Linux · high · Core system faults

"No space left on device" but df shows free space

The block usage is fine but the filesystem has run out of inodes, or a deleted file is still held open by a process and its space has not been released.

What you see

Writes fail with ENOSPC. df -h shows plenty of free space on the filesystem in question, so the error looks nonsensical.

What is actually wrong

Either inode exhaustion — millions of tiny files, typically in a mail spool, a session directory or a cache — or a process holding a deleted file open, which keeps the blocks allocated until it closes or exits.

Codes and articles

ENOSPCNo space left on deviceerrno 28

Fixes (2)

Find and clear the inode consumer

Shell as root30–60 minutesmedium risknot reversible

df -i shows the inode table full. Deleting files is not reversible — check what they are before removing them.

Confirm which filesystem is out of inodes.
Shell
```
df -i
```
Find the directories holding the most files. This walks the tree, so run it on the affected mount only.
Shell
```
sudo du --inodes -x -d 3 /var 2>/dev/null | sort -rn | head -20
```
-x keeps it on one filesystem so it does not wander into /proc or a network mount and take all day.

On older systems without du --inodes, count directly.

Shell

sudo find /var -xdev -type f -printf '%h\n' 2>/dev/null | sort | uniq -c | sort -rn | head -20

Look at what the files actually are before deleting anything.
Shell
```
sudo ls -la /var/spool/postfix/maildrop | head -20
```
Delete safely in batches — a plain rm * will fail with 'argument list too long' at this scale.
Shell
```
sudo find /var/spool/postfix/maildrop -type f -mtime +7 -delete
```
find -delete streams the deletions instead of building one enormous argument list, and the -mtime guard stops it taking files still in use.
Fix the cause: rotate the logs, set a cron job to clear the cache, or fix the application filling the directory.

Confirm it workedIUse% has dropped and writes succeed.

Shell

df -i && touch /var/tmp/write-test && rm /var/tmp/write-test && echo OK

If you need to undo itNone — deleted files are gone. Restore from backup if you removed something needed.

Release space held by a deleted-but-open file

Shell as root15 minutesmedium riskreversible

Inodes are fine but space is not being returned. Classic after someone deletes a huge log that a running daemon still has open.

List deleted files still held open, largest first.
Shell
```
sudo lsof -nP +L1 | sort -k7 -rn | head -20
```
+L1 selects files with a link count below 1 — that is exactly the set that has been unlinked but not closed.
Note the PID and the size. The space comes back when that file descriptor closes.
Preferred: reload the service so it reopens its logs.
Shell
```
sudo systemctl reload nginx
```
If reload is not supported, restart it.
Shell
```
sudo systemctl restart nginx
```
As a last resort — and only if you cannot restart the service — truncate the descriptor in place. Replace PID and FD with the values from lsof.
Shell
```
sudo truncate -s 0 /proc/12345/fd/4
```
This frees the blocks without stopping the process. It discards the file contents, so do not do it to something you still need to read.
Set up logrotate with copytruncate for whatever produced the file, so it does not recur.

Confirm it workeddf shows the space returned.

Shell

df -h

If you need to undo itNothing to undo once the descriptor is closed; the data is unrecoverable either way since the file was already unlinked.

Related faults

Where this stops. This write-up was written and checked by hand. It says what each step changes, how to confirm it worked and how to reverse it, and anything destructive is flagged before you reach it. If it does not match what your machine is doing, search the Support Centre for the exact code or message — and when something needs a person, get in touch.