Linux · critical · Day-to-day operations

"bash: fork: retry: Resource temporarily unavailable"

The system or the user has hit a process or thread limit. Nothing new can start, including the commands you would use to diagnose it.

What you see

Commands fail to run, SSH refuses new sessions, and an already-open shell can barely do anything. The machine is otherwise responsive.

What is actually wrong

A per-user process limit reached (often by a runaway service forking), the system-wide pid_max exhausted, or a cgroup pids limit in a container.

Codes and articles

fork: retry: Resource temporarily unavailableEAGAINCannot forkpthread_create failederrno 11

Fixes (2)

Find the runaway and raise the right limit

Shell as root30 minutesmedium riskreversible

One account is affected. Keep any working root shell you have open — you may not get another.

Count processes per user. Use an existing shell; starting a pipeline may fail.
Shell
```
ps -eo user= | sort | uniq -c | sort -rn | head
```

Look at the limits that apply to that user.

Shell

sudo -u appuser bash -c 'ulimit -u'cat /proc/sys/kernel/threads-max

For a systemd service, the limit is TasksMax on the unit — not limits.conf, which is why editing that appears to do nothing.
Shell
```
systemctl show myapp.service -p TasksMax -p TasksCurrent
```
systemd applies a default TasksMax of 15% of pid_max to every unit. A service that forks per connection hits it long before any ulimit.

Raise it for that unit.

Shell

sudo systemctl set-property myapp.service TasksMax=8192sudo systemctl daemon-reload

For an interactive user, raise nproc in the limits file.

Shell

echo 'appuser soft nproc 4096' | sudo tee -a /etc/security/limits.d/90-nproc.confecho 'appuser hard nproc 8192' | sudo tee -a /etc/security/limits.d/90-nproc.conf

Then find why it needed so many. A limit raised without understanding the cause just moves the failure later.
Shell
```
ps -eo pid,ppid,user,etime,cmd --sort=start_time | grep appuser | tail -30
```

Confirm it workedThe user can start processes and the count is stable.

Shell

systemctl show myapp.service -p TasksCurrent -p TasksMax

If you need to undo itsystemctl revert myapp.service; remove the limits.d file.

Recover a machine that cannot fork at all

Existing shell or console20 minuteshigh riskreversible

Nothing can start. Use shell builtins, which need no fork, wherever possible.

Use builtins to look around — echo and read are builtins and will work when /bin/ls will not.
Shell
```
echo /proc/[0-9]*/ | wc -w
```
Globbing and echo are handled inside bash with no fork. Anything that calls an external binary needs a process slot you do not have.
Read the system-wide pid ceiling.
Shell
```
cat /proc/sys/kernel/pid_max
```
Raise it temporarily to buy room to work.
Shell
```
echo 131072 > /proc/sys/kernel/pid_max
```
Now that you can fork, find the offender.
Shell
```
ps -eo user,pid,ppid,cmd --sort=-ppid | head -40
```
Kill the whole process group rather than individual processes, which respawn faster than you can kill them.
Shell
```
sudo systemctl stop runaway.servicesudo pkill -9 -u appuser
```

Make the ceiling permanent if it was genuinely too low for the workload.

Shell

echo 'kernel.pid_max = 131072' | sudo tee /etc/sysctl.d/99-pidmax.confsudo sysctl --system

Confirm it workedCommands run normally and the process count is stable.

Shell

ps -eo user= | wc -l; cat /proc/sys/kernel/pid_max

If you need to undo itDelete /etc/sysctl.d/99-pidmax.conf and run sysctl --system.

Related faults

Where this stops. This write-up was written and checked by hand. It says what each step changes, how to confirm it worked and how to reverse it, and anything destructive is flagged before you reach it. If it does not match what your machine is doing, search the Support Centre for the exact code or message — and when something needs a person, get in touch.