Linux  ·  medium  ·  Day-to-day operations

Load average is high but the CPU looks idle

Linux counts uninterruptible processes in the load average, so a storage or NFS problem produces a huge load figure with almost no CPU use.

What you see

uptime shows a load of 30 on an 8-core box, but top shows the CPU 95% idle. The machine feels sluggish and some commands hang.

What is actually wrong

Processes stuck in D state waiting on I/O — a slow or failing disk, a hung NFS mount, or a saturated storage path. Occasionally very high context switching.

Codes and articles

high load averageiowaitD stateuninterruptible sleepload average

The fix

Separate CPU load from I/O wait
Shell30 minuteslow riskreversible

Load is high and CPU is not. The load average alone cannot tell you which, so start by splitting them.

  1. Look at the CPU breakdown. The %wa column is I/O wait.

    Shell
    top -bn2 -d1 | grep -E '^%Cpu' | tail -1vmstat 1 5

    In vmstat, the 'b' column counts processes blocked on I/O. If b is high and 'r' is low, no amount of CPU would help.

  2. List the processes actually in uninterruptible sleep.

    Shell
    ps -eo state,pid,user,wchan:30,cmd | awk '$1 ~ /D/'

    The wchan column names the kernel function they are stuck in — nfs_wait, io_schedule and similar point straight at the subsystem.

  3. Find which device is saturated.

    Shell
    iostat -xz 1 5

    %util near 100 with a high await means that device is the bottleneck. A high await with low %util usually means the storage behind it, not the disk itself.

  4. Attribute the I/O to a process.

    Shell
    sudo iotop -oPa -n 5
  5. Check for hung NFS mounts, a very common cause of unkillable D-state processes.

    Shell
    mount -t nfs,nfs4sudo dmesg -T | grep -i 'nfs.*not responding' | tail
  6. Look for kernel hung-task warnings, which name the stuck process and its stack.

    Shell
    sudo dmesg -T | grep -iA10 'hung_task\|blocked for more than' | tail -40
Confirm it workedThe load figure tracks the CPU run queue again once the I/O source is dealt with.
Shell
uptime; vmstat 1 3
If you need to undo itNone — diagnostic.

Where this stops. This write-up was written and checked by hand. It says what each step changes, how to confirm it worked and how to reverse it, and anything destructive is flagged before you reach it. If it does not match what your machine is doing, search the Support Centre for the exact code or message — and when something needs a person, get in touch.