# Understanding top and load average

I'm observing a high load average on a certain machine (about 9) in all three load fields. I understand load as the number of processes in state "run" / currently desiring CPU time. Am I correct at reasoning that if N processes are running on my machine this cannot produce a load greater than N?

Also, does the load count in respect to processes or threads? In other words, can a multithreaded process produce a load greater than 1?

• Note that (because they're usually expected to be scheduled very soon), tasks in the "uninterruptible" state are also counted as running. Just a detail, not something that changes your question. Mar 17, 2011 at 14:43

Load average is usually described as "average length of run queue". So few CPU-consuming processes or threads can raise LA above 1. There is no problem if LA is less than total number of CPU cores. But if it gets higher than number of CPUs, this means some threads/processes will stay in queue, ready to run, but waiting for free CPU.

The numbers that are used to calculate load average are tasks in the run or uninterruptable state and the amount of work done in the time slice of the moving average. These tasks can be part of a multithreaded process. The fields are fuzzy the farther back in time due to smoothing results from the algorithm used.

A load of 1 is equal to 100% of one CPUs worth of work. If you had a multithreaded application that managed to have a number of active threads in excess of the number of available CPUs, then you could have a single process drive the load above 1. This would likely be a short term spike and not reflected in the longer time slice views of the load average.

Also, since the load average was developed before there were multi-core systems, its important to divide the load numbers by the total available number of cores. If this is a sustained load of 9 on a 4 socket quad core system, then this is a 9 of 16 load and not really a problem.

• — I've yet to see a system where multiple cores on a socket are exposed up to user-space in a way which makes them look like anything but more CPUs. So in top, on the 4-socket quad-core system, 16 CPUs are seen. (So one can readily see that the load of 9 is below the number of CPUs.) Mar 17, 2011 at 18:48
• I understand that the numbers are logarithmic, so they are just under or above 1. One system I have, I saw the system report this: load average: 10.41, 9.57, 9.26 . What?? Why would one linux system report load average: 0.81, 1.57, 0.98 whereas this system reports load average: 10.41, 9.57, 9.26 ? Are all unix's not the same on this? Jun 20, 2011 at 21:56

See kernel/sched/loadavg.c which has a long and excellent comment at the start explaining the derivation of load average from a exponentially decaying average of the number of runnable threads (the "run queue") plus the number of uninterruptable threads (waiting on I/O or waiting on a lock).

Here's the essence of the comment, but it is worthwhile reading in full:

`````` * The global load average is an exponentially decaying average of
* nr_running + nr_uninterruptible.
*
*     nr_active = 0;
*     for_each_possible_cpu(cpu)
*         nr_active += cpu_of(cpu)->nr_running +
*                      cpu_of(cpu)->nr_uninterruptible;
*     avenrun[n] = avenrun *
*                  exp_n + nr_active *
*                  (1 - exp_n)
``````

Real life makes the code somewhat complex: per-CPU counters, tickless kernels, hotswap CPUs, lack of floaing point code requiring a fixed-point implementation of exp(n). But it's easy to see that these are all working towards faithfully implementing the method described in the comment.

The 3 load averages are something like a logarithmic function revolving around the number 1. Something similar to f(x) = eX (e to the X exponent). Technically a fixed-point representation of a exponential decay function simulating a average. They are additive, per CPU, so a full load might look like 4.00 on a quad-core system. The first number is the average over the last minute, the second is the average over the last five minutes, and the third is the average over the last 15 minutes. I thought an answer should be dropped here that mentions that.

Instant load: number of tasks running or waiting to run, or in another way, the number of tasks willing to run

Load average: the measure above but exponentially averaged with previous samples of the same measure

Both of these numbers are unbounded, and often much larger than N.

To be clear: the load count on Linux includes threads, there is no doubt about it. You can produce an arbitrarily large load with a single process that creates many threads.

More on this here