0

I'm running some matrix multiplications with NumPy and Python's 3 Pool multiprocessing on a server with 4 Intel Xeon E5-4620, totalling 64 cores.

top in Solaris mode shows (the):

top - 16:00:55 up 7 days, 16:42,  2 users,  load average: 292.74, 295.09, 297.44
Tasks: 704 total,   8 running, 696 sleeping,   0 stopped,   0 zombie
%Cpu(s): 76.6 us, 23.4 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 52830358+total, 52306323+free,  3022900 used,  2217456 buff/cache
KiB Swap:  6553596 total,  6553596 free,        0 used. 52376304+avail Mem 

PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
75266 splitti+  20   0 3050540 294912  10808 R 25.0  0.1  27649:20 python3
75264 splitti+  20   0 3050028 292448  10936 R 21.7  0.1  27750:10 python3
75286 splitti+  20   0 3006504 325704  10668 R 20.3  0.1  26831:30 python3
75598 splitti+  20   0 3006820 326264  11164 R 20.2  0.1  27221:56 python3
75288 splitti+  20   0 3006504 325696  10660 R 10.5  0.1  26351:29 python3
75265 splitti+  20   0 3050028 294292  10728 R  1.6  0.1  22889:05 python3
...

In Irix mode:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND    
75266 splitti+  20   0 3050028 292292  10808 R  1494  0.1  27712:54 python3    
75264 splitti+  20   0 3050540 295028  10936 R  1390  0.1  27803:50 python3    
75288 splitti+  20   0 3006504 325696  10660 R  1305  0.1  26395:29 python3    
75598 splitti+  20   0 3006820 326112  11164 R  1240  0.1  27275:08 python3    
75286 splitti+  20   0 3006504 325628  10668 R 752.6  0.1  26874:41 python3    
75265 splitti+  20   0 3050028 294292  10728 R 153.9  0.1  22893:03 python3
... 

In per core mode:

top - 16:03:40 up 7 days, 16:45,  2 users,  load average: 299.56, 297.12, 297.83
Tasks: 704 total,   8 running, 696 sleeping,   0 stopped,   0 zombie
%Cpu0  : 76.9 us, 23.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  : 71.9 us, 28.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  : 70.4 us, 29.6 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  : 74.6 us, 25.4 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  : 73.9 us, 26.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  : 79.9 us, 20.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  : 80.9 us, 19.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  : 78.9 us, 21.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu8  : 80.3 us, 19.7 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu9  : 78.9 us, 21.1 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu10 : 80.2 us, 19.8 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
...

The question: How are these values connected?

If top shows ~80% in per core usage and 25% in per process Solaris mode, where do 50% of core usage disappear? How can these 9 Python processes (I pasted only 6) use all 64 cores?

0

The top manpage describes the CPU percentage as follows:

The task's share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time.

In a true SMP environment, if a process is multi-threaded and top is not operating in Threads mode, amounts greater than 100% may be reported. You toggle Threads mode with the `H' interactive command.

Also for multi-processor environments, if Irix mode is Off, top will operate in Solaris mode where a task's cpu usage will be divided by the total number of CPUs. You toggle Irix/Solaris modes with the `I' interactive command.

In Solaris mode, the percentage shows how much of your total CPU capacity is being used by each process; thus the process with identifier 75266 used 25% of all the CPU capacity in the last refresh cycle. In Irix mode, the percentage shows how much of one CPU “unit” is being used by each process; thus the process with identifier 75266 used CPU time equivalent to 14.94 cores in the last refresh cycle (presumably it’s running 15 threads or more, which explains how nine processes can use more than nine cores).

You can go from one type of calculation to the other by dividing or multiplying by the number of execution units: 15 is nearly 25% of 64 (close enough to match, given that your screen “shots” weren’t of the same refresh cycle).

  • Seems legit. If these processes spawn more sub-processes, and each process uses ~20% overall, than they use ~100% of all CPU pool available. – homocomputeris Feb 7 '18 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.