6
top - 08:43:16 up 96 days, 22:16,  1 user,  load average: 4.03, 3.92, 3.98
Tasks: 199 total,   1 running, 198 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  0.5%sy, 50.0%ni, 49.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  3.9%sy, 46.8%ni, 49.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  0.0%us,  3.0%sy, 47.5%ni, 49.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  5.0%sy, 45.5%ni, 49.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3593 foldinga  39  19  276m  80m 2972 S  402  1.0  12:55.42 FahCore_a3.exe

Now, how come top says it's using 100% of the CPU (400% / 4 cores) while exactly half of the processors are idle??

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 30
model name      : Intel(R) Xeon(R) CPU           X3440  @ 2.53GHz
stepping        : 5
cpu MHz         : 2526.932
cache size      : 8192 KB
physical id     : 0
siblings        : 4
core id         : 3
cpu cores       : 4
apicid          : 6
initial apicid  : 6
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid
bogomips        : 5054.02
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

EDIT: In response to i_grok and Max Alginin, I made sure Hyperthreading was enabled on the server. Once I got it turned on, here are the results of top now. Notice, the same symptoms are evident.

top - 10:17:01 up 47 days, 10:28,  3 users,  load average: 7.93, 7.96, 8.02
Tasks: 150 total,   1 running, 149 sleeping,   0 stopped,   0 zombie
Cpu0  :  0.0%us,  2.8%sy, 42.0%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  0.0%us,  2.2%sy, 42.5%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  1.2%us,  3.7%sy, 95.1%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  0.0%us,  1.7%sy, 43.1%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  1.1%sy, 43.6%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy, 44.8%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  2.2%sy, 42.5%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  1.7%sy, 43.1%ni, 55.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8177700k total,  6258704k used,  1918996k free,    29248k buffers
Swap:        0k total,        0k used,        0k free,  5203172k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 8132 foldinga  39  19  557m  99m 3060 S  796  1.2   1510:53 FahCore_a3.exe
|improve this question

migrated from serverfault.com Feb 19 '12 at 15:44

This question came from our site for system and network administrators.

5

To expand upon Max's answer: Processors with hyperthreading report two cores to the operating system when there is really a single physical core. Linux sees 4 cores, but your processor has 2 physical cores.

The physical core has advanced scheduling capabilities and some extra transistors to handle the second thread, but for many processor-intensive applications the hyperthreading does not provide an actual benefit.

In my field, we benchmark the applications that will be running on a system and disable hyperthreading in the BIOS if it hinders performance.

EDIT:

If I load half the cores using cat /dev/urandom > /dev/null here is what I get on a modern Xeon running CentOS 6:

top - 19:32:38 up 1 day,  8:17,  3 users,  load average: 12.13, 8.73, 4.07
Tasks: 694 total,  13 running, 681 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us, 50.4%sy,  0.0%ni, 49.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  49552656k total,  2693452k used, 46859204k free,   186448k buffers
Swap: 15999992k total,        0k used, 15999992k free,   469560k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                           
21193 root      20   0 98.5m  560  452 R 100.0  0.0   6:23.31 cat                                                                                               
21194 root      20   0 98.5m  560  452 R 100.0  0.0   6:22.76 cat                                                                                               
21195 root      20   0 98.5m  560  452 R 100.0  0.0   6:22.38 cat                                                                                               
21201 root      20   0 98.5m  560  452 R 100.0  0.0   6:21.98 cat                                                                                               
21202 root      20   0 98.5m  560  452 R 100.0  0.0   6:21.62 cat                                                                                               
21203 root      20   0 98.5m  560  452 R 100.0  0.0   6:21.36 cat                                                                                               
21206 root      20   0 98.5m  560  452 R 100.0  0.0   6:20.43 cat                                                                                               
21187 root      20   0 98.5m  560  452 R 99.6  0.0   6:24.45 cat                                                                                                
21204 root      20   0 98.5m  560  452 R 99.6  0.0   6:21.03 cat                                                                                                
21205 root      20   0 98.5m  560  452 R 99.6  0.0   6:20.73 cat                                                                                                
21213 root      20   0 98.5m  560  452 R 99.6  0.0   6:20.09 cat                                                                                                
21214 root      20   0 98.5m  560  452 R 99.6  0.0   6:19.70 cat
|improve this answer
  • What difference does it make whether they are hyperthreaded or not? If linux sees 4 cpus that are each 50% used, then you can't end up with a total of 400% utilization. This looks like a bug in top. – psusi Feb 21 '12 at 0:04
  • Symptoms persist with HT enabled... – Theodore R. Smith Feb 21 '12 at 16:21
  • @psusi I suspected that top did not properly account for hyperthreading and thus showed misleading results (I hadn't really tested and have updated my answer). – i_grok Feb 23 '12 at 0:32
3

It's been over four years, and to be fair, I completely forgot about this question and only came back because I saw that I had received a Notable Question badge for it.

The problem was tangentially related to hyperthreading, but as I continually pointed out to responders, that was not the cause for the 50% idling.

The cause had to do with an inappropriately configured kernel dynamic ticks configuration. I was running Gentoo and using a custom-built kernel. After I upgraded the kernel, sometime in mid-2012, the issue resolved itself.

|improve this answer
1

I don't think Hyperthreading is the issue here. If you look at one of these lines:

Cpu0  :  0.0%us,  0.5%sy, 50.0%ni, 49.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

The issue isn't that half of your cores are staying idle, it is that each core is idle 50% of the time! Contradicting this though is the line:

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3593 foldinga  39  19  276m  80m 2972 S  402  1.0  12:55.42 FahCore_a3.exe

Which says that foldingathome maxed out all four cores. This is just weird, and makes me suspect you discovered a bug in the kernel's reporting of CPU stats. Here are some things to try:

  • Run foldingathome without renice'ing it.
  • Run a different program that will swamp all cores, and see what numbers you get.

Also, is foldingathome using only your CPU for computation or is their a GPU in the mix as well?

|improve this answer
1

I suspect the top summary and the process line are averaging over different time periods.

The process line is saying 'right then' for some value of n the process is consuming 100% of one logical core, and the summary is saying over a longer period of time it's consuming 50%, so it's probably running flat out when it runs but only half the time.

I've seen this when outputting from top in batch mode, particularly if it is only given one cycle. If looped in bash for instance the process line can say 100% on multiple incarnations but the summary be way less than the 1/n times 100% (for n logical cores).

Check the CPU time consumed over time - does it show the process line or the summary as being correct ?

How are you running top?

|improve this answer
0

I tested i_groks' scenario on Centos 6 on an AMD double quad core (8 "apparent" cores per processor since AMD has an HT equivalent) and running 22 processes like this one

cat /dev/urandom > /dev/null

we observe that the total CPU usage is saturated around 95%. I double checked with atop and the total usage was consistent.

Additional processes will not increase the total usage, probably due to contention.

Looking at core level: the 16 "apparent" cores (each physical cores has 2 sibblings) are shown as running at nearly 100%.

So, I can't reproduce the initially reported situation / problem.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.