Frequency scaling governor scales up too many cores

Question

I have two machines on which I run the same OS, the same kernel, the same CPU freq scaling driver, and the same CPU freq scaling governor.

One will boost all core frequencies if one process pins a core.

One will boost only one physical core if there is one process pinning a core.

Machine A

CPU: Intel(R) Xeon(R) W-2140B CPU @ 3.20GHz
Scaling driver: intel_pstate
Scaling governor: powersave
OS: Ubuntu 21.10
Kernel: 5.13.0-22-generic

Machine B

CPU: 11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz
Scaling driver: intel_pstate
Scaling governor: powersave
OS: Ubuntu 21.10
Kernel: 5.13.0-22-generic

Observe these examples:

Pinning two cores on machine A: (NOTE that this machine has Hyperthreading, so scales up 4 of the 16 available virtual cores.)

Pinning two cores on machine B: (NOTE that this machine has 6 cores with HT, and scales up all 12 virtual cores.)

The screenshots are of freqtop, an open sourced frequency monitor that I wrote myself. The orange ticks show the load on each core defined as a percentage of user+kernel cycles in relation of total cycles (user+kernel+idle.)

Why does the i5-11600K CPU throttle up all cores, even if there is demand for just one?

UPDATE: A difference I found between machines is the number of pstates.

For the Xeon /sys/devices/system/cpu/intel_pstate is set to 33, and the corei5 has it set to 42. I am not sure what the significance is for this.

Answer 1

Among Intel CPUs there are several different ways to handle the core throttling. It's pretty technical to learn them all. Most people just learn the CPUs they use. But I am sure it's either the level of control permitted by the firmware (BIOS, UEFI) and/or the internals of the processor that are giving you these differing outcomes.

I see the same on two of our machines, both running Debian Bullseye, but different processors. Sometimes $ powertop can be revealing. It's meant to tune power consumption on laptops, but the info revealed on desktops is interesting for different reasons.

I use powertop to check whether the cores are running in turbo mode, and how much they're throttling, among other things. It works for AMD too.

You've wrote that monitor, so it seems you might know more than I on the subject.

This forum would not be the place to get into many details, simply because the material is so voluminous and technical. I hope this helps.

Answer 2

I have been able to make the corei5 machine behave more appropriately, and only scale up the frequency of the cores that we actually have work for.

To do so, I found that I had to switch the intel_pstate driver mode.

In /sys/devices/system/cpu/intel_pstate/status you can switch between active and passive modes.

active mode: In this mode the driver bypasses the scaling governors layer of CPUFreq and provides its own scaling algorithms for P-state selection.

passive mode: ...the driver behaves like a regular CPUFreq scaling driver. That is, it is invoked by generic scaling governors when necessary to talk to the hardware in order to change the P-state of a CPU...

If I read this correctly, I think in active mode the CPU has less feedback from the OS itself. I think the CPU incorrectly guesses that all cores need to be throttled up.

What is still unexplained, is the fact that the Xeon CPU does just fine in active move using the same intel_pstate driver. But that could just be a guess of not all CPUs being created equal. Or maybe even a suspect motherboard or BIOS that doesn't properly do pstates.