I have two dual-core Linux systems installed using Linux cgroups with relatively recent kernels; one is running Debian Squeeze, the other Ubuntu 11.04 Natty Narwhal. I've gotten CPU load balancing with cgroups working a bit better on the Debian system despite its older kernel. But it's not right for everything, and the specific oddity I'm asking about here happens on both systems.
If you read Resource Management in Linux with Control Groups it gives an example showing how to reproduce the problem. Here's the Ubuntu version (run this as root):
cd /sys/fs/cgroup/cpu [On Debian Squeeze start at /mnt/cgroups/cpu instead] mkdir low high echo 512 > low/cpu.shares echo 2048 > high/cpu.shares yes low > /dev/null & echo $! > low/tasks yes high > /dev/null & echo $! > high/tasks ps -C yes -opid,%cpu,psr,args [repeat that a few times] killall -9 yes
I was expecting the "high" process to be allocated more time than the "low" one; what actually happens with this test case is always more like this:
root@black:/sys/fs/cgroup/cpu# ps -C yes -opid,%cpu,psr,args PID %CPU PSR COMMAND 3105 88.3 1 yes low 3106 94.5 0 yes high
Where the times are almost equal. Here's my question: why is that happening?
In the presentation, this problem is shown going away by pinning each process to the same CPU; additional lines to test that:
taskset -c 1 yes high > /dev/null & echo $! > high/tasks taskset -c 1 yes low > /dev/null & echo $! > low/tasks ps -C yes -opid,%cpu,psr,args [later, rinse, repeat] killall -9 yes
The result then is what I was expecting to see all the time: the "high" process getting a much higher percentage of the CPU:
root@black:/sys/fs/cgroup/cpu# ps -C yes -opid,%cpu,psr,args PID %CPU PSR COMMAND 3128 83.3 1 yes high 3129 20.7 1 yes low
Explaining why this works would be a useful step toward figuring out why the earlier one doesn't too.