5

I'd like to limit the container to 25% of the system's total CPU bandwidth.
Here's my setup:

  • LXC version 1.0.2
  • kernel 3.2.45
  • one user created cgroup (foo) for an LXC container
  • 40 available cores on the host
  • the host and container have default values for every other cgroup subsystem except:
  • /sys/fs/cgroup/cpu/lxc/foo/cpu.cfs_quota_us = 400000
  • /sys/fs/cgroup/cpu/lxc/foo/cpu.cfs_period_us = 100000
  • /sys/fs/cgroup/cpuset/lxc/foo/cpuset.cpus = 0-15

I calculated the quota using this formula:

(# of cpus available to container) * (cpu.cfs_period_us) * (.25) so 16 * 100000 * .25 = 400000

I ran a basic stress-ng inside and outside the container at the same time to get a gauge of how many operations per second were being allowed inside and out and the results were basically the same as running with a quota of "-1", which is to say no quota.

Outside Run:

$ ./stress-ng  --cpu-load 50 -c 40 --timeout 20s --metrics-brief
stress-ng: info: [25649] dispatching hogs: 40 cpu  
stress-ng: info: [25649] successful run completed in 20.44s  
stress-ng: info: [25649] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s  
stress-ng: info: [25649]                          (secs)    (secs)    (secs)   (real time) (usr+sys time)  
stress-ng: info: [25649] cpu              37348     20.18    380.56      0.58      1850.85        97.99  

Inside Run:

$ ./stress-ng --cpu-load 100 -c 16 --timeout 20s --metrics-brief  
stress-ng: info: [34256] dispatching hogs: 16 cpu  
stress-ng: info: [34256] successful run completed in 20.10s  
stress-ng: info: [34256] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s  
stress-ng: info: [34256]                          (secs)    (secs)    (secs)   (real time) (usr+sys time)  
stress-ng: info: [34256] cpu              24147     20.03    205.20      0.17      1205.67       117.58  

Based on the ops/s I'm getting 39%. Why does this happen? Shouldn't it be limited by cpu.cfs_quota_us?

Thanks for the help in advance.

3

Wanted to post the answer to this question in case anyone else sees a similar confusing result. It looks like I had two problems:

  • Need to use # of cpus on host, not # available CPUs in the cgroups cpuset to estimate CPU bandwidth:

    (# of cpus on the host) * (cpu.cfs_period_us) * (.25) so 40 * 100000 * .25 = 1000000

  • My run of stress-ng inside the container was using the cpu and cpuset controllers of the /lxc/foo cgroup while the run of stress-ng outside of the container was using the /system/sshd.service cgroup

To better model my real world application I should have specified which controllers to use by using cgexec:

$ cgexec -g cpuset:/lxc/foo -g cpu:/lxc/foo ./stress-ng --cpu-load 100 -c 48 --times --timeout 10s --metrics-brief  
stress-ng: info: [6252] dispatching hogs: 48 cpu  
stress-ng: info: [6252] successful run completed in 10.36s  
stress-ng: info: [6252] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s  
stress-ng: info: [6252]                          (secs)    (secs)    (secs)   (real time) (usr+sys time)  
stress-ng: info: [6252] cpu              11152     10.09    102.83      0.12      1105.60       108.32  
stress-ng: info: [6252] for a 10.36s run time:  
stress-ng: info: [6252]     414.46s available CPU time  
stress-ng: info: [6252]     102.85s user time   ( 24.82%)  
stress-ng: info: [6252]       0.12s system time (  0.03%)  
stress-ng: info: [6252]     102.97s total time  ( 24.84%)  

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.