What's the Linux kernel's behaviour when processes in a cgroup hit their memory limit?

Question

The memory resource controller for cgroups v1 allows for setting limits on memory usage on a particular cgroup using the memory.limit_in_bytes file. What is the Linux kernel's behavior when this limit is reached?

In particular:

• Does the kernel OOM kill the process and if so is the oom_score of the process taken into account, or is it the process that asked for the memory that caused the limit to be hit that gets killed?
• Or would the request for memory just be rejected in which case the process would only die if it didn't deal with such an event?

Answer 1

By default OOM is overseeing cgroups.

memory.oom_control

contains a flag (0 or 1) that enables or disables the Out of Memory killer for a cgroup. If enabled (0), tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer. The OOM killer is enabled by default in every cgroup using the memory subsystem; to disable it, write 1 to the memory.oom_control file:

  ~]# echo 1 > /cgroup/memory/lab1/memory.oom_control

When the OOM killer is disabled, tasks that attempt to use more memory than they are allowed are paused until additional memory is freed.

References

• Redhat docs - 3.7. MEMORY