0

I am using a CentOS 7.5 instance in AWS which has 16 CPUs and 32GB memory. I found when I run the following command, the whole system will be unresponsive, I cannot run any commands on it anymore, cannot even establish a new SSH session (but can still ping it). And I do not see OOM killer triggered at all, it seems the whole system just hang forever.

stress --vm 1 --vm-bytes 29800M --vm-hang 0 

However if I run stress --vm 1 --vm-bytes 29850M --vm-hang 0 to consume a bit more memory (50MB), OOM kill will be successfully triggered (I can see it in dmesg). And if I run the stress command to consume less memory than 29800MB (e.g. stress --vm 1 --vm-bytes 29700M --vm-hang 0), the system will be responsive (and no OOM kill) and I can run any commands as usual.

So it seems 29800MB is a "magic number" for this instance, if I run stress command to use more memory than it, the command will be OOM killed, and if I run stress command to use less memory than it, everything is OK, if I run stress command to just use 29800MB memory, the whole system will be unresponsive. And I also observed the same behavior in Linux host with different spec, e.g. in a CentOS 7.5 instance which has 72 CPUs and 144GB memory, the "magic number" is "137600MB".

My question is, why won't OOM kill be triggered when the "magic number" of memory is used?

2
  • You claim OOM cannot be triggered. Did you try to trigger it with the Sysrq sequence? – Quasímodo Jun 8 '20 at 11:00
  • No, I did not do that. – Eric Jun 9 '20 at 11:26
1

Oh, this topic has been discussed ad neseum already.

See:

The Linux kernel's inability to gracefully handle low memory pressure

Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure

The solution? Various daemons like earlyoom (installed and enabled by default in Fedora 32) which is my favourite. There are others:

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.