4

Is it possible to deny malloc instead of killing process selected by OOM killer? I think out of memory can be at least logged by the process which is better than being killed. Also, part of the problem is that system is unresponsive for long time (couple of minutes) before killing the offensive process.

Are control groups better tool to guarantee other processes memory?

  • You should read about memory overcommit if you aren't familiar with that. – Mat Sep 22 '16 at 12:27
3

You can set limits before starting the process. For example set the virtual (soft) memory limit for the current shell to 1G

ulimit -S 1048576

You can also set such limits system wide, for example I have this:

  • /etc/security/limits.conf:

    *               soft    as             25165824
    *               hard    as             25165824
    
  • /etc/systemd/system.conf:

    DefaultLimitAS=25769803776
    

To limit all processes to 24G on my 32G systems. Although two processes together could still cause such OOM situation, it never happened in practice.

Regarding "unresponsive for long time before killing the offensive process":

  • Don't use a swap partition, then things will be killed quickly.

Note, actually the malloc (virtual memory) is not the real problem. OOM happens only when the process starts using that malloc'ed memory. Unfortunately on Linux you cannot limit the resident memory. See "memory overcommit":

  • Your suggestion probably wins in the "simplicity" category (vs control-groups for example), but I'm hoping for more dynamic approach like don't overcommit any new pages above 80% of RAM under same assumption that already overcommited ones will likely not be requested. Note that I do not have swap. Page scanning, nasty malloc loops somewhere? Not sure how to debug this when system is unresponsive. – sevo Sep 23 '16 at 15:25
  • Maybe it's "unresponsive" is because there is no RAM left for HD cache. – rudimeier Sep 26 '16 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.