I know about memory overcommitment and I profoundly dislike it and usually disable it. I am not thinking of setuid-based system processes (like those running sudo
or postfix) but of an ordinary Linux process started on some command line by some user not having admin privileges.
A well written program could malloc
(or mmap
which is often used by malloc
) more memory than available and crash when using it. Without memory overcommitment, that malloc
or mmap
would fail and the well written program would catch that failure. The poorly written program (using malloc
without checks against failure) would crash when using the result of a failed malloc
.
Of course virtual address space (which gets extended by mmap
so by malloc
) is not the same as RAM (RAM is a resource managed by the kernel, see this; processes have their virtual address space initialized by execve(2) and extended by mmap
& sbrk
so don't consume directly RAM, only virtual memory).
Notice that optimizing RAM usage could be done with madvise(2) (which could give a hint, using MADV_DONTNEED
to the kernel to swap some pages onto the disk), when really needed. Programs wanting some overcommitment could use mmap(2) with MAP_NORESERVE
. My understanding of memory overcommitment is as if every memory mapping (by execve
or mmap
) is using implicitly MAP_NORESERVE
My perception of it is that it is simply useful for very buggy programs. But IMHO a real developer should always check failure of malloc
, mmap
and related virtual address space changing functions (e.g. like here). And most free software programs whose source code I have studied have such check, perhaps as some xmalloc
function....
Are there real life programs, e.g. packaged in a typical Linux distributions, which actually need and are using memory overcommitment in a sane and useful way? I know none of them!
What are the disadvantages of disabling memory overcommitment? Many older Unixes (e.g. SunOS4, SunOS5 from the previous century) did not have it, and IMHO their malloc
(and perhaps even the general full-system performance, malloc
-wise) was not much worse (and improvements since then are unrelated to memory overcommitment).
I believe that memory overcommitment is a misfeature for lazy programmers.
The user of that program could setup some resource limit for setrlimit(2) called with RLIMIT_AS
by the parent process (e.g. ulimit
builtin of /bin/bash
; or limit
builtin of zsh, or any modern equivalent for e.g. at
, crontab
, batch
, ...), or a grand-parent process (up to eventually /sbin/init
of pid 1 or its modern systemd variant).
R
have trouble on OpenBSD because they want large amounts of virtual memory and OpenBSD says nope. Those same packages are fine on Linux and do not cause the crazy drunk oom killer to start blasting away at the process table.madvise
. The kernel usually gets the hint automatically anyway, by keeping a count of which pages have been recenlty used and which have not. TheMAP_NORESERVE
flag to themmap
system call only means no swap space is reserved for the mapping, it does not disable demand paging.