I have a 64-bit server running linux 3.x that reports "bash: fork: cannot allocate memory". This server has plenty of RAM (64G) and swap (128G) and I cannot see a particular process being the culript (no apparent memory leak).
I was not limited by "ulimit" as nothing was running under this shell, and the problem also existed as root. It is slightly complicated to debug a server when even typing top or cat /proc/whatever fires "bash: fork: cannot allocate memory" 95% of the time.
A few check I already did: - some memory was remaining (total allocated space less than swap space). - no ulimit - no clear issue, no messages in log, no messages in dmesg - virtual memory not exhausted (as far as I checked)
What should I check when I encounter this same problem again ? What can be the causes of no possible memory allocation apart ulimit / limited swap space / non exhausted virtual space ? Of course a reboot solved the issue (for now) but I am pretty surprised by this issue.