How to create memory dumps for out of memory.
If system is out of memory then oom-killer kills the process which occupies highest memory by some calculations.
How can we get core dump of the process killed
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.Sign up to join this community
Even if your kernel was configured to generate a core dump, the OOM killer would not cause a core dump to be produced because it sends a SIGKILL.
As decribed in signal's man page (7), SIGKILL does not produce a core dump.
The only way to change that behavior would therefore require a custom patch to the kernel.
Configured vm.panic_on_oom=2 in /etc/sysctl.conf, which solves my problem.
This enables or disables panic on out-of-memory feature.
If this is set to 0, the kernel will kill some rogue process, called oom_killer. Usually, oom_killer can kill rogue processes and system will survive.
If this is set to 1, the kernel panics when out-of-memory happens. However, if a process limits using nodes by mempolicy/cpusets, and those nodes become memory exhaustion status, one process may be killed by oom-killer. No panic occurs in this case. Because other nodes' memory may be free. This means system total status may be not fatal yet.
If this is set to 2, the kernel panics compulsorily even on the above-mentioned. Even oom happens under memory cgroup, the whole system panics.
The default value is 0. 1 and 2 are for failover of clustering. Please select either according to your policy of failover. panic_on_oom=2+kdump gives you very strong tool to investigate why oom happens. You can get snapshot.