Is there a way, I can find the memory leak of a running process? I can use Valgrind for finding memory leaks before the start of a process. I can use GDB to attach it to a running process. How could I debug a memory leaks of a running process?
I think memleax is exact what you want.
It debugs memory leak of a running process by attaching it, without recompiling program or restarting target process. It's very convenient and suitable for production environment.
It works on GNU/Linux and FreeBSD.
NOTE: I'm the author, any suggestion is welcomed
== EDIT ==
I write another tool libleak, which hooks memory functions by LD_PRELOAD.
There is also no need to modify the target program. Although you have to restart the progress with LD_PRELOAD, you can enable/disable the detection during running.
There is much less impact on performance since no signal trap.
Compared with similar tools (such as mtrace), it print the full call-stack at suspicious memory leak point.
Here is almost guarantee steps to find who is leaking the memory:
Find out the PID of the process which causing memory leak.
/proc/PID/smapsand save into some file like
- wait till memory gets increased.
- capture again
/proc/PID/smapsand save it has
find the difference between first
smaps, e. g. with
diff -u beforeMemInc.txt afterMemInc.txt
note down the address range where memory got increased, for example:
beforeMemInc.txt afterMemInc.txt --------------------------------------------------- 2b3289290000-2b3289343000 2b3289290000-2b3289343000 #ADDRESS Shared_Clean: 0 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Clean: 0 kB Private_Dirty: 28 kB Private_Dirty: 36 kB Referenced: 28 kB Referenced: 36 kB Anonymous: 28 kB Anonymous: 36 kB #INCREASE MEM AnonHugePages: 0 kB AnonHugePages: 0 kB Swap: 0 kB Swap: 0 kB KernelPageSize: 4 kB KernelPageSize: 4 kB MMUPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB Locked: 0 kB VmFlags: rd wr mr mw me ac VmFlags: rd wr mr mw me ac
use GDB to dump memory on running process or get the coredump using
gcore -o process
I used gdb on running process to dump the memory to some file.
gdb -p PID dump memory ./dump_outputfile.dump 0x2b3289290000 0x2b3289343000
hexdump -Cto print the
You get readable form where you can locate those strings into your source code.
Analyze your source to find the leak.
I think without providing support for allocation monitoring after program start directly in the source code, you're out of luck. Here are two reasons I can think of:
- Heap checkers initialize when the program begins. Some offer the ability to tweak the exact timing, but the environment variables that start them must be set when the program runs. This is because they watch to make sure each allocation has a corresponding deallocation, and they would miss some otherwise.
- Heap checking usually requires elevated privileges, or hooks, to be provided by the operating system. If those hooks aren't provided at the time of program start, the heap checkers cannot leverage them. I don't believe OSes provide these privileges after the program in question is started.
If, however, you're program is running inside a virtual machine, that environment may provide support for monitoring allocations. I know Java has several allocation and garbage collection monitoring tools (like visualVM) that attach to running programs or VMs.
IBM's Purify is probably the oldest and most sophisticated tool of all. It will flag the line number in the code which causes the memory leak.