29

I want to monitor memory usage of a process, and I want this data to be logged. Does such a tool exist?

1

7 Answers 7

29

Occasionally when the need arises I just do:

$ top -d 1 -b |grep <process> >>somefile

It's not an elegant solution, but gets the job done if you want the quick crude value to verify your hypothesis.

3
  • 3
    I think it is elegant in it's simplicity. You may want to do grep --line-buffered <process> >>somefile to force grep to output each line without buffering
    – Ott Toomet
    Oct 7, 2017 at 21:04
  • 1
    An explanation of the top arguments: -b sets top to batch mode, suitable for outputting to other processes, and -d sets a delay in seconds for each update. Altogether this command outputs lines from top every 1 second and appends to somefile. Nov 4, 2020 at 23:00
  • 1
    This doesn't seem to append anything to somefile for me. I am using top -d 1 -b | grep 2597 >>somefile, somefile gets created, but is always empty. Any ideas on what's wrong?
    – Dahn
    Feb 25, 2021 at 18:19
12

I have written a script to do exactly this. It basically samples ps at specific intervals, to build up a profile of a particular process. The process can be launched by the monitoring tool itself, or it can be an independent process (specified by pid or command pattern).

0
8

sar (System Activity Reporter) from the sysstat package is your friend in case like these.

Another way would be monitoring combined with historical data, e.g. Munin, pnp4nagios, rrdtools, ...

3
  • 2
    but can sar be focussed on just one process? mostly it seems to monitor the system as a whole Jan 13, 2011 at 15:38
  • 3
    The pidstat command also from the sysstat package provides a fairly nice interface for reporting statistics on single process.
    – Steven D
    Jan 13, 2011 at 22:21
  • @xenoterracide Steven D had the answer. I wasn't aware of this command before.
    – Christian
    Jan 14, 2011 at 7:10
4

Besides the aforementioned sar, I'd recommend atop. It saves a binary log that you can peruse afterwards, and besides memory saves a lot of other information.

2

You could try Valgrind.

Valgrind is an instrumentation framework for building dynamic analysis tools. There are Valgrind tools that can automatically detect many memory management and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools.

The Valgrind distribution currently includes six production-quality tools: a memory error detector, two thread error detectors, a cache and branch-prediction profiler, a call-graph generating cache and branch-prediction profiler, and a heap profiler.

2
  • Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference.
    – saji89
    Jan 3, 2013 at 8:36
  • This document massif explain in detail.
    – Shawn Xie
    Feb 7, 2013 at 7:23
1

I like Sridhar's simple answer but I rolled my own before I tried his:

#! /usr/bin/python3

import json, psutil, datetime, sys, time

while True:
  print(json.dumps((datetime.datetime.now().isoformat(),
                    psutil.Process(int(sys.argv[1])).memory_info()._asdict())))
  time.sleep(1)

It's basically only useful if you want structured output. Note: _asdict() was broken in some versions of Python 3.5; it works again in Python 3.6.9 though (as well as in Python 2.7.17).

Output looks like:

["2019-03-19T11:21:53.784670", {"rss": 220389376, "vms": 538984448, "shared": 15724544, "text": 303104, "lib": 0, "data": 221364224, "dirty": 0}]
["2019-03-19T11:21:54.786136", {"rss": 220438528, "vms": 539119616, "shared": 15724544, "text": 303104, "lib": 0, "data": 221499392, "dirty": 0}]
["2019-03-19T11:21:55.787555", {"rss": 220495872, "vms": 539119616, "shared": 15724544, "text": 303104, "lib": 0, "data": 221499392, "dirty": 0}]
["2019-03-19T11:21:56.788754", {"rss": 220528640, "vms": 539119616, "shared": 15724544, "text": 303104, "lib": 0, "data": 221499392, "dirty": 0}]

For me, it was important to have structured output so that I could more easily consume it for analysis.

1
1

I want to monitor memory usage of a process, and I want this data to be logged. Does such a tool exist?

Yes, can achieve exactly this, a bit more with Procpath (author here).

In the simplest case you point it to the PID of interest and let it record its Procfs metrics including Resident Set Size (RSS). By default it makes a recording per 10 seconds (for the target process, its ancestors and descendants).

$ procpath record -d tgt.sqlite -p $PID

When you think it has recorded enough datapoints, Ctrl+C it and this will give you the recordings.

$ sqlite3 tgt.sqlite "SELECT ts, stat_rss * 4 rss_kib FROM record WHERE stat_pid = $PID"
1635349330.11141|478460
1635349340.12125|476068
1635349350.13124|480232
1635349360.14117|483416
...

Using a SQLite GUI can be a more convenient to explore and query the database (e.g. apt install sqlitebrowser).

Procpath can also visualise RSS of the target process into an SVG out of the box like this:

$ procpath plot -d tgt.sqlite -f tgt.svg -q rss -p $PID

resident set size plot

If you're running Kernel 4.14+ you can get more advanced Linux memory usage metrics like Proportional Set Size (PSS), Unique Set Size (USS), proportional swap usage and others (see the documentation for what you can get from /proc/{pid}/smaps_rollup):

procpath record -d tgt.sqlite -p $PID -f stat,cmdline,smaps_rollup

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.