1

I'm trying to check how much memory strain a process is actually putting on the system, but ps, top and friends are almost useless for that purpose as they only report 3 statistics:

  • RES - the resident memory set includes only data pages that in physical memory (not including swapped out pages) but also includes loaded shared libraries.
  • VIRT - includes all pages mapped to the memory by the kernel, including swapped out pages but also memory mapped files, shared libraries, etc.
  • SHR - possibly the most useless of all, includes just the memory used by libraries that can be shared, but as I understand it, it does not actually account for memory used by the process but counts the entire size of the library even if only part of it is actually resident.

In multi-process computing software, I want to know how much memory will be used/freed by running or killing another process with a similar or identical shared memory/libraries as existing ones, which means I need to know how large is the data set used by the process - including all swapped out data pages but excluding all non-data pages, such as shared libraries, shared memory pages, memory mapped files, etc.

I'm not afraid of some coding but it will be better if there's already a top replacement I'm not aware of that shows that information.

2

Take a look at this script https://github.com/pixelb/scripts/commits/master/scripts/ps_mem.py which we are using regularly to debug our applications. It is not a simple task and the methods differ from kernel to kernel sometimes.

From the description of the script you can read the following.

# Try to determine how much RAM is currently being used per program.
# Note per _program_, not per process. So for example this script
# will report RAM used by all httpd process together. In detail it reports:
# sum(private RAM for program processes) + sum(Shared RAM for program processes)
# The shared RAM is problematic to calculate, and this script automatically
# selects the most accurate method available for your kernel.
  • You probably meant:github.com/pixelb/ps_mem :-) Does this summary includes memory mapped files? does "shared" includes both shared library and shared data pages? – Guss Oct 1 '14 at 13:07
1

I figured you can analyze the /proc/ID/maps file for each process in question - if you list all the mapped pages, discard all executable pages, shared pages and pages that are not mapped to an inode. If you then sum up their sizes (which can be computed from the beginning and ending addresses) then the result is the actual memory pressure of the process.

I got the following PoC ruby code that does this:

sudo ruby -le '
  puts $<.read.split("\n").collect{|l|l.split(/\s+/)}. # create data records
    select{|r|r[4].to_i==0&&r[1]!~/x|s/}. # remove mapped, exec and shared pages
    collect{|r|b,e=r[0].split(/-/,2).collect{|a|a.to_i(16)};r[0]=e-b;r}. # size
    inject(0){|s,r|s+=r[0]} # sum
  ' /proc/17099/maps

of course running this for actual analysis is going to be tedious at best.

  • Replacing 17099 with $(pgrep PROCESS_NAME | head -1) (e.g. $(pgrep firefox | head -1) would make it a bit quicker for finding the PID of single instances of programs as you don't have to find the PID first. Also, do you need to run the script as root with sudo? – Wilf Oct 1 '14 at 12:01
  • Thanks for the pgrep suggestion. At least on my system, a process' proc entries cannot be read without root permissions. – Guss Oct 1 '14 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.