How to monitor use of Huge Pages per process

Question

I'm trying to determine which process is using a large number of Huge Pages, but I can't find a simple Linux command (like top) to view the Huge Page usage. The best I could find was

$ cat /sys/devices/system/node/node*/meminfo | fgrep Huge
Node 0 HugePages_Total:   512
Node 0 HugePages_Free:    159
Node 0 HugePages_Surp:      0
Node 1 HugePages_Total:   512
Node 1 HugePages_Free:      0
Node 1 HugePages_Surp:      0

which tells me at the granularity of Nodes where the Huge Pages are in use, but I would like to see the Huge Page usage per process. I wouldn't mind iterating over all processes and cating some /sys special device to get this information.

A similiar question here got no reponses: https://stackoverflow.com/q/25731343/364818

I am not running Oracle, btw.

Answer 1

I found a discussion on ServerFault that discusses this. Basically,

$ sudo grep huge /proc/*/numa_maps
/proc/4131/numa_maps:80000000 default file=/anon_hugepage\040(deleted) huge anon=4 dirty=4 N0=3 N1=1
/proc/4131/numa_maps:581a00000 default file=/anon_hugepage\040(deleted) huge anon=258 dirty=258 N0=150 N1=108
/proc/4131/numa_maps:7f6c40400000 default file=/anon_hugepage\040(deleted) huge
/proc/4131/numa_maps:7f6ce5000000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1
/proc/4153/numa_maps:80000000 default file=/anon_hugepage\040(deleted) huge anon=7 dirty=7 N0=6 N1=1
/proc/4153/numa_maps:581a00000 default file=/anon_hugepage\040(deleted) huge anon=265 dirty=265 N0=162 N1=103
/proc/4153/numa_maps:7f3dc8400000 default file=/anon_hugepage\040(deleted) huge
/proc/4153/numa_maps:7f3e00600000 default file=/anon_hugepage\040(deleted) huge anon=1 dirty=1 N0=1

and getting the process name

$ ps 4131
  PID TTY      STAT   TIME COMMAND
 4131 ?        Sl     1:08 /var/lib/jenkins/java/bin/java -jar slave.jar
$ ps 4153
  PID TTY      STAT   TIME COMMAND
 4153 ?        Sl     1:09 /var/lib/jenkins/java/bin/java -jar slave.jar

will give you an idea of what processes are using huge memory.

$ grep HugePages /proc/meminfo
AnonHugePages:   1079296 kB
HugePages_Total:    4096
HugePages_Free:     3560
HugePages_Rsvd:      234
HugePages_Surp:        0

$ sudo ~/bin/counthugepages.pl 4153
273 huge pages
$ sudo ~/bin/counthugepages.pl 4131
263 huge pages

The sum of free pages (3560) plus the pages from the 2 process (273+263) equals 4096. All accounted for!

The perl script to sum the dirty= fields is here:

https://serverfault.com/questions/527085/linux-non-transparent-per-process-hugepage-accounting/644471#644471

Answer 2

Looking through the /proc documentation, I see that huge page usage is recorded in /proc/PID/smaps with the ht flag in VmFlags and (other than file-backed pages) with the AnonHugePages field.

grep '^VmFlags:.* ht' /proc/[0-9]*/smaps

Answer 3

To see the huge pages usage for a given process, run

numastat -p PID

with the relevant process id.

Answer 4

Red Hat recommends this **:

grep -B 11 'KernelPageSize:     2048 kB' /proc/[PID]/smaps \
   | grep "^Size:" \
   | awk 'BEGIN{sum=0}{sum+=$2}END{print sum/1024}'

More at my question on serverfault.

** Copy and paste can be finicky and if you don't input the the proper number of spaces between "KernelPageSize:" and "2048", you may not get proper results. Another, perhaps more reliable way to perform this grep may be to do this:

grep -B 11 -E 'KernelPageSize:\s+2048 kB' /proc/[PID]/smaps \
   | grep "^Size:" \
   | awk 'BEGIN{sum=0}{sum+=$2}END{print sum/1024}'

Your results should concur with the "Huge" line of

numastat -p [PID]

Results are in MiB. [PID] represents an integer; do not include the brackets.