4

Looking for the largests among the open files by all processes. lsof already has the open files with their sizes. It may be passing the right parameters to lsof and processing the output.

3

You can use the -F option of lsof to get almost unambiguous output which is machine-parseable with only moderate pain. The output is ambiguous because lsof rewrites newlines in file names to \n.

The lsof output consists of one field per line. The first character of each name indicates the field type and the rest of the line is the field value. The fields are: p=PID (only for the first descriptor in a given process), f=descriptor, t=type (REG for regular files, the only type that has a size), s=size (only if available), n=name. The awk code below collects entries that have a size and prints the size and the file name. The rest of the pipelines sorts the output and retains the entry with the largest size.

lsof -Fnst | awk '
    { field = substr($0,1,1); sub(/^./,""); }
    field == "p" { pid = $0; }
    field == "t" { if ($0 == "REG") size = 0; else next; }
    field == "s" { size = $0; }
    field == "n" && size != 0 { print size, $0; }
' | sort -k1n -u | tail -n42 | sed 's/^[0-9]* //'
  • Could you briefly mention what is happening in the awk program ? Specifically, why do you need sub(/^./,""); ? – Hakan Baba Jul 31 '17 at 17:41
  • @HakanBaba See my edit. – Gilles Aug 1 '17 at 13:46
  • I understand that with sub(/^./,""); you remove the first character after saving it to field. – Hakan Baba Aug 1 '17 at 17:44
  • Would you consider unifying while sorting? (sort -k1n -u) This way no duplicate files will be reported in the output. Otherwise if a large file is opened by multiple processes,threads, that is all you see in the output. – Hakan Baba Aug 1 '17 at 17:48
3

You can do the following

lsof | grep REG | awk '{ print $1,$7,$9 }' | sort -t ' ' -k 2 -V

Using the awk you filter the output to include command, size and filename and sort it based on the 2nd column, which is size. -t specifies the delimiter, -V sort 'naturally' - so 1, 2, 10 will be sorted this way instead of 1, 10, 2. -k is the key for the sort (the column you want to sort by)

  • I have found the lsof output quite irregular to parse. In my answer above, I have an example where some REG files have ThreadId specified also some other REG files are "Linux map files that have been deleted". For them the size is not the 7th column. In my answer I attempted to get to size by counting backwards from the end of the line. There I had to consider deleted files separately since they add one more column at the end. – Hakan Baba Jul 30 '17 at 7:31
2

One convoluted way looks like this:

lsof \
| grep REG \
| grep -v "stat: No such file or directory" \
| grep -v DEL \
| awk '{if ($NF=="(deleted)") {x=3;y=1} else {x=2;y=0}; {print $(NF-x) "  " $(NF-y) } }'  \
| sort -n -u  \
| numfmt  --field=1 --to=iec

Output

...
....
129M  /var/log/maillog
166M  /var/log/nginx/access_log
172M  /var/log/metrics/kubernetes/kubelet.log
185M  /var/log/metrics/kubernetes/etcd.log
257M  /var/log/metrics/kubernetes/etcd.log.1
335M  /var/log/metrics/kubernetes/kubelet.log.1

I know this is not perfect. For example if the file name contains "DEL" this would purge that file from the output list. lsof also has a -F option described in the OUTPUT FOR OTHER PROGRAMS section. Using that may be simpler.

Details

lsof prints something like this:

COMMAND      PID    TID           USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME
systemd        1                  root  cwd       DIR              253,0      4096        128 /
tuned       2975                  root    7u      REG              253,0      4096  805307770 /tmp/ffiKkVeXD (deleted)
python2    49888  49890           root  DEL       REG               0,18            196039884 /dev/shm/sem.NXPFow
systemd        1                  root  mem       REG              253,0     90664      10063 /usr/lib64/libz.so.1.2.7
java      149435 175229            box   69r      REG              253,0 350872273  808108999 /box/var/log/metrics/kubernetes/kubelet.log.1
java      149435 149580            box  107w     FIFO                0,8       0t0  272526226 pipe
prometheu 147867 148211           root  mem       REG              253,6             31457463 /lib64/ld-2.12.so (stat: No such file or directory)

grep REG keep regular files


grep -v "stat: No such file or directory" Remove the files that have stat error. (I do not know why this happens)


grep -v DEL Discard the Linux map files that have been deleted;

From lsof documentation:

''DEL'' for a Linux map file that have been deleted;


After this processing we are left with something like this:

tuned       2975                  root    7u      REG              253,0      4096  805307770 /tmp/ffiKkVeXD (deleted)
systemd        1                  root  mem       REG              253,0     90664      10063 /usr/lib64/libz.so.1.2.7
java      149435 175229            box   69r      REG              253,0 350872273  808108999 /box/var/log/metrics/kubernetes/kubelet.log.1

The size is either the 3rd 2nd column from last depending on the value of the last column. If the last column is (deleted) pick the 3rd from last otherwise 2nd.

awk '{if ($NF=="(deleted)") {x=3;y=1} else {x=2;y=0}; {print $(NF-x) " " $(NF-y) } }'


sort -n -u | numfmt --field=1 --to=iec

Sort, uniquify and make the bytes count human readable

  • My lsof version is 4.87 – Hakan Baba Jul 30 '17 at 7:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.