I encountered this strange behavior yesterday on one of our servers. ps, pgrep and htop (on startup) were very slow. strace ps showed that read('/proc/$pid/cmdline) took several seconds on some processes. Why did this happen?

Some observations:

  • The processes executable was on NFS
  • The processes (about 20+) were doing unlink and symlink operations on files also on NFS, in parallel
  • They're forked from the same parent process
  • There're 80GB of RAM available (mostly cached), but swap (only 4GB) is in full use
  • I run while true; do cat /proc/$pid/status; sleep .1; done, cat returned immediately if State is S or R, but took several seconds when State is D

I did some Google'ing and found some SO answers suggesting that when State is D, reading /proc/$pid/cmdline would stall. Is that true? And how does that work? Why was /proc/$pid/cmdline, which was set before the program started, affected by what it was doing after that?

  • Could you post your kernel version and the contents of /proc/$PID/stack of the hanging cat process? – Danila Kiver Jun 30 '18 at 9:05
  • I upvote for the question AND the links, very useful. – Massimo Sep 14 '18 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.