I encountered this strange behavior yesterday on one of our servers.
htop (on startup) were very slow.
strace ps showed that
read('/proc/$pid/cmdline) took several seconds on some processes. Why did this happen?
- The processes executable was on NFS
- The processes (about 20+) were doing
symlinkoperations on files also on NFS, in parallel
- They're forked from the same parent process
- There're 80GB of RAM available (mostly cached), but swap (only 4GB) is in full use
- I run
while true; do cat /proc/$pid/status; sleep .1; done,
catreturned immediately if
R, but took several seconds when
I did some Google'ing and found some SO answers suggesting that when
/proc/$pid/cmdline would stall. Is that true? And how does that work? Why was
/proc/$pid/cmdline, which was set before the program started, affected by what it was doing after that?