I run a CentOS 5.7 machine (64bit) with 24GB ram and 4x SAS drives in RAID10 setup. This machine runs nginx/1.0.10, php-fpm & xcache. About a month back the RAM usage of this machine has changed.

About every few hours the 'CACHE' is flushed from the RAM, this happens exactly when the 'Inode table usage' drops. I'm pretty sure these drops are related. (see the 2 attached images).

enter image description here enter image description here

This server hosts quite a lot of small files (20M all a few KB big). Not many files are deleted (maybe 100 per hour (total size a few MB max)), not enough to account for the huge Inode table drops.

I also have no crons running which could cause these drops.

Sar -r output: http://pastebin.com/C4D0B79i

My question: Why are these huge RAM/Inode usage drops happening? How can I get Nginx/PHP to use all of my servers RAM?

  • 1
    Are you unmounting a filesystem? Like a snapshot you made to backup, then discard? – psusi Feb 7 '12 at 0:26
  • Hi, no I run no such process during these drops. – Mr.Boon Feb 7 '12 at 8:00
  • Some extra data: I have put my configs here: pastebin.com/iEWJchc4 and the output of LSOF here: hostlogr.com/lsof.txt. The thing i do notice the VERY large number of php-fpm processes that go to /dev/zero. Which is specified in my xcache configuration. Could that possibly be wrong? – Mr.Boon Feb 7 '12 at 8:33
  • The only 3 ways I can think of for the cache to drop like that are 1) deleting files, 2) Unmounting a fs, and 3) A process has opened the block device directly, used it a bunch to load up the cache, then closes the fd, causing the cache to be discarded. – psusi Feb 7 '12 at 15:19
  • Thank you, any way I can detect which of the 3 is actually happening? – Mr.Boon Feb 8 '12 at 8:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.