14

Related to this question on StackOverflow, I am wondering if there is a way for me to flush the NFS cache / force Linux to see the most up-to-date copy of a file that's on an NFS share.

I have a situation where four Apache servers mount the same directory via NFS, and when one server make a change to a file, it takes about 5 - 10 seconds for the other servers to see that change. If a second change is made to that file within this window, it may overwrite the first change.

The fstab entry for the filesystem is:

172.16.1.15:/home               /media/home     nfs     vers=3,defaults,noauto,sync,acregmin=1          0       0

Is there a command which will force Linux to flush cache of one file on an NFS share?

  • What apache caching mechanism you using? – usermane Oct 27 '11 at 17:18
  • @usermane: I'm not using an Apache caching mechanism. The files in question are PHP files, and when they're modified on one host, the other hosts don't see that change for a few seconds. – Josh Oct 27 '11 at 19:47
12

Check these items, and see if any work for you:

  1. On the client, if you're not already using the cto option in the options column of the /etc/fstab line for your NFS filesystem, add it. cto tells the nfs client to open files via close-to-open, which makes them refresh the file whenever they open it.

  2. On the server, make sure your filesystem is exported with the sync option, and not async. With synchronous writes, the client will flush to disk when the file is closed. (or with nfs V3, whenever a commit() is made). There may be a performance hit that way, but if you're doing writes to an NFS filesystem, you definitely want sync set.

  3. Following on the heels of that stackoverflow post, opening file with O_DIRECT works only if the kernel was compiled with CONFIG_NFS_DIRECTIO.

  4. Also, make sure you have the following settings in your httpd.conf file:

    • EnableMMAP off
    • EnableSendfile off

    From the apache performance tuning documentation:

    • If you memory-map a file located on an NFS-mounted filesystem and a process on another NFS client machine deletes or truncates the file, your process may get a bus error the next time it tries to access the mapped file content.
    • Turning off EnableSendfile won't specifically help with the sync/async behavior of NFS, but it needs to be turned off if you're using apache with NFS.
  • Thanks! These are great suggestions. I am not using cto on the client and I will try that. I don't have either sync nor async on the server; I just added sync. O_DIRECT didn't help me because it's actually PHP that's reading these files and it can't open them with O_DIRECT in any way I can find. – Josh Oct 27 '11 at 20:29
  • @Josh, did it solve your problem? We are blocked on the same issue! Could you please update this post? – rafa.ferreira Aug 28 '14 at 16:35
3

Within a given process, calling opendir and closedir on the parent directory of a file invalidates the NFS cache. I used this while programming a job scheduler. Very, very helpful. Try it!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.