23

The Perl 5.x documentation states that its implementation of flock(..) will use one of the following native calls, starting at 1 and working toward 3 if unavailable:

  1. flock(2)
  2. fcntl(2)
  3. lockf(3)

That's fine. However, you may have noticed their disclaimer that flock(2) should not be used over a NFS. The doc suggests using a -Ud_flock flag to force Perl to use flock(2). The man page of flock(2) (on Redhat) states a similar disclaimer about NFS issues.

My question is, why!?!? I can't seem to find an in-depth article or explanation of WHY flock(2) is unsafe over a NFS.

I've written several test scripts in C and Perl, on both Redhat (where flock(2) is being used) and on Solaris (where fcntl(2) is being used). I ran strace/truss to ensure that Perl was indeed using flock(2) and fcntl(2) respectively. I could not replicate any issues where a lock was not being honored! What gives??

4 Answers 4

18

I'm pretty sure that you are looking at legacy concerns. Recall that the Perl5 manual was released in 1994 and that it was just an edit of Perl4's manual from 1991. In those days it could probably be said about the oft-named Nightmare File System that "it isn't how well the bear dances that amazes, but that it dances at all".

NFS2 in the 1991 epoch was slowly crawling out of Sun into other platforms and was relatively crude. The security model was essentially non-existent (root on a client machine could read the full contents of an NFS mount) and locking - via nfs.lockd - was this side of experimental. You would have been foolish to expect flock semantics to work properly if at all between two different allegedly interoperable implementations. Coax was the dominant Ethernet PHY at the time which many network users have never had the displeasure of using (what do you mean you forgot to put the 50š¯›€ termination resistor on?) if that gives you a better grip on the state of intranets then.

Larry Wall and crew had every reason to make pessimistic assumptions about the correctness of NFS locks at the time, and this is the sort of defensive programming that future code jockeys are loathe to remove because it is so hard to prove the absence of a defect by removing old code which is re-introduced in interoperability with a legacy system that you never even heard of.

Since then, NFS has improved considerably, and lockd has migrated in time to a feature of the Linux 2.6 kernel. For a collection of 2003+ systems, NFS file locking can probably be trusted, especially if tested well within your application across the many platforms it may be running on.

All of the above was cribbed from memory, and could likely be substantiated through research (e.g. http://nfs.sourceforge.net/) but the proof - as they say - is in the locking, and if you didn't test it, it is presumed broken.

2
  • That's a great analysis. In fact, I've come to the same conclusion so far. I read over the nfs sourceforge page again after you posted that link, and I've finally found what I'm looking for! Here is an in-depth analysis directly from the horse's mouth!
    – Jmoney38
    Sep 8, 2010 at 12:49
  • 2
    oops, I hit enter... go to nfs.sourceforge.net, section D10 towards the bottom discusses this issue in detail.
    – Jmoney38
    Sep 8, 2010 at 12:50
5

This is out of date now. NFS4 supports locking inside the protocol (no lockd daemon or RPC callback mechanism is required) and Perl's flock() method works fine - we are using it in production.

Very old versions of the kernel implemented flock (the syscall) as a no-op on NFS, and other stuff like byte range locking wasn't properly supported. This is where the hysteria comes from.

2
4

Lennart Poettering recently did some digging into linux filesystem locking behaviour, which doesn't paint a particularly rosy picture for locking over NFS (especially the follow-up he links to at the bottom of the post).

http://0pointer.net/blog/projects/locking.html

1
  • 2
    That is the exact type of information I was looking for. Thank you! After several weeks of investigation, it's a very similar resolution that I came to, but it's great to read an article that confirms my suspicions (and suggests others). The link from that page's comments was also a good reference, and a good article about POSIX and its history): samba.org/samba/news/articles/low_point/tale_two_stds_os2.html
    – Jmoney38
    Sep 14, 2010 at 2:07
3

Another one, straight from Linux-NFS FAQ: nfs.sf.net

I'm trying to use flock()/BSD locks to lock files used on multiple clients, but the files become corrupted. How come? A. flock()/BSD locks act only locally on Linux NFS clients prior to 2.6.12. Use fcntl()/POSIX locks to ensure that file locks are visible to other clients.

Here are some ways to serialize access to an NFS file.

Use the fcntl()/POSIX locking API. This type of locking provides byte-range locking across multiple clients via the NLM protocol, or via NFSv4. Use a separate lockfile, and create hard links to it. See the description in the O_EXCL section of the creat(2) man page. It's worth noting that until early 2.6 kernels, O_EXCL creates were not atomic on Linux NFS clients. Don't use O_EXCL creates and expect atomic behavior among multiple NFS client unless you are running a kernel newer than 2.6.5.

It's a known issue that Perl uses flock()/BSD locking by default. This can break programs ported from other operating systems, such as Solaris, that expect flock/BSD locks to work like POSIX locks.

On Linux, using file locking instead of a hard link has the added benefit of checkpointing the client's cache with the server. When a file lock is acquired, the client will flush the page cache for that file so that any subsequent reads get new data from the server. When a file lock is released, any changes to the file on that client are flushed back to the server before the lock is released so that other clients waiting to lock that file can see the changes.

The NFS client in 2.6.12 provides support for flock()/BSD locks on NFS files by emulating the BSD-style locks in terms of POSIX byte range locks. Other NFS clients that use the same emulation mechanism, or that use fcntl()/POSIX locks, will then see the same locks that the Linux NFS client sees.

On local Linux filesystems, POSIX locks and BSD locks are invisible to one another. Thus, due to this emulation, applications running on a Linux NFS server will still see files locked by NFS clients as being locked with a fcntl()/POSIX lock, whether the application on the client is using a BSD-style or a POSIX-style lock. If the server application uses flock()BSD locks, it will not see the locks the NFS clients use.

3
  • So do two NFS clients running kernel 3.13.* see each other's flock()s? Aug 17, 2016 at 16:11
  • If I'm understanding correctly, the answer is no. Unless I missed something, flock has not, does not, and will not lock across nfs mounts.
    – erik258
    Oct 25, 2017 at 0:34
  • It does, at least on NFS4.
    – rjh
    Mar 21, 2018 at 9:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .