8

I've recently been trying to create a shell script for a minor project, and for some reason, the flock command isn't working for me properly. Whenever I invoke it in a subshell the atomic way and put it into the background, other programs seem to be able to read from/write to the locked file.

Bash session:

guest@guest ~ $ touch ./temp
guest@guest ~ $ ( flock -x 3 && sleep 99999999999; ) 3>./temp &
[1] 22874
guest@guest ~ $ cat ./temp 
guest@guest ~ $ echo this is a test >./temp 
guest@guest ~ $ cat ./temp 
this is a test
guest@guest ~ $ jobs
[1]+  Running                 ( flock -x 3 && sleep 99999999999 ) 3>./temp &
guest@guest ~ $ bash --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I feel like I might be missing something very simple relating to the internals of the 'flock' program, but don't know what it could be.

1 Answer 1

10

flock does advisory locking, which is a cooperative locking scheme. This means that you will be able to override the lock if you don't cooperate. You cooperate by requesting the lock before doing the operation, and then by releasing the lock after you're done. It's the operation that is protected by the lock, not (necessarily) the lock file itself.

From the flock(2) manual on my system:

Advisory locks allow cooperating processes to perform consistent operations on files, but do not guarantee consistency (i.e., processes may still access files without using advisory locks possibly resulting in inconsistencies).

Consider this script:

#!/bin/sh

( flock -x 9 || exit 1
  echo '1: Locking for 5 secs'; sleep 5; echo '1: Done' ) 9>/tmp/lock &

sleep 1
echo '2: Will now attempt to get lock'

( flock -x 9 || exit 1
  echo '2: Got lock' ) 9>/tmp/lock

# Since the second flock call only performs one operation, the whole last 
# subshell may be replaced by just
#    flock -x /tmp/lock -c echo '2: Got lock'
#
#  (-x and -c are not needed, a lock is exclusive ("write lock")
#   unless -s is used to create a shared lock ("read lock"),
#   and the -c is optional)

Output:

1: Locking for 5 secs
2: Will now attempt to get lock
1: Done
2: Got lock

You can see that the lock was acquired by the background process and that the other flock invocation had to wait for it to be released before being able to lock it.

Also note that the lock file is not what's protected here, it's the echo operations in the subshells that are guaranteed to be exclusive. In particular, the lock file is not protected from uncooperative processes that write or read from it.

This means that each flock subshell, by locking /tmp/lock in this example, is guaranteed that operations (on files or other shared data resources) will not be intermingled with conflicting operations from any other program that uses flock with /tmp/lock as the locking file.

To illustrate the last paragraph above, run my script above in two different terminals (possibly with slightly increased sleep time), as simultaneously as possible, and verify that the two competing scripts acquire the locks appropriately (waiting for each other). Since one lock is requested in a background process, this means that the locks might be acquired out of order from what specified in the script when running two instances of the script concurrently.

In your example, the interactive shell is not cooperating with the locking mechanism. This is why you're able to read from and write to the file even though the lock is held by the backgrounded subshell.

Note also that not all filesystems may support file locking with flock (or its C library equivalent, flock()). For example, the network filesystems AFS and NFS may be problematic in this regard. See https://en.wikipedia.org/wiki/File_locking#Problems

1
  • Ugh, the unfortunate truths... Commented Aug 26, 2017 at 6:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .