What Unix commands can be used as a semaphore/lock?

Question

I want to run multiple Bash shell scripts in parallel. However, I want to avoid race conditions. What Unix commands are truly atomic that I could use for this purpose, and how can I use them?

Answer 1

If lockfile is not installed on your system, then mkdir will do the work: it's an atomic operation, and it fails if the directory is already existing (as long as you don't add the -p command-line switch).

create_lock_or_wait () {
  path="$1"
  wait_time="${2:-10}
  while true; do
        if mkdir "${path}.lock.d"; then
           break;
        fi
        sleep $wait_time
  done
}

remove_lock () {
  path="$1"
  rmdir "${path}.lock.d"
}

Answer 2

flock(1)

#!/bin/bash

# Makes sure we exit if flock fails.
set -e

(
  # Wait for lock on /var/lock/.myscript.exclusivelock (fd 200) for 10 seconds
  flock -x -w 10 200

  # Do stuff

) 200>/var/lock/.myscript.exclusivelock

This ensures that code between "(" and ")" is run only by one process at a time and that the process does wait for a lock too long.

Answer 3

lockfile(1) looks like a good candidate, though beware that it's part of the procmail package, which you may not have installed on your machine yet. It's a popular enough package that it should be packaged for your system if it's not installed yet. Three of the four systems I checked have it, and the other has it available.

Using it is simple:

#!/bin/sh
LOCKFILE=$HOME/.myscript/lock
mkdir -p `dirname $LOCKFILE`

echo Waiting for lock $LOCKFILE...
if lockfile -1 -r15 $LOCKFILE
then
    # Do protected stuff here
    echo Doing protected stuff...

    # Then, afterward, clean up so another instance of this script can run
    rm -f $LOCKFILE
else
    echo "Failed to acquire lock!  lockfile(1) returned $?"
    exit 1
fi

The options I've given make it retry once a second for up to 15 seconds. Drop the "-r" flag if you want it to wait forever.

Answer 4

The system call mkdir() is atomic on POSIX filesystems. So, using the mkdir command in such a way that it involves exactly one call to mkdir() would achieve your purpose. (IOW, don't use mkdir -p). The corresponding unlock is rmdir of course.

Caveat emptor: mkdir() might not be atomic on network filesystems.

Answer 5

Maybe the lockfile command will do what you need.

lockfile ~/.config/mylockfile.lock
.....
rm -f important.lock