38

I want to run multiple Bash shell scripts in parallel. However, I want to avoid race conditions. What Unix commands are truly atomic that I could use for this purpose, and how can I use them?

34

If lockfile is not installed on your system, then mkdir will do the work: it's an atomic operation, and it fails if the directory already exists (as long as you don't add the -p command-line switch).

create_lock_or_wait () {
  path="$1"
  wait_time="${2:-10}"
  while true; do
        if mkdir "${path}.lock.d"; then
           break;
        fi
        sleep $wait_time
  done
}

remove_lock () {
  path="$1"
  rmdir "${path}.lock.d"
}
|improve this answer|||||
29

flock(1)

#!/bin/bash

# Makes sure we exit if flock fails.
set -e

(
  # Wait for lock on /var/lock/.myscript.exclusivelock (fd 200) for 10 seconds
  flock -x -w 10 200

  # Do stuff

) 200>/var/lock/.myscript.exclusivelock

This ensures that code between "(" and ")" is run only by one process at a time and that the process does wait for a lock too long.

|improve this answer|||||
  • Nice one, didn't know about it. However, it's apparently Linux-specific... – Riccardo Murri Sep 7 '10 at 19:52
  • 1
    @Riccardo, FreeBSD has a similar command: lockf(1). – Alex B Sep 8 '10 at 11:08
  • lockf(1) doesn't work in the way used in this example, though. It can't take a file descriptor number as an argument. – Charley Oct 28 '16 at 7:36
12

lockfile(1) looks like a good candidate, though beware that it's part of the procmail package, which you may not have installed on your machine yet. It's a popular enough package that it should be packaged for your system if it's not installed yet. Three of the four systems I checked have it, and the other has it available.

Using it is simple:

#!/bin/sh
LOCKFILE=$HOME/.myscript/lock
mkdir -p `dirname $LOCKFILE`

echo Waiting for lock $LOCKFILE...
if lockfile -1 -r15 $LOCKFILE
then
    # Do protected stuff here
    echo Doing protected stuff...

    # Then, afterward, clean up so another instance of this script can run
    rm -f $LOCKFILE
else
    echo "Failed to acquire lock!  lockfile(1) returned $?"
    exit 1
fi

The options I've given make it retry once a second for up to 15 seconds. Drop the "-r" flag if you want it to wait forever.

|improve this answer|||||
  • 2
    Just for reference - the man page: linux.die.net/man/1/lockfile. :) – Lucas Jones Aug 10 '10 at 21:50
  • Be aware that (according to the manpage), "Once a file is locked, the lock must be touched at least once every five minutes or the lock will be considered stale, and subsequent lock attempts will succeed." – Jay Jan 5 '19 at 19:19
7

The system call mkdir() is atomic on POSIX filesystems. So, using the mkdir command in such a way that it involves exactly one call to mkdir() would achieve your purpose. (IOW, don't use mkdir -p). The corresponding unlock is rmdir of course.

Caveat emptor: mkdir() might not be atomic on network filesystems.

|improve this answer|||||
  • is rmdir therefore also atomic? – Alexej Magura May 11 '17 at 23:20
3

Maybe the lockfile command will do what you need.

lockfile ~/.config/mylockfile.lock
.....
rm -f important.lock
|improve this answer|||||
  • This seems to delete the wrong file. – Benjamin W. Jul 13 '19 at 19:19
1

If you're only running on Unix, use fifos. You can write work records to the fifo and have processes read from this file and your readers will block on the fifos.

Lock files are ok, but for what you describe I would go with fifos

|improve this answer|||||
0

As posted here: " Correct locking in shell scripts? ", using FLOM (Free LOck Manager) tool, serializing commands and shell scripts becomes as easy as running

flom -- command_to_serialize

FLOM allows you to implement more sofisticate use cases (distributed locking, readers/writers, numeric resources, etc...) as explained here: http://sourceforge.net/p/flom/wiki/FLOM%20by%20examples/

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.