Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to run multiple Bash shell scripts in parallel. However, I want to avoid race conditions. What Unix commands are truly atomic that I could use for this purpose, and how can I use them?

share|improve this question
    
What are you doing that requires parallel work? Can't you express the dependencies in a way that allows a parallel make(1) to take over? (i.e., do a make -j 9 if you have 8 cores)? This has the added advantage of interleaving work with finer granularity. –  vonbrand Mar 15 '13 at 14:50
add comment

5 Answers

up vote 16 down vote accepted

If lockfile is not installed on your system, then mkdir will do the work: it's an atomic operation, and it fails if the directory is already existing (as long as you don't add the -p command-line switch).

create_lock_or_wait () {
  path="$1"
  wait_time="${2:-10}
  while true; do
        if mkdir "${path}.lock.d"; then
           break;
        fi
        sleep $wait_time
  done
}

remove_lock () {
  path="$1"
  rmdir "${path}.lock.d"
}
share|improve this answer
add comment

flock(1)

#!/bin/bash

# Makes sure we exit if flock fails.
set -e

(
  # Wait for lock on /var/lock/.myscript.exclusivelock (fd 200) for 10 seconds
  flock -x -w 10 200

  # Do stuff

) 200>/var/lock/.myscript.exclusivelock

This ensures that code between "(" and ")" is run only by one process at a time and that the process does wait for a lock too long.

share|improve this answer
    
Nice one, didn't know about it. However, it's apparently Linux-specific... –  Riccardo Murri Sep 7 '10 at 19:52
1  
@Riccardo, FreeBSD has a similar command: lockf(1). –  Alex B Sep 8 '10 at 11:08
add comment

lockfile(1) looks like a good candidate, though beware that it's part of the procmail package, which you may not have installed on your machine yet. It's a popular enough package that it should be packaged for your system if it's not installed yet. Three of the four systems I checked have it, and the other has it available.

Using it is simple:

#!/bin/sh
LOCKFILE=$HOME/.myscript/lock
mkdir -p `dirname $LOCKFILE`

echo Waiting for lock $LOCKFILE...
if lockfile -1 -r15 $LOCKFILE
then
    # Do protected stuff here
    echo Doing protected stuff...

    # Then, afterward, clean up so another instance of this script can run
    rm -f $LOCKFILE
else
    echo "Failed to acquire lock!  lockfile(1) returned $?"
    exit 1
fi

The options I've given make it retry once a second for up to 15 seconds. Drop the "-r" flag if you want it to wait forever.

share|improve this answer
1  
Just for reference - the man page: linux.die.net/man/1/lockfile. :) –  Lucas Jones Aug 10 '10 at 21:50
add comment

The system call mkdir() is atomic on POSIX filesystems. So, using the mkdir command in such a way that it involves exactly one call to mkdir() would achieve your purpose. (IOW, don't use mkdir -p). The corresponding unlock is rmdir of course.

Caveat emptor: mkdir() might not be atomic on network filesystems.

share|improve this answer
add comment

Maybe the lockfile command will do what you need.

lockfile ~/.config/mylockfile.lock
.....
rm -f important.lock
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.