Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Sometimes you have to make sure that only one instance of a shell script is running at the same time.

For example a cron job which is executed via crond that does not provide locking on its own (e.g. the default Solaris crond).

A common pattern to implement locking is code like this:

#!/bin/sh
LOCK=/var/tmp/mylock
if [ -f $LOCK ]; then
  echo Job is already running\!
  exit 6
fi
touch $LOCK
# do some work
rm $LOCK

Of course, such code has a race condition. There is a time window where the execution of two instances can both advance after line 3 before one is able to touch the $LOCK file.

For a cron job this is usually not a problem because you have an interval of minutes between two invocations.

But things can go wrong - for example when the lockfile is on a NFS server - that hangs. In that case several cron jobs can block on line 3 and queue up. If the NFS server is active again then you have thundering herd of parallel running jobs.

Searching on the web I found the tool lockrun which seems like a good solution to that problem. With it you run a script that needs locking like this:

$ lockrun --lockfile=/var/tmp/mylock myscript.sh

You can put this in a wrapper or use it from your crontab.

On current Linux systems you probably want to apply following patch to get working locking via NFS:

--- a/lockrun.c Tue Oct 04 20:39:42 2011 +0200
+++ b/lockrun.c Tue Oct 04 20:40:39 2011 +0200
@@ -97,11 +97,7 @@
                __attribute__((noreturn))
                __attribute__((format(printf, 1, 2)));

-#ifdef __sun
 # define WAIT_AND_LOCK(fd) lockf(fd, F_TLOCK,0)
-#else
-# define WAIT_AND_LOCK(fd) flock(fd, LOCK_EX | LOCK_NB)
-#endif

This should also work on all current UNIX-like systems because lockf is POSIX.

Are there alternatives to lockrun?

What about other cron daemons? Are there common crond's that support locking in a sane way? A quick look into the man page of Vixie Crond (default on Debian/Ubuntu systems) does not show anything about locking.

Would it be a good idea to include a tool like lockrun into coreutils?

In my opinion it implements a theme very similar to timeout, nice and friends.

share|improve this question
2  
Tangentially, and for the benefit of others who may consider your initial pattern Good Enough(tm), that shell code should possibly trap TERM in order to remove its lockfile when killed; and it seems to be good practice to store one's own pid in the lockfile, rather than just touching it. –  Ulrich Schwarz Oct 4 '11 at 19:13
    
possible duplicate of What Unix commands can be used as a semaphore/lock? –  Shawn J. Goff Oct 4 '11 at 20:57
    
@Shawn, not really, does not mention crond and NFS. –  maxschlepzig Oct 4 '11 at 21:02
    
related question on SO: stackoverflow.com/questions/185451/… –  maxschlepzig Oct 4 '11 at 21:07

10 Answers 10

up vote 21 down vote accepted

Here's another way to do locking in shell script that can prevent the race condition you describe above, where two jobs may both pass line 3. The noclobber option will work in ksh and bash. Don't use set noclobber because you shouldn't be scripting in csh/tcsh. ;)

lockfile=/var/tmp/mylock

if ( set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null; then

        trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT

        # do stuff here

        # clean up after yourself, and release your trap
        rm -f "$lockfile"
        trap - INT TERM EXIT
else
        echo "Lock Exists: $lockfile owned by $(cat $lockfile)"
fi

YMMV with locking on NFS (you know, when NFS servers are not reachable), but in general it's much more robust than it used to be. (10 years ago)

If you have cron jobs that do the same thing at the same time, from multiple servers, but you only need 1 instance to actually run, the something like this might work for you.

I have no experience with lockrun, but having a pre-set lock environment prior to the script actually running might help. Or it might not. You're just setting the test for the lockfile outside your script in a wrapper, and theoretically, couldn't you just hit the same race condition if two jobs were called by lockrun at exactly the same time, just as with the 'inside-the-script' solution?

File locking is pretty much honor system behavior anyways, and any scripts that don't check for the lockfile's existence prior to running will do whatever they're going to do. Just by putting in the lockfile test, and proper behavior, you'll be solving 99% of potential problems, if not 100%.

If you run into lockfile race conditions a lot, it may be an indicator of a larger problem, like not having your jobs timed right, or perhaps if interval is not as important as the job completing, maybe your job is better suited to be daemonized.

share|improve this answer
1  
No, with lockrun you don't have a problem - when a NFS server hangs, all lockrun calls will hang (at least) in the lockf() system call - when it is back up all processes are resumed but only one process will win the lock. No race condition. I don't run into such problems with cronjobs a lot - the opposite is the case - but this is a problem when it hits you it has the potential to create a lot of pain. –  maxschlepzig Oct 4 '11 at 20:15
    
Your noclobber method seems to be safe: nfs.sourceforge.net/#faq_d10 and mywiki.wooledge.org/BashFAQ/045#Discussion –  maxschlepzig Oct 4 '11 at 21:11
1  
I have accepted this answer because the method is safe and so far the most elegant one. I suggest a small variant: set -o noclobber && echo "$$" > "$lockfile" to get a safe fallback when the shell does not support the noclobber option. –  maxschlepzig Oct 7 '11 at 9:45
1  
Good answer, but you should also 'kill -0' the value in lockfile to ensure that the process that created the lock still exists. –  Nigel Horne 2 days ago

I prefer to use hard links.

lockfile=/var/lock/mylock
tmpfile=${lockfile}.$$
echo $$ > $tmpfile
if ln $tmpfile $lockfile 2>&-; then
    echo locked
else
    echo locked by $(<$lockfile)
    rm $tmpfile
    exit
fi
trap "rm ${tmpfile} ${lockfile}" 0 1 2 3 15
# do what you need to

Hard links are atomic over NFS and for the most part, mkdir is as well. Using mkdir(2) or link(2) are about the same, at a practical level; I just prefer using hard links because more implementations of NFS allowed atomic hard links than atomic mkdir. With modern releases of NFS, you shouldn't have to worry using either.

share|improve this answer

I understand that mkdir is atomic, so perhaps:

lockdir=/var/tmp/myapp
if mkdir $lockdir; then
  # this is a new instance, store the pid
  echo $$ > $lockdir/PID
else
  echo Job is already running, pid $(<$lockdir/PID) >&2
  exit 6
fi

# then set traps to cleanup upon script termination 
# ref http://www.shelldorado.com/goodcoding/tempfiles.html
trap 'rm -r "$lockdir" >/dev/null 2>&1' 0
trap "exit 2" 1 2 3 13 15
share|improve this answer
    
Ok, but I could not find informations whether mkdir() over NFS (>=3) is standardized to be atomic. –  maxschlepzig Oct 4 '11 at 21:05
    
@maxschlepzig RFC 1813 does not explicitly call out for mkdir to be atomic (it does for rename). In practice, it is known that some implementations are not. Related: an interesting thread, including a contribution by the author of GNU arch. –  Gilles Oct 4 '11 at 22:12

I use the command-line tool "flock" to manage locks in my bash scripts, as described here and here. I have used this simple method from the flock manpage, to run some commands in a subshell...

   (
     flock -n 9
     # ... commands executed under lock ...
   ) 9>/var/lock/mylockfile

In that example, it fails with exit code of 1 if it can't acquire the lockfile. But flock can also be used in ways that don't require commands to be run in a sub-shell :-)

share|improve this answer
1  
The flock() system call does not work over NFS. –  maxschlepzig Oct 5 '11 at 8:03
    
BSD has a similar tool, "lockf". –  dubiousjim Nov 14 '12 at 14:06

I use dtach.

$ dtach -n /tmp/socket long_running_task ; echo $?
0
$ dtach -n /tmp/socket long_running_task ; echo $?
dtach: /tmp/socket: Address already in use
1
share|improve this answer

An easy way is to use lockfile coming usually with the procmail package.

LOCKFILE="/tmp/mylockfile.lock"
# try once to get the lock else exit
lockfile -r 0 "$LOCKFILE" || exit 0

# here the actual job

rm -f "$LOCKFILE"
share|improve this answer

Don't use a file.

If your script is executed like this e.g.:

bash my_script

You can detect if it's running using:

running_proc=$(ps -C bash -o pid=,cmd= | grep my_script);
if [[ "$running_proc" != "$$ bash my_script" ]]; do 
  echo Already locked
  exit 6
fi
share|improve this answer
    
Hm, the ps checking code runs from within my_script? In the case another instance is running - does not running_proc contain two matching lines? I like the idea, but of course - you will get false results when another user is running a script with the same name ... –  maxschlepzig Oct 5 '11 at 8:20
    
It also includes a race condition: if 2 instances execute the first line in parallel then none gets the 'lock' and both exit with status 6. This would be a kind of one round mutual starvation. Btw, I am not sure why you use $! instead of $$ in your example. –  maxschlepzig Oct 5 '11 at 12:02
    
@maxschlepzig indeed sorry about the incorrect $! vs. $$ –  frogstarr78 Oct 5 '11 at 17:29
    
@maxschlepzig to handle multiple users running the script add euser= to the -o argument. –  frogstarr78 Oct 5 '11 at 17:33
    
@maxschlepzig to prevent multiple lines you can also change the arguments to grep, or additional "filters" (e.g. grep -v $$). Basically I was attempting to provide a different approach to the problem. –  frogstarr78 Oct 5 '11 at 17:39

Using FLOM (Free LOck Manager) tool, serializing commands becomes as easy as running

flom -- command_to_serialize

FLOM allows you to implement more sofisticate use cases (distributed locking, readers/writers, numeric resources, etc...) as explained here: http://sourceforge.net/p/flom/wiki/FLOM%20by%20examples/

share|improve this answer

I offer the following solution, in a script named 'flocktest'

#!/bin/bash
export LOGFILE=`basename $0`.logfile
logit () {
echo "$1" >>$LOGFILE
}
PROGPATH=$0
(
flock -x -n 257
(($?)) && logit "'$PROGPATH' is already running!" && exit 0
logit "'$PROGPATH', proc($$): sleeping 30 seconds"
sleep 30
)257<$PROGPATH
share|improve this answer

Check my script ...

You may LOVE it....

[rambabu@Server01 ~]$ sh Prevent_cron-OR-Script_against_parallel_run.sh
Parallel RUN Enabled
Now running
Task completed in Parallel RUN...
[rambabu@Server01 ~]$ cat Prevent_cron-OR-Script_against_parallel_run.sh
#!/bin/bash
#Created by RambabuKella
#Date : 12-12-2013

#LOCK file name
Parallel_RUN="yes"
#Parallel_RUN="no"
PS_GREP=0
LOCK=/var/tmp/mylock_`whoami`_"$0"
#Checking for the process
PS_GREP=`ps -ef |grep "sh $0" |grep -v grep|wc -l`
if [ "$Parallel_RUN" == "no" ] ;then
echo "Parallel RUN Disabled"

 if [ -f $LOCK ] || [ $PS_GREP -gt 2   ] ;then
        echo -e "\nJob is already running OR LOCK file exists. "
        echo -e "\nDetail are : "
        ps -ef |grep  "$0" |grep -v grep
        cat "$LOCK"
  exit 6
 fi
echo -e "LOCK file \" $LOCK \" created on : `date +%F-%H-%M` ." &> $LOCK
# do some work
echo "Now running"
echo "Task completed on with single RUN ..."
#done

rm -v $LOCK 2>/dev/null
exit 0
else

echo "Parallel RUN Enabled"

# do some work
echo "Now running"
echo "Task completed in Parallel RUN..."
#done

exit 0
fi
echo "some thing wrong"
exit 2
[rambabu@Server01 ~]$
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.