2

I want to use lockfile functionality for a bash script that will run in parallel. The problem is that in a functionality as in the script below, a starvation can happen in case that the first one to lock is called immediately again. It seems that lockfile is polling with a big enough timeout to allow other processes to acquire the lock.

Is that true? How should that be handled?

lockfile test.lock
echo "TESTING $$"
sleep 10
rm test.lock

For example:

Two shells :
SH1, SH2 run the script together,
SH1 acquires lock, SH2 is blocked.
PROBLEM:
When SH1 is done and removes the lock, SH2 is still blocked (for about a second...), and if SH1 runs the script in a loop, SH2 will be blocked permanently.

|improve this question
  • lockfile test$$.lock can help – Costas Jan 19 '15 at 13:56
  • 1
    How? Cause if the script is run from different sessions, the $$ if different, and the lockfile isn't locking anything... I need the lockfile to be as global as it gets – csny Jan 19 '15 at 14:00
  • It has return as to the beginning. Firstly you should explain your main goal to recive adequate advise. – Costas Jan 19 '15 at 14:04
  • 1
    From man: "If lockfile can't create all the specified files (in the specified order), it waits sleeptime (defaults to 8) seconds and retries the last file that didn't succeed. You can specify the number of retries (-r parameter) to do until failure is returned." So try to change sleeptime and number of retry. – Costas Jan 19 '15 at 14:40
  • 1
    May want to try the flock command instead of lockfile. It will hang until the lock is available (usig the flock system call) rather than polling. – Mark Plotnick Jan 19 '15 at 21:11
1

You should put the lockfile on a tmpfs, because poking lockfiles will prevent it from sleeping.

Here are two solutions:

Static Sleep; Sufficient For a Few Procs

WAIT=5  # max seconds between two procs; larger is nicer for the system
lockfile -${WAIT} -r-1 "${LOCKFILE}"

## payload

rm -f "${LOCKFILE}"
# sleep outside the lock for at least $WAIT seconds
#  to give another proc a chance to lock it
sleep ${WAIT}s

Randomized Sleep; Handles Many Procs, But Not Super Nice

MAX=5
MIN=3
WAIT=$((MIN+RANDOM/(1+MAX-MIN))) # random sleep between min and max
lockfile -${WAIT} -r-1 "${LOCKFILE}"

## payload

rm -f "${LOCKFILE}"
# sleep outside the lock for at least $WAIT seconds
#  to give another proc a chance to lock it
sleep ${WAIT}s

Longer Waiters Become More Aggressive; Nice For Many Procs

MAX=600
MIN=1
WAIT=${MAX}  # max seconds between two procs; larger is nicer
while ! lockfile -r0 "${LOCKFILE}"
do
    sleep ${WAIT}s
    WAIT=$(( WAIT / 2 )) # backoff formula
    if [ ${WAIT} -lt ${MIN} ]; then
        WAIT=${MIN}
    fi
done

## payload

rm -f "${LOCKFILE}"
# reduced WAIT means likely to run again; alternatively use MAX
sleep ${WAIT}s
|improve this answer
0

I am not sure there is a problem.

The kernel code for fs/locks.c contains the following comment:

 519 /* Insert waiter into blocker's block list.
 520  * We use a circular list so that processes can be easily woken up in
 521  * the order they blocked. The documentation doesn't require this but
 522  * it seems like the reasonable thing to do.
 523  */

If you still do not trust this, the only reasonable alternative is to use semaphores, semop(2), semget(2).

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.