6

In my script /usr/local/bin/backup, that I call every hour from /etc/crontab, I use rsync to copy data to an off-site server. That all worked fine, even in cases where we had somewhat more new data than can be pushed out in an hour.

Last week someone copied an 11GB file on the data partition and when I found out the next day there were 14 rsync programs running in parallel, each of then getting no bandwidth and each probably working on the same huge file. I killed them all (before realising I should have kept the first one running), stopped the cron job and ran the backup script by hand.

I can write out a file in the script before starting rsync and check in the script if that file is already there to prevent backup from running in parallel. Is there an easier way of doing this?

My /etc/crontab entry:

5  *  *  *  *   root  /usr/local/bin/backup
6

There are different ways of doing this, but IMO the easiest is inserting flock before the command in the crontab file:

5  *  *  *  *   root  flock -n /var/lock/backup /usr/local/bin/backup

The /var/lock/backup file is the lock that flock uses and -n immediately makes the command fail if the lock already exists.

This could of course mean that if one backup takes 1 hour and 1 minute, that the next one starts 59 minutes later. If that is a problem you could look into using -x.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.