stop parallel processing for a cron triggered process

Question

I have configured my webservers cron jobs to trigger some PHP script, every 15min, but sometimes the script takes more than 15 min to complete execution, by the time another process of the same script starts. which is creating problems in our logic. I want only one instance of the same process running, and if it is completed then only let the other process execute (don't want to put process in a queue, just terminate at that moment and execute at next interval).

Example: I have three PHP file which will be called by the webservers cron job every 15 min. let say a.php, b.php, c.php. the script might take more than 15 min to complete execution. on end of execution it updates our database, hence can't run two copy of same script file (this will break our logic). Like say a.php is triggered at 4.00 pm and at 4.15pm again cron will trigger a.php. here I want to check if a.php has completed execution, if not skip the call and trigger it at 4.30pm by checking the same case, but b.php, c.php and a.php are run parallel (but same instances can't).

Answer 1

The standard practice is to use a pid file. Every time the script executes, have it check a file to see if it's already running somewhere. The order of operations is outlined below.

• read pid out of /var/run/yourscriptname.pid
• check if pid is running, exit if it is
• get your pid, write it to /var/run/yourscriptname.pid
• execute as normal
• remove pid file

In your case this will do, but there are some cases where a script might be called fast enough that in the time between checking the pid and writing the pidfile, another process starts up and also sees no pidfile and continues to execute. You can further tighten things by using advisory locking. It is a facility the OS provides on just about every Unix.

References:

• man flock(2)
• Locking files with flock() (in PHP)

Answer 2

It is super simple via flock:

* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/local/bin/frequent_cron_job

Credit to this answer. I came here by google, so I thought sharing it here as well.

Answer 3

It's hard to tell from your question if you want to start with a new job every time or just skip a rotation if it is still running.

If you want to start fresh each time, the simplest hack is to add a killall your_command; your_command to your cron.

Otherwise try pgrep you_command || your_command in your cron line to only execute if a current one is NOT found running.

You could also get fancier and register the PID's of your job in some kind of run file and then check that run file for active jobs before you start work. If you find one you can either exit and let it run or kill the other one off and mark your own PID as the currently running one.