Cron job for every minute executing out of order?

Question

I'm setting up some shell scripting to be executed every five minutes, then every minute on our client's system, to poll a log and pull some timing information to be stored and accessed by another application through an external file. The current implementation we have in place works fine as it writes a single line to two separate files. We're refining the process so now I need to write two lines to one file every five minutes, and four lines to another, for every minute. However, I've noticed in testing that every few minutes the lines seem to execute out of order.

My scripts are included below:

 */5 * * * *     ~/myscript.pl ~/mylog | tail -3 | head -1 > ~/myreport1
 */5 * * * *     ~/myscript.pl ~/mylog | tail -2 | head -1 >> ~/myreport1

 * * * * *       ~/myscript.pl ~/mylog | tail -8 | head -1 > ~/myreport2
 * * * * *       ~/myscript.pl ~/mylog | tail -7 | head -1 >> ~/myreport2
 * * * * *       ~/myscript.pl ~/mylog | tail -3 | head -1 >> ~/myreport2
 * * * * *       ~/myscript.pl ~/mylog | tail -2 | head -1 >> ~/myreport2

In some cases, it seems that only a handful of the lines execute properly while in others, only one line is written. I don't even see the full number of lines written to the file all the time, otherwise I'd just assume the values I was pulling weren't being collected properly. I'm not sure how to judge whether all the cron lines are being executed, and what could be causing them to happen out of order, or not at all.

Answer 1

There is no guarantee that cron will run tasks in the order in which they appear in the cronfile. In fact, it may well run two tasks simultaneously. So it's definitely not a good idea to have the tasks depend on each other. For example, in your cronfile, one task creates a file and another one (or three) appends to it. If the appender starts first, the creator will effectively delete the appender's work.

Better would be to create a driver script with the four every-minute runs of myscript and another one with the two every-five-minute runs. Then you can cron the two driver scripts, resulting in only one cron task for each time interval.

Answer 2

As far as cron is concerned, you have a lot of commands that are executed at the same time. Cron just creates some child processes that run 'in parallel' - i.e they are somehow multi-tasked/scheduled which introduces data-races for your use case.

For your problem you don't really need cron. A simple shell script like this one is enough:

#!/bin/sh
function f1() {
  ~/myscript.pl ~/mylog | tail -3 | head -1 > ~/myreport1
  ~/myscript.pl ~/mylog | tail -2 | head -1 >> ~/myreport1
}

function f2() {
 ~/myscript.pl ~/mylog | tail -8 | head -1 > ~/myreport2
 ~/myscript.pl ~/mylog | tail -7 | head -1 >> ~/myreport2
 ~/myscript.pl ~/mylog | tail -3 | head -1 >> ~/myreport2
 ~/myscript.pl ~/mylog | tail -2 | head -1 >> ~/myreport2
}

while true; do
  f1
  f2
  sleep 1
  f2
  sleep 1
  f2
  sleep 1
  f2
  sleep 1
  f2
  sleep 1
done

Now everything is well-defined, i.e. the order of execution is guaranteed.

You could start it (as a background job) via at - or call from a screen session.