10

I want to ask about cron jobs. OK, we place the scripts in the crontab and the cron daemon executes them.

Now if I understand this, each minute, cron checks the crontab of each user and executes the configured scripts. But how is this actually done? Does it fork child processes etc?

It can not execute the tasks sequentially since the timings would be lost (e.g. due to waiting a long-running script to finish). So how is this actually implemented.

Just to help out, I'm not looking for low level code. A high level description (of the algorithm perhaps?) or how this is implemented in most distros would suffice to me.

2 Answers 2

9

I found this Q&A over on StackOverflow titled: How does cron internally schedule jobs?.

excerpt from that post & the wikipedia article on cron

The algorithm used by this cron is as follows:

1. On start-up, look for a file named .crontab in the home directories of 
   all account holders.

2. For each crontab file found, determine the next time in the future that
   each command is to be run.

3. Place those commands on the Franta-Maly event list with their corresponding
   time and their "five field" time specifier.

4. Enter main loop:

   1. Examine the task entry at the head of the queue, compute how far in 
      the future it is to be run.

   2. Sleep for that period of time.

   3. On awakening and after verifying the correct time, execute the task 
      at the head of the queue (in background) with the privileges of the 
      user who created it.

   4. Determine the next time in the future to run this command and place 
      it back on the event list at that time

This SuperUser Q&A titled: How does cron work? covers some of your additional questions. For example your question as to how does cron deal with jobs that are scheduled for the same time. One of the answers in that thread states that as the cron daemon processes each task, it forks each scheduled job so that no single job will act as a blocker for jobs that have overlapping times.

7
  • If it is a queue then tasks run one after the other (those in the same queue). So if a task needs 3 mins to execute and the next task in the queue is expected to be scheduled in the next minute, how does this happen?
    – Jim
    Jul 11, 2013 at 5:58
  • @Jim - I think one simplification that you're assuming is that it's reading the files every minute when it's doesn't do this at all. When you save the file, the crond daemon is incorporating the contents of the crontab for a given user then and adding it to the queue in a sorted order based on the time in the future that it's suppose to run.
    – slm
    Jul 11, 2013 at 11:22
  • Ok.So in this case, the queue could have tasks that the should be run concurrently.So child processes are forked or something?
    – Jim
    Jul 11, 2013 at 19:23
  • @Jim - exactly. Also see my additional comments in the answer.
    – slm
    Jul 11, 2013 at 19:42
  • How does it detect new tasks that we create for cron? Does it use something like inotify to watch the filesystem or something else? May 28, 2015 at 16:47
0

I wrote a blog post describing it.
Quoting the relevant text from there:

  • We can have a finite thread-pool which will execute all the tasks by picking them up from a PriorityBlockingQueue (thread-safe heap) prioritized on job.nextExecutionTime().
  • Meaning that the top element of this heap will be always be the one that will fire the soonest.
  • We will be following the standard threadpool producer-consumer pattern.
  • We will have one thread which will be running in an infinite loop and submitting new jobs to the thread pool after consuming them from the queue. Lets call it QueueConsumerThread:
void goToSleep(job, jobQueue){
    jobQueue.push(job);
    sleep(job.nextExecutionTime() - getCurrentTime());
}

void executeJob(job, jobQueue){
    threadpool.submit(job); // async call
    job = job.copy();
    job.setNextExecutionTime(getCurrentTime() + job.getExecutionInterval());
    jobQueue.add(job);
}

@Override
void run(){
    while(true)
    {
        job = jobQueue.pop()
        if(job.nextExecutionTime() > getCurrentTime()){
            // Nothing to do
            goToSleep(job, jobQueue)
        }
        else{
            executeJob(job, jobQueue)
        }
    }
}
  • There will be one more thread which will be monitoring the crontab file for any new job additions and will push them to the queue.
  • Lets call it QueueProducerThread:
@Override
void run()
{
    while(true)
    {
        newJob = getNewJobFromCrontabFile() // blocking call
        jobQueue.push(newJob)
    }
}
  • However, there is a problem with this:
    • Imagine that Thread1 is sleeping and will wake up after an hour.
    • Meanwhile a new task arrives which is supposed to run every minute.
    • This new task will not be able to start executing until an hour later.
  • To solve this problem, we can have ProducerThread wakeup ConsumerThread from its sleep forcefully whenever the new task has to run sooner than the front task in the queue:
@Override
void run()
{
    while(true)
    {
        newJob = getNewJobFromCrontabFile() // blocking call
        jobQueue.push(newJob)
        if(newJob == jobQueue.peek())
        {
            // The new job is the one that will be scheduled next.
            // So wakeup consumer thread so that it does not oversleep.
            jobQueueConsumerThread.interrupt()
        }
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.