2

Does UNIX offer any way of creating queues that start regularly at a particular time, at then lets you gradually add a number of tasks to that queue that will be run sequentially at that time?

I know about cron and at, however:

  • cron does not have a queue system.
  • at doesn't seem to support sequential tasks, and lists tasks in a very unfriendly way ("list all" just prints task number and date, while info about a single a task prints the entire env.)

I envision something like this:

$ create queue a at midnight
$ add "ffmpeg -i a.mkv a.mp4" to a
$ add "ffmpeg -i b.mkv b.mp4" to a
$ add "ffmpeg -i c.mkv c.mp4" to a

$ create queue b at noon
$ add "mysqldump ..." to b

$ list queue a
RUNNING  ffmpeg -i b.mkv b.mp4
NEXT     ffmpeg -i c.mkv c.mp4

$ mail
>N 1 Task Daemon Thu Sep 1 00:23 <user@localhost> ffmpeg -i a.mkv a.mp4

For the record, my actual need is to schedule CPU/time demanding tasks at night while my laptop is in a room with a closed door so I can't hear the fan (or feel the boiling heat from the CPU).

3 Answers 3

2

You may decry at, but it does part of what you want if you use its batch features. If you give the at command the single uppercase letter name of a queue, eg -q M, then any commands on that queue will only be run at the requested time if the system load is less than 0.8, and it will space out starting the jobs by 60 seconds.

You can alter the load and interval with options -l and -b to the atd daemon. The jobs are run in order of submission, and atq -q M lists just the jobs on the given queue.

1

Cron and at are the only things that are commonly installed on Unix systems. But there are fancier job scheduling systems out there. There is a standard interface to queue a job: qsub. If you want to schedule a job for the next midnight:

{ echo '#!/bin/sh'; echo 'ffmpeg -i a.mkv a.mp4'; } >a.script
qsub -a $(date -d 'tomorrow 0:00' +%Y%m%d%H%M) a.script

Ubuntu includes two job schedulers that implement the POSIX (NQS-derived) interface: TORQUE and Slurm. There are other open-source implementations and I have no particular advice on choosing one.

0

If these are single run jobs (as opposed to periodic ones), then the easiest method I think is to just use tmux and sleep. Open a tmux session, and do this:

$ echo "ffmpeg -i a.mkv a.mp4" > midnight.sh
$ echo "ffmpeg -i b.mkv b.mp4" >> midnight.sh
$ let secs_till_midnight=$(date -d "`date -d '+1 day' +%F`" +%s)-$(date +%s)
$ sleep $secs_till_midnight && sh midnight.sh

Schedule more jobs in new panes/windows from the same session, and when you're done, detach from that session. That's it! Your very own, homemade scheduler!

Even though you can't list running tasks, add these 2 lines at the beginning of the midnight.sh for extra visibility:

set -x
exec 2>midnight_err >midnight_out

Now, each command will be output to terminal before it is executed, and all output from the command gets redirected to midnight_* files.

Now simply attach to that session or tail the log files to check the status!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .