Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The supercomputer I run my Analysis uses SGE to do the job management and requires each job duration not longer than 24 hours. And this feature is causing me quite a headache.

Sadly, I was asked to use checkpoint mechanism to output results and resubmit the script qsub myJobScript. On program's part, it's not difficult for me to have it output a temporary file when the clock time hits 23:30:00 (I give the IO operation half of hour), let's say the output file name is called 1.00137/U and 1.00137/p, etc.
Note: when you submit a job, you never know when it is going to start. but it does have a log file under my folder when it is started.

So my question is how to use crontab or at or bash scripts to submit my job automatically when "new files" are created and detected? Any ideas?

share|improve this question
What unix variant is it? For Linux, use an inotify-based mechanism, see Can a bash script be hooked to a file? – Gilles Dec 11 '12 at 23:48

Run a simple script with cron every 5 minutes. Your crontab could look e.g. like this:

*/5 * * * *  /path/to/your/script

Script pseudocode:

IF (lockfile exists) { exit }
create lockfile
IF (old job has not finished yet) { 
    remove lockfile
IF (new job has already been queued) {
    remove lockfile
schedule new task
remove lockfile

See Correct locking in shell scripts? for how to lock correctly (as race-condition-free as possible).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.