Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

The supercomputer I run my Analysis uses SGE to do the job management and requires each job duration not longer than 24 hours. And this feature is causing me quite a headache.

Sadly, I was asked to use checkpoint mechanism to output results and resubmit the script qsub myJobScript. On program's part, it's not difficult for me to have it output a temporary file when the clock time hits 23:30:00 (I give the IO operation half of hour), let's say the output file name is called 1.00137/U and 1.00137/p, etc.
Note: when you submit a job, you never know when it is going to start. but it does have a log file under my folder when it is started.

So my question is how to use crontab or at or bash scripts to submit my job automatically when "new files" are created and detected? Any ideas?

share|improve this question
1  
What unix variant is it? For Linux, use an inotify-based mechanism, see Can a bash script be hooked to a file? –  Gilles Dec 11 '12 at 23:48
add comment

1 Answer

Run a simple script with cron every 5 minutes. Your crontab could look e.g. like this:

*/5 * * * *  /path/to/your/script

Script pseudocode:

IF (lockfile exists) { exit }
create lockfile
IF (old job has not finished yet) { 
    remove lockfile
    exit
}
IF (new job has already been queued) {
    remove lockfile
    exit
}
schedule new task
remove lockfile

See Correct locking in shell scripts? for how to lock correctly (as race-condition-free as possible).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.