The supercomputer I run my Analysis uses SGE to do the job management and requires each job duration not longer than 24 hours. And this feature is causing me quite a headache.
Sadly, I was asked to use checkpoint mechanism to output results and resubmit the script
qsub myJobScript. On program's part, it's not difficult for me to have it output a temporary file when the clock time hits 23:30:00 (I give the IO operation half of hour), let's say the output file name is called
Note: when you submit a job, you never know when it is going to start. but it does have a log file under my folder when it is started.
So my question is how to use
at or bash scripts to submit my job automatically when "new files" are created and detected? Any ideas?