scheduling background tasks in succession

Question

I've got three SAS programs to run in the background over a weekend. The three programs output information into files of the same names, so I'll need to remove these output files after the first program finishes and before the second program starts, as well as after the second program finishes and before the third program starts. Obviously the simplest way to get around this issue is to alter each of the programs in such a way that the output filenames are different, but that wouldn't help me learn how to use Unix. So this is what I've got so far:

# Begin by running the first SAS program in the background.
sas program1.sas & | at 5:00 PM JAN 11

# I'd like to wait until the first program finishes to remove
# the output files and run the second program.
wait ???
rm file1.sas7bdat file2.sas7bdat file3.sas7bdat file4.sas7bdat file5.sas7bdat
sas program2.sas & | at 5:00 PM JAN 12

# And now I repeat.
wait ???
rm file1.sas7bdat file2.sas7bdat file3.sas7bdat file4.sas7bdat file5.sas7bdat
sas program3.sas & | at 5:00 PM JAN 13

I set each program to run at the same time on consecutive days for two reasons: (1) I know the previous program will finish within 24 hours, and (2) I am assuming the load on the server will be similar at the same time on consecutive days. Ideally I could somehow grab the process ID for the first program and pass it to the first wait command, and similarly for the second program and wait command. Any ideas?

EDIT

I am more interested in a general solution for determining a process ID and passing it to the wait command than this specific situation.

Answer 1

The problem is you're using at for something its not supposed to be used for, really. You also have two conflicting goals: "run program 2 after program 1" and "run program 2 at 5pm jan 12".

One part of your problem is simple to solve: at allows multiple commands. So, instead of just using at to schedule the sas script, you can put the rm in the job as well.

at 5:00 PM JAN 11 <<EOJ
sas program1.sas
rm file1.sas7bdat file2.sas7bdat file3.sas7bdat file4.sas7bdat file5.sas7bdat
EOJ

There is no reason for the & in the sas line; at always runs jobs "in the background".

Next, you must figure how you want to solve the conflict mentioned earlier. In particular, if the Jan. 11 job hasn't finished 24 hours later (by 5pm Jan. 12). There are a couple of ways to go about it:

• It isn't really the Jan 12 job. It should run as soon as program1.sas finishes. In which case, just make it part of the first job above, just like the rm.
• This will never happen. If it does, you'll fix it manually. In which case, just schedule it like the one above.
• It needs to wait. In which case, you could either schedule it at the end of the first at job, or use a lock file, or calculate the appropriate amount of time to sleep (or use sleepenh, if available, to avoid doing the calculation yourself).

Answer 2

So what you have and want to do basically boils down to the following.

command1 &
wait <on command1>
sleep <until specified time>

command2 &
wait <on command2>
sleep <until specified time>

...

In this case, removing the backgrounding (&) as well as the wait and sleep will cause the commands to execute in sequence. This, however, will make them execute immediately after each other.

To wait until a specific time, you can sleep for the appropriate amount of time. sleep takes a number of seconds as input, and Unix systems traditionally keep and measure time in seconds, so this boils down to simple arithmetic:

1. Convert the wait-for date and time to seconds since epoch.
2. Convert the current time to seconds since epoch.
3. Compute the difference.
4. Sleep for that long.

How to do that is nicely laid out in this Stack Overflow answer, but to copy the essential part:

current_epoch=$(date +%s)
target_epoch=$(date -d '01/01/2010 12:00' +%s)
sleep_seconds=$(( $target_epoch - $current_epoch ))
sleep $sleep_seconds

That particular example is bash syntax, but it should be rather easy to convert to just about any shell scripting language. $(...) executes a command and replaces with the output of that command, and $(( ... )) evaluates an arithmetic expression.

GNU date -d also supports time-only timestamps, so you can say date -d '05:00' and it will translate that to the coming 05:00.

Putting this together, it should be easy to make a script like the one you want.

Answer 3

Well, there is no wait command. At least not that I know of. Here's my alternative logic:

Create 3 script files for each sas program. These scripts will only execute if the previous sas program has finished execution. This can be done by capturing pid of the previous sas program and verifying with ps command that it's not running anymore before removing the output files.

Schedule the scripts with crontab to be run on every Friday through Sunday, respectively.

So, if I named the first script file sas_script1.sh, it would look like this:

if [ -f /var/run/sas.pid ]; then
    /bin/ps $( cat /var/run/sas.pid ) >/dev/null
    [ $? ] && exit               #  $? checks the exit status of the last command
fi

rm -f /path/to/file{1..5}.sas7bdat

/usr/bin/sas /path/to/program1.sas &

echo $! >/var/run/sas.pid       #  $! gives you the PID of the last program that's sent background

Write the same script for rest of the two sas programs e.g just edit program1.sas to program2.sas

Create a file and write the following in it:

00 17 * * 5 /path/to/sas_script1.sh
00 17 * * 6 /path/to/sas_script2.sh
00 17 * * 0 /path/to/sas_script3.sh

Now, upload the file as cron job:

crontab /path/to/file

You can learn more about Un*X this way :D