email notification upon completion or error of nohup background process

Question

How would I send an email if a background process I run via nohup fails or is completed? I am running a lot of background processes via nohup in a for loop:

for afolder in $dir/dothis*
do 
    nohup nice COMMAND afolder & 
done

I guess because of the background process, when I do

nohup nice COMMAND afolder & ; tail nohup.out | mail [email protected] -s "job done"

I get an email upon execution, not completion. And how would I send a different email subject line depending on whether the process failed to complete vs. successful completion?

Thanks in advance!

Answer 1

One option is to do the mailing in the background job:

for afolder in "$dir"/dothis*; do
    nohup sh -c '
        outfile=$(mktemp)

        if nice COMMAND "$1" >"$outfile" 2>&1; then
            success_or_fail="success"
        else
            success_or_fail="failure"
        fi

        tail "$outfile" |
        mail -s "job done ($success_or_fail)" my[email protected]

        rm "$outfile"' sh "$afolder" &
done

This does the same loop as you, but then starts a child script running under nohup. This script takes the current value of $afolder as its first command line argument and runs the command in an if statement. It sets success_or_fail appropriately depending on the outcome of the command, and then sends the email.

All output of the command is redirected to a temporary file which is then tail-ed for the email and deleted when the shell exits.