timeout without killing process in bash

Question

I have a main script that I'm running, and from it I have a second "slow process" I want to kick off, and "do something" in the main script if it doesn't complete in the time limit -- depending on if it completed or not. N.B. If the "slow process" finishes before my time limit, I don't want to have to wait an entire time limit.

I want the "slow process" to keep going so I can gather stats and forensics about it's performance.

I've looked into using timeout, however it will kill my script when finished.

Suppose this simplified example.

main.sh

result=`timeout 3 ./slowprocess.sh`
if [ "$result" = "Complete" ]
then
 echo "Cool it completed, do stuff..."
else
 echo "It didn't complete, do something else..."
fi

slowprocess.sh

#!/bin/bash
start=`date +%s`
sleep 5
end=`date +%s`
total=`expr $end - $start`
echo $total >> /tmp/performance.log
echo "Complete"

Here, it uses timeout -- so the script dies, so nothing winds up in /tmp/performance.log -- I want slowprocess.sh to complete, but, I want main.sh to go onto its next step even if it doesn't finish in the 3 seconds.

Answer 1

With ksh/bash/zsh:

{
  (./slowprocess.sh >&3 3>&-; echo "$?") |
    if read -t 3 status; then
      echo "Cool it completed with status $status, do stuff..."
    else
      echo "It didn't complete, do something else..."
    fi
} 3>&1

We duplicate the original stdout onto fd 3 (3>&1) so we can restore it for slowprocess.sh (>&3), while stdout for the rest of the (...) subshell goes to the pipe to read -t 3.

Alternatively, if you want to use timeout (here assuming GNU timeout):

timeout --foreground 3 sh -c './slowprocess.sh;exit'

would avoid slowprocess.sh being killed (the ;exit is necessary for sh implementations that optimise by executing the last command in the shell process).

Answer 2

Here's a solution using only ubiquitous shell tools.

This should be easily done by forking the slow process and a sleep in the background and waiting for the first to finish, except that the wait shell builtin waits for all jobs to finish rather than only for the first one.

So instead, fork the slow process and a sleep in the background, have them both report their status through a pipe, and read the first status that comes out of the pipe.

fifo=$(mktemp -u)  # if not on Linux, adapt to what your OS provides
mkfifo -m 600 "$fifo"
{ ./slowprocess.sh; echo z >"$fifo"; } &
sh -c 'sleep 3; echo a' >"$fifo" &
sleep_pgid=$!
read status <$fifo
case $status in
  a) echo "That process is taking a long time"; read ignored <$fifo;;
  z) echo "Done already"; kill -INT -$sleep_pgid;;
esac
rm "$fifo"

Answer 3

Whatever is blocking in the slowprocess.sh could be backgrounded so that it can continue even after the timeout.

Also you might be able to use a signal from timeout back to your main.sh. See the man page for timeout.

excerpt from timeout info page

`-s SIGNAL'
`--signal=SIGNAL'
     Send this SIGNAL to COMMAND on timeout, rather than the default
     `TERM' signal. SIGNAL may be a name like `HUP' or a number. Also
     see *Note Signal specifications::.

You'd have to trap for this signal inside of main.sh.

Also timeout returns different statuses for different types of failures. You might be able to make use of these as well:

   Exit status:

     124 if COMMAND times out
     125 if `timeout' itself fails
     126 if COMMAND is found but cannot be invoked
     127 if COMMAND cannot be found
     the exit status of COMMAND otherwise

These are present in the variable $?, after the timeout ... line runs.

Answer 4

Here's another approach that is fairly simple, though it produces some extraneous job control output: start a background process "sleep nnn" then use "wait -n" to wait for the first of the sleep process and the process you're monitoring. Here's a somewhat clumsy example of the technique (with "set -o notify" immediate notifications):

$ (for I in $(seq 10); do sleep 1; echo $I ; done) & \
more> J1=$! ; \
more> sleep 5 & \
more> J2=$! ; \
more> time wait -n $J1 $J2
[1] 288160
[2] 288161
1
2
3
4
[2]+  Done                    sleep 5

real    0m5.006s
user    0m0.002s
sys 0m0.004s
$ 5
6
7
8
9
10
[1]+  Done                    ( for I in $(seq 10);
do
sleep 1; echo $I;
done )
$

$ (for I in $(seq 3); do sleep 1; echo $I ; done) & \
more> J1=$! ; \
more> sleep 5 & \
more> J2=$! ; \
more> time wait -n $J1 $J2
[1] 288256
[2] 288257
1
2
3
[1]-  Done                    ( for I in $(seq 3);
do
sleep 1; echo $I;
done )

real    0m3.025s
user    0m0.004s
sys 0m0.015s
$ [2]+  Done                    sleep 5
$