9

In my script I split up a data set into input_aa, input_ab, etc. Then, I run each through the same Python script, as such:

# Execute program on each split file
for part in input_*; do
        python3 $part &
done
wait

My question is two-fold: how do I detect that a Python process has failed, and when detected, how do I kill all spawned children and exit the script with a fail?

10

You could use a process group:

set -m
(
   for part in input_*; do
     (python3 "$part" || kill 0) &
   done
   wait
)

set -m (and optional POSIX shell feature, required Unix shell feature) runs jobs in their own process group. In bash, yash, zsh, mksh, that's jobs of the subshell where set -m is enabled so the outer (...) and all the processes created within that will be placed in that same process group.

For dash and other ash based shells, that only works at the top level shell process. So that code will work unless it's put in a subshell.

That won't work in AT&T ksh or the old SysV/Bourne shell at all.

kill 0 sends a SIGTERM signal to to all the members of the current process group.

  • In bash. Why I included a shebang - the required shell is not clear. Good answer – jim mcnamara Jan 14 '15 at 22:27
  • @jimmcnamara, that works in bash, dash, yash, mksh, zsh. Basically any POSIX shell but AT&T ksh. set -m is (under-)specified in POSIX but as an optional feature. – Stéphane Chazelas Jan 14 '15 at 22:31
  • I use Solaris. /bin/sh will not fly. – jim mcnamara Jan 15 '15 at 2:00
  • @jimmcnamara, no /bin/sh on Solaris 10 and before is the Bourne shell (not a POSIX shell), and on 11, AT&T ksh. As I said, it works in bash, dash, yash, mksh, zsh. – Stéphane Chazelas Jan 15 '15 at 8:27
  • 1
    @mikeserv, that would reparent the process to 1, but would not take it out of the process group. kill 0 kills all the members of the process group whatever their parent is. See ps -j to see process group ids. – Stéphane Chazelas Jan 15 '15 at 11:20
3

This is an example. PLAY with this first to get exactly what you need. It cannot break much as is.

#!/bin/bash
# Example of killing off all children

> killfile
> outfile.err
kill_em()
{
   echo 'killing all children ' > 2
   while read pid
   do
      kill -0 $pid && kill -9 $pid  # if still running kill it
   done < killfile
   exit 1
}

export grandparentpid=$$
trap 'kill_em' 6
for i in 2 2 3 4 5 6 7 8 9 10
do
        ( sleep $i && ls oinkle  >> outfile 2>> outfile.err &
          pid=$!
          echo $pid >> killfile
          wait $!
          [ $? -ne 0 ] && kill -6 $grandparentpid
        ) &
done
wait

This is setup to fail deliberately because ls oinkle will fail (on my machine).

When you get what you need after tinkering with the starter script--- Change:

for i in 2 2 3 4 5 6 7 8 9 10

to:

for part in input_* 

change:

sleep $i && ls oinkle 

to:

python3 $part 

The redirections are there to save logs . You may not want them.

  • It's a bit racy. If one of the jobs fails before all the other ones have started, then your killfile may not contain all the pids of the jobs that have been started. – Stéphane Chazelas Jan 15 '15 at 11:05
  • A few bad practices like: unquoted variables, use of signal numbers instead of names, use signal 6 (ABRT on Linux amd64 for instance) instead of USR1/USR2 as a user signal, [ $? -ne 0 ]... – Stéphane Chazelas Jan 15 '15 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.