4

I am running multiple processes and want to exit with the appropriate exit code (this means error on failure, success otherwise) if any of them fails or exists.

Additionally if any child process exits or fails, any other child processes should also be shut down.

My current non-functional solution (yarn is just an example; may be any other command):

#!/bin/bash -e

# Run other process before start
sh ./bin/optimize.sh

trap 'exit' INT TERM
trap 'kill -INT 0' EXIT

# Run Schedule
sh ./bin/schedule.sh &
# Run a long running task
yarn task &

wait

./bin/schedule.sh:

#!/bin/bash -e

while true; do
  yarn schedule
  sleep 30
done

If something in yarn schedule fails, everything exists correctly. But when I kill the process via ctrl+c or yarn task exits yarn schedule keeps running.

How to get this working regardless of what the child processes are (bash, yarn, php or whatever)?

I can't use GNU parallel.

  • sorry ... is "Scheduler" the ./bin/schedule.sh referred to above? – trs Dec 10 '18 at 20:13
  • I can't reproduce this with small test scripts. Is yarn ignoring signals when used independently? – Kusalananda Dec 10 '18 at 20:32
  • @trs yes, I made it more clear in the question – timw Dec 10 '18 at 20:35
  • @Kusalananda yarn is just an example – timw Dec 10 '18 at 20:35
  • To have the script exit when yarn task exits, you would have to start yarn task with (yarn task; kill "$$") &. I can't at the moment see a good reason why the backgrounded shell script does not exit when you press Ctrl+C (it does when I'm testing this with yarn schedule replaced by a sleep). – Kusalananda Dec 10 '18 at 20:53
2

This is painful in shells because the wait builtin doesn't do “wait for any”, it does ”wait for all“. wait with no argument waits for all the children to exit, and returns 0. wait with an explicit list of processes waits for all of them to exit, and returns the status of the last argument. To wait for multiple children and obtain their exit status, you need a different approach. wait can give you the exit status only if you know which child is already dead.

One possible approach is to use a dedicated named pipe to report each child's status. The following snippet (untested!) returns the largest of the children's status.

mkfifo status_pipe
children=0
{ child1; echo 1 $? >status_pipe; } & children=$((children+1))
{ child2; echo 2 $? >status_pipe; } & children=$((children+1))
max_status=0
while [ $children -ne 0 ]; do
  read -r child status <status_pipe
  children=$((children-1))
  if [ $status -gt $max_status ]; then
    max_status=$status
  fi
done
rm status_pipe

Note that this will block forever if one of the subshells dies without reporting its status. This won't happen under typical conditions, but it could happen if the subshell was killed manually, or if the subshell ran out of memory.

If you want to do something as soon as one of the children fails, replace if [ $status -gt $max_status ]; then … by if [ $status -ne 0 ]; then ….

  • 1
    bash 4.3 has wait -n, which waits for any job to complete – iruvar May 20 at 23:13
2

GNU Parallel has --halt. It will kill all running jobs if one of the jobs finishes or dies and will return false if the job failed:

parallel --halt now,done=1 ::: 'sleep 1;echo a' 'sleep 2;echo b' ||
  echo the job that finished failed

parallel --halt now,done=1 ::: 'sleep 1;echo a;false' 'sleep 2;echo b' ||
  echo the job that finished failed

For systems that do not have GNU Parallel installed, you can typically write your script on a system that has GNU Parallel, and use --embed to embed GNU Parallel directly into the script:

parallel --embed > myscript.sh

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.