Wait for background processes to finish with inverse match process ID

Question

I'm looking to run a number of background processes in a loop, but only want to wait for all of those processes to finish, still allowing another background process to continue.

After doing some research, I only see solutions that would require me to list every process ID I wish to wait for instead of matching them inversely against a single process ID. It seems that something like wait != $pid would be very useful.

example script:

#!/bin/bash

command1 &
pid=$!
for i in ${array[@]}; do
    for n in $(seq 3); do
        if [ $n -eq 1 ];
            command2 > file &
        else
            command2 >> file &
        fi
    done
done

In this example I'm looking to wait for every process to finish except $pid without having to list them all using the wait command. Is this possible?

Answer 1

What's wrong with just

wait

As help wait notes:

Waits for the process identified by ID, which may be a process ID or a job specification, and reports its termination status. If ID is not given, waits for all currently active child processes, and the return status is zero.

You want to "still allow another background process to continue"? Just disown it. Following your example:

command1 & disown %1
command2 & command2 & command2 &
wait

Answer 2

As others have said, there isn't any way to wait on "not" a process ID. However this pattern of code doesn't seem that bad to me, so I offer it as a suggested way of achieving what you're after.

It makes use of Bash arrays which you can then give as a list to the wait command.

Example

A modified version of your example code, cmd.bash.

!/bin/bash

array[0]=1
array[0]=2

sleep 30 &
pid=$!

pidArr=()
for i in ${array[@]}; do
  for n in $(seq 3); do
    if [ $n -eq 1 ]; then
      sleep 31 > file &
      pidArr+=($!)
    else
      sleep 32 >> file &
      pidArr+=($!)
    fi
  done
done

echo ${pidArr[@]}

I've substituted sleep commands in this example just to simulate some long running jobs that we can background. They have nothing to do with this other than to act as stand-ins.

Also the final echo command is there purely so we can see what happens when the for loops complete. Ultimately we'll replace it with an actual wait command, but let's not get ahead of ourselves.

Example run #1

So let's run our program and see what happens.

$ ./cmd.bash 
24666 24667 24668

Now if we check ps:

$ ps -eaf|grep sleep
saml     24664     1  0 22:28 pts/9    00:00:00 sleep 30
saml     24666     1  0 22:28 pts/9    00:00:00 sleep 31
saml     24667     1  0 22:28 pts/9    00:00:00 sleep 32
saml     24668     1  0 22:28 pts/9    00:00:00 sleep 32

If we wait a bit, these will ultimately finish. But this shows that if we add the process IDs to an array we can echo them out afterwards.

Example #2

So let's change that echo line out and swap in a wait line now, something like this:

  wait ${pidArr[@]}

Now when we run our modified version we see that it's waiting on ALL the process IDs:

$ ./cmd.bash 

...after ~30 seconds passes

$

We get back our prompt. So we successfully waited for all the processes. A quick check of ps:

$ ps -eaf|grep sleep
$

So it worked.