The Stack Overflow podcast is back! Listen to an interview with our new CEO.
2 of 3 edited tags

How do I wait for all child processes but one to finish executing

I am starting a number of processes in the background of the scrip that I'm writing. I want to wait for some of them to finish before the end of each iteration of a for loop, but there is one process that I need to leave running and end separately. What I'm doing right now doesn't work as I seem to run out of memory, and more and more processes are sacrificed, including the script that I'm running. What I currently have is:

loadAvg &
pid=$!
echo "LA pid $pid"
for (( j=0; j < $units_to_fill; j=$((j+1)) )); do
    #Print all files in dir | shuffle file names | delete 1st 1024 files found
    if [ $from_zero -ne 1 ];then
        stat -c %n * | sort -rz | awk -v n=0 '/file_[0-9]*_[abcd].bin/ { if( ++n <= 1024) {print $1 ; }}' | xargs -r rm -f
    fi
    echo "Creating file $j"
    pids=$(writeFile "file_${j}.bin" $src_file $block_size $block_count "${wd}/${dd_log_file}") #Writes a file of (dd) block size 1MB, with 1024 blocks
    for job in `jobs -p`; do
        if [[ $job -ne $pid ]]; then
            echo "Waiting for $job"             
            wait $job
        fi
    done
done
kill $pid

The function writeFile spawns a number of child processes, and I can't guarantee the order that they'll exit in, as I was previously doing wait $! in place of the

for job in `jobs -p`

I can capture the pids of the child processes that are spawned in the function, and they are currently being returned in a format that's essentially pid1 pid2 pid3 ..., but running wait $pids doesn't work as some of the processes finish before that line is reached, so wait errors out.

Note: I'm running on an embedded system with busybox linux installed, and as such it's been quite cut down and a lot of standard functions are unavailable.