3

I've looked and looked and can't find a working solution to a bash script I'm trying to create to shut a process down and wait for it and spawned processes to finish. I'm still learning a lot of Linux.

Context: Process FOO runs. Process BAR is used to check FOO and is also used to kill it (I have no control over the internals of these 2 processes). All I can do it is pass commands to BAR and it performs them.

In this case, I send command to BAR to kill FOO and it spawns another process and backgrounds it.

Goal: I'm trying to run 20 commands simultaneously to execute kill on 20 FOO's (via BAR) and WAIT for all FOO's to die before moving on to the next part of the script.

Problem: So far all I can do is wait for BAR to execute and it moves on before the backgrounded process kills FOO.

BAR exit FOO1
BAR exit FOO2
...
BAR exit FOO20
wait
do more stuff

I've also tried

BAR exit FOO1
PID1=$!
BAR exit FOO2
PID2=$!
wait $PID1 $PID2

without luck.

  • The wait and $! constructs are shell tools for controlling jobs / processes your shell spawns. In your description, the only process your shell spawns is BAR, and by your description, BAR exits immediately after spawning some opaque background stuff. Your shell can't see that background stuff BAR does, only BAR, so it doesn't even have any background jobs / processes to use wait / $! on. If you need a mechanism to know whether the internal background process spawned by BAR exit FOO1 worked, it has to come from BAR. – jw013 Jan 15 '16 at 23:26
  • @jw013 Ahh that makes perfect sense. Thank you. – M. Massey Jan 15 '16 at 23:37
  • The whole cgroups zoo was created in Linux (in part at least) to be able to solve exactly this kind of problem cleanly. There is no simple, clean solution to this problem. – vonbrand Jan 16 '16 at 0:03
  • I know the OP said he has no control over the internals of BAR, but for the sake of argument, I think the best solution would be to have BAR wait on its spawned processes. Is this correct? – gardenhead Jan 16 '16 at 11:20
  • How is FOO started? In general you can track processes reliably only from an ancestor process. Doing it from an unrelated process requires some knowledge and possibly some influence about the process. For example, can you arrange for those processes to keep a particular file open? – Gilles Jan 16 '16 at 20:55
0

First try a simpler scenario such as only 2 processes FOO1 and FOO2, and if you were to run within a script for example named parent.sh:

#!/bin/bash

FOO1 &
pid1=$!

echo "Child PID=$pid1 launched"

FOO2 &
pid2=$!

echo "Child PID=$pid2 launched"

BAR exit FOO1
BAR exit FOO2

echo "Pausing to wait for children to finish..."

wait $pid1
wait $pid2

echo "Children finished, moving on."

See if this works with the two FOOs and if so, then implement for the 20.

Explanation

Since we are unable to see the internals of FOOs and BAR process, I am relying only on what you've posted and, as I understand it, you said

  • "Process BAR is used to check FOO and is also used to kill it"
  • meaning, at some point prior to your posted snippet, you started FOO1 somehow, in order for BAR exit FOO1 to affect it

From among your originally posted snippet, you also wrote:

BAR exit FOO1
PID1=$!
  • $! captures the most recently background-ed process
  • but BAR exit FOO1 is not a process going into the background, it is as you claim, a means of using BAR to control FOO: "BAR is used to check FOO and is also used to kill it"
  • thus $! is unlikely to be capturing FOO1's pid

So instead, ensure that right where you actually start a FOO* process, you capture pid. For example if you are running all in parent.sh, and have only 2 processes, you would do:

#!/bin/sh

FOO1 &
pid1=$!

echo "Child PID=$pid1 launched"

FOO2 &
pid2=$!

echo "Child PID=$pid2 launched"
  • lowercase pid1, you can use your original uppercase too, as long as consistent, I just prefer to use lowercase as a convention to differentiate from being mistaken as environment variables
  • The echo is an optional trace so we know what is going on, use it while we are troubleshooting and comment it out with # or delete it once you've solved this problem

We then do the commands you claimed you use to check and kill FOO1 and FOO2:

BAR exit FOO1
BAR exit FOO2

Then,

echo "Pausing to wait for children to finish..."

wait $pid1
wait $pid2

echo "Children finished, moving on."
  • again the echos are optional but informative while you are still troubleshooting this, lets us know what is happening
  • wait for each individual $pid...
  • followed by another optional echo to let you know the waiting is over

CREDIT

Shotts 2009 asynchronous scripts example on the free pdf's page 506

  • Unfortunately BAR starts and stops all FOOs. But it looks like BAR can give me the PID of a FOO so I can use that to check for when it dies. – M. Massey Jan 18 '16 at 16:51
0

You could touch a file for each command and check on that file. Here is a simple example:

#!/usr/local/bin/bash

# Flag to knkow when completed
finished=0;

# The commands t run
declare -a commands=('cmd1' 'cmd2' 'cmd3' 'cmd4');

# Fork all commands
for cmd in "${commands[@]}"; do
        ./$cmd &
done

# Loop to wait for all processes to finish
loop=0;
while [ $finished -eq 0 ]; do
        # Just for visual effect
        loop=$((loop+1));
        echo "Loop $loop";
        sleep 1;

        files=0;
        for cmd in "${commands[@]}"; do
                # Each program shoudl touch a file so that we
                # can know it has completed successfully
                if [ -e ".$cmd" ]; then
                        echo "Process $cmd completed";
                        files=$((files+1));
                else
                        # No need to continue if anything is not done
                        # Error checking here like a TTL value or ps -ef | grep pid etc
                        echo "Waiting for process $cmd";
                        break;
                fi;
        done

        # if we have as many files as executed commands
        if [ "$files" -eq ${#commands[@]} ]; then
                finished=1;
        fi
done;

echo 'All jobs completed successfully';
# Clean up after yourself
for cmd in "${commands[@]}"; do
        rm -f ".$cmd";
done;

Then for testing each cmd1, cmd2 ... cmdN can have something as follows:

#!/usr/local/bin/bash
num=1;

echo "Enter cmd$num";
sleep 1; # use different sleep values to simulate execution
touch ".cmd$num"
echo "Exit cmd$num";
-1

This will evaluate to true if the $PID1 is running: if [kill -s 0 $PID1 ]

You could have multiple if clauses. They will evaluate to false if the PID isn't running. This clause (combined with others) should control the flow of the script as you desire.

  • This doesn't help. If the process with PID x that was started by the script has existed, there may now be an unrelated process with PID x. – Gilles Jan 16 '16 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.