Semi-asynchronous pipe

Question

Assume I have the following pipe:

a | b | c | d

How can I wait for the completion of c (or b) in sh or bash? This means that script d can start any time (and does not need to be waited for) but requires complete output from c to work correctly.

The use case is a difftool for git that compares images. It is called by git and needs to process its input (the a | b | c part) and display the results of the comparison (the d part). The caller will delete input that is required for a and b. This means that before returning from the script, process c (or b) must terminate. On the other hand, I cannot wait for d because this means I'm waiting for user input.

I know I can write the results of c to a temporary file, or perhaps use a FIFO in bash. (Not sure if the FIFO will help, though.) Is it possible to achieve this without temporary files in sh?

EDIT

Perhaps it would be sufficient if I could find out the process ID of the c (or b) process in a reliable fashion. Then the whole pipe could be started asynchronously, and I could wait for a process ID. Something along the lines of

wait $(a | b | { c & [print PID of c] ; } | d)

EDIT^2

I have found a solution, comments (or still better solutions) are welcome.

Answer 1

a | b | { c; [notify];} | d

The notification can be done e.g. by a signal to a PID which has been passed in an environment variable (kill -USR1 $EXTPID) or by creating a file (touch /path/to/file).

Another idea:

You execute the next process (the one to be able to start you are waiting for) from the pipeline:

a | b | { c; exec >&-; nextprocess;} | d

or

a | b | { c; exec >&-; nextprocess &} | d

Answer 2

If I understand your question correctly, this should work:

a | b | c | { (exec <&3 3<&-; d) &} 3<&0

(the fd 3 trick is because some (most) shells redirect stdin to /dev/null with &).

Answer 3

In bash, you can use a process substitution. The command in the process substitution runs asynchronously, it is not waited for.

a | b | c > >(d)

Answer 4

This is what I have found by trial and error, with the help of Hauke's input:

a | b | { c; kill -PIPE $$; } | d

Equivalently:

a | b | ( c; kill -PIPE $$; ) | d

(The latter is more explicit, because {} will run in a subshell anyway if inside a pipe.)

Some other signals (including QUIT, TERM and USR1) work, too, however in this case the description of the signal is shown on the terminal.

I wonder if this is the original intention of the PIPE signal. According to the manual:

SIGPIPE: The PIPE signal is sent to a process when it attempts to write to a pipe without a process connected to the other end.

This means that when I artificially send a pipe signal to the subshell, it silently terminates, leaving the final consumer (d) alone.

This works in both sh and bash.

Answer 5

You can use the sponge program from the "moreutils" package:

a | b | c | sponge | d

sponge will for the end of c's output before piping it to d. Hope that's what you wanted.

Answer 6

You can just do:

a | b | c | (d ; cat > /dev/null)

So, when d finishes, the cat will absorb the rest of c's output until it finishes.

Ok. After comments, I think the answer is to directly start d in the background.

Do:

a | b | c | (d &)

or use Stephane Chazelas's solution if there are problems with d reading from stdin.