11

Assume I have the following pipe:

a | b | c | d

How can I wait for the completion of c (or b) in sh or bash? This means that script d can start any time (and does not need to be waited for) but requires complete output from c to work correctly.

The use case is a difftool for git that compares images. It is called by git and needs to process its input (the a | b | c part) and display the results of the comparison (the d part). The caller will delete input that is required for a and b. This means that before returning from the script, process c (or b) must terminate. On the other hand, I cannot wait for d because this means I'm waiting for user input.

I know I can write the results of c to a temporary file, or perhaps use a FIFO in bash. (Not sure if the FIFO will help, though.) Is it possible to achieve this without temporary files in sh?

EDIT

Perhaps it would be sufficient if I could find out the process ID of the c (or b) process in a reliable fashion. Then the whole pipe could be started asynchronously, and I could wait for a process ID. Something along the lines of

wait $(a | b | { c & [print PID of c] ; } | d)

EDIT^2

I have found a solution, comments (or still better solutions) are welcome.

|improve this question
  • You mean you want d to start processing c's output only after c has completed? You don't want d to start processing each output line as it comes? – terdon♦ Feb 24 '14 at 13:03
  • @terdon: No, d is free to start whenever it likes, but c needs to finish before I can move on. – krlmlr Feb 24 '14 at 13:04
  • That seems to be self-contradictory, if d can start when it likes, what exactly are you waiting for? – terdon♦ Feb 24 '14 at 13:05
  • @terdon: Expanded to show use case. – krlmlr Feb 24 '14 at 13:11
  • If d does not use the output of c then it seems not to make any sense to make d part of the pipeline. But if d does use the input then d must work a while on its input after having read all of it for your approach to make any difference. – Hauke Laging Feb 24 '14 at 13:12
6
a | b | { c; [notify];} | d

The notification can be done e.g. by a signal to a PID which has been passed in an environment variable (kill -USR1 $EXTPID) or by creating a file (touch /path/to/file).

Another idea:

You execute the next process (the one to be able to start you are waiting for) from the pipeline:

a | b | { c; exec >&-; nextprocess;} | d

or

a | b | { c; exec >&-; nextprocess &} | d
|improve this answer
  • Thanks. But then, creating a temporary file for the intermediate output is more verbose and easier to parse (for a human). Also, how do I avoid the race condition with the signal? ... I was hoping for a cleaner solution, but if this is the way to go, then so be it. – krlmlr Feb 24 '14 at 13:16
  • @krlmlr What race condition? – Hauke Laging Feb 24 '14 at 13:19
  • notify could be executed before I set up the signal trap. How can I make sure not to miss that signal? – krlmlr Feb 24 '14 at 13:22
  • @krlmlr You stop the script which calls the pipeline until it receives a signal itself which is sent after the trap has been defined. – Hauke Laging Feb 24 '14 at 13:26
  • Your edit: The pipe is called in a loop, over which I have no control. Unfortunately, { c; bg; } or { c; exit 0; } doesn't seem to work. – krlmlr Feb 24 '14 at 13:34
5

If I understand your question correctly, this should work:

a | b | c | { (exec <&3 3<&-; d) &} 3<&0

(the fd 3 trick is because some (most) shells redirect stdin to /dev/null with &).

|improve this answer
4

In bash, you can use a process substitution. The command in the process substitution runs asynchronously, it is not waited for.

a | b | c > >(d)
|improve this answer
  • Thanks. This is bash only, right -- no sh support? – krlmlr Feb 25 '14 at 6:17
  • @krlmlr Bash/zsh only (and ksh93 with a few modifications). – Gilles Feb 25 '14 at 8:34
3

This is what I have found by trial and error, with the help of Hauke's input:

a | b | { c; kill -PIPE $$; } | d

Equivalently:

a | b | ( c; kill -PIPE $$; ) | d

(The latter is more explicit, because {} will run in a subshell anyway if inside a pipe.)

Some other signals (including QUIT, TERM and USR1) work, too, however in this case the description of the signal is shown on the terminal.

I wonder if this is the original intention of the PIPE signal. According to the manual:

SIGPIPE: The PIPE signal is sent to a process when it attempts to write to a pipe without a process connected to the other end.

This means that when I artificially send a pipe signal to the subshell, it silently terminates, leaving the final consumer (d) alone.

This works in both sh and bash.

|improve this answer
0

You can use the sponge program from the "moreutils" package:

a | b | c | sponge | d

sponge will for the end of c's output before piping it to d. Hope that's what you wanted.

|improve this answer
0

You can just do:

a | b | c | (d ; cat > /dev/null)

So, when d finishes, the cat will absorb the rest of c's output until it finishes.

Ok. After comments, I think the answer is to directly start d in the background.

Do:

a | b | c | (d &)

or use Stephane Chazelas's solution if there are problems with d reading from stdin.

|improve this answer
  • I'm afraid my use case is rather the other way round: c finishes earlier than d, and I have to wait until c finishes but don't care for d. – krlmlr Feb 24 '14 at 19:53
  • Sorry, I don't get it. What do you want to do when c finishes and d is still running? Kill d? – angus Feb 24 '14 at 20:10
  • d has a GUI and can run until the user closes it. – krlmlr Feb 24 '14 at 20:27
  • OK... but this already happens. When a, b, and c finish their work, they close stdout and exit; and only d remains running. Do you want to send d to background when c finishes? Is that it? – angus Feb 24 '14 at 20:49
  • Yes, d should be sent to the background once c finishes. – krlmlr Feb 24 '14 at 20:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.