Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Assume I have the following pipe:

a | b | c | d

How to wait for completion of c (or b) in sh or bash? This means that script d can start any time (and does not need to be waited for) but requires complete output from c to work correctly.

The use case is a difftool for git that compares images. It is called by git and needs to process its input (the a | b | c part) and display the results of the comparison (the d part). The caller will delete input that is required for a and b, this means that before returning from the script, process c (or b) must terminate. On the other hand, I cannot wait for d because this means I'm waiting for user input.

I know I can write the results of c to a temporary file, or perhaps use a FIFO in bash. (Not sure if the FIFO will help, though.) Is it possible to achieve this without temporary files in sh?

EDIT: Perhaps it would be sufficient if I could find out the process ID of the c (or b) process in a reliable fashion. Then the whole pipe could be started asynchronously, and I could wait for a process ID. Something along the lines of

wait $(a | b | { c & [print PID of c] ; } | d)

EDIT^2: I have found a solution, comments (or still better solutions) are welcome.

share|improve this question
    
You mean you want d to start processing c's output only after c has completed? You don't want d to start processing each output line as it comes? –  terdon Feb 24 at 13:03
    
@terdon: No, d is free to start whenever it likes, but c needs to finish before I can move on. –  krlmlr Feb 24 at 13:04
    
That seems to be self-contradictory, if d can start when it likes, what exactly are you waiting for? –  terdon Feb 24 at 13:05
    
@terdon: Expanded to show use case. –  krlmlr Feb 24 at 13:11
    
If d does not use the output of c then it seems not to make any sense to make d part of the pipeline. But if d does use the input then d must work a while on its input after having read all of it for your approach to make any difference. –  Hauke Laging Feb 24 at 13:12

5 Answers 5

a | b | { c; [notify];} | d

The notification can be done e.g. by a signal to a PID which has been passed in an environment variable (kill -USR1 $EXTPID) or by creating a file (touch /path/to/file).

Another idea:

You execute the next process (the one to be able to start you are waiting for) from the pipeline:

a | b | { c; exec >&-; nextprocess;} | d

or

a | b | { c; exec >&-; nextprocess &} | d
share|improve this answer
    
Thanks. But then, creating a temporary file for the intermediate output is more verbose and easier to parse (for a human). Also, how do I avoid the race condition with the signal? ... I was hoping for a cleaner solution, but if this is the way to go, then so be it. –  krlmlr Feb 24 at 13:16
    
@krlmlr What race condition? –  Hauke Laging Feb 24 at 13:19
    
notify could be executed before I set up the signal trap. How can I make sure not to miss that signal? –  krlmlr Feb 24 at 13:22
    
@krlmlr You stop the script which calls the pipeline until it receives a signal itself which is sent after the trap has been defined. –  Hauke Laging Feb 24 at 13:26
    
Your edit: The pipe is called in a loop, over which I have no control. Unfortunately, { c; bg; } or { c; exit 0; } doesn't seem to work. –  krlmlr Feb 24 at 13:34

This is what I have found by trial and error, with the help of Hauke's input:

a | b | { c; kill -PIPE $$; } | d

Equivalently:

a | b | ( c; kill -PIPE $$; ) | d

(The latter is more explicit, because {} will run in a subshell anyway if inside a pipe.)

Some other signals (including QUIT, TERM and USR1) work, too, however in this case the description of the signal is shown on the terminal.

I wonder if this is the original intention of the PIPE signal. According to the manual:

SIGPIPE: The PIPE signal is sent to a process when it attempts to write to a pipe without a process connected to the other end.

This means that when I artificially send a pipe signal to the subshell, it silently terminates, leaving the final consumer (d) alone.

This works in both sh and bash.

share|improve this answer

If I understand your question correctly, this should work:

a | b | c | { (exec <&3 3<&-; d) &} 3<&0

(the fd 3 trick is because some (most) shells redirect stdin to /dev/null with &).

share|improve this answer

In bash, you can use a process substitution. The command in the process substitution runs asynchronously, it is not waited for.

a | b | c > >(d)
share|improve this answer
    
Thanks. This is bash only, right -- no sh support? –  krlmlr Feb 25 at 6:17
    
@krlmlr Bash/zsh only (and ksh93 with a few modifications). –  Gilles Feb 25 at 8:34

You can just do

a | b | c | (d ; cat > /dev/null)

So, when d finishes, the cat will absorb the rest of c's output until it finishes.

Ok. After comments, I think the answer is to directly start d in the background.

Do either

a | b | c | (d &)

or use Stephane Chazelas's solution if there are problems with d reading from stdin.

share|improve this answer
    
I'm afraid my use case is rather the other way round: c finishes earlier than d, and I have to wait until c finishes but don't care for d. –  krlmlr Feb 24 at 19:53
    
Sorry, I don't get it. What do you want to do when c finishes and d is still running? Kill d? –  angus Feb 24 at 20:10
    
d has a GUI and can run until the user closes it. –  krlmlr Feb 24 at 20:27
    
OK... but this already happens. When a, b, and c finish their work, they close stdout and exit; and only d remains running. Do you want to send d to background when c finishes? Is that it? –  angus Feb 24 at 20:49
    
Yes, d should be sent to the background once c finishes. –  krlmlr Feb 24 at 20:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.