Here is my script:

#! /bin/bash


(echo first start; sleep 2s; echo first finish;) &> $file_out1 &
(echo second start; sleep 1s; echo second finish;) &> $file_out2 &

wait $P1 $P2 || ret=$?

cat $file_out1
cat $file_out2

exit $ret

Here, the output is:

first start
first finish
second start
second finish

But the output should be:

second start
second finish
first start
first finish

What I mean is, cat $file_out1 should run as soon as P1 is complete and cat $file_out2 should run as soon as P2 is complete.

Moreover, I want to make sure P1 ended successfully, and P2 ended successfully. At the end, how can I know if there was an error in P1 or P2 or both.

In this script, $ret is not working properly (it is always exit 0). After it works, It would be nice to know whether the $ret was from P1 or P2. Say something like P1 has return code x1, P2 has return code x2. And exit the program with a return code which is non-zero if there is a non-zero exit code.


One answer in askubuntu, suggests that we can use --group argument of parallel command to ensures, that the output of each command waits till the command has finished (avoids mixed output).

The command it showed is parallel --group 'echo -n {};sleep {};echo {}' ::: 1 3 2 4.

This command works in my machine. However, I am not understanding how to use this command in my context.

I mean how can I get

second start
second finish
first start
first finish

When the input commands are echo first start; sleep 2s; echo first finish; and echo second start; sleep 1s; echo second finish; using gnu parallel.

Update 2:

The code I am currently at is:

#! /bin/bash

function first {
    echo first start
    sleep 2s
    echo first finish
function second {
    echo second start
    sleep 1s
    echo second finish

export -f first && export -f second

parallel -j2 ::: first second

This actually solves my problem.

  • 1
    your script does not parallelize as you expect, it is false to assert that cat file_out1 or file_out2 do occur "as soon as". They occur one after the other as it is written in the main script. What did you expect exactly ? May 29 at 13:19

1 Answer 1


I suppose you expect each of your task to show output as soon as they terminate, but avoid folding their output, that's why you bufferize it in a file - that's ok, you could also prefix output with sed, let it flow as it comes, then sort it again on demand : that would be in case of unpredictable quantity of data and risk of storage overflow.

Your mistakes :

  • when you fork background processes with &, you lose access to their final status (what you expect in ret=$? is something else)
  • when you wait for their completion, you cannot predict which one will terminate first, and whether you will wait on time to catch every process final status : too late after the process termination, wait will get 127 even if the process final status were 0. And that is unpredictable.

Your wish is to have the same feedback as running sequentially, but with the duration advantage of parallelism, and minimizing the sauce.

Let them execute in their own box, manage the race for output - flock on a file descriptor is clean - and force them to record their final status somewhere - again, a file descriptor is clean.

EDIT GNU parallel was all-in-one solution, as @AhmadIsmail edit say. Let this one anyway.

# wrap background
 tmp=$(mktemp) && trap "rm -f $tmp" RETURN
 exec {safe1}>&1 {safe2}>&2 1>$tmp 2>$tmp # swap descriptors
 (eval $@) # work
 echo $? >&$fdstatus # report status 
 exec >&$safe1 2>&$safe2 # restore descriptors
 flock 1 cat $tmp # report output
mybackground "echo first start; sleep 2s; echo first finish;exit 2" &
mybackground "echo second start; sleep 1s; echo second finish" &
) {fdstatus}> >(
  while read rc;do ((sum+=rc));done;echo sum=$sum;exit $sum ## collect & sum status
) | cat # wait {fdstatus} termination
  • Why does it print sum=2 May 29 at 15:09
  • 1
    Not exactly. It returns control before really terminating. It is because of the >( ... ) that is not waited for. to fix that, put the whole in ( ... ) | cat May 29 at 15:25
  • 1
    I think there is a better solution using parallel --group (please check the update). Your answer is till now the accepted answer. However, I want to keep it open a bit longer to find an easier solution. Till then i am upvoting this answer. May 29 at 15:33
  • 1
    I made a cosmetic refactory Of course, GNU parallel seems to be exactly the right tool, I forgot that, I never used it, damned ! May 29 at 15:50
  • 1
    I found that it had options for showing progress and everything. You can check gnu.org/software/parallel/… May 29 at 15:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .