1

I have several processes that are producing output on STDOUT and STDERR, which I have redirected to numbered file descriptors, and I want to collate all the output together into a single file.

I have naively tried

[input processes] | cat <3 <4 <5 2>&1 >[output file]

but of course, this does not work as cat will wait until it's STDIN pipe is closed before reading data from any of the subsequent ones, causing my process to hang when the other pipes' buffers become full.

Any suggestions?

4

Collating output together is not really the dual of tee. tee makes multiple copies of its input, whereas collating output does not involve any merging of data.

To merge output sources, just redirect them all to the same file descriptor. The interleaving of the sources is somewhat unpredictable in general, but sufficiently small writes to a pipe are guaranteed to be atomic. (Being able to tell the boundaries from the read side is another story.)

{ data_source_1 &
  data_source_2 &
  wait; } >merged_output

If you're getting input from multiple file descriptors and you want to merge them, pass each of them through.

{ cat <&3 & cat <&4 & wait; } >merged_ouput

But usually you'd be able to redirect all the file descriptors to the same destination.

… 3>merged_ouput 4>&3
0

You need to add STDERR and STDOUT into STDOUT in each process.

Example:
processA 2>&1 | processB | processC

STDOUT and STDERR would be redirected via the PIPE as the STDIN for processB. However processC will only get the STDOUT of ProcessB as STDIN.

So if you need STDOUT and STDERR of processB as STDIN for processC it would be: processA 2>&1 | processB 2>&1 | processC

If you really don't want to use pipes, but want to just redirect all kind of output into a single file, you can do:

(processA; processB) >./just_STDOUT.txt

(processA 2>&1; processB 2>&1) >./STDOUT_AND_STDERR.txt

  • 4
    Or just (process A; process B) > ./STDOUT_AND_STDERR.txt 2>&1 for your last example. Or (process A & process B & wait) > ./STDOUT_AND_STDERR.txt 2>&1 if they need to run in parallel. – Stéphane Chazelas May 10 '17 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.