6

Suppose that you redirect, in bash, the standard output of a command cmd to a file named f.out, and the standard error to f.err, using tee to preserve console printing:

cmd 1> >(tee f.out) 2> >(tee f.err)

Then f.out contains the output as well as the error (at least on my system).

Now, if you change the order of redirections:

cmd 2> >(tee f.err) 1> >(tee f.out)

f.out only contains the output (and f.err only contains the error in both cases).

So my question is double: how stderr can be redirected to f.out, and why does the order of redirections impact the result?

Note that if you don't use tee, but for example cat, like this:

cmd 1> >(cat>f.out) 2> >(cat>f.err)

you don't have this issue, and the order of redirections doesn't matter, as expected, and as it would be the case without process substitution (cmd 1>f.out 2>f.err).

2
  • Please have a look at the other answer(s). I deleted mine because it was quite wrong (redirections are read from left to right and not from right to left as I had said).
    – terdon
    Apr 3, 2019 at 18:51
  • @terdon Yes indeed, redirections are read from left to right, but I thought @LL3 and you were finally saying the same thing in different ways. Anyway, like I've said in my previous comment to your deleted answer, my surprise comes from the connection between tee f.err’s stdout and tee f.out’s stdin, via cmd's stdout. It seems that I confused "different processes" and "independent processes". Thank you for your response anyway!
    – Glyph
    Apr 4, 2019 at 7:20

2 Answers 2

8

Order of redirection is important because Bash applies them in the order it finds them on the command it interprets.

This is on purpose so that you can have idioms like > file 2>&1 working as expected i.e. having stderr the same as stdout. This idiom works as in "assign file to stdout and then make stderr equal to stdout", which yields the expected outcome because by the time stderr gets stdout's same value, stdout's value is file. The other way around (ie 2>&1 1> file) won't yield the same outcome because stdout's value is changed after it has been copied to stderr's value. File-descriptors can be considered analogous to regular variables, which have their own values and can be made to get a copy of another variable's value, as in var1="${var2}", and much like such var1 won't follow var2's subsequent value changes, file-descriptor's value won't too.

It is also handy so that you can e.g. swap file-descriptors on the same line, like in 3>&1 1>&2 2>&3-. This swaps fds 1 and 2 using fd 3 as temporary “helper” fd.

As such you can consider redirections as instructions executed sequentially, just as if they were on two separate lines of your command or script.

On your specific case there are also Process Substitutions involved, and those too get executed in the specified sequence inheriting the redirections expressed up to that point

That is, to cap it all:

  1. you first redirect stdout to the process running tee f.out; at this point cmd’s stdout is connected to tee f.out’s stdin, as desired
  2. then you redirect stderr to the process running tee f.err; but this inherits its stdout as per the redirection expressed before, i.e. connected to tee f.out’s stdin

Therefore tee f.err, by innocently outputting to its stdout as well as to f.err file, pipes your cmd’s error messages to tee f.out’s stdin which will therefore receive all messages, outputting them to f.out file as well as to your terminal window.

12
  • 1
    Turns out you're right and I'm wrong: Redirections are processed in the order they appear, from left to right. I deleted my answer since it was accepted and misleading. Thanks for making me look into it more deeply!
    – terdon
    Apr 3, 2019 at 18:51
  • 1
    This is interesting as it provides a way for making one of the two streams take an "extra trip" through a generic command before recombining them. That could, most simply, be used for things like, eh, upper-casing all errors (replace the second tee by tr '[:lower:]' '[:upper:]').
    – Kusalananda
    Apr 3, 2019 at 19:03
  • 1
    Yeah, that seems about right, apart from not quoting those globbing patterns ;-) Thanks. The pipe in that command plays the role of the first process substitution of the command in the question. Still, it uses a non-standard process substitution. But it's of minor importance with regards to this question.
    – Kusalananda
    Apr 3, 2019 at 19:58
  • 1
    Sorry, I thought a saw a pipe in there somewhere, which I now see is not there. I'm tired... Process substitutions is an extension to the POSIX standard for the shell, implemented by a handful of shells, bash being one of them. The yash shell has the exact same syntax for something it calls "process redirection", which is different.
    – Kusalananda
    Apr 3, 2019 at 20:08
  • 1
    Having both process substitutions modify the data might help in visualizing how the data flows, e.g. something like cmd 1> >(sed -e 's/^/x /' | tee f.out) 2> >(sed -e 's/^/y /' | tee f.err)
    – ilkkachu
    Apr 3, 2019 at 21:54
0

I came across this question which was part of the answer to a test I am working on. The general case of the command is to run a cmd and display on terminal, with stderr modified (indented & coloured red) and then save what appears on screen in a log file. Removing the colour coding from the log.

So, the example cmd creating 2 stdout and 2 stderr lines, is

{ ls -l /usr/bin/col{1,2,a,b} 2>&1 >&3 | sed 's/.*/\t\o33[31mSTDERR: &\o33[0m/'; } 3>&1  

Now, putting a copy into a log file

{ ls -l /usr/bin/col{1,2,a,b} 2>&1 >&3 | sed 's/.*/\t\o33[31mSTDERR: &\o33[0m/'; } 3>&1 | tee >( sed 's/\o33//g;s/\[31m//g;s/\[0m//' > /tmp/file )

This works fine, however; is there a better way to achieve this?

1
  • Well, your "answer" is not really an answer to my question, but… Concerning your question, I would say ls -l /usr/bin/col{1,2,a,b} 2> >(sed 's/^/\tSTDERR: /' | tee -a /tmp/file | sed 's/.*/\o33[31m&\o33[0m/') 1> >(tee -a /tmp/file)
    – Glyph
    Apr 23, 2020 at 9:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .