3

I'm trying to understand what causes the difference in these 2 constructs that I thought were functionally equivalent:

$ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') | sed 's/^/stdout: /'   
stdout: stderr: foo
$ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') > >(sed 's/^/stdout: /')
stderr: foo

EDIT: If I understood user1133275 right, he suggests that >(sed 's/^/stdout: /') is not run unless the subshell ( echo foo >&2 ) outputs to stdout. However, that would mean that the following:

$ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') > >(echo baz)   
baz
stderr: foo

should not display baz.

EDIT 2: Perhaps also of interest, sed doesn't output stdout: on empty input, even when piped:

$ sed 's/^/stdout: /' < /dev/null
$ printf "" | sed 's/^/stdout: /'
$
  • You've tagged with both bash and zsh. Which one is it? – Kusalananda Jan 18 at 17:45
  • 1
    @Kusalananda I checked both. – JoL Jan 18 at 17:45
  • Regarding your "EDIT 2": sed only outputs lines that are fed to it, or that it creates itself. In neither of those two commands are you giving sed a single properly terminated line of input. – Kusalananda Jan 18 at 18:14
  • @Kusalananda The results are expected, but when reality doesn't seem to follow common sense, it's time to recheck common sense. user1133275's previous answer also seemed to depend on sed printing even on empty input, that's why I added it. – JoL Jan 18 at 18:15
3

Your first command,

( echo foo >&2 ) 2> >(sed 's/^/stderr: /') | sed 's/^/stdout: /'

in simplified form (using a temporary file to hold the data produced by echo):

{ echo foo 2>file >&2; sed 's/^/stderr: /' file; } | sed 's/^/stdout: /'

I.e., the first sed reads what is produced on standard error from the echo and writes to standard output and the second sed reads and modifies that.

Your second command,

( echo foo >&2 ) 2> >(sed 's/^/stderr: /') > >(sed 's/^/stdout: /')

in simplified form,

echo foo 2>file >&2; sed 's/^/stderr: /' file; sed 's/^/stdout: /' </dev/null

Here, the sed that gets the standard error output produces output while the other sed that gets the standard output output (which is nothing) does not produce any output (since it didn't get any input and since it did not insert or append any data).

Another way of formulating it:

First command:

( echo foo >&2 ) 2>file
sed 's/^/stderr: /' file | sed 's/^/stdout: /'

Second command:

( echo foo >&2 ) 2>file >otherfile
sed 's/^/stderr: /' file
sed 's/^/stdout: /' otherfile

In short, the second sed in the second command never reads anything. In particular, it does not read the output from the first sed as in the first command.


Using extremely simplified symbols, the first command is something like

utility-writing-to-stderr 2> >(something1) | something2

where something1 writes to standard output, which is read by something2.

The second command, using the same notation,

utility-writing-to-stderr 2> >(something1) >(something2)

i.e. something1 and something2 is not even connected to each other, and something2 can not in any way read what something1 is producing. Furthermore, since utility-writing-to-stderr does not produce anything on its standard output stream, something2 will have nothing to read from its standard input.

  • @P_Yadav Ha! It was backwards for me. I was more confused by seeing that | sed 's/^/stdout: /' would take input from >(sed 's/^/stderr: /'), when >(sed 's/^/stderr: /')'s output should be inherited to be the TTY. It seems that the shell assigns a >() subshell a different output to what it inherits if it's used as part of a redirection. That different output is that of the command it's redirecting. Indeed, when I asked the question, I thought sed was differentiating its input filetype (named vs unnamed pipe?) and oddly outputting to the TTY on empty input. – JoL Jan 18 at 19:13
  • @JoL Nothing is outputting directly to the TTY (/dev/tty) here. It's the shell that dumps the final standard output stream to the console. – Kusalananda Jan 18 at 19:35
  • I made a followup question about that, but I see what I thought was natural behavior was only done by zsh on some cases and never by bash: unix.stackexchange.com/questions/495346/… – JoL Jan 18 at 20:05
1

> is operating on ( echo foo >&2 ) and | is operating on >(sed 's/^/stderr: /') because of the order of redirection in shells. E.G.

$ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /' > >(sed 's/^/stdout: /') )
stdout: stderr: foo

or

$ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') | cat > >(sed 's/^/stdout: /')
stdout: stderr: foo

vs explicit examples the order in the question

$ ( ( echo foo >&2 ) > >(sed 's/^/stdout: /') ) 2> >(sed 's/^/stderr: /')
stderr: foo

or

$ ( ( echo foo >&2 )  | sed 's/^/stdout: /' ) 2> >(sed 's/^/stderr: /')
stderr: foo
  • Do you mean that sed is not run unless that subshell outputs? But then why if you replace sed with an echo baz it does output? Let me edit the question to make that more clear. – JoL Jan 18 at 17:59
  • I've edited the question to include what I've asked in my previous comment. – JoL Jan 18 at 18:03
  • 1
    You're right that changing the order of redirections "fixes" the issue, but the part that I was misunderstanding in particular was that these >() subshells get assigned same piped output when they're part of a redirection, instead of outputting to the inherited output that is the TTY, like they would if they were not part of a redirection. E.g. >() get's a different output when used like tee >(...) | ... than when used like tee 2> >(...) | .... I upvoted both answers, but I think I'm accepting Kusalananda's because it more clearly highlights the file descriptor inheritance issue. – JoL Jan 18 at 20:23
1

The two commands are not equivalent.

The command (1):

( one ) 2> >(two) | three 

Sends the (std) output of two to three (More about stderr below).

The command (2):

( one ) 2> >(two) > >(three)

Sends the (std) output of one to three (stderr goes to two).


The painful detail is:

Command 1: ( one ) 2> >(two) | three

  • Command three is started (sed 's/^/stdout: /') waiting for input on stdin.
  • Redirection of 2 (stderr) to the stdin of two takes place.
  • Internal redirection of one takes place (echo foo >&2).
  • The command gets executed sending foo to stderr (nothing to stdout).
  • The word foo gets redirected from (stderr of one) to (stdin of two).
  • The command in two gets executed (sed 's/^/stderr: /')
  • The now modified output of two (stderr: foo) is sent to the stdout of two.
  • The present stdout of two goes through the pipe to three.
  • The command three which was waiting for input from the start gets the output.
  • The output gets modified adding a leading stdout:.
  • The final string stdout: stderr: foo goes to tty.

Command 2: ( one ) 2> >(two) > >(three)

  • The last redirection (>) gets built first.
  • The command three is started waiting for input on stdin only (>()).
  • The command three gets only the stdout of one.
  • The command two is started waiting for input.
  • The command two gets (only) the stderr of one.
  • The internal redirection of one connects the stdout of one to its stderr.
  • The command one gets executed sending foo to the stderr (of one).
  • foo goes to two
  • two change the string received to stderr: foo and sends it to tty.
  • three receives an empty input form the stdout of one.
  • three prints nothing (as there is no line to process).

Understand that this command:

printf '' | sed 's/^/initial: /'

has no output (and there is no way to make it work).


Changing the order: Command 3: ( one ) > >(three) 2> >(two)

Makes the output of two go to three (not the tty) detail left as an exercise.

0

I upvoted @Kusalananda's very clear-cut answer, but I venture to (belatedly) add this to his answer, as a small illustration of process substitution at work:

Your second proposition would work with a small typographical change (and a comparatively large change in its faulty logic):

  $ ( echo foo >&2 ) 2> >(sed 's/^/stderr: /') > >(sed 's/^/stdout: /')
  stderr: foo    # for the well explained reason above

But:

    v                                          vv
  $ ( (echo foo >&2) 2> >(sed 's/^/stderr: /') )> >(sed 's/^/stdout: /')  
  stdout: stderr: foo

Because adding ( )> as illustrated (or >( ), both work fine) actually allows you to create a file via process substitution, whose output can be redirected as input toward >(sed 's/^/stdout: /').

I thought about adding this to illustrate how process substitution can be embedded in another process substitution. For readability I would not push it to more levels of embedding. But if for any reason you need to avoid a pipe and the associated shell, this is a good solution.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.