Is this a typo in Bash manual's redirection section?

Question

Note that the order of redirections is significant.  For example, the command

          ls > dirlist 2>&1

   directs both standard output and standard error to the file dirlist, 
   while the command

          ls 2>&1 > dirlist

   directs  only  the  standard  output  to  file  dirlist,  because the 
   standard error was duplicated from the standard output before the standard
   output was redirected to dirlist.

Now, that last part is confusing to me. In that case, any standard error would be printed to the terminal and any STDOUT would go to the dirlist file. That is what would happen, but that is not how I understand the manual.

It seems like it should say "because the standard error was duplicated from the standard output AFTER the standard output was redirected to dirlist". If STDERR was sent to STDOUT before STDOUT was directed to a file, then wouldn't the file contain STDOUT AND STDERR?

Can someone please clear this up for me? Is it just poor reading comprehension on my part? The use of the word duplication seems a little strange to me in this context. Perhaps that is throwing me.

Answer 1

Duplication is really the important part here.

Let's see where the file descriptors are going to before redirection. This is normally the current terminal, e.g.:

STDOUT ---> /dev/pts/1
STDERR ---> /dev/pts/1

Now, if we call ls -l without redirection, output and error messages go to my terminal under /dev/pts/1.

If we first redirect the STDOUT to a file (ls -l > dirlist), it looks like this:

STDOUT ---> /home/bon/dirlist
STDERR ---> /dev/pts/1

When we then redirect STDERR to a duplicate of STDOUT's file descriptor (ls -l > dirlist 2>&1), STDERR goes to a duplicate of /home/bon/dirlist:

STDOUT ---> /home/bon/dirlist
STDERR ---> /home/bon/dirlist

If we would first redirect STDERR to a duplicate of STDOUT's file descriptor (ls -l 2>&1):

STDOUT ---> /dev/pts/1
STDERR ---> /dev/pts/1

and then STDOUT to a file (ls -l 2>&1 > dirlist), we would get this:

STDOUT ---> /home/bon/dirlist
STDERR ---> /dev/pts/1

Here, STDERR is still going to the terminal.

You see, the order in the man page is correct.

---

Testing Redirection

Now, you can test that yourself. Using ls -l /proc/$$/fd/, you see where STDOUT (with fd 1) and STDERR (with fd 2), are going for the current process:

$ ls -l /proc/$$/fd/
total 0
lrwx------ 1 bon bon 64 Jul 24 18:19 0 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 18:19 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 07:41 2 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 18:19 255 -> /dev/pts/1

Let's create a small shell script that shows where your file descriptors are pointed. This way, we always get the state when calling ls, including any redirection from the calling shell.

$ cat > lookfd.sh
#!/bin/sh
ls -l /proc/$$/fd/
^D
$ chmod +x lookfd.sh

(With CtrlD, you send an end-of-file and so stop the cat command reading from STDIN.)

Now, call this script with varying combinations of redirection:

$ ./lookfd.sh 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:08 0 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:08 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:08 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:08 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh > foo.out
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:10 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:10 1 -> /home/bon/foo.out
lrwx------ 1 bon bon 64 Jul 24 19:10 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:10 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh 2>&1 > foo.out
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:10 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:10 1 -> /home/bon/foo.out
lrwx------ 1 bon bon 64 Jul 24 19:10 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:10 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh > foo.out 2>&1
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:11 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:11 1 -> /home/bon/foo.out
l-wx------ 1 bon bon 64 Jul 24 19:11 2 -> /home/bon/foo.out
lr-x------ 1 bon bon 64 Jul 24 19:11 255 -> /home/bon/lookfd.sh

You can see, that the file descriptors 1 (for STDOUT) and 2 (for STDERR) vary. For fun, you could also redirect STDIN and see the result:

$ ./lookfd.sh < /dev/zero
total 0
lr-x------ 1 bon bon 64 Jul 24 19:18 0 -> /dev/zero
lrwx------ 1 bon bon 64 Jul 24 19:18 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:18 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:18 255 -> /home/bon/lookfd.sh

(Question left to the reader: Where does file descriptor 255 point? ;-))

Answer 2

No, the manual is right.

If at first 1 points to the terminal, and 2 also to the terminal, then:

command  2>&1   1>somewhere

redirection evaluatino will happen from left to right.

So it will FIRST evaluate 2>&1, and thus FIRST copy what fd 1 used to point to (ie, the file descriptor of the terminal, usually /dev/tty) into fd 2.

So at that point fd 2 now points to where fd 1 used to point to (the terminal)

And THEN it evaluates the 1>somewhere part, and thus will copy the file descriptor of somewhere in fd 1 (so at that point, fd 1 now points to somewhere, and fd 2 still points to the terminal)

So it does indeed print 1 into "somewhere" and 2 into the terminal, as 2 was duplicated from 1 BEFORE 1 has been changed.

The other order:

command  1>somewhere 2>&1

will first redirect fd 1 to somewhere, and then copy that same reference into fd 2, so at the end 2 also points to somewhere. But they are not "linked" from now on. Each can still be redirected separately.

ex:

command  1>somewhere 2>&1
exec 2>/dev/null

At the end of that one, fd 1 points to somewhere, and fd 2 is directed to /dev/null

Usual names for fd 1 is STDOUT (standard output), and usual name for fd 2 is STDERR (standard error, as it's commonly used to display errors without interfering with STDOUT)

Answer 3

I think the confusing part here is the misapprehension that redirecting stderr to stdout actually connects the two streams.

A perfectly reasonable idea but what happens when you write 2>&1 is stderr takes a peekaboo at what stdout is writing to and writes to the same place itself. Therefore if you subsequently tell stdout to go write somewhere else it has no effect on the destination of stderr which has already been moved.

I think it's a tad counterintuitive myself but that's how it works. Set up where you want to write to first then tell everyone "copy me". Hope that clarifies...

Answer 4

DUPLICATION...

is important, but rather in the sense that it is the source of much confusion. It is really quite simple. This answer is just a "radical" illustration.

The accepted answer is good, but too long and it emphasizes "duplicatition".

The Q wisely ends with:

The use of the word duplication seems a little strange to me in this context. Perhaps that is throwing me.

I use bash notation and define variables "one" and "two" as filehandles "1" and "2". The (output) redirection operator > is an assignment =. & and $ mean "value" of.

The man bash examples (with default "1" added)

ls 1>dirlist 2>&1      # both to dirlist
ls 2>&1 1>dirlist      # 1 to dirlist, 2 stays on tty/screen 

become:

one=dirlist  two=$one

and

two=$one   one=dirlist

And even this is non-automatic to me, and some others I guess. The first line leaves you with $one and $two both containing "dirlist". Of course.

The second line starts with a useless assignment. Both start by definition with "TTY" (a bit symbolic) as their direction; no value is changed by this assignment, and with variables as with filehandles, nothing is magically linked. Variable two is not affected by the following one=dirlist. Of course not.

Sombody here (6 years ago) suggested "point to" instead of "copy" or "duplicate", and then realized: that would be confusing too.

This duplication or pointer semantic is not even needed. Maybe it's the ampersand that needs more attention. The "value of" operator/token/whatever.

If - and only if - you are looking for a way to get a surprising job number on your console, then a "done" message plus as a bonus a file named "2", then you go:

ls 1>2& 2>/dev/null

It reads naturally as "copy"/"duplicate" 1 to 2, and then both together to null. But the idea is wrong, and also the syntax. (but no syntax error, it is valid)

The right way to plan it is to redirect any of the two to null, and then redirect the OTHER to the SAME place:

ls 1>/dev/null 2>&1
# or 
ls 2>/dev/null 1>&2

(the leading "1" can be left away)

(OK the acc. A is not too long, but is too much of a list - or: very good visualisation, not so good explanation)