12
Note that the order of redirections is significant.  For example, the command

          ls > dirlist 2>&1

   directs both standard output and standard error to the file dirlist, 
   while the command

          ls 2>&1 > dirlist

   directs  only  the  standard  output  to  file  dirlist,  because the 
   standard error was duplicated from the standard output before the standard
   output was redirected to dirlist.

Now, that last part is confusing to me. In that case, any standard error would be printed to the terminal and any STDOUT would go to the dirlist file. That is what would happen, but that is not how I understand the manual.

It seems like it should say "because the standard error was duplicated from the standard output AFTER the standard output was redirected to dirlist". If STDERR was sent to STDOUT before STDOUT was directed to a file, then wouldn't the file contain STDOUT AND STDERR?

Can someone please clear this up for me? Is it just poor reading comprehension on my part? The use of the word duplication seems a little strange to me in this context. Perhaps that is throwing me.

  • See stackoverflow.com/q/14615653/1030675 – choroba Jul 24 '13 at 15:48
  • 1
    A classic case of mixing up operations that are "by value" vs "by reference". When you duplicate a file descriptor it is a by value operation. In programming, after a = 1; b = a; a = 2 you expect a == 2 && b == 1 to be true. The redirection 2>&1 is similar to the b = a assignment - it is by value, not by reference. 2>&1 does not wed file descriptor 2 to file descriptor 1 for all of eternity - they are still 2 distinct file descriptors, who happen to point to the same file. – jw013 Jul 30 '13 at 20:15
22

Duplication is really the important part here.

Let's see where the file descriptors are going to before redirection. This is normally the current terminal, e.g.:

STDOUT ---> /dev/pts/1
STDERR ---> /dev/pts/1

Now, if we call ls -l without redirection, output and error messages go to my terminal under /dev/pts/1.

If we first redirect the STDOUT to a file (ls -l > dirlist), it looks like this:

STDOUT ---> /home/bon/dirlist
STDERR ---> /dev/pts/1

When we then redirect STDERR to a duplicate of STDOUT's file descriptor (ls -l > dirlist 2>&1), STDERR goes to a duplicate of /home/bon/dirlist:

STDOUT ---> /home/bon/dirlist
STDERR ---> /home/bon/dirlist

If we would first redirect STDERR to a duplicate of STDOUT's file descriptor (ls -l 2>&1):

STDOUT ---> /dev/pts/1
STDERR ---> /dev/pts/1

and then STDOUT to a file (ls -l 2>&1 > dirlist), we would get this:

STDOUT ---> /home/bon/dirlist
STDERR ---> /dev/pts/1

Here, STDERR is still going to the terminal.

You see, the order in the man page is correct.


Testing Redirection

Now, you can test that yourself. Using ls -l /proc/$$/fd/, you see where STDOUT (with fd 1) and STDERR (with fd 2), are going for the current process:

$ ls -l /proc/$$/fd/
total 0
lrwx------ 1 bon bon 64 Jul 24 18:19 0 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 18:19 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 07:41 2 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 18:19 255 -> /dev/pts/1

Let's create a small shell script that shows where your file descriptors are pointed. This way, we always get the state when calling ls, including any redirection from the calling shell.

$ cat > lookfd.sh
#!/bin/sh
ls -l /proc/$$/fd/
^D
$ chmod +x lookfd.sh

(With CtrlD, you send an end-of-file and so stop the cat command reading from STDIN.)

Now, call this script with varying combinations of redirection:

$ ./lookfd.sh 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:08 0 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:08 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:08 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:08 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh > foo.out
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:10 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:10 1 -> /home/bon/foo.out
lrwx------ 1 bon bon 64 Jul 24 19:10 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:10 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh 2>&1 > foo.out
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:10 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:10 1 -> /home/bon/foo.out
lrwx------ 1 bon bon 64 Jul 24 19:10 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:10 255 -> /home/bon/lookfd.sh
$ ./lookfd.sh > foo.out 2>&1
$ cat foo.out 
total 0
lrwx------ 1 bon bon 64 Jul 24 19:11 0 -> /dev/pts/1
l-wx------ 1 bon bon 64 Jul 24 19:11 1 -> /home/bon/foo.out
l-wx------ 1 bon bon 64 Jul 24 19:11 2 -> /home/bon/foo.out
lr-x------ 1 bon bon 64 Jul 24 19:11 255 -> /home/bon/lookfd.sh

You can see, that the file descriptors 1 (for STDOUT) and 2 (for STDERR) vary. For fun, you could also redirect STDIN and see the result:

$ ./lookfd.sh < /dev/zero
total 0
lr-x------ 1 bon bon 64 Jul 24 19:18 0 -> /dev/zero
lrwx------ 1 bon bon 64 Jul 24 19:18 1 -> /dev/pts/1
lrwx------ 1 bon bon 64 Jul 24 19:18 2 -> /dev/pts/1
lr-x------ 1 bon bon 64 Jul 24 19:18 255 -> /home/bon/lookfd.sh

(Question left to the reader: Where does file descriptor 255 point? ;-))

  • +1 - excellent answer. Extremely well written and terrific examples. Thank you!!! – slm Jul 24 '13 at 18:50
  • I see, I think my misunderstanding was that the redirection would be persistent for all following commands, so that any STDERR for the rest of the line would go to STDOUT. – Gregg Leventhal Jul 31 '13 at 13:56
2

No, the manual is right.

If at first 1 points to the terminal, and 2 also to the terminal, then:

command  2>&1   1>somewhere

redirection evaluatino will happen from left to right.

So it will FIRST evaluate 2>&1, and thus FIRST copy what fd 1 used to point to (ie, the file descriptor of the terminal, usually /dev/tty) into fd 2.

So at that point fd 2 now points to where fd 1 used to point to (the terminal)

And THEN it evaluates the 1>somewhere part, and thus will copy the file descriptor of somewhere in fd 1 (so at that point, fd 1 now points to somewhere, and fd 2 still points to the terminal)

So it does indeed print 1 into "somewhere" and 2 into the terminal, as 2 was duplicated from 1 BEFORE 1 has been changed.

The other order:

command  1>somewhere 2>&1

will first redirect fd 1 to somewhere, and then copy that same reference into fd 2, so at the end 2 also points to somewhere. But they are not "linked" from now on. Each can still be redirected separately.

ex:

command  1>somewhere 2>&1
exec 2>/dev/null

At the end of that one, fd 1 points to somewhere, and fd 2 is directed to /dev/null

Usual names for fd 1 is STDOUT (standard output), and usual name for fd 2 is STDERR (standard error, as it's commonly used to display errors without interfering with STDOUT)

  • @Michael-mrozek : thanks for the edit, but I insist on saying "copy" instead of "duplicate" as "duplicate" could lead one to believe that from now on both are "the same thing", which is not true. ex: cmd 1>somewhere 2>&1 ; exec 2>/dev/null : after the exec, only 2 has been redirected to /dev/null (1 is still going to "somewhere"). I do need help to come up with a way to say "what 1 points to" instead of "the fd 1", however... as that is also confusing... – Olivier Dulac Jul 24 '13 at 17:19
  • 1
    I'm not sure what you mean; you're the one who changed it from "copy" to "duplicate". All I did was capitalize and format things, I didn't change a word – Michael Mrozek Jul 24 '13 at 17:23
  • doh... ^^ sorry. And I edited again to reformulate to make more precise what is copied into what ^^ – Olivier Dulac Jul 24 '13 at 17:26
1

I think the confusing part here is the misapprehension that redirecting stderr to stdout actually connects the two streams.

A perfectly reasonable idea but what happens when you write 2>&1 is stderr takes a peekaboo at what stdout is writing to and writes to the same place itself. Therefore if you subsequently tell stdout to go write somewhere else it has no effect on the destination of stderr which has already been moved.

I think it's a tad counterintuitive myself but that's how it works. Set up where you want to write to first then tell everyone "copy me". Hope that clarifies...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.