How to redirect stdout to a file, and stdout+stderr to another one?

Question

How can I achieve

cmd >> file1 2>&1 1>>file2

That is, the stdout and stderr should redirect to one file (file1) and only stdout (file2) should redirect to another (both in append mode)?

Answer 1

Problem is that when you redirect your output, it's not available anymore for the next redirect. You can pipe to tee in a subshell to keep the output for the second redirection:

( cmd | tee -a file2 ) >> file1 2>&1

or if you like to see the output in terminal:

( cmd | tee -a file2 ) 2>&1 | tee -a file1

---

To avoid adding the stderr of the first tee to file1, you should redirect the stderr of your command to some file descriptor (e.g. 3), and later add this to stdout again:

( 2>&3 cmd | tee -a file2 ) >> file1 3>&1
# or
( 2>&3 cmd | tee -a file2 ) 3>&1 | tee -a file1

(thanks @fra-san)

Answer 2

With zsh:

cmd >& out+err.log > out.log

In append mode:

cmd >>& out+err.log >> out.log

In zsh, and provided the mult_ios option has not been disabled, when a file descriptor (here 1) is redirected several times for writing, then the shell implements a built-in tee to duplicate the output to all targets.

Answer 3

You could : tag stdout (using an UNBUFFERED sed, ie: sed -u ...), have stderr also go to stdout (untagged, as it didn't go through that tagging sed), and thus be able to differentiate the 2 in the resulting logfile.

The following: is slow (It can be seriously optimized, by using for exemple a perl script instead of the while ... ; do ... ;done, for exemple, that will spawning subshells & commands at every lines!), weird (it seems I need the 2 {} stages to in one rename stdout and then in the other one add the "falled through" stderr to it), etc. But it is : a "proof of concept", that will try to keep the output's order the most of stdout & stderr as much as possible:

#basic principle (some un-necessary "{}" to visually help see the layers):
# { { complex command ;} | sed -e "s/^/TAGstdout/" ;} 2>&1 | read_stdin_and_redispatch

#exemple:
# complex command = a (slowed) ls of several things (some existing, others not)
#  to see if the order of stdout&stderr is kept

#preparation, not needed for the "proof of concept", but needed for our specific exemple setup:
\rm out.file out_AND_err.file unknown unknown2 
touch existing existing2 existing3

#and the (slow, too many execs, etc) "proof of concept":
uniquetag="_stdout_" # change this to something unique, that will NOT appear in all the commands outputs... 
                     # avoid regexp characters ("+" "?" "*" etc) to make it easy to remove with another sed later on.

{
   { for f in existing unknown existing2 unknown2 existing3 ; do ls -l "$f" ; sleep 1; done ;
   } | sed -u -e "s/^/${uniquetag}/" ;
} 2>&1 | while IFS="" read -r line ; do
    case "$line" in
       ${uniquetag}*) printf "%s\n" "$line" | tee -a out_AND_err.file | sed -e "s/^${uniquetag}//" >> out.file ;; 
        *)            printf "%s\n" "$line"       >> out_AND_err.file ;;   
    esac; 
done;

# see the results:
grep "^" out.file out_AND_err.file

Answer 4

If order of output must be: stdout then stderr; there is no solution with redirection only.
The stderr must be stored to a temporal file

cmd 2>>file-err | tee -a file1 >>file2
cat file-err >> file1
rm file-err

Description:

The only way to redirect one output (an fd like stdout or stderr) to two files is to reproduce it. The command tee is the correct tool to reproduce a file descriptor content. So, an initial idea to have one output on two files would be to use:

... |  tee file1 file2

That reproduces the stdin of tee to both files (1 & 2) leaving the output of tee still unused. But we need to append (use -a) and only need one copy. This solve both issues:

... | tee -a file1 >>file2

To supply tee with stdout (the one to repeat) we need to consume stderr directly out of the command. One way, if order is not important (order of output will (most probably) be preserved as generated, whichever is output first will be stored first). Either:

1. cmd 2>>file1 | tee -a file2 >>file1
2. cmd 2>>file1 > >( tee -a file2 >>file1 )
3. ( cmd | tee -a file2 ) >> file1 2>&1

Option 2 works only in some shells. Option 3 uses an additional subshell (slower) but use the file names only once.

But if stdout must be first (whichever order output is generated) we need to store stderr to append it to file at the end (first solution posted).

Answer 5

In the interests of diversity:

If your system supports /dev/stderr, then

(cmd | tee -a /dev/stderr) 2>> file1 >> file2

will work.  The standard output of the cmd is sent to both the stdout and the stderr of the pipeline. The standard error of the cmd bypasses the tee and comes out the stderr of the pipeline.

So

• the stdout of the pipeline is just the stdout of the cmd, and
• the stderr of the pipeline is the stdout and stderr of the cmd, intermixed.

It's then a simple matter of sending those streams to the correct files.

As with almost any approach like this (including Stéphane’s answer), file1 may get lines out of order.