How can I use BASH magic to achieve this?
I want to only see stderr output on the screen,
but I want both stdout and stderr to be written to a file.

Clarification: I want both stdout and stderr to end up in the same file. In the order they happen.
Unfortunately none of the answers below does this.


9 Answers 9


Even without any redirection, or with nothing but >logfile 2>&1, you're not guaranteed to see output in order of generation.

For starters, the stdout from the application will be line-buffered (to tty) or buffered (to a pipeline) but stderr is unbuffered, so relationships between the order of output are broken as far as a reader is concerned. Subsequent stages in any pipeline you could concoct will not get deterministically ordered access to the two streams (they are conceptually things happening in parallel, and you're always subject to the scheduler - if by the time your reader gets a slice the writer has already written to both pipes, you cannot tell which came first).

"[T]he order they happen" is only really known to the application. Ordering of output across stdout/stderr is a well-known - classic, perhaps - problem.

  • I think you dismiss the problem prematurely. A carefully written program will flush stdout before writing to stderr, making the order of output deterministic; that is, subsequent stages in carefully_written_program 2>&1 | subsequent_stages will see the combined stream in deterministic order. That solves the classic "ordering of output across stdout/stderr" problem, leaving the question the OP asked: can the deterministic ordering be preserved while also filtering each of the two streams in different ways?
    – Don Hatch
    Dec 26, 2019 at 12:28

When you use the construction: 1>stdout.log 2>&1 both stderr and stdout get redirected to the file because the stdout redirection is set up before the stderr redirection.

If you invert order you can get stdout redirected to a file and then copy stderr to stdout so you can pipe it to tee.

$ cat test
echo OUT! >&1
echo ERR! >&2

$ ./test 2>&1 1>stdout.log | tee stderr.log

$ cat stdout.log

$ cat stderr.log
  • That doesn't let you log the two streams to the same file.
    – wnrph
    Mar 19, 2011 at 16:19
  • 2
    @artistoex: You just can use tee -a with the same file used in 1> to sum both outputs in the same file. Something like: ./test 2>&1 1>out+err.log | tee -a out+err.log
    – mmoya
    Mar 19, 2011 at 17:41
  • I have no idea why, but this does not work with wget -O - www.google.de for me. (compare with solution below)
    – wnrph
    Mar 19, 2011 at 17:56
  • 5
    Using tee -a won't work reliably to have the same output that would be on the console if nothing was redirected. The output that goes via tee might be slightly delayed compared with the output that comes directly from the command, and so might appear later in the log. Mar 19, 2011 at 20:33
  • This does not work for me, out+err.log is empty. And yes, I want stderror and standard out in the same file
    – Andreas
    Mar 19, 2011 at 20:41

I have been able to accomplish this by placing the following line at the top of a bash script:

exec 1>>log 2> >(tee -a log >&2)

This will redirect stdout to the file log (1>>log), then tee stderr to the file log (2> >(tee -a log) and direct it back to stderr (>&2). In this way, I get a single file, log, which shows both stdout and stderr in order, and stderr is also displayed on the screen as usual.

The catch is that it only seems to work when I append to the file. If I don't append, it appears that the two redirections clobber each other, and I only get whichever one outputs last.

  • Any way to get this to "modify" the STDERR lines to include a custom string so you can easily grep on it? Oct 18, 2018 at 2:35

To write stderr to screen and write BOTH stderr and stdout to a file -- AND have the lines for stderr and stdout come out in the same sequence they would if both were written to the screen:

Turns out to be a difficult problem, especially the part about having "same sequence" you would expect if you simply wrote them to the screen. In simple terms: Write each to its own file, do some background-process magic to mark each line (in each file) with the exact time the line was produced, and then: "tail --follow" the stderr file to the screen, but to see BOTH "stderr" and "stdout" together -- in sequence -- sort the two files (with exact-time markings on each line) together.


# Set the location of output and the "first name" of the log file(s)
date >>$pth/$ffn.out
date >>$pth/$ffn.err
# Start background processes to handle 2 files, by rewriting each one line-by-line as each line is added, putting a label at front of line
tail -f $pth/$ffn.out | perl -nle 'use Time::HiRes qw(time);print substr(time."0000",0,16)."|1|".$_' >>$pth/$ffn.out.txt &
tail -f $pth/$ffn.err | perl -nle 'use Time::HiRes qw(time);print substr(time."0000",0,16)."|2|".$_' >>$pth/$ffn.err.txt &
sleep 1
# Remember the process id of each of 2 background processes
export idout=`ps -ef | grep "tail -f $pth/$ffn.out" | grep -v 'grep' | perl -pe 's/\s+/\t/g' | cut -f2`
export iderr=`ps -ef | grep "tail -f $pth/$ffn.err" | grep -v 'grep' | perl -pe 's/\s+/\t/g' | cut -f2`
# Run the command, sending stdout to one file, and stderr to a 2nd file
bash mycommand.sh 1>>$pth/$ffn.out 2>>$pth/$ffn.err
# Remember the exit code of the command
# Kill the two background processes
ps -ef | perl -lne 'print if m/^\S+\s+$ENV{"idout"}/'
echo kill $idout
kill $idout
ps -ef | perl -lne 'print if m/^\S+\s+$ENV{"iderr"}/'
echo kill $iderr
kill $iderr
echo "Exit code: $myexit for '$listname', list item# '$ix', bookcode '$bookcode'"

Yes, this seems elaborate, and it results in 4 output files (2 of which you can delete). It does appear this is a hard problem to solve, so it took several mechanisms.

In the end, to see the results from BOTH stdout and stderr in the sequence you would expect, run this:

cat $pth/$ffn.out.txt $pth/$ffn.err.txt | sort

The only reason the sequence is at least very close to what it would have been had both stdout and stderr simply gone to the screen is: Every single line is marked with a timestamp down to the millisecond.

To see stderr on screen as the process goes along, use this:

tail -f $pth/$ffn.out

Hope that helps someone out, who arrived here long after the original question was asked.

  • +1 for the sheer effort, I had a similar idea (timestamps both streams via perl) but was struggling to implement it . I'll investigate if it works for me (ie, if it really keeps the order of outputs, as others solutions here shift things a bit due to asynchronicity between tagging of stderr and stdout) Dec 21, 2016 at 16:50

I've struggled with this exact requirement, and in the end couldn't find a simple solution. What I ended up doing instead was this:

myCommand > $TMPFILE 2>&1
[ $? != 0 ] && cat $TMPFILE
date >> myCommand.log
cat $TMPFILE >> myCommand.log
rm -f $TMPFILE

It appends stdout and stderr, in the proper interleaved order, to the log file. And if any errors occurred (as determined by the exit status of the command), it sends the entire output (stdout and stderr) to stdout, again in proper interleaved order. I found this to be a reasonable compromise for my needs. If you want a single-run log file rather than a growing multi-run one, it's even simpler:

myCommand > myCommand.log 2>&1
[ $? != 0 ] && cat myCommand.log

You want to duplicate the error stream so that it appears both on the console and in the log file. The tool for that is tee, and all you need to do is apply it to the error stream. Unfortunately, there's no standard shell construct to pipe a command's error stream into another command, so a little file descriptor rearrangement is required.

{ { echo out; echo err 1>&2; } 2>&1 >&3 | tee /dev/tty; } >log 3>&1
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^   ^^^^^^^^^^^^    ^^^^^^^^^
  command produces output      stdout→3                    →log
  command produces error       stderr→1   dup to terminal  →log
  • This also, does not quite work for me, I get stdout lines first, and then all stderr lines in 'log' file. example: { { echo out; echo err 1>&2;echo out2;echo err2 1>&2; } 2>&1 >&3 | tee /dev/tty; } >log 3>&1
    – Andreas
    Mar 19, 2011 at 20:56
  • 1
    @Andreas: Oh, duh, tee introduces a delay here too. I don't think there is a pure shell solution. In fact I wonder whether there is a solution without modifying the application in some way. Mar 19, 2011 at 21:28

stderr to command-substituted tee -a, and stdout appending to same file:

./script.sh 2> >(tee -a outputfile) >>outputfile

and note: too ensure correct order (but without stderr-show) there is 'unbuffer' command from 'expect' tools, that simulates tty and keeps stdout/err in order as it would show on terminal:

unbuffer ./script.sh > outputfile

Let f be the command you'd like be executed, then this

( exec 3>/tmp/log; f 2>&1 1>&3 |tee >(cat)>&3 )

should give you what you wish. For example wget -O - www.google.de would look like this:

( exec 3>/tmp/googlelog; wget -O - www.google.de 2>&1 1>&3 |tee >(cat)>&3 )
  • 2
    This almost works for me, but in the log file I first get all the stdout lines, and then all stderror lines. I want them to be interleaved in the natural order as they happen.
    – Andreas
    Mar 19, 2011 at 20:44

Given what I've read here, the only solution I can see would be to prepend each line whether STOUT or STERR with a timeslice/timestamp. date +%s would get to seconds, but thats to slow to maintain order. Also the log would somehow have to be sorted. More work than I'm capable of.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .