Is it possible to read from stderr in a bash script?

Question

I have a bash script like this:

#!/bin/bash
while read -r -a line
do
   ... parse $line in some way
done

This script is executed by piping the command from another program:

some-random-program | myscript.sh

This works as long as some-random-program sends its output to stdout.

However, if the script sends output to stderr, then my script sees no output. I know I can do this:

some-random-program 2>&1 | myscript.sh

My question is this: is there a way to get myscript.sh to read from stderr so that the "2>&1" (or any variation of it) is not necessary? I thought I could do this:

while read -r -a line
do
   ... parse $line in some way
done < /dev/stderr

But that didn't work. The script never sees any input.

Answer 1

No, this is not possible.

Your attempt doesn't work for a fundamental reason: stderr is an attribute of each process, not some system-wide thing. Standard error (stderr) just means file descriptor 2 of the process. read … </dev/stderr is the same thing as read <&2: it reads from whatever file is open on file descriptor 2 of the shell process. If you run this in a terminal, it reads from the terminal.

I strongly recommend reading what is meant by connecting STDOUT and STDIN?.

What you're trying to do isn't actually “read from stderr”, it's read what some-random-program writes to stderr, i.e. read what the program writes to file descriptor 2. You can only do that if the program is writing to a pipe and you're reading to the other end. Or you could set up some different kind of file: a socket would also work (but is harder to set up), or you could have the program write to a regular file and read from that file (but with no direct way to know when to stop reading: if you reach the end of the file, the program might still write more), or you could have the program write to a terminal for which you're a terminal emulator, or you could have the program write to a character device and be a kernel driver for that device. All of these methods have in common that the program is writing to something, and you read from the “other end” of that thing.

All of these methods have in common that _the caller of some-random-program needs to set something up (or some-random-program itself, but here it isn't collaborating). First someone has to set up the communication channel (and if not the caller, nobody will), then the caller of some-random-program has to connect the write end of the channel to file descriptor 2 of some-random-program, and the caller needs to hand you the read end of the channel. This is what the pipe (|) operator does in the shell, except it connects the write end to file descriptor 1 (stdout), not file descriptor 2 (stderr).

In the construct some-random-program | myscript.sh, the two sides some-random-program and myscript.sh run in parallel. You have no control over the order in which the two sides do thing. Depending on how fast the programs are, some-random-program might write to stderr before myscript.sh runs the first instruction in the script. So even if there was a way to cause some-random-program's stderr to change, it would be too late.

(Technically, there is a way myscript.sh could spy on some-random-program: have it work like a debugger and either snoop on what the program writes, or have it inject a system call into the program's process to open something different on file descriptor 2. Injecting a system call may or may not confuse the program, depending on how much it cares. On many systems, you'd have to disable security restrictions to allow a process to debug another process that isn't its child, e.g. sysctl kernel.yama.ptrace_scope on Linux.)

So, if you want your script to see both stdout and stderr from some-random-program, you need to set up the file descriptors of some-random-program accordingly:

some-random-program 2>&1 | myscript.sh

Bash, ksh and zsh let you abbreviate this to some-random-program |& myscript.sh.

If you want to handle the program's stdout and stderr separately, see How to grep standard error stream (stderr)?. Note that you lose synchronization.

Answer 2

The documentation, man bash has this to say about the read command

read [-ers] [-a aname] [-d delim] [-i text] [-n nchars] [-N nchars] [-p prompt] [-t timeout] [-u fd] [name ...]

One line is read from the standard input, or from the file descriptor fd supplied as an argument to the -u option [...]

Semantically it doesn't make sense to read from stderr, as that's an output error stream. However, there's nothing stopping you reading from file descriptor 2, which happens to be where error output is usually sent.

Now, an example implementation. The | symbol only connects stdout to stdin so you will need to "roll your own" pipe for stderr to fd 2. Be very careful of blocking on the FIFO. Also note that because you have repurposed fd 2 for reading the previous command's stderr, there's now nowhere for real error messages to go. You would probably be better off reading from fd 3 (say) and leaving fd 2 as stderr.

#!/bin/sh
mkfifo 'f'                        # Create the FIFO (pipe)
(
    exec 2>'f'                    # Connect stderr to the FIFO
    echo 'stdout'                 # Write to stdout
    echo 'stderr' >&2             # Write to stderr
) |
(
    read one                      # Read from stdin
    read two <&2                  # Read from fd 2
    echo "one=$one, two=$two."
    echo 'hyperspace?' >&2        # This disappears
) 2<'f'                           # Redirect fd 2 from the FIFO
rm -f 'f'

In bash you could rewrite read two <&2 as read -u 2 two, but it's really just semantic sugar. Note that the timeout specified with read -t 1 … will have no effect when reading from a blocking FIFO. There is no easy way to determine whether there is data in the pipe ready for reading.