Append string on redirect

Question

When redirecting all output to stderr:

stat file >&2

... if the command throws an error (e.g. file does not exist) how can one prepend some arbitrary text, such as:

<<MY ARBITRARY TEXT>>: stat: cannot stat 'file': No such file or directory

The required logic is:

let res = stat file
if error then output "<<MY ARBITRARY TEXT>>"+error to stderr
else output res to stderr

More context: all command output is being redirected to stderr because stdout and stderr streams do always return in the right order, i.e. if multiple commands are sent and one throw an error, the stdout of the next command may return before the stderr of the original command. To avoid this, I am merging all commands onto stderr, but ideally would like to be able to easily identify errors.

Answer 1

TL;DR: Parsing stderr on the fly can be complicated, so simplify it by storing stderr into variable and operate variable contents later.

What you may want to consider doing is capturing stderr to variable, and afterwards checking if it's not empty. Consider the example below:

$ foo=$(stat /etc/passwd 2>&1  > /dev/tty )
  File: /etc/passwd
  Size: 1710        Blocks: 8          IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 537594      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2019-03-08 15:28:10.481688274 -0700
Modify: 2019-03-08 15:28:10.333669255 -0700
Change: 2019-03-08 15:28:10.341670283 -0700
 Birth: -
$ test "x$foo" = "x" && echo "empty"
empty

Command substitution $(...) runs a subshell and captures stdout only, hence we want to duplicate the file descriptor 1 onto 2 2>&1 so that stderr now can be captured by command substitution, however we still want to see normal output on screen, hence afterwards we point stdout at the controlling terminal /dev/tty.

Now, what if we actually capture stderr into the variable ?

$ foo=$(stat nonexistent.txt 2>&1  > /dev/tty )
$ test "x$foo" = "x" && echo "empty" || echo "not empty"
not empty
$ printf "<Arbitrary Text>%s\n" "$foo"
<Arbitrary Text>stat: cannot stat 'nonexistent.txt': No such file or directory

As you can see the because stderr is captured we can wrap arbitrary text around it as we wish. The main reason for using a variable is that standard shell methods such as pipeline and command substitution operate on stdout file descriptor 1, so there's not a lot we can do to parse stderr on the fly. Of course, we could use a named pipe and redirect to it as stat foobar.txt 2> /tmp/named_pipe.fifo, but the problem with that is blocking - information sent to pipes is buffered until consumed by another process, so your shell script will be stuck. Of course, this could be handled via starting a background process, but IMHO it's more complexity than necessary, and doing multiple processes is far better done in a programming language like Python where you can actually directly access multiprocessing calls.

In case you want both stdout and stderr in separate variables for processing, there is a good example on Stack Overflow already of that can be achieved via named pipes, along with a few shell-specific approaches/tricks.

Side note: test and [ are the same command, so you could just as well write [ "x$foo" = "x" ] and use that in a more elaborate if-else statement

Answer 2

You can achieve what you want to do quite simply.

1) If you need to recover your output in file foobar:

 $ (echo "Arbitrary text" && stat /etc/passwd ) 2>&1 | tee foobar

2) More simply, if all you care for is output to stdout:

 $ (echo "Arbitrary text" && stat /etc/passwd ) 2>&1