Piping data to a script or function, determining whether the output is coming to stdout or stderr?

Question

I'm trying to create a general-purpose function or script that I can pipe data into, and prepend the output showing which fd the data "arrived" on (stdout or stderr). My Bash skills are intermediate at best, and I find myself in over my head trying to wrap my head around statements like { foo 2>&1 >&3 3>&- | bar 3>&-; } 3>&1.

Can anyone suggest if this is possible, and a way to do it?

Essentially I want to do something like this:

$ { echo "foo" ; echo "bar" 1>&2 ; } | my_output_processor.sh
stdout: foo
stderr: bar 

I've read Piping STDERR vs. STDOUT but that didn't answer it for me. I think the answer at https://unix.stackexchange.com/a/440439/307184 might contain a clue, but it seems that would work within the running script only, not with piped in data.

My Bash version is 5.0 on macOS.

edit: @Glenn's answer below is working! I made a couple of small helper functions to save a bit of typing and prevent errors, and tested like this:

$ out(){ sed "s/^/out: /" ;} ; err(){ sed "s/^/err: /" 1>&2 ;}
$ { echo "good" ; echo "bad" >&2; } 2> >(err) 1> >(out)
err: bad
out: good

Followup question about the ordering of the redirection: why does 2> >(err) 1> >(out) work great, while 1> >(out) 2> >(err) does not? (update: answered by @RudiC below - helper functions updated so they work in any order now)

Answer 1

Use Process Substitution not pipes:

{ echo "foo" ; echo "bar" >&2; } 2> >(sed "s/^/err: /") > >(sed "s/^/out: /")

err: bar
out: foo

If you're doing this in a script, use the exec command to set up the redirection for the whole duration of the script:

bash -c '
    exec 2> >(sed "s/^/err: /") > >(sed "s/^/out: /")
    echo foo
    echo bar >&2
'

err: bar
out: foo

---

If you do

exec > >(sed "s/^/out: /") 2> >(sed "s/^/err: /")

then the output is

out: foo
out: err: bar

I assume this is what you mean by "doesn't work". That's because the output of the "err" process substitution by default goes to stdout, and that's now pointing at the "out" process sub. You would have to do this to get around it:

exec 3>&1 > >(sed "s/^/out: /") 2> >(sed "s/^/err: /" >&3)

Create another file descriptor (3) that points to the default stdout, then the stderr redirection prints to fd3.

Answer 2

Follow up question answers itself when executed:

The err function prints to stdout as well. Obviously the first form has the original stdout, while the second already uses the "redirected to out() function" one.

Answer 3

This should work in any standard shell, not just bash:

{ cmd 2>&1 >&3 | sed 's/^/err: /' >&2; } 3>&1 | sed 's/^/out: /'

This could be made into a function, for convenience:

# usage louterr cmd [args ...]
louterr(){ { "$@" 2>&1 >&3 | sed 's/^/err: /' >&2; } 3>&1 | sed 's/^/out: /'; }

cmd(){ echo ERR >&2; echo OUT; }
louterr cmd
out: OUT
err: ERR
louterr louterr louterr cmd
err: err: err: ERR
out: out: out: OUT

or, with configurable "filters":

# usage: louterrx out_filter err_filter cmd [args ...]
louterrx(){
   oflt=$1; eflt=$2; shift; shift
   eval "{ \"\$@\" 2>&1 >&3 | $eflt >&2; } 3>&1 | $oflt"
}

louterrx "sed 's/^/out: /'" "sed 's/^/err: /'" cmd

A disadvantage of this is that the exit status of cmd will be hidden (as with any right side of a pipeline). To work around that, you could use a subshell and set -o pipefail (supported in bash, zsh, ksh, etc but not in dash yet):

louterr()(
   set -o pipefail
   { "$@" 2>&1 >&3 | sed 's/^/err: /' >&2; } 3>&1 | sed 's/^/out: /'
)
cmd(){ echo ERR >&2; echo OUT; return 13; }
louterr cmd; echo $?
out: OUT
err: ERR
13

---

I do not recommend using the > >(...) zsh/bash-ism in this case, since it relies on quirks related to the order in which expansions and redirections are performed in different shells (and is subject to a lot of other quirks and bugs). For example, in

ls no_such_file >/dev/null 2> >(cat)

the output of cat (the error message of ls) will be redirected to /dev/null in bash but not in zsh.

~the only situation where you have to use the cmd > >(...) form is when cmd is a function whose purpose is to modify the current environment and which cannot be run in a subshell.