I'm trying to pipe a command through the output of two other commands and then merge the results of the two process substitutions. An example that gets me close is:

command | tee >(sed -rn 's/.*foo (bar).*/1/p') >(awk '{print $3}')

However, I would like to achieve the following:

  • I don't need to see the input stream of the original command
  • I would like to use 'paste' to merge the results

I suppose one option is to run two separate commands and put them into files, but that isn't as elegant as I would like. What is the most elegant (single liner, clearly understood) way to do this in bash?

up vote 8 down vote accepted

The reason you are seeing the output of the original command is because tee outputs to stdout as well as the files specified. To discard this you can put >/dev/null at the end of the command or redirect this output to one of your process substitutions by adding an extra >, eg:

command | tee >(sed -rn 's/.*foo (bar).*/1/p') > >(awk '{print $3}')

Or simpler just use another pipe:

command | tee >(sed -rn 's/.*foo (bar).*/1/p') | awk '{print $3}'

As for combining the result of the two process substitutions using paste, unless there is some obscure shell trick that I don't know about, there is no way to do this without using a named pipe. Ultimately this is two lines (formatted to more for clarity):

mkfifo /tmp/myfifo
command |
  tee >(sed -rn 's/.*foo (bar).*/1/p' >/tmp/myfifo) |
  awk '{print $3}' |
  paste /tmp/myfifo -

If you are putting this in a script, also consider using the recommendations for creating a temporary named pipe here.

  • Thanks, I knew about the tee redirect option. I did indeed achieve what I wanted with fifos, I was just hoping there was something more elegant. – Arima Jun 12 '14 at 1:09
  • @Arima, it definitely seems like it should (there is certainly a lot you can do with process substitution). There is a connect command proposed for moreutils looks like it would come close, but it just looks like it is a proposal just now without an implementation. – Graeme Jun 12 '14 at 7:52

You can do all of that just in sed.

command | 
sed '/\([^ ][^ ]*  *\)\{2\}/{h
         s///;s/^  *//;s/ .*/p
     g};s/.*foo \(bar\).*/\1/p;d'  

But as to the other thing, you have tee around the |pipe files:

cmd1 | {     
    {   tee /dev/fd/3 | 
        cmd2 >&2
    } 3>&1 | 
    cmd3
} 2>&1 |paste

But since you're already using sed you can use it like a smart tee and only dup/redirect when you have to:

cmd | { 
    sed -n 'p;s/.*foo \(bar\).*/\1/w /dev/fd/3' |
    awk ...
} 2>&1

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.