Run command once and filter output in multiple files [duplicate]

Question

Is there anyway to redirect filtered output to multiple files by running mycommand.sh only once?

Example output should be similar to this:

mycommand.sh | grep --line-buffered -B 1 A >> file1 ; my command.sh | grep --line-buffered -A 1 B >> file2

Or maybe with egrep somehow...

Answer 1

With awk command:

mycommand.sh | awk '/A/{ print > "file1" }/B/{ print > "file2" }'

• /A/{ print > "file1" } - if a record matches the pattern A print/redirect the whole record into file1

Answer 2

awk is a better tool for this because it's much faster, doesn't need to fork grep or whatever multiple times, and easily scales up to writing to any number files. Ditto for most other scripting languages, like perl or python.

In shell, you can use tee and process substitution. e.g.

mycommand.sh |
  tee >(grep --line-buffered -A 1 B >> file2) |
  grep --line-buffered -B 1 A >> file1

(extra line feeds and indentation added to improve readability. works all on one line if you prefer it ugly and unreadable)

tee writes its input to both stdout (e.g. to the terminal, redirected to a file, or piped to the next command in the pipleline) AND to a file specified on the command line.

In this case, that "file" is a file descriptor provided by process substitution (e.g. another grep command with redirected output, as in the example above).

tee can write to multiple output files (or process substitution file-descriptors) simultaneously. For example:

mycommand.sh |
  tee >(grep --line-buffered -A 1 B >> file2) \
    >(grep --line-buffered -A 1 C >> file3) \
    >(grep --line-buffered -A 1 D >> file4) |
  grep --line-buffered -B 1 A >> file1

---

This works, but I'd personally use something like @RomanPerekhrest's awk answer.