1

I'm trying to distribute very long make output using following method

mkfifo myfifo 
make 2>&1 | tee myfifo | grep -E "errors|warnings" myfifo > errors.log | cat myfifo

Idea is that subset of output should be copied to log files, while whole output would be redirected to stdout of console.

It seem to work at first, but suddenly stops and stays frozen until i press Ctrl-Z.

A use of process substitution

make 2>&1 | tee >(grep -E "errors|warnings" > errors.log)

is not possible, because shell compiled with default strict POSIX compliance.

platform - ubuntu-flavored distro. shell bash 4.2 (and only one available)

  • 4
    tee is outputting to a pipe (in addition to that fifo), but grep is never reading from it, so after awhile the pipe fills up and the writes from tee will block. You also have a cat that's not reading from a pipe, so I'm not sure what you're trying to do. But I think what you want to do can be done, in a modern shell, with just make 2>&1 | tee >(grep -E "errors|warnings" > errors.log). – Mark Plotnick Apr 24 '17 at 15:02
  • @Mark Plotnick Thank you. I followed only source i had based literrally on analog of system V, Linux is a "obscure" OS here due to literature and language barrier. so proving queue as file doesnt work as reading it? there is proper content in errors.log and i suspected that something like is happening. i need output to TWO different logs AND to console though – Swift Apr 24 '17 at 15:08
  • @Mark Plotnick syntax error near unexpected token (. That doesn't work in bash 4.2 – Swift Apr 24 '17 at 15:18
  • @Mark Plotnik no luck, quick check on that error suggested that it requires non-posix feature, bash runs as if with --posix flag – Swift Apr 24 '17 at 16:05
  • That explains it - process substitution won't work with strictly-conforming POSIX shells. – Mark Plotnick Apr 24 '17 at 16:06
1

That

make 2>&1 | tee myfifo | grep -E "errors|warnings" myfifo > errors.log | cat myfifo
          P1           P2                                              P3

command makes little sense. It starts those 4 commands concurrently with pipes in between:

  • make 2>&1: stdout and stderr goes to P1
  • tee myfifo: stdin from P1, stdout to P2. So tee writes the output to both myfifo and P2.
  • grep -E "errors|warnings" myfifo > errors.log: stdin from P2 but you're passing a filename (myfifo) to grep, so grep won't read from its stdin. stdout goes to errors.log, nothing writes to P3
  • cat myfifo: stdin from P3 but again as cat is given a filename, it reads it instead. So both grep and cat read from myfifo concurrently.

Since nothing reads from P2, tee will hang when P2 is full. Also cat will not display the portions of myfifo that have been read by grep.

make 2>&1 | tee myfifo | grep -e errors -e warnings > errors.log | cat myfifo

is closer to what you meant I think. Again P3 is not used, but we're using the | to start cat concurrently and wait for it.

Or you could do:

make 2>&1 | tee myfifo & grep -e errors -e warnings > errors.log
wait

If you need more greps, you need more fifos:

make 2>&1 | tee fifo2grep1 fifo2grep2 &
grep errors fifo2grep1 > errors.log &
grep warnings fifo2grep2 > warnings.log
wait

On systems that support /dev/fd/x, you can also do:

make 2>&1 | { tee /dev/fd/3 | grep -e errors -e warnings 3>&-; } 3>&1

(that's typically what process substitution does in shells that support it).

With 2 greps, that becomes:

make 2>&1 |
 {
   {
     tee /dev/fd/3 /dev/fd/4 |
       grep errors > errors.log 3>&- 4>&-
   } 4>&1 |
     grep warnings > warnings.log 3>&-
 } 3>&1
  • Marvelous explanation. so if i need two different greps, i reiterate, using additional fd/n ? – Swift Apr 25 '17 at 13:50
  • 1
    @Swift, see edit – Stéphane Chazelas Apr 25 '17 at 14:13
  • is >&2 a typo? i didnt spotted it at first, because i tested in console with edited string , that retained old start. found descrepancylater, >&2 variant doesnt work – Swift Apr 26 '17 at 10:31
  • 1
    @Swift, yes sorry. Typo fixed now. Well spotted. – Stéphane Chazelas Apr 26 '17 at 10:34
0

After grep -E "errors|warnings" myfifo > errors.log the pipe no longer contains data. So the next command cat myfifo which read from the pipe will be blocked.

If I understand your question correctly, you want to print all messages to stdout, and redirect all error and warning messages to errors.log. So if you want to use pipe, use two:

 mkfifo pipe1 pipe2 
 make 2>&1 | tee pipe1 pipe2 | grep -E "errors|warnings" pipe1 > errors.log | cat pipe2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.