Pipe and filter bash outputs in realtime

Question

I am running some commands in bash (basically some ffmpeg commands), that I'm using grep and awk to filter out the results. The command takes some time and continously outputs some results as it progresses through the video. The grep pipe is the same. But the awk pipe waits until the command is completed and prints all at once, which is not good. I want it to output as soon as it found a match.

How to change this command to output results in real-time as it progresses?

ffmpeg -i freeze.mp4 -vf "freezedetect=n=-60dB:d=2" -map 0:v:0 -f null - 2>&1 | grep freezedetect | awk '{print $4,$5}' | tr -d , | grep lavfi

Answer 1

With standard tools, try

ffmpeg -i freeze.mp4 -vf "freezedetect=n=-60dB:d=2" -map 0:v:0 -f null - 2>&1 |  stdbuf -o 0 grep freezedetect | stdbuf -o 0 awk '{print $4,$5}' | stdbuf -o 0 tr -d , | stdbuf -o 0 grep lavfi

Answer 2

expect provides a tool, unbuffer, which you can use to solve this. It will connect to your long-running ffmpeg process with a pseudoterminal, which makes the system act as though it's interactive rather than using the 4K buffer that pipelines usually incorporate:

$ unbuffer ffmpeg [...] | grep [...] | awk [...]

Further, you can combine the grep and awk into one operation:

Rather than:

[...] | grep freezedetect | awk '{print $4,$5}' | tr -d , | grep lavfi

Instead, you can:

[...] | awk '/freezedetect/ && ( $4 /lavfi/ || $5 /lavfi/ ) {print $4,$5}' | tr -d ,