How to correctly unbuffer a pipeline?

Question

I'm trying to follow various answers in this, this and this threads, but my pipeline still appears buffered. Here's an example attempt:

unbuffer sh -c 'echo -n test; sleep 1; echo; sleep 1; echo -n hello; sleep 1; echo' | grep .

The above command should immediately print test, then after on second print hello, and after one more second end. Instead, I get a one-second delay, only then test, then, after a two-second delay, hello with command exit.

Note that here I'm trying to completely get rid of buffering, and --line-buffered option of grep won't help. So, how do I truly unbuffer this pipeline?

Answer 1

grep will output the lines that match a pattern, so it can't output anything until a full line (or end-of-file upon which, if the last line was not terminated, the behaviour varies depending on the implementation) has been read.

So in:

(echo -n foo; sleep 1; echo bar) | grep .

It's not because echo buffers its output. It doesn't. How could it, once it has terminated? It's because grep is waiting for the end of the line (that will come, immediately followed by end-of-file with the second echo).

To do what you seem to want, you'd need to read lines with a timeout for each line like (with bash):

(echo -n foo; sleep 1; echo bar) |
  while true; do
    TMOUT=0.1 IFS= read -r line; ret=$?
    if [ "$(kill -l "$ret")" = ALRM ]; then
      [ -n "$line" ] || continue
      ors=
    elif [ "$ret" -eq 0 ]; then
      ors=$'\n'
    elif [ -z "$line" ]; then
      break
    fi
    [[ $line =~ . ]] && printf %s "$line$ors"
  done

(you get a pass in the loop at least every 0.1 second).