The following chain of commands are to

  1. ping with datestamp (UNIX),

  2. convert the UNIX datestamp to more human-readable format, and

  3. output to the terminal and a log file.

ping -D localhost 2>&1 | sed 's/^\[\([0-9]*\.[0-9]*\)\]\(.*$\)/echo "[`date -d @\1 +"%Y-%m-%d %H:%M:%S"`] \2"/e' | tee -a ping.log

Problem is that as written, the output seems to be buffered in chunks of almost a minute or ~50 lines, unlike the usual second-by-second and line-by-line output from ping.

What is causing the buffering and how can it be avoided?


If it's available in your system, use unbuffer. Should be as easy as:

unbuffer ping -D localhost 2>&1 | unbuffer sed 's/^\[\([0-9]*\.[0-9]*\)\]\(.*$\)/echo "[`date -d @\1 +"%Y-%m-%d %H:%M:%S"`] \2"/e' | tee -a ping.log
  • sed -u looks like a simpler and cleaner solution for now, but do you know when it would be preferable to use unbuffer? And which commands would need it? – adatum Sep 11 '18 at 1:10
  • unbuffer is generic and works with any command. I've used it extensively with various programs including awk, perl and others. It's needed whenever piping is buffered automatically. – msb Sep 11 '18 at 1:13
  • Good to know. When is piping buffered automatically? It confused me that the piped commands, depending on how many and which commands, sometimes get buffered and sometimes don't. I initially thought it was a problem with the code. – adatum Sep 11 '18 at 4:22
  • 1
    That's a good question, idk myself. I've read around about stdbuf which is another program analogous to unbuffer that controls stdin/stdout/stderr streams. I guess it's something that either the shell or the kernel controls, but I'd have to do some research on it to know who controls. All I know is that it's external to the program. I had my own scripts with autoflush and everything, and they'd still get buffered, which was really puzzling and how I found unbuffer. – msb Sep 11 '18 at 4:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.