I'm reading from a serial port connected to a gps device sending nmea strings.

A simplified invocation to illustrate my point:

  $ awk '{ print $0 }' /dev/ttyPSC9 

If I instead try to read from a pipe, awk buffers the input before sending it to stdout.

$ cat /dev/ttyPSC9 | awk '{ print $0 }'
<long pause>

How can I avoid the buffering?

Edit: Kyle Jones suggested that cat is buffering it's output but that doesn't appear to be happening:

$ strace cat /dev/ttyPSC9 | awk '{ print $0 }'
write(1, "2,"..., 2)                    = 2
read(3, "E"..., 4096)                   = 1
write(1, "E"..., 1)                     = 1
read(3, ",0"..., 4096)                  = 2

When I think about it: I thought that a program used line buffering when writing to a terminal and "regular buffering" for all other cases. Then, why is cat not buffering more? Is the serial port signaling EOF? Then why is cat not terminated?

  • 1
    BashFAQ 009 may be useful.
    – jw013
    Mar 7 '12 at 8:21
  • @jw013: Thanks for the link, a great sum up of how buffering works in bash. Mar 7 '12 at 8:31

It is likely to be buffering in awk, not cat. In the first case, awk believes it is interactive because its input and output are TTYs (even though they're different TTYs - I'm guessing that awk is not checking that). In the second, the input is a pipe so it runs non-interactively.

You will need to explicitly flush in your awk program. This is not portable, though.

For more background and details on how to flush output, read: http://www.gnu.org/software/gawk/manual/html_node/I_002fO-Functions.html

  • 9
    Thanks for the explanation. awk -W interactive '{print $0}' appears to do the trick. The 'W interactive option is available on my awk version (mawk 1.2) but I dunno if it's a standard option. Mar 7 '12 at 8:29
  • 3
    @dannas -W is not in the POSIX standard for awk. I'm not sure what to do if you need maximum portability.
    – jw013
    Mar 7 '12 at 8:46
  • I'm accepting this answer since it explains why awk is doing full buffering in my example, instead of line buffering - it checks the if the input is a tty as well as the output. I only thought that it would check the output. Mar 7 '12 at 8:57
  • @jw013: Thanks for looking up the standard. For me, I just wanted to understand why awk was doing full buffering and I think I do now. Mar 7 '12 at 9:00
  • @dannas I can confirm that -W interactive is at least supported in the Ubuntu 12.04 (and presumably newer) distribution of awk, which is mawk.
    – Jason C
    Jun 11 '14 at 19:38

I know it is an old question, but a one-liner may help those who come here searching:

cat /dev/ttyPSC9 | awk '{ print $0; system("")}'

system("") does the trick, and is POSIX compliant. Non-posix systems: beware.

There exists a more specific function fflush() that does the same, but is not available in older versions of awk.

An important piece of information from the docs regarding the use of system(""):

gawk treats this use of the system() function as a special case and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with gawk, this idiom is not only useful, it is also efficient.

  • This worked for me
    – redolent
    Jan 1 '14 at 3:20
  • 3
    My awk doesn't do anything on neither fflush() nor system(""). My gawk honored it though. Jul 14 '17 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.