It appears the problem here is
awk. Your program elides output - trims it. This can be a very useful thing to do, but when you do this you reduce the output queued for
awk. Most UNIX programs which use the stdio.h (which is most UNIX programs) will buffer their output depending on the type of their output device. When the output device is your terminal, most UNIX programs line-buffer their output - and do a
write() per line. When it is not though, (for example when it is a pipe) said programs will buffer to some defined number of bytes per write.
This is standard behavior and is generally a good thing. Every call to
write() is a system call - it's a hook into the system kernel's core and most basic functionality. UNIX systems are time-shared - the kernel shares out the CPU's processing time to each requesting process as necessary. And so it pays to be efficient and block your i/o into manageable chunks. That's why applications buffer.
awk is doing. And this problem is conflated in that its filtering of its input reduces its own output to some 8 or 10 bytes every second or so. If
awk buffers output into 4kb chunks, (as is common), and it only generates approximately 10 bytes of output per second, it will wind up writing only once every 6 minutes. It's not going to be a very useful application for measuring latency unless you're checking some frequency on the scale of continental drift.
There are ways to handle this. You can, for example, wrap your pipeline in a pty -
screen can do this for you and several other applications besides. Possibly there are
awk-native options for handling this - but I wouldn't know (even if terdon definitely does). There is GNU's
stdbuf tool which injects a libc call for setting the output buffer at program exec time. This is usually quite effective, but fails to be very useful if the program it execs later adjusts its buffer in the course of its execution.
Now I blame
ping might be buffering output as well. However - at least on my linux system - when I call it like:
ping -O -n 18.104.22.168 | ...
...this is not the case. In case you're curious, I managed to make it write every output line consistently as soon as one was available using only very basic POSIX-options to standard UNIX commands.
ping -On 22.214.171.124 |
sed -u 's/^64.*=\(.*\) ttl.*=/\1 /' |
...will work with either a GNU, BSD, or AST
sed. Also, there is...
ping -On 126.96.36.199 |
sed 's/^64.*=\(.*\) ttl.*=/\1 /w target_file' |
...which will copy all input lines following any possible processing to stdout - and this will most likely be block-buffered. However, it will also immediately
write the results of every successful
s///ubstitution to target_file. Immediate
writes for any/all specified
write files is POSIX-spec'd
sed behavior - and the same is (almost) true for
read files as well. As a slight variation on that theme:
ping -On 188.8.131.52 |
sed 's/^64.*=\(.*\) ttl.*=/\1 /w /dev/fd/1' |
On systems which support addressing file descriptors with
/dev links, the above should result in line-buffered output through the standard shell pipeline - at least as far as
sed is concerned. For those systems which do not support the
/dev/fd links directly, it is yet a popular
sed feature to handle
/dev/std(in|out|err) anyway - even the old
minised does this. Use
-n to disable the default stdout output altogether.
Here is a more flexible - and more complicated - solution based on that construct:
mkfifo /tmp/ipipe /tmp/opipe
exec 9<>/tmp/opipe 8<>/tmp/ipipe
trap ' printf \\nTTY:STOP\\n >&8' INT
sed -n 's/^64.*=\(.*\) ttl.*=/\1\t/
}' <&8 >/dev/null 2>&1 & SEDPID=$!
ping -On 184.108.40.206 >&8 2>&8 & PNGPID=$!
printf \\nPIP:START\\n >&8;rm /tmp/?pipe
Now that sets up a backgrounded dispatcher process that will always write its output line-buffered. It will write either to the file-descriptor it saved when it is was first called and which corresponds to the parent shell's
<>&9 descriptor, or it will write to the tty depending on whether you send it the commands:
You can send these like:
printf \\nPIP:START\\n >&8
printf \\nTTY:START\\n >&8
It will handle both simultaneously as you like. The parent shell has a
trap configured for sending it the command TTY:STOP when it receives an INT signal - and so just pressing CTRL+C on your keyboard should be enough to make it stop writing to your terminal at any given time. You will need to explicitly tell it to stop writing to a pipe though, which might also be handled in a
trap if you liked. You can tell it to stop like:
printf \\nPIP:STOP\\n >&8
sed will always
write on a line-buffer - regardless of how it might buffer its standard out. And so with this you can read each line written by
ping as soon as it writes it. You can do it by calling up another process to read on file-descriptor 9 - which is the PIP
sed writes to when instructed to do so - or you can just have it print straight to the terminal. To call up another process to read from PIP just do:
...or similar. Also note that I explicitly
mkfifo created file-system links for the pipes used as soon as all processes that might need one have definitely established their file handles on those pipes. This means that you will not be able to interact with the processes involved via the filesystem - all IPC must be coordinated through the parent shell's descriptors 8 and 9. The PIDs for the backgrounded
ping processes are saved in the shell variables
$SEDPID. Either can be addressed by
Now having run the script example above, then doing sending the TTY:START command the results were:
[mikeserv@localhost ~]$ printf \\nTTY:START\\n >&8
[mikeserv@localhost ~]$ 218 24.2 ms
219 21.2 ms
220 23.1 ms
221 21.3 ms
222 21.9 ms
...but both of the processes are still running -
sed just isn't writing output anywhere.
ps -Fp "$SEDPID" "$PNGPID"
UID PID PPID C SZ RSS PSR STIME TTY STAT TIME CMD
mikeserv 31601 28945 0 2143 1712 4 14:22 pts/0 SN 0:00 sed -n s/^64.*q=\(.*\) ttl.*=/\1\
mikeserv 31602 28945 0 2106 740 3 14:22 pts/0 SN 0:00 ping -On 220.127.116.11