Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.
Sign up to join this community
Why does the following command not produce any output?
$ tail -f /etc/passwd | tail
After reading about buffering, I tried the following to no avail:
$ tail -f /etc/passwd | stdbuf -oL tail
Note that the following does produce output:
$ tail /etc/passwd | tail
So does this:
$ tail -f /etc/passwd | head
I am using tail version 8.21 (GNU coreutils).
tail -f's tail is actually something unknown in the present, so how should the next tail know it. On the other hand tail -f's head is something already known and could therefor be processed.
Or to put it simpler: tail is relative to the end of the file, but the output stream of tail -f got no EOF (at least not before its termination).
If you find the first tail's pid and kill it, you should then see the output from the second.
When running with a stream as input, tail keeps an n-line buffer that it fills as it reads the stream, but it cannot output those lines until it reaches the end of the stream, i.e. it receives a special EOF code when trying to read from the input stream. The invocation tail -f does not exit, thus it will never close its stream, which makes it impossible to e.g. return the 10 last lines of that stream.
tail shows the last X lines. tail -f does the same, but essentially in an infinite loop: on start-up, show the last X lines of the file, then using some OS magic (like inotify), monitor and show new lines.
To do its job, tail must be able to locate the end of the file. If tail cannot find the file's end, it cannot show the last X lines, because "last" is undefined. So what does tail do in this case? It waits until it does find the end of the file.
Consider this:
$ chatter() { while :; do date; sleep 1; done; }
$ chatter | tail -f
This never appears to make progress, because there is never a definite end of file from chatter.
You get the same behavior if you ask tail to give you the last lines from a file system pipe. Consider:
$ mkfifo test.pipe
$ tail test.pipe
stdbuf to get around the perceived problem was a noble attempt. The key fact though is that I/O buffering isn't the root cause: the lack of a definite end-of-file is. If you check out the tail.c source code, you'll see the file_lines function comment reads:
END_POS is the file offset of EOF (one larger than offset of last byte).
and that's the magic. You need an end-of-file for tail to work in any configuration. head doesn't have that restriction, it just needs a start of file (which it might not have, try head test.pipe). Stream oriented tools like sed and awk need neither a start or end of file: they work on buffers.
The function of tail is to show the last part - "tail" - of the input or file. (The option -f is about what it does later, so that is not relevant here.)
Let's think about a file:
What is the last part of a file?
Let's say it's the last n lines of a file.
When we read line i of the input file, how to decide it needs to be printed or
not?
We do not know whether it is in the last part - because we do not know which the last line will be. So we can not print it now.
We need to keep the line until it becomes clear that is's part of the last n lines, or can no longer be part of it, because we know n further lines
If we now come to the end of the file, we know that the last n lines we kept are in fact the last n lines of the file.
Now, in the case of
tail -f /etc/passwd | tail
the first tail reads the file, and then waits to get more data from it, to write that out too. So it will not signal an end of file to the second tail when it comes to the end of the file it reads. Without that, the second tail never gets notified of the end of file, so it can never find out which are the last lines it should print.
