Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to read a big log file with 10000 lines everytime. and I use this command:

tail -c +offset somefile | head -n 10000

my question is, with the piping of head, will tail read the whole file?

updated:

I know that tail will read from the offset pos.

what I want to know is that:

if the file has 20000 lines left from the offset, and the head will just print first 10000, will the tail read the left 20000 lines? or just read the first 10000 lines?

share|improve this question
add comment

2 Answers 2

up vote 3 down vote accepted

tail will die of a SIGPIPE signal as soon as it tries to write to the pipe when it has no reader.

So tail will die soon after head has finished outputting its 10000 lines and exited.

Because pipes can hold some data (64kiB on Linux) and because tail buffers its output when not to a terminal (8kiB in my test) and head and tail read their input in chunks (of up to 8kiB in my tests), tail may read up to 64 + 8 + 8 kiB after the end of the 10000th line after offset before dying.

The worst case scenario is when head is slower to empty the pipe than tail is to fill it up (for instance if writing to a slow output like a terminal) and if the last byte of the 10000th line is at the start of a 8kiB block.

Then, in the end, head will have read the block with that last newline at the beginning (8191 more bytes than necessary), but is busy writing its output. tail, during that time has filled up the pipe (64kiB), has read another 8kiB block, but as the pipe is full is blocked writing to it. As soon as head writes its last line, and exits, tail will die but will have read 64kiB + 8 kiB + 8191 bytes past the last of those 10000 lines.

The best case scenario is when the last of the 10000 lines is at the very end of a 8kiB block and head reads data as soon as it's put in the pipe by tail. Then, if on the last block, head manages to read it from the pipe and exit before tail writes the next one, then tail will die upon writing the next block, so it will read only 8192 bytes past the end of that 10000th line.

That assumes somefile is a regular file. It could terminate sooner if somefile was some king of special file that trickles one byte at a time for instance (like a pipe from a while :; do echo; done)

share|improve this answer
    
thanks very much –  Zava Feb 28 at 9:52
add comment

No. And piping out put will have no affect what so ever on what tail reads.

With your character offset option -c +offset tail will seek to position, then read from that point.

share|improve this answer
    
if the file has 20000 lines left after the offset, will tail read the whole 20000 lines, or just the first 10000 lines? –  Zava Feb 28 at 3:58
    
No, Tail will fail when head exits after 10000 lines –  X Tian Feb 28 at 11:27
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.