Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm using the following statement (simplified version):

tail -f -c+1 <filename>

in order to pipe a file to a process.

What I've found, though, is that there is a number of lines at the end which are not piped.

A specific example is piping a mysql file, and stopping when the end is reached:

tail -f -c+1 mysqdump.sql | sed '/^-- Dump completed/ q0'

This doesn't work - the last line of the dump -- Dump completed [...] is not piped to sed.

My guess is that the tail -f buffer, in this case, is flushed only when it's full.

Does anybody know how can I workaround this?

=================

I've found the cause - the description is not complete (and the code doesn't exhibit the behavior).

The problem happens when piping from a compressed (lzma) file:

tail -f -c+1 <filename.lzma> | lzma -d | sed '/^-- Dump completed/ q0'

Most likely, tail is not sending the last compressed block, because it doesn't detect any new line, as the input is binary.

share|improve this question
2  
See e.g. stackoverflow.com/questions/5295430/… for an answer. –  jofel Aug 30 '12 at 13:40
    
I don't understand your means.before tail and after tail what do you do? –  Mohsen Pahlevanzadeh Aug 30 '12 at 18:34
    
and especially read the BashFAQ entry linked to in that stackoverflow question, mywiki.wooledge.org/BashFAQ/009 –  cas Aug 30 '12 at 22:59

1 Answer 1

up vote 2 down vote accepted

tail -f flushes after every input line. You can confirm this with strace (or truss or whatever your unix variant provides to trace processes' system calls).

If there is an incomplete line, tail -f keeps waiting for the next newline. It's a tool designed for text files. If you need to tail a binary file (e.g. if -- Dump completed is not followed by a newline), you'll have to use a custom tool.

If you've redirected sed's output away from the terminal, it will be doing its own buffering. Try stdbuf or unbuffer.

share|improve this answer
    
I was in fact executing on a binary file being remotely copied (I purposely don't want to use "ssh cat"). I worked the problem around using scp on a named pipe. –  Marcus Aug 31 '12 at 8:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.