I'm writing continuously to a file, and I want to tail it from another shell. But it seems that as long as the file is open for writing,

tail -f filename

can't show all the new lines until the file is closed again.

Unfortunately, opening and closing the file many times a second slows my script down to the point where data is received faster than it can be written, so I can't close the file during the process.

The writing is being done by a python program:

ser = serial.Serial('/dev/ttyACM2',9600)
f   = open("filename", "a+")
while True:
    s    = ser.readline()
    f.write(s + "\r\n")

Isn't there a way to view the contents of a file that is in the process of being written to?

  • Do you mean tail -f <file>? Or better: tail -F <file>, the uppercase F will follow even if the file does not exist yet or is logrotated etc. – eblock Feb 22 '19 at 16:27
  • 1
    This is either a matter of not using tail -f on the file, or of the program writing to the file doing buffering (either explicitly or, more likely, implicitly). How are you writing to the file? Buffering of output is done for performance reasons, but the data would not be written until the buffer is full. Sometimes buffering can be avoided. – Kusalananda Feb 22 '19 at 16:28
  • 1
    Seconding the thought that this is most likely a buffering issue. – DopeGhoti Feb 22 '19 at 16:29

If you can't see it with tail, it's not in the file.

It might be in the write buffer of the Python program, though. The customary behavior when writing to files is that the C library buffers outputs until a full block is written (e.g. 4096 bytes or such) instead of sending all the writes to the OS immediately. This saves on system call overhead. This is similar in Python. (I'm not sure if it uses the C library, or if it does similar buffering itself, but the end result is the same.)

For output to the terminal, the default is line buffering: the buffer is written out when a newline is written. Of course, any buffers are also flushed if the file is closed, but that's indeed not very efficient.

To fix that, either call f.flush() after each f.write(), use disable the buffering altogether. There are some ways to do that in this question on SO: Disable output buffering

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.