I have some code that is supposed to run more or less indefinitely, however in the case that it crashes I would like to see its output. Is there any command or combination of commands I can use in Linux to store only the last X lines of my command's standard out so that if my code has been running for a few days I don't end up with a 100,000 lines text file but only the last say, 100 lines that have been output?
More so, is it possible to have this text file always have only the last 100 or so lines in it so that if I were to view it at anytime while my code is running I could see what my code has been doing?

  • 1
    Ummm...tail -n 100 file.log? – heemayl Jun 7 '16 at 14:32
  • @heemayl this will only show the last 100 lines, not the continuous stream. – roottraveller Aug 15 at 5:56

You are looking for a "ring buffer". Serverfault Sister site link to accepted answer for linux:

Turning a log file into a sort of circular buffer

  • Thanks, this is exactly what I needed, although I couldn't get it set up on my system it led me to ulogbufd (panix.com/~jdw/ulogbufd) which works great, and is to my understanding quite similar in what it accomplishes. – roboctopus Jun 7 '16 at 17:17

The easiest way to do this would be to have the program log to a single file, then just use the tail command on that file anytime you wanted to see what had been going on recently.

  • You didn't read my question all the way through, I specifically didn't want to end up with a couple hundred thousand lined text file. – roboctopus Jun 7 '16 at 17:19

100000 lines of say 100 characters each is still only about 10 MB, so it's not even impossible to save all of it and search for what you want, or to just read the tail.

However, if you don't want to save it all, you have some options. For log files, it's usual to rotate the logs, i.e. after a file is large enough, it's renamed and the program notified to reopen its output, which then goes to a new file with the same name. This might require modifying the program if it doesn't know how to do this natively. (Just renaming the file will not help, since the open file handle will still point to the renamed file.)

Workaround for the above problem: pipe the output to logger, sending it to syslog, and then do log rotation on the output, since syslog will definitely support it. Negatives: a bit messy in that it will pollute unrelated log files, unless you take care to configure that particular log data to a separate file. Also adds a bit of overhead.

Three: run the program in screen and set the size of the scrollback window to whatever you like. (screen -h <N> when starting it, or use :scrollback <N> inside screen to change the size. Scrolling the scrollback in screen might be a bit unintuitive to begin with, but you can save the whole buffer with :hardcopy -h <filename>.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.