I am trying to get the last line of a file and then delete it:

Read the line:

sed -n '$p' file
# or
tail -n 1 file

Remove the line:

sed -i '$d' file

This works, but is there a way to do this with just a single command ?

  • I'm not focused on sed. Any default tool will be fine. – pLumo Oct 11 '18 at 13:46
  • In what way does sed not satisfy your requirements? To be able to suggest another way of doing it, we would need to know in what particular way sed is not good enough. Or, do you want to perform both operations in one command? – Kusalananda Oct 11 '18 at 13:47
  • sed is fine, but I'd like to open the file just once. If there is no way to do this, I'll be fine. – pLumo Oct 11 '18 at 13:49

Whatever the prevailing wisdom is, having to read a whole file each time in order to cut off its last line is dumb, no matter if it's done in a single process/command or two or three.

Unlike sed -i, tail is smart enough not to read a file from the beginning in order to determine what its last line is; also, linux has a truncate(1) utility, which combined with tail allows you to "pop" the last line(s) of a huge file in O(1) time:

# usage popline file [line_count, 1 by default]
popline(){ LC_CTYPE=C l=`tail -"${2:-1}" "$1"; echo t`; l=${l%t}; truncate -s "-${#l}" "$1"; printf %s "$l"; }

$ wc -l /tmp/foo
687800 /tmp/foo
$ cp /tmp/foo /tmp/foo1 && time sed -i '$d' /tmp/foo1

real    0m1.185s
user    0m0.196s
sys     0m0.984s
$ cp /tmp/foo /tmp/foo1 && time popline /tmp/foo1

real    0m0.082s
user    0m0.000s
sys     0m0.000s
  • +1 very cool. time popline file -> 0m0.008s <-----> time sed -i -e '${w /dev/stdout' -e 'd;}' file --> 0m0.160s – pLumo Oct 11 '18 at 16:19
  • 1
    Nice clean command! What's the reason for the "t" marker character? Just to add one byte for the newline on the previous line? – Mike Hill Oct 11 '18 at 20:29
  • 2
    @MikeHill command substitutions like `...` in shell always strip the trailing newline, that's trick to preserve it without adding a newline if there isn't one already there. See here for a nice complete explanation. – mosvy Oct 11 '18 at 20:37
  • I wouldn't say tail finds the last line in O(1) time, but for the vast majority of files, the last line is not going to be a large fraction of the total contents of the file. Even tail still has to scan backwards from the end of the file to ensure that it finds the last newline character. (tail can't actually read backwards, but I assume it does something like read X bytes, and if there is no newline to mark the end of the next-to-last line, read the last 2X bytes, etc, until it finally finds the full last line.) – chepner Oct 11 '18 at 20:53
  • 2
    Files are collections of bytes, not lines; it's really O(n) in the worst case because the last line might be the entire file, and you would have to read all the bytes to determine that. In practice, you only need to read a small fraction of those bytes, so it "feels" like O(1). (Also, you don't necessarily have to buffer more than X bytes; once you find the last newline in a particular block, you can simply re-read every byte after that and write them directly to standard out.) – chepner Oct 11 '18 at 21:48

The goal is to have a single command that outputs the last line of a file, and at the same time deletes that line from the original file.

sed -i -e '${w /dev/stdout' -e 'd;}' file

This would run the following sed script:

    w /dev/stdout

This writes the last line to /dev/stdout and then deletes it. All other lines are written back into original file through the -i option.

The script on the command line has to be split in two as there is no way to otherwise delimit the output filename of the w command (other than inserting a literal newline).

With ed:

ed -s file <<END_ED

ed opens the file file and places the cursor on the last line of the file. The first command prints that line to standard output, the second deletes it, and the last command writes the buffer back to the file. Using ed in this way may not be advisable on huge files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.