17

I have many text files in a directory, and I want to remove the last line of every file in the directory.

How can I do it?

  • 6
    What have you tried? unix.stackexchange.com/help/how-to-ask: "Sharing your research helps everyone. Tell us what you found and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and above all, it helps you get a more specific and relevant answer!" – Patrick Jan 31 '17 at 13:23
  • Why does the thought of somebody accidentally applying that to /etc have a very special quality of cringe induction :) – rackandboneman Feb 1 '17 at 11:27
4

If you have access to vim, you can use:

for file in ./*
do
  if [ -f "${file}" ]
  then
    vim -c '$d' -c "wq" "${file}"
  fi
done
16

You can use this nice oneliner if you have GNU sed.

 sed -i '$ d' ./*

It will remove the last line of each non-hidden file in the current directory. Switch -i for GNU sed means to operate in place and '$ d' commands the sed to delete the last line ($ meaning last and d meaning delete).

  • 3
    This will throw up errors (and not do anything) if the folder contains anything other than a regular file, e.g. another folder... – Najib Idrissi Jan 31 '17 at 14:27
  • 1
    @StefanR You're using -i which is a GNUism, so this is moot, but I would lose my old beard standing if I failed to point out that some older versions of sed do not allow you to put any whitespace between the $ and the d (or, in general, between the pattern and the command). – zwol Jan 31 '17 at 15:02
  • 1
    @zwol As I wrote, this will result in an error, not a warning, and sed will give up once it reach that file (at least with the version of sed I have). The next files won't be processed. Throwing away the error messages would be a terrible idea since you wouldn't even know it had happened! With zsh you could use *(.) to glob for regular files, I don't know about other shells. – Najib Idrissi Jan 31 '17 at 15:05
  • @NajibIdrissi Hmm, you're right. That surprises me; I would have expected it to complain about the directory but then go on to the next file on the command line. In fact, I think I'm going to report that as a bug. – zwol Jan 31 '17 at 15:07
  • @don_crissti I have GNU sed v4.3 too... I don't know what to tell you, I just tested again. gist.github.com/nidrissi/66fad6be334234f5dbb41c539d84d61e – Najib Idrissi Jan 31 '17 at 15:18
11

The other answers all have problems if the directory contains something other than a regular file, or a file with spaces/newlines in the file name. Here's something that works regardless:

find "$dir" -type f -exec sed -i '$d' '{}' '+'
  • find "$dir" -type f: find the files in the directory $dir
    • -type f which are regular files;
    • -exec execute the command on each file found
    • sed -i: edit the files in place;
    • '$d': delete (d) the last ($) line.
    • '+': tells find to keep adding arguments to sed (a bit more efficient than running the command for each file separately, thanks to @zwol).

If you don't want to descend into subdirectories, then you can add the argument -maxdepth 1 to find.

  • 1
    Hmm, but unlike the other answers this descends into subdirectories. (Also, with current versions of find this is more efficiently written find $dir -type f -exec sed -i '$d' '{}' '+'.) – zwol Jan 31 '17 at 15:14
  • @zwol Thanks, I've added that into the answer. – Najib Idrissi Jan 31 '17 at 15:23
  • -print0 isn't present in the full command, why put it into the explanation? – Ruslan Jan 31 '17 at 16:58
  • 1
    Also, -depth 0 doesn't work (findutils 4.4.2), it should instead be -maxdepth 1. – Ruslan Jan 31 '17 at 17:02
  • @Ruslan I had a first version where I used xargs but then remembered -exec. – Najib Idrissi Jan 31 '17 at 17:15
9

Using GNU sed -i '$d' means reading the full file and making a copy of it without the last line, while it would be a lot more efficient to just truncate the file in place (at least for big files).

With GNU truncate, you can do:

for file in ./*; do
  [ -f "$file" ] &&
    length=$(tail -n 1 "$file" | wc -c) &&
    [ "$length" -gt 0 ] &&
    truncate -s "-$length" "$file"
done

If the files are relatively small, that would probably be less efficient though as it runs several commands per file.

Note that for files that contain extra bytes after the last newline character (after the last line) or in other words if they have a non-delimited last line, then depending on the tail implementation, tail -n 1 will return only those extra bytes (like GNU tail), or the last (properly delimited) line and those extra bytes.

  • do you need a |wc -c in the tail call? (or a ${#length}) – Jeff Schaller Jan 31 '17 at 19:57
  • @JeffSchaller. Oops. wc -c was intended indeed. ${#length} would not work as it counts characters, not bytes and $(...) would remove the tailing newline character so ${#...} would be off by one even if all the characters were single-byte. – Stéphane Chazelas Jan 31 '17 at 20:09
6

A more portable approach:

for f in ./*
do
test -f "$f" && ed -s "$f" <<\IN
d
w
q
IN
done

I don't think this needs any explanation... except maybe that in this case d is the same as $d since ed by default selects the last line.
This will not search recursively and will not process hidden files (aka dotfiles).
If you want to edit those too see How to match * with hidden files inside a directory

  • Nice! If you change [[]] to [] then it will be fully POSIX compliant. ([[ ... ]] is a Bashism.) – Wildcard Feb 1 '17 at 3:50
  • @Wildcard - thanks, changed (though [[ is not a bashism) – don_crissti Feb 1 '17 at 11:57
  • A non-POSIX-ism, I should say. :) – Wildcard Feb 1 '17 at 12:00
3

POSIX-compliant one-liner for all files recursively starting in current directory, including dot-files:

find . -type f -exec sh -c 'for f; do printf "\$d\nx\n" | ex "$f"; done' sh {} +

For .txt files only, non-recursively:

find . -path '*/*/*' -prune -o -type f -name '*.txt' -exec sh -c 'for f; do printf "\$d\nx\n" | ex "$f"; done' sh {} +

Also see:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.