I want to free up some space by deleting redundant files within many subdirectories. However I always want to retain the last file of a file stem. For example I have an assortment (next to other files that should not be touched) similar to this:

FILE_1-1 FILE_1-2 FILE_1-3 FILE_2-1 FILE_2-2

within a whole bunch of branching subdirectories. I want to keep FILE_2-2 in this instance, but in another folder it would be FILE_3-1 or whatever. Always the last file. I know how to get rid of all of them using the find command which traverses the subdirectories nicely:

find . -name "FILE_*" -type f -delete

How can I exclude the last FILE_* within each directory where removal happens using the find command? Googling (https://www.unix.com/shell-programming-and-scripting/36639-find-remove-all-but-latest-file.html) turned up the following:

ls -1 FILE_* | sort -r | tail +2 | xargs -i rm {}

But this command does not work with branching directories, does it?

  • to make matters worse, you'll also need to worry about the sort functionality if you have more than one digit in the sequence. FILE_1-10 comes lexicographically before FILE_1-2 . Padding with zeros (eg FILE_1-002 ) would make numeric and lexicographic order consistent. – Daniel Farrell Sep 27 '19 at 15:04
  • For a general answer, yes. In my case the highest number is fortunately FILE_3-5. I hope this helps to keep it simple. – And Sep 27 '19 at 15:07
  • You mention branching subdirectories, but they need to be considered when finding "the last file", or is the "last file" the one in each separate subdirectory? – Jeff Schaller Sep 27 '19 at 16:13
  • There's a slight inconsistency; your example command shows ls ... FILE_*.txt while your sample filenames do not end in .txt – Jeff Schaller Sep 27 '19 at 16:16
  • You are correct, I edited that out. And It's always the last file within each separate subdirectory. – And Sep 30 '19 at 8:01

In a directory you can find the "stems" with:

ls * | sed -r 's/[0-9]+$//' | sort | uniq

For each stem you can list all files with the last with:

ls  FILE_1-* | head -n -1

Or if you want to keep the most recent:

ls -rt FILE_1-* | head -n -1

Both commands can be followed by | xargs rm to delete their output

Put all that in a short script(*) and run:

find . -type d -exec {the_script} {} \;

(*) the short script would look like:

#! /bin/bash
cd "$1"  # takes the directory as a parameter
for stem in $(ls FILE* | sed -r 's/[0-9]+$//' | sort | uniq)
    ls -rt ${stem}* | head -n -1 | xargs echo rm 

Of course test thoroughly and replace echo rm by just rm when you are confident that it will erase the right files.

  • I put the first and the third line into the script and then executed the last line and I'm getting the error find: ‘{the_script}’: No such file or directory. I put the script into the parent folder where I am executing the find command. What do I do wrong? – And Sep 30 '19 at 10:24
  • {the script} is of course a place holder for your actual script name. Note that your script is also assumed to loop over the stems: line 3 is executed once for each stem found with line 1. – xenoid Sep 30 '19 at 11:17
  • Added sample script, use at your own risk. – xenoid Sep 30 '19 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.