How do I delete files with the same names in multiple sub-folders from the command line?

Question

I have a folder called simulations, which contains 101 subfolders. These sub-folders are named posterior_predictive_sim_1 to posterior_predictive_sim_101.

Each subfolder contains five files: seq.nex, seq[1].nex, seq[2].nex, seq[3].nex, and seq[4].nex.

I want to delete the files seq[1].nex, seq[2].nex, seq[3].nex, and seq[4].nex from each subfolder (so that only seq.nex remains).

How do I do that from the command line?

Answer 1

With just rm:

rm simulations/posterior_predictive_sim_*/seq\[[1-4]\].nex

The escaped \[ and \] are literal square brackets. The unescaped [1-4] inside them is a glob pattern that matches the digits 1 to 4.

This only works if the glob doesn't expand to more filenames than will fit into one command-line, the ARG_MAX limit - this varies depending on the OS, but is about 2 million characters on modern Linux. ARG_MAX applies to external commands like rm, not to shell built-ins like for or echo.

Also, test this with /bin/echo instead of rm first. Because: 1. /bin/echo is safe, won't delete anything. and 2. /bin/echo is an external command, not the shell built-in version of echo so will also test if the total command length will be <= ARG_MAX.

Using GNU find:

find simulations/ -type f -name 'seq\[[1-4]\].nex' -delete

I suggest running this with -ls or -print before running it with -delete (or -exec rm as below), just to verify that it will do what you want.

Alternatively:

find simulations/ -type f -name 'seq\[[1-4]\].nex' -exec rm {} +

Some ancient versions of find don't support + at the end of the -exec predicate, so use \; instead:

find simulations/ -type f -name 'seq\[[1-4]\].nex' -exec rm {} \;

The difference between the + and the \; is that with +, find tries to fit as many filename arguments into the rm command as it can (ARG_MAX again). With \;, it runs rm once per filename - this is obviously much slower, but it still useful (e.g. when you need to run a program that only takes one filename argument).

BTW, \; isn't special to find. In fact, find itself uses just ; as a terminator for -exec. The \ is to make sure the shell passes the semi-colon on to find, i.e. prevent the shell from interpreting the ; as the end of the find statement. shell statements are separated by newlines OR semi-colons.

Answer 2

First you need the list of files. You can use ls, find, fd... Save this to a text file: ls posterior_predictive_sim_*/ > all.txt. People complain about parsing ls, tl;dr is it doesn't matter here.

Then you want to filter that down to those files you actually want to delete: cat all.txt | grep -v 'seq.nex' > del.txt (you can also try to fight your shell in accepting the more conservative regex pattern seq[\d+]\.nex).

Now open del.txt and check that you actually want to delete all that. You can just manually edit the file if you want.

Finally, actually delete them. rm can take many filenames at once, so unless you have many thousands of files, you could try to do that. I like to use GNU Parallel: cat del.txt | parallel --dry-run rm {}. It will print the actual commands it's planning to run, remove --dry-run to do it for real.

If you like living on the edge you can also pipe all these commands to each other instead of saving them to files. Hope you didn't make a mistake, rm'd files can't be recovered. Btw use https://github.com/andreafrancia/trash-cli instead, it lets you undelete files.

Answer 3

With a loop:

for i in $(ls); do rm posterior_predictive_sim_$i/seq\[*.nex; done

---

^{Leaving this up just as a reminder to myself and others why NOT to do it this way - see cas's comments below}