file mass deletion

Question

I would like to delete all the txt, xls,pdf files in a directory as well as its sub directories. I would like to save everything else.

find . -type f ! -iname '*.xml$,.png$,.jpeg$,.gif$,' -delete

that seemed to have done it, but it delete some other files that I need. How can I achieve that without deleting anything else?

Answer 1

Do this instead:

find . -type f -iname '*.xml' -o -iname '*.png'\
       -o -iname '*.jpeg' -o -iname '*.gif' -delete

You could also use regular expressions:

find . -type f -iregex '.*\.\(xml\|png\|jpeg\|gif\)$' -delete

Answer 2

There are basically 4 ways in which to approach this problem using find.

Method #1 - using -delete

$ find . -type f -iname '*.xml' -o -iname '*.png'\
       -o -iname '*.jpeg' -o -iname '*.gif' -delete

As others have mentioned within this Q&A, this method is the fastest and least resource intensive. Quoting from the find online docs:

10.1.6 Using the `-delete' action

---

The most efficient and secure method of solving this problem is to use the `-delete' action:

 find /var/tmp/stuff -mtime +90 -delete

This alternative is more efficient than any of the -exec' or-execdir' actions, since it entirely avoids the overhead of forking a new process and using exec' to run/bin/rm'. It is also normally more efficient than xargs' for the same reason. The file deletion is performed from the directory containing the entry to be deleted, so the-delete' action has the same security advantages as the `-execdir' action has.

The `-delete' action was introduced by the BSD family of operating systems.

NOTE: One thing to keep in mind with this approach, the use of -delete implies also the switch -depth. What does this mean? Here's an example of how -delete can burn you if you're not careful.

For example, say I have a subversion work directory where I want to clean up some files, but leave its .svn sub-directories intact. I might use the following command to accomplish this:

$ find . -not "(" -name .svn -type d -prune ")" -type f -print
./a.txt

But because -delete includes a -depth switch, the files that would actually be getting dealt with:

$ find . -not "(" -name .svn -type d -prune ")" -type f -print -depth
./.svn/all-wcprops
./.svn/entries
./.svn/format
./.svn/text-base/a.txt.svn-base
./a.txt

For this reason, when using -delete, care must be taken.

Method #2 - -exec command {} +

$ find . -type f -iname '*.xml' -o -iname '*.png'\
       -o -iname '*.jpeg' -o -iname '*.gif' -exec rm {} \+

Compared to the -delete method, this is most likely the next best option, in terms of performance & portability across Unixes. The -exec ... {} + notation works as follows:

from find man page

This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of `{}' is allowed within the command. The command is executed in the starting directory.

So in effect, this method works similar to xargs, but without having to jump through hoops of passing the output of a find through a pipe to xargs.

Method #3 - xargs

$ find . -type f -iname '*.xml' -o -iname '*.png'\
       -o -iname '*.jpeg' -o -iname '*.gif' -print0 | xargs -0 rm -f

The find ... -print0 will build a list of files matching the specified criteria. This list is then passed through the pipe to xargs. The -print0 switch puts an ASCII NUL character as a separator between each result from find. The -0 switch on xargs makes it assume that the files being passed in are seperated by ASCII NUL characters.

Compared to methods #1 & #2, this one will have similar performance to #2, however the -print0 switch is not universally supported across all Unixes.

Method #4 - -exec command {} \;

$ find . -type f -iname '*.xml' -o -iname '*.png'\
       -o -iname '*.jpeg' -o -iname '*.gif' | exec rm -f {} \;

Compared to the first 3 methods, this is the least performant. It literally calls the rm command for each individual file that the find command finds.

Additional considerations around security

One thing that might not be that obvious when using any of the above methods is that some of the methods are more secure than the others. You're probably saying to yourself, ... security? .. what? Here's an example.

Assume your root and you run the following command:

$ find /var/tmp/somedir -type f -exec rm {} \;

Unbeknownst to you, someone has maliciously created a link to the /etc directory under /var/tmp/somedir. When the above command runs, the /etc directory will also be deleted. This issue exists with any of the methods for deleting files, except for the -delete option (method #1).

tl;dr;

The fastest and most secure way to delete files with the help of find is to use -delete. Using xargs -0 can be similar in performance, but it is not as secure. The -delete action is not completely portable. The most efficient portable alternative is -exec ... +, but this is insecure and isn't supported by versions of GNU findutils prior to 4.2.12.

References

• Deleting lots of files with find - Debian Administrators Blog
• Find Users Manual
• Efficiency of find exec vs. xargs
• Deleting Files with find

Answer 3

There is an slight inaccuracy into slm answer.

NOTE & Disclamer : this have to be a comment to slm answer but righ now I can't do comments yet.

The example "someone has maliciously created a link " given into Additional considerations around security is not totally accurate both about Unix hard links and for Unix soft links.

To understand the difference between the two please see Hard link and Symbolic links in Unix or google it up.

For soft links (the most used type anyways) both GNU find and BDS find does not follow simbolic links unless is used a a specific -L flag to force symbolic link following. [See man find]

So this example is likely to not be a prolem, unless you force find to follow soft links using the -L flag. This is the dangerous choice, anyways.

For hard links, find will follow the link to another file but please note that hard linking to a another directory "will probably fail " as see into man ln for GNU ln :

   -d, -F, --directory
          allow the superuser to attempt to hard link directories (note: will probably
          fail due to system restrictions, even for the superuser)

So it is probably impossible to create the hard link to a directory in the first place and find will have nothing to follow.

Note that some BDS ln implementation does not have a -d option at all.