1

Trying to delete thousands of files on multiple servers.
rm will not work.
Wrote a generic script to run from cron to delete on a regular basis:

...  
/bin/perl -e 'unlink ( <`find ${DIRECTORY} -name "*.${FILETYPE}" -type f -mtime +${DELETE_OLDER_THAN}`> ) or die "Nothing deleted\n" ' >> ${REMOVAL_LOGFILE} 2>&1  
...  

I want to be very specific about what I am deleting.
I'm using perl because it's appears to be faster than -exec rm {} :/
Is find allowed in this context?

4
  • 2
    perl has a find module here
    – Jeff Schaller
    Commented Jun 7, 2017 at 19:18
  • There is also find2perl perldoc.perl.org/find2perl.html
    – user14755
    Commented Jun 8, 2017 at 3:24
  • 1
    There is seldom if ever any reason to mix Perl with find(1). Perl's File::Find is generally much more versatile than find(1). Commented Jun 8, 2017 at 6:05
  • I used perl -e unlink - worked well for what I wanted to accomplish
    – Marinaio
    Commented Jun 22, 2017 at 19:47

1 Answer 1

4

If your find supports the -delete action you'll find this about as fast as you can go without spawning multiple delete processes:

find "$DIRECTORY" -name "*.$FILETYPE" -type f -mtime +"$DELETE_OLDER_THAN" \
    -delete -print >> "$REMOVAL_LOGFILE" 2>&1

You'll find that -exec rm {} \; is slow because it creates a new process for every delete operation. If your find has it, -exec rm {} + would be faster because it would create one rm process for a considerable number of files. But fastest of all single-threaded solutions would be -delete.

With somewhat more complexity you could fork one find .. -delete process per directory, in a tree of directories, and have (at least some of) those running in parallel. But I don't know how many directories you're processing so I can't determine whether this would be a net gain.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .