Deleting lots of files

Question

I accidentally created 8 million files and every time I'm trying to delete them the server almost dies because of the rm process eating all disk IO (the server is remote without console).

Should something like this work i.e. invoking ionice?

sudo find /var/lib/php5/ -type f -exec ionice -c3 rm -f {} \;

Answer 1

I would do it like this...

sudo ionice -c3 find /var/lib/php5/ -type f -exec  rm -f '{}' +

the + is more xargs like (I think), and thus more efficient at very large numbers of files. putting ionice on the whole command should make every sub command also ioniced, as well as the search itself. Have you tried this?

you could also make it really nice nice -n 19 ionice -c 3 programname though it shouldn't be necessary.

or taking a cue from @alex you could

ionice -c3 find /var/lib/php5/ -type f -delete

Answer 2

I'd do something like this:

import os
os.nice(19)
dir = "/var/lib/php5/"
bad_files = [os.join(dir, file) for file in os.listdir(dir)
                     if is_bad(os.join(dir, file))]
for junk in bad_files:
  os.unlink(junk)
  time.sleep(0.1)

Answer 3

Maybe too silly.

What about deleting the whole directory structure?

find /var/lib/php5 -type d -print > /tmp/directories
rm -r /var/lib/php5
cat /tmp/directories | xargs mkdir