How can I delete a folder with lots of subfolders fast? [duplicate]

Question

I have a folder with 266778 subfolders. How can I delete it?

I have tried

cd ~/.local/share/Trash/
sudo rm -rf *

but it takes much time. After 1 minute 25 seconds real time and 0.072 seconds user time it only deleted 2500 folders. This way, it will take over two hours to delete this folder.

Is there a faster way to delete this folder? Why is there such a big difference between user time and real time?

real    1m25.474s
user    0m0.072s
sys     0m28.142s

I use Linux 2.6.32 (Ubuntu 10.04.4 LTS).

Answer 1

It depends on your definition of fast. The answers already here give a good solution for actually removing the directories from the filesystem, but if what you really need is to free the directory name as fast as possible, a rename on the same filesystem is instantaneous:

{ mv directory directory.gone && rm -rf directory.gone; } &

Technically this is cheating since I haven't sped up the actual deletion, but practically it's very useful: I use this trick all the time so I don't have to wait for slow deletion operations.

Answer 2

If your version of "find" implements the -delete sub-command, then you can try

find directory -delete

In this case:

find ~/.local/share/Trash/ -delete

Some commands, like rm, perform most of their work in the kernel. In the file-system routines, to be exact. Time spent performing system calls are accounted for in that way, so whilst your "rm" command runs for a long time, it doesn't do much work in user-land - the system calls performs most of the work.

Answer 3

rm -rf directory or rm -rf * of course is the fastest method unless your local rm implementation is broken.

Using find gives no advantages.

Whether this is fast or slow mainly depends on the filesystem and OS implementation. So the question seems to be inappropriate.

UFS and ZFS on Solaris are known to be very fast with this kind of task as both filesystem implementations include delayed background delete code that causes the unlink() and rmdir() calls to return fast even when the related object will take more time in total.

With the delayed background delete in the kernel, the directory updates can be done fast as well and this help to speed up the whole operation.

Answer 4

This is only a partial answer, sheding light on the three values the command returns; quoted from the time(1) manpage:

(i) the elapsed real time between invocation and termination, (ii) the user CPU time (the sum of the tms_utime and tms_cutime values in a struct tms as returned by times(2)), and (iii) the system CPU time (the sum of the tms_stime and tms_cstime values in a struct tms as returned by times(2))."

Answer 5

If you don't want to wait, and you want to avoid downtime, or you just need to get rid of the folder fast, queue the delete operation via mv on your next reboot. Also, the mv file operation is always faster than anything else, and no need to wait for blocking file-io operation, and continue what you are currently doing on that folder.

Just mv folder_to_be_deleted /tmp/folder_queue_for_deletion. Files in the /tmp directory will be deleted upon your next reboot.

Benchmark:

$ cat make_million_files.sh
#!/usr/bin/env bash

mkdir folder_to_be_deleted
for i in $(seq 0 1000000); do
  touch folder_to_be_deleted/$i;
done

$ ./make_million_files.sh
real    66m3.613s
user    5m47.507s
sys     61m15.432s

IO blocking operation

$ rm -rf folder_to_be_deleted
real    0m32.451s
user    0m2.086s
sys     0m25.094s

Non-IO blocking operation (Queue deletion on next reboot)

$ mv folder_to_be_deleted /tmp/folder_queue_for_deletion
real    0m0.012s
user    0m0.001s
sys     0m0.010s

In essence, the benefit is noticeable if you have multiple folders with lots of subfolders and you want to delete them fast, without downtime in your work, you might consider this solution, as it only takes 60 seconds to delete a million folders with lots of subfolders.

For 1000 folders with lots of subfolders, it would take ~1 hour of blocking IO using rm -rf, vs. 12 seconds via mv. And it would just take 60 seconds of boot time to delete everything. Finally, if you don't want to reboot, just mv the folder out of your way, and rm -rf it somewhere (other TTY session, etc.).