2

I'm trying to recursively delete a directory with rm -rf, but this fails because some inner directories don't have the w permission set. I know how to fix this with chmod. However, this requires to iterate over the whole directory twice, which can be slow.
Is there a way to remove such a directory in one go? (Assuming you have enough permissions to give yourself enough permissions)
sudo is not an option (limited user on pc in question).

11
  • what's wrong with chmod -R 0777 <dir> && rm -rf <dir>
    – alecxs
    Commented Jul 22, 2020 at 9:06
  • 1
    same but slower find <dir> -depth \( -exec chmod 0777 {} + -a -exec rm -r {} + \)
    – alecxs
    Commented Jul 22, 2020 at 9:32
  • That it has to go over the whole directory twice (once for chmod -R and once for rm -r, which seems unnecessarily slow). I'm looking for rm -rf but adjust permissions on the fly if necessary
    – Koen G.
    Commented Jul 22, 2020 at 9:34
  • 1
    Indeed, upon recheck, with + it works, while it fails with \; (what I was doing out of bad habit). This suggests + might also fail when reaching the maximum command length. Whether that is something worth worrying about will depend on the complexity of the directory structure.
    – Koen G.
    Commented Jul 22, 2020 at 15:06
  • 1
    No, find, like xargs batches calls to the command to limit the size fo the arguments.
    – xenoid
    Commented Jul 22, 2020 at 19:15

4 Answers 4

2

rsync with an empty dummy directory seems fine:
mkdir empty; rsync -r --delete empty/ targetdir/; rmdir empty targetdir
With a 10x repeated test on a simple example, this took 10-14s (14 was an outlier, all others took 10 or 11s),
vs. chmod -R u+w targetdir && rm -rf targetdir, which took 19-25s
and find targetdir -type d -exec chmod 755 {} \; && rm -rf targetdir, which took 12-16s but will likely deteriorate more than rsync with more complex folder structures.

1
  • this should be accepted answer because it's exactly what asked and the only one that doesn't chmod (besides the fact that every find construct will badly fail on missing read permissions)
    – alecxs
    Commented Jul 23, 2020 at 5:44
1
find directory -type d -exec chmod 775 {} \;
rm -rf directory
2
  • Is this not iterating over the whole directory tree twice as well?
    – Koen G.
    Commented Jul 22, 2020 at 8:59
  • 2
    +1 for -type d because it makes it faster as it is not unnecessarily changing permissions on files. You do not need permissions on a file to delete it, this is managed by the w permission of the directory it is in.
    – Koen G.
    Commented Jul 22, 2020 at 9:47
1

I did not do performance measurements. I just guess it makes sense to delete all the files immediately when a directory is read (because it is read completely anyway and you might risk losing already read metadata from the page cache with -depth) and change the permissions of those 1st level subdirectories then for which that is necessary. With lots of files and subdirectories it seems hard to avoid reading the same directory content again and without the files that should be much faster.

I suggest to use a wrapper script which is called for each directory. That script makes find

  • delete all files in the argument directory
  • fix the permissions of level-1 subdirectories where necessary
  • call the script for each level-1 subdirectory
#!/bin/bash
  
set -x

dir_path="$1"

test -d "$dir_path" || exit 1

declare -i tests_x=0
declare -i tests_x_limit=30
while true; do
    # there is a race condition between the -exec + and xargs
    if [ -x "$dir_path" ]; then
        break
    else
        sleep 1
    fi
    tests_x+=1
    if [ "$tests_x" -gt "$tests_x_limit" ]; then
        exit 1
    fi
done

find "$dir_path" -mindepth 1 -maxdepth 1 -type f -delete -o -type l -delete \
-o -\( -type d ! -perm -700 -exec  chmod u=rwx {} + , -print0 \) |
    xargs -0 -n 1 -r "$0"

And run this with

deletion.sh /start/dir

You need rm -r afterwards as the above approach does not delete the directories (and cannot do so in a useful (i.e. faster than rm -r) way IMHO).

Independently of the command approach it may help to increase the filesystem commit time. So for ext4:

mount -o remount,commit=60 /path/to/mp
4
  • I like this answer because it follows the question's spirit of not reading directories twice. Unfortunately, it doesn't seem to delete the directory tree itself (also after adding - to type in -o and adding -o type l after -type f
    – Koen G.
    Commented Jul 23, 2020 at 9:19
  • @KoenG. Right, I forgot to mention that you need rm -r afterwards. It seems to me that it is impossible to do this with just one directory read. So it is just about minimizing the pain. You can reduce it to reading large directories just once (assuming most of the entries are files) once with my solution so that rm -r is fast. My solution could be improved by writing the directory paths to a FIFO and have some background process read them from there and read those directories every few seconds in order to keep their metadata in the page cache so that the rm -r is as fast as it can be Commented Jul 26, 2020 at 21:06
  • Changing to \; i.o. + for the chmod and adding rmdir "$dir_path at the end deletes the directories on the fly. As far as I can see all directory contents are only read once in that case (by the find). Very disappointed that this kind of behaviour is not an option of rm, but an approach like this is what I was looking for. Unfortunately, it is very slow in my testing. This might be the overhead of bash scripting though (and lack of parallellism). I'll set up a test on a slower drive to shift the balance (this will take some days due to practicalities).
    – Koen G.
    Commented Jul 28, 2020 at 8:41
  • 1
    Calling rmdir for just one directory is not very efficient, though. I also just realize that my approach keeps one instance of bash and xargs running for every directory. This whole thing should rather be done in Python, Perl or whatever so that you do not have the overhead of new processes. Commented Jul 28, 2020 at 9:24
0

You can run a small shell in each directory to change its mode and delete its contents:

find . -depth -type d -exec sh -c 'echo "Removing $1 contents"; chmod +w "$1" ; rm -r "$1"/*' anything {} \;
2
  • That is a terrible idea because pathname expansion results are sorted. Exactly the problem that is to be avoided. Commented Jul 22, 2020 at 20:37
  • Using $1/* instead of just $1 does fix the problem with find and subdir1/subdir2 with w missing on both subdir1 and subdir2 (it never tries to remove any folder's content without setting its w permission first). Using multiple -exec instead of -exec sh -c (because it was faster), this took 18-20s on the test case of the rsync answer
    – Koen G.
    Commented Jul 23, 2020 at 8:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .