rm -rf failing if deleting in parallel

Question

rm -rf will fail if something tries to delete the same file tree (I think because rm enumerates the files first, then deletes).

A simple test:

# Terminal 1
for i in `seq 1 1000`; do mkdir -p /tmp/dirtest/$i; done

# Now at teh same time in terminal 1 and 2
rm -rf /tmp/dirtest

There will be some output into stderr, e.g.:

...
rm: cannot remove directory `/tmp/dirtest/294': No such file or directory
rm: cannot remove directory `/tmp/dirtest/297': No such file or directory
rm: cannot remove directory `/tmp/dirtest/304': No such file or directory

I can ignore all the stderr output by redirecting it to /dev/null, but removing of /tmp/dirtest actually fails! After both commands are finished, /tmp/dirtest is still there.

How can I make rm delete the directory tree properly and really ignore all the errors?

Answer 1

I'm curious how the build system ended up like this. Are you able to change it? At a minimum, you can create a flag that lets the scripts know the other one is already doing the job...

mkdir /tmp/flag.rmdirtest && rm -rf /tmp/dirtest && rmdir /tmp/flag.rmdirtest

It would be better to re-architect the thing so that this isn't necessary.

Answer 2

Nasty. But in a sense, you're looking for trouble when two concurrent processes are manipulating a directory tree. Unix provides primitives for atomic manipulation of a single file, but not for whole directory trees.

A simple workaround would be for your script to rename the directory before removing it. Since your use case has cooperating scripts, it's ok for the new name to be predictable.

mv /build/tree /build/tree.rm.$$
mkdir /build/tree
rm -rf /build/tree.rm.$$

Maybe you can even do the rm in the background later, while your build performs some CPU-bound tasks.