3

I'm writing Makefile rules like this:

out/%.x: ...
    mkdir -p $(@D)/
    ...

I intend to parallelize this Makefile (-j), which will trigger many mkdir -p's to simultaneously.

Is this safe? Or do I need to guard against problems by ensuring mkdir -p only happens once?

(Note that mkdir -p isn't necessarily a single operation at the file system level, unlike mkdir.)

  • 1
    @muru: Thanks for the links, but the serverfault answers say "This should not be a problem if this is properly handled (i.e. good error handling)" and "I guess it comes down to what mkdir -p does" which is... kinda useless when the entire question is about what mkdir -p does? And the second one also seems to suggest the problem in that question might be something else for that OP. – Mehrdad Jun 7 at 1:14
4

The rationale for POSIX mkdir says

Because no error is produced if any path component already exists, the -p option is also useful to ensure that a particular directory exists.

This suggests that parallel use should be safe. I can understand your doubts, given the potential for races, so let’s look at an implementation...

In GNU coreutils, the -p option ends up being handled by mkancesdirs, which has a helpful description which should allay your fears:

Ensure that the ancestor directories of FILE exist, using an algorithm that should work even if two processes execute this function in parallel.

(Instead of immediately acting on errors from mkdir, it tries to chdir into each component directory and only fails if that fails.)

If you’re using GNU Make, you could use its secondary expansion to handle directories in the same way as other prerequisites:

.SECONDEXPANSION:

%/:
        mkdir -p $@

out/%.x: $$(@D)/ ...
        ...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.