How can I partially serialize with GNU make

Question

When I have a make task where a specific target needs to be made before another, while in parallel mode, this is simple when using SunPro Make (dmake). The following makefile:

install:  dir dir/file

dir:
    mkdir dir

dir/file:
    cp source dir/file

could be made parallel make safe by changing the first line to:

install: dir .WAIT dir/file

or by using:

.NO_PARALLEL: install

The rest of the makefile will still be handled in parallel mode and even a list if targets to the left or to the right of .WAIT will be handled in parallel mode.

See: http://schilytools.sourceforge.net/man/man1/make.1s.html and http://schilytools.sourceforge.net/man/man1/dmake.1.html

but GNU make does not seem to have a similar option. Is there really no way to do this with GNU make?

To be more specific, the solution needs to be written in a way that allows the makefile to be processed with other make implementations.

The special target .WAIT could e.g. be in a make program dependent macro that is called e.g. $(WAIT)

Note that there is an apparent solution that does not help:

In theory one could make dir a dependency of dir/file in order to enforce to create dir before trying to copy to dir/file, but if you later copy another file into dir, this makes dir newer than dir/file. As a result, calling make more than once would copy to dir/fileagain and again, without the expected result from make to do the copy only in case that the source has become newer that dir/file.

This raises the alternate question whether there may be a topology of dependencies that enforces make to create dir before copying to dir/file without making dira dependency of dir/file.

Answer 1

In GNU make, you can use order-only-prerequisites to make sure dir is created before the rule to create dir/file is run.

See this page for more details.

You can see from the example in GNU make's manual that it's really tailored for adding dependency on a directory being created before creating files under it.

In the case of your specific example, something like this should do it:

dir/file: | dir
        cp source dir/file

Though you don't need to modify the existing rule, you could just add that dependency somewhere else in your Makefile...

To be more specific, the solution needs to be written in a way that allows the makefile to be processed with other make implementations.

I don't think this would work in other make implementations... One possible solution is to create a GNUmakefile that would include the actual Makefile and also add GNU make specific extensions.

You could also probably make GNU make ignore a rule such as .WAIT by defining it as a dummy rule in your GNUmakefile.

I hope these pointers are helpful!

Answer 2

You need to add dependencies so that directories are created before the files under them.

It is possible to use a "marker" file for the dependency on the directory, that way creating a file under it won't touch the timestamp of the directory. That doesn't require any GNU make specific features either.

In your example:

install: dir/file

dir/.marker:
        mkdir -p dir
        touch dir/.marker

dir/file: dir/.marker
        cp source dir/file

You can use something like .directory or .dir or .folder for the marker file...

Answer 3

Here is the my own answer that has been derived from the idea presented by Filipe Brandenburger and from generic methods used in the Schily Makefile system:

The makefile system makes sure that the following make macros are set up this way:

WAIT=       # empty with GNU make
WAIT= .WAIT # .WAIT special target with SunPro Make

MAKEPROG= <name of the make program> # This is from: smake, gmake, sunpro

_UNIQ= .XxZzy-

Now the makefile that makes use from the macro definitions above:

_NORULE=    $(_UNIQ)$(MAKEPROG)
__NORULE=   $(_NORULE:$(_UNIQ)gmake=)
NORULE=     $(__NORULE:$(_UNIQ)%=%)

install: dir $(WAIT) dir/file

dir/file: source
        cp source dir/file

dir:
        mkdir -p dir

$(NORULE)dir/file: | dir

$(NORULE) expands to nothing with gmake and to sunprowith SunPro Make.

In case of gmake, the whole makefile expands to:

install: dir dir/file

dir/file: source
        cp source dir/file

dir:
        mkdir -p dir

dir/file: | dir

In case of SunPro Make, the whole makefile expands to:

install: dir .WAIT dir/file

dir/file: source
        cp source dir/file

dir:
        mkdir -p dir

sunprodir/file: | dir

The last line is seen as a junk rule with no relevance.