Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

In a makefile, I have several rules that look like this:

out.txt: foo.sh input.txt
  ./foo.sh -i input.txt > out.txt

If foo.sh fails, then out.txt will be created as a 0-sized file. If I run make again, it will mistakenly assume that the out.txt file was created successfully, and it won't run the rule again.

What's the right way to handle this sort of scenario?

share|improve this question

4 Answers 4

up vote 9 down vote accepted

You can request that make delete the target file if the rule fails, by defining a special target named .DELETE_ON_ERROR. It doesn't need to do anything or have any dependencies, so just add this to your makefile:

.DELETE_ON_ERROR:

Then you get the following:

$ cat Makefile
.DELETE_ON_ERROR:
foo:
    false > foo

$ make
false > foo
make: *** [foo] Error 1
make: *** Deleting file `foo'
zsh: exit 2     make

$ stat foo
stat: cannot stat `foo': No such file or directory

From Errors in Recipes:

Usually when a recipe line fails, if it has changed the target file at all, the file is corrupted and cannot be used—or at least it is not completely updated. Yet the file's time stamp says that it is now up to date, so the next time make runs, it will not try to update that file. The situation is just the same as when the shell is killed by a signal; see Interrupts. So generally the right thing to do is to delete the target file if the recipe fails after beginning to change the file. make will do this if .DELETE_ON_ERROR appears as a target. This is almost always what you want make to do, but it is not historical practice; so for compatibility, you must explicitly request it.

share|improve this answer
3  
The advantage of .DELETE_ON_ERROR is that it automatically applies to every rule. It has downsides though: it's specific to GNU make; it doesn't trigger if the build is interrupted violently (power failure, unmaskable signal); it removes the partial file, which can be useful in investigating failed builds. –  Gilles Jul 6 '11 at 18:58

I suppose that foo.sh properly returns non-zero on error. Then you should make a temporary, and only overwrite the output on success.

tmp=$$(mktemp) && ./foo.sh input.txt > $$tmp && mv $$tmp $@ || rm -f $$tmp && false
share|improve this answer
    
I wouldn't delete $tmp if the generation fails, partial files can be useful to debug failed builds. If you do, you must make sure the command fails if rm is invoked, e.g. || rm $$tmp && false. –  Gilles Jul 6 '11 at 17:54
    
Yes, it is a good idea to keep partial files around for debugging, but that would complicate the solution a little. I added the bit about false, good. –  Juliano Jul 6 '11 at 18:12

Whenever possible, makefile rules should first create their target under a temporary name, then move it into place. That way, if the build process is interrupted for any reason, there won't be a half-written target file that can't be distinguished from a fully written file.

out.txt: foo.sh input.txt
        ./foo.sh -i input.txt >$@.tmp
        mv -f $@.tmp $@

mv -f $@.tmp $@ is a common makefile idiom.

Juliano's answer shows a variant where the name of the temporary file is generated dynamically. The dynamic name generation is required if there may be more than one process generating the same target or if the directory can be written to by other users. This is very rarely the case for a build tree (if these were issues, a lot of what's going on in a typical makefile would break), so the extra complexity is usually not necessary.

share|improve this answer
    
Is that actually necessary? I thought make would clean up half-written files if it got interrupted or the process has a signal (and experimentation seems to indicate it does). By default it doesn't clean up files if the process exits non-zero, but that's what .DELETE_ON_ERROR is in my answer; with that on there should be no need for this temp file juggling –  Michael Mrozek Jul 6 '11 at 17:58
    
@Michael What if make dies without having time to clean up (power failure, OOM, user sending a KILL signal, …)? –  Gilles Jul 6 '11 at 18:09
2  
Well, how often does that happen? If you're kill -9ing make you deserve what you get, and I would hope somebody is going to do a clean build if there's something exceptional like a power failure –  Michael Mrozek Jul 6 '11 at 18:32

Since it seems that foo.sh is something you have written (or is generated by something you wrote), I would consider adding a -o flag to it, which takes the output file name. Then you do not have to depend on the default I/O redirection behavior. Your script can clean up partial output files, or possibly even avoid creating the file in the first place if you can detect the error early enough.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.