1

I'm looking for a means to complete the following:

  1. Run some command that is intended to produce foo.out
  2. The solution substitutes in foo.out.tmp for foo.out so the target command actually operates on a temp file on the same mount as foo.out ends up.
  3. Process terminates:
    • If the process finishes with exit code 0, atomically mv -f foo.out.tmp foo.out
    • If it doesn't, leave the foo.out.tmp in place, or remove it, since it's likely garbage.

I saw Cleanup redirected output file from failed command however it relies on redirecting stdout, and I have some commands that produce multiple foos (foo.1.out, foo.2.out, foo.3.out, ...).

I can imagine some sort of wrapper script that if told that it was expected to create foo.out[, ...] could sed the command line in before running the subprocess then waiting and check the exit code, or doing some LD_PRELOAD voodoo to intercept the open call to modify the parameters on their way.

For my use case, discarding the temp file(s) is also perfectly satisfactory. I couldn't think of a way to use e.g. a Docker container and some cunning overlay mount options such that on failure the whole output volume would disappear.

On a reasonably modern Linux (kernel 4.x) if that helps.

4
  • 1
    It sounds like we'd need to know more about the command you are running. Can you know what output files it will produce at the time of launching it?
    – terdon
    Oct 11, 2020 at 17:15
  • Is the location of foo.out fixed? Maybe it's enough to run the command in a temporary subdirectory, so it creates foo.out there. In case of success you just mv foo.out ../. Oct 11, 2020 at 17:19
  • @terdon - sadly I'm looking for a general solution, hence why I'm chasing some very esoteric rabbits into their burrows. This is to integrate to an in-house 'job scheduler', so there could be any number of files involved, but the job spec is at least explicit enough about what it's trying to create.
    – Rich L
    Oct 12, 2020 at 1:17
  • @KamilMaciorowski thanks for the idea, unfortunately, it's a .. fog? ... mist?... whatever is much less impressive than a cloud system, and the backend store is shared over NFS and many overlapping tasks want to either get the data, or produce it themselves otherwise. That's why the atomic rename in place, which fortunately NFS honors is a big part of the design the Powers That Be like.... :/
    – Rich L
    Oct 12, 2020 at 1:20

0

You must log in to answer this question.

Browse other questions tagged .