I sometimes wind up running the same, fairly-expensive command over and over again, to get the same output. E.g.,
ffprobe to get information about a media file. Given the same input, the same output should be produced always—so caching should be possible.
I've see Memoizing/caching command line output but I'm looking for a more thorough implementation: in particular, that one just seems to compare the command line—if one of the files passed is modified, it won't notice. (It also has a bunch of fixed-length buffers which make me suspicious and is oddly a daemon.)
Before I go off and write my own, I'm curious if one already exists. The key requirements:
- Must re-run the command if any of the input files (on the command line) change
- Must re-run the command if any of the command line options change
- I'm OK (and honestly expect) commands to be run "non-interactive": e.g., with
/dev/nullas stdin, and a two different files as stdout and stderr.
- If the command errors out, I'm OK with either that being cached along with the exit code or alternatively it just not being cached at all.
- Should return cached content as often as possible, given the above. But correctness comes first.
- Preferable if the cache can be shared between several machines (all under common control) e.g., via NFS.
Basically what I'm thinking of doing, if I write my own, is (skipping some locking and error checking for briefness): take command line + stat results of each item on the command line (error or dev, inode, size, mtime) and pass that whole mess through SHA-512 or SHA-256. That'll give a key that is a fixed size but will change if the command or the files change (unless someone makes a size- and mtime-preserving change, in which case they deserve what they get). Check if that key is in the cache directory. If it already exists, copy their contents to stdout & stderr. Otherwise, run the command in a subprocesses with stdin /dev/null and two files as stdout and stderr. If successful, put the files in to the cache directory. Then copy their contents to stdout and stderr. If it turns out I wind up writing my own, design feedback welcome. And the result will be free software.