What I want to do is something like this:

inotifywait -m ./input -e close_write -e moved_to |
    parallel --max-args 1 "someSlowSingleThreadedTaskThatActsOnThisFile"

I can't imagine that that would work as-is (it doesn't seem to). I assume parallel has to be given all of its inputs at the start, before it begins running. Therefore my question is this: Is there a way that I could add more inputs to parallel as it's already going? I imagine the inputs as a pool of work, and it just processes that pool as fast as possible, using the maximum sensible number of threads for the current system. What I want to do is add to that pool while it's still going.

How can I continuously feed parallel with more inputs as it's running (specifically, as more files are added to the folder)? Alternatively, if parallel isn't a good choice for this task, what is?


How about using the entr command you supply entr a file or list of files and then whenever a change to that file is made it runs an arbitrary command. Would this help I use it to rebuild previews and projects automatically when the files are changed. I don't see why you couldn't use parallel with it.

Jason C.

  • This is what inotifywait already does. The issue is not finding a way to run a command when the directory changes, but rather finding a way to process those files in parallel while still being able to add to the "pool" of files left to be processed. – Clonkex Feb 22 at 3:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.