Is there some mechanism to make find with -exec use multiple jobs?

Lets use this command for example

find ./* -exec flac --best {} \;

Is there some way to set the number of concurrent -exec commands to run easier than something like this solution:

run find ./* and save the results to a file/pipe them for another command to split into 16 parts (or however many threads you want) then run commands on these?

  • 1
    Pipe to xargs and use -P – muru Sep 2 '19 at 1:40
  • 1
    Good call. This is what I ended up using: find ./* -type f -print0|xargs -0 -P 16 -I % opusenc --bitrate 256 % %.opus – SultanLegend Sep 2 '19 at 3:52
  • 1
    Nice! You can post that as an answer. – muru Sep 2 '19 at 4:54
  • 1
    No need for the ./*. A dot would be enough, and that would not blow up if you have many names in the current directory. I would also suggest a -name test to get names you want and not everything (and -type f to avoid running flac (or opusenc) on directories). If you have one single directory, you shouldn't need find at all, and if you're using bash or zsh, you could use ** instead of find. – Kusalananda Sep 2 '19 at 6:57

You can set the number of workers if you use xargs

find . -type f -print0|xargs -0 -P 16 -I % opusenc --bitrate 256 % %.opus

xargs -P 16

(since I struggled to find more details online or understand the man page section)

It does your command on 16, or the number you pass to -P, of the inputs at a time.

For my command I was trying to convert an organized folders of cd music files [artist/album/music file] to opus. This was a copied folder so I wanted to delete the originals. So a better command than the one I used would be

find . -type f -print0|xargs -0 -P 16 -I % sh -c 'opusenc --bitrate 256 % %.opus; rm %'

After conversion the original file is removed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.