1

Let's say I have N bash commands stored in a file:

$ cat list_of_commands.txt
do_thing_1
do_thing_2
...
do_thing_N

The goal is to run them in parallel, but only 10 at a time (to avoid overloading the CPU). So, the first batch of commands (do_thing_1 to do_thing_10) starts running in parallel, and as soon as one of them are done, do_thing_11 should start running. When another command is done, do_thing_12 starts. Etc. Until all N commands are done.

I've found the GNU parallel tool but after tinkering with it, I'm unsure if it's the right tool for the job.

Thanks in advance!
~ Rijk

2
  • GNU parallel tool is the right tool
    – gapsf
    Commented Oct 14, 2022 at 15:19
  • xargs can also do this, and will typically be available by default when parallel is not. (See the -P option to xargs).
    – larsks
    Commented Oct 14, 2022 at 15:35

1 Answer 1

0

The simplest use:

cat list_of_commands.txt | parallel -j10

Read more:

1
  • Oh wow, it was that easy all along. Thank you for the reply and for your work on parallel!
    – rijkdw
    Commented Oct 17, 2022 at 7:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .