@Jinpeng was on the right track with GNU Parallel, just not the implementation.
Example: Run 10 parallel instances of your program, with each thread running your program only once:
parallel -j10 './myprog argument1 argument2 #' ::: {1..10}
Example: Run 10 parallel threads, with each of those threads running your program infinitely:
parallel -j10 'while true ; do ./myprog argument1 argument2 ; done #' ::: {1..10}
You can easily scale this to the hundreds of threads by replacing the 10
in my examples.
parallel -j200 ... ::: {1..200}
If your program produces any stdout messages and you want to see them as they are produced (rather than the default which collates them), the --ungroup
option to parallel may be useful.
parallel --ungroup ...
If you are running a lot of threads from your workstation and don't want things to become unresponsive, consider nice
ing the whole process subtree at launch time.
nice -n19 parallel ...
Side note, GNU Parallel is usually not installed by default but is usually in your normal package repos, so just install it like any other package: dnf install parallel
, apt-get install parallel
, brew install parallel
, etc.
myprog
instances and aggregates their output for you.