I have to run a script in 100 parts while making sure that only 10 parts run at parallel at any point of time.

Below script will trigger all the 100 processes at the same time :-

for i in {1..100}
    nohup ksh my_background_script.ksh -mod ${i} &

There are two tools commonly used for this, xargs and GNU parallel:

  1. xargs is older, very often installed by default, but more limited. You would use it like this:

     seq 1 100 | xargs -P 10 -I {} nohup ksh my_background_script.ksh -mod {} &

    The -P 10 is telling it to run 10 processes in parallel.

  2. GNU parallel, which although not installed by default on many systems, should be readily available for any GNU/Linux system and is far more powerful than xargs. You would use it like this (probably, the details depend on what you are running):

     parallel -j 10 nohup ksh my_background_script.ksh -mod ::: {1..100}

Some more details on the two programs and their respective uses can be found here: GNU parallel vs & (I mean background) vs xargs -P


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.