1

How to make this loop parallel :

for a in $(seq 1 3)
do
  for b in 0.1 0.2 
  do
    echo process with a=$a and b=$b &
  done
done

is it in parallel or not ? In fact, I want to run the instruction echo process with a=$a and b=$b in parallel for each combination of the values a and b Here is the result of runing the above shell :

process with a=1 and b=0.1
process with a=2 and b=0.2
process with a=2 and b=0.1
process with a=3 and b=0.2
process with a=3 and b=0.1
process with a=1 and b=0.2

Thanks.

  • 1
    The fact that the output appears nondeterministic should already suffice to demonstrate that the background jobs run in parallel. – tripleee Apr 18 '17 at 18:45
3

With GNU Parallel it looks like this:

parallel echo process with a={1} and b={2} ::: 1 2 3 ::: 0.1 0.2 
seq 1 3 | parallel echo process with a={1} and b={2} :::: - ::: 0.1 
parallel echo process with a={1} and b={2} :::: <(seq 1 3) ::: 0.1 0.2

I assume echo is just an example, as parallelizing echo is hardly worth it.

0

If the thing you run in the inner loop takes any amount of time (an echo is very quick to run), then, when the loops have started all the asynchronous processes, they will be running concurrently.

The loops themselves are not "parallel".

0

This should have be a comment, but comments are too small.

Make sure your solution works for output of arbitrary sizes and does not mix output from one job with another. Here is a small example to test that:

#!/bin/bash                                                                               

# stress test of parallelizing programs                                                   

longline() {
    # print one big line with 'a's followed by 'b's followed by 'c's                      
    perl -e 'print "@ARGV ", map { "$_"x10000000 } (a..c)' "$@"
    echo
}


echo "Run testprogram in parallel"
for a in $(seq 1 3)
do
    for b in 0.1 0.2
    do
        longline $a $b &
    done
done |
# use 'tr' to compress many 'a's into a single 'a'                                        
# if the output is not 'abc' on every line, lines have been mixed                         
  tr -cs '@'

echo "Run testprogram in serial"
for a in $(seq 1 3)
do
    for b in 0.1 0.2
    do
        longline $a $b
    done
done | tr -cs '@'


echo "Compare with GNU Parallel"
export -f longline
parallel -kj0 longline :::: <(seq 1 3) ::: 0.1 0.2 | tr -cs '@'
  • Thanks, I found the response in your exemple, I'd try it :) – Belkacem Thiziri Apr 19 '17 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.