I have script I'd always like to run 'x' instances in parallel.
The code looks a like that:
for A in
do
for B in
do
(script1.sh $A $B;script2.sh $A $B) &
done #B
done #A
The scripts itself run DB queries, so it would benefit from parallel running. Problem is
1) 'wait' doesn't work (because it finished all background jobs and starts new ones (even if I include a threadcounter), that wastes lots of time.
2) I couldn't figure out how to get parallel to do that. I only found examples where the same script gets run multiple times, but not with different parameters.
3) the alternative solution would be:
for A in
do
for B in
do
while threadcount>X
do
sleep 60
done
(script1.sh $A $B;script2.sh $A $B) &
done #B
done #A
But I didn't really figure out how to get the thread count reliable.
Some hints into the right direction are very much welcomed.
I'd love to use parallel, but the thing just doesn't work as the documentation tells me.
I do
parallel echo ::: A B C ::: D E F
(from the doc) and it tells me
parallel: Input is read from the terminal. Only experts do this on purpose. Press CTRL-D to exit.
and that is just the simplest example of the man pages.