2

I pass the variables from a script to the main script in a parallel command like

./script1 | parallel -u --jobs 3 "./script2 {}"

How can I pass the job number as the second argument of ./script2?

something like

./script1 | parallel -u --jobs 3 "./script2 {} {}" ::: {1..3}

but the first {} should come from ./script1.

Note that I do not want the combination of arguments. Instead, the running jobs should be

./script2 var1 1
./script2 var2 2
./script2 var3 3
./script2 var4 1
./script2 var5 2
./script2 var6 3
./script2 var7 1

Imagine ./script2' writes to filesfile1.txt,files2.txt, andfile3.txtwhere the number is{1..3}` or the job number.

I want to make sure only one script is writing to its corresponding file. In other words, three parallel jobs write to three designated files.

2
  • Are you just looking for {#}?
    – meuh
    Apr 24, 2020 at 16:11
  • @meuh yes, just the number of enough. {#} is perfect if the first {} reads the pipeline from the script.
    – Googlebot
    Apr 24, 2020 at 16:31

2 Answers 2

1

You are most likely looking for {%} which is not the job number, but the job slot number.

LC_ALL=C seq 10 -0.1 1 | shuf |
  parallel --lb 'echo Job {} grabs {%}; sleep {}; echo Job {} releases {%}'

Notice how when a number is released it is grabbed by the next job. So no 2 jobs running in parallel will have the same {%} value.

This is covered in chapter 5 of GNU Parallel 2018 (http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html or https://doi.org/10.5281/zenodo.1146014).

0

I found out that the solution is --link flag to aggregate the combinations.

parallel -u --jobs 3 --link ./script2 {1} {2} :::: <(./script1) ::: {1..3}

Then, {1} comes from the script, and {2} just matches periodically.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.