1

I want to run a task where I specify two commands which will be run in an alternating fashion with different parameters. E.g:

1. exec --foo $inputfile1 $inputfile.outfile1
2. exec --bar $inputfile2 $inputfile.outfile2
3. exec --foo $inputfile3 $inputfile.outfile3
4. exec --bar $inputfile4 $inputfile.outfile4

I could probably get away with specyfing two parallel commands or with specyfying two inputs but I need something more universal. The files will be specified using pipelined "find" command.

EDIT: My command for one action would look like this:

find . -name 'somefiles*' -print0 | parallel -0 -j10 --verbose 'exec --foo {} {.}.outfile'

I just do not know how to do this in alternate fashion between two commands

So basically what I need parallel -j10 to do is to run 5 of this commands with foo parameter and 5 of them with bar parameter on a single set of files. I could probably get away with it not being alternating but i want parallel to take care about it being exactly 5/5 split so I don't end with more foos or more bars

Improve this question

3 Answers 3

Reset to default
2

You can first put all parameters in a file and then use

parallel -a filename command

For example:

echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *MAY*.pdf).out" >> /tmp/a 
echo "--page-label=3 $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JUNE*.pdf).out" >> /tmp/a 
echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JULY*.pdf).out" >> /tmp/a

Then run the command: 

parallel -a /tmp/a evince
Improve this answer
9
  • I just noticed your edit (and the example). I am your will be able to redirect the output of your find commands to the file file.prall.
    – user5325
    Apr 5, 2019 at 11:53
  • so basically i would need to provide a list of commands to parallel? That would probably work but is not very practical and usable with find. Can you provide an example where i could redirect output of find in an alternating fashion (probably without using a temp file if you think it's possible)
    – kulzaus
    Apr 5, 2019 at 12:02
  • 1
    Collect all the required parameters. Here I am collecting pdf files, and passing few parameters for evince. The third column is pdffile.out. echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *MAY*.pdf).out" >> /tmp/a echo "--page-label=3 $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JUNE*.pdf).out" >> /tmp/a echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JULY*.pdf).out" >> /tmp/a Then run the parallel -a /tmp/a evince
    – user5325
    Apr 5, 2019 at 12:10
  • well, that could probably work i'll test it. Am I assuming correctly that you run the same find command both for input and output parameter to get the name of the file? It could probably work nice in most usecases but it probably result in some discrepancies in filenames if files were added/changed between those two finds.
    – kulzaus
    Apr 5, 2019 at 12:20
  • @kulzaus The issues you are raising are outside the scope of this question. My answer assumes that you can generate the parameters to exec command and as you said they are generated by find. If the your find command is producing output which is ambiguous, then you should modify your find command.
    – Mike V.D.C.
    Apr 5, 2019 at 12:23
1 
exec --foo $inputfile1 $inputfile.outfile1 & \
exec --bar $inputfile2 $inputfile.outfile2 & \
exec --foo $inputfile3 $inputfile.outfile3 & \
exec --bar $inputfile4 $inputfile.outfile4 &
Improve this answer
2
  • Well i asked it in gnu parallel tag. Basically i need to parallelize more than 10 jobs at once on few hundred files so that probably won't work
    – kulzaus
    Apr 5, 2019 at 11:38
  • I added an example of what i need to do
    – kulzaus
    Apr 5, 2019 at 11:45
0

This should alternate between --foo and --bar:

find . -name 'somefiles*' -print0 |
  parallel -0 -j10 --verbose 'exec {= $_=seq() % 2 ? "--foo" : "--bar" =} {} {.}.outfile'
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .