0

I want to run a task where I specify two commands which will be run in an alternating fashion with different parameters. E.g:

1. exec --foo $inputfile1 $inputfile.outfile1
2. exec --bar $inputfile2 $inputfile.outfile2
3. exec --foo $inputfile3 $inputfile.outfile3
4. exec --bar $inputfile4 $inputfile.outfile4

I could probably get away with specyfing two parallel commands or with specyfying two inputs but I need something more universal. The files will be specified using pipelined "find" command.

EDIT: My command for one action would look like this:

find . -name 'somefiles*' -print0 | parallel -0 -j10 --verbose 'exec --foo {} {.}.outfile' 

I just do not know how to do this in alternate fashion between two commands

So basically what I need parallel -j10 to do is to run 5 of this commands with foo parameter and 5 of them with bar parameter on a single set of files. I could probably get away with it not being alternating but i want parallel to take care about it being exactly 5/5 split so I don't end with more foos or more bars

2

You can first put all parameters in a file and then use

parallel -a filename command

For example:

echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *MAY*.pdf).out" >> /tmp/a 
echo "--page-label=3 $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JUNE*.pdf).out" >> /tmp/a 
echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JULY*.pdf).out" >> /tmp/a 

Then run the command:

parallel -a /tmp/a evince
  • I just noticed your edit (and the example). I am your will be able to redirect the output of your find commands to the file file.prall. – user5325 Apr 5 at 11:53
  • so basically i would need to provide a list of commands to parallel? That would probably work but is not very practical and usable with find. Can you provide an example where i could redirect output of find in an alternating fashion (probably without using a temp file if you think it's possible) – kulzaus Apr 5 at 12:02
  • 1
    Collect all the required parameters. Here I am collecting pdf files, and passing few parameters for evince. The third column is pdffile.out. echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *MAY*.pdf).out" >> /tmp/a echo "--page-label=3 $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JUNE*.pdf).out" >> /tmp/a echo "--fullscreen $(find /tmp -name *MAY*.pdf) $(find /tmp -name *JULY*.pdf).out" >> /tmp/a Then run the parallel -a /tmp/a evince – user5325 Apr 5 at 12:10
  • well, that could probably work i'll test it. Am I assuming correctly that you run the same find command both for input and output parameter to get the name of the file? It could probably work nice in most usecases but it probably result in some discrepancies in filenames if files were added/changed between those two finds. – kulzaus Apr 5 at 12:20
  • @kulzaus The issues you are raising are outside the scope of this question. My answer assumes that you can generate the parameters to exec command and as you said they are generated by find. If the your find command is producing output which is ambiguous, then you should modify your find command. – Mike V.D.C. Apr 5 at 12:23
1
exec --foo $inputfile1 $inputfile.outfile1 & \
exec --bar $inputfile2 $inputfile.outfile2 & \
exec --foo $inputfile3 $inputfile.outfile3 & \
exec --bar $inputfile4 $inputfile.outfile4 &
  • Well i asked it in gnu parallel tag. Basically i need to parallelize more than 10 jobs at once on few hundred files so that probably won't work – kulzaus Apr 5 at 11:38
  • I added an example of what i need to do – kulzaus Apr 5 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.