I have a file with many lines, and on each line I have the arguments I want to pass to parallel with a tab separator.

I run this script

cat $hdd_file | grep $ssd | parallel -C '\t' clean_and_destroy

And it works, $hdd_file is the filename, the grep collects the lines which hdds have a certain $ssd as a cache, and then the parallel calls a function which destroys their connection.

Now that I made new partitions to the cleaned ssds, I try to call parallel like this:

cat $hdd_file | grep $ssd | parallel -C '\t' create_new_cache :::+ `seq $partitions_per_ssd`

Which should get the arguments from the pipe and pair them with the numbers given, but it does not.

cat $hdd_file | grep $ssd | parallel -C '\t' create_new_cache ::: {} :::+ `seq $partitions_per_ssd`

I also tried this and it still doesn't work. The {} :::+ are passed as arguments for some reason


GNU parallel solution:

Sample input.txt (for demonstration):

a   b
c   d
e   f

grep '^[ac]' input.txt will be used to emulate command(or pipeline) acting like input source file

parallel -C '\t' echo :::: <(grep '^[ac]' input.txt) ::: $(seq 1 3)

The output:

a b 1
a b 2
a b 3
c d 1
c d 2
c d 3

  • :::: argfiles - treat argfiles as input source. ::: and :::: can be mixed.

To aggregate elements from each input source - add --xapply option:

parallel -C '\t' --xapply echo :::: <(grep '^[ac]' input.txt) ::: $(seq 1 2)

The output:

a b 1
c d 2
| improve this answer | |
  • thanks, that's a step closer to what i want. I actually want this: parallel -C '\t' echo :::: <(grep '^[ac]' input.txt) :::+ $(seq 1 2) a b 1, c d 2 but i get parallel: Error: Cannot open input file :::+': No such file or directory.` – Θεόφιλος Μουρατίδης Mar 13 '18 at 14:49
  • @ΘεόφιλοςΜουρατίδης, you need --xapply option. See my update – RomanPerekhrest Mar 13 '18 at 14:58
  • @ΘεόφιλοςΜουρατίδης :::+ is from 20160422. You probably have an older version. – Ole Tange Mar 14 '18 at 7:58
  • You can also use '-' as input file: grep '^[ac]' input.txt | parallel -C '\t' echo :::: - :::+ {1..10} – Ole Tange Mar 14 '18 at 8:00
  • @OleTange your example doesn't work, I tried it many times with different variations, it gets the :::+ as an argument of the function – Θεόφιλος Μουρατίδης Mar 14 '18 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.