3

I have a file with many lines, and on each line I have the arguments I want to pass to parallel with a tab separator.

I run this script

cat $hdd_file | grep $ssd | parallel -C '\t' clean_and_destroy

And it works, $hdd_file is the filename, the grep collects the lines which hdds have a certain $ssd as a cache, and then the parallel calls a function which destroys their connection.

Now that I made new partitions to the cleaned ssds, I try to call parallel like this:

cat $hdd_file | grep $ssd | parallel -C '\t' create_new_cache :::+ `seq $partitions_per_ssd`

Which should get the arguments from the pipe and pair them with the numbers given, but it does not.

cat $hdd_file | grep $ssd | parallel -C '\t' create_new_cache ::: {} :::+ `seq $partitions_per_ssd`

I also tried this and it still doesn't work. The {} :::+ are passed as arguments for some reason

1 Answer 1

4

GNU parallel solution:

Sample input.txt (for demonstration):

a   b
c   d
e   f

grep '^[ac]' input.txt will be used to emulate command(or pipeline) acting like input source file


parallel -C '\t' echo :::: <(grep '^[ac]' input.txt) ::: $(seq 1 3)

The output:

a b 1
a b 2
a b 3
c d 1
c d 2
c d 3

  • :::: argfiles - treat argfiles as input source. ::: and :::: can be mixed.

To aggregate elements from each input source - add --xapply option:

parallel -C '\t' --xapply echo :::: <(grep '^[ac]' input.txt) ::: $(seq 1 2)

The output:

a b 1
c d 2
6
  • thanks, that's a step closer to what i want. I actually want this: parallel -C '\t' echo :::: <(grep '^[ac]' input.txt) :::+ $(seq 1 2) a b 1, c d 2 but i get parallel: Error: Cannot open input file :::+': No such file or directory.` Commented Mar 13, 2018 at 14:49
  • @ΘεόφιλοςΜουρατίδης, you need --xapply option. See my update Commented Mar 13, 2018 at 14:58
  • @ΘεόφιλοςΜουρατίδης :::+ is from 20160422. You probably have an older version.
    – Ole Tange
    Commented Mar 14, 2018 at 7:58
  • You can also use '-' as input file: grep '^[ac]' input.txt | parallel -C '\t' echo :::: - :::+ {1..10}
    – Ole Tange
    Commented Mar 14, 2018 at 8:00
  • @OleTange your example doesn't work, I tried it many times with different variations, it gets the :::+ as an argument of the function Commented Mar 14, 2018 at 8:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .