Extract the pattern of Directory in GNU parallel

Question

I am running a command line software on multiple folder/samples.Each folder has such files *fastq.gz.

Below is an example of a folder.

Sample_EC_only/EC_only_S1_L005_I1_001.fastq.gz Sample_EC_only/EC_only_S1_L005_R1_001.fastq.gz Sample_EC_only/EC_only_S1_L005_R2_001.fastq.gz Sample_EC_only/EC_only_S1_L006_I1_001.fastq.gz Sample_EC_only/EC_only_S1_L006_R1_001.fastq.gz

I am trying to run this using gnu parallel for multiple softwares,but having issues with extracting the "ID" of the folder.

parallel -j $NSLOTS --xapply \
" echo {1}  \
/home/rob2056/software/cellranger-2.2.0/cellranger count --id  = "{basename} {1}"  \
                   --transcriptome=$ref_data \
                   --fastqs={1} \
 " ::: $TMPDIR/FASTQ/Sample* 

I want to extract for e.g. "Sample_EC_only" as a pattern from the folder inside gnu parallel. --fastqs is able to get the path using {1} ,but having issues with --id option.I have tried various options to extract a pattern from the paths in {1} but not working.

The --id parameter needs a pattern extracted from the path in {1} so that it can create a output dir.

Each {1} consists of e.g. (shown below only for one sample)

/tmp/FASTQ/Sample_EC_only

Answer 1

If I understand you correctly, all you are looking for is {1/} instead of {1}. It is the "basename" of the argument. See man parallel_tutorial and the discussion of --rpl where we have that replacement strings are implemented as

     --rpl '{/} s:.*/::'

and The positional replacement strings can also be modified using / etc. So {1/} is like removing all characters upto the final /.

---

You can create your own replacement shorthand strings using --rpl followed by a string which begins with a tag ({/} in the example above), then a perl expression, such as the substitute command above (s:pattern:replacement:).

I'm not sure what is allowed as tags, but we can use the tutorial example {..} for a positional tag, i.e. that can be used with {number}. The perl expression to remove everything upto the last / followed by the word "Sample_" woudl be: s:.*/Sample_:: so you need to add before --xapply the arguments

--rpl '{..} s:.*/Sample_::'

and then use --id={1..} to apply this replacement to arg 1. If, for example, you want to remove the word upto the first underline _, rather than a fixed word Sample, you can use a pattern such as

--rpl '{..} s:.*/[^_]*_::'

---

The final command should look something like this:

parallel -j $NSLOTS --rpl '{..} s:.*/Sample_::' --xapply \
" echo {1}  \
/home/rob2056/software/cellranger-2.2.0/cellranger count --id={1/} \
                   --id2={1..} \
           --transcriptome=$ref_data \
           --fastqs={1} \
 " ::: $TMPDIR/FASTQ/Sample*