8

I have two files in my current folder (MA502) whose names are -

MA502_TAAGGCGA-TCGCAGG_L001_R1_001.at.fastq
MA502_TAAGGCGA-TCGCAGG_L001_R2_001.at.fastq

I have many such folders - ex MA503, MA504 etc, and I want to loop over those.

I assign my variable names using wild cards -

Forward=*R1*.at.fastq
Reverse=*R2*.at.fastq

I want to process these files in a script, and I want my output to replace .at to .atqt, so that the final name would look like -

MA502_TAAGGCGA-TCGCAGG_L001_R1_001.atqt.fastq
MA502_TAAGGCGA-TCGCAGG_L001_R2_001.atqt.fastq

I tried

awk 'script' $Forward > ${Forward/.at/.atqt}

My final file name looks like -

*R1*.atqt.fastq

instead of my expectation which was

MA502_TAAGGCGA-TCGCAGG_L001_R1_001.atqt.fastq

I've learnt everything by necessity on unix, so I'm not sure how variable names are processed. Any help is appreciated!

11

The command Forward=*R1*.at.fastq sets the variable Forward to the string *R1*.at.fastq (star, capital R, digit 1, star, dot, lowercase A, etc.). Wildcards are only expanded in contexts that allow multiple words; the right-hand size of a variable assignment expects a single word, so no wildcard expansion occurs.

In a command like cat $Forward, the wildcards in the value of Forward are expanded. When a variable is expanded outside double quotes, its value is intepreted as a whitespace-delimited list of wildcard patterns, and if any pattern matches one or more files, it's replaced by the list of files.

In ${Forward/.at/.atqt}, first the variable's value is looked up: *R1*.at.fastq. Then the text substitution is applied to this string, yielding *R1*.atqt.fastq. The result is an unquoted variable expansion so it is interpreted as a wildcard pattern. But *R1*.atqt.fastq doesn't match any file, so it's left unchanged.

To expand the wildcard when setting Forward, you could make it an array.

Forward=(*R1*.at.fastq)

This sets Forward to a 1-element array, the element being the string MA502_TAAGGCGA-TCGCAGG_L001_R1_001.at.fastq. The wildcard pattern is expanded to the list of matches because it's in a context (the parentheses of array assignment) where multiple words are expected.

In bash, $Forward when Forward is an array is equivalent to ${Forward[0]} — referencing an array variable with the same syntax as a scalar variable refers to the first element of the array. So you can leave your awk command unchanged.

  • The script worked, and your explanation comes very easily to me. Thanks a lot! – Ayush Saxena Jul 4 '15 at 2:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.