I have 96 *fastqc.gz raw read files from 24 samples. Each sample was sequenced on two lanees for each pair.

I would like to merge reads for each pair from both lanes into one output file with same name identifier from sample file name (2271_merged_R1_001.fastq.gz).

File names are in this order:



I tried the following short script but only two output files are being generated (first and the last).

FOR R1 files

  for rf in 22[71-94]*R1_001.fastq.gz; do zcat $rf > 22"${71-94}"_merged_R1_001.fastq.gz ; done

FOR R2 files

for rf in 22[71-94]*R2_001.fastq.gz; do zcat $rf > 22"${71-94}"_merged_R2_001.fastq.gz ; done

My Questions are: 1. Why only two output files are generated? 2. The number of reads in the out put files are not the sum of the merged files from both lanes. 3. Is there a nice way, I could do the merging of reads from both lanes for both (R1 and R2) in single step instead of running it two times for each read type.

What went wrong in the code? and how could I verify that the output files are completely merged?


  • Are there literal asterisk characters in your file names, or were you trying to do bold formatting in the code block? Also, what are the filenames of your two output files? – cunninghamp3 Apr 25 '18 at 15:40

First, 22[71-94]*R1_001.fastq.gz does not expand to what you think it expands to

This is effectively 22[1-9]*R1_001.fastq.gz - in this, [71-94] is a character grouping where "7 OR 1 to 9 OR 4" simplifies to "1 to 9".

22{71..94}*R1_001.fastq.gz is most likely the expansion you were looking for, but your loop will perform zcat once for each file found, not concatenate things together. Effectively, every R1 file sequentially gets zcat-ed into the same outfile, overwriting the previous writes.

I believe this is what you're actually looking for given an assumption(see below):

for num in {71..94}; do zcat 22"$num"*R1_{L001,L002}.fastq.gz > "22${num}_merged_R1_001.fastq" ; done

Assumption: You want the result to be decompressed, thus I removed the .gz from the resulting files (as pointed out by Patrick)
If this is not the case, simply change zcat to cat and append .gz to the resulting file again.

  • Your solution is zcating the files, so they're uncompressed, and then storing them in a file with the extension .gz. Either it should not be .gz, or you should recompress. Note however if your end result is compressed, there's no reason to zcat at all, just use cat and avoid the decompress/compression steps. – Patrick Apr 25 '18 at 16:58
  • @Patrick True, I've removed the .gz extension from the resulting files, as that was simply duplicating what the intended filenames were in the OP's post. – cunninghamp3 Apr 25 '18 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.