1

I'm very new to scripting and working in Unix/Linux systems, so I appreciate any help. I'm sure this has been answered before but I cannot figure out why none of the other solutions are working for my case.

I have a directory with several gzip-compressed ASCII text files. I need to separate these based on the file name depending on the presence of the strings rb, it, v3 and v4 at the 5th and 6th position of the name.

Here is an example of what my filenames look like within the directory:

M07Zv3REP1_S442_L001_R1_001.fastq.gz  
M07Zv3REP1_S442_L001_R2_001.fastq.gz  
M18ZitREP3_S276_L001_R2_001.fastq.gz  
M10ZrbREP3_S535_L001_R1_001.fastq.gz  
M10ZrbREP3_S535_L001_R2_001.fastq.gz  
M09Zv4REP1_S300_L001_R1_001.fastq.gz  
M09Zv4REP1_S300_L001_R2_001.fastq.gz  

I've tried using a number of different grep options, but they return every file. For example

grep -FLZ "rb" *.fastq.gz

I figure I'll use xargs to move the grepped files to a new directory, but I'm stuck on how to get the correct zipped files to match. It may be the problem is that grep is searching WITHIN each of the fastq files, in which case there may be an "rb" or an "it" in there somewhere so everything is getting returned.

Any help is much appreciated!

1
  • This question is 100% on topic and welcome here, but given the data you are working on, you might also be interested in our sister site, Bioinformatics.
    – terdon
    Commented Oct 31, 2022 at 14:22

1 Answer 1

0

Yes, as you say, the grep command will search inside file names. And since the files are compressed, it will search through them as binary data and if the file is large enough, you're bound to find most random two-character strings there. Note that even if thy were not compressed, you could still find those strings in the fastq quality lines.

Anyway, grep isn't the right tool here since that searches the contents of files and not the file names. All you need in this case is simple shell globbing. If, as is the case in the example you have shown, you can be 100% sure that the presence of one of the strings v3, v4, rb or it is enough to define the file, you can just do:

mkdir v3 v4 rb it
for pat in v3 v4 rb it; do mv -- *"$pat"*gz "$pat"/; done

If you need to also take the specific position into account so that a file with v3 in its name but not in the 5th and 6th position is not counted, you would need something like this:

mkdir v3 v4 rb it
for file in *gz; do 
    pat=$(printf '%s' "$file" | cut -c 5-6)
    mv -- "$file" "$pat"/
done

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .