-1

I have a file with over 10,000 rows:

head samples   
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192170/type/READ_SET_FASTQ/filename/HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192170/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192171/type/READ_SET_FASTQ/filename/HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192171/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192168/type/READ_SET_FASTQ/filename/HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192168/type/READ_SET_FASTQ_PE/filename/HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz.md5
https://genomequebec.mcgill.ca/nanuqMPS/readSetMd5Download/id/192169/type/READ_SET_FASTQ/filename/HI.2613.007.Index_14.ED14_IA2A35-2_310_R1.fastq.gz.md5

I want to print only part of each line that starts with "HI.*"

This is my desired output:

HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz
HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz
3

Using awk

awk -F'/' '$NF ~ /^HI\./{ print $NF }' infile

to remove the .md5 suffix, you could do:

awk -F'(/|.md5)' '$(NF-1) ~ /^HI\./{ print $(NF-1) }' infile
  • in awk, the $0 is referring to the whole line/record and $1, $2, $3, ... are referring to the first, second, third, ... respectively; and $NF referring to the last field and accordingly the $(NF-1) is the second last field.

  • the tild ~ operator in awk treat the right-hand operator as (extended) regular-expression match against the left-hand operand as string string ~ /regular-expression/

The sed solution:

sed 's:.*/\([^/]*\)\.md5$:\1: ; /^HI\./!d' infile
  • this /\([^/]*\)\.md5 matches last slash followed by anything but not a slash that ends with .md5. We take \([^/]*\) (everything between last slash and .md5 as a group match and print just that in replacement part with its back-reference \1.

  • this /^HI\./!d deletes the lines which doesn't start with HI. from the result of previous sed command.

  • we used different sed delimiter : since we have special / character in the input.

1

Try this,

awk -F '/' '$NF ~ /^HI/ {print substr($NF, 1, length($NF)-4)}' file.txt
  • prints the last field if last field starts with HI
  • excludes the last 4 charecters .md5

Output

HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R1.fastq.gz
HI.2613.007.Custom_0022.ED9_SD2A27-1_180_R2.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R1.fastq.gz
HI.2613.007.Custom_0021.ED4_KS1A29-7_338_R2.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R1.fastq.gz
HI.2613.007.Index_18.ED17_MO1A26-7_353_R2.fastq.gz
HI.2613.007.Index_14.ED14_IA2A35-2_310_R1.fastq.gz
0
awk -F"filename/" '{gsub (".md5","");print $2}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.