1

I have a file1.txt and file2.txt i would like to print the matching lines to a new files

file1.txt

FOMPING00002015
FOMPING00008750 
FOMPING00003379 
FOMPING00009073
FOMPING00007164 
FOMPING00009598

file2.txt

>FOMPING00013293 Protein of unknown function
ATGCCCTGCTCGTCGCTCGAGCGGGATCATAGCCAGCATGAAGTTATACCGTCATCGCAG
AGCCAGGAACGCGACTTTGTGCCGCCTAATGGTGACATCAGGAGTCGGGCGAGAACGACA
TCCGACGAAATTGTACCCACATCGCAG
>FOMPING00003379 Protein of unknown function
ATGCCCTGCTCGTCGCTCGAGCGGGATCATAGCCAGCATGAAGTTATACCGTCATCGCAG
AGCCAGGAACGCGACTTTGTGCCGCCTAATGGTGACATCAGGAGTCGGGCGAGAACGACA
TCCGACGAAATTGTACCCACATCGCAGTA
>FOMPING00009073 Protein of unknown function 
ATGTCCTCTTGGTCTGGTTCTTCTTACCCTCCACCTCCACGCGCACGTTCGCGCTCTCGC
TCCCCTTATCGTGGGTCTTATCCTGCGAGACCCGGGTATCCAGAGCCTGGATACTCGCAG
>FOMPING00000581 Similar to mcs4: Response regulator mcs4  
ATGTCCTCTTGGTCTGGTTCTTCTTACCCTCCACCTCCACGCGCACGTTCGCGCTCTCGC
TCCCCTTATCGTGGGTCTTATCCTGCGAGACCCGGGTATCCAGAGCCTGGATACTCGCAG
GATCCATACCGTGCCGACTGGGAGGCTTATGACAGAGAGCGCGCATGGGCCTCCTACGAG

I have tried few commands

grep -F file1.txt file2.txt > output.txt
grep -Ff file1.txt file2.txt > output.txt

both the commands outputs just the first line from file2.txt

output.txt

>FOMPING00013293 Protein of unknown function
>FOMPING00000581 Similar to mcs4: Response regulator mcs4.

I want the output file just like the file2.txt with sequences in it.

Thank you

  • Diff can print out X lines after match with -A X. Is the number of lines after each match constant? If not, you'll have to parse file2.txt and match "headers" with their "sequence" lines. But if it's always just one line, you can do -A 1. – Daniel Farrell Oct 24 '19 at 18:52
  • Thank you. number of lines varies after each match. – kapr0001 Oct 24 '19 at 19:14
  • Does headers in file2 literally start with >? – George Vasiliou Oct 24 '19 at 19:27
  • Yes every block starts with ">" – kapr0001 Oct 24 '19 at 19:28
  • 1
    Of the three you list, awk is what you need. But if I were doing this and similar things with arbitrary, "inconveniently" structured input data (briefly looking at your question history), I would skip awk and shoot straight to my favorite programming language. awk is a very powerful tool but also antiquated. In the modern age you'll (probably) enjoy modeling your data in something like golang/ruby/python/etc compared to awk. I cannot tell you how to do this in awk. – Daniel Farrell Oct 24 '19 at 19:31
2

This seems to work ok in my tests. The trick is to use ">" as record / block separator.

awk 'NR==FNR{a[$0];next};$1 in a{print ">" $0}' file1.txt RS=">" file2.txt
#or alternativelly, due to the whitespace present in the end of each line of file1.txt
awk 'NR==FNR{a[$1];next};$1 in a{print ">" $0}' file1.txt RS=">" file2.txt

The position of RS in the end of awk, affects the file that comes after RS. In my command, file1 is parsed with default RS="\n" , but file2 is parsed with RS=">".

| improve this answer | |
  • thank you, but the output has only one from the test file (>FOMPING00009073) but not the other one – kapr0001 Oct 24 '19 at 20:51
  • 1
    @kapr0001 Your file1.txt has some space characters at the end of some lines. It should work if you remove these. – Freddy Oct 24 '19 at 20:54
  • @Freddy i spent 15 minutes to realize that .... my solution was not working in the beginning due to this fact. Maybe is safer to use {a[$1];next} in the first part to be sure that white space at the end will be ignored. – George Vasiliou Oct 24 '19 at 20:59
  • Yes, I realized the same (a[$1]). Btw., I upvoted your solution, very creative! – Freddy Oct 24 '19 at 21:03
  • I couldn't realize there were white spaces in file one. thank you – kapr0001 Oct 25 '19 at 8:35
1

Using awk with two input field separators > and the space character:

awk -F'[> ]' '{
  if (NR==FNR){
    a[$1]
  }
  else {
    if (substr($0,0,1) == ">"){
      printline=($2 in a)
    }
    if (printline){
      print
    }
  }
}' file1.txt file2.txt

When the first file is processed, store the first field in an array.
When the second file is processed, test if the current line begins with > and set a flag printline testing if the second field is present in the array. Print the current line if the flag is set.

| improve this answer | |
  • It working on the test files, but i dont know for some reason its not working on my original files, will try to figure it out – kapr0001 Oct 24 '19 at 20:52
  • it worked great. thank you – kapr0001 Oct 24 '19 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.