3

I want to use grep to search pattern from one file in a second one. My pattern file is something like:

K02217
K07448
KO8980

The file to search in is:

>aai:AARI_24510  proP; proline/betaine transporter; K03762 MFS transporter, MHS family, proline/betaine transporter
>aai:AARI_26600  ferritin-like protein; K02217 ferritin [EC:1.16.3.1]
>aai:AARI_28260  hypothetical protein
>aai:AARI_29060  ABC drug resistance transporter, inner membrane subunit; K09686 antibiotic transport system permease protein
>aai:AARI_29070  ABC drug resistance transporter, ATP-binding subunit (EC:3.6.3.-); K09687 antibiotic transport system ATP-binding protein
>aai:AARI_29650  hypothetical protein
>aai:AARI_32480  iron-siderophore ABC transporter ATP-binding subunit (EC:3.6.3.-); K02013 iron complex transport system ATP-binding protein [EC:3.6.3.34]
>aai:AARI_33320  mrr; restriction system protein Mrr; K07448 restriction system protein

The command I tried is:

fgrep --file=pattern.txt file.txt >> output.txt

This prints the lines of file.txt where the pattern is found. I need it to print also a column with the pattern found. So something like:

K07448 mrr; restriction system protein Mrr; K07448 restriction system
K02217 ferritin-like protein; K02217 ferritin [EC:1.16.3.1]

Anyone can suggest me how to do?

  • 1
    Actually my case is a bit different, because the patterns in the file pattern.txt are repeated more times in file.txt. I want it to search recursively and pronti g all the lines where the pattern is found... – Francesca de Filippis Nov 17 '14 at 0:08
  • Then I can't search for the pattern in a specific column, because the position in file.txt is random... – Francesca de Filippis Nov 17 '14 at 0:14
  • Can a single line match more than one pattern? – terdon Nov 17 '14 at 0:27
  • No, each line of file.txt match only one of pattern.txt – Francesca de Filippis Nov 17 '14 at 0:29
  • I just closed and re-opened since it already had three wrong votes to close as a duplicate. It is not. At least not of the proposed one. – terdon Nov 17 '14 at 13:02
1

You can use a shell loop:

$ while read pat; do 
    grep "$pat" file | 
        while read match do 
            echo -e "$pat\t$match"
        done
 done < patterns 
KO3435  Xxxxx KO3435;folding factor
KO3435  Yyyyy KO3435,xxxx
KO3322  proteinaseK (KO3322)

I tested by running this on the UniProt flat file for human (625M) and using 1000 UniProt IDs as the patterns. It took ~6 minutes on my Pentium i7 laptop. It took ~35 seconds when I only looked for 100 patterns.


As pointed out in the comments below, you can make this slightly faster by skipping the echo and using grep's --label and -H options:

$ while read pat; do 
    grep "$pat" --label="$pat" -H < file
done < patterns

Running this on your example files produces:

$ while read pat; do 
    grep "$pat" --label="$pat" -H < kegg.annotations; 
  done < allKO.IDs.txt > test1
terdon@oregano foo $ cat test1 
K02217:>aai:AARI_26600  ferritin-like protein; K02217 ferritin [EC:1.16.3.1]
K07448:>aai:AARI_33320  mrr; restriction system protein Mrr; K07448 restriction system protein
  • Possible problem with this is that you get duplicate lines if they contain two different patterns. Or this is possibly an improvement; I can not know what OP wants. – CameronNemo Nov 17 '14 at 1:41
  • 2
    @CameronNemo good point, which is why I asked the OP who said that each line can only match one pattern. – terdon Nov 17 '14 at 1:43
  • 2
    @terdon - I wonder if this would be any faster (using --label): while read pat; do < file grep "$pat" --label="$pat" -H; done < patterns (output is a bit different but still along the same lines). – don_crissti Nov 17 '14 at 1:54
  • @don_crissti looks like that cuts about 1 second from my test with 100 patterns. Nice! I'd never noticed the --label option before. Shamelessly stolen and added to my answer. – terdon Nov 17 '14 at 1:59
  • I tried terdon solution as follows: while read allKO.IDs.txt; do grep "$pat" --label="$pat" -H < kegg.annotations; done > test1 but I had this error allKO.IDs.txt': not a valid identifier Am I doing something wrong? – Francesca de Filippis Nov 17 '14 at 10:23
3

If you don't mind an extra column with a number in it you can use join and grep to do this.

$ join <(grep -of patterns.txt file.txt | nl) \
       <(grep -f patterns.txt file.txt | nl)
1 KO3322 proteinaseK (KO3322)
2 KO3435 Xxxxx KO3435;folding factor
3 KO3435 Yyyyy KO3435,xxxx
  • Nice. And of course we can strip off that first field using awk or sed, if desired. – PM 2Ring Nov 17 '14 at 5:17
  • @PM2Ring - correct, didn't want to muddle up the A any more. OP or whomever can take this from here and use as needed. – slm Nov 17 '14 at 5:56
1

You can use ack:

$ ack "$(tr '\n' '|' < pattern.txt | sed -e 's/.$//')" --print0 --output='$& $_' file.txt
KO3322 proteinaseK (KO3322)
KO3435 Xxxxx KO3435;folding factor
KO3435 Yyyyy KO3435,xxxx

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.