3

I want to use grep to search pattern from one file in a second one. My pattern file is something like:

K02217
K07448
KO8980

The file to search in is:

>aai:AARI_24510  proP; proline/betaine transporter; K03762 MFS transporter, MHS family, proline/betaine transporter
>aai:AARI_26600  ferritin-like protein; K02217 ferritin [EC:1.16.3.1]
>aai:AARI_28260  hypothetical protein
>aai:AARI_29060  ABC drug resistance transporter, inner membrane subunit; K09686 antibiotic transport system permease protein
>aai:AARI_29070  ABC drug resistance transporter, ATP-binding subunit (EC:3.6.3.-); K09687 antibiotic transport system ATP-binding protein
>aai:AARI_29650  hypothetical protein
>aai:AARI_32480  iron-siderophore ABC transporter ATP-binding subunit (EC:3.6.3.-); K02013 iron complex transport system ATP-binding protein [EC:3.6.3.34]
>aai:AARI_33320  mrr; restriction system protein Mrr; K07448 restriction system protein

The command I tried is:

fgrep --file=pattern.txt file.txt >> output.txt

This prints the lines of file.txt where the pattern is found. I need it to print also a column with the pattern found. So something like:

K07448 mrr; restriction system protein Mrr; K07448 restriction system
K02217 ferritin-like protein; K02217 ferritin [EC:1.16.3.1]

Anyone can suggest me how to do?

5
  • 1
    Actually my case is a bit different, because the patterns in the file pattern.txt are repeated more times in file.txt. I want it to search recursively and pronti g all the lines where the pattern is found... Nov 17, 2014 at 0:08
  • Then I can't search for the pattern in a specific column, because the position in file.txt is random... Nov 17, 2014 at 0:14
  • Can a single line match more than one pattern?
    – terdon
    Nov 17, 2014 at 0:27
  • No, each line of file.txt match only one of pattern.txt Nov 17, 2014 at 0:29
  • I just closed and re-opened since it already had three wrong votes to close as a duplicate. It is not. At least not of the proposed one.
    – terdon
    Nov 17, 2014 at 13:02

3 Answers 3

3

If you don't mind an extra column with a number in it you can use join and grep to do this.

$ join <(grep -of patterns.txt file.txt | nl) \
       <(grep -f patterns.txt file.txt | nl)
1 KO3322 proteinaseK (KO3322)
2 KO3435 Xxxxx KO3435;folding factor
3 KO3435 Yyyyy KO3435,xxxx
2
  • Nice. And of course we can strip off that first field using awk or sed, if desired.
    – PM 2Ring
    Nov 17, 2014 at 5:17
  • @PM2Ring - correct, didn't want to muddle up the A any more. OP or whomever can take this from here and use as needed.
    – slm
    Nov 17, 2014 at 5:56
1

You can use a shell loop:

$ while read pat; do 
    grep "$pat" file | 
        while read match do 
            echo -e "$pat\t$match"
        done
 done < patterns 
KO3435  Xxxxx KO3435;folding factor
KO3435  Yyyyy KO3435,xxxx
KO3322  proteinaseK (KO3322)

I tested by running this on the UniProt flat file for human (625M) and using 1000 UniProt IDs as the patterns. It took ~6 minutes on my Pentium i7 laptop. It took ~35 seconds when I only looked for 100 patterns.


As pointed out in the comments below, you can make this slightly faster by skipping the echo and using grep's --label and -H options:

$ while read pat; do 
    grep "$pat" --label="$pat" -H < file
done < patterns

Running this on your example files produces:

$ while read pat; do 
    grep "$pat" --label="$pat" -H < kegg.annotations; 
  done < allKO.IDs.txt > test1
terdon@oregano foo $ cat test1 
K02217:>aai:AARI_26600  ferritin-like protein; K02217 ferritin [EC:1.16.3.1]
K07448:>aai:AARI_33320  mrr; restriction system protein Mrr; K07448 restriction system protein
16
  • Possible problem with this is that you get duplicate lines if they contain two different patterns. Or this is possibly an improvement; I can not know what OP wants. Nov 17, 2014 at 1:41
  • 2
    @CameronNemo good point, which is why I asked the OP who said that each line can only match one pattern.
    – terdon
    Nov 17, 2014 at 1:43
  • 2
    @terdon - I wonder if this would be any faster (using --label): while read pat; do < file grep "$pat" --label="$pat" -H; done < patterns (output is a bit different but still along the same lines). Nov 17, 2014 at 1:54
  • @don_crissti looks like that cuts about 1 second from my test with 100 patterns. Nice! I'd never noticed the --label option before. Shamelessly stolen and added to my answer.
    – terdon
    Nov 17, 2014 at 1:59
  • I tried terdon solution as follows: while read allKO.IDs.txt; do grep "$pat" --label="$pat" -H < kegg.annotations; done > test1 but I had this error allKO.IDs.txt': not a valid identifier Am I doing something wrong? Nov 17, 2014 at 10:23
1

You can use ack:

$ ack "$(tr '\n' '|' < pattern.txt | sed -e 's/.$//')" --print0 --output='$& $_' file.txt
KO3322 proteinaseK (KO3322)
KO3435 Xxxxx KO3435;folding factor
KO3435 Yyyyy KO3435,xxxx
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .