grep and print how many times my pattern in file 1 is present in file2

Question

I have a file1 (list of my pattern) like this:

file1

Fatty_acid_degradation
Aminobenzoate_degradation
Amino_sugar_and_nucleotide_sugar_metabolism
Amoebiasis

and I have a file2 (list of all the patterns).

file2

Fatty_acid_degradation
Fatty_acid_degradation
Fatty_acid_degradation
Bacterial_invasion_of_epithelial_cells
Bacterial_invasion_of_epithelial_cells
Bacterial_invasion_of_epithelial_cells
Bacterial_invasion_of_epithelial_cells

I would like to grep and count how many times each of my patterns in file1 is present in file2 and obtain a table (tab separated) like this:

Fatty_acid_degradation 3

Answer 1

The simplest approach would be to grep each of the patterns and then count them:

$ grep -Fwf file1 file2 | sort | uniq -c
      3 Fatty_acid_degradation

The grep options are -f to give a file as a list of patterns to search for, -F to specify that the pattern should be treated as a string and not a regular expression and -w to ensure that the pattern is matched only against entire words (so that regulation_of_expression is not matched against upregulation_of_excpression for example).

Then, you can use whatever tool you prefer to change the format:

$ grep -Fwf file1 file2 | sort | uniq -c | sed -r 's/.*([0-9]+) *(.*)/\2\t\1/'
$ grep -Fwf file1 file2 | sort | uniq -c | perl -lane 'print "$F[1]\t$F[0]"'
$ grep -Fwf file1 file2 | sort | uniq -c | awk -vOFS="\t" '{print $2,$1}'

All of the above return

Fatty_acid_degradation  3

Answer 2

grep -f file1 file2 | sort | uniq -c

This gives the output in the format:

  3 Fatty_acid_degradation

Can you live with that?

Answer 3

so many quick answer, make me feel awkward ...

awk 'FNR == NR { pat[$1]=0 ; next ; }
{ if ( $0 in pat ) pat[$0]++ ; }
END { for ( p in pat ) if ( pat[p]) printf "%s %d\n",p,pat[p] ;}' f1 f2

where

• FNR == NR { pat[$1]=0 ; next ; } record pattern in pat array
• { if ( $0 in pat ) pat[$0]++ ; } whenever one is matched, count
• END { for ( p in pat ) if ( pat[p]) printf "%s %d\n",p,pat[p] ;} at the end, dump non zero count

Answer 4

You can also try the following solution using Python:

#!/usr/bin/env python2
import collections
with open('file_1') as f1, open('file_2') as f2:
    counts = collections.Counter(f2)
    for line in f1:
        if line in counts:
            print line.rstrip() + '\t' + str(counts[line])

Here we have used the Counter class of collections module which will generate a dictionary containing the occurrences of each element of an iterable.