grep match fields for patten in two files

Question

Is it possible to match fields from two different files using grep? for instance, if I have a file1. i want to match the field POS from the two files if there is a macth i output only the lines that match like

1   10366   rs58108140  G   A 
1  51954  rs185832753 G C 

file 1

#CHROM  POS ID  REF ALT QUAL
1   10366   rs58108140  G   A   
1   10611   rs189107123 C   G   
1   51954   rs185832753 G   C   
1   13327   rs144762171 G   C   
1   13957   .   TC  T   28  
1   13980   rs151276478 T   C   

file 2

#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51935   rs181754315 C   T   
1   51954   rs185832753 G   C   
1   52058   rs62637813  G   C   
1   52144   rs190291950 T   A   
1   52238   rs150021059 T   G   
1   54353   rs140052487 C   A

Answer 1

You can't do this in grep, no. While you can use a file with patterns to search for as input to grep, you cannot specify a specific field in that file. So if you want to only match the 2nd positions, you would need something like this:

$ awk  '{ if(NR==FNR){a[$2]=$0}else{if($2 in a){print}}}' file1.vcf file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

Of course, this will also print cases where the position is the same but the variant is on another chromosome. That's what your question asked for, but it might make more sense to look for unique combinations of chr and pos:

$ awk  '{ if(NR==FNR){a[$1$2]=$0}else{if($1$2 in a){print}}}' file1.vcf file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

---

To do it with grep, you would need to first extract the POS field into a new file:

awk '{print $2}' file1.vcf > file3
$ cat file3
POS
10366
10611
51954
13327
13957
13980

Now, use this file with grep:

$ grep -f file3 file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

But this really isn't a good idea. For example, it would also find a match for 1103668since10366is a substring of1103668`.

---

Finally, assuming these are two valid vcf files (which your examples are not), you can use bedtools intersect. But that's not really on topic here.

Answer 2

Use grep:

$ grep -Fof file1 file2
    #CHROM  POS ID  REF ALT QUAL
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

• -F use fixed strings instead of regex
• -o only print matching
• -f file1 search for all patterns/fixed strings, one for each line of file1
• file2 the file in which to search these

Alternative using awk:

awk '
    NR==FNR{s[$0]=1}
    NR!=FNR && s[$0]
' file1 file2

Answer 3

If any part of the "matching" line (including the amount of space between fields) can change between the two files, then join would be a better tool for the job.

sort -n -k 2 < file1 > sort1
sort -n -k 2 < file2 > sort2
join --header -1 2 -2 2 -o '1.1 1.2 1.3 1.4 1.5'  sort1 sort2

The sort steps are necessary because join requires its input to have been sorted (on the common field).

Result:

#CHROM POS ID REF ALT
1 10366 rs58108140 G A
1 51954 rs185832753 G C