2

Is it possible to match fields from two different files using grep? for instance, if I have a file1. i want to match the field POS from the two files if there is a macth i output only the lines that match like

1   10366   rs58108140  G   A 
1  51954  rs185832753 G C 

file 1

#CHROM  POS ID  REF ALT QUAL
1   10366   rs58108140  G   A   
1   10611   rs189107123 C   G   
1   51954   rs185832753 G   C   
1   13327   rs144762171 G   C   
1   13957   .   TC  T   28  
1   13980   rs151276478 T   C   

file 2

#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51935   rs181754315 C   T   
1   51954   rs185832753 G   C   
1   52058   rs62637813  G   C   
1   52144   rs190291950 T   A   
1   52238   rs150021059 T   G   
1   54353   rs140052487 C   A
  • Does the entire line (not just field 2) have to match? When they don't match, which input would you like to be shown in the results? – JigglyNaga Sep 25 '19 at 12:57
  • You're probably looking for bcftools. You might want to check out our sister site: Bioinformatics. – terdon Sep 25 '19 at 15:29
0

You can't do this in grep, no. While you can use a file with patterns to search for as input to grep, you cannot specify a specific field in that file. So if you want to only match the 2nd positions, you would need something like this:

$ awk  '{ if(NR==FNR){a[$2]=$0}else{if($2 in a){print}}}' file1.vcf file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

Of course, this will also print cases where the position is the same but the variant is on another chromosome. That's what your question asked for, but it might make more sense to look for unique combinations of chr and pos:

$ awk  '{ if(NR==FNR){a[$1$2]=$0}else{if($1$2 in a){print}}}' file1.vcf file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

To do it with grep, you would need to first extract the POS field into a new file:

awk '{print $2}' file1.vcf > file3
$ cat file3
POS
10366
10611
51954
13327
13957
13980

Now, use this file with grep:

$ grep -f file3 file2.vcf 
#CHROM  POS ID  REF ALT QUAL    
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   

But this really isn't a good idea. For example, it would also find a match for 1103668since10366is a substring of1103668`.


Finally, assuming these are two valid vcf files (which your examples are not), you can use bedtools intersect. But that's not really on topic here.

  • is there a way to work with huge files. my two file a very large 2GB and 6GB which causes my script to hang – user11766958 Sep 26 '19 at 0:23
0

Use grep:

$ grep -Fof file1 file2
    #CHROM  POS ID  REF ALT QUAL
1   10366   rs58108140  G   A   
1   51954   rs185832753 G   C   
  • -F use fixed strings instead of regex
  • -o only print matching
  • -f file1 search for all patterns/fixed strings, one for each line of file1
  • file2 the file in which to search these

Alternative using awk:

awk '
    NR==FNR{s[$0]=1}
    NR!=FNR && s[$0]
' file1 file2
  • i did someting like grep -Fof FamilyVarient.txt merged_Small.vcf > out.txt but it only output the header #CHROM POS ID REF ALT QUAL. i must have done something wrong – user11766958 Sep 25 '19 at 11:51
  • maybe some extra spaces or so – pLumo Sep 25 '19 at 11:52
  • @pLumo What's the idea behind using option -o? It's GNU grep only and doesn't add value it whole lines match. – Philippos Sep 25 '19 at 13:57
  • @user11766958 If there are extra characters in one of the files, try switching the file names grep -Ff merged_Small.vcf FamilyVarient.txt instead. – Philippos Sep 25 '19 at 13:59
  • This will only work if the entire line is the same. The question states that it should print whenever the second field (POS) is the same. – terdon Sep 25 '19 at 15:34
0

If any part of the "matching" line (including the amount of space between fields) can change between the two files, then join would be a better tool for the job.

sort -n -k 2 < file1 > sort1
sort -n -k 2 < file2 > sort2
join --header -1 2 -2 2 -o '1.1 1.2 1.3 1.4 1.5'  sort1 sort2

The sort steps are necessary because join requires its input to have been sorted (on the common field).

Result:

#CHROM POS ID REF ALT
1 10366 rs58108140 G A
1 51954 rs185832753 G C

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.