I have 2 genetic datasets. File1 is a txt file and has chromosome IDs which all also appear in file 2. File 2 is a csv file and is bigger than file 1 and has more unique chromosome IDs. How can I extract only the lines in file 2 which have unique chromosome IDs in comparison to file 1?
For example my data looks like this:
File 1 (chromosome position is actually my 125th column, implied by the ...):
Gene pval ... Chromosome position ID
ACE 0.002 ... 01:3290834_CT_C_1
NOS 0.01 ... 03:3304593_GA_G_1
BRCA 0.004 . ... 06:6265733_GA_G_1
CYP3 0.34 ... 09:9433933_GA_G_1
File 2 (chromosome position is my first column):
Chromosome position ID Gene pval
01:1243933_GA_G_1 ACE 0.002
03:3304593_GA_G_1 NOS 0.01
06:6265733_GA_G_1 BRCA 0.004
09:9433933_GA_G_1 CYP3 0.34
Based only on the chromosome position ID, the output unique line(s) which only appear in file 2 would be:
Chromosome position ID Gene pval
01:1243933_GA_G_1 ACE 0.002
I've tried to get this output with uniq
and sort
but currently I'm only finding examples online which use multiple text files but want the opposite where they take the matching ID lines not the unique ID lines. I have also tried grep
but with these files being large the command is killed.