I have 2 files. If field 1, 2, 4 and 5 of both file1 and file2 matches, then I want to print the whole line of file1 and file2 one after another in my output file.


sc2/80         20      .        A       T         86   F=5;U=4
sc2/60         55      .        G       T         76   F=5;U=4 
sc2/68         20      .        T       C         71   F=5;U=4
sc2/24         24      .        T       G         31   F=5;U=4


sc2/99         84      .        C       G         61   F=5;U=4
sc2/80         20      .        A       T         30   F=5;U=4
sc2/60         40      .        G       T         76   F=5;U=4 
sc2/30         20      .        T       C         71   F=5;U=4
sc2/24         24      .        T       G         91   F=5;U=4

Expected Output:

sc2/80         20      .        A       T         86       F=5;U=4
sc2/80         20      .        A       T         30       F=5;U=4
sc2/24         24      .        T       G         31       F=5;U=4
sc2/24         24      .        T       G         91       F=5;U=4

I am new in the field and I appreciate your help.

4 Answers 4


You can use a multidimensional array:

awk 'FNR==NR{a[$1,$2,$4,$5]=$0;next}{if(b=a[$1,$2,$4,$5]){print b;print}}' file1 file2

FNR (file number of record) is equal to NR when awk is processing the first file.

a[$1,$2]=$0 is the same as a[$1 SUBSEP $2]=$0 or a[$1"\034"$2]=$0, and ($1,$2)in a is the same as ($1 SUBSEP $2)in a or ($1"\034"$2)in a.

You could also replace if(b=a[$1,$2,$4,$5]){print b;print} with if(($1,$2,$4,$5)in a){print a[$1,$2,$4,$5];print}. If !("index" in a), a["index"] is like a["index"]="".

  • As I understand, in the second file you check if(b=a[$1,$2,$4,$5]) but where is b defined? Is that an implicit definition?
    – mike
    Aug 27, 2013 at 11:59
  • = is an assignment, not a comparison.
    – Lri
    Aug 27, 2013 at 12:12
  • Okay, then I don't understand it. Could you please elaborate on the control flow?
    – mike
    Aug 27, 2013 at 16:12
  • If a has an index like $1"\034"$2"\034"$4"\034"$5, a[$1,$2,$4,$5] returns the value of the element at that index. Otherwise a[$1,$2,$4,$5] adds a new element at that index whose value is an empty string, and b is assigned to an empty string, which evaluates to false.
    – Lri
    Aug 28, 2013 at 4:16

I know you said you didn't want a Perl or Python solution but it might be useful to someone else (and you really really should learn one of those languages if you are doing bioinformatics).

perl -ane '$f=$F[0].$F[1]; print "$k{$f}$_" if $k{$f}; $k{$f}=$_;' file1 file2 


The -a option will cause Perl to split input into the @F array, -n means read input files line by line and -e means "run the script I give on the command line".

So, $f is set to the concatenation of the first ($F[0]) and second ($F[1]) fields. $k{$f}=$_ means save the current line ($_) as the value in a hash (associative arrays in Perl) called k with the key $f. As we read through the files, print the current line and the value of $k{$f} if that value exists. In other words, if we have already seen a line that has the same two first fields, print that line and the current one.


You want to print the intersection of the two files without reordering them (so it's not a set intersection)? I would look up string similarity algorithms and treat each line as a letter. You'll need to modify the algorithm to keep track which letters (lines) are the same and which differ. The big problem is that order is significant but position is not. Also, you may find it easier to massage the data by rewriting it to drop the fields you don't care about. (Or to write a comparison function that ignores these fields.)

Have you considered python or perl? I've heard they're popular in the bioinformatics field. And this really seems like a programming task.

  • True, I need the intersection of two files based on 4 files. Even if the other fields doesnot match, I don't care. I would prefer Awk and unix.
    – Namrata
    Aug 27, 2013 at 10:19

If you can guarantee that each file is supposed to have unique entries. Sort the files with sort -u concatenate the files. Sort again without the -u and then search for duplicate entries.

I would write the little script, but I can't do it out of my head. But it shouldn't be to hard given my approach.

I have a console in front of me now. Here you go:

rm -rf all; sort -u file1 > all; sort -u file2 >> all
sort all | uniq --all-repeated=separate -w 32

If file1 file2 are already sorted and do not contain duplicate entries you can use this command:

sort -m file1 file2 | uniq --all-repeated=separate -w 32

Oh, looks like this is not quite what you asked, since I compare whole lines. Maybe someone else will find this useful though.

  • I want to match only 4 fields in both the files. If they match they are redirected to output. About other fields I dont care. They can be different.
    – Namrata
    Aug 27, 2013 at 14:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .