I have three files file1.txt, file2.txt, file3.txt and they are of same format(same number of columns with same data type).

I want to select records from file1.txt which are not present in file2.txt, file3.txt by comparing column 2, column 3 respectively to the output file out.txt. Could you please help me how to do this ?

Sample Input:


abc 1 a f11 f13 f14 
abd 2 b f12 f14 f13  
abe 4 d f13 f16 f12 
acf 6 s f14 f15 f19


 abc 1 a f21 f23 f24 
 abd 1 b f21 f24 f23  
 abe 4 d f24 f26 f22 
 acf 6 s f23 f25 f29


 abc 1 a f31 f33 f34 
 abd 3 b f31 f34 f33  
 acf 6 s f33 f35 f39 
 abe 3 d f34 f36 f32

Desired output


 abd 2 b f12 f14 f13 

Except this row, the other rows are present in either of file2.txt, file3.txt

I have found these post useful for working on three files, comparing two files.

2 Answers 2


If you so like awk

awk '
    m[$2,$3] = 1
!(($2,$3) in m)
' file3.txt file2.txt file1.txt > out.txt

As for me much easy

cut -d" " -f 2,3 file2.txt file3.txt | grep -v -f - file1.txt > out.txt
  • There's no not in operator in awk; the code only concatenates $1, $2, and variable not (which is empty). The correct form would be !(($1 $2) in m). Also note that multiple indices should be written as m[$2,$3] and !(($2,$3) in m), so that there's no key clashes.
    – Janis
    May 20, 2015 at 12:57
  • @Costas Thank you for sharing more knowledge. I was more curious in the second way that how to tweak it to store the output into out.txt.
    – Prradep
    May 20, 2015 at 16:30

Possible solution with awk:

awk 'FILENAME == ARGV[1] {
    m[$2,$3] = $0;
    if (!(($2,$3) in m)) {
        m[$2,$3] = $0;
    if (!(($2,$3) in m)) {
        print $0 >"out.txt";
}' file3.txt file2.txt file1.txt

First we read the first file and create array with keys column 2 and 3.
Then we read second file and we check if key from column 2 and 3 exists in first array, if it not exists we add it to array.
Finally we read the first file, check for existing key in array and if it not exists that is the line we are interested in.

out.txt should contain line(s) in the first file which is not presented in other two files:

abd 2 b f12 f14 f13
  • 1
    In awk there's a safer syntax if you want to compare two keys; multiple indices should be written as m[$2,$3] and !(($2,$3) in m), so that there's no key clashes.
    – Janis
    May 20, 2015 at 12:58
  • @Janis thanks, but I wanted to explain it better to the OP how it could work.
    – taliezin
    May 20, 2015 at 13:00
  • I hope the "key clash" argument was clear? (With concatenation of keys, e.g. $2="abc" and $3="d", would clash with $2="ab" $3="cd".) If you want to keep the assignment s = $2 $3 one can use s= $2 SUBSEP $3 (which creates the index equivalent to the safe form using commas in ($2,$3) and m[$2,$3]).
    – Janis
    May 20, 2015 at 13:08
  • @Janis thanks for your wonderful comments and possible issues arising out of different usage of keys.
    – Prradep
    May 20, 2015 at 16:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .