1

I have three files file1.txt, file2.txt, file3.txt and they are of same format(same number of columns with same data type).

I want to select records from file1.txt which are not present in file2.txt, file3.txt by comparing column 2, column 3 respectively to the output file out.txt. Could you please help me how to do this ?

Sample Input:

file1.txt

abc 1 a f11 f13 f14 
abd 2 b f12 f14 f13  
abe 4 d f13 f16 f12 
acf 6 s f14 f15 f19

file2.txt

 abc 1 a f21 f23 f24 
 abd 1 b f21 f24 f23  
 abe 4 d f24 f26 f22 
 acf 6 s f23 f25 f29

file3.txt

 abc 1 a f31 f33 f34 
 abd 3 b f31 f34 f33  
 acf 6 s f33 f35 f39 
 abe 3 d f34 f36 f32

Desired output

out.txt

 abd 2 b f12 f14 f13 

Except this row, the other rows are present in either of file2.txt, file3.txt

I have found these post useful for working on three files, comparing two files.

3

If you so like awk

awk '
FILENAME != ARGV[3] {
    m[$2,$3] = 1
    next
    }
!(($2,$3) in m)
' file3.txt file2.txt file1.txt > out.txt

As for me much easy

cut -d" " -f 2,3 file2.txt file3.txt | grep -v -f - file1.txt > out.txt
  • There's no not in operator in awk; the code only concatenates $1, $2, and variable not (which is empty). The correct form would be !(($1 $2) in m). Also note that multiple indices should be written as m[$2,$3] and !(($2,$3) in m), so that there's no key clashes. – Janis May 20 '15 at 12:57
  • @Costas Thank you for sharing more knowledge. I was more curious in the second way that how to tweak it to store the output into out.txt. – Prradep May 20 '15 at 16:30
1

Possible solution with awk:

awk 'FILENAME == ARGV[1] {
    m[$2,$3] = $0;
    next;
}
FILENAME == ARGV[2] {
    if (!(($2,$3) in m)) {
        m[$2,$3] = $0;
    }
    next;
}
{
    if (!(($2,$3) in m)) {
        print $0 >"out.txt";
    }
}' file3.txt file2.txt file1.txt

First we read the first file and create array with keys column 2 and 3.
Then we read second file and we check if key from column 2 and 3 exists in first array, if it not exists we add it to array.
Finally we read the first file, check for existing key in array and if it not exists that is the line we are interested in.

out.txt should contain line(s) in the first file which is not presented in other two files:

abd 2 b f12 f14 f13
  • 1
    In awk there's a safer syntax if you want to compare two keys; multiple indices should be written as m[$2,$3] and !(($2,$3) in m), so that there's no key clashes. – Janis May 20 '15 at 12:58
  • @Janis thanks, but I wanted to explain it better to the OP how it could work. – taliezin May 20 '15 at 13:00
  • I hope the "key clash" argument was clear? (With concatenation of keys, e.g. $2="abc" and $3="d", would clash with $2="ab" $3="cd".) If you want to keep the assignment s = $2 $3 one can use s= $2 SUBSEP $3 (which creates the index equivalent to the safe form using commas in ($2,$3) and m[$2,$3]). – Janis May 20 '15 at 13:08
  • @Janis thanks for your wonderful comments and possible issues arising out of different usage of keys. – Prradep May 20 '15 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.