Comparing text files using Bash and AWK

Question

I have two text files, and I want to compare their corresponding values according to their rows and columns. By comparing, I mean to check if the values are equal and echo if the values are the same or not. Here are the files:

file1.txt

Name  Col1  Col2  Col3  
-----------------------
row1  1     4     7        
row2  2     5     8         
row3  3     6     9   

file2.txt

Name  Col1  Col2  Col3  
-----------------------
row2  1     4     11        
row1  2     5     12

Here are the constraints:

• compare only the rows that exist (that is, since row3 exists in file1.txt, but not in file2.txt, no comparison is needed)
• Must use AWK
• The rows may be out of order in the files
• No double arrays since my version of AWK doesn't support it

I am thinking of something like this:

awk 'NR>2 {next}
{
    for (i=2;i<NR;i++)              #For each row of file1.txt
    {     
        for(j=1;i<NF;j++)           #For each column of file1.txt
        {
             // Check if row and column of file1.txt is equal to row and column of file2.txt.
        } 
    }
}
' file1.txt file2.txt

How do you compare values of two different text files? Let me know if more explanation is needed.

All I'm asking for is a general structure using AWK. Hopefully I can figure out the rest.

Answer 1

This one outputs the lines from file2 where the name is in both files but any of the values differ

awk 'NR==FNR {f1[$1]=$0; next} $1 in f1 && $0 != f1[$1]' file1.txt file2.txt 

row2  1     4     11        
row1  2     5     12

I realized that even a difference in whitespace will give false results. We can "normalize" the lines:

awk '
    NR==FNR  {$1=$1; f1[$1]=$0; next} 
    $1 in f1 {$1=$1; if ($0 != f1[$1]) print}
' file1.txt file2.txt 

row2 1 4 11
row1 2 5 12

Answer 2

There are many ways to do it, here's a solution with gawk 4.x:

$ awk '
    FNR < 2 { next }
    FNR == NR {
        for (i = 2; i <= NF; i++) {
            a[$1][i] = $i;
        }
        next;
    }
    ($1 in a) {
      for (i = 2; i <= NF; i++) {
          if (a[$1][i] == $i) {
              printf("%s->Col%d: Equal\n", $1, i-1);
          } else {
              printf("%s->Col%d: Not Equal\n", $1, i-1);
          }
       }
  }
' file1.txt file2.txt
row2->Col1: Not Equal
row2->Col2: Not Equal
row2->Col3: Not Equal
row1->Col1: Not Equal
row1->Col2: Not Equal
row1->Col3: Not Equal

Explanation

• FNR < 2 { next }: skip first two lines
• FNR == NR: only true when we processing first file. We save each column value in associative array a, with form a[ROW][COLUMN].
• ($1 in a): check if row in file2 existed in file1. If true, we loop through all its column value, compare with value in file1.

With older gawk version, you can try:

$ awk '
    FNR < 2 { next }
    FNR == NR {
        for (i = 2; i <= NF; i++) {
            a[i,$1] = $i;
            b[$1];
        }
        next;
    }
    ($1 in b) {
      for (i = 2; i <= NF; i++) {
          if (a[i,$1] == $i) {
              printf("%s->Col%d: Equal\n", $1, i-1);
          } else {
              printf("%s->Col%d: Not Equal\n", $1, i-1);
          }
       }
  }
' file1.txt file2.txt

Answer 3

Try this out inside for inner for loop:

awk ' {
      if(i == j)
           print "Same value";
      else
           print "Not the same";
       }'

This is a simple conditional statement that might help, but it may need some additional tweaking with comparing two different text files...