1

I have two files which I need to compare by some fields:

reference file:

42:B:0
43:A:1
44:A:1
45:A:1

target file:

42:!:1
43:B:0
44:A:1
45:B:2

What I need I can already achieve with combined 'while' loop + awk:

$ cat reference|while IFS=: read a b c;do awk -F: -va=$a -vb=$b -vc=$c '$1==a{if($2!=b){if($3>c)if($2!="!"){ print a":target has bigger $3("$3">"c") and $2 different ("$2")" } else { print a":target has bigger $3("$3">"c") but $2 disabled ("$2")" }}}' target;done
42:target has bigger $3(1>0) but $2 disabled (!)
45:target has bigger $3(2>1) and $2 different (B)

how could I rid of the 'while' loop and process both files in awk directly?

1
awk -F: '
    FNR == NR { c2[$1] = $2; c3[$1] = $3; next }
    !($1 in c2) {
        printf("%d: $1 not found in reference\n", $1)
        next
    }
    $3 > c3[$1] && $2 == "!"    {
        printf("%d: target has bigger $3 (%d>%d) but disabled $2 (%s)\n", $1,$3,c3[$1],$2)
        next
    }
    $3 > c3[$1] && $2 != c2[$1] {
        printf("%d: target has bigger $3 (%d>%d) but different $2 (%s)\n", $1,$3,c3[$1],$2)
    }' reference target

This reads the reference file and then the target file.

When reading the reference file (FNR == NR), it collects the values of the second and third column in the two arrays c2 and c3. The index used is the value in the first column.

When reading the target file (FNR != NR), it will compare the value in the third column against the saved value in the c3 array. It then also compares the the second column against ! or against the saved second column from the reference file in c2.

The code will also complain with an additional message if the target file's first column is not found in the reference file.

The output, given the data in the question with an added target line (56:C:9):

42: target has bigger $3 (1>0) but disabled $2 (!)
45: target has bigger $3 (2>1) but different $2 (B)
56: $1 not found in reference
1

paste + awk magic:

paste -d':' reference target | awk -F':' \
'$1 == $4 && $2 != $5 && $6 > $3{ 
     if ($5 == "!"){ p = "but"; state = "disabled" }
     else { p = "and"; state = "different" }
     printf "%s:target has bigger $3(%d > %d) %s $2 %s (%s)\n", $1, $6, $3, p, state, $5 
}'

The output:

42:target has bigger $3(1 > 0) but $2 disabled (!)
45:target has bigger $3(2 > 1) and $2 different (B)

Bonus gawk solution (considering that the 1st field value are ordered and unique):

awk -F':' \
'NR == FNR{ 
    a[NR][1] = $1; a[NR][2] = $2; a[NR][3] = $3; next 
}
$1 == a[FNR][1] && $2 != a[FNR][2] && $3 > a[FNR][3]{ 
   if ($2 == "!"){ p = "but"; state = "disabled" }
   else { p = "and"; state = "different" }
   printf "%s:target has bigger $3(%d > %d) %s $2 %s (%s)\n", $1, $3, a[FNR][3], p, state, $2 
}' reference target
  • I don't know about the input data, but it would fail to pick up something that is "disabled" in both reference and target. It would also require the two files to be of the same lengths and sorted the same way. These things are not mentioned by the user, so it may not be an issue. – Kusalananda Mar 28 '18 at 7:03
  • @Kusalananda, that's your subjective. It's obvious that both files should correlate by record numbers (the OP said What I need I can already achieve), it's obvious from the OP's effort. As you said I don't know about the input data - that's right, you don't know – RomanPerekhrest Mar 28 '18 at 7:04
  • I thank you both of you, reference file can have more records than target, all other which exist in target but not in reference should be ignored. – DonJ Mar 28 '18 at 8:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.