3

What is the syntax inside for loop if i in [2,4,7] in awk. Also any idea how to achieve to do in below code. Thanks

Input :

$ cat f1
col1,col2,col3,col4,col5,col6,col7
col11,col12,col13,col14,col15,col16,col17
$ cat f2
col1,col2,col03,col4,col5,col06,col7
col11,col12,col13,col14,col015,col16,col17

$   awk -F,  '
   NR==FNR {
     a[FNR][0]=$0        #populate base file records
     for(i=1;i<=NF;i++)
       a[FNR][i]=$i      #populate base file fields
         next
   }
   {
     for(i=1;i<=NF;i++)
       {
       #if(i in [2,4,7])  <***How to chieve this column in list of values*** >
         #continue;
       if($i!=a[FNR][i])
         {
         printf "Line#%d, column:%d is different in two files.\n",FNR,i
         # ***<TODO print record from first file and second file after printing all mismatch columns>***
         }
       }
   }' f1 f2

Expected output:

Line#1, column:3 is different in two files.
Line#1, column:6 is different in two files.
col1,col2,col3,col4,col5,col6,col7
col1,col2,col03,col4,col5,col06,col7>
Line#2, column:5 is different in two files.
col11,col12,col13,col14,col15,col16,col17
col11,col12,col13,col14,col015,col16,col17>
0

3 Answers 3

3

Basically you are doing line by line comparison on two files and on specific columns excluding some columns; that, all you can do with GNU awk for the word-boundaries support \< & \>:

awk -F, -v skip='2,4,7' 'BEGIN{ filetwo=ARGV[1]; ARGV[1]=""; };{
    getline lf2 <filetwo; split(lf2, arr, ",");
    for (i=1; i<=NF; i++) {
        if ( (skip !~ "\\<"i"\\>") && $i!=arr[i] ) {
            print "Line#"FNR, "Column#" i " is different in two files."; mismatch=1; };
    };
}; mismatch { print $0; print lf2; mismatch=0; };' file2 file1

Or in any awk versions:

awk -F, -v skip_cols='2,4,7' '
    BEGIN{ filetwo=ARGV[1]; ARGV[1]=""; split(skip_cols, skip, ","); };{
    getline lf2 <filetwo; split(lf2, arr, ",");
    for (i=1; i<=NF; i++) {
        if ( !(i in skip) && $i!=arr[i] ) {
            print "Line#"FNR, "Column#" i " is different in two files."; mismatch=1; };
    };
}; mismatch { print $0; print lf2; mismatch=0; };' file2 file1

explaining the code:

  • The BEGIN { ... } block:
    this execute at very first and once before awk want to read any input.

    • Using ARGV, filetwo=ARGV[1];:
      read second argument passed to the command (that is file2) and save that into filetwo variable; first argument ARGV[0] is awk itself and the third one ARGV[2] is file1.
    • after we read the parameter's value, with ARGV[1]="" we unset its value, so awk will not found that parameter (file2) for reading.
    • skip="2 4 7";:
      we defined a variable (see Assignment Expressions) skip and set with the columns number we want to ignore the later.
  • getline command- see Using getline into a Variable from a File:
    we are reading a line from the file2 and assign it to variable lf2 (note the above that filetwo variable now contains the name of the second argument we read from ARGV[1])

  • split() function:
    we split the line we read from file2 which is in lf2 variable on comma character , and store in into array called arr.; now every fields of that line addressed by arr[1] (first field), arr[2] (second field), arr[3] (third), etc.

  • Within for-loop statement we checks two things below:

    • The value of variable i that indicates column number is not seen ! ~ within skip variable value (skip !~ "\\<"i"\\>"; \< and \> are word boundaries anchors, GNU awk specific, so i=2 will not match on 22); next
    • checking that value of column from file1 is not equal with the same column of file2 with same indexes: $i!=arr[i]; if those were not same print the mismatched line number FNR and the diff column index i and also set a control variable mismatch=1.
  • mismatch { print ... }: print both lines from file1 followed by line from file2 in lf2 only if mismatch was detected and mismatch variable was set within if statement; and reset the variable mismatch=0 for next line.

0
2

If I understand correctly :

  • you want to do a for loop on all fields : for(i=1;i<=NF;i++) { ... }
  • and inside: you want to SKIP when i is one of 4 values (in awk, "continue" will bypass the rest of the current for loop and go to the next iteration

A simple way: if you want to be able to skip fields, you can do this by using the following technique

BEGIN { skip[2]++; skip[3]++; skip[22]++; skip[23]++ }

....
 for(i=1;i<=NF;i++) {
   if (i in skip) { continue ; rem="Will skip for values defined in skip array indexes" }
   ...

Instead of defining "skip" from a BEGIN section, you could also have a file with the 4 indexes to be skipped (1 on each line), and read that file using the NR==FNR condition, populating the skip array with this, and then when NR!=FNR (when reading the source file) you use the above method to skip those fields.

1
  • 1
    Thank you for your inputs, yes this is the first part I want to achieve. I will check this and try to implement in my code and will update in a while. Thank you Nov 23, 2020 at 15:08
1
$ cat tst.awk
BEGIN {
    FS=","
    split("2,4,7",tmp)
    for (i in tmp) {
        skipFldNrs[tmp[i]]
    }
}
NR==FNR {
    old[FNR] = $0
    next
}
FNR == 1 {
    for (fldNr=1; fldNr<=NF; fldNr++) {
        if ( !(fldNr in skipFldNrs) ) {
            chkFldNrs[++numToChk] = fldNr
        }
    }
}
old[FNR] != $0 {
    split(old[FNR],o)
    for (i=1; i<=numToChk; i++) {
        fldNr = chkFldNrs[i]
        if ( o[fldNr] != $fldNr ) {
            printf "Line#%d, column:%d is different in two files (\"%s\" vs \"%s\").\n", FNR, fldNr, o[fldNr], $fldNr
        }
    }
}

$ awk  -f tst.awk f1 f2
Line#1, column:3 is different in two files ("col3" vs "col03").
Line#1, column:6 is different in two files ("col6" vs "col06").
Line#2, column:5 is different in two files ("col15" vs "col015").

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .