I read Comparing two files using Unix and Awk. It is really interesting. I read and tested it, but I can't understand it completely and use it in other cases.

I have two files. file1 has one field and the other one has 16 fields. I want to read elements of file1 and compare them with 3rd field of file2. If there was a match for each element, I sum value of field 5 in file2. As an example:

file 1


file 2

2 2 2 1 2
3 6 1 2 4 
4 1 1 2 3
6 3 3 3 4 

For element 1 in file1 I want to add values in field 5 of file2 where value of field 3 is 1. And do the same for element 2 and 3 in file1. The output for 1 is (3+4=7) and for 2 is 2 and for 3 is 4.

I don't know how I should write it with awk.


Here's one way. I have written it as an awk script so I can add comments:

#!/usr/local/bin/awk -f

    ## FNR is the line number of the current file, NR is the number of 
    ## lines that have been processed. If you only give one file to
    ## awk, FNR will always equal NR. If you give more than one file,
    ## FNR will go back to 1 when the next file is reached but NR
    ## will continue incrementing. Therefore, NR == FNR only while
    ## the first file is being processed.
    if(NR == FNR){
      ## If this is the first file, save the values of $1
      ## in the array n.
      n[$1] = 0
    ## If we have moved on to the 2nd file
      ## If the 3rd field of the second file exists in
      ## the first file.
      if($3 in n){
        ## Add the value of the 5th field to the corresponding value
        ## of the n array.
## The END{} block is executed after all files have been processed.
## This is useful since you may have more than one line whose 3rd
## field was specified in the first file so you don't want to print
## as you process the files.
    ## For each element in the n array
    for (i in n){
    ## print the element itself and then its value
    print i,":",n[i];

You can save that as a file, make it executable and run it like so:

$ chmod a+x foo.awk
$ ./foo.awk file1 file2
1 : 7
2 : 2
3 : 4

Or, you can condense it into a one-liner:

awk '
     (NR == FNR){n[$1] = 0; next}
     {if($3 in n){n[$3]+=$5}}
     END{for (i in n){print i,":",n[i]} }' file1 file2
awk '
  NR == FNR {n[$3] += $5; next}
  {print $1 ": " n[$1]}' file2 file1
  • It does some extra work by summing non matched fields.
    – Emmanuel
    Dec 26 '13 at 12:52
  • @Emmanuel, that's still one awk instructions per line of file2, which makes it shorter and faster than terdon's Dec 26 '13 at 16:28
  • brillant solution ! Aug 4 '15 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.