7

I've been trying to get awk to do some trivial arithmetic, which involves carrying some values from one line to the next.

Here is a minimal example pair, for comparison. The first example is expected behaviour, since 99.16 - 20.85 = 78.31

$ echo -e "0,99.16\n20.85,78.31" | awk -F, '{
  if (NR != 1 && (prior_tot - $1) != $2) {
    print "Arithmetic fail..." $0
  } else {
    print "OK"
  };
  prior_tot = $2
}'

Returns

OK
OK

The second example is not expected behaviour, since 99.15 - 20.85 = 78.30

$ echo -e "0,99.15\n20.85,78.30" | awk -F, '{
  if (NR != 1 && (prior_tot - $1) != $2) {
    print "Arithmetic fail..." $0
  } else {
    print "OK"
  };
  prior_tot = $2
}'

Returns

OK
Arithmetic fail...20.85,78.30

Can anybody explain what is going on here?

7

The floating point numbers 99.15 and 28.85 and 78.30 don't have exact IEEE 754 binary representations. You can see this with a C program that does the same calculation:

#include <stdio.h>
int
main(int ac, char **av)
{
        float a = 99.15;
        float b = 20.85;
        float c;

        printf("a = %.7f\n", a);
        printf("b = %.7f\n", b);
        c = a - b;
        printf("c = %.7f\n", c);

        return 0;
}

I get these answers on by an x86 and an x86_64 machine probably because they both do IEEE 754 floating point math:

a = 99.1500015 b = 20.8500004 c = 78.3000031

Here's what happens: floating point numbers get represented with a sign bit (positive or negative), a number of bits, and an exponent. Not every rational number (which is what a "floating point" number is in this context) can be represented exactly in IEEE 754 format. So, the hardware gets as close as it can. Unfortunately, in your test case, the hardware doesn't get an exact representation of any of the 3 values. It won't even if you use double instead of float, which awk probably does.

Here's a further explanation of the spacing of floating point numbers that have exact binary representations.

You can probably find some values that pass your test and others that don't. There's a lot more that don't.

Usually people solve a floating point problem by doing something like this:

if (abs(c) <= epsilon) {
    // We'll call it equal
} else {
    // Not equal
}

That's a lot harder to do in awk. If you're doing money with monetary units and two significant digits of sub-unit (dollars and cents, say), you should just carry out all calculations in the sub-units (cents in the USA). Do not use floating point to do monetary calculations. You will only find yourself regretting that decision.

  • 1
    awk could also show the numbers: awk 'BEGIN{a=99.15;b=20.85;c=78.30;printf("%22.20f %22.20f %22.20f",a,b,c)}' 99.15000000000000568434 20.85000000000000142109 78.29999999999999715783 Also note that the awk representation is in double, your c code used single (thus the bigger error). For double more than 17 exact digits is not reasonable. – Isaac Aug 17 '18 at 22:35
  • 1
    And there is the fact that IEEE784 rounds to even. Some number simply won't match, no matter how long they are printed. – Isaac Aug 17 '18 at 22:37
7

You're being bitten by a floating point arithmetic issue.

$ awk 'BEGIN { printf "%.17f\n", 99.15-20.85 }'
78.30000000000001137

http://floating-point-gui.de/ might be able to help clear things up for you - it tries to explain what floating point is, and why arithmetic errors like this happen, and how to avoid these sorts of issues in your programs.

5

You can avoid such kind of mistakes by numbers formating:

awk -F, '{
    if (NR != 1 && sprintf(CONVFMT,prior_tot-$1) != $2)
        {print "Arithmetic fail..." $0}
    else
        {print "OK"}
    prior_tot = $2}'
  • It seems that it doesn't try echo -e "0,99.16\n20.85,78.31\n78.31,0\n-99.15,99.15\n20.85,78.30" | awk -F, '{ if (NR != 1 && sprintf(CONVFMT,prior_tot-$1) != $2) {printf("Arithmetic fail...%s %30.30f",$0,prior_tot-$1)} else {print "OK"} prior_tot = $2}' – Isaac Aug 17 '18 at 22:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.