Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am looking for a way to tell awk to do high-precision arithmetic in a substitution operation. This involves, reading a field from a file and substituting it with a 1% increment on that value. However, I am losing precision there. Here is a simplified reproduction of the problem:

 $ echo 0.4970436865354813 | awk '{gsub($1, $1*1.1)}; {print}'
   0.546748

Here, I have a 16 digit after decimal precision but awk gives only six. Using printf, I am getting the same result:

$ echo 0.4970436865354813 | awk '{gsub($1, $1*1.1)}; {printf("%.16G\n", $1)}'
0.546748

Any suggestions on to how to get the desired precision?

share|improve this question
    
Perhaps awk has higher resolution but it's just your output formatting is truncating. Use printf. –  dubiousjim Nov 28 '12 at 15:25
    
No changes in result value after using printf. Question edited accordingly. –  Ketan Nov 28 '12 at 15:32
    
As @manatwork has pointed out, that gsub is unnecessary. The problem is gsub works on strings, not numbers, so a conversion is done first using CONVFMT, and the default value for that is %.6g. –  jw013 Nov 28 '12 at 15:46
    
@jw013, As I mentioned in the question, my original problem requires gsub since I need to substitute a number with a 1% increment. Agreed, in the simplified example, it is not required. –  Ketan Nov 28 '12 at 15:48

1 Answer 1

up vote 6 down vote accepted
$ echo 0.4970436865354813 | awk -v CONVFMT=%.17g '{gsub($1, $1*1.1)}; {print}'
0.54674805518902947

is probably the best you can achieve. Use bc instead for arbitrary precision.

$ echo 'scale=1000; 0.4970436865354813 * 1.1' | bc
.54674805518902943
share|improve this answer
    
Thanks, that solves it. –  Ketan Nov 28 '12 at 15:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.