Precision Problem

[penney]

Please help me understand what's going on:

I have the following code and it does not appear to do rounding as I would expect it to:

  double val=323.700000,rval;
  double t=26.975;
  int i=12;

  rval = val / (double)i;
  printf("t           = [%6.2f]\n",t);
  printf("t   NF    = [%f]\n",t);
  printf("rval NF  = [%f]\n",rval);
  printf("rval WF = [%6.2f]\n",rval);

Here's the output that I receive:
t = [ 26.98]
t NF = [26.975000]
lmp NF = [26.975000]
lmp WF = [ 26.97]

Why doesn't rval WF round the number to 26.98???

[Codeplug]

What every computer scientist should know about floating-point arithmetic

The value, as "seen by the CPU", is almost 26.975.

gg

[Stack Overflow]

Hi penney,

I've ran into these problems before, but I have found a simple way to accomplish this. I'll display and example using the math library "math.h"

#include <math.h>
#include <stdlib.h>	

double roundNum(double);

// Example
int main() {
	printf("%6.2f\n", roundNum(55.634592) );

	return 0;
}

// Rounding code
double roundNum(double val) {
	return (val > 0.0) ? floor(val + 0.5) : ceil(val - 0.5);
}

Using a new function called roundNum()

Now roundNum consists of using floor and ceil, probably guessing floor rounds down, and ceil(ing) rounds up

Back on topic, here is a link that might help you learn more about these math functions: floor() and ceil()

Also the way roundNum was written and implemented was to use the ternary operator "? :", example:

expr1 ? expr2 : expr3

expr1 is evaluated first, and if it is non-zero (true), then expr2 is evaluated, and that is the value of the conditional expression, otherwise expr3 is evaluated.

Another example of somthing like z = max(a, b);, would look like:

z = (a > b) ? a : b;

More information can be learned by searching for the ternary operator and a few other ways.

Edit: Now for something like your code, I'd do something like:

printf("t = [%6.2f]\n", roundNum(t) );
printf("t NF = [%f]\n", roundNum(t) );
printf("rval NF = [%f]\n", roundNum(rva) l);
printf("rval WF = [%6.2f]\n", roundNum(rval) );

That's just an example, of course only round the numbers you want rounded and so forth...


Hope this helps,
Stack Overflow

[penney]

Thank you Stack Overflow and CodePlug for your replies. I wasn't so much interested in a "fix" for the problem as much as I was a good explanation of the problem itself. I can easily fix the problem by just declaring the variables as float instead of double so that the precision is not so high. I guess I don't understand why the internal representation of this math would result in a number very near to the calculation but not exact when the calculation is not a repeating sequence of digits after decimal point. The answer is 26.97500000000 no matter how many digits it's carried out. I could understand an issue with precision if the calculation was something like 323.69 / 12. Can you provide any more details as to internally why the CPU is coming up with something like 26.7499999999999999999999?

[Stack Overflow]

Hi penney,

I'm not sure about the CPU question or anything but what I did just learn was something like this:

Floating Point Information
#define FLT_RADIX 2 // radix of exponent representation
#define FLT_DIG 6 // decimal digits of precision
#define FLT_MAX 1E+37 // maximum floating-point number
#define FLT_MIN 1E-37
#define DBL_DIG 10 // decimal digits of precision

I'm not sure this will help any, most of this I found in <float.h> or <limits.h>, but it may very well have to do with the CPU too. I know for myself I usually have like a 6 or 10 digit max after the decimal so that might be it, other than that for anyone else that may know about the floating-point feel free to explain.


Stack Overflow

[Codeplug]

>> I wasn't so much interested in a "fix" for the problem as much as I was a good explanation

Here's a better link to the paper which contains an explanation (amoung other things):
http://www.validlab.com/goldberg/paper.pdf

In short (and from the above paper):

Squeezing infinitely many real numbers into a finite number of bits requires an approximate representation. Although there are infinitely many integers, in most programs the result of integer computation can be stored in 32 bits. In contrast, given any fixed number of bits, most calculations with real numbers will produce quantities that cannot be exactly represented using that many bits. Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation. The rounding error is the characteristic feature of floating-point computation.

gg

[Jez]

I guess I don't understand why the internal representation of this math would result in a number very near to the calculation but not exact when the calculation is not a repeating sequence of digits after decimal point. The answer is 26.97500000000 no matter how many digits it's carried out. I could understand an issue with precision if the calculation was something like 323.69 / 12. Can you provide any more details as to internally why the CPU is coming up with something like 26.7499999999999999999999?

Ah, well the digits don't repeat in decimal, but they do in binary, and floats (and doubles of course) are stored in binary. (usually).

This is the source of the anomaly.