Messages in this thread | | | Date | Wed, 24 Feb 1999 01:56:28 +1100 | From | Andrew Morton <> | Subject | Re: [OFFTOPIC] Rounding problem |
| |
Andrew Logan wrote: > > Hi all, > > I've found a worrying rounding error which is manifested by the > following simple test program. > [ snip ]
Wow. One I can answer..
Here's the degenerate example:
double d1 = 2.4, d2 = 1000, d3 = 2400;
main() { if (d1 * d2 == d3) printf("just lucky\n");
printf("%d\n", (long)(d1 * d2)); }
You'll find that the floating point representation of one or both of 2.4 and 1000 isn't exact, so the product of them comes to a tiny, tiny bit less that 2400. The Pentium's hardware float to long conversion then rounds down to 2399.
Note that this program doesn't say "just lucky". This is why floats should never be compared for equality.
It works with 'cc -msoft-float'. Just lucky.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |