Messages in this thread | | | From | Andrew Logan <> | Date | Tue, 23 Feb 1999 09:16:58 +0000 | Subject | [OFFTOPIC] Rounding problem |
| |
Hi all,
I've found a worrying rounding error which is manifested by the following simple test program.
I ran this on linux2.0.26, and compiled with gcc version 2.7.2
double d1 = 2.4, d2 = 1000, d3;
func1( double val ) { printf( "%ld\n", (long) val ); }
func( long val ) { printf( "%ld\n", val ); }
main() { d3 = d1 * d2; printf( "func( (long) d3 ) -> " ); func( (long) d3 ); printf( "func( (long) (d1 * d2) ) -> " ); func( (long) (d1 * d2) ); printf( "func1( d1 * d2 ) -> " ); func1( d1 * d2 ); }
This produced:
func( (long) d3 ) -> 2400 func( (long) (d1 * d2) ) -> 2399 func1( d1 * d2 ) -> 2400
The second line of output manifests the rounding problem. I compiled and ran exactly the same program on Non-StopUX with the native C compiler and Win98 using M$ 32-bit compiler version 10.20.6166 which both produced the correct value on the second output line. Would anyone like to comment?
Andy aml@bsl.co.uk
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |