lkml.org 
[lkml]   [2008]   [Jun]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Subjectmath-emu issue with fp divide
Date
David,

You seem to be the last person to touch the math-emu code in the
kernel so I figured you might have some insights. I'm trying to move
the powerpc math-emu code to use the include/math-emu bits.

In doing so I've been using TestFloat to see how good or bad we are
doing. For the most part the current math-emu code that PPC uses has
a number of issues that the code in include/math-emu seems to solve
(plus bugs we've had for ever that no one every realized).

Anyways, I've come across a case that we are flagging underflow and
inexact because we think we have a denormalized result from a double
precision divide:

000.FFFFFFFFFFFFF / 3FE.FFFFFFFFFFFFE
soft: 001.0000000000000 ..... syst: 001.0000000000000 ...ux

What it looks like is the results out of FP_DIV_D are:

D:
sign: 0
mantissa: 01000000 00000000
exp: -1023 (0)

The problem seems like we aren't normalizing the result and bumping
the exp. So I was wondering if you had any idea if FP_DIV_D should be
doing that or if when we call FP_PACK_DP/_FP_PACK_CANONICAL if that
could should be doing it.

thanks

- k


\
 
 \ /
  Last update: 2008-06-04 22:41    [W:0.184 / U:1.228 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site