Patchwork fpu: Correct edgecase in float64_muladd

login
register
mail settings
Submitter Peter Maydell
Date April 12, 2013, 3:37 p.m.
Message ID <1365781072-24979-1-git-send-email-peter.maydell@linaro.org>
Download mbox | patch
Permalink /patch/236136/
State New
Headers show

Comments

Peter Maydell - April 12, 2013, 3:37 p.m.
In handling float64_muladd, if we end up doing a subtraction of the
product and c, and the 128 bit result of this subtraction happens to
have its most significant bit in bit 63, we weren't handling this
correctly when attempting to normalize to put the most significant
bit into bit 126.  We would end up doing a right shift by a negative
number (undefined behaviour in C) so at best we would return an
incorrect result to the guest.  MSB in bit 63 has to be handled as a
special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
in 127 is not possible.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
Specific test vector which triggers this:
 a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000

Also tested with my usual set of random test vectors.

 fpu/softfloat.c |   12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
Aurelien Jarno - April 13, 2013, 1:57 p.m.
On Fri, Apr 12, 2013 at 04:37:52PM +0100, Peter Maydell wrote:
> In handling float64_muladd, if we end up doing a subtraction of the
> product and c, and the 128 bit result of this subtraction happens to
> have its most significant bit in bit 63, we weren't handling this
> correctly when attempting to normalize to put the most significant
> bit into bit 126.  We would end up doing a right shift by a negative
> number (undefined behaviour in C) so at best we would return an
> incorrect result to the guest.  MSB in bit 63 has to be handled as a
> special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
> in 127 is not possible.)
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Specific test vector which triggers this:
>  a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000
> 
> Also tested with my usual set of random test vectors.
> 
>  fpu/softfloat.c |   12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/fpu/softfloat.c b/fpu/softfloat.c
> index 83ccc4b..7ba51b6 100644
> --- a/fpu/softfloat.c
> +++ b/fpu/softfloat.c
> @@ -3898,9 +3898,15 @@ float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
>              }
>              zExp -= shiftcount;
>          } else {
> -            shiftcount = countLeadingZeros64(zSig1) - 1;
> -            zSig0 = zSig1 << shiftcount;
> -            zExp -= (shiftcount + 64);
> +            shiftcount = countLeadingZeros64(zSig1);
> +            if (shiftcount == 0) {
> +                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
> +                zExp -= 63;
> +            } else {
> +                shiftcount--;
> +                zSig0 = zSig1 << shiftcount;
> +                zExp -= (shiftcount + 64);
> +            }
>          }
>          return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
>      }
> 
Reviewed-by: Aurelien Jarno <aurelien@aurel32.net>
Aurelien Jarno - April 15, 2013, 2:25 p.m.
On Fri, Apr 12, 2013 at 04:37:52PM +0100, Peter Maydell wrote:
> In handling float64_muladd, if we end up doing a subtraction of the
> product and c, and the 128 bit result of this subtraction happens to
> have its most significant bit in bit 63, we weren't handling this
> correctly when attempting to normalize to put the most significant
> bit into bit 126.  We would end up doing a right shift by a negative
> number (undefined behaviour in C) so at best we would return an
> incorrect result to the guest.  MSB in bit 63 has to be handled as a
> special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
> in 127 is not possible.)
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Specific test vector which triggers this:
>  a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000
> 
> Also tested with my usual set of random test vectors.
> 
>  fpu/softfloat.c |   12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/fpu/softfloat.c b/fpu/softfloat.c
> index 83ccc4b..7ba51b6 100644
> --- a/fpu/softfloat.c
> +++ b/fpu/softfloat.c
> @@ -3898,9 +3898,15 @@ float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
>              }
>              zExp -= shiftcount;
>          } else {
> -            shiftcount = countLeadingZeros64(zSig1) - 1;
> -            zSig0 = zSig1 << shiftcount;
> -            zExp -= (shiftcount + 64);
> +            shiftcount = countLeadingZeros64(zSig1);
> +            if (shiftcount == 0) {
> +                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
> +                zExp -= 63;
> +            } else {
> +                shiftcount--;
> +                zSig0 = zSig1 << shiftcount;
> +                zExp -= (shiftcount + 64);
> +            }
>          }
>          return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
>      }

Thanks, applied.

Patch

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 83ccc4b..7ba51b6 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -3898,9 +3898,15 @@  float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
             }
             zExp -= shiftcount;
         } else {
-            shiftcount = countLeadingZeros64(zSig1) - 1;
-            zSig0 = zSig1 << shiftcount;
-            zExp -= (shiftcount + 64);
+            shiftcount = countLeadingZeros64(zSig1);
+            if (shiftcount == 0) {
+                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
+                zExp -= 63;
+            } else {
+                shiftcount--;
+                zSig0 = zSig1 << shiftcount;
+                zExp -= (shiftcount + 64);
+            }
         }
         return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
     }