Patchwork [v2,01/20] softfloat: fix floatx80 handling of NaN

login
register
mail settings
Submitter Aurelien Jarno
Date April 20, 2011, 10:11 a.m.
Message ID <1303294329-22634-2-git-send-email-aurelien@aurel32.net>
Download mbox | patch
Permalink /patch/92134/
State New
Headers show

Comments

Aurelien Jarno - April 20, 2011, 10:11 a.m.
The floatx80 format uses an explicit bit that should be taken into account
when converting to and from commonNaN format.

When converting to commonNaN, the explicit bit should be removed if it is
a 1, and a default NaN should be used if it is 0.

When converting from commonNan, the explicit bit should be added.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
---
 fpu/softfloat-specialize.h |   19 +++++++++++++------
 1 files changed, 13 insertions(+), 6 deletions(-)

v1 -> v2: fix wrong condition that may create an infinity instead of a NaN.
Peter Maydell - April 20, 2011, 10:35 a.m.
On 20 April 2011 11:11, Aurelien Jarno <aurelien@aurel32.net> wrote:
> @@ -624,10 +630,11 @@ static floatx80 commonNaNToFloatx80( commonNaNT a STATUS_PARAM)
>         return z;
>     }
>
> -    if (a.high)
> -        z.low = a.high;
> -    else
> +    if (a.high >> 1) {
> +        z.low = LIT64( 0x8000000000000000 ) | a.high >> 1;
> +    } else {
>         z.low = floatx80_default_nan_low;
> +    }
>     z.high = ( ( (uint16_t) a.sign )<<15 ) | 0x7FFF;
>     return z;
>  }

This is still retaining the sign bit from the input if it generates
a default NaN because the mantissa would have been zero. This isn't
consistent with the commonNaNToFloat64/32, which just return the
float64/32_default_nan with whatever sign it has.

-- PMM

Patch

diff --git a/fpu/softfloat-specialize.h b/fpu/softfloat-specialize.h
index b110187..29e30ce 100644
--- a/fpu/softfloat-specialize.h
+++ b/fpu/softfloat-specialize.h
@@ -603,9 +603,15 @@  static commonNaNT floatx80ToCommonNaN( floatx80 a STATUS_PARAM)
     commonNaNT z;
 
     if ( floatx80_is_signaling_nan( a ) ) float_raise( float_flag_invalid STATUS_VAR);
-    z.sign = a.high>>15;
-    z.low = 0;
-    z.high = a.low;
+    if ( a.low >> 63 ) {
+        z.sign = a.high >> 15;
+        z.low = 0;
+        z.high = a.low << 1;
+    } else {
+        z.sign = floatx80_default_nan_high >> 15;
+        z.low = 0;
+        z.high = floatx80_default_nan_low << 1;
+    }
     return z;
 }
 
@@ -624,10 +630,11 @@  static floatx80 commonNaNToFloatx80( commonNaNT a STATUS_PARAM)
         return z;
     }
 
-    if (a.high)
-        z.low = a.high;
-    else
+    if (a.high >> 1) {
+        z.low = LIT64( 0x8000000000000000 ) | a.high >> 1;
+    } else {
         z.low = floatx80_default_nan_low;
+    }
     z.high = ( ( (uint16_t) a.sign )<<15 ) | 0x7FFF;
     return z;
 }