Fix scalb spurious "invalid" exceptions (bug 16770)

This patch fixes bug 16770, spurious "invalid" exceptions from scalb
when testing whether the second argument is an integer, by inserting
appropriate range checks to determine whether a cast to int is safe.
(Note that invalid_fn is a function that handles both nonintegers and
large integers, distinguishing them reliably using functions such as
__rint; note also that there are no issues with scalb needing to avoid
spurious "inexact" exceptions - it's an old-POSIX XSI function, not a
standard C function bound to an IEEE 754 operation - although the
return value is still fully determined.)

Tested x86_64 and x86.

2014-03-29  Joseph Myers  <joseph@codesourcery.com>

	[BZ #16770]
	* math/e_scalb.c (__ieee754_scalb): Check second argument is not
	too large before casting to int.
	* math/e_scalbf.c (__ieee754_scalbf): Likewise.
	* math/e_scalbl.c (__ieee754_scalbl): Likewise.
	* math/libm-test.inc (scalb_test_data): Add more tests.

On 03/29/2014 05:21 PM, Joseph S. Myers wrote:
> This patch fixes bug 16770, spurious "invalid" exceptions from scalb
> when testing whether the second argument is an integer, by inserting
> appropriate range checks to determine whether a cast to int is safe.
> (Note that invalid_fn is a function that handles both nonintegers and
> large integers, distinguishing them reliably using functions such as
> __rint; note also that there are no issues with scalb needing to avoid
> spurious "inexact" exceptions - it's an old-POSIX XSI function, not a
> standard C function bound to an IEEE 754 operation - although the
> return value is still fully determined.)

Thanks,
Andreas

On Sat, Mar 29, 2014 at 04:21:23PM +0000, Joseph S. Myers wrote:
> This patch fixes bug 16770, spurious "invalid" exceptions from scalb
> when testing whether the second argument is an integer, by inserting
> appropriate range checks to determine whether a cast to int is safe.
> (Note that invalid_fn is a function that handles both nonintegers and
> large integers, distinguishing them reliably using functions such as
> __rint; note also that there are no issues with scalb needing to avoid
> spurious "inexact" exceptions - it's an old-POSIX XSI function, not a
> standard C function bound to an IEEE 754 operation - although the
> return value is still fully determined.)
> 
> Tested x86_64 and x86.
> 
> 2014-03-29  Joseph Myers  <joseph@codesourcery.com>
> 
> 	[BZ #16770]
> 	* math/e_scalb.c (__ieee754_scalb): Check second argument is not
> 	too large before casting to int.
> 	* math/e_scalbf.c (__ieee754_scalbf): Likewise.
> 	* math/e_scalbl.c (__ieee754_scalbl): Likewise.
> 	* math/libm-test.inc (scalb_test_data): Add more tests.
> 
> diff --git a/math/e_scalb.c b/math/e_scalb.c
> index bddedfa..146d49e 100644
> --- a/math/e_scalb.c
> +++ b/math/e_scalb.c
> @@ -50,7 +50,7 @@ __ieee754_scalb (double x, double fn)
>  	return x;
>        return x / -fn;
>      }
> -  if (__glibc_unlikely ((double) (int) fn != fn))
> +  if (__glibc_unlikely (fabs (fn) >= 0x1p31 || (double) (int) fn != fn))

Off-by-one for the negative case: -0x1p31 is representable as int.
Maybe it doesn't matter anyway, but if it does, you could test
fn>=0x1p31 and fn<-0x1p31 separately.

Rich

On Sat, 29 Mar 2014, Rich Felker wrote:

> > -  if (__glibc_unlikely ((double) (int) fn != fn))
> > +  if (__glibc_unlikely (fabs (fn) >= 0x1p31 || (double) (int) fn != fn))
> 
> Off-by-one for the negative case: -0x1p31 is representable as int.
> Maybe it doesn't matter anyway, but if it does, you could test
> fn>=0x1p31 and fn<-0x1p31 separately.

The purpose of this check is to be safe: to avoid the cast if it might be 
out of range; there is no functional need for the cast to occur just 
because it's in range.  Any threshold above 65000.0 (the arbitrary value 
used in invalid_fn to decide whether to call __scalbn with a positive or 
negative scale) would work as well here.