diff mbox series

Use floor functions not __floor functions in glibc libm

Message ID alpine.DEB.2.21.1809121216000.28943@digraph.polyomino.org.uk
State New
Headers show
Series Use floor functions not __floor functions in glibc libm | expand

Commit Message

Joseph Myers Sept. 12, 2018, 12:17 p.m. UTC 
Similar to the changes that were made to call sqrt functions directly
in glibc, instead of __ieee754_sqrt variants, so that the compiler
could inline them automatically without needing special inline
definitions in lots of math_private.h headers, this patch makes libm
code call floor functions directly instead of __floor variants,
removing the inlines / macros for x86_64 (SSE4.1) and powerpc
(POWER5).

The redirection used to ensure that __ieee754_sqrt does still get
called when the compiler doesn't inline a built-in function expansion
is refactored so it can be applied to other functions; the refactoring
is arranged so it's not limited to unary functions either (it would be
reasonable to use this mechanism for copysign - removing the inline in
math_private_calls.h but also eliminating unnecessary local PLT entry
use in the cases (powerpc soft-float and e500v1, for IBM long double)
where copysign calls don't get inlined).

The point of this change is that more architectures can get floor
calls inlined where they weren't previously (AArch64, for example),
without needing special inline definitions in their math_private.h,
and existing such definitions in math_private.h headers can be
removed.

Note that it's possible that in some cases an inline may be used where
an IFUNC call was previously used - this is the case on x86_64, for
example.  I think the direct calls to floor are still appropriate; if
there's any significant performance cost from inline SSE2 floor
instead of an IFUNC call ending up with SSE4.1 floor, that indicates
that either the function should be doing something else that's faster
than using floor at all, or it should itself have IFUNC variants, or
that the compiler choice of inlining for generic tuning should change
to allow for the possibility that, by not inlining, an SSE4.1 IFUNC
might be called at runtime - but not that glibc should avoid calling
floor internally.  (After all, all the same considerations would apply
to any user program calling floor, where it might either be inlined or
left as an out-of-line call allowing for a possible IFUNC.)  Any
comments on this point?

Tested for x86_64, and with build-many-glibcs.py.

Note that this depends on 
<https://sourceware.org/ml/libc-alpha/2018-09/msg00147.html>, to avoid 
problems with the call to floor from elf/tst-relsort1mod2.c.

2018-09-12  Joseph Myers  <joseph@codesourcery.com>

	* include/math.h [!_ISOMAC && !(__FINITE_MATH_ONLY__ &&
	__FINITE_MATH_ONLY__ > 0) && !NO_MATH_REDIRECT] (MATH_REDIRECT):
	New macro.
	[!_ISOMAC && !(__FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
	&& !NO_MATH_REDIRECT] (MATH_REDIRECT_LDBL): Likewise.
	[!_ISOMAC && !(__FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
	&& !NO_MATH_REDIRECT] (MATH_REDIRECT_F128): Likewise.
	[!_ISOMAC && !(__FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
	&& !NO_MATH_REDIRECT] (MATH_REDIRECT_UNARY_ARGS): Likewise.
	[!_ISOMAC && !(__FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
	&& !NO_MATH_REDIRECT] (sqrt): Redirect using MATH_REDIRECT.
	[!_ISOMAC && !(__FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
	&& !NO_MATH_REDIRECT] (floor): Likewise.
	* sysdeps/aarch64/fpu/s_floor.c: Define NO_MATH_REDIRECT before
	header inclusion.
	* sysdeps/aarch64/fpu/s_floorf.c: Likewise.
	* sysdeps/ieee754/dbl-64/s_floor.c: Likewise.
	* sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c: Likewise.
	* sysdeps/ieee754/float128/s_floorf128.c: Likewise.
	* sysdeps/ieee754/flt-32/s_floorf.c: Likewise.
	* sysdeps/ieee754/ldbl-128/s_floorl.c: Likewise.
	* sysdeps/ieee754/ldbl-128ibm/s_floorl.c: Likewise.
	* sysdeps/m68k/m680x0/fpu/s_floor_template.c: Likewise.
	* sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floor.c: Likewise.
	* sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floorf.c: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/multiarch/s_floor.c: Likewise.
	* sysdeps/powerpc/powerpc64/fpu/multiarch/s_floorf.c: Likewise.
	* sysdeps/riscv/rv64/rvd/s_floor.c: Likewise.
	* sysdeps/riscv/rvf/s_floorf.c: Likewise.
	* sysdeps/sparc/sparc64/fpu/multiarch/s_floor.c: Likewise.
	* sysdeps/sparc/sparc64/fpu/multiarch/s_floorf.c: Likewise.
	* sysdeps/x86_64/fpu/multiarch/s_floor.c: Likewise.
	* sysdeps/x86_64/fpu/multiarch/s_floorf.c: Likewise.
	* sysdeps/powerpc/fpu/math_private.h [_ARCH_PWR5X] (__floor):
	Remove macro.
	[_ARCH_PWR5X] (__floorf): Likewise.
	* sysdeps/x86_64/fpu/math_private.h [__SSE4_1__] (__floor): Remove
	inline function.
	[__SSE4_1__] (__floorf): Likewise.
	* math/w_lgamma_main.c (LGFUNC (__lgamma)): Use floor functions
	instead of __floor variants.
	* math/w_lgamma_r_compat.c (__lgamma_r): Likewise.
	* math/w_lgammaf_main.c (LGFUNC (__lgammaf)): Likewise.
	* math/w_lgammaf_r_compat.c (__lgammaf_r): Likewise.
	* math/w_lgammal_main.c (LGFUNC (__lgammal)): Likewise.
	* math/w_lgammal_r_compat.c (__lgammal_r): Likewise.
	* math/w_tgamma_compat.c (__tgamma): Likewise.
	* math/w_tgamma_template.c (M_DECL_FUNC (__tgamma)): Likewise.
	* math/w_tgammaf_compat.c (__tgammaf): Likewise.
	* math/w_tgammal_compat.c (__tgammal): Likewise.
	* sysdeps/ieee754/dbl-64/e_lgamma_r.c (sin_pi): Likewise.
	* sysdeps/ieee754/dbl-64/k_rem_pio2.c (__kernel_rem_pio2):
	Likewise.
	* sysdeps/ieee754/dbl-64/lgamma_neg.c (__lgamma_neg): Likewise.
	* sysdeps/ieee754/flt-32/e_lgammaf_r.c (sin_pif): Likewise.
	* sysdeps/ieee754/flt-32/lgamma_negf.c (__lgamma_negf): Likewise.
	* sysdeps/ieee754/ldbl-128/e_lgammal_r.c (__ieee754_lgammal_r):
	Likewise.
	* sysdeps/ieee754/ldbl-128/e_powl.c (__ieee754_powl): Likewise.
	* sysdeps/ieee754/ldbl-128/lgamma_negl.c (__lgamma_negl):
	Likewise.
	* sysdeps/ieee754/ldbl-128/s_expm1l.c (__expm1l): Likewise.
	* sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c (__ieee754_lgammal_r):
	Likewise.
	* sysdeps/ieee754/ldbl-128ibm/e_powl.c (__ieee754_powl): Likewise.
	* sysdeps/ieee754/ldbl-128ibm/lgamma_negl.c (__lgamma_negl):
	Likewise.
	* sysdeps/ieee754/ldbl-128ibm/s_expm1l.c (__expm1l): Likewise.
	* sysdeps/ieee754/ldbl-128ibm/s_truncl.c (__truncl): Likewise.
	* sysdeps/ieee754/ldbl-96/e_lgammal_r.c (sin_pi): Likewise.
	* sysdeps/ieee754/ldbl-96/lgamma_negl.c (__lgamma_negl): Likewise.
	* sysdeps/powerpc/power5+/fpu/s_modf.c (__modf): Likewise.
	* sysdeps/powerpc/power5+/fpu/s_modff.c (__modff): Likewise.
diff mbox series

Patch

diff --git a/include/math.h b/include/math.h
index 9ab7aa4403..eed953072b 100644
--- a/include/math.h
+++ b/include/math.h
@@ -79,16 +79,29 @@  fabsf128 (_Float128 x)
 
 # if !(defined __FINITE_MATH_ONLY__ && __FINITE_MATH_ONLY__ > 0)
 #  ifndef NO_MATH_REDIRECT
-/* Declare sqrt for use within GLIBC.  Compilers typically inline sqrt as a
-   single instruction.  Use an asm to avoid use of PLTs if it doesn't.  */
-float (sqrtf) (float) asm ("__ieee754_sqrtf");
-double (sqrt) (double) asm ("__ieee754_sqrt");
-#   ifndef __NO_LONG_DOUBLE_MATH
-long double (sqrtl) (long double) asm ("__ieee754_sqrtl");
+/* Declare some functions for use within GLIBC.  Compilers typically
+   inline those functions as a single instruction.  Use an asm to
+   avoid use of PLTs if it doesn't.  */
+#   define MATH_REDIRECT(FUNC, PREFIX, ARGS)			\
+  float (FUNC ## f) (ARGS (float)) asm (PREFIX #FUNC "f");	\
+  double (FUNC) (ARGS (double)) asm (PREFIX #FUNC );		\
+  MATH_REDIRECT_LDBL (FUNC, PREFIX, ARGS)			\
+  MATH_REDIRECT_F128 (FUNC, PREFIX, ARGS)
+#   ifdef __NO_LONG_DOUBLE_MATH
+#    define MATH_REDIRECT_LDBL(FUNC, PREFIX, ARGS)
+#   else
+#    define MATH_REDIRECT_LDBL(FUNC, PREFIX, ARGS)			\
+  long double (FUNC ## l) (ARGS (long double)) asm (PREFIX #FUNC "l");
 #   endif
-#   if __HAVE_DISTINCT_FLOAT128 > 0
-_Float128 (sqrtf128) (_Float128) asm ("__ieee754_sqrtf128");
+#   if __HAVE_DISTINCT_FLOAT128
+#    define MATH_REDIRECT_F128(FUNC, PREFIX, ARGS)			\
+  _Float128 (FUNC ## f128) (ARGS (_Float128)) asm (PREFIX #FUNC "f128");
+#   else
+#    define MATH_REDIRECT_F128(FUNC, PREFIX, ARGS)
 #   endif
+#   define MATH_REDIRECT_UNARY_ARGS(TYPE) TYPE
+MATH_REDIRECT (sqrt, "__ieee754_", MATH_REDIRECT_UNARY_ARGS)
+MATH_REDIRECT (floor, "__", MATH_REDIRECT_UNARY_ARGS)
 #  endif
 # endif
 
diff --git a/math/w_lgamma_main.c b/math/w_lgamma_main.c
index ac91b9d9e6..573bcb88fb 100644
--- a/math/w_lgamma_main.c
+++ b/math/w_lgamma_main.c
@@ -31,7 +31,7 @@  LGFUNC (__lgamma) (double x)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard(x, x,
-					 __floor(x)==x&&x<=0.0
+					 floor(x)==x&&x<=0.0
 					 ? 15 /* lgamma pole */
 					 : 14); /* lgamma overflow */
 
diff --git a/math/w_lgamma_r_compat.c b/math/w_lgamma_r_compat.c
index 5b3ceaa2ea..c68ce67174 100644
--- a/math/w_lgamma_r_compat.c
+++ b/math/w_lgamma_r_compat.c
@@ -28,7 +28,7 @@  __lgamma_r(double x, int *signgamp)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard(x, x,
-					 __floor(x)==x&&x<=0.0
+					 floor(x)==x&&x<=0.0
 					 ? 15 /* lgamma pole */
 					 : 14); /* lgamma overflow */
 
diff --git a/math/w_lgammaf_main.c b/math/w_lgammaf_main.c
index ef6065994f..2a85e3cadf 100644
--- a/math/w_lgammaf_main.c
+++ b/math/w_lgammaf_main.c
@@ -28,7 +28,7 @@  LGFUNC (__lgammaf) (float x)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard_f(x, x,
-					   __floorf(x)==x&&x<=0.0f
+					   floorf(x)==x&&x<=0.0f
 					   ? 115 /* lgamma pole */
 					   : 114); /* lgamma overflow */
 
diff --git a/math/w_lgammaf_r_compat.c b/math/w_lgammaf_r_compat.c
index c7751f13f5..78fc4fcb23 100644
--- a/math/w_lgammaf_r_compat.c
+++ b/math/w_lgammaf_r_compat.c
@@ -31,7 +31,7 @@  __lgammaf_r(float x, int *signgamp)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard_f(x, x,
-					   __floorf(x)==x&&x<=0.0f
+					   floorf(x)==x&&x<=0.0f
 					   ? 115 /* lgamma pole */
 					   : 114); /* lgamma overflow */
 
diff --git a/math/w_lgammal_main.c b/math/w_lgammal_main.c
index f269cef6ab..04440cd29f 100644
--- a/math/w_lgammal_main.c
+++ b/math/w_lgammal_main.c
@@ -35,7 +35,7 @@  LGFUNC (__lgammal) (long double x)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard_l(x, x,
-					   __floorl(x)==x&&x<=0.0L
+					   floorl(x)==x&&x<=0.0L
 					   ? 215 /* lgamma pole */
 					   : 214); /* lgamma overflow */
 
diff --git a/math/w_lgammal_r_compat.c b/math/w_lgammal_r_compat.c
index 09a8070b46..6ebfa41333 100644
--- a/math/w_lgammal_r_compat.c
+++ b/math/w_lgammal_r_compat.c
@@ -32,7 +32,7 @@  __lgammal_r(long double x, int *signgamp)
 	if(__builtin_expect(!isfinite(y), 0)
 	   && isfinite(x) && _LIB_VERSION != _IEEE_)
 		return __kernel_standard(x, x,
-					 __floorl(x)==x&&x<=0.0
+					 floorl(x)==x&&x<=0.0
 					 ? 215 /* lgamma pole */
 					 : 214); /* lgamma overflow */
 
diff --git a/math/w_tgamma_compat.c b/math/w_tgamma_compat.c
index 219aa10862..910d2fe490 100644
--- a/math/w_tgamma_compat.c
+++ b/math/w_tgamma_compat.c
@@ -33,7 +33,7 @@  __tgamma(double x)
 	   && _LIB_VERSION != _IEEE_) {
 	  if (x == 0.0)
 	    return __kernel_standard(x,x,50); /* tgamma pole */
-	  else if(__floor(x)==x&&x<0.0)
+	  else if(floor(x)==x&&x<0.0)
 	    return __kernel_standard(x,x,41); /* tgamma domain */
 	  else if (y == 0)
 	    __set_errno (ERANGE); /* tgamma underflow */
diff --git a/math/w_tgamma_template.c b/math/w_tgamma_template.c
index 032f27a3f7..f570615d7b 100644
--- a/math/w_tgamma_template.c
+++ b/math/w_tgamma_template.c
@@ -41,7 +41,7 @@  M_DECL_FUNC (__tgamma) (FLOAT x)
       if (x == 0)
 	/* Pole error: tgamma(x=0).  */
 	__set_errno (ERANGE);
-      else if (M_SUF (__floor) (x) == x && x < 0)
+      else if (M_SUF (floor) (x) == x && x < 0)
 	/* Domain error: tgamma(integer x<0).  */
 	__set_errno (EDOM);
       else
diff --git a/math/w_tgammaf_compat.c b/math/w_tgammaf_compat.c
index e9ffddb796..ed509885ae 100644
--- a/math/w_tgammaf_compat.c
+++ b/math/w_tgammaf_compat.c
@@ -32,7 +32,7 @@  __tgammaf(float x)
 	  if (x == (float)0.0)
 	    /* tgammaf pole */
 	    return __kernel_standard_f(x, x, 150);
-	  else if(__floorf(x)==x&&x<0.0f)
+	  else if(floorf(x)==x&&x<0.0f)
 	    /* tgammaf domain */
 	    return __kernel_standard_f(x, x, 141);
 	  else if (y == 0)
diff --git a/math/w_tgammal_compat.c b/math/w_tgammal_compat.c
index 3695b7fbb3..d79c788dfc 100644
--- a/math/w_tgammal_compat.c
+++ b/math/w_tgammal_compat.c
@@ -36,7 +36,7 @@  __tgammal(long double x)
 	   && _LIB_VERSION != _IEEE_) {
 	  if(x==0.0)
 	    return __kernel_standard_l(x,x,250); /* tgamma pole */
-	  else if(__floorl(x)==x&&x<0.0L)
+	  else if(floorl(x)==x&&x<0.0L)
 	    return __kernel_standard_l(x,x,241); /* tgamma domain */
 	  else if (y == 0)
 	    __set_errno (ERANGE); /* tgamma underflow */
diff --git a/sysdeps/aarch64/fpu/s_floor.c b/sysdeps/aarch64/fpu/s_floor.c
index 7258246896..9580a5313d 100644
--- a/sysdeps/aarch64/fpu/s_floor.c
+++ b/sysdeps/aarch64/fpu/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <libm-alias-double.h>
 
diff --git a/sysdeps/aarch64/fpu/s_floorf.c b/sysdeps/aarch64/fpu/s_floorf.c
index bd197013ad..a874aaedaf 100644
--- a/sysdeps/aarch64/fpu/s_floorf.c
+++ b/sysdeps/aarch64/fpu/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <libm-alias-float.h>
 
diff --git a/sysdeps/ieee754/dbl-64/e_lgamma_r.c b/sysdeps/ieee754/dbl-64/e_lgamma_r.c
index 17717d915f..88051bc75b 100644
--- a/sysdeps/ieee754/dbl-64/e_lgamma_r.c
+++ b/sysdeps/ieee754/dbl-64/e_lgamma_r.c
@@ -168,10 +168,10 @@  sin_pi(double x)
      * argument reduction, make sure inexact flag not raised if input
      * is an integer
      */
-	z = __floor(y);
+	z = floor(y);
 	if(z!=y) {				/* inexact anyway */
 	    y  *= 0.5;
-	    y   = 2.0*(y - __floor(y));		/* y = |x| mod 2.0 */
+	    y   = 2.0*(y - floor(y));		/* y = |x| mod 2.0 */
 	    n   = (int) (y*4.0);
 	} else {
 	    if(ix>=0x43400000) {
diff --git a/sysdeps/ieee754/dbl-64/k_rem_pio2.c b/sysdeps/ieee754/dbl-64/k_rem_pio2.c
index d8403dc345..6221a1a474 100644
--- a/sysdeps/ieee754/dbl-64/k_rem_pio2.c
+++ b/sysdeps/ieee754/dbl-64/k_rem_pio2.c
@@ -196,7 +196,7 @@  recompute:
 
   /* compute n */
   z = __scalbn (z, q0);                 /* actual value of z */
-  z -= 8.0 * __floor (z * 0.125);               /* trim off integer >= 8 */
+  z -= 8.0 * floor (z * 0.125);               /* trim off integer >= 8 */
   n = (int32_t) z;
   z -= (double) n;
   ih = 0;
diff --git a/sysdeps/ieee754/dbl-64/lgamma_neg.c b/sysdeps/ieee754/dbl-64/lgamma_neg.c
index eaa6f641eb..56a3fca1ba 100644
--- a/sysdeps/ieee754/dbl-64/lgamma_neg.c
+++ b/sysdeps/ieee754/dbl-64/lgamma_neg.c
@@ -282,7 +282,7 @@  __lgamma_neg (double x, int *signgamp)
 {
   /* Determine the half-integer region X lies in, handle exact
      integers and determine the sign of the result.  */
-  int i = __floor (-2 * x);
+  int i = floor (-2 * x);
   if ((i & 1) == 0 && i == -2 * x)
     return 1.0 / 0.0;
   double xn = ((i & 1) == 0 ? -i / 2 : (-i - 1) / 2);
@@ -299,7 +299,7 @@  __lgamma_neg (double x, int *signgamp)
      approximations to an adjusted version of the gamma function.  */
   if (i < 2)
     {
-      int j = __floor (-8 * x) - 16;
+      int j = floor (-8 * x) - 16;
       double xm = (-33 - 2 * j) * 0.0625;
       double x_adj = x - xm;
       size_t deg = poly_deg[j];
diff --git a/sysdeps/ieee754/dbl-64/s_floor.c b/sysdeps/ieee754/dbl-64/s_floor.c
index f27c6f3ad2..bebc018658 100644
--- a/sysdeps/ieee754/dbl-64/s_floor.c
+++ b/sysdeps/ieee754/dbl-64/s_floor.c
@@ -17,6 +17,7 @@ 
  *	Bit twiddling.
  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <libm-alias-double.h>
diff --git a/sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c b/sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c
index f7e0a77ec3..44c1e19754 100644
--- a/sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c
+++ b/sysdeps/ieee754/dbl-64/wordsize-64/s_floor.c
@@ -30,6 +30,7 @@ 
  * ====================================================
  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <stdint.h>
diff --git a/sysdeps/ieee754/float128/s_floorf128.c b/sysdeps/ieee754/float128/s_floorf128.c
index 18298436a1..e0a6412980 100644
--- a/sysdeps/ieee754/float128/s_floorf128.c
+++ b/sysdeps/ieee754/float128/s_floorf128.c
@@ -1,2 +1,3 @@ 
+#define NO_MATH_REDIRECT
 #include <float128_private.h>
 #include "../ldbl-128/s_floorl.c"
diff --git a/sysdeps/ieee754/flt-32/e_lgammaf_r.c b/sysdeps/ieee754/flt-32/e_lgammaf_r.c
index ebe2d3462f..a7f9b9fbad 100644
--- a/sysdeps/ieee754/flt-32/e_lgammaf_r.c
+++ b/sysdeps/ieee754/flt-32/e_lgammaf_r.c
@@ -104,10 +104,10 @@  sin_pif(float x)
      * argument reduction, make sure inexact flag not raised if input
      * is an integer
      */
-	z = __floorf(y);
+	z = floorf(y);
 	if(z!=y) {				/* inexact anyway */
 	    y  *= (float)0.5;
-	    y   = (float)2.0*(y - __floorf(y));	/* y = |x| mod 2.0 */
+	    y   = (float)2.0*(y - floorf(y));	/* y = |x| mod 2.0 */
 	    n   = (int) (y*(float)4.0);
 	} else {
 	    if(ix>=0x4b800000) {
diff --git a/sysdeps/ieee754/flt-32/lgamma_negf.c b/sysdeps/ieee754/flt-32/lgamma_negf.c
index 7951cde5c7..c20b816acc 100644
--- a/sysdeps/ieee754/flt-32/lgamma_negf.c
+++ b/sysdeps/ieee754/flt-32/lgamma_negf.c
@@ -197,7 +197,7 @@  __lgamma_negf (float x, int *signgamp)
 {
   /* Determine the half-integer region X lies in, handle exact
      integers and determine the sign of the result.  */
-  int i = __floorf (-2 * x);
+  int i = floorf (-2 * x);
   if ((i & 1) == 0 && i == -2 * x)
     return 1.0f / 0.0f;
   float xn = ((i & 1) == 0 ? -i / 2 : (-i - 1) / 2);
@@ -214,7 +214,7 @@  __lgamma_negf (float x, int *signgamp)
      approximations to an adjusted version of the gamma function.  */
   if (i < 2)
     {
-      int j = __floorf (-8 * x) - 16;
+      int j = floorf (-8 * x) - 16;
       float xm = (-33 - 2 * j) * 0.0625f;
       float x_adj = x - xm;
       size_t deg = poly_deg[j];
diff --git a/sysdeps/ieee754/flt-32/s_floorf.c b/sysdeps/ieee754/flt-32/s_floorf.c
index 12aed343a0..b34d967f01 100644
--- a/sysdeps/ieee754/flt-32/s_floorf.c
+++ b/sysdeps/ieee754/flt-32/s_floorf.c
@@ -20,6 +20,7 @@ 
  *	Bit twiddling.
  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <libm-alias-float.h>
diff --git a/sysdeps/ieee754/ldbl-128/e_lgammal_r.c b/sysdeps/ieee754/ldbl-128/e_lgammal_r.c
index 5c50e4616a..f470034f9c 100644
--- a/sysdeps/ieee754/ldbl-128/e_lgammal_r.c
+++ b/sysdeps/ieee754/ldbl-128/e_lgammal_r.c
@@ -776,11 +776,11 @@  __ieee754_lgammal_r (_Float128 x, int *signgamp)
       if (x < -2 && x > -50)
 	return __lgamma_negl (x, signgamp);
       q = -x;
-      p = __floorl (q);
+      p = floorl (q);
       if (p == q)
 	return (one / fabsl (p - p));
       _Float128 halfp = p * L(0.5);
-      if (halfp == __floorl (halfp))
+      if (halfp == floorl (halfp))
 	*signgamp = -1;
       else
 	*signgamp = 1;
@@ -801,7 +801,7 @@  __ieee754_lgammal_r (_Float128 x, int *signgamp)
   if (x < L(13.5))
     {
       p = 0;
-      nx = __floorl (x + L(0.5));
+      nx = floorl (x + L(0.5));
       nn = nx;
       switch (nn)
 	{
diff --git a/sysdeps/ieee754/ldbl-128/e_powl.c b/sysdeps/ieee754/ldbl-128/e_powl.c
index dc748c3a0b..47226f8688 100644
--- a/sysdeps/ieee754/ldbl-128/e_powl.c
+++ b/sysdeps/ieee754/ldbl-128/e_powl.c
@@ -198,10 +198,10 @@  __ieee754_powl (_Float128 x, _Float128 y)
 	yisint = 2;		/* even integer y */
       else if (iy >= 0x3fff0000)	/* 1.0 */
 	{
-	  if (__floorl (y) == y)
+	  if (floorl (y) == y)
 	    {
 	      z = 0.5 * y;
-	      if (__floorl (z) == z)
+	      if (floorl (z) == z)
 		yisint = 2;
 	      else
 		yisint = 1;
@@ -413,7 +413,7 @@  __ieee754_powl (_Float128 x, _Float128 y)
   n = 0;
   if (i > 0x3ffe0000)
     {				/* if |z| > 0.5, set n = [z+0.5] */
-      n = __floorl (z + L(0.5));
+      n = floorl (z + L(0.5));
       t = n;
       p_h -= t;
     }
diff --git a/sysdeps/ieee754/ldbl-128/lgamma_negl.c b/sysdeps/ieee754/ldbl-128/lgamma_negl.c
index cb64d4b0a9..c0bf1661dc 100644
--- a/sysdeps/ieee754/ldbl-128/lgamma_negl.c
+++ b/sysdeps/ieee754/ldbl-128/lgamma_negl.c
@@ -449,7 +449,7 @@  __lgamma_negl (_Float128 x, int *signgamp)
 {
   /* Determine the half-integer region X lies in, handle exact
      integers and determine the sign of the result.  */
-  int i = __floorl (-2 * x);
+  int i = floorl (-2 * x);
   if ((i & 1) == 0 && i == -2 * x)
     return L(1.0) / L(0.0);
   _Float128 xn = ((i & 1) == 0 ? -i / 2 : (-i - 1) / 2);
@@ -466,7 +466,7 @@  __lgamma_negl (_Float128 x, int *signgamp)
      approximations to an adjusted version of the gamma function.  */
   if (i < 2)
     {
-      int j = __floorl (-8 * x) - 16;
+      int j = floorl (-8 * x) - 16;
       _Float128 xm = (-33 - 2 * j) * L(0.0625);
       _Float128 x_adj = x - xm;
       size_t deg = poly_deg[j];
diff --git a/sysdeps/ieee754/ldbl-128/s_expm1l.c b/sysdeps/ieee754/ldbl-128/s_expm1l.c
index 66881af01a..08f9ec848b 100644
--- a/sysdeps/ieee754/ldbl-128/s_expm1l.c
+++ b/sysdeps/ieee754/ldbl-128/s_expm1l.c
@@ -136,7 +136,7 @@  __expm1l (_Float128 x)
 
   /* Express x = ln 2 (k + remainder), remainder not exceeding 1/2. */
   xx = C1 + C2;			/* ln 2. */
-  px = __floorl (0.5 + x / xx);
+  px = floorl (0.5 + x / xx);
   k = px;
   /* remainder times ln 2 */
   x -= px * C1;
diff --git a/sysdeps/ieee754/ldbl-128/s_floorl.c b/sysdeps/ieee754/ldbl-128/s_floorl.c
index f9c5e014f9..f340a3fbca 100644
--- a/sysdeps/ieee754/ldbl-128/s_floorl.c
+++ b/sysdeps/ieee754/ldbl-128/s_floorl.c
@@ -24,6 +24,7 @@  static char rcsid[] = "$NetBSD: $";
  *	Bit twiddling.
  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <libm-alias-ldouble.h>
diff --git a/sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c b/sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c
index 5b628bedc1..ae2e6559b0 100644
--- a/sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c
+++ b/sysdeps/ieee754/ldbl-128ibm/e_lgammal_r.c
@@ -726,11 +726,11 @@  __ieee754_lgammal_r (long double x, int *signgamp)
       if (x < -2 && x > -48)
 	return __lgamma_negl (x, signgamp);
       q = -x;
-      p = __floorl (q);
+      p = floorl (q);
       if (p == q)
 	return (one / fabsl (p - p));
       long double halfp = p * 0.5L;
-      if (halfp == __floorl (halfp))
+      if (halfp == floorl (halfp))
 	*signgamp = -1;
       else
 	*signgamp = 1;
@@ -751,7 +751,7 @@  __ieee754_lgammal_r (long double x, int *signgamp)
   if (x < 13.5L)
     {
       p = 0;
-      nx = __floorl (x + 0.5L);
+      nx = floorl (x + 0.5L);
       nn = nx;
       switch (nn)
 	{
diff --git a/sysdeps/ieee754/ldbl-128ibm/e_powl.c b/sysdeps/ieee754/ldbl-128ibm/e_powl.c
index f59ad4e113..1f0f2b4c04 100644
--- a/sysdeps/ieee754/ldbl-128ibm/e_powl.c
+++ b/sysdeps/ieee754/ldbl-128ibm/e_powl.c
@@ -195,10 +195,10 @@  __ieee754_powl (long double x, long double y)
 	yisint = 2;		/* even integer y */
       else if (iy >= 0x3ff00000)	/* 1.0 */
 	{
-	  if (__floorl (y) == y)
+	  if (floorl (y) == y)
 	    {
 	      z = 0.5 * y;
-	      if (__floorl (z) == z)
+	      if (floorl (z) == z)
 		yisint = 2;
 	      else
 		yisint = 1;
@@ -392,7 +392,7 @@  __ieee754_powl (long double x, long double y)
   n = 0;
   if (i > 0x3fe00000)
     {				/* if |z| > 0.5, set n = [z+0.5] */
-      n = __floorl (z + 0.5L);
+      n = floorl (z + 0.5L);
       t = n;
       p_h -= t;
     }
diff --git a/sysdeps/ieee754/ldbl-128ibm/lgamma_negl.c b/sysdeps/ieee754/ldbl-128ibm/lgamma_negl.c
index c96b641e02..c359657269 100644
--- a/sysdeps/ieee754/ldbl-128ibm/lgamma_negl.c
+++ b/sysdeps/ieee754/ldbl-128ibm/lgamma_negl.c
@@ -430,7 +430,7 @@  __lgamma_negl (long double x, int *signgamp)
 {
   /* Determine the half-integer region X lies in, handle exact
      integers and determine the sign of the result.  */
-  int i = __floorl (-2 * x);
+  int i = floorl (-2 * x);
   if ((i & 1) == 0 && i == -2 * x)
     return 1.0L / 0.0L;
   long double xn = ((i & 1) == 0 ? -i / 2 : (-i - 1) / 2);
@@ -447,7 +447,7 @@  __lgamma_negl (long double x, int *signgamp)
      approximations to an adjusted version of the gamma function.  */
   if (i < 2)
     {
-      int j = __floorl (-8 * x) - 16;
+      int j = floorl (-8 * x) - 16;
       long double xm = (-33 - 2 * j) * 0.0625L;
       long double x_adj = x - xm;
       size_t deg = poly_deg[j];
diff --git a/sysdeps/ieee754/ldbl-128ibm/s_expm1l.c b/sysdeps/ieee754/ldbl-128ibm/s_expm1l.c
index 42d57c6eec..22abee4d99 100644
--- a/sysdeps/ieee754/ldbl-128ibm/s_expm1l.c
+++ b/sysdeps/ieee754/ldbl-128ibm/s_expm1l.c
@@ -120,7 +120,7 @@  __expm1l (long double x)
 
   /* Express x = ln 2 (k + remainder), remainder not exceeding 1/2. */
   xx = C1 + C2;			/* ln 2. */
-  px = __floorl (0.5 + x / xx);
+  px = floorl (0.5 + x / xx);
   k = px;
   /* remainder times ln 2 */
   x -= px * C1;
diff --git a/sysdeps/ieee754/ldbl-128ibm/s_floorl.c b/sysdeps/ieee754/ldbl-128ibm/s_floorl.c
index 4aae5ae608..f33b6ad701 100644
--- a/sysdeps/ieee754/ldbl-128ibm/s_floorl.c
+++ b/sysdeps/ieee754/ldbl-128ibm/s_floorl.c
@@ -17,6 +17,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <math_ldbl_opt.h>
diff --git a/sysdeps/ieee754/ldbl-128ibm/s_truncl.c b/sysdeps/ieee754/ldbl-128ibm/s_truncl.c
index a915a01ac6..a366c42599 100644
--- a/sysdeps/ieee754/ldbl-128ibm/s_truncl.c
+++ b/sysdeps/ieee754/ldbl-128ibm/s_truncl.c
@@ -47,7 +47,7 @@  __truncl (long double x)
       else
 	{
 	  /* The high part is a nonzero integer.  */
-	  lo = xh > 0 ? __floor (xl) : __ceil (xl);
+	  lo = xh > 0 ? floor (xl) : __ceil (xl);
 	  xh = hi;
 	  xl = lo;
 	  ldbl_canonicalize_int (&xh, &xl);
diff --git a/sysdeps/ieee754/ldbl-96/e_lgammal_r.c b/sysdeps/ieee754/ldbl-96/e_lgammal_r.c
index 200421f5cc..a5d6d0dea5 100644
--- a/sysdeps/ieee754/ldbl-96/e_lgammal_r.c
+++ b/sysdeps/ieee754/ldbl-96/e_lgammal_r.c
@@ -221,11 +221,11 @@  sin_pi (long double x)
    * argument reduction, make sure inexact flag not raised if input
    * is an integer
    */
-  z = __floorl (y);
+  z = floorl (y);
   if (z != y)
     {				/* inexact anyway */
       y  *= 0.5;
-      y = 2.0*(y - __floorl(y));		/* y = |x| mod 2.0 */
+      y = 2.0*(y - floorl(y));		/* y = |x| mod 2.0 */
       n = (int) (y*4.0);
     }
   else
diff --git a/sysdeps/ieee754/ldbl-96/lgamma_negl.c b/sysdeps/ieee754/ldbl-96/lgamma_negl.c
index 90bb8345a0..eaa41d6b3a 100644
--- a/sysdeps/ieee754/ldbl-96/lgamma_negl.c
+++ b/sysdeps/ieee754/ldbl-96/lgamma_negl.c
@@ -316,7 +316,7 @@  __lgamma_negl (long double x, int *signgamp)
 {
   /* Determine the half-integer region X lies in, handle exact
      integers and determine the sign of the result.  */
-  int i = __floorl (-2 * x);
+  int i = floorl (-2 * x);
   if ((i & 1) == 0 && i == -2 * x)
     return 1.0L / 0.0L;
   long double xn = ((i & 1) == 0 ? -i / 2 : (-i - 1) / 2);
@@ -333,7 +333,7 @@  __lgamma_negl (long double x, int *signgamp)
      approximations to an adjusted version of the gamma function.  */
   if (i < 2)
     {
-      int j = __floorl (-8 * x) - 16;
+      int j = floorl (-8 * x) - 16;
       long double xm = (-33 - 2 * j) * 0.0625L;
       long double x_adj = x - xm;
       size_t deg = poly_deg[j];
diff --git a/sysdeps/m68k/m680x0/fpu/s_floor_template.c b/sysdeps/m68k/m680x0/fpu/s_floor_template.c
index 7779813d84..1bead5aa3e 100644
--- a/sysdeps/m68k/m680x0/fpu/s_floor_template.c
+++ b/sysdeps/m68k/m680x0/fpu/s_floor_template.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library.  If not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 
 FLOAT
diff --git a/sysdeps/powerpc/fpu/math_private.h b/sysdeps/powerpc/fpu/math_private.h
index 36fcb3c654..dc6dbfa474 100644
--- a/sysdeps/powerpc/fpu/math_private.h
+++ b/sysdeps/powerpc/fpu/math_private.h
@@ -97,26 +97,6 @@  __ieee754_sqrtf128 (_Float128 __x)
      __z; })
 # endif
 
-# ifndef __floor
-#  define __floor(x)			\
-    ({ double __z;			\
-     __asm __volatile (			\
-	"	frim %0,%1\n"		\
-		: "=f" (__z)		\
-		: "f" (x));		\
-     __z; })
-# endif
-# ifndef __floorf
-#  define __floorf(x)			\
-    ({ float __z;			\
-     __asm __volatile (			\
-	"	frim %0,%1\n"		\
-	"	frsp %0,%0\n"		\
-		: "=f" (__z)		\
-		: "f" (x));		\
-     __z; })
-# endif
-
 #endif	/* defined _ARCH_PWR5X */
 
 #endif /* _PPC_MATH_PRIVATE_H_ */
diff --git a/sysdeps/powerpc/power5+/fpu/s_modf.c b/sysdeps/powerpc/power5+/fpu/s_modf.c
index 7a32d31303..3d3b3e7866 100644
--- a/sysdeps/powerpc/power5+/fpu/s_modf.c
+++ b/sysdeps/powerpc/power5+/fpu/s_modf.c
@@ -36,7 +36,7 @@  __modf (double x, double *iptr)
 
   if (x >= 0.0)
     {
-      *iptr = __floor (x);
+      *iptr = floor (x);
       return __copysign (x - *iptr, x);
     }
   else
diff --git a/sysdeps/powerpc/power5+/fpu/s_modff.c b/sysdeps/powerpc/power5+/fpu/s_modff.c
index c1bbae6479..de44218982 100644
--- a/sysdeps/powerpc/power5+/fpu/s_modff.c
+++ b/sysdeps/powerpc/power5+/fpu/s_modff.c
@@ -35,7 +35,7 @@  __modff (float x, float *iptr)
 
   if (x >= 0.0)
     {
-      *iptr = __floorf (x);
+      *iptr = floorf (x);
       return __copysignf (x - *iptr, x);
     }
   else
diff --git a/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floor.c b/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floor.c
index 0152ea606e..5380471af7 100644
--- a/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floor.c
+++ b/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_ldbl_opt.h>
 #include <shlib-compat.h>
diff --git a/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floorf.c b/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floorf.c
index 57ae50c9f6..cd95393e82 100644
--- a/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floorf.c
+++ b/sysdeps/powerpc/powerpc32/power4/fpu/multiarch/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_ldbl_opt.h>
 #include <shlib-compat.h>
diff --git a/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floor.c b/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floor.c
index d1a21a1d12..3b21afc911 100644
--- a/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floor.c
+++ b/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_ldbl_opt.h>
 #include <shlib-compat.h>
diff --git a/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floorf.c b/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floorf.c
index d1bbc14614..7b89aececc 100644
--- a/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floorf.c
+++ b/sysdeps/powerpc/powerpc64/fpu/multiarch/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_ldbl_opt.h>
 #include <shlib-compat.h>
diff --git a/sysdeps/riscv/rv64/rvd/s_floor.c b/sysdeps/riscv/rv64/rvd/s_floor.c
index da3b11cdb4..84a609fd19 100644
--- a/sysdeps/riscv/rv64/rvd/s_floor.c
+++ b/sysdeps/riscv/rv64/rvd/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library.  If not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <fenv_private.h>
diff --git a/sysdeps/riscv/rvf/s_floorf.c b/sysdeps/riscv/rvf/s_floorf.c
index 560553ffba..ba91d37914 100644
--- a/sysdeps/riscv/rvf/s_floorf.c
+++ b/sysdeps/riscv/rvf/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library.  If not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <math.h>
 #include <math_private.h>
 #include <fenv_private.h>
diff --git a/sysdeps/sparc/sparc64/fpu/multiarch/s_floor.c b/sysdeps/sparc/sparc64/fpu/multiarch/s_floor.c
index 6e5ecd2b5d..910e7c1486 100644
--- a/sysdeps/sparc/sparc64/fpu/multiarch/s_floor.c
+++ b/sysdeps/sparc/sparc64/fpu/multiarch/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <sparc-ifunc.h>
 #include <math.h>
 #include <libm-alias-double.h>
diff --git a/sysdeps/sparc/sparc64/fpu/multiarch/s_floorf.c b/sysdeps/sparc/sparc64/fpu/multiarch/s_floorf.c
index b79fd83fa4..788d69cd96 100644
--- a/sysdeps/sparc/sparc64/fpu/multiarch/s_floorf.c
+++ b/sysdeps/sparc/sparc64/fpu/multiarch/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <sparc-ifunc.h>
 #include <math.h>
 #include <libm-alias-float.h>
diff --git a/sysdeps/x86_64/fpu/math_private.h b/sysdeps/x86_64/fpu/math_private.h
index 306875f92b..e5bf46f6b1 100644
--- a/sysdeps/x86_64/fpu/math_private.h
+++ b/sysdeps/x86_64/fpu/math_private.h
@@ -27,30 +27,6 @@  __rintf (float d)
 # endif
   return res;
 }
-
-extern __always_inline double
-__floor (double d)
-{
-  double res;
-# if defined __AVX__ || defined SSE2AVX
-  asm ("vroundsd $1, %1, %0, %0" : "=x" (res) : "xm" (d));
-# else
-  asm ("roundsd $1, %1, %0" : "=x" (res) : "xm" (d));
-# endif
-  return res;
-}
-
-extern __always_inline float
-__floorf (float d)
-{
-  float res;
-# if defined __AVX__ || defined SSE2AVX
-  asm ("vroundss $1, %1, %0, %0" : "=x" (res) : "xm" (d));
-# else
-  asm ("roundss $1, %1, %0" : "=x" (res) : "xm" (d));
-#  endif
-  return res;
-}
 #endif /* __SSE4_1__ */
 
 #endif /* X86_64_MATH_PRIVATE_H */
diff --git a/sysdeps/x86_64/fpu/multiarch/s_floor.c b/sysdeps/x86_64/fpu/multiarch/s_floor.c
index 58f8ed8eaf..2171863609 100644
--- a/sysdeps/x86_64/fpu/multiarch/s_floor.c
+++ b/sysdeps/x86_64/fpu/multiarch/s_floor.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <libm-alias-double.h>
 
 #define floor __redirect_floor
diff --git a/sysdeps/x86_64/fpu/multiarch/s_floorf.c b/sysdeps/x86_64/fpu/multiarch/s_floorf.c
index 5ef2fec2e3..82987974de 100644
--- a/sysdeps/x86_64/fpu/multiarch/s_floorf.c
+++ b/sysdeps/x86_64/fpu/multiarch/s_floorf.c
@@ -16,6 +16,7 @@ 
    License along with the GNU C Library; if not, see
    <http://www.gnu.org/licenses/>.  */
 
+#define NO_MATH_REDIRECT
 #include <libm-alias-float.h>
 
 #define floorf __redirect_floorf