[1/6] ldbl-128: Simplify fma usage
diff mbox

Message ID 20ecdde4-db40-0f8e-63f6-ed50a1875681@linux.vnet.ibm.com
State New
Headers show

Commit Message

Paul E. Murphy Aug. 17, 2016, 7:24 p.m. UTC
On 08/16/2016 01:03 PM, Joseph Myers wrote:
> On Tue, 16 Aug 2016, Paul E. Murphy wrote:
> 
>> Several ldbl-128 files share identical code which does
>> not ammend itself to the macro renaming necessary to
>> support building with another type.
>>
>> This moves the duplicated function mul_split into its
>> own header, and refactors the fma usage into a single
>> selection macro.
> 
> This seems like something that if done for one type, should be done for 
> all types together (optionally also removing the "compiler before GCC 4.6" 
> cases as no longer relevant to building glibc).
> 

This expands it to the users in ldbl-128ibm, dbl-64, and ldbl-96.  Likewise,
the long double mul_split is renamed mul_splitl for clarity.

The new headers are also moved into math/ to reflect the shared usage.

I've verified the generated code for x86_64, ppc64, s390x, and aarch64 libm
has not changed.

Comments

Joseph Myers Aug. 18, 2016, 4:43 p.m. UTC | #1
On Wed, 17 Aug 2016, Paul E. Murphy wrote:

> +#ifndef _MUL_SPLIT_H
> +#define _MUL_SPLIT_H
> +
> +#include <float.h>
> +
> +/* Choose an appropriate fused multiply-add function, if any.  */
> +#ifdef __FP_FAST_FMA
> +# define __FMA __builtin_fma
> +#elif defined FP_FAST_FMA
> +# define __FMA __fma
> +#endif

By testing FP_FAST_FMA here, this header has a nonobvious dependency on 
include ordering: it would quietly behave differently if <math.h>, the 
header that defines FP_FAST_FMA, were not included first.  Such a 
nonobvious quiet dependency is a bad idea; either this header (and 
likewise the one for long double) should include <math.h>, or the tests of 
FP_FAST_FMA and FP_FAST_FMAL should be removed as no longer relevant now 
we require GCC versions recent enough to have built-in fma and predefine 
__FP_FAST_FMA when a fast inline expansion is available.

(The bug is of course theoretical, since (a) the include ordering in the 
patch is correct and (b) the #elif case is dead code because of the 
compiler version requirements.  But we shouldn't introduce headers that 
look suspicious like that.)

Patch
diff mbox

From 39e4d10ddeca3ffcc401cdbb6af690b50c10cc0c Mon Sep 17 00:00:00 2001
From: "Paul E. Murphy" <murphyp@linux.vnet.ibm.com>
Date: Mon, 8 Aug 2016 15:58:28 -0500
Subject: [PATCH 1/5] Merge common usage of mul_split function

A number of files share identical code for the
mul_split function.

This moves the duplicated function mul_split into its
own header, and refactors the fma usage into a single
selection macro.  Likewise, mul_split when used by a
long double implementation is renamed mul_splitl for
clarity.

	* sysdeps/ieee754/ldbl-128/gamma_productl.c:
	(mul_split) Remove, rename as mul_splitl, remove
	redundant float.h include, and include via mul_splitl.h

	* sysdeps/ieee754/ldbl-128/lgamma_productl.c: Likewise.
	* sysdeps/ieee754/ldbl-128/x2y2m1l.c: Likewise.
	* sysdeps/ieee754/ldbl-96/gamma_product.c: Likewise.
	* sysdeps/ieee754/ldbl-96/lgamma_product.c: Likewise.
	* sysdeps/ieee754/ldbl-96/x2y2m1.c: Likewise.

	* math/mul_splitl.h: New file.

	* sysdeps/ieee754/dbl-64/gamma_product.c (mul_split):
	Move into mul_split.h, and remove redundant float.h include.

	* sysdeps/ieee754/dbl-64/lgamma_product.c: Likewise.
	* sysdeps/ieee754/dbl-64/x2y2m1.c: Likewise.
	* sysdeps/ieee754/ldbl-128ibm/s_fmal.c: Likewise.
	* sysdeps/ieee754/ldbl-128ibm/x2y2m1.c: Likewise.

	* math/mul_split.h: New file.
---
 math/mul_split.h                           | 57 ++++++++++++++++++++++++++++++
 math/mul_splitl.h                          | 57 ++++++++++++++++++++++++++++++
 sysdeps/ieee754/dbl-64/gamma_product.c     | 32 +----------------
 sysdeps/ieee754/dbl-64/lgamma_product.c    | 32 +----------------
 sysdeps/ieee754/dbl-64/x2y2m1.c            | 32 +----------------
 sysdeps/ieee754/ldbl-128/gamma_productl.c  | 34 ++----------------
 sysdeps/ieee754/ldbl-128/lgamma_productl.c | 36 ++-----------------
 sysdeps/ieee754/ldbl-128/x2y2m1l.c         | 37 +++----------------
 sysdeps/ieee754/ldbl-128ibm/s_fmal.c       | 27 +-------------
 sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c      | 32 +----------------
 sysdeps/ieee754/ldbl-96/gamma_productl.c   | 34 ++----------------
 sysdeps/ieee754/ldbl-96/lgamma_productl.c  | 36 ++-----------------
 sysdeps/ieee754/ldbl-96/x2y2m1l.c          | 36 ++-----------------
 13 files changed, 136 insertions(+), 346 deletions(-)
 create mode 100644 math/mul_split.h
 create mode 100644 math/mul_splitl.h

diff --git a/math/mul_split.h b/math/mul_split.h
new file mode 100644
index 0000000..8cbf01a
--- /dev/null
+++ b/math/mul_split.h
@@ -0,0 +1,57 @@ 
+/* Compute full X * Y for double type.
+   Copyright (C) 2013-2016 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _MUL_SPLIT_H
+#define _MUL_SPLIT_H
+
+#include <float.h>
+
+/* Choose an appropriate fused multiply-add function, if any.  */
+#ifdef __FP_FAST_FMA
+# define __FMA __builtin_fma
+#elif defined FP_FAST_FMA
+# define __FMA __fma
+#endif
+
+/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
+   given that the values are small enough that no overflow occurs and
+   large enough (or zero) that no underflow occurs.  */
+
+static void
+mul_split (double *hi, double *lo, double x, double y)
+{
+#ifdef __FMA
+  /* Fast built-in fused multiply-add.  */
+  *hi = x * y;
+  *lo = __FMA (x, y, -*hi);
+#else
+  /* Apply Dekker's algorithm.  */
+  *hi = x * y;
+# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
+  double x1 = x * C;
+  double y1 = y * C;
+# undef C
+  x1 = (x - x1) + x1;
+  y1 = (y - y1) + y1;
+  double x2 = x - x1;
+  double y2 = y - y1;
+  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
+#endif
+}
+
+#endif /* _MUL_SPLIT_H */
diff --git a/math/mul_splitl.h b/math/mul_splitl.h
new file mode 100644
index 0000000..fc5c229
--- /dev/null
+++ b/math/mul_splitl.h
@@ -0,0 +1,57 @@ 
+/* Compute full X * Y for long double type.
+   Copyright (C) 2013-2016 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _MUL_SPLITL_H
+#define _MUL_SPLITL_H
+
+#include <float.h>
+
+/* Choose an appropriate fused multiply-add function, if any.  */
+#ifdef __FP_FAST_FMAL
+# define __FMAL __builtin_fmal
+#elif defined FP_FAST_FMAL
+# define __FMAL __fmal
+#endif
+
+/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
+   given that the values are small enough that no overflow occurs and
+   large enough (or zero) that no underflow occurs.  */
+
+static inline void
+mul_splitl (long double *hi, long double *lo, long double x, long double y)
+{
+#ifdef __FMAL
+  /* Fast library fused multiply-add, if supported.  */
+  *hi = x * y;
+  *lo = __FMAL (x, y, -*hi);
+#else
+  /* Apply Dekker's algorithm.  */
+  *hi = x * y;
+# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
+  long double x1 = x * C;
+  long double y1 = y * C;
+# undef C
+  x1 = (x - x1) + x1;
+  y1 = (y - y1) + y1;
+  long double x2 = x - x1;
+  long double y2 = y - y1;
+  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
+#endif
+}
+
+#endif /* _MUL_SPLITL_H */
diff --git a/sysdeps/ieee754/dbl-64/gamma_product.c b/sysdeps/ieee754/dbl-64/gamma_product.c
index 7ae144a..51407b1 100644
--- a/sysdeps/ieee754/dbl-64/gamma_product.c
+++ b/sysdeps/ieee754/dbl-64/gamma_product.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (double *hi, double *lo, double x, double y)
-{
-#ifdef __FP_FAST_FMA
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fma (x, y, -*hi);
-#elif defined FP_FAST_FMA
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fma (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
-  double x1 = x * C;
-  double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  double x2 = x - x1;
-  double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_split.h>
 
 /* Compute the product of X + X_EPS, X + X_EPS + 1, ..., X + X_EPS + N
    - 1, in the form R * (1 + *EPS) where the return value R is an
diff --git a/sysdeps/ieee754/dbl-64/lgamma_product.c b/sysdeps/ieee754/dbl-64/lgamma_product.c
index d956575..fa89337 100644
--- a/sysdeps/ieee754/dbl-64/lgamma_product.c
+++ b/sysdeps/ieee754/dbl-64/lgamma_product.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (double *hi, double *lo, double x, double y)
-{
-#ifdef __FP_FAST_FMA
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fma (x, y, -*hi);
-#elif defined FP_FAST_FMA
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fma (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
-  double x1 = x * C;
-  double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  double x2 = x - x1;
-  double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_split.h>
 
 /* Compute the product of 1 + (T / (X + X_EPS)), 1 + (T / (X + X_EPS +
    1)), ..., 1 + (T / (X + X_EPS + N - 1)), minus 1.  X is such that
diff --git a/sysdeps/ieee754/dbl-64/x2y2m1.c b/sysdeps/ieee754/dbl-64/x2y2m1.c
index 9607888..7505215 100644
--- a/sysdeps/ieee754/dbl-64/x2y2m1.c
+++ b/sysdeps/ieee754/dbl-64/x2y2m1.c
@@ -18,7 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
+#include <mul_split.h>
 #include <stdlib.h>
 
 /* Calculate X + Y exactly and store the result in *HI + *LO.  It is
@@ -33,36 +33,6 @@  add_split (double *hi, double *lo, double x, double y)
   *lo = (x - *hi) + y;
 }
 
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (double *hi, double *lo, double x, double y)
-{
-#ifdef __FP_FAST_FMA
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fma (x, y, -*hi);
-#elif defined FP_FAST_FMA
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fma (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
-  double x1 = x * C;
-  double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  double x2 = x - x1;
-  double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
-
 /* Compare absolute values of floating-point values pointed to by P
    and Q for qsort.  */
 
diff --git a/sysdeps/ieee754/ldbl-128/gamma_productl.c b/sysdeps/ieee754/ldbl-128/gamma_productl.c
index 849b57d..8ad0452 100644
--- a/sysdeps/ieee754/ldbl-128/gamma_productl.c
+++ b/sysdeps/ieee754/ldbl-128/gamma_productl.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static inline void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_splitl.h>
 
 /* Compute the product of X + X_EPS, X + X_EPS + 1, ..., X + X_EPS + N
    - 1, in the form R * (1 + *EPS) where the return value R is an
@@ -68,7 +38,7 @@  __gamma_productl (long double x, long double x_eps, int n, long double *eps)
     {
       *eps += x_eps / (x + i);
       long double lo;
-      mul_split (&ret, &lo, ret, x + i);
+      mul_splitl (&ret, &lo, ret, x + i);
       *eps += lo / ret;
     }
   return ret;
diff --git a/sysdeps/ieee754/ldbl-128/lgamma_productl.c b/sysdeps/ieee754/ldbl-128/lgamma_productl.c
index de67cbe..9aa0137 100644
--- a/sysdeps/ieee754/ldbl-128/lgamma_productl.c
+++ b/sysdeps/ieee754/ldbl-128/lgamma_productl.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_splitl.h>
 
 /* Compute the product of 1 + (T / (X + X_EPS)), 1 + (T / (X + X_EPS +
    1)), ..., 1 + (T / (X + X_EPS + N - 1)), minus 1.  X is such that
@@ -65,11 +35,11 @@  __lgamma_productl (long double t, long double x, long double x_eps, int n)
       long double xi = x + i;
       long double quot = t / xi;
       long double mhi, mlo;
-      mul_split (&mhi, &mlo, quot, xi);
+      mul_splitl (&mhi, &mlo, quot, xi);
       long double quot_lo = (t - mhi - mlo) / xi - t * x_eps / (xi * xi);
       /* We want (1 + RET + RET_EPS) * (1 + QUOT + QUOT_LO) - 1.  */
       long double rhi, rlo;
-      mul_split (&rhi, &rlo, ret, quot);
+      mul_splitl (&rhi, &rlo, ret, quot);
       long double rpq = ret + quot;
       long double rpq_eps = (ret - rpq) + quot;
       long double nret = rpq + rhi;
diff --git a/sysdeps/ieee754/ldbl-128/x2y2m1l.c b/sysdeps/ieee754/ldbl-128/x2y2m1l.c
index 733742d..4e97885 100644
--- a/sysdeps/ieee754/ldbl-128/x2y2m1l.c
+++ b/sysdeps/ieee754/ldbl-128/x2y2m1l.c
@@ -18,9 +18,10 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
+#include <mul_splitl.h>
 #include <stdlib.h>
 
+
 /* Calculate X + Y exactly and store the result in *HI + *LO.  It is
    given that |X| >= |Y| and the values are small enough that no
    overflow occurs.  */
@@ -33,36 +34,6 @@  add_split (long double *hi, long double *lo, long double x, long double y)
   *lo = (x - *hi) + y;
 }
 
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static inline void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
-
 /* Compare absolute values of floating-point values pointed to by P
    and Q for qsort.  */
 
@@ -88,8 +59,8 @@  __x2y2m1l (long double x, long double y)
 {
   long double vals[5];
   SET_RESTORE_ROUNDL (FE_TONEAREST);
-  mul_split (&vals[1], &vals[0], x, x);
-  mul_split (&vals[3], &vals[2], y, y);
+  mul_splitl (&vals[1], &vals[0], x, x);
+  mul_splitl (&vals[3], &vals[2], y, y);
   vals[4] = -1.0L;
   qsort (vals, 5, sizeof (long double), compare);
   /* Add up the values so that each element of VALS has absolute value
diff --git a/sysdeps/ieee754/ldbl-128ibm/s_fmal.c b/sysdeps/ieee754/ldbl-128ibm/s_fmal.c
index 177a048..1405763 100644
--- a/sysdeps/ieee754/ldbl-128ibm/s_fmal.c
+++ b/sysdeps/ieee754/ldbl-128ibm/s_fmal.c
@@ -22,6 +22,7 @@ 
 #include <math.h>
 #include <math_private.h>
 #include <math_ldbl_opt.h>
+#include <mul_split.h>
 #include <stdlib.h>
 
 /* Calculate X + Y exactly and store the result in *HI + *LO.  It is
@@ -36,32 +37,6 @@  add_split (double *hi, double *lo, double x, double y)
   *lo = (x - *hi) + y;
 }
 
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (double *hi, double *lo, double x, double y)
-{
-#ifdef __FP_FAST_FMA
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fma (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
-  double x1 = x * C;
-  double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  double x2 = x - x1;
-  double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
-
 /* Value with extended range, used in intermediate computations.  */
 typedef struct
 {
diff --git a/sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c b/sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c
index da2e929..a8b4895 100644
--- a/sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c
+++ b/sysdeps/ieee754/ldbl-128ibm/x2y2m1l.c
@@ -18,7 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
+#include <mul_split.h>
 #include <stdlib.h>
 
 /* Calculate X + Y exactly and store the result in *HI + *LO.  It is
@@ -33,36 +33,6 @@  add_split (double *hi, double *lo, double x, double y)
   *lo = (x - *hi) + y;
 }
 
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static inline void
-mul_split (double *hi, double *lo, double x, double y)
-{
-#ifdef __FP_FAST_FMA
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fma (x, y, -*hi);
-#elif defined FP_FAST_FMA
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fma (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1 << (DBL_MANT_DIG + 1) / 2) + 1)
-  double x1 = x * C;
-  double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  double x2 = x - x1;
-  double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
-
 /* Compare absolute values of floating-point values pointed to by P
    and Q for qsort.  */
 
diff --git a/sysdeps/ieee754/ldbl-96/gamma_productl.c b/sysdeps/ieee754/ldbl-96/gamma_productl.c
index 849b57d..8ad0452 100644
--- a/sysdeps/ieee754/ldbl-96/gamma_productl.c
+++ b/sysdeps/ieee754/ldbl-96/gamma_productl.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static inline void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_splitl.h>
 
 /* Compute the product of X + X_EPS, X + X_EPS + 1, ..., X + X_EPS + N
    - 1, in the form R * (1 + *EPS) where the return value R is an
@@ -68,7 +38,7 @@  __gamma_productl (long double x, long double x_eps, int n, long double *eps)
     {
       *eps += x_eps / (x + i);
       long double lo;
-      mul_split (&ret, &lo, ret, x + i);
+      mul_splitl (&ret, &lo, ret, x + i);
       *eps += lo / ret;
     }
   return ret;
diff --git a/sysdeps/ieee754/ldbl-96/lgamma_productl.c b/sysdeps/ieee754/ldbl-96/lgamma_productl.c
index de67cbe..9aa0137 100644
--- a/sysdeps/ieee754/ldbl-96/lgamma_productl.c
+++ b/sysdeps/ieee754/ldbl-96/lgamma_productl.c
@@ -18,37 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
-
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
+#include <mul_splitl.h>
 
 /* Compute the product of 1 + (T / (X + X_EPS)), 1 + (T / (X + X_EPS +
    1)), ..., 1 + (T / (X + X_EPS + N - 1)), minus 1.  X is such that
@@ -65,11 +35,11 @@  __lgamma_productl (long double t, long double x, long double x_eps, int n)
       long double xi = x + i;
       long double quot = t / xi;
       long double mhi, mlo;
-      mul_split (&mhi, &mlo, quot, xi);
+      mul_splitl (&mhi, &mlo, quot, xi);
       long double quot_lo = (t - mhi - mlo) / xi - t * x_eps / (xi * xi);
       /* We want (1 + RET + RET_EPS) * (1 + QUOT + QUOT_LO) - 1.  */
       long double rhi, rlo;
-      mul_split (&rhi, &rlo, ret, quot);
+      mul_splitl (&rhi, &rlo, ret, quot);
       long double rpq = ret + quot;
       long double rpq_eps = (ret - rpq) + quot;
       long double nret = rpq + rhi;
diff --git a/sysdeps/ieee754/ldbl-96/x2y2m1l.c b/sysdeps/ieee754/ldbl-96/x2y2m1l.c
index 733742d..b86ba1c 100644
--- a/sysdeps/ieee754/ldbl-96/x2y2m1l.c
+++ b/sysdeps/ieee754/ldbl-96/x2y2m1l.c
@@ -18,7 +18,7 @@ 
 
 #include <math.h>
 #include <math_private.h>
-#include <float.h>
+#include <mul_splitl.h>
 #include <stdlib.h>
 
 /* Calculate X + Y exactly and store the result in *HI + *LO.  It is
@@ -33,36 +33,6 @@  add_split (long double *hi, long double *lo, long double x, long double y)
   *lo = (x - *hi) + y;
 }
 
-/* Calculate X * Y exactly and store the result in *HI + *LO.  It is
-   given that the values are small enough that no overflow occurs and
-   large enough (or zero) that no underflow occurs.  */
-
-static inline void
-mul_split (long double *hi, long double *lo, long double x, long double y)
-{
-#ifdef __FP_FAST_FMAL
-  /* Fast built-in fused multiply-add.  */
-  *hi = x * y;
-  *lo = __builtin_fmal (x, y, -*hi);
-#elif defined FP_FAST_FMAL
-  /* Fast library fused multiply-add, compiler before GCC 4.6.  */
-  *hi = x * y;
-  *lo = __fmal (x, y, -*hi);
-#else
-  /* Apply Dekker's algorithm.  */
-  *hi = x * y;
-# define C ((1LL << (LDBL_MANT_DIG + 1) / 2) + 1)
-  long double x1 = x * C;
-  long double y1 = y * C;
-# undef C
-  x1 = (x - x1) + x1;
-  y1 = (y - y1) + y1;
-  long double x2 = x - x1;
-  long double y2 = y - y1;
-  *lo = (((x1 * y1 - *hi) + x1 * y2) + x2 * y1) + x2 * y2;
-#endif
-}
-
 /* Compare absolute values of floating-point values pointed to by P
    and Q for qsort.  */
 
@@ -88,8 +58,8 @@  __x2y2m1l (long double x, long double y)
 {
   long double vals[5];
   SET_RESTORE_ROUNDL (FE_TONEAREST);
-  mul_split (&vals[1], &vals[0], x, x);
-  mul_split (&vals[3], &vals[2], y, y);
+  mul_splitl (&vals[1], &vals[0], x, x);
+  mul_splitl (&vals[3], &vals[2], y, y);
   vals[4] = -1.0L;
   qsort (vals, 5, sizeof (long double), compare);
   /* Add up the values so that each element of VALS has absolute value
-- 
2.4.11