From patchwork Sun Aug 6 04:25:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 798329 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-82780-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="qWbP5KTJ"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xQ7256kHLz9sDB for ; Sun, 6 Aug 2017 14:28:12 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; q=dns; s=default; b=Zeie1HouKf1dK0XIbx6an9fxHarff nuwiJx9dFA9U9sss9ROyjVOajPtiVlfwxsUpZsNc8AVTMt05+8H63b7LiB8oQWCB sVXLbUV1E1RsolOhTVxahkehZxcmSUyysiUWlYwECq8CD3gsmcM9JiSLwjTHr+SP fzQa83YXCxtFaU= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; s=default; bh=51ny56EzGS4LAZDiNVCaVBPlJ5s=; b=qWb P5KTJgQklvoc2NAa9hfCGhzB79AADLSCKuDcyg/BkEeqsm49ZshskgW/Ydn2LltE w4nStOACzaJh3IHH2wyef+NcKEzWj72ROHgKHesDYVqbJY+jwCQvFOcZz+5NGaai nsoGwIt80SZVV7MWcNcTVIA/9KHsdJC+utDF6XHQ= Received: (qmail 76721 invoked by alias); 6 Aug 2017 04:27:48 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 67746 invoked by uid 89); 6 Aug 2017 04:25:53 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.3 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_STOCKGEN, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-pg0-f65.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=Qqj28fmuCPgppI3NVXeV/jzXezfAKzopol5qQmUlhJ8=; b=J54zi1oyO2h3pH+NHQOUd/dDUB5MRG8gTn/59Nh3mGNwlzfm39WensoiOA6rt7NciN VQ13z5TPMtbtvdl3BFl5EIByiglS9VuSt2xfN5fkYJa1DzR+kaZGfKfH7T2LmL0ydvz8 zqCieWiJgsUT11N/8SDFmzibs7Cl5KzNAXZrHkspEaWauMLiW85MJkjtwyUVyXXrMOYY Mdw/bUqNnANst1Ggu0taa9cBjJb6W5trl71r71unHAOS3CnSxipf/ix2sPzDAYUQp9B8 1Q3aAz7G2IgdJjmcgJrXZ2oK/WQMTQtedCr6A/2WPmf40QsLjxM1c7NX63nxgVjqxTQ8 xdfw== X-Gm-Message-State: AHYfb5gDL+DXz/liQhMD16RyJNsLZSNoNk4fOS4ZNLHUsDgiPn/CKPXk 7KjFFiFlNRjwmm2K X-Received: by 10.98.159.155 with SMTP id v27mr5435873pfk.124.1501993545911; Sat, 05 Aug 2017 21:25:45 -0700 (PDT) Date: Sat, 5 Aug 2017 21:25:44 -0700 From: "H.J. Lu" To: GNU C Library Subject: [PATCH] x86-64: Add FMA multiarch functions to libm Message-ID: <20170806042544.GA11557@gmail.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.8.3 (2017-05-23) This patch adds multiarch functions optimized with -mfma -mavx2 to libm. e_pow-fma.c is compiled with -mno-fma -mavx2 due to PR 19003. Tested on Skylake and Ivybridge. Any comments? H.J. --- * sysdeps/x86_64/fpu/multiarch/Makefile (libm-sysdep_routines): Add e_exp-fma, e_log-fma, e_pow-fma, s_atan-fma, e_asin-fma, e_atan2-fma, s_sin-fma, s_tan-fma, mplog-fma, mpa-fma, slowexp-fma, slowpow-fma, sincos32-fma, doasin-fma, dosincos-fma, halfulp-fma, mpexp-fma, mpatan2-fma, mpatan-fma, mpsqrt-fma, and mptan-fma. (CFLAGS-doasin-fma.c): New. (CFLAGS-dosincos-fma.c): Likewise. (CFLAGS-e_asin-fma.c): Likewise. (CFLAGS-e_atan2-fma.c): Likewise. (CFLAGS-e_exp-fma.c): Likewise. (CFLAGS-e_log-fma.c): Likewise. (CFLAGS-e_pow-fma.c): Likewise. (CFLAGS-halfulp-fma.c): Likewise. (CFLAGS-mpa-fma.c): Likewise. (CFLAGS-mpatan-fma.c): Likewise. (CFLAGS-mpatan2-fma.c): Likewise. (CFLAGS-mpexp-fma.c): Likewise. (CFLAGS-mplog-fma.c): Likewise. (CFLAGS-mpsqrt-fma.c): Likewise. (CFLAGS-mptan-fma.c): Likewise. (CFLAGS-s_atan-fma.c): Likewise. (CFLAGS-sincos32-fma.c): Likewise. (CFLAGS-slowexp-fma.c): Likewise. (CFLAGS-slowpow-fma.c): Likewise. (CFLAGS-s_sin-fma.c): Likewise. (CFLAGS-s_tan-fma.c): Likewise. * sysdeps/x86_64/fpu/multiarch/doasin-fma.c: New file. * sysdeps/x86_64/fpu/multiarch/dosincos-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_asin-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_exp-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_log-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_pow-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/halfulp-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h: Likewise. * sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h: Likewise. * sysdeps/x86_64/fpu/multiarch/mpa-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mpatan-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mpexp-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mplog-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/mptan-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_atan-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_sin-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_tan-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/sincos32-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/slowexp-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/slowpow-fma.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_asin.c: Rewrite. * sysdeps/x86_64/fpu/multiarch/e_atan2.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_exp.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_log.c: Likewise. * sysdeps/x86_64/fpu/multiarch/e_pow.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_atan.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_sin.c: Likewise. * sysdeps/x86_64/fpu/multiarch/s_tan.c: Likewise. --- sysdeps/x86_64/fpu/multiarch/Makefile | 30 +++++++++++++++ sysdeps/x86_64/fpu/multiarch/doasin-fma.c | 4 ++ sysdeps/x86_64/fpu/multiarch/dosincos-fma.c | 6 +++ sysdeps/x86_64/fpu/multiarch/e_asin-fma.c | 11 ++++++ sysdeps/x86_64/fpu/multiarch/e_asin.c | 54 +++++++++++++++++---------- sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c | 10 +++++ sysdeps/x86_64/fpu/multiarch/e_atan2.c | 35 +++++++++++------ sysdeps/x86_64/fpu/multiarch/e_exp-fma.c | 6 +++ sysdeps/x86_64/fpu/multiarch/e_exp.c | 35 +++++++++++------ sysdeps/x86_64/fpu/multiarch/e_log-fma.c | 8 ++++ sysdeps/x86_64/fpu/multiarch/e_log.c | 35 +++++++++++------ sysdeps/x86_64/fpu/multiarch/e_pow-fma.c | 6 +++ sysdeps/x86_64/fpu/multiarch/e_pow.c | 34 +++++++++++------ sysdeps/x86_64/fpu/multiarch/halfulp-fma.c | 4 ++ sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h | 42 +++++++++++++++++++++ sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h | 39 +++++++++++++++++++ sysdeps/x86_64/fpu/multiarch/mpa-fma.c | 14 +++++++ sysdeps/x86_64/fpu/multiarch/mpatan-fma.c | 10 +++++ sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c | 9 +++++ sysdeps/x86_64/fpu/multiarch/mpexp-fma.c | 9 +++++ sysdeps/x86_64/fpu/multiarch/mplog-fma.c | 8 ++++ sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c | 8 ++++ sysdeps/x86_64/fpu/multiarch/mptan-fma.c | 7 ++++ sysdeps/x86_64/fpu/multiarch/s_atan-fma.c | 9 +++++ sysdeps/x86_64/fpu/multiarch/s_atan.c | 30 ++++++++++----- sysdeps/x86_64/fpu/multiarch/s_sin-fma.c | 11 ++++++ sysdeps/x86_64/fpu/multiarch/s_sin.c | 51 +++++++++++++++---------- sysdeps/x86_64/fpu/multiarch/s_tan-fma.c | 8 ++++ sysdeps/x86_64/fpu/multiarch/s_tan.c | 30 ++++++++++----- sysdeps/x86_64/fpu/multiarch/sincos32-fma.c | 15 ++++++++ sysdeps/x86_64/fpu/multiarch/slowexp-fma.c | 9 +++++ sysdeps/x86_64/fpu/multiarch/slowpow-fma.c | 11 ++++++ 32 files changed, 493 insertions(+), 105 deletions(-) create mode 100644 sysdeps/x86_64/fpu/multiarch/doasin-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/dosincos-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_asin-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_exp-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_log-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/e_pow-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/halfulp-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h create mode 100644 sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h create mode 100644 sysdeps/x86_64/fpu/multiarch/mpa-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mpatan-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mpexp-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mplog-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/mptan-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/s_atan-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/s_sin-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/s_tan-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/sincos32-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/slowexp-fma.c create mode 100644 sysdeps/x86_64/fpu/multiarch/slowpow-fma.c diff --git a/sysdeps/x86_64/fpu/multiarch/Makefile b/sysdeps/x86_64/fpu/multiarch/Makefile index f9ceb09a4e..309624960f 100644 --- a/sysdeps/x86_64/fpu/multiarch/Makefile +++ b/sysdeps/x86_64/fpu/multiarch/Makefile @@ -6,6 +6,36 @@ libm-sysdep_routines += s_ceil-sse4_1 s_ceilf-sse4_1 s_floor-sse4_1 \ s_floorf-sse4_1 s_nearbyint-sse4_1 \ s_nearbyintf-sse4_1 s_rint-sse4_1 s_rintf-sse4_1 +libm-sysdep_routines += e_exp-fma e_log-fma e_pow-fma s_atan-fma \ + e_asin-fma e_atan2-fma s_sin-fma s_tan-fma \ + mplog-fma mpa-fma slowexp-fma slowpow-fma \ + sincos32-fma doasin-fma dosincos-fma \ + halfulp-fma mpexp-fma \ + mpatan2-fma mpatan-fma mpsqrt-fma mptan-fma + +CFLAGS-doasin-fma.c = -mfma -mavx2 +CFLAGS-dosincos-fma.c = -mfma -mavx2 +CFLAGS-e_asin-fma.c = -mfma -mavx2 +CFLAGS-e_atan2-fma.c = -mfma -mavx2 +CFLAGS-e_exp-fma.c = -mfma -mavx2 +CFLAGS-e_log-fma.c = -mfma -mavx2 +# FMA is disabled due to [BZ #19003]. +CFLAGS-e_pow-fma.c = -mno-fma -mavx2 +CFLAGS-halfulp-fma.c = -mfma -mavx2 +CFLAGS-mpa-fma.c = -mfma -mavx2 +CFLAGS-mpatan-fma.c = -mfma -mavx2 +CFLAGS-mpatan2-fma.c = -mfma -mavx2 +CFLAGS-mpexp-fma.c = -mfma -mavx2 +CFLAGS-mplog-fma.c = -mfma -mavx2 +CFLAGS-mpsqrt-fma.c = -mfma -mavx2 +CFLAGS-mptan-fma.c = -mfma -mavx2 +CFLAGS-s_atan-fma.c = -mfma -mavx2 +CFLAGS-sincos32-fma.c = -mfma -mavx2 +CFLAGS-slowexp-fma.c = -mfma -mavx2 +CFLAGS-slowpow-fma.c = -mfma -mavx2 +CFLAGS-s_sin-fma.c = -mfma -mavx2 +CFLAGS-s_tan-fma.c = -mfma -mavx2 + libm-sysdep_routines += e_exp-fma4 e_log-fma4 e_pow-fma4 s_atan-fma4 \ e_asin-fma4 e_atan2-fma4 s_sin-fma4 s_tan-fma4 \ mplog-fma4 mpa-fma4 slowexp-fma4 slowpow-fma4 \ diff --git a/sysdeps/x86_64/fpu/multiarch/doasin-fma.c b/sysdeps/x86_64/fpu/multiarch/doasin-fma.c new file mode 100644 index 0000000000..7a09865fca --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/doasin-fma.c @@ -0,0 +1,4 @@ +#define __doasin __doasin_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/dosincos-fma.c b/sysdeps/x86_64/fpu/multiarch/dosincos-fma.c new file mode 100644 index 0000000000..5744586bdb --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/dosincos-fma.c @@ -0,0 +1,6 @@ +#define __docos __docos_fma +#define __dubcos __dubcos_fma +#define __dubsin __dubsin_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_asin-fma.c b/sysdeps/x86_64/fpu/multiarch/e_asin-fma.c new file mode 100644 index 0000000000..50e9c64247 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_asin-fma.c @@ -0,0 +1,11 @@ +#define __ieee754_acos __ieee754_acos_fma +#define __ieee754_asin __ieee754_asin_fma +#define __cos32 __cos32_fma +#define __doasin __doasin_fma +#define __docos __docos_fma +#define __dubcos __dubcos_fma +#define __dubsin __dubsin_fma +#define __sin32 __sin32_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_asin.c b/sysdeps/x86_64/fpu/multiarch/e_asin.c index 111a5b99bd..37a44b2388 100644 --- a/sysdeps/x86_64/fpu/multiarch/e_asin.c +++ b/sysdeps/x86_64/fpu/multiarch/e_asin.c @@ -1,26 +1,40 @@ -#include -#include -#include - -extern double __ieee754_acos_sse2 (double); -extern double __ieee754_asin_sse2 (double); -extern double __ieee754_acos_fma4 (double); -extern double __ieee754_asin_fma4 (double); - -libm_ifunc (__ieee754_acos, - HAS_ARCH_FEATURE (FMA4_Usable) - ? __ieee754_acos_fma4 - : __ieee754_acos_sse2); -strong_alias (__ieee754_acos, __acos_finite) +/* Multiple versions of IEEE 754 asin and acos. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_ieee754_asin (double); +extern double __redirect_ieee754_acos (double); + +#define SYMBOL_NAME ieee754_asin +#include "ifunc-fma4.h" -libm_ifunc (__ieee754_asin, - HAS_ARCH_FEATURE (FMA4_Usable) - ? __ieee754_asin_fma4 - : __ieee754_asin_sse2); +libc_ifunc_redirected (__redirect_ieee754_asin, __ieee754_asin, + IFUNC_SELECTOR ()); strong_alias (__ieee754_asin, __asin_finite) -#define __ieee754_acos __ieee754_acos_sse2 -#define __ieee754_asin __ieee754_asin_sse2 +#undef SYMBOL_NAME +#define SYMBOL_NAME ieee754_acos +#include "ifunc-fma4.h" + +libc_ifunc_redirected (__redirect_ieee754_acos, __ieee754_acos, + IFUNC_SELECTOR ()); +strong_alias (__ieee754_acos, __acos_finite) +#define __ieee754_acos __ieee754_acos_sse2 +#define __ieee754_asin __ieee754_asin_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c b/sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c new file mode 100644 index 0000000000..caba686496 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_atan2-fma.c @@ -0,0 +1,10 @@ +#define __ieee754_atan2 __ieee754_atan2_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __dvd __dvd_fma +#define __mpatan2 __mpatan2_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_atan2.c b/sysdeps/x86_64/fpu/multiarch/e_atan2.c index 9ca3c02a44..a2c8cfc159 100644 --- a/sysdeps/x86_64/fpu/multiarch/e_atan2.c +++ b/sysdeps/x86_64/fpu/multiarch/e_atan2.c @@ -1,18 +1,29 @@ -#include -#include -#include +/* Multiple versions of IEEE 754 atan. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __ieee754_atan2_sse2 (double, double); -extern double __ieee754_atan2_avx (double, double); -extern double __ieee754_atan2_fma4 (double, double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (__ieee754_atan2, - HAS_ARCH_FEATURE (FMA4_Usable) ? __ieee754_atan2_fma4 - : (HAS_ARCH_FEATURE (AVX_Usable) - ? __ieee754_atan2_avx : __ieee754_atan2_sse2)); -strong_alias (__ieee754_atan2, __atan2_finite) + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define __ieee754_atan2 __ieee754_atan2_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_ieee754_atan2 (double, double); +#define SYMBOL_NAME ieee754_atan2 +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_ieee754_atan2, + __ieee754_atan2, IFUNC_SELECTOR ()); +strong_alias (__ieee754_atan2, __atan2_finite) + +#define __ieee754_atan2 __ieee754_atan2_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/e_exp-fma.c b/sysdeps/x86_64/fpu/multiarch/e_exp-fma.c new file mode 100644 index 0000000000..6e0fdb7941 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_exp-fma.c @@ -0,0 +1,6 @@ +#define __ieee754_exp __ieee754_exp_fma +#define __exp1 __exp1_fma +#define __slowexp __slowexp_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_exp.c b/sysdeps/x86_64/fpu/multiarch/e_exp.c index b7d7b5ff27..81211f4ace 100644 --- a/sysdeps/x86_64/fpu/multiarch/e_exp.c +++ b/sysdeps/x86_64/fpu/multiarch/e_exp.c @@ -1,18 +1,29 @@ -#include -#include -#include +/* Multiple versions of IEEE 754 exp. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __ieee754_exp_sse2 (double); -extern double __ieee754_exp_avx (double); -extern double __ieee754_exp_fma4 (double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (__ieee754_exp, - HAS_ARCH_FEATURE (FMA4_Usable) ? __ieee754_exp_fma4 - : (HAS_ARCH_FEATURE (AVX_Usable) - ? __ieee754_exp_avx : __ieee754_exp_sse2)); -strong_alias (__ieee754_exp, __exp_finite) + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define __ieee754_exp __ieee754_exp_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_ieee754_exp (double); +#define SYMBOL_NAME ieee754_exp +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_ieee754_exp, __ieee754_exp, + IFUNC_SELECTOR ()); +strong_alias (__ieee754_exp, __exp_finite) + +#define __ieee754_exp __ieee754_exp_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/e_log-fma.c b/sysdeps/x86_64/fpu/multiarch/e_log-fma.c new file mode 100644 index 0000000000..a7123b1a06 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_log-fma.c @@ -0,0 +1,8 @@ +#define __ieee754_log __ieee754_log_fma +#define __mplog __mplog_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_log.c b/sysdeps/x86_64/fpu/multiarch/e_log.c index cf9533d6c0..de1fbf1696 100644 --- a/sysdeps/x86_64/fpu/multiarch/e_log.c +++ b/sysdeps/x86_64/fpu/multiarch/e_log.c @@ -1,18 +1,29 @@ -#include -#include -#include +/* Multiple versions of IEEE 754 log. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __ieee754_log_sse2 (double); -extern double __ieee754_log_avx (double); -extern double __ieee754_log_fma4 (double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (__ieee754_log, - HAS_ARCH_FEATURE (FMA4_Usable) ? __ieee754_log_fma4 - : (HAS_ARCH_FEATURE (AVX_Usable) - ? __ieee754_log_avx : __ieee754_log_sse2)); -strong_alias (__ieee754_log, __log_finite) + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define __ieee754_log __ieee754_log_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_ieee754_log (double); +#define SYMBOL_NAME ieee754_log +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_ieee754_log, __ieee754_log, + IFUNC_SELECTOR ()); +strong_alias (__ieee754_log, __log_finite) + +#define __ieee754_log __ieee754_log_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/e_pow-fma.c b/sysdeps/x86_64/fpu/multiarch/e_pow-fma.c new file mode 100644 index 0000000000..6fd408342e --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/e_pow-fma.c @@ -0,0 +1,6 @@ +#define __ieee754_pow __ieee754_pow_fma +#define __exp1 __exp1_fma +#define __slowpow __slowpow_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/e_pow.c b/sysdeps/x86_64/fpu/multiarch/e_pow.c index a5c5d89c3e..07a7929d32 100644 --- a/sysdeps/x86_64/fpu/multiarch/e_pow.c +++ b/sysdeps/x86_64/fpu/multiarch/e_pow.c @@ -1,17 +1,29 @@ -#include -#include -#include +/* Multiple versions of IEEE 754 pow. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __ieee754_pow_sse2 (double, double); -extern double __ieee754_pow_fma4 (double, double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (__ieee754_pow, - HAS_ARCH_FEATURE (FMA4_Usable) - ? __ieee754_pow_fma4 - : __ieee754_pow_sse2); -strong_alias (__ieee754_pow, __pow_finite) + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define __ieee754_pow __ieee754_pow_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_ieee754_pow (double, double); +#define SYMBOL_NAME ieee754_pow +#include "ifunc-fma4.h" +libc_ifunc_redirected (__redirect_ieee754_pow, + __ieee754_pow, IFUNC_SELECTOR ()); +strong_alias (__ieee754_pow, __pow_finite) + +#define __ieee754_pow __ieee754_pow_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/halfulp-fma.c b/sysdeps/x86_64/fpu/multiarch/halfulp-fma.c new file mode 100644 index 0000000000..6ca70462ca --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/halfulp-fma.c @@ -0,0 +1,4 @@ +#define __halfulp __halfulp_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h b/sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h new file mode 100644 index 0000000000..6850763f18 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/ifunc-avx-fma4.h @@ -0,0 +1,42 @@ +/* Common definition for ifunc selections optimized with AVX, AVX2/FMA + and FMA4. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (avx) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (fma) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (fma4) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable) + && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)) + + if (CPU_FEATURES_ARCH_P (cpu_features, FMA4_Usable)) + return OPTIMIZE (fma4); + + if (CPU_FEATURES_ARCH_P (cpu_features, AVX_Usable)) + return OPTIMIZE (avx); + + return OPTIMIZE (sse2); +} diff --git a/sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h b/sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h new file mode 100644 index 0000000000..5928984e54 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/ifunc-fma4.h @@ -0,0 +1,39 @@ +/* Common definition for ifunc selections optimized with AVX2/FMA and + FMA4. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (fma) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (fma4) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (void) +{ + const struct cpu_features* cpu_features = __get_cpu_features (); + + if (CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable) + && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)) + return OPTIMIZE (fma); + + if (CPU_FEATURES_ARCH_P (cpu_features, FMA4_Usable)) + return OPTIMIZE (fma4); + + return OPTIMIZE (sse2); +} diff --git a/sysdeps/x86_64/fpu/multiarch/mpa-fma.c b/sysdeps/x86_64/fpu/multiarch/mpa-fma.c new file mode 100644 index 0000000000..177cc2517f --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mpa-fma.c @@ -0,0 +1,14 @@ +#define __add __add_fma +#define __mul __mul_fma +#define __sqr __sqr_fma +#define __sub __sub_fma +#define __dbl_mp __dbl_mp_fma +#define __dvd __dvd_fma + +#define NO___CPY 1 +#define NO___MP_DBL 1 +#define NO___ACR 1 +#define NO__CONST 1 +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mpatan-fma.c b/sysdeps/x86_64/fpu/multiarch/mpatan-fma.c new file mode 100644 index 0000000000..d216f9142d --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mpatan-fma.c @@ -0,0 +1,10 @@ +#define __mpatan __mpatan_fma +#define __add __add_fma +#define __dvd __dvd_fma +#define __mpsqrt __mpsqrt_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define AVOID_MPATAN_H 1 +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c b/sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c new file mode 100644 index 0000000000..98df336f79 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mpatan2-fma.c @@ -0,0 +1,9 @@ +#define __mpatan2 __mpatan2_fma +#define __add __add_fma +#define __dvd __dvd_fma +#define __mpatan __mpatan_fma +#define __mpsqrt __mpsqrt_fma +#define __mul __mul_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mpexp-fma.c b/sysdeps/x86_64/fpu/multiarch/mpexp-fma.c new file mode 100644 index 0000000000..637631ba06 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mpexp-fma.c @@ -0,0 +1,9 @@ +#define __mpexp __mpexp_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __dvd __dvd_fma +#define __mul __mul_fma +#define AVOID_MPEXP_H 1 +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mplog-fma.c b/sysdeps/x86_64/fpu/multiarch/mplog-fma.c new file mode 100644 index 0000000000..645b6b7c57 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mplog-fma.c @@ -0,0 +1,8 @@ +#define __mplog __mplog_fma +#define __add __add_fma +#define __mpexp __mpexp_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c b/sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c new file mode 100644 index 0000000000..44d7a23ae3 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mpsqrt-fma.c @@ -0,0 +1,8 @@ +#define __mpsqrt __mpsqrt_fma +#define __dbl_mp __dbl_mp_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define AVOID_MPSQRT_H 1 +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/mptan-fma.c b/sysdeps/x86_64/fpu/multiarch/mptan-fma.c new file mode 100644 index 0000000000..d1a691413c --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/mptan-fma.c @@ -0,0 +1,7 @@ +#define __mptan __mptan_fma +#define __c32 __c32_fma +#define __dvd __dvd_fma +#define __mpranred __mpranred_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/s_atan-fma.c b/sysdeps/x86_64/fpu/multiarch/s_atan-fma.c new file mode 100644 index 0000000000..bedb3f2053 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/s_atan-fma.c @@ -0,0 +1,9 @@ +#define atan __atan_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __mpatan __mpatan_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/s_atan.c b/sysdeps/x86_64/fpu/multiarch/s_atan.c index 742e95cb96..f81919cbbe 100644 --- a/sysdeps/x86_64/fpu/multiarch/s_atan.c +++ b/sysdeps/x86_64/fpu/multiarch/s_atan.c @@ -1,15 +1,27 @@ -#include -#include +/* Multiple versions of atan. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __atan_sse2 (double); -extern double __atan_avx (double); -extern double __atan_fma4 (double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (atan, (HAS_ARCH_FEATURE (FMA4_Usable) ? __atan_fma4 : - HAS_ARCH_FEATURE (AVX_Usable) - ? __atan_avx : __atan_sse2)); + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define atan __atan_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_atan (double); +#define SYMBOL_NAME atan +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_atan, atan, IFUNC_SELECTOR ()); + +#define atan __atan_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/s_sin-fma.c b/sysdeps/x86_64/fpu/multiarch/s_sin-fma.c new file mode 100644 index 0000000000..15f3c394d5 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/s_sin-fma.c @@ -0,0 +1,11 @@ +#define __cos __cos_fma +#define __sin __sin_fma +#define __docos __docos_fma +#define __dubsin __dubsin_fma +#define __mpcos __mpcos_fma +#define __mpcos1 __mpcos1_fma +#define __mpsin __mpsin_fma +#define __mpsin1 __mpsin1_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/s_sin.c b/sysdeps/x86_64/fpu/multiarch/s_sin.c index 8ffd3e7125..eafc06374f 100644 --- a/sysdeps/x86_64/fpu/multiarch/s_sin.c +++ b/sysdeps/x86_64/fpu/multiarch/s_sin.c @@ -1,26 +1,37 @@ -#include -#include -#undef NAN - -extern double __cos_sse2 (double); -extern double __sin_sse2 (double); -extern double __cos_avx (double); -extern double __sin_avx (double); -extern double __cos_fma4 (double); -extern double __sin_fma4 (double); - -libm_ifunc (__cos, (HAS_ARCH_FEATURE (FMA4_Usable) ? __cos_fma4 : - HAS_ARCH_FEATURE (AVX_Usable) - ? __cos_avx : __cos_sse2)); -weak_alias (__cos, cos) +/* Multiple versions of sin and cos. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_sin (double); +extern double __redirect_cos (double); + +#define SYMBOL_NAME sin +#include "ifunc-avx-fma4.h" -libm_ifunc (__sin, (HAS_ARCH_FEATURE (FMA4_Usable) ? __sin_fma4 : - HAS_ARCH_FEATURE (AVX_Usable) - ? __sin_avx : __sin_sse2)); +libc_ifunc_redirected (__redirect_sin, __sin, IFUNC_SELECTOR ()); weak_alias (__sin, sin) -#define __cos __cos_sse2 -#define __sin __sin_sse2 +#undef SYMBOL_NAME +#define SYMBOL_NAME cos +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_cos, __cos, IFUNC_SELECTOR ()); +weak_alias (__cos, cos) +#define __cos __cos_sse2 +#define __sin __sin_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/s_tan-fma.c b/sysdeps/x86_64/fpu/multiarch/s_tan-fma.c new file mode 100644 index 0000000000..c85f8bceed --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/s_tan-fma.c @@ -0,0 +1,8 @@ +#define tan __tan_fma +#define __dbl_mp __dbl_mp_fma +#define __mpranred __mpranred_fma +#define __mptan __mptan_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/s_tan.c b/sysdeps/x86_64/fpu/multiarch/s_tan.c index 25f3bca07e..96a73811f3 100644 --- a/sysdeps/x86_64/fpu/multiarch/s_tan.c +++ b/sysdeps/x86_64/fpu/multiarch/s_tan.c @@ -1,15 +1,27 @@ -#include -#include +/* Multiple versions of tan. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. -extern double __tan_sse2 (double); -extern double __tan_avx (double); -extern double __tan_fma4 (double); + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. -libm_ifunc (tan, (HAS_ARCH_FEATURE (FMA4_Usable) ? __tan_fma4 : - HAS_ARCH_FEATURE (AVX_Usable) - ? __tan_avx : __tan_sse2)); + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. -#define tan __tan_sse2 + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +extern double __redirect_tan (double); +#define SYMBOL_NAME tan +#include "ifunc-avx-fma4.h" +libc_ifunc_redirected (__redirect_tan, tan, IFUNC_SELECTOR ()); + +#define tan __tan_sse2 #include diff --git a/sysdeps/x86_64/fpu/multiarch/sincos32-fma.c b/sysdeps/x86_64/fpu/multiarch/sincos32-fma.c new file mode 100644 index 0000000000..dcd44bc5e8 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/sincos32-fma.c @@ -0,0 +1,15 @@ +#define __cos32 __cos32_fma +#define __sin32 __sin32_fma +#define __c32 __c32_fma +#define __mpsin __mpsin_fma +#define __mpsin1 __mpsin1_fma +#define __mpcos __mpcos_fma +#define __mpcos1 __mpcos1_fma +#define __mpranred __mpranred_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/slowexp-fma.c b/sysdeps/x86_64/fpu/multiarch/slowexp-fma.c new file mode 100644 index 0000000000..6fffca1a93 --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/slowexp-fma.c @@ -0,0 +1,9 @@ +#define __slowexp __slowexp_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __mpexp __mpexp_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include diff --git a/sysdeps/x86_64/fpu/multiarch/slowpow-fma.c b/sysdeps/x86_64/fpu/multiarch/slowpow-fma.c new file mode 100644 index 0000000000..160ed683ab --- /dev/null +++ b/sysdeps/x86_64/fpu/multiarch/slowpow-fma.c @@ -0,0 +1,11 @@ +#define __slowpow __slowpow_fma +#define __add __add_fma +#define __dbl_mp __dbl_mp_fma +#define __mpexp __mpexp_fma +#define __mplog __mplog_fma +#define __mul __mul_fma +#define __sub __sub_fma +#define __halfulp __halfulp_fma +#define SECTION __attribute__ ((section (".text.fma"))) + +#include