| Message ID | 20231215163635.21673-1-Joe.Ramsay@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | aarch64: Add SIMD attributes to math functions with vector versions | expand |
The 12/15/2023 16:36, Joe Ramsay wrote: > Added annotations for autovec by GCC and GFortran - this enables GCC > >= 9 to autovectorise math calls at -Ofast. The compiler may emit > calls to 'half-width' routines (two-lane single-precision > variants). These have been added in the form of wrappers around the > full-width versions, where the low half of the vector is simply > duplicated. This will perform poorly when one lane triggers the > special-case handler, as there will be a redundant call to the scalar > version, however this is expected to be rare at Ofast. this is ok, but i'd like the public header changes to be in a separate commit from the ABI symbol changes. so in case something goes wrong with compiler autovec (gcc vs glibc abi) the public headers can be reverted independently of the ABI symbols. > --- > Thanks, > Joe > include/libc-symbols.h | 2 + > sysdeps/aarch64/fpu/Versions | 15 +++++ > sysdeps/aarch64/fpu/acosf_advsimd.c | 2 + > sysdeps/aarch64/fpu/advsimd_f32_protos.h | 34 ++++++++++ > sysdeps/aarch64/fpu/asinf_advsimd.c | 2 + > sysdeps/aarch64/fpu/atan2f_advsimd.c | 2 + > sysdeps/aarch64/fpu/atanf_advsimd.c | 2 + > sysdeps/aarch64/fpu/bits/math-vector.h | 66 +++++++++++++++++++ > sysdeps/aarch64/fpu/cosf_advsimd.c | 2 + > sysdeps/aarch64/fpu/exp10f_advsimd.c | 2 + > sysdeps/aarch64/fpu/exp2f_advsimd.c | 2 + > sysdeps/aarch64/fpu/expf_advsimd.c | 2 + > sysdeps/aarch64/fpu/expm1f_advsimd.c | 2 + > .../fpu/finclude/math-vector-fortran.h | 47 +++++++++++++ > sysdeps/aarch64/fpu/log10f_advsimd.c | 2 + > sysdeps/aarch64/fpu/log1pf_advsimd.c | 2 + > sysdeps/aarch64/fpu/log2f_advsimd.c | 2 + > sysdeps/aarch64/fpu/logf_advsimd.c | 2 + > sysdeps/aarch64/fpu/sinf_advsimd.c | 2 + > sysdeps/aarch64/fpu/tanf_advsimd.c | 2 + > sysdeps/aarch64/fpu/v_math.h | 15 +++++ > .../unix/sysv/linux/aarch64/libmvec.abilist | 15 +++++ > 22 files changed, 224 insertions(+) > create mode 100644 sysdeps/aarch64/fpu/advsimd_f32_protos.h > create mode 100644 sysdeps/aarch64/fpu/finclude/math-vector-fortran.h > > diff --git a/include/libc-symbols.h b/include/libc-symbols.h > index 5794614488..a226119295 100644 > --- a/include/libc-symbols.h > +++ b/include/libc-symbols.h > @@ -600,8 +600,10 @@ for linking") > #endif > > #if IS_IN (libmvec) > +# define libmvec_hidden_proto(name, attrs...) hidden_proto (name, ##attrs) > # define libmvec_hidden_def(name) hidden_def (name) > #else > +# define libmvec_hidden_proto(name, attrs...) > # define libmvec_hidden_def(name) > #endif > > diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions > index aaacacaebe..accd101184 100644 > --- a/sysdeps/aarch64/fpu/Versions > +++ b/sysdeps/aarch64/fpu/Versions > @@ -18,47 +18,62 @@ libmvec { > _ZGVsMxv_sinf; > } > GLIBC_2.39 { > + _ZGVnN2v_cosf; > + _ZGVnN2v_expf; > + _ZGVnN2v_logf; > + _ZGVnN2v_sinf; > _ZGVnN4v_acosf; > + _ZGVnN2v_acosf; > _ZGVnN2v_acos; > _ZGVsMxv_acosf; > _ZGVsMxv_acos; > _ZGVnN4v_asinf; > + _ZGVnN2v_asinf; > _ZGVnN2v_asin; > _ZGVsMxv_asinf; > _ZGVsMxv_asin; > _ZGVnN4v_atanf; > + _ZGVnN2v_atanf; > _ZGVnN2v_atan; > _ZGVsMxv_atanf; > _ZGVsMxv_atan; > _ZGVnN4vv_atan2f; > + _ZGVnN2vv_atan2f; > _ZGVnN2vv_atan2; > _ZGVsMxvv_atan2f; > _ZGVsMxvv_atan2; > _ZGVnN4v_exp10f; > + _ZGVnN2v_exp10f; > _ZGVnN2v_exp10; > _ZGVsMxv_exp10f; > _ZGVsMxv_exp10; > _ZGVnN4v_exp2f; > + _ZGVnN2v_exp2f; > _ZGVnN2v_exp2; > _ZGVsMxv_exp2f; > _ZGVsMxv_exp2; > _ZGVnN4v_expm1f; > + _ZGVnN2v_expm1f; > _ZGVnN2v_expm1; > _ZGVsMxv_expm1f; > _ZGVsMxv_expm1; > _ZGVnN4v_log10f; > + _ZGVnN2v_log10f; > _ZGVnN2v_log10; > _ZGVsMxv_log10f; > _ZGVsMxv_log10; > _ZGVnN4v_log1pf; > + _ZGVnN2v_log1pf; > _ZGVnN2v_log1p; > _ZGVsMxv_log1pf; > _ZGVsMxv_log1p; > _ZGVnN4v_log2f; > + _ZGVnN2v_log2f; > _ZGVnN2v_log2; > _ZGVsMxv_log2f; > _ZGVsMxv_log2; > _ZGVnN4v_tanf; > + _ZGVnN2v_tanf; > _ZGVnN2v_tan; > _ZGVsMxv_tanf; > _ZGVsMxv_tan; > diff --git a/sysdeps/aarch64/fpu/acosf_advsimd.c b/sysdeps/aarch64/fpu/acosf_advsimd.c > index 7d39e9b805..e28c200a96 100644 > --- a/sysdeps/aarch64/fpu/acosf_advsimd.c > +++ b/sysdeps/aarch64/fpu/acosf_advsimd.c > @@ -111,3 +111,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (acos) (float32x4_t x) > > return vfmaq_f32 (add, mul, y); > } > +libmvec_hidden_def (V_NAME_F1(acos)) > +HALF_WIDTH_ALIAS_F1 (acos) > diff --git a/sysdeps/aarch64/fpu/advsimd_f32_protos.h b/sysdeps/aarch64/fpu/advsimd_f32_protos.h > new file mode 100644 > index 0000000000..b406ad7156 > --- /dev/null > +++ b/sysdeps/aarch64/fpu/advsimd_f32_protos.h > @@ -0,0 +1,34 @@ > +/* Hidden prototypes for single-precision AdvSIMD routines > + > + Copyright (C) 2023 Free Software Foundation, Inc. > + This file is part of the GNU C Library. > + > + The GNU C Library is free software; you can redistribute it and/or > + modify it under the terms of the GNU Lesser General Public > + License as published by the Free Software Foundation; either > + version 2.1 of the License, or (at your option) any later version. > + > + The GNU C Library is distributed in the hope that it will be useful, > + but WITHOUT ANY WARRANTY; without even the implied warranty of > + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > + Lesser General Public License for more details. > + > + You should have received a copy of the GNU Lesser General Public > + License along with the GNU C Library; if not, see > + <https://www.gnu.org/licenses/>. */ > + > +libmvec_hidden_proto (V_NAME_F1(acos)); > +libmvec_hidden_proto (V_NAME_F1(asin)); > +libmvec_hidden_proto (V_NAME_F1(atan)); > +libmvec_hidden_proto (V_NAME_F1(cos)); > +libmvec_hidden_proto (V_NAME_F1(exp10)); > +libmvec_hidden_proto (V_NAME_F1(exp2)); > +libmvec_hidden_proto (V_NAME_F1(exp)); > +libmvec_hidden_proto (V_NAME_F1(expm1)); > +libmvec_hidden_proto (V_NAME_F1(log10)); > +libmvec_hidden_proto (V_NAME_F1(log1p)); > +libmvec_hidden_proto (V_NAME_F1(log2)); > +libmvec_hidden_proto (V_NAME_F1(log)); > +libmvec_hidden_proto (V_NAME_F1(sin)); > +libmvec_hidden_proto (V_NAME_F1(tan)); > +libmvec_hidden_proto (V_NAME_F2(atan2)); > diff --git a/sysdeps/aarch64/fpu/asinf_advsimd.c b/sysdeps/aarch64/fpu/asinf_advsimd.c > index 3180ae7c8e..842037fd56 100644 > --- a/sysdeps/aarch64/fpu/asinf_advsimd.c > +++ b/sysdeps/aarch64/fpu/asinf_advsimd.c > @@ -102,3 +102,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (asin) (float32x4_t x) > /* Copy sign. */ > return vbslq_f32 (v_u32 (AbsMask), y, x); > } > +libmvec_hidden_def (V_NAME_F1 (asin)) > +HALF_WIDTH_ALIAS_F1 (asin) > diff --git a/sysdeps/aarch64/fpu/atan2f_advsimd.c b/sysdeps/aarch64/fpu/atan2f_advsimd.c > index 5a5a6202d1..fa161b1458 100644 > --- a/sysdeps/aarch64/fpu/atan2f_advsimd.c > +++ b/sysdeps/aarch64/fpu/atan2f_advsimd.c > @@ -114,3 +114,5 @@ float32x4_t VPCS_ATTR V_NAME_F2 (atan2) (float32x4_t y, float32x4_t x) > > return ret; > } > +libmvec_hidden_def (V_NAME_F2 (atan2)) > +HALF_WIDTH_ALIAS_F2(atan2) > diff --git a/sysdeps/aarch64/fpu/atanf_advsimd.c b/sysdeps/aarch64/fpu/atanf_advsimd.c > index 589b0e8c96..3c08c51965 100644 > --- a/sysdeps/aarch64/fpu/atanf_advsimd.c > +++ b/sysdeps/aarch64/fpu/atanf_advsimd.c > @@ -107,3 +107,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (atan) (float32x4_t x) > > return y; > } > +libmvec_hidden_def (V_NAME_F1 (atan)) > +HALF_WIDTH_ALIAS_F1 (atan) > diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h > index 52aad95e3b..67dc65bb76 100644 > --- a/sysdeps/aarch64/fpu/bits/math-vector.h > +++ b/sysdeps/aarch64/fpu/bits/math-vector.h > @@ -25,6 +25,72 @@ > /* Get default empty definitions for simd declarations. */ > #include <bits/libm-simd-decl-stubs.h> > > +#if defined __aarch64__ && defined __FAST_MATH__ && __GNUC_PREREQ (6, 0) > +/* Requires GCC >= 6 for __attribute__ ((__simd__)). */ > +# define __DECL_SIMD_aarch64 __attribute__ ((__simd__ ("notinbranch"), const)) > + > +# undef __DECL_SIMD_acos > +# define __DECL_SIMD_acos __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_acosf > +# define __DECL_SIMD_acosf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_asin > +# define __DECL_SIMD_asin __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_asinf > +# define __DECL_SIMD_asinf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_atan > +# define __DECL_SIMD_atan __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_atanf > +# define __DECL_SIMD_atanf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_atan2 > +# define __DECL_SIMD_atan2 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_atan2f > +# define __DECL_SIMD_atan2f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_cos > +# define __DECL_SIMD_cos __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_cosf > +# define __DECL_SIMD_cosf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_exp > +# define __DECL_SIMD_exp __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_expf > +# define __DECL_SIMD_expf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_exp10 > +# define __DECL_SIMD_exp10 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_exp10f > +# define __DECL_SIMD_exp10f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_exp2 > +# define __DECL_SIMD_exp2 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_exp2f > +# define __DECL_SIMD_exp2f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_expm1 > +# define __DECL_SIMD_expm1 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_expm1f > +# define __DECL_SIMD_expm1f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log > +# define __DECL_SIMD_log __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_logf > +# define __DECL_SIMD_logf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log10 > +# define __DECL_SIMD_log10 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log10f > +# define __DECL_SIMD_log10f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log1p > +# define __DECL_SIMD_log1p __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log1pf > +# define __DECL_SIMD_log1pf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log2 > +# define __DECL_SIMD_log2 __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_log2f > +# define __DECL_SIMD_log2f __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_sin > +# define __DECL_SIMD_sin __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_sinf > +# define __DECL_SIMD_sinf __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_tan > +# define __DECL_SIMD_tan __DECL_SIMD_aarch64 > +# undef __DECL_SIMD_tanf > +# define __DECL_SIMD_tanf __DECL_SIMD_aarch64 > +#endif > + > #if __GNUC_PREREQ(9, 0) > # define __ADVSIMD_VEC_MATH_SUPPORTED > typedef __Float32x4_t __f32x4_t; > diff --git a/sysdeps/aarch64/fpu/cosf_advsimd.c b/sysdeps/aarch64/fpu/cosf_advsimd.c > index f05dd2bcda..9f82e7c5d6 100644 > --- a/sysdeps/aarch64/fpu/cosf_advsimd.c > +++ b/sysdeps/aarch64/fpu/cosf_advsimd.c > @@ -92,3 +92,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (cos) (float32x4_t x) > return special_case (x, y, odd, cmp); > return vreinterpretq_f32_u32 (veorq_u32 (vreinterpretq_u32_f32 (y), odd)); > } > +libmvec_hidden_def (V_NAME_F1 (cos)) > +HALF_WIDTH_ALIAS_F1 (cos) > diff --git a/sysdeps/aarch64/fpu/exp10f_advsimd.c b/sysdeps/aarch64/fpu/exp10f_advsimd.c > index 9e754c46fa..10c9dedad7 100644 > --- a/sysdeps/aarch64/fpu/exp10f_advsimd.c > +++ b/sysdeps/aarch64/fpu/exp10f_advsimd.c > @@ -138,3 +138,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp10) (float32x4_t x) > > return vfmaq_f32 (scale, poly, scale); > } > +libmvec_hidden_def (V_NAME_F1 (exp10)) > +HALF_WIDTH_ALIAS_F1 (exp10) > diff --git a/sysdeps/aarch64/fpu/exp2f_advsimd.c b/sysdeps/aarch64/fpu/exp2f_advsimd.c > index 70b3ab66c1..cf84a5b9cd 100644 > --- a/sysdeps/aarch64/fpu/exp2f_advsimd.c > +++ b/sysdeps/aarch64/fpu/exp2f_advsimd.c > @@ -122,3 +122,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp2) (float32x4_t x) > > return vfmaq_f32 (scale, poly, scale); > } > +libmvec_hidden_def (V_NAME_F1 (exp2)) > +HALF_WIDTH_ALIAS_F1 (exp2) > diff --git a/sysdeps/aarch64/fpu/expf_advsimd.c b/sysdeps/aarch64/fpu/expf_advsimd.c > index 69d5d1ea77..d70d1add95 100644 > --- a/sysdeps/aarch64/fpu/expf_advsimd.c > +++ b/sysdeps/aarch64/fpu/expf_advsimd.c > @@ -131,3 +131,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp) (float32x4_t x) > > return vfmaq_f32 (scale, poly, scale); > } > +libmvec_hidden_def (V_NAME_F1 (exp)) > +HALF_WIDTH_ALIAS_F1 (exp) > diff --git a/sysdeps/aarch64/fpu/expm1f_advsimd.c b/sysdeps/aarch64/fpu/expm1f_advsimd.c > index b27b75068a..e3b6e1b01b 100644 > --- a/sysdeps/aarch64/fpu/expm1f_advsimd.c > +++ b/sysdeps/aarch64/fpu/expm1f_advsimd.c > @@ -115,3 +115,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (expm1) (float32x4_t x) > /* expm1(x) ~= p * t + (t - 1). */ > return vfmaq_f32 (vsubq_f32 (t, v_f32 (1.0f)), p, t); > } > +libmvec_hidden_def (V_NAME_F1 (expm1)) > +HALF_WIDTH_ALIAS_F1 (expm1) > diff --git a/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h b/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h > new file mode 100644 > index 0000000000..af489f4769 > --- /dev/null > +++ b/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h > @@ -0,0 +1,47 @@ > +! Platform-specific declarations of SIMD math functions for Fortran. -*- f90 -*- > +! Copyright (C) 2019-2023 Free Software Foundation, Inc. > +! This file is part of the GNU C Library. > +! > +! The GNU C Library is free software; you can redistribute it and/or > +! modify it under the terms of the GNU Lesser General Public > +! License as published by the Free Software Foundation; either > +! version 2.1 of the License, or (at your option) any later version. > +! > +! The GNU C Library is distributed in the hope that it will be useful, > +! but WITHOUT ANY WARRANTY; without even the implied warranty of > +! MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU > +! Lesser General Public License for more details. > +! > +! You should have received a copy of the GNU Lesser General Public > +! License along with the GNU C Library; if not, see > +! <https://www.gnu.org/licenses/>. > +!GCC$ builtin (acos) attributes simd (notinbranch) > +!GCC$ builtin (acosf) attributes simd (notinbranch) > +!GCC$ builtin (asin) attributes simd (notinbranch) > +!GCC$ builtin (asinf) attributes simd (notinbranch) > +!GCC$ builtin (atan) attributes simd (notinbranch) > +!GCC$ builtin (atanf) attributes simd (notinbranch) > +!GCC$ builtin (atan2) attributes simd (notinbranch) > +!GCC$ builtin (atan2f) attributes simd (notinbranch) > +!GCC$ builtin (cos) attributes simd (notinbranch) > +!GCC$ builtin (cosf) attributes simd (notinbranch) > +!GCC$ builtin (exp) attributes simd (notinbranch) > +!GCC$ builtin (expf) attributes simd (notinbranch) > +!GCC$ builtin (exp10) attributes simd (notinbranch) > +!GCC$ builtin (exp10f) attributes simd (notinbranch) > +!GCC$ builtin (exp2) attributes simd (notinbranch) > +!GCC$ builtin (exp2f) attributes simd (notinbranch) > +!GCC$ builtin (expm1) attributes simd (notinbranch) > +!GCC$ builtin (expm1f) attributes simd (notinbranch) > +!GCC$ builtin (log) attributes simd (notinbranch) > +!GCC$ builtin (logf) attributes simd (notinbranch) > +!GCC$ builtin (log10) attributes simd (notinbranch) > +!GCC$ builtin (log10f) attributes simd (notinbranch) > +!GCC$ builtin (log1p) attributes simd (notinbranch) > +!GCC$ builtin (log1pf) attributes simd (notinbranch) > +!GCC$ builtin (log2) attributes simd (notinbranch) > +!GCC$ builtin (log2f) attributes simd (notinbranch) > +!GCC$ builtin (sin) attributes simd (notinbranch) > +!GCC$ builtin (sinf) attributes simd (notinbranch) > +!GCC$ builtin (tan) attributes simd (notinbranch) > +!GCC$ builtin (tanf) attributes simd (notinbranch) > diff --git a/sysdeps/aarch64/fpu/log10f_advsimd.c b/sysdeps/aarch64/fpu/log10f_advsimd.c > index ba02060bbe..4c4e7cb2f4 100644 > --- a/sysdeps/aarch64/fpu/log10f_advsimd.c > +++ b/sysdeps/aarch64/fpu/log10f_advsimd.c > @@ -80,3 +80,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log10) (float32x4_t x) > return special_case (x, y, poly, r2, special); > return vfmaq_f32 (y, poly, r2); > } > +libmvec_hidden_def (V_NAME_F1 (log10)) > +HALF_WIDTH_ALIAS_F1 (log10) > diff --git a/sysdeps/aarch64/fpu/log1pf_advsimd.c b/sysdeps/aarch64/fpu/log1pf_advsimd.c > index 3748830de8..0530dc2002 100644 > --- a/sysdeps/aarch64/fpu/log1pf_advsimd.c > +++ b/sysdeps/aarch64/fpu/log1pf_advsimd.c > @@ -126,3 +126,5 @@ VPCS_ATTR float32x4_t V_NAME_F1 (log1p) (float32x4_t x) > return special_case (special_arg, y, special_cases); > return y; > } > +libmvec_hidden_def (V_NAME_F1 (log1p)) > +HALF_WIDTH_ALIAS_F1 (log1p) > diff --git a/sysdeps/aarch64/fpu/log2f_advsimd.c b/sysdeps/aarch64/fpu/log2f_advsimd.c > index e913bcda18..4049db242e 100644 > --- a/sysdeps/aarch64/fpu/log2f_advsimd.c > +++ b/sysdeps/aarch64/fpu/log2f_advsimd.c > @@ -75,3 +75,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log2) (float32x4_t x) > return special_case (x, n, p, r, special); > return vfmaq_f32 (n, p, r); > } > +libmvec_hidden_def (V_NAME_F1 (log2)) > +HALF_WIDTH_ALIAS_F1 (log2) > diff --git a/sysdeps/aarch64/fpu/logf_advsimd.c b/sysdeps/aarch64/fpu/logf_advsimd.c > index 93903c7962..930e4b9c8f 100644 > --- a/sysdeps/aarch64/fpu/logf_advsimd.c > +++ b/sysdeps/aarch64/fpu/logf_advsimd.c > @@ -83,3 +83,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log) (float32x4_t x) > return special_case (x, y, r2, p, cmp); > return vfmaq_f32 (p, y, r2); > } > +libmvec_hidden_def (V_NAME_F1 (log)) > +HALF_WIDTH_ALIAS_F1 (log) > diff --git a/sysdeps/aarch64/fpu/sinf_advsimd.c b/sysdeps/aarch64/fpu/sinf_advsimd.c > index 0e78cf55f0..3638ab5508 100644 > --- a/sysdeps/aarch64/fpu/sinf_advsimd.c > +++ b/sysdeps/aarch64/fpu/sinf_advsimd.c > @@ -92,3 +92,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (sin) (float32x4_t x) > return special_case (x, y, odd, cmp); > return vreinterpretq_f32_u32 (veorq_u32 (vreinterpretq_u32_f32 (y), odd)); > } > +libmvec_hidden_def (V_NAME_F1 (sin)) > +HALF_WIDTH_ALIAS_F1 (sin) > diff --git a/sysdeps/aarch64/fpu/tanf_advsimd.c b/sysdeps/aarch64/fpu/tanf_advsimd.c > index 4c8a7f740e..dbc7b4dd6e 100644 > --- a/sysdeps/aarch64/fpu/tanf_advsimd.c > +++ b/sysdeps/aarch64/fpu/tanf_advsimd.c > @@ -127,3 +127,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (tan) (float32x4_t x) > return special_case (special_arg, vbslq_f32 (pred_alt, inv_y, y), special); > return vbslq_f32 (pred_alt, inv_y, y); > } > +libmvec_hidden_def (V_NAME_F1 (tan)) > +HALF_WIDTH_ALIAS_F1 (tan) > diff --git a/sysdeps/aarch64/fpu/v_math.h b/sysdeps/aarch64/fpu/v_math.h > index d286eb81b3..e8ac0e2332 100644 > --- a/sysdeps/aarch64/fpu/v_math.h > +++ b/sysdeps/aarch64/fpu/v_math.h > @@ -29,6 +29,21 @@ > #define V_NAME_F2(fun) _ZGVnN4vv_##fun##f > #define V_NAME_D2(fun) _ZGVnN2vv_##fun > > +#include "advsimd_f32_protos.h" > + > +#define HALF_WIDTH_ALIAS_F1(fun) \ > + float32x2_t VPCS_ATTR _ZGVnN2v_##fun##f (float32x2_t x) \ > + { \ > + return vget_low_f32 (_ZGVnN4v_##fun##f (vcombine_f32 (x, x))); \ > + } > + > +#define HALF_WIDTH_ALIAS_F2(fun) \ > + float32x2_t VPCS_ATTR _ZGVnN2vv_##fun##f (float32x2_t x, float32x2_t y) \ > + { \ > + return vget_low_f32 ( \ > + _ZGVnN4vv_##fun##f (vcombine_f32 (x, x), vcombine_f32 (y, y))); \ > + } > + > /* Shorthand helpers for declaring constants. */ > #define V2(X) { X, X } > #define V4(X) { X, X, X, X } > diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist > index 2bf4ea6332..580952b4de 100644 > --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist > +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist > @@ -15,16 +15,31 @@ GLIBC_2.38 _ZGVsMxv_logf F > GLIBC_2.38 _ZGVsMxv_sin F > GLIBC_2.38 _ZGVsMxv_sinf F > GLIBC_2.39 _ZGVnN2v_acos F > +GLIBC_2.39 _ZGVnN2v_acosf F > GLIBC_2.39 _ZGVnN2v_asin F > +GLIBC_2.39 _ZGVnN2v_asinf F > GLIBC_2.39 _ZGVnN2v_atan F > +GLIBC_2.39 _ZGVnN2v_atanf F > +GLIBC_2.39 _ZGVnN2v_cosf F > GLIBC_2.39 _ZGVnN2v_exp10 F > +GLIBC_2.39 _ZGVnN2v_exp10f F > GLIBC_2.39 _ZGVnN2v_exp2 F > +GLIBC_2.39 _ZGVnN2v_exp2f F > +GLIBC_2.39 _ZGVnN2v_expf F > GLIBC_2.39 _ZGVnN2v_expm1 F > +GLIBC_2.39 _ZGVnN2v_expm1f F > GLIBC_2.39 _ZGVnN2v_log10 F > +GLIBC_2.39 _ZGVnN2v_log10f F > GLIBC_2.39 _ZGVnN2v_log1p F > +GLIBC_2.39 _ZGVnN2v_log1pf F > GLIBC_2.39 _ZGVnN2v_log2 F > +GLIBC_2.39 _ZGVnN2v_log2f F > +GLIBC_2.39 _ZGVnN2v_logf F > +GLIBC_2.39 _ZGVnN2v_sinf F > GLIBC_2.39 _ZGVnN2v_tan F > +GLIBC_2.39 _ZGVnN2v_tanf F > GLIBC_2.39 _ZGVnN2vv_atan2 F > +GLIBC_2.39 _ZGVnN2vv_atan2f F > GLIBC_2.39 _ZGVnN4v_acosf F > GLIBC_2.39 _ZGVnN4v_asinf F > GLIBC_2.39 _ZGVnN4v_atanf F > -- > 2.27.0 >
diff --git a/include/libc-symbols.h b/include/libc-symbols.h index 5794614488..a226119295 100644 --- a/include/libc-symbols.h +++ b/include/libc-symbols.h @@ -600,8 +600,10 @@ for linking") #endif #if IS_IN (libmvec) +# define libmvec_hidden_proto(name, attrs...) hidden_proto (name, ##attrs) # define libmvec_hidden_def(name) hidden_def (name) #else +# define libmvec_hidden_proto(name, attrs...) # define libmvec_hidden_def(name) #endif diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index aaacacaebe..accd101184 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,47 +18,62 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN2v_cosf; + _ZGVnN2v_expf; + _ZGVnN2v_logf; + _ZGVnN2v_sinf; _ZGVnN4v_acosf; + _ZGVnN2v_acosf; _ZGVnN2v_acos; _ZGVsMxv_acosf; _ZGVsMxv_acos; _ZGVnN4v_asinf; + _ZGVnN2v_asinf; _ZGVnN2v_asin; _ZGVsMxv_asinf; _ZGVsMxv_asin; _ZGVnN4v_atanf; + _ZGVnN2v_atanf; _ZGVnN2v_atan; _ZGVsMxv_atanf; _ZGVsMxv_atan; _ZGVnN4vv_atan2f; + _ZGVnN2vv_atan2f; _ZGVnN2vv_atan2; _ZGVsMxvv_atan2f; _ZGVsMxvv_atan2; _ZGVnN4v_exp10f; + _ZGVnN2v_exp10f; _ZGVnN2v_exp10; _ZGVsMxv_exp10f; _ZGVsMxv_exp10; _ZGVnN4v_exp2f; + _ZGVnN2v_exp2f; _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; _ZGVnN4v_expm1f; + _ZGVnN2v_expm1f; _ZGVnN2v_expm1; _ZGVsMxv_expm1f; _ZGVsMxv_expm1; _ZGVnN4v_log10f; + _ZGVnN2v_log10f; _ZGVnN2v_log10; _ZGVsMxv_log10f; _ZGVsMxv_log10; _ZGVnN4v_log1pf; + _ZGVnN2v_log1pf; _ZGVnN2v_log1p; _ZGVsMxv_log1pf; _ZGVsMxv_log1p; _ZGVnN4v_log2f; + _ZGVnN2v_log2f; _ZGVnN2v_log2; _ZGVsMxv_log2f; _ZGVsMxv_log2; _ZGVnN4v_tanf; + _ZGVnN2v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; _ZGVsMxv_tan; diff --git a/sysdeps/aarch64/fpu/acosf_advsimd.c b/sysdeps/aarch64/fpu/acosf_advsimd.c index 7d39e9b805..e28c200a96 100644 --- a/sysdeps/aarch64/fpu/acosf_advsimd.c +++ b/sysdeps/aarch64/fpu/acosf_advsimd.c @@ -111,3 +111,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (acos) (float32x4_t x) return vfmaq_f32 (add, mul, y); } +libmvec_hidden_def (V_NAME_F1(acos)) +HALF_WIDTH_ALIAS_F1 (acos) diff --git a/sysdeps/aarch64/fpu/advsimd_f32_protos.h b/sysdeps/aarch64/fpu/advsimd_f32_protos.h new file mode 100644 index 0000000000..b406ad7156 --- /dev/null +++ b/sysdeps/aarch64/fpu/advsimd_f32_protos.h @@ -0,0 +1,34 @@ +/* Hidden prototypes for single-precision AdvSIMD routines + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +libmvec_hidden_proto (V_NAME_F1(acos)); +libmvec_hidden_proto (V_NAME_F1(asin)); +libmvec_hidden_proto (V_NAME_F1(atan)); +libmvec_hidden_proto (V_NAME_F1(cos)); +libmvec_hidden_proto (V_NAME_F1(exp10)); +libmvec_hidden_proto (V_NAME_F1(exp2)); +libmvec_hidden_proto (V_NAME_F1(exp)); +libmvec_hidden_proto (V_NAME_F1(expm1)); +libmvec_hidden_proto (V_NAME_F1(log10)); +libmvec_hidden_proto (V_NAME_F1(log1p)); +libmvec_hidden_proto (V_NAME_F1(log2)); +libmvec_hidden_proto (V_NAME_F1(log)); +libmvec_hidden_proto (V_NAME_F1(sin)); +libmvec_hidden_proto (V_NAME_F1(tan)); +libmvec_hidden_proto (V_NAME_F2(atan2)); diff --git a/sysdeps/aarch64/fpu/asinf_advsimd.c b/sysdeps/aarch64/fpu/asinf_advsimd.c index 3180ae7c8e..842037fd56 100644 --- a/sysdeps/aarch64/fpu/asinf_advsimd.c +++ b/sysdeps/aarch64/fpu/asinf_advsimd.c @@ -102,3 +102,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (asin) (float32x4_t x) /* Copy sign. */ return vbslq_f32 (v_u32 (AbsMask), y, x); } +libmvec_hidden_def (V_NAME_F1 (asin)) +HALF_WIDTH_ALIAS_F1 (asin) diff --git a/sysdeps/aarch64/fpu/atan2f_advsimd.c b/sysdeps/aarch64/fpu/atan2f_advsimd.c index 5a5a6202d1..fa161b1458 100644 --- a/sysdeps/aarch64/fpu/atan2f_advsimd.c +++ b/sysdeps/aarch64/fpu/atan2f_advsimd.c @@ -114,3 +114,5 @@ float32x4_t VPCS_ATTR V_NAME_F2 (atan2) (float32x4_t y, float32x4_t x) return ret; } +libmvec_hidden_def (V_NAME_F2 (atan2)) +HALF_WIDTH_ALIAS_F2(atan2) diff --git a/sysdeps/aarch64/fpu/atanf_advsimd.c b/sysdeps/aarch64/fpu/atanf_advsimd.c index 589b0e8c96..3c08c51965 100644 --- a/sysdeps/aarch64/fpu/atanf_advsimd.c +++ b/sysdeps/aarch64/fpu/atanf_advsimd.c @@ -107,3 +107,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (atan) (float32x4_t x) return y; } +libmvec_hidden_def (V_NAME_F1 (atan)) +HALF_WIDTH_ALIAS_F1 (atan) diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 52aad95e3b..67dc65bb76 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -25,6 +25,72 @@ /* Get default empty definitions for simd declarations. */ #include <bits/libm-simd-decl-stubs.h> +#if defined __aarch64__ && defined __FAST_MATH__ && __GNUC_PREREQ (6, 0) +/* Requires GCC >= 6 for __attribute__ ((__simd__)). */ +# define __DECL_SIMD_aarch64 __attribute__ ((__simd__ ("notinbranch"), const)) + +# undef __DECL_SIMD_acos +# define __DECL_SIMD_acos __DECL_SIMD_aarch64 +# undef __DECL_SIMD_acosf +# define __DECL_SIMD_acosf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_asin +# define __DECL_SIMD_asin __DECL_SIMD_aarch64 +# undef __DECL_SIMD_asinf +# define __DECL_SIMD_asinf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_atan +# define __DECL_SIMD_atan __DECL_SIMD_aarch64 +# undef __DECL_SIMD_atanf +# define __DECL_SIMD_atanf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_atan2 +# define __DECL_SIMD_atan2 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_atan2f +# define __DECL_SIMD_atan2f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_cos +# define __DECL_SIMD_cos __DECL_SIMD_aarch64 +# undef __DECL_SIMD_cosf +# define __DECL_SIMD_cosf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_exp +# define __DECL_SIMD_exp __DECL_SIMD_aarch64 +# undef __DECL_SIMD_expf +# define __DECL_SIMD_expf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_exp10 +# define __DECL_SIMD_exp10 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_exp10f +# define __DECL_SIMD_exp10f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_exp2 +# define __DECL_SIMD_exp2 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_exp2f +# define __DECL_SIMD_exp2f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_expm1 +# define __DECL_SIMD_expm1 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_expm1f +# define __DECL_SIMD_expm1f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log +# define __DECL_SIMD_log __DECL_SIMD_aarch64 +# undef __DECL_SIMD_logf +# define __DECL_SIMD_logf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log10 +# define __DECL_SIMD_log10 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log10f +# define __DECL_SIMD_log10f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log1p +# define __DECL_SIMD_log1p __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log1pf +# define __DECL_SIMD_log1pf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log2 +# define __DECL_SIMD_log2 __DECL_SIMD_aarch64 +# undef __DECL_SIMD_log2f +# define __DECL_SIMD_log2f __DECL_SIMD_aarch64 +# undef __DECL_SIMD_sin +# define __DECL_SIMD_sin __DECL_SIMD_aarch64 +# undef __DECL_SIMD_sinf +# define __DECL_SIMD_sinf __DECL_SIMD_aarch64 +# undef __DECL_SIMD_tan +# define __DECL_SIMD_tan __DECL_SIMD_aarch64 +# undef __DECL_SIMD_tanf +# define __DECL_SIMD_tanf __DECL_SIMD_aarch64 +#endif + #if __GNUC_PREREQ(9, 0) # define __ADVSIMD_VEC_MATH_SUPPORTED typedef __Float32x4_t __f32x4_t; diff --git a/sysdeps/aarch64/fpu/cosf_advsimd.c b/sysdeps/aarch64/fpu/cosf_advsimd.c index f05dd2bcda..9f82e7c5d6 100644 --- a/sysdeps/aarch64/fpu/cosf_advsimd.c +++ b/sysdeps/aarch64/fpu/cosf_advsimd.c @@ -92,3 +92,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (cos) (float32x4_t x) return special_case (x, y, odd, cmp); return vreinterpretq_f32_u32 (veorq_u32 (vreinterpretq_u32_f32 (y), odd)); } +libmvec_hidden_def (V_NAME_F1 (cos)) +HALF_WIDTH_ALIAS_F1 (cos) diff --git a/sysdeps/aarch64/fpu/exp10f_advsimd.c b/sysdeps/aarch64/fpu/exp10f_advsimd.c index 9e754c46fa..10c9dedad7 100644 --- a/sysdeps/aarch64/fpu/exp10f_advsimd.c +++ b/sysdeps/aarch64/fpu/exp10f_advsimd.c @@ -138,3 +138,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp10) (float32x4_t x) return vfmaq_f32 (scale, poly, scale); } +libmvec_hidden_def (V_NAME_F1 (exp10)) +HALF_WIDTH_ALIAS_F1 (exp10) diff --git a/sysdeps/aarch64/fpu/exp2f_advsimd.c b/sysdeps/aarch64/fpu/exp2f_advsimd.c index 70b3ab66c1..cf84a5b9cd 100644 --- a/sysdeps/aarch64/fpu/exp2f_advsimd.c +++ b/sysdeps/aarch64/fpu/exp2f_advsimd.c @@ -122,3 +122,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp2) (float32x4_t x) return vfmaq_f32 (scale, poly, scale); } +libmvec_hidden_def (V_NAME_F1 (exp2)) +HALF_WIDTH_ALIAS_F1 (exp2) diff --git a/sysdeps/aarch64/fpu/expf_advsimd.c b/sysdeps/aarch64/fpu/expf_advsimd.c index 69d5d1ea77..d70d1add95 100644 --- a/sysdeps/aarch64/fpu/expf_advsimd.c +++ b/sysdeps/aarch64/fpu/expf_advsimd.c @@ -131,3 +131,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (exp) (float32x4_t x) return vfmaq_f32 (scale, poly, scale); } +libmvec_hidden_def (V_NAME_F1 (exp)) +HALF_WIDTH_ALIAS_F1 (exp) diff --git a/sysdeps/aarch64/fpu/expm1f_advsimd.c b/sysdeps/aarch64/fpu/expm1f_advsimd.c index b27b75068a..e3b6e1b01b 100644 --- a/sysdeps/aarch64/fpu/expm1f_advsimd.c +++ b/sysdeps/aarch64/fpu/expm1f_advsimd.c @@ -115,3 +115,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (expm1) (float32x4_t x) /* expm1(x) ~= p * t + (t - 1). */ return vfmaq_f32 (vsubq_f32 (t, v_f32 (1.0f)), p, t); } +libmvec_hidden_def (V_NAME_F1 (expm1)) +HALF_WIDTH_ALIAS_F1 (expm1) diff --git a/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h b/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h new file mode 100644 index 0000000000..af489f4769 --- /dev/null +++ b/sysdeps/aarch64/fpu/finclude/math-vector-fortran.h @@ -0,0 +1,47 @@ +! Platform-specific declarations of SIMD math functions for Fortran. -*- f90 -*- +! Copyright (C) 2019-2023 Free Software Foundation, Inc. +! This file is part of the GNU C Library. +! +! The GNU C Library is free software; you can redistribute it and/or +! modify it under the terms of the GNU Lesser General Public +! License as published by the Free Software Foundation; either +! version 2.1 of the License, or (at your option) any later version. +! +! The GNU C Library is distributed in the hope that it will be useful, +! but WITHOUT ANY WARRANTY; without even the implied warranty of +! MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +! Lesser General Public License for more details. +! +! You should have received a copy of the GNU Lesser General Public +! License along with the GNU C Library; if not, see +! <https://www.gnu.org/licenses/>. +!GCC$ builtin (acos) attributes simd (notinbranch) +!GCC$ builtin (acosf) attributes simd (notinbranch) +!GCC$ builtin (asin) attributes simd (notinbranch) +!GCC$ builtin (asinf) attributes simd (notinbranch) +!GCC$ builtin (atan) attributes simd (notinbranch) +!GCC$ builtin (atanf) attributes simd (notinbranch) +!GCC$ builtin (atan2) attributes simd (notinbranch) +!GCC$ builtin (atan2f) attributes simd (notinbranch) +!GCC$ builtin (cos) attributes simd (notinbranch) +!GCC$ builtin (cosf) attributes simd (notinbranch) +!GCC$ builtin (exp) attributes simd (notinbranch) +!GCC$ builtin (expf) attributes simd (notinbranch) +!GCC$ builtin (exp10) attributes simd (notinbranch) +!GCC$ builtin (exp10f) attributes simd (notinbranch) +!GCC$ builtin (exp2) attributes simd (notinbranch) +!GCC$ builtin (exp2f) attributes simd (notinbranch) +!GCC$ builtin (expm1) attributes simd (notinbranch) +!GCC$ builtin (expm1f) attributes simd (notinbranch) +!GCC$ builtin (log) attributes simd (notinbranch) +!GCC$ builtin (logf) attributes simd (notinbranch) +!GCC$ builtin (log10) attributes simd (notinbranch) +!GCC$ builtin (log10f) attributes simd (notinbranch) +!GCC$ builtin (log1p) attributes simd (notinbranch) +!GCC$ builtin (log1pf) attributes simd (notinbranch) +!GCC$ builtin (log2) attributes simd (notinbranch) +!GCC$ builtin (log2f) attributes simd (notinbranch) +!GCC$ builtin (sin) attributes simd (notinbranch) +!GCC$ builtin (sinf) attributes simd (notinbranch) +!GCC$ builtin (tan) attributes simd (notinbranch) +!GCC$ builtin (tanf) attributes simd (notinbranch) diff --git a/sysdeps/aarch64/fpu/log10f_advsimd.c b/sysdeps/aarch64/fpu/log10f_advsimd.c index ba02060bbe..4c4e7cb2f4 100644 --- a/sysdeps/aarch64/fpu/log10f_advsimd.c +++ b/sysdeps/aarch64/fpu/log10f_advsimd.c @@ -80,3 +80,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log10) (float32x4_t x) return special_case (x, y, poly, r2, special); return vfmaq_f32 (y, poly, r2); } +libmvec_hidden_def (V_NAME_F1 (log10)) +HALF_WIDTH_ALIAS_F1 (log10) diff --git a/sysdeps/aarch64/fpu/log1pf_advsimd.c b/sysdeps/aarch64/fpu/log1pf_advsimd.c index 3748830de8..0530dc2002 100644 --- a/sysdeps/aarch64/fpu/log1pf_advsimd.c +++ b/sysdeps/aarch64/fpu/log1pf_advsimd.c @@ -126,3 +126,5 @@ VPCS_ATTR float32x4_t V_NAME_F1 (log1p) (float32x4_t x) return special_case (special_arg, y, special_cases); return y; } +libmvec_hidden_def (V_NAME_F1 (log1p)) +HALF_WIDTH_ALIAS_F1 (log1p) diff --git a/sysdeps/aarch64/fpu/log2f_advsimd.c b/sysdeps/aarch64/fpu/log2f_advsimd.c index e913bcda18..4049db242e 100644 --- a/sysdeps/aarch64/fpu/log2f_advsimd.c +++ b/sysdeps/aarch64/fpu/log2f_advsimd.c @@ -75,3 +75,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log2) (float32x4_t x) return special_case (x, n, p, r, special); return vfmaq_f32 (n, p, r); } +libmvec_hidden_def (V_NAME_F1 (log2)) +HALF_WIDTH_ALIAS_F1 (log2) diff --git a/sysdeps/aarch64/fpu/logf_advsimd.c b/sysdeps/aarch64/fpu/logf_advsimd.c index 93903c7962..930e4b9c8f 100644 --- a/sysdeps/aarch64/fpu/logf_advsimd.c +++ b/sysdeps/aarch64/fpu/logf_advsimd.c @@ -83,3 +83,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (log) (float32x4_t x) return special_case (x, y, r2, p, cmp); return vfmaq_f32 (p, y, r2); } +libmvec_hidden_def (V_NAME_F1 (log)) +HALF_WIDTH_ALIAS_F1 (log) diff --git a/sysdeps/aarch64/fpu/sinf_advsimd.c b/sysdeps/aarch64/fpu/sinf_advsimd.c index 0e78cf55f0..3638ab5508 100644 --- a/sysdeps/aarch64/fpu/sinf_advsimd.c +++ b/sysdeps/aarch64/fpu/sinf_advsimd.c @@ -92,3 +92,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (sin) (float32x4_t x) return special_case (x, y, odd, cmp); return vreinterpretq_f32_u32 (veorq_u32 (vreinterpretq_u32_f32 (y), odd)); } +libmvec_hidden_def (V_NAME_F1 (sin)) +HALF_WIDTH_ALIAS_F1 (sin) diff --git a/sysdeps/aarch64/fpu/tanf_advsimd.c b/sysdeps/aarch64/fpu/tanf_advsimd.c index 4c8a7f740e..dbc7b4dd6e 100644 --- a/sysdeps/aarch64/fpu/tanf_advsimd.c +++ b/sysdeps/aarch64/fpu/tanf_advsimd.c @@ -127,3 +127,5 @@ float32x4_t VPCS_ATTR V_NAME_F1 (tan) (float32x4_t x) return special_case (special_arg, vbslq_f32 (pred_alt, inv_y, y), special); return vbslq_f32 (pred_alt, inv_y, y); } +libmvec_hidden_def (V_NAME_F1 (tan)) +HALF_WIDTH_ALIAS_F1 (tan) diff --git a/sysdeps/aarch64/fpu/v_math.h b/sysdeps/aarch64/fpu/v_math.h index d286eb81b3..e8ac0e2332 100644 --- a/sysdeps/aarch64/fpu/v_math.h +++ b/sysdeps/aarch64/fpu/v_math.h @@ -29,6 +29,21 @@ #define V_NAME_F2(fun) _ZGVnN4vv_##fun##f #define V_NAME_D2(fun) _ZGVnN2vv_##fun +#include "advsimd_f32_protos.h" + +#define HALF_WIDTH_ALIAS_F1(fun) \ + float32x2_t VPCS_ATTR _ZGVnN2v_##fun##f (float32x2_t x) \ + { \ + return vget_low_f32 (_ZGVnN4v_##fun##f (vcombine_f32 (x, x))); \ + } + +#define HALF_WIDTH_ALIAS_F2(fun) \ + float32x2_t VPCS_ATTR _ZGVnN2vv_##fun##f (float32x2_t x, float32x2_t y) \ + { \ + return vget_low_f32 ( \ + _ZGVnN4vv_##fun##f (vcombine_f32 (x, x), vcombine_f32 (y, y))); \ + } + /* Shorthand helpers for declaring constants. */ #define V2(X) { X, X } #define V4(X) { X, X, X, X } diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 2bf4ea6332..580952b4de 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -15,16 +15,31 @@ GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F GLIBC_2.39 _ZGVnN2v_acos F +GLIBC_2.39 _ZGVnN2v_acosf F GLIBC_2.39 _ZGVnN2v_asin F +GLIBC_2.39 _ZGVnN2v_asinf F GLIBC_2.39 _ZGVnN2v_atan F +GLIBC_2.39 _ZGVnN2v_atanf F +GLIBC_2.39 _ZGVnN2v_cosf F GLIBC_2.39 _ZGVnN2v_exp10 F +GLIBC_2.39 _ZGVnN2v_exp10f F GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN2v_exp2f F +GLIBC_2.39 _ZGVnN2v_expf F GLIBC_2.39 _ZGVnN2v_expm1 F +GLIBC_2.39 _ZGVnN2v_expm1f F GLIBC_2.39 _ZGVnN2v_log10 F +GLIBC_2.39 _ZGVnN2v_log10f F GLIBC_2.39 _ZGVnN2v_log1p F +GLIBC_2.39 _ZGVnN2v_log1pf F GLIBC_2.39 _ZGVnN2v_log2 F +GLIBC_2.39 _ZGVnN2v_log2f F +GLIBC_2.39 _ZGVnN2v_logf F +GLIBC_2.39 _ZGVnN2v_sinf F GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN2v_tanf F GLIBC_2.39 _ZGVnN2vv_atan2 F +GLIBC_2.39 _ZGVnN2vv_atan2f F GLIBC_2.39 _ZGVnN4v_acosf F GLIBC_2.39 _ZGVnN4v_asinf F GLIBC_2.39 _ZGVnN4v_atanf F
Added annotations for autovec by GCC and GFortran - this enables GCC >= 9 to autovectorise math calls at -Ofast. The compiler may emit calls to 'half-width' routines (two-lane single-precision variants). These have been added in the form of wrappers around the full-width versions, where the low half of the vector is simply duplicated. This will perform poorly when one lane triggers the special-case handler, as there will be a redundant call to the scalar version, however this is expected to be rare at Ofast. --- Thanks, Joe include/libc-symbols.h | 2 + sysdeps/aarch64/fpu/Versions | 15 +++++ sysdeps/aarch64/fpu/acosf_advsimd.c | 2 + sysdeps/aarch64/fpu/advsimd_f32_protos.h | 34 ++++++++++ sysdeps/aarch64/fpu/asinf_advsimd.c | 2 + sysdeps/aarch64/fpu/atan2f_advsimd.c | 2 + sysdeps/aarch64/fpu/atanf_advsimd.c | 2 + sysdeps/aarch64/fpu/bits/math-vector.h | 66 +++++++++++++++++++ sysdeps/aarch64/fpu/cosf_advsimd.c | 2 + sysdeps/aarch64/fpu/exp10f_advsimd.c | 2 + sysdeps/aarch64/fpu/exp2f_advsimd.c | 2 + sysdeps/aarch64/fpu/expf_advsimd.c | 2 + sysdeps/aarch64/fpu/expm1f_advsimd.c | 2 + .../fpu/finclude/math-vector-fortran.h | 47 +++++++++++++ sysdeps/aarch64/fpu/log10f_advsimd.c | 2 + sysdeps/aarch64/fpu/log1pf_advsimd.c | 2 + sysdeps/aarch64/fpu/log2f_advsimd.c | 2 + sysdeps/aarch64/fpu/logf_advsimd.c | 2 + sysdeps/aarch64/fpu/sinf_advsimd.c | 2 + sysdeps/aarch64/fpu/tanf_advsimd.c | 2 + sysdeps/aarch64/fpu/v_math.h | 15 +++++ .../unix/sysv/linux/aarch64/libmvec.abilist | 15 +++++ 22 files changed, 224 insertions(+) create mode 100644 sysdeps/aarch64/fpu/advsimd_f32_protos.h create mode 100644 sysdeps/aarch64/fpu/finclude/math-vector-fortran.h