From patchwork Thu Oct 5 13:51:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 821787 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-85441-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="OzpknSh5"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y7Dhy2NZFz9sRq for ; Fri, 6 Oct 2017 00:52:02 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id; q=dns; s= default; b=CjPn8tWnheOkxFgxPzzOcuPM/clAW2B6FC7oZtKp6cCpg8iIk7zoS 8MTHJcoJOzirbA34alti/pP4rkU7uPiUTgkCZzP+E2Kmw5LVY8vyjGBc0A6WuxuL 7nJPEsQEQ3m7D96Gw9Bavc6e819MlyU2tlixCDsVfhgm1WaXwRd8zg= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:subject:date:message-id; s=default; bh=tG7Od6MSMMcns9QwJQQ8yKMYA4o=; b=OzpknSh5osNfnhRwRJDCIIctepBS CTUNc+vT+n/c/V0uHRqqd9lW4+CR7wuwUXI7/LeQc/fSuy/Tne3g1ejY1a6kostY RMQ+PQZ6RPpLVz+6jWxkwqYXU+k4vB75PGyGOSJ+07M02zUSWcK9jZQp83CGgsdG z9Aavd5C/+/2y+I= Received: (qmail 130470 invoked by alias); 5 Oct 2017 13:51:27 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 129863 invoked by uid 89); 5 Oct 2017 13:51:27 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-26.3 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_STOCKGEN, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy=8, 5, xy X-HELO: mail-qk0-f178.google.com X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id; bh=5tYOK8g/WMIe6FW30LDhkHfpTBcwCNKh6AxH5uBebvg=; b=r7XNZRzhs7erAODqE2Xwj1qoKYt4RQVyTVfC3kYX7n6cBKQL7bmApWPD+11hirVIAo d8bP4LHtC+0PJJpUEB/PBqQ8DRdPd+FvPJwQ/HCvLCYCwKufR5f/KiTz4D7mjvhDShQ+ J0WqvWkD9r5OWh6EnILYR+Ca2GepNCd0gHtItD+b6lzSZvRC2oMSgANezQsYlSjAUPMu 2Rf3JT3yfqnbuqMixjkJZuRUJ0YEVkJbGky4ApQc0klHUOHElb9FonA7YCkRfB0VDpv7 +CN1zheSv1q7DEadA97TRRTX6+8+Z7YvLhpBpgLdhO2AmodTJly9mCa1bbZ+FzK0VvRp ltVg== X-Gm-Message-State: AMCzsaUZhyELd+0tfVGH4KhIoAl900thheMJmse/pPyJPbScaduPZGs6 KGKekvBbs1sV/SWW8BOKKkdl2olNuM4= X-Google-Smtp-Source: AOwi7QBN9IncjHTor2Ya2UuyIwzluiegLxLS/+1hi4GSdcrvYioQZnkeDYnOxluvFIQpt3dCsd9RMA== X-Received: by 10.55.25.85 with SMTP id k82mr28032219qkh.223.1507211480074; Thu, 05 Oct 2017 06:51:20 -0700 (PDT) From: Adhemerval Zanella To: libc-alpha@sourceware.org Subject: [PATCH 1/4] sparc: Implement memcpy/mempcpy ifunc selection in C Date: Thu, 5 Oct 2017 10:51:09 -0300 Message-Id: <1507211472-1194-1-git-send-email-adhemerval.zanella@linaro.org> This patch refactor the SPARC64 ifunc selector to a C implementation. The x86_64 implementation is used as default, which resulted in common definitions (ifunc-init.h) used on both architectures. No functional change is expected, including ifunc resolution rules. Checked on sparc64-linux-gnu, sparcv9-linux-gnu and x86_64-linux-gnu. * sysdeps/generic/ifunc-init.h: New file. * sysdeps/sparc/init-arch.h: Likewise. * sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S: New file. * sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c: Likewise. * sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c: Likewise. * sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S: Likewise. * sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h: Likewise. * sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S: Likewise. * sysdeps/sparc/sparc64/multiarch/memcpy.c: Likewise. * sysdeps/sparc/sparc64/multiarch/mempcpy.c: Likewise. * sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S: Likewise. * sysdeps/sparc/sparc-ifunc.h (sparc_libc_ifunc_redirected): New macro. * sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile [$(subdir) = string] (sysdep_routines): Add memcpy-ultra1. * sysdeps/sparc/sparc64/multiarch/Makefile [$(subdir) = string] (sysdep_routines): Add memcpy-ultra1. * sysdeps/sparc/sparc64/multiarch/memcpy.S: Remove file. * sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S: Likewise. * sysdeps/x86/init-arch.h: Use generic ifunc-init.h. --- ChangeLog | 24 +++ sysdeps/generic/ifunc-init.h | 54 +++++++ sysdeps/sparc/init-arch.h | 23 +++ sysdeps/sparc/sparc-ifunc.h | 3 + sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile | 3 +- .../sparc32/sparcv9/multiarch/memcpy-ultra1.S | 35 +++++ sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S | 4 - sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c | 1 + sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c | 1 + .../sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S | 1 + sysdeps/sparc/sparc64/multiarch/Makefile | 3 +- sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h | 40 +++++ sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S | 35 +++++ sysdeps/sparc/sparc64/multiarch/memcpy.S | 167 --------------------- sysdeps/sparc/sparc64/multiarch/memcpy.c | 38 +++++ sysdeps/sparc/sparc64/multiarch/mempcpy.c | 47 ++++++ sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S | 1 + sysdeps/x86/init-arch.h | 40 +---- 18 files changed, 308 insertions(+), 212 deletions(-) create mode 100644 sysdeps/generic/ifunc-init.h create mode 100644 sysdeps/sparc/init-arch.h create mode 100644 sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S delete mode 100644 sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S create mode 100644 sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c create mode 100644 sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c create mode 100644 sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S create mode 100644 sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h create mode 100644 sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S delete mode 100644 sysdeps/sparc/sparc64/multiarch/memcpy.S create mode 100644 sysdeps/sparc/sparc64/multiarch/memcpy.c create mode 100644 sysdeps/sparc/sparc64/multiarch/mempcpy.c create mode 100644 sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S diff --git a/sysdeps/generic/ifunc-init.h b/sysdeps/generic/ifunc-init.h new file mode 100644 index 0000000..59d5ac6 --- /dev/null +++ b/sysdeps/generic/ifunc-init.h @@ -0,0 +1,54 @@ +/* IFUNC generic definitions. + This file is part of the GNU C Library. + Copyright (C) 2017 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* These macros are used to implement ifunc selection in C. To implement + an ifunc function, foo, which returns the address of __foo_impl1 or + __foo_impl2: + + #define foo __redirect_foo + #include + #undef foo + #define SYMBOL_NAME foo + #include + + extern __typeof (REDIRECT_NAME) OPTIMIZE (impl1) attribute_hidden; + extern __typeof (REDIRECT_NAME) OPTIMIZE (impl2) attribute_hidden; + + static inline void * + foo_selector (void) + { + if (condition) + return OPTIMIZE (impl2); + + return OPTIMIZE (impl1); + } + + libc_ifunc_redirected (__redirect_foo, foo, IFUNC_SELECTOR ()); +*/ + +#define PASTER1(x,y) x##_##y +#define EVALUATOR1(x,y) PASTER1 (x,y) +#define PASTER2(x,y) __##x##_##y +#define EVALUATOR2(x,y) PASTER2 (x,y) + +/* Basically set '__redirect_' to use as type definition, + '___' as the optimized implementation and + '_ifunc_selector' as the IFUNC selector. */ +#define REDIRECT_NAME EVALUATOR1 (__redirect, SYMBOL_NAME) +#define OPTIMIZE(name) EVALUATOR2 (SYMBOL_NAME, name) +#define IFUNC_SELECTOR EVALUATOR1 (SYMBOL_NAME, ifunc_selector) diff --git a/sysdeps/sparc/init-arch.h b/sysdeps/sparc/init-arch.h new file mode 100644 index 0000000..30972cd --- /dev/null +++ b/sysdeps/sparc/init-arch.h @@ -0,0 +1,23 @@ +/* IFUNC common definition for SPARC64. + This file is part of the GNU C Library. + Copyright (C) 2017 Free Software Foundation, Inc. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include + +#define INIT_ARCH() +#include diff --git a/sysdeps/sparc/sparc-ifunc.h b/sysdeps/sparc/sparc-ifunc.h index 4b1ea00..3d81b52 100644 --- a/sysdeps/sparc/sparc-ifunc.h +++ b/sysdeps/sparc/sparc-ifunc.h @@ -162,6 +162,9 @@ END (__##name) #else /* __ASSEMBLER__ */ +# define sparc_libc_ifunc_redirected(redirected_name, name, expr) \ + __ifunc (redirected_name, name, expr(hwcap), int hwcap, INIT_ARCH) + # define sparc_libm_ifunc(name, expr) \ __ifunc (name, name, expr, int hwcap, libm_ifunc_init) diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile b/sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile index 4ad7aff..ca44798 100644 --- a/sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile +++ b/sysdeps/sparc/sparc32/sparcv9/multiarch/Makefile @@ -8,5 +8,6 @@ endif ifeq ($(subdir),string) sysdep_routines += memcpy-ultra3 memcpy-niagara1 memcpy-niagara2 \ - memset-niagara1 memcpy-niagara4 memset-niagara4 + memset-niagara1 memcpy-niagara4 memset-niagara4 \ + memcpy-ultra1 endif diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S b/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S new file mode 100644 index 0000000..b56438d --- /dev/null +++ b/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy-ultra1.S @@ -0,0 +1,35 @@ +/* Default SPARC32 memcpy implementation. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) + +# include + +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# undef weak_alias +# define weak_alias(x, y) +# undef libc_hidden_def +# define libc_hidden_def(name) + +# define memcpy __memcpy_ultra1 +# define __mempcpy __mempcpy_ultra1 + +#endif + +#include diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S b/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S deleted file mode 100644 index 14df91e..0000000 --- a/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.S +++ /dev/null @@ -1,4 +0,0 @@ -#define ASI_PNF 0x82 -#define ASI_BLK_P 0xf0 -#define XCC icc -#include diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c b/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c new file mode 100644 index 0000000..369acac --- /dev/null +++ b/sysdeps/sparc/sparc32/sparcv9/multiarch/memcpy.c @@ -0,0 +1 @@ +#include diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c b/sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c new file mode 100644 index 0000000..616a538 --- /dev/null +++ b/sysdeps/sparc/sparc32/sparcv9/multiarch/mempcpy.c @@ -0,0 +1 @@ +#include diff --git a/sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S b/sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S new file mode 100644 index 0000000..f18c9f9 --- /dev/null +++ b/sysdeps/sparc/sparc32/sparcv9/multiarch/rtld-mempcpy.S @@ -0,0 +1 @@ +/* rtld-mempcpy is implemented by rtld-memcpy.S. */ diff --git a/sysdeps/sparc/sparc64/multiarch/Makefile b/sysdeps/sparc/sparc64/multiarch/Makefile index 55b757f..4e52526 100644 --- a/sysdeps/sparc/sparc64/multiarch/Makefile +++ b/sysdeps/sparc/sparc64/multiarch/Makefile @@ -8,7 +8,8 @@ endif ifeq ($(subdir),string) sysdep_routines += memcpy-ultra3 memcpy-niagara1 memcpy-niagara2 \ - memset-niagara1 memcpy-niagara4 memset-niagara4 + memset-niagara1 memcpy-niagara4 memset-niagara4 \ + memcpy-ultra1 endif ifeq ($(subdir),stdlib) diff --git a/sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h b/sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h new file mode 100644 index 0000000..f497108 --- /dev/null +++ b/sysdeps/sparc/sparc64/multiarch/ifunc-memcpy.h @@ -0,0 +1,40 @@ +/* Common definition for memcpy and mempcpy implementation. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +extern __typeof (REDIRECT_NAME) OPTIMIZE (niagara4) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (niagara2) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (niagara1) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ultra3) attribute_hidden; +extern __typeof (REDIRECT_NAME) OPTIMIZE (ultra1) attribute_hidden; + +static inline void * +IFUNC_SELECTOR (int hwcap) +{ + if (hwcap & HWCAP_SPARC_CRYPTO) + return OPTIMIZE (niagara4); + if (hwcap & HWCAP_SPARC_N2) + return OPTIMIZE (niagara2); + if (hwcap & HWCAP_SPARC_BLKINIT) + return OPTIMIZE (niagara1); + if (hwcap & HWCAP_SPARC_ULTRA3) + return OPTIMIZE (ultra3); + return OPTIMIZE (ultra1); +} diff --git a/sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S b/sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S new file mode 100644 index 0000000..1a0aaf3 --- /dev/null +++ b/sysdeps/sparc/sparc64/multiarch/memcpy-ultra1.S @@ -0,0 +1,35 @@ +/* Default SPARC64 memcpy implementation. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) + +# include + +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(name) +# undef weak_alias +# define weak_alias(x, y) +# undef libc_hidden_def +# define libc_hidden_def(name) + +# define memcpy __memcpy_ultra1 +# define __mempcpy __mempcpy_ultra1 + +#endif + +#include diff --git a/sysdeps/sparc/sparc64/multiarch/memcpy.S b/sysdeps/sparc/sparc64/multiarch/memcpy.S deleted file mode 100644 index b6396ee..0000000 --- a/sysdeps/sparc/sparc64/multiarch/memcpy.S +++ /dev/null @@ -1,167 +0,0 @@ -/* Multiple versions of memcpy - All versions must be listed in ifunc-impl-list.c. - Copyright (C) 2010-2017 Free Software Foundation, Inc. - Contributed by David S. Miller (davem@davemloft.net) - This file is part of the GNU C Library. - - The GNU C Library is free software; you can redistribute it and/or - modify it under the terms of the GNU Lesser General Public - License as published by the Free Software Foundation; either - version 2.1 of the License, or (at your option) any later version. - - The GNU C Library is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - Lesser General Public License for more details. - - You should have received a copy of the GNU Lesser General Public - License along with the GNU C Library; if not, see - . */ - -#include - -#if IS_IN (libc) - .text -ENTRY(memcpy) - .type memcpy, @gnu_indirect_function -# ifdef SHARED - SETUP_PIC_REG_LEAF(o3, o5) -# endif - set HWCAP_SPARC_CRYPTO, %o1 - andcc %o0, %o1, %g0 - be 1f - andcc %o0, HWCAP_SPARC_N2, %g0 -# ifdef SHARED - sethi %gdop_hix22(__memcpy_niagara4), %o1 - xor %o1, %gdop_lox10(__memcpy_niagara4), %o1 -# else - set __memcpy_niagara4, %o1 -# endif - ba 10f - nop -1: be 1f - andcc %o0, HWCAP_SPARC_BLKINIT, %g0 -# ifdef SHARED - sethi %gdop_hix22(__memcpy_niagara2), %o1 - xor %o1, %gdop_lox10(__memcpy_niagara2), %o1 -# else - set __memcpy_niagara2, %o1 -# endif - ba 10f - nop -1: be 1f - andcc %o0, HWCAP_SPARC_ULTRA3, %g0 -# ifdef SHARED - sethi %gdop_hix22(__memcpy_niagara1), %o1 - xor %o1, %gdop_lox10(__memcpy_niagara1), %o1 -# else - set __memcpy_niagara1, %o1 -# endif - ba 10f - nop -1: be 9f - nop -# ifdef SHARED - sethi %gdop_hix22(__memcpy_ultra3), %o1 - xor %o1, %gdop_lox10(__memcpy_ultra3), %o1 -# else - set __memcpy_ultra3, %o1 -# endif - ba 10f - nop -9: -# ifdef SHARED - sethi %gdop_hix22(__memcpy_ultra1), %o1 - xor %o1, %gdop_lox10(__memcpy_ultra1), %o1 -# else - set __memcpy_ultra1, %o1 -# endif -10: -# ifdef SHARED - add %o3, %o1, %o1 -# endif - retl - mov %o1, %o0 -END(memcpy) - -ENTRY(__mempcpy) - .type __mempcpy, @gnu_indirect_function -# ifdef SHARED - SETUP_PIC_REG_LEAF(o3, o5) -# endif - set HWCAP_SPARC_CRYPTO, %o1 - andcc %o0, %o1, %g0 - be 1f - andcc %o0, HWCAP_SPARC_N2, %g0 -# ifdef SHARED - sethi %gdop_hix22(__mempcpy_niagara4), %o1 - xor %o1, %gdop_lox10(__mempcpy_niagara4), %o1 -# else - set __mempcpy_niagara4, %o1 -# endif - ba 10f - nop -1: be 1f - andcc %o0, HWCAP_SPARC_BLKINIT, %g0 -# ifdef SHARED - sethi %gdop_hix22(__mempcpy_niagara2), %o1 - xor %o1, %gdop_lox10(__mempcpy_niagara2), %o1 -# else - set __mempcpy_niagara2, %o1 -# endif - ba 10f - nop -1: be 1f - andcc %o0, HWCAP_SPARC_ULTRA3, %g0 -# ifdef SHARED - sethi %gdop_hix22(__mempcpy_niagara1), %o1 - xor %o1, %gdop_lox10(__mempcpy_niagara1), %o1 -# else - set __mempcpy_niagara1, %o1 -# endif - ba 10f - nop -1: be 9f - nop -# ifdef SHARED - sethi %gdop_hix22(__mempcpy_ultra3), %o1 - xor %o1, %gdop_lox10(__mempcpy_ultra3), %o1 -# else - set __mempcpy_ultra3, %o1 -# endif - ba 10f - nop -9: -# ifdef SHARED - sethi %gdop_hix22(__mempcpy_ultra1), %o1 - xor %o1, %gdop_lox10(__mempcpy_ultra1), %o1 -# else - set __mempcpy_ultra1, %o1 -# endif -10: -# ifdef SHARED - add %o3, %o1, %o1 -# endif - retl - mov %o1, %o0 -END(__mempcpy) - -libc_hidden_builtin_def (memcpy) - -libc_hidden_def (__mempcpy) -weak_alias (__mempcpy, mempcpy) -libc_hidden_builtin_def (mempcpy) - -#undef libc_hidden_builtin_def -#define libc_hidden_builtin_def(name) -#undef weak_alias -#define weak_alias(x, y) -#undef libc_hidden_def -#define libc_hidden_def(name) - -#define memcpy __memcpy_ultra1 -#define __mempcpy __mempcpy_ultra1 - -#endif - -#include "../memcpy.S" diff --git a/sysdeps/sparc/sparc64/multiarch/memcpy.c b/sysdeps/sparc/sparc64/multiarch/memcpy.c new file mode 100644 index 0000000..2c3d3f3 --- /dev/null +++ b/sysdeps/sparc/sparc64/multiarch/memcpy.c @@ -0,0 +1,38 @@ +/* Multiple versions of memcpy. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +# define memcpy __redirect_memcpy +# include +# undef memcpy + +# include + +# define SYMBOL_NAME memcpy +# include "ifunc-memcpy.h" + +sparc_libc_ifunc_redirected (__redirect_memcpy, memcpy, IFUNC_SELECTOR); + +/* It essentially does libc_hidden_builtin_def (memcpy) and redirect + the internal symbol to ifunc implementation. */ +# ifdef SHARED +__hidden_ver1 (memcpy, __GI_memcpy, __redirect_memcpy) + __attribute__ ((visibility ("hidden"))); +# endif +#endif diff --git a/sysdeps/sparc/sparc64/multiarch/mempcpy.c b/sysdeps/sparc/sparc64/multiarch/mempcpy.c new file mode 100644 index 0000000..f52dca6 --- /dev/null +++ b/sysdeps/sparc/sparc64/multiarch/mempcpy.c @@ -0,0 +1,47 @@ +/* Multiple versions of memcpy + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +# define mempcpy __redirect_mempcpy +# define __mempcpy __redirect___mempcpy +# define NO_MEMPCPY_STPCPY_REDIRECT +# define __NO_STRING_INLINES +# include +# undef mempcpy +# undef __mempcpy + +# include + +# define SYMBOL_NAME mempcpy +# include "ifunc-memcpy.h" + +sparc_libc_ifunc_redirected (__redirect_mempcpy, __mempcpy, IFUNC_SELECTOR); + +weak_alias (__mempcpy, mempcpy) + +/* It essentially does libc_hidden_builtin_def (mempcpy) and + and libc_hidden_def (__mempcpy) and redirects the internal symbol to + ifunc implementation. */ +# ifdef SHARED +__hidden_ver1 (__mempcpy, __GI___mempcpy, __redirect___mempcpy) + __attribute__ ((visibility ("hidden"))); +__hidden_ver1 (mempcpy, __GI_mempcpy, __redirect_mempcpy) + __attribute__ ((visibility ("hidden"))); +# endif +#endif diff --git a/sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S b/sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S new file mode 100644 index 0000000..f18c9f9 --- /dev/null +++ b/sysdeps/sparc/sparc64/multiarch/rtld-mempcpy.S @@ -0,0 +1 @@ +/* rtld-mempcpy is implemented by rtld-memcpy.S. */ diff --git a/sysdeps/x86/init-arch.h b/sysdeps/x86/init-arch.h index 15d3f09..9228fe5 100644 --- a/sysdeps/x86/init-arch.h +++ b/sysdeps/x86/init-arch.h @@ -21,45 +21,7 @@ # include #endif -/* These macros are used to implement ifunc selection in C. To implement - an ifunc function, foo, which returns the address of __foo_sse2 or - __foo_avx2: - - #define foo __redirect_foo - #define __foo __redirect___foo - #include - #undef foo - #undef __foo - #define SYMBOL_NAME foo - #include - - extern __typeof (REDIRECT_NAME) OPTIMIZE (sse2) attribute_hidden; - extern __typeof (REDIRECT_NAME) OPTIMIZE (avx2) attribute_hidden; - - static inline void * - foo_selector (void) - { - if (use AVX2) - return OPTIMIZE (avx2); - - return OPTIMIZE (sse2); - } - - libc_ifunc_redirected (__redirect_foo, foo, foo_selector ()); - -*/ - -#define PASTER1(x,y) x##_##y -#define EVALUATOR1(x,y) PASTER1 (x,y) -#define PASTER2(x,y) __##x##_##y -#define EVALUATOR2(x,y) PASTER2 (x,y) - -/* Basically set '__redirect_' to use as type definition, - '___' as the optimized implementation and - '_ifunc_selector' as the IFUNC selector. */ -#define REDIRECT_NAME EVALUATOR1 (__redirect, SYMBOL_NAME) -#define OPTIMIZE(name) EVALUATOR2 (SYMBOL_NAME, name) -#define IFUNC_SELECTOR EVALUATOR1 (SYMBOL_NAME, ifunc_selector) +#include #ifndef __x86_64__ /* Due to the reordering and the other nifty extensions in i686, it is