From patchwork Fri Jul 3 13:38:26 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Liebler X-Patchwork-Id: 491075 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 19477140757 for ; Fri, 3 Jul 2015 23:41:36 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=sourceware.org header.i=@sourceware.org header.b=f0KDJcRb; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:in-reply-to :references; q=dns; s=default; b=AXSRM8PIGAofLbWy8dsa6N+EW2X9K0Q 6bMoeBLjK+5v4ywgQYBcR7/oRv2Ku0Jq4VnPnxjsjqwYXHYhiJeZ9xkhUz7OYJiu cLA/+hKlrpzdtnq3EPVQkPS1eiAZ5h2AmAqL7MKPjZm09w2mVkYglrUUhnWfzqBE CoFz4eO6p/CU= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:in-reply-to :references; s=default; bh=ntDmtdFuJKOtfC1YxUhovOhPcpo=; b=f0KDJ cRbtmoF0btsenqSp296fro/Si0EtjNVbNFB/5u4TBK8xDnscpHIVdfAHy7cPGXLl vwhnFEcq3Yv0nhMbIV4Clz4sLie9ycTm5VfykXWdGgxhDad5LjlD5+IAnERSNqqQ qtJnRu0xv95wzEEulppLfQhcyjcGj3DtEUrRNI= Received: (qmail 98973 invoked by alias); 3 Jul 2015 13:39:30 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 98863 invoked by uid 89); 3 Jul 2015 13:39:29 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=0.9 required=5.0 tests=AWL, BAYES_50, KAM_LAZY_DOMAIN_SECURITY, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-HELO: e06smtp11.uk.ibm.com X-MailFrom: stli@linux.vnet.ibm.com X-RcptTo: libc-alpha@sourceware.org From: Stefan Liebler To: libc-alpha@sourceware.org Cc: Stefan Liebler Subject: [PATCH 12/27] S390: Optimize stpncpy and wcpncpy. Date: Fri, 3 Jul 2015 15:38:26 +0200 Message-Id: <1435930721-27922-13-git-send-email-stli@linux.vnet.ibm.com> In-Reply-To: <1435930721-27922-1-git-send-email-stli@linux.vnet.ibm.com> References: <1435930721-27922-1-git-send-email-stli@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15070313-0041-0000-0000-000004F9EA97 This patch provides optimized versions of stpncpy and wcpncpy with the z13 vector instructions. ChangeLog: * sysdeps/s390/multiarch/stpncpy-c.c: New File. * sysdeps/s390/multiarch/stpncpy-vx.S: Likewise. * sysdeps/s390/multiarch/stpncpy.c: Likewise. * sysdeps/s390/multiarch/wcpncpy-c.c: Likewise. * sysdeps/s390/multiarch/wcpncpy-vx.S: Likewise. * sysdeps/s390/multiarch/wcpncpy.c: Likewise. * sysdeps/s390/multiarch/Makefile (sysdep_routines): Add stpncpy and wcpncpy functions. * sysdeps/s390/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list): Add ifunc test for stpncpy, wcpncpy. * wcsmbs/wcpncpy.c: Use WCPNCPY if defined. * string/test-stpncpy.c: Add wcpncpy support. * wcsmbs/test-wcpncpy.c: New File. * wcsmbs/Makefile (strop-tests): Add wcpncpy. * benchtests/bench-stpncpy.c: Add wcpncpy support. * benchtests/bench-wcpncpy.c: New File. * benchtests/Makefile (wcsmbs-bench): Add wcpncpy. --- benchtests/Makefile | 2 +- benchtests/bench-stpncpy.c | 41 +++++-- benchtests/bench-wcpncpy.c | 20 ++++ string/test-stpncpy.c | 41 +++++-- sysdeps/s390/multiarch/Makefile | 6 +- sysdeps/s390/multiarch/ifunc-impl-list.c | 3 + sysdeps/s390/multiarch/stpncpy-c.c | 28 +++++ sysdeps/s390/multiarch/stpncpy-vx.S | 176 ++++++++++++++++++++++++++++ sysdeps/s390/multiarch/stpncpy.c | 28 +++++ sysdeps/s390/multiarch/wcpncpy-c.c | 25 ++++ sysdeps/s390/multiarch/wcpncpy-vx.S | 195 +++++++++++++++++++++++++++++++ sysdeps/s390/multiarch/wcpncpy.c | 28 +++++ wcsmbs/Makefile | 2 +- wcsmbs/test-wcpncpy.c | 20 ++++ wcsmbs/wcpncpy.c | 5 + 15 files changed, 594 insertions(+), 26 deletions(-) create mode 100644 benchtests/bench-wcpncpy.c create mode 100644 sysdeps/s390/multiarch/stpncpy-c.c create mode 100644 sysdeps/s390/multiarch/stpncpy-vx.S create mode 100644 sysdeps/s390/multiarch/stpncpy.c create mode 100644 sysdeps/s390/multiarch/wcpncpy-c.c create mode 100644 sysdeps/s390/multiarch/wcpncpy-vx.S create mode 100644 sysdeps/s390/multiarch/wcpncpy.c create mode 100644 wcsmbs/test-wcpncpy.c diff --git a/benchtests/Makefile b/benchtests/Makefile index 3785328..c3531a9 100644 --- a/benchtests/Makefile +++ b/benchtests/Makefile @@ -36,7 +36,7 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ strncasecmp strncat strncmp strncpy strnlen strpbrk strrchr \ strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok \ strcoll -wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy +wcsmbs-bench := wcslen wcsnlen wcscpy wcpcpy wcsncpy wcpncpy string-bench-all := $(string-bench) ${wcsmbs-bench} # We have to generate locales diff --git a/benchtests/bench-stpncpy.c b/benchtests/bench-stpncpy.c index 53539c7..822db1f 100644 --- a/benchtests/bench-stpncpy.c +++ b/benchtests/bench-stpncpy.c @@ -18,18 +18,36 @@ #define STRNCPY_RESULT(dst, len, n) ((dst) + ((len) > (n) ? (n) : (len))) #define TEST_MAIN -#define TEST_NAME "stpncpy" +#ifndef WIDE +# define TEST_NAME "stpncpy" +#else +# define TEST_NAME "wcpncpy" +#endif /* WIDE */ #include "bench-string.h" +#ifndef WIDE +# define CHAR char +# define SIMPLE_STPNCPY simple_stpncpy +# define STUPID_STPNCPY stupid_stpncpy +# define STPNCPY stpncpy +# define STRNLEN strnlen +#else +# include +# define CHAR wchar_t +# define SIMPLE_STPNCPY simple_wcpncpy +# define STUPID_STPNCPY stupid_wcpncpy +# define STPNCPY wcpncpy +# define STRNLEN wcsnlen +#endif /* WIDE */ -char *simple_stpncpy (char *, const char *, size_t); -char *stupid_stpncpy (char *, const char *, size_t); +CHAR *SIMPLE_STPNCPY (CHAR *, const CHAR *, size_t); +CHAR *STUPID_STPNCPY (CHAR *, const CHAR *, size_t); -IMPL (stupid_stpncpy, 0) -IMPL (simple_stpncpy, 0) -IMPL (stpncpy, 1) +IMPL (STUPID_STPNCPY, 0) +IMPL (SIMPLE_STPNCPY, 0) +IMPL (STPNCPY, 1) -char * -simple_stpncpy (char *dst, const char *src, size_t n) +CHAR * +SIMPLE_STPNCPY (CHAR *dst, const CHAR *src, size_t n) { while (n--) if ((*dst++ = *src++) == '\0') @@ -43,10 +61,10 @@ simple_stpncpy (char *dst, const char *src, size_t n) return dst; } -char * -stupid_stpncpy (char *dst, const char *src, size_t n) +CHAR * +STUPID_STPNCPY (CHAR *dst, const CHAR *src, size_t n) { - size_t nc = strnlen (src, n); + size_t nc = STRNLEN (src, n); size_t i; for (i = 0; i < nc; ++i) @@ -56,4 +74,5 @@ stupid_stpncpy (char *dst, const char *src, size_t n) return dst + nc; } +#undef CHAR #include "bench-strncpy.c" diff --git a/benchtests/bench-wcpncpy.c b/benchtests/bench-wcpncpy.c new file mode 100644 index 0000000..8aa529e --- /dev/null +++ b/benchtests/bench-wcpncpy.c @@ -0,0 +1,20 @@ +/* Measure wcpncpy functions. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define WIDE 1 +#include "bench-stpncpy.c" diff --git a/string/test-stpncpy.c b/string/test-stpncpy.c index 1adab34..8624bba 100644 --- a/string/test-stpncpy.c +++ b/string/test-stpncpy.c @@ -19,18 +19,36 @@ #define STRNCPY_RESULT(dst, len, n) ((dst) + ((len) > (n) ? (n) : (len))) #define TEST_MAIN -#define TEST_NAME "stpncpy" +#ifndef WIDE +# define TEST_NAME "stpncpy" +#else +# define TEST_NAME "wcpncpy" +#endif /* WIDE */ #include "test-string.h" +#ifndef WIDE +# define CHAR char +# define SIMPLE_STPNCPY simple_stpncpy +# define STUPID_STPNCPY stupid_stpncpy +# define STPNCPY stpncpy +# define STRNLEN strnlen +#else +# include +# define CHAR wchar_t +# define SIMPLE_STPNCPY simple_wcpncpy +# define STUPID_STPNCPY stupid_wcpncpy +# define STPNCPY wcpncpy +# define STRNLEN wcsnlen +#endif /* WIDE */ -char *simple_stpncpy (char *, const char *, size_t); -char *stupid_stpncpy (char *, const char *, size_t); +CHAR *SIMPLE_STPNCPY (CHAR *, const CHAR *, size_t); +CHAR *STUPID_STPNCPY (CHAR *, const CHAR *, size_t); -IMPL (stupid_stpncpy, 0) -IMPL (simple_stpncpy, 0) -IMPL (stpncpy, 1) +IMPL (STUPID_STPNCPY, 0) +IMPL (SIMPLE_STPNCPY, 0) +IMPL (STPNCPY, 1) -char * -simple_stpncpy (char *dst, const char *src, size_t n) +CHAR * +SIMPLE_STPNCPY (CHAR *dst, const CHAR *src, size_t n) { while (n--) if ((*dst++ = *src++) == '\0') @@ -44,10 +62,10 @@ simple_stpncpy (char *dst, const char *src, size_t n) return dst; } -char * -stupid_stpncpy (char *dst, const char *src, size_t n) +CHAR * +STUPID_STPNCPY (CHAR *dst, const CHAR *src, size_t n) { - size_t nc = strnlen (src, n); + size_t nc = STRNLEN (src, n); size_t i; for (i = 0; i < nc; ++i) @@ -57,4 +75,5 @@ stupid_stpncpy (char *dst, const char *src, size_t n) return dst + nc; } +#undef CHAR #include "test-strncpy.c" diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile index 0dff2dc..98b588f 100644 --- a/sysdeps/s390/multiarch/Makefile +++ b/sysdeps/s390/multiarch/Makefile @@ -3,7 +3,8 @@ sysdep_routines += strlen strlen-vx strlen-c \ strnlen strnlen-vx strnlen-c \ strcpy strcpy-vx \ stpcpy stpcpy-vx stpcpy-c \ - strncpy strncpy-vx + strncpy strncpy-vx \ + stpncpy stpncpy-vx stpncpy-c endif ifeq ($(subdir),wcsmbs) @@ -11,5 +12,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \ wcsnlen wcsnlen-vx wcsnlen-c \ wcscpy wcscpy-vx wcscpy-c \ wcpcpy wcpcpy-vx wcpcpy-c \ - wcsncpy wcsncpy-vx wcsncpy-c + wcsncpy wcsncpy-vx wcsncpy-c \ + wcpncpy wcpncpy-vx wcpncpy-c endif diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c index 940421d..ca69983 100644 --- a/sysdeps/s390/multiarch/ifunc-impl-list.c +++ b/sysdeps/s390/multiarch/ifunc-impl-list.c @@ -94,6 +94,9 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, IFUNC_VX_IMPL (strncpy); IFUNC_VX_IMPL (wcsncpy); + IFUNC_VX_IMPL (stpncpy); + IFUNC_VX_IMPL (wcpncpy); + #endif /* HAVE_S390_VX_ASM_SUPPORT */ return i; diff --git a/sysdeps/s390/multiarch/stpncpy-c.c b/sysdeps/s390/multiarch/stpncpy-c.c new file mode 100644 index 0000000..bea15fa --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy-c.c @@ -0,0 +1,28 @@ +/* Default stpncpy implementation for S/390. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define STPNCPY __stpncpy_c +# ifdef SHARED +# undef libc_hidden_def +# define libc_hidden_def(name) \ + __hidden_ver1 (__stpncpy_c, __GI___stpncpy, __stpncpy_c); +# endif /* SHARED */ + +# include +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpncpy-vx.S b/sysdeps/s390/multiarch/stpncpy-vx.S new file mode 100644 index 0000000..0f1e507 --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy-vx.S @@ -0,0 +1,176 @@ +/* Vector optimized 32/64 bit S/390 version of stpncpy. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* char * stpncpy (char *dest, const char *src, size_t n) + Copies at most n characters of string src to dest + returning a pointer to its end or dest+n + if src is smaller than n. + + Register usage: + -%r0 = return value + -%r1 = zero byte index + -%r2 = curr dst pointer + -%r3 = curr src pointer + -%r4 = n + -%r5 = current_len + -%r6 = loaded bytes + -%r7 = border, tmp +*/ +ENTRY(__stpncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */ + vlvgp %v31,%r6,%r7 /* Save registers. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + lghi %r5,0 /* current_len = 0. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + +.Lpreloop1: + /* Now we are 16byte aligned, so we can load a full vreg + without page fault. */ + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <= 16, process remaining + bytes. */ + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + j .Lloop1 + + /* Find zero in 16byte aligned loop. */ +.Lloop2: + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + +.Lloop1: + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezbs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Save previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezbs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezbs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezbs %v17,%v18,%v18 + jo .Lloop2 /* No zero found -> loop. */ + +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = maxlen - (currlen + zero_index + 1) */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + lgr %r0,%r2 /* Save return-pointer to found zero. */ + clgije %r4,1,.Lend /* Skip zero-filling, if found zero is last + possible character. + (1 is substracted from r4 below!). */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezb %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */ + clrjle %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination! */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 +.Lend2: + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 +END(__stpncpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/stpncpy.c b/sysdeps/s390/multiarch/stpncpy.c new file mode 100644 index 0000000..cf8e56c --- /dev/null +++ b/sysdeps/s390/multiarch/stpncpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of stpncpy. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include +# include + +s390_vx_libc_ifunc (__stpncpy) +weak_alias (__stpncpy, stpncpy) + +#else +# include +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/sysdeps/s390/multiarch/wcpncpy-c.c b/sysdeps/s390/multiarch/wcpncpy-c.c new file mode 100644 index 0000000..9810e52 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy-c.c @@ -0,0 +1,25 @@ +/* Default wcsncpy implementation for S/390. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# define WCPNCPY __wcpncpy_c + +# include +extern __typeof (__wcpncpy) __wcpncpy_c; +# include +#endif diff --git a/sysdeps/s390/multiarch/wcpncpy-vx.S b/sysdeps/s390/multiarch/wcpncpy-vx.S new file mode 100644 index 0000000..9c7ca10 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy-vx.S @@ -0,0 +1,195 @@ +/* Vector optimized 32/64 bit S/390 version of wcpncpy. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) + +# include "sysdep.h" +# include "asm-syntax.h" + + .text + +/* wchar_t * wcpncpy (wchar_t *dest, const wchar_t *src, size_t n) + Copies at most n characters of string src to dest + returning a pointer to its end or dest+n + if src is smaller than n. + + Register usage: + -%r0 = return value + -%r1 = zero byte index + -%r2 = curr dst pointer + -%r3 = curr src pointer + -%r4 = n + -%r5 = current_len + -%r6 = loaded bytes + -%r7 = border, tmp +*/ +ENTRY(__wcpncpy_vx) + .machine "z13" + .machinemode "zarch_nohighgprs" + +# if !defined __s390x__ + llgfr %r4,%r4 +# endif /* !defined __s390x__ */ + + clgfi %r4,0 + ber %r14 /* Nothing to do, if n == 0. */ + + vlbb %v16,0(%r3),6 /* Load s until next 4k-byte boundary. */ + + tmll %r3,3 /* Test if s is 4-byte aligned? */ + jne .Lfallback /* And use common-code variant if not. */ + + vlvgp %v31,%r6,%r7 /* Save registers. */ + lghi %r5,0 /* current_len = 0. */ + + lcbb %r6,0(%r3),6 /* Get bytes to 4k-byte boundary or 16. */ + llgfr %r6,%r6 /* Convert 32bit to 64bit. */ + + /* Check range of maxlen and convert to byte-count. */ +# ifdef __s390x__ + tmhh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + lghi %r1,-4 /* Max byte-count is 18446744073709551612. */ +# else + tmlh %r4,49152 /* Test bit 0 or 1 of maxlen. */ + llilf %r1,4294967292 /* Max byte-count is 4294967292. */ +# endif /* !__s390x__ */ + sllg %r4,%r4,2 /* Convert character-count to byte-count. */ + locgrne %r4,%r1 /* Use max byte-count, if bit 0/1 was one. */ + + la %r0,0(%r4,%r2) /* Save destination pointer + n for return. */ + + clgrjle %r4,%r6,.Lremaining_v16 /* If n <= loaded-bytes + -> process remaining. */ + + /* n > loaded-byte-count */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + clrjl %r1,%r6,.Lfound_v16_store /* Found zero within loaded bytes, + copy and return. */ + + /* Align s to 16 byte. */ + risbgn %r7,%r3,60,128+63,0 /* %r3 = bits 60-63 of %r2 'and' 15. */ + lghi %r5,15 /* current_len = 15. */ + slr %r5,%r7 /* Compute highest index to 16byte boundary. */ + + /* Zero not found and n > loaded-byte-count. */ + vstl %v16,%r5,0(%r2) /* Copy loaded characters - no zero. */ + ahi %r5,1 /* Start loop at next character. */ + +.Lpreloop1: + /* Now we are 16byte aligned, so we can load a full vreg + without page fault. */ + vl %v16,0(%r5,%r3) /* Load s. */ + clgijl %r4,17,.Lremaining_v16 /* If n <=16, + process remaining bytes. */ + lgr %r7,%r4 + slgfi %r7,16 /* border_len = n - 16. */ + j .Lloop1 + + /* Find zero in 16byte aligned loop. */ +.Lloop2: + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + +.Lloop1: + clgrjhe %r5,%r7,.Lremaining_v16 /* If current_len >= border + then process remaining bytes. */ + vfenezfs %v17,%v16,%v16 /* Find element not equal with zero search. */ + je .Lfound_v16 /* Jump away if zero was found. */ + vl %v18,16(%r5,%r3) /* Load next part of s. */ + vst %v16,0(%r5,%r2) /* Store previous part without zero to dst. */ + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezfs %v17,%v18,%v18 + je .Lfound_v18 + vl %v16,16(%r5,%r3) + vst %v18,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v16 + vfenezfs %v17,%v16,%v16 + je .Lfound_v16 + vl %v18,16(%r5,%r3) + vst %v16,0(%r5,%r2) + aghi %r5,16 + + clgrjhe %r5,%r7,.Lremaining_v18 + vfenezfs %v17,%v18,%v18 + jo .Lloop2 /* No zero found -> loop. */ + +.Lfound_v18: + vlr %v16,%v18 +.Lfound_v16: + /* v16 contains a zero. Store remaining bytes to zero. current_len + has not reached border, thus checking for n is not needed! */ + vlgvb %r1,%v17,7 /* Load byte index of zero. */ + la %r2,0(%r5,%r2) /* vstl has no support for index-register. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ +.Lfound_v16_store: + vstl %v16,%r1,0(%r2) /* Copy characters including zero. */ + /* Fill remaining bytes with zero - remaining byte count always > 0. */ + algr %r5,%r1 /* Remaining bytes (=%r4) = ... */ + slgr %r4,%r5 /* = n - (currlen + zero_index + 1) */ + la %r2,0(%r1,%r2) /* Pointer to zero. start filling beyond. */ + lay %r0,-3(%r2) /* Save return-pointer to found zero. */ + clgije %r4,1,.Lend /* Skip zero-filling, if found-zero is last + possible character. + (1 is substracted from r4 below!). */ + aghi %r4,-2 /* mvc with exrl needs count - 1. + (additional -1, see remaining bytes above) */ + srlg %r6,%r4,8 /* Split into 256 byte blocks. */ + ltgr %r6,%r6 + je .Lzero_lt256 +.Lzero_loop256: + mvc 1(256,%r2),0(%r2) /* Fill 256 zeros at once. */ + la %r2,256(%r2) + brctg %r6,.Lzero_loop256 /* Loop until all blocks are processed. */ +.Lzero_lt256: + exrl %r4,.Lmvc_lt256 + j .Lend +.Lmvc_lt256: + mvc 1(1,%r2),0(%r2) + +.Lremaining_v18: + vlr %v16,%v18 +.Lremaining_v16: + /* v16 contains the remaining bytes [1...16]. + Store remaining bytes and append string-termination. */ + vfenezf %v17,%v16,%v16 /* Find element not equal with zero search. */ + slgrk %r7,%r4,%r5 /* Remaining bytes = maxlen - current_len */ + aghi %r7,-1 /* vstl needs highest index. */ + la %r2,0(%r5,%r2) /* vstl has no index register. */ + vlgvb %r1,%v17,7 /* Load zero index or 16 if not found. */ + aghi %r1,3 /* Also copy remaining bytes of zero. */ + /* Zero in remaining bytes? -> jump away (zero-index <= max-index). */ + clrjle %r1,%r7,.Lfound_v16_store + vstl %v16,%r7,0(%r2) /* Store remaining bytes without null + termination! */ +.Lend: + /* Restore saved registers. */ + vlgvg %r6,%v31,0 + vlgvg %r7,%v31,1 + lgr %r2,%r0 /* Load saved dest-ptr. */ + br %r14 +.Lfallback: + jg __wcpncpy_c +END(__wcpncpy_vx) +#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */ diff --git a/sysdeps/s390/multiarch/wcpncpy.c b/sysdeps/s390/multiarch/wcpncpy.c new file mode 100644 index 0000000..e15acb7 --- /dev/null +++ b/sysdeps/s390/multiarch/wcpncpy.c @@ -0,0 +1,28 @@ +/* Multiple versions of wcpncpy. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) +# include +# include + +s390_vx_libc_ifunc (__wcpncpy) +weak_alias (__wcpncpy, wcpncpy) + +#else +# include +#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */ diff --git a/wcsmbs/Makefile b/wcsmbs/Makefile index 426eab0..1d07c51 100644 --- a/wcsmbs/Makefile +++ b/wcsmbs/Makefile @@ -43,7 +43,7 @@ routines := wcscat wcschr wcscmp wcscpy wcscspn wcsdup wcslen wcsncat \ mbrtoc16 c16rtomb strop-tests := wcscmp wcsncmp wmemcmp wcslen wcschr wcsrchr wcscpy wcsnlen \ - wcpcpy wcsncpy + wcpcpy wcsncpy wcpncpy tests := tst-wcstof wcsmbs-tst1 tst-wcsnlen tst-btowc tst-mbrtowc \ tst-wcrtomb tst-wcpncpy tst-mbsrtowcs tst-wchar-h tst-mbrtowc2 \ tst-c16c32-1 wcsatcliff $(addprefix test-,$(strop-tests)) diff --git a/wcsmbs/test-wcpncpy.c b/wcsmbs/test-wcpncpy.c new file mode 100644 index 0000000..de302b4 --- /dev/null +++ b/wcsmbs/test-wcpncpy.c @@ -0,0 +1,20 @@ +/* Test wcpncpy functions. + Copyright (C) 2015 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define WIDE 1 +#include "../string/test-stpncpy.c" diff --git a/wcsmbs/wcpncpy.c b/wcsmbs/wcpncpy.c index 98c77a5..e19e44c 100644 --- a/wcsmbs/wcpncpy.c +++ b/wcsmbs/wcpncpy.c @@ -18,6 +18,9 @@ #include +#ifdef WCPNCPY +# define __wcpncpy WCPNCPY +#endif /* Copy no more than N wide-characters of SRC to DEST, returning the address of the last character written into DEST. */ @@ -82,4 +85,6 @@ __wcpncpy (dest, src, n) return dest - 1; } +#ifndef WCPNCPY weak_alias (__wcpncpy, wcpncpy) +#endif