From patchwork Thu Aug 20 18:29:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raphael M Zinsly X-Patchwork-Id: 1348550 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=libc-alpha-bounces@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=sourceware.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=DcWLWGyi; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BXY8X58XJz9sPB for ; Fri, 21 Aug 2020 04:29:32 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5EFEE3851C3B; Thu, 20 Aug 2020 18:29:29 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5EFEE3851C3B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1597948169; bh=NjEYVO8LbxQYXYIEKE3h5bkBr6LNEkHnP81zEZ7Fowo=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:Cc:From; b=DcWLWGyiDFJe+ye1s1PP0yv3ZO8yTfMAMLDWo3dQBRgZfhky8Bx1q5latqV2GdCuv dSQsjaE3xAJepQFV5syT5GMUpCSwpUcuf0QZGTlce+WH6SwWKt8dtoW89DHtoNNGad I5xzQWfprBCsP/baPYHr41TR5jeZ3GFpLsPO30NQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by sourceware.org (Postfix) with ESMTPS id 9FF563857C5E for ; Thu, 20 Aug 2020 18:29:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 9FF563857C5E Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 07KI1wOb161965 for ; Thu, 20 Aug 2020 14:29:25 -0400 Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 331rvvbvd1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 20 Aug 2020 14:29:25 -0400 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 07KIKHNJ030543 for ; Thu, 20 Aug 2020 18:29:24 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma02wdc.us.ibm.com with ESMTP id 3304sd69uk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 20 Aug 2020 18:29:24 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 07KITNFG20316418 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Aug 2020 18:29:23 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7ED9B7805C; Thu, 20 Aug 2020 18:29:23 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0AB5B7805E; Thu, 20 Aug 2020 18:29:22 +0000 (GMT) Received: from localhost (unknown [9.163.29.102]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Thu, 20 Aug 2020 18:29:22 +0000 (GMT) To: libc-alpha@sourceware.org Subject: [PATCH 1/2] powerpc: Optimized strncpy for POWER9 Date: Thu, 20 Aug 2020 15:29:16 -0300 Message-Id: <20200820182917.12602-1-rzinsly@linux.ibm.com> X-Mailer: git-send-email 2.26.2 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-20_03:2020-08-19, 2020-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 mlxscore=0 malwarescore=0 bulkscore=0 priorityscore=1501 adultscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 clxscore=1015 spamscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008200142 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, KAM_NUMSUBJECT, KAM_SHORT, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Raphael Moreira Zinsly via Libc-alpha From: Raphael M Zinsly Reply-To: Raphael Moreira Zinsly Cc: Raphael Moreira Zinsly Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" Similar to the strcpy P9 optimization, this version uses VSX to improve performance. --- sysdeps/powerpc/powerpc64/le/power9/strncpy.S | 276 ++++++++++++++++++ sysdeps/powerpc/powerpc64/multiarch/Makefile | 2 +- .../powerpc64/multiarch/ifunc-impl-list.c | 5 + .../powerpc64/multiarch/strncpy-power9.S | 26 ++ sysdeps/powerpc/powerpc64/multiarch/strncpy.c | 7 + 5 files changed, 315 insertions(+), 1 deletion(-) create mode 100644 sysdeps/powerpc/powerpc64/le/power9/strncpy.S create mode 100644 sysdeps/powerpc/powerpc64/multiarch/strncpy-power9.S diff --git a/sysdeps/powerpc/powerpc64/le/power9/strncpy.S b/sysdeps/powerpc/powerpc64/le/power9/strncpy.S new file mode 100644 index 0000000000..cde68384d4 --- /dev/null +++ b/sysdeps/powerpc/powerpc64/le/power9/strncpy.S @@ -0,0 +1,276 @@ +/* Optimized strncpy implementation for PowerPC64/POWER9. + Copyright (C) 2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +# ifndef STRNCPY +# define FUNC_NAME strncpy +# else +# define FUNC_NAME STRNCPY +# endif + +/* Implements the function + + char * [r3] strncpy (char *dest [r3], const char *src [r4], size_t n [r5]) + + The implementation can load bytes past a null terminator, but only + up to the next 16B boundary, so it never crosses a page. */ + +.machine power9 +ENTRY_TOCLESS (FUNC_NAME, 4) + CALL_MCOUNT 2 + + cmpwi r5, 0 + beqlr + /* NULL string optimisation */ + lbz r0,0(r4) + stb r0,0(r3) + addi r11,r3,1 + addi r5,r5,-1 + vspltisb v18,0 /* Zeroes in v18 */ + cmpwi r0,0 + beq L(zero_padding_loop) + + cmpwi r5,0 + beqlr + +L(cont): + addi r4,r4,1 + neg r7,r4 + rldicl r9,r7,0,60 /* How many bytes to get source 16B aligned? */ + + /* Get source 16B aligned */ + lvx v0,0,r4 + lvsr v1,0,r4 + vperm v0,v18,v0,v1 + + vcmpequb v6,v0,v18 /* 0xff if byte is NULL, 0x00 otherwise */ + vctzlsbb r7,v6 /* Number of trailing zeroes */ + addi r8,r7,1 /* Add null terminator */ + + /* r8 = bytes including null + r9 = bytes to get source 16B aligned + if r8 > r9 + no null, copy r9 bytes + else + there is a null, copy r8 bytes and return. */ + cmpd r8,r9 + bgt L(no_null) + + cmpd r8,r5 /* r8 <= n? */ + ble L(null) + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + stxvl 32+v0,r11,r10 /* Partial store */ + + blr + +L(null): + sldi r10,r8,56 /* stxvl wants size in top 8 bits */ + stxvl 32+v0,r11,r10 /* Partial store */ + + add r11,r11,r8 + sub r5,r5,r8 + b L(zero_padding_loop) + +L(no_null): + cmpd r9,r5 /* Check if length was reached. */ + bge L(n_tail1) + sldi r10,r9,56 /* stxvl wants size in top 8 bits */ + stxvl 32+v0,r11,r10 /* Partial store */ + + add r4,r4,r9 + add r11,r11,r9 + sub r5,r5,r9 + +L(loop): + cmpldi cr6,r5,64 /* Check if length was reached. */ + ble cr6,L(final_loop) + + lxv 32+v0,0(r4) + vcmpequb. v6,v0,v18 /* Any zero bytes? */ + bne cr6,L(prep_tail1) + + lxv 32+v1,16(r4) + vcmpequb. v6,v1,v18 /* Any zero bytes? */ + bne cr6,L(prep_tail2) + + lxv 32+v2,32(r4) + vcmpequb. v6,v2,v18 /* Any zero bytes? */ + bne cr6,L(prep_tail3) + + lxv 32+v3,48(r4) + vcmpequb. v6,v3,v18 /* Any zero bytes? */ + bne cr6,L(prep_tail4) + + stxv 32+v0,0(r11) + stxv 32+v1,16(r11) + stxv 32+v2,32(r11) + stxv 32+v3,48(r11) + + addi r4,r4,64 + addi r11,r11,64 + addi r5,r5,-64 + + b L(loop) + +L(final_loop): + cmpldi cr5,r5,16 + lxv 32+v0,0(r4) + vcmpequb. v6,v0,v18 /* Any zero bytes? */ + ble cr5,L(prep_n_tail1) + bne cr6,L(count_tail1) + addi r5,r5,-16 + + cmpldi cr5,r5,16 + lxv 32+v1,16(r4) + vcmpequb. v6,v1,v18 /* Any zero bytes? */ + ble cr5,L(prep_n_tail2) + bne cr6,L(count_tail2) + addi r5,r5,-16 + + cmpldi cr5,r5,16 + lxv 32+v2,32(r4) + vcmpequb. v6,v2,v18 /* Any zero bytes? */ + ble cr5,L(prep_n_tail3) + bne cr6,L(count_tail3) + addi r5,r5,-16 + + lxv 32+v3,48(r4) + vcmpequb. v6,v3,v18 /* Any zero bytes? */ + beq cr6,L(n_tail4) + + vctzlsbb r8,v6 /* Number of trailing zeroes */ + cmpd r8,r5 /* r8 < n? */ + blt L(tail4) +L(n_tail4): + stxv 32+v0,0(r11) + stxv 32+v1,16(r11) + stxv 32+v2,32(r11) + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,48 /* Offset */ + stxvl 32+v3,r11,r10 /* Partial store */ + blr + +L(prep_n_tail1): + beq cr6,L(n_tail1) /* Any zero bytes? */ + vctzlsbb r8,v6 /* Number of trailing zeroes */ + cmpd r8,r5 /* r8 < n? */ + blt L(tail1) +L(n_tail1): + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + stxvl 32+v0,r11,r10 /* Partial store */ + blr + +L(prep_n_tail2): + beq cr6,L(n_tail2) /* Any zero bytes? */ + vctzlsbb r8,v6 /* Number of trailing zeroes */ + cmpd r8,r5 /* r8 < n? */ + blt L(tail2) +L(n_tail2): + stxv 32+v0,0(r11) + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,16 /* offset */ + stxvl 32+v1,r11,r10 /* Partial store */ + blr + +L(prep_n_tail3): + beq cr6,L(n_tail3) /* Any zero bytes? */ + vctzlsbb r8,v6 /* Number of trailing zeroes */ + cmpd r8,r5 /* r8 < n? */ + blt L(tail3) +L(n_tail3): + stxv 32+v0,0(r11) + stxv 32+v1,16(r11) + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,32 /* Offset */ + stxvl 32+v2,r11,r10 /* Partial store */ + blr + +L(prep_tail1): +L(count_tail1): + vctzlsbb r8,v6 /* Number of trailing zeroes */ +L(tail1): + addi r9,r8,1 /* Add null terminator */ + sldi r10,r9,56 /* stxvl wants size in top 8 bits */ + stxvl 32+v0,r11,r10 /* Partial store */ + add r11,r11,r9 + sub r5,r5,r9 + b L(zero_padding_loop) + +L(prep_tail2): + addi r5,r5,-16 +L(count_tail2): + vctzlsbb r8,v6 /* Number of trailing zeroes */ +L(tail2): + addi r9,r8,1 /* Add null terminator */ + stxv 32+v0,0(r11) + sldi r10,r9,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,16 /* offset */ + stxvl 32+v1,r11,r10 /* Partial store */ + add r11,r11,r9 + sub r5,r5,r9 + b L(zero_padding_loop) + +L(prep_tail3): + addi r5,r5,-32 +L(count_tail3): + vctzlsbb r8,v6 /* Number of trailing zeroes */ +L(tail3): + addi r9,r8,1 /* Add null terminator */ + stxv 32+v0,0(r11) + stxv 32+v1,16(r11) + sldi r10,r9,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,32 /* offset */ + stxvl 32+v2,r11,r10 /* Partial store */ + add r11,r11,r9 + sub r5,r5,r9 + b L(zero_padding_loop) + +L(prep_tail4): + addi r5,r5,-48 + vctzlsbb r8,v6 /* Number of trailing zeroes */ +L(tail4): + addi r9,r8,1 /* Add null terminator */ + stxv 32+v0,0(r11) + stxv 32+v1,16(r11) + stxv 32+v2,32(r11) + sldi r10,r9,56 /* stxvl wants size in top 8 bits */ + addi r11,r11,48 /* offset */ + stxvl 32+v3,r11,r10 /* Partial store */ + add r11,r11,r9 + sub r5,r5,r9 + +/* This code pads the remainder of dest with NULL bytes. */ +L(zero_padding_loop): + cmpldi cr6,r5,16 /* Check if length was reached. */ + ble cr6,L(zero_padding_end) + + stxv v18,0(r11) + addi r11,r11,16 + addi r5,r5,-16 + + b L(zero_padding_loop) + +L(zero_padding_end): + sldi r10,r5,56 /* stxvl wants size in top 8 bits */ + stxvl v18,r11,r10 /* Partial store */ + blr + +L(n_tail): + +END (FUNC_NAME) diff --git a/sysdeps/powerpc/powerpc64/multiarch/Makefile b/sysdeps/powerpc/powerpc64/multiarch/Makefile index 19acb6c64a..cd2b47b403 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/Makefile +++ b/sysdeps/powerpc/powerpc64/multiarch/Makefile @@ -33,7 +33,7 @@ sysdep_routines += memcpy-power8-cached memcpy-power7 memcpy-a2 memcpy-power6 \ ifneq (,$(filter %le,$(config-machine))) sysdep_routines += strcmp-power9 strncmp-power9 strcpy-power9 stpcpy-power9 \ - rawmemchr-power9 strlen-power9 + rawmemchr-power9 strlen-power9 strncpy-power9 endif CFLAGS-strncase-power7.c += -mcpu=power7 -funroll-loops CFLAGS-strncase_l-power7.c += -mcpu=power7 -funroll-loops diff --git a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c index ea10b00417..aa63e1c23f 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c +++ b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c @@ -301,6 +301,11 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/powerpc/powerpc64/multiarch/strncpy.c. */ IFUNC_IMPL (i, name, strncpy, +#ifdef __LITTLE_ENDIAN__ + IFUNC_IMPL_ADD (array, i, strncpy, + hwcap2 & PPC_FEATURE2_ARCH_3_00, + __strncpy_power9) +#endif IFUNC_IMPL_ADD (array, i, strncpy, hwcap2 & PPC_FEATURE2_ARCH_2_07, __strncpy_power8) diff --git a/sysdeps/powerpc/powerpc64/multiarch/strncpy-power9.S b/sysdeps/powerpc/powerpc64/multiarch/strncpy-power9.S new file mode 100644 index 0000000000..b9b6092f7b --- /dev/null +++ b/sysdeps/powerpc/powerpc64/multiarch/strncpy-power9.S @@ -0,0 +1,26 @@ +/* Optimized strncpy implementation for POWER9/PPC64. + Copyright (C) 2016-2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if defined __LITTLE_ENDIAN__ && IS_IN (libc) +#define STRNCPY __strncpy_power9 + +#undef libc_hidden_builtin_def +#define libc_hidden_builtin_def(name) + +#include +#endif diff --git a/sysdeps/powerpc/powerpc64/multiarch/strncpy.c b/sysdeps/powerpc/powerpc64/multiarch/strncpy.c index 7bacf28aca..822ceb2003 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/strncpy.c +++ b/sysdeps/powerpc/powerpc64/multiarch/strncpy.c @@ -28,11 +28,18 @@ extern __typeof (strncpy) __strncpy_ppc attribute_hidden; extern __typeof (strncpy) __strncpy_power7 attribute_hidden; extern __typeof (strncpy) __strncpy_power8 attribute_hidden; +# ifdef __LITTLE_ENDIAN__ +extern __typeof (strncpy) __strncpy_power9 attribute_hidden; +# endif # undef strncpy /* Avoid DWARF definition DIE on ifunc symbol so that GDB can handle ifunc symbol properly. */ libc_ifunc_redirected (__redirect_strncpy, strncpy, +# ifdef __LITTLE_ENDIAN__ + (hwcap2 & PPC_FEATURE2_ARCH_3_00) + ? __strncpy_power9 : +# endif (hwcap2 & PPC_FEATURE2_ARCH_2_07) ? __strncpy_power8 : (hwcap & PPC_FEATURE_HAS_VSX) From patchwork Thu Aug 20 18:29:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raphael M Zinsly X-Patchwork-Id: 1348551 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=libc-alpha-bounces@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=sourceware.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=VW905vmw; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BXY8n3sN8z9sPB for ; Fri, 21 Aug 2020 04:29:45 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id E4B903870896; Thu, 20 Aug 2020 18:29:42 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org E4B903870896 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1597948183; bh=FUPTxga7ZeSxFPuj62saU3k8y/2+BqEY0cSrqbdGgZ4=; h=To:Subject:Date:In-Reply-To:References:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=VW905vmwp2PSDkJ8Uk/mPutLuOCdv+s1nwPfS59ByyEnganO4bZXENS4QTV7YGuMu /aT+zB0ablQf512i9Rd0L8zbnizwcFwIB63tCgJKcA1Ng4TPBj8LBDzblZq3dapsXS FsjfhOLOr7fbHMt4rX5EzFEoBkkQHPcw2jtJaYZQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id 62AFC386F823 for ; Thu, 20 Aug 2020 18:29:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 62AFC386F823 Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 07KI1jQR151703 for ; Thu, 20 Aug 2020 14:29:39 -0400 Received: from ppma03dal.us.ibm.com (b.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.11]) by mx0a-001b2d01.pphosted.com with ESMTP id 3304rvj6vb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 20 Aug 2020 14:29:39 -0400 Received: from pps.filterd (ppma03dal.us.ibm.com [127.0.0.1]) by ppma03dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 07KIJXPp010967 for ; Thu, 20 Aug 2020 18:29:38 GMT Received: from b01cxnp22034.gho.pok.ibm.com (b01cxnp22034.gho.pok.ibm.com [9.57.198.24]) by ppma03dal.us.ibm.com with ESMTP id 3304cd3yew-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 20 Aug 2020 18:29:38 +0000 Received: from b01ledav004.gho.pok.ibm.com (b01ledav004.gho.pok.ibm.com [9.57.199.109]) by b01cxnp22034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 07KITbFD41419100 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 20 Aug 2020 18:29:37 GMT Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ADC4E112061; Thu, 20 Aug 2020 18:29:37 +0000 (GMT) Received: from b01ledav004.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 22059112062; Thu, 20 Aug 2020 18:29:37 +0000 (GMT) Received: from localhost (unknown [9.163.29.102]) by b01ledav004.gho.pok.ibm.com (Postfix) with ESMTP; Thu, 20 Aug 2020 18:29:36 +0000 (GMT) To: libc-alpha@sourceware.org Subject: [PATCH 2/2] powerpc: Optimzed stpncpy for POWER9 Date: Thu, 20 Aug 2020 15:29:17 -0300 Message-Id: <20200820182917.12602-2-rzinsly@linux.ibm.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200820182917.12602-1-rzinsly@linux.ibm.com> References: <20200820182917.12602-1-rzinsly@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-08-20_03:2020-08-19, 2020-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 mlxlogscore=868 priorityscore=1501 suspectscore=0 phishscore=0 spamscore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2008200142 X-Spam-Status: No, score=-11.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, GIT_PATCH_0, KAM_NUMSUBJECT, KAM_SHORT, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Raphael Moreira Zinsly via Libc-alpha From: Raphael M Zinsly Reply-To: Raphael Moreira Zinsly Cc: Raphael Moreira Zinsly Errors-To: libc-alpha-bounces@sourceware.org Sender: "Libc-alpha" Adds stpncpy support into the POWER9 strncpy. --- sysdeps/powerpc/powerpc64/le/power9/stpncpy.S | 24 ++++++ sysdeps/powerpc/powerpc64/le/power9/strncpy.S | 74 +++++++++++++++++++ sysdeps/powerpc/powerpc64/multiarch/Makefile | 2 +- .../powerpc64/multiarch/ifunc-impl-list.c | 5 ++ .../powerpc64/multiarch/stpncpy-power9.S | 24 ++++++ sysdeps/powerpc/powerpc64/multiarch/stpncpy.c | 7 ++ 6 files changed, 135 insertions(+), 1 deletion(-) create mode 100644 sysdeps/powerpc/powerpc64/le/power9/stpncpy.S create mode 100644 sysdeps/powerpc/powerpc64/multiarch/stpncpy-power9.S diff --git a/sysdeps/powerpc/powerpc64/le/power9/stpncpy.S b/sysdeps/powerpc/powerpc64/le/power9/stpncpy.S new file mode 100644 index 0000000000..a96840bb6f --- /dev/null +++ b/sysdeps/powerpc/powerpc64/le/power9/stpncpy.S @@ -0,0 +1,24 @@ +/* Optimized stpncpy implementation for PowerPC64/POWER9. + Copyright (C) 2015-2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define USE_AS_STPNCPY +#include + +weak_alias (__stpncpy, stpncpy) +libc_hidden_def (__stpncpy) +libc_hidden_builtin_def (stpncpy) diff --git a/sysdeps/powerpc/powerpc64/le/power9/strncpy.S b/sysdeps/powerpc/powerpc64/le/power9/strncpy.S index cde68384d4..64b06a9040 100644 --- a/sysdeps/powerpc/powerpc64/le/power9/strncpy.S +++ b/sysdeps/powerpc/powerpc64/le/power9/strncpy.S @@ -18,16 +18,30 @@ #include +#ifdef USE_AS_STPNCPY +# ifndef STPNCPY +# define FUNC_NAME __stpncpy +# else +# define FUNC_NAME STPNCPY +# endif +#else # ifndef STRNCPY # define FUNC_NAME strncpy # else # define FUNC_NAME STRNCPY # endif +#endif /* !USE_AS_STPNCPY */ /* Implements the function char * [r3] strncpy (char *dest [r3], const char *src [r4], size_t n [r5]) + or + + char * [r3] stpncpy (char *dest [r3], const char *src [r4], size_t n [r5]) + + if USE_AS_STPNCPY is defined. + The implementation can load bytes past a null terminator, but only up to the next 16B boundary, so it never crosses a page. */ @@ -47,6 +61,13 @@ ENTRY_TOCLESS (FUNC_NAME, 4) beq L(zero_padding_loop) cmpwi r5,0 +#ifdef USE_AS_STPNCPY + bgt L(cont) + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + addi r3,r3,1 + blr +#endif beqlr L(cont): @@ -77,12 +98,22 @@ L(cont): sldi r10,r5,56 /* stxvl wants size in top 8 bits */ stxvl 32+v0,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r5 +#endif blr L(null): sldi r10,r8,56 /* stxvl wants size in top 8 bits */ stxvl 32+v0,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r7 +#endif add r11,r11,r8 sub r5,r5,r8 b L(zero_padding_loop) @@ -164,6 +195,11 @@ L(n_tail4): sldi r10,r5,56 /* stxvl wants size in top 8 bits */ addi r11,r11,48 /* Offset */ stxvl 32+v3,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r5 +#endif blr L(prep_n_tail1): @@ -174,6 +210,11 @@ L(prep_n_tail1): L(n_tail1): sldi r10,r5,56 /* stxvl wants size in top 8 bits */ stxvl 32+v0,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r5 +#endif blr L(prep_n_tail2): @@ -186,6 +227,11 @@ L(n_tail2): sldi r10,r5,56 /* stxvl wants size in top 8 bits */ addi r11,r11,16 /* offset */ stxvl 32+v1,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r5 +#endif blr L(prep_n_tail3): @@ -199,6 +245,11 @@ L(n_tail3): sldi r10,r5,56 /* stxvl wants size in top 8 bits */ addi r11,r11,32 /* Offset */ stxvl 32+v2,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r5 +#endif blr L(prep_tail1): @@ -208,6 +259,11 @@ L(tail1): addi r9,r8,1 /* Add null terminator */ sldi r10,r9,56 /* stxvl wants size in top 8 bits */ stxvl 32+v0,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r8 +#endif add r11,r11,r9 sub r5,r5,r9 b L(zero_padding_loop) @@ -222,6 +278,11 @@ L(tail2): sldi r10,r9,56 /* stxvl wants size in top 8 bits */ addi r11,r11,16 /* offset */ stxvl 32+v1,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r8 +#endif add r11,r11,r9 sub r5,r5,r9 b L(zero_padding_loop) @@ -237,6 +298,11 @@ L(tail3): sldi r10,r9,56 /* stxvl wants size in top 8 bits */ addi r11,r11,32 /* offset */ stxvl 32+v2,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r8 +#endif add r11,r11,r9 sub r5,r5,r9 b L(zero_padding_loop) @@ -252,6 +318,11 @@ L(tail4): sldi r10,r9,56 /* stxvl wants size in top 8 bits */ addi r11,r11,48 /* offset */ stxvl 32+v3,r11,r10 /* Partial store */ +#ifdef USE_AS_STPNCPY + /* stpncpy returns the dest address plus the size not counting the + final '\0'. */ + add r3,r11,r8 +#endif add r11,r11,r9 sub r5,r5,r9 @@ -274,3 +345,6 @@ L(zero_padding_end): L(n_tail): END (FUNC_NAME) +#ifndef USE_AS_STPNCPY +libc_hidden_builtin_def (strncpy) +#endif diff --git a/sysdeps/powerpc/powerpc64/multiarch/Makefile b/sysdeps/powerpc/powerpc64/multiarch/Makefile index cd2b47b403..f46bf50732 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/Makefile +++ b/sysdeps/powerpc/powerpc64/multiarch/Makefile @@ -33,7 +33,7 @@ sysdep_routines += memcpy-power8-cached memcpy-power7 memcpy-a2 memcpy-power6 \ ifneq (,$(filter %le,$(config-machine))) sysdep_routines += strcmp-power9 strncmp-power9 strcpy-power9 stpcpy-power9 \ - rawmemchr-power9 strlen-power9 strncpy-power9 + rawmemchr-power9 strlen-power9 strncpy-power9 stpncpy-power9 endif CFLAGS-strncase-power7.c += -mcpu=power7 -funroll-loops CFLAGS-strncase_l-power7.c += -mcpu=power7 -funroll-loops diff --git a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c index aa63e1c23f..56790bcfe3 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c +++ b/sysdeps/powerpc/powerpc64/multiarch/ifunc-impl-list.c @@ -317,6 +317,11 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, /* Support sysdeps/powerpc/powerpc64/multiarch/stpncpy.c. */ IFUNC_IMPL (i, name, stpncpy, +#ifdef __LITTLE_ENDIAN__ + IFUNC_IMPL_ADD (array, i, stpncpy, + hwcap2 & PPC_FEATURE2_ARCH_3_00, + __stpncpy_power9) +#endif IFUNC_IMPL_ADD (array, i, stpncpy, hwcap2 & PPC_FEATURE2_ARCH_2_07, __stpncpy_power8) diff --git a/sysdeps/powerpc/powerpc64/multiarch/stpncpy-power9.S b/sysdeps/powerpc/powerpc64/multiarch/stpncpy-power9.S new file mode 100644 index 0000000000..ecbbb5c8e9 --- /dev/null +++ b/sysdeps/powerpc/powerpc64/multiarch/stpncpy-power9.S @@ -0,0 +1,24 @@ +/* Optimized stpncpy implementation for POWER9/PPC64. + Copyright (C) 2015-2020 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define STPNCPY __stpncpy_power9 + +#undef libc_hidden_builtin_def +#define libc_hidden_builtin_def(name) + +#include diff --git a/sysdeps/powerpc/powerpc64/multiarch/stpncpy.c b/sysdeps/powerpc/powerpc64/multiarch/stpncpy.c index 17df886431..21702716a3 100644 --- a/sysdeps/powerpc/powerpc64/multiarch/stpncpy.c +++ b/sysdeps/powerpc/powerpc64/multiarch/stpncpy.c @@ -26,10 +26,17 @@ extern __typeof (__stpncpy) __stpncpy_ppc attribute_hidden; extern __typeof (__stpncpy) __stpncpy_power7 attribute_hidden; extern __typeof (__stpncpy) __stpncpy_power8 attribute_hidden; +# ifdef __LITTLE_ENDIAN__ +extern __typeof (__stpncpy) __stpncpy_power9 attribute_hidden; +# endif # undef stpncpy # undef __stpncpy libc_ifunc_redirected (__redirect___stpncpy, __stpncpy, +# ifdef __LITTLE_ENDIAN__ + (hwcap2 & PPC_FEATURE2_ARCH_3_00) + ? __stpncpy_power9 : +# endif (hwcap2 & PPC_FEATURE2_ARCH_2_07) ? __stpncpy_power8 : (hwcap & PPC_FEATURE_HAS_VSX)