From patchwork Thu Aug 3 12:27:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1816430 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=fb5ZnGc/; dkim-atps=neutral Received: from server2.sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RGp7L1SBDz1yZl for ; Thu, 3 Aug 2023 22:29:18 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3873B385C6DF for ; Thu, 3 Aug 2023 12:29:16 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 3873B385C6DF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691065756; bh=BP4LtrHZvsEHOtqRWC7wC+e5+0Mx5dS495VmPuUkWx8=; h=To:CC:Subject:Date:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:From; b=fb5ZnGc/ia04C02ERb0fqpeA11Hoxl7pjY1wIJIQMFav2uCH9DEKi1R5knzhMxzvT 6LiJEKzs6xh0hsqgpbIYSvEanCgP9F9kwB2NhqljVqW5Vw/aJFV2YHfa4eWdibupUv v/dGjHI85KzT1bGBh2k2EloGi682uAQefaf+dh0E= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04on2085.outbound.protection.outlook.com [40.107.8.85]) by sourceware.org (Postfix) with ESMTPS id 080333858C20 for ; Thu, 3 Aug 2023 12:27:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 080333858C20 Received: from DB6PR0301CA0086.eurprd03.prod.outlook.com (2603:10a6:6:30::33) by DU0PR08MB9049.eurprd08.prod.outlook.com (2603:10a6:10:471::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:31 +0000 Received: from DBAEUR03FT048.eop-EUR03.prod.protection.outlook.com (2603:10a6:6:30:cafe::f3) by DB6PR0301CA0086.outlook.office365.com (2603:10a6:6:30::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Thu, 3 Aug 2023 12:27:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT048.mail.protection.outlook.com (100.127.142.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:29 +0000 Received: ("Tessian outbound 997ae1cc9f47:v145"); Thu, 03 Aug 2023 12:27:29 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: e5dd423b9559ed51 X-CR-MTA-TID: 64aa7808 Received: from d980dd0ec4b3.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 0AD3358A-48FB-4428-9F8F-FBA5A7FCA590.1; Thu, 03 Aug 2023 12:27:24 +0000 Received: from EUR02-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d980dd0ec4b3.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 03 Aug 2023 12:27:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aRQCe9xsH9E2FyY/Nl4l7kzEvkjRtHwFrnMX0s3db1PO3iCImIP8ZV7Vv2eOp1PJ47mD36zXZyrUFm7jrLjRGxAGx5BqWm9zSfxzlId24Nl2pqGpZrHVroPlPN6ZIeD7gnmbtqJFlcfGljNMeWIztCIrTtYK1hFpB1vzGsAN05R0Jev91uSKabTsr6RM16QfffTxG6fjoKh5bnO2XAdyq7LmNl+Cxrb0WaZ9/rlhkG62jFaf3aJvMb5vdIVKDe7QduBLbHzeC7zwB0wgbtLJ6CB7L1568RUwYSgdONRCTg73H/jjKyQkFFeeFkQyi0Tojxw62lfW50Dn7IO+5G1NQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BP4LtrHZvsEHOtqRWC7wC+e5+0Mx5dS495VmPuUkWx8=; b=AMrRCtXz6nfcBEhwEprKSVAN253A+ts4SnisOtmW2uv8PZIW2CVE1bcey7HZN4DxkqiWcqOLBbaXsB5WiCMA4k40aIAS+nqKMe7f4prevjloGpnrb90/761h3LR0XDkuePH8X02/0jYFkybXxXbUgtO7sfdI/ugemngN8OB4qlVYbxOuqDyWBcL+oqjNEzlIssvucvtKiYy/AiETUJ4t8PcdxL2n8b+8gXKt2OMA0r9rMYnkNX9kARnMBjdxEq5z7aME8DFNNrFaFaL+CDx4Wc6KKtQfgahXILltiuaUKFY2leD3NIVzLsmy6FeqskHSEsIIFVh7Y5rKz4fkd9ayeQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from DB7PR02CA0003.eurprd02.prod.outlook.com (2603:10a6:10:52::16) by DBBPR08MB6140.eurprd08.prod.outlook.com (2603:10a6:10:20a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:20 +0000 Received: from DBAEUR03FT049.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:52:cafe::7c) by DB7PR02CA0003.outlook.office365.com (2603:10a6:10:52::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT049.mail.protection.outlook.com (100.127.142.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 Received: from AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:19 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:18 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 3 Aug 2023 12:27:18 +0000 To: CC: Joe Ramsay Subject: [PATCH 1/5] aarch64: Add vector implementations of tan routines Date: Thu, 3 Aug 2023 13:27:13 +0100 Message-ID: <20230803122717.33413-1-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DBAEUR03FT049:EE_|DBBPR08MB6140:EE_|DBAEUR03FT048:EE_|DU0PR08MB9049:EE_ X-MS-Office365-Filtering-Correlation-Id: e33d13db-f55a-4d9e-4e52-08db941d0144 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: lx9/KXHftZ9+iNUfJrYiVXeDqJcATZJc0Y2Y5GPJVAZLGdBiKeuKC2diq+EMvbSsxxDbSylzFz6QQGF/ur4V9HqWbf4CXnD38RZzqoGKMfxPqd33o0dgFWh145ueluC8FrTcLjmHdXRQthsiNFBvVsZNtC1jnsx4qGKIEYHT0FEwt/SBfv70R6xuM1g2S2dH+7NxdriiZRQx+X4bmN8Mtldj6b3W2p7rHLqdLGHT4gjprBm4kpCVGbQwWlRbQodnWT+S7d4S6CIVZtOzQZIGxGdN+axY9/+pI2WSHK2kpqw7zWQaYUSsHMYLtyn7XM1qb2LksSI6OTlHfpr2mxhbQ7tAYeb9UlHfnz2X3tQ7gZRK0ELq8gvPscFtcCuF/tWCgXnzGJbvorPmfI/iYHLU6GLPxgYkLHXlgiOmgjj8YxvSQ6TccD1B+mlkD8mS6aEUEXrFXUW/1k5YRDdAvcVlqkNIZ1tPq0MNcZLNfekJ90/hnbWDb5u4FVyT4qCDbIyFmqlR9wsPwsXRqWn0JcmfZq9V5XXKxKggTHWHJMbzvoNS8pBobegoo40KHZDPiR2HQJPqACWJYnCvTXtXElId02EJ3NgI0SmuaWn7fEYeaBlVXHh215j+cvkGMQ6ySckQWkxHcMssRBw2YmOkyfhH9zx+QYLvp7PyNaOBaW3bm9A63JlDmu3xE4VjyDcoP0bulbaMoXUaTaS7qw7AGb9htEm92cGPvAZOTPsMpFGk/kj+susgynzZirxpYcuHjipBmtPX2iLJnspintHWq/k3K8GLoJ2IJHqg/Cu0j20wsssTHRpWE8Np7Woxw4H9AYbn X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(346002)(39860400002)(376002)(451199021)(82310400008)(40470700004)(46966006)(36840700001)(40460700003)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(6666004)(478600001)(40480700001)(356005)(81166007)(86362001)(36756003)(82740400003)(2004002)(36900700001)(579004)(559001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6140 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT048.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 8b851f16-170f-4d08-b908-08db941cfbce X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DSBMvPnT5vPubTiDBepC/gteKFN1hg+XovqWGv70xwFjvyR6ghXgKH0nq/pyE5tfDJPgZaqKR7+Nh95BwIuRlnhpcShDbiHJrzui9fTh/8UkAAmRomql44gk/Fs81UoXu4xUeiKwXT9/bIhBtEKxIUdYzo60r1OGB4uKIzuBPDvZbOx/B386wLRIDoa7SVUUL4MXwnqpxEzY5g02enlWhQKARWUIqwQXdGoD7s/8ELHdUDZa75W4DmrNcDXiaUOwTz1o/nxgrn4UANFP+N1QJGBh2SBBrIeLu1VZoEYaBgShUMw53caIZq0XDsErLGbl1V6pnJ60sjfaTX2zUROvabyk+OTnJoExi+DGPOuNAevsG0VaW0R/uEoZz/5Gt6BQP786KPauKTit+U4lofQp5+88RLjm2qz0ZP06Vy8Dm0rJFFhxBI9ouHvwzo8JQ6D/wcrIysV0kxD/TeE0pLwH6moB1FHDFHkTnb7xbyEQqYSPZ8mjhu0Jb9yhFbBoVdBret3WkBfaLu0KPga7Px8YdY9rTvdpRFtxW+J7D+0k0vlHu51/TX+woUvWXh+0xxaPQ/Afl5fykEckiTCX3V2iznHpphdhGoiplYcFUmooLcgDzsyeoFbIJ2HF//Tdpk0BePoaD92TzHSZHgFzZzCD83AFisVq3iXnxcPJrBLI46H7gMPnD25+F/DFYW/DSIOLJc8rx5KreZ1i/CZS11WZaaXzZfuBPgW7qhEluAo0ILc+VbmMHMgZuLC3Djz6f/Kk9xPROyW8PpuNzmrmqnydHQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(376002)(136003)(39860400002)(346002)(396003)(451199021)(82310400008)(36840700001)(46966006)(40470700004)(40460700003)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(6666004)(478600001)(40480700001)(81166007)(86362001)(36756003)(82740400003)(2004002)(559001)(579004)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 12:27:29.9273 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e33d13db-f55a-4d9e-4e52-08db941d0144 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT048.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9049 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" This includes some utility headers for evaluating polynomials using various schemes. --- Thanks, Joe sysdeps/aarch64/fpu/Makefile | 3 +- sysdeps/aarch64/fpu/Versions | 6 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/poly_advsimd_f32.h | 36 ++ sysdeps/aarch64/fpu/poly_advsimd_f64.h | 36 ++ sysdeps/aarch64/fpu/poly_generic.h | 285 ++++++++++++++++ sysdeps/aarch64/fpu/poly_sve_f32.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_f64.h | 38 +++ sysdeps/aarch64/fpu/poly_sve_generic.h | 313 ++++++++++++++++++ sysdeps/aarch64/fpu/tan_advsimd.c | 123 +++++++ sysdeps/aarch64/fpu/tan_sve.c | 106 ++++++ sysdeps/aarch64/fpu/tanf_advsimd.c | 134 ++++++++ sysdeps/aarch64/fpu/tanf_sve.c | 123 +++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 19 files changed, 1260 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_advsimd_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_generic.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f32.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_f64.h create mode 100644 sysdeps/aarch64/fpu/poly_sve_generic.h create mode 100644 sysdeps/aarch64/fpu/tan_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tan_sve.c create mode 100644 sysdeps/aarch64/fpu/tanf_advsimd.c create mode 100644 sysdeps/aarch64/fpu/tanf_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 04aa2e37ca..a1bbc9bcaa 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,7 +1,8 @@ libmvec-supported-funcs = cos \ exp \ log \ - sin + sin \ + tan float-advsimd-funcs = $(libmvec-supported-funcs) double-advsimd-funcs = $(libmvec-supported-funcs) diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index c85c0f3efb..f0ca0940a9 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -17,4 +17,10 @@ libmvec { _ZGVsMxv_sin; _ZGVsMxv_sinf; } + GLIBC_2.39 { + _ZGVnN4v_tanf; + _ZGVnN2v_tan; + _ZGVsMxv_tanf; + _ZGVsMxv_tan; + } } diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 7c200599c1..6193213147 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,11 +53,13 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); # undef __ADVSIMD_VEC_MATH_SUPPORTED #endif /* __ADVSIMD_VEC_MATH_SUPPORTED */ @@ -68,11 +70,13 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); # undef __SVE_VEC_MATH_SUPPORTED #endif /* __SVE_VEC_MATH_SUPPORTED */ diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f32.h b/sysdeps/aarch64/fpu/poly_advsimd_f32.h new file mode 100644 index 0000000000..9e2ad9ad94 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f32.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on single-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F32_H +#define AARCH64_FPU_POLY_ADVSIMD_F32_H + +#include + +/* Wrap AdvSIMD f32 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f32. */ +#define VTYPE float32x4_t +#define FMA(x, y, z) vfmaq_f32 (z, x, y) +#define VWRAP(f) v_##f##_f32 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_advsimd_f64.h b/sysdeps/aarch64/fpu/poly_advsimd_f64.h new file mode 100644 index 0000000000..955cfc08ce --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_advsimd_f64.h @@ -0,0 +1,36 @@ +/* Helpers for evaluating polynomials on double-precision AdvSIMD input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_ADVSIMD_F64_H +#define AARCH64_FPU_POLY_ADVSIMD_F64_H + +#include + +/* Wrap AdvSIMD f64 helpers: evaluation of some scheme/order has form: + v_[scheme]_[order]_f64. */ +#define VTYPE float64x2_t +#define FMA(x, y, z) vfmaq_f64 (z, x, y) +#define VWRAP(f) v_##f##_f64 +#include "poly_generic.h" +#undef VWRAP +#undef FMA +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_generic.h b/sysdeps/aarch64/fpu/poly_generic.h new file mode 100644 index 0000000000..84f042182b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_generic.h @@ -0,0 +1,285 @@ +/* Generic helpers for evaluating polynomials with various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef FMA +# error Cannot use poly_generic without defining FMA +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (VTYPE x, VTYPE x2, + const VTYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + return FMA (p23, x2, p01); +} + +static inline VTYPE VWRAP (estrin_4) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + return FMA (poly[4], x4, p03); +} +static inline VTYPE VWRAP (estrin_5) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + return FMA (p45, x4, p03); +} +static inline VTYPE VWRAP (estrin_6) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p46 = FMA (poly[6], x2, p45); + return FMA (p46, x4, p03); +} +static inline VTYPE VWRAP (estrin_7) (VTYPE x, VTYPE x2, VTYPE x4, + const VTYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (x, x2, poly + 4); + return FMA (p47, x4, p03); +} +static inline VTYPE VWRAP (estrin_8) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (poly[8], x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_9) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + return FMA (p89, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_10) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p89 = FMA (poly[9], x, poly[8]); + VTYPE p8_10 = FMA (poly[10], x2, p89); + return FMA (p8_10, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_11) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (x, x2, poly + 8); + return FMA (p8_11, x8, VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_12) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_4) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_13) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_5) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_14) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_6) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_15) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + const VTYPE *poly) +{ + return FMA (VWRAP (estrin_7) (x, x2, x4, poly + 8), x8, + VWRAP (estrin_7) (x, x2, x4, poly)); +} +static inline VTYPE VWRAP (estrin_16) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + return FMA (poly[16], x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_17) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + return FMA (p16_17, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_18) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_17 = FMA (poly[17], x, poly[16]); + VTYPE p16_18 = FMA (poly[18], x2, p16_17); + return FMA (p16_18, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} +static inline VTYPE VWRAP (estrin_19) (VTYPE x, VTYPE x2, VTYPE x4, VTYPE x8, + VTYPE x16, const VTYPE *poly) +{ + VTYPE p16_19 = VWRAP (pairwise_poly_3) (x, x2, poly + 16); + return FMA (p16_19, x16, VWRAP (estrin_15) (x, x2, x4, x8, poly)); +} + +static inline VTYPE VWRAP (horner_3) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[3], x, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (VTYPE x, const VTYPE *poly) +{ + VTYPE p = FMA (poly[4], x, poly[3]); + p = FMA (x, p, poly[2]); + p = FMA (x, p, poly[1]); + p = FMA (x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_4) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_5) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_6) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_7) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_8) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_10) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_9) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_11) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_10) (x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_12) (VTYPE x, const VTYPE *poly) +{ + return FMA (x, VWRAP (horner_11) (x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p; + p = FMA (x2, poly[4], p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p01 = FMA (poly[1], x, poly[0]); + VTYPE p23 = FMA (poly[3], x, poly[2]); + VTYPE p45 = FMA (poly[5], x, poly[4]); + VTYPE p; + p = FMA (x2, p45, p23); + p = FMA (x2, p, p01); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p26, p01); +} +static inline VTYPE VWRAP (pw_horner_7) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p27, p01); +} +static inline VTYPE VWRAP (pw_horner_8) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p28, p01); +} +static inline VTYPE VWRAP (pw_horner_9) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p29, p01); +} +static inline VTYPE VWRAP (pw_horner_10) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_10, p01); +} +static inline VTYPE VWRAP (pw_horner_11) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_11, p01); +} +static inline VTYPE VWRAP (pw_horner_12) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_12, p01); +} +static inline VTYPE VWRAP (pw_horner_13) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_13, p01); +} +static inline VTYPE VWRAP (pw_horner_14) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_14, p01); +} +static inline VTYPE VWRAP (pw_horner_15) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_15, p01); +} +static inline VTYPE VWRAP (pw_horner_16) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_16, p01); +} +static inline VTYPE VWRAP (pw_horner_17) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_17, p01); +} +static inline VTYPE VWRAP (pw_horner_18) (VTYPE x, VTYPE x2, const VTYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (x, x2, poly + 2); + VTYPE p01 = FMA (poly[1], x, poly[0]); + return FMA (x2, p2_18, p01); +} diff --git a/sysdeps/aarch64/fpu/poly_sve_f32.h b/sysdeps/aarch64/fpu/poly_sve_f32.h new file mode 100644 index 0000000000..dcf2fab8dd --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f32.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on single-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F32_H +#define AARCH64_FPU_POLY_SVE_F32_H + +#include + +/* Wrap SVE f32 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f32_x. */ +#define VTYPE svfloat32_t +#define STYPE float +#define VWRAP(f) sv_##f##_f32_x +#define DUP svdup_n_f32 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_f64.h b/sysdeps/aarch64/fpu/poly_sve_f64.h new file mode 100644 index 0000000000..97a0b76637 --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_f64.h @@ -0,0 +1,38 @@ +/* Helpers for evaluating polynomials on double-precision SVE input, using + various schemes. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef AARCH64_FPU_POLY_SVE_F64_H +#define AARCH64_FPU_POLY_SVE_F64_H + +#include + +/* Wrap SVE f64 helpers: evaluation of some scheme/order has form: + sv_[scheme]_[order]_f64_x. */ +#define VTYPE svfloat64_t +#define STYPE double +#define VWRAP(f) sv_##f##_f64_x +#define DUP svdup_n_f64 +#include "poly_sve_generic.h" +#undef DUP +#undef VWRAP +#undef STYPE +#undef VTYPE + +#endif diff --git a/sysdeps/aarch64/fpu/poly_sve_generic.h b/sysdeps/aarch64/fpu/poly_sve_generic.h new file mode 100644 index 0000000000..0ecf5ce45b --- /dev/null +++ b/sysdeps/aarch64/fpu/poly_sve_generic.h @@ -0,0 +1,313 @@ +/* Helpers for evaluating polynomials with various schemes - specific to SVE + but precision-agnostic. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#ifndef VTYPE +# error Cannot use poly_generic without defining VTYPE +#endif +#ifndef STYPE +# error Cannot use poly_generic without defining STYPE +#endif +#ifndef VWRAP +# error Cannot use poly_generic without defining VWRAP +#endif +#ifndef DUP +# error Cannot use poly_generic without defining DUP +#endif + +static inline VTYPE VWRAP (pairwise_poly_3) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + /* At order 3, Estrin and Pairwise Horner are identical. */ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + return svmla_x (pg, p01, p23, x2); +} + +static inline VTYPE VWRAP (estrin_4) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + return svmla_x (pg, p03, x4, poly[4]); +} +static inline VTYPE VWRAP (estrin_5) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + return svmla_x (pg, p03, p45, x4); +} +static inline VTYPE VWRAP (estrin_6) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p46 = svmla_x (pg, p45, x, poly[6]); + return svmla_x (pg, p03, p46, x4); +} +static inline VTYPE VWRAP (estrin_7) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + const STYPE *poly) +{ + VTYPE p03 = VWRAP (pairwise_poly_3) (pg, x, x2, poly); + VTYPE p47 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 4); + return svmla_x (pg, p03, p47, x4); +} +static inline VTYPE VWRAP (estrin_8) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), x8, poly[8]); +} +static inline VTYPE VWRAP (estrin_9) (svbool_t pg, VTYPE x, VTYPE x2, VTYPE x4, + VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p89, x8); +} +static inline VTYPE VWRAP (estrin_10) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p89 = svmla_x (pg, DUP (poly[8]), x, poly[9]); + VTYPE p8_10 = svmla_x (pg, p89, x2, poly[10]); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_10, x8); +} +static inline VTYPE VWRAP (estrin_11) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + VTYPE p8_11 = VWRAP (pairwise_poly_3) (pg, x, x2, poly + 8); + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), p8_11, x8); +} +static inline VTYPE VWRAP (estrin_12) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_4) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_13) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_5) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_14) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_6) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_15) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_7) (pg, x, x2, x4, poly), + VWRAP (estrin_7) (pg, x, x2, x4, poly + 8), x8); +} +static inline VTYPE VWRAP (estrin_16) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), x16, + poly[16]); +} +static inline VTYPE VWRAP (estrin_17) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_17, + x16); +} +static inline VTYPE VWRAP (estrin_18) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + VTYPE p16_17 = svmla_x (pg, DUP (poly[16]), x, poly[17]); + VTYPE p16_18 = svmla_x (pg, p16_17, x2, poly[18]); + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), p16_18, + x16); +} +static inline VTYPE VWRAP (estrin_19) (svbool_t pg, VTYPE x, VTYPE x2, + VTYPE x4, VTYPE x8, VTYPE x16, + const STYPE *poly) +{ + return svmla_x (pg, VWRAP (estrin_15) (pg, x, x2, x4, x8, poly), + VWRAP (pairwise_poly_3) (pg, x, x2, poly + 16), x16); +} + +static inline VTYPE VWRAP (horner_3) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[2]), x, poly[3]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_4) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + VTYPE p = svmla_x (pg, DUP (poly[3]), x, poly[4]); + p = svmad_x (pg, x, p, poly[2]); + p = svmad_x (pg, x, p, poly[1]); + p = svmad_x (pg, x, p, poly[0]); + return p; +} +static inline VTYPE VWRAP (horner_5) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_4) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_6) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_5) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_7) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_6) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_8) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_7) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE VWRAP (horner_9) (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_8) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_10_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, VWRAP (horner_9) (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_11_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_10_f32_x (pg, x, poly + 1), poly[0]); +} +static inline VTYPE +sv_horner_12_f32_x (svbool_t pg, VTYPE x, const STYPE *poly) +{ + return svmad_x (pg, x, sv_horner_11_f32_x (pg, x, poly + 1), poly[0]); +} + +static inline VTYPE VWRAP (pw_horner_4) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p; + p = svmla_x (pg, p23, x2, poly[4]); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_5) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + VTYPE p23 = svmla_x (pg, DUP (poly[2]), x, poly[3]); + VTYPE p45 = svmla_x (pg, DUP (poly[4]), x, poly[5]); + VTYPE p; + p = svmla_x (pg, p23, x2, p45); + p = svmla_x (pg, p01, x2, p); + return p; +} +static inline VTYPE VWRAP (pw_horner_6) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p26 = VWRAP (pw_horner_4) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p26); +} +static inline VTYPE VWRAP (pw_horner_7) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p27 = VWRAP (pw_horner_5) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p27); +} +static inline VTYPE VWRAP (pw_horner_8) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p28 = VWRAP (pw_horner_6) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p28); +} +static inline VTYPE VWRAP (pw_horner_9) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p29 = VWRAP (pw_horner_7) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p29); +} +static inline VTYPE VWRAP (pw_horner_10) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_10 = VWRAP (pw_horner_8) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_10); +} +static inline VTYPE VWRAP (pw_horner_11) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_11 = VWRAP (pw_horner_9) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_11); +} +static inline VTYPE VWRAP (pw_horner_12) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_12 = VWRAP (pw_horner_10) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_12); +} +static inline VTYPE VWRAP (pw_horner_13) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_13 = VWRAP (pw_horner_11) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_13); +} +static inline VTYPE VWRAP (pw_horner_14) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_14 = VWRAP (pw_horner_12) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_14); +} +static inline VTYPE VWRAP (pw_horner_15) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_15 = VWRAP (pw_horner_13) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_15); +} +static inline VTYPE VWRAP (pw_horner_16) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_16 = VWRAP (pw_horner_14) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_16); +} +static inline VTYPE VWRAP (pw_horner_17) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_17 = VWRAP (pw_horner_15) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_17); +} +static inline VTYPE VWRAP (pw_horner_18) (svbool_t pg, VTYPE x, VTYPE x2, + const STYPE *poly) +{ + VTYPE p2_18 = VWRAP (pw_horner_16) (pg, x, x2, poly + 2); + VTYPE p01 = svmla_x (pg, DUP (poly[0]), x, poly[1]); + return svmla_x (pg, p01, x2, p2_18); +} diff --git a/sysdeps/aarch64/fpu/tan_advsimd.c b/sysdeps/aarch64/fpu/tan_advsimd.c new file mode 100644 index 0000000000..936a0569c8 --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_advsimd.c @@ -0,0 +1,123 @@ +/* Double-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +static const struct data +{ + float64x2_t poly[9]; + float64x2_t half_pi_hi, half_pi_lo, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V2 (0x1.5555555555556p-2), V2 (0x1.1111111110a63p-3), + V2 (0x1.ba1ba1bb46414p-5), V2 (0x1.664f47e5b5445p-6), + V2 (0x1.226e5e5ecdfa3p-7), V2 (0x1.d6c7ddbf87047p-9), + V2 (0x1.7ea75d05b583ep-10), V2 (0x1.289f22964a03cp-11), + V2 (0x1.4e4fd14147622p-12) }, + .half_pi_hi = V2 (0x1.921fb54442d18p0), + .half_pi_lo = V2 (0x1.1a62633145c07p-54), + .two_over_pi = V2 (0x1.45f306dc9c883p-1), + .shift = V2 (0x1.8p52), +#if !WANT_SIMD_EXCEPT + .range_val = V2 (0x1p23), +#endif +}; + +#define RangeVal 0x4160000000000000 /* asuint64(0x1p23). */ +#define TinyBound 0x3e50000000000000 /* asuint64(2^-26). */ +#define Thresh 0x310000000000000 /* RangeVal - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (tan, x, x, v_u64 (-1)); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + __v_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +float64x2_t VPCS_ATTR V_NAME_D1 (tan) (float64x2_t x) +{ + const struct data *dat = ptr_barrier (&data); + /* Our argument reduction cannot calculate q with sufficient accuracy for very + large inputs. Fall back to scalar routine for all lanes if any are too + large, or Inf/NaN. If fenv exceptions are expected, also fall back for tiny + input to avoid underflow. */ +#if WANT_SIMD_EXCEPT + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + /* iax - tiny_bound > range_val - tiny_bound. */ + uint64x2_t special + = vcgtq_u64 (vsubq_u64 (iax, v_u64 (TinyBound)), v_u64 (Thresh)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + /* q = nearest integer to 2 * x / pi. */ + float64x2_t q + = vsubq_f64 (vfmaq_f64 (dat->shift, x, dat->two_over_pi), dat->shift); + int64x2_t qi = vcvtq_s64_f64 (q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + float64x2_t r = x; + r = vfmsq_f64 (r, q, dat->half_pi_hi); + r = vfmsq_f64 (r, q, dat->half_pi_lo); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = vmulq_n_f64 (r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + float64x2_t r2 = vmulq_f64 (r, r), r4 = vmulq_f64 (r2, r2), + r8 = vmulq_f64 (r4, r4); + /* Offset coefficients to evaluate from C1 onwards. */ + float64x2_t p = v_estrin_7_f64 (r2, r4, r8, dat->poly + 1); + p = vfmaq_f64 (dat->poly[0], p, r2); + p = vfmaq_f64 (r, r2, vmulq_f64 (p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator, dependent on odd/even-ness of q (hence quadrant). */ + float64x2_t n = vfmaq_f64 (v_f64 (-1), p, p); + float64x2_t d = vaddq_f64 (p, p); + + uint64x2_t no_recip = vtstq_u64 (vreinterpretq_u64_s64 (qi), v_u64 (1)); + +#if !WANT_SIMD_EXCEPT + uint64x2_t special = vceqzq_u64 (vcaleq_f64 (x, dat->range_val)); + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x); +#endif + + return vdivq_f64 (vbslq_f64 (no_recip, n, vnegq_f64 (d)), + vbslq_f64 (no_recip, d, n)); +} diff --git a/sysdeps/aarch64/fpu/tan_sve.c b/sysdeps/aarch64/fpu/tan_sve.c new file mode 100644 index 0000000000..e19d8890e5 --- /dev/null +++ b/sysdeps/aarch64/fpu/tan_sve.c @@ -0,0 +1,106 @@ +/* Double-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +static const struct data +{ + double poly[9]; + double half_pi_hi, half_pi_lo, inv_half_pi, range_val, shift; +} data = { + /* Polynomial generated with FPMinimax. */ + .poly = { 0x1.5555555555556p-2, 0x1.1111111110a63p-3, 0x1.ba1ba1bb46414p-5, + 0x1.664f47e5b5445p-6, 0x1.226e5e5ecdfa3p-7, 0x1.d6c7ddbf87047p-9, + 0x1.7ea75d05b583ep-10, 0x1.289f22964a03cp-11, + 0x1.4e4fd14147622p-12, }, + .half_pi_hi = 0x1.921fb54442d18p0, + .half_pi_lo = 0x1.1a62633145c07p-54, + .inv_half_pi = 0x1.45f306dc9c883p-1, + .range_val = 0x1p23, + .shift = 0x1.8p52, +}; + +static svfloat64_t NOINLINE +special_case (svfloat64_t x) +{ + return sv_call_f64 (tan, x, x, svptrue_b64 ()); +} + +/* Vector approximation for double-precision tan. + Maximum measured error is 3.48 ULP: + _ZGVsMxv_tan(0x1.4457047ef78d8p+20) got -0x1.f6ccd8ecf7dedp+37 + want -0x1.f6ccd8ecf7deap+37. */ +svfloat64_t SV_NAME_D1 (tan) (svfloat64_t x, svbool_t pg) +{ + const struct data *dat = ptr_barrier (&data); + + /* Invert condition to catch NaNs and Infs as well as large values. */ + svbool_t special = svnot_z (pg, svaclt (pg, x, dat->range_val)); + + /* Fallback for all lanes if any are out of bounds. */ + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x); + + /* q = nearest integer to 2 * x / pi. */ + svfloat64_t shift = sv_f64 (dat->shift); + svfloat64_t q = svmla_x (pg, shift, x, dat->inv_half_pi); + q = svsub_x (pg, q, shift); + svint64_t qi = svcvt_s64_x (pg, q); + + /* Use q to reduce x to r in [-pi/4, pi/4], by: + r = x - q * pi/2, in extended precision. */ + svfloat64_t r = x; + svfloat64_t half_pi = svld1rq (svptrue_b64 (), &dat->half_pi_hi); + r = svmls_lane (r, q, half_pi, 0); + r = svmls_lane (r, q, half_pi, 1); + /* Further reduce r to [-pi/8, pi/8], to be reconstructed using double angle + formula. */ + r = svmul_x (pg, r, 0.5); + + /* Approximate tan(r) using order 8 polynomial. + tan(x) is odd, so polynomial has the form: + tan(x) ~= x + C0 * x^3 + C1 * x^5 + C3 * x^7 + ... + Hence we first approximate P(r) = C1 + C2 * r^2 + C3 * r^4 + ... + Then compute the approximation by: + tan(r) ~= r + r^3 * (C0 + r^2 * P(r)). */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t r4 = svmul_x (pg, r2, r2); + svfloat64_t r8 = svmul_x (pg, r4, r4); + /* Use offset version coeff array by 1 to evaluate from C1 onwards. */ + svfloat64_t p = sv_estrin_7_f64_x (pg, r2, r4, r8, dat->poly + 1); + p = svmad_x (pg, p, r2, dat->poly[0]); + p = svmla_x (pg, r, r2, svmul_x (pg, p, r)); + + /* Recombination uses double-angle formula: + tan(2x) = 2 * tan(x) / (1 - (tan(x))^2) + and reciprocity around pi/2: + tan(x) = 1 / (tan(pi/2 - x)) + to assemble result using change-of-sign and conditional selection of + numerator/denominator dependent on odd/even-ness of q (hence quadrant). */ + svbool_t use_recip + = svcmpeq (pg, svand_x (pg, svreinterpret_u64 (qi), 1), 0); + + svfloat64_t n = svmad_x (pg, p, p, -1); + svfloat64_t d = svmul_x (pg, p, 2); + svfloat64_t swap = n; + n = svneg_m (n, use_recip, d); + d = svsel (use_recip, swap, d); + return svdiv_x (pg, n, d); +} diff --git a/sysdeps/aarch64/fpu/tanf_advsimd.c b/sysdeps/aarch64/fpu/tanf_advsimd.c new file mode 100644 index 0000000000..b9a055bba0 --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_advsimd.c @@ -0,0 +1,134 @@ +/* Single-precision vector (Advanced SIMD) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[6]; + float32x4_t neg_half_pi_1, neg_half_pi_2, neg_half_pi_3, two_over_pi, shift; +#if !WANT_SIMD_EXCEPT + float32x4_t range_val; +#endif +} data = { + /* Coefficients generated using FPMinimax. */ + .poly = { V4 (0x1.55555p-2f), V4 (0x1.11166p-3f), V4 (0x1.b88a78p-5f), + V4 (0x1.7b5756p-6f), V4 (0x1.4ef4cep-8f), V4 (0x1.0e1e74p-7f) }, + .neg_half_pi_1 = V4 (-0x1.921fb6p+0f), + .neg_half_pi_2 = V4 (0x1.777a5cp-25f), + .neg_half_pi_3 = V4 (0x1.ee59dap-50f), + .two_over_pi = V4 (0x1.45f306p-1f), + .shift = V4 (0x1.8p+23f), +#if !WANT_SIMD_EXCEPT + .range_val = V4 (0x1p15f), +#endif +}; + +#define RangeVal v_u32 (0x47000000) /* asuint32(0x1p15f). */ +#define TinyBound v_u32 (0x30000000) /* asuint32 (0x1p-31f). */ +#define Thresh v_u32 (0x16000000) /* asuint32(RangeVal) - TinyBound. */ + +/* Special cases (fall back to scalar calls). */ +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + return v_call_f32 (tanf, x, y, cmp); +} + +/* Use a full Estrin scheme to evaluate polynomial. */ +static inline float32x4_t +eval_poly (float32x4_t z, const struct data *d) +{ + float32x4_t z2 = vmulq_f32 (z, z); +#if WANT_SIMD_EXCEPT + /* Tiny z (<= 0x1p-31) will underflow when calculating z^4. If fp exceptions + are to be triggered correctly, sidestep this by fixing such lanes to 0. */ + uint32x4_t will_uflow + = vcleq_u32 (vreinterpretq_u32_f32 (vabsq_f32 (z)), TinyBound); + if (__glibc_unlikely (v_any_u32 (will_uflow))) + z2 = vbslq_f32 (will_uflow, v_f32 (0), z2); +#endif + float32x4_t z4 = vmulq_f32 (z2, z2); + return v_estrin_5_f32 (z, z2, z4, d->poly); +} + +/* Fast implementation of AdvSIMD tanf. + Maximum error is 3.45 ULP: + __v_tanf(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +float32x4_t VPCS_ATTR V_NAME_F1 (tan) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t special_arg = x; + + /* iax >= RangeVal means x, if not inf or NaN, is too large to perform fast + regression. */ +#if WANT_SIMD_EXCEPT + uint32x4_t iax = vreinterpretq_u32_f32 (vabsq_f32 (x)); + /* If fp exceptions are to be triggered correctly, also special-case tiny + input, as this will load to overflow later. Fix any special lanes to 1 to + prevent any exceptions being triggered. */ + uint32x4_t special = vcgeq_u32 (vsubq_u32 (iax, TinyBound), Thresh); + if (__glibc_unlikely (v_any_u32 (special))) + x = vbslq_f32 (special, v_f32 (1.0f), x); +#else + /* Otherwise, special-case large and special values. */ + uint32x4_t special = vcageq_f32 (x, d->range_val); +#endif + + /* n = rint(x/(pi/2)). */ + float32x4_t q = vfmaq_f32 (d->shift, d->two_over_pi, x); + float32x4_t n = vsubq_f32 (q, d->shift); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + uint32x4_t pred_alt = vtstq_u32 (vreinterpretq_u32_f32 (q), v_u32 (1)); + + /* r = x - n * (pi/2) (range reduction into -pi./4 .. pi/4). */ + float32x4_t r; + r = vfmaq_f32 (x, d->neg_half_pi_1, n); + r = vfmaq_f32 (r, d->neg_half_pi_2, n); + r = vfmaq_f32 (r, d->neg_half_pi_3, n); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Invert sign of r if odd quadrant. */ + float32x4_t z = vmulq_f32 (r, vbslq_f32 (pred_alt, v_f32 (-1), v_f32 (1))); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4]. */ + float32x4_t z2 = vmulq_f32 (r, r); + float32x4_t p = eval_poly (z2, d); + float32x4_t y = vfmaq_f32 (z, vmulq_f32 (z, z2), p); + + /* Compute reciprocal and apply if required. */ + float32x4_t inv_y = vdivq_f32 (v_f32 (1.0f), y); + y = vbslq_f32 (pred_alt, inv_y, y); + + /* Fast reduction does not handle the x = -0.0 case well, + therefore it is fixed here. */ + y = vbslq_f32 (vceqzq_f32 (x), x, y); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (special_arg, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/tanf_sve.c b/sysdeps/aarch64/fpu/tanf_sve.c new file mode 100644 index 0000000000..48343ae197 --- /dev/null +++ b/sysdeps/aarch64/fpu/tanf_sve.c @@ -0,0 +1,123 @@ +/* Single-precision vector (SVE) tan function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float pio2_1, pio2_2, pio2_3, invpio2; + float c1, c3, c5; + float c0, c2, c4, range_val, shift; +} data = { + /* Coefficients generated using: + poly = fpminimax((tan(sqrt(x))-sqrt(x))/x^(3/2), + deg, + [|single ...|], + [a*a;b*b]); + optimize relative error + final prec : 23 bits + deg : 5 + a : 0x1p-126 ^ 2 + b : ((pi) / 0x1p2) ^ 2 + dirty rel error: 0x1.f7c2e4p-25 + dirty abs error: 0x1.f7c2ecp-25. */ + .c0 = 0x1.55555p-2, .c1 = 0x1.11166p-3, + .c2 = 0x1.b88a78p-5, .c3 = 0x1.7b5756p-6, + .c4 = 0x1.4ef4cep-8, .c5 = 0x1.0e1e74p-7, + + .pio2_1 = 0x1.921fb6p+0f, .pio2_2 = -0x1.777a5cp-25f, + .pio2_3 = -0x1.ee59dap-50f, .invpio2 = 0x1.45f306p-1f, + .range_val = 0x1p15f, .shift = 0x1.8p+23f +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (tanf, x, y, cmp); +} + +/* Fast implementation of SVE tanf. + Maximum error is 3.45 ULP: + SV_NAME_F1 (tan)(-0x1.e5f0cap+13) got 0x1.ff9856p-1 + want 0x1.ff9850p-1. */ +svfloat32_t SV_NAME_F1 (tan) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + /* Determine whether input is too large to perform fast regression. */ + svbool_t cmp = svacge (pg, x, d->range_val); + svbool_t peqz = svcmpeq (pg, x, 0.0); + + svfloat32_t odd_coeffs = svld1rq (svptrue_b32 (), &d->c1); + svfloat32_t pi_vals = svld1rq (svptrue_b32 (), &d->pio2_1); + + /* n = rint(x/(pi/2)). */ + svfloat32_t q = svmla_lane (sv_f32 (d->shift), x, pi_vals, 3); + svfloat32_t n = svsub_x (pg, q, d->shift); + /* n is already a signed integer, simply convert it. */ + svint32_t in = svcvt_s32_x (pg, n); + /* Determine if x lives in an interval, where |tan(x)| grows to infinity. */ + svint32_t alt = svand_x (pg, in, 1); + svbool_t pred_alt = svcmpne (pg, alt, 0); + + /* r = x - n * (pi/2) (range reduction into 0 .. pi/4). */ + svfloat32_t r; + r = svmls_lane (x, n, pi_vals, 0); + r = svmls_lane (r, n, pi_vals, 1); + r = svmls_lane (r, n, pi_vals, 2); + + /* If x lives in an interval, where |tan(x)| + - is finite, then use a polynomial approximation of the form + tan(r) ~ r + r^3 * P(r^2) = r + r * r^2 * P(r^2). + - grows to infinity then use symmetries of tangent and the identity + tan(r) = cotan(pi/2 - r) to express tan(x) as 1/tan(-r). Finally, use + the same polynomial approximation of tan as above. */ + + /* Perform additional reduction if required. */ + svfloat32_t z = svneg_m (r, pred_alt, r); + + /* Evaluate polynomial approximation of tangent on [-pi/4, pi/4], + using Estrin on z^2. */ + svfloat32_t z2 = svmul_x (pg, z, z); + svfloat32_t p01 = svmla_lane (sv_f32 (d->c0), z2, odd_coeffs, 0); + svfloat32_t p23 = svmla_lane (sv_f32 (d->c2), z2, odd_coeffs, 1); + svfloat32_t p45 = svmla_lane (sv_f32 (d->c4), z2, odd_coeffs, 2); + + svfloat32_t z4 = svmul_x (pg, z2, z2); + svfloat32_t p = svmla_x (pg, p01, z4, p23); + + svfloat32_t z8 = svmul_x (pg, z4, z4); + p = svmla_x (pg, p, z8, p45); + + svfloat32_t y = svmla_x (pg, z, p, svmul_x (pg, z, z2)); + + /* Transform result back, if necessary. */ + svfloat32_t inv_y = svdivr_x (pg, y, 1.0f); + y = svsel (pred_alt, inv_y, y); + + /* Fast reduction does not handle the x = -0.0 case well, + therefore it is fixed here. */ + y = svsel (peqz, x, y); + + /* No need to pass pg to specialcase here since cmp is a strict subset, + guaranteed by the cmpge above. */ + if (__glibc_unlikely (svptest_any (pg, cmp))) + return special_case (x, y, cmp); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 3b6b1e343d..f76726e324 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) +VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index d7ac47ca22..5a9e5b552a 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) +SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index d4a9ac7154..66c6227087 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,3 +27,4 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) +VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index d44033eab0..3cbf8b48b7 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,3 +36,4 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) +SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6b25ed43e0..ec84381c75 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1340,11 +1340,19 @@ Function: "tan": float: 1 ldouble: 1 +Function: "tan_advsimd": +double: 2 +float: 2 + Function: "tan_downward": double: 1 float: 2 ldouble: 1 +Function: "tan_sve": +double: 2 +float: 2 + Function: "tan_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index ae46ef8c34..52ef1c34b5 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,3 +14,7 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_tan F +GLIBC_2.39 _ZGVnN4v_tanf F +GLIBC_2.39 _ZGVsMxv_tan F +GLIBC_2.39 _ZGVsMxv_tanf F \ No newline at end of file From patchwork Thu Aug 3 12:27:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1816429 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=RbzT1XEs; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RGp6m5sk6z1yZl for ; Thu, 3 Aug 2023 22:28:48 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id DC1223857714 for ; Thu, 3 Aug 2023 12:28:46 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org DC1223857714 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691065726; bh=prG5lSyXuSvQ2+gVWGOh8gmv6meujqsQSJHTufM/N2g=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=RbzT1XEshmSFCTVrOH+rNvfChzvjN9jNE+/ZJsB7IBpcNEL2YkfHLf9ath0OIMm8G umJOVqerTTebJ6vD+Z3S2PUCCJPtS4IbaGUShzV/wV/vBYpZVYUiJzVhsLtLLPs0T2 25I74znTUydYjmzCXOhMFirfe8cIym6szwVpbyQE= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on2044.outbound.protection.outlook.com [40.107.7.44]) by sourceware.org (Postfix) with ESMTPS id 8182F3858C5E for ; Thu, 3 Aug 2023 12:27:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 8182F3858C5E Received: from AS9PR07CA0045.eurprd07.prod.outlook.com (2603:10a6:20b:46b::17) by GV1PR08MB10371.eurprd08.prod.outlook.com (2603:10a6:150:8a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:30 +0000 Received: from AM7EUR03FT017.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:46b:cafe::97) by AS9PR07CA0045.outlook.office365.com (2603:10a6:20b:46b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Thu, 3 Aug 2023 12:27:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT017.mail.protection.outlook.com (100.127.140.184) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:29 +0000 Received: ("Tessian outbound f5de790fcf89:v145"); Thu, 03 Aug 2023 12:27:29 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 77f7dfb24a77f043 X-CR-MTA-TID: 64aa7808 Received: from 5bab0a83e54f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id CB92432B-9232-43E7-9BB7-221B1A528395.1; Thu, 03 Aug 2023 12:27:23 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 5bab0a83e54f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 03 Aug 2023 12:27:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LD186dInUhX6IZbA4VR7Sw6qaayieoQ8SkNMt/2zR7wuFEhgQtYTKVupePe6/cWWW0XSs6oAWQD9+AJY6qN7+L2JSlu+JDodRrAND3s7aWGBbxTzTo7oSKPk4JxsCmPpBu2PyYvCQjZbK1Xakd54p5neP8qw1nl02qDrZ3IbMZPvXmJP8Pne95ObwdQPsRrdxw6jlBZFYBQ5OpHzHZ+yj340DrWuqtqB/F1DYtftfSxigCTKBERFoUZzeUd98As/l9YskWDBb/8fXWzgQJQs6pf7KWMQjmE4Be9ACriYWg2i9AyVEBeoQ4a3dlNUtVJmr+8GDGnZvHiISVpClaEXOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=prG5lSyXuSvQ2+gVWGOh8gmv6meujqsQSJHTufM/N2g=; b=NVNVqS+HKCvU59LINABfBrPWoNsb5mH3t+WfLBiBobbx8rLnzzZ/sOCfLDuyV1meoPdOYdptHqgsZWAQJOP/J94lulsVwZbKXq8Ij0kVlzfna+g2lUfSf+kJoNqot+5TXFjKialnqi5MdnwZn9AEXMtH34UtfiehDeQrwcJqjIPrA7tzaYltdwT1mcorgzKdJC/9hAWUHZsPKPYcccWc8oKxnX6P2Iw2AVyqS769R4kSzt7ofBmadXawuKFGcYFH6d45Ec8MDLVxsOpbWp7gZTIxnjdjBtNco9kOdmr1+QVTUi/lhh6cjFOsZ8kTC4a2byaIrIUMNnQXpND0KajbIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from DB7PR02CA0028.eurprd02.prod.outlook.com (2603:10a6:10:52::41) by AS8PR08MB10150.eurprd08.prod.outlook.com (2603:10a6:20b:62a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20; Thu, 3 Aug 2023 12:27:20 +0000 Received: from DBAEUR03FT049.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:52:cafe::97) by DB7PR02CA0028.outlook.office365.com (2603:10a6:10:52::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT049.mail.protection.outlook.com (100.127.142.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:18 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 3 Aug 2023 12:27:18 +0000 To: CC: Joe Ramsay Subject: [PATCH 2/5] aarch64: Add vector implementations of exp2 routines Date: Thu, 3 Aug 2023 13:27:14 +0100 Message-ID: <20230803122717.33413-2-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230803122717.33413-1-Joe.Ramsay@arm.com> References: <20230803122717.33413-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DBAEUR03FT049:EE_|AS8PR08MB10150:EE_|AM7EUR03FT017:EE_|GV1PR08MB10371:EE_ X-MS-Office365-Filtering-Correlation-Id: ce96e2a5-5a91-4723-5519-08db941d0156 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: HhWECTBuj+kEsxdiWriLaC9R425S5+yXu4chMXwaMCcnvucP4728s5fYugcRLiH0cd9jG09tMqWr5jVZSx82604RYi2Clnf9FUFlzKklfRMGh9jxKHC3Wwo1pb38p7FTieFZvvH/aleJmkSwkFxsaYt+QXzCQqMYBjphpxZSytR+j9BLNOKG1Mtl3b5G92K7IMNvSMGNRgpYdZjM2o70siPT6cETBT5LpG5s/ej3jzPfs/d0AND1DQDrQ0WwyAY6bmymjD+z0sVxGwFoTsXWJyWOjFSgbj1E7LfTkn/KDFOEqJvvIsl8F7+Wf+pydCXA574Plx/X35BKEr9/QoADidjtQVhVuBwuoerSWiLpD6SymEPEKBfsqpqQgdzgs7FGq8Qvxx7182fOfZ5VneKvOGw5hVUhHRA5WzdI9oYRbv4VF2qDvFkTETwNgMwCifrCJPOCbesiOm9E36O3IvZXfkOrO1udciOkv8YOlRSYkmV4fk5Z5nXT5OJYKnbD5UzHIU7HkkaPEQoqeK5kUV28MzqjC4fh2Ib1/FP6C5A6AP1+0AsZSg/3C9cmRD7Dt1aDY8mX0AZyAbHCXFbQuoeiAq2LEJ0rOilEFDiO6OhYDjBxTy3kXj8shcC8Cg5C/W64iiiiiI8DyEpof18Lj+4uHnVkSSXPB+wKsTwqdhTirBFDPa5SnWWWlxxZp4MsCCBbCagXUyKi84juD+CvCHoWefTzCQlKX4PgylEFgZnFz/7nBzW6zSZMbvez+8t7wf5CcBzwu7PG/U+/5jbgTownqw== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(396003)(346002)(376002)(39860400002)(136003)(451199021)(82310400008)(40470700004)(36840700001)(46966006)(40460700003)(41300700001)(8936002)(8676002)(36860700001)(47076005)(83380400001)(336012)(186003)(1076003)(26005)(86362001)(426003)(2616005)(81166007)(356005)(82740400003)(316002)(6666004)(7696005)(40480700001)(478600001)(36756003)(6916009)(4326008)(70206006)(70586007)(30864003)(2906002)(5660300002)(2004002)(36900700001); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB10150 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT017.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 2fd3d015-70f8-4799-5a74-08db941cfb8e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JKL0syCsURcjy/8Yy7iv1XOYCUNU9/Tix3yTymcJ0EI8qwq7egGz60+5X1T0FWGCDwS3u5TQVMitOR7lBOCkjfGpR1gPTPWgMlufiWpwE5ORjOSjFoNUmmM8ca9unA1oOMzw/wiYyV4ThPEODN8ltfl21mQTGyOa5EPf5bK+CrdzIoDvqxSf7KmLZmoSk1z919qSiepRRkV77J9M7ZrD8GF3Z6tTAH9vt8TZRjiywKCz7fSnPnlQCKnWDcMxmNbbW5vSymqDYthk/b/w4450dBjRlt2E8l5FL1kKhz8X970MtwdC56Mkg+TwtO5D4Wo/WSvcfSTWNc3DLkNsJ34pDNR2470kHWUbLulDO7S0O9FhvBLxK4bdnB/scuBxFHbPVYf3DNFbHnVvBflBqpoy4SGmgxklk6ZcptzsQnbQqD1QvaYEOzieDuzGWI2mYV7JH9qzzOcHxCLb1/5viIlSKyY38REqvtR4fG9mfO6sZcteGFmV5Q1CJKaOrNjeyslCVR4yESmWekiNPQQeJNXEHA+SIwloPknjeL4t5X3FBBt9eGr1eheZwnHwTlVzv0G3ss8zCFX4A7yl1/WSs00mDgbZ2sox03i5vpcvYv5KOFmSTtcKzrKMP5hZmTB4UN76A94zp4tsikcSPEuCzofnDaLjTQz5eDpUP9aulu52kjqw9v27gtP7v0hJbFp/GCzCdVW0ZvX3an3aDOtf6H4G7GkbWDLkF/ILZm/nvcVsEIyLyyxkRHaswb5SptXSXVDM X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(39860400002)(376002)(346002)(451199021)(82310400008)(46966006)(40470700004)(36840700001)(41300700001)(40460700003)(8936002)(8676002)(36860700001)(83380400001)(47076005)(336012)(186003)(1076003)(26005)(86362001)(426003)(2616005)(81166007)(82740400003)(316002)(6666004)(40480700001)(478600001)(7696005)(36756003)(6916009)(4326008)(70586007)(70206006)(30864003)(2906002)(5660300002)(2004002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 12:27:29.9673 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ce96e2a5-5a91-4723-5519-08db941d0156 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT017.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB10371 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Some routines reuse table from v_exp_data.c --- Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp2_advsimd.c | 128 ++++++++++++++++++ sysdeps/aarch64/fpu/exp2_sve.c | 111 +++++++++++++++ sysdeps/aarch64/fpu/exp2f_advsimd.c | 124 +++++++++++++++++ sysdeps/aarch64/fpu/exp2f_sve.c | 75 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 ++ .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 463 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2_sve.c create mode 100644 sysdeps/aarch64/fpu/exp2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp2f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index a1bbc9bcaa..9c7c768301 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp2 \ log \ sin \ tan diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index f0ca0940a9..05de4325d5 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp2f; + _ZGVnN2v_exp2; + _ZGVsMxv_exp2f; + _ZGVsMxv_exp2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 6193213147..50921b22e5 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,12 +51,14 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -68,12 +70,14 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp2_advsimd.c b/sysdeps/aarch64/fpu/exp2_advsimd.c new file mode 100644 index 0000000000..8e26033cb9 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_advsimd.c @@ -0,0 +1,128 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask (N - 1) +#define BigBound 1022.0 +#define UOFlowBound 1280.0 + +static const struct data +{ + float64x2_t poly[4]; + float64x2_t shift, scale_big_bound, scale_uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { V2 (0x1.62e42fefa3686p-1), V2 (0x1.ebfbdff82c241p-3), + V2 (0x1.c6b09b16de99ap-5), V2 (0x1.3b2abf5571ad8p-7) }, + .shift = V2 (0x1.8p52 / N), + .scale_big_bound = V2 (BigBound), + .scale_uoflow_bound = V2 (UOFlowBound), +}; + +static inline uint64x2_t +lookup_sbits (uint64x2_t i) +{ + return (uint64x2_t){ __v_exp_data[i[0] & IndexMask], + __v_exp_data[i[1] & IndexMask] }; +} + +#if WANT_SIMD_EXCEPT + +# define TinyBound 0x2000000000000000 /* asuint64(0x1p-511). */ +# define Thres 0x2080000000000000 /* asuint64(512.0) - TinyBound. */ + +/* Call scalar exp2 as a fallback. */ +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x) +{ + return v_call_f64 (exp2, x, x, v_u64 (0xffffffffffffffff)); +} + +#else + +# define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +# define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vclezq_f64 (n), v_u64 (SpecialOffset)); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (v_u64 (SpecialBias1), b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), v_u64 (SpecialBias2)), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_uoflow_bound); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, s2, y), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVnN2v_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +VPCS_ATTR +float64x2_t V_NAME_D1 (exp2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + uint64x2_t ia = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (ia, v_u64 (TinyBound)), v_u64 (Thres)); + /* If any special case (inf, nan, small and large x) is detected, + fall back to scalar for all lanes. */ + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (x); +#else + cmp = vcagtq_f64 (x, d->scale_big_bound); +#endif + + /* n = round(x/N). */ + float64x2_t z = vaddq_f64 (d->shift, x); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n/N. */ + float64x2_t r = vsubq_f64 (x, n); + + /* s = 2^(n/N). */ + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + u = lookup_sbits (u); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + /* y ~ exp2(r) - 1. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pairwise_poly_3_f64 (r, r2, d->poly); + y = vmulq_f64 (r, y); + +#if !WANT_SIMD_EXCEPT + if (__glibc_unlikely (v_any_u64 (cmp))) + return special_case (s, y, n, d); +#endif + return vfmaq_f64 (s, s, y); +} diff --git a/sysdeps/aarch64/fpu/exp2_sve.c b/sysdeps/aarch64/fpu/exp2_sve.c new file mode 100644 index 0000000000..c9314a9fee --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2_sve.c @@ -0,0 +1,111 @@ +/* Double-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_EXP_TABLE_BITS) + +#define BigBound 1022 +#define UOFlowBound 1280 + +static const struct data +{ + double poly[4]; + double shift, big_bound, uoflow_bound; +} data = { + /* Coefficients are computed using Remez algorithm with + minimisation of the absolute error. */ + .poly = { 0x1.62e42fefa3686p-1, 0x1.ebfbdff82c241p-3, 0x1.c6b09b16de99ap-5, + 0x1.3b2abf5571ad8p-7 }, + .shift = 0x1.8p52 / N, + .uoflow_bound = UOFlowBound, + .big_bound = BigBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->uoflow_bound); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp2. + Maximum measured error is 1.65 ulp. + _ZGVsMxv_exp2(-0x1.4c264ab5b559bp-6) got 0x1.f8db0d4df721fp-1 + want 0x1.f8db0d4df721dp-1. */ +svfloat64_t SV_NAME_D1 (exp2) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->big_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* Reduce x to k/N + r, where k is integer and r in [-1/2N, 1/2N]. */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t kd = svadd_x (pg, x, shift); + svuint64_t ki = svreinterpret_u64 (kd); + /* kd = k/N. */ + kd = svsub_x (pg, kd, shift); + svfloat64_t r = svsub_x (pg, x, kd); + + /* scale ~= 2^(k/N). */ + svuint64_t idx = svand_x (pg, ki, N - 1); + svuint64_t sbits = svld1_gather_index (pg, __v_exp_data, idx); + /* This is only a valid scale when -1023*N < k < 1024*N. */ + svuint64_t top = svlsl_x (pg, ki, 52 - V_EXP_TABLE_BITS); + svfloat64_t scale = svreinterpret_f64 (svadd_x (pg, sbits, top)); + + /* Approximate exp2(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t p = sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly); + svfloat64_t y = svmul_x (pg, r, p); + + /* Assemble exp2(x) = exp2(r) * scale. */ + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (pg, scale, y, kd, d); + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp2f_advsimd.c b/sysdeps/aarch64/fpu/exp2f_advsimd.c new file mode 100644 index 0000000000..70b3ab66c1 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_advsimd.c @@ -0,0 +1,124 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +static const struct data +{ + float32x4_t poly[5]; + uint32x4_t exponent_bias; +#if !WANT_SIMD_EXCEPT + float32x4_t special_bound, scale_thresh; +#endif +} data = { + /* maxerr: 1.962 ulp. */ + .poly = { V4 (0x1.59977ap-10f), V4 (0x1.3ce9e4p-7f), V4 (0x1.c6bd32p-5f), + V4 (0x1.ebf9bcp-3f), V4 (0x1.62e422p-1f) }, + .exponent_bias = V4 (0x3f800000), +#if !WANT_SIMD_EXCEPT + .special_bound = V4 (126.0f), + .scale_thresh = V4 (192.0f), +#endif +}; + +#define C(i) d->poly[i] + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42800000) /* asuint (0x1p6). */ +# define SpecialBound v_u32 (0x22800000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f32 (exp2f, x, y, cmp); +} + +#else + +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +float32x4_t VPCS_ATTR V_NAME_F1 (exp2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + float32x4_t n, r, r2, scale, p, q, poly; + uint32x4_t cmp, e; + +#if WANT_SIMD_EXCEPT + /* asuint(|x|) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t ia = vreinterpretq_u32_f32 (vabsq_f32 (x)); + cmp = vcgeq_u32 (vsubq_u32 (ia, TinyBound), SpecialBound); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + n = vrndaq_f32 (x); + r = vsubq_f32 (x, n); + e = vshlq_n_u32 (vreinterpretq_u32_s32 (vcvtaq_s32_f32 (x)), 23); + scale = vreinterpretq_f32_u32 (vaddq_u32 (e, d->exponent_bias)); + +#if !WANT_SIMD_EXCEPT + cmp = vcagtq_f32 (n, d->special_bound); +#endif + + r2 = vmulq_f32 (r, r); + p = vfmaq_f32 (C (1), C (0), r); + q = vfmaq_f32 (C (3), C (2), r); + q = vfmaq_f32 (q, p, r2); + p = vmulq_f32 (C (4), r); + poly = vfmaq_f32 (p, q, r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp2f_sve.c b/sysdeps/aarch64/fpu/exp2f_sve.c new file mode 100644 index 0000000000..88d10daefc --- /dev/null +++ b/sysdeps/aarch64/fpu/exp2f_sve.c @@ -0,0 +1,75 @@ +/* Single-precision vector (SVE) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +static const struct data +{ + float poly[5]; + float shift, thres; +} data = { + /* Coefficients copied from the polynomial in AdvSIMD variant, reversed for + compatibility with polynomial helpers. */ + .poly = { 0x1.62e422p-1f, 0x1.ebf9bcp-3f, 0x1.c6bd32p-5f, 0x1.3ce9e4p-7f, + 0x1.59977ap-10f }, + /* 1.5*2^17 + 127. */ + .shift = 0x1.903f8p17f, + /* Roughly 87.3. For x < -Thres, the result is subnormal and not handled + correctly by FEXPA. */ + .thres = 0x1.5d5e2ap+6f, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp2f, x, y, special); +} + +/* Single-precision SVE exp2f routine. Implements the same algorithm + as AdvSIMD exp2f. + Worst case error is 1.04 ULPs. + SV_NAME_F1 (exp2)(0x1.943b9p-1) got 0x1.ba7eb2p+0 + want 0x1.ba7ebp+0. */ +svfloat32_t SV_NAME_F1 (exp2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp2(x) = 2^n (1 + poly(r)), with 1 + poly(r) in [1/sqrt(2),sqrt(2)] + x = n + r, with r in [-1/2, 1/2]. */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svadd_x (pg, x, shift); + svfloat32_t n = svsub_x (pg, z, shift); + svfloat32_t r = svsub_x (pg, x, n); + + svbool_t special = svacgt (pg, x, d->thres); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp2(r)-1. + Evaluate polynomial use hybrid scheme - offset ESTRIN by 1 for + coefficients 1 to 4, and apply most significant coefficient directly. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t p14 = sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1); + svfloat32_t p0 = svmul_x (pg, r, d->poly[0]); + svfloat32_t poly = svmla_x (pg, p0, r2, p14); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index f76726e324..b2b36fd847 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 5a9e5b552a..88b76ed678 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 66c6227087..02ab609b5a 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 3cbf8b48b7..fa41ce09d8 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index ec84381c75..a1e5651c87 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -990,11 +990,19 @@ double: 1 float: 1 ldouble: 1 +Function: "exp2_advsimd": +double: 1 +float: 1 + Function: "exp2_downward": double: 1 float: 1 ldouble: 1 +Function: "exp2_sve": +double: 1 +float: 1 + Function: "exp2_towardzero": double: 1 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 52ef1c34b5..75550bf840 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,6 +14,10 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp2 F +GLIBC_2.39 _ZGVnN4v_exp2f F +GLIBC_2.39 _ZGVsMxv_exp2 F +GLIBC_2.39 _ZGVsMxv_exp2f F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_tan F From patchwork Thu Aug 3 12:27:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1816426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=fUX7gYiK; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RGp5p4dw0z1yds for ; Thu, 3 Aug 2023 22:27:58 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1B083385773C for ; Thu, 3 Aug 2023 12:27:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 1B083385773C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691065676; bh=bRQjOg2lopNL9nQOouyo1EmsdkX8flRav+Jd0tGJj5E=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=fUX7gYiKg5Tq4iE+oO3BFoKXgscE/MUZhWguVIhYFret7m0HnGNaq58LVqnrNOAHb srzukavTLr3Zog3lRf2W0mFVR60Gi8bSOTD6dx3j/xSvf/V2etQq89ik+pGOkZKqA1 J7ZCTgXBvITNpsT8NAdu78wnqqSWB97tpGy9GMkQ= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2068.outbound.protection.outlook.com [40.107.105.68]) by sourceware.org (Postfix) with ESMTPS id DC1A23858D35 for ; Thu, 3 Aug 2023 12:27:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org DC1A23858D35 Received: from DUZPR01CA0213.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b4::27) by VE1PR08MB5744.eurprd08.prod.outlook.com (2603:10a6:800:1af::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:30 +0000 Received: from DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:4b4:cafe::69) by DUZPR01CA0213.outlook.office365.com (2603:10a6:10:4b4::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Thu, 3 Aug 2023 12:27:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT012.mail.protection.outlook.com (100.127.142.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:29 +0000 Received: ("Tessian outbound e1fdbe8a48d3:v145"); Thu, 03 Aug 2023 12:27:29 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 299765efb5627884 X-CR-MTA-TID: 64aa7808 Received: from 0d47cd273caa.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 132EA143-EB96-4312-B9FD-42D334E31A94.1; Thu, 03 Aug 2023 12:27:23 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 0d47cd273caa.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 03 Aug 2023 12:27:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VBS5HRTX8Dji7wVRlK62GLTdE1xby7Rqq8i4l7eO5XRbHAp9qI7PWOmJXR2q5RNpiHIEtXA31MKFT/CyBRQbNdDBE/mf0fdgNWdPZO/+r/l5/uffgpGI0xkTFUiIPcnbtVfA3JnLH0UUTzMhT/+HoRRp+K+Zg/YtGwDunzQrtTLM5FAAGPsxlzrHwLcHS6xxGSmZMQFFgAbTYnL2NBByPmcgtA0snf6XVs/w66bGcl+hSE/FYyx2jJuTKQqNmM9dakoQh5Fpo9gGsKKrYucXXCJF0By1ct11EpwEpygWEH2DwaLdqwKiYKNP/AQup6qEHZyx96ykEuhpt33Q+g/45A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bRQjOg2lopNL9nQOouyo1EmsdkX8flRav+Jd0tGJj5E=; b=LxJa8l1bIWwx4lwlCkyktWf+WfSPjslRgH2AP7CEJrvim8E7n89wrHOESYd6pjwzXtWxO6K0ML6CEgGIhuoej8/nJl2/wWVHIDnthThbICkBtrIrJgUjKPy9Bcc+hm5FhVCLmJiH8ep9d0L/jo4TH7LHZT0fguCK4bJ4P55Jy9BSvT0//ijnR0SS5jdcyxf0Nsh2T2hbCdspH2SsFfB1L2bjoiHc/TfTlY8pLYVTQA8LC63RIjtB9lNGI6dN/yW+rxMYRXRjmfZcejpXPiotJx4J8l6r/IrCplj764Om25ZsZqvCDhn8LEIiAhYvW7t/fkbiJulr+Y4zr6BbVsEqEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from DUZPR01CA0189.eurprd01.prod.exchangelabs.com (2603:10a6:10:4b6::7) by DU0PR08MB8164.eurprd08.prod.outlook.com (2603:10a6:10:3ef::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:20 +0000 Received: from DBAEUR03FT019.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:4b6:cafe::5e) by DUZPR01CA0189.outlook.office365.com (2603:10a6:10:4b6::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT019.mail.protection.outlook.com (100.127.142.129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6631.45 via Frontend Transport; Thu, 3 Aug 2023 12:27:20 +0000 Received: from AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) by AZ-NEU-EX03.Arm.com (10.251.24.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:19 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX02.Emea.Arm.com (10.251.26.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:19 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 3 Aug 2023 12:27:19 +0000 To: CC: Joe Ramsay Subject: [PATCH 3/5] aarch64: Add vector implementations of log2 routines Date: Thu, 3 Aug 2023 13:27:15 +0100 Message-ID: <20230803122717.33413-3-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230803122717.33413-1-Joe.Ramsay@arm.com> References: <20230803122717.33413-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DBAEUR03FT019:EE_|DU0PR08MB8164:EE_|DBAEUR03FT012:EE_|VE1PR08MB5744:EE_ X-MS-Office365-Filtering-Correlation-Id: 3fd75797-1324-4695-a354-08db941d0137 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: b5A3xLqhCqnEVw20B92Qk9WM8t8/9KnQj8qK9LyIpZgfwsC/AaOtQE4wrWvvMlDCQYwHlDhfIdnSygl9jRppsSJvjLlJVFYJM476TE14mklN9XiSwctDtenaJYpj52QMAqyiRdMqSgDYwLdhnNoCGdhbYPG8iEnIGwvJcwGrMAk1Z8AVoCWCZOZj2MUQe1WVzUQed6PML66QGlxjpp/T/hd9AceBd9XaQ5QcwEh7+qp+QNJWfqc7T+CL7emE1TqT7T9msBjBvOcs8jwnnhHeppynmMj2YR6zpXdlY+EDezH294Jfx2EU2/Q+p9NYTXYoKpGuq/DDIOksuGgG2+lCk4b0vlfg0F6MBe/0sY3ahyE5IFhFrwITixm4TVRtwmbskj+pnZym1Lpge/YkzHh6eQJP7ee8VGlPuK6UAdBpn+s7aHh7kYphffkmIpBHlbgSRpRtg78Oe7efN82joGYXJopG3SazNPI5UtzwyTmVdtimiHGXuJAgDGNO6WdODuLc0QxBJNi5DKpUkQJUMr7iQU8fUmoTP3wZzUGw+kBfidtNu2Xs0+NqbjrbANnOFdoN8mdQCaWv1POGfcuVZE73n9U52Zu2FhL2JUUUB3lO4B6uLz1EFcUMRhcLAJADJSd6gW1ROky2/oLXyZIcK65q+c9knk+yCfAe3rBNAHjJpQTKLy5IwYLWLl+HbPEbsD3lMMFCJaJqd0lr3pE2J+vSnC7txeHLzq/aylgjqFF5TfR9ItTzPGT1TUe/Xblve+qm0bxu0zIenweMvgSTbetdpt33sIfRiiFCLyXBtSqGLiDC72F3edcpbP9KFBDADvjq X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(136003)(39860400002)(451199021)(82310400008)(40470700004)(36840700001)(46966006)(30864003)(41300700001)(40460700003)(316002)(2906002)(4326008)(70206006)(70586007)(6916009)(8936002)(8676002)(5660300002)(356005)(81166007)(82740400003)(26005)(1076003)(36756003)(186003)(86362001)(83380400001)(2616005)(426003)(336012)(47076005)(36860700001)(478600001)(7696005)(6666004)(40480700001)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8164 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 35f7e024-1c81-4a4e-7e5f-08db941cfb8a X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p1ntG4H7fGuTGOCqzibOroHkbCR/yw1qst/fqQpFnMaGTEk7Fjui7FJE8690UeIVagh10FJdQAIU406L3GusYB+kXfM/rD+HiIld072DNB3uGRwJ03cwdevp0vyQSDH8WYcBUfZ8cxLaabI8bNIcFu8u2kWca0JGuLxvdhdfG7qAwDxmeUdFmdnks5u5xpgETLwAbAodP8ehrRgTnIPVQHykaOrBtzhJGTp5e3n/QXLEzJ9sBBAcoDz8NZcp6CBR7Yid+u1rYUkglyIbgULDYHG6E+tdbiZs8nogK6OJ7R6oMkCe+cWhMq1s8D+c3xEhgis9sqmuvt6nemmrkYcTQi8l0Am0kgx+jDg7/91F80FHIhkcgtGv8KFIaJzzOvUobkm/cv8xlfcmKJp7QsMK/NszqVfA9gH/leLluorXuMAZw1kolu53GZKDtJq/pHLDogV/o1CKBZQfePc6CDyRMmEdB+av3/j/xPbUx+eZ5RIKEGujJRslEkd8dQt52nfjJLK/8YHzsZlgMW2Ab5mLsZrmJJ0ZzVt/xyJ6Z5FOCxvwOXvRFU8lNI9mdaQvqiRxrsmHj3fM8lGwpuGSILGwkcRFSNofEClevcd5BBARL5waEoWsGpTmyB2ne6aAMsoj33HDOuep16k8dlnbVcxQ1mEmN8HmajnUn0Mg9PsqUbvNOobASX63ZW+aBC9DJrKx8Pvd5gxVsKq7VSQhh5RjH2vTQbLIz0B/ZbJ8CKskHU81fK26tLNV71rad7AsSvH4naAfB2i2ahfv5UeSsRlg6w== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(136003)(376002)(346002)(39860400002)(82310400008)(451199021)(36840700001)(40470700004)(46966006)(82740400003)(40460700003)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(6666004)(478600001)(40480700001)(81166007)(36756003)(86362001)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 12:27:29.8286 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3fd75797-1324-4695-a354-08db941d0137 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5744 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" A table is also added, which is shared between AdvSIMD and SVE log2. --- Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log2_advsimd.c | 113 ++++++++++++ sysdeps/aarch64/fpu/log2_sve.c | 71 ++++++++ sysdeps/aarch64/fpu/log2f_advsimd.c | 77 ++++++++ sysdeps/aarch64/fpu/log2f_sve.c | 87 +++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log2_data.c | 165 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 10 ++ sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 550 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log2_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2_sve.c create mode 100644 sysdeps/aarch64/fpu/log2f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log2f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log2_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 9c7c768301..c3f204ff0d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log2 \ sin \ tan @@ -16,7 +17,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix f_sve,$(float-sve-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ - v_exp_data + v_exp_data \ + v_log2_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 05de4325d5..ffe62a6f65 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log2f; + _ZGVnN2v_log2; + _ZGVsMxv_log2f; + _ZGVsMxv_log2; _ZGVnN4v_tanf; _ZGVnN2v_tan; _ZGVsMxv_tanf; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 50921b22e5..92f214b194 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -72,6 +74,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -79,6 +82,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log2_advsimd.c b/sysdeps/aarch64/fpu/log2_advsimd.c new file mode 100644 index 0000000000..7617ac57e0 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_advsimd.c @@ -0,0 +1,113 @@ +/* Double-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +#define V_LOG2_TABLE_BITS 7 +#define N (1 << V_LOG2_TABLE_BITS) + +static const struct data +{ + float64x2_t poly[5]; + float64x2_t invln2; + uint64x2_t min_norm, special_bound, sign_exp_mask; +} data = { + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { V2 (-0x1.71547652b83p-1), V2 (0x1.ec709dc340953p-2), + V2 (-0x1.71547651c8f35p-2), V2 (0x1.2777ebe12dda5p-2), + V2 (-0x1.ec738d616fe26p-3) }, + .invln2 = V2 (0x1.71547652b82fep0), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V2 (0x7fe0000000000000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask v_u64 (N - 1) + +#define T(s, i) __v_log2_data.s[i] +#define C(i) d->poly[i] + +struct entry +{ + float64x2_t invc; + float64x2_t log2c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + e.invc[0] = T (invc, i[0]); + e.log2c[0] = T (log2c, i[0]); + e.invc[1] = T (invc, i[1]); + e.log2c[1] = T (log2c, i[1]); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, uint64x2_t special) +{ + return v_call_f64 (log2, x, y, special); +} + +/* Double-precision vector log2 routine. Implements the same algorithm as + vector log10, with coefficients and table entries scaled in extended + precision. The maximum observed error is 2.58 ULP: + _ZGVnN2v_log2(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log2) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint64x2_t special + = vcgeq_u64 (vsubq_u64 (ix, d->min_norm), d->special_bound); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + uint64x2_t i + = vandq_u64 (vshrq_n_u64 (tmp, 52 - V_LOG2_TABLE_BITS), IndexMask); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (i); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + float64x2_t w = vfmaq_f64 (e.log2c, r, d->invln2); + + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t p_23 = vfmaq_f64 (C (2), C (3), r); + float64x2_t p_01 = vfmaq_f64 (C (0), C (1), r); + float64x2_t y; + y = vfmaq_f64 (p_23, C (4), r2); + y = vfmaq_f64 (p_01, r2, y); + y = vfmaq_f64 (vaddq_f64 (kd, w), r2, y); + + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2_sve.c b/sysdeps/aarch64/fpu/log2_sve.c new file mode 100644 index 0000000000..6645e0369b --- /dev/null +++ b/sysdeps/aarch64/fpu/log2_sve.c @@ -0,0 +1,71 @@ +/* Double-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define N (1 << V_LOG2_TABLE_BITS) +#define Off 0x3fe6900900000000 +#define Max (0x7ff0000000000000) +#define Min (0x0010000000000000) +#define Thresh (0x7fe0000000000000) /* Max - Min. */ + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, const svbool_t cmp) +{ + return sv_call_f64 (log2, x, y, cmp); +} + +/* Double-precision SVE log2 routine. + Implements the same algorithm as AdvSIMD log10, with coefficients and table + entries scaled in extended precision. + The maximum observed error is 2.58 ULP: + SV_NAME_D1 (log2)(0x1.0b556b093869bp+0) got 0x1.fffb34198d9dap-5 + want 0x1.fffb34198d9ddp-5. */ +svfloat64_t SV_NAME_D1 (log2) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thresh); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i + = sv_mod_n_u64_x (pg, svlsr_x (pg, tmp, 52 - V_LOG2_TABLE_BITS), N); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + svfloat64_t invc = svld1_gather_index (pg, __v_log2_data.invc, i); + svfloat64_t log2c = svld1_gather_index (pg, __v_log2_data.log2c, i); + + /* log2(x) = log1p(z/c-1)/log(2) + log2(c) + k. */ + + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + svfloat64_t w = svmla_x (pg, log2c, r, __v_log2_data.invln2); + + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log2_data.poly); + y = svmla_x (pg, svadd_x (pg, k, w), r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2f_advsimd.c b/sysdeps/aarch64/fpu/log2f_advsimd.c new file mode 100644 index 0000000000..085c18489f --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_advsimd.c @@ -0,0 +1,77 @@ +/* Single-precision vector (AdvSIMD) exp2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[9]; + float32x4_t ln2; + uint32x4_t min_norm, special_bound, off, mantissa_mask; +} data = { + /* Coefficients generated using Remez algorithm approximate + log2(1+r)/r for r in [ -1/3, 1/3 ]. + rel error: 0x1.c4c4b0cp-26. */ + .poly = { V4 (0x1.715476p0f), /* (float)(1 / ln(2)). */ + V4 (-0x1.715458p-1f), V4 (0x1.ec701cp-2f), V4 (-0x1.7171a4p-2f), + V4 (0x1.27a0b8p-2f), V4 (-0x1.e5143ep-3f), V4 (0x1.9d8ecap-3f), + V4 (-0x1.c675bp-3f), V4 (0x1.9e495p-3f) }, + .ln2 = V4 (0x1.62e43p-1f), + .min_norm = V4 (0x00800000), + .special_bound = V4 (0x7f000000), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log2f, x, y, cmp); +} + +/* Fast implementation for single precision AdvSIMD log2, + relies on same argument reduction as AdvSIMD logf. + Maximum error: 2.48 ULPs + _ZGVnN4v_log2f(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log2) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint32x4_t special + = vcgeq_u32 (vsubq_u32 (u, d->min_norm), d->special_bound); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log2(1+r) + n. */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t p = v_pw_horner_8_f32 (r, r2, d->poly); + float32x4_t y = vfmaq_f32 (n, p, r); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log2f_sve.c b/sysdeps/aarch64/fpu/log2f_sve.c new file mode 100644 index 0000000000..1efdd00ab1 --- /dev/null +++ b/sysdeps/aarch64/fpu/log2f_sve.c @@ -0,0 +1,87 @@ +/* Single-precision vector (SVE) log2 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_02468[5]; + float poly_1357[4]; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + -0x1.715458p-1f, -0x1.7171a4p-2f, -0x1.e5143ep-3f, -0x1.c675bp-3f + }, + .poly_02468 = { 0x1.715476p0f, 0x1.ec701cp-2f, 0x1.27a0b8p-2f, + 0x1.9d8ecap-3f, 0x1.9e495p-3f }, +}; + +#define Min (0x00800000) +#define Max (0x7f800000) +#define Thres (0x7f000000) /* Max - Min. */ +#define MantissaMask (0x007fffff) +#define Off (0x3f2aaaab) /* 0.666667. */ + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t cmp) +{ + return sv_call_f32 (log2f, x, y, cmp); +} + +/* Optimised implementation of SVE log2f, using the same algorithm + and polynomial as AdvSIMD log2f. + Maximum error is 2.48 ULPs: + SV_NAME_F1 (log2)(0x1.558174p+0) got 0x1.a9be84p-2 + want 0x1.a9be8p-2. */ +svfloat32_t SV_NAME_F1 (log2) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + + svuint32_t u = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, u, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = svsub_x (pg, u, Off); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (u), 23)); /* Sign-extend. */ + u = svand_x (pg, u, MantissaMask); + u = svadd_x (pg, u, Off); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (u), 1.0f); + + /* y = log2(1+r) + n. */ + svfloat32_t r2 = svmul_x (pg, r, r); + + /* Evaluate polynomial using pairwise Horner scheme. */ + svfloat32_t p_1357 = svld1rq (pg, &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_02468[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_02468[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_02468[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_02468[3]), r, p_1357, 3); + svfloat32_t y = svmla_x (pg, q_67, r2, sv_f32 (d->poly_02468[4])); + y = svmla_x (pg, q_45, r2, y); + y = svmla_x (pg, q_23, r2, y); + y = svmla_x (pg, q_01, r2, y); + y = svmla_x (pg, n, r, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index b2b36fd847..d30dcd6f95 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 88b76ed678..22a8479100 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 02ab609b5a..e8f7f47c67 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,5 +27,6 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index fa41ce09d8..f5e9584265 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,5 +36,6 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log2_data.c b/sysdeps/aarch64/fpu/v_log2_data.c new file mode 100644 index 0000000000..2b62c59fe1 --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log2_data.c @@ -0,0 +1,165 @@ +/* Coefficients and table entries for vector log2 + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +#define N (1 << V_LOG2_TABLE_BITS) + +const struct v_log2_data __v_log2_data = { + + /* Each coefficient was generated to approximate log(r) for |r| < 0x1.fp-9 + and N = 128, then scaled by log2(e) in extended precision and rounded back + to double precision. */ + .poly = { -0x1.71547652b83p-1, 0x1.ec709dc340953p-2, -0x1.71547651c8f35p-2, + 0x1.2777ebe12dda5p-2, -0x1.ec738d616fe26p-3 }, + + .invln2 = 0x1.71547652b82fep0, + + /* Derived from tables in v_log_data.c in a similar way as v_log10_data.c. + This means invc is unchanged and log2c was calculated by scaling log(c) by + log2(e) in extended precision and rounding back to double precision. */ + .invc = { 0x1.6a133d0dec120p+0, 0x1.6815f2f3e42edp+0, + 0x1.661e39be1ac9ep+0, 0x1.642bfa30ac371p+0, + 0x1.623f1d916f323p+0, 0x1.60578da220f65p+0, + 0x1.5e75349dea571p+0, 0x1.5c97fd387a75ap+0, + 0x1.5abfd2981f200p+0, 0x1.58eca051dc99cp+0, + 0x1.571e526d9df12p+0, 0x1.5554d555b3fcbp+0, + 0x1.539015e2a20cdp+0, 0x1.51d0014ee0164p+0, + 0x1.50148538cd9eep+0, 0x1.4e5d8f9f698a1p+0, + 0x1.4cab0edca66bep+0, 0x1.4afcf1a9db874p+0, + 0x1.495327136e16fp+0, 0x1.47ad9e84af28fp+0, + 0x1.460c47b39ae15p+0, 0x1.446f12b278001p+0, + 0x1.42d5efdd720ecp+0, 0x1.4140cfe001a0fp+0, + 0x1.3fafa3b421f69p+0, 0x1.3e225c9c8ece5p+0, + 0x1.3c98ec29a211ap+0, 0x1.3b13442a413fep+0, + 0x1.399156baa3c54p+0, 0x1.38131639b4cdbp+0, + 0x1.36987540fbf53p+0, 0x1.352166b648f61p+0, + 0x1.33adddb3eb575p+0, 0x1.323dcd99fc1d3p+0, + 0x1.30d129fefc7d2p+0, 0x1.2f67e6b72fe7dp+0, + 0x1.2e01f7cf8b187p+0, 0x1.2c9f518ddc86ep+0, + 0x1.2b3fe86e5f413p+0, 0x1.29e3b1211b25cp+0, + 0x1.288aa08b373cfp+0, 0x1.2734abcaa8467p+0, + 0x1.25e1c82459b81p+0, 0x1.2491eb1ad59c5p+0, + 0x1.23450a54048b5p+0, 0x1.21fb1bb09e578p+0, + 0x1.20b415346d8f7p+0, 0x1.1f6fed179a1acp+0, + 0x1.1e2e99b93c7b3p+0, 0x1.1cf011a7a882ap+0, + 0x1.1bb44b97dba5ap+0, 0x1.1a7b3e66cdd4fp+0, + 0x1.1944e11dc56cdp+0, 0x1.18112aebb1a6ep+0, + 0x1.16e013231b7e9p+0, 0x1.15b1913f156cfp+0, + 0x1.14859cdedde13p+0, 0x1.135c2dc68cfa4p+0, + 0x1.12353bdb01684p+0, 0x1.1110bf25b85b4p+0, + 0x1.0feeafd2f8577p+0, 0x1.0ecf062c51c3bp+0, + 0x1.0db1baa076c8bp+0, 0x1.0c96c5bb3048ep+0, + 0x1.0b7e20263e070p+0, 0x1.0a67c2acd0ce3p+0, + 0x1.0953a6391e982p+0, 0x1.0841c3caea380p+0, + 0x1.07321489b13eap+0, 0x1.062491aee9904p+0, + 0x1.05193497a7cc5p+0, 0x1.040ff6b5f5e9fp+0, + 0x1.0308d19aa6127p+0, 0x1.0203beedb0c67p+0, + 0x1.010037d38bcc2p+0, 1.0, + 0x1.fc06d493cca10p-1, 0x1.f81e6ac3b918fp-1, + 0x1.f44546ef18996p-1, 0x1.f07b10382c84bp-1, + 0x1.ecbf7070e59d4p-1, 0x1.e91213f715939p-1, + 0x1.e572a9a75f7b7p-1, 0x1.e1e0e2c530207p-1, + 0x1.de5c72d8a8be3p-1, 0x1.dae50fa5658ccp-1, + 0x1.d77a71145a2dap-1, 0x1.d41c51166623ep-1, + 0x1.d0ca6ba0bb29fp-1, 0x1.cd847e8e59681p-1, + 0x1.ca4a499693e00p-1, 0x1.c71b8e399e821p-1, + 0x1.c3f80faf19077p-1, 0x1.c0df92dc2b0ecp-1, + 0x1.bdd1de3cbb542p-1, 0x1.baceb9e1007a3p-1, + 0x1.b7d5ef543e55ep-1, 0x1.b4e749977d953p-1, + 0x1.b20295155478ep-1, 0x1.af279f8e82be2p-1, + 0x1.ac5638197fdf3p-1, 0x1.a98e2f102e087p-1, + 0x1.a6cf5606d05c1p-1, 0x1.a4197fc04d746p-1, + 0x1.a16c80293dc01p-1, 0x1.9ec82c4dc5bc9p-1, + 0x1.9c2c5a491f534p-1, 0x1.9998e1480b618p-1, + 0x1.970d9977c6c2dp-1, 0x1.948a5c023d212p-1, + 0x1.920f0303d6809p-1, 0x1.8f9b698a98b45p-1, + 0x1.8d2f6b81726f6p-1, 0x1.8acae5bb55badp-1, + 0x1.886db5d9275b8p-1, 0x1.8617ba567c13cp-1, + 0x1.83c8d27487800p-1, 0x1.8180de3c5dbe7p-1, + 0x1.7f3fbe71cdb71p-1, 0x1.7d055498071c1p-1, + 0x1.7ad182e54f65ap-1, 0x1.78a42c3c90125p-1, + 0x1.767d342f76944p-1, 0x1.745c7ef26b00ap-1, + 0x1.7241f15769d0fp-1, 0x1.702d70d396e41p-1, + 0x1.6e1ee3700cd11p-1, 0x1.6c162fc9cbe02p-1 }, + .log2c = { -0x1.00130d57f5fadp-1, -0x1.f802661bd725ep-2, + -0x1.efea1c6f73a5bp-2, -0x1.e7dd1dcd06f05p-2, + -0x1.dfdb4ae024809p-2, -0x1.d7e484d101958p-2, + -0x1.cff8ad452f6ep-2, -0x1.c817a666c997fp-2, + -0x1.c04152d640419p-2, -0x1.b87595a3f64b2p-2, + -0x1.b0b4526c44d07p-2, -0x1.a8fd6d1a90f5ep-2, + -0x1.a150ca2559fc6p-2, -0x1.99ae4e62cca29p-2, + -0x1.9215df1a1e842p-2, -0x1.8a8761fe1f0d9p-2, + -0x1.8302bd1cc9a54p-2, -0x1.7b87d6fb437f6p-2, + -0x1.741696673a86dp-2, -0x1.6caee2b3c6fe4p-2, + -0x1.6550a3666c27ap-2, -0x1.5dfbc08de02a4p-2, + -0x1.56b022766c84ap-2, -0x1.4f6db1c955536p-2, + -0x1.4834579063054p-2, -0x1.4103fd2249a76p-2, + -0x1.39dc8c3fe6dabp-2, -0x1.32bdeed4b5c8fp-2, + -0x1.2ba80f41e20ddp-2, -0x1.249ad8332f4a7p-2, + -0x1.1d96347e7f3ebp-2, -0x1.169a0f7d6604ap-2, + -0x1.0fa654a221909p-2, -0x1.08baefcf8251ap-2, + -0x1.01d7cd14deecdp-2, -0x1.f5f9b1ad55495p-3, + -0x1.e853ff76a77afp-3, -0x1.dabe5d624cba1p-3, + -0x1.cd38a5cef4822p-3, -0x1.bfc2b38d315f9p-3, + -0x1.b25c61f5edd0fp-3, -0x1.a5058d18e9cacp-3, + -0x1.97be1113e47a3p-3, -0x1.8a85cafdf5e27p-3, + -0x1.7d5c97e8fc45bp-3, -0x1.704255d6486e4p-3, + -0x1.6336e2cedd7bfp-3, -0x1.563a1d9b0cc6ap-3, + -0x1.494be541aaa6fp-3, -0x1.3c6c1964dd0f2p-3, + -0x1.2f9a99f19a243p-3, -0x1.22d747344446p-3, + -0x1.1622020d4f7f5p-3, -0x1.097aabb3553f3p-3, + -0x1.f9c24b48014c5p-4, -0x1.e0aaa3bdc858ap-4, + -0x1.c7ae257c952d6p-4, -0x1.aecc960a03e58p-4, + -0x1.9605bb724d541p-4, -0x1.7d595ca7147cep-4, + -0x1.64c74165002d9p-4, -0x1.4c4f31c86d344p-4, + -0x1.33f0f70388258p-4, -0x1.1bac5abb3037dp-4, + -0x1.0381272495f21p-4, -0x1.d6de4eba2de2ap-5, + -0x1.a6ec4e8156898p-5, -0x1.772be542e3e1bp-5, + -0x1.479cadcde852dp-5, -0x1.183e4265faa5p-5, + -0x1.d2207fdaa1b85p-6, -0x1.742486cb4a6a2p-6, + -0x1.1687d77cfc299p-6, -0x1.7293623a6b5dep-7, + -0x1.70ec80ec8f25dp-8, 0.0, + 0x1.704c1ca6b6bc9p-7, 0x1.6eac8ba664beap-6, + 0x1.11e67d040772dp-5, 0x1.6bc665e2105dep-5, + 0x1.c4f8a9772bf1dp-5, 0x1.0ebff10fbb951p-4, + 0x1.3aaf4d7805d11p-4, 0x1.664ba81a4d717p-4, + 0x1.9196387da6de4p-4, 0x1.bc902f2b7796p-4, + 0x1.e73ab5f584f28p-4, 0x1.08cb78510d232p-3, + 0x1.1dd2fe2f0dcb5p-3, 0x1.32b4784400df4p-3, + 0x1.47706f3d49942p-3, 0x1.5c0768ee4a4dcp-3, + 0x1.7079e86fc7c6dp-3, 0x1.84c86e1183467p-3, + 0x1.98f377a34b499p-3, 0x1.acfb803bc924bp-3, + 0x1.c0e10098b025fp-3, 0x1.d4a46efe103efp-3, + 0x1.e8463f45b8d0bp-3, 0x1.fbc6e3228997fp-3, + 0x1.079364f2e5aa8p-2, 0x1.1133306010a63p-2, + 0x1.1ac309631bd17p-2, 0x1.24432485370c1p-2, + 0x1.2db3b5449132fp-2, 0x1.3714ee1d7a32p-2, + 0x1.406700ab52c94p-2, 0x1.49aa1d87522b2p-2, + 0x1.52de746d7ecb2p-2, 0x1.5c0434336b343p-2, + 0x1.651b8ad6c90d1p-2, 0x1.6e24a56ab5831p-2, + 0x1.771fb04ec29b1p-2, 0x1.800cd6f19c25ep-2, + 0x1.88ec441df11dfp-2, 0x1.91be21b7c93f5p-2, + 0x1.9a8298f8c7454p-2, 0x1.a339d255c04ddp-2, + 0x1.abe3f59f43db7p-2, 0x1.b48129deca9efp-2, + 0x1.bd119575364c1p-2, 0x1.c5955e23ebcbcp-2, + 0x1.ce0ca8f4e1557p-2, 0x1.d6779a5a75774p-2, + 0x1.ded6563550d27p-2, 0x1.e728ffafd840ep-2, + 0x1.ef6fb96c8d739p-2, 0x1.f7aaa57907219p-2 } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index e7d30b477f..82a68fd35d 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -48,4 +48,14 @@ extern const struct v_log_data #define V_EXP_TABLE_BITS 7 extern const uint64_t __v_exp_data[1 << V_EXP_TABLE_BITS] attribute_hidden; + +#define V_LOG2_TABLE_BITS 7 +extern const struct v_log2_data +{ + double poly[5]; + double invln2; + double invc[1 << V_LOG2_TABLE_BITS]; + double log2c[1 << V_LOG2_TABLE_BITS]; +} __v_log2_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index a1e5651c87..7a5af571e2 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1220,11 +1220,19 @@ double: 1 float: 1 ldouble: 3 +Function: "log2_advsimd": +double: 1 +float: 2 + Function: "log2_downward": double: 3 float: 3 ldouble: 3 +Function: "log2_sve": +double: 1 +float: 2 + Function: "log2_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 75550bf840..376dcdd6aa 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -18,6 +18,10 @@ GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVnN2v_log2 F +GLIBC_2.39 _ZGVnN4v_log2f F +GLIBC_2.39 _ZGVsMxv_log2 F +GLIBC_2.39 _ZGVsMxv_log2f F GLIBC_2.39 _ZGVnN2v_tan F GLIBC_2.39 _ZGVnN4v_tanf F GLIBC_2.39 _ZGVsMxv_tan F From patchwork Thu Aug 3 12:27:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1816427 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=UVgObFbq; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RGp6M6FLRz1yZl for ; Thu, 3 Aug 2023 22:28:27 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D3FC63857025 for ; Thu, 3 Aug 2023 12:28:25 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org D3FC63857025 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691065705; bh=cbpWkqVXCLHDEyZUANPiUaF/XoctFVLCjZ2aHzGjEAo=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=UVgObFbqcXfkjCdp2a1rhpHQ1BI55jdjPa5PyVIRdo/vPiUI2z6gEEqvtb2qvT7Dp RNS4IhxSW8OzpDaRz0WqzGxSZSeB6B7sFS2AEgsQK+A59IuX116JvJrkVcQYp5jQyv AfHlhKv0dzY+1Rpd9FahQ4fawdXmukVJpTi89DYM= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR02-VI1-obe.outbound.protection.outlook.com (mail-vi1eur02on2043.outbound.protection.outlook.com [40.107.241.43]) by sourceware.org (Postfix) with ESMTPS id 6B78A3858426 for ; Thu, 3 Aug 2023 12:27:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6B78A3858426 Received: from AS9PR05CA0041.eurprd05.prod.outlook.com (2603:10a6:20b:489::21) by GV1PR08MB8059.eurprd08.prod.outlook.com (2603:10a6:150:95::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:32 +0000 Received: from AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:489:cafe::7d) by AS9PR05CA0041.outlook.office365.com (2603:10a6:20b:489::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT024.mail.protection.outlook.com (100.127.140.238) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:32 +0000 Received: ("Tessian outbound d7adc65d10b4:v145"); Thu, 03 Aug 2023 12:27:32 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 4a574cb53fab6e55 X-CR-MTA-TID: 64aa7808 Received: from 2146d68436d3.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 2743FE57-498A-4ABD-A444-994E9E78F0E0.1; Thu, 03 Aug 2023 12:27:24 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2146d68436d3.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 03 Aug 2023 12:27:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eZne7tHdCinv65lVzfL+16reU4eG4WBbHP8tqlXL8fkAb4O1yk8S9WLR1jsds6AyLoBuNG+82UuXtWcrSLiZ/7T7CoiTDuCdD7Iyv35H7MxdJmtspVsdP+G+cprVf2GfzJaHkwMtFd+3cyi7vNhdiySy6S976ZJqfVhh+h7hX5mXd37iu46R7I6/CUcc8LLPKB/HNtGTgkuUA4jx1ZSQ7ocedk1jngO06o991gU+StE7TWEzxNu4k9U/b0LTnRhw0Fix9N3GOD66DEt4CsuDFs58s5Ik/X5n0BhxacsG3K6WA2igzZBTaP8Qv+uIBF3molJWRcx6r1v0nVYIbd4kYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cbpWkqVXCLHDEyZUANPiUaF/XoctFVLCjZ2aHzGjEAo=; b=RUBAfWmmM91U5DuLS/9+cEpmxJwaR1mm1s+gAIKan4qOB/n849hGIzd6ansDuCK2zRK3XXkzavCIZHNmNYau9oNrBVeatd55Kp7rnzskkcv37k/QsInDoJ0Xn98tTgBVxZSdXOcC8iFNSCx867vX82gTDORJgjrpvYBfFXNHAbOYcAWrlBfoDuniCig45DAZ0XrsSmo/aGwhrYmOTh98pJCJ0SBvNPQnrbOo5IptKiTVCyhfaD8UqdWF2ub2HU3/PygpcSo/FwK6pFU7k7WLghwJdaubXC8gWPivwoB/2ABBWNQvoYuLKT1uUlVcuxkT/KiMq+wZIGS0Fqq4r/5i7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from DB7PR02CA0015.eurprd02.prod.outlook.com (2603:10a6:10:52::28) by DB4PR08MB10341.eurprd08.prod.outlook.com (2603:10a6:10:3ff::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:21 +0000 Received: from DBAEUR03FT049.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:52:cafe::23) by DB7PR02CA0015.outlook.office365.com (2603:10a6:10:52::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.19 via Frontend Transport; Thu, 3 Aug 2023 12:27:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT049.mail.protection.outlook.com (100.127.142.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:21 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:19 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 3 Aug 2023 12:27:19 +0000 To: CC: Joe Ramsay Subject: [PATCH 4/5] aarch64: Add vector implementations of log10 routines Date: Thu, 3 Aug 2023 13:27:16 +0100 Message-ID: <20230803122717.33413-4-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230803122717.33413-1-Joe.Ramsay@arm.com> References: <20230803122717.33413-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DBAEUR03FT049:EE_|DB4PR08MB10341:EE_|AM7EUR03FT024:EE_|GV1PR08MB8059:EE_ X-MS-Office365-Filtering-Correlation-Id: 406f93e8-478d-4e4b-fad4-08db941d02b0 x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: ej8h0CeycOTQfvnCmiXzGCvpUV0P8CS0Es1pEkh6NTCl2+rebm3Jz5HuIHE/dSr8erZFLHltb00NwX/LbDvuyypIVAU/5FyAA5p1o7i1Zi668Sl1mGiZ+ovJB/B911RJOsAdmdgkS8UXtvC6uyhFSWtFdnBBHepR6phCVrMFTCs8AVgqhgErEUCa1qwMjtFfKNi326d6jOgoKUuxIQJxHW8dzMRtaSxcQjyQ43XvXjy2ZhONhaleDfiWO9UxUroVYQtJoDcP2j9DYAnjNnyHhABev4livzovLjpUCULpn6d+ube3AqZg5ZLvzVsPPASAoEUZ4NHWPZz7t+/w8hTer9GZ9cpyt7FZWZ63E6BlkwxDnZtoE6KQMYkpKtiu3OU6bwKxV7ENrYZy96eG8ceTgiqE88wYMxfbVXDAA1kekIx/rHfNe6CzQhKh9voQAWUcLVFEcqC+6x05lFMUYVTf72ZTNNbMGsg/WokSw+2w2CMnzV+/n8f5TGhesYfAUOnG8eiySODyFi78XgWE60YM4wLb4A+8WOkpztefpoC5sToHeFMIpOH332IQW+f3KfxnIOhqJoEQ2vcxziAZwNxRXEna1GC9AOuCyQYRxqnQBCSBd3+9tOEw8rBiXfKOL3E0C47S9LG1gTIiOyfbWe6ubXrX3Waje4cyqy1kLPvWkQQaXpsZZBuO0Z61BCULB3oHIeCcqrMqhW8HB64t8ATynFzX5b6JUaBLAZvmSENTOYRxY2WAAOC6TJwEh233XsN9YzDcs5JInW6WFnd7IOeffg== X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(346002)(39860400002)(376002)(451199021)(82310400008)(46966006)(36840700001)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(6666004)(478600001)(40480700001)(356005)(81166007)(86362001)(36756003)(82740400003)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB10341 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 57afea9f-60f9-42dd-d995-08db941cfc32 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O3ROmcT6ELODmtbnGqORG+tWxNluodg+E9gEt4ZSHnju/CyKDn7B74ehafEBtFajG+xG7+5i3qO1/NDGZZH7jIq7NGyq8K4Hr+gdd1IEN2paxnOfOCy4DdF5OeCmTk8c5YJxX2bDkMJ5yD7V6wVd15GnsjARIMqkqUTvyboABXGdviITbPRCB6Qt7M1XlQ2guvOz43E6Dh5Vbl7CF6z0+sj0dcra44pR1gZZ3mUrFamDJjCACIVKoRgFexiXxqjhpRCkzVtGvqwnMa/JnSshv2a07W0YaFYSwGyWOPIzUBY6CXHR5Rkdvghf2VXnxvYT765NyJc80cDuLBRHvuz/yRHazTvdMyTpbXpPekmgLzWDHgQ/MCqtzAP/f+bNvcwADRyuza1vC4+BbG27RsLp5aOrQyeKfV4/eCiTGCuq536Y0+5m+ud5fJDwUBGqXJcj+YVJp/Am7YGL2Y3t7Z/MwNBM4U+jybxz8GPNk5AyZU67oRWREneZf6QA6o2wt5ariUWoKlZLpyCTo+6HfINi5YGbjhDVz0iaDqg1Z31RKGGOQYK0eaQDKekgAZklIexIedj6Q/o0xeuEc2q9DFrA7r9v/hvEEYnVAtKAQ9y0y7FAUgst4iimrkNyq0SfmtYZcO1PM+Ubz4YHAPD6gGaLiLZ0x+WN9z2ngO0nhgIf0RsI1srydfmFBVRnBAt02rtkumqFsUNT6TW8L18Kh6E8qXuWCTGnKAw49z6UrqBkEXRCfGh4Lq+7T3idJULAh1n1yCS7a/DeiBPfUbpyq+V85g== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(376002)(39860400002)(346002)(396003)(136003)(82310400008)(451199021)(36840700001)(46966006)(40470700004)(40460700003)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(6666004)(478600001)(40480700001)(81166007)(36756003)(86362001)(82740400003)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 12:27:32.2391 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 406f93e8-478d-4e4b-fad4-08db941d02b0 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT024.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB8059 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" A table is also added, which is shared between AdvSIMD and SVE log10. --- Thanks, Joe sysdeps/aarch64/fpu/Makefile | 4 +- sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/log10_advsimd.c | 117 ++++++++++++ sysdeps/aarch64/fpu/log10_sve.c | 77 ++++++++ sysdeps/aarch64/fpu/log10f_advsimd.c | 79 ++++++++ sysdeps/aarch64/fpu/log10f_sve.c | 94 ++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/fpu/v_log10_data.c | 175 ++++++++++++++++++ sysdeps/aarch64/fpu/vecmath_config.h | 9 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 15 files changed, 578 insertions(+), 1 deletion(-) create mode 100644 sysdeps/aarch64/fpu/log10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10_sve.c create mode 100644 sysdeps/aarch64/fpu/log10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/log10f_sve.c create mode 100644 sysdeps/aarch64/fpu/v_log10_data.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index c3f204ff0d..0a047a150d 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -2,6 +2,7 @@ libmvec-supported-funcs = cos \ exp \ exp2 \ log \ + log10 \ log2 \ sin \ tan @@ -18,7 +19,8 @@ libmvec-support = $(addsuffix f_advsimd,$(float-advsimd-funcs)) \ $(addsuffix _sve,$(double-sve-funcs)) \ v_log_data \ v_exp_data \ - v_log2_data + v_log2_data \ + v_log10_data endif sve-cflags = -march=armv8-a+sve diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index ffe62a6f65..358efed5ee 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -22,6 +22,10 @@ libmvec { _ZGVnN2v_exp2; _ZGVsMxv_exp2f; _ZGVsMxv_exp2; + _ZGVnN4v_log10f; + _ZGVnN2v_log10; + _ZGVsMxv_log10f; + _ZGVsMxv_log10; _ZGVnN4v_log2f; _ZGVnN2v_log2; _ZGVsMxv_log2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 92f214b194..59f2efa6d7 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -53,6 +53,7 @@ __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_sinf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); @@ -61,6 +62,7 @@ __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_sin (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); @@ -74,6 +76,7 @@ __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_sinf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); @@ -82,6 +85,7 @@ __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_sin (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_tan (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/log10_advsimd.c b/sysdeps/aarch64/fpu/log10_advsimd.c new file mode 100644 index 0000000000..f518b55b60 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_advsimd.c @@ -0,0 +1,117 @@ +/* Double-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f64.h" + +#define N (1 << V_LOG10_TABLE_BITS) + +static const struct data +{ + float64x2_t poly[5]; + float64x2_t invln10, log10_2, ln2; + uint64x2_t min_norm, special_bound, sign_exp_mask; +} data = { + /* Computed from log coefficients divided by log(10) then rounded to double + precision. */ + .poly = { V2 (-0x1.bcb7b1526e506p-3), V2 (0x1.287a7636be1d1p-3), + V2 (-0x1.bcb7b158af938p-4), V2 (0x1.63c78734e6d07p-4), + V2 (-0x1.287461742fee4p-4) }, + .ln2 = V2 (0x1.62e42fefa39efp-1), + .invln10 = V2 (0x1.bcb7b1526e50ep-2), + .log10_2 = V2 (0x1.34413509f79ffp-2), + .min_norm = V2 (0x0010000000000000), /* asuint64(0x1p-1022). */ + .special_bound = V2 (0x7fe0000000000000), /* asuint64(inf) - min_norm. */ + .sign_exp_mask = V2 (0xfff0000000000000), +}; + +#define Off v_u64 (0x3fe6900900000000) +#define IndexMask v_u64 (N - 1) + +#define T(s, i) __v_log10_data.s[i] + +struct entry +{ + float64x2_t invc; + float64x2_t log10c; +}; + +static inline struct entry +lookup (uint64x2_t i) +{ + struct entry e; + e.invc[0] = T (invc, i[0]); + e.log10c[0] = T (log10c, i[0]); + e.invc[1] = T (invc, i[1]); + e.log10c[1] = T (log10c, i[1]); + return e; +} + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t x, float64x2_t y, uint64x2_t special) +{ + return v_call_f64 (log10, x, y, special); +} + +/* Fast implementation of double-precision vector log10 + is a slight modification of double-precision vector log. + Max ULP error: < 2.5 ulp (nearest rounding.) + Maximum measured at 2.46 ulp for x in [0.96, 0.97] + _ZGVnN2v_log10(0x1.13192407fcb46p+0) got 0x1.fff6be3cae4bbp-6 + want 0x1.fff6be3cae4b9p-6. */ +float64x2_t VPCS_ATTR V_NAME_D1 (log10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t ix = vreinterpretq_u64_f64 (x); + uint64x2_t special + = vcgeq_u64 (vsubq_u64 (ix, d->min_norm), d->special_bound); + + /* x = 2^k z; where z is in range [OFF,2*OFF) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + uint64x2_t tmp = vsubq_u64 (ix, Off); + uint64x2_t i + = vandq_u64 (vshrq_n_u64 (tmp, 52 - V_LOG10_TABLE_BITS), IndexMask); + int64x2_t k = vshrq_n_s64 (vreinterpretq_s64_u64 (tmp), 52); + uint64x2_t iz = vsubq_u64 (ix, vandq_u64 (tmp, d->sign_exp_mask)); + float64x2_t z = vreinterpretq_f64_u64 (iz); + + struct entry e = lookup (i); + + /* log10(x) = log1p(z/c-1)/log(10) + log10(c) + k*log10(2). */ + float64x2_t r = vfmaq_f64 (v_f64 (-1.0), z, e.invc); + float64x2_t kd = vcvtq_f64_s64 (k); + + /* hi = r / log(10) + log10(c) + k*log10(2). + Constants in v_log10_data.c are computed (in extended precision) as + e.log10c := e.logc * ivln10. */ + float64x2_t w = vfmaq_f64 (e.log10c, r, d->invln10); + + /* y = log10(1+r) + n * log10(2). */ + float64x2_t hi = vfmaq_f64 (w, kd, d->log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t y = v_pw_horner_4_f64 (r, r2, d->poly); + y = vfmaq_f64 (hi, r2, y); + + if (__glibc_unlikely (v_any_u64 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10_sve.c b/sysdeps/aarch64/fpu/log10_sve.c new file mode 100644 index 0000000000..4522879f99 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10_sve.c @@ -0,0 +1,77 @@ +/* Double-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define Min 0x0010000000000000 +#define Max 0x7ff0000000000000 +#define Thres 0x7fe0000000000000 /* Max - Min. */ +#define Off 0x3fe6900900000000 +#define N (1 << V_LOG10_TABLE_BITS) + +#define T(s, i) __v_log10_data.s[i] + +static svfloat64_t NOINLINE +special_case (svfloat64_t x, svfloat64_t y, svbool_t special) +{ + return sv_call_f64 (log10, x, y, special); +} + +/* SVE log10 algorithm. + Maximum measured error is 2.46 ulps. + SV_NAME_D1 (log10)(0x1.131956cd4b627p+0) got 0x1.fffbdf6eaa669p-6 + want 0x1.fffbdf6eaa667p-6. */ +svfloat64_t SV_NAME_D1 (log10) (svfloat64_t x, const svbool_t pg) +{ + svuint64_t ix = svreinterpret_u64 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^k z; where z is in range [Off,2*Off) and exact. + The range is split into N subintervals. + The ith subinterval contains z and c is near its center. */ + svuint64_t tmp = svsub_x (pg, ix, Off); + svuint64_t i + = sv_mod_n_u64_x (pg, svlsr_x (pg, tmp, 52 - V_LOG10_TABLE_BITS), N); + svfloat64_t k = svcvt_f64_x (pg, svasr_x (pg, svreinterpret_s64 (tmp), 52)); + svfloat64_t z = svreinterpret_f64 ( + svsub_x (pg, ix, svand_x (pg, tmp, 0xfffULL << 52))); + + /* log(x) = k*log(2) + log(c) + log(z/c). */ + svfloat64_t invc = svld1_gather_index (pg, &T (invc, 0), i); + svfloat64_t logc = svld1_gather_index (pg, &T (log10c, 0), i); + + /* We approximate log(z/c) with a polynomial P(x) ~= log(x + 1): + r = z/c - 1 (we look up precomputed 1/c) + log(z/c) ~= P(r). */ + svfloat64_t r = svmad_x (pg, invc, z, -1.0); + + /* hi = log(c) + k*log(2). */ + svfloat64_t w = svmla_x (pg, logc, r, __v_log10_data.invln10); + svfloat64_t hi = svmla_x (pg, w, k, __v_log10_data.log10_2); + + /* y = r2*(A0 + r*A1 + r2*(A2 + r*A3 + r2*A4)) + hi. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = sv_pw_horner_4_f64_x (pg, r, r2, __v_log10_data.poly); + y = svmla_x (pg, hi, r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10f_advsimd.c b/sysdeps/aarch64/fpu/log10f_advsimd.c new file mode 100644 index 0000000000..371495b541 --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_advsimd.c @@ -0,0 +1,79 @@ +/* Single-precision vector (AdvSIMD) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +static const struct data +{ + float32x4_t poly[8]; + float32x4_t inv_ln10, ln2; + uint32x4_t min_norm, special_bound, off, mantissa_mask; +} data = { + /* Use order 9 for log10(1+x), i.e. order 8 for log10(1+x)/x, with x in + [-1/3, 1/3] (offset=2/3). Max. relative error: 0x1.068ee468p-25. */ + .poly = { V4 (-0x1.bcb79cp-3f), V4 (0x1.2879c8p-3f), V4 (-0x1.bcd472p-4f), + V4 (0x1.6408f8p-4f), V4 (-0x1.246f8p-4f), V4 (0x1.f0e514p-5f), + V4 (-0x1.0fc92cp-4f), V4 (0x1.f5f76ap-5f) }, + .ln2 = V4 (0x1.62e43p-1f), + .inv_ln10 = V4 (0x1.bcb7b2p-2f), + .min_norm = V4 (0x00800000), + .special_bound = V4 (0x7f000000), /* asuint32(inf) - min_norm. */ + .off = V4 (0x3f2aaaab), /* 0.666667. */ + .mantissa_mask = V4 (0x007fffff), +}; + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* Fall back to scalar code. */ + return v_call_f32 (log10f, x, y, cmp); +} + +/* Fast implementation of AdvSIMD log10f, + uses a similar approach as AdvSIMD logf with the same offset (i.e., 2/3) and + an order 9 polynomial. + Maximum error: 3.305ulps (nearest rounding.) + _ZGVnN4v_log10f(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +float32x4_t VPCS_ATTR V_NAME_F1 (log10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); + uint32x4_t u = vreinterpretq_u32_f32 (x); + uint32x4_t special + = vcgeq_u32 (vsubq_u32 (u, d->min_norm), d->special_bound); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + u = vsubq_u32 (u, d->off); + float32x4_t n = vcvtq_f32_s32 ( + vshrq_n_s32 (vreinterpretq_s32_u32 (u), 23)); /* signextend. */ + u = vaddq_u32 (vandq_u32 (u, d->mantissa_mask), d->off); + float32x4_t r = vsubq_f32 (vreinterpretq_f32_u32 (u), v_f32 (1.0f)); + + /* y = log10(1+r) + n * log10(2). */ + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly = v_pw_horner_7_f32 (r, r2, d->poly); + /* y = Log10(2) * n + poly * InvLn(10). */ + float32x4_t y = vfmaq_f32 (r, d->ln2, n); + y = vfmaq_f32 (vmulq_f32 (y, d->inv_ln10), poly, r2); + + if (__glibc_unlikely (v_any_u32 (special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/log10f_sve.c b/sysdeps/aarch64/fpu/log10f_sve.c new file mode 100644 index 0000000000..22c63bb2fa --- /dev/null +++ b/sysdeps/aarch64/fpu/log10f_sve.c @@ -0,0 +1,94 @@ +/* Single-precision vector (SVE) log10 function + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" + +static const struct data +{ + float poly_0246[4]; + float poly_1357[4]; + float ln2, inv_ln10; +} data = { + .poly_1357 = { + /* Coefficients copied from the AdvSIMD routine, then rearranged so that coeffs + 1, 3, 5 and 7 can be loaded as a single quad-word, hence used with _lane + variant of MLA intrinsic. */ + 0x1.2879c8p-3f, 0x1.6408f8p-4f, 0x1.f0e514p-5f, 0x1.f5f76ap-5f + }, + .poly_0246 = { -0x1.bcb79cp-3f, -0x1.bcd472p-4f, -0x1.246f8p-4f, + -0x1.0fc92cp-4f }, + .ln2 = 0x1.62e43p-1f, + .inv_ln10 = 0x1.bcb7b2p-2f, +}; + +#define Min 0x00800000 +#define Max 0x7f800000 +#define Thres 0x7f000000 /* Max - Min. */ +#define Offset 0x3f2aaaab /* 0.666667. */ +#define MantissaMask 0x007fffff + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (log10f, x, y, special); +} + +/* Optimised implementation of SVE log10f using the same algorithm and + polynomial as AdvSIMD log10f. + Maximum error is 3.31ulps: + SV_NAME_F1 (log10)(0x1.555c16p+0) got 0x1.ffe2fap-4 + want 0x1.ffe2f4p-4. */ +svfloat32_t SV_NAME_F1 (log10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svuint32_t ix = svreinterpret_u32 (x); + svbool_t special = svcmpge (pg, svsub_x (pg, ix, Min), Thres); + + /* x = 2^n * (1+r), where 2/3 < 1+r < 4/3. */ + ix = svsub_x (pg, ix, Offset); + svfloat32_t n = svcvt_f32_x ( + pg, svasr_x (pg, svreinterpret_s32 (ix), 23)); /* signextend. */ + ix = svand_x (pg, ix, MantissaMask); + ix = svadd_x (pg, ix, Offset); + svfloat32_t r = svsub_x (pg, svreinterpret_f32 (ix), 1.0f); + + /* y = log10(1+r) + n*log10(2) + log10(1+r) ~ r * InvLn(10) + P(r) + where P(r) is a polynomial. Use order 9 for log10(1+x), i.e. order 8 for + log10(1+x)/x, with x in [-1/3, 1/3] (offset=2/3). */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t r4 = svmul_x (pg, r2, r2); + svfloat32_t p_1357 = svld1rq (pg, &d->poly_1357[0]); + svfloat32_t q_01 = svmla_lane (sv_f32 (d->poly_0246[0]), r, p_1357, 0); + svfloat32_t q_23 = svmla_lane (sv_f32 (d->poly_0246[1]), r, p_1357, 1); + svfloat32_t q_45 = svmla_lane (sv_f32 (d->poly_0246[2]), r, p_1357, 2); + svfloat32_t q_67 = svmla_lane (sv_f32 (d->poly_0246[3]), r, p_1357, 3); + svfloat32_t q_47 = svmla_x (pg, q_45, r2, q_67); + svfloat32_t q_03 = svmla_x (pg, q_01, r2, q_23); + svfloat32_t y = svmla_x (pg, q_03, r4, q_47); + + /* Using hi = Log10(2)*n + r*InvLn(10) is faster but less accurate. */ + svfloat32_t hi = svmla_x (pg, r, n, d->ln2); + hi = svmul_x (pg, hi, d->inv_ln10); + y = svmla_x (pg, hi, r2, y); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, y, special); + return y; +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index d30dcd6f95..8d05498ec9 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) +VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) VPCS_VECTOR_WRAPPER (log2_advsimd, _ZGVnN2v_log2) VPCS_VECTOR_WRAPPER (sin_advsimd, _ZGVnN2v_sin) VPCS_VECTOR_WRAPPER (tan_advsimd, _ZGVnN2v_tan) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index 22a8479100..b65bc6f1e6 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) +SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) SVE_VECTOR_WRAPPER (log2_sve, _ZGVsMxv_log2) SVE_VECTOR_WRAPPER (sin_sve, _ZGVsMxv_sin) SVE_VECTOR_WRAPPER (tan_sve, _ZGVsMxv_tan) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index e8f7f47c67..6ced0d4488 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -27,6 +27,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) +VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) VPCS_VECTOR_WRAPPER (log2f_advsimd, _ZGVnN4v_log2f) VPCS_VECTOR_WRAPPER (sinf_advsimd, _ZGVnN4v_sinf) VPCS_VECTOR_WRAPPER (tanf_advsimd, _ZGVnN4v_tanf) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index f5e9584265..2ed8d0659a 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -36,6 +36,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) +SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) SVE_VECTOR_WRAPPER (log2f_sve, _ZGVsMxv_log2f) SVE_VECTOR_WRAPPER (sinf_sve, _ZGVsMxv_sinf) SVE_VECTOR_WRAPPER (tanf_sve, _ZGVsMxv_tanf) diff --git a/sysdeps/aarch64/fpu/v_log10_data.c b/sysdeps/aarch64/fpu/v_log10_data.c new file mode 100644 index 0000000000..b0b07bafe7 --- /dev/null +++ b/sysdeps/aarch64/fpu/v_log10_data.c @@ -0,0 +1,175 @@ +/* Lookup table for double-precision log10(x) vector function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "vecmath_config.h" + +const struct v_log10_data __v_log10_data = { + /* Computed from log's coefficients div by log(10) then rounded to double + precision. */ + .poly = { -0x1.bcb7b1526e506p-3, 0x1.287a7636be1d1p-3, -0x1.bcb7b158af938p-4, + 0x1.63c78734e6d07p-4, -0x1.287461742fee4p-4 }, + .invln10 = 0x1.bcb7b1526e50ep-2, + .log10_2 = 0x1.34413509f79ffp-2, + /* Algorithm: + + x = 2^k z + log10(x) = k log10(2) + log10(c) + poly(z/c - 1) / log(10) + + where z is in [a;2a) which is split into N subintervals (a=0x1.69009p-1, + N=128) and log(c) and 1/c for the ith subinterval comes from lookup + tables: + + invc[i] = 1/c + log10c[i] = (double)log10(c) + + where c is near the center of the subinterval and is chosen by trying + several floating point invc candidates around 1/center and selecting one + for which the error in (double)log(c) is minimized (< 0x1p-74), except the + subinterval that contains 1 and the previous one got tweaked to avoid + cancellation. NB: invc should be optimized to minimize error in + (double)log10(c) instead. */ + .invc = { 0x1.6a133d0dec120p+0, 0x1.6815f2f3e42edp+0, + 0x1.661e39be1ac9ep+0, 0x1.642bfa30ac371p+0, + 0x1.623f1d916f323p+0, 0x1.60578da220f65p+0, + 0x1.5e75349dea571p+0, 0x1.5c97fd387a75ap+0, + 0x1.5abfd2981f200p+0, 0x1.58eca051dc99cp+0, + 0x1.571e526d9df12p+0, 0x1.5554d555b3fcbp+0, + 0x1.539015e2a20cdp+0, 0x1.51d0014ee0164p+0, + 0x1.50148538cd9eep+0, 0x1.4e5d8f9f698a1p+0, + 0x1.4cab0edca66bep+0, 0x1.4afcf1a9db874p+0, + 0x1.495327136e16fp+0, 0x1.47ad9e84af28fp+0, + 0x1.460c47b39ae15p+0, 0x1.446f12b278001p+0, + 0x1.42d5efdd720ecp+0, 0x1.4140cfe001a0fp+0, + 0x1.3fafa3b421f69p+0, 0x1.3e225c9c8ece5p+0, + 0x1.3c98ec29a211ap+0, 0x1.3b13442a413fep+0, + 0x1.399156baa3c54p+0, 0x1.38131639b4cdbp+0, + 0x1.36987540fbf53p+0, 0x1.352166b648f61p+0, + 0x1.33adddb3eb575p+0, 0x1.323dcd99fc1d3p+0, + 0x1.30d129fefc7d2p+0, 0x1.2f67e6b72fe7dp+0, + 0x1.2e01f7cf8b187p+0, 0x1.2c9f518ddc86ep+0, + 0x1.2b3fe86e5f413p+0, 0x1.29e3b1211b25cp+0, + 0x1.288aa08b373cfp+0, 0x1.2734abcaa8467p+0, + 0x1.25e1c82459b81p+0, 0x1.2491eb1ad59c5p+0, + 0x1.23450a54048b5p+0, 0x1.21fb1bb09e578p+0, + 0x1.20b415346d8f7p+0, 0x1.1f6fed179a1acp+0, + 0x1.1e2e99b93c7b3p+0, 0x1.1cf011a7a882ap+0, + 0x1.1bb44b97dba5ap+0, 0x1.1a7b3e66cdd4fp+0, + 0x1.1944e11dc56cdp+0, 0x1.18112aebb1a6ep+0, + 0x1.16e013231b7e9p+0, 0x1.15b1913f156cfp+0, + 0x1.14859cdedde13p+0, 0x1.135c2dc68cfa4p+0, + 0x1.12353bdb01684p+0, 0x1.1110bf25b85b4p+0, + 0x1.0feeafd2f8577p+0, 0x1.0ecf062c51c3bp+0, + 0x1.0db1baa076c8bp+0, 0x1.0c96c5bb3048ep+0, + 0x1.0b7e20263e070p+0, 0x1.0a67c2acd0ce3p+0, + 0x1.0953a6391e982p+0, 0x1.0841c3caea380p+0, + 0x1.07321489b13eap+0, 0x1.062491aee9904p+0, + 0x1.05193497a7cc5p+0, 0x1.040ff6b5f5e9fp+0, + 0x1.0308d19aa6127p+0, 0x1.0203beedb0c67p+0, + 0x1.010037d38bcc2p+0, 1.0, + 0x1.fc06d493cca10p-1, 0x1.f81e6ac3b918fp-1, + 0x1.f44546ef18996p-1, 0x1.f07b10382c84bp-1, + 0x1.ecbf7070e59d4p-1, 0x1.e91213f715939p-1, + 0x1.e572a9a75f7b7p-1, 0x1.e1e0e2c530207p-1, + 0x1.de5c72d8a8be3p-1, 0x1.dae50fa5658ccp-1, + 0x1.d77a71145a2dap-1, 0x1.d41c51166623ep-1, + 0x1.d0ca6ba0bb29fp-1, 0x1.cd847e8e59681p-1, + 0x1.ca4a499693e00p-1, 0x1.c71b8e399e821p-1, + 0x1.c3f80faf19077p-1, 0x1.c0df92dc2b0ecp-1, + 0x1.bdd1de3cbb542p-1, 0x1.baceb9e1007a3p-1, + 0x1.b7d5ef543e55ep-1, 0x1.b4e749977d953p-1, + 0x1.b20295155478ep-1, 0x1.af279f8e82be2p-1, + 0x1.ac5638197fdf3p-1, 0x1.a98e2f102e087p-1, + 0x1.a6cf5606d05c1p-1, 0x1.a4197fc04d746p-1, + 0x1.a16c80293dc01p-1, 0x1.9ec82c4dc5bc9p-1, + 0x1.9c2c5a491f534p-1, 0x1.9998e1480b618p-1, + 0x1.970d9977c6c2dp-1, 0x1.948a5c023d212p-1, + 0x1.920f0303d6809p-1, 0x1.8f9b698a98b45p-1, + 0x1.8d2f6b81726f6p-1, 0x1.8acae5bb55badp-1, + 0x1.886db5d9275b8p-1, 0x1.8617ba567c13cp-1, + 0x1.83c8d27487800p-1, 0x1.8180de3c5dbe7p-1, + 0x1.7f3fbe71cdb71p-1, 0x1.7d055498071c1p-1, + 0x1.7ad182e54f65ap-1, 0x1.78a42c3c90125p-1, + 0x1.767d342f76944p-1, 0x1.745c7ef26b00ap-1, + 0x1.7241f15769d0fp-1, 0x1.702d70d396e41p-1, + 0x1.6e1ee3700cd11p-1, 0x1.6c162fc9cbe02p-1 }, + .log10c = { -0x1.345825f221684p-3, -0x1.2f71a1f0c554ep-3, + -0x1.2a91fdb30b1f4p-3, -0x1.25b9260981a04p-3, + -0x1.20e7081762193p-3, -0x1.1c1b914aeefacp-3, + -0x1.1756af5de404dp-3, -0x1.12985059c90bfp-3, + -0x1.0de0628f63df4p-3, -0x1.092ed492e08eep-3, + -0x1.0483954caf1dfp-3, -0x1.ffbd27a9adbcp-4, + -0x1.f67f7f2e3d1ap-4, -0x1.ed4e1071ceebep-4, + -0x1.e428bb47413c4p-4, -0x1.db0f6003028d6p-4, + -0x1.d201df6749831p-4, -0x1.c9001ac5c9672p-4, + -0x1.c009f3c78c79p-4, -0x1.b71f4cb642e53p-4, + -0x1.ae400818526b2p-4, -0x1.a56c091954f87p-4, + -0x1.9ca3332f096eep-4, -0x1.93e56a3f23e55p-4, + -0x1.8b3292a3903bp-4, -0x1.828a9112d9618p-4, + -0x1.79ed4ac35f5acp-4, -0x1.715aa51ed28c4p-4, + -0x1.68d2861c999e9p-4, -0x1.6054d40ded21p-4, + -0x1.57e17576bc9a2p-4, -0x1.4f7851798bb0bp-4, + -0x1.47194f5690ae3p-4, -0x1.3ec456d58ec47p-4, + -0x1.36794ff3e5f55p-4, -0x1.2e382315725e4p-4, + -0x1.2600b8ed82e91p-4, -0x1.1dd2fa85efc12p-4, + -0x1.15aed136e3961p-4, -0x1.0d94269d1a30dp-4, + -0x1.0582e4a7659f5p-4, -0x1.faf5eb655742dp-5, + -0x1.eaf888487e8eep-5, -0x1.db0d75ef25a82p-5, + -0x1.cb348a49e6431p-5, -0x1.bb6d9c69acdd8p-5, + -0x1.abb88368aa7ap-5, -0x1.9c1517476af14p-5, + -0x1.8c833051bfa4dp-5, -0x1.7d02a78e7fb31p-5, + -0x1.6d93565e97c5fp-5, -0x1.5e351695db0c5p-5, + -0x1.4ee7c2ba67adcp-5, -0x1.3fab35ba16c01p-5, + -0x1.307f4ad854bc9p-5, -0x1.2163ddf4f988cp-5, + -0x1.1258cb5d19e22p-5, -0x1.035defdba3188p-5, + -0x1.e8e651191bce4p-6, -0x1.cb30a62be444cp-6, + -0x1.ad9a9b3043823p-6, -0x1.9023ecda1ccdep-6, + -0x1.72cc592bd82dp-6, -0x1.55939eb1f9c6ep-6, + -0x1.38797ca6cc5ap-6, -0x1.1b7db35c2c072p-6, + -0x1.fd400812ee9a2p-7, -0x1.c3c05fb4620f1p-7, + -0x1.8a7bf3c40e2e3p-7, -0x1.517249c15a75cp-7, + -0x1.18a2ea5330c91p-7, -0x1.c01abc8cdc4e2p-8, + -0x1.4f6261750dec9p-8, -0x1.be37b6612afa7p-9, + -0x1.bc3a8398ac26p-10, 0.0, + 0x1.bb796219f30a5p-9, 0x1.b984fdcba61cep-8, + 0x1.49cf12adf8e8cp-7, 0x1.b6075b5217083p-7, + 0x1.10b7466fc30ddp-6, 0x1.4603e4db6a3a1p-6, + 0x1.7aeb10e99e105p-6, 0x1.af6e49b0f0e36p-6, + 0x1.e38f064f41179p-6, 0x1.0ba75abbb7623p-5, + 0x1.25575ee2dba86p-5, 0x1.3ed83f477f946p-5, + 0x1.582aa79af60efp-5, 0x1.714f400fa83aep-5, + 0x1.8a46ad3901cb9p-5, 0x1.a311903b6b87p-5, + 0x1.bbb086f216911p-5, 0x1.d4242bdda648ep-5, + 0x1.ec6d167c2af1p-5, 0x1.0245ed8221426p-4, + 0x1.0e40856c74f64p-4, 0x1.1a269a31120fep-4, + 0x1.25f8718fc076cp-4, 0x1.31b64ffc95bfp-4, + 0x1.3d60787ca5063p-4, 0x1.48f72ccd187fdp-4, + 0x1.547aad6602f1cp-4, 0x1.5feb3989d3acbp-4, + 0x1.6b490f3978c79p-4, 0x1.76946b3f5e703p-4, + 0x1.81cd895717c83p-4, 0x1.8cf4a4055c30ep-4, + 0x1.9809f4c48c0ebp-4, 0x1.a30db3f9899efp-4, + 0x1.ae001905458fcp-4, 0x1.b8e15a2e3a2cdp-4, + 0x1.c3b1ace2b0996p-4, 0x1.ce71456edfa62p-4, + 0x1.d9205759882c4p-4, 0x1.e3bf1513af0dfp-4, + 0x1.ee4db0412c414p-4, 0x1.f8cc5998de3a5p-4, + 0x1.019da085eaeb1p-3, 0x1.06cd4acdb4e3dp-3, + 0x1.0bf542bef813fp-3, 0x1.11159f14da262p-3, + 0x1.162e761c10d1cp-3, 0x1.1b3fddc60d43ep-3, + 0x1.2049ebac86aa6p-3, 0x1.254cb4fb7836ap-3, + 0x1.2a484e8d0d252p-3, 0x1.2f3ccce1c860bp-3 } +}; diff --git a/sysdeps/aarch64/fpu/vecmath_config.h b/sysdeps/aarch64/fpu/vecmath_config.h index 82a68fd35d..4eb7b587b3 100644 --- a/sysdeps/aarch64/fpu/vecmath_config.h +++ b/sysdeps/aarch64/fpu/vecmath_config.h @@ -58,4 +58,13 @@ extern const struct v_log2_data double log2c[1 << V_LOG2_TABLE_BITS]; } __v_log2_data attribute_hidden; +#define V_LOG10_TABLE_BITS 7 +extern const struct v_log10_data +{ + double poly[5]; + double invln10, log10_2; + double invc[1 << V_LOG10_TABLE_BITS]; + double log10c[1 << V_LOG10_TABLE_BITS]; +} __v_log10_data attribute_hidden; + #endif diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 7a5af571e2..6641c7fa0b 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -1180,11 +1180,19 @@ double: 2 float: 2 ldouble: 2 +Function: "log10_advsimd": +double: 1 +float: 2 + Function: "log10_downward": double: 2 float: 3 ldouble: 1 +Function: "log10_sve": +double: 1 +float: 2 + Function: "log10_towardzero": double: 2 float: 2 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index 376dcdd6aa..a92585bf8b 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -18,6 +18,10 @@ GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVsMxv_exp2 F GLIBC_2.39 _ZGVsMxv_exp2f F +GLIBC_2.39 _ZGVnN2v_log10 F +GLIBC_2.39 _ZGVnN4v_log10f F +GLIBC_2.39 _ZGVsMxv_log10 F +GLIBC_2.39 _ZGVsMxv_log10f F GLIBC_2.39 _ZGVnN2v_log2 F GLIBC_2.39 _ZGVnN4v_log2f F GLIBC_2.39 _ZGVsMxv_log2 F From patchwork Thu Aug 3 12:27:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joe Ramsay X-Patchwork-Id: 1816425 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=vMOzZUcy; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RGp5p3xKSz1yZl for ; Thu, 3 Aug 2023 22:27:58 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 431063857717 for ; Thu, 3 Aug 2023 12:27:56 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 431063857717 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1691065676; bh=sNanniTyzBeaj02Rwb+nDniQ0Z0wDWSVMcAbL4JXcUI=; h=To:CC:Subject:Date:In-Reply-To:References:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=vMOzZUcyuC1b60Xiy0E9d3QvG2q8snx+PDNG0Gy5JsmKVIafJcYbJkfYYl0JW6hro g9z3M99XD5I8T6L5RRLrkRgNp6N5DVG+OHvfAuo4zlnRlOlQsZ6UEa7MTSJFDhh3hc Ennbibt88CAW51H4C2UUx42tDIEeEdjeMFn5Hyv8= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2069.outbound.protection.outlook.com [40.107.20.69]) by sourceware.org (Postfix) with ESMTPS id 51BDC3858D1E for ; Thu, 3 Aug 2023 12:27:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 51BDC3858D1E Received: from AM4PR0302CA0032.eurprd03.prod.outlook.com (2603:10a6:205:2::45) by DU0PR08MB8931.eurprd08.prod.outlook.com (2603:10a6:10:466::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:31 +0000 Received: from AM7EUR03FT065.eop-EUR03.prod.protection.outlook.com (2603:10a6:205:2:cafe::d) by AM4PR0302CA0032.outlook.office365.com (2603:10a6:205:2::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT065.mail.protection.outlook.com (100.127.140.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.45 via Frontend Transport; Thu, 3 Aug 2023 12:27:31 +0000 Received: ("Tessian outbound e1fdbe8a48d3:v145"); Thu, 03 Aug 2023 12:27:30 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 7462ee85b1a44999 X-CR-MTA-TID: 64aa7808 Received: from eba24d55b73f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id DF0DBDD8-17B3-4EBD-8798-325D81A6A21D.1; Thu, 03 Aug 2023 12:27:24 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id eba24d55b73f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 03 Aug 2023 12:27:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qj+2uU1ozZGa9fcfEyz2dLZoSwNNyFK5Ku/K+87Qf1BmiKi7lS0F1Ig4y0SvdoRhf6kTmrez4Nxrg2B2x8UrV+L5qVJBISHF+tPx9CqSA/k3lo0UTvBRjGyKMTuRSjB6SQ19BEru77Ts/q8Em7K7y7fo5Xo5rcBgE1wkIVEzeqlKkJEdVr0dz3CA6QXYqJn4SroJgEkAbaFISrRF1y832PC00Scy41A8UzYN7miSMVQVUzSoUHihPyCk8y8KIdIjESqCD/Ec00Aa5x6tXcYxV4UvKjf/fIiaW69L6PSAzuzS06r79NXS86BA6WI8z3hqAC80zliC31qMb3PqvZgQhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sNanniTyzBeaj02Rwb+nDniQ0Z0wDWSVMcAbL4JXcUI=; b=ckq4m0eo3+If7H45kHstiHCUKhcPmLiO4JUcVrl9OWKWTJoqeAdshQNHWOEcOmMvc4X1HMrtCG5owYLDvkvQ8hHyiPlP3VXU+y6nbkaiQN3kiPXk2oGcZ6bdKDq9ee+EBf0oODXIoMVi4FeE1RE15U9B8ZfaJw0fpzCOdjBNhNzB3jiPrkVQ128bZKR1oH+KDerPeKncdu0NsNePf/JfuO+bO4bQ78YbluwuhrhxtWg6Ly5Iybso+1xWTd/i0Ar2VBIoO2SC7K9UkiUis2P4skeEzpJPHFfmFQUOGH+4YSZ21BATxhX/Mha9UU28kehmp2BWq698WQa8mT0Et6tUvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 40.67.248.234) smtp.rcpttodomain=sourceware.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=none (message not signed); arc=none Received: from DB7PR02CA0019.eurprd02.prod.outlook.com (2603:10a6:10:52::32) by DBBPR08MB6313.eurprd08.prod.outlook.com (2603:10a6:10:202::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.47; Thu, 3 Aug 2023 12:27:21 +0000 Received: from DBAEUR03FT049.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:52:cafe::23) by DB7PR02CA0019.outlook.office365.com (2603:10a6:10:52::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.20 via Frontend Transport; Thu, 3 Aug 2023 12:27:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 40.67.248.234) smtp.mailfrom=arm.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 40.67.248.234 as permitted sender) receiver=protection.outlook.com; client-ip=40.67.248.234; helo=nebula.arm.com; pr=C Received: from nebula.arm.com (40.67.248.234) by DBAEUR03FT049.mail.protection.outlook.com (100.127.142.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6631.47 via Frontend Transport; Thu, 3 Aug 2023 12:27:21 +0000 Received: from AZ-NEU-EX04.Arm.com (10.251.24.32) by AZ-NEU-EX04.Arm.com (10.251.24.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 3 Aug 2023 12:27:19 +0000 Received: from vcn-man-apps.manchester.arm.com (10.32.108.22) by mail.arm.com (10.251.24.32) with Microsoft SMTP Server id 15.1.2507.27 via Frontend Transport; Thu, 3 Aug 2023 12:27:19 +0000 To: CC: Joe Ramsay Subject: [PATCH 5/5] aarch64: Add vector implementations of exp10 routines Date: Thu, 3 Aug 2023 13:27:17 +0100 Message-ID: <20230803122717.33413-5-Joe.Ramsay@arm.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20230803122717.33413-1-Joe.Ramsay@arm.com> References: <20230803122717.33413-1-Joe.Ramsay@arm.com> MIME-Version: 1.0 X-EOPAttributedMessage: 1 X-MS-TrafficTypeDiagnostic: DBAEUR03FT049:EE_|DBBPR08MB6313:EE_|AM7EUR03FT065:EE_|DU0PR08MB8931:EE_ X-MS-Office365-Filtering-Correlation-Id: 53b0015d-e49c-414b-73de-08db941d01ff x-checkrecipientrouted: true NoDisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: t07LupkSy0OwSZHYpKZxtFGArPQpd4jCf89N6TcaaJ15zMAm0u1mal/fkSiln514hO1EPJVMv6P8dDyZfg/6Lng0awpqqbNjVEqOGMcnbr+n6MBeGIS8pYjlSCiUkSHjTXucK75xC1sHvRirx5udzpjv/U3Tc/zqitlfyLDKb7aZJumGgfT8fxl5qzRXhwUJ5sAiqTH/jm6T6pfwchs9sBVy5zOmXjkRrVBgcnd8w0BCQ8nU/30A6NtlFfbjs+ND2d6B87/K9maAJy2mxlgW39qOfRkkFF8rzvhqrIOwiUR+K+j6Lkoc3P+sTobWstxp7oJn9kfufMMnn6iv0up5oTYNm7GDbkO/iQf7QIWgp6J0wCgJhKy0IlqeLHATuZggppHH8+gVYrVx5VM23bNcn8V2OtGzflNvwusNn65TXIetP/MOCSP/xIzTVT028C74pwIs36jYedZYlNGhz6uKl6paFrPLxXOLjsGmpAHeCw5tk2aiwdwGY5A5pOeYRrKYBQi2CRHOrmniO9dyPaZwQaCvIMI3gLX2w99KZLC+gTREoOs7v15NzLNOyqJAfXTyiJ+r8xq4p0Nq01qtzWaBI/z3G5Eh7Zga6rPl7wXDM/8QhYmvM1V7bpy04Dsp6PfOZyIlB2me2W0VJW5DVY6ij1mBjyto8czyWT0G6gUNK2Q98IIlXLUo3nODEU0Us0Thk568bZCKvftuhkCzmfYCYoboSseCwSLpIC0MEtpKNd5uQnNpTQzytYjp/fTLYxvZSduZC+HtVq0ugxePrbGYoDabFH9qK2ocTtH5y7sPHhj6AeDLyd5+KT8x3R3YIR0O X-Forefront-Antispam-Report-Untrusted: CIP:40.67.248.234; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:nebula.arm.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230028)(4636009)(376002)(136003)(39860400002)(346002)(396003)(451199021)(82310400008)(36840700001)(46966006)(40470700004)(40460700003)(426003)(2616005)(1076003)(83380400001)(26005)(186003)(336012)(47076005)(36860700001)(316002)(2906002)(70586007)(70206006)(4326008)(5660300002)(6916009)(30864003)(41300700001)(8676002)(8936002)(7696005)(478600001)(40480700001)(356005)(81166007)(86362001)(36756003)(82740400003)(2004002)(36900700001)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB6313 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT065.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: f9e8cc41-4876-496c-fe7c-08db941cfc58 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Q0qkCF/tIQLA4u4xeuM50BnPtieeiuyoOaBfmMRPwpbEcH2komWgKr4n7LOVO0CxR9O5NSIcvifZWOIgPs7HQvJx3/bfzpvqD35bxKFeefDNR7s4ofgVEbL7PhduuCF4BI3zhsaHRDUX0vnhm/fJXCsuY35DnzvS5BPcIDs0sC7PS1Jl7KoTevRK0HIMFSyqzJbjyd47JbFBTdFrIE3hF22E9F0yeLqxydFFYSbabHs8dW6CtSGDil24wjWjAebxIx4dd5iehdIK3FVtokPiZaekU3NyUwje+PuGlqoSDDrJDpVLQdxxSAxc8GWPMbyuRRQVsAUB2QG6ZWDS5MLzjxrZWsCP0vtV61ml+rKkpDAOfMrpIMn+zcUNpk16otgrFweYtysQ29sxSiWEFFbRujWJPdVI8CftcQxCW0GpDp8Eco2DGdafwcKdllVQJx4kPFMjHCZ2DD/teWu2KBcICsLtOJXEHoCPa2Dhjr1xG35bYaAN7BVxK4Eq3LNloOP1H8dEgKl1k6ausQmqtOWnMjP2X+kU/zrvfuhCp5u5uEJG1cSDNrGaWU1bbi0b8B24F1/oo4v6XUWEOfBdQ8mJSXr9I6WdAWDL7nk/3NWA7ptztNpVj9rGjVy8ak32FWvCbqZLzEZmQOj8lk2ZcB44J/ZV1RzZHk2UgcTjjHs9o4Ldcw/IfS6+84GcPdmkk/EGCHlaeiztF4LhzpL2etnJcoVUcjJZiiz9GW68U6YDt4QE7CyQXBJ8Gd5JUhKZNtA5jmtvRX4ggai7B1o1J8KnDg== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(396003)(376002)(346002)(136003)(39860400002)(451199021)(82310400008)(40470700004)(36840700001)(46966006)(30864003)(41300700001)(40460700003)(316002)(2906002)(4326008)(70206006)(70586007)(6916009)(8936002)(8676002)(5660300002)(81166007)(82740400003)(26005)(1076003)(36756003)(186003)(86362001)(83380400001)(2616005)(426003)(336012)(47076005)(36860700001)(478600001)(7696005)(40480700001)(2004002)(473944003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Aug 2023 12:27:31.0913 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 53b0015d-e49c-414b-73de-08db941d01ff X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT065.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB8931 X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, FORGED_SPF_HELO, GIT_PATCH_0, KAM_DMARC_NONE, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Joe Ramsay via Libc-alpha From: Joe Ramsay Reply-To: Joe Ramsay Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" Double-precision routines either reuse the exp table (AdvSIMD) or use SVE FEXPA intruction. --- Thanks, Joe sysdeps/aarch64/fpu/Makefile | 1 + sysdeps/aarch64/fpu/Versions | 4 + sysdeps/aarch64/fpu/bits/math-vector.h | 4 + sysdeps/aarch64/fpu/exp10_advsimd.c | 145 ++++++++++++++++++ sysdeps/aarch64/fpu/exp10_sve.c | 127 +++++++++++++++ sysdeps/aarch64/fpu/exp10f_advsimd.c | 140 +++++++++++++++++ sysdeps/aarch64/fpu/exp10f_sve.c | 91 +++++++++++ .../fpu/test-double-advsimd-wrappers.c | 1 + .../aarch64/fpu/test-double-sve-wrappers.c | 1 + .../aarch64/fpu/test-float-advsimd-wrappers.c | 1 + sysdeps/aarch64/fpu/test-float-sve-wrappers.c | 1 + sysdeps/aarch64/libm-test-ulps | 8 + .../unix/sysv/linux/aarch64/libmvec.abilist | 4 + 13 files changed, 528 insertions(+) create mode 100644 sysdeps/aarch64/fpu/exp10_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10_sve.c create mode 100644 sysdeps/aarch64/fpu/exp10f_advsimd.c create mode 100644 sysdeps/aarch64/fpu/exp10f_sve.c diff --git a/sysdeps/aarch64/fpu/Makefile b/sysdeps/aarch64/fpu/Makefile index 0a047a150d..1f1ac2a2b8 100644 --- a/sysdeps/aarch64/fpu/Makefile +++ b/sysdeps/aarch64/fpu/Makefile @@ -1,5 +1,6 @@ libmvec-supported-funcs = cos \ exp \ + exp10 \ exp2 \ log \ log10 \ diff --git a/sysdeps/aarch64/fpu/Versions b/sysdeps/aarch64/fpu/Versions index 358efed5ee..eb5ad50017 100644 --- a/sysdeps/aarch64/fpu/Versions +++ b/sysdeps/aarch64/fpu/Versions @@ -18,6 +18,10 @@ libmvec { _ZGVsMxv_sinf; } GLIBC_2.39 { + _ZGVnN4v_exp10f; + _ZGVnN2v_exp10; + _ZGVsMxv_exp10f; + _ZGVsMxv_exp10; _ZGVnN4v_exp2f; _ZGVnN2v_exp2; _ZGVsMxv_exp2f; diff --git a/sysdeps/aarch64/fpu/bits/math-vector.h b/sysdeps/aarch64/fpu/bits/math-vector.h index 59f2efa6d7..06587ffa91 100644 --- a/sysdeps/aarch64/fpu/bits/math-vector.h +++ b/sysdeps/aarch64/fpu/bits/math-vector.h @@ -51,6 +51,7 @@ typedef __SVBool_t __sv_bool_t; __vpcs __f32x4_t _ZGVnN4v_cosf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_expf (__f32x4_t); +__vpcs __f32x4_t _ZGVnN4v_exp10f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_exp2f (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_logf (__f32x4_t); __vpcs __f32x4_t _ZGVnN4v_log10f (__f32x4_t); @@ -60,6 +61,7 @@ __vpcs __f32x4_t _ZGVnN4v_tanf (__f32x4_t); __vpcs __f64x2_t _ZGVnN2v_cos (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp (__f64x2_t); +__vpcs __f64x2_t _ZGVnN2v_exp10 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_exp2 (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log (__f64x2_t); __vpcs __f64x2_t _ZGVnN2v_log10 (__f64x2_t); @@ -74,6 +76,7 @@ __vpcs __f64x2_t _ZGVnN2v_tan (__f64x2_t); __sv_f32_t _ZGVsMxv_cosf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_expf (__sv_f32_t, __sv_bool_t); +__sv_f32_t _ZGVsMxv_exp10f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_exp2f (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_logf (__sv_f32_t, __sv_bool_t); __sv_f32_t _ZGVsMxv_log10f (__sv_f32_t, __sv_bool_t); @@ -83,6 +86,7 @@ __sv_f32_t _ZGVsMxv_tanf (__sv_f32_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_cos (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp (__sv_f64_t, __sv_bool_t); +__sv_f64_t _ZGVsMxv_exp10 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_exp2 (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log (__sv_f64_t, __sv_bool_t); __sv_f64_t _ZGVsMxv_log10 (__sv_f64_t, __sv_bool_t); diff --git a/sysdeps/aarch64/fpu/exp10_advsimd.c b/sysdeps/aarch64/fpu/exp10_advsimd.c new file mode 100644 index 0000000000..c584e2a270 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_advsimd.c @@ -0,0 +1,145 @@ +/* Double-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" + +/* Value of |x| above which scale overflows without special treatment. */ +#define SpecialBound 306.0 /* floor (log10 (2^1023)) - 1. */ +/* Value of n above which scale overflows even with special treatment. */ +#define ScaleBound 163840.0 /* 1280.0 * N. */ + +const static struct data +{ + float64x2_t poly[4]; + float64x2_t log10_2, log2_10_hi, log2_10_lo, shift; +#if !WANT_SIMD_EXCEPT + float64x2_t special_bound, scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.5ddf8f28p-54 + abs error: 0x1.5ed266c8p-54 in [ -log10(2)/256, log10(2)/256 ] + maxerr: 1.14432 +0.5 ulp. */ + .poly = { V2 (0x1.26bb1bbb5524p1), V2 (0x1.53524c73cecdap1), + V2 (0x1.047060efb781cp1), V2 (0x1.2bd76040f0d16p0) }, + .log10_2 = V2 (0x1.a934f0979a371p8), /* N/log2(10). */ + .log2_10_hi = V2 (0x1.34413509f79ffp-9), /* log2(10)/N. */ + .log2_10_lo = V2 (-0x1.9dc1da994fd21p-66), + .shift = V2 (0x1.8p+52), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V2 (ScaleBound), + .special_bound = V2 (SpecialBound), +#endif +}; + +#define N (1 << V_EXP_TABLE_BITS) +#define IndexMask v_u64 (N - 1) + +#if WANT_SIMD_EXCEPT + +# define TinyBound v_u64 (0x2000000000000000) /* asuint64 (0x1p-511). */ +# define BigBound v_u64 (0x4070000000000000) /* asuint64 (0x1p8). */ +# define Thres v_u64 (0x2070000000000000) /* BigBound - TinyBound. */ + +static inline float64x2_t VPCS_ATTR +special_case (float64x2_t x, float64x2_t y, uint64x2_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine for special lanes. */ + return v_call_f64 (exp10, x, y, cmp); +} + +#else + +# define SpecialOffset v_u64 (0x6000000000000000) /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +# define SpecialBias1 v_u64 (0x7000000000000000) /* 0x1p769. */ +# define SpecialBias2 v_u64 (0x3010000000000000) /* 0x1p-254. */ + +static float64x2_t VPCS_ATTR NOINLINE +special_case (float64x2_t s, float64x2_t y, float64x2_t n, + const struct data *d) +{ + /* 2^(n/N) may overflow, break it up into s1*s2. */ + uint64x2_t b = vandq_u64 (vcltzq_f64 (n), SpecialOffset); + float64x2_t s1 = vreinterpretq_f64_u64 (vsubq_u64 (SpecialBias1, b)); + float64x2_t s2 = vreinterpretq_f64_u64 ( + vaddq_u64 (vsubq_u64 (vreinterpretq_u64_f64 (s), SpecialBias2), b)); + uint64x2_t cmp = vcagtq_f64 (n, d->scale_thresh); + float64x2_t r1 = vmulq_f64 (s1, s1); + float64x2_t r0 = vmulq_f64 (vfmaq_f64 (s2, y, s2), s1); + return vbslq_f64 (cmp, r1, r0); +} + +#endif + +/* Fast vector implementation of exp10. + Maximum measured error is 1.64 ulp. + _ZGVnN2v_exp10(0x1.ccd1c9d82cc8cp+0) got 0x1.f8dab6d7fed0cp+5 + want 0x1.f8dab6d7fed0ap+5. */ +float64x2_t VPCS_ATTR V_NAME_D1 (exp10) (float64x2_t x) +{ + const struct data *d = ptr_barrier (&data); + uint64x2_t cmp; +#if WANT_SIMD_EXCEPT + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special_case to fix special lanes later. This is only necessary if fenv + exceptions are to be triggered correctly. */ + float64x2_t xm = x; + uint64x2_t iax = vreinterpretq_u64_f64 (vabsq_f64 (x)); + cmp = vcgeq_u64 (vsubq_u64 (iax, TinyBound), Thres); + if (__glibc_unlikely (v_any_u64 (cmp))) + x = vbslq_f64 (cmp, v_f64 (1), x); +#else + cmp = vcageq_f64 (x, d->special_bound); +#endif + + /* n = round(x/(log10(2)/N)). */ + float64x2_t z = vfmaq_f64 (d->shift, x, d->log10_2); + uint64x2_t u = vreinterpretq_u64_f64 (z); + float64x2_t n = vsubq_f64 (z, d->shift); + + /* r = x - n*log10(2)/N. */ + float64x2_t r = x; + r = vfmsq_f64 (r, d->log2_10_hi, n); + r = vfmsq_f64 (r, d->log2_10_lo, n); + + uint64x2_t e = vshlq_n_u64 (u, 52 - V_EXP_TABLE_BITS); + uint64x2_t i = vandq_u64 (u, IndexMask); + + /* y = exp10(r) - 1 ~= C0 r + C1 r^2 + C2 r^3 + C3 r^4. */ + float64x2_t r2 = vmulq_f64 (r, r); + float64x2_t p = vfmaq_f64 (d->poly[0], r, d->poly[1]); + float64x2_t y = vfmaq_f64 (d->poly[2], r, d->poly[3]); + p = vfmaq_f64 (p, y, r2); + y = vmulq_f64 (r, p); + + /* s = 2^(n/N). */ + u = v_lookup_u64 (__v_exp_data, i); + float64x2_t s = vreinterpretq_f64_u64 (vaddq_u64 (u, e)); + + if (__glibc_unlikely (v_any_u64 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f64 (s, y, s), cmp); +#else + return special_case (s, y, n, d); +#endif + + return vfmaq_f64 (s, y, s); +} diff --git a/sysdeps/aarch64/fpu/exp10_sve.c b/sysdeps/aarch64/fpu/exp10_sve.c new file mode 100644 index 0000000000..a8cef7b692 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10_sve.c @@ -0,0 +1,127 @@ +/* Double-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f64.h" + +#define SpecialBound 307.0 /* floor (log10 (2^1023)). */ + +static const struct data +{ + double poly[5]; + double shift, log10_2, log2_10_hi, log2_10_lo, scale_thres, special_bound; +} data = { + /* Coefficients generated using Remez algorithm. + rel error: 0x1.9fcb9b3p-60 + abs error: 0x1.a20d9598p-60 in [ -log10(2)/128, log10(2)/128 ] + max ulp err 0.52 +0.5. */ + .poly = { 0x1.26bb1bbb55516p1, 0x1.53524c73cd32ap1, 0x1.0470591daeafbp1, + 0x1.2bd77b1361ef6p0, 0x1.142b5d54e9621p-1 }, + /* 1.5*2^46+1023. This value is further explained below. */ + .shift = 0x1.800000000ffc0p+46, + .log10_2 = 0x1.a934f0979a371p1, /* 1/log2(10). */ + .log2_10_hi = 0x1.34413509f79ffp-2, /* log2(10). */ + .log2_10_lo = -0x1.9dc1da994fd21p-59, + .scale_thres = 1280.0, + .special_bound = SpecialBound, +}; + +#define SpecialOffset 0x6000000000000000 /* 0x1p513. */ +/* SpecialBias1 + SpecialBias1 = asuint(1.0). */ +#define SpecialBias1 0x7000000000000000 /* 0x1p769. */ +#define SpecialBias2 0x3010000000000000 /* 0x1p-254. */ + +/* Update of both special and non-special cases, if any special case is + detected. */ +static inline svfloat64_t +special_case (svbool_t pg, svfloat64_t s, svfloat64_t y, svfloat64_t n, + const struct data *d) +{ + /* s=2^n may overflow, break it up into s=s1*s2, + such that exp = s + s*y can be computed as s1*(s2+s2*y) + and s1*s1 overflows only if n>0. */ + + /* If n<=0 then set b to 0x6, 0 otherwise. */ + svbool_t p_sign = svcmple (pg, n, 0.0); /* n <= 0. */ + svuint64_t b = svdup_u64_z (p_sign, SpecialOffset); + + /* Set s1 to generate overflow depending on sign of exponent n. */ + svfloat64_t s1 = svreinterpret_f64 (svsubr_x (pg, b, SpecialBias1)); + /* Offset s to avoid overflow in final result if n is below threshold. */ + svfloat64_t s2 = svreinterpret_f64 ( + svadd_x (pg, svsub_x (pg, svreinterpret_u64 (s), SpecialBias2), b)); + + /* |n| > 1280 => 2^(n) overflows. */ + svbool_t p_cmp = svacgt (pg, n, d->scale_thres); + + svfloat64_t r1 = svmul_x (pg, s1, s1); + svfloat64_t r2 = svmla_x (pg, s2, s2, y); + svfloat64_t r0 = svmul_x (pg, r2, s1); + + return svsel (p_cmp, r1, r0); +} + +/* Fast vector implementation of exp10 using FEXPA instruction. + Maximum measured error is 1.02 ulp. + SV_NAME_D1 (exp10)(-0x1.2862fec805e58p+2) got 0x1.885a89551d782p-16 + want 0x1.885a89551d781p-16. */ +svfloat64_t SV_NAME_D1 (exp10) (svfloat64_t x, svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + svbool_t no_big_scale = svacle (pg, x, d->special_bound); + svbool_t special = svnot_z (pg, no_big_scale); + + /* n = round(x/(log10(2)/N)). */ + svfloat64_t shift = sv_f64 (d->shift); + svfloat64_t z = svmla_x (pg, shift, x, d->log10_2); + svfloat64_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat64_t log2_10 = svld1rq (svptrue_b64 (), &d->log2_10_hi); + svfloat64_t r = x; + r = svmls_lane (r, n, log2_10, 0); + r = svmls_lane (r, n, log2_10, 1); + + /* scale = 2^(n/N), computed using FEXPA. FEXPA does not propagate NaNs, so + for consistent NaN handling we have to manually propagate them. This + comes at significant performance cost. */ + svuint64_t u = svreinterpret_u64 (z); + svfloat64_t scale = svexpa (u); + + /* Approximate exp10(r) using polynomial. */ + svfloat64_t r2 = svmul_x (pg, r, r); + svfloat64_t y = svmla_x (pg, svmul_x (pg, r, d->poly[0]), r2, + sv_pairwise_poly_3_f64_x (pg, r, r2, d->poly + 1)); + + /* Assemble result as exp10(x) = 2^n * exp10(r). If |x| > SpecialBound + multiplication may overflow, so use special case routine. */ + if (__glibc_unlikely (svptest_any (pg, special))) + { + /* FEXPA zeroes the sign bit, however the sign is meaningful to the + special case function so needs to be copied. + e = sign bit of u << 46. */ + svuint64_t e = svand_x (pg, svlsl_x (pg, u, 46), 0x8000000000000000); + /* Copy sign to scale. */ + scale = svreinterpret_f64 (svadd_x (pg, e, svreinterpret_u64 (scale))); + return special_case (pg, scale, y, n, d); + } + + /* No special case. */ + return svmla_x (pg, scale, scale, y); +} diff --git a/sysdeps/aarch64/fpu/exp10f_advsimd.c b/sysdeps/aarch64/fpu/exp10f_advsimd.c new file mode 100644 index 0000000000..9e754c46fa --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_advsimd.c @@ -0,0 +1,140 @@ +/* Single-precision vector (AdvSIMD) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "v_math.h" +#include "poly_advsimd_f32.h" + +#define ScaleBound 192.0f + +static const struct data +{ + float32x4_t poly[5]; + float32x4_t shift, log10_2, log2_10_hi, log2_10_lo; +#if !WANT_SIMD_EXCEPT + float32x4_t scale_thresh; +#endif +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 1.85943 +0.5 ulp. */ + .poly = { V4 (0x1.26bb16p+1f), V4 (0x1.5350d2p+1f), V4 (0x1.04744ap+1f), + V4 (0x1.2d8176p+0f), V4 (0x1.12b41ap-1f) }, + .shift = V4 (0x1.8p23f), + .log10_2 = V4 (0x1.a934fp+1), + .log2_10_hi = V4 (0x1.344136p-2), + .log2_10_lo = V4 (-0x1.ec10cp-27), +#if !WANT_SIMD_EXCEPT + .scale_thresh = V4 (ScaleBound) +#endif +}; + +#define ExponentBias v_u32 (0x3f800000) + +#if WANT_SIMD_EXCEPT + +# define SpecialBound 38.0f /* rint(log10(2^127)). */ +# define TinyBound v_u32 (0x20000000) /* asuint (0x1p-63). */ +# define BigBound v_u32 (0x42180000) /* asuint (SpecialBound). */ +# define Thres v_u32 (0x22180000) /* BigBound - TinyBound. */ + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t x, float32x4_t y, uint32x4_t cmp) +{ + /* If fenv exceptions are to be triggered correctly, fall back to the scalar + routine to special lanes. */ + return v_call_f32 (exp10f, x, y, cmp); +} + +#else + +# define SpecialBound 126.0f /* rint (log2 (2^127 / (1 + sqrt (2)))). */ +# define SpecialOffset v_u32 (0x82000000) +# define SpecialBias v_u32 (0x7f000000) + +static float32x4_t VPCS_ATTR NOINLINE +special_case (float32x4_t poly, float32x4_t n, uint32x4_t e, uint32x4_t cmp1, + float32x4_t scale, const struct data *d) +{ + /* 2^n may overflow, break it up into s1*s2. */ + uint32x4_t b = vandq_u32 (vclezq_f32 (n), SpecialOffset); + float32x4_t s1 = vreinterpretq_f32_u32 (vaddq_u32 (b, SpecialBias)); + float32x4_t s2 = vreinterpretq_f32_u32 (vsubq_u32 (e, b)); + uint32x4_t cmp2 = vcagtq_f32 (n, d->scale_thresh); + float32x4_t r2 = vmulq_f32 (s1, s1); + float32x4_t r1 = vmulq_f32 (vfmaq_f32 (s2, poly, s2), s1); + /* Similar to r1 but avoids double rounding in the subnormal range. */ + float32x4_t r0 = vfmaq_f32 (scale, poly, scale); + float32x4_t r = vbslq_f32 (cmp1, r1, r0); + return vbslq_f32 (cmp2, r2, r); +} + +#endif + +/* Fast vector implementation of single-precision exp10. + Algorithm is accurate to 2.36 ULP. + _ZGVnN4v_exp10f(0x1.be2b36p+1) got 0x1.7e79c4p+11 + want 0x1.7e79cp+11. */ +float32x4_t VPCS_ATTR V_NAME_F1 (exp10) (float32x4_t x) +{ + const struct data *d = ptr_barrier (&data); +#if WANT_SIMD_EXCEPT + /* asuint(x) - TinyBound >= BigBound - TinyBound. */ + uint32x4_t cmp = vcgeq_u32 ( + vsubq_u32 (vandq_u32 (vreinterpretq_u32_f32 (x), v_u32 (0x7fffffff)), + TinyBound), + Thres); + float32x4_t xm = x; + /* If any lanes are special, mask them with 1 and retain a copy of x to allow + special case handler to fix special lanes later. This is only necessary if + fenv exceptions are to be triggered correctly. */ + if (__glibc_unlikely (v_any_u32 (cmp))) + x = vbslq_f32 (cmp, v_f32 (1), x); +#endif + + /* exp10(x) = 2^n * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10 (2), with r in [-log10(2)/2, log10(2)/2]. */ + float32x4_t z = vfmaq_f32 (d->shift, x, d->log10_2); + float32x4_t n = vsubq_f32 (z, d->shift); + float32x4_t r = vfmsq_f32 (x, n, d->log2_10_hi); + r = vfmsq_f32 (r, n, d->log2_10_lo); + uint32x4_t e = vshlq_n_u32 (vreinterpretq_u32_f32 (z), 23); + + float32x4_t scale = vreinterpretq_f32_u32 (vaddq_u32 (e, ExponentBias)); + +#if !WANT_SIMD_EXCEPT + uint32x4_t cmp = vcagtq_f32 (n, v_f32 (SpecialBound)); +#endif + + float32x4_t r2 = vmulq_f32 (r, r); + float32x4_t poly + = vfmaq_f32 (vmulq_f32 (r, d->poly[0]), + v_pairwise_poly_3_f32 (r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (v_any_u32 (cmp))) +#if WANT_SIMD_EXCEPT + return special_case (xm, vfmaq_f32 (scale, poly, scale), cmp); +#else + return special_case (poly, n, e, cmp, scale, d); +#endif + + return vfmaq_f32 (scale, poly, scale); +} diff --git a/sysdeps/aarch64/fpu/exp10f_sve.c b/sysdeps/aarch64/fpu/exp10f_sve.c new file mode 100644 index 0000000000..3df3246e05 --- /dev/null +++ b/sysdeps/aarch64/fpu/exp10f_sve.c @@ -0,0 +1,91 @@ +/* Single-precision vector (SVE) exp10 function. + + Copyright (C) 2023 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include "sv_math.h" +#include "poly_sve_f32.h" + +/* For x < -SpecialBound, the result is subnormal and not handled correctly by + FEXPA. */ +#define SpecialBound 37.9 + +static const struct data +{ + float poly[5]; + float shift, log10_2, log2_10_hi, log2_10_lo, special_bound; +} data = { + /* Coefficients generated using Remez algorithm with minimisation of relative + error. + rel error: 0x1.89dafa3p-24 + abs error: 0x1.167d55p-23 in [-log10(2)/2, log10(2)/2] + maxerr: 0.52 +0.5 ulp. */ + .poly = { 0x1.26bb16p+1f, 0x1.5350d2p+1f, 0x1.04744ap+1f, 0x1.2d8176p+0f, + 0x1.12b41ap-1f }, + /* 1.5*2^17 + 127, a shift value suitable for FEXPA. */ + .shift = 0x1.903f8p17f, + .log10_2 = 0x1.a934fp+1, + .log2_10_hi = 0x1.344136p-2, + .log2_10_lo = -0x1.ec10cp-27, + .special_bound = SpecialBound, +}; + +static svfloat32_t NOINLINE +special_case (svfloat32_t x, svfloat32_t y, svbool_t special) +{ + return sv_call_f32 (exp10f, x, y, special); +} + +/* Single-precision SVE exp10f routine. Implements the same algorithm + as AdvSIMD exp10f. + Worst case error is 1.02 ULPs. + _ZGVsMxv_exp10f(-0x1.040488p-4) got 0x1.ba5f9ep-1 + want 0x1.ba5f9cp-1. */ +svfloat32_t SV_NAME_F1 (exp10) (svfloat32_t x, const svbool_t pg) +{ + const struct data *d = ptr_barrier (&data); + /* exp10(x) = 2^(n/N) * 10^r = 2^n * (1 + poly (r)), + with poly(r) in [1/sqrt(2), sqrt(2)] and + x = r + n * log10(2) / N, with r in [-log10(2)/2N, log10(2)/2N]. */ + + /* Load some constants in quad-word chunks to minimise memory access (last + lane is wasted). */ + svfloat32_t log10_2_and_inv = svld1rq (svptrue_b32 (), &d->log10_2); + + /* n = round(x/(log10(2)/N)). */ + svfloat32_t shift = sv_f32 (d->shift); + svfloat32_t z = svmla_lane (shift, x, log10_2_and_inv, 0); + svfloat32_t n = svsub_x (pg, z, shift); + + /* r = x - n*log10(2)/N. */ + svfloat32_t r = svmls_lane (x, n, log10_2_and_inv, 1); + r = svmls_lane (r, n, log10_2_and_inv, 2); + + svbool_t special = svacgt (pg, x, d->special_bound); + svfloat32_t scale = svexpa (svreinterpret_u32 (z)); + + /* Polynomial evaluation: poly(r) ~ exp10(r)-1. */ + svfloat32_t r2 = svmul_x (pg, r, r); + svfloat32_t poly + = svmla_x (pg, svmul_x (pg, r, d->poly[0]), + sv_pairwise_poly_3_f32_x (pg, r, r2, d->poly + 1), r2); + + if (__glibc_unlikely (svptest_any (pg, special))) + return special_case (x, svmla_x (pg, scale, scale, poly), special); + + return svmla_x (pg, scale, scale, poly); +} diff --git a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c index 8d05498ec9..26d5ecf66f 100644 --- a/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cos_advsimd, _ZGVnN2v_cos) VPCS_VECTOR_WRAPPER (exp_advsimd, _ZGVnN2v_exp) +VPCS_VECTOR_WRAPPER (exp10_advsimd, _ZGVnN2v_exp10) VPCS_VECTOR_WRAPPER (exp2_advsimd, _ZGVnN2v_exp2) VPCS_VECTOR_WRAPPER (log_advsimd, _ZGVnN2v_log) VPCS_VECTOR_WRAPPER (log10_advsimd, _ZGVnN2v_log10) diff --git a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c index b65bc6f1e6..86efd60779 100644 --- a/sysdeps/aarch64/fpu/test-double-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-double-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cos_sve, _ZGVsMxv_cos) SVE_VECTOR_WRAPPER (exp_sve, _ZGVsMxv_exp) +SVE_VECTOR_WRAPPER (exp10_sve, _ZGVsMxv_exp10) SVE_VECTOR_WRAPPER (exp2_sve, _ZGVsMxv_exp2) SVE_VECTOR_WRAPPER (log_sve, _ZGVsMxv_log) SVE_VECTOR_WRAPPER (log10_sve, _ZGVsMxv_log10) diff --git a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c index 6ced0d4488..8f7ebea1ac 100644 --- a/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-advsimd-wrappers.c @@ -25,6 +25,7 @@ VPCS_VECTOR_WRAPPER (cosf_advsimd, _ZGVnN4v_cosf) VPCS_VECTOR_WRAPPER (expf_advsimd, _ZGVnN4v_expf) +VPCS_VECTOR_WRAPPER (exp10f_advsimd, _ZGVnN4v_exp10f) VPCS_VECTOR_WRAPPER (exp2f_advsimd, _ZGVnN4v_exp2f) VPCS_VECTOR_WRAPPER (logf_advsimd, _ZGVnN4v_logf) VPCS_VECTOR_WRAPPER (log10f_advsimd, _ZGVnN4v_log10f) diff --git a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c index 2ed8d0659a..885e58ac39 100644 --- a/sysdeps/aarch64/fpu/test-float-sve-wrappers.c +++ b/sysdeps/aarch64/fpu/test-float-sve-wrappers.c @@ -34,6 +34,7 @@ SVE_VECTOR_WRAPPER (cosf_sve, _ZGVsMxv_cosf) SVE_VECTOR_WRAPPER (expf_sve, _ZGVsMxv_expf) +SVE_VECTOR_WRAPPER (exp10f_sve, _ZGVsMxv_exp10f) SVE_VECTOR_WRAPPER (exp2f_sve, _ZGVsMxv_exp2f) SVE_VECTOR_WRAPPER (logf_sve, _ZGVsMxv_logf) SVE_VECTOR_WRAPPER (log10f_sve, _ZGVsMxv_log10f) diff --git a/sysdeps/aarch64/libm-test-ulps b/sysdeps/aarch64/libm-test-ulps index 6641c7fa0b..d117209c06 100644 --- a/sysdeps/aarch64/libm-test-ulps +++ b/sysdeps/aarch64/libm-test-ulps @@ -970,11 +970,19 @@ double: 2 float: 1 ldouble: 2 +Function: "exp10_advsimd": +double: 1 +float: 2 + Function: "exp10_downward": double: 2 float: 1 ldouble: 3 +Function: "exp10_sve": +double: 1 +float: 1 + Function: "exp10_towardzero": double: 2 float: 1 diff --git a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist index a92585bf8b..6567bbd474 100644 --- a/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist +++ b/sysdeps/unix/sysv/linux/aarch64/libmvec.abilist @@ -14,6 +14,10 @@ GLIBC_2.38 _ZGVsMxv_log F GLIBC_2.38 _ZGVsMxv_logf F GLIBC_2.38 _ZGVsMxv_sin F GLIBC_2.38 _ZGVsMxv_sinf F +GLIBC_2.39 _ZGVnN2v_exp10 F +GLIBC_2.39 _ZGVnN4v_exp10f F +GLIBC_2.39 _ZGVsMxv_exp10 F +GLIBC_2.39 _ZGVsMxv_exp10f F GLIBC_2.39 _ZGVnN2v_exp2 F GLIBC_2.39 _ZGVnN4v_exp2f F GLIBC_2.39 _ZGVsMxv_exp2 F