From patchwork Thu Jul 15 16:39:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1505791 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=jLCpJmuA; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GQgBr1ZKMz9sPf for ; Fri, 16 Jul 2021 02:42:12 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CA1583989C0D for ; Thu, 15 Jul 2021 16:42:09 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CA1583989C0D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1626367329; bh=HljsPO54IOTtuCe+LSYFvaETddQe4Wh/VTYwpH10C5U=; h=Date:To:Subject:In-Reply-To:List-Id:List-Unsubscribe:List-Archive: List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:From; b=jLCpJmuA6Hx/KrYSLQzHPq+aOjOwufo7VCftNLE7bGwY9hvlAVa/TzlXynNOYZ9/I nA+SaR69H11sE+BGrnYENF2ZunSNidtvExHVa2lsUnnl49Ajy3XpVAcNp+/O11MMUt 6xMP+eRugncrwLmZ7xR3lSxp4L8d0FNJ/QkQTGwg= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70080.outbound.protection.outlook.com [40.107.7.80]) by sourceware.org (Postfix) with ESMTPS id AB2B13989C1D for ; Thu, 15 Jul 2021 16:40:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org AB2B13989C1D Received: from DB7PR03CA0088.eurprd03.prod.outlook.com (2603:10a6:10:72::29) by DB7PR08MB3178.eurprd08.prod.outlook.com (2603:10a6:5:24::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Thu, 15 Jul 2021 16:40:08 +0000 Received: from DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:72:cafe::2) by DB7PR03CA0088.outlook.office365.com (2603:10a6:10:72::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:40:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT032.mail.protection.outlook.com (10.152.20.162) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:40:08 +0000 Received: ("Tessian outbound ef2da60907d5:v99"); Thu, 15 Jul 2021 16:40:08 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: b408e487f256cf8e X-CR-MTA-TID: 64aa7808 Received: from eb60b93b33fc.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 03B024ED-3F0B-4522-867A-B0CA82D912E1.1; Thu, 15 Jul 2021 16:40:01 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id eb60b93b33fc.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 15 Jul 2021 16:40:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jmGnOwNaTybp8kpgVIPn1fcuvYL9d0mIeRl8Okpr/oe4ucLfRt+LTJ1iKVhU3Ru2Y/M74dPZoLxzoP6VBldvFZI/cBJldNoQGyPnvxXp90bGGnZYMkcVu0mHlYwujqM6aBBQDL6JIK/hav36r8LEeFb7Q8CFsuHzXzanFE80PuTsb89BN1fnjbKYK/SszoWS4VZ08zyL1NMpwP5Gbcty8S0Cy2CYe0bHUmZJzyQZXMaOZhVcIcCwby5hq8ChSUCH0ggyQXxZ8sEVmDDTgq7hP5dzcJIMBHjQBMFoBL0oiD9YXCpmB+FxBKn1WygNSp2xVZSuxrIhastWBX+Yzdo/qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HljsPO54IOTtuCe+LSYFvaETddQe4Wh/VTYwpH10C5U=; b=ZtpBEQVMRVe75d8se/ngQONSrtQx65WA/6rqq0ma3Vyjdze2eibU4DGmGMxO5rplkupfuwN8y/UfWrijktPwLE4ZZFOXZcCOD6AsRdSNRhnT3+2ERn4zYYQ1qR2P1KLC3Ij0n5lmC3F9zfJWn0x2Z/KxfFvn0+oFDEwSEOtl38krGPDW3Yoe7CqQB+LAXUnqJEupdz/TP5uYAieKiIpLtEGNjlBbPiZc8gPaBPBGe6VU1VRITROiX8uZbEWQt9j4BKIVf69swoO41y+pSNhqhRgxav44GkKx1e9rMFO3z5TWYr3pKDLytrbEphCW8XKdrdbYibBenwetFDYmQSSVpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VI1PR0801MB1727.eurprd08.prod.outlook.com (2603:10a6:800:5a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.25; Thu, 15 Jul 2021 16:40:00 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::cdbb:a6b9:e814:19c7]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::cdbb:a6b9:e814:19c7%7]) with mapi id 15.20.4331.021; Thu, 15 Jul 2021 16:40:00 +0000 Date: Thu, 15 Jul 2021 17:39:58 +0100 To: gcc-patches@gcc.gnu.org Subject: [PATCH 2/4]AArch64: correct usdot vectorizer and intrinsics optabs Message-ID: <20210715163953.GA2861@arm.com> Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: LO4P123CA0076.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:190::9) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.51) by LO4P123CA0076.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:190::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Thu, 15 Jul 2021 16:39:59 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 32c39ca7-685d-4e41-a458-08d947af34d6 X-MS-TrafficTypeDiagnostic: VI1PR0801MB1727:|DB7PR08MB3178: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:7691;OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: VJ/Uxk7zFof0MOdyBvq+BU1fsKJ8/0KXM6c3/cPLiTtdEllq3O9+zSq1jdCId0UQkR6ahb07IJOylysrAHGnUFxgJ/N0q2IuD9BI8TL68M+caQjzPl+FyciHhQyPXzG2a0v0Gq+VMCKl2xhTHwsU0J4PCS2C4hEYP95nMijZ8zlDdl05TmR66u4DLnIw9R+brbLXaMDg6MhxdEaPo6MAehqzOoTsERHR0ZuILv5sdEQkyVg0aY5kaz7khUmsXV+o854ReexjbUZG3OVzTJSu95ndaikZ1yFur2EwR7xvfg9Z9DHvPqzE6aOeCfZp4xSRcUwn3bVuacOHfVSolOZsS0axY9r/m190bVsDO8rmoDSYdswBchTjq0wGIWB6c0wSQ2nwZ0LBo8x8hDtvLhHEt0IBOcUQPOoMH3qfW+o6FxG18rV9QKjfcmK0L+v1kgom1XisQg/CldriYqJ9AqLmfhvHvxR7YwUNoOo+wUefN4h3EkMtbKfn9mdyynCw+Fs45l/2b341w1/oaXcmlrMfx3Ix++KRm3rhiyQ1vaIh6bwfeG17B4HCxOQPywlhMknVGMMVKgkCvAhoQgmUhxXZS+UzinxPySq7NDPW27IyAQL8vWlJx8Spt28ijy/tapXmUm7ljzLPRAbdbwRG/pGcm4ebLBt3u1VCadorauYRy0Dpt3c/pmdEbszsPFhE6DrRMm37PSDtbcW4HeBEc9pabJlw5gSSpihxJhipaQ9YO1o= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(376002)(39830400003)(396003)(136003)(346002)(186003)(2906002)(5660300002)(55016002)(4743002)(8676002)(36756003)(478600001)(1076003)(6916009)(26005)(52116002)(66556008)(7696005)(66616009)(86362001)(8886007)(8936002)(235185007)(44832011)(33656002)(4326008)(66476007)(33964004)(316002)(66946007)(956004)(38100700002)(44144004)(2616005)(38350700002)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?BpFh1EzT2//ZDsWcogalKS8Iw5Fr?= =?utf-8?q?XJidJPhiVnucIwFxzaonQjSQgSgZIhczIqhaAhUTcuA+upAkKyx2xkC2DZxyDKt2y?= =?utf-8?q?fV+977ZMOHAscUn7IKdQ52xnGz1z3PtT7xv+s6l9KQNhall3rHnRnnrPtlXc7An4p?= =?utf-8?q?hl6/x4as3n6qHdMa8TOrMbTACuvUZbrSbVinqNVBlVNN3fa+5TqDGSqILBmneP0Sm?= =?utf-8?q?XtOuuLdJIazsqhzAbx6yGShCZitFHK6/tq5RX5YvAnu9QBoTkx+sFt1yfRFjkpF2p?= =?utf-8?q?fKU3s91lTanOdw+rVwvAY52yCTxJuDvTm8E8qgfWt87Jh7qJ9kThYMwF1WRAAtKxp?= =?utf-8?q?G2CaVSMg+a6q1QqcCy4R8Sn5cRG6jkBpraKeRMHJQxiYONPpX3LyDlURXfWdvq7Rr?= =?utf-8?q?AijszvT9REgV/wY7MfxdkneK6KDssxr2ns8w0K/IoO3KCOIh/VRuNeI9H+sQODCDH?= =?utf-8?q?cJTnd2wi5kM0DJjQ0Jx08kcTWeRqhwXNmnq/yY8P45UEaBCu9kD5AunJT70H95nyL?= =?utf-8?q?8qNziwkSfakUMDppbupRl/GVP2zrX7XUPxG+wGuN0PgkNgR+Q4L1bsAfIrSUl8TXL?= =?utf-8?q?HUery+BP+wR5s3YHzrjN/MSZLDpfns2V5vt0GycXlZ/NCPJQdYYamGN67sXDL+BQH?= =?utf-8?q?TWtMPgPFCcURvzwR+lgmYOibpnBdtkfJaZfz+0IMDVfX0jKJQWdfPMUcQicnx9KZJ?= =?utf-8?q?z6EiPTXBRv0UeySPd0MqYd/AZOzaRqSEu0XNZMIKxeYvHMor7HHEkYZlPrL0/kJdM?= =?utf-8?q?W3L56KMkjTb0GRwJafL6+F9Gv0wU8Z+HrfuSPdHqn2p8+JmtVnax7bFTtHzfNeDIk?= =?utf-8?q?JC08WuiUK/vm3Pm/7SnPuu2nrDniRZ8Wq8OfthHQNK/4nYjdjw24cVGkSf68M1W2M?= =?utf-8?q?Y6sxAdqsaG5MgYT6lHC9SaOCWHTn/15hbAJt0MVoUnniX6PUGM66862H7nJgS+iLu?= =?utf-8?q?31niHfqtacgTIa2wr+3pDGJ8QnASSTWvBz5Q2iLoccdnuIB5WC2sO0FklP2p6KyDE?= =?utf-8?q?43GkuVcVqvdCRTLrzC6F2TgLkyQnHt2WoJcKQ/ZLFMESMdSiQZs+TrJ+f9XY4Go+h?= =?utf-8?q?Wem8HcJg7VpFwZG4xNWzdwO8escV4792nJO5B+1iBbHEs08EFABhwT4Ws0R6HSbB0?= =?utf-8?q?ka/MxtoCNGiIpqMKMdl8KUPNWUrs/ZpqUzqDs582L0CjM4VpetQmopkLFfqZt4v3O?= =?utf-8?q?2f4z8siPdDI0gEZOkaUGMwlMC8jKHIuwm8oxel2feRtKcLmT62FGXgVAcZsg9HBSY?= =?utf-8?q?f5L6LszgkAYxaSOp?= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0801MB1727 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 8c1df05f-913e-4fef-ba65-08d947af2fec X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zGOUuBJO9Ih9cJj4LZ6foX2+DQReCLu3VzPl5v3LZ8RL3F7JHhcWEzJveQu/pZKigGE1RP/NA9VdSlZtkFebyZEtKu7BZbeUNS4COyLUdYxfWE8iXGhUvGvYrIcTZfEFMknxD8M/CEiteEtnA1CyscUAU0KNhdsWi79xFWksUbwTZQIJjpMXwS0OOIismrZqPAjCyWiSDzXFvjZTKZDVdPVH7JAacRqSf3ZyaA/8pYnxgqc9H9Oom7fLxBe13a8pC0KiaFHJQ6ohjbIFfuxKrNvTvVKKD7wVd8hOr0Z3pVRt11A3vIJyTjV8xXVP1dK1pWqCWiyLNYKH8Sc7GLmdeu6B6XuY5rlNb5zQTwwUerPcOV1pyoGveaSm4NDtGB16L+5j7EKBy74qqt136e/pfAI0kQAcqiaHWm0warEggUQFyS27FMPUFXsVA9OGusdFB07LMSq6W21EM5R30YAPJT+Co1blhCQ5/HDY0NAbZhHSM846cxM8EaqpRbNi/c8/Kqde1Jiw66fzDzj/nxpU7gjvZnfGGspb7bv7/llqpOV9wGmXAZLVxcQLCWlwhsFZXDEjXu3hNIBOXPT7mkNG4Z8l/I3R7wD2t3RAtRrd8afTrvWZzY8q2Gkavm5p2R+nzvhGOWGhMfYlu2qHY06r1yKGfQwsxZbMp64c0p0vWuzC2iC//knK4vozZPcvXOJ4ixsQTDuCGhyagyZC3G+vqCRS+fnvB3fpla1qa83i+vI= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(346002)(39860400002)(396003)(376002)(136003)(36840700001)(46966006)(44832011)(956004)(1076003)(2616005)(6916009)(186003)(336012)(235185007)(44144004)(66616009)(7696005)(5660300002)(4326008)(26005)(70206006)(33964004)(70586007)(8886007)(8676002)(4743002)(82740400003)(356005)(81166007)(36860700001)(55016002)(478600001)(86362001)(33656002)(316002)(2906002)(8936002)(47076005)(82310400003)(36756003)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2021 16:40:08.0896 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 32c39ca7-685d-4e41-a458-08d947af34d6 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT032.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3178 X-Spam-Status: No, score=-13.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, KAM_LOTSOFHASH, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tamar Christina via Gcc-patches From: Tamar Christina Reply-To: Tamar Christina Cc: Richard.Earnshaw@arm.com, nd@arm.com, richard.sandiford@arm.com, Marcus.Shawcroft@arm.com Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Hi All, There's a slight mismatch between the vectorizer optabs and the intrinsics patterns for NEON. The vectorizer expects operands[3] and operands[0] to be the same but the aarch64 intrinsics expanders expect operands[0] and operands[1] to be the same. This means we need different patterns here. This adds a separate usdot vectorizer pattern which just shuffles around the RTL params. There's also an inconsistency between the usdot and (u|s)dot intrinsics RTL patterns which is not corrected here. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * config/aarch64/aarch64-simd.md (usdot_prod): Rename to... (aarch64_usdot): ..This (usdot_prod): New. * config/aarch64/arm_neon.h (vusdot_s32, vusdotq_s32): Use aarch64_usdot. * config/aarch64/aarch64-simd-builtins.def: Likewise. --- inline copy of patch -- diff --git a/gcc/config/aarch64/aarch64-simd-builtins.def b/gcc/config/aarch64/aarch64-simd-builtins.def index 063f503ebd96657f017dfaa067cb231991376bda..ac5d4fc7ff1e61d404e66193b629986382ee4ffd 100644 diff --git a/gcc/config/aarch64/aarch64-simd-builtins.def b/gcc/config/aarch64/aarch64-simd-builtins.def index 063f503ebd96657f017dfaa067cb231991376bda..ac5d4fc7ff1e61d404e66193b629986382ee4ffd 100644 --- a/gcc/config/aarch64/aarch64-simd-builtins.def +++ b/gcc/config/aarch64/aarch64-simd-builtins.def @@ -374,11 +374,10 @@ BUILTIN_VSDQ_I_DI (BINOP, srshl, 0, NONE) BUILTIN_VSDQ_I_DI (BINOP_UUS, urshl, 0, NONE) - /* Implemented by _prod. */ + /* Implemented by aarch64_{_lane}{q}. */ BUILTIN_VB (TERNOP, sdot, 0, NONE) BUILTIN_VB (TERNOPU, udot, 0, NONE) - BUILTIN_VB (TERNOP_SSUS, usdot_prod, 10, NONE) - /* Implemented by aarch64__lane{q}. */ + BUILTIN_VB (TERNOP_SSUS, usdot, 0, NONE) BUILTIN_VB (QUADOP_LANE, sdot_lane, 0, NONE) BUILTIN_VB (QUADOPU_LANE, udot_lane, 0, NONE) BUILTIN_VB (QUADOP_LANE, sdot_laneq, 0, NONE) diff --git a/gcc/config/aarch64/aarch64-simd.md b/gcc/config/aarch64/aarch64-simd.md index 74890989cb3045798bf8d0241467eaaf72238297..7397f1ec5ca0cb9e3cdd5c46772f604e640666e4 100644 --- a/gcc/config/aarch64/aarch64-simd.md +++ b/gcc/config/aarch64/aarch64-simd.md @@ -601,7 +601,7 @@ (define_insn "aarch64_dot" ;; These instructions map to the __builtins for the armv8.6a I8MM usdot ;; (vector) Dot Product operation. -(define_insn "usdot_prod" +(define_insn "aarch64_usdot" [(set (match_operand:VS 0 "register_operand" "=w") (plus:VS (unspec:VS [(match_operand: 2 "register_operand" "w") @@ -648,6 +648,17 @@ (define_expand "dot_prod" DONE; }) +;; Auto-vectorizer pattern for usdot. The operand[3] and operand[0] are the +;; RMW parameters that when it comes to the vectorizer. +(define_expand "usdot_prod" + [(set (match_operand:VS 0 "register_operand") + (plus:VS (unspec:VS [(match_operand: 1 "register_operand") + (match_operand: 2 "register_operand")] + UNSPEC_USDOT) + (match_operand:VS 3 "register_operand")))] + "TARGET_I8MM" +) + ;; These instructions map to the __builtins for the Dot Product ;; indexed operations. (define_insn "aarch64_dot_lane" diff --git a/gcc/config/aarch64/arm_neon.h b/gcc/config/aarch64/arm_neon.h index 00d76ea937ace5763746478cbdfadf6479e0b15a..17e059efb80fa86a8a32127ace4fc7f43e2040a8 100644 --- a/gcc/config/aarch64/arm_neon.h +++ b/gcc/config/aarch64/arm_neon.h @@ -34039,14 +34039,14 @@ __extension__ extern __inline int32x2_t __attribute__ ((__always_inline__, __gnu_inline__, __artificial__)) vusdot_s32 (int32x2_t __r, uint8x8_t __a, int8x8_t __b) { - return __builtin_aarch64_usdot_prodv8qi_ssus (__r, __a, __b); + return __builtin_aarch64_usdotv8qi_ssus (__r, __a, __b); } __extension__ extern __inline int32x4_t __attribute__ ((__always_inline__, __gnu_inline__, __artificial__)) vusdotq_s32 (int32x4_t __r, uint8x16_t __a, int8x16_t __b) { - return __builtin_aarch64_usdot_prodv16qi_ssus (__r, __a, __b); + return __builtin_aarch64_usdotv16qi_ssus (__r, __a, __b); } __extension__ extern __inline int32x2_t