From patchwork Sat May 25 14:54:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Feng Xue OS X-Patchwork-Id: 1939269 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=os.amperecomputing.com header.i=@os.amperecomputing.com header.a=rsa-sha256 header.s=selector2 header.b=LhhQCdLe; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VmlLb2M3Fz20PT for ; Sun, 26 May 2024 00:54:47 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8C8083858CD9 for ; Sat, 25 May 2024 14:54:45 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2114.outbound.protection.outlook.com [40.107.212.114]) by sourceware.org (Postfix) with ESMTPS id B0A053858D29 for ; Sat, 25 May 2024 14:54:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B0A053858D29 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=os.amperecomputing.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=os.amperecomputing.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org B0A053858D29 Authentication-Results: server2.sourceware.org; arc=pass smtp.remote-ip=40.107.212.114 ARC-Seal: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1716648864; cv=pass; b=gjXc4ZbpfFDxq/eK5uvNHLIS1M4VZFngYdGUkEIlMuerFiSL6Mt0t8h8EV9QOmJOuX8a7pvTm7SrOZxPK48UEInnRyUn+AU+FV8R4PN6/sUuRvaZSy9XyoAdpBaQrkbYYOw/l/4JUg4VmAuit7TVo8r+GsMa1LCazpCrt9HWqc4= ARC-Message-Signature: i=2; a=rsa-sha256; d=sourceware.org; s=key; t=1716648864; c=relaxed/simple; bh=oZlYegiPnir7cKQLTMaVX5hIRoWXvCPnyHhRQfcnjPA=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=vmLIdrtqWv/eOL3LGFjVmPgJ+FfQLu4G4fN0UEEesN+m8+F1pYzycGT5RfANR4bwemycuQhHmy7T+FKtXk8fYSKfNSTb1rWVFFkClhq43PCfMth4nqkYSdG6ajcsTREQawRTQaHUZEAmqps7PjvNMsCo/ONYLPDn9TWcL9smR/k= ARC-Authentication-Results: i=2; server2.sourceware.org ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FzpxV9gZODDlEuoDrpy3fSY3WEL0ibpxk1DKhpSsdVKrFngIdWBIXIqpK0j+cpwJ5dX3F0szC4hVWtVEAWbdOKpSW8eMCfWXrAu74aUkwj++t3QH0gypPPNftZQxPx48TAs6kFc6Z1xnh+S2ToOSjrjmDJAne0oJmNyEoKP2XhvVrrPNeLSrBl5Y/Oz1H93ZsssPMqKjNyr3DpZUeLZRxnZ6fmBZzMyQrJF94t2c83L3EiAL/Ae7vAORrLXunYf2JXIkysGKkGNrPRvyK1VV8PVPSzxxQRLYWPZjDJp14Gtr28gLVKBh06O8Gw7IhST6tt+cu/nLs3aAGjNVvn+USQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z5qEL8P/dTo4JO05hRus51IZP+tE3JLZMBR7S7E5L/Q=; b=dM3213wDkwdPTyF0qpdQMnSny1vMAI5znB7Hm4ONoLKQNFEjV02LBK4HoPhWoGSDOo9de6HcZifQ1U2/8grE1pMwmS6kaPTf3ZePqgYKqZKZ8zTaqiz1FPxT8FI4pN7d0iVfV9raDb4T40Wm8lQ0Nak32JtwcND49NU1PRsL+iKASQ3zrQL1gOTWQM0x8yTYFbn0rN5VXGmuNX6mHpjxYyAKyX7twszBoPlQKHiIZy/66+KCkUvKiqDDkfewBp9ppyXvQYXTA4GD14PTnR9HNw/11scecos6GX/3RCaLHUtqAI3gqjZpiN0ynQ6FpPlTB3bSggqyRAYy7l1EZQ3YvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z5qEL8P/dTo4JO05hRus51IZP+tE3JLZMBR7S7E5L/Q=; b=LhhQCdLeVs7w6H3rflzpBPTN5Vikj+HC1XiDQbJKLrMUGz41Osg1E4D6iNpwE7Reyw13xZeXyfq1SLSs7mlp32+HmzVe54TlJt6ttMrsWPGzlC52cBqu+QcGWdl/qiZ9jHJXfMKydSJpT5B3UqFo3fKfGQQDG9hqP2SsXTiU9BU= Received: from LV2PR01MB7839.prod.exchangelabs.com (2603:10b6:408:14f::13) by LV3PR01MB8559.prod.exchangelabs.com (2603:10b6:408:197::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.19; Sat, 25 May 2024 14:54:19 +0000 Received: from LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63]) by LV2PR01MB7839.prod.exchangelabs.com ([fe80::2ac3:5a77:36fd:9c63%3]) with mapi id 15.20.7611.025; Sat, 25 May 2024 14:54:18 +0000 From: Feng Xue OS To: Richard Biener CC: "gcc-patches@gcc.gnu.org" Subject: [PATCH] vect: Unify bbs in loop_vec_info and bb_vec_info Thread-Topic: [PATCH] vect: Unify bbs in loop_vec_info and bb_vec_info Thread-Index: AQHarrL/TAdoaz0EzU6936381/1nNA== Date: Sat, 25 May 2024 14:54:18 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Enabled=True; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SiteId=3bc2b170-fd94-476d-b0ce-4229bdc904a7; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_SetDate=2024-05-25T14:54:18.303Z; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Name=Confidential; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_ContentBits=0; MSIP_Label_5b82cb1d-c2e0-4643-920a-bbe7b2d7cc47_Method=Standard; authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: LV2PR01MB7839:EE_|LV3PR01MB8559:EE_ x-ms-office365-filtering-correlation-id: 30d20b08-8cb9-4aff-bf04-08dc7cca8e19 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230031|1800799015|366007|376005|38070700009; x-microsoft-antispam-message-info: =?iso-8859-1?q?bi3EzmAHzL1unDlH6eP3pZlJWk?= =?iso-8859-1?q?NfL+iZwyFBHyHPMkiUfzJZLJNyp9lvP7Jt5iCakUeZ26JdZmbB3Rhn6c35Sr?= =?iso-8859-1?q?jFPVUxM42HSGuNEUbZ9oWD6FKNkX2BYijRdL4kkShKhWtvMJfdi+a28Pqw8x?= =?iso-8859-1?q?FMuZYHSUmAqiQEId345bwG1zXK3SPI1fNmN/VJw1IG9mGRhJnmMFa37LUBzu?= =?iso-8859-1?q?lXjY4IZ1kSauVhNTdc5/u4q6kr5hxpf9qNJ9IsxHRPMWd3UMEywLgHiV+cPj?= =?iso-8859-1?q?vmYzKNd1dSjJqIPSPVJWjqYeFjSU9bfCzRf20Ihrc2hP0mt9Mhpd9tm8nLbQ?= =?iso-8859-1?q?TU2CFpvDGYbBNIOQ89dm2IgoXv5QGuDcQKKU5kuXgid0G6Brt1rm7jTGIfKy?= =?iso-8859-1?q?RwEZg04wemVCuWFLTINs9c0l8lUVycGxQRrQpuFvHx2C1ri9umWph+b4VKuJ?= =?iso-8859-1?q?wQDNJg4V0UwmnA5EV/S87VplI8dwKqwEWN7hiGXnoF9v9qxl+Ntj1wXBnzrp?= =?iso-8859-1?q?HEhlC0rYyC/VaB8UXM2XzVKPuGzUqsuHiHamrJiW+b8eHsIfVEvgLpMvfra/?= =?iso-8859-1?q?Az0soQMPXggeQ9gDPaFYcuhqXmB/+VVfEqUhnays1NH06nf9yYl/ShDjD/bI?= =?iso-8859-1?q?HiBSXw3xzmB6femjpbXwFYnsUaAKdMXRB79c2gCAiPVy+yNMiSrWg3cfLRuk?= =?iso-8859-1?q?ZkHHwaMCdiME79LKadtdF3EkRcZ5kfe5nsA+U9QnjhHls8XELTLWO4Zrleyu?= =?iso-8859-1?q?aeyoRIuj15lxOMD2jloPIH9w3T1EPE8gQxP7HFfLUWJuLI5UbzSCbVO7+ya+?= =?iso-8859-1?q?GqetMLzdV0dOQVQmKGYVlTsKeOQ5iUiNco0OQCCV/pYBZDp17yUBF79h6mc5?= =?iso-8859-1?q?NSZuH5ZcTTe1yQUs5U68Xmkd7/OFHTp86I3fDY4r3fSScPUd/BumdAnDrN6X?= =?iso-8859-1?q?ukTGIvzEiKajzmn6mFqGNvRDF5YrCPPdVz/z0FiMwVAfk4iHFxegZpr3vVby?= =?iso-8859-1?q?/JnfFWiPFvYm7nGkQMkwx+LzPIztrBN2uQeBPtd6glMTxrG+sgC8iguEnYLb?= =?iso-8859-1?q?PeZmyv8x8eRGEzpXohMe+wqEOrPg5AFC/MtA+ijd8KYlapn7H5ANtxYprSNj?= =?iso-8859-1?q?5ig66Ul2hC/HEaB4esUqRqP8QqqvhG356moDXgREdzed7G9uo0GbMwEDnsmX?= =?iso-8859-1?q?6ln6Ls9ddQIa+vKydbV6e2MdOgK33go/ohJr6CJ2fW6FXc7O7hInIyVID6c0?= =?iso-8859-1?q?w0LtbLFbHZSijlh1S8is4pysl4dtzrhk7Udde7T/pBrIwtJoiFur0MfLsxMt?= =?iso-8859-1?q?PFl936A9aAyZ6CdrDv0n9xkCJoLI5BuZWiWg+gUzZvV35/18yRSXu1E2RBu1?= =?iso-8859-1?q?NWjnoGDyHb4XgppjklBQ=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:LV2PR01MB7839.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230031)(1800799015)(366007)(376005)(38070700009); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?5ci1hHrE+WXMeKCM0nJ5Rkt?= =?iso-8859-1?q?BICQjrI6Y+2OzIR5lqmVYAnDPnvYedCmwORWIRJ19DsYeSM1cqqxEf07rP0z?= =?iso-8859-1?q?E7TKo3GAARWaCo8IuFuaqpThpLjfBj+NyTbrWpWd6wo6Thp9jAWodxMNeuYp?= =?iso-8859-1?q?fv7XS7fx4QXSdRC7TUmuz9dJ+/6voEG1SBSjcp7CosPfuTbzJwYduexSY0/f?= =?iso-8859-1?q?UELn4RQfivP/91Lv2YIPKNTCfMDqy2JvgQnvPrqL6ZY1TYe7Fgmc48df57IZ?= =?iso-8859-1?q?0tsxCLRRlrg6Er8Ie4LN7y+nN8Om7uUGD5BwqpW9FWTbRWyRsWLj56hAxKfH?= =?iso-8859-1?q?UhYa75I+fFq5fMPNSUqGx4hNcaROTZW+ZeqmYMrtJAA+4TbLB3ik5XZFgEiA?= =?iso-8859-1?q?TgZ7kWS5itXu006/RcR8uDer012gWoJDf0pRsAYE+xGcxMl8G59jQoHQjCCs?= =?iso-8859-1?q?nHDfazVWyKGxD7extNqViqx2zNUr+WeXmo9urf4Q4U4JyNwnMkNXI5woesVr?= =?iso-8859-1?q?CPgcqHabIVNpvhDHw6nZhL6gwOORtCEFejc5d0mSCi8jS75RbvgbjSwax94k?= =?iso-8859-1?q?d+dML9UEBZ3cDOmzlaM6y+GU6aZHhAtDftvxr49zDLE/nPxIIAik5oyrbA6G?= =?iso-8859-1?q?tE9JeGBToUdrZnngftiUlsfSDb4rdIGEsmZOa1Mgy2PwmPp7QY0/y1hhqDIC?= =?iso-8859-1?q?WBvf//w9j3U4c97LvGwhe6Qoh90AMyvdh9jFtTbst9E0dvJL5lPJ3efnGiWh?= =?iso-8859-1?q?cej//ao/6cBqcFjvZ+kaikli8Eby2wzCUvPYa0UkcvCwXZ2bfsqGE5UJ6+jD?= =?iso-8859-1?q?J8oOnHWsXJEbjaghwRi4a85d9Vslqm/XsEJPtsIzAF+jUciMJhLkOTwMg9Dx?= =?iso-8859-1?q?Q2bLxvtp2ViqNpJ1C5V35K6MMiLb5/EKvqaBsb0ATX1MUlmkViqW8bVuu06Y?= =?iso-8859-1?q?jGPsd/h4gUjZ/7BS0QKo8SRNGfJJjWyVs0DK/dJy7BLnr3TIri0/nR+rsMas?= =?iso-8859-1?q?wUTxcEpcX4ofmgWSprDzCMYYYXdvk3PfWGSmpVq9YpGqZVb2t0Gwh/C92zI4?= =?iso-8859-1?q?YMAqrYWH9Nc1Xto331fj0MTjUmk5ng5M2MV5eF8UpmejFyI5I9OCfcqjw/FT?= =?iso-8859-1?q?+ltBWh6WXg0BJplA5vzZarl21IcX39x0zZy+zoSQbcH4busK3ajQvBn4FQj2?= =?iso-8859-1?q?GQ+UqVhjLp1vwd5rIp0g0rpsYGbK8NAVFHlMnaLquRVV3SklWlKPzUl0vD6+?= =?iso-8859-1?q?rrK8sFPEYHuEsnsAnI+a6DeWK9zt83A6J8Elf/q/PZg6RfPwA9A+py1XKkh3?= =?iso-8859-1?q?pctP31e1CKCXVCNncKeDHOIwN//hGeDXP7df2J5bxZGci8YChKctMXRvD7a+?= =?iso-8859-1?q?BEhOshwR+cku5q+fP1KXDjsq8KLfl3KK7r3AGDSdHdbanRTxt2U4DZQcQvl2?= =?iso-8859-1?q?W7elAUahLNF1Q0ZcBWA/uC9eaoIy1qaiNSYIAJdR9HPUh5TKRJwPoT7e4kCy?= =?iso-8859-1?q?USZyvbGpTtY2MeWkIcoPQ/JhnSPwlgWb2IuO9clJDPGaQUrp1lfrUu5aSFuI?= =?iso-8859-1?q?wRNDCEUn5TZF2ng8lLQKA5bJukldTrxgggCCGBul29jslh+d/3rgbSiOh6fU?= =?iso-8859-1?q?GV+m8BoyAP5L4xBhFdcNasUbvP4IqFwG6KdgrWQ=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: LV2PR01MB7839.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30d20b08-8cb9-4aff-bf04-08dc7cca8e19 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 May 2024 14:54:18.8703 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: GHwiRPmlPwQv6pIwJ+QBL2yBmtNz7d6iSCvvuvO4vV6vePfYY8n5rrIQPWsSaP8nGyVcAdFiH7qCY68HsDc0yQ2t9Er7KRJ9d8HktVVLx4I= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR01MB8559 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, GIT_PATCH_0, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Both derived classes ( loop_vec_info/bb_vec_info) have their own "bbs" field, which have exactly same purpose of recording all basic blocks inside the corresponding vect region, while the fields are composed by different data type, one is normal array, the other is auto_vec. This difference causes some duplicated code even handling the same stuff, almost in tree-vect-patterns. One refinement is lifting this field into the base class "vec_info", and reset its value to the continuous memory area pointed by two old "bbs" in each constructor of derived classes. Regression test on x86-64 and aarch64. Feng --- gcc/ * tree-vect-loop.cc (_loop_vec_info::_loop_vec_info): Move initialization of bbs to explicit construction code. Adjust the definition of nbbs. * tree-vect-pattern.cc (vect_determine_precisions): Make loop_vec_info and bb_vec_info share same code. (vect_pattern_recog): Remove duplicated vect_pattern_recog_1 loop. * tree-vect-slp.cc (vect_get_and_check_slp_defs): Access to bbs[0] via base vec_info class. (_bb_vec_info::_bb_vec_info): Initialize bbs and nbbs using data fields of input auto_vec<> bbs. (_bb_vec_info::_bb_vec_info): Add assertions on bbs and nbbs to ensure they are not changed externally. (vect_slp_region): Use access to nbbs to replace original bbs.length(). (vect_schedule_slp_node): Access to bbs[0] via base vec_info class. * tree-vectorizer.cc (vec_info::vec_info): Add initialization of bbs and nbbs. (vec_info::insert_seq_on_entry): Access to bbs[0] via base vec_info class. * tree-vectorizer.h (vec_info): Add new fields bbs and nbbs. (_loop_vec_info): Remove field bbs. (_bb_vec_info): Rename old bbs field to bbs_as_vector, and make it be private. --- gcc/tree-vect-loop.cc | 6 +- gcc/tree-vect-patterns.cc | 142 +++++++++++--------------------------- gcc/tree-vect-slp.cc | 24 ++++--- gcc/tree-vectorizer.cc | 7 +- gcc/tree-vectorizer.h | 19 ++--- 5 files changed, 72 insertions(+), 126 deletions(-) diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc index 83c0544b6aa..aef17420a5f 100644 --- a/gcc/tree-vect-loop.cc +++ b/gcc/tree-vect-loop.cc @@ -1028,7 +1028,6 @@ bb_in_loop_p (const_basic_block bb, const void *data) _loop_vec_info::_loop_vec_info (class loop *loop_in, vec_info_shared *shared) : vec_info (vec_info::loop, shared), loop (loop_in), - bbs (XCNEWVEC (basic_block, loop->num_nodes)), num_itersm1 (NULL_TREE), num_iters (NULL_TREE), num_iters_unchanged (NULL_TREE), @@ -1079,8 +1078,9 @@ _loop_vec_info::_loop_vec_info (class loop *loop_in, vec_info_shared *shared) case of the loop forms we allow, a dfs order of the BBs would the same as reversed postorder traversal, so we are safe. */ - unsigned int nbbs = dfs_enumerate_from (loop->header, 0, bb_in_loop_p, - bbs, loop->num_nodes, loop); + bbs = XCNEWVEC (basic_block, loop->num_nodes); + nbbs = dfs_enumerate_from (loop->header, 0, bb_in_loop_p, bbs, + loop->num_nodes, loop); gcc_assert (nbbs == loop->num_nodes); for (unsigned int i = 0; i < nbbs; i++) diff --git a/gcc/tree-vect-patterns.cc b/gcc/tree-vect-patterns.cc index a313dc64643..848a3195a93 100644 --- a/gcc/tree-vect-patterns.cc +++ b/gcc/tree-vect-patterns.cc @@ -6925,81 +6925,41 @@ vect_determine_stmt_precisions (vec_info *vinfo, stmt_vec_info stmt_info) void vect_determine_precisions (vec_info *vinfo) { + basic_block *bbs = vinfo->bbs; + unsigned int nbbs = vinfo->nbbs; + DUMP_VECT_SCOPE ("vect_determine_precisions"); - if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) + for (unsigned int i = 0; i < nbbs; i++) { - class loop *loop = LOOP_VINFO_LOOP (loop_vinfo); - basic_block *bbs = LOOP_VINFO_BBS (loop_vinfo); - unsigned int nbbs = loop->num_nodes; - - for (unsigned int i = 0; i < nbbs; i++) + basic_block bb = bbs[i]; + for (auto gsi = gsi_start_phis (bb); !gsi_end_p (gsi); gsi_next (&gsi)) { - basic_block bb = bbs[i]; - for (auto gsi = gsi_start_phis (bb); - !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); - if (stmt_info) - vect_determine_mask_precision (vinfo, stmt_info); - } - for (auto si = gsi_start_bb (bb); !gsi_end_p (si); gsi_next (&si)) - if (!is_gimple_debug (gsi_stmt (si))) - vect_determine_mask_precision - (vinfo, vinfo->lookup_stmt (gsi_stmt (si))); + stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); + if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) + vect_determine_mask_precision (vinfo, stmt_info); } - for (unsigned int i = 0; i < nbbs; i++) + for (auto gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) { - basic_block bb = bbs[nbbs - i - 1]; - for (gimple_stmt_iterator si = gsi_last_bb (bb); - !gsi_end_p (si); gsi_prev (&si)) - if (!is_gimple_debug (gsi_stmt (si))) - vect_determine_stmt_precisions - (vinfo, vinfo->lookup_stmt (gsi_stmt (si))); - for (auto gsi = gsi_start_phis (bb); - !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); - if (stmt_info) - vect_determine_stmt_precisions (vinfo, stmt_info); - } + stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (gsi)); + if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) + vect_determine_mask_precision (vinfo, stmt_info); } } - else + for (unsigned int i = 0; i < nbbs; i++) { - bb_vec_info bb_vinfo = as_a (vinfo); - for (unsigned i = 0; i < bb_vinfo->bbs.length (); ++i) + basic_block bb = bbs[nbbs - i - 1]; + for (auto gsi = gsi_last_bb (bb); !gsi_end_p (gsi); gsi_prev (&gsi)) { - basic_block bb = bb_vinfo->bbs[i]; - for (auto gsi = gsi_start_phis (bb); !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); - if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) - vect_determine_mask_precision (vinfo, stmt_info); - } - for (auto gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (gsi)); - if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) - vect_determine_mask_precision (vinfo, stmt_info); - } + stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (gsi)); + if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) + vect_determine_stmt_precisions (vinfo, stmt_info); } - for (int i = bb_vinfo->bbs.length () - 1; i != -1; --i) + for (auto gsi = gsi_start_phis (bb); !gsi_end_p (gsi); gsi_next (&gsi)) { - for (gimple_stmt_iterator gsi = gsi_last_bb (bb_vinfo->bbs[i]); - !gsi_end_p (gsi); gsi_prev (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (gsi)); - if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) - vect_determine_stmt_precisions (vinfo, stmt_info); - } - for (auto gsi = gsi_start_phis (bb_vinfo->bbs[i]); - !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); - if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) - vect_determine_stmt_precisions (vinfo, stmt_info); - } + stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi.phi ()); + if (stmt_info && STMT_VINFO_VECTORIZABLE (stmt_info)) + vect_determine_stmt_precisions (vinfo, stmt_info); } } } @@ -7328,56 +7288,32 @@ vect_pattern_recog_1 (vec_info *vinfo, void vect_pattern_recog (vec_info *vinfo) { - class loop *loop; - basic_block *bbs; - unsigned int nbbs; - gimple_stmt_iterator si; - unsigned int i, j; + basic_block *bbs = vinfo->bbs; + unsigned int nbbs = vinfo->nbbs; vect_determine_precisions (vinfo); DUMP_VECT_SCOPE ("vect_pattern_recog"); - if (loop_vec_info loop_vinfo = dyn_cast (vinfo)) + /* Scan through the stmts in the region, applying the pattern recognition + functions starting at each stmt visited. */ + for (unsigned i = 0; i < nbbs; i++) { - loop = LOOP_VINFO_LOOP (loop_vinfo); - bbs = LOOP_VINFO_BBS (loop_vinfo); - nbbs = loop->num_nodes; + basic_block bb = bbs[i]; - /* Scan through the loop stmts, applying the pattern recognition - functions starting at each stmt visited: */ - for (i = 0; i < nbbs; i++) + for (auto si = gsi_start_bb (bb); !gsi_end_p (si); gsi_next (&si)) { - basic_block bb = bbs[i]; - for (si = gsi_start_bb (bb); !gsi_end_p (si); gsi_next (&si)) - { - if (is_gimple_debug (gsi_stmt (si))) - continue; - stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (si)); - /* Scan over all generic vect_recog_xxx_pattern functions. */ - for (j = 0; j < NUM_PATTERNS; j++) - vect_pattern_recog_1 (vinfo, &vect_vect_recog_func_ptrs[j], - stmt_info); - } + stmt_vec_info stmt_info = vinfo->lookup_stmt (gsi_stmt (si)); + + if (!stmt_info || !STMT_VINFO_VECTORIZABLE (stmt_info)) + continue; + + /* Scan over all generic vect_recog_xxx_pattern functions. */ + for (unsigned j = 0; j < NUM_PATTERNS; j++) + vect_pattern_recog_1 (vinfo, &vect_vect_recog_func_ptrs[j], + stmt_info); } } - else - { - bb_vec_info bb_vinfo = as_a (vinfo); - for (unsigned i = 0; i < bb_vinfo->bbs.length (); ++i) - for (gimple_stmt_iterator gsi = gsi_start_bb (bb_vinfo->bbs[i]); - !gsi_end_p (gsi); gsi_next (&gsi)) - { - stmt_vec_info stmt_info = bb_vinfo->lookup_stmt (gsi_stmt (gsi)); - if (!stmt_info || !STMT_VINFO_VECTORIZABLE (stmt_info)) - continue; - - /* Scan over all generic vect_recog_xxx_pattern functions. */ - for (j = 0; j < NUM_PATTERNS; j++) - vect_pattern_recog_1 (vinfo, - &vect_vect_recog_func_ptrs[j], stmt_info); - } - } /* After this no more add_stmt calls are allowed. */ vinfo->stmt_vec_info_ro = true; diff --git a/gcc/tree-vect-slp.cc b/gcc/tree-vect-slp.cc index c7ed520b629..2e0244cf582 100644 --- a/gcc/tree-vect-slp.cc +++ b/gcc/tree-vect-slp.cc @@ -748,8 +748,7 @@ vect_get_and_check_slp_defs (vec_info *vinfo, unsigned char swap, && is_a (vinfo) && TREE_CODE (oprnd) == SSA_NAME && !SSA_NAME_IS_DEFAULT_DEF (oprnd) - && !dominated_by_p (CDI_DOMINATORS, - as_a (vinfo)->bbs[0], + && !dominated_by_p (CDI_DOMINATORS, vinfo->bbs[0], gimple_bb (SSA_NAME_DEF_STMT (oprnd)))) { if (dump_enabled_p ()) @@ -6288,10 +6287,13 @@ vect_detect_hybrid_slp (loop_vec_info loop_vinfo) _bb_vec_info::_bb_vec_info (vec _bbs, vec_info_shared *shared) : vec_info (vec_info::bb, shared), - bbs (_bbs), + bbs_as_vector (_bbs), roots (vNULL) { - for (unsigned i = 0; i < bbs.length (); ++i) + bbs = bbs_as_vector.address (); + nbbs = bbs_as_vector.length (); + + for (unsigned i = 0; i < nbbs; ++i) { if (i != 0) for (gphi_iterator si = gsi_start_phis (bbs[i]); !gsi_end_p (si); @@ -6319,8 +6321,12 @@ _bb_vec_info::_bb_vec_info (vec _bbs, vec_info_shared *shared) _bb_vec_info::~_bb_vec_info () { + /* Check that bbs stored in the vector is not changed externally. */ + gcc_assert (bbs == bbs_as_vector.address ()); + gcc_assert (nbbs = bbs_as_vector.length ()); + /* Reset region marker. */ - for (unsigned i = 0; i < bbs.length (); ++i) + for (unsigned i = 0; i < nbbs; ++i) { if (i != 0) for (gphi_iterator si = gsi_start_phis (bbs[i]); !gsi_end_p (si); @@ -7625,7 +7631,7 @@ vect_slp_is_lane_insert (gimple *use_stmt, tree vec, unsigned *this_lane) static void vect_slp_check_for_roots (bb_vec_info bb_vinfo) { - for (unsigned i = 0; i < bb_vinfo->bbs.length (); ++i) + for (unsigned i = 0; i < bb_vinfo->nbbs; ++i) for (gimple_stmt_iterator gsi = gsi_start_bb (bb_vinfo->bbs[i]); !gsi_end_p (gsi); gsi_next (&gsi)) { @@ -8109,7 +8115,7 @@ vect_slp_region (vec bbs, vec datarefs, we vectorized all if-converted code. */ if ((!profitable_subgraphs.is_empty () || force_clear) && orig_loop) { - gcc_assert (bb_vinfo->bbs.length () == 1); + gcc_assert (bb_vinfo->nbbs == 1); for (gimple_stmt_iterator gsi = gsi_start_bb (bb_vinfo->bbs[0]); !gsi_end_p (gsi); gsi_next (&gsi)) { @@ -9613,14 +9619,14 @@ vect_schedule_slp_node (vec_info *vinfo, if (!last_stmt) { gcc_assert (seen_vector_def); - si = gsi_after_labels (as_a (vinfo)->bbs[0]); + si = gsi_after_labels (vinfo->bbs[0]); } else if (is_ctrl_altering_stmt (last_stmt)) { /* We split regions to vectorize at control altering stmts with a definition so this must be an external which we can insert at the start of the region. */ - si = gsi_after_labels (as_a (vinfo)->bbs[0]); + si = gsi_after_labels (vinfo->bbs[0]); } else if (is_a (vinfo) && gimple_bb (last_stmt) != gimple_bb (stmt_info->stmt) diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc index 9001b738bf3..1fb4fb36ed4 100644 --- a/gcc/tree-vectorizer.cc +++ b/gcc/tree-vectorizer.cc @@ -463,7 +463,9 @@ shrink_simd_arrays vec_info::vec_info (vec_info::vec_kind kind_in, vec_info_shared *shared_) : kind (kind_in), shared (shared_), - stmt_vec_info_ro (false) + stmt_vec_info_ro (false), + bbs (NULL), + nbbs (0) { stmt_vec_infos.create (50); } @@ -660,9 +662,8 @@ vec_info::insert_seq_on_entry (stmt_vec_info context, gimple_seq seq) } else { - bb_vec_info bb_vinfo = as_a (this); gimple_stmt_iterator gsi_region_begin - = gsi_after_labels (bb_vinfo->bbs[0]); + = gsi_after_labels (bbs[0]); gsi_insert_seq_before (&gsi_region_begin, seq, GSI_SAME_STMT); } } diff --git a/gcc/tree-vectorizer.h b/gcc/tree-vectorizer.h index 93bc30ef660..ffc2b223dd2 100644 --- a/gcc/tree-vectorizer.h +++ b/gcc/tree-vectorizer.h @@ -499,6 +499,12 @@ public: made any decisions about which vector modes to use. */ machine_mode vector_mode; + /* The basic blocks in the vectorization region. */ + basic_block *bbs; + + /* The count of the basic blocks in the vectorization region. */ + unsigned int nbbs; + private: stmt_vec_info new_stmt_vec_info (gimple *stmt); void set_vinfo_for_stmt (gimple *, stmt_vec_info, bool = true); @@ -679,9 +685,6 @@ public: /* The loop to which this info struct refers to. */ class loop *loop; - /* The loop basic blocks. */ - basic_block *bbs; - /* Number of latch executions. */ tree num_itersm1; /* Number of iterations. */ @@ -1090,15 +1093,15 @@ struct slp_root typedef class _bb_vec_info : public vec_info { -public: - _bb_vec_info (vec bbs, vec_info_shared *); - ~_bb_vec_info (); - /* The region we are operating on. bbs[0] is the entry, excluding its PHI nodes. In the future we might want to track an explicit entry edge to cover bbs[0] PHI nodes and have a region entry insert location. */ - vec bbs; + vec bbs_as_vector; + +public: + _bb_vec_info (vec bbs, vec_info_shared *); + ~_bb_vec_info (); vec roots; } *bb_vec_info;