From patchwork Fri Sep 25 14:28:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tamar Christina X-Patchwork-Id: 1371340 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=f/OnIYJ4; dkim=pass (1024-bit key) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-armh-onmicrosoft-com header.b=f/OnIYJ4; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByZ6Z2TKYz9sR4 for ; Sat, 26 Sep 2020 00:29:10 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7979B398EC1D; Fri, 25 Sep 2020 14:29:08 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10041.outbound.protection.outlook.com [40.107.1.41]) by sourceware.org (Postfix) with ESMTPS id B5D11398B400 for ; Fri, 25 Sep 2020 14:29:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org B5D11398B400 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Tamar.Christina@arm.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lymhjkwZcYPRiQgmuc9g76ix/RB5yMmoo/YZKYhDGWY=; b=f/OnIYJ4mCBCFadYicntkoGk8djoy+EX7iQOyEX68cQtEWqm5vy8lXRfTlLnyDZdrgnNnCuySEC9sc3gpzysdAoSYi6JfHJ9uJ+DXc0Nv+MVSqrkFEzNO1DI8WVaSd+of49MvX4UcWgr64csrYO5/eqnpu2Ofav1V/1yX/x7G+Y= Received: from DB3PR08CA0028.eurprd08.prod.outlook.com (2603:10a6:8::41) by AM0PR08MB4481.eurprd08.prod.outlook.com (2603:10a6:208:148::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.22; Fri, 25 Sep 2020 14:29:02 +0000 Received: from DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com (2603:10a6:8:0:cafe::52) by DB3PR08CA0028.outlook.office365.com (2603:10a6:8::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21 via Frontend Transport; Fri, 25 Sep 2020 14:29:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; gcc.gnu.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;gcc.gnu.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT055.mail.protection.outlook.com (10.152.21.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21 via Frontend Transport; Fri, 25 Sep 2020 14:29:02 +0000 Received: ("Tessian outbound a0bffebca527:v64"); Fri, 25 Sep 2020 14:29:02 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: d82809f173d0db63 X-CR-MTA-TID: 64aa7808 Received: from e561700e7cf0.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C5D29E95-2C2A-400C-B318-53C6E7CB3659.1; Fri, 25 Sep 2020 14:28:48 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id e561700e7cf0.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 25 Sep 2020 14:28:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=csI/iRRJJpXznvEg1hvzZ5qCSjw9J9jjZgguSvNRYSJJCbFnrc0gaG3gjgHh8SE15n7+bBFwRRPmdxftocVdlgP3iR6MSVGL8tjZZ81o+tiq+fToGjxbK+zGLuzmMQavBcPNYiBeWM9cfsj64KZYk+ADUx5KZf8rLUs5N0cXZW/1Dqx3GzZs1fWPAZKLWvZqT7Ojj3k7vZolLpNtcWcWDvFMNwzwMScQER3WZq8ZoyLQnLUN7n4SjH51pv9a/5sU3KTYMM7zeswbK7Dj0QLdYQktcdl5h9IxU1HYeWVr38cUvROqhG11i1E01LfOLLCBvvhw8P32f4MlTNqQbPDA0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lymhjkwZcYPRiQgmuc9g76ix/RB5yMmoo/YZKYhDGWY=; b=Og2t+qbhmI0IgwqponLbM5I4xLNnMFLQBy6uE0gLaULO5qAnx+14eLNeNXsc1VYOttR+rvZ2++zr1Buh7S1URg9tv6yDJSGc3yKomYR8FD6Fo19Zhd/LJkAOJa21g6zUBFnFsRKLrTxJ9Vlpe/5dN7RCLJLGIRWq3E1lLa2iM4inM3dzu+ADzSTpZnqKF9TxIaynLd//tgvXIF39NJIAHixqRs21QUlWlryWpWCffRNf8HUnf4fuZR0dEbJ9fa0C1FYhHkZJWY7AlXt0aOR3AMF/qoLm3yBObyPW9TrynLUmni09nVFIbF0wQusHTcQQK5CS67ZYWOLLi3UcQ63eIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lymhjkwZcYPRiQgmuc9g76ix/RB5yMmoo/YZKYhDGWY=; b=f/OnIYJ4mCBCFadYicntkoGk8djoy+EX7iQOyEX68cQtEWqm5vy8lXRfTlLnyDZdrgnNnCuySEC9sc3gpzysdAoSYi6JfHJ9uJ+DXc0Nv+MVSqrkFEzNO1DI8WVaSd+of49MvX4UcWgr64csrYO5/eqnpu2Ofav1V/1yX/x7G+Y= Authentication-Results-Original: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; Received: from VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) by VE1PR08MB5678.eurprd08.prod.outlook.com (2603:10a6:800:1a0::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.11; Fri, 25 Sep 2020 14:28:47 +0000 Received: from VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::d0e7:49cd:4dae:a2a2]) by VI1PR08MB5325.eurprd08.prod.outlook.com ([fe80::d0e7:49cd:4dae:a2a2%7]) with mapi id 15.20.3412.024; Fri, 25 Sep 2020 14:28:47 +0000 Date: Fri, 25 Sep 2020 15:28:39 +0100 From: Tamar Christina To: gcc-patches@gcc.gnu.org Subject: [PATCH v2 5/16]middle-end: Add shared machinery for matching patterns involving complex numbers. Message-ID: <20200925142837.GA16579@arm.com> Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: SN4PR0201CA0048.namprd02.prod.outlook.com (2603:10b6:803:2e::34) To VI1PR08MB5325.eurprd08.prod.outlook.com (2603:10a6:803:13e::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from arm.com (217.140.106.53) by SN4PR0201CA0048.namprd02.prod.outlook.com (2603:10b6:803:2e::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.20 via Frontend Transport; Fri, 25 Sep 2020 14:28:45 +0000 X-Originating-IP: [217.140.106.53] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 556acdbc-4fa3-4680-fcac-08d8615f597d X-MS-TrafficTypeDiagnostic: VE1PR08MB5678:|AM0PR08MB4481: X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true NoDisclaimer: true X-MS-Oob-TLC-OOBClassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: AyxKyccSqNEfXlh/zdB8orlbHSyIrwgOf2rHZk5Ajj0SVy88jC9FESHflvw4EPtICh0JI0LMkwcgkvW8QhNNfxBAA9BkYOkvBeUm9s+gPxQgf21JOWBL5rsbaIfcuJs1vyKAWzQVLwNK9qCrNJIp/G63g8ELzSpHfdg1L3zNgaEIMnPk6aPNsbYPkzlErLOe5OQlDp6It1jgFJY1cENiICHQlHDFLwgXZ1/onoK4z94FI1mZaLZDFib482nNCXQYZ3ALy3Yp91CpnucFWtL4sPxhipF/A7PovSAoE/HarVDFkjVoA0R9iLaQjb6HW2ru8B9efH/lMa9gyJbUHilCjYEncEBk2uKKGKnPbqDmG88AMp09q/6SVyHKZBvnOVeigs4JDjVywwMCpMqNCKqU+GDViZ2pOO0ptMSiXjrQr9w= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5325.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(136003)(39860400002)(346002)(376002)(8936002)(1076003)(55016002)(33656002)(8886007)(66946007)(66556008)(86362001)(66616009)(83380400001)(36756003)(66476007)(478600001)(6666004)(6916009)(235185007)(44144004)(5660300002)(16526019)(186003)(4743002)(4326008)(33964004)(52116002)(44832011)(7696005)(316002)(2616005)(26005)(2906002)(8676002)(956004)(2700100001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: d3F8WBc2lkdc524+8g5kITFNRiwx87atKpH967wqVznEHb9FpotTH1LzquUtlHFCltXnsuu2bnIXYzHJVIWSt7eml+GtN0ZpF6a8ZpWrgPy5EFBb27ruet1Manlb4jdSHCya8csEjFfuQsEO75yRU7eoMOUwkru3MSWse55Y8yw+h7L70nyVRb6vBxcZ+0poTsGL90dR5S2KRhaI6SDyct/9+2AmnAdEocJHT2SWuX2RtzokdzbP3oqXyp0I8jjGFdx9uF68PWMeb9TBj6Wc8A1VltHk3Dp56Xt152UFVzSIsewsJOpUtmOShZwBqGwxUeQAkEr0gjyCnLDa58epHojljUZt2MxpyEYo9qVLQlsOe2RAZvBz4TenumL6AB3DEPkYFz5kpQciSMBdvGboCcuvyU0Eeu4ZHERUcUScEKstb/0QCpIntvSFNjTUnm3qTdXhN8tbRkDFG0urHV2F0WSzku0UfBSoFKQJkHNZOQ68GX+NrkdBbcKEJWmru4chrPdkdM+NRiBgVdz9Wumr0tkN+ZmmZY0fTIynXrCaO8I/Uzg6G5yMFbcWZmcT7LAqR0htToOmPmbZa+AKjIXqpnqO99gkh+8v/mEK7hQXepDUeeS9JWp7azmhGqMcrKYiYOLQyd8+IKlwgSQRQTOTRQ== X-MS-Exchange-Transport-Forked: True X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5678 Original-Authentication-Results: gcc.gnu.org; dkim=none (message not signed) header.d=none;gcc.gnu.org; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 1585d413-586d-4893-f554-08d8615f500e X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JPbpHjGV073B0PX/JP7K1VBTpJ95TxTvUi3rx3jNMHtVa5HUtdw6Z0xwRFg5IAJOp8/QYMpDGDdnBa7MIZmvf4TiBspUi7D4+h2khll0oVe+qe4glOAO5aZ1VV35pXzhi+R6pfgSEgjfBvQJJMLT32wJ4mbk3LkGGkJ7ffDscWuTS3zMz+VUr8KC2adtVD7rW1VCG+7Q8T7ATlETLnf4gpNWYLR4ddhAyVHx0uPd+mZe/rbk6Hbg57ilCIDrtPYDpPOx3EhlPYcyCTo30KHzWOQcDfk9NRikTSFZDFZ7OMv7e9CgBEnu6m6iFHCZrmioJvObDXM257Xi4wNysoJkD2YhuBWE4+Cdb6JbmYB/ufbrPLkuxnp7m7uJqUY/VKYnnwX1KtpcLyWtvs14RLk+OiJMXy9AG0CpO+UkjU5zi0A= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(396003)(39860400002)(136003)(376002)(346002)(46966005)(8936002)(2906002)(33656002)(356005)(81166007)(70206006)(956004)(316002)(2616005)(83380400001)(82740400003)(82310400003)(47076004)(478600001)(44832011)(186003)(7696005)(4326008)(86362001)(36756003)(16526019)(70586007)(8886007)(55016002)(6666004)(44144004)(336012)(26005)(8676002)(6916009)(4743002)(66616009)(1076003)(33964004)(235185007)(5660300002)(2700100001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Sep 2020 14:29:02.4061 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 556acdbc-4fa3-4680-fcac-08d8615f597d X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB4481 X-Spam-Status: No, score=-14.1 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, MSGID_FROM_MTA_HEADER, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: nd@arm.com, rguenther@suse.de, ook@ucw.cz Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Hi All, This patch adds shared machinery for detecting patterns having to do with complex number operations. The class ComplexPattern provides helpers for matching and ultimately undoing the permutation in the tree by rebuilding the graph. Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. Ok for master? Thanks, Tamar gcc/ChangeLog: * tree-vect-slp-patterns.c (complex_operation_t,class ComplexPattern): New. diff --git a/gcc/tree-vect-slp-patterns.c b/gcc/tree-vect-slp-patterns.c index f605f68d2a14c4bf4941f97b7c1d57f6acb5ffb1..6453a5b1b6464dba833adc2c2a194db5e712bb79 100644 --- a/gcc/tree-vect-slp-patterns.c +++ b/gcc/tree-vect-slp-patterns.c @@ -134,6 +134,19 @@ along with GCC; see the file COPYING3. If not see To add a new pattern, implement the VectPattern class and add the type to slp_patterns. */ +/* The COMPLEX_OPERATION enum denotes the possible pair of operations that can + be matched when looking for expressions that we are interested matching for + complex numbers addition and mla. */ + +typedef enum _complex_operation { + PLUS_PLUS, + MINUS_PLUS, + PLUS_MINUS, + MULT_MULT, + NEG_NEG, + CMPLX_NONE +} complex_operation_t; + /* VectSimplePatternMatch holds contextual information about a single match found in the SLP tree. The use of the class is to allow you to defer performing any modifications to the SLP tree until they are to be done. By @@ -298,6 +311,358 @@ class VectSimplePatternMatch : public VectPatternMatch } }; +/* The ComplexPattern class contains common code for pattern matchers that work + on complex numbers. These provide functionality to allow de-construction and + validation of sequences depicting/transforming REAL and IMAG pairs. */ + +class ComplexPattern : public VectPattern +{ + protected: + /* Current list of arguments that were found during the current invocation + of the pattern matcher. */ + vec m_vects; + + /* Representative statement for the current match being performed. */ + stmt_vec_info m_stmt_info; + + /* A list of all arguments found between all invocations of the current + pattern matcher. */ + vec> m_defs; + + /* Checks to see of the expression EXPR is a gimple assign with code CODE + and if this is the case the two operands of EXPR is returned in OP1 and + OP2. + + If the matching and extraction is successful TRUE is returned otherwise + FALSE in which case the value of OP1 and OP2 will not have been touched. + */ + + bool + vect_match_expression_p (slp_tree node, tree_code code, int base, int idx, + stmt_vec_info *op1, stmt_vec_info *op2) + { + + vec scalar_stmts = SLP_TREE_SCALAR_STMTS (node); + + /* Calculate the index of the statement in the node to inspect. */ + int n = base + idx; + if (scalar_stmts.length () < (unsigned)n) // can use group_size + return false; + + gimple* expr = STMT_VINFO_STMT (scalar_stmts[n]); + if (!is_gimple_assign (expr) + || gimple_expr_code (expr) != code) + return false; + + vec children = SLP_TREE_CHILDREN (node); + + /* If it's a VEC_PERM_EXPR we need to look one deeper. VEC_PERM_EXPR + only have one entry. So pick on. */ + if (node->code == VEC_PERM_EXPR) + children = SLP_TREE_CHILDREN (children.last ()); + + if (children.length () != (op2 ? 2 : 1)) + return false; + + if (op1) + { + if (SLP_TREE_DEF_TYPE (children[0]) != vect_internal_def) + return false; + *op1 = SLP_TREE_SCALAR_STMTS (children[0])[n]; + } + + if (op2) + { + if (SLP_TREE_DEF_TYPE (children[1]) != vect_internal_def) + return false; + *op2 = SLP_TREE_SCALAR_STMTS (children[1])[n]; + } + + return true; + } + + /* This function will match two gimple expressions STMT_0 and STMT_1 in + parallel and returns the pair operation that represents the two + expressions in the two statements. The statements are located in NODE1 + and NODE2 at offset base + offset1 and base + offset2 respectively. + + If match is successful then the corresponding complex_operation is + returned and the arguments to the two matched operations are returned in + OPS. + + If unsuccessful then CMPLX_NONE is returned and OPS is untouched. + + e.g. the following gimple statements + + stmt 0 _39 = _37 + _12; + stmt 1 _6 = _38 - _36; + + will return PLUS_MINUS along with OPS containing {_37, _12, _38, _36}. + */ + + complex_operation_t + vect_detect_pair_op (int base, slp_tree node1, int offset1, slp_tree node2, + int offset2, vec *ops) + { + stmt_vec_info op1 = NULL, op2 = NULL, op3 = NULL, op4 = NULL; + complex_operation_t result = CMPLX_NONE; + #define CHECK_FOR(x, y, z) \ + (vect_match_expression_p (node1, x, base, offset1, &op1, \ + z ? &op2 : NULL) \ + && vect_match_expression_p (node2, y, base, offset2, &op3, \ + z ? &op4 : NULL)) + + if (CHECK_FOR (MINUS_EXPR, PLUS_EXPR, true)) + result = MINUS_PLUS; + else if (CHECK_FOR (PLUS_EXPR, MINUS_EXPR, true)) + result = PLUS_MINUS; + else if (CHECK_FOR (PLUS_EXPR, PLUS_EXPR, true)) + result = PLUS_PLUS; + else if (CHECK_FOR (MULT_EXPR, MULT_EXPR, true)) + result = MULT_MULT; + else if (CHECK_FOR (NEGATE_EXPR, NEGATE_EXPR, false)) + result = NEG_NEG; + #undef CHECK_FOR + + if (result != CMPLX_NONE && ops != NULL) + { + ops->create (4); + ops->quick_push (op1); + ops->quick_push (op2); + ops->quick_push (op3); + ops->quick_push (op4); + } + return result; + } + + /* Overload of vect_detect_pair_op where the statements are assumed to be + one after the other. This inspects node[base] and node[base+1]. */ + + complex_operation_t + vect_detect_pair_op (int base, slp_tree node, vec *ops) + { + return vect_detect_pair_op (base, node, 0, node, 1, ops); + } + + /* Create the intermediate states that are needed and generate a new match + object with the information. */ + + bool + store_results () + { + this->m_defs.safe_push (this->m_vects); + save_match (); + return true; + } + + /* This function marks every statement that is being replaced during the + the pattern matching as PURE. Normally when replacing a statement due + to a pattern we add the statement to the STMT_VINFO_PATTERN_DEF_SEQ of + the pattern that is replacing them. In this case however this won't + work as when doing the replacement we are changing the nodes that are + used by the statements. This means that when vectorized the SSA chain + is different than in the BB. + + Declaring the statements as part of the sequence will then cause SSA + verification to fail as we may refer to statements that were not in the + original USE-DEF chain of the statement we are replacing. + + The downside of this approach is that the statements will still be + seen as relevant and so we will still generate code for them and they + will be in the output, unconnected until DSE. We could mark them as + irrelevant, but that is only safe if there are no more uses of the node + in the SLP graph (So perhaps this should be done in free_slp_tree + instead of here. */ + + static void + vect_mark_stmts_as_in_pattern (hash_set *cache, vec orig, + slp_tree node) + { + if (cache->contains (node)) + return; + + unsigned i; + stmt_vec_info stmt_info; + slp_tree child; + + cache->add (node); + + FOR_EACH_VEC_ELT (SLP_TREE_SCALAR_STMTS (node), i, stmt_info) + { + if (gimple_assign_load_p (STMT_VINFO_STMT (stmt_info))) + return; + + STMT_SLP_TYPE (stmt_info) = pure_slp; + } + + FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (node), i, child) + vect_mark_stmts_as_in_pattern (cache, orig, child); + } + + protected: + ComplexPattern (slp_tree node, vec_info *vinfo) + : VectPattern (node, vinfo) + { } + + /* Create and store a new VectPatternMatch object with the current match + that was found. */ + + void save_match () + { + tree type = gimple_expr_type (STMT_VINFO_STMT (this->m_stmt_info)); + tree vectype = get_vectype_for_scalar_type (this->m_vinfo, type, + this->m_node); + VectPatternMatch *match + = new VectSimplePatternMatch (this->m_arity, this->m_defs.last (), + this->m_last_ifn, this->m_vinfo, + this->m_last_idx, this->m_node, type, + vectype, this->m_num_args); + this->m_matches.safe_push (match); + } + + public: + + /* Check to see if all loads rooted in ROOT are linear. Linearity is + defined as having no gaps between values loaded. */ + static bool + linear_loads_p (slp_tree root) + { + if (!root) + return false; + + unsigned i; + + if (SLP_TREE_LOAD_PERMUTATION (root).exists ()) + { + vec loads = SLP_TREE_LOAD_PERMUTATION (root); + unsigned leader = loads[0]; + unsigned load; + FOR_EACH_VEC_ELT_FROM (loads, i, load, 1) + if (load != ++leader) + return false; + } + + slp_tree child; + FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (root), i, child) + if (!linear_loads_p (child)) + return false; + + return true; + } + + /* The post transform and validation function for the complex number + patterns. This will re-arrange the tree and re-organize the nodes such + that they can be used by the complex number instructions that are to be + created. It does this by doing the following steps: + + 1) It looks up the definition nodes of each statement in DEFS which are + the new arguments to be used in the new patterns that will be created. + From this new SLP trees are created by calling vect_build_slp_tree + with the statements in the order we expect them to be. A majority of + these will be found in the cache and so this call will be fast. + + 2) After the new trees are created we check to see if all of them are + linear. If they are not linear we abort and undo the bookkeeping + information that vect_build_slp_tree created for them. + + 3) The children of NODE are replaced with the new set of nodes we + created. + + 4) The new sub-tree rooted in NODE are marked as relevant. + + This sequence of operations does an implicit re-ordering nodes. After + which DSE can remove the unused nodes. e.g. it will undo as much of the + permutes as it possibly can. This is required such that pattern + matchers running on the newly created statements match the correct + operations. */ + + bool validate_p (poly_uint64 *max_nunits, bool *matches, + unsigned *npermutes, unsigned *tree_size, + scalar_stmts_to_slp_tree_map_t * bst_map) + { + int group_size = SLP_TREE_SCALAR_STMTS (this->m_node).length (); + auto_vec nodes; + auto_vec stmts; + stmts.create (0); + stmts.safe_grow_cleared (group_size); + nodes.create (0); + nodes.safe_grow_cleared (this->m_num_args); + slp_tree tmp = NULL; + vec iters = SLP_TREE_SCALAR_STMTS (this->m_node); + + VectPatternMatch *match; + unsigned int i, count; + int idx = -1; + hash_set *visited = new hash_set (); + + for (idx = 0; idx < this->m_num_args; idx++) + { + count = 0; + FOR_EACH_VEC_ELT (this->m_matches, i, match) + { + vec def = this->m_defs[i]; + for (int x = 0; x < this->m_arity; x++) + { + stmt_vec_info op = def[idx + (x * this->m_num_args)]; + stmts[count++] = op; + } + } + + /* We need top copy the statements in case the node is not in the + cache. But if it is in the cache we leak? */ + vec new_stmts = stmts.copy (); + tmp = vect_build_slp_tree (this->m_vinfo, new_stmts, group_size, + max_nunits, matches, npermutes, tree_size, + bst_map); + + gimple *info + = STMT_VINFO_STMT (SLP_TREE_REPRESENTATIVE (this->m_node)); + if (!tmp) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "Could not build new SLP tree for %G\n", info); + + goto graceful_exit; + } + + nodes[idx] = tmp; + visited->add (tmp); + + if (!linear_loads_p (tmp)) + { + if (dump_enabled_p ()) + dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, + "Loads could not be made linear %G\n", info); + + goto graceful_exit; + } + } + + /* Mark all statements that are unused between the new and old nodes as in + a pattern. */ + FOR_EACH_VEC_ELT (SLP_TREE_CHILDREN (this->m_node), i, tmp) + vect_mark_stmts_as_in_pattern (visited, iters, tmp); + + delete visited; + + SLP_TREE_CHILDREN (this->m_node).truncate (0); + SLP_TREE_CHILDREN (this->m_node).safe_splice (nodes); + + return true; + +graceful_exit: + + delete visited; + + FOR_EACH_VEC_ELT (nodes, i, tmp) + if (tmp) + vect_free_slp_tree (tmp); + + return false; + } +}; + #define SLP_PATTERN(x) &x::create VectPatternDecl slp_patterns[] {