{"id":2198181,"url":"http://patchwork.ozlabs.org/api/1.0/patches/2198181/?format=json","project":{"id":17,"url":"http://patchwork.ozlabs.org/api/1.0/projects/17/?format=json","name":"GNU Compiler Collection","link_name":"gcc","list_id":"gcc-patches.gcc.gnu.org","list_email":"gcc-patches@gcc.gnu.org","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"","date":"2026-02-19T14:02:03","name":"[v2,2/7,Vectorizer] : SLP MATCH Pattern: Identify pattern and build the new SLP nodes","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"4ddabe00cf16adb3814a09fffe7ab74878559593","submitter":{"id":92460,"url":"http://patchwork.ozlabs.org/api/1.0/people/92460/?format=json","name":"Andrei Tirziu via Sourceware Forge","email":"forge-bot+why@forge-stage.sourceware.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/gcc/patch/bmm.hfy4c1w8xq.gcc.gcc-TEST.why.135.2.2@forge-stage.sourceware.org/mbox/","series":[{"id":492684,"url":"http://patchwork.ozlabs.org/api/1.0/series/492684/?format=json","date":"2026-02-19T14:02:02","name":"Vectorizer: New SLP Pattern","version":2,"mbox":"http://patchwork.ozlabs.org/series/492684/mbox/"}],"check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2198181/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","gcc-patches@gcc.gnu.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","gcc-patches@gcc.gnu.org"],"Authentication-Results":["legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org\n (client-ip=2620:52:6:3111::32; helo=vm01.sourceware.org;\n envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org;\n receiver=patchwork.ozlabs.org)","sourceware.org; dmarc=none (p=none dis=none)\n header.from=forge-stage.sourceware.org","sourceware.org;\n spf=pass smtp.mailfrom=forge-stage.sourceware.org","server2.sourceware.org;\n arc=none smtp.remote-ip=38.145.34.39"],"Received":["from vm01.sourceware.org (vm01.sourceware.org\n [IPv6:2620:52:6:3111::32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fGw9Q2cJKz1xvg\n\tfor ; Fri, 20 Feb 2026 01:04:26 +1100 (AEDT)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id 08BB74BAD155\n\tfor ; Thu, 19 Feb 2026 14:04:24 +0000 (GMT)","from forge-stage.sourceware.org (vm08.sourceware.org [38.145.34.39])\n by sourceware.org (Postfix) with ESMTPS id CCF064B9DB48\n for ; Thu, 19 Feb 2026 14:02:36 +0000 (GMT)","from forge-stage.sourceware.org (localhost [IPv6:::1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange x25519 server-signature ECDSA (prime256v1) server-digest SHA256)\n (No client certificate requested)\n by forge-stage.sourceware.org (Postfix) with ESMTPS id 9A1334422F;\n Thu, 19 Feb 2026 14:02:36 +0000 (UTC)"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org 08BB74BAD155","OpenDKIM Filter v2.11.0 sourceware.org CCF064B9DB48"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org CCF064B9DB48","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org CCF064B9DB48","ARC-Seal":"i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1771509756; cv=none;\n b=m2Ha+mIS6aY1zR7DTQKBG8iWTyQ7+HrcLAJ3Men789l/9shG75FWOfq+5zIIyqrv5or7S4K31/IpwV9WsCjTwfPObsLoXvxN3nqE1lzL+CooOMZGhDyDKR+ixnNBs/dWyJaMp1que/KpJUd/ceVTbZICSeUd1TfAHRFftqrec/U=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1771509756; c=relaxed/simple;\n bh=tRh+V/kh+ftG0V02xCS+hnqlbKcoJsIeFT0x1edZ2i8=;\n h=From:Date:Subject:To:Message-ID;\n b=Bu465yBE1G+J4i7w5LaKaw1zM8yaECWMPI0yh+511xGbb7XeuuqmOqRmZF814bmtyBY88BANuO7F1zuBqIyOljjeBfl0u+7zGyYxJaxybujh+92PUnkj794KX/xR9vLXEKmoPjk/WtmA6qmjGfFKq6DfbJ9Kuq3sxSnPQRV6AWw=","ARC-Authentication-Results":"i=1; server2.sourceware.org","From":"Andrei Tirziu via Sourceware Forge\n ","Date":"Thu, 19 Feb 2026 14:02:03 +0000","Subject":"[PATCH v2 2/7] [Vectorizer]: SLP MATCH Pattern: Identify pattern and\n build the new SLP nodes","To":"gcc-patches mailing list ","Cc":"Tamar Christina ,\n Victor Do Nascimento ","Message-ID":"\n ","X-Mailer":"batrachomyomachia","X-Pull-Request-Organization":"gcc","X-Pull-Request-Repository":"gcc-TEST","X-Pull-Request":"https://forge.sourceware.org/gcc/gcc-TEST/pulls/135","References":"\n ","In-Reply-To":"\n ","X-Patch-URL":"\n https://forge.sourceware.org/why/gcc/commit/8897b688945729b9c45bdcde5ddbed9ffbde6156","X-BeenThere":"gcc-patches@gcc.gnu.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Gcc-patches mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Reply-To":"gcc-patches mailing list ,\n Tamar Christina ,\n Victor Do Nascimento ,\n andreinichita.tirziu@arm.com","Errors-To":"gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org"},"content":"From: Andrei Nichita Tirziu \n\nThe new SLP pattern identifies a MATCH pattern:\n - OR-chain (for equality comparisons disjunctions): `(p0) || (p1) || ...`,\n leaves are `EQ_EXPR`.\n - AND-chain (for inequality comparisons conjunctions): `(q0) && (q1) && ...`,\n leaves are `NE_EXPR`.\n\nFor the pattern to be recognized, the leaves (also called \"predicates\")\nmust be formed of a common variant and some invariants.\n\nExample (using an array - the variant, and 3 invariants -\nthe variables `x`, `y`, `z`):\n for (int i = 0; i < n; i++)\n {\n if (a[i] == x || a[i] == y || a[i] == z)\n {\n ...\n }\n }\n\nIf such a pattern is identified, a replacement is also created for it, such\nthat the GIMPLE expression used by the if-statement, of the form\n exp = (a[i] == x || a[i] == y || a[i] == z)\nis rewritten as (the actual code is slightly different, but this exemplifies\nthe main idea):\n exp = IFN_MATCH_ANY_FROM (a[i], x, y, z)\n\nUsing this new node, the vectorizer is able to transform the loop to use\nan SVE MATCH instruction instead of the intitial comparisons.\n\nThe pattern is enabled through the subsequent commits.\n\ngcc/ChangeLog:\n\n\t* tree-vect-slp-patterns.cc: New pattern.\n---\n gcc/tree-vect-slp-patterns.cc | 561 +++++++++++++++++++++++++++++++++-\n 1 file changed, 560 insertions(+), 1 deletion(-)","diff":"diff --git a/gcc/tree-vect-slp-patterns.cc b/gcc/tree-vect-slp-patterns.cc\nindex f30b1eb91a35..ae3e62750f3a 100644\n--- a/gcc/tree-vect-slp-patterns.cc\n+++ b/gcc/tree-vect-slp-patterns.cc\n@@ -42,6 +42,7 @@ along with GCC; see the file COPYING3. If not see\n #include \"vec-perm-indices.h\"\n #include \"gimple-fold.h\"\n #include \"internal-fn.h\"\n+#include \n \n /* SLP Pattern matching mechanism.\n \n@@ -1705,6 +1706,564 @@ addsub_pattern::build (vec_info *vinfo)\n }\n }\n \n+/**\n+ * Recognize and replace a \"match-any equality\" predicate with an IFN.\n+ *\n+ * Detects the idiom:\n+ * - Disjunction of equalities (OR-chain):\n+ * `(a == c0) || (a == c1) || ... || (a == ck)`\n+ *\t (the classic \"a equals any of the set {c0..ck}\")\n+ * - Conjuction of inequalities (AND-chain):\n+ * `(a != c0) && (a != c1) && ... && (a != ck)`\n+ *\n+ * This pattern is useful for loops that contain:\n+ * if (x[i] == c0 || x[i] == c1 || ...) { }\n+ *\n+ * Throughout the comments and code, we use:\n+ * - `leaves` or `terms` to denote the structures similar to (a == c0);\n+ * - `operand` to refer to `x[i]` or `{c0...ck}`;\n+ * - `variant operand` for `x[i]`; this is the variant operand within a single\n+ * iteration;\n+ * - `invariant operand` for `{c0...ck}`, which are invariant with respect to\n+ * the loop.\n+ */\n+class match_pattern : public vect_pattern\n+{\n+private:\n+ std::vector all_terms {};\n+\n+public:\n+ match_pattern (slp_tree* node, internal_fn ifn, std::vector& terms)\n+\t\t : vect_pattern (node, NULL, ifn), all_terms (terms) {};\n+\n+ /**\n+ * Build the replacement statement.\n+ *\n+ * @param vinfo Vectorizer context.\n+ */\n+ void build (vec_info *vinfo) final override;\n+\n+ /**\n+ * Check the current SLP node if it matches the MATCH pattern\n+ * (see class description for more details).\n+ *\n+ * @param slp_tree node The SLP node we are analyzing.\n+ *\n+ * @return An instance of the `match_pattern` class if the pattern\n+ *\t was detected, or NULL otherwise.\n+ */\n+ static vect_pattern* recognize (slp_tree_to_load_perm_map_t*,\n+\t\t\t\t slp_compat_nodes_map_t*,\n+\t\t\t\t slp_tree* node);\n+\n+ /**\n+ * Identify the root SLP node representing an OR/AND predicate chain\n+ * and collect all its leaf predicate SLP nodes.\n+ *\n+ * Typical intended shapes:\n+ * - OR-chain (for equality disjunctions): `(p0) || (p1) || ...`,\n+ * leaves are `EQ_EXPR`.\n+ * - AND-chain (for inequality conjunctions): `(q0) && (q1) && ...`,\n+ * leaves are `NE_EXPR`.\n+ *\n+ * If we have for example this GIMPLE sequence of statements:\n+ * _1 = x[i]\n+ * _2 = ( _1 == a )\n+ * _3 = ( _1 == b )\n+ * _4 = ( _1 == c )\n+ * _5 = ( _3 || _4 )\n+ * _6 = ( _2 || _4 )\n+ *\n+ * The leaves (sometimes called the predicates) correspond\n+ * to `_2`, `_3`, and `_4`.\n+ * In the SLP form, this function locates those comparison nodes within the\n+ * SLP graph and records their corresponding SLP nodes as the collected terms.\n+ *\n+ * @param slp_node The SLP node we are analyzing.\n+ * @param terms Output vector that receives all discovered predicate-leaf\n+ *\t\t SLP nodes (`EQ_EXPR` or `NE_EXPR`).\n+ *\n+ * @return , if `slp_node` denotes the root of an\n+ *\t OR/AND chain, oterwise.\n+ */\n+ static std::pair\n+ identify_and_collect_match_terms (slp_tree* slp_node,\n+\t\t\t\t std::vector& terms);\n+\n+ /**\n+ * Perform a depth-first walk over an SLP subtree to collect the leaf\n+ * SLP nodes corresponding to predicate comparisons (EQ or NE) that form\n+ * an OR/AND chain.\n+ *\n+ * @param node\t Root SLP node representing the top OR/AND expression.\n+ * @param collected_terms Vector to append the predicate leaf SLP nodes to.\n+ * @param is_or_chain When true, collect EQ leaves under ORs, otherwise\n+ *\t\t\t collect NE leaves under ANDs.\n+ */\n+ static bool collect_match_terms (slp_tree node,\n+\t\t\t\t std::vector& collected_terms,\n+\t\t\t\t bool is_or_chain);\n+\n+ /**\n+ * Given a boolean SSA name `pred_ssa` that is produced by some statement,\n+ * check whether any of its immediate uses is itself an OR/AND-producing\n+ * GIMPLE_ASSIGN. If so, this `pred_ssa` has a parent in the OR/AND chain.\n+ *\n+ * @param pred_ssa\t Boolean SSA name to inspect.\n+ * @param look_for_or_chain If true, look for OR parents; else\n+ *\t\t\t look for AND parents.\n+ *\n+ * @return True if an OR/AND consumer exists; False otherwise.\n+ */\n+ static bool has_or_and_parent (tree pred_ssa, bool look_for_or_chain);\n+\n+private:\n+ /**\n+ * Determine whether an operand is invariant within the current\n+ * vectorization region.\n+ *\n+ * @param vinfo Vectorization context.\n+ * @param term Operand to classify.\n+ *\n+ * @return True if `term` is invariant, False otherwise.\n+ */\n+ bool is_term_invariant_operand_inside_for_loop (vec_info* vinfo, tree term);\n+\n+ /**\n+ * Extract the common (variant) operand and the unique invariant candidates\n+ * from a set of SLP leaf nodes.\n+ *\n+ * @param vinfo\t Vectorization context.\n+ * @param terms\t Vector of SLP leaf nodes representing EQ/NE\n+ *\t\t\t comparisons collected from an\n+ *\t\t\t OR/AND predicate chain.\n+ * @param variant_operand Shared non-invariant SLP operand present in\n+ *\t\t\t all predicate leaves.\n+ * @param invariant_operands List of SLP operands that are invariant within\n+ *\t\t\t the loop and distinct across leaves.\n+ *\n+ * @return True, if the common variant exists and the other terms are\n+ *\t invariant, False otherwise.\n+ */\n+ bool identify_match_operands_from_terms (vec_info *vinfo,\n+\t\t\t\t\t const std::vector& terms,\n+\t\t\t\t\t slp_tree& variant_operand,\n+\t\t\t\t\t std::vector& invariant_operands);\n+};\n+\n+void match_pattern::build (vec_info *vinfo)\n+{\n+ stmt_vec_info current_vec_info = SLP_TREE_REPRESENTATIVE (*m_node);\n+ const gimple* current_stmt = STMT_VINFO_STMT (current_vec_info);\n+\n+ // Identify the unique invariant operands.\n+ std::vector invariant_operands {};\n+\n+ // The non-invariant side that must be the same across all leaves.\n+ slp_tree variant_operand = nullptr;\n+\n+ // Collect all operands.\n+ if (!identify_match_operands_from_terms (vinfo, all_terms,\n+\t\t\t\t\t variant_operand, invariant_operands))\n+ {\n+ return;\n+ }\n+\n+ // Only at this point we know that the pattern fully matches.\n+ if (dump_enabled_p ())\n+ dump_printf_loc (MSG_NOTE, vect_location,\n+\t\t \"Found SLP MATCH pattern starting from statement %G\",\n+\t\t current_stmt);\n+\n+ // Check if the backend supports a direct optab for this internal function.\n+ if (!direct_internal_fn_supported_p (m_ifn, SLP_TREE_VECTYPE (variant_operand),\n+ OPTIMIZE_FOR_SPEED))\n+ {\n+ if (dump_enabled_p ())\n+ dump_printf_loc (MSG_NOTE, vect_location,\n+\t\t \"SLP MATCH pattern: %s is not supported by an optab. \"\n+\t\t \"Aborting transformation\\n\",\n+\t\t internal_fn_name (m_ifn));\n+\n+ return;\n+ }\n+\n+ // Create a new SLP node to hold all invariants as scalar operands.\n+ slp_tree invariant_node = vect_create_new_slp_node (0, CONSTRUCTOR);\n+ SLP_TREE_DEF_TYPE (invariant_node) = vect_external_def;\n+ SLP_TREE_VECTYPE (invariant_node) = SLP_TREE_VECTYPE (variant_operand);\n+ SLP_TREE_REPRESENTATIVE (invariant_node) = nullptr;\n+\n+ /* At this point, we know that we have a vector mode and we are\n+ interested in how many elements we can pack in a segment\n+ (usually 128 bits long). */\n+ const unsigned int lanes_in_segment =\n+\tGET_MODE_NUNITS (vinfo->vector_mode).coeffs[0];\n+\n+ // We can't fit more invariants than the number of lanes in a segment.\n+ if (lanes_in_segment < invariant_operands.size ())\n+ {\n+ if (dump_enabled_p ())\n+ dump_printf_loc (MSG_NOTE, vect_location,\n+\t\t \"SLP MATCH pattern: Too many invariants (got %d \"\n+\t\t \"invariants, maximum for the current data type was %d).\"\n+\t\t \"Aborting transformation\\n\",\n+\t\t invariant_operands.size (),\n+\t\t lanes_in_segment);\n+\n+ return;\n+ }\n+\n+ const std::size_t number_invariants = invariant_operands.size ();\n+\n+ /* Most likely, the number of invariants won't match the number of lanes,\n+ so we have to fill the lanes in a circular way.\n+ The lanes can't just be left empty, since we would then match with zero,\n+ which isn't necessarily among our invariants. */\n+ for (unsigned int i = 0; i < lanes_in_segment; i++)\n+ {\n+ const tree scalar_invariant =\n+\tSLP_TREE_SCALAR_OPS (invariant_operands[i % number_invariants])[0];\n+ SLP_TREE_SCALAR_OPS (invariant_node).safe_push (scalar_invariant);\n+ }\n+\n+ /* We need a value here, that ensures that the\n+ scalar operands end up vectorized. */\n+ SLP_TREE_LANES (invariant_node) = 1;\n+\n+ /* Build a dummy IFN (the IFN will get its proper arguments\n+ in vectorizable_call). */\n+ auto_vec ifn_args {};\n+ tree dummy_arg_type = TREE_TYPE (gimple_get_lhs (current_stmt));\n+ tree dummy_arg = fold_convert (dummy_arg_type, integer_zero_node);\n+\n+ ifn_args.safe_push (dummy_arg);\n+ ifn_args.safe_push (dummy_arg);\n+\n+ gcall *ifn_call = gimple_build_call_internal_vec (m_ifn, ifn_args);\n+\n+ // Create an SSA LHS of the vector mask type and attach it to the call.\n+ const tree lhs = make_temp_ssa_name (SLP_TREE_VECTYPE (*m_node),\n+ ifn_call,\n+ \"match_temp_ifn\");\n+ gimple_call_set_lhs (ifn_call, lhs);\n+ gimple_set_location (ifn_call, gimple_location (current_stmt));\n+ gimple_call_set_nothrow (ifn_call, true);\n+ gimple_set_bb (ifn_call, gimple_bb (current_stmt));\n+\n+ // Create new vector info for the newly created IFN call.\n+ stmt_vec_info new_vec_info = vinfo->add_pattern_stmt (ifn_call,\n+\t\t\t\t\t\t\tcurrent_vec_info);\n+ STMT_VINFO_RELEVANT (new_vec_info) = vect_used_in_scope;\n+ STMT_SLP_TYPE (new_vec_info) = pure_slp;\n+ STMT_VINFO_VECTYPE (new_vec_info) = SLP_TREE_VECTYPE (*m_node);\n+ STMT_VINFO_SLP_VECT_ONLY_PATTERN (new_vec_info) = true;\n+\n+ /* We'll replace the node's children with the variant operand node\n+ and the newly created invariants node. */\n+ SLP_TREE_CHILDREN (*m_node).release ();\n+\n+ SLP_TREE_CHILDREN (*m_node).safe_push (variant_operand);\n+ SLP_TREE_REF_COUNT (variant_operand)++;\n+\n+ SLP_TREE_CHILDREN (*m_node).safe_push (invariant_node);\n+ SLP_TREE_REF_COUNT (invariant_node)++;\n+\n+ if (dump_enabled_p ())\n+ dump_printf_loc (MSG_NOTE, vect_location,\n+\t\t \"SLP MATCH pattern: Successfully built replacement for \"\n+\t\t \"statement %G\",\n+\t\t current_stmt);\n+\n+ SLP_TREE_REPRESENTATIVE (*m_node) = new_vec_info;\n+ SLP_TREE_CODE (*m_node) = CALL_EXPR;\n+}\n+\n+vect_pattern*\n+match_pattern::recognize (slp_tree_to_load_perm_map_t*, slp_compat_nodes_map_t*,\n+\t\t\t slp_tree* node)\n+{\n+ /* Collect leaf predicate SSA names that are defined by EQ or NE leaves\n+ across all lanes. */\n+ std::vector terms {};\n+\n+ auto [match_success, is_or_chain] =\n+\tidentify_and_collect_match_terms (node, terms);\n+ if (!match_success)\n+ return nullptr;\n+\n+ return new match_pattern (node,\n+\t\t\t is_or_chain ? IFN_MATCH_ANY_FROM : IFN_MATCH_NONE_FROM,\n+\t\t\t terms);\n+}\n+\n+std::pair\n+match_pattern::identify_and_collect_match_terms (slp_tree* slp_node,\n+\t\t\t\t\t\t std::vector& terms)\n+{\n+ // VecInfo of representative statement of the SLP node.\n+ const stmt_vec_info vec_info = SLP_TREE_REPRESENTATIVE (*slp_node);\n+ if (!vec_info)\n+ return {false, false};\n+\n+ // The representative scalar statement we're inspecting.\n+ const gimple *stmt = STMT_VINFO_STMT (vec_info);\n+ if (!stmt)\n+ return {false, false};\n+\n+ // If True, we have an OR chain, otherwise it is an AND chain.\n+ bool is_or_chain;\n+\n+ // We only start from an OR / AND-producing GIMPLE_ASSIGN statement.\n+ if (is_gimple_assign (stmt))\n+ {\n+ const tree_code stmt_code = gimple_assign_rhs_code (stmt);\n+\n+ // Only consider OR, AND nodes.\n+ if (stmt_code == TRUTH_OR_EXPR || stmt_code == BIT_IOR_EXPR)\n+ is_or_chain = true;\n+ else if (stmt_code == TRUTH_AND_EXPR || stmt_code == BIT_AND_EXPR)\n+ is_or_chain = false;\n+ else\n+ return {false, false};\n+\n+ // LHS is an SSA name of type bool, produced by this OR / AND node.\n+ const tree lhs = gimple_get_lhs (stmt);\n+\n+ /* Ensure this OR / AND node is the root of the chain\n+ (if this is an interior OR / AND, we will skip it). */\n+ if (has_or_and_parent (lhs, is_or_chain))\n+ return {false, false};\n+ }\n+ else\n+ return {false, false};\n+\n+ // Ensure the statement has the expected shape, and collect its leaves.\n+ if (!collect_match_terms (*slp_node, terms, is_or_chain))\n+ return {false, false};\n+\n+ // We need at least one leaf.\n+ if (terms.empty ())\n+ return {false, false};\n+\n+ return {true, is_or_chain};\n+}\n+\n+bool\n+match_pattern::collect_match_terms (slp_tree node,\n+\t\t\t\t std::vector& collected_terms,\n+\t\t\t\t bool is_or_chain)\n+{\n+ const stmt_vec_info vec_info = SLP_TREE_REPRESENTATIVE (node);\n+ if (!vec_info)\n+ return false;\n+\n+ const gimple *stmt = STMT_VINFO_STMT (vec_info);\n+ if (!is_gimple_assign (stmt))\n+ return false;\n+\n+ const tree lhs = gimple_get_lhs (stmt);\n+ /* We only expect SSA names of boolean types here\n+ (the result of an OR/AND/EQ/NE assign). */\n+ if (TREE_CODE (lhs) != SSA_NAME\n+ || TREE_CODE (TREE_TYPE (lhs)) != BOOLEAN_TYPE)\n+ return false;\n+\n+ if (is_or_chain)\n+ {\n+ switch (gimple_assign_rhs_code (stmt))\n+ {\n+ case TRUTH_OR_EXPR:\n+ case BIT_IOR_EXPR:\n+ {\n+ /* Internal OR node: recurse into both boolean children.\n+\t Expect exactly 2 children in the SLP tree for a binary op. */\n+ const auto& children = SLP_TREE_CHILDREN (node);\n+ if (children.length () != 2)\n+\treturn false;\n+\n+ return collect_match_terms (children[0], collected_terms, is_or_chain)\n+\t && collect_match_terms (children[1], collected_terms, is_or_chain);\n+ }\n+ case EQ_EXPR:\n+ // Leaf predicate for the OR case: equality comparison.\n+ collected_terms.push_back (node);\n+ return true;\n+ default:\n+ return false;\n+ }\n+ }\n+ else\n+ {\n+ switch (gimple_assign_rhs_code (stmt))\n+ {\n+ case TRUTH_AND_EXPR:\n+ case BIT_AND_EXPR:\n+ {\n+ /* Internal AND node: recurse into both boolean children.\n+\t Expect exactly 2 children in the SLP tree for a binary op. */\n+ const auto& children = SLP_TREE_CHILDREN (node);\n+ if (children.length () != 2)\n+\treturn false;\n+\n+ return collect_match_terms (children[0], collected_terms, is_or_chain)\n+\t && collect_match_terms (children[1], collected_terms, is_or_chain);\n+ }\n+ case NE_EXPR:\n+ // Leaf predicate for the AND case: inequality comparison.\n+ collected_terms.push_back (node);\n+ return true;\n+ default:\n+ return false;\n+ }\n+ }\n+}\n+\n+bool match_pattern::has_or_and_parent (tree pred_ssa, bool look_for_or_chain)\n+{\n+ imm_use_iterator it;\n+ use_operand_p use_p;\n+\n+ /* Iterate over all immediate consumers of this SSA name\n+ (look at the other GIMPLE statements that use this SSA variable). */\n+ FOR_EACH_IMM_USE_FAST (use_p, it, pred_ssa)\n+ {\n+ const gimple *u = USE_STMT (use_p);\n+\n+ // Only assignments can have an RHS code we care about.\n+ if (is_gimple_assign (u))\n+ {\n+ const tree_code c = gimple_assign_rhs_code (u);\n+\n+ if (look_for_or_chain)\n+ {\n+\tif (c == TRUTH_OR_EXPR || c == BIT_IOR_EXPR)\n+\t return true;\n+ }\n+ else\n+ {\n+\tif (c == TRUTH_AND_EXPR || c == BIT_AND_EXPR)\n+\t return true;\n+ }\n+ }\n+ }\n+\n+ return false;\n+}\n+\n+bool\n+match_pattern::is_term_invariant_operand_inside_for_loop (vec_info* vinfo,\n+\t\t\t\t\t\t\t tree term)\n+{\n+ // GIMPLE minimal invariants.\n+ if (is_gimple_min_invariant (term))\n+ return true;\n+\n+ if (TREE_CODE (term) != SSA_NAME)\n+ return false;\n+\n+ /* Will hold how the value enters the vectorization region:\n+\t- vect_constant_def : compile-time constant\n+\t- vect_external_def : defined outside the vectorized region\n+\t- vect_internal_def : defined inside (variant)\n+\t- vect_induction_def / vect_reduction_def / etc.: also variant for us\n+ */\n+ enum vect_def_type dt;\n+\n+ /* Ask the vectorizer to classify the use/definition of `term` relative\n+ to `vinfo`. */\n+ if (!vect_is_simple_use (term, vinfo, &dt))\n+ return false;\n+\n+ return dt == vect_constant_def || dt == vect_external_def;\n+}\n+\n+bool\n+match_pattern::identify_match_operands_from_terms (vec_info *vinfo,\n+\t\t\t\t\t\t const std::vector& terms,\n+\t\t\t\t\t\t slp_tree& variant_operand,\n+\t\t\t\t\t\t std::vector& invariant_operands)\n+{\n+ for (const slp_tree& term : terms)\n+ {\n+ const stmt_vec_info term_vec_info = SLP_TREE_REPRESENTATIVE (term);\n+ if (!term_vec_info)\n+ return false;\n+\n+ const gimple* term_node = STMT_VINFO_STMT (term_vec_info);\n+ if (!is_gimple_assign (term_node))\n+ return false;\n+\n+ const tree expr_lhs = gimple_assign_rhs1 (term_node);\n+ const tree expr_rhs = gimple_assign_rhs2 (term_node);\n+\n+ // Classify invariance for each side: we require exactly one invariant side.\n+ const bool is_lhs_invariant =\n+\tis_term_invariant_operand_inside_for_loop (vinfo, expr_lhs);\n+ const bool is_rhs_invariant =\n+\tis_term_invariant_operand_inside_for_loop (vinfo, expr_rhs);\n+\n+ if (is_lhs_invariant && is_rhs_invariant)\n+ return false;\n+\n+ if (!is_lhs_invariant && !is_rhs_invariant)\n+ return false;\n+\n+ auto &children = SLP_TREE_CHILDREN (term);\n+ if (children.length () != 2)\n+ return false;\n+\n+ const slp_tree lhs_node = children[0];\n+ const slp_tree rhs_node = children[1];\n+\n+ /* The invariants is designated as \"not common\", the variant is\n+ \"maybe common\". */\n+ const slp_tree maybe_common_op = is_lhs_invariant ? rhs_node : lhs_node;\n+ const slp_tree not_common_op = is_rhs_invariant ? rhs_node : lhs_node;\n+\n+ if (!variant_operand)\n+ {\n+ // First leaf establishes the \"common\" variant operand.\n+ variant_operand = maybe_common_op;\n+ invariant_operands.push_back (not_common_op);\n+ }\n+ else\n+ {\n+ auto is_same_slp_operand = [] (slp_tree a, slp_tree b) -> bool {\n+\t/* Prefer pointer equality, but fall back to scalar operand equality\n+\t if we have representatives. */\n+\tif (a == b)\n+\t return true;\n+\tif (a == nullptr || b == nullptr)\n+\t return false;\n+\tauto &a_ops = SLP_TREE_SCALAR_OPS (a);\n+\tauto &b_ops = SLP_TREE_SCALAR_OPS (b);\n+\tif (a_ops.is_empty () || b_ops.is_empty ())\n+\t return false;\n+\treturn operand_equal_p (a_ops[0], b_ops[0]);\n+ };\n+\n+ // Enforce that all leaves share the same variant \"common\" operand.\n+ if (!is_same_slp_operand (variant_operand, maybe_common_op))\n+\treturn false;\n+\n+ // Deduplicate the invariant candidate across leaves.\n+ bool operand_exists = false;\n+ for (const slp_tree &op : invariant_operands)\n+\tif (is_same_slp_operand (op, not_common_op))\n+\t{\n+\t operand_exists = true;\n+\t break;\n+\t}\n+\n+ if (!operand_exists)\n+\tinvariant_operands.push_back (not_common_op);\n+ }\n+ }\n+\n+ return true;\n+}\n+\n /*******************************************************************************\n * Pattern matching definitions\n ******************************************************************************/\n@@ -1717,7 +2276,7 @@ vect_pattern_decl_t slp_patterns[]\n overlap in what they can detect. */\n \n SLP_PATTERN (complex_operations_pattern),\n- SLP_PATTERN (addsub_pattern)\n+ SLP_PATTERN (match_pattern)\n };\n #undef SLP_PATTERN\n \n","prefixes":["v2","2/7","Vectorizer"]}