{"id":2198178,"url":"http://patchwork.ozlabs.org/api/1.0/covers/2198178/?format=json","project":{"id":17,"url":"http://patchwork.ozlabs.org/api/1.0/projects/17/?format=json","name":"GNU Compiler Collection","link_name":"gcc","list_id":"gcc-patches.gcc.gnu.org","list_email":"gcc-patches@gcc.gnu.org","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"","date":"2026-02-19T14:02:01","name":"[v2,0/7] Vectorizer: New SLP Pattern","submitter":{"id":92460,"url":"http://patchwork.ozlabs.org/api/1.0/people/92460/?format=json","name":"Andrei Tirziu via Sourceware Forge","email":"forge-bot+why@forge-stage.sourceware.org"},"series":[{"id":492684,"url":"http://patchwork.ozlabs.org/api/1.0/series/492684/?format=json","date":"2026-02-19T14:02:02","name":"Vectorizer: New SLP Pattern","version":2,"mbox":"http://patchwork.ozlabs.org/series/492684/mbox/"}],"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","gcc-patches@gcc.gnu.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","gcc-patches@gcc.gnu.org"],"Authentication-Results":["legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org\n (client-ip=38.145.34.32; helo=vm01.sourceware.org;\n envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org;\n receiver=patchwork.ozlabs.org)","sourceware.org; dmarc=none (p=none dis=none)\n header.from=forge-stage.sourceware.org","sourceware.org;\n spf=pass smtp.mailfrom=forge-stage.sourceware.org","server2.sourceware.org;\n arc=none smtp.remote-ip=38.145.34.39"],"Received":["from vm01.sourceware.org (vm01.sourceware.org [38.145.34.32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fGw834ScSz1xvg\n\tfor ; Fri, 20 Feb 2026 01:03:15 +1100 (AEDT)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id 8E32B4BAE7D6\n\tfor ; Thu, 19 Feb 2026 14:03:08 +0000 (GMT)","from forge-stage.sourceware.org (vm08.sourceware.org [38.145.34.39])\n by sourceware.org (Postfix) with ESMTPS id A75254BA23FF\n for ; Thu, 19 Feb 2026 14:02:36 +0000 (GMT)","from forge-stage.sourceware.org (localhost [IPv6:::1])\n (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n key-exchange x25519 server-signature ECDSA (prime256v1) server-digest SHA256)\n (No client certificate requested)\n by forge-stage.sourceware.org (Postfix) with ESMTPS id 7D4C44422D;\n Thu, 19 Feb 2026 14:02:36 +0000 (UTC)"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org 8E32B4BAE7D6","OpenDKIM Filter v2.11.0 sourceware.org A75254BA23FF"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org A75254BA23FF","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org A75254BA23FF","ARC-Seal":"i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1771509756; cv=none;\n b=Nb7S5fx2ouu5yx6svIp6Jxg8bMNl8n6qg6Q5XBpghKm/gTxL6luUPa4j5m0jme6JoZHqjbdAgStjbVTg2N5g73XGIyKFZ1uHJNKvx6WYwLnZSpgs3nK8X8+Z+E6PMpMrMNqCQcCVBL8LWXsSSSGRiS7FWUOONv278bpw4GlpCRo=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1771509756; c=relaxed/simple;\n bh=Tt+QsUfCV7r3RMo8OAj3a0OIVD7US8IQVYAk3sB4fbQ=;\n h=From:Date:Subject:To:Message-ID;\n b=w/9y75G73nH9sCQBUC0i1SGynplBZ58SPtWOrAzIQh3CEJi6+QXP+iP9CSUUGYvblrdu0zaTag8L3I9XLVNKNcuWF91imK7zmQ3asLujX34Xd2F+0mxyFDs8/f1xR/KI6KPetJcBdZgtDfwHbB8wPbu+Ne/iHC9oxqt3G7mdXgQ=","ARC-Authentication-Results":"i=1; server2.sourceware.org","From":"Andrei Tirziu via Sourceware Forge\n ","Date":"Thu, 19 Feb 2026 14:02:01 +0000","Subject":"[PATCH v2 0/7] Vectorizer: New SLP Pattern","To":"gcc-patches mailing list ","Cc":"Tamar Christina ,\n Victor Do Nascimento ","Message-ID":"\n ","X-Mailer":"batrachomyomachia","X-Pull-Request-Organization":"gcc","X-Pull-Request-Repository":"gcc-TEST","X-Pull-Request":"https://forge.sourceware.org/gcc/gcc-TEST/pulls/135","References":"\n ","In-Reply-To":"\n ","X-BeenThere":"gcc-patches@gcc.gnu.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Gcc-patches mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Reply-To":"gcc-patches mailing list ,\n Tamar Christina ,\n Victor Do Nascimento ,\n andreinichita.tirziu@arm.com","Errors-To":"gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org"},"content":"Hi gcc-patches mailing list,\nAndrei Tirziu has requested that the following forgejo pull request\nbe published on the mailing list.\n\nCreated on: 2026-01-20 16:35:05+00:00\nLatest update: 2026-02-19 14:02:06+00:00\nChanges: 19 changed files, 1804 additions, 76 deletions\nHead revision: why/gcc ref idiom-match-upstream commit 9de0c72932c609f582ab88807528c475f0f095ed\nBase revision: gcc/gcc-TEST ref master commit ffd205ff5ce78d30d44a204e152204626bf17dae r16-6927-gffd205ff5ce78d\nMerge base: ffd205ff5ce78d30d44a204e152204626bf17dae\nFull diff url: https://forge.sourceware.org/gcc/gcc-TEST/pulls/135.diff\nDiscussion: https://forge.sourceware.org/gcc/gcc-TEST/pulls/135\nRequested Reviewers:\n\nThis series of patches introduces a new SLP Pattern in the Vectorizer.\nThe \"MATCH\" pattern has a direct correspondence in the AArch64 SVE instructions (`match` and `nmatch`), thus allowing for optimizations of the initial code.\n\nThe MATCH pattern is either:\n- OR-chain (for equality disjunctions): `(p0) || (p1) || ...`, leaves are `EQ_EXPR`.\n- AND-chain (for inequality conjunctions): `(q0) && (q1) && ...`, leaves are `NE_EXPR`.\n\nIn addition to having an OR-chain / AND-chain structure, the leaves (also called \"predicates\") must be formed of a common variant and some invariants for the pattern to be recognized.\n\nA simple example using an array (the variant), and 3 invariants (the variables `x`, `y`, `z`) would be:\n`int count = 0;`\n`for (int i = 0; i < n; i++)`\n`{`\n` if (a[i] == x || a[i] == y || a[i] == z)`\n` {`\n` count++;`\n` }`\n`}`\n\nIf such a pattern is identified, a replacement is created for it, such that the GIMPLE expression used by the if-statement, of the form\n`exp = (a[i] == x || a[i] == y || a[i] == z)`\nis rewritten as (the actual code is slightly different, but this exemplifies the main idea):\n`exp = IFN_MATCH_ANY_FROM (a[i], x, y, z)`.\n\nSince we are dealing with SLP nodes, a more accurate representation would be:\n`exp = IFN_MATCH_ANY_FROM (a[i..i+N], inv)` where `a[i..i+N]` is a slice of the array (equivalent to having `N` iterations of the loop), and `inv = {x, y, z, x, y, z, ...}` where the invariants are organized in a cyclic way such that `inv.size() = N`.\n\nThe vectorizer then transforms this Internal Function to its conditional version, thus adding the loop mask to it: `exp = IFN_COND_MATCH_ANY_FROM (a[i..i+N], inv, mask, else)`.\n\nWith the help of a direct optab, and its implementation in the AArch64 backend, the vectorizer can transform this IFN into a `match` instruction in the final assembly output.\n\n\nChanged files:\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_char_small.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_char_small_run.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large_run.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium_run.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small_run.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small.c\n- A: gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small_run.c\n- M: gcc/config/aarch64/aarch64-sve2.md\n- M: gcc/config/aarch64/iterators.md\n- M: gcc/doc/md.texi\n- M: gcc/internal-fn.cc\n- M: gcc/internal-fn.def\n- M: gcc/internal-fn.h\n- M: gcc/optabs.def\n- M: gcc/tree-vect-slp-patterns.cc\n- M: gcc/tree-vect-stmts.cc\n\n\nAndrei Nichita Tirziu (7):\n [Vectorizer]: SLP MATCH Pattern: Add Internal Functions and Optabs\n [Vectorizer]: SLP MATCH Pattern: Identify pattern and build the new\n SLP nodes\n [Vectorizer][Aarch64]: SLP MATCH Pattern: Add implementation for match\n / nmatch SVE instructions\n [Vectorizer]: Improve vectorizable_call to support transformation from\n standard IFN to its conditional version\n [Vectorizer]: SLP MATCH Pattern: Enable pattern\n [Vectorizer]: SLP MATCH Pattern: Allow for more invariants\n [Vectorizer]: SLP MATCH Pattern: Add tests\n\n gcc/config/aarch64/aarch64-sve2.md | 70 ++\n gcc/config/aarch64/iterators.md | 3 +\n gcc/doc/md.texi | 18 +\n gcc/internal-fn.cc | 18 +-\n gcc/internal-fn.def | 11 +\n gcc/internal-fn.h | 40 +\n gcc/optabs.def | 6 +\n .../aarch64/sve2/match_char_small.c | 44 ++\n .../aarch64/sve2/match_char_small_run.c | 39 +\n .../aarch64/sve2/match_short_int_large.c | 60 ++\n .../aarch64/sve2/match_short_int_large_run.c | 71 ++\n .../aarch64/sve2/match_short_int_medium.c | 58 ++\n .../aarch64/sve2/match_short_int_medium_run.c | 69 ++\n .../aarch64/sve2/match_short_int_small.c | 48 ++\n .../aarch64/sve2/match_short_int_small_run.c | 53 ++\n .../aarch64/sve2/nmatch_short_int_small.c | 55 ++\n .../aarch64/sve2/nmatch_short_int_small_run.c | 53 ++\n gcc/tree-vect-slp-patterns.cc | 705 +++++++++++++++++-\n gcc/tree-vect-stmts.cc | 457 ++++++++++--\n 19 files changed, 1803 insertions(+), 75 deletions(-)\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_char_small.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_char_small_run.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large_run.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium_run.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small_run.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small.c\n create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small_run.c\n\nRange-diff against v1:\n1: 5b9fabd680e3 ! 1: 38a9e7b1b9c0 [Vectorizer]: SLP MATCH Pattern: Add Internal Functions and Optabs\n @@ Commit message\n is created for the initial expression.\n \n This requires several new Internal Functions:\n - - `IFN_MATCH_EQ`, `IFN_MATCH_NE`: the basic IFNs for\n + - `IFN_MATCH_ANY_FROM`, `IFN_MATCH_NONE_FROM`: the basic IFNs for\n unpredicated `match / nmatch` instructions.\n These are used when building the new node, but are expected to\n get transformed into the conditional versions.\n - - `IFN_COND_MATCH_EQ`, `IFN_COND_MATCH_NE`: conditional versions.\n - - `IFN_COND_LEN_MATCH_EQ`, `IFN_COND_LEN_MATCH_NE`: conditional\n + - `IFN_COND_MATCH_ANY_FROM`, `IFN_COND_MATCH_NONE_FROM`: conditional versions.\n + - `IFN_COND_LEN_MATCH_ANY_FROM`, `IFN_COND_LEN_MATCH_NONE_FROM`: conditional\n with length versions.\n \n The arguments of these functions are:\n - - `IFN_MATCH_EQ (variants, invariants)`\n - - `IFN_MATCH_NE (variants, invariants)`\n - - `IFN_COND_MATCH_EQ (variants, invariants, mask, else)`\n - - `IFN_COND_MATCH_NE (variants, invariants, mask, else)`\n - - `IFN_COND_LEN_MATCH_EQ (variants, invariants, mask, else, len, bias)`\n - - `IFN_COND_LEN_MATCH_NE (variants, invariants, mask, else, len, bias)`\n + - `IFN_MATCH_ANY_FROM (variants, invariants)`\n + - `IFN_MATCH_NONE_FROM (variants, invariants)`\n + - `IFN_COND_MATCH_ANY_FROM (variants, invariants, mask, else)`\n + - `IFN_COND_MATCH_NONE_FROM (variants, invariants, mask, else)`\n + - `IFN_COND_LEN_MATCH_ANY_FROM (variants, invariants, mask, else, len, bias)`\n + - `IFN_COND_LEN_MATCH_NONE_FROM (variants, invariants, mask, else, len, bias)`\n \n These IFNs have corresponding optabs. It is up to the backends to support\n - them further. All the optabs are declared as \"conversion direct\" optabs,\n - since two modes are being used. It is expected that the first mode\n - corresponds to the result, while the second mode represents the input.\n -\n - The modes are required because the output of the `match / nmatch`\n - instructions is a vector of booleans (or a mask), while the inputs\n - are the arrays of variants and invariants (which are INT types\n - of different sizes).\n + them further. All the optabs are declared as \"direct\" optabs,\n + since only a mode is being used. It is expected that the mode\n + corresponds to the input vectors (the variants / invariants).\n \n gcc/ChangeLog:\n \n - * internal-fn.def: New IFN for MATCH and NMATCH (including\n - conditional versions).\n + * internal-fn.def: New IFNs for MATCH ANY FROM and MATCH\n + NONE FROM (including conditional versions).\n * internal-fn.cc: Define paramater positions for new IFNs.\n - * optabs.def: New optabs for MATCH and NMATCH (including\n - conditional versions).\n -\n - Change-Id: I173c5989b844137a600622ebfa5c474f8829321b\n + * optabs.def: New optabs for MATCH ANY FROM and MATCH\n + NONE FROM (including conditional versions).\n \n - ## gcc/internal-fn.cc ##\n -@@ gcc/internal-fn.cc: init_internal_fns ()\n - #define unary_direct { 0, 0, true }\n - #define unary_convert_direct { -1, 0, true }\n - #define binary_direct { 0, 0, true }\n -+#define binary_convert_direct { -1, 1, true }\n - #define ternary_direct { 0, 0, true }\n - #define cond_unary_direct { 1, 1, true }\n - #define cond_binary_direct { 1, 1, true }\n -@@ gcc/internal-fn.cc: init_internal_fns ()\n - #define cond_len_unary_direct { 1, 1, true }\n - #define cond_len_binary_direct { 1, 1, true }\n - #define cond_len_ternary_direct { 1, 1, true }\n -+#define cond_binary_convert_direct { -1, 1, true }\n -+#define cond_len_binary_convert_direct { -1, 1, true }\n - #define while_direct { 0, 2, false }\n - #define fold_extract_direct { 2, 2, false }\n - #define fold_len_extract_direct { 2, 2, false }\n -@@ gcc/internal-fn.cc: expand_reduc_sbool_optab_fn (internal_fn fn, gcall *stmt, direct_optab optab)\n - #define expand_unary_convert_optab_fn(FN, STMT, OPTAB) \\\n - expand_convert_optab_fn (FN, STMT, OPTAB, 1)\n + ## gcc/doc/md.texi ##\n +@@ gcc/doc/md.texi: with mode @var{m}.\n \n -+#define expand_binary_convert_optab_fn(FN, STMT, OPTAB) \\\n -+ expand_convert_optab_fn (FN, STMT, OPTAB, 2)\n -+\n -+#define expand_cond_binary_convert_optab_fn(FN, STMT, OPTAB) \\\n -+ expand_convert_optab_fn (FN, STMT, OPTAB, 4)\n + These instructions are not allowed to @code{FAIL}.\n + \n ++@cindex @code{vec_match_any_from_@var{m}} instruction pattern\n ++@cindex @code{vec_match_none_from_@var{m}} instruction pattern\n ++@cindex @code{cond_vec_match_any_from_@var{m}} instruction pattern\n ++@cindex @code{cond_vec_match_none_from_@var{m}} instruction pattern\n ++@cindex @code{cond_len_vec_match_any_from_@var{m}} instruction pattern\n ++@cindex @code{cond_len_vec_match_none_from_@var{m}} instruction pattern\n ++@item @samp{vec_match_any_from_@var{m}}, @samp{cond_vec_match_any_from_@var{m}}, @samp{cond_len_vec_match_any_from_@var{m}}\n ++@itemx @samp{vec_match_none_from_@var{m}}, @samp{cond_vec_match_none_from_@var{m}}, @samp{cond_len_vec_match_none_from_@var{m}}\n ++Used for Match ANY FROM and Match NONE FROM patterns, which handle\n ++disjunctions of equality comparisons and conjunctions of inequality comparisons.\n +\n -+#define expand_cond_len_binary_convert_optab_fn(FN, STMT, OPTAB) \\\n -+ expand_convert_optab_fn (FN, STMT, OPTAB, 6)\n ++@smallexample\n ++short int a[1000], x, y, z, sum = 0;\n ++for (int i = 0; i < 1000; i++)\n ++ if (a[i] == x || a[i] == y || a[i] == z)\n ++ sum++;\n ++@end smallexample\n +\n - #define expand_vec_extract_optab_fn(FN, STMT, OPTAB) \\\n - expand_convert_optab_fn (FN, STMT, OPTAB, 2)\n - \n -@@ gcc/internal-fn.cc: multi_vector_optab_supported_p (convert_optab optab, tree_pair types,\n - #define direct_unary_optab_supported_p direct_optab_supported_p\n - #define direct_unary_convert_optab_supported_p convert_optab_supported_p\n - #define direct_binary_optab_supported_p direct_optab_supported_p\n -+#define direct_binary_convert_optab_supported_p convert_optab_supported_p\n - #define direct_ternary_optab_supported_p direct_optab_supported_p\n - #define direct_cond_unary_optab_supported_p direct_optab_supported_p\n - #define direct_cond_binary_optab_supported_p direct_optab_supported_p\n -@@ gcc/internal-fn.cc: multi_vector_optab_supported_p (convert_optab optab, tree_pair types,\n - #define direct_cond_len_unary_optab_supported_p direct_optab_supported_p\n - #define direct_cond_len_binary_optab_supported_p direct_optab_supported_p\n - #define direct_cond_len_ternary_optab_supported_p direct_optab_supported_p\n -+#define direct_cond_binary_convert_optab_supported_p convert_optab_supported_p\n -+#define direct_cond_len_binary_convert_optab_supported_p convert_optab_supported_p\n - #define direct_crc_optab_supported_p convert_optab_supported_p\n - #define direct_mask_load_optab_supported_p convert_optab_supported_p\n - #define direct_load_lanes_optab_supported_p multi_vector_optab_supported_p\n + @cindex @code{mulhisi3} instruction pattern\n + @item @samp{mulhisi3}\n + Multiply operands 1 and 2, which have mode @code{HImode}, and store\n +\n + ## gcc/internal-fn.cc ##\n @@ gcc/internal-fn.cc: get_conditional_len_internal_fn (tree_code code)\n T (ROUND) \\\n T (FLOOR) \\\n T (RINT) \\\n - T (CEIL)\n + T (CEIL) \\\n -+ T (MATCH_EQ) \\\n -+ T (MATCH_NE)\n ++ T (MATCH_ANY_FROM) \\\n ++ T (MATCH_NONE_FROM)\n \n /* Return a function that only performs internal function FN when a\n certain condition is met and that uses a given fallback value otherwise.\n @@ gcc/internal-fn.cc: get_len_internal_fn (internal_fn fn)\n return IFN_MASK_LEN_LOAD_LANES;\n case IFN_MASK_GATHER_LOAD:\n return IFN_MASK_LEN_GATHER_LOAD;\n -+ case IFN_MATCH_EQ:\n -+ return IFN_COND_LEN_MATCH_EQ;\n -+ case IFN_MATCH_NE:\n -+ return IFN_COND_LEN_MATCH_NE;\n ++ case IFN_MATCH_ANY_FROM:\n ++ return IFN_COND_LEN_MATCH_ANY_FROM;\n ++ case IFN_MATCH_NONE_FROM:\n ++ return IFN_COND_LEN_MATCH_NONE_FROM;\n default:\n return IFN_LAST;\n }\n @@ gcc/internal-fn.cc: internal_fn_len_index (internal_fn fn)\n case IFN_COND_LEN_XOR:\n case IFN_COND_LEN_SHL:\n case IFN_COND_LEN_SHR:\n -+ case IFN_COND_LEN_MATCH_EQ:\n -+ case IFN_COND_LEN_MATCH_NE:\n ++ case IFN_COND_LEN_MATCH_ANY_FROM:\n ++ case IFN_COND_LEN_MATCH_NONE_FROM:\n case IFN_MASK_LEN_STRIDED_STORE:\n return 4;\n \n @@ gcc/internal-fn.cc: internal_fn_else_index (internal_fn fn)\n case IFN_COND_LEN_XOR:\n case IFN_COND_LEN_SHL:\n case IFN_COND_LEN_SHR:\n -+ case IFN_COND_MATCH_EQ:\n -+ case IFN_COND_MATCH_NE:\n -+ case IFN_COND_LEN_MATCH_EQ:\n -+ case IFN_COND_LEN_MATCH_NE:\n ++ case IFN_COND_MATCH_ANY_FROM:\n ++ case IFN_COND_MATCH_NONE_FROM:\n ++ case IFN_COND_LEN_MATCH_ANY_FROM:\n ++ case IFN_COND_LEN_MATCH_NONE_FROM:\n return 3;\n \n case IFN_MASK_LOAD:\n @@ gcc/internal-fn.cc: internal_fn_mask_index (internal_fn fn)\n case IFN_MASK_LEN_STORE_LANES:\n case IFN_MASK_LEN_LOAD:\n case IFN_MASK_LEN_STORE:\n -+ case IFN_COND_MATCH_EQ:\n -+ case IFN_COND_MATCH_NE:\n -+ case IFN_COND_LEN_MATCH_EQ:\n -+ case IFN_COND_LEN_MATCH_NE:\n ++ case IFN_COND_MATCH_ANY_FROM:\n ++ case IFN_COND_MATCH_NONE_FROM:\n ++ case IFN_COND_LEN_MATCH_ANY_FROM:\n ++ case IFN_COND_LEN_MATCH_NONE_FROM:\n return 2;\n \n case IFN_MASK_LEN_STRIDED_LOAD:\n @@ gcc/internal-fn.def: DEF_INTERNAL_OPTAB_FN (VEC_SET, ECF_CONST | ECF_NOTHROW, ve\n DEF_INTERNAL_OPTAB_FN (VEC_EXTRACT, ECF_CONST | ECF_NOTHROW,\n \t\t vec_extract, vec_extract)\n \n -+// Internal functions for Match EQ and Match NE patterns\n ++// Internal functions for Match ANY FROM and Match NONE FROM patterns\n +// (conjuction of inequalities or disjunction of equalities)\n -+DEF_INTERNAL_OPTAB_FN (MATCH_EQ, ECF_PURE | ECF_NOTHROW,\n -+\t\t vec_match_eq, binary_convert)\n -+DEF_INTERNAL_OPTAB_FN (MATCH_NE, ECF_PURE | ECF_NOTHROW,\n -+\t\t vec_match_ne, binary_convert)\n -+DEF_INTERNAL_COND_FN (MATCH_EQ, ECF_PURE | ECF_NOTHROW,\n -+\t\t vec_match_eq, binary_convert)\n -+DEF_INTERNAL_COND_FN (MATCH_NE, ECF_PURE | ECF_NOTHROW,\n -+\t\t vec_match_ne, binary_convert)\n ++DEF_INTERNAL_OPTAB_FN (MATCH_ANY_FROM, ECF_PURE | ECF_NOTHROW,\n ++\t\t vec_match_any_from, binary)\n ++DEF_INTERNAL_OPTAB_FN (MATCH_NONE_FROM, ECF_PURE | ECF_NOTHROW,\n ++\t\t vec_match_none_from, binary)\n ++DEF_INTERNAL_COND_FN (MATCH_ANY_FROM, ECF_PURE | ECF_NOTHROW,\n ++\t\t vec_match_any_from, binary)\n ++DEF_INTERNAL_COND_FN (MATCH_NONE_FROM, ECF_PURE | ECF_NOTHROW,\n ++\t\t vec_match_none_from, binary)\n +\n DEF_INTERNAL_OPTAB_FN (LEN_STORE, 0, len_store, len_store)\n DEF_INTERNAL_OPTAB_FN (MASK_LEN_STORE, 0, mask_len_store, mask_len_store)\n \n \n ## gcc/optabs.def ##\n -@@ gcc/optabs.def: OPTAB_CD(vcond_mask_optab, \"vcond_mask_$a$b\")\n - OPTAB_CD(vec_cmp_optab, \"vec_cmp$a$b\")\n - OPTAB_CD(vec_cmpu_optab, \"vec_cmpu$a$b\")\n - OPTAB_CD(vec_cmpeq_optab, \"vec_cmpeq$a$b\")\n -+OPTAB_CD(vec_match_eq_optab, \"vec_match_$a$b\")\n -+OPTAB_CD(vec_match_ne_optab, \"vec_nmatch_$a$b\")\n -+OPTAB_CD(cond_vec_match_eq_optab, \"vec_match_cond_$a$b\")\n -+OPTAB_CD(cond_vec_match_ne_optab, \"vec_nmatch_cond_$a$b\")\n -+OPTAB_CD(cond_len_vec_match_eq_optab, \"vec_match_cond_len_$a$b\")\n -+OPTAB_CD(cond_len_vec_match_ne_optab, \"vec_nmatch_cond_len_$a$b\")\n - OPTAB_CD(maskload_optab, \"maskload$a$b\")\n - OPTAB_CD(maskstore_optab, \"maskstore$a$b\")\n - OPTAB_CD(mask_len_load_optab, \"mask_len_load$a$b\")\n +@@ gcc/optabs.def: OPTAB_D (vec_trunc_add_high_optab, \"vec_trunc_add_high$a\")\n + OPTAB_D (vec_addsub_optab, \"vec_addsub$a3\")\n + OPTAB_D (vec_fmaddsub_optab, \"vec_fmaddsub$a4\")\n + OPTAB_D (vec_fmsubadd_optab, \"vec_fmsubadd$a4\")\n ++OPTAB_D (vec_match_any_from_optab, \"vec_match_any_from_$a\")\n ++OPTAB_D (vec_match_none_from_optab, \"vec_match_none_from_$a\")\n ++OPTAB_D (cond_vec_match_any_from_optab, \"vec_match_any_from_cond_$a\")\n ++OPTAB_D (cond_vec_match_none_from_optab, \"vec_match_none_from_cond_$a\")\n ++OPTAB_D (cond_len_vec_match_any_from_optab, \"vec_match_any_from_cond_len_$a\")\n ++OPTAB_D (cond_len_vec_match_none_from_optab, \"vec_match_none_from_cond_len_$a\")\n + \n + OPTAB_D (sync_add_optab, \"sync_add$I$a\")\n + OPTAB_D (sync_and_optab, \"sync_and$I$a\")\n2: 6236a6440278 ! 2: 8897b6889457 [Vectorizer]: SLP MATCH Pattern: Identify pattern and build the new SLP nodes\n @@ Commit message\n [Vectorizer]: SLP MATCH Pattern: Identify pattern and build the new SLP nodes\n \n The new SLP pattern identifies a MATCH pattern:\n - - OR-chain (for equality disjunctions): `(p0) || (p1) || ...`,\n + - OR-chain (for equality comparisons disjunctions): `(p0) || (p1) || ...`,\n leaves are `EQ_EXPR`.\n - - AND-chain (for inequality conjunctions): `(q0) && (q1) && ...`,\n + - AND-chain (for inequality comparisons conjunctions): `(q0) && (q1) && ...`,\n leaves are `NE_EXPR`.\n \n For the pattern to be recognized, the leaves (also called \"predicates\")\n @@ Commit message\n exp = (a[i] == x || a[i] == y || a[i] == z)\n is rewritten as (the actual code is slightly different, but this exemplifies\n the main idea):\n - exp = IFN_MATCH_EQ (a[i], x, y, z)\n + exp = IFN_MATCH_ANY_FROM (a[i], x, y, z)\n \n Using this new node, the vectorizer is able to transform the loop to use\n an SVE MATCH instruction instead of the intitial comparisons.\n @@ Commit message\n \n * tree-vect-slp-patterns.cc: New pattern.\n \n - Change-Id: Ica57d1297720e4f763d3894af03e0003904883f8\n -\n ## gcc/tree-vect-slp-patterns.cc ##\n @@ gcc/tree-vect-slp-patterns.cc: along with GCC; see the file COPYING3. If not see\n #include \"vec-perm-indices.h\"\n @@ gcc/tree-vect-slp-patterns.cc: addsub_pattern::build (vec_info *vinfo)\n }\n \n +/**\n -+ * Recognize and replace a “match-any equality” predicate with an IFN.\n ++ * Recognize and replace a \"match-any equality\" predicate with an IFN.\n + *\n + * Detects the idiom:\n + * - Disjunction of equalities (OR-chain):\n + * `(a == c0) || (a == c1) || ... || (a == ck)`\n -+ *\t (the classic “a equals any of the set {c0..ck}”)\n ++ *\t (the classic \"a equals any of the set {c0..ck}\")\n + * - Conjuction of inequalities (AND-chain):\n + * `(a != c0) && (a != c1) && ... && (a != ck)`\n + *\n @@ gcc/tree-vect-slp-patterns.cc: addsub_pattern::build (vec_info *vinfo)\n +\t\t \"Found SLP MATCH pattern starting from statement %G\",\n +\t\t current_stmt);\n +\n -+ /* The first element will be a boolean mask, and the second will\n -+ be an integer vector (we know this from all previous checks). */\n -+ const tree_pair ifn_types = tree_pair (SLP_TREE_VECTYPE (*m_node),\n -+\t\t\t\t\t SLP_TREE_VECTYPE (variant_operand));\n -+\n + // Check if the backend supports a direct optab for this internal function.\n -+ if (!direct_internal_fn_supported_p (m_ifn, ifn_types, OPTIMIZE_FOR_SPEED))\n ++ if (!direct_internal_fn_supported_p (m_ifn, SLP_TREE_VECTYPE (variant_operand),\n ++ OPTIMIZE_FOR_SPEED))\n + {\n + if (dump_enabled_p ())\n + dump_printf_loc (MSG_NOTE, vect_location,\n @@ gcc/tree-vect-slp-patterns.cc: addsub_pattern::build (vec_info *vinfo)\n + return nullptr;\n +\n + return new match_pattern (node,\n -+\t\t\t is_or_chain ? IFN_MATCH_EQ : IFN_MATCH_NE,\n ++\t\t\t is_or_chain ? IFN_MATCH_ANY_FROM : IFN_MATCH_NONE_FROM,\n +\t\t\t terms);\n +}\n +\n3: daac8761a3c8 ! 3: d5cec2a1672e [Vectorizer][Aarch64]: SLP MATCH Pattern: Add implementation for match / nmatch SVE instructions\n @@ Commit message\n * config/aarch64/iterators.md: New iterator for MATCH,\n NMATCH modes.\n \n - Change-Id: I1e343a332752ae34863d7b5b760176dc0eb81104\n -\n ## gcc/config/aarch64/aarch64-sve2.md ##\n @@\n ;; - NMATCH\n ;; -------------------------------------------------------------------------\n \n -+;; SVE MATCH and NMATCH instructions. These are used by the optabs\n -+;; `vec_match_eq_optab` and `vec_match_ne_optab`.\n ++;; SVE MATCH instruction. This is used by the `vec_match_any_from_optab`.\n ++;;\n ++;; The initial operands are (result, variants_vector, invariants_vector).\n ++;; A full-true mask is generated to transform it into a valid instruction.\n ++(define_expand \"@vec_match_any_from_\"\n ++ [(set (match_operand: 0 \"register_operand\" \"=Upa\")\n ++\t(unspec:\n ++\t [(match_operand:SVE_MATCH_AND_NMATCH 1 \"register_operand\" \"w\")\n ++\t (match_operand:SVE_MATCH_AND_NMATCH 2 \"register_operand\" \"w\")]\n ++\t UNSPEC_MATCH))]\n ++ \"TARGET_SVE2 && TARGET_NON_STREAMING\"\n ++ {\n ++ rtx true_pred = aarch64_ptrue_reg (mode);\n ++ emit_insn (gen_vec_match_any_from_cond_\n ++\t (operands[0], operands[1], operands[2], true_pred, operands[1]));\n ++ DONE;\n ++ }\n ++)\n ++\n ++;; SVE MATCH instruction. This is used by the `cond_vec_match_any_from_optab`.\n ++;;\n ++;; The operands are\n ++;; (result, variants_vector, invariants_vector, mask, ignored_vector).\n ++(define_insn \"@vec_match_any_from_cond_\"\n ++ [(set (match_operand: 0 \"register_operand\" \"=Upa\")\n ++\t(unspec:\n ++\t [(match_operand:SVE_MATCH_AND_NMATCH 1 \"register_operand\" \"w\")\n ++\t (match_operand:SVE_MATCH_AND_NMATCH 2 \"register_operand\" \"w\")\n ++\t (match_operand: 3 \"register_operand\" \"Upa\")\n ++\t (match_operand:SVE_MATCH_AND_NMATCH 4 \"register_operand\" \"w\")]\n ++\t UNSPEC_MATCH))]\n ++ \"TARGET_SVE2 && TARGET_NON_STREAMING\"\n ++ \"match\\t%0., %3/z, %1., %2.\"\n ++)\n ++\n ++;; SVE NMATCH instruction. This is used by the `vec_match_none_from_optab`.\n +;;\n +;; The initial operands are (result, variants_vector, invariants_vector).\n +;; A full-true mask is generated to transform it into a valid instruction.\n -+(define_expand \"@vec__\"\n ++(define_expand \"@vec_match_none_from_\"\n + [(set (match_operand: 0 \"register_operand\" \"=Upa\")\n +\t(unspec:\n +\t [(match_operand:SVE_MATCH_AND_NMATCH 1 \"register_operand\" \"w\")\n +\t (match_operand:SVE_MATCH_AND_NMATCH 2 \"register_operand\" \"w\")]\n -+\t SVE2_MATCH))]\n ++\t UNSPEC_NMATCH))]\n + \"TARGET_SVE2 && TARGET_NON_STREAMING\"\n + {\n + rtx true_pred = aarch64_ptrue_reg (mode);\n -+ emit_insn (gen_vec__cond_\n ++ emit_insn (gen_vec_match_none_from_cond_\n +\t (operands[0], operands[1], operands[2], true_pred, operands[1]));\n + DONE;\n + }\n +)\n +\n -+;; SVE MATCH and NMATCH instructions. These are used by the optabs\n -+;; `cond_vec_match_eq_optab` and `cond_vec_match_ne_optab`.\n ++;; SVE NMATCH instruction. This is used by the `cond_vec_match_none_from_optab`.\n +;;\n +;; The operands are\n +;; (result, variants_vector, invariants_vector, mask, ignored_vector).\n -+(define_insn \"@vec__cond_\"\n ++(define_insn \"@vec_match_none_from_cond_\"\n + [(set (match_operand: 0 \"register_operand\" \"=Upa\")\n +\t(unspec:\n +\t [(match_operand:SVE_MATCH_AND_NMATCH 1 \"register_operand\" \"w\")\n +\t (match_operand:SVE_MATCH_AND_NMATCH 2 \"register_operand\" \"w\")\n +\t (match_operand: 3 \"register_operand\" \"Upa\")\n +\t (match_operand:SVE_MATCH_AND_NMATCH 4 \"register_operand\" \"w\")]\n -+\t SVE2_MATCH))]\n ++\t UNSPEC_NMATCH))]\n + \"TARGET_SVE2 && TARGET_NON_STREAMING\"\n -+ \"\\t%0., %3/z, %1., %2.\"\n ++ \"nmatch\\t%0., %3/z, %1., %2.\"\n +)\n +\n ;; Predicated string matching.\n4: 95cf5d8b25d2 ! 4: 05205cea0e04 [Vectorizer]: Improve vectorizable_call to support transformation from standard IFN to its conditional version\n @@ Commit message\n * internal-fn.h: New helper functions.\n * tree-vect-stmts.cc: Improve vectorizable_call function.\n \n - Change-Id: Ib1f95f0b391e55b5f8935b5221cfe2b07a783c80\n -\n ## gcc/internal-fn.h ##\n @@ gcc/internal-fn.h: vectorizable_internal_fn_p (internal_fn fn)\n return direct_internal_fn_array[fn].vectorizable;\n @@ gcc/internal-fn.h: vectorizable_internal_fn_p (internal_fn fn)\n +{\n + switch (fn)\n + {\n -+ case IFN_MATCH_EQ:\n -+ case IFN_MATCH_NE:\n ++ case IFN_MATCH_ANY_FROM:\n ++ case IFN_MATCH_NONE_FROM:\n + return true;\n + default:\n + return false;\n @@ gcc/internal-fn.h: vectorizable_internal_fn_p (internal_fn fn)\n +{\n + switch (fn)\n + {\n -+ case IFN_MATCH_EQ:\n -+ case IFN_MATCH_NE:\n -+ case IFN_COND_MATCH_EQ:\n -+ case IFN_COND_MATCH_NE:\n ++ case IFN_MATCH_ANY_FROM:\n ++ case IFN_MATCH_NONE_FROM:\n ++ case IFN_COND_MATCH_ANY_FROM:\n ++ case IFN_COND_MATCH_NONE_FROM:\n + return true;\n + default:\n + return false;\n @@ gcc/tree-vect-stmts.cc: vect_finish_stmt_generation (vec_info *vinfo,\n gsi_insert_before (gsi, vec_stmt, GSI_SAME_STMT);\n vect_finish_stmt_generation_1 (vinfo, stmt_info, vec_stmt);\n }\n +@@ gcc/tree-vect-stmts.cc: vectorizable_internal_function (combined_fn cfn, tree fndecl,\n + \t exactly the same when vectype_out isn't participating the optab.\n + \t While there is no restriction for type size when vectype_out\n + \t is part of the optab query. */\n +-\t if (type0 != vectype_out && type1 != vectype_out && !same_size_p)\n +-\t return IFN_LAST;\n ++ if (ifn != IFN_MATCH_ANY_FROM && ifn != IFN_MATCH_NONE_FROM &&\n ++\t ifn != IFN_COND_MATCH_ANY_FROM && ifn != IFN_COND_MATCH_NONE_FROM)\n ++\t {\n ++\t if (type0 != vectype_out && type1 != vectype_out && !same_size_p)\n ++\t return IFN_LAST;\n ++\t }\n + \n + \t if (direct_internal_fn_supported_p (ifn, tree_pair (type0, type1),\n + \t\t\t\t\t OPTIMIZE_FOR_SPEED))\n @@ gcc/tree-vect-stmts.cc: vectorizable_call (vec_info *vinfo,\n return false;\n }\n \n - if (VECTOR_BOOLEAN_TYPE_P (vectype_out)\n - != VECTOR_BOOLEAN_TYPE_P (vectype_in))\n -+ /* In the case of IFN_MATCH_EQ and IFN_MATCH_NE, we know that the\n ++ /* In the case of IFN_MATCH_ANY_FROM and IFN_MATCH_NONE_FROM, we know that the\n + input vectype is an integer vector, while the output is a boolean mask.\n + So we only check if the vectypes match if we don't have an IFN\n -+ or if we have an IFN different from IFN_MATCH_EQ and IFN_MATCH_NE. */\n ++ or if we have an IFN different from IFN_MATCH_ANY_FROM\n ++ and IFN_MATCH_NONE_FROM. */\n + if (!gimple_call_internal_p (stmt) ||\n -+\t(gimple_call_internal_fn (stmt) != IFN_MATCH_EQ &&\n -+\t gimple_call_internal_fn (stmt) != IFN_MATCH_NE))\n ++\t(gimple_call_internal_fn (stmt) != IFN_MATCH_ANY_FROM &&\n ++\t gimple_call_internal_fn (stmt) != IFN_MATCH_NONE_FROM))\n + {\n + if (VECTOR_BOOLEAN_TYPE_P (vectype_out) !=\n +\tVECTOR_BOOLEAN_TYPE_P (vectype_in))\n @@ gcc/tree-vect-stmts.cc: vectorizable_call (vec_info *vinfo,\n +\t };\n +\n +\t /* If the initial call had no explicit mask but we need it now,\n -+\t synthesize the loop mask into the conditional IFN’s mask operand.\n ++\t synthesize the loop mask into the conditional IFN's mask operand.\n +\t */\n +\t if (needs_new_loop_mask)\n \t {\n5: 740e963135fe ! 5: a6f3c926a92b [Vectorizer]: SLP MATCH Pattern: Enable pattern\n @@ Commit message\n \n * tree-vect-slp-patterns.cc: Enable MATCH pattern.\n \n - Change-Id: Iaec87e7f11f7195cf52870b6f434a93f40a81a05\n -\n ## gcc/tree-vect-slp-patterns.cc ##\n @@ gcc/tree-vect-slp-patterns.cc: vect_pattern_decl_t slp_patterns[]\n overlap in what they can detect. */\n6: 86850a02bc5c ! 6: edf34a05054d [Vectorizer]: SLP MATCH Pattern: Allow for more invariants\n @@ Commit message\n obtain the final result.\n \n This would be equivalent to the following:\n - x1 = IFN_MATCH_EQ (variant, invariant_1);\n - x2 = IFN_MATCH_EQ (variant, invariant_1);\n - x3 = IFN_MATCH_EQ (variant, invariant_3);\n + x1 = IFN_MATCH_ANY_FROM (variant, invariant_1);\n + x2 = IFN_MATCH_ANY_FROM (variant, invariant_1);\n + x3 = IFN_MATCH_ANY_FROM (variant, invariant_3);\n final_result = x1 || x2 || x3;\n (similar for the NMATCH case).\n \n @@ Commit message\n * tree-vect-slp-patterns.cc: Changes to the MATCH\n SLP pattern.\n \n - Change-Id: I67755646bb7ecf76079b203641f78e47d308f79b\n -\n ## gcc/tree-vect-slp-patterns.cc ##\n @@ gcc/tree-vect-slp-patterns.cc: void match_pattern::build (vec_info *vinfo)\n return;\n @@ gcc/tree-vect-slp-patterns.cc: void match_pattern::build (vec_info *vinfo)\n + {\n + /* Create new \"match nodes\". Each newly created IFN will form a new\n + internal node, whose children are the variant operand and the IFN's\n -+ corresponding invariant node. */ \n ++ corresponding invariant node. */\n + for (unsigned int i = 0; i < total_invariant_nodes; i++)\n + {\n + slp_tree new_match_node = vect_create_new_slp_node (0, CONSTRUCTOR);\n @@ gcc/tree-vect-slp-patterns.cc: void match_pattern::build (vec_info *vinfo)\n +\t combine_result = result_of_match_ifn0 OR result_of_match_ifn1. */\n + gassign *result_of_match_combine =\n +\tgimple_build_assign (lhs,\n -+\t\t\t m_ifn == IFN_MATCH_EQ ? BIT_IOR_EXPR : BIT_AND_EXPR,\n ++\t\t\t m_ifn == IFN_MATCH_ANY_FROM ? BIT_IOR_EXPR : BIT_AND_EXPR,\n +\t\t\t accumulator_lhs,\n +\t\t\t gimple_call_lhs (new_ifn_calls[i + 1]));\n +\n @@ gcc/tree-vect-slp-patterns.cc: void match_pattern::build (vec_info *vinfo)\n - gimple_call_set_nothrow (ifn_call, true);\n - gimple_set_bb (ifn_call, gimple_bb (current_stmt));\n +\tSLP_TREE_CODE (new_combine_node) =\n -+\t m_ifn == IFN_MATCH_EQ ? BIT_IOR_EXPR : BIT_AND_EXPR;\n ++\t m_ifn == IFN_MATCH_ANY_FROM ? BIT_IOR_EXPR : BIT_AND_EXPR;\n \n - // Create new vector info for the newly created IFN call.\n - stmt_vec_info new_vec_info = vinfo->add_pattern_stmt (ifn_call,\n @@ gcc/tree-vect-slp-patterns.cc: void match_pattern::build (vec_info *vinfo)\n - SLP_TREE_CHILDREN (*m_node).safe_push (invariant_node);\n - SLP_TREE_REF_COUNT (invariant_node)++;\n +\tSLP_TREE_CODE (*m_node) =\n -+\t m_ifn == IFN_MATCH_EQ ? BIT_IOR_EXPR : BIT_AND_EXPR;\n ++\t m_ifn == IFN_MATCH_ANY_FROM ? BIT_IOR_EXPR : BIT_AND_EXPR;\n +\tSLP_TREE_REPRESENTATIVE (*m_node) = new_vec_info;\n + }\n + }\n7: 9a7d99f3f4bc ! 7: 9de0c72932c6 [Vectorizer]: SLP MATCH Pattern: Add tests\n @@ Commit message\n * gcc.target/aarch64/sve2/nmatch_short_int_small.c: New test.\n * gcc.target/aarch64/sve2/nmatch_short_int_small_run.c: New test.\n \n - Change-Id: If0880d8228453c5f01ba503ac04f3e9256f7f362\n -\n ## gcc/testsuite/gcc.target/aarch64/sve2/match_char_small.c (new) ##\n @@\n +/**\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_char_small.c (new)\n + *\n + * Test with 3 invariants, of type `char` (8-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants is small enough\n + * to only use a MATCH instruction.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_char_small_run.c (new)\n + *\n + * Test with 3 invariants, of type `char` (8-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants is small enough\n + * to only use a MATCH instruction.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large.c (new)\n + *\n + * Test with 20 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants should require\n + * three MATCH instructions.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_large_run.c (new)\n + *\n + * Test with 20 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants should require\n + * three MATCH instructions.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium.c (new)\n + *\n + * Test with 10 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants should require\n + * two MATCH instructions.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_medium_run.c (new)\n + *\n + * Test with 10 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants should require\n + * two MATCH instructions.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small.c (new)\n + *\n + * Test with 4 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants is small enough\n + * to only use a MATCH instruction.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/match_short_int_small_run.c (new)\n + *\n + * Test with 4 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-EQ pattern should be identified and replaced with\n ++ * A Match ANY FROM pattern should be identified and replaced with\n + * a MATCH instruction. In this test, the number of invariants is small enough\n + * to only use a MATCH instruction.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small.c (new)\n + *\n + * Test with 4 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-NE pattern should be identified and replaced with\n ++ * A Match NONE FROM pattern should be identified and replaced with\n + * a NMATCH instruction. In this test, the number of invariants is small enough\n + * to only use a NMATCH instruction.\n + *\n @@ gcc/testsuite/gcc.target/aarch64/sve2/nmatch_short_int_small_run.c (new)\n + *\n + * Test with 4 invariants, of type `short int` (16-bits).\n + *\n -+ * A Match-NE pattern should be identified and replaced with\n ++ * A Match NONE FROM pattern should be identified and replaced with\n + * a NMATCH instruction. In this test, the number of invariants is small enough\n + * to only use a NMATCH instruction.\n + *"}