From patchwork Wed May 24 07:03:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?6ZKf5bGF5ZOy?= X-Patchwork-Id: 1785555 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QR2Gz1QNrz20Q0 for ; Wed, 24 May 2023 17:04:09 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 6C0F13857719 for ; Wed, 24 May 2023 07:04:06 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg151.qq.com (smtpbg151.qq.com [18.169.211.239]) by sourceware.org (Postfix) with ESMTPS id 245773858C54 for ; Wed, 24 May 2023 07:03:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 245773858C54 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai X-QQ-mid: bizesmtp90t1684911817temmr99m Received: from rios-cad5.localdomain ( [58.60.1.11]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 24 May 2023 15:03:36 +0800 (CST) X-QQ-SSF: 01400000000000F0R000000A0000000 X-QQ-FEAT: /rrU+puPB7TeGVf+cbfN+SKXydyl7GZHwAMnf8sPxnIdOOWyomBhjWINRVfmx BzE7L5Xt/iNUfCeOSl4hoc3PE8Kqqd6Dxkr1LXTSuzHA4EO/mfikH1iQGFEi+x90yGT1yqM Rc0S8WdVQXpXA944HOwTZNsExFJTymqA7tAsoEgYYxNMtd0vU+5sJFtJqlgcOO280k2iJxe /edd4QMKn4bmppooY4W+LLRIn059ydrlZ++pPeY2a5LSosIRw94r7fdwmyRqEd/kL/r08ok BXniqilJiIE/C9JmV9ckxqlEEyC5vRMtzkkB63pEURh6CN6peGx4GZ595sJlQC3ZV1zLf25 qsSwvrRyOEqzUjf58indoHj8Kb/hm5ZnNgROU6obi6Jvt5OWxy1C4K/L9QS+FBuKFrcBlDt zNur5FctE4I= X-QQ-GoodBg: 2 X-BIZMAIL-ID: 9890980608651169434 From: juzhe.zhong@rivai.ai To: gcc-patches@gcc.gnu.org Cc: kito.cheng@gmail.com, kito.cheng@sifive.com, palmer@dabbelt.com, palmer@rivosinc.com, jeffreyalaw@gmail.com, rdapp.gcc@gmail.com, Juzhe-Zhong Subject: [PATCH] RISC-V: Add RVV mask logic auto-vectorization Date: Wed, 24 May 2023 15:03:29 +0800 Message-Id: <20230524070329.1163656-1-juzhe.zhong@rivai.ai> X-Mailer: git-send-email 2.36.3 MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz7a-one-0 X-Spam-Status: No, score=-11.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" From: Juzhe-Zhong This patch is adding mask logic auto-vectorization. define the pattern as "define_insn_and_split" to allow combine PASS easily combine series instructions. For example: combine vmxor.mm + vmnot.m into vmxnor.mm Build success and regression PASS Ok for trunk ? gcc/ChangeLog: * config/riscv/autovec.md (3): New pattern. (one_cmpl2): Ditto. (*not): Ditto. (*n): Ditto. * config/riscv/riscv-v.cc (expand_vec_cmp_float): Change to one_cmpl. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/cmp/vcond-4.c: New test. * gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c: New test. --- gcc/config/riscv/autovec.md | 95 +++++++++++++++++++ gcc/config/riscv/riscv-v.cc | 7 +- .../riscv/rvv/autovec/cmp/vcond-4.c | 53 +++++++++++ .../riscv/rvv/autovec/cmp/vcond_run-4.c | 35 +++++++ 4 files changed, 187 insertions(+), 3 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 4eeeab624a4..cacf27e4e60 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -163,6 +163,101 @@ DONE; }) +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmand.mm +;; - vmxor.mm +;; - vmor.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "3" + [(set (match_operand:VB 0 "register_operand" "=vr") + (any_bitwise:VB (match_operand:VB 1 "register_operand" " vr") + (match_operand:VB 2 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred (, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Inverse +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmnot.m +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "one_cmpl2" + [(set (match_operand:VB 0 "register_operand" "=vr") + (not:VB (match_operand:VB 1 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_not (mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations (inverted second input) +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmandnot.mm +;; - vmornot.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "*not" + [(set (match_operand:VB 0 "register_operand" "=vr") + (bitmanip_bitwise:VB + (not:VB (match_operand:VB 2 "register_operand" " vr")) + (match_operand:VB 1 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_not (, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations (inverted result) +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmnand.mm +;; - vmnor.mm +;; - vmxnor.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "*n" + [(set (match_operand:VB 0 "register_operand" "=vr") + (not:VB + (any_bitwise:VB + (match_operand:VB 1 "register_operand" " vr") + (match_operand:VB 2 "register_operand" " vr"))))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_n (, mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "")]) + ;; ========================================================================= ;; == Comparisons and selects ;; ========================================================================= diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 10de5a19937..f71ad9e46a1 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -1550,9 +1550,10 @@ expand_vec_cmp_float (rtx target, rtx_code code, rtx op0, rtx op1, emit_move_insn (target, eq0); return true; } - insn_code icode = code_for_pred_not (mask_mode); - rtx ops[] = {target, eq0}; - emit_vlmax_insn (icode, RVV_UNOP, ops); + + /* We use one_cmpl2 to make Combine PASS to combine mask instructions + into: vmand.mm/vmnor.mm/vmnand.mm/vmnor.mm/vmxnor.mm. */ + emit_insn (gen_rtx_SET (target, gen_rtx_NOT (mask_mode, eq0))); return false; } diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c new file mode 100644 index 00000000000..435a59c97f2 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c @@ -0,0 +1,53 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include + +#define b_and(A, B) ((A) & (B)) +#define b_orr(A, B) ((A) | (B)) +#define b_xor(A, B) ((A) ^ (B)) +#define b_nand(A, B) (!((A) & (B))) +#define b_nor(A, B) (!((A) | (B))) +#define b_xnor(A, B) (!(A) ^ (B)) +#define b_andnot(A, B) ((A) & !(B)) +#define b_ornot(A, B) ((A) | !(B)) + +#define LOOP(TYPE, BINOP) \ + void __attribute__ ((noinline, noclone)) \ + test_##TYPE##_##BINOP (TYPE *restrict dest, TYPE *restrict src, \ + TYPE *restrict a, TYPE *restrict b, TYPE *restrict c, \ + TYPE *restrict d, TYPE fallback, int count) \ + { \ + for (int i = 0; i < count; ++i) \ + { \ + TYPE srcv = src[i]; \ + dest[i] = (BINOP (__builtin_isunordered (a[i], b[i]), \ + __builtin_isunordered (c[i], d[i])) \ + ? srcv \ + : fallback); \ + } \ + } + +#define TEST_BINOP(T, BINOP) \ + T (float, BINOP) \ + T (double, BINOP) + +#define TEST_ALL(T) \ + TEST_BINOP (T, b_and) \ + TEST_BINOP (T, b_orr) \ + TEST_BINOP (T, b_xor) \ + TEST_BINOP (T, b_nand) \ + TEST_BINOP (T, b_nor) \ + TEST_BINOP (T, b_xnor) \ + TEST_BINOP (T, b_andnot) \ + TEST_BINOP (T, b_ornot) + +TEST_ALL (LOOP) + +/* { dg-final { scan-assembler-times {\tvmand\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmxor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmnot\.m} 4 } } */ +/* { dg-final { scan-assembler-times {\tvmxnor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmandn\.mm} 4 } } */ +/* { dg-final { scan-assembler-times {\tvmorn\.mm} 4 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c new file mode 100644 index 00000000000..6c45c274c33 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c @@ -0,0 +1,35 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include "vcond-4.c" + +#define N 401 + +#define RUN_LOOP(TYPE, BINOP) \ + { \ + TYPE dest[N], src[N], a[N], b[N], c[N], d[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + src[i] = i * i; \ + a[i] = i % 5 < 3 ? __builtin_nan("") : i; \ + b[i] = i % 7 < 4 ? __builtin_nan("") : i; \ + c[i] = i % 9 < 5 ? __builtin_nan("") : i; \ + d[i] = i % 11 < 6 ? __builtin_nan("") : i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##BINOP (dest, src, a, b, c, d, 100, N); \ + for (int i = 0; i < N; ++i) \ + { \ + int res = BINOP (__builtin_isunordered (a[i], b[i]), \ + __builtin_isunordered (c[i], d[i])); \ + if (dest[i] != (res ? src[i] : 100.0)) \ + __builtin_abort (); \ + } \ + } + +int __attribute__ ((optimize (1))) +main (void) +{ + TEST_ALL (RUN_LOOP) + return 0; +}