Message ID | 20230524070329.1163656-1-juzhe.zhong@rivai.ai |
---|---|
State | New |
Headers | show |
Series | RISC-V: Add RVV mask logic auto-vectorization | expand |
Just one comment: define_insn_and_split should be used in this scenario rather than define_insn_and_rewrite since you are not really rewriting. You can commit after updating to define_insn_and_split :) On Wed, May 24, 2023 at 3:04 PM <juzhe.zhong@rivai.ai> wrote: > > From: Juzhe-Zhong <juzhe.zhong@rivai.ai> > > This patch is adding mask logic auto-vectorization. > define the pattern as "define_insn_and_split" to allow > combine PASS easily combine series instructions. > > For example: > combine vmxor.mm + vmnot.m into vmxnor.mm > > Build success and regression PASS > > Ok for trunk ? > > gcc/ChangeLog: > > * config/riscv/autovec.md (<optab><mode>3): New pattern. > (one_cmpl<mode>2): Ditto. > (*<optab>not<mode>): Ditto. > (*n<optab><mode>): Ditto. > * config/riscv/riscv-v.cc (expand_vec_cmp_float): Change to one_cmpl. > > gcc/testsuite/ChangeLog: > > * gcc.target/riscv/rvv/autovec/cmp/vcond-4.c: New test. > * gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c: New test. > > --- > gcc/config/riscv/autovec.md | 95 +++++++++++++++++++ > gcc/config/riscv/riscv-v.cc | 7 +- > .../riscv/rvv/autovec/cmp/vcond-4.c | 53 +++++++++++ > .../riscv/rvv/autovec/cmp/vcond_run-4.c | 35 +++++++ > 4 files changed, 187 insertions(+), 3 deletions(-) > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > > diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md > index 4eeeab624a4..cacf27e4e60 100644 > --- a/gcc/config/riscv/autovec.md > +++ b/gcc/config/riscv/autovec.md > @@ -163,6 +163,101 @@ > DONE; > }) > > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmand.mm > +;; - vmxor.mm > +;; - vmor.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "<optab><mode>3" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (any_bitwise:VB (match_operand:VB 1 "register_operand" " vr") > + (match_operand:VB 2 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Inverse > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmnot.m > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "one_cmpl<mode>2" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (not:VB (match_operand:VB 1 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_not (<MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations (inverted second input) > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmandnot.mm > +;; - vmornot.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "*<optab>not<mode>" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (bitmanip_bitwise:VB > + (not:VB (match_operand:VB 2 "register_operand" " vr")) > + (match_operand:VB 1 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_not (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations (inverted result) > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmnand.mm > +;; - vmnor.mm > +;; - vmxnor.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "*n<optab><mode>" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (not:VB > + (any_bitwise:VB > + (match_operand:VB 1 "register_operand" " vr") > + (match_operand:VB 2 "register_operand" " vr"))))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_n (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > ;; ========================================================================= > ;; == Comparisons and selects > ;; ========================================================================= > diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc > index 10de5a19937..f71ad9e46a1 100644 > --- a/gcc/config/riscv/riscv-v.cc > +++ b/gcc/config/riscv/riscv-v.cc > @@ -1550,9 +1550,10 @@ expand_vec_cmp_float (rtx target, rtx_code code, rtx op0, rtx op1, > emit_move_insn (target, eq0); > return true; > } > - insn_code icode = code_for_pred_not (mask_mode); > - rtx ops[] = {target, eq0}; > - emit_vlmax_insn (icode, RVV_UNOP, ops); > + > + /* We use one_cmpl<mode>2 to make Combine PASS to combine mask instructions > + into: vmand.mm/vmnor.mm/vmnand.mm/vmnor.mm/vmxnor.mm. */ > + emit_insn (gen_rtx_SET (target, gen_rtx_NOT (mask_mode, eq0))); > return false; > } > > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > new file mode 100644 > index 00000000000..435a59c97f2 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > @@ -0,0 +1,53 @@ > +/* { dg-do compile } */ > +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ > + > +#include <stdint-gcc.h> > + > +#define b_and(A, B) ((A) & (B)) > +#define b_orr(A, B) ((A) | (B)) > +#define b_xor(A, B) ((A) ^ (B)) > +#define b_nand(A, B) (!((A) & (B))) > +#define b_nor(A, B) (!((A) | (B))) > +#define b_xnor(A, B) (!(A) ^ (B)) > +#define b_andnot(A, B) ((A) & !(B)) > +#define b_ornot(A, B) ((A) | !(B)) > + > +#define LOOP(TYPE, BINOP) \ > + void __attribute__ ((noinline, noclone)) \ > + test_##TYPE##_##BINOP (TYPE *restrict dest, TYPE *restrict src, \ > + TYPE *restrict a, TYPE *restrict b, TYPE *restrict c, \ > + TYPE *restrict d, TYPE fallback, int count) \ > + { \ > + for (int i = 0; i < count; ++i) \ > + { \ > + TYPE srcv = src[i]; \ > + dest[i] = (BINOP (__builtin_isunordered (a[i], b[i]), \ > + __builtin_isunordered (c[i], d[i])) \ > + ? srcv \ > + : fallback); \ > + } \ > + } > + > +#define TEST_BINOP(T, BINOP) \ > + T (float, BINOP) \ > + T (double, BINOP) > + > +#define TEST_ALL(T) \ > + TEST_BINOP (T, b_and) \ > + TEST_BINOP (T, b_orr) \ > + TEST_BINOP (T, b_xor) \ > + TEST_BINOP (T, b_nand) \ > + TEST_BINOP (T, b_nor) \ > + TEST_BINOP (T, b_xnor) \ > + TEST_BINOP (T, b_andnot) \ > + TEST_BINOP (T, b_ornot) > + > +TEST_ALL (LOOP) > + > +/* { dg-final { scan-assembler-times {\tvmand\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmxor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmnot\.m} 4 } } */ > +/* { dg-final { scan-assembler-times {\tvmxnor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmandn\.mm} 4 } } */ > +/* { dg-final { scan-assembler-times {\tvmorn\.mm} 4 } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > new file mode 100644 > index 00000000000..6c45c274c33 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > @@ -0,0 +1,35 @@ > +/* { dg-do run { target { riscv_vector } } } */ > +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ > + > +#include "vcond-4.c" > + > +#define N 401 > + > +#define RUN_LOOP(TYPE, BINOP) \ > + { \ > + TYPE dest[N], src[N], a[N], b[N], c[N], d[N]; \ > + for (int i = 0; i < N; ++i) \ > + { \ > + src[i] = i * i; \ > + a[i] = i % 5 < 3 ? __builtin_nan("") : i; \ > + b[i] = i % 7 < 4 ? __builtin_nan("") : i; \ > + c[i] = i % 9 < 5 ? __builtin_nan("") : i; \ > + d[i] = i % 11 < 6 ? __builtin_nan("") : i; \ > + asm volatile ("" ::: "memory"); \ > + } \ > + test_##TYPE##_##BINOP (dest, src, a, b, c, d, 100, N); \ > + for (int i = 0; i < N; ++i) \ > + { \ > + int res = BINOP (__builtin_isunordered (a[i], b[i]), \ > + __builtin_isunordered (c[i], d[i])); \ > + if (dest[i] != (res ? src[i] : 100.0)) \ > + __builtin_abort (); \ > + } \ > + } > + > +int __attribute__ ((optimize (1))) > +main (void) > +{ > + TEST_ALL (RUN_LOOP) > + return 0; > +} > -- > 2.36.3 >
Thanks kito., change it into define_insn_and_split send V2 soon. juzhe.zhong@rivai.ai From: Kito Cheng Date: 2023-05-24 15:18 To: juzhe.zhong CC: gcc-patches; kito.cheng; palmer; palmer; jeffreyalaw; rdapp.gcc Subject: Re: [PATCH] RISC-V: Add RVV mask logic auto-vectorization Just one comment: define_insn_and_split should be used in this scenario rather than define_insn_and_rewrite since you are not really rewriting. You can commit after updating to define_insn_and_split :) On Wed, May 24, 2023 at 3:04 PM <juzhe.zhong@rivai.ai> wrote: > > From: Juzhe-Zhong <juzhe.zhong@rivai.ai> > > This patch is adding mask logic auto-vectorization. > define the pattern as "define_insn_and_split" to allow > combine PASS easily combine series instructions. > > For example: > combine vmxor.mm + vmnot.m into vmxnor.mm > > Build success and regression PASS > > Ok for trunk ? > > gcc/ChangeLog: > > * config/riscv/autovec.md (<optab><mode>3): New pattern. > (one_cmpl<mode>2): Ditto. > (*<optab>not<mode>): Ditto. > (*n<optab><mode>): Ditto. > * config/riscv/riscv-v.cc (expand_vec_cmp_float): Change to one_cmpl. > > gcc/testsuite/ChangeLog: > > * gcc.target/riscv/rvv/autovec/cmp/vcond-4.c: New test. > * gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c: New test. > > --- > gcc/config/riscv/autovec.md | 95 +++++++++++++++++++ > gcc/config/riscv/riscv-v.cc | 7 +- > .../riscv/rvv/autovec/cmp/vcond-4.c | 53 +++++++++++ > .../riscv/rvv/autovec/cmp/vcond_run-4.c | 35 +++++++ > 4 files changed, 187 insertions(+), 3 deletions(-) > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > > diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md > index 4eeeab624a4..cacf27e4e60 100644 > --- a/gcc/config/riscv/autovec.md > +++ b/gcc/config/riscv/autovec.md > @@ -163,6 +163,101 @@ > DONE; > }) > > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmand.mm > +;; - vmxor.mm > +;; - vmor.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "<optab><mode>3" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (any_bitwise:VB (match_operand:VB 1 "register_operand" " vr") > + (match_operand:VB 2 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Inverse > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmnot.m > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "one_cmpl<mode>2" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (not:VB (match_operand:VB 1 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_not (<MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations (inverted second input) > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmandnot.mm > +;; - vmornot.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "*<optab>not<mode>" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (bitmanip_bitwise:VB > + (not:VB (match_operand:VB 2 "register_operand" " vr")) > + (match_operand:VB 1 "register_operand" " vr")))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_not (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > +;; ------------------------------------------------------------------------- > +;; ---- [BOOL] Binary logical operations (inverted result) > +;; ------------------------------------------------------------------------- > +;; Includes: > +;; - vmnand.mm > +;; - vmnor.mm > +;; - vmxnor.mm > +;; ------------------------------------------------------------------------- > + > +(define_insn_and_rewrite "*n<optab><mode>" > + [(set (match_operand:VB 0 "register_operand" "=vr") > + (not:VB > + (any_bitwise:VB > + (match_operand:VB 1 "register_operand" " vr") > + (match_operand:VB 2 "register_operand" " vr"))))] > + "TARGET_VECTOR" > + "#" > + "&& can_create_pseudo_p ()" > + { > + insn_code icode = code_for_pred_n (<CODE>, <MODE>mode); > + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); > + DONE; > + } > + [(set_attr "type" "vmalu") > + (set_attr "mode" "<MODE>")]) > + > ;; ========================================================================= > ;; == Comparisons and selects > ;; ========================================================================= > diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc > index 10de5a19937..f71ad9e46a1 100644 > --- a/gcc/config/riscv/riscv-v.cc > +++ b/gcc/config/riscv/riscv-v.cc > @@ -1550,9 +1550,10 @@ expand_vec_cmp_float (rtx target, rtx_code code, rtx op0, rtx op1, > emit_move_insn (target, eq0); > return true; > } > - insn_code icode = code_for_pred_not (mask_mode); > - rtx ops[] = {target, eq0}; > - emit_vlmax_insn (icode, RVV_UNOP, ops); > + > + /* We use one_cmpl<mode>2 to make Combine PASS to combine mask instructions > + into: vmand.mm/vmnor.mm/vmnand.mm/vmnor.mm/vmxnor.mm. */ > + emit_insn (gen_rtx_SET (target, gen_rtx_NOT (mask_mode, eq0))); > return false; > } > > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > new file mode 100644 > index 00000000000..435a59c97f2 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c > @@ -0,0 +1,53 @@ > +/* { dg-do compile } */ > +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ > + > +#include <stdint-gcc.h> > + > +#define b_and(A, B) ((A) & (B)) > +#define b_orr(A, B) ((A) | (B)) > +#define b_xor(A, B) ((A) ^ (B)) > +#define b_nand(A, B) (!((A) & (B))) > +#define b_nor(A, B) (!((A) | (B))) > +#define b_xnor(A, B) (!(A) ^ (B)) > +#define b_andnot(A, B) ((A) & !(B)) > +#define b_ornot(A, B) ((A) | !(B)) > + > +#define LOOP(TYPE, BINOP) \ > + void __attribute__ ((noinline, noclone)) \ > + test_##TYPE##_##BINOP (TYPE *restrict dest, TYPE *restrict src, \ > + TYPE *restrict a, TYPE *restrict b, TYPE *restrict c, \ > + TYPE *restrict d, TYPE fallback, int count) \ > + { \ > + for (int i = 0; i < count; ++i) \ > + { \ > + TYPE srcv = src[i]; \ > + dest[i] = (BINOP (__builtin_isunordered (a[i], b[i]), \ > + __builtin_isunordered (c[i], d[i])) \ > + ? srcv \ > + : fallback); \ > + } \ > + } > + > +#define TEST_BINOP(T, BINOP) \ > + T (float, BINOP) \ > + T (double, BINOP) > + > +#define TEST_ALL(T) \ > + TEST_BINOP (T, b_and) \ > + TEST_BINOP (T, b_orr) \ > + TEST_BINOP (T, b_xor) \ > + TEST_BINOP (T, b_nand) \ > + TEST_BINOP (T, b_nor) \ > + TEST_BINOP (T, b_xnor) \ > + TEST_BINOP (T, b_andnot) \ > + TEST_BINOP (T, b_ornot) > + > +TEST_ALL (LOOP) > + > +/* { dg-final { scan-assembler-times {\tvmand\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmxor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmnot\.m} 4 } } */ > +/* { dg-final { scan-assembler-times {\tvmxnor\.mm} 2 } } */ > +/* { dg-final { scan-assembler-times {\tvmandn\.mm} 4 } } */ > +/* { dg-final { scan-assembler-times {\tvmorn\.mm} 4 } } */ > diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > new file mode 100644 > index 00000000000..6c45c274c33 > --- /dev/null > +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c > @@ -0,0 +1,35 @@ > +/* { dg-do run { target { riscv_vector } } } */ > +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ > + > +#include "vcond-4.c" > + > +#define N 401 > + > +#define RUN_LOOP(TYPE, BINOP) \ > + { \ > + TYPE dest[N], src[N], a[N], b[N], c[N], d[N]; \ > + for (int i = 0; i < N; ++i) \ > + { \ > + src[i] = i * i; \ > + a[i] = i % 5 < 3 ? __builtin_nan("") : i; \ > + b[i] = i % 7 < 4 ? __builtin_nan("") : i; \ > + c[i] = i % 9 < 5 ? __builtin_nan("") : i; \ > + d[i] = i % 11 < 6 ? __builtin_nan("") : i; \ > + asm volatile ("" ::: "memory"); \ > + } \ > + test_##TYPE##_##BINOP (dest, src, a, b, c, d, 100, N); \ > + for (int i = 0; i < N; ++i) \ > + { \ > + int res = BINOP (__builtin_isunordered (a[i], b[i]), \ > + __builtin_isunordered (c[i], d[i])); \ > + if (dest[i] != (res ? src[i] : 100.0)) \ > + __builtin_abort (); \ > + } \ > + } > + > +int __attribute__ ((optimize (1))) > +main (void) > +{ > + TEST_ALL (RUN_LOOP) > + return 0; > +} > -- > 2.36.3 >
diff --git a/gcc/config/riscv/autovec.md b/gcc/config/riscv/autovec.md index 4eeeab624a4..cacf27e4e60 100644 --- a/gcc/config/riscv/autovec.md +++ b/gcc/config/riscv/autovec.md @@ -163,6 +163,101 @@ DONE; }) +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmand.mm +;; - vmxor.mm +;; - vmor.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "<optab><mode>3" + [(set (match_operand:VB 0 "register_operand" "=vr") + (any_bitwise:VB (match_operand:VB 1 "register_operand" " vr") + (match_operand:VB 2 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred (<CODE>, <MODE>mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "<MODE>")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Inverse +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmnot.m +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "one_cmpl<mode>2" + [(set (match_operand:VB 0 "register_operand" "=vr") + (not:VB (match_operand:VB 1 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_not (<MODE>mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_UNOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "<MODE>")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations (inverted second input) +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmandnot.mm +;; - vmornot.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "*<optab>not<mode>" + [(set (match_operand:VB 0 "register_operand" "=vr") + (bitmanip_bitwise:VB + (not:VB (match_operand:VB 2 "register_operand" " vr")) + (match_operand:VB 1 "register_operand" " vr")))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_not (<CODE>, <MODE>mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "<MODE>")]) + +;; ------------------------------------------------------------------------- +;; ---- [BOOL] Binary logical operations (inverted result) +;; ------------------------------------------------------------------------- +;; Includes: +;; - vmnand.mm +;; - vmnor.mm +;; - vmxnor.mm +;; ------------------------------------------------------------------------- + +(define_insn_and_rewrite "*n<optab><mode>" + [(set (match_operand:VB 0 "register_operand" "=vr") + (not:VB + (any_bitwise:VB + (match_operand:VB 1 "register_operand" " vr") + (match_operand:VB 2 "register_operand" " vr"))))] + "TARGET_VECTOR" + "#" + "&& can_create_pseudo_p ()" + { + insn_code icode = code_for_pred_n (<CODE>, <MODE>mode); + riscv_vector::emit_vlmax_insn (icode, riscv_vector::RVV_BINOP, operands); + DONE; + } + [(set_attr "type" "vmalu") + (set_attr "mode" "<MODE>")]) + ;; ========================================================================= ;; == Comparisons and selects ;; ========================================================================= diff --git a/gcc/config/riscv/riscv-v.cc b/gcc/config/riscv/riscv-v.cc index 10de5a19937..f71ad9e46a1 100644 --- a/gcc/config/riscv/riscv-v.cc +++ b/gcc/config/riscv/riscv-v.cc @@ -1550,9 +1550,10 @@ expand_vec_cmp_float (rtx target, rtx_code code, rtx op0, rtx op1, emit_move_insn (target, eq0); return true; } - insn_code icode = code_for_pred_not (mask_mode); - rtx ops[] = {target, eq0}; - emit_vlmax_insn (icode, RVV_UNOP, ops); + + /* We use one_cmpl<mode>2 to make Combine PASS to combine mask instructions + into: vmand.mm/vmnor.mm/vmnand.mm/vmnor.mm/vmxnor.mm. */ + emit_insn (gen_rtx_SET (target, gen_rtx_NOT (mask_mode, eq0))); return false; } diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c new file mode 100644 index 00000000000..435a59c97f2 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c @@ -0,0 +1,53 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-march=rv32gcv -mabi=ilp32d --param=riscv-autovec-preference=scalable" } */ + +#include <stdint-gcc.h> + +#define b_and(A, B) ((A) & (B)) +#define b_orr(A, B) ((A) | (B)) +#define b_xor(A, B) ((A) ^ (B)) +#define b_nand(A, B) (!((A) & (B))) +#define b_nor(A, B) (!((A) | (B))) +#define b_xnor(A, B) (!(A) ^ (B)) +#define b_andnot(A, B) ((A) & !(B)) +#define b_ornot(A, B) ((A) | !(B)) + +#define LOOP(TYPE, BINOP) \ + void __attribute__ ((noinline, noclone)) \ + test_##TYPE##_##BINOP (TYPE *restrict dest, TYPE *restrict src, \ + TYPE *restrict a, TYPE *restrict b, TYPE *restrict c, \ + TYPE *restrict d, TYPE fallback, int count) \ + { \ + for (int i = 0; i < count; ++i) \ + { \ + TYPE srcv = src[i]; \ + dest[i] = (BINOP (__builtin_isunordered (a[i], b[i]), \ + __builtin_isunordered (c[i], d[i])) \ + ? srcv \ + : fallback); \ + } \ + } + +#define TEST_BINOP(T, BINOP) \ + T (float, BINOP) \ + T (double, BINOP) + +#define TEST_ALL(T) \ + TEST_BINOP (T, b_and) \ + TEST_BINOP (T, b_orr) \ + TEST_BINOP (T, b_xor) \ + TEST_BINOP (T, b_nand) \ + TEST_BINOP (T, b_nor) \ + TEST_BINOP (T, b_xnor) \ + TEST_BINOP (T, b_andnot) \ + TEST_BINOP (T, b_ornot) + +TEST_ALL (LOOP) + +/* { dg-final { scan-assembler-times {\tvmand\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmxor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmnot\.m} 4 } } */ +/* { dg-final { scan-assembler-times {\tvmxnor\.mm} 2 } } */ +/* { dg-final { scan-assembler-times {\tvmandn\.mm} 4 } } */ +/* { dg-final { scan-assembler-times {\tvmorn\.mm} 4 } } */ diff --git a/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c new file mode 100644 index 00000000000..6c45c274c33 --- /dev/null +++ b/gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c @@ -0,0 +1,35 @@ +/* { dg-do run { target { riscv_vector } } } */ +/* { dg-additional-options "--param=riscv-autovec-preference=scalable" } */ + +#include "vcond-4.c" + +#define N 401 + +#define RUN_LOOP(TYPE, BINOP) \ + { \ + TYPE dest[N], src[N], a[N], b[N], c[N], d[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + src[i] = i * i; \ + a[i] = i % 5 < 3 ? __builtin_nan("") : i; \ + b[i] = i % 7 < 4 ? __builtin_nan("") : i; \ + c[i] = i % 9 < 5 ? __builtin_nan("") : i; \ + d[i] = i % 11 < 6 ? __builtin_nan("") : i; \ + asm volatile ("" ::: "memory"); \ + } \ + test_##TYPE##_##BINOP (dest, src, a, b, c, d, 100, N); \ + for (int i = 0; i < N; ++i) \ + { \ + int res = BINOP (__builtin_isunordered (a[i], b[i]), \ + __builtin_isunordered (c[i], d[i])); \ + if (dest[i] != (res ? src[i] : 100.0)) \ + __builtin_abort (); \ + } \ + } + +int __attribute__ ((optimize (1))) +main (void) +{ + TEST_ALL (RUN_LOOP) + return 0; +}
From: Juzhe-Zhong <juzhe.zhong@rivai.ai> This patch is adding mask logic auto-vectorization. define the pattern as "define_insn_and_split" to allow combine PASS easily combine series instructions. For example: combine vmxor.mm + vmnot.m into vmxnor.mm Build success and regression PASS Ok for trunk ? gcc/ChangeLog: * config/riscv/autovec.md (<optab><mode>3): New pattern. (one_cmpl<mode>2): Ditto. (*<optab>not<mode>): Ditto. (*n<optab><mode>): Ditto. * config/riscv/riscv-v.cc (expand_vec_cmp_float): Change to one_cmpl. gcc/testsuite/ChangeLog: * gcc.target/riscv/rvv/autovec/cmp/vcond-4.c: New test. * gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c: New test. --- gcc/config/riscv/autovec.md | 95 +++++++++++++++++++ gcc/config/riscv/riscv-v.cc | 7 +- .../riscv/rvv/autovec/cmp/vcond-4.c | 53 +++++++++++ .../riscv/rvv/autovec/cmp/vcond_run-4.c | 35 +++++++ 4 files changed, 187 insertions(+), 3 deletions(-) create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond-4.c create mode 100644 gcc/testsuite/gcc.target/riscv/rvv/autovec/cmp/vcond_run-4.c