| Message ID | ZkSOUIG1FPZlgCB5@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | AArch64: support conditional early clobbers on certain operations. | expand |
Hi Tamar, On Wed, 15 May 2024 at 11:28, Tamar Christina <tamar.christina@arm.com> wrote: > Hi All, > > This converts the single alternative patterns to the new compact syntax > such > that when I add the new alternatives it's clearer what's being changed. > > Note that this will spew out a bunch of warnings from geninsn as it'll > warn that > @ is useless for a single alternative pattern. These are not fatal so > won't > break the build and are only temporary. > > No change in functionality is expected with this patch. > > Bootstrapped Regtested on aarch64-none-linux-gnu and no issues. > > Ok for master? Ok. Thanks, Kyrill > > Thanks, > Tamar > > gcc/ChangeLog: > > * config/aarch64/aarch64-sve.md (and<mode>3, > @aarch64_pred_<optab><mode>_z, *<optab><mode>3_cc, > *<optab><mode>3_ptest, aarch64_pred_<nlogical><mode>_z, > *<nlogical><mode>3_cc, *<nlogical><mode>3_ptest, > aarch64_pred_<logical_nn><mode>_z, *<logical_nn><mode>3_cc, > *<logical_nn><mode>3_ptest, *cmp<cmp_op><mode>_ptest, > @aarch64_pred_cmp<cmp_op><mode>_wide, > *aarch64_pred_cmp<cmp_op><mode>_wide_cc, > *aarch64_pred_cmp<cmp_op><mode>_wide_ptest, > *aarch64_brk<brk_op>_cc, > *aarch64_brk<brk_op>_ptest, @aarch64_brk<brk_op>, *aarch64_brkn_cc, > *aarch64_brkn_ptest, *aarch64_brk<brk_op>_cc, > *aarch64_brk<brk_op>_ptest, aarch64_rdffr_z, > *aarch64_rdffr_z_ptest, > *aarch64_rdffr_ptest, *aarch64_rdffr_z_cc, *aarch64_rdffr_cc): > Convert > to compact syntax. > * config/aarch64/aarch64-sve2.md > (@aarch64_pred_<sve_int_op><mode>): Likewise. > > --- > diff --git a/gcc/config/aarch64/aarch64-sve.md > b/gcc/config/aarch64/aarch64-sve.md > index > 0434358122d2fde71bd0e0f850338e739e9be02c..839ab0627747d7a49bef7b0192ee9e7a42587ca0 > 100644 > --- a/gcc/config/aarch64/aarch64-sve.md > +++ b/gcc/config/aarch64/aarch64-sve.md > @@ -1156,76 +1156,86 @@ (define_insn "aarch64_rdffr" > > ;; Likewise with zero predication. > (define_insn "aarch64_rdffr_z" > - [(set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + [(set (match_operand:VNx16BI 0 "register_operand") > (and:VNx16BI > (reg:VNx16BI FFRT_REGNUM) > - (match_operand:VNx16BI 1 "register_operand" "Upa")))] > + (match_operand:VNx16BI 1 "register_operand")))] > "TARGET_SVE && TARGET_NON_STREAMING" > - "rdffr\t%0.b, %1/z" > + {@ [ cons: =0, 1 ] > + [ Upa , Upa ] rdffr\t%0.b, %1/z > + } > ) > > ;; Read the FFR to test for a fault, without using the predicate result. > (define_insn "*aarch64_rdffr_z_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 2 "aarch64_sve_ptrue_flag") > (and:VNx16BI > (reg:VNx16BI FFRT_REGNUM) > (match_dup 1))] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE && TARGET_NON_STREAMING" > - "rdffrs\t%0.b, %1/z" > + {@ [ cons: =0, 1 , 2 ] > + [ Upa , Upa, ] rdffrs\t%0.b, %1/z > + } > ) > > ;; Same for unpredicated RDFFR when tested with a known PTRUE. > (define_insn "*aarch64_rdffr_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (const_int SVE_KNOWN_PTRUE) > (reg:VNx16BI FFRT_REGNUM)] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE && TARGET_NON_STREAMING" > - "rdffrs\t%0.b, %1/z" > + {@ [ cons: =0, 1 ] > + [ Upa , Upa ] rdffrs\t%0.b, %1/z > + } > ) > > ;; Read the FFR with zero predication and test the result. > (define_insn "*aarch64_rdffr_z_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 2 "aarch64_sve_ptrue_flag") > (and:VNx16BI > (reg:VNx16BI FFRT_REGNUM) > (match_dup 1))] > UNSPEC_PTEST)) > - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + (set (match_operand:VNx16BI 0 "register_operand") > (and:VNx16BI > (reg:VNx16BI FFRT_REGNUM) > (match_dup 1)))] > "TARGET_SVE && TARGET_NON_STREAMING" > - "rdffrs\t%0.b, %1/z" > + {@ [ cons: =0, 1 , 2 ] > + [ Upa , Upa, ] rdffrs\t%0.b, %1/z > + } > ) > > ;; Same for unpredicated RDFFR when tested with a known PTRUE. > (define_insn "*aarch64_rdffr_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (const_int SVE_KNOWN_PTRUE) > (reg:VNx16BI FFRT_REGNUM)] > UNSPEC_PTEST)) > - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + (set (match_operand:VNx16BI 0 "register_operand") > (reg:VNx16BI FFRT_REGNUM))] > "TARGET_SVE && TARGET_NON_STREAMING" > - "rdffrs\t%0.b, %1/z" > + {@ [ cons: =0, 1 , 2 ] > + [ Upa , Upa, ] rdffrs\t%0.b, %1/z > + } > ) > > ;; [R3 in the block comment above about FFR handling] > @@ -6637,11 +6647,13 @@ (define_insn "@aarch64_pred_<optab><mode>" > ;; Doubling the second operand is the preferred implementation > ;; of the MOV alias, so we use that instead of %1/z, %1, %2. > (define_insn "and<mode>3" > - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > - (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand" "Upa") > - (match_operand:PRED_ALL 2 "register_operand" > "Upa")))] > + [(set (match_operand:PRED_ALL 0 "register_operand") > + (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") > + (match_operand:PRED_ALL 2 "register_operand")))] > "TARGET_SVE" > - "and\t%0.b, %1/z, %2.b, %2.b" > + {@ [ cons: =0, 1 , 2 ] > + [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b > + } > ) > > ;; Unpredicated predicate EOR and ORR. > @@ -6660,14 +6672,16 @@ (define_expand "<optab><mode>3" > > ;; Predicated predicate AND, EOR and ORR. > (define_insn "@aarch64_pred_<optab><mode>_z" > - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + [(set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL > (LOGICAL:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa") > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] > + (match_operand:PRED_ALL 2 "register_operand") > + (match_operand:PRED_ALL 3 "register_operand")) > + (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - "<logical>\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 ] > + [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Perform a logical operation on operands 2 and 3, using operand 1 as > @@ -6676,38 +6690,42 @@ (define_insn "@aarch64_pred_<optab><mode>_z" > (define_insn "*<optab><mode>3_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (LOGICAL:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa") > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > + (match_operand:PRED_ALL 2 "register_operand") > + (match_operand:PRED_ALL 3 "register_operand")) > (match_dup 4))] > UNSPEC_PTEST)) > - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + (set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) > (match_dup 4)))] > "TARGET_SVE" > - "<logical>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Same with just the flags result. > (define_insn "*<optab><mode>3_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (LOGICAL:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa") > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > + (match_operand:PRED_ALL 2 "register_operand") > + (match_operand:PRED_ALL 3 "register_operand")) > (match_dup 4))] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "<logical>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; > ------------------------------------------------------------------------- > @@ -6720,56 +6738,62 @@ (define_insn "*<optab><mode>3_ptest" > > ;; Predicated predicate BIC and ORN. > (define_insn "aarch64_pred_<nlogical><mode>_z" > - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + [(set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > - (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand" > "Upa")) > - (match_operand:PRED_ALL 2 "register_operand" "Upa")) > - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] > + (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand")) > + (match_operand:PRED_ALL 2 "register_operand")) > + (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - "<nlogical>\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 ] > + [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Same, but set the flags as a side-effect. > (define_insn "*<nlogical><mode>3_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > (not:PRED_ALL > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > - (match_operand:PRED_ALL 2 "register_operand" "Upa")) > + (match_operand:PRED_ALL 3 "register_operand")) > + (match_operand:PRED_ALL 2 "register_operand")) > (match_dup 4))] > UNSPEC_PTEST)) > - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + (set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL (NLOGICAL:PRED_ALL > (not:PRED_ALL (match_dup 3)) > (match_dup 2)) > (match_dup 4)))] > "TARGET_SVE" > - "<nlogical>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Same with just the flags result. > (define_insn "*<nlogical><mode>3_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > (not:PRED_ALL > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > - (match_operand:PRED_ALL 2 "register_operand" "Upa")) > + (match_operand:PRED_ALL 3 "register_operand")) > + (match_operand:PRED_ALL 2 "register_operand")) > (match_dup 4))] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "<nlogical>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <nlogical>s\t%0.b, %1/z, %2.b, > %3.b > + } > ) > > ;; > ------------------------------------------------------------------------- > @@ -6782,58 +6806,64 @@ (define_insn "*<nlogical><mode>3_ptest" > > ;; Predicated predicate NAND and NOR. > (define_insn "aarch64_pred_<logical_nn><mode>_z" > - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + [(set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > - (not:PRED_ALL (match_operand:PRED_ALL 2 "register_operand" > "Upa")) > - (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand" > "Upa"))) > - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] > + (not:PRED_ALL (match_operand:PRED_ALL 2 "register_operand")) > + (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) > + (match_operand:PRED_ALL 1 "register_operand")))] > "TARGET_SVE" > - "<logical_nn>\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 ] > + [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Same, but set the flags as a side-effect. > (define_insn "*<logical_nn><mode>3_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > (not:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa")) > + (match_operand:PRED_ALL 2 "register_operand")) > (not:PRED_ALL > - (match_operand:PRED_ALL 3 "register_operand" "Upa"))) > + (match_operand:PRED_ALL 3 "register_operand"))) > (match_dup 4))] > UNSPEC_PTEST)) > - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + (set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL (NLOGICAL:PRED_ALL > (not:PRED_ALL (match_dup 2)) > (not:PRED_ALL (match_dup 3))) > (match_dup 4)))] > "TARGET_SVE" > - "<logical_nn>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <logical_nn>s\t%0.b, %1/z, %2.b, > %3.b > + } > ) > > ;; Same with just the flags result. > (define_insn "*<logical_nn><mode>3_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (NLOGICAL:PRED_ALL > (not:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa")) > + (match_operand:PRED_ALL 2 "register_operand")) > (not:PRED_ALL > - (match_operand:PRED_ALL 3 "register_operand" "Upa"))) > + (match_operand:PRED_ALL 3 "register_operand"))) > (match_dup 4))] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "<logical_nn>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <logical_nn>s\t%0.b, %1/z, %2.b, > %3.b > + } > ) > > ;; > ========================================================================= > @@ -8133,12 +8163,12 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_ptest" > (match_operand:SVE_I 3 > "aarch64_sve_cmp_<sve_imm_con>_operand"))] > UNSPEC_PRED_Z)] > UNSPEC_PTEST)) > - (clobber (match_scratch:<VPRED> 0 "=Upa, Upa"))] > + (clobber (match_scratch:<VPRED> 0))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - {@ [ cons: 1 , 2 , 3 ] > - [ Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, #%3 > - [ Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %2.<Vetype>, %3.<Vetype> > + {@ [ cons: =0, 1 , 2 , 3 ] > + [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, #%3 > + [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, %3.<Vetype> > } > "&& !rtx_equal_p (operands[4], operands[6])" > { > @@ -8180,18 +8210,20 @@ (define_insn_and_split "*cmp<cmp_op><mode>_and" > > ;; Predicated integer wide comparisons. > (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" > - [(set (match_operand:<VPRED> 0 "register_operand" "=Upa") > + [(set (match_operand:<VPRED> 0 "register_operand") > (unspec:<VPRED> > - [(match_operand:VNx16BI 1 "register_operand" "Upl") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand:SI 2 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:SVE_FULL_BHSI 3 "register_operand" "w") > - (match_operand:VNx2DI 4 "register_operand" "w")] > + [(match_operand:SVE_FULL_BHSI 3 "register_operand") > + (match_operand:VNx2DI 4 "register_operand")] > SVE_COND_INT_CMP_WIDE)] > UNSPEC_PRED_Z)) > (clobber (reg:CC_NZC CC_REGNUM))] > "TARGET_SVE" > - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d" > + {@ [ cons: =0, 1 , 2, 3, 4 ] > + [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.d > + } > ) > > ;; Predicated integer wide comparisons in which both the flag and > @@ -8199,19 +8231,19 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" > (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upl") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:VNx16BI 6 "register_operand" "Upl") > + [(match_operand:VNx16BI 6 "register_operand") > (match_operand:SI 7 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:SVE_FULL_BHSI 2 "register_operand" "w") > - (match_operand:VNx2DI 3 "register_operand" "w")] > + [(match_operand:SVE_FULL_BHSI 2 "register_operand") > + (match_operand:VNx2DI 3 "register_operand")] > SVE_COND_INT_CMP_WIDE)] > UNSPEC_PRED_Z)] > UNSPEC_PTEST)) > - (set (match_operand:<VPRED> 0 "register_operand" "=Upa") > + (set (match_operand:<VPRED> 0 "register_operand") > (unspec:<VPRED> > [(match_dup 6) > (match_dup 7) > @@ -8222,7 +8254,9 @@ (define_insn > "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" > UNSPEC_PRED_Z))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d" > + {@ [ cons: =0, 1 , 2, 3, 4, 5, 6 , 7 ] > + [ Upa , Upl, w, w, , , Upl, ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, %3.d > + } > ) > > ;; Predicated integer wide comparisons in which only the flags result > @@ -8230,22 +8264,24 @@ (define_insn > "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" > (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upl") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:VNx16BI 6 "register_operand" "Upl") > + [(match_operand:VNx16BI 6 "register_operand") > (match_operand:SI 7 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:SVE_FULL_BHSI 2 "register_operand" "w") > - (match_operand:VNx2DI 3 "register_operand" "w")] > + [(match_operand:SVE_FULL_BHSI 2 "register_operand") > + (match_operand:VNx2DI 3 "register_operand")] > SVE_COND_INT_CMP_WIDE)] > UNSPEC_PRED_Z)] > UNSPEC_PTEST)) > - (clobber (match_scratch:<VPRED> 0 "=Upa"))] > + (clobber (match_scratch:<VPRED> 0))] > "TARGET_SVE > && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" > - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d" > + {@ [ cons: =0, 1 , 2, 3, 4, 5, 6 , 7 ] > + [ Upa , Upl, w, w, , , Upl, ] cmp<cmp_op>\t%0.<Vetype>, > %1/z, %2.<Vetype>, %3.d > + } > ) > > ;; > ------------------------------------------------------------------------- > @@ -9922,41 +9958,45 @@ (define_insn "@aarch64_brk<brk_op>" > (define_insn "*aarch64_brk<brk_op>_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 4 "aarch64_sve_ptrue_flag") > (unspec:VNx16BI > [(match_dup 1) > - (match_operand:VNx16BI 2 "register_operand" "Upa") > + (match_operand:VNx16BI 2 "register_operand") > (match_operand:VNx16BI 3 "aarch64_simd_imm_zero")] > SVE_BRK_UNARY)] > UNSPEC_PTEST)) > - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + (set (match_operand:VNx16BI 0 "register_operand") > (unspec:VNx16BI > [(match_dup 1) > (match_dup 2) > (match_dup 3)] > SVE_BRK_UNARY))] > "TARGET_SVE" > - "brk<brk_op>s\t%0.b, %1/z, %2.b" > + {@ [ cons: =0, 1 , 2 , 3, 4 ] > + [ Upa , Upa, Upa, , ] brk<brk_op>s\t%0.b, %1/z, %2.b > + } > ) > > ;; Same, but with only the flags result being interesting. > (define_insn "*aarch64_brk<brk_op>_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 4 "aarch64_sve_ptrue_flag") > (unspec:VNx16BI > [(match_dup 1) > - (match_operand:VNx16BI 2 "register_operand" "Upa") > + (match_operand:VNx16BI 2 "register_operand") > (match_operand:VNx16BI 3 "aarch64_simd_imm_zero")] > SVE_BRK_UNARY)] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "brk<brk_op>s\t%0.b, %1/z, %2.b" > + {@ [ cons: =0, 1 , 2 , 3, 4 ] > + [ Upa , Upa, Upa, , ] brk<brk_op>s\t%0.b, %1/z, %2.b > + } > ) > > ;; > ------------------------------------------------------------------------- > @@ -9973,14 +10013,16 @@ (define_insn "*aarch64_brk<brk_op>_ptest" > > ;; Binary BRKs (BRKN, BRKPA, BRKPB). > (define_insn "@aarch64_brk<brk_op>" > - [(set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + [(set (match_operand:VNx16BI 0 "register_operand") > (unspec:VNx16BI > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > - (match_operand:VNx16BI 2 "register_operand" "Upa") > - (match_operand:VNx16BI 3 "register_operand" "<brk_reg_con>")] > + [(match_operand:VNx16BI 1 "register_operand") > + (match_operand:VNx16BI 2 "register_operand") > + (match_operand:VNx16BI 3 "register_operand")] > SVE_BRK_BINARY))] > "TARGET_SVE" > - "brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b" > + {@ [ cons: =0, 1 , 2 , 3 ] > + [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, > %<brk_reg_opno>.b > + } > ) > > ;; BRKN, producing both a predicate and a flags result. Unlike other > @@ -9992,19 +10034,21 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" > (match_operand:VNx16BI 5) > (const_int SVE_KNOWN_PTRUE) > (unspec:VNx16BI > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > - (match_operand:VNx16BI 2 "register_operand" "Upa") > - (match_operand:VNx16BI 3 "register_operand" "0")] > + [(match_operand:VNx16BI 1 "register_operand") > + (match_operand:VNx16BI 2 "register_operand") > + (match_operand:VNx16BI 3 "register_operand")] > UNSPEC_BRKN)] > UNSPEC_PTEST)) > - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + (set (match_operand:VNx16BI 0 "register_operand") > (unspec:VNx16BI > [(match_dup 1) > (match_dup 2) > (match_dup 3)] > UNSPEC_BRKN))] > "TARGET_SVE" > - "brkns\t%0.b, %1/z, %2.b, %0.b" > + {@ [ cons: =0, 1 , 2 , 3, 4, 5 ] > + [ Upa , Upa, Upa, 0, , ] brkns\t%0.b, %1/z, %2.b, %0.b > + } > "&& (operands[4] != CONST0_RTX (VNx16BImode) > || operands[5] != CONST0_RTX (VNx16BImode))" > { > @@ -10021,14 +10065,16 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" > (match_operand:VNx16BI 5) > (const_int SVE_KNOWN_PTRUE) > (unspec:VNx16BI > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > - (match_operand:VNx16BI 2 "register_operand" "Upa") > - (match_operand:VNx16BI 3 "register_operand" "0")] > + [(match_operand:VNx16BI 1 "register_operand") > + (match_operand:VNx16BI 2 "register_operand") > + (match_operand:VNx16BI 3 "register_operand")] > UNSPEC_BRKN)] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "brkns\t%0.b, %1/z, %2.b, %0.b" > + {@ [ cons: =0, 1 , 2 , 3, 4, 5 ] > + [ Upa , Upa, Upa, 0, , ] brkns\t%0.b, %1/z, %2.b, %0.b > + } > "&& (operands[4] != CONST0_RTX (VNx16BImode) > || operands[5] != CONST0_RTX (VNx16BImode))" > { > @@ -10041,41 +10087,45 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" > (define_insn "*aarch64_brk<brk_op>_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 4 "aarch64_sve_ptrue_flag") > (unspec:VNx16BI > [(match_dup 1) > - (match_operand:VNx16BI 2 "register_operand" "Upa") > - (match_operand:VNx16BI 3 "register_operand" "Upa")] > + (match_operand:VNx16BI 2 "register_operand") > + (match_operand:VNx16BI 3 "register_operand")] > SVE_BRKP)] > UNSPEC_PTEST)) > - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") > + (set (match_operand:VNx16BI 0 "register_operand") > (unspec:VNx16BI > [(match_dup 1) > (match_dup 2) > (match_dup 3)] > SVE_BRKP))] > "TARGET_SVE" > - "brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4 ] > + [ Upa , Upa, Upa, Upa, ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; Same, but with only the flags result being interesting. > (define_insn "*aarch64_brk<brk_op>_ptest" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_dup 1) > (match_operand:SI 4 "aarch64_sve_ptrue_flag") > (unspec:VNx16BI > [(match_dup 1) > - (match_operand:VNx16BI 2 "register_operand" "Upa") > - (match_operand:VNx16BI 3 "register_operand" "Upa")] > + (match_operand:VNx16BI 2 "register_operand") > + (match_operand:VNx16BI 3 "register_operand")] > SVE_BRKP)] > UNSPEC_PTEST)) > - (clobber (match_scratch:VNx16BI 0 "=Upa"))] > + (clobber (match_scratch:VNx16BI 0))] > "TARGET_SVE" > - "brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4 ] > + [ Upa , Upa, Upa, Upa, ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b > + } > ) > > ;; > ------------------------------------------------------------------------- > diff --git a/gcc/config/aarch64/aarch64-sve2.md > b/gcc/config/aarch64/aarch64-sve2.md > index > 934e57055d3419e5dcc89b473fd110a0d4978b4f..aa12baf48355358ca4fefe88157df3aac6eb09bd > 100644 > --- a/gcc/config/aarch64/aarch64-sve2.md > +++ b/gcc/config/aarch64/aarch64-sve2.md > @@ -3338,18 +3338,20 @@ (define_insn "@aarch64_sve2_histseg<mode>" > > ;; Predicated string matching. > (define_insn "@aarch64_pred_<sve_int_op><mode>" > - [(set (match_operand:<VPRED> 0 "register_operand" "=Upa") > + [(set (match_operand:<VPRED> 0 "register_operand") > (unspec:<VPRED> > - [(match_operand:<VPRED> 1 "register_operand" "Upl") > + [(match_operand:<VPRED> 1 "register_operand") > (match_operand:SI 2 "aarch64_sve_ptrue_flag") > (unspec:<VPRED> > - [(match_operand:SVE_FULL_BHI 3 "register_operand" "w") > - (match_operand:SVE_FULL_BHI 4 "register_operand" "w")] > + [(match_operand:SVE_FULL_BHI 3 "register_operand") > + (match_operand:SVE_FULL_BHI 4 "register_operand")] > SVE2_MATCH)] > UNSPEC_PRED_Z)) > (clobber (reg:CC_NZC CC_REGNUM))] > "TARGET_SVE2 && TARGET_NON_STREAMING" > - "<sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype>" > + {@ [ cons: =0, 1 , 2, 3, 4 ] > + [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, > %3.<Vetype>, %4.<Vetype> > + } > ) > > ;; Predicated string matching in which both the flag and predicate results > > > > > -- >
Thanks for doing this a pre-patch. Minor request below: Tamar Christina <tamar.christina@arm.com> writes: > ;; Perform a logical operation on operands 2 and 3, using operand 1 as > @@ -6676,38 +6690,42 @@ (define_insn "@aarch64_pred_<optab><mode>_z" > (define_insn "*<optab><mode>3_cc" > [(set (reg:CC_NZC CC_REGNUM) > (unspec:CC_NZC > - [(match_operand:VNx16BI 1 "register_operand" "Upa") > + [(match_operand:VNx16BI 1 "register_operand") > (match_operand 4) > (match_operand:SI 5 "aarch64_sve_ptrue_flag") > (and:PRED_ALL > (LOGICAL:PRED_ALL > - (match_operand:PRED_ALL 2 "register_operand" "Upa") > - (match_operand:PRED_ALL 3 "register_operand" "Upa")) > + (match_operand:PRED_ALL 2 "register_operand") > + (match_operand:PRED_ALL 3 "register_operand")) > (match_dup 4))] > UNSPEC_PTEST)) > - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") > + (set (match_operand:PRED_ALL 0 "register_operand") > (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) > (match_dup 4)))] > "TARGET_SVE" > - "<logical>s\t%0.b, %1/z, %2.b, %3.b" > + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] > + [ Upa , Upa, Upa, Upa, , ] <logical>s\t%0.b, %1/z, %2.b, %3.b > + } > ) Could we leave out these empty trailing constraints? They're quite common in SVE & SME patterns and are specifically not meant to influence instruction selection. E.g. we've done the same thing for *cnot<mode> (to pick a random example). Agree with Kyrill's ok otherwise. Richard
diff --git a/gcc/config/aarch64/aarch64-sve.md b/gcc/config/aarch64/aarch64-sve.md index 0434358122d2fde71bd0e0f850338e739e9be02c..839ab0627747d7a49bef7b0192ee9e7a42587ca0 100644 --- a/gcc/config/aarch64/aarch64-sve.md +++ b/gcc/config/aarch64/aarch64-sve.md @@ -1156,76 +1156,86 @@ (define_insn "aarch64_rdffr" ;; Likewise with zero predication. (define_insn "aarch64_rdffr_z" - [(set (match_operand:VNx16BI 0 "register_operand" "=Upa") + [(set (match_operand:VNx16BI 0 "register_operand") (and:VNx16BI (reg:VNx16BI FFRT_REGNUM) - (match_operand:VNx16BI 1 "register_operand" "Upa")))] + (match_operand:VNx16BI 1 "register_operand")))] "TARGET_SVE && TARGET_NON_STREAMING" - "rdffr\t%0.b, %1/z" + {@ [ cons: =0, 1 ] + [ Upa , Upa ] rdffr\t%0.b, %1/z + } ) ;; Read the FFR to test for a fault, without using the predicate result. (define_insn "*aarch64_rdffr_z_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 2 "aarch64_sve_ptrue_flag") (and:VNx16BI (reg:VNx16BI FFRT_REGNUM) (match_dup 1))] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - "rdffrs\t%0.b, %1/z" + {@ [ cons: =0, 1 , 2 ] + [ Upa , Upa, ] rdffrs\t%0.b, %1/z + } ) ;; Same for unpredicated RDFFR when tested with a known PTRUE. (define_insn "*aarch64_rdffr_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (const_int SVE_KNOWN_PTRUE) (reg:VNx16BI FFRT_REGNUM)] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE && TARGET_NON_STREAMING" - "rdffrs\t%0.b, %1/z" + {@ [ cons: =0, 1 ] + [ Upa , Upa ] rdffrs\t%0.b, %1/z + } ) ;; Read the FFR with zero predication and test the result. (define_insn "*aarch64_rdffr_z_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 2 "aarch64_sve_ptrue_flag") (and:VNx16BI (reg:VNx16BI FFRT_REGNUM) (match_dup 1))] UNSPEC_PTEST)) - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") + (set (match_operand:VNx16BI 0 "register_operand") (and:VNx16BI (reg:VNx16BI FFRT_REGNUM) (match_dup 1)))] "TARGET_SVE && TARGET_NON_STREAMING" - "rdffrs\t%0.b, %1/z" + {@ [ cons: =0, 1 , 2 ] + [ Upa , Upa, ] rdffrs\t%0.b, %1/z + } ) ;; Same for unpredicated RDFFR when tested with a known PTRUE. (define_insn "*aarch64_rdffr_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (const_int SVE_KNOWN_PTRUE) (reg:VNx16BI FFRT_REGNUM)] UNSPEC_PTEST)) - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") + (set (match_operand:VNx16BI 0 "register_operand") (reg:VNx16BI FFRT_REGNUM))] "TARGET_SVE && TARGET_NON_STREAMING" - "rdffrs\t%0.b, %1/z" + {@ [ cons: =0, 1 , 2 ] + [ Upa , Upa, ] rdffrs\t%0.b, %1/z + } ) ;; [R3 in the block comment above about FFR handling] @@ -6637,11 +6647,13 @@ (define_insn "@aarch64_pred_<optab><mode>" ;; Doubling the second operand is the preferred implementation ;; of the MOV alias, so we use that instead of %1/z, %1, %2. (define_insn "and<mode>3" - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") - (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand" "Upa") - (match_operand:PRED_ALL 2 "register_operand" "Upa")))] + [(set (match_operand:PRED_ALL 0 "register_operand") + (and:PRED_ALL (match_operand:PRED_ALL 1 "register_operand") + (match_operand:PRED_ALL 2 "register_operand")))] "TARGET_SVE" - "and\t%0.b, %1/z, %2.b, %2.b" + {@ [ cons: =0, 1 , 2 ] + [ Upa , Upa, Upa ] and\t%0.b, %1/z, %2.b, %2.b + } ) ;; Unpredicated predicate EOR and ORR. @@ -6660,14 +6672,16 @@ (define_expand "<optab><mode>3" ;; Predicated predicate AND, EOR and ORR. (define_insn "@aarch64_pred_<optab><mode>_z" - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + [(set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (LOGICAL:PRED_ALL - (match_operand:PRED_ALL 2 "register_operand" "Upa") - (match_operand:PRED_ALL 3 "register_operand" "Upa")) - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] + (match_operand:PRED_ALL 2 "register_operand") + (match_operand:PRED_ALL 3 "register_operand")) + (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - "<logical>\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 ] + [ Upa , Upa, Upa, Upa ] <logical>\t%0.b, %1/z, %2.b, %3.b + } ) ;; Perform a logical operation on operands 2 and 3, using operand 1 as @@ -6676,38 +6690,42 @@ (define_insn "@aarch64_pred_<optab><mode>_z" (define_insn "*<optab><mode>3_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (LOGICAL:PRED_ALL - (match_operand:PRED_ALL 2 "register_operand" "Upa") - (match_operand:PRED_ALL 3 "register_operand" "Upa")) + (match_operand:PRED_ALL 2 "register_operand") + (match_operand:PRED_ALL 3 "register_operand")) (match_dup 4))] UNSPEC_PTEST)) - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + (set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (LOGICAL:PRED_ALL (match_dup 2) (match_dup 3)) (match_dup 4)))] "TARGET_SVE" - "<logical>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <logical>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same with just the flags result. (define_insn "*<optab><mode>3_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (LOGICAL:PRED_ALL - (match_operand:PRED_ALL 2 "register_operand" "Upa") - (match_operand:PRED_ALL 3 "register_operand" "Upa")) + (match_operand:PRED_ALL 2 "register_operand") + (match_operand:PRED_ALL 3 "register_operand")) (match_dup 4))] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "<logical>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <logical>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; ------------------------------------------------------------------------- @@ -6720,56 +6738,62 @@ (define_insn "*<optab><mode>3_ptest" ;; Predicated predicate BIC and ORN. (define_insn "aarch64_pred_<nlogical><mode>_z" - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + [(set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (NLOGICAL:PRED_ALL - (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand" "Upa")) - (match_operand:PRED_ALL 2 "register_operand" "Upa")) - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] + (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand")) + (match_operand:PRED_ALL 2 "register_operand")) + (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - "<nlogical>\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 ] + [ Upa , Upa, Upa, Upa ] <nlogical>\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same, but set the flags as a side-effect. (define_insn "*<nlogical><mode>3_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL - (match_operand:PRED_ALL 3 "register_operand" "Upa")) - (match_operand:PRED_ALL 2 "register_operand" "Upa")) + (match_operand:PRED_ALL 3 "register_operand")) + (match_operand:PRED_ALL 2 "register_operand")) (match_dup 4))] UNSPEC_PTEST)) - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + (set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL (match_dup 3)) (match_dup 2)) (match_dup 4)))] "TARGET_SVE" - "<nlogical>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same with just the flags result. (define_insn "*<nlogical><mode>3_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL - (match_operand:PRED_ALL 3 "register_operand" "Upa")) - (match_operand:PRED_ALL 2 "register_operand" "Upa")) + (match_operand:PRED_ALL 3 "register_operand")) + (match_operand:PRED_ALL 2 "register_operand")) (match_dup 4))] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "<nlogical>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <nlogical>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; ------------------------------------------------------------------------- @@ -6782,58 +6806,64 @@ (define_insn "*<nlogical><mode>3_ptest" ;; Predicated predicate NAND and NOR. (define_insn "aarch64_pred_<logical_nn><mode>_z" - [(set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + [(set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (NLOGICAL:PRED_ALL - (not:PRED_ALL (match_operand:PRED_ALL 2 "register_operand" "Upa")) - (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand" "Upa"))) - (match_operand:PRED_ALL 1 "register_operand" "Upa")))] + (not:PRED_ALL (match_operand:PRED_ALL 2 "register_operand")) + (not:PRED_ALL (match_operand:PRED_ALL 3 "register_operand"))) + (match_operand:PRED_ALL 1 "register_operand")))] "TARGET_SVE" - "<logical_nn>\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 ] + [ Upa , Upa, Upa, Upa ] <logical_nn>\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same, but set the flags as a side-effect. (define_insn "*<logical_nn><mode>3_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL - (match_operand:PRED_ALL 2 "register_operand" "Upa")) + (match_operand:PRED_ALL 2 "register_operand")) (not:PRED_ALL - (match_operand:PRED_ALL 3 "register_operand" "Upa"))) + (match_operand:PRED_ALL 3 "register_operand"))) (match_dup 4))] UNSPEC_PTEST)) - (set (match_operand:PRED_ALL 0 "register_operand" "=Upa") + (set (match_operand:PRED_ALL 0 "register_operand") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL (match_dup 2)) (not:PRED_ALL (match_dup 3))) (match_dup 4)))] "TARGET_SVE" - "<logical_nn>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same with just the flags result. (define_insn "*<logical_nn><mode>3_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (and:PRED_ALL (NLOGICAL:PRED_ALL (not:PRED_ALL - (match_operand:PRED_ALL 2 "register_operand" "Upa")) + (match_operand:PRED_ALL 2 "register_operand")) (not:PRED_ALL - (match_operand:PRED_ALL 3 "register_operand" "Upa"))) + (match_operand:PRED_ALL 3 "register_operand"))) (match_dup 4))] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "<logical_nn>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4, 5 ] + [ Upa , Upa, Upa, Upa, , ] <logical_nn>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; ========================================================================= @@ -8133,12 +8163,12 @@ (define_insn_and_rewrite "*cmp<cmp_op><mode>_ptest" (match_operand:SVE_I 3 "aarch64_sve_cmp_<sve_imm_con>_operand"))] UNSPEC_PRED_Z)] UNSPEC_PTEST)) - (clobber (match_scratch:<VPRED> 0 "=Upa, Upa"))] + (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - {@ [ cons: 1 , 2 , 3 ] - [ Upl , w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 - [ Upl , w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> + {@ [ cons: =0, 1 , 2 , 3 ] + [ Upa , Upl, w , <sve_imm_con> ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, #%3 + [ Upa , Upl, w , w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.<Vetype> } "&& !rtx_equal_p (operands[4], operands[6])" { @@ -8180,18 +8210,20 @@ (define_insn_and_split "*cmp<cmp_op><mode>_and" ;; Predicated integer wide comparisons. (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" - [(set (match_operand:<VPRED> 0 "register_operand" "=Upa") + [(set (match_operand:<VPRED> 0 "register_operand") (unspec:<VPRED> - [(match_operand:VNx16BI 1 "register_operand" "Upl") + [(match_operand:VNx16BI 1 "register_operand") (match_operand:SI 2 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:SVE_FULL_BHSI 3 "register_operand" "w") - (match_operand:VNx2DI 4 "register_operand" "w")] + [(match_operand:SVE_FULL_BHSI 3 "register_operand") + (match_operand:VNx2DI 4 "register_operand")] SVE_COND_INT_CMP_WIDE)] UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE" - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d" + {@ [ cons: =0, 1 , 2, 3, 4 ] + [ Upa , Upl, , w, w ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.d + } ) ;; Predicated integer wide comparisons in which both the flag and @@ -8199,19 +8231,19 @@ (define_insn "@aarch64_pred_cmp<cmp_op><mode>_wide" (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upl") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:VNx16BI 6 "register_operand" "Upl") + [(match_operand:VNx16BI 6 "register_operand") (match_operand:SI 7 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:SVE_FULL_BHSI 2 "register_operand" "w") - (match_operand:VNx2DI 3 "register_operand" "w")] + [(match_operand:SVE_FULL_BHSI 2 "register_operand") + (match_operand:VNx2DI 3 "register_operand")] SVE_COND_INT_CMP_WIDE)] UNSPEC_PRED_Z)] UNSPEC_PTEST)) - (set (match_operand:<VPRED> 0 "register_operand" "=Upa") + (set (match_operand:<VPRED> 0 "register_operand") (unspec:<VPRED> [(match_dup 6) (match_dup 7) @@ -8222,7 +8254,9 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" UNSPEC_PRED_Z))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d" + {@ [ cons: =0, 1 , 2, 3, 4, 5, 6 , 7 ] + [ Upa , Upl, w, w, , , Upl, ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + } ) ;; Predicated integer wide comparisons in which only the flags result @@ -8230,22 +8264,24 @@ (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_cc" (define_insn "*aarch64_pred_cmp<cmp_op><mode>_wide_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upl") + [(match_operand:VNx16BI 1 "register_operand") (match_operand 4) (match_operand:SI 5 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:VNx16BI 6 "register_operand" "Upl") + [(match_operand:VNx16BI 6 "register_operand") (match_operand:SI 7 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:SVE_FULL_BHSI 2 "register_operand" "w") - (match_operand:VNx2DI 3 "register_operand" "w")] + [(match_operand:SVE_FULL_BHSI 2 "register_operand") + (match_operand:VNx2DI 3 "register_operand")] SVE_COND_INT_CMP_WIDE)] UNSPEC_PRED_Z)] UNSPEC_PTEST)) - (clobber (match_scratch:<VPRED> 0 "=Upa"))] + (clobber (match_scratch:<VPRED> 0))] "TARGET_SVE && aarch64_sve_same_pred_for_ptest_p (&operands[4], &operands[6])" - "cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d" + {@ [ cons: =0, 1 , 2, 3, 4, 5, 6 , 7 ] + [ Upa , Upl, w, w, , , Upl, ] cmp<cmp_op>\t%0.<Vetype>, %1/z, %2.<Vetype>, %3.d + } ) ;; ------------------------------------------------------------------------- @@ -9922,41 +9958,45 @@ (define_insn "@aarch64_brk<brk_op>" (define_insn "*aarch64_brk<brk_op>_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 4 "aarch64_sve_ptrue_flag") (unspec:VNx16BI [(match_dup 1) - (match_operand:VNx16BI 2 "register_operand" "Upa") + (match_operand:VNx16BI 2 "register_operand") (match_operand:VNx16BI 3 "aarch64_simd_imm_zero")] SVE_BRK_UNARY)] UNSPEC_PTEST)) - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") + (set (match_operand:VNx16BI 0 "register_operand") (unspec:VNx16BI [(match_dup 1) (match_dup 2) (match_dup 3)] SVE_BRK_UNARY))] "TARGET_SVE" - "brk<brk_op>s\t%0.b, %1/z, %2.b" + {@ [ cons: =0, 1 , 2 , 3, 4 ] + [ Upa , Upa, Upa, , ] brk<brk_op>s\t%0.b, %1/z, %2.b + } ) ;; Same, but with only the flags result being interesting. (define_insn "*aarch64_brk<brk_op>_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 4 "aarch64_sve_ptrue_flag") (unspec:VNx16BI [(match_dup 1) - (match_operand:VNx16BI 2 "register_operand" "Upa") + (match_operand:VNx16BI 2 "register_operand") (match_operand:VNx16BI 3 "aarch64_simd_imm_zero")] SVE_BRK_UNARY)] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "brk<brk_op>s\t%0.b, %1/z, %2.b" + {@ [ cons: =0, 1 , 2 , 3, 4 ] + [ Upa , Upa, Upa, , ] brk<brk_op>s\t%0.b, %1/z, %2.b + } ) ;; ------------------------------------------------------------------------- @@ -9973,14 +10013,16 @@ (define_insn "*aarch64_brk<brk_op>_ptest" ;; Binary BRKs (BRKN, BRKPA, BRKPB). (define_insn "@aarch64_brk<brk_op>" - [(set (match_operand:VNx16BI 0 "register_operand" "=Upa") + [(set (match_operand:VNx16BI 0 "register_operand") (unspec:VNx16BI - [(match_operand:VNx16BI 1 "register_operand" "Upa") - (match_operand:VNx16BI 2 "register_operand" "Upa") - (match_operand:VNx16BI 3 "register_operand" "<brk_reg_con>")] + [(match_operand:VNx16BI 1 "register_operand") + (match_operand:VNx16BI 2 "register_operand") + (match_operand:VNx16BI 3 "register_operand")] SVE_BRK_BINARY))] "TARGET_SVE" - "brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b" + {@ [ cons: =0, 1 , 2 , 3 ] + [ Upa , Upa, Upa, <brk_reg_con> ] brk<brk_op>\t%0.b, %1/z, %2.b, %<brk_reg_opno>.b + } ) ;; BRKN, producing both a predicate and a flags result. Unlike other @@ -9992,19 +10034,21 @@ (define_insn_and_rewrite "*aarch64_brkn_cc" (match_operand:VNx16BI 5) (const_int SVE_KNOWN_PTRUE) (unspec:VNx16BI - [(match_operand:VNx16BI 1 "register_operand" "Upa") - (match_operand:VNx16BI 2 "register_operand" "Upa") - (match_operand:VNx16BI 3 "register_operand" "0")] + [(match_operand:VNx16BI 1 "register_operand") + (match_operand:VNx16BI 2 "register_operand") + (match_operand:VNx16BI 3 "register_operand")] UNSPEC_BRKN)] UNSPEC_PTEST)) - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") + (set (match_operand:VNx16BI 0 "register_operand") (unspec:VNx16BI [(match_dup 1) (match_dup 2) (match_dup 3)] UNSPEC_BRKN))] "TARGET_SVE" - "brkns\t%0.b, %1/z, %2.b, %0.b" + {@ [ cons: =0, 1 , 2 , 3, 4, 5 ] + [ Upa , Upa, Upa, 0, , ] brkns\t%0.b, %1/z, %2.b, %0.b + } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" { @@ -10021,14 +10065,16 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" (match_operand:VNx16BI 5) (const_int SVE_KNOWN_PTRUE) (unspec:VNx16BI - [(match_operand:VNx16BI 1 "register_operand" "Upa") - (match_operand:VNx16BI 2 "register_operand" "Upa") - (match_operand:VNx16BI 3 "register_operand" "0")] + [(match_operand:VNx16BI 1 "register_operand") + (match_operand:VNx16BI 2 "register_operand") + (match_operand:VNx16BI 3 "register_operand")] UNSPEC_BRKN)] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "brkns\t%0.b, %1/z, %2.b, %0.b" + {@ [ cons: =0, 1 , 2 , 3, 4, 5 ] + [ Upa , Upa, Upa, 0, , ] brkns\t%0.b, %1/z, %2.b, %0.b + } "&& (operands[4] != CONST0_RTX (VNx16BImode) || operands[5] != CONST0_RTX (VNx16BImode))" { @@ -10041,41 +10087,45 @@ (define_insn_and_rewrite "*aarch64_brkn_ptest" (define_insn "*aarch64_brk<brk_op>_cc" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 4 "aarch64_sve_ptrue_flag") (unspec:VNx16BI [(match_dup 1) - (match_operand:VNx16BI 2 "register_operand" "Upa") - (match_operand:VNx16BI 3 "register_operand" "Upa")] + (match_operand:VNx16BI 2 "register_operand") + (match_operand:VNx16BI 3 "register_operand")] SVE_BRKP)] UNSPEC_PTEST)) - (set (match_operand:VNx16BI 0 "register_operand" "=Upa") + (set (match_operand:VNx16BI 0 "register_operand") (unspec:VNx16BI [(match_dup 1) (match_dup 2) (match_dup 3)] SVE_BRKP))] "TARGET_SVE" - "brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4 ] + [ Upa , Upa, Upa, Upa, ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; Same, but with only the flags result being interesting. (define_insn "*aarch64_brk<brk_op>_ptest" [(set (reg:CC_NZC CC_REGNUM) (unspec:CC_NZC - [(match_operand:VNx16BI 1 "register_operand" "Upa") + [(match_operand:VNx16BI 1 "register_operand") (match_dup 1) (match_operand:SI 4 "aarch64_sve_ptrue_flag") (unspec:VNx16BI [(match_dup 1) - (match_operand:VNx16BI 2 "register_operand" "Upa") - (match_operand:VNx16BI 3 "register_operand" "Upa")] + (match_operand:VNx16BI 2 "register_operand") + (match_operand:VNx16BI 3 "register_operand")] SVE_BRKP)] UNSPEC_PTEST)) - (clobber (match_scratch:VNx16BI 0 "=Upa"))] + (clobber (match_scratch:VNx16BI 0))] "TARGET_SVE" - "brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b" + {@ [ cons: =0, 1 , 2 , 3 , 4 ] + [ Upa , Upa, Upa, Upa, ] brk<brk_op>s\t%0.b, %1/z, %2.b, %3.b + } ) ;; ------------------------------------------------------------------------- diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md index 934e57055d3419e5dcc89b473fd110a0d4978b4f..aa12baf48355358ca4fefe88157df3aac6eb09bd 100644 --- a/gcc/config/aarch64/aarch64-sve2.md +++ b/gcc/config/aarch64/aarch64-sve2.md @@ -3338,18 +3338,20 @@ (define_insn "@aarch64_sve2_histseg<mode>" ;; Predicated string matching. (define_insn "@aarch64_pred_<sve_int_op><mode>" - [(set (match_operand:<VPRED> 0 "register_operand" "=Upa") + [(set (match_operand:<VPRED> 0 "register_operand") (unspec:<VPRED> - [(match_operand:<VPRED> 1 "register_operand" "Upl") + [(match_operand:<VPRED> 1 "register_operand") (match_operand:SI 2 "aarch64_sve_ptrue_flag") (unspec:<VPRED> - [(match_operand:SVE_FULL_BHI 3 "register_operand" "w") - (match_operand:SVE_FULL_BHI 4 "register_operand" "w")] + [(match_operand:SVE_FULL_BHI 3 "register_operand") + (match_operand:SVE_FULL_BHI 4 "register_operand")] SVE2_MATCH)] UNSPEC_PRED_Z)) (clobber (reg:CC_NZC CC_REGNUM))] "TARGET_SVE2 && TARGET_NON_STREAMING" - "<sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype>" + {@ [ cons: =0, 1 , 2, 3, 4 ] + [ Upa , Upl, , w, w ] <sve_int_op>\t%0.<Vetype>, %1/z, %3.<Vetype>, %4.<Vetype> + } ) ;; Predicated string matching in which both the flag and predicate results