| Message ID | 16675a67-3dd2-fc62-fd38-6eaa24da66f7@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [nvptx,1/2] Reimplement libgomp barriers for nvptx | expand |
On 9/21/22 09:45, Chung-Lin Tang wrote: > Hi Tom, following the first patch. > > This new barrier implementation I posted in the first patch uses the > 'bar.red' instruction. > Usually this could've been easily done with a single line of inline > assembly. However I quickly > realized that because the NVPTX GCC port is implemented with all virtual > general registers, > we don't have a register constraint usable to select "predicate registers". > Since bar.red uses predicate typed values, I can't create it directly > using inline asm. > > So it appears that the most simple way of accessing it is with a target > builtin. > The attached patch adds bar.red instructions to the nvptx port, and > __builtin_nvptx_bar_red_* builtins > to use it. The code should support all variations of bar.red (and, or, > and popc operations). > > (This support was used to implement the first libgomp barrier patch, so > must be approved together) > What I conclude from what you're telling me here is that this is the first patch in the series rather than the second. So, LGTM, please apply it, unless it cannot be applied by itself without causing regressions, in which case you need to fix those first. IWBN if this also included standalone test-cases in gcc/testsuite/gcc.target/nvptx, but I suppose we can live without for now. Thanks, - Tom > Thanks, > Chung-Lin > > 2022-09-21 Chung-Lin Tang <cltang@codesourcery.com> > > gcc/ChangeLog: > > * config/nvptx/nvptx.cc (nvptx_print_operand): Add 'p' > case, adjust comments. > (enum nvptx_builtins): Add NVPTX_BUILTIN_BAR_RED_AND, > NVPTX_BUILTIN_BAR_RED_OR, and NVPTX_BUILTIN_BAR_RED_POPC. > (nvptx_expand_bar_red): New function. > (nvptx_init_builtins): > Add DEFs of __builtin_nvptx_bar_red_[and/or/popc]. > (nvptx_expand_builtin): Use nvptx_expand_bar_red to expand > NVPTX_BUILTIN_BAR_RED_[AND/OR/POPC] cases. > > * config/nvptx/nvptx.md (define_c_enum "unspecv"): Add > UNSPECV_BARRED_AND, UNSPECV_BARRED_OR, and UNSPECV_BARRED_POPC. > (BARRED): New int iterator. > (barred_op,barred_mode,barred_ptxtype): New int attrs. > (nvptx_barred_<barred_op>): New define_insn.
Hi Chung-Lin! On 2022-09-21T15:45:54+0800, Chung-Lin Tang via Gcc-patches <gcc-patches@gcc.gnu.org> wrote: > [...] The attached patch adds bar.red instructions to the nvptx port [...] I see GCC report: [...] build/genrecog [...]/source-gcc/gcc/common.md [...]/source-gcc/gcc/config/nvptx/nvptx.md \ insn-conditions.md > tmp-recog.cc [...]/source-gcc/gcc/config/nvptx/nvptx.md:2297:1: warning: source missing a mode? [...]/source-gcc/gcc/config/nvptx/nvptx.md:2297:1: warning: source missing a mode? [...]/source-gcc/gcc/config/nvptx/nvptx.md:2297:1: warning: source missing a mode? Statistics for recog: [...] 2297 (define_insn "nvptx_barred_<barred_op>" 2298 [(set (match_operand:<barred_mode> 0 "nvptx_register_operand" "=R") 2299 (unspec_volatile 2300 [(match_operand:SI 1 "nvptx_nonmemory_operand" "Ri") 2301 (match_operand:SI 2 "nvptx_nonmemory_operand" "Ri") 2302 (match_operand:SI 3 "const_int_operand" "i") 2303 (match_operand:BI 4 "nvptx_register_operand" "R")] 2304 BARRED))] 2305 "" 2306 "\\tbar.red.<barred_op>.<barred_ptxtype> \\t%0, %1, %2, %p3%4;";" 2307 [(set_attr "predicable" "no")]) Grüße Thomas > gcc/ChangeLog: > > * config/nvptx/nvptx.cc (nvptx_print_operand): Add 'p' > case, adjust comments. > (enum nvptx_builtins): Add NVPTX_BUILTIN_BAR_RED_AND, > NVPTX_BUILTIN_BAR_RED_OR, and NVPTX_BUILTIN_BAR_RED_POPC. > (nvptx_expand_bar_red): New function. > (nvptx_init_builtins): > Add DEFs of __builtin_nvptx_bar_red_[and/or/popc]. > (nvptx_expand_builtin): Use nvptx_expand_bar_red to expand > NVPTX_BUILTIN_BAR_RED_[AND/OR/POPC] cases. > > * config/nvptx/nvptx.md (define_c_enum "unspecv"): Add > UNSPECV_BARRED_AND, UNSPECV_BARRED_OR, and UNSPECV_BARRED_POPC. > (BARRED): New int iterator. > (barred_op,barred_mode,barred_ptxtype): New int attrs. > (nvptx_barred_<barred_op>): New define_insn. > diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc > index 49cc681..afc3a890 100644 > --- a/gcc/config/nvptx/nvptx.cc > +++ b/gcc/config/nvptx/nvptx.cc > @@ -2879,6 +2879,7 @@ nvptx_mem_maybe_shared_p (const_rtx x) > t -- print a type opcode suffix, promoting QImode to 32 bits > T -- print a type size in bits > u -- print a type opcode suffix without promotions. > + p -- print a '!' for constant 0. > x -- print a destination operand that may also be a bit bucket. */ > > static void > @@ -3012,6 +3013,11 @@ nvptx_print_operand (FILE *file, rtx x, int code) > fprintf (file, "@!"); > goto common; > > + case 'p': > + if (INTVAL (x) == 0) > + fprintf (file, "!"); > + break; > + > case 'c': > mode = GET_MODE (XEXP (x, 0)); > switch (x_code) > @@ -6151,9 +6157,90 @@ enum nvptx_builtins > NVPTX_BUILTIN_CMP_SWAPLL, > NVPTX_BUILTIN_MEMBAR_GL, > NVPTX_BUILTIN_MEMBAR_CTA, > + NVPTX_BUILTIN_BAR_RED_AND, > + NVPTX_BUILTIN_BAR_RED_OR, > + NVPTX_BUILTIN_BAR_RED_POPC, > NVPTX_BUILTIN_MAX > }; > > +/* Expander for 'bar.red' instruction builtins. */ > + > +static rtx > +nvptx_expand_bar_red (tree exp, rtx target, > + machine_mode ARG_UNUSED (m), int ARG_UNUSED (ignore)) > +{ > + int code = DECL_MD_FUNCTION_CODE (TREE_OPERAND (CALL_EXPR_FN (exp), 0)); > + machine_mode mode = TYPE_MODE (TREE_TYPE (exp)); > + > + if (!target) > + target = gen_reg_rtx (mode); > + > + rtx pred, dst; > + rtx bar = expand_expr (CALL_EXPR_ARG (exp, 0), > + NULL_RTX, SImode, EXPAND_NORMAL); > + rtx nthr = expand_expr (CALL_EXPR_ARG (exp, 1), > + NULL_RTX, SImode, EXPAND_NORMAL); > + rtx cpl = expand_expr (CALL_EXPR_ARG (exp, 2), > + NULL_RTX, SImode, EXPAND_NORMAL); > + rtx redop = expand_expr (CALL_EXPR_ARG (exp, 3), > + NULL_RTX, SImode, EXPAND_NORMAL); > + if (CONST_INT_P (bar)) > + { > + if (INTVAL (bar) < 0 || INTVAL (bar) > 15) > + { > + error_at (EXPR_LOCATION (exp), > + "barrier value must be within [0,15]"); > + return const0_rtx; > + } > + } > + else if (!REG_P (bar)) > + bar = copy_to_mode_reg (SImode, bar); > + > + if (!CONST_INT_P (nthr) && !REG_P (nthr)) > + nthr = copy_to_mode_reg (SImode, nthr); > + > + if (!CONST_INT_P (cpl)) > + { > + error_at (EXPR_LOCATION (exp), > + "complement argument must be constant"); > + return const0_rtx; > + } > + > + pred = gen_reg_rtx (BImode); > + if (!REG_P (redop)) > + redop = copy_to_mode_reg (SImode, redop); > + emit_insn (gen_rtx_SET (pred, gen_rtx_NE (BImode, redop, GEN_INT (0)))); > + redop = pred; > + > + rtx pat; > + switch (code) > + { > + case NVPTX_BUILTIN_BAR_RED_AND: > + dst = gen_reg_rtx (BImode); > + pat = gen_nvptx_barred_and (dst, bar, nthr, cpl, redop); > + break; > + case NVPTX_BUILTIN_BAR_RED_OR: > + dst = gen_reg_rtx (BImode); > + pat = gen_nvptx_barred_or (dst, bar, nthr, cpl, redop); > + break; > + case NVPTX_BUILTIN_BAR_RED_POPC: > + dst = gen_reg_rtx (SImode); > + pat = gen_nvptx_barred_popc (dst, bar, nthr, cpl, redop); > + break; > + default: > + gcc_unreachable (); > + } > + emit_insn (pat); > + if (GET_MODE (dst) == BImode) > + { > + rtx tmp = gen_reg_rtx (mode); > + emit_insn (gen_rtx_SET (tmp, gen_rtx_NE (mode, dst, GEN_INT (0)))); > + dst = tmp; > + } > + emit_move_insn (target, dst); > + return target; > +} > + > static GTY(()) tree nvptx_builtin_decls[NVPTX_BUILTIN_MAX]; > > /* Return the NVPTX builtin for CODE. */ > @@ -6194,6 +6281,13 @@ nvptx_init_builtins (void) > DEF (MEMBAR_GL, "membar_gl", (VOID, VOID, NULL_TREE)); > DEF (MEMBAR_CTA, "membar_cta", (VOID, VOID, NULL_TREE)); > > + DEF (BAR_RED_AND, "bar_red_and", > + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); > + DEF (BAR_RED_OR, "bar_red_or", > + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); > + DEF (BAR_RED_POPC, "bar_red_popc", > + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); > + > #undef DEF > #undef ST > #undef UINT > @@ -6236,6 +6330,11 @@ nvptx_expand_builtin (tree exp, rtx target, rtx ARG_UNUSED (subtarget), > emit_insn (gen_nvptx_membar_cta ()); > return NULL_RTX; > > + case NVPTX_BUILTIN_BAR_RED_AND: > + case NVPTX_BUILTIN_BAR_RED_OR: > + case NVPTX_BUILTIN_BAR_RED_POPC: > + return nvptx_expand_bar_red (exp, target, mode, ignore); > + > default: gcc_unreachable (); > } > } > diff --git a/gcc/config/nvptx/nvptx.md b/gcc/config/nvptx/nvptx.md > index 8ed6850..740c4de 100644 > --- a/gcc/config/nvptx/nvptx.md > +++ b/gcc/config/nvptx/nvptx.md > @@ -58,6 +58,9 @@ > UNSPECV_CAS_LOCAL > UNSPECV_XCHG > UNSPECV_ST > + UNSPECV_BARRED_AND > + UNSPECV_BARRED_OR > + UNSPECV_BARRED_POPC > UNSPECV_BARSYNC > UNSPECV_WARPSYNC > UNSPECV_UNIFORM_WARP_CHECK > @@ -2274,6 +2277,35 @@ > "TARGET_PTX_6_0" > "%.\\tbar.warp.sync\\t0xffffffff;") > > +(define_int_iterator BARRED > + [UNSPECV_BARRED_AND > + UNSPECV_BARRED_OR > + UNSPECV_BARRED_POPC]) > +(define_int_attr barred_op > + [(UNSPECV_BARRED_AND "and") > + (UNSPECV_BARRED_OR "or") > + (UNSPECV_BARRED_POPC "popc")]) > +(define_int_attr barred_mode > + [(UNSPECV_BARRED_AND "BI") > + (UNSPECV_BARRED_OR "BI") > + (UNSPECV_BARRED_POPC "SI")]) > +(define_int_attr barred_ptxtype > + [(UNSPECV_BARRED_AND "pred") > + (UNSPECV_BARRED_OR "pred") > + (UNSPECV_BARRED_POPC "u32")]) > + > +(define_insn "nvptx_barred_<barred_op>" > + [(set (match_operand:<barred_mode> 0 "nvptx_register_operand" "=R") > + (unspec_volatile > + [(match_operand:SI 1 "nvptx_nonmemory_operand" "Ri") > + (match_operand:SI 2 "nvptx_nonmemory_operand" "Ri") > + (match_operand:SI 3 "const_int_operand" "i") > + (match_operand:BI 4 "nvptx_register_operand" "R")] > + BARRED))] > + "" > + "\\tbar.red.<barred_op>.<barred_ptxtype> \\t%0, %1, %2, %p3%4;";" > + [(set_attr "predicable" "no")]) > + > (define_insn "nvptx_uniform_warp_check" > [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)] > "" ----------------- Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
diff --git a/gcc/config/nvptx/nvptx.cc b/gcc/config/nvptx/nvptx.cc index 49cc681..afc3a890 100644 --- a/gcc/config/nvptx/nvptx.cc +++ b/gcc/config/nvptx/nvptx.cc @@ -2879,6 +2879,7 @@ nvptx_mem_maybe_shared_p (const_rtx x) t -- print a type opcode suffix, promoting QImode to 32 bits T -- print a type size in bits u -- print a type opcode suffix without promotions. + p -- print a '!' for constant 0. x -- print a destination operand that may also be a bit bucket. */ static void @@ -3012,6 +3013,11 @@ nvptx_print_operand (FILE *file, rtx x, int code) fprintf (file, "@!"); goto common; + case 'p': + if (INTVAL (x) == 0) + fprintf (file, "!"); + break; + case 'c': mode = GET_MODE (XEXP (x, 0)); switch (x_code) @@ -6151,9 +6157,90 @@ enum nvptx_builtins NVPTX_BUILTIN_CMP_SWAPLL, NVPTX_BUILTIN_MEMBAR_GL, NVPTX_BUILTIN_MEMBAR_CTA, + NVPTX_BUILTIN_BAR_RED_AND, + NVPTX_BUILTIN_BAR_RED_OR, + NVPTX_BUILTIN_BAR_RED_POPC, NVPTX_BUILTIN_MAX }; +/* Expander for 'bar.red' instruction builtins. */ + +static rtx +nvptx_expand_bar_red (tree exp, rtx target, + machine_mode ARG_UNUSED (m), int ARG_UNUSED (ignore)) +{ + int code = DECL_MD_FUNCTION_CODE (TREE_OPERAND (CALL_EXPR_FN (exp), 0)); + machine_mode mode = TYPE_MODE (TREE_TYPE (exp)); + + if (!target) + target = gen_reg_rtx (mode); + + rtx pred, dst; + rtx bar = expand_expr (CALL_EXPR_ARG (exp, 0), + NULL_RTX, SImode, EXPAND_NORMAL); + rtx nthr = expand_expr (CALL_EXPR_ARG (exp, 1), + NULL_RTX, SImode, EXPAND_NORMAL); + rtx cpl = expand_expr (CALL_EXPR_ARG (exp, 2), + NULL_RTX, SImode, EXPAND_NORMAL); + rtx redop = expand_expr (CALL_EXPR_ARG (exp, 3), + NULL_RTX, SImode, EXPAND_NORMAL); + if (CONST_INT_P (bar)) + { + if (INTVAL (bar) < 0 || INTVAL (bar) > 15) + { + error_at (EXPR_LOCATION (exp), + "barrier value must be within [0,15]"); + return const0_rtx; + } + } + else if (!REG_P (bar)) + bar = copy_to_mode_reg (SImode, bar); + + if (!CONST_INT_P (nthr) && !REG_P (nthr)) + nthr = copy_to_mode_reg (SImode, nthr); + + if (!CONST_INT_P (cpl)) + { + error_at (EXPR_LOCATION (exp), + "complement argument must be constant"); + return const0_rtx; + } + + pred = gen_reg_rtx (BImode); + if (!REG_P (redop)) + redop = copy_to_mode_reg (SImode, redop); + emit_insn (gen_rtx_SET (pred, gen_rtx_NE (BImode, redop, GEN_INT (0)))); + redop = pred; + + rtx pat; + switch (code) + { + case NVPTX_BUILTIN_BAR_RED_AND: + dst = gen_reg_rtx (BImode); + pat = gen_nvptx_barred_and (dst, bar, nthr, cpl, redop); + break; + case NVPTX_BUILTIN_BAR_RED_OR: + dst = gen_reg_rtx (BImode); + pat = gen_nvptx_barred_or (dst, bar, nthr, cpl, redop); + break; + case NVPTX_BUILTIN_BAR_RED_POPC: + dst = gen_reg_rtx (SImode); + pat = gen_nvptx_barred_popc (dst, bar, nthr, cpl, redop); + break; + default: + gcc_unreachable (); + } + emit_insn (pat); + if (GET_MODE (dst) == BImode) + { + rtx tmp = gen_reg_rtx (mode); + emit_insn (gen_rtx_SET (tmp, gen_rtx_NE (mode, dst, GEN_INT (0)))); + dst = tmp; + } + emit_move_insn (target, dst); + return target; +} + static GTY(()) tree nvptx_builtin_decls[NVPTX_BUILTIN_MAX]; /* Return the NVPTX builtin for CODE. */ @@ -6194,6 +6281,13 @@ nvptx_init_builtins (void) DEF (MEMBAR_GL, "membar_gl", (VOID, VOID, NULL_TREE)); DEF (MEMBAR_CTA, "membar_cta", (VOID, VOID, NULL_TREE)); + DEF (BAR_RED_AND, "bar_red_and", + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); + DEF (BAR_RED_OR, "bar_red_or", + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); + DEF (BAR_RED_POPC, "bar_red_popc", + (UINT, UINT, UINT, UINT, UINT, NULL_TREE)); + #undef DEF #undef ST #undef UINT @@ -6236,6 +6330,11 @@ nvptx_expand_builtin (tree exp, rtx target, rtx ARG_UNUSED (subtarget), emit_insn (gen_nvptx_membar_cta ()); return NULL_RTX; + case NVPTX_BUILTIN_BAR_RED_AND: + case NVPTX_BUILTIN_BAR_RED_OR: + case NVPTX_BUILTIN_BAR_RED_POPC: + return nvptx_expand_bar_red (exp, target, mode, ignore); + default: gcc_unreachable (); } } diff --git a/gcc/config/nvptx/nvptx.md b/gcc/config/nvptx/nvptx.md index 8ed6850..740c4de 100644 --- a/gcc/config/nvptx/nvptx.md +++ b/gcc/config/nvptx/nvptx.md @@ -58,6 +58,9 @@ UNSPECV_CAS_LOCAL UNSPECV_XCHG UNSPECV_ST + UNSPECV_BARRED_AND + UNSPECV_BARRED_OR + UNSPECV_BARRED_POPC UNSPECV_BARSYNC UNSPECV_WARPSYNC UNSPECV_UNIFORM_WARP_CHECK @@ -2274,6 +2277,35 @@ "TARGET_PTX_6_0" "%.\\tbar.warp.sync\\t0xffffffff;") +(define_int_iterator BARRED + [UNSPECV_BARRED_AND + UNSPECV_BARRED_OR + UNSPECV_BARRED_POPC]) +(define_int_attr barred_op + [(UNSPECV_BARRED_AND "and") + (UNSPECV_BARRED_OR "or") + (UNSPECV_BARRED_POPC "popc")]) +(define_int_attr barred_mode + [(UNSPECV_BARRED_AND "BI") + (UNSPECV_BARRED_OR "BI") + (UNSPECV_BARRED_POPC "SI")]) +(define_int_attr barred_ptxtype + [(UNSPECV_BARRED_AND "pred") + (UNSPECV_BARRED_OR "pred") + (UNSPECV_BARRED_POPC "u32")]) + +(define_insn "nvptx_barred_<barred_op>" + [(set (match_operand:<barred_mode> 0 "nvptx_register_operand" "=R") + (unspec_volatile + [(match_operand:SI 1 "nvptx_nonmemory_operand" "Ri") + (match_operand:SI 2 "nvptx_nonmemory_operand" "Ri") + (match_operand:SI 3 "const_int_operand" "i") + (match_operand:BI 4 "nvptx_register_operand" "R")] + BARRED))] + "" + "\\tbar.red.<barred_op>.<barred_ptxtype> \\t%0, %1, %2, %p3%4;";" + [(set_attr "predicable" "no")]) + (define_insn "nvptx_uniform_warp_check" [(unspec_volatile [(const_int 0)] UNSPECV_UNIFORM_WARP_CHECK)] ""