From patchwork Wed Jun 15 16:41:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1643908 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=oq4V8tu0; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LNWKY14Zmz9sGF for ; Thu, 16 Jun 2022 02:41:33 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 309733857033 for ; Wed, 15 Jun 2022 16:41:27 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 309733857033 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1655311287; bh=g+b9p8bOFC/ZuuQxiF7VDNRENPymWwwFPdH7Y7Jt4I4=; h=To:Subject:Date:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=oq4V8tu0wmVaFQVWbS6xKelMf13hzbVsZrSE7vNfd0u00TMGoWP2ydmLqvLVA/wpi c+Syd+Uds4fQW6BTY0fkCKGsThVkrFWnmCC6xwUmu8R5pXChqwBBJiQyiGOJ3GJVpN oI9SmbCu43p5xWuh3Q93G1vkcVVjUR0LBsy2pegQ= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 6BBF13858D32 for ; Wed, 15 Jun 2022 16:41:04 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 6BBF13858D32 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D33E153B for ; Wed, 15 Jun 2022 09:41:04 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.37]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 91E493F7F5 for ; Wed, 15 Jun 2022 09:41:03 -0700 (PDT) To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [pushed] Revert recent internal-fn changes [PR105975] Date: Wed, 15 Jun 2022 17:41:02 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 X-Spam-Status: No, score=-57.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Richard Sandiford via Gcc-patches From: Richard Sandiford Reply-To: Richard Sandiford Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" The recent internal-fn “clean-ups” triggered problems on nvptx because some of the omp_simt_* patterns had modeless operands. I wondered about adapting expand_fn_using_insn to cope with that, but then the problem becomes: what should the mode of operand 0 be when there is no lhs? The answer depends on the target insn. For GOMP_SIMT_ENTER_ALLOC the answer was: use Pmode. For GOMP_SIMT_ORDERED_PRED and others the answer was: elide the call. (However, GOMP_SIMT_ORDERED_PRED doesn't seem to have ECF_* flags that would normally allow it to be dropped at the gimple level.) So these instructions seem to be special enough that they need their own code after all. This patch reverts the second patch and most of the first. The only part retained from the first is splitting expand_fn_using_insn out of expand_direct_optab_fn, since I think expand_fn_using_insn could still be useful in future. Tested on aarch64-linux-gnu & pushed. Richard gcc/ PR middle-end/105975 Revert everything apart from the expand_fn_using_insn and expand_direct_optab_fn changes from: * internal-fn.def (DEF_INTERNAL_INSN_FN): New macro. (GOMP_SIMT_ENTER_ALLOC, GOMP_SIMT_EXIT, GOMP_SIMT_LANE) (GOMP_SIMT_LAST_LANE, GOMP_SIMT_ORDERED_PRED, GOMP_SIMT_VOTE_ANY) (GOMP_SIMT_XCHG_BFLY, GOMP_SIMT_XCHG_IDX): Use it. * internal-fn.h (direct_internal_fn_info::directly_mapped): New member variable. (direct_internal_fn_info::vectorizable): Reduce to 1 bit. (direct_internal_fn_p): Also return true for internal functions that map directly to instructions defined target-insns.def. (direct_internal_fn): Adjust comment accordingly. * internal-fn.cc (direct_insn, optab1, optab2, vectorizable_optab1) (vectorizable_optab2): New local macros. (not_direct): Initialize directly_mapped. (mask_load_direct, load_lanes_direct, mask_load_lanes_direct) (gather_load_direct, len_load_direct, mask_store_direct) (store_lanes_direct, mask_store_lanes_direct, vec_cond_mask_direct) (vec_cond_direct, scatter_store_direct, len_store_direct) (vec_set_direct, unary_direct, binary_direct, ternary_direct) (cond_unary_direct, cond_binary_direct, cond_ternary_direct) (while_direct, fold_extract_direct, fold_left_direct) (mask_fold_left_direct, check_ptrs_direct): Use the macros above. (expand_GOMP_SIMT_ENTER_ALLOC, expand_GOMP_SIMT_EXIT): Delete (expand_GOMP_SIMT_LANE, expand_GOMP_SIMT_LAST_LANE): Likewise; (expand_GOMP_SIMT_ORDERED_PRED, expand_GOMP_SIMT_VOTE_ANY): Likewise. (expand_GOMP_SIMT_XCHG_BFLY, expand_GOMP_SIMT_XCHG_IDX): Likewise. (direct_internal_fn_types): Handle functions that map to instructions defined in target-insns.def. (direct_internal_fn_types): Likewise. (direct_internal_fn_supported_p): Likewise. (internal_fn_expanders): Likewise. (expand_fn_using_insn): New function, split out and adapted from... (expand_direct_optab_fn): ...here. (expand_GOMP_SIMT_ENTER_ALLOC): Use it. (expand_GOMP_SIMT_EXIT): Likewise. (expand_GOMP_SIMT_LANE): Likewise. (expand_GOMP_SIMT_LAST_LANE): Likewise. (expand_GOMP_SIMT_ORDERED_PRED): Likewise. (expand_GOMP_SIMT_VOTE_ANY): Likewise. (expand_GOMP_SIMT_XCHG_BFLY): Likewise. (expand_GOMP_SIMT_XCHG_IDX): Likewise. --- gcc/internal-fn.cc | 239 +++++++++++++++++++++++++++++++++++--------- gcc/internal-fn.def | 34 ++----- gcc/internal-fn.h | 20 ++-- 3 files changed, 206 insertions(+), 87 deletions(-) diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc index a809953ce6f..91588f8bc9f 100644 --- a/gcc/internal-fn.cc +++ b/gcc/internal-fn.cc @@ -105,44 +105,37 @@ init_internal_fns () /* Create static initializers for the information returned by direct_internal_fn. */ -#define not_direct { -2, -2, false, false } -#define direct_insn { -2, -2, true, false } -#define optab1(TYPE0) { TYPE0, TYPE0, true, false } -#define optab2(TYPE0, TYPE1) { TYPE0, TYPE1, true, false } -#define vectorizable_optab1(TYPE0) { TYPE0, TYPE0, true, true } - -#define mask_load_direct optab2 (-1, 2) -#define load_lanes_direct optab1 (-1) -#define mask_load_lanes_direct optab1 (-1) -#define gather_load_direct optab2 (3, 1) -#define len_load_direct optab1 (-1) -#define mask_store_direct optab2 (3, 2) -#define store_lanes_direct optab1 (0) -#define mask_store_lanes_direct optab1 (0) -#define vec_cond_mask_direct optab2 (1, 0) -#define vec_cond_direct optab2 (2, 0) -#define scatter_store_direct optab2 (3, 1) -#define len_store_direct optab1 (3) -#define vec_set_direct optab1 (3) -#define unary_direct vectorizable_optab1 (0) -#define binary_direct vectorizable_optab1 (0) -#define ternary_direct vectorizable_optab1 (0) -#define cond_unary_direct vectorizable_optab1 (1) -#define cond_binary_direct vectorizable_optab1 (1) -#define cond_ternary_direct vectorizable_optab1 (1) -#define while_direct optab2 (0, 2) -#define fold_extract_direct optab1 (2) -#define fold_left_direct optab1 (1) -#define mask_fold_left_direct optab1 (1) -#define check_ptrs_direct optab1 (0) +#define not_direct { -2, -2, false } +#define mask_load_direct { -1, 2, false } +#define load_lanes_direct { -1, -1, false } +#define mask_load_lanes_direct { -1, -1, false } +#define gather_load_direct { 3, 1, false } +#define len_load_direct { -1, -1, false } +#define mask_store_direct { 3, 2, false } +#define store_lanes_direct { 0, 0, false } +#define mask_store_lanes_direct { 0, 0, false } +#define vec_cond_mask_direct { 1, 0, false } +#define vec_cond_direct { 2, 0, false } +#define scatter_store_direct { 3, 1, false } +#define len_store_direct { 3, 3, false } +#define vec_set_direct { 3, 3, false } +#define unary_direct { 0, 0, true } +#define binary_direct { 0, 0, true } +#define ternary_direct { 0, 0, true } +#define cond_unary_direct { 1, 1, true } +#define cond_binary_direct { 1, 1, true } +#define cond_ternary_direct { 1, 1, true } +#define while_direct { 0, 2, false } +#define fold_extract_direct { 2, 2, false } +#define fold_left_direct { 1, 1, false } +#define mask_fold_left_direct { 1, 1, false } +#define check_ptrs_direct { 0, 0, false } const direct_internal_fn_info direct_internal_fn_array[IFN_LAST + 1] = { #define DEF_INTERNAL_FN(CODE, FLAGS, FNSPEC) not_direct, #define DEF_INTERNAL_OPTAB_FN(CODE, FLAGS, OPTAB, TYPE) TYPE##_direct, #define DEF_INTERNAL_SIGNED_OPTAB_FN(CODE, FLAGS, SELECTOR, SIGNED_OPTAB, \ UNSIGNED_OPTAB, TYPE) TYPE##_direct, -#define DEF_INTERNAL_INSN_FN(CODE, FLAGS, INSN, NOUTPUTS, NINPUTS) \ - direct_insn, #include "internal-fn.def" not_direct }; @@ -315,6 +308,57 @@ expand_GOMP_SIMT_ENTER (internal_fn, gcall *) gcc_unreachable (); } +/* Allocate per-lane storage and begin non-uniform execution region. */ + +static void +expand_GOMP_SIMT_ENTER_ALLOC (internal_fn, gcall *stmt) +{ + rtx target; + tree lhs = gimple_call_lhs (stmt); + if (lhs) + target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + else + target = gen_reg_rtx (Pmode); + rtx size = expand_normal (gimple_call_arg (stmt, 0)); + rtx align = expand_normal (gimple_call_arg (stmt, 1)); + class expand_operand ops[3]; + create_output_operand (&ops[0], target, Pmode); + create_input_operand (&ops[1], size, Pmode); + create_input_operand (&ops[2], align, Pmode); + gcc_assert (targetm.have_omp_simt_enter ()); + expand_insn (targetm.code_for_omp_simt_enter, 3, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + +/* Deallocate per-lane storage and leave non-uniform execution region. */ + +static void +expand_GOMP_SIMT_EXIT (internal_fn, gcall *stmt) +{ + gcc_checking_assert (!gimple_call_lhs (stmt)); + rtx arg = expand_normal (gimple_call_arg (stmt, 0)); + class expand_operand ops[1]; + create_input_operand (&ops[0], arg, Pmode); + gcc_assert (targetm.have_omp_simt_exit ()); + expand_insn (targetm.code_for_omp_simt_exit, 1, ops); +} + +/* Lane index on SIMT targets: thread index in the warp on NVPTX. On targets + without SIMT execution this should be expanded in omp_device_lower pass. */ + +static void +expand_GOMP_SIMT_LANE (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + gcc_assert (targetm.have_omp_simt_lane ()); + emit_insn (targetm.gen_omp_simt_lane (target)); +} + /* This should get expanded in omp_device_lower pass. */ static void @@ -323,6 +367,119 @@ expand_GOMP_SIMT_VF (internal_fn, gcall *) gcc_unreachable (); } +/* Lane index of the first SIMT lane that supplies a non-zero argument. + This is a SIMT counterpart to GOMP_SIMD_LAST_LANE, used to represent the + lane that executed the last iteration for handling OpenMP lastprivate. */ + +static void +expand_GOMP_SIMT_LAST_LANE (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx cond = expand_normal (gimple_call_arg (stmt, 0)); + machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); + class expand_operand ops[2]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], cond, mode); + gcc_assert (targetm.have_omp_simt_last_lane ()); + expand_insn (targetm.code_for_omp_simt_last_lane, 2, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + +/* Non-transparent predicate used in SIMT lowering of OpenMP "ordered". */ + +static void +expand_GOMP_SIMT_ORDERED_PRED (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx ctr = expand_normal (gimple_call_arg (stmt, 0)); + machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); + class expand_operand ops[2]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], ctr, mode); + gcc_assert (targetm.have_omp_simt_ordered ()); + expand_insn (targetm.code_for_omp_simt_ordered, 2, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + +/* "Or" boolean reduction across SIMT lanes: return non-zero in all lanes if + any lane supplies a non-zero argument. */ + +static void +expand_GOMP_SIMT_VOTE_ANY (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx cond = expand_normal (gimple_call_arg (stmt, 0)); + machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); + class expand_operand ops[2]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], cond, mode); + gcc_assert (targetm.have_omp_simt_vote_any ()); + expand_insn (targetm.code_for_omp_simt_vote_any, 2, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + +/* Exchange between SIMT lanes with a "butterfly" pattern: source lane index + is destination lane index XOR given offset. */ + +static void +expand_GOMP_SIMT_XCHG_BFLY (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx src = expand_normal (gimple_call_arg (stmt, 0)); + rtx idx = expand_normal (gimple_call_arg (stmt, 1)); + machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); + class expand_operand ops[3]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], src, mode); + create_input_operand (&ops[2], idx, SImode); + gcc_assert (targetm.have_omp_simt_xchg_bfly ()); + expand_insn (targetm.code_for_omp_simt_xchg_bfly, 3, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + +/* Exchange between SIMT lanes according to given source lane index. */ + +static void +expand_GOMP_SIMT_XCHG_IDX (internal_fn, gcall *stmt) +{ + tree lhs = gimple_call_lhs (stmt); + if (!lhs) + return; + + rtx target = expand_expr (lhs, NULL_RTX, VOIDmode, EXPAND_WRITE); + rtx src = expand_normal (gimple_call_arg (stmt, 0)); + rtx idx = expand_normal (gimple_call_arg (stmt, 1)); + machine_mode mode = TYPE_MODE (TREE_TYPE (lhs)); + class expand_operand ops[3]; + create_output_operand (&ops[0], target, mode); + create_input_operand (&ops[1], src, mode); + create_input_operand (&ops[2], idx, SImode); + gcc_assert (targetm.have_omp_simt_xchg_idx ()); + expand_insn (targetm.code_for_omp_simt_xchg_idx, 3, ops); + if (!rtx_equal_p (target, ops[0].value)) + emit_move_insn (target, ops[0].value); +} + /* This should get expanded in adjust_simduid_builtins. */ static void @@ -3563,10 +3720,6 @@ tree_pair direct_internal_fn_types (internal_fn fn, tree return_type, tree *args) { const direct_internal_fn_info &info = direct_internal_fn (fn); - if (info.type0 == -2) - /* Functions created by DEF_INTERNAL_INSN_FN are not type-dependent. */ - return tree_pair {}; - tree type0 = (info.type0 < 0 ? return_type : TREE_TYPE (args[info.type0])); tree type1 = (info.type1 < 0 ? return_type : TREE_TYPE (args[info.type1])); return tree_pair (type0, type1); @@ -3580,10 +3733,6 @@ tree_pair direct_internal_fn_types (internal_fn fn, gcall *call) { const direct_internal_fn_info &info = direct_internal_fn (fn); - if (info.type0 == -2) - /* Functions created by DEF_INTERNAL_INSN_FN are not type-dependent. */ - return tree_pair {}; - tree op0 = (info.type0 < 0 ? gimple_call_lhs (call) : gimple_call_arg (call, info.type0)); @@ -3728,8 +3877,6 @@ direct_internal_fn_supported_p (internal_fn fn, tree_pair types, return direct_##TYPE##_optab_supported_p (which_optab, types, \ opt_type); \ } -#define DEF_INTERNAL_INSN_FN(CODE, FLAGS, INSN, NOUTPUTS, NINPUTS) \ - case IFN_##CODE: return targetm.have_##INSN (); #include "internal-fn.def" case IFN_LAST: @@ -3881,14 +4028,6 @@ set_edom_supported_p (void) optab which_optab = direct_internal_fn_optab (fn, types); \ expand_##TYPE##_optab_fn (fn, stmt, which_optab); \ } -#define DEF_INTERNAL_INSN_FN(CODE, FLAGS, INSN, NOUTPUTS, NINPUTS) \ - static void \ - expand_##CODE (internal_fn, gcall *stmt) \ - { \ - gcc_assert (targetm.have_##INSN ()); \ - expand_fn_using_insn (stmt, targetm.code_for_##INSN, \ - NOUTPUTS, NINPUTS); \ - } #include "internal-fn.def" /* Routines to expand each internal function, indexed by function number. diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def index 76fd050b57c..d2d550d3586 100644 --- a/gcc/internal-fn.def +++ b/gcc/internal-fn.def @@ -34,7 +34,6 @@ along with GCC; see the file COPYING3. If not see UNSIGNED_OPTAB, TYPE) DEF_INTERNAL_FLT_FN (NAME, FLAGS, OPTAB, TYPE) DEF_INTERNAL_INT_FN (NAME, FLAGS, OPTAB, TYPE) - DEF_INTERNAL_INSN_FN (NAME, FLAGS, INSN, NOUTPUTS, NINPUTS) where NAME is the name of the function, FLAGS is a set of ECF_* flags and FNSPEC is a string describing functions fnspec. @@ -83,11 +82,6 @@ along with GCC; see the file COPYING3. If not see says that the function extends the C-level BUILT_IN_{,L,LL,IMAX} group of functions to any integral mode (including vector modes). - DEF_INTERNAL_INSN_FN defines an internal function that maps to target - instruction INSN, which is one of those defined in target-insns.def. - The instruction has NOUTPUTS output operands (either 0 or 1) and - NINPUTS input operands. - Each entry must have a corresponding expander of the form: void expand_NAME (gimple_call stmt) @@ -126,11 +120,6 @@ along with GCC; see the file COPYING3. If not see DEF_INTERNAL_OPTAB_FN (NAME, FLAGS, OPTAB, TYPE) #endif -#ifndef DEF_INTERNAL_INSN_FN -#define DEF_INTERNAL_INSN_FN(NAME, FLAGS, INSN, NOUTPUTS, NINPUTS) \ - DEF_INTERNAL_FN (NAME, FLAGS | ECF_LEAF, NULL) -#endif - DEF_INTERNAL_OPTAB_FN (MASK_LOAD, ECF_PURE, maskload, mask_load) DEF_INTERNAL_OPTAB_FN (LOAD_LANES, ECF_CONST, vec_load_lanes, load_lanes) DEF_INTERNAL_OPTAB_FN (MASK_LOAD_LANES, ECF_PURE, @@ -326,21 +315,15 @@ DEF_INTERNAL_INT_FN (POPCOUNT, ECF_CONST | ECF_NOTHROW, popcount, unary) DEF_INTERNAL_FN (GOMP_USE_SIMT, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (GOMP_SIMT_ENTER, ECF_LEAF | ECF_NOTHROW, NULL) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_ENTER_ALLOC, ECF_NOTHROW, omp_simt_enter, 1, 2) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_EXIT, ECF_NOTHROW, omp_simt_exit, 0, 1) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_LANE, ECF_NOVOPS | ECF_NOTHROW, - omp_simt_lane, 1, 0) +DEF_INTERNAL_FN (GOMP_SIMT_ENTER_ALLOC, ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_EXIT, ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_LANE, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (GOMP_SIMT_VF, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_LAST_LANE, ECF_NOVOPS | ECF_NOTHROW, - omp_simt_last_lane, 1, 1) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_ORDERED_PRED, ECF_NOTHROW, - omp_simt_ordered, 1, 1) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_VOTE_ANY, ECF_NOVOPS | ECF_NOTHROW, - omp_simt_vote_any, 1, 1) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_XCHG_BFLY, ECF_NOVOPS | ECF_NOTHROW, - omp_simt_xchg_bfly, 1, 2) -DEF_INTERNAL_INSN_FN (GOMP_SIMT_XCHG_IDX, ECF_NOVOPS | ECF_NOTHROW, - omp_simt_xchg_idx, 1, 2) +DEF_INTERNAL_FN (GOMP_SIMT_LAST_LANE, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_ORDERED_PRED, ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_VOTE_ANY, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_XCHG_BFLY, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) +DEF_INTERNAL_FN (GOMP_SIMT_XCHG_IDX, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (GOMP_SIMD_LANE, ECF_NOVOPS | ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (GOMP_SIMD_VF, ECF_CONST | ECF_LEAF | ECF_NOTHROW, NULL) DEF_INTERNAL_FN (GOMP_SIMD_LAST_LANE, ECF_CONST | ECF_LEAF | ECF_NOTHROW, NULL) @@ -450,7 +433,6 @@ DEF_INTERNAL_FN (SHUFFLEVECTOR, ECF_CONST | ECF_LEAF | ECF_NOTHROW, NULL) /* <=> optimization. */ DEF_INTERNAL_FN (SPACESHIP, ECF_CONST | ECF_LEAF | ECF_NOTHROW, NULL) -#undef DEF_INTERNAL_INSN_FN #undef DEF_INTERNAL_INT_FN #undef DEF_INTERNAL_FLT_FN #undef DEF_INTERNAL_FLT_FLOATN_FN diff --git a/gcc/internal-fn.h b/gcc/internal-fn.h index 3409c42e4d0..23c014a963c 100644 --- a/gcc/internal-fn.h +++ b/gcc/internal-fn.h @@ -133,8 +133,7 @@ internal_fn_fnspec (enum internal_fn fn) return internal_fn_fnspec_array[(int) fn]; } -/* Describes an internal function that maps directly to either an optab - or an instruction defined in target-insns.def. */ +/* Describes an internal function that maps directly to an optab. */ struct direct_internal_fn_info { /* optabs can be parameterized by one or two modes. These fields describe @@ -145,25 +144,24 @@ struct direct_internal_fn_info function isn't directly mapped to an optab. */ signed int type0 : 8; signed int type1 : 8; - /* True if the function is directly mapped to either an optab or an - instruction defined in target-insns.def. */ - unsigned int directly_mapped : 1; /* True if the function is pointwise, so that it can be vectorized by converting the return type and all argument types to vectors of the same number of elements. E.g. we can vectorize an IFN_SQRT on - floats as an IFN_SQRT on vectors of N floats. */ - unsigned int vectorizable : 1; + floats as an IFN_SQRT on vectors of N floats. + + This only needs 1 bit, but occupies the full 16 to ensure a nice + layout. */ + unsigned int vectorizable : 16; }; extern const direct_internal_fn_info direct_internal_fn_array[IFN_LAST + 1]; -/* Return true if FN is mapped directly to either an optab or an instruction - defined in target-insns.def. */ +/* Return true if FN is mapped directly to an optab. */ inline bool direct_internal_fn_p (internal_fn fn) { - return direct_internal_fn_array[fn].directly_mapped; + return direct_internal_fn_array[fn].type0 >= -1; } /* Return true if FN is a direct internal function that can be vectorized by @@ -177,7 +175,7 @@ vectorizable_internal_fn_p (internal_fn fn) return direct_internal_fn_array[fn].vectorizable; } -/* Return information about internal function FN. Only meaningful +/* Return optab information about internal function FN. Only meaningful if direct_internal_fn_p (FN). */ inline const direct_internal_fn_info &