From patchwork Wed Jun 27 09:31:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 935355 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-480542-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="p9IT4DGM"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41FyNl09KFz9s2R for ; Wed, 27 Jun 2018 19:32:05 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:references:date:in-reply-to:message-id :mime-version:content-type; q=dns; s=default; b=vk+IsyRlThmZ6xPu Uc/9DiH0e3AptW9k5MTYMndMvq4ybSDHcCZ2iwmzyjsd4KmFb2El5eZYwuoCZ7EM 5lT/+SBNjhyxUMM4rzgNdjJzBNFgjIK3vanT8tjOo3gNj/snkTkjdv6vTz4z/SSF Rjf2MH4RG0UZk4JsxxHu77l/gwo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:references:date:in-reply-to:message-id :mime-version:content-type; s=default; bh=BfSoOtds8khh6cqp/P2Nur wS7Jk=; b=p9IT4DGM77im4Ob60d5AIkcpcR3LJK7Ybtn9OCzUNcpKsp+f2VEzlu rYmVRNiChzbDRMhShWMV3e7Q7OawGsmk3geZsYLPo+daEol070fBMfBujDD71trF 1uVq1bSZJaS4JrHvJtWA0kI63Wf/ShLDqHlMprJ6xzQ6x+ylhse24= Received: (qmail 108035 invoked by alias); 27 Jun 2018 09:31:35 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 107893 invoked by uid 89); 27 Jun 2018 09:31:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-16.1 required=5.0 tests=BAYES_00, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, SPF_PASS autolearn=ham version=3.3.2 spammy=oddly, (unknown) X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 27 Jun 2018 09:31:21 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7C85918A; Wed, 27 Jun 2018 02:31:05 -0700 (PDT) Received: from localhost (unknown [10.32.98.107]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AF9C53F5AD; Wed, 27 Jun 2018 02:31:04 -0700 (PDT) From: Richard Sandiford To: Richard Biener Mail-Followup-To: Richard Biener , GCC Patches , richard.sandiford@arm.com Cc: GCC Patches Subject: [9a/n] PR85694: Reorder vect_is_simple_use arguments References: <87d0wof858.fsf@arm.com> <87r2l1eoxk.fsf@arm.com> Date: Wed, 27 Jun 2018 10:31:03 +0100 In-Reply-To: <87r2l1eoxk.fsf@arm.com> (Richard Sandiford's message of "Wed, 20 Jun 2018 11:24:07 +0100") Message-ID: <87y3f0bmp4.fsf_-_@arm.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Richard Sandiford writes: > Richard Biener writes: >> On Mon, Jun 18, 2018 at 5:04 PM Richard Sandiford >> wrote: >>> >>> When following the definitions of SSA names, some recognisers >>> already cope with statements that have been replaced by patterns. >>> This patch makes that happen automatically for users of >>> type_conversion_p and vect_get_internal_def. It also adds >>> a vect_look_through_pattern helper that can be used directly. >>> >>> The reason for doing this is that the main patch for PR85694 >>> makes over_widening handle more general cases. These over-widened >>> patterns can still be useful when matching later statements; >>> e.g. an overwidened MULT_EXPR could be the input to a DOT_PROD_EXPR. >>> >>> The patch doesn't do anything with the STMT_VINFO_IN_PATTERN_P checks >>> in vect_recog_over_widening_pattern or vect_recog_widen_shift_pattern >>> since later patches rewrite them anyway. >>> >>> Doing this fixed an XFAIL in vect-reduc-dot-u16b.c. >>> >>> Tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install? >> >> Hmm. It seems to me that *def_stmt for vect_is_simple_use should >> eventually be the pattern def given the vectype overload takes the >> vectype from the pattern def already but oddly enough the >> DEF_TYPE is taken from the non-pattern stmt. >> >> I wonder which callers look at def_stmt at all (and how...) >> >> I guess swapping the def_stmt and dt arguments and adding yet another >> overload to remove all unused &def_stmt args might this easier to review... >> >> So - I'm suggesting to change vect_is_simple_use. > > OK, I'll try that. Might end up being its own mini-series. :-) Turned out to be simpler than feared. :-) This patch does the first bit: reorder the arguments to vect_is_simple_use so that def_stmt comes last and is optional. Many callers can then drop it, making it more obvious which of the remaining calls would be affected by the next patch. Tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install? Richard 2018-06-27 Richard Sandiford gcc/ * tree-vectorizer.h (vect_is_simple_use): Move the gimple ** to the end and default to null. * tree-vect-loop.c (vect_create_epilog_for_reduction) (vectorizable_reduction): Update calls accordingly, dropping the gimple ** argument if the passed-back statement isn't needed. * tree-vect-patterns.c (vect_get_internal_def, type_conversion_p) (vect_recog_rotate_pattern): Likewise. (vect_recog_mask_conversion_pattern): Likewise. * tree-vect-slp.c (vect_get_and_check_slp_defs): Likewise. (vect_mask_constant_operand_p): Likewise. * tree-vect-stmts.c (is_simple_and_all_uses_invariant, process_use): (vect_model_simple_cost, vect_get_vec_def_for_operand): Likewise. (get_group_load_store_type, get_load_store_type): Likewise. (vect_check_load_store_mask, vect_check_store_rhs): Likewise. (vectorizable_call, vectorizable_simd_clone_call): Likewise. (vectorizable_conversion, vectorizable_assignment): Likewise. (vectorizable_shift, vectorizable_operation): Likewise. (vectorizable_store, vect_is_simple_cond): Likewise. (vectorizable_condition, vectorizable_comparison): Likewise. (get_same_sized_vectype, vect_get_mask_type_for_stmt): Likewise. (vect_is_simple_use): Rename the def_stmt argument to def_stmt_out and move it to the end. Cope with null def_stmt_outs. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2018-06-27 10:27:12.366628072 +0100 +++ gcc/tree-vectorizer.h 2018-06-27 10:27:31.782458413 +0100 @@ -1476,10 +1476,10 @@ extern tree get_vectype_for_scalar_type_ extern tree get_mask_type_for_scalar_type (tree); extern tree get_same_sized_vectype (tree, tree); extern bool vect_get_loop_mask_type (loop_vec_info); -extern bool vect_is_simple_use (tree, vec_info *, gimple **, - enum vect_def_type *); -extern bool vect_is_simple_use (tree, vec_info *, gimple **, - enum vect_def_type *, tree *); +extern bool vect_is_simple_use (tree, vec_info *, enum vect_def_type *, + gimple ** = NULL); +extern bool vect_is_simple_use (tree, vec_info *, enum vect_def_type *, + tree *, gimple ** = NULL); extern bool supportable_widening_operation (enum tree_code, gimple *, tree, tree, enum tree_code *, enum tree_code *, int *, Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2018-06-27 10:27:09.038657152 +0100 +++ gcc/tree-vect-loop.c 2018-06-27 10:27:31.782458413 +0100 @@ -4567,7 +4567,6 @@ vect_create_epilog_for_reduction (vecvinfo, def_stmt, &dt)) + if (!vect_is_simple_use (name, stmt_vinfo->vinfo, &dt, def_stmt)) return false; if (dt != vect_internal_def @@ -219,7 +218,7 @@ type_conversion_p (tree name, gimple *us else *promotion = false; - if (!vect_is_simple_use (oprnd0, stmt_vinfo->vinfo, &dummy_gimple, &dt)) + if (!vect_is_simple_use (oprnd0, stmt_vinfo->vinfo, &dt)) return false; return true; @@ -1795,7 +1794,7 @@ vect_recog_rotate_pattern (vec || !TYPE_UNSIGNED (type)) return NULL; - if (!vect_is_simple_use (oprnd1, vinfo, &def_stmt, &dt)) + if (!vect_is_simple_use (oprnd1, vinfo, &dt, &def_stmt)) return NULL; if (dt != vect_internal_def @@ -3930,13 +3929,10 @@ vect_recog_mask_conversion_pattern (vec< && known_le (TYPE_VECTOR_SUBPARTS (vectype1), TYPE_VECTOR_SUBPARTS (vectype2))) { - gimple *dummy; enum vect_def_type dt; - if (vect_is_simple_use (TREE_OPERAND (rhs1, 0), stmt_vinfo->vinfo, - &dummy, &dt) + if (vect_is_simple_use (TREE_OPERAND (rhs1, 0), vinfo, &dt) && dt == vect_external_def - && vect_is_simple_use (TREE_OPERAND (rhs1, 1), stmt_vinfo->vinfo, - &dummy, &dt) + && vect_is_simple_use (TREE_OPERAND (rhs1, 1), vinfo, &dt) && (dt == vect_external_def || dt == vect_constant_def)) { Index: gcc/tree-vect-slp.c =================================================================== --- gcc/tree-vect-slp.c 2018-06-27 10:27:10.106647819 +0100 +++ gcc/tree-vect-slp.c 2018-06-27 10:27:31.782458413 +0100 @@ -348,7 +348,7 @@ vect_get_and_check_slp_defs (vec_info *v oprnd_info = (*oprnds_info)[i]; - if (!vect_is_simple_use (oprnd, vinfo, &def_stmt, &dt)) + if (!vect_is_simple_use (oprnd, vinfo, &dt, &def_stmt)) { if (dump_enabled_p ()) { @@ -3105,7 +3105,6 @@ vect_mask_constant_operand_p (gimple *st stmt_vec_info stmt_vinfo = vinfo_for_stmt (stmt); enum tree_code code = gimple_expr_code (stmt); tree op, vectype; - gimple *def_stmt; enum vect_def_type dt; /* For comparison and COND_EXPR type is chosen depending @@ -3117,8 +3116,7 @@ vect_mask_constant_operand_p (gimple *st else op = gimple_assign_rhs2 (stmt); - if (!vect_is_simple_use (op, stmt_vinfo->vinfo, &def_stmt, - &dt, &vectype)) + if (!vect_is_simple_use (op, stmt_vinfo->vinfo, &dt, &vectype)) gcc_unreachable (); return !vectype || VECTOR_BOOLEAN_TYPE_P (vectype); @@ -3135,8 +3133,7 @@ vect_mask_constant_operand_p (gimple *st else op = TREE_OPERAND (cond, 0); - if (!vect_is_simple_use (op, stmt_vinfo->vinfo, &def_stmt, - &dt, &vectype)) + if (!vect_is_simple_use (op, stmt_vinfo->vinfo, &dt, &vectype)) gcc_unreachable (); return !vectype || VECTOR_BOOLEAN_TYPE_P (vectype); Index: gcc/tree-vect-stmts.c =================================================================== --- gcc/tree-vect-stmts.c 2018-06-27 10:27:09.042657117 +0100 +++ gcc/tree-vect-stmts.c 2018-06-27 10:27:31.782458413 +0100 @@ -260,7 +260,6 @@ vect_mark_relevant (vec *workl is_simple_and_all_uses_invariant (gimple *stmt, loop_vec_info loop_vinfo) { tree op; - gimple *def_stmt; ssa_op_iter iter; if (!is_gimple_assign (stmt)) @@ -270,7 +269,7 @@ is_simple_and_all_uses_invariant (gimple { enum vect_def_type dt = vect_uninitialized_def; - if (!vect_is_simple_use (op, loop_vinfo, &def_stmt, &dt)) + if (!vect_is_simple_use (op, loop_vinfo, &dt)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -472,7 +471,7 @@ process_use (gimple *stmt, tree use, loo if (!force && !exist_non_indexing_operands_for_use_p (use, stmt)) return true; - if (!vect_is_simple_use (use, loop_vinfo, &def_stmt, &dt)) + if (!vect_is_simple_use (use, loop_vinfo, &dt, &def_stmt)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -908,11 +907,10 @@ vect_model_simple_cost (stmt_vec_info st for (unsigned i = 0; i < gimple_num_ops (stmt); ++i) { tree op = gimple_op (stmt, i); - gimple *def_stmt; enum vect_def_type dt; if (!op || op == lhs) continue; - if (vect_is_simple_use (op, stmt_info->vinfo, &def_stmt, &dt) + if (vect_is_simple_use (op, stmt_info->vinfo, &dt) && (dt == vect_constant_def || dt == vect_external_def)) prologue_cost += vect_prologue_cost_for_slp_op (node, stmt_info, i, dt, cost_vec); @@ -1567,7 +1565,7 @@ vect_get_vec_def_for_operand (tree op, g dump_printf (MSG_NOTE, "\n"); } - is_simple_use = vect_is_simple_use (op, loop_vinfo, &def_stmt, &dt); + is_simple_use = vect_is_simple_use (op, loop_vinfo, &dt, &def_stmt); gcc_assert (is_simple_use); if (def_stmt && dump_enabled_p ()) { @@ -2336,9 +2334,8 @@ get_group_load_store_type (gimple *stmt, while (next_stmt) { tree op = vect_get_store_rhs (next_stmt); - gimple *def_stmt; enum vect_def_type dt; - if (!vect_is_simple_use (op, vinfo, &def_stmt, &dt)) + if (!vect_is_simple_use (op, vinfo, &dt)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -2436,10 +2433,9 @@ get_load_store_type (gimple *stmt, tree if (STMT_VINFO_GATHER_SCATTER_P (stmt_info)) { *memory_access_type = VMAT_GATHER_SCATTER; - gimple *def_stmt; if (!vect_check_gather_scatter (stmt, loop_vinfo, gs_info)) gcc_unreachable (); - else if (!vect_is_simple_use (gs_info->offset, vinfo, &def_stmt, + else if (!vect_is_simple_use (gs_info->offset, vinfo, &gs_info->offset_dt, &gs_info->offset_vectype)) { @@ -2536,11 +2532,9 @@ vect_check_load_store_mask (gimple *stmt } stmt_vec_info stmt_info = vinfo_for_stmt (stmt); - gimple *def_stmt; enum vect_def_type mask_dt; tree mask_vectype; - if (!vect_is_simple_use (mask, stmt_info->vinfo, &def_stmt, &mask_dt, - &mask_vectype)) + if (!vect_is_simple_use (mask, stmt_info->vinfo, &mask_dt, &mask_vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -2601,11 +2595,9 @@ vect_check_store_rhs (gimple *stmt, tree } stmt_vec_info stmt_info = vinfo_for_stmt (stmt); - gimple *def_stmt; enum vect_def_type rhs_dt; tree rhs_vectype; - if (!vect_is_simple_use (rhs, stmt_info->vinfo, &def_stmt, &rhs_dt, - &rhs_vectype)) + if (!vect_is_simple_use (rhs, stmt_info->vinfo, &rhs_dt, &rhs_vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -3136,7 +3128,6 @@ vectorizable_call (gimple *gs, gimple_st bb_vec_info bb_vinfo = STMT_VINFO_BB_VINFO (stmt_info); vec_info *vinfo = stmt_info->vinfo; tree fndecl, new_temp, rhs_type; - gimple *def_stmt; enum vect_def_type dt[3] = {vect_unknown_def_type, vect_unknown_def_type, vect_unknown_def_type}; int ndts = 3; @@ -3210,7 +3201,7 @@ vectorizable_call (gimple *gs, gimple_st if (!rhs_type) rhs_type = TREE_TYPE (op); - if (!vect_is_simple_use (op, vinfo, &def_stmt, &dt[i], &opvectype)) + if (!vect_is_simple_use (op, vinfo, &dt[i], &opvectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -3731,7 +3722,6 @@ vectorizable_simd_clone_call (gimple *st vec_info *vinfo = stmt_info->vinfo; struct loop *loop = loop_vinfo ? LOOP_VINFO_LOOP (loop_vinfo) : NULL; tree fndecl, new_temp; - gimple *def_stmt; gimple *new_stmt = NULL; int ncopies, j; auto_vec arginfo; @@ -3794,7 +3784,7 @@ vectorizable_simd_clone_call (gimple *st thisarginfo.simd_lane_linear = false; op = gimple_call_arg (stmt, i); - if (!vect_is_simple_use (op, vinfo, &def_stmt, &thisarginfo.dt, + if (!vect_is_simple_use (op, vinfo, &thisarginfo.dt, &thisarginfo.vectype) || thisarginfo.dt == vect_uninitialized_def) { @@ -4575,7 +4565,6 @@ vectorizable_conversion (gimple *stmt, g enum tree_code codecvt1 = ERROR_MARK, codecvt2 = ERROR_MARK; tree decl1 = NULL_TREE, decl2 = NULL_TREE; tree new_temp; - gimple *def_stmt; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; int ndts = 2; gimple *new_stmt = NULL; @@ -4651,7 +4640,7 @@ vectorizable_conversion (gimple *stmt, g } /* Check the operands of the operation. */ - if (!vect_is_simple_use (op0, vinfo, &def_stmt, &dt[0], &vectype_in)) + if (!vect_is_simple_use (op0, vinfo, &dt[0], &vectype_in)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -4667,9 +4656,9 @@ vectorizable_conversion (gimple *stmt, g /* For WIDEN_MULT_EXPR, if OP0 is a constant, use the type of OP1. */ if (CONSTANT_CLASS_P (op0)) - ok = vect_is_simple_use (op1, vinfo, &def_stmt, &dt[1], &vectype_in); + ok = vect_is_simple_use (op1, vinfo, &dt[1], &vectype_in); else - ok = vect_is_simple_use (op1, vinfo, &def_stmt, &dt[1]); + ok = vect_is_simple_use (op1, vinfo, &dt[1]); if (!ok) { @@ -5171,7 +5160,6 @@ vectorizable_assignment (gimple *stmt, g stmt_vec_info stmt_info = vinfo_for_stmt (stmt); loop_vec_info loop_vinfo = STMT_VINFO_LOOP_VINFO (stmt_info); tree new_temp; - gimple *def_stmt; enum vect_def_type dt[1] = {vect_unknown_def_type}; int ndts = 1; int ncopies; @@ -5224,7 +5212,7 @@ vectorizable_assignment (gimple *stmt, g gcc_assert (ncopies >= 1); - if (!vect_is_simple_use (op, vinfo, &def_stmt, &dt[0], &vectype_in)) + if (!vect_is_simple_use (op, vinfo, &dt[0], &vectype_in)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5380,7 +5368,6 @@ vectorizable_shift (gimple *stmt, gimple optab optab; int icode; machine_mode optab_op2_mode; - gimple *def_stmt; enum vect_def_type dt[2] = {vect_unknown_def_type, vect_unknown_def_type}; int ndts = 2; gimple *new_stmt = NULL; @@ -5430,7 +5417,7 @@ vectorizable_shift (gimple *stmt, gimple } op0 = gimple_assign_rhs1 (stmt); - if (!vect_is_simple_use (op0, vinfo, &def_stmt, &dt[0], &vectype)) + if (!vect_is_simple_use (op0, vinfo, &dt[0], &vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5457,7 +5444,7 @@ vectorizable_shift (gimple *stmt, gimple return false; op1 = gimple_assign_rhs2 (stmt); - if (!vect_is_simple_use (op1, vinfo, &def_stmt, &dt[1], &op1_vectype)) + if (!vect_is_simple_use (op1, vinfo, &dt[1], &op1_vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5746,7 +5733,6 @@ vectorizable_operation (gimple *stmt, gi int op_type; optab optab; bool target_support_p; - gimple *def_stmt; enum vect_def_type dt[3] = {vect_unknown_def_type, vect_unknown_def_type, vect_unknown_def_type}; int ndts = 3; @@ -5817,7 +5803,7 @@ vectorizable_operation (gimple *stmt, gi } op0 = gimple_assign_rhs1 (stmt); - if (!vect_is_simple_use (op0, vinfo, &def_stmt, &dt[0], &vectype)) + if (!vect_is_simple_use (op0, vinfo, &dt[0], &vectype)) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5871,7 +5857,7 @@ vectorizable_operation (gimple *stmt, gi if (op_type == binary_op || op_type == ternary_op) { op1 = gimple_assign_rhs2 (stmt); - if (!vect_is_simple_use (op1, vinfo, &def_stmt, &dt[1])) + if (!vect_is_simple_use (op1, vinfo, &dt[1])) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -5882,7 +5868,7 @@ vectorizable_operation (gimple *stmt, gi if (op_type == ternary_op) { op2 = gimple_assign_rhs3 (stmt); - if (!vect_is_simple_use (op2, vinfo, &def_stmt, &dt[2])) + if (!vect_is_simple_use (op2, vinfo, &dt[2])) { if (dump_enabled_p ()) dump_printf_loc (MSG_MISSED_OPTIMIZATION, vect_location, @@ -6196,7 +6182,6 @@ vectorizable_store (gimple *stmt, gimple machine_mode vec_mode; tree dummy; enum dr_alignment_support alignment_support_scheme; - gimple *def_stmt; enum vect_def_type rhs_dt = vect_unknown_def_type; enum vect_def_type mask_dt = vect_unknown_def_type; stmt_vec_info prev_stmt_info = NULL; @@ -6757,7 +6742,7 @@ vectorizable_store (gimple *stmt, gimple vec_oprnd = vec_oprnds[j]; else { - vect_is_simple_use (op, vinfo, &def_stmt, &rhs_dt); + vect_is_simple_use (op, vinfo, &rhs_dt); vec_oprnd = vect_get_vec_def_for_stmt_copy (rhs_dt, vec_oprnd); } @@ -7003,7 +6988,7 @@ vectorizable_store (gimple *stmt, gimple for (i = 0; i < group_size; i++) { op = oprnds[i]; - vect_is_simple_use (op, vinfo, &def_stmt, &rhs_dt); + vect_is_simple_use (op, vinfo, &rhs_dt); vec_oprnd = vect_get_vec_def_for_stmt_copy (rhs_dt, op); dr_chain[i] = vec_oprnd; oprnds[i] = vec_oprnd; @@ -8588,9 +8573,7 @@ vect_is_simple_cond (tree cond, vec_info if (TREE_CODE (cond) == SSA_NAME && VECT_SCALAR_BOOLEAN_TYPE_P (TREE_TYPE (cond))) { - gimple *lhs_def_stmt = SSA_NAME_DEF_STMT (cond); - if (!vect_is_simple_use (cond, vinfo, &lhs_def_stmt, - &dts[0], comp_vectype) + if (!vect_is_simple_use (cond, vinfo, &dts[0], comp_vectype) || !*comp_vectype || !VECTOR_BOOLEAN_TYPE_P (*comp_vectype)) return false; @@ -8605,8 +8588,7 @@ vect_is_simple_cond (tree cond, vec_info if (TREE_CODE (lhs) == SSA_NAME) { - gimple *lhs_def_stmt = SSA_NAME_DEF_STMT (lhs); - if (!vect_is_simple_use (lhs, vinfo, &lhs_def_stmt, &dts[0], &vectype1)) + if (!vect_is_simple_use (lhs, vinfo, &dts[0], &vectype1)) return false; } else if (TREE_CODE (lhs) == INTEGER_CST || TREE_CODE (lhs) == REAL_CST @@ -8617,8 +8599,7 @@ vect_is_simple_cond (tree cond, vec_info if (TREE_CODE (rhs) == SSA_NAME) { - gimple *rhs_def_stmt = SSA_NAME_DEF_STMT (rhs); - if (!vect_is_simple_use (rhs, vinfo, &rhs_def_stmt, &dts[1], &vectype2)) + if (!vect_is_simple_use (rhs, vinfo, &dts[1], &vectype2)) return false; } else if (TREE_CODE (rhs) == INTEGER_CST || TREE_CODE (rhs) == REAL_CST @@ -8750,12 +8731,9 @@ vectorizable_condition (gimple *stmt, gi || !comp_vectype) return false; - gimple *def_stmt; - if (!vect_is_simple_use (then_clause, stmt_info->vinfo, &def_stmt, &dts[2], - &vectype1)) + if (!vect_is_simple_use (then_clause, stmt_info->vinfo, &dts[2], &vectype1)) return false; - if (!vect_is_simple_use (else_clause, stmt_info->vinfo, &def_stmt, &dts[3], - &vectype2)) + if (!vect_is_simple_use (else_clause, stmt_info->vinfo, &dts[3], &vectype2)) return false; if (vectype1 && !useless_type_conversion_p (vectype, vectype1)) @@ -8892,26 +8870,24 @@ vectorizable_condition (gimple *stmt, gi } else { - gimple *gtemp; if (masked) { vec_cond_lhs = vect_get_vec_def_for_operand (cond_expr, stmt, comp_vectype); - vect_is_simple_use (cond_expr, stmt_info->vinfo, - >emp, &dts[0]); + vect_is_simple_use (cond_expr, stmt_info->vinfo, &dts[0]); } else { vec_cond_lhs = vect_get_vec_def_for_operand (cond_expr0, stmt, comp_vectype); - vect_is_simple_use (cond_expr0, loop_vinfo, >emp, &dts[0]); + vect_is_simple_use (cond_expr0, loop_vinfo, &dts[0]); vec_cond_rhs = vect_get_vec_def_for_operand (cond_expr1, stmt, comp_vectype); - vect_is_simple_use (cond_expr1, loop_vinfo, >emp, &dts[1]); + vect_is_simple_use (cond_expr1, loop_vinfo, &dts[1]); } if (reduc_index == 1) vec_then_clause = reduc_def; @@ -8919,8 +8895,7 @@ vectorizable_condition (gimple *stmt, gi { vec_then_clause = vect_get_vec_def_for_operand (then_clause, stmt); - vect_is_simple_use (then_clause, loop_vinfo, - >emp, &dts[2]); + vect_is_simple_use (then_clause, loop_vinfo, &dts[2]); } if (reduc_index == 2) vec_else_clause = reduc_def; @@ -8928,7 +8903,7 @@ vectorizable_condition (gimple *stmt, gi { vec_else_clause = vect_get_vec_def_for_operand (else_clause, stmt); - vect_is_simple_use (else_clause, loop_vinfo, >emp, &dts[3]); + vect_is_simple_use (else_clause, loop_vinfo, &dts[3]); } } } @@ -9089,7 +9064,6 @@ vectorizable_comparison (gimple *stmt, g bb_vec_info bb_vinfo = STMT_VINFO_BB_VINFO (stmt_info); vec vec_oprnds0 = vNULL; vec vec_oprnds1 = vNULL; - gimple *def_stmt; tree mask_type; tree mask; @@ -9132,12 +9106,10 @@ vectorizable_comparison (gimple *stmt, g rhs1 = gimple_assign_rhs1 (stmt); rhs2 = gimple_assign_rhs2 (stmt); - if (!vect_is_simple_use (rhs1, stmt_info->vinfo, &def_stmt, - &dts[0], &vectype1)) + if (!vect_is_simple_use (rhs1, stmt_info->vinfo, &dts[0], &vectype1)) return false; - if (!vect_is_simple_use (rhs2, stmt_info->vinfo, &def_stmt, - &dts[1], &vectype2)) + if (!vect_is_simple_use (rhs2, stmt_info->vinfo, &dts[1], &vectype2)) return false; if (vectype1 && vectype2 @@ -10043,7 +10015,7 @@ get_same_sized_vectype (tree scalar_type VINFO - the vect info of the loop or basic block that is being vectorized. OPERAND - operand in the loop or bb. Output: - DEF_STMT - the defining stmt in case OPERAND is an SSA_NAME. + DEF_STMT_OUT (optional) - the defining stmt in case OPERAND is an SSA_NAME. DT - the type of definition Returns whether a stmt with OPERAND can be vectorized. @@ -10055,10 +10027,11 @@ get_same_sized_vectype (tree scalar_type For now, operands defined outside the basic block are not supported. */ bool -vect_is_simple_use (tree operand, vec_info *vinfo, - gimple **def_stmt, enum vect_def_type *dt) +vect_is_simple_use (tree operand, vec_info *vinfo, enum vect_def_type *dt, + gimple **def_stmt_out) { - *def_stmt = NULL; + if (def_stmt_out) + *def_stmt_out = NULL; *dt = vect_unknown_def_type; if (dump_enabled_p ()) @@ -10095,18 +10068,20 @@ vect_is_simple_use (tree operand, vec_in return true; } - *def_stmt = SSA_NAME_DEF_STMT (operand); + gimple *def_stmt = SSA_NAME_DEF_STMT (operand); + if (def_stmt_out) + *def_stmt_out = def_stmt; if (dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, "def_stmt: "); - dump_gimple_stmt (MSG_NOTE, TDF_SLIM, *def_stmt, 0); + dump_gimple_stmt (MSG_NOTE, TDF_SLIM, def_stmt, 0); } - if (! vect_stmt_in_region_p (vinfo, *def_stmt)) + if (! vect_stmt_in_region_p (vinfo, def_stmt)) *dt = vect_external_def; else { - stmt_vec_info stmt_vinfo = vinfo_for_stmt (*def_stmt); + stmt_vec_info stmt_vinfo = vinfo_for_stmt (def_stmt); *dt = STMT_VINFO_DEF_TYPE (stmt_vinfo); } @@ -10153,7 +10128,7 @@ vect_is_simple_use (tree operand, vec_in return false; } - switch (gimple_code (*def_stmt)) + switch (gimple_code (def_stmt)) { case GIMPLE_PHI: case GIMPLE_ASSIGN: @@ -10179,12 +10154,16 @@ vect_is_simple_use (tree operand, vec_in scalar operand. */ bool -vect_is_simple_use (tree operand, vec_info *vinfo, - gimple **def_stmt, enum vect_def_type *dt, tree *vectype) +vect_is_simple_use (tree operand, vec_info *vinfo, enum vect_def_type *dt, + tree *vectype, gimple **def_stmt_out) { - if (!vect_is_simple_use (operand, vinfo, def_stmt, dt)) + gimple *def_stmt; + if (!vect_is_simple_use (operand, vinfo, dt, &def_stmt)) return false; + if (def_stmt_out) + *def_stmt_out = def_stmt; + /* Now get a vector type if the def is internal, otherwise supply NULL_TREE and leave it up to the caller to figure out a proper type for the use stmt. */ @@ -10194,7 +10173,7 @@ vect_is_simple_use (tree operand, vec_in || *dt == vect_double_reduction_def || *dt == vect_nested_cycle) { - stmt_vec_info stmt_info = vinfo_for_stmt (*def_stmt); + stmt_vec_info stmt_info = vinfo_for_stmt (def_stmt); if (STMT_VINFO_IN_PATTERN_P (stmt_info) && !STMT_VINFO_RELEVANT (stmt_info) @@ -10869,13 +10848,11 @@ vect_get_mask_type_for_stmt (stmt_vec_in { tree rhs; ssa_op_iter iter; - gimple *def_stmt; enum vect_def_type dt; FOR_EACH_SSA_TREE_OPERAND (rhs, stmt, iter, SSA_OP_USE) { - if (!vect_is_simple_use (rhs, stmt_info->vinfo, - &def_stmt, &dt, &vectype)) + if (!vect_is_simple_use (rhs, stmt_info->vinfo, &dt, &vectype)) { if (dump_enabled_p ()) {