From patchwork Sun Oct 20 13:30:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1180001 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-511376-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="dICKIXJu"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46x0ym41vLz9sP7 for ; Mon, 21 Oct 2019 00:30:56 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=default; b=ImPcPRhVSKxP0GmjGyLWLV38ipK6l DxYwLDyoYJ7f69HNhdwZwhS+9fFh9zT+h4CYpfqyia+CdLpiH836XXTTwqcne8Jr sudjt8aj2meMMIF0yii8l4RCFpXzx5tybHx3LxyGglYVCN0Sb1pMDdoo3V6PWW4J HR81XZDYZrXWp8= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:references:date:in-reply-to:message-id:mime-version :content-type; s=default; bh=I/f1RlGPLntEiS8Mqwp1iBL/OeY=; b=dIC KIXJuKA+H8pgiWnTja+DQfTfUXxpI4cs9KiUhFYtPd0MmvEqKc/ok7fMe/R/LBQA 5w1Ik6ZhGGNne3whnFhWF27zllUzX0KxIkWyXVIORV2WNI5lMUnmWnaSlJ8RqVJ1 /imWDOM1qY//cz1UqdP3AkrZC2JGmIoPQi1+SO+E= Received: (qmail 37433 invoked by alias); 20 Oct 2019 13:30:49 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 37396 invoked by uid 89); 20 Oct 2019 13:30:44 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-9.1 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, SPF_PASS autolearn=ham version=3.3.1 spammy=rational, aux, 23807, unroll X-HELO: foss.arm.com Received: from Unknown (HELO foss.arm.com) (217.140.110.172) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Sun, 20 Oct 2019 13:30:42 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D570493 for ; Sun, 20 Oct 2019 06:30:34 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.98.126]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 433763F718 for ; Sun, 20 Oct 2019 06:30:34 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [3/3] Replace current_vector_size with vec_info::vector_size References: Date: Sun, 20 Oct 2019 14:30:33 +0100 In-Reply-To: (Richard Sandiford's message of "Sun, 20 Oct 2019 14:21:32 +0100") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 X-IsSubscribed: yes Now that all necessary routines have access to the vec_info, it's trivial to convert current_vector_size to a member variable. 2019-10-20 Richard Sandiford gcc/ * tree-vectorizer.h (vec_info::vector_size): New member variable. (vect_update_max_nunits): Update comment. (current_vector_size): Delete. * tree-vect-stmts.c (current_vector_size): Likewise. (get_vectype_for_scalar_type): Use vec_info::vector_size instead of current_vector_size. (get_mask_type_for_scalar_type): Likewise. * tree-vectorizer.c (try_vectorize_loop_1): Likewise. * tree-vect-loop.c (vect_update_vf_for_slp): Likewise. (vect_analyze_loop, vect_halve_mask_nunits): Likewise. (vect_double_mask_nunits, vect_transform_loop): Likewise. * tree-vect-slp.c (can_duplicate_and_interleave_p): Likewise. (vect_make_slp_decision, vect_slp_bb_region): Likewise. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2019-10-20 14:14:33.692550581 +0100 +++ gcc/tree-vectorizer.h 2019-10-20 14:14:36.768528611 +0100 @@ -326,6 +326,10 @@ typedef std::pair vec_object /* Cost data used by the target cost model. */ void *target_cost_data; + /* The vector size for this loop in bytes, or 0 if we haven't picked + a size yet. */ + poly_uint64 vector_size; + private: stmt_vec_info new_stmt_vec_info (gimple *stmt); void set_vinfo_for_stmt (gimple *, stmt_vec_info); @@ -1472,7 +1476,7 @@ vect_get_num_copies (loop_vec_info loop_ static inline void vect_update_max_nunits (poly_uint64 *max_nunits, poly_uint64 nunits) { - /* All unit counts have the form current_vector_size * X for some + /* All unit counts have the form vec_info::vector_size * X for some rational X, so two unit sizes must have a common multiple. Everything is a multiple of the initial value of 1. */ *max_nunits = force_common_multiple (*max_nunits, nunits); @@ -1588,7 +1592,6 @@ extern dump_user_location_t find_loop_lo extern bool vect_can_advance_ivs_p (loop_vec_info); /* In tree-vect-stmts.c. */ -extern poly_uint64 current_vector_size; extern tree get_vectype_for_scalar_type (vec_info *, tree); extern tree get_vectype_for_scalar_type_and_size (tree, poly_uint64); extern tree get_mask_type_for_scalar_type (vec_info *, tree); Index: gcc/tree-vect-stmts.c =================================================================== --- gcc/tree-vect-stmts.c 2019-10-20 14:14:33.692550581 +0100 +++ gcc/tree-vect-stmts.c 2019-10-20 14:14:36.768528611 +0100 @@ -11133,22 +11133,20 @@ get_vectype_for_scalar_type_and_size (tr return vectype; } -poly_uint64 current_vector_size; - /* Function get_vectype_for_scalar_type. Returns the vector type corresponding to SCALAR_TYPE as supported by the target. */ tree -get_vectype_for_scalar_type (vec_info *, tree scalar_type) +get_vectype_for_scalar_type (vec_info *vinfo, tree scalar_type) { tree vectype; vectype = get_vectype_for_scalar_type_and_size (scalar_type, - current_vector_size); + vinfo->vector_size); if (vectype - && known_eq (current_vector_size, 0U)) - current_vector_size = GET_MODE_SIZE (TYPE_MODE (vectype)); + && known_eq (vinfo->vector_size, 0U)) + vinfo->vector_size = GET_MODE_SIZE (TYPE_MODE (vectype)); return vectype; } @@ -11166,7 +11164,7 @@ get_mask_type_for_scalar_type (vec_info return NULL; return build_truth_vector_type (TYPE_VECTOR_SUBPARTS (vectype), - current_vector_size); + vinfo->vector_size); } /* Function get_same_sized_vectype Index: gcc/tree-vectorizer.c =================================================================== --- gcc/tree-vectorizer.c 2019-10-20 14:13:50.784857051 +0100 +++ gcc/tree-vectorizer.c 2019-10-20 14:14:36.768528611 +0100 @@ -971,7 +971,7 @@ try_vectorize_loop_1 (hash_tablevector_size.is_constant (&bytes)) dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location, "loop vectorized using %wu byte vectors\n", bytes); else Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2019-10-20 14:14:33.692550581 +0100 +++ gcc/tree-vect-loop.c 2019-10-20 14:14:36.764528643 +0100 @@ -1414,7 +1414,7 @@ vect_update_vf_for_slp (loop_vec_info lo dump_printf_loc (MSG_NOTE, vect_location, "Loop contains SLP and non-SLP stmts\n"); /* Both the vectorization factor and unroll factor have the form - current_vector_size * X for some rational X, so they must have + loop_vinfo->vector_size * X for some rational X, so they must have a common multiple. */ vectorization_factor = force_common_multiple (vectorization_factor, @@ -2311,7 +2311,6 @@ vect_analyze_loop (class loop *loop, loo auto_vector_sizes vector_sizes; /* Autodetect first vector size we try. */ - current_vector_size = 0; targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, loop->simdlen != 0); unsigned int next_size = 0; @@ -2333,7 +2332,7 @@ vect_analyze_loop (class loop *loop, loo unsigned n_stmts = 0; poly_uint64 autodetected_vector_size = 0; opt_loop_vec_info first_loop_vinfo = opt_loop_vec_info::success (NULL); - poly_uint64 first_vector_size = 0; + poly_uint64 next_vector_size = 0; while (1) { /* Check the CFG characteristics of the loop (nesting, entry/exit). */ @@ -2347,6 +2346,7 @@ vect_analyze_loop (class loop *loop, loo gcc_checking_assert (first_loop_vinfo == NULL); return loop_vinfo; } + loop_vinfo->vector_size = next_vector_size; bool fatal = false; @@ -2365,7 +2365,6 @@ vect_analyze_loop (class loop *loop, loo if (first_loop_vinfo == NULL) { first_loop_vinfo = loop_vinfo; - first_vector_size = current_vector_size; loop->aux = NULL; } else @@ -2381,7 +2380,7 @@ vect_analyze_loop (class loop *loop, loo delete loop_vinfo; if (next_size == 0) - autodetected_vector_size = current_vector_size; + autodetected_vector_size = loop_vinfo->vector_size; if (next_size < vector_sizes.length () && known_eq (vector_sizes[next_size], autodetected_vector_size)) @@ -2394,17 +2393,16 @@ vect_analyze_loop (class loop *loop, loo } if (next_size == vector_sizes.length () - || known_eq (current_vector_size, 0U)) + || known_eq (loop_vinfo->vector_size, 0U)) { if (first_loop_vinfo) { - current_vector_size = first_vector_size; loop->aux = (loop_vec_info) first_loop_vinfo; if (dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, "***** Choosing vector size "); - dump_dec (MSG_NOTE, current_vector_size); + dump_dec (MSG_NOTE, first_loop_vinfo->vector_size); dump_printf (MSG_NOTE, "\n"); } return first_loop_vinfo; @@ -2414,13 +2412,13 @@ vect_analyze_loop (class loop *loop, loo } /* Try the next biggest vector size. */ - current_vector_size = vector_sizes[next_size++]; + next_vector_size = vector_sizes[next_size++]; if (dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, "***** Re-trying analysis with " "vector size "); - dump_dec (MSG_NOTE, current_vector_size); + dump_dec (MSG_NOTE, next_vector_size); dump_printf (MSG_NOTE, "\n"); } } @@ -7745,19 +7743,19 @@ loop_niters_no_overflow (loop_vec_info l /* Return a mask type with half the number of elements as TYPE. */ tree -vect_halve_mask_nunits (vec_info *, tree type) +vect_halve_mask_nunits (vec_info *vinfo, tree type) { poly_uint64 nunits = exact_div (TYPE_VECTOR_SUBPARTS (type), 2); - return build_truth_vector_type (nunits, current_vector_size); + return build_truth_vector_type (nunits, vinfo->vector_size); } /* Return a mask type with twice as many elements as TYPE. */ tree -vect_double_mask_nunits (vec_info *, tree type) +vect_double_mask_nunits (vec_info *vinfo, tree type) { poly_uint64 nunits = TYPE_VECTOR_SUBPARTS (type) * 2; - return build_truth_vector_type (nunits, current_vector_size); + return build_truth_vector_type (nunits, vinfo->vector_size); } /* Record that a fully-masked version of LOOP_VINFO would need MASKS to @@ -8243,7 +8241,7 @@ vect_transform_loop (loop_vec_info loop_ { dump_printf_loc (MSG_NOTE, vect_location, "LOOP EPILOGUE VECTORIZED (VS="); - dump_dec (MSG_NOTE, current_vector_size); + dump_dec (MSG_NOTE, loop_vinfo->vector_size); dump_printf (MSG_NOTE, ")\n"); } } @@ -8295,14 +8293,14 @@ vect_transform_loop (loop_vec_info loop_ unsigned int ratio; while (next_size < vector_sizes.length () - && !(constant_multiple_p (current_vector_size, + && !(constant_multiple_p (loop_vinfo->vector_size, vector_sizes[next_size], &ratio) && eiters >= lowest_vf / ratio)) next_size += 1; } else while (next_size < vector_sizes.length () - && maybe_lt (current_vector_size, vector_sizes[next_size])) + && maybe_lt (loop_vinfo->vector_size, vector_sizes[next_size])) next_size += 1; if (next_size == vector_sizes.length ()) Index: gcc/tree-vect-slp.c =================================================================== --- gcc/tree-vect-slp.c 2019-10-20 14:14:16.688672033 +0100 +++ gcc/tree-vect-slp.c 2019-10-20 14:14:36.764528643 +0100 @@ -233,7 +233,7 @@ vect_get_place_in_interleaving_chain (st (if nonnull). */ bool -can_duplicate_and_interleave_p (vec_info *, unsigned int count, +can_duplicate_and_interleave_p (vec_info *vinfo, unsigned int count, machine_mode elt_mode, unsigned int *nvectors_out, tree *vector_type_out, @@ -246,7 +246,7 @@ can_duplicate_and_interleave_p (vec_info { scalar_int_mode int_mode; poly_int64 elt_bits = elt_bytes * BITS_PER_UNIT; - if (multiple_p (current_vector_size, elt_bytes, &nelts) + if (multiple_p (vinfo->vector_size, elt_bytes, &nelts) && int_mode_for_size (elt_bits, 0).exists (&int_mode)) { tree int_type = build_nonstandard_integer_type @@ -431,7 +431,7 @@ vect_get_and_check_slp_defs (vec_info *v } if ((dt == vect_constant_def || dt == vect_external_def) - && !current_vector_size.is_constant () + && !vinfo->vector_size.is_constant () && (TREE_CODE (type) == BOOLEAN_TYPE || !can_duplicate_and_interleave_p (vinfo, stmts.length (), TYPE_MODE (type)))) @@ -2250,7 +2250,7 @@ vect_make_slp_decision (loop_vec_info lo FOR_EACH_VEC_ELT (slp_instances, i, instance) { /* FORNOW: SLP if you can. */ - /* All unroll factors have the form current_vector_size * X for some + /* All unroll factors have the form vinfo->vector_size * X for some rational X, so they must have a common multiple. */ unrolling_factor = force_common_multiple (unrolling_factor, @@ -2986,7 +2986,7 @@ vect_slp_bb_region (gimple_stmt_iterator auto_vector_sizes vector_sizes; /* Autodetect first vector size we try. */ - current_vector_size = 0; + poly_uint64 next_vector_size = 0; targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); unsigned int next_size = 0; @@ -3005,6 +3005,7 @@ vect_slp_bb_region (gimple_stmt_iterator bb_vinfo->shared->save_datarefs (); else bb_vinfo->shared->check_datarefs (); + bb_vinfo->vector_size = next_vector_size; if (vect_slp_analyze_bb_1 (bb_vinfo, n_stmts, fatal) && dbg_cnt (vect_slp)) @@ -3018,7 +3019,7 @@ vect_slp_bb_region (gimple_stmt_iterator unsigned HOST_WIDE_INT bytes; if (dump_enabled_p ()) { - if (current_vector_size.is_constant (&bytes)) + if (bb_vinfo->vector_size.is_constant (&bytes)) dump_printf_loc (MSG_OPTIMIZED_LOCATIONS, vect_location, "basic block part vectorized using %wu byte " "vectors\n", bytes); @@ -3030,10 +3031,11 @@ vect_slp_bb_region (gimple_stmt_iterator vectorized = true; } - delete bb_vinfo; if (next_size == 0) - autodetected_vector_size = current_vector_size; + autodetected_vector_size = bb_vinfo->vector_size; + + delete bb_vinfo; if (next_size < vector_sizes.length () && known_eq (vector_sizes[next_size], autodetected_vector_size)) @@ -3041,20 +3043,20 @@ vect_slp_bb_region (gimple_stmt_iterator if (vectorized || next_size == vector_sizes.length () - || known_eq (current_vector_size, 0U) + || known_eq (bb_vinfo->vector_size, 0U) /* If vect_slp_analyze_bb_1 signaled that analysis for all vector sizes will fail do not bother iterating. */ || fatal) return vectorized; /* Try the next biggest vector size. */ - current_vector_size = vector_sizes[next_size++]; + next_vector_size = vector_sizes[next_size++]; if (dump_enabled_p ()) { dump_printf_loc (MSG_NOTE, vect_location, "***** Re-trying analysis with " "vector size "); - dump_dec (MSG_NOTE, current_vector_size); + dump_dec (MSG_NOTE, next_vector_size); dump_printf (MSG_NOTE, "\n"); } }