| Message ID | mptblu59f21.fsf@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | [8/n] Replace autovectorize_vector_sizes with autovectorize_vector_modes | expand |
On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford <richard.sandiford@arm.com> wrote: > > This is another patch in the series to remove the assumption that > all modes involved in vectorisation have to be the same size. > Rather than have the target provide a list of vector sizes, > it makes the target provide a list of vector "approaches", > with each approach represented by a mode. > > A later patch will pass this mode to targetm.vectorize.related_mode > to get the vector mode for a given element mode. Until then, the modes > simply act as an alternative way of specifying the vector size. Is there a restriction to use integer vector modes for the hook or would FP vector modes be OK as well? Note that your x86 change likely disables word_mode vectorization with -mno-sse? That is, how do we represent GPR vectorization "size" here? The preferred SIMD mode hook may return an integer mode, are non-vector modes OK for autovectorize_vector_modes? Thanks, Richard. > > 2019-10-24 Richard Sandiford <richard.sandiford@arm.com> > > gcc/ > * target.h (vector_sizes, auto_vector_sizes): Delete. > (vector_modes, auto_vector_modes): New typedefs. > * target.def (autovectorize_vector_sizes): Replace with... > (autovectorize_vector_modes): ...this new hook. > * doc/tm.texi.in (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): > Replace with... > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): ...this new hook. > * doc/tm.texi: Regenerate. > * targhooks.h (default_autovectorize_vector_sizes): Delete. > (default_autovectorize_vector_modes): New function. > * targhooks.c (default_autovectorize_vector_sizes): Delete. > (default_autovectorize_vector_modes): New function. > * omp-general.c (omp_max_vf): Use autovectorize_vector_modes instead > of autovectorize_vector_sizes. Use the number of units in the mode > to calculate the maximum VF. > * omp-low.c (omp_clause_aligned_alignment): Use > autovectorize_vector_modes instead of autovectorize_vector_sizes. > Use a loop based on related_mode to iterate through all supported > vector modes for a given scalar mode. > * optabs-query.c (can_vec_mask_load_store_p): Use > autovectorize_vector_modes instead of autovectorize_vector_sizes. > * tree-vect-loop.c (vect_analyze_loop, vect_transform_loop): Likewise. > * tree-vect-slp.c (vect_slp_bb_region): Likewise. > * config/aarch64/aarch64.c (aarch64_autovectorize_vector_sizes): > Replace with... > (aarch64_autovectorize_vector_modes): ...this new function. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. > * config/arc/arc.c (arc_autovectorize_vector_sizes): Replace with... > (arc_autovectorize_vector_modes): ...this new function. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. > * config/arm/arm.c (arm_autovectorize_vector_sizes): Replace with... > (arm_autovectorize_vector_modes): ...this new function. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. > * config/i386/i386.c (ix86_autovectorize_vector_sizes): Replace with... > (ix86_autovectorize_vector_modes): ...this new function. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. > * config/mips/mips.c (mips_autovectorize_vector_sizes): Replace with... > (mips_autovectorize_vector_modes): ...this new function. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. > (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. > > Index: gcc/target.h > =================================================================== > --- gcc/target.h 2019-09-30 17:19:39.843166118 +0100 > +++ gcc/target.h 2019-10-25 13:27:15.525762975 +0100 > @@ -205,11 +205,11 @@ enum vect_cost_model_location { > class vec_perm_indices; > > /* The type to use for lists of vector sizes. */ > -typedef vec<poly_uint64> vector_sizes; > +typedef vec<machine_mode> vector_modes; > > /* Same, but can be used to construct local lists that are > automatically freed. */ > -typedef auto_vec<poly_uint64, 8> auto_vector_sizes; > +typedef auto_vec<machine_mode, 8> auto_vector_modes; > > /* The target structure. This holds all the backend hooks. */ > #define DEFHOOKPOD(NAME, DOC, TYPE, INIT) TYPE NAME; > Index: gcc/target.def > =================================================================== > --- gcc/target.def 2019-10-25 13:26:59.309877555 +0100 > +++ gcc/target.def 2019-10-25 13:27:15.525762975 +0100 > @@ -1894,20 +1894,28 @@ reached. The default is @var{mode} whic > /* Returns a mask of vector sizes to iterate over when auto-vectorizing > after processing the preferred one derived from preferred_simd_mode. */ > DEFHOOK > -(autovectorize_vector_sizes, > - "If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not\n\ > -the only one that is worth considering, this hook should add all suitable\n\ > -vector sizes to @var{sizes}, in order of decreasing preference. The first\n\ > -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}.\n\ > -If @var{all} is true, add suitable vector sizes even when they are generally\n\ > +(autovectorize_vector_modes, > + "If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}\n\ > +is not the only approach worth considering, this hook should add one mode to\n\ > +@var{modes} for each useful alternative approach. These modes are then\n\ > +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode\n\ > +for a given element mode.\n\ > +\n\ > +The modes returned in @var{modes} should use the smallest element mode\n\ > +possible for the vectorization approach that they represent, preferring\n\ > +integer modes over floating-poing modes in the event of a tie. The first\n\ > +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its\n\ > +element mode.\n\ > +\n\ > +If @var{all} is true, add suitable vector modes even when they are generally\n\ > not expected to be worthwhile.\n\ > \n\ > The hook does not need to do anything if the vector returned by\n\ > @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is the only one relevant\n\ > for autovectorization. The default implementation does nothing.", > void, > - (vector_sizes *sizes, bool all), > - default_autovectorize_vector_sizes) > + (vector_modes *modes, bool all), > + default_autovectorize_vector_modes) > > DEFHOOK > (related_mode, > Index: gcc/doc/tm.texi.in > =================================================================== > --- gcc/doc/tm.texi.in 2019-10-25 13:26:59.009879675 +0100 > +++ gcc/doc/tm.texi.in 2019-10-25 13:27:15.521763003 +0100 > @@ -4179,7 +4179,7 @@ address; but often a machine-dependent > > @hook TARGET_VECTORIZE_SPLIT_REDUCTION > > -@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > +@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > > @hook TARGET_VECTORIZE_RELATED_MODE > > Index: gcc/doc/tm.texi > =================================================================== > --- gcc/doc/tm.texi 2019-10-25 13:26:59.305877583 +0100 > +++ gcc/doc/tm.texi 2019-10-25 13:27:15.521763003 +0100 > @@ -6016,12 +6016,20 @@ against lower halves of vectors recursiv > reached. The default is @var{mode} which means no splitting. > @end deftypefn > > -@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES (vector_sizes *@var{sizes}, bool @var{all}) > -If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not > -the only one that is worth considering, this hook should add all suitable > -vector sizes to @var{sizes}, in order of decreasing preference. The first > -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}. > -If @var{all} is true, add suitable vector sizes even when they are generally > +@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES (vector_modes *@var{modes}, bool @var{all}) > +If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} > +is not the only approach worth considering, this hook should add one mode to > +@var{modes} for each useful alternative approach. These modes are then > +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode > +for a given element mode. > + > +The modes returned in @var{modes} should use the smallest element mode > +possible for the vectorization approach that they represent, preferring > +integer modes over floating-poing modes in the event of a tie. The first > +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its > +element mode. > + > +If @var{all} is true, add suitable vector modes even when they are generally > not expected to be worthwhile. > > The hook does not need to do anything if the vector returned by > Index: gcc/targhooks.h > =================================================================== > --- gcc/targhooks.h 2019-10-25 13:26:59.309877555 +0100 > +++ gcc/targhooks.h 2019-10-25 13:27:15.525762975 +0100 > @@ -113,7 +113,7 @@ default_builtin_support_vector_misalignm > int, bool); > extern machine_mode default_preferred_simd_mode (scalar_mode mode); > extern machine_mode default_split_reduction (machine_mode); > -extern void default_autovectorize_vector_sizes (vector_sizes *, bool); > +extern void default_autovectorize_vector_modes (vector_modes *, bool); > extern opt_machine_mode default_vectorize_related_mode (machine_mode, > scalar_mode, > poly_uint64); > Index: gcc/targhooks.c > =================================================================== > --- gcc/targhooks.c 2019-10-25 13:26:59.309877555 +0100 > +++ gcc/targhooks.c 2019-10-25 13:27:15.525762975 +0100 > @@ -1299,11 +1299,10 @@ default_split_reduction (machine_mode mo > return mode; > } > > -/* By default only the size derived from the preferred vector mode > - is tried. */ > +/* By default only the preferred vector mode is tried. */ > > void > -default_autovectorize_vector_sizes (vector_sizes *, bool) > +default_autovectorize_vector_modes (vector_modes *, bool) > { > } > > Index: gcc/omp-general.c > =================================================================== > --- gcc/omp-general.c 2019-10-25 09:21:28.798326303 +0100 > +++ gcc/omp-general.c 2019-10-25 13:27:15.521763003 +0100 > @@ -508,13 +508,16 @@ omp_max_vf (void) > && global_options_set.x_flag_tree_loop_vectorize)) > return 1; > > - auto_vector_sizes sizes; > - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); > - if (!sizes.is_empty ()) > + auto_vector_modes modes; > + targetm.vectorize.autovectorize_vector_modes (&modes, true); > + if (!modes.is_empty ()) > { > poly_uint64 vf = 0; > - for (unsigned int i = 0; i < sizes.length (); ++i) > - vf = ordered_max (vf, sizes[i]); > + for (unsigned int i = 0; i < modes.length (); ++i) > + /* The returned modes use the smallest element size (and thus > + the largest nunits) for the vectorization approach that they > + represent. */ > + vf = ordered_max (vf, GET_MODE_NUNITS (modes[i])); > return vf; > } > > Index: gcc/omp-low.c > =================================================================== > --- gcc/omp-low.c 2019-10-11 15:43:51.283513446 +0100 > +++ gcc/omp-low.c 2019-10-25 13:27:15.525762975 +0100 > @@ -3947,11 +3947,8 @@ omp_clause_aligned_alignment (tree claus > /* Otherwise return implementation defined alignment. */ > unsigned int al = 1; > opt_scalar_mode mode_iter; > - auto_vector_sizes sizes; > - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); > - poly_uint64 vs = 0; > - for (unsigned int i = 0; i < sizes.length (); ++i) > - vs = ordered_max (vs, sizes[i]); > + auto_vector_modes modes; > + targetm.vectorize.autovectorize_vector_modes (&modes, true); > static enum mode_class classes[] > = { MODE_INT, MODE_VECTOR_INT, MODE_FLOAT, MODE_VECTOR_FLOAT }; > for (int i = 0; i < 4; i += 2) > @@ -3962,19 +3959,18 @@ omp_clause_aligned_alignment (tree claus > machine_mode vmode = targetm.vectorize.preferred_simd_mode (mode); > if (GET_MODE_CLASS (vmode) != classes[i + 1]) > continue; > - while (maybe_ne (vs, 0U) > - && known_lt (GET_MODE_SIZE (vmode), vs) > - && GET_MODE_2XWIDER_MODE (vmode).exists ()) > - vmode = GET_MODE_2XWIDER_MODE (vmode).require (); > + machine_mode alt_vmode; > + for (unsigned int j = 0; j < modes.length (); ++j) > + if (related_vector_mode (modes[j], mode).exists (&alt_vmode) > + && known_ge (GET_MODE_SIZE (alt_vmode), GET_MODE_SIZE (vmode))) > + vmode = alt_vmode; > > tree type = lang_hooks.types.type_for_mode (mode, 1); > if (type == NULL_TREE || TYPE_MODE (type) != mode) > continue; > - poly_uint64 nelts = exact_div (GET_MODE_SIZE (vmode), > - GET_MODE_SIZE (mode)); > - type = build_vector_type (type, nelts); > - if (TYPE_MODE (type) != vmode) > - continue; > + type = build_vector_type_for_mode (type, vmode); > + /* The functions above are not allowed to return invalid modes. */ > + gcc_assert (TYPE_MODE (type) == vmode); > if (TYPE_ALIGN_UNIT (type) > al) > al = TYPE_ALIGN_UNIT (type); > } > Index: gcc/optabs-query.c > =================================================================== > --- gcc/optabs-query.c 2019-10-25 13:26:59.305877583 +0100 > +++ gcc/optabs-query.c 2019-10-25 13:27:15.525762975 +0100 > @@ -589,11 +589,11 @@ can_vec_mask_load_store_p (machine_mode > && convert_optab_handler (op, vmode, mask_mode) != CODE_FOR_nothing) > return true; > > - auto_vector_sizes vector_sizes; > - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, true); > - for (unsigned int i = 0; i < vector_sizes.length (); ++i) > + auto_vector_modes vector_modes; > + targetm.vectorize.autovectorize_vector_modes (&vector_modes, true); > + for (unsigned int i = 0; i < vector_modes.length (); ++i) > { > - poly_uint64 cur = vector_sizes[i]; > + poly_uint64 cur = GET_MODE_SIZE (vector_modes[i]); > poly_uint64 nunits; > if (!multiple_p (cur, GET_MODE_SIZE (smode), &nunits)) > continue; > Index: gcc/tree-vect-loop.c > =================================================================== > --- gcc/tree-vect-loop.c 2019-10-25 13:26:59.137878771 +0100 > +++ gcc/tree-vect-loop.c 2019-10-25 13:27:15.525762975 +0100 > @@ -2319,12 +2319,12 @@ vect_analyze_loop_2 (loop_vec_info loop_ > vect_analyze_loop (class loop *loop, loop_vec_info orig_loop_vinfo, > vec_info_shared *shared) > { > - auto_vector_sizes vector_sizes; > + auto_vector_modes vector_modes; > > /* Autodetect first vector size we try. */ > - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, > + targetm.vectorize.autovectorize_vector_modes (&vector_modes, > loop->simdlen != 0); > - unsigned int next_size = 0; > + unsigned int mode_i = 0; > > DUMP_VECT_SCOPE ("analyze_loop_nest"); > > @@ -2343,7 +2343,7 @@ vect_analyze_loop (class loop *loop, loo > unsigned n_stmts = 0; > poly_uint64 autodetected_vector_size = 0; > opt_loop_vec_info first_loop_vinfo = opt_loop_vec_info::success (NULL); > - poly_uint64 next_vector_size = 0; > + machine_mode next_vector_mode = VOIDmode; > while (1) > { > /* Check the CFG characteristics of the loop (nesting, entry/exit). */ > @@ -2357,7 +2357,7 @@ vect_analyze_loop (class loop *loop, loo > gcc_checking_assert (first_loop_vinfo == NULL); > return loop_vinfo; > } > - loop_vinfo->vector_size = next_vector_size; > + loop_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); > > bool fatal = false; > > @@ -2365,7 +2365,7 @@ vect_analyze_loop (class loop *loop, loo > LOOP_VINFO_ORIG_LOOP_INFO (loop_vinfo) = orig_loop_vinfo; > > opt_result res = vect_analyze_loop_2 (loop_vinfo, fatal, &n_stmts); > - if (next_size == 0) > + if (mode_i == 0) > autodetected_vector_size = loop_vinfo->vector_size; > > if (res) > @@ -2399,11 +2399,12 @@ vect_analyze_loop (class loop *loop, loo > return opt_loop_vec_info::propagate_failure (res); > } > > - if (next_size < vector_sizes.length () > - && known_eq (vector_sizes[next_size], autodetected_vector_size)) > - next_size += 1; > + if (mode_i < vector_modes.length () > + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), > + autodetected_vector_size)) > + mode_i += 1; > > - if (next_size == vector_sizes.length () > + if (mode_i == vector_modes.length () > || known_eq (autodetected_vector_size, 0U)) > { > if (first_loop_vinfo) > @@ -2423,15 +2424,11 @@ vect_analyze_loop (class loop *loop, loo > } > > /* Try the next biggest vector size. */ > - next_vector_size = vector_sizes[next_size++]; > + next_vector_mode = vector_modes[mode_i++]; > if (dump_enabled_p ()) > - { > - dump_printf_loc (MSG_NOTE, vect_location, > - "***** Re-trying analysis with " > - "vector size "); > - dump_dec (MSG_NOTE, next_vector_size); > - dump_printf (MSG_NOTE, "\n"); > - } > + dump_printf_loc (MSG_NOTE, vect_location, > + "***** Re-trying analysis with vector mode %s\n", > + GET_MODE_NAME (next_vector_mode)); > } > } > > @@ -8277,9 +8274,9 @@ vect_transform_loop (loop_vec_info loop_ > > if (epilogue) > { > - auto_vector_sizes vector_sizes; > - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); > - unsigned int next_size = 0; > + auto_vector_modes vector_modes; > + targetm.vectorize.autovectorize_vector_modes (&vector_modes, false); > + unsigned int next_i = 0; > > /* Note LOOP_VINFO_NITERS_KNOWN_P and LOOP_VINFO_INT_NITERS work > on niters already ajusted for the iterations of the prologue. */ > @@ -8295,18 +8292,20 @@ vect_transform_loop (loop_vec_info loop_ > epilogue->any_upper_bound = true; > > unsigned int ratio; > - while (next_size < vector_sizes.length () > - && !(constant_multiple_p (loop_vinfo->vector_size, > - vector_sizes[next_size], &ratio) > + while (next_i < vector_modes.length () > + && !(constant_multiple_p > + (loop_vinfo->vector_size, > + GET_MODE_SIZE (vector_modes[next_i]), &ratio) > && eiters >= lowest_vf / ratio)) > - next_size += 1; > + next_i += 1; > } > else > - while (next_size < vector_sizes.length () > - && maybe_lt (loop_vinfo->vector_size, vector_sizes[next_size])) > - next_size += 1; > + while (next_i < vector_modes.length () > + && maybe_lt (loop_vinfo->vector_size, > + GET_MODE_SIZE (vector_modes[next_i]))) > + next_i += 1; > > - if (next_size == vector_sizes.length ()) > + if (next_i == vector_modes.length ()) > epilogue = NULL; > } > > Index: gcc/tree-vect-slp.c > =================================================================== > --- gcc/tree-vect-slp.c 2019-10-25 13:26:59.141878743 +0100 > +++ gcc/tree-vect-slp.c 2019-10-25 13:27:15.525762975 +0100 > @@ -3087,12 +3087,12 @@ vect_slp_bb_region (gimple_stmt_iterator > unsigned int n_stmts) > { > bb_vec_info bb_vinfo; > - auto_vector_sizes vector_sizes; > + auto_vector_modes vector_modes; > > /* Autodetect first vector size we try. */ > - poly_uint64 next_vector_size = 0; > - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); > - unsigned int next_size = 0; > + machine_mode next_vector_mode = VOIDmode; > + targetm.vectorize.autovectorize_vector_modes (&vector_modes, false); > + unsigned int mode_i = 0; > > vec_info_shared shared; > > @@ -3109,7 +3109,7 @@ vect_slp_bb_region (gimple_stmt_iterator > bb_vinfo->shared->save_datarefs (); > else > bb_vinfo->shared->check_datarefs (); > - bb_vinfo->vector_size = next_vector_size; > + bb_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); > > if (vect_slp_analyze_bb_1 (bb_vinfo, n_stmts, fatal) > && dbg_cnt (vect_slp)) > @@ -3136,17 +3136,18 @@ vect_slp_bb_region (gimple_stmt_iterator > vectorized = true; > } > > - if (next_size == 0) > + if (mode_i == 0) > autodetected_vector_size = bb_vinfo->vector_size; > > delete bb_vinfo; > > - if (next_size < vector_sizes.length () > - && known_eq (vector_sizes[next_size], autodetected_vector_size)) > - next_size += 1; > + if (mode_i < vector_modes.length () > + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), > + autodetected_vector_size)) > + mode_i += 1; > > if (vectorized > - || next_size == vector_sizes.length () > + || mode_i == vector_modes.length () > || known_eq (autodetected_vector_size, 0U) > /* If vect_slp_analyze_bb_1 signaled that analysis for all > vector sizes will fail do not bother iterating. */ > @@ -3154,15 +3155,11 @@ vect_slp_bb_region (gimple_stmt_iterator > return vectorized; > > /* Try the next biggest vector size. */ > - next_vector_size = vector_sizes[next_size++]; > + next_vector_mode = vector_modes[mode_i++]; > if (dump_enabled_p ()) > - { > - dump_printf_loc (MSG_NOTE, vect_location, > - "***** Re-trying analysis with " > - "vector size "); > - dump_dec (MSG_NOTE, next_vector_size); > - dump_printf (MSG_NOTE, "\n"); > - } > + dump_printf_loc (MSG_NOTE, vect_location, > + "***** Re-trying analysis with vector mode %s\n", > + GET_MODE_NAME (next_vector_mode)); > } > } > > Index: gcc/config/aarch64/aarch64.c > =================================================================== > --- gcc/config/aarch64/aarch64.c 2019-10-25 13:26:59.177878488 +0100 > +++ gcc/config/aarch64/aarch64.c 2019-10-25 13:27:15.505763118 +0100 > @@ -15203,12 +15203,12 @@ aarch64_preferred_simd_mode (scalar_mode > /* Return a list of possible vector sizes for the vectorizer > to iterate over. */ > static void > -aarch64_autovectorize_vector_sizes (vector_sizes *sizes, bool) > +aarch64_autovectorize_vector_modes (vector_modes *modes, bool) > { > if (TARGET_SVE) > - sizes->safe_push (BYTES_PER_SVE_VECTOR); > - sizes->safe_push (16); > - sizes->safe_push (8); > + modes->safe_push (VNx16QImode); > + modes->safe_push (V16QImode); > + modes->safe_push (V8QImode); > } > > /* Implement TARGET_MANGLE_TYPE. */ > @@ -20915,9 +20915,9 @@ #define TARGET_VECTORIZE_BUILTINS > #define TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION \ > aarch64_builtin_vectorized_function > > -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ > - aarch64_autovectorize_vector_sizes > +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ > + aarch64_autovectorize_vector_modes > > #undef TARGET_ATOMIC_ASSIGN_EXPAND_FENV > #define TARGET_ATOMIC_ASSIGN_EXPAND_FENV \ > Index: gcc/config/arc/arc.c > =================================================================== > --- gcc/config/arc/arc.c 2019-10-25 09:21:25.974346475 +0100 > +++ gcc/config/arc/arc.c 2019-10-25 13:27:15.505763118 +0100 > @@ -607,15 +607,15 @@ arc_preferred_simd_mode (scalar_mode mod > } > > /* Implements target hook > - TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ > + TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ > > static void > -arc_autovectorize_vector_sizes (vector_sizes *sizes, bool) > +arc_autovectorize_vector_modes (vector_modes *modes, bool) > { > if (TARGET_PLUS_QMACW) > { > - sizes->quick_push (8); > - sizes->quick_push (4); > + modes->quick_push (V4HImode); > + modes->quick_push (V2HImode); > } > } > > @@ -726,8 +726,8 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a > #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE > #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arc_preferred_simd_mode > > -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES arc_autovectorize_vector_sizes > +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES arc_autovectorize_vector_modes > > #undef TARGET_CAN_USE_DOLOOP_P > #define TARGET_CAN_USE_DOLOOP_P arc_can_use_doloop_p > Index: gcc/config/arm/arm.c > =================================================================== > --- gcc/config/arm/arm.c 2019-10-23 11:29:47.933883742 +0100 > +++ gcc/config/arm/arm.c 2019-10-25 13:27:15.513763059 +0100 > @@ -289,7 +289,7 @@ static bool arm_builtin_support_vector_m > static void arm_conditional_register_usage (void); > static enum flt_eval_method arm_excess_precision (enum excess_precision_type); > static reg_class_t arm_preferred_rename_class (reg_class_t rclass); > -static void arm_autovectorize_vector_sizes (vector_sizes *, bool); > +static void arm_autovectorize_vector_modes (vector_modes *, bool); > static int arm_default_branch_cost (bool, bool); > static int arm_cortex_a5_branch_cost (bool, bool); > static int arm_cortex_m_branch_cost (bool, bool); > @@ -522,9 +522,9 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a > #define TARGET_ARRAY_MODE_SUPPORTED_P arm_array_mode_supported_p > #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE > #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arm_preferred_simd_mode > -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ > - arm_autovectorize_vector_sizes > +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ > + arm_autovectorize_vector_modes > > #undef TARGET_MACHINE_DEPENDENT_REORG > #define TARGET_MACHINE_DEPENDENT_REORG arm_reorg > @@ -29012,12 +29012,12 @@ arm_vector_alignment (const_tree type) > } > > static void > -arm_autovectorize_vector_sizes (vector_sizes *sizes, bool) > +arm_autovectorize_vector_modes (vector_modes *modes, bool) > { > if (!TARGET_NEON_VECTORIZE_DOUBLE) > { > - sizes->safe_push (16); > - sizes->safe_push (8); > + modes->safe_push (V16QImode); > + modes->safe_push (V8QImode); > } > } > > Index: gcc/config/i386/i386.c > =================================================================== > --- gcc/config/i386/i386.c 2019-10-25 13:26:59.277877782 +0100 > +++ gcc/config/i386/i386.c 2019-10-25 13:27:15.517763031 +0100 > @@ -21387,35 +21387,35 @@ ix86_preferred_simd_mode (scalar_mode mo > 256bit and 128bit vectors. */ > > static void > -ix86_autovectorize_vector_sizes (vector_sizes *sizes, bool all) > +ix86_autovectorize_vector_modes (vector_modes *modes, bool all) > { > if (TARGET_AVX512F && !TARGET_PREFER_AVX256) > { > - sizes->safe_push (64); > - sizes->safe_push (32); > - sizes->safe_push (16); > + modes->safe_push (V64QImode); > + modes->safe_push (V32QImode); > + modes->safe_push (V16QImode); > } > else if (TARGET_AVX512F && all) > { > - sizes->safe_push (32); > - sizes->safe_push (16); > - sizes->safe_push (64); > + modes->safe_push (V32QImode); > + modes->safe_push (V16QImode); > + modes->safe_push (V64QImode); > } > else if (TARGET_AVX && !TARGET_PREFER_AVX128) > { > - sizes->safe_push (32); > - sizes->safe_push (16); > + modes->safe_push (V32QImode); > + modes->safe_push (V16QImode); > } > else if (TARGET_AVX && all) > { > - sizes->safe_push (16); > - sizes->safe_push (32); > + modes->safe_push (V16QImode); > + modes->safe_push (V32QImode); > } > else if (TARGET_MMX_WITH_SSE) > - sizes->safe_push (16); > + modes->safe_push (V16QImode); > > if (TARGET_MMX_WITH_SSE) > - sizes->safe_push (8); > + modes->safe_push (V8QImode); > } > > /* Implemenation of targetm.vectorize.get_mask_mode. */ > @@ -22954,9 +22954,9 @@ #define TARGET_VECTORIZE_PREFERRED_SIMD_ > #undef TARGET_VECTORIZE_SPLIT_REDUCTION > #define TARGET_VECTORIZE_SPLIT_REDUCTION \ > ix86_split_reduction > -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ > - ix86_autovectorize_vector_sizes > +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ > + ix86_autovectorize_vector_modes > #undef TARGET_VECTORIZE_GET_MASK_MODE > #define TARGET_VECTORIZE_GET_MASK_MODE ix86_get_mask_mode > #undef TARGET_VECTORIZE_INIT_COST > Index: gcc/config/mips/mips.c > =================================================================== > --- gcc/config/mips/mips.c 2019-10-17 14:22:54.903313423 +0100 > +++ gcc/config/mips/mips.c 2019-10-25 13:27:15.517763031 +0100 > @@ -13453,13 +13453,13 @@ mips_preferred_simd_mode (scalar_mode mo > return word_mode; > } > > -/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ > +/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ > > static void > -mips_autovectorize_vector_sizes (vector_sizes *sizes, bool) > +mips_autovectorize_vector_modes (vector_modes *modes, bool) > { > if (ISA_HAS_MSA) > - sizes->safe_push (16); > + modes->safe_push (V16QImode); > } > > /* Implement TARGET_INIT_LIBFUNCS. */ > @@ -22694,9 +22694,9 @@ #define TARGET_SCALAR_MODE_SUPPORTED_P m > > #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE > #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE mips_preferred_simd_mode > -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES > -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ > - mips_autovectorize_vector_sizes > +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES > +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ > + mips_autovectorize_vector_modes > > #undef TARGET_INIT_BUILTINS > #define TARGET_INIT_BUILTINS mips_init_builtins
Richard Biener <richard.guenther@gmail.com> writes: > On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford > <richard.sandiford@arm.com> wrote: >> >> This is another patch in the series to remove the assumption that >> all modes involved in vectorisation have to be the same size. >> Rather than have the target provide a list of vector sizes, >> it makes the target provide a list of vector "approaches", >> with each approach represented by a mode. >> >> A later patch will pass this mode to targetm.vectorize.related_mode >> to get the vector mode for a given element mode. Until then, the modes >> simply act as an alternative way of specifying the vector size. > > Is there a restriction to use integer vector modes for the hook > or would FP vector modes be OK as well? Conceptually, each mode returned by the hook represents a set of vector modes, with the set containing one member for each supported element type. The idea is to represent the set using the member with the smallest element type, preferring integer modes over floating-point modes in the event of a tie. So using a floating-point mode as the representative mode is fine if floating-point elements are the smallest (or only) supported element type. > Note that your x86 change likely disables word_mode vectorization with > -mno-sse? No, that still works, because... > That is, how do we represent GPR vectorization "size" here? > The preferred SIMD mode hook may return an integer mode, > are non-vector modes OK for autovectorize_vector_modes? ...at least with all current targets, preferred_simd_mode is only an integer mode if the target has no "real" vectorisation support for that element type. There's no need to handle that case in autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing when SSE is disabled. So while preferred_simd_mode can continue to return integer modes, autovectorize_vector_modes always returns vector modes. This patch just treats the mode as an alternative way of specifying the vector size. 11/n then tries to use related_vector_mode to choose the vector mode for each element type instead. But 11/n only uses related_vector_mode if vec_info::vector_mode is a vector mode. If it's an integer mode (as for -mno-sse), or if related_vector_mode fails to find a vector mode, then we still fall back to mode_for_vector and so pick an integer mode in the same cases as before. Thanks, Richard
Ping Richard Sandiford <richard.sandiford@arm.com> writes: > Richard Biener <richard.guenther@gmail.com> writes: >> On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford >> <richard.sandiford@arm.com> wrote: >>> >>> This is another patch in the series to remove the assumption that >>> all modes involved in vectorisation have to be the same size. >>> Rather than have the target provide a list of vector sizes, >>> it makes the target provide a list of vector "approaches", >>> with each approach represented by a mode. >>> >>> A later patch will pass this mode to targetm.vectorize.related_mode >>> to get the vector mode for a given element mode. Until then, the modes >>> simply act as an alternative way of specifying the vector size. >> >> Is there a restriction to use integer vector modes for the hook >> or would FP vector modes be OK as well? > > Conceptually, each mode returned by the hook represents a set of vector > modes, with the set containing one member for each supported element > type. The idea is to represent the set using the member with the > smallest element type, preferring integer modes over floating-point > modes in the event of a tie. So using a floating-point mode as the > representative mode is fine if floating-point elements are the smallest > (or only) supported element type. > >> Note that your x86 change likely disables word_mode vectorization with >> -mno-sse? > > No, that still works, because... > >> That is, how do we represent GPR vectorization "size" here? >> The preferred SIMD mode hook may return an integer mode, >> are non-vector modes OK for autovectorize_vector_modes? > > ...at least with all current targets, preferred_simd_mode is only > an integer mode if the target has no "real" vectorisation support > for that element type. There's no need to handle that case in > autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing > when SSE is disabled. > > So while preferred_simd_mode can continue to return integer modes, > autovectorize_vector_modes always returns vector modes. > > This patch just treats the mode as an alternative way of specifying > the vector size. 11/n then tries to use related_vector_mode to choose > the vector mode for each element type instead. But 11/n only uses > related_vector_mode if vec_info::vector_mode is a vector mode. If it's > an integer mode (as for -mno-sse), or if related_vector_mode fails to > find a vector mode, then we still fall back to mode_for_vector and so > pick an integer mode in the same cases as before. > > Thanks, > Richard 2019-10-24 Richard Sandiford <richard.sandiford@arm.com> gcc/ * target.h (vector_sizes, auto_vector_sizes): Delete. (vector_modes, auto_vector_modes): New typedefs. * target.def (autovectorize_vector_sizes): Replace with... (autovectorize_vector_modes): ...this new hook. * doc/tm.texi.in (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Replace with... (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): ...this new hook. * doc/tm.texi: Regenerate. * targhooks.h (default_autovectorize_vector_sizes): Delete. (default_autovectorize_vector_modes): New function. * targhooks.c (default_autovectorize_vector_sizes): Delete. (default_autovectorize_vector_modes): New function. * omp-general.c (omp_max_vf): Use autovectorize_vector_modes instead of autovectorize_vector_sizes. Use the number of units in the mode to calculate the maximum VF. * omp-low.c (omp_clause_aligned_alignment): Use autovectorize_vector_modes instead of autovectorize_vector_sizes. Use a loop based on related_mode to iterate through all supported vector modes for a given scalar mode. * optabs-query.c (can_vec_mask_load_store_p): Use autovectorize_vector_modes instead of autovectorize_vector_sizes. * tree-vect-loop.c (vect_analyze_loop, vect_transform_loop): Likewise. * tree-vect-slp.c (vect_slp_bb_region): Likewise. * config/aarch64/aarch64.c (aarch64_autovectorize_vector_sizes): Replace with... (aarch64_autovectorize_vector_modes): ...this new function. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. * config/arc/arc.c (arc_autovectorize_vector_sizes): Replace with... (arc_autovectorize_vector_modes): ...this new function. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. * config/arm/arm.c (arm_autovectorize_vector_sizes): Replace with... (arm_autovectorize_vector_modes): ...this new function. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. * config/i386/i386.c (ix86_autovectorize_vector_sizes): Replace with... (ix86_autovectorize_vector_modes): ...this new function. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. * config/mips/mips.c (mips_autovectorize_vector_sizes): Replace with... (mips_autovectorize_vector_modes): ...this new function. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES): Delete. (TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES): Define. Index: gcc/target.h =================================================================== --- gcc/target.h 2019-11-05 10:31:39.017103686 +0000 +++ gcc/target.h 2019-11-05 10:34:34.443861724 +0000 @@ -205,11 +205,11 @@ enum vect_cost_model_location { class vec_perm_indices; /* The type to use for lists of vector sizes. */ -typedef vec<poly_uint64> vector_sizes; +typedef vec<machine_mode> vector_modes; /* Same, but can be used to construct local lists that are automatically freed. */ -typedef auto_vec<poly_uint64, 8> auto_vector_sizes; +typedef auto_vec<machine_mode, 8> auto_vector_modes; /* First argument of targetm.omp.device_kind_arch_isa. */ enum omp_device_kind_arch_isa { Index: gcc/target.def =================================================================== --- gcc/target.def 2019-11-05 10:31:39.017103686 +0000 +++ gcc/target.def 2019-11-05 10:34:34.443861724 +0000 @@ -1909,20 +1909,28 @@ reached. The default is @var{mode} whic /* Returns a mask of vector sizes to iterate over when auto-vectorizing after processing the preferred one derived from preferred_simd_mode. */ DEFHOOK -(autovectorize_vector_sizes, - "If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not\n\ -the only one that is worth considering, this hook should add all suitable\n\ -vector sizes to @var{sizes}, in order of decreasing preference. The first\n\ -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}.\n\ -If @var{all} is true, add suitable vector sizes even when they are generally\n\ +(autovectorize_vector_modes, + "If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}\n\ +is not the only approach worth considering, this hook should add one mode to\n\ +@var{modes} for each useful alternative approach. These modes are then\n\ +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode\n\ +for a given element mode.\n\ +\n\ +The modes returned in @var{modes} should use the smallest element mode\n\ +possible for the vectorization approach that they represent, preferring\n\ +integer modes over floating-poing modes in the event of a tie. The first\n\ +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its\n\ +element mode.\n\ +\n\ +If @var{all} is true, add suitable vector modes even when they are generally\n\ not expected to be worthwhile.\n\ \n\ The hook does not need to do anything if the vector returned by\n\ @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is the only one relevant\n\ for autovectorization. The default implementation does nothing.", void, - (vector_sizes *sizes, bool all), - default_autovectorize_vector_sizes) + (vector_modes *modes, bool all), + default_autovectorize_vector_modes) DEFHOOK (related_mode, Index: gcc/doc/tm.texi.in =================================================================== --- gcc/doc/tm.texi.in 2019-11-05 10:31:39.013103714 +0000 +++ gcc/doc/tm.texi.in 2019-11-05 10:34:34.443861724 +0000 @@ -4179,7 +4179,7 @@ address; but often a machine-dependent @hook TARGET_VECTORIZE_SPLIT_REDUCTION -@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES +@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES @hook TARGET_VECTORIZE_RELATED_MODE Index: gcc/doc/tm.texi =================================================================== --- gcc/doc/tm.texi 2019-11-05 10:31:39.013103714 +0000 +++ gcc/doc/tm.texi 2019-11-05 10:34:34.443861724 +0000 @@ -6016,12 +6016,20 @@ against lower halves of vectors recursiv reached. The default is @var{mode} which means no splitting. @end deftypefn -@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES (vector_sizes *@var{sizes}, bool @var{all}) -If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not -the only one that is worth considering, this hook should add all suitable -vector sizes to @var{sizes}, in order of decreasing preference. The first -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}. -If @var{all} is true, add suitable vector sizes even when they are generally +@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES (vector_modes *@var{modes}, bool @var{all}) +If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} +is not the only approach worth considering, this hook should add one mode to +@var{modes} for each useful alternative approach. These modes are then +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode +for a given element mode. + +The modes returned in @var{modes} should use the smallest element mode +possible for the vectorization approach that they represent, preferring +integer modes over floating-poing modes in the event of a tie. The first +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its +element mode. + +If @var{all} is true, add suitable vector modes even when they are generally not expected to be worthwhile. The hook does not need to do anything if the vector returned by Index: gcc/targhooks.h =================================================================== --- gcc/targhooks.h 2019-11-05 10:31:39.017103686 +0000 +++ gcc/targhooks.h 2019-11-05 10:34:34.447861696 +0000 @@ -113,7 +113,7 @@ default_builtin_support_vector_misalignm int, bool); extern machine_mode default_preferred_simd_mode (scalar_mode mode); extern machine_mode default_split_reduction (machine_mode); -extern void default_autovectorize_vector_sizes (vector_sizes *, bool); +extern void default_autovectorize_vector_modes (vector_modes *, bool); extern opt_machine_mode default_vectorize_related_mode (machine_mode, scalar_mode, poly_uint64); Index: gcc/targhooks.c =================================================================== --- gcc/targhooks.c 2019-11-05 10:31:39.017103686 +0000 +++ gcc/targhooks.c 2019-11-05 10:34:34.447861696 +0000 @@ -1299,11 +1299,10 @@ default_split_reduction (machine_mode mo return mode; } -/* By default only the size derived from the preferred vector mode - is tried. */ +/* By default only the preferred vector mode is tried. */ void -default_autovectorize_vector_sizes (vector_sizes *, bool) +default_autovectorize_vector_modes (vector_modes *, bool) { } Index: gcc/omp-general.c =================================================================== --- gcc/omp-general.c 2019-11-05 10:31:39.013103714 +0000 +++ gcc/omp-general.c 2019-11-05 10:34:34.443861724 +0000 @@ -509,13 +509,16 @@ omp_max_vf (void) && global_options_set.x_flag_tree_loop_vectorize)) return 1; - auto_vector_sizes sizes; - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); - if (!sizes.is_empty ()) + auto_vector_modes modes; + targetm.vectorize.autovectorize_vector_modes (&modes, true); + if (!modes.is_empty ()) { poly_uint64 vf = 0; - for (unsigned int i = 0; i < sizes.length (); ++i) - vf = ordered_max (vf, sizes[i]); + for (unsigned int i = 0; i < modes.length (); ++i) + /* The returned modes use the smallest element size (and thus + the largest nunits) for the vectorization approach that they + represent. */ + vf = ordered_max (vf, GET_MODE_NUNITS (modes[i])); return vf; } Index: gcc/omp-low.c =================================================================== --- gcc/omp-low.c 2019-11-05 10:31:39.013103714 +0000 +++ gcc/omp-low.c 2019-11-05 10:34:34.443861724 +0000 @@ -3959,11 +3959,8 @@ omp_clause_aligned_alignment (tree claus /* Otherwise return implementation defined alignment. */ unsigned int al = 1; opt_scalar_mode mode_iter; - auto_vector_sizes sizes; - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); - poly_uint64 vs = 0; - for (unsigned int i = 0; i < sizes.length (); ++i) - vs = ordered_max (vs, sizes[i]); + auto_vector_modes modes; + targetm.vectorize.autovectorize_vector_modes (&modes, true); static enum mode_class classes[] = { MODE_INT, MODE_VECTOR_INT, MODE_FLOAT, MODE_VECTOR_FLOAT }; for (int i = 0; i < 4; i += 2) @@ -3974,19 +3971,18 @@ omp_clause_aligned_alignment (tree claus machine_mode vmode = targetm.vectorize.preferred_simd_mode (mode); if (GET_MODE_CLASS (vmode) != classes[i + 1]) continue; - while (maybe_ne (vs, 0U) - && known_lt (GET_MODE_SIZE (vmode), vs) - && GET_MODE_2XWIDER_MODE (vmode).exists ()) - vmode = GET_MODE_2XWIDER_MODE (vmode).require (); + machine_mode alt_vmode; + for (unsigned int j = 0; j < modes.length (); ++j) + if (related_vector_mode (modes[j], mode).exists (&alt_vmode) + && known_ge (GET_MODE_SIZE (alt_vmode), GET_MODE_SIZE (vmode))) + vmode = alt_vmode; tree type = lang_hooks.types.type_for_mode (mode, 1); if (type == NULL_TREE || TYPE_MODE (type) != mode) continue; - poly_uint64 nelts = exact_div (GET_MODE_SIZE (vmode), - GET_MODE_SIZE (mode)); - type = build_vector_type (type, nelts); - if (TYPE_MODE (type) != vmode) - continue; + type = build_vector_type_for_mode (type, vmode); + /* The functions above are not allowed to return invalid modes. */ + gcc_assert (TYPE_MODE (type) == vmode); if (TYPE_ALIGN_UNIT (type) > al) al = TYPE_ALIGN_UNIT (type); } Index: gcc/optabs-query.c =================================================================== --- gcc/optabs-query.c 2019-11-05 10:31:39.013103714 +0000 +++ gcc/optabs-query.c 2019-11-05 10:34:34.443861724 +0000 @@ -589,11 +589,11 @@ can_vec_mask_load_store_p (machine_mode && convert_optab_handler (op, vmode, mask_mode) != CODE_FOR_nothing) return true; - auto_vector_sizes vector_sizes; - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, true); - for (unsigned int i = 0; i < vector_sizes.length (); ++i) + auto_vector_modes vector_modes; + targetm.vectorize.autovectorize_vector_modes (&vector_modes, true); + for (unsigned int i = 0; i < vector_modes.length (); ++i) { - poly_uint64 cur = vector_sizes[i]; + poly_uint64 cur = GET_MODE_SIZE (vector_modes[i]); poly_uint64 nunits; if (!multiple_p (cur, GET_MODE_SIZE (smode), &nunits)) continue; Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2019-11-05 10:31:39.017103686 +0000 +++ gcc/tree-vect-loop.c 2019-11-05 10:34:34.447861696 +0000 @@ -2364,12 +2364,12 @@ vect_analyze_loop_2 (loop_vec_info loop_ opt_loop_vec_info vect_analyze_loop (class loop *loop, vec_info_shared *shared) { - auto_vector_sizes vector_sizes; + auto_vector_modes vector_modes; /* Autodetect first vector size we try. */ - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, + targetm.vectorize.autovectorize_vector_modes (&vector_modes, loop->simdlen != 0); - unsigned int next_size = 0; + unsigned int mode_i = 0; DUMP_VECT_SCOPE ("analyze_loop_nest"); @@ -2388,7 +2388,7 @@ vect_analyze_loop (class loop *loop, vec unsigned n_stmts = 0; poly_uint64 autodetected_vector_size = 0; opt_loop_vec_info first_loop_vinfo = opt_loop_vec_info::success (NULL); - poly_uint64 next_vector_size = 0; + machine_mode next_vector_mode = VOIDmode; poly_uint64 lowest_th = 0; unsigned vectorized_loops = 0; @@ -2407,7 +2407,7 @@ vect_analyze_loop (class loop *loop, vec gcc_checking_assert (first_loop_vinfo == NULL); return loop_vinfo; } - loop_vinfo->vector_size = next_vector_size; + loop_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); bool fatal = false; @@ -2415,7 +2415,7 @@ vect_analyze_loop (class loop *loop, vec LOOP_VINFO_ORIG_LOOP_INFO (loop_vinfo) = first_loop_vinfo; res = vect_analyze_loop_2 (loop_vinfo, fatal, &n_stmts); - if (next_size == 0) + if (mode_i == 0) autodetected_vector_size = loop_vinfo->vector_size; loop->aux = NULL; @@ -2478,24 +2478,21 @@ vect_analyze_loop (class loop *loop, vec } } - if (next_size < vector_sizes.length () - && known_eq (vector_sizes[next_size], autodetected_vector_size)) - next_size += 1; + if (mode_i < vector_modes.length () + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), + autodetected_vector_size)) + mode_i += 1; - if (next_size == vector_sizes.length () + if (mode_i == vector_modes.length () || known_eq (autodetected_vector_size, 0U)) break; /* Try the next biggest vector size. */ - next_vector_size = vector_sizes[next_size++]; + next_vector_mode = vector_modes[mode_i++]; if (dump_enabled_p ()) - { - dump_printf_loc (MSG_NOTE, vect_location, - "***** Re-trying analysis with " - "vector size "); - dump_dec (MSG_NOTE, next_vector_size); - dump_printf (MSG_NOTE, "\n"); - } + dump_printf_loc (MSG_NOTE, vect_location, + "***** Re-trying analysis with vector mode %s\n", + GET_MODE_NAME (next_vector_mode)); } if (first_loop_vinfo) Index: gcc/tree-vect-slp.c =================================================================== --- gcc/tree-vect-slp.c 2019-11-05 10:31:39.021103658 +0000 +++ gcc/tree-vect-slp.c 2019-11-05 10:34:34.447861696 +0000 @@ -3171,12 +3171,12 @@ vect_slp_bb_region (gimple_stmt_iterator unsigned int n_stmts) { bb_vec_info bb_vinfo; - auto_vector_sizes vector_sizes; + auto_vector_modes vector_modes; /* Autodetect first vector size we try. */ - poly_uint64 next_vector_size = 0; - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); - unsigned int next_size = 0; + machine_mode next_vector_mode = VOIDmode; + targetm.vectorize.autovectorize_vector_modes (&vector_modes, false); + unsigned int mode_i = 0; vec_info_shared shared; @@ -3193,7 +3193,7 @@ vect_slp_bb_region (gimple_stmt_iterator bb_vinfo->shared->save_datarefs (); else bb_vinfo->shared->check_datarefs (); - bb_vinfo->vector_size = next_vector_size; + bb_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); if (vect_slp_analyze_bb_1 (bb_vinfo, n_stmts, fatal) && dbg_cnt (vect_slp)) @@ -3220,17 +3220,18 @@ vect_slp_bb_region (gimple_stmt_iterator vectorized = true; } - if (next_size == 0) + if (mode_i == 0) autodetected_vector_size = bb_vinfo->vector_size; delete bb_vinfo; - if (next_size < vector_sizes.length () - && known_eq (vector_sizes[next_size], autodetected_vector_size)) - next_size += 1; + if (mode_i < vector_modes.length () + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), + autodetected_vector_size)) + mode_i += 1; if (vectorized - || next_size == vector_sizes.length () + || mode_i == vector_modes.length () || known_eq (autodetected_vector_size, 0U) /* If vect_slp_analyze_bb_1 signaled that analysis for all vector sizes will fail do not bother iterating. */ @@ -3238,15 +3239,11 @@ vect_slp_bb_region (gimple_stmt_iterator return vectorized; /* Try the next biggest vector size. */ - next_vector_size = vector_sizes[next_size++]; + next_vector_mode = vector_modes[mode_i++]; if (dump_enabled_p ()) - { - dump_printf_loc (MSG_NOTE, vect_location, - "***** Re-trying analysis with " - "vector size "); - dump_dec (MSG_NOTE, next_vector_size); - dump_printf (MSG_NOTE, "\n"); - } + dump_printf_loc (MSG_NOTE, vect_location, + "***** Re-trying analysis with vector mode %s\n", + GET_MODE_NAME (next_vector_mode)); } } Index: gcc/config/aarch64/aarch64.c =================================================================== --- gcc/config/aarch64/aarch64.c 2019-11-05 10:31:38.985103913 +0000 +++ gcc/config/aarch64/aarch64.c 2019-11-05 10:34:34.423861866 +0000 @@ -15926,12 +15926,12 @@ aarch64_preferred_simd_mode (scalar_mode /* Return a list of possible vector sizes for the vectorizer to iterate over. */ static void -aarch64_autovectorize_vector_sizes (vector_sizes *sizes, bool) +aarch64_autovectorize_vector_modes (vector_modes *modes, bool) { if (TARGET_SVE) - sizes->safe_push (BYTES_PER_SVE_VECTOR); - sizes->safe_push (16); - sizes->safe_push (8); + modes->safe_push (VNx16QImode); + modes->safe_push (V16QImode); + modes->safe_push (V8QImode); } /* Implement TARGET_MANGLE_TYPE. */ @@ -21765,9 +21765,9 @@ #define TARGET_VECTORIZE_BUILTINS #define TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION \ aarch64_builtin_vectorized_function -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - aarch64_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + aarch64_autovectorize_vector_modes #undef TARGET_ATOMIC_ASSIGN_EXPAND_FENV #define TARGET_ATOMIC_ASSIGN_EXPAND_FENV \ Index: gcc/config/arc/arc.c =================================================================== --- gcc/config/arc/arc.c 2019-11-05 10:31:38.989103884 +0000 +++ gcc/config/arc/arc.c 2019-11-05 10:34:34.427861838 +0000 @@ -607,15 +607,15 @@ arc_preferred_simd_mode (scalar_mode mod } /* Implements target hook - TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ + TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ static void -arc_autovectorize_vector_sizes (vector_sizes *sizes, bool) +arc_autovectorize_vector_modes (vector_modes *modes, bool) { if (TARGET_PLUS_QMACW) { - sizes->quick_push (8); - sizes->quick_push (4); + modes->quick_push (V4HImode); + modes->quick_push (V2HImode); } } @@ -726,8 +726,8 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arc_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES arc_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES arc_autovectorize_vector_modes #undef TARGET_CAN_USE_DOLOOP_P #define TARGET_CAN_USE_DOLOOP_P arc_can_use_doloop_p Index: gcc/config/arm/arm.c =================================================================== --- gcc/config/arm/arm.c 2019-11-05 10:31:39.001103800 +0000 +++ gcc/config/arm/arm.c 2019-11-05 10:34:34.435861782 +0000 @@ -289,7 +289,7 @@ static bool arm_builtin_support_vector_m static void arm_conditional_register_usage (void); static enum flt_eval_method arm_excess_precision (enum excess_precision_type); static reg_class_t arm_preferred_rename_class (reg_class_t rclass); -static void arm_autovectorize_vector_sizes (vector_sizes *, bool); +static void arm_autovectorize_vector_modes (vector_modes *, bool); static int arm_default_branch_cost (bool, bool); static int arm_cortex_a5_branch_cost (bool, bool); static int arm_cortex_m_branch_cost (bool, bool); @@ -522,9 +522,9 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a #define TARGET_ARRAY_MODE_SUPPORTED_P arm_array_mode_supported_p #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arm_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - arm_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + arm_autovectorize_vector_modes #undef TARGET_MACHINE_DEPENDENT_REORG #define TARGET_MACHINE_DEPENDENT_REORG arm_reorg @@ -29016,12 +29016,12 @@ arm_vector_alignment (const_tree type) } static void -arm_autovectorize_vector_sizes (vector_sizes *sizes, bool) +arm_autovectorize_vector_modes (vector_modes *modes, bool) { if (!TARGET_NEON_VECTORIZE_DOUBLE) { - sizes->safe_push (16); - sizes->safe_push (8); + modes->safe_push (V16QImode); + modes->safe_push (V8QImode); } } Index: gcc/config/i386/i386.c =================================================================== --- gcc/config/i386/i386.c 2019-11-05 10:31:39.005103774 +0000 +++ gcc/config/i386/i386.c 2019-11-05 10:34:34.435861782 +0000 @@ -21386,35 +21386,35 @@ ix86_preferred_simd_mode (scalar_mode mo 256bit and 128bit vectors. */ static void -ix86_autovectorize_vector_sizes (vector_sizes *sizes, bool all) +ix86_autovectorize_vector_modes (vector_modes *modes, bool all) { if (TARGET_AVX512F && !TARGET_PREFER_AVX256) { - sizes->safe_push (64); - sizes->safe_push (32); - sizes->safe_push (16); + modes->safe_push (V64QImode); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); } else if (TARGET_AVX512F && all) { - sizes->safe_push (32); - sizes->safe_push (16); - sizes->safe_push (64); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); + modes->safe_push (V64QImode); } else if (TARGET_AVX && !TARGET_PREFER_AVX128) { - sizes->safe_push (32); - sizes->safe_push (16); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); } else if (TARGET_AVX && all) { - sizes->safe_push (16); - sizes->safe_push (32); + modes->safe_push (V16QImode); + modes->safe_push (V32QImode); } else if (TARGET_MMX_WITH_SSE) - sizes->safe_push (16); + modes->safe_push (V16QImode); if (TARGET_MMX_WITH_SSE) - sizes->safe_push (8); + modes->safe_push (V8QImode); } /* Implemenation of targetm.vectorize.get_mask_mode. */ @@ -22953,9 +22953,9 @@ #define TARGET_VECTORIZE_PREFERRED_SIMD_ #undef TARGET_VECTORIZE_SPLIT_REDUCTION #define TARGET_VECTORIZE_SPLIT_REDUCTION \ ix86_split_reduction -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - ix86_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + ix86_autovectorize_vector_modes #undef TARGET_VECTORIZE_GET_MASK_MODE #define TARGET_VECTORIZE_GET_MASK_MODE ix86_get_mask_mode #undef TARGET_VECTORIZE_INIT_COST Index: gcc/config/mips/mips.c =================================================================== --- gcc/config/mips/mips.c 2019-11-05 10:31:39.009103742 +0000 +++ gcc/config/mips/mips.c 2019-11-05 10:34:34.439861754 +0000 @@ -13453,13 +13453,13 @@ mips_preferred_simd_mode (scalar_mode mo return word_mode; } -/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ +/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ static void -mips_autovectorize_vector_sizes (vector_sizes *sizes, bool) +mips_autovectorize_vector_modes (vector_modes *modes, bool) { if (ISA_HAS_MSA) - sizes->safe_push (16); + modes->safe_push (V16QImode); } /* Implement TARGET_INIT_LIBFUNCS. */ @@ -22716,9 +22716,9 @@ #define TARGET_SCALAR_MODE_SUPPORTED_P m #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE mips_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - mips_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + mips_autovectorize_vector_modes #undef TARGET_INIT_BUILTINS #define TARGET_INIT_BUILTINS mips_init_builtins
On Wed, Oct 30, 2019 at 4:58 PM Richard Sandiford <richard.sandiford@arm.com> wrote: > > Richard Biener <richard.guenther@gmail.com> writes: > > On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford > > <richard.sandiford@arm.com> wrote: > >> > >> This is another patch in the series to remove the assumption that > >> all modes involved in vectorisation have to be the same size. > >> Rather than have the target provide a list of vector sizes, > >> it makes the target provide a list of vector "approaches", > >> with each approach represented by a mode. > >> > >> A later patch will pass this mode to targetm.vectorize.related_mode > >> to get the vector mode for a given element mode. Until then, the modes > >> simply act as an alternative way of specifying the vector size. > > > > Is there a restriction to use integer vector modes for the hook > > or would FP vector modes be OK as well? > > Conceptually, each mode returned by the hook represents a set of vector > modes, with the set containing one member for each supported element > type. The idea is to represent the set using the member with the > smallest element type, preferring integer modes over floating-point > modes in the event of a tie. So using a floating-point mode as the > representative mode is fine if floating-point elements are the smallest > (or only) supported element type. > > > Note that your x86 change likely disables word_mode vectorization with > > -mno-sse? > > No, that still works, because... > > > That is, how do we represent GPR vectorization "size" here? > > The preferred SIMD mode hook may return an integer mode, > > are non-vector modes OK for autovectorize_vector_modes? > > ...at least with all current targets, preferred_simd_mode is only > an integer mode if the target has no "real" vectorisation support > for that element type. There's no need to handle that case in > autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing > when SSE is disabled. > > So while preferred_simd_mode can continue to return integer modes, > autovectorize_vector_modes always returns vector modes. Hmm, I see. IIRC I was playing with a patch for x86 that enabled word-mode vectorization (64bits) for SSE before (I see we don't do that at the moment). The MMX-with-SSE has made that somewhat moot but with iterating over modes we could even make MMX-with-SSE (MMX modes) and word-mode vectors coexist by allowing the hook to return V4SI, V2SI, DImode? Because MMX-with-SSE might be more costly than word-mode but can of course handle more cases. So you say the above isn't supported and cannot be made supported? Thanks, Richard. > This patch just treats the mode as an alternative way of specifying > the vector size. 11/n then tries to use related_vector_mode to choose > the vector mode for each element type instead. But 11/n only uses > related_vector_mode if vec_info::vector_mode is a vector mode. If it's > an integer mode (as for -mno-sse), or if related_vector_mode fails to > find a vector mode, then we still fall back to mode_for_vector and so > pick an integer mode in the same cases as before. > > Thanks, > Richard
Richard Biener <richard.guenther@gmail.com> writes: > On Wed, Oct 30, 2019 at 4:58 PM Richard Sandiford > <richard.sandiford@arm.com> wrote: >> >> Richard Biener <richard.guenther@gmail.com> writes: >> > On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford >> > <richard.sandiford@arm.com> wrote: >> >> >> >> This is another patch in the series to remove the assumption that >> >> all modes involved in vectorisation have to be the same size. >> >> Rather than have the target provide a list of vector sizes, >> >> it makes the target provide a list of vector "approaches", >> >> with each approach represented by a mode. >> >> >> >> A later patch will pass this mode to targetm.vectorize.related_mode >> >> to get the vector mode for a given element mode. Until then, the modes >> >> simply act as an alternative way of specifying the vector size. >> > >> > Is there a restriction to use integer vector modes for the hook >> > or would FP vector modes be OK as well? >> >> Conceptually, each mode returned by the hook represents a set of vector >> modes, with the set containing one member for each supported element >> type. The idea is to represent the set using the member with the >> smallest element type, preferring integer modes over floating-point >> modes in the event of a tie. So using a floating-point mode as the >> representative mode is fine if floating-point elements are the smallest >> (or only) supported element type. >> >> > Note that your x86 change likely disables word_mode vectorization with >> > -mno-sse? >> >> No, that still works, because... >> >> > That is, how do we represent GPR vectorization "size" here? >> > The preferred SIMD mode hook may return an integer mode, >> > are non-vector modes OK for autovectorize_vector_modes? >> >> ...at least with all current targets, preferred_simd_mode is only >> an integer mode if the target has no "real" vectorisation support >> for that element type. There's no need to handle that case in >> autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing >> when SSE is disabled. >> >> So while preferred_simd_mode can continue to return integer modes, >> autovectorize_vector_modes always returns vector modes. > > Hmm, I see. IIRC I was playing with a patch for x86 that > enabled word-mode vectorization (64bits) for SSE before (I see > we don't do that at the moment). The MMX-with-SSE has made > that somewhat moot but with iterating over modes we could > even make MMX-with-SSE (MMX modes) and word-mode vectors > coexist by allowing the hook to return V4SI, V2SI, DImode? > Because MMX-with-SSE might be more costly than word-mode > but can of course handle more cases. > > So you say the above isn't supported and cannot be made supported? It isn't supported as things stand. It shouldn't be hard to make it work, but I'm not sure what the best semantics would be. AIUI, before the series, returning word_mode from preferred_simd_mode just means that vectors should have the same size as word_mode. If the target defines V2SI, we'll use that as the raw type mode for SI vectors, regardless of whether V2SI is enabled. If the mode *is* enabled, the TYPE_MODE will also be V2SI and so returning word_mode from preferred_simd_mode is equivalent to returning V2SImode. If the mode isn't enabled, the TYPE_MODE will be word_mode if that's suitable and BLKmode otherwise. The situation's similar for SF; if the target defines and supports V2SF, returning word_mode would be equivalent to returning V2SFmode. But it sounds like returning word_mode for the new hook would behave differently, in that we'd force the raw type mode to be DImode even if V2SImode is defined and supported. So what should happen for float types? Should we reject those, or behave as above and apply the usual mode_for_vector treatment for a word_mode-sized vector? If code contains a mixture of HImode and SImode elements, should we use DImode for both of them, or SImode for HImode elements? Should the modes be passed to the target's related_vector_mode hook in the same way as for vectors, or handled before then? I could implement one of these. I'm just not sure it'd turn out to be the right one, once someone actually tries to use it. :-) FWIW, another way of doing the same thing would be to define emulated vector modes, e.g. EMUL_V2SI, giving them a lower priority than the real V2SI. This is already possible with VECTOR_MODES_WITH_PREFIX. Because these emulated modes would be permanently unsupported, the associated TYPE_MODE would always be the equivalent integer mode (if appropriate). So we could force integer modes that way too. This has the advantage that we never lose sight of what the element type is, and so can choose between pairing EMUL_V2SI and EMUL_V4HI vs. pairing EMUL_V2SI and EMUL_V2HI, just like we can for "real" vector modes. Of course that's "a bit" of a hack. But then so IMO is using integer modes for this kind of choice. :-) Another option I'd considered was having the hook return a list of abstract identifiers that are only meaningful to the target, either with accompanying information like the maximum size and maximum nunits, or with a separate hook to provide that information. Or we could even return a list of virtual objects. But that seemed like an abstraction too far when in practice a vector mode should be descriptive enough. Thanks, Richard
On Tue, Nov 12, 2019 at 6:54 PM Richard Sandiford <richard.sandiford@arm.com> wrote: > > Richard Biener <richard.guenther@gmail.com> writes: > > On Wed, Oct 30, 2019 at 4:58 PM Richard Sandiford > > <richard.sandiford@arm.com> wrote: > >> > >> Richard Biener <richard.guenther@gmail.com> writes: > >> > On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford > >> > <richard.sandiford@arm.com> wrote: > >> >> > >> >> This is another patch in the series to remove the assumption that > >> >> all modes involved in vectorisation have to be the same size. > >> >> Rather than have the target provide a list of vector sizes, > >> >> it makes the target provide a list of vector "approaches", > >> >> with each approach represented by a mode. > >> >> > >> >> A later patch will pass this mode to targetm.vectorize.related_mode > >> >> to get the vector mode for a given element mode. Until then, the modes > >> >> simply act as an alternative way of specifying the vector size. > >> > > >> > Is there a restriction to use integer vector modes for the hook > >> > or would FP vector modes be OK as well? > >> > >> Conceptually, each mode returned by the hook represents a set of vector > >> modes, with the set containing one member for each supported element > >> type. The idea is to represent the set using the member with the > >> smallest element type, preferring integer modes over floating-point > >> modes in the event of a tie. So using a floating-point mode as the > >> representative mode is fine if floating-point elements are the smallest > >> (or only) supported element type. > >> > >> > Note that your x86 change likely disables word_mode vectorization with > >> > -mno-sse? > >> > >> No, that still works, because... > >> > >> > That is, how do we represent GPR vectorization "size" here? > >> > The preferred SIMD mode hook may return an integer mode, > >> > are non-vector modes OK for autovectorize_vector_modes? > >> > >> ...at least with all current targets, preferred_simd_mode is only > >> an integer mode if the target has no "real" vectorisation support > >> for that element type. There's no need to handle that case in > >> autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing > >> when SSE is disabled. > >> > >> So while preferred_simd_mode can continue to return integer modes, > >> autovectorize_vector_modes always returns vector modes. > > > > Hmm, I see. IIRC I was playing with a patch for x86 that > > enabled word-mode vectorization (64bits) for SSE before (I see > > we don't do that at the moment). The MMX-with-SSE has made > > that somewhat moot but with iterating over modes we could > > even make MMX-with-SSE (MMX modes) and word-mode vectors > > coexist by allowing the hook to return V4SI, V2SI, DImode? > > Because MMX-with-SSE might be more costly than word-mode > > but can of course handle more cases. > > > > So you say the above isn't supported and cannot be made supported? > > It isn't supported as things stand. It shouldn't be hard to make > it work, but I'm not sure what the best semantics would be. > > AIUI, before the series, returning word_mode from preferred_simd_mode > just means that vectors should have the same size as word_mode. If the > target defines V2SI, we'll use that as the raw type mode for SI vectors, > regardless of whether V2SI is enabled. If the mode *is* enabled, > the TYPE_MODE will also be V2SI and so returning word_mode from > preferred_simd_mode is equivalent to returning V2SImode. If the mode > isn't enabled, the TYPE_MODE will be word_mode if that's suitable and > BLKmode otherwise. > > The situation's similar for SF; if the target defines and supports V2SF, > returning word_mode would be equivalent to returning V2SFmode. > > But it sounds like returning word_mode for the new hook would behave > differently, in that we'd force the raw type mode to be DImode even > if V2SImode is defined and supported. Yes. > So what should happen for float > types? Should we reject those, or behave as above and apply the usual > mode_for_vector treatment for a word_mode-sized vector? I wasn't aware we're doing this mode_for_vector dance, certainly we didn't that before? I expected targets would return vector modes that are enabled only. What's the reason to not do that? > If code contains a mixture of HImode and SImode elements, should > we use DImode for both of them, or SImode for HImode elements? > Should the modes be passed to the target's related_vector_mode > hook in the same way as for vectors, or handled before then? I'd say SImode and HImode are naturally "related" "vector" modes for word_mode - so word_mode ideally is just a placeholder for "try to vectorize w/o actual SIMD instructions" which we can do for a small set of operations. > I could implement one of these. I'm just not sure it'd turn out > to be the right one, once someone actually tries to use it. :-) Heh. I was merely trying to make sure we're not designing us into a corner where we can't mix the word_mode vectorization facility with the SIMD one (because on GIMPLE it's still all vectors, only vector lowering exposes the ints IIRC). > FWIW, another way of doing the same thing would be to define > emulated vector modes, e.g. EMUL_V2SI, giving them a lower > priority than the real V2SI. This is already possible with > VECTOR_MODES_WITH_PREFIX. Because these emulated modes would > be permanently unsupported, the associated TYPE_MODE would always > be the equivalent integer mode (if appropriate). So we could force > integer modes that way too. This has the advantage that we never lose > sight of what the element type is, and so can choose between pairing > EMUL_V2SI and EMUL_V4HI vs. pairing EMUL_V2SI and EMUL_V2HI, > just like we can for "real" vector modes. > > Of course that's "a bit" of a hack. But then so IMO is using integer > modes for this kind of choice. :-) > > Another option I'd considered was having the hook return a list of > abstract identifiers that are only meaningful to the target, either > with accompanying information like the maximum size and maximum nunits, > or with a separate hook to provide that information. Or we could even > return a list of virtual objects. But that seemed like an abstraction > too far when in practice a vector mode should be descriptive enough. Anyway, I think we have testsuite coverage for word_mode vectorization and IIRC i?86 enables that when there's no SSE support via the preferred_simd_mode hook which you do not change (and at the moment that doesn't have to be in the list of sizes we should iterate over). When I understand you correctly what is not yet supported with your patch is adding word_mode (or other integer modes) to autovectorize_vector_modes but behavior could be emulated via non-enabled vector modes that get mapped to integer modes. Thus the patch is OK. Thanks, Richard. > Thanks, > Richard
Richard Biener <richard.guenther@gmail.com> writes: > On Tue, Nov 12, 2019 at 6:54 PM Richard Sandiford > <richard.sandiford@arm.com> wrote: >> >> Richard Biener <richard.guenther@gmail.com> writes: >> > On Wed, Oct 30, 2019 at 4:58 PM Richard Sandiford >> > <richard.sandiford@arm.com> wrote: >> >> >> >> Richard Biener <richard.guenther@gmail.com> writes: >> >> > On Fri, Oct 25, 2019 at 2:37 PM Richard Sandiford >> >> > <richard.sandiford@arm.com> wrote: >> >> >> >> >> >> This is another patch in the series to remove the assumption that >> >> >> all modes involved in vectorisation have to be the same size. >> >> >> Rather than have the target provide a list of vector sizes, >> >> >> it makes the target provide a list of vector "approaches", >> >> >> with each approach represented by a mode. >> >> >> >> >> >> A later patch will pass this mode to targetm.vectorize.related_mode >> >> >> to get the vector mode for a given element mode. Until then, the modes >> >> >> simply act as an alternative way of specifying the vector size. >> >> > >> >> > Is there a restriction to use integer vector modes for the hook >> >> > or would FP vector modes be OK as well? >> >> >> >> Conceptually, each mode returned by the hook represents a set of vector >> >> modes, with the set containing one member for each supported element >> >> type. The idea is to represent the set using the member with the >> >> smallest element type, preferring integer modes over floating-point >> >> modes in the event of a tie. So using a floating-point mode as the >> >> representative mode is fine if floating-point elements are the smallest >> >> (or only) supported element type. >> >> >> >> > Note that your x86 change likely disables word_mode vectorization with >> >> > -mno-sse? >> >> >> >> No, that still works, because... >> >> >> >> > That is, how do we represent GPR vectorization "size" here? >> >> > The preferred SIMD mode hook may return an integer mode, >> >> > are non-vector modes OK for autovectorize_vector_modes? >> >> >> >> ...at least with all current targets, preferred_simd_mode is only >> >> an integer mode if the target has no "real" vectorisation support >> >> for that element type. There's no need to handle that case in >> >> autovectorize_vector_sizes/modes, and e.g. the x86 hook does nothing >> >> when SSE is disabled. >> >> >> >> So while preferred_simd_mode can continue to return integer modes, >> >> autovectorize_vector_modes always returns vector modes. >> > >> > Hmm, I see. IIRC I was playing with a patch for x86 that >> > enabled word-mode vectorization (64bits) for SSE before (I see >> > we don't do that at the moment). The MMX-with-SSE has made >> > that somewhat moot but with iterating over modes we could >> > even make MMX-with-SSE (MMX modes) and word-mode vectors >> > coexist by allowing the hook to return V4SI, V2SI, DImode? >> > Because MMX-with-SSE might be more costly than word-mode >> > but can of course handle more cases. >> > >> > So you say the above isn't supported and cannot be made supported? >> >> It isn't supported as things stand. It shouldn't be hard to make >> it work, but I'm not sure what the best semantics would be. >> >> AIUI, before the series, returning word_mode from preferred_simd_mode >> just means that vectors should have the same size as word_mode. If the >> target defines V2SI, we'll use that as the raw type mode for SI vectors, >> regardless of whether V2SI is enabled. If the mode *is* enabled, >> the TYPE_MODE will also be V2SI and so returning word_mode from >> preferred_simd_mode is equivalent to returning V2SImode. If the mode >> isn't enabled, the TYPE_MODE will be word_mode if that's suitable and >> BLKmode otherwise. >> >> The situation's similar for SF; if the target defines and supports V2SF, >> returning word_mode would be equivalent to returning V2SFmode. >> >> But it sounds like returning word_mode for the new hook would behave >> differently, in that we'd force the raw type mode to be DImode even >> if V2SImode is defined and supported. > > Yes. > >> So what should happen for float >> types? Should we reject those, or behave as above and apply the usual >> mode_for_vector treatment for a word_mode-sized vector? > > I wasn't aware we're doing this mode_for_vector dance, certainly we > didn't that before? We do that without the series too, just a bit more indirectly. get_vectype_for_scalar_type_and_size has: /* If no size was supplied use the mode the target prefers. Otherwise lookup a vector mode of the specified size. */ if (known_eq (size, 0U)) simd_mode = targetm.vectorize.preferred_simd_mode (inner_mode); else if (!multiple_p (size, nbytes, &nunits) || !mode_for_vector (inner_mode, nunits).exists (&simd_mode)) return NULL_TREE; /* NOTE: nunits == 1 is allowed to support single element vector types. */ if (!multiple_p (GET_MODE_SIZE (simd_mode), nbytes, &nunits)) return NULL_TREE; vectype = build_vector_type (scalar_type, nunits); So all we use the simd_mode for is to get a size. Then that size determines the nunits, and build_vector_type picks whichever mode works best for that nunits and element type. That mode might not have anything to do with simd_mode, except for having the same size. So although the new mode_for_vector calls make that explicit, it was the behaviour before the series too. > I expected targets would return vector modes that > are enabled only. What's the reason to not do that? I think all current targets do only return supported modes or word_mode from preferred_simd_mode. But returning word_mode isn't for example an effective way of preventing V2SI being used, since if V2SI is enabled for 64-bit targets, word_mode would actually choose V2SI rather than DI. >> If code contains a mixture of HImode and SImode elements, should >> we use DImode for both of them, or SImode for HImode elements? >> Should the modes be passed to the target's related_vector_mode >> hook in the same way as for vectors, or handled before then? > > I'd say SImode and HImode are naturally "related" "vector" modes > for word_mode - so word_mode ideally is just a placeholder for > "try to vectorize w/o actual SIMD instructions" which we can do > for a small set of operations. > >> I could implement one of these. I'm just not sure it'd turn out >> to be the right one, once someone actually tries to use it. :-) > > Heh. I was merely trying to make sure we're not designing us > into a corner where we can't mix the word_mode vectorization > facility with the SIMD one (because on GIMPLE it's still all > vectors, only vector lowering exposes the ints IIRC). > >> FWIW, another way of doing the same thing would be to define >> emulated vector modes, e.g. EMUL_V2SI, giving them a lower >> priority than the real V2SI. This is already possible with >> VECTOR_MODES_WITH_PREFIX. Because these emulated modes would >> be permanently unsupported, the associated TYPE_MODE would always >> be the equivalent integer mode (if appropriate). So we could force >> integer modes that way too. This has the advantage that we never lose >> sight of what the element type is, and so can choose between pairing >> EMUL_V2SI and EMUL_V4HI vs. pairing EMUL_V2SI and EMUL_V2HI, >> just like we can for "real" vector modes. >> >> Of course that's "a bit" of a hack. But then so IMO is using integer >> modes for this kind of choice. :-) >> >> Another option I'd considered was having the hook return a list of >> abstract identifiers that are only meaningful to the target, either >> with accompanying information like the maximum size and maximum nunits, >> or with a separate hook to provide that information. Or we could even >> return a list of virtual objects. But that seemed like an abstraction >> too far when in practice a vector mode should be descriptive enough. > > Anyway, I think we have testsuite coverage for word_mode vectorization > and IIRC i?86 enables that when there's no SSE support via > the preferred_simd_mode hook which you do not change (and at the moment > that doesn't have to be in the list of sizes we should iterate over). Yeah. > When I understand you correctly what is not yet supported with your patch > is adding word_mode (or other integer modes) to autovectorize_vector_modes > but behavior could be emulated via non-enabled vector modes that get > mapped to integer modes. Right. And we could support word_mode with its current meaning from autovectorize_vector_modes too if that turns out to be the best way of handling it. Thanks, Richard
Index: gcc/target.h =================================================================== --- gcc/target.h 2019-09-30 17:19:39.843166118 +0100 +++ gcc/target.h 2019-10-25 13:27:15.525762975 +0100 @@ -205,11 +205,11 @@ enum vect_cost_model_location { class vec_perm_indices; /* The type to use for lists of vector sizes. */ -typedef vec<poly_uint64> vector_sizes; +typedef vec<machine_mode> vector_modes; /* Same, but can be used to construct local lists that are automatically freed. */ -typedef auto_vec<poly_uint64, 8> auto_vector_sizes; +typedef auto_vec<machine_mode, 8> auto_vector_modes; /* The target structure. This holds all the backend hooks. */ #define DEFHOOKPOD(NAME, DOC, TYPE, INIT) TYPE NAME; Index: gcc/target.def =================================================================== --- gcc/target.def 2019-10-25 13:26:59.309877555 +0100 +++ gcc/target.def 2019-10-25 13:27:15.525762975 +0100 @@ -1894,20 +1894,28 @@ reached. The default is @var{mode} whic /* Returns a mask of vector sizes to iterate over when auto-vectorizing after processing the preferred one derived from preferred_simd_mode. */ DEFHOOK -(autovectorize_vector_sizes, - "If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not\n\ -the only one that is worth considering, this hook should add all suitable\n\ -vector sizes to @var{sizes}, in order of decreasing preference. The first\n\ -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}.\n\ -If @var{all} is true, add suitable vector sizes even when they are generally\n\ +(autovectorize_vector_modes, + "If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}\n\ +is not the only approach worth considering, this hook should add one mode to\n\ +@var{modes} for each useful alternative approach. These modes are then\n\ +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode\n\ +for a given element mode.\n\ +\n\ +The modes returned in @var{modes} should use the smallest element mode\n\ +possible for the vectorization approach that they represent, preferring\n\ +integer modes over floating-poing modes in the event of a tie. The first\n\ +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its\n\ +element mode.\n\ +\n\ +If @var{all} is true, add suitable vector modes even when they are generally\n\ not expected to be worthwhile.\n\ \n\ The hook does not need to do anything if the vector returned by\n\ @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is the only one relevant\n\ for autovectorization. The default implementation does nothing.", void, - (vector_sizes *sizes, bool all), - default_autovectorize_vector_sizes) + (vector_modes *modes, bool all), + default_autovectorize_vector_modes) DEFHOOK (related_mode, Index: gcc/doc/tm.texi.in =================================================================== --- gcc/doc/tm.texi.in 2019-10-25 13:26:59.009879675 +0100 +++ gcc/doc/tm.texi.in 2019-10-25 13:27:15.521763003 +0100 @@ -4179,7 +4179,7 @@ address; but often a machine-dependent @hook TARGET_VECTORIZE_SPLIT_REDUCTION -@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES +@hook TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES @hook TARGET_VECTORIZE_RELATED_MODE Index: gcc/doc/tm.texi =================================================================== --- gcc/doc/tm.texi 2019-10-25 13:26:59.305877583 +0100 +++ gcc/doc/tm.texi 2019-10-25 13:27:15.521763003 +0100 @@ -6016,12 +6016,20 @@ against lower halves of vectors recursiv reached. The default is @var{mode} which means no splitting. @end deftypefn -@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES (vector_sizes *@var{sizes}, bool @var{all}) -If the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} is not -the only one that is worth considering, this hook should add all suitable -vector sizes to @var{sizes}, in order of decreasing preference. The first -one should be the size of @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE}. -If @var{all} is true, add suitable vector sizes even when they are generally +@deftypefn {Target Hook} void TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES (vector_modes *@var{modes}, bool @var{all}) +If using the mode returned by @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} +is not the only approach worth considering, this hook should add one mode to +@var{modes} for each useful alternative approach. These modes are then +passed to @code{TARGET_VECTORIZE_RELATED_MODE} to obtain the vector mode +for a given element mode. + +The modes returned in @var{modes} should use the smallest element mode +possible for the vectorization approach that they represent, preferring +integer modes over floating-poing modes in the event of a tie. The first +mode should be the @code{TARGET_VECTORIZE_PREFERRED_SIMD_MODE} for its +element mode. + +If @var{all} is true, add suitable vector modes even when they are generally not expected to be worthwhile. The hook does not need to do anything if the vector returned by Index: gcc/targhooks.h =================================================================== --- gcc/targhooks.h 2019-10-25 13:26:59.309877555 +0100 +++ gcc/targhooks.h 2019-10-25 13:27:15.525762975 +0100 @@ -113,7 +113,7 @@ default_builtin_support_vector_misalignm int, bool); extern machine_mode default_preferred_simd_mode (scalar_mode mode); extern machine_mode default_split_reduction (machine_mode); -extern void default_autovectorize_vector_sizes (vector_sizes *, bool); +extern void default_autovectorize_vector_modes (vector_modes *, bool); extern opt_machine_mode default_vectorize_related_mode (machine_mode, scalar_mode, poly_uint64); Index: gcc/targhooks.c =================================================================== --- gcc/targhooks.c 2019-10-25 13:26:59.309877555 +0100 +++ gcc/targhooks.c 2019-10-25 13:27:15.525762975 +0100 @@ -1299,11 +1299,10 @@ default_split_reduction (machine_mode mo return mode; } -/* By default only the size derived from the preferred vector mode - is tried. */ +/* By default only the preferred vector mode is tried. */ void -default_autovectorize_vector_sizes (vector_sizes *, bool) +default_autovectorize_vector_modes (vector_modes *, bool) { } Index: gcc/omp-general.c =================================================================== --- gcc/omp-general.c 2019-10-25 09:21:28.798326303 +0100 +++ gcc/omp-general.c 2019-10-25 13:27:15.521763003 +0100 @@ -508,13 +508,16 @@ omp_max_vf (void) && global_options_set.x_flag_tree_loop_vectorize)) return 1; - auto_vector_sizes sizes; - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); - if (!sizes.is_empty ()) + auto_vector_modes modes; + targetm.vectorize.autovectorize_vector_modes (&modes, true); + if (!modes.is_empty ()) { poly_uint64 vf = 0; - for (unsigned int i = 0; i < sizes.length (); ++i) - vf = ordered_max (vf, sizes[i]); + for (unsigned int i = 0; i < modes.length (); ++i) + /* The returned modes use the smallest element size (and thus + the largest nunits) for the vectorization approach that they + represent. */ + vf = ordered_max (vf, GET_MODE_NUNITS (modes[i])); return vf; } Index: gcc/omp-low.c =================================================================== --- gcc/omp-low.c 2019-10-11 15:43:51.283513446 +0100 +++ gcc/omp-low.c 2019-10-25 13:27:15.525762975 +0100 @@ -3947,11 +3947,8 @@ omp_clause_aligned_alignment (tree claus /* Otherwise return implementation defined alignment. */ unsigned int al = 1; opt_scalar_mode mode_iter; - auto_vector_sizes sizes; - targetm.vectorize.autovectorize_vector_sizes (&sizes, true); - poly_uint64 vs = 0; - for (unsigned int i = 0; i < sizes.length (); ++i) - vs = ordered_max (vs, sizes[i]); + auto_vector_modes modes; + targetm.vectorize.autovectorize_vector_modes (&modes, true); static enum mode_class classes[] = { MODE_INT, MODE_VECTOR_INT, MODE_FLOAT, MODE_VECTOR_FLOAT }; for (int i = 0; i < 4; i += 2) @@ -3962,19 +3959,18 @@ omp_clause_aligned_alignment (tree claus machine_mode vmode = targetm.vectorize.preferred_simd_mode (mode); if (GET_MODE_CLASS (vmode) != classes[i + 1]) continue; - while (maybe_ne (vs, 0U) - && known_lt (GET_MODE_SIZE (vmode), vs) - && GET_MODE_2XWIDER_MODE (vmode).exists ()) - vmode = GET_MODE_2XWIDER_MODE (vmode).require (); + machine_mode alt_vmode; + for (unsigned int j = 0; j < modes.length (); ++j) + if (related_vector_mode (modes[j], mode).exists (&alt_vmode) + && known_ge (GET_MODE_SIZE (alt_vmode), GET_MODE_SIZE (vmode))) + vmode = alt_vmode; tree type = lang_hooks.types.type_for_mode (mode, 1); if (type == NULL_TREE || TYPE_MODE (type) != mode) continue; - poly_uint64 nelts = exact_div (GET_MODE_SIZE (vmode), - GET_MODE_SIZE (mode)); - type = build_vector_type (type, nelts); - if (TYPE_MODE (type) != vmode) - continue; + type = build_vector_type_for_mode (type, vmode); + /* The functions above are not allowed to return invalid modes. */ + gcc_assert (TYPE_MODE (type) == vmode); if (TYPE_ALIGN_UNIT (type) > al) al = TYPE_ALIGN_UNIT (type); } Index: gcc/optabs-query.c =================================================================== --- gcc/optabs-query.c 2019-10-25 13:26:59.305877583 +0100 +++ gcc/optabs-query.c 2019-10-25 13:27:15.525762975 +0100 @@ -589,11 +589,11 @@ can_vec_mask_load_store_p (machine_mode && convert_optab_handler (op, vmode, mask_mode) != CODE_FOR_nothing) return true; - auto_vector_sizes vector_sizes; - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, true); - for (unsigned int i = 0; i < vector_sizes.length (); ++i) + auto_vector_modes vector_modes; + targetm.vectorize.autovectorize_vector_modes (&vector_modes, true); + for (unsigned int i = 0; i < vector_modes.length (); ++i) { - poly_uint64 cur = vector_sizes[i]; + poly_uint64 cur = GET_MODE_SIZE (vector_modes[i]); poly_uint64 nunits; if (!multiple_p (cur, GET_MODE_SIZE (smode), &nunits)) continue; Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2019-10-25 13:26:59.137878771 +0100 +++ gcc/tree-vect-loop.c 2019-10-25 13:27:15.525762975 +0100 @@ -2319,12 +2319,12 @@ vect_analyze_loop_2 (loop_vec_info loop_ vect_analyze_loop (class loop *loop, loop_vec_info orig_loop_vinfo, vec_info_shared *shared) { - auto_vector_sizes vector_sizes; + auto_vector_modes vector_modes; /* Autodetect first vector size we try. */ - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, + targetm.vectorize.autovectorize_vector_modes (&vector_modes, loop->simdlen != 0); - unsigned int next_size = 0; + unsigned int mode_i = 0; DUMP_VECT_SCOPE ("analyze_loop_nest"); @@ -2343,7 +2343,7 @@ vect_analyze_loop (class loop *loop, loo unsigned n_stmts = 0; poly_uint64 autodetected_vector_size = 0; opt_loop_vec_info first_loop_vinfo = opt_loop_vec_info::success (NULL); - poly_uint64 next_vector_size = 0; + machine_mode next_vector_mode = VOIDmode; while (1) { /* Check the CFG characteristics of the loop (nesting, entry/exit). */ @@ -2357,7 +2357,7 @@ vect_analyze_loop (class loop *loop, loo gcc_checking_assert (first_loop_vinfo == NULL); return loop_vinfo; } - loop_vinfo->vector_size = next_vector_size; + loop_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); bool fatal = false; @@ -2365,7 +2365,7 @@ vect_analyze_loop (class loop *loop, loo LOOP_VINFO_ORIG_LOOP_INFO (loop_vinfo) = orig_loop_vinfo; opt_result res = vect_analyze_loop_2 (loop_vinfo, fatal, &n_stmts); - if (next_size == 0) + if (mode_i == 0) autodetected_vector_size = loop_vinfo->vector_size; if (res) @@ -2399,11 +2399,12 @@ vect_analyze_loop (class loop *loop, loo return opt_loop_vec_info::propagate_failure (res); } - if (next_size < vector_sizes.length () - && known_eq (vector_sizes[next_size], autodetected_vector_size)) - next_size += 1; + if (mode_i < vector_modes.length () + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), + autodetected_vector_size)) + mode_i += 1; - if (next_size == vector_sizes.length () + if (mode_i == vector_modes.length () || known_eq (autodetected_vector_size, 0U)) { if (first_loop_vinfo) @@ -2423,15 +2424,11 @@ vect_analyze_loop (class loop *loop, loo } /* Try the next biggest vector size. */ - next_vector_size = vector_sizes[next_size++]; + next_vector_mode = vector_modes[mode_i++]; if (dump_enabled_p ()) - { - dump_printf_loc (MSG_NOTE, vect_location, - "***** Re-trying analysis with " - "vector size "); - dump_dec (MSG_NOTE, next_vector_size); - dump_printf (MSG_NOTE, "\n"); - } + dump_printf_loc (MSG_NOTE, vect_location, + "***** Re-trying analysis with vector mode %s\n", + GET_MODE_NAME (next_vector_mode)); } } @@ -8277,9 +8274,9 @@ vect_transform_loop (loop_vec_info loop_ if (epilogue) { - auto_vector_sizes vector_sizes; - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); - unsigned int next_size = 0; + auto_vector_modes vector_modes; + targetm.vectorize.autovectorize_vector_modes (&vector_modes, false); + unsigned int next_i = 0; /* Note LOOP_VINFO_NITERS_KNOWN_P and LOOP_VINFO_INT_NITERS work on niters already ajusted for the iterations of the prologue. */ @@ -8295,18 +8292,20 @@ vect_transform_loop (loop_vec_info loop_ epilogue->any_upper_bound = true; unsigned int ratio; - while (next_size < vector_sizes.length () - && !(constant_multiple_p (loop_vinfo->vector_size, - vector_sizes[next_size], &ratio) + while (next_i < vector_modes.length () + && !(constant_multiple_p + (loop_vinfo->vector_size, + GET_MODE_SIZE (vector_modes[next_i]), &ratio) && eiters >= lowest_vf / ratio)) - next_size += 1; + next_i += 1; } else - while (next_size < vector_sizes.length () - && maybe_lt (loop_vinfo->vector_size, vector_sizes[next_size])) - next_size += 1; + while (next_i < vector_modes.length () + && maybe_lt (loop_vinfo->vector_size, + GET_MODE_SIZE (vector_modes[next_i]))) + next_i += 1; - if (next_size == vector_sizes.length ()) + if (next_i == vector_modes.length ()) epilogue = NULL; } Index: gcc/tree-vect-slp.c =================================================================== --- gcc/tree-vect-slp.c 2019-10-25 13:26:59.141878743 +0100 +++ gcc/tree-vect-slp.c 2019-10-25 13:27:15.525762975 +0100 @@ -3087,12 +3087,12 @@ vect_slp_bb_region (gimple_stmt_iterator unsigned int n_stmts) { bb_vec_info bb_vinfo; - auto_vector_sizes vector_sizes; + auto_vector_modes vector_modes; /* Autodetect first vector size we try. */ - poly_uint64 next_vector_size = 0; - targetm.vectorize.autovectorize_vector_sizes (&vector_sizes, false); - unsigned int next_size = 0; + machine_mode next_vector_mode = VOIDmode; + targetm.vectorize.autovectorize_vector_modes (&vector_modes, false); + unsigned int mode_i = 0; vec_info_shared shared; @@ -3109,7 +3109,7 @@ vect_slp_bb_region (gimple_stmt_iterator bb_vinfo->shared->save_datarefs (); else bb_vinfo->shared->check_datarefs (); - bb_vinfo->vector_size = next_vector_size; + bb_vinfo->vector_size = GET_MODE_SIZE (next_vector_mode); if (vect_slp_analyze_bb_1 (bb_vinfo, n_stmts, fatal) && dbg_cnt (vect_slp)) @@ -3136,17 +3136,18 @@ vect_slp_bb_region (gimple_stmt_iterator vectorized = true; } - if (next_size == 0) + if (mode_i == 0) autodetected_vector_size = bb_vinfo->vector_size; delete bb_vinfo; - if (next_size < vector_sizes.length () - && known_eq (vector_sizes[next_size], autodetected_vector_size)) - next_size += 1; + if (mode_i < vector_modes.length () + && known_eq (GET_MODE_SIZE (vector_modes[mode_i]), + autodetected_vector_size)) + mode_i += 1; if (vectorized - || next_size == vector_sizes.length () + || mode_i == vector_modes.length () || known_eq (autodetected_vector_size, 0U) /* If vect_slp_analyze_bb_1 signaled that analysis for all vector sizes will fail do not bother iterating. */ @@ -3154,15 +3155,11 @@ vect_slp_bb_region (gimple_stmt_iterator return vectorized; /* Try the next biggest vector size. */ - next_vector_size = vector_sizes[next_size++]; + next_vector_mode = vector_modes[mode_i++]; if (dump_enabled_p ()) - { - dump_printf_loc (MSG_NOTE, vect_location, - "***** Re-trying analysis with " - "vector size "); - dump_dec (MSG_NOTE, next_vector_size); - dump_printf (MSG_NOTE, "\n"); - } + dump_printf_loc (MSG_NOTE, vect_location, + "***** Re-trying analysis with vector mode %s\n", + GET_MODE_NAME (next_vector_mode)); } } Index: gcc/config/aarch64/aarch64.c =================================================================== --- gcc/config/aarch64/aarch64.c 2019-10-25 13:26:59.177878488 +0100 +++ gcc/config/aarch64/aarch64.c 2019-10-25 13:27:15.505763118 +0100 @@ -15203,12 +15203,12 @@ aarch64_preferred_simd_mode (scalar_mode /* Return a list of possible vector sizes for the vectorizer to iterate over. */ static void -aarch64_autovectorize_vector_sizes (vector_sizes *sizes, bool) +aarch64_autovectorize_vector_modes (vector_modes *modes, bool) { if (TARGET_SVE) - sizes->safe_push (BYTES_PER_SVE_VECTOR); - sizes->safe_push (16); - sizes->safe_push (8); + modes->safe_push (VNx16QImode); + modes->safe_push (V16QImode); + modes->safe_push (V8QImode); } /* Implement TARGET_MANGLE_TYPE. */ @@ -20915,9 +20915,9 @@ #define TARGET_VECTORIZE_BUILTINS #define TARGET_VECTORIZE_BUILTIN_VECTORIZED_FUNCTION \ aarch64_builtin_vectorized_function -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - aarch64_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + aarch64_autovectorize_vector_modes #undef TARGET_ATOMIC_ASSIGN_EXPAND_FENV #define TARGET_ATOMIC_ASSIGN_EXPAND_FENV \ Index: gcc/config/arc/arc.c =================================================================== --- gcc/config/arc/arc.c 2019-10-25 09:21:25.974346475 +0100 +++ gcc/config/arc/arc.c 2019-10-25 13:27:15.505763118 +0100 @@ -607,15 +607,15 @@ arc_preferred_simd_mode (scalar_mode mod } /* Implements target hook - TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ + TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ static void -arc_autovectorize_vector_sizes (vector_sizes *sizes, bool) +arc_autovectorize_vector_modes (vector_modes *modes, bool) { if (TARGET_PLUS_QMACW) { - sizes->quick_push (8); - sizes->quick_push (4); + modes->quick_push (V4HImode); + modes->quick_push (V2HImode); } } @@ -726,8 +726,8 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arc_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES arc_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES arc_autovectorize_vector_modes #undef TARGET_CAN_USE_DOLOOP_P #define TARGET_CAN_USE_DOLOOP_P arc_can_use_doloop_p Index: gcc/config/arm/arm.c =================================================================== --- gcc/config/arm/arm.c 2019-10-23 11:29:47.933883742 +0100 +++ gcc/config/arm/arm.c 2019-10-25 13:27:15.513763059 +0100 @@ -289,7 +289,7 @@ static bool arm_builtin_support_vector_m static void arm_conditional_register_usage (void); static enum flt_eval_method arm_excess_precision (enum excess_precision_type); static reg_class_t arm_preferred_rename_class (reg_class_t rclass); -static void arm_autovectorize_vector_sizes (vector_sizes *, bool); +static void arm_autovectorize_vector_modes (vector_modes *, bool); static int arm_default_branch_cost (bool, bool); static int arm_cortex_a5_branch_cost (bool, bool); static int arm_cortex_m_branch_cost (bool, bool); @@ -522,9 +522,9 @@ #define TARGET_VECTOR_MODE_SUPPORTED_P a #define TARGET_ARRAY_MODE_SUPPORTED_P arm_array_mode_supported_p #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE arm_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - arm_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + arm_autovectorize_vector_modes #undef TARGET_MACHINE_DEPENDENT_REORG #define TARGET_MACHINE_DEPENDENT_REORG arm_reorg @@ -29012,12 +29012,12 @@ arm_vector_alignment (const_tree type) } static void -arm_autovectorize_vector_sizes (vector_sizes *sizes, bool) +arm_autovectorize_vector_modes (vector_modes *modes, bool) { if (!TARGET_NEON_VECTORIZE_DOUBLE) { - sizes->safe_push (16); - sizes->safe_push (8); + modes->safe_push (V16QImode); + modes->safe_push (V8QImode); } } Index: gcc/config/i386/i386.c =================================================================== --- gcc/config/i386/i386.c 2019-10-25 13:26:59.277877782 +0100 +++ gcc/config/i386/i386.c 2019-10-25 13:27:15.517763031 +0100 @@ -21387,35 +21387,35 @@ ix86_preferred_simd_mode (scalar_mode mo 256bit and 128bit vectors. */ static void -ix86_autovectorize_vector_sizes (vector_sizes *sizes, bool all) +ix86_autovectorize_vector_modes (vector_modes *modes, bool all) { if (TARGET_AVX512F && !TARGET_PREFER_AVX256) { - sizes->safe_push (64); - sizes->safe_push (32); - sizes->safe_push (16); + modes->safe_push (V64QImode); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); } else if (TARGET_AVX512F && all) { - sizes->safe_push (32); - sizes->safe_push (16); - sizes->safe_push (64); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); + modes->safe_push (V64QImode); } else if (TARGET_AVX && !TARGET_PREFER_AVX128) { - sizes->safe_push (32); - sizes->safe_push (16); + modes->safe_push (V32QImode); + modes->safe_push (V16QImode); } else if (TARGET_AVX && all) { - sizes->safe_push (16); - sizes->safe_push (32); + modes->safe_push (V16QImode); + modes->safe_push (V32QImode); } else if (TARGET_MMX_WITH_SSE) - sizes->safe_push (16); + modes->safe_push (V16QImode); if (TARGET_MMX_WITH_SSE) - sizes->safe_push (8); + modes->safe_push (V8QImode); } /* Implemenation of targetm.vectorize.get_mask_mode. */ @@ -22954,9 +22954,9 @@ #define TARGET_VECTORIZE_PREFERRED_SIMD_ #undef TARGET_VECTORIZE_SPLIT_REDUCTION #define TARGET_VECTORIZE_SPLIT_REDUCTION \ ix86_split_reduction -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - ix86_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + ix86_autovectorize_vector_modes #undef TARGET_VECTORIZE_GET_MASK_MODE #define TARGET_VECTORIZE_GET_MASK_MODE ix86_get_mask_mode #undef TARGET_VECTORIZE_INIT_COST Index: gcc/config/mips/mips.c =================================================================== --- gcc/config/mips/mips.c 2019-10-17 14:22:54.903313423 +0100 +++ gcc/config/mips/mips.c 2019-10-25 13:27:15.517763031 +0100 @@ -13453,13 +13453,13 @@ mips_preferred_simd_mode (scalar_mode mo return word_mode; } -/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES. */ +/* Implement TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES. */ static void -mips_autovectorize_vector_sizes (vector_sizes *sizes, bool) +mips_autovectorize_vector_modes (vector_modes *modes, bool) { if (ISA_HAS_MSA) - sizes->safe_push (16); + modes->safe_push (V16QImode); } /* Implement TARGET_INIT_LIBFUNCS. */ @@ -22694,9 +22694,9 @@ #define TARGET_SCALAR_MODE_SUPPORTED_P m #undef TARGET_VECTORIZE_PREFERRED_SIMD_MODE #define TARGET_VECTORIZE_PREFERRED_SIMD_MODE mips_preferred_simd_mode -#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES -#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_SIZES \ - mips_autovectorize_vector_sizes +#undef TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES +#define TARGET_VECTORIZE_AUTOVECTORIZE_VECTOR_MODES \ + mips_autovectorize_vector_modes #undef TARGET_INIT_BUILTINS #define TARGET_INIT_BUILTINS mips_init_builtins