| Message ID | mptilln4uo0.fsf@arm.com |
|---|---|
| State | New |
| Headers | show |
| Series | vect: Fix missed gather load opportunity | expand |
On Fri, 16 Sep 2022, Richard Sandiford wrote: > While writing a testcase for PR106794, I noticed that we failed > to vectorise the testcase in the patch for SVE. The code that > recognises gather loads tries to optimise the point at which > the offset is calculated, to avoid unnecessary extensions or > truncations: > > /* Don't include the conversion if the target is happy with > the current offset type. */ > > But breaking only makes sense if we're at an SSA_NAME (which could > then be vectorised). We shouldn't break on a conversion embedded > in a generic expression. > > Tested on aarch64-linux-gnu & x86_64-linux-gnu. OK to install? OK, Thanks, Richard. > Richard > > > gcc/ > * tree-vect-data-refs.cc (vect_check_gather_scatter): Restrict > early-out optimisation to SSA_NAMEs. > > gcc/testsuite/ > * gcc.dg/vect/vect-gather-5.c: New test. > --- > gcc/testsuite/gcc.dg/vect/vect-gather-5.c | 42 +++++++++++++++++++++++ > gcc/tree-vect-data-refs.cc | 1 + > 2 files changed, 43 insertions(+) > create mode 100644 gcc/testsuite/gcc.dg/vect/vect-gather-5.c > > diff --git a/gcc/testsuite/gcc.dg/vect/vect-gather-5.c b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c > new file mode 100644 > index 00000000000..8b5074bba88 > --- /dev/null > +++ b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c > @@ -0,0 +1,42 @@ > +/* { dg-do compile } */ > +/* { dg-additional-options "-O3" } */ > + > +#ifdef __aarch64__ > +#pragma GCC target "+sve" > +#endif > + > +long a[100], b[100], c[100]; > + > +void g1 () > +{ > + for (int i = 0; i < 100; i += 2) > + { > + c[i] += a[b[i]] + 1; > + c[i + 1] += a[b[i + 1]] + 2; > + } > +} > + > +long g2 () > +{ > + long res = 0; > + for (int i = 0; i < 100; i += 2) > + { > + res += a[b[i + 1]]; > + res += a[b[i]]; > + } > + return res; > +} > + > +long g3 () > +{ > + long res = 0; > + for (int i = 0; i < 100; i += 2) > + { > + res += a[b[i]]; > + res += a[b[i + 1]]; > + } > + return res; > +} > + > +/* { dg-final { scan-tree-dump-times {add new stmt[^\n]*GATHER_LOAD} 3 "vect" { target aarch64*-*-* } } } */ > +/* { dg-final { scan-tree-dump-not {add new stmt[^\n]*VEC_PERM_EXPR} "vect" { target aarch64*-*-* } } } */ > diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc > index b279a82551e..e03b50498d1 100644 > --- a/gcc/tree-vect-data-refs.cc > +++ b/gcc/tree-vect-data-refs.cc > @@ -4151,6 +4151,7 @@ vect_check_gather_scatter (stmt_vec_info stmt_info, loop_vec_info loop_vinfo, > /* Don't include the conversion if the target is happy with > the current offset type. */ > if (use_ifn_p > + && TREE_CODE (off) == SSA_NAME > && !POINTER_TYPE_P (TREE_TYPE (off)) > && vect_gather_scatter_fn_p (loop_vinfo, DR_IS_READ (dr), > masked_p, vectype, memory_type, > -- > 2.25.1 > >
diff --git a/gcc/testsuite/gcc.dg/vect/vect-gather-5.c b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c new file mode 100644 index 00000000000..8b5074bba88 --- /dev/null +++ b/gcc/testsuite/gcc.dg/vect/vect-gather-5.c @@ -0,0 +1,42 @@ +/* { dg-do compile } */ +/* { dg-additional-options "-O3" } */ + +#ifdef __aarch64__ +#pragma GCC target "+sve" +#endif + +long a[100], b[100], c[100]; + +void g1 () +{ + for (int i = 0; i < 100; i += 2) + { + c[i] += a[b[i]] + 1; + c[i + 1] += a[b[i + 1]] + 2; + } +} + +long g2 () +{ + long res = 0; + for (int i = 0; i < 100; i += 2) + { + res += a[b[i + 1]]; + res += a[b[i]]; + } + return res; +} + +long g3 () +{ + long res = 0; + for (int i = 0; i < 100; i += 2) + { + res += a[b[i]]; + res += a[b[i + 1]]; + } + return res; +} + +/* { dg-final { scan-tree-dump-times {add new stmt[^\n]*GATHER_LOAD} 3 "vect" { target aarch64*-*-* } } } */ +/* { dg-final { scan-tree-dump-not {add new stmt[^\n]*VEC_PERM_EXPR} "vect" { target aarch64*-*-* } } } */ diff --git a/gcc/tree-vect-data-refs.cc b/gcc/tree-vect-data-refs.cc index b279a82551e..e03b50498d1 100644 --- a/gcc/tree-vect-data-refs.cc +++ b/gcc/tree-vect-data-refs.cc @@ -4151,6 +4151,7 @@ vect_check_gather_scatter (stmt_vec_info stmt_info, loop_vec_info loop_vinfo, /* Don't include the conversion if the target is happy with the current offset type. */ if (use_ifn_p + && TREE_CODE (off) == SSA_NAME && !POINTER_TYPE_P (TREE_TYPE (off)) && vect_gather_scatter_fn_p (loop_vinfo, DR_IS_READ (dr), masked_p, vectype, memory_type,