| Message ID | f63f904c-910a-58ea-33b8-24f0db35b007@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | issue consistent warning for past-the-end array stores (PR 91457) | expand |
On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: > > With the recent enhancement to the strlen handling of multibyte > stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays > started failing on hppa (and probably elsewhere as well). This > is partly the result of the added detection of past-the-end > writes into the strlen pass which detects more instances of > the problem than -Warray-bounds. Since the IL each warning > works with varies between targets, the same invalid code can > be diagnosed by one warning one target and different warning > on another. > > The attached patch does three things: > > 1) It enhances compute_objsize to also determine the size of > a flexible array member (and its various variants), including > from its initializer if necessary. (This resolves 91457 but > introduces another warning where was previously just one.) > 2) It guards the new instance of -Wstringop-overflow with > the no-warning bit on the assignment to avoid warning on code > that's already been diagnosed. > 3) It arranges for -Warray-bounds to set the no-warning bit on > the enclosing expression to keep -Wstringop-overflow from issuing > another warning for the same problem. > > Testing the compute_objsize enhancement to bring it up to par > with -Warray-bounds in turn exposed a weakness in the latter > warning for flexible array members. Rather than snowballing > additional improvements into this one I decided to put that > off until later, so the new -Warray-bounds test has a bunch > of XFAILs. I'll see if I can find the time to deal with those > either still in stage 1 or in stage 3 (one of them is actually > an ancient regression). +static tree +get_initializer_for (tree init, tree decl) +{ can't you use fold_ctor_reference here? +/* Determine the size of the flexible array FLD from the initializer + expression for the struct object DECL in which the meber is declared + (possibly recursively). Return the size or zero constant if it isn't + initialized. */ + +static tree +get_flexarray_size (tree decl, tree fld) +{ + if (tree init = DECL_INITIAL (decl)) + { + init = get_initializer_for (init, fld); + if (init) + return TYPE_SIZE_UNIT (TREE_TYPE (init)); + } + + return integer_zero_node; so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for returns has a complete type but the initialized object didn't get it completed. Isnt that wishful thinking? And why return integer_zero_node rather than NULL_TREE here? + if (TREE_CODE (dest) == COMPONENT_REF) + { + *pdecl = TREE_OPERAND (dest, 1); + + /* If the member has a size return it. Otherwise it's a flexible + array member. */ + if (tree size = DECL_SIZE_UNIT (*pdecl)) + return size; because here you do. Also once you have an underlying VAR_DECL you can compute the flexarray size by DECL_SIZE (var) - offset-of flexarray member. Isn't that way cheaper than walking the initializer (possibly many times?) Richard. > > Martin > > PS I imagine the new get_flexarray_size function will probably > need to move somewhere more appropriate so that other warnings > (like -Warray-bounds to remove the XFAILs) and optimizations > can make use of it.
On 8/19/19 8:10 AM, Richard Biener wrote: > On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: >> >> With the recent enhancement to the strlen handling of multibyte >> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays >> started failing on hppa (and probably elsewhere as well). This >> is partly the result of the added detection of past-the-end >> writes into the strlen pass which detects more instances of >> the problem than -Warray-bounds. Since the IL each warning >> works with varies between targets, the same invalid code can >> be diagnosed by one warning one target and different warning >> on another. >> >> The attached patch does three things: >> >> 1) It enhances compute_objsize to also determine the size of >> a flexible array member (and its various variants), including >> from its initializer if necessary. (This resolves 91457 but >> introduces another warning where was previously just one.) >> 2) It guards the new instance of -Wstringop-overflow with >> the no-warning bit on the assignment to avoid warning on code >> that's already been diagnosed. >> 3) It arranges for -Warray-bounds to set the no-warning bit on >> the enclosing expression to keep -Wstringop-overflow from issuing >> another warning for the same problem. >> >> Testing the compute_objsize enhancement to bring it up to par >> with -Warray-bounds in turn exposed a weakness in the latter >> warning for flexible array members. Rather than snowballing >> additional improvements into this one I decided to put that >> off until later, so the new -Warray-bounds test has a bunch >> of XFAILs. I'll see if I can find the time to deal with those >> either still in stage 1 or in stage 3 (one of them is actually >> an ancient regression). > > +static tree > +get_initializer_for (tree init, tree decl) > +{ > > can't you use fold_ctor_reference here? Yes, but only with an additional enhancement. Char initializers for flexible array members aren't transformed to STRING_CSTs yet, so without the size of the initializer specified, the function returns the initializer for the smallest subobject, or char in this case. I've enhanced the function to handle them. > > +/* Determine the size of the flexible array FLD from the initializer > + expression for the struct object DECL in which the meber is declared > + (possibly recursively). Return the size or zero constant if it isn't > + initialized. */ > + > +static tree > +get_flexarray_size (tree decl, tree fld) > +{ > + if (tree init = DECL_INITIAL (decl)) > + { > + init = get_initializer_for (init, fld); > + if (init) > + return TYPE_SIZE_UNIT (TREE_TYPE (init)); > + } > + > + return integer_zero_node; > > so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for > returns has a complete type but the initialized object didn't get it > completed. Isnt that wishful thinking? I don't know what you mean. When might a CONSTRUCTOR not have a complete type, and if/when it doesn't, why would that be a problem here? TYPE_SIZE_UNIT will evaluate to null meaning "don't know" and that's fine. Could you try to be more specific about the problem you're pointing out? > And why return integer_zero_node > rather than NULL_TREE here? Because the size of a flexible array member with no initializer is zero. > > + if (TREE_CODE (dest) == COMPONENT_REF) > + { > + *pdecl = TREE_OPERAND (dest, 1); > + > + /* If the member has a size return it. Otherwise it's a flexible > + array member. */ > + if (tree size = DECL_SIZE_UNIT (*pdecl)) > + return size; > > because here you do. Not sure what you mean here either. (This code was also a bit out of date WRT to the patch I had tested. Not sure how that happened. The attached patch is up to date.) > > Also once you have an underlying VAR_DECL you can compute > the flexarray size by DECL_SIZE (var) - offset-of flexarray member. > Isn't that way cheaper than walking the initializer (possibly many > times?) It would be nice if it were this easy. Is the value of DECL_SIZE (var) supposed to include the size of the flexible array member? I don't see it mentioned in the comments in tree.h and in my tests it only does in C but not in C++. Is that a bug that in C++ it doesn't? Attached is an updated patch that uses fold_ctor_reference as you suggested. I've also made a few other minor changes to diagnose a few more invalid strlen calls with out-of-bounds offsets. (More still remain.) Martin > > Richard. > >> >> Martin >> >> PS I imagine the new get_flexarray_size function will probably >> need to move somewhere more appropriate so that other warnings >> (like -Warray-bounds to remove the XFAILs) and optimizations >> can make use of it.
On Tue, Aug 20, 2019 at 4:32 AM Martin Sebor <msebor@gmail.com> wrote: > > On 8/19/19 8:10 AM, Richard Biener wrote: > > On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: > >> > >> With the recent enhancement to the strlen handling of multibyte > >> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays > >> started failing on hppa (and probably elsewhere as well). This > >> is partly the result of the added detection of past-the-end > >> writes into the strlen pass which detects more instances of > >> the problem than -Warray-bounds. Since the IL each warning > >> works with varies between targets, the same invalid code can > >> be diagnosed by one warning one target and different warning > >> on another. > >> > >> The attached patch does three things: > >> > >> 1) It enhances compute_objsize to also determine the size of > >> a flexible array member (and its various variants), including > >> from its initializer if necessary. (This resolves 91457 but > >> introduces another warning where was previously just one.) > >> 2) It guards the new instance of -Wstringop-overflow with > >> the no-warning bit on the assignment to avoid warning on code > >> that's already been diagnosed. > >> 3) It arranges for -Warray-bounds to set the no-warning bit on > >> the enclosing expression to keep -Wstringop-overflow from issuing > >> another warning for the same problem. > >> > >> Testing the compute_objsize enhancement to bring it up to par > >> with -Warray-bounds in turn exposed a weakness in the latter > >> warning for flexible array members. Rather than snowballing > >> additional improvements into this one I decided to put that > >> off until later, so the new -Warray-bounds test has a bunch > >> of XFAILs. I'll see if I can find the time to deal with those > >> either still in stage 1 or in stage 3 (one of them is actually > >> an ancient regression). > > > > +static tree > > +get_initializer_for (tree init, tree decl) > > +{ > > > > can't you use fold_ctor_reference here? > > Yes, but only with an additional enhancement. Char initializers > for flexible array members aren't transformed to STRING_CSTs yet, > so without the size of the initializer specified, the function > returns the initializer for the smallest subobject, or char in > this case. I've enhanced the function to handle them. So at the moment it returns an empty array constructor, correct? Isn't that the more canonical representation? The STRING_CST index type doesn't really handle "empty" strings and I expect code more confused about that than about an empty CTOR? > > > > +/* Determine the size of the flexible array FLD from the initializer > > + expression for the struct object DECL in which the meber is declared > > + (possibly recursively). Return the size or zero constant if it isn't > > + initialized. */ > > + > > +static tree > > +get_flexarray_size (tree decl, tree fld) > > +{ > > + if (tree init = DECL_INITIAL (decl)) > > + { > > + init = get_initializer_for (init, fld); > > + if (init) > > + return TYPE_SIZE_UNIT (TREE_TYPE (init)); > > + } > > + > > + return integer_zero_node; > > > > so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for > > returns has a complete type but the initialized object didn't get it > > completed. Isnt that wishful thinking? > > I don't know what you mean. When might a CONSTRUCTOR not have > a complete type, and if/when it doesn't, why would that be > a problem here? TYPE_SIZE_UNIT will evaluate to null meaning > "don't know" and that's fine. Could you try to be more specific > about the problem you're pointing out? > > > And why return integer_zero_node > > rather than NULL_TREE here? > > Because the size of a flexible array member with no initializer > is zero. > > > > > + if (TREE_CODE (dest) == COMPONENT_REF) > > + { > > + *pdecl = TREE_OPERAND (dest, 1); > > + > > + /* If the member has a size return it. Otherwise it's a flexible > > + array member. */ > > + if (tree size = DECL_SIZE_UNIT (*pdecl)) > > + return size; > > > > because here you do. > > Not sure what you mean here either. (This code was also a bit You return NULL_TREE. > out of date WRT to the patch I had tested. Not sure how that > happened. The attached patch is up to date.) > > > > > Also once you have an underlying VAR_DECL you can compute > > the flexarray size by DECL_SIZE (var) - offset-of flexarray member. > > Isn't that way cheaper than walking the initializer (possibly many > > times?) > > It would be nice if it were this easy. Is the value of DECL_SIZE > (var) supposed to include the size of the flexible array member? Yes, DECL_SIZE of the VAR_DECL is the size we use for assembling. It is usually only the types that remain incomplete (or too small) since the FE doesn't create many variants of a struct S { int n; char x[]; } when "instantiating" it via struct S s = { 5, "abcde" }; that then holds true for the DECL_SIZE of the FIELD_DECL of x as well. > I don't see it mentioned in the comments in tree.h and in my tests > it only does in C but not in C++. Is that a bug that in C++ it > doesn't? tree.h dcoumentation isn't complete. And for me the C++ FE for the above simple example has <var_decl 0x7ffff7fedb40 s type <record_type 0x7ffff6d94f18 S cxx-odr-p type_5 type_6 BLK size <integer_cst 0x7ffff6c5e0a8 constant 32> unit-size <integer_cst 0x7ffff6c5e0c0 constant 4> align:32 warn_if_not_align:0 symtab:0 alias-set -1 canonical-type 0x7ffff6d94f18 fields <function_decl 0x7ffff6d98b00 __dt type <method_type 0x7ffff6da8150> public abstract external autoinline decl_3 QI t.ii:1:8 align:16 warn_if_not_align:0 context <record_type 0x7ffff6d94f18 S> full-name "S::~S() noexcept (<uninstantiated>)" not-really-extern chain <function_decl 0x7ffff6d98d00 __dt_base >> context <translation_unit_decl 0x7ffff6c4a168 t.ii> full-name "struct S" X() X(constX&) this=(X&) n_parents=0 use_template=0 interface-unknown pointer_to_this <pointer_type 0x7ffff6da85e8> reference_to_this <reference_type 0x7ffff6da8a80> chain <type_decl 0x7ffff6c6b850 S>> public static tree_1 tree_2 tree_3 BLK t.ii:3:3 size <integer_cst 0x7ffff6c5e0a8 32> unit-size <integer_cst 0x7ffff6c5e0c0 4> align:32 warn_if_not_align:0 context <translation_unit_decl 0x7ffff6c4a168 t.ii> initial <constructor 0x7ffff6da52a0> chain <type_decl 0x7ffff6c6b850 S>> and we assemble it like .globl s .data .align 4 .type s, @object .size s, 4 s: .long 5 .string "abcde" note how we have .size s, 4 here... of course in the end nobody cares about a symbols size ...(?) I expected layout_decl to fixup DECL_SIZE according to the iniitalizer but it looks like the initializer isn't present when it is layouted. I think this is a middle-end flaw we should try to fix that either in the FEs or in the middle-end. The C FE calls add_flexible_array_elts_to_size and complete_flexible_array_elts in its finish_decl routine (layouting happens at build_decl time, too early, but if we want to move this functionality then re-layouting might be posible). I'm not sure about other FEs but I guess the easiest point we could do sanity checking is when assemble_variable outputs the constructor. > Attached is an updated patch that uses fold_ctor_reference as you > suggested. I've also made a few other minor changes to diagnose > a few more invalid strlen calls with out-of-bounds offsets. > (More still remain.) Thanks, but the patch was already large ... it would be nice to strip it down to the bare bones again and do followup adjustments instead. Richard. > Martin > > > > > Richard. > > > >> > >> Martin > >> > >> PS I imagine the new get_flexarray_size function will probably > >> need to move somewhere more appropriate so that other warnings > >> (like -Warray-bounds to remove the XFAILs) and optimizations > >> can make use of it. >
On 8/20/19 1:26 AM, Richard Biener wrote: > On Tue, Aug 20, 2019 at 4:32 AM Martin Sebor <msebor@gmail.com> wrote: >> >> On 8/19/19 8:10 AM, Richard Biener wrote: >>> On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: >>>> >>>> With the recent enhancement to the strlen handling of multibyte >>>> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays >>>> started failing on hppa (and probably elsewhere as well). This >>>> is partly the result of the added detection of past-the-end >>>> writes into the strlen pass which detects more instances of >>>> the problem than -Warray-bounds. Since the IL each warning >>>> works with varies between targets, the same invalid code can >>>> be diagnosed by one warning one target and different warning >>>> on another. >>>> >>>> The attached patch does three things: >>>> >>>> 1) It enhances compute_objsize to also determine the size of >>>> a flexible array member (and its various variants), including >>>> from its initializer if necessary. (This resolves 91457 but >>>> introduces another warning where was previously just one.) >>>> 2) It guards the new instance of -Wstringop-overflow with >>>> the no-warning bit on the assignment to avoid warning on code >>>> that's already been diagnosed. >>>> 3) It arranges for -Warray-bounds to set the no-warning bit on >>>> the enclosing expression to keep -Wstringop-overflow from issuing >>>> another warning for the same problem. >>>> >>>> Testing the compute_objsize enhancement to bring it up to par >>>> with -Warray-bounds in turn exposed a weakness in the latter >>>> warning for flexible array members. Rather than snowballing >>>> additional improvements into this one I decided to put that >>>> off until later, so the new -Warray-bounds test has a bunch >>>> of XFAILs. I'll see if I can find the time to deal with those >>>> either still in stage 1 or in stage 3 (one of them is actually >>>> an ancient regression). >>> >>> +static tree >>> +get_initializer_for (tree init, tree decl) >>> +{ >>> >>> can't you use fold_ctor_reference here? >> >> Yes, but only with an additional enhancement. Char initializers >> for flexible array members aren't transformed to STRING_CSTs yet, >> so without the size of the initializer specified, the function >> returns the initializer for the smallest subobject, or char in >> this case. I've enhanced the function to handle them. > > So at the moment it returns an empty array constructor, correct? > Isn't that the more canonical representation? The STRING_CST > index type doesn't really handle "empty" strings and I expect code > more confused about that than about an empty CTOR? Yes. Returning an empty CTOR is more general than an empty string and enables more optimizations. It requires changing the caller(s) that look for a string but I think that's fine. Thanks for the hint! >>> +/* Determine the size of the flexible array FLD from the initializer >>> + expression for the struct object DECL in which the meber is declared >>> + (possibly recursively). Return the size or zero constant if it isn't >>> + initialized. */ >>> + >>> +static tree >>> +get_flexarray_size (tree decl, tree fld) >>> +{ >>> + if (tree init = DECL_INITIAL (decl)) >>> + { >>> + init = get_initializer_for (init, fld); >>> + if (init) >>> + return TYPE_SIZE_UNIT (TREE_TYPE (init)); >>> + } >>> + >>> + return integer_zero_node; >>> >>> so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for >>> returns has a complete type but the initialized object didn't get it >>> completed. Isnt that wishful thinking? >> >> I don't know what you mean. When might a CONSTRUCTOR not have >> a complete type, and if/when it doesn't, why would that be >> a problem here? TYPE_SIZE_UNIT will evaluate to null meaning >> "don't know" and that's fine. Could you try to be more specific >> about the problem you're pointing out? >> >>> And why return integer_zero_node >>> rather than NULL_TREE here? >> >> Because the size of a flexible array member with no initializer >> is zero. >> >>> >>> + if (TREE_CODE (dest) == COMPONENT_REF) >>> + { >>> + *pdecl = TREE_OPERAND (dest, 1); >>> + >>> + /* If the member has a size return it. Otherwise it's a flexible >>> + array member. */ >>> + if (tree size = DECL_SIZE_UNIT (*pdecl)) >>> + return size; >>> >>> because here you do. >> >> Not sure what you mean here either. (This code was also a bit > > You return NULL_TREE. > >> out of date WRT to the patch I had tested. Not sure how that >> happened. The attached patch is up to date.) >> >>> >>> Also once you have an underlying VAR_DECL you can compute >>> the flexarray size by DECL_SIZE (var) - offset-of flexarray member. >>> Isn't that way cheaper than walking the initializer (possibly many >>> times?) >> >> It would be nice if it were this easy. Is the value of DECL_SIZE >> (var) supposed to include the size of the flexible array member? > > Yes, DECL_SIZE of the VAR_DECL is the size we use for assembling. > It is usually only the types that remain incomplete (or too small) since > the FE doesn't create many variants of a struct S { int n; char x[]; } > when "instantiating" it via struct S s = { 5, "abcde" }; that then holds > true for the DECL_SIZE of the FIELD_DECL of x as well. > >> I don't see it mentioned in the comments in tree.h and in my tests >> it only does in C but not in C++. Is that a bug that in C++ it >> doesn't? > > tree.h dcoumentation isn't complete. And for me the C++ FE for > the above simple example has > > <var_decl 0x7ffff7fedb40 s > type <record_type 0x7ffff6d94f18 S cxx-odr-p type_5 type_6 BLK > size <integer_cst 0x7ffff6c5e0a8 constant 32> > unit-size <integer_cst 0x7ffff6c5e0c0 constant 4> > align:32 warn_if_not_align:0 symtab:0 alias-set -1 > canonical-type 0x7ffff6d94f18 > fields <function_decl 0x7ffff6d98b00 __dt type <method_type > 0x7ffff6da8150> > public abstract external autoinline decl_3 QI t.ii:1:8 > align:16 warn_if_not_align:0 context <record_type 0x7ffff6d94f18 S> > full-name "S::~S() noexcept (<uninstantiated>)" > not-really-extern chain <function_decl 0x7ffff6d98d00 > __dt_base >> context <translation_unit_decl 0x7ffff6c4a168 t.ii> > full-name "struct S" > X() X(constX&) this=(X&) n_parents=0 use_template=0 interface-unknown > pointer_to_this <pointer_type 0x7ffff6da85e8> > reference_to_this <reference_type 0x7ffff6da8a80> chain <type_decl > 0x7ffff6c6b850 S>> > public static tree_1 tree_2 tree_3 BLK t.ii:3:3 size <integer_cst > 0x7ffff6c5e0a8 32> unit-size <integer_cst 0x7ffff6c5e0c0 4> > align:32 warn_if_not_align:0 context <translation_unit_decl > 0x7ffff6c4a168 t.ii> initial <constructor 0x7ffff6da52a0> chain > <type_decl 0x7ffff6c6b850 S>> > > and we assemble it like > > .globl s > .data > .align 4 > .type s, @object > .size s, 4 > s: > .long 5 > .string "abcde" > > note how we have .size s, 4 here... > > of course in the end nobody cares about a symbols size ...(?) > > I expected layout_decl to fixup DECL_SIZE according to the iniitalizer > but it looks like the initializer isn't present when it is layouted. > > I think this is a middle-end flaw we should try to fix that either in > the FEs or in the middle-end. The C FE calls add_flexible_array_elts_to_size > and complete_flexible_array_elts in its finish_decl routine (layouting happens > at build_decl time, too early, but if we want to move this functionality then > re-layouting might be posible). I'm not sure about other FEs but I guess > the easiest point we could do sanity checking is when assemble_variable > outputs the constructor. I don't know if I followed all this but it sounds like you're saying this is a separate issue. > >> Attached is an updated patch that uses fold_ctor_reference as you >> suggested. I've also made a few other minor changes to diagnose >> a few more invalid strlen calls with out-of-bounds offsets. >> (More still remain.) > > Thanks, but the patch was already large ... it would be nice to strip it > down to the bare bones again and do followup adjustments instead. I'm not sure what you view as the bare bones bits but I took a guess and split off the changes for PR 91490 (missing folding of constant flexible array members) into a separate patch and posted that. Once that's approved I'll update this one and post what's left of it. Martin > > Richard. > >> Martin >> >>> >>> Richard. >>> >>>> >>>> Martin >>>> >>>> PS I imagine the new get_flexarray_size function will probably >>>> need to move somewhere more appropriate so that other warnings >>>> (like -Warray-bounds to remove the XFAILs) and optimizations >>>> can make use of it. >>
On Wed, Aug 21, 2019 at 10:10 PM Martin Sebor <msebor@gmail.com> wrote: > > On 8/20/19 1:26 AM, Richard Biener wrote: > > On Tue, Aug 20, 2019 at 4:32 AM Martin Sebor <msebor@gmail.com> wrote: > >> > >> On 8/19/19 8:10 AM, Richard Biener wrote: > >>> On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: > >>>> > >>>> With the recent enhancement to the strlen handling of multibyte > >>>> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays > >>>> started failing on hppa (and probably elsewhere as well). This > >>>> is partly the result of the added detection of past-the-end > >>>> writes into the strlen pass which detects more instances of > >>>> the problem than -Warray-bounds. Since the IL each warning > >>>> works with varies between targets, the same invalid code can > >>>> be diagnosed by one warning one target and different warning > >>>> on another. > >>>> > >>>> The attached patch does three things: > >>>> > >>>> 1) It enhances compute_objsize to also determine the size of > >>>> a flexible array member (and its various variants), including > >>>> from its initializer if necessary. (This resolves 91457 but > >>>> introduces another warning where was previously just one.) > >>>> 2) It guards the new instance of -Wstringop-overflow with > >>>> the no-warning bit on the assignment to avoid warning on code > >>>> that's already been diagnosed. > >>>> 3) It arranges for -Warray-bounds to set the no-warning bit on > >>>> the enclosing expression to keep -Wstringop-overflow from issuing > >>>> another warning for the same problem. > >>>> > >>>> Testing the compute_objsize enhancement to bring it up to par > >>>> with -Warray-bounds in turn exposed a weakness in the latter > >>>> warning for flexible array members. Rather than snowballing > >>>> additional improvements into this one I decided to put that > >>>> off until later, so the new -Warray-bounds test has a bunch > >>>> of XFAILs. I'll see if I can find the time to deal with those > >>>> either still in stage 1 or in stage 3 (one of them is actually > >>>> an ancient regression). > >>> > >>> +static tree > >>> +get_initializer_for (tree init, tree decl) > >>> +{ > >>> > >>> can't you use fold_ctor_reference here? > >> > >> Yes, but only with an additional enhancement. Char initializers > >> for flexible array members aren't transformed to STRING_CSTs yet, > >> so without the size of the initializer specified, the function > >> returns the initializer for the smallest subobject, or char in > >> this case. I've enhanced the function to handle them. > > > > So at the moment it returns an empty array constructor, correct? > > Isn't that the more canonical representation? The STRING_CST > > index type doesn't really handle "empty" strings and I expect code > > more confused about that than about an empty CTOR? > > Yes. Returning an empty CTOR is more general than an empty > string and enables more optimizations. It requires changing > the caller(s) that look for a string but I think that's fine. > Thanks for the hint! > > >>> +/* Determine the size of the flexible array FLD from the initializer > >>> + expression for the struct object DECL in which the meber is declared > >>> + (possibly recursively). Return the size or zero constant if it isn't > >>> + initialized. */ > >>> + > >>> +static tree > >>> +get_flexarray_size (tree decl, tree fld) > >>> +{ > >>> + if (tree init = DECL_INITIAL (decl)) > >>> + { > >>> + init = get_initializer_for (init, fld); > >>> + if (init) > >>> + return TYPE_SIZE_UNIT (TREE_TYPE (init)); > >>> + } > >>> + > >>> + return integer_zero_node; > >>> > >>> so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for > >>> returns has a complete type but the initialized object didn't get it > >>> completed. Isnt that wishful thinking? > >> > >> I don't know what you mean. When might a CONSTRUCTOR not have > >> a complete type, and if/when it doesn't, why would that be > >> a problem here? TYPE_SIZE_UNIT will evaluate to null meaning > >> "don't know" and that's fine. Could you try to be more specific > >> about the problem you're pointing out? > >> > >>> And why return integer_zero_node > >>> rather than NULL_TREE here? > >> > >> Because the size of a flexible array member with no initializer > >> is zero. > >> > >>> > >>> + if (TREE_CODE (dest) == COMPONENT_REF) > >>> + { > >>> + *pdecl = TREE_OPERAND (dest, 1); > >>> + > >>> + /* If the member has a size return it. Otherwise it's a flexible > >>> + array member. */ > >>> + if (tree size = DECL_SIZE_UNIT (*pdecl)) > >>> + return size; > >>> > >>> because here you do. > >> > >> Not sure what you mean here either. (This code was also a bit > > > > You return NULL_TREE. > > > >> out of date WRT to the patch I had tested. Not sure how that > >> happened. The attached patch is up to date.) > >> > >>> > >>> Also once you have an underlying VAR_DECL you can compute > >>> the flexarray size by DECL_SIZE (var) - offset-of flexarray member. > >>> Isn't that way cheaper than walking the initializer (possibly many > >>> times?) > >> > >> It would be nice if it were this easy. Is the value of DECL_SIZE > >> (var) supposed to include the size of the flexible array member? > > > > Yes, DECL_SIZE of the VAR_DECL is the size we use for assembling. > > It is usually only the types that remain incomplete (or too small) since > > the FE doesn't create many variants of a struct S { int n; char x[]; } > > when "instantiating" it via struct S s = { 5, "abcde" }; that then holds > > true for the DECL_SIZE of the FIELD_DECL of x as well. > > > >> I don't see it mentioned in the comments in tree.h and in my tests > >> it only does in C but not in C++. Is that a bug that in C++ it > >> doesn't? > > > > tree.h dcoumentation isn't complete. And for me the C++ FE for > > the above simple example has > > > > <var_decl 0x7ffff7fedb40 s > > type <record_type 0x7ffff6d94f18 S cxx-odr-p type_5 type_6 BLK > > size <integer_cst 0x7ffff6c5e0a8 constant 32> > > unit-size <integer_cst 0x7ffff6c5e0c0 constant 4> > > align:32 warn_if_not_align:0 symtab:0 alias-set -1 > > canonical-type 0x7ffff6d94f18 > > fields <function_decl 0x7ffff6d98b00 __dt type <method_type > > 0x7ffff6da8150> > > public abstract external autoinline decl_3 QI t.ii:1:8 > > align:16 warn_if_not_align:0 context <record_type 0x7ffff6d94f18 S> > > full-name "S::~S() noexcept (<uninstantiated>)" > > not-really-extern chain <function_decl 0x7ffff6d98d00 > > __dt_base >> context <translation_unit_decl 0x7ffff6c4a168 t.ii> > > full-name "struct S" > > X() X(constX&) this=(X&) n_parents=0 use_template=0 interface-unknown > > pointer_to_this <pointer_type 0x7ffff6da85e8> > > reference_to_this <reference_type 0x7ffff6da8a80> chain <type_decl > > 0x7ffff6c6b850 S>> > > public static tree_1 tree_2 tree_3 BLK t.ii:3:3 size <integer_cst > > 0x7ffff6c5e0a8 32> unit-size <integer_cst 0x7ffff6c5e0c0 4> > > align:32 warn_if_not_align:0 context <translation_unit_decl > > 0x7ffff6c4a168 t.ii> initial <constructor 0x7ffff6da52a0> chain > > <type_decl 0x7ffff6c6b850 S>> > > > > and we assemble it like > > > > .globl s > > .data > > .align 4 > > .type s, @object > > .size s, 4 > > s: > > .long 5 > > .string "abcde" > > > > note how we have .size s, 4 here... > > > > of course in the end nobody cares about a symbols size ...(?) > > > > I expected layout_decl to fixup DECL_SIZE according to the iniitalizer > > but it looks like the initializer isn't present when it is layouted. > > > > I think this is a middle-end flaw we should try to fix that either in > > the FEs or in the middle-end. The C FE calls add_flexible_array_elts_to_size > > and complete_flexible_array_elts in its finish_decl routine (layouting happens > > at build_decl time, too early, but if we want to move this functionality then > > re-layouting might be posible). I'm not sure about other FEs but I guess > > the easiest point we could do sanity checking is when assemble_variable > > outputs the constructor. > > I don't know if I followed all this but it sounds like you're > saying this is a separate issue. Yeah, it looks like a separate issue to me but one that would be nice to have fixed so we can compute the length of the trailing array without needing to parse the initializer. I'd even say we should delay that effort if the above is confirmed as a bug and a fix is possible. Jason, is the above intended behavior? I realize "trailing" arrays is a GNU extension in C++ but as seen above it doesn't match GNU C behavior here. As said, I'm not sure if a wrong symbol size has an impact on any target binary format we support. > > > >> Attached is an updated patch that uses fold_ctor_reference as you > >> suggested. I've also made a few other minor changes to diagnose > >> a few more invalid strlen calls with out-of-bounds offsets. > >> (More still remain.) > > > > Thanks, but the patch was already large ... it would be nice to strip it > > down to the bare bones again and do followup adjustments instead. > > I'm not sure what you view as the bare bones bits but I took > a guess and split off the changes for PR 91490 (missing folding > of constant flexible array members) into a separate patch and > posted that. Once that's approved I'll update this one and post > what's left of it. Thanks, Richard. > Martin > > > > > Richard. > > > >> Martin > >> > >>> > >>> Richard. > >>> > >>>> > >>>> Martin > >>>> > >>>> PS I imagine the new get_flexarray_size function will probably > >>>> need to move somewhere more appropriate so that other warnings > >>>> (like -Warray-bounds to remove the XFAILs) and optimizations > >>>> can make use of it. > >> >
On 8/20/19 1:26 AM, Richard Biener wrote: > On Tue, Aug 20, 2019 at 4:32 AM Martin Sebor <msebor@gmail.com> wrote: >> >> On 8/19/19 8:10 AM, Richard Biener wrote: >>> On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: >>>> >>>> With the recent enhancement to the strlen handling of multibyte >>>> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays >>>> started failing on hppa (and probably elsewhere as well). This >>>> is partly the result of the added detection of past-the-end >>>> writes into the strlen pass which detects more instances of >>>> the problem than -Warray-bounds. Since the IL each warning >>>> works with varies between targets, the same invalid code can >>>> be diagnosed by one warning one target and different warning >>>> on another. >>>> >>>> The attached patch does three things: >>>> >>>> 1) It enhances compute_objsize to also determine the size of >>>> a flexible array member (and its various variants), including >>>> from its initializer if necessary. (This resolves 91457 but >>>> introduces another warning where was previously just one.) >>>> 2) It guards the new instance of -Wstringop-overflow with >>>> the no-warning bit on the assignment to avoid warning on code >>>> that's already been diagnosed. >>>> 3) It arranges for -Warray-bounds to set the no-warning bit on >>>> the enclosing expression to keep -Wstringop-overflow from issuing >>>> another warning for the same problem. >>>> >>>> Testing the compute_objsize enhancement to bring it up to par >>>> with -Warray-bounds in turn exposed a weakness in the latter >>>> warning for flexible array members. Rather than snowballing >>>> additional improvements into this one I decided to put that >>>> off until later, so the new -Warray-bounds test has a bunch >>>> of XFAILs. I'll see if I can find the time to deal with those >>>> either still in stage 1 or in stage 3 (one of them is actually >>>> an ancient regression). >>> >>> +static tree >>> +get_initializer_for (tree init, tree decl) >>> +{ >>> >>> can't you use fold_ctor_reference here? >> >> Yes, but only with an additional enhancement. Char initializers >> for flexible array members aren't transformed to STRING_CSTs yet, >> so without the size of the initializer specified, the function >> returns the initializer for the smallest subobject, or char in >> this case. I've enhanced the function to handle them. > > So at the moment it returns an empty array constructor, correct? > Isn't that the more canonical representation? The STRING_CST > index type doesn't really handle "empty" strings and I expect code > more confused about that than about an empty CTOR? > >>> >>> +/* Determine the size of the flexible array FLD from the initializer >>> + expression for the struct object DECL in which the meber is declared >>> + (possibly recursively). Return the size or zero constant if it isn't >>> + initialized. */ >>> + >>> +static tree >>> +get_flexarray_size (tree decl, tree fld) >>> +{ >>> + if (tree init = DECL_INITIAL (decl)) >>> + { >>> + init = get_initializer_for (init, fld); >>> + if (init) >>> + return TYPE_SIZE_UNIT (TREE_TYPE (init)); >>> + } >>> + >>> + return integer_zero_node; >>> >>> so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for >>> returns has a complete type but the initialized object didn't get it >>> completed. Isnt that wishful thinking? >> >> I don't know what you mean. When might a CONSTRUCTOR not have >> a complete type, and if/when it doesn't, why would that be >> a problem here? TYPE_SIZE_UNIT will evaluate to null meaning >> "don't know" and that's fine. Could you try to be more specific >> about the problem you're pointing out? >> >>> And why return integer_zero_node >>> rather than NULL_TREE here? >> >> Because the size of a flexible array member with no initializer >> is zero. >> >>> >>> + if (TREE_CODE (dest) == COMPONENT_REF) >>> + { >>> + *pdecl = TREE_OPERAND (dest, 1); >>> + >>> + /* If the member has a size return it. Otherwise it's a flexible >>> + array member. */ >>> + if (tree size = DECL_SIZE_UNIT (*pdecl)) >>> + return size; >>> >>> because here you do. >> >> Not sure what you mean here either. (This code was also a bit > > You return NULL_TREE. > >> out of date WRT to the patch I had tested. Not sure how that >> happened. The attached patch is up to date.) >> >>> >>> Also once you have an underlying VAR_DECL you can compute >>> the flexarray size by DECL_SIZE (var) - offset-of flexarray member. >>> Isn't that way cheaper than walking the initializer (possibly many >>> times?) >> >> It would be nice if it were this easy. Is the value of DECL_SIZE >> (var) supposed to include the size of the flexible array member? > > Yes, DECL_SIZE of the VAR_DECL is the size we use for assembling. > It is usually only the types that remain incomplete (or too small) since > the FE doesn't create many variants of a struct S { int n; char x[]; } > when "instantiating" it via struct S s = { 5, "abcde" }; that then holds > true for the DECL_SIZE of the FIELD_DECL of x as well. > >> I don't see it mentioned in the comments in tree.h and in my tests >> it only does in C but not in C++. Is that a bug that in C++ it >> doesn't? > > tree.h dcoumentation isn't complete. And for me the C++ FE for > the above simple example has > > <var_decl 0x7ffff7fedb40 s > type <record_type 0x7ffff6d94f18 S cxx-odr-p type_5 type_6 BLK > size <integer_cst 0x7ffff6c5e0a8 constant 32> > unit-size <integer_cst 0x7ffff6c5e0c0 constant 4> > align:32 warn_if_not_align:0 symtab:0 alias-set -1 > canonical-type 0x7ffff6d94f18 > fields <function_decl 0x7ffff6d98b00 __dt type <method_type > 0x7ffff6da8150> > public abstract external autoinline decl_3 QI t.ii:1:8 > align:16 warn_if_not_align:0 context <record_type 0x7ffff6d94f18 S> > full-name "S::~S() noexcept (<uninstantiated>)" > not-really-extern chain <function_decl 0x7ffff6d98d00 > __dt_base >> context <translation_unit_decl 0x7ffff6c4a168 t.ii> > full-name "struct S" > X() X(constX&) this=(X&) n_parents=0 use_template=0 interface-unknown > pointer_to_this <pointer_type 0x7ffff6da85e8> > reference_to_this <reference_type 0x7ffff6da8a80> chain <type_decl > 0x7ffff6c6b850 S>> > public static tree_1 tree_2 tree_3 BLK t.ii:3:3 size <integer_cst > 0x7ffff6c5e0a8 32> unit-size <integer_cst 0x7ffff6c5e0c0 4> > align:32 warn_if_not_align:0 context <translation_unit_decl > 0x7ffff6c4a168 t.ii> initial <constructor 0x7ffff6da52a0> chain > <type_decl 0x7ffff6c6b850 S>> > > and we assemble it like > > .globl s > .data > .align 4 > .type s, @object > .size s, 4 > s: > .long 5 > .string "abcde" > > note how we have .size s, 4 here... > > of course in the end nobody cares about a symbols size ...(?) The linker may care. I've certainly run across those which would warn/error when the size of an object with global scope differed across TUs. I wouldn't expect to be running into that kind of issue in the modern world though. Jeff
On Thu, Aug 22, 2019 at 3:43 AM Richard Biener <richard.guenther@gmail.com> wrote: > On Wed, Aug 21, 2019 at 10:10 PM Martin Sebor <msebor@gmail.com> wrote: > > On 8/20/19 1:26 AM, Richard Biener wrote: > > > On Tue, Aug 20, 2019 at 4:32 AM Martin Sebor <msebor@gmail.com> wrote: > > >> On 8/19/19 8:10 AM, Richard Biener wrote: > > >>> On Sat, Aug 17, 2019 at 12:43 AM Martin Sebor <msebor@gmail.com> wrote: > > >>>> > > >>>> With the recent enhancement to the strlen handling of multibyte > > >>>> stores the g++.dg/warn/Warray-bounds-4.C for zero-length arrays > > >>>> started failing on hppa (and probably elsewhere as well). This > > >>>> is partly the result of the added detection of past-the-end > > >>>> writes into the strlen pass which detects more instances of > > >>>> the problem than -Warray-bounds. Since the IL each warning > > >>>> works with varies between targets, the same invalid code can > > >>>> be diagnosed by one warning one target and different warning > > >>>> on another. > > >>>> > > >>>> The attached patch does three things: > > >>>> > > >>>> 1) It enhances compute_objsize to also determine the size of > > >>>> a flexible array member (and its various variants), including > > >>>> from its initializer if necessary. (This resolves 91457 but > > >>>> introduces another warning where was previously just one.) > > >>>> 2) It guards the new instance of -Wstringop-overflow with > > >>>> the no-warning bit on the assignment to avoid warning on code > > >>>> that's already been diagnosed. > > >>>> 3) It arranges for -Warray-bounds to set the no-warning bit on > > >>>> the enclosing expression to keep -Wstringop-overflow from issuing > > >>>> another warning for the same problem. > > >>>> > > >>>> Testing the compute_objsize enhancement to bring it up to par > > >>>> with -Warray-bounds in turn exposed a weakness in the latter > > >>>> warning for flexible array members. Rather than snowballing > > >>>> additional improvements into this one I decided to put that > > >>>> off until later, so the new -Warray-bounds test has a bunch > > >>>> of XFAILs. I'll see if I can find the time to deal with those > > >>>> either still in stage 1 or in stage 3 (one of them is actually > > >>>> an ancient regression). > > >>> > > >>> +static tree > > >>> +get_initializer_for (tree init, tree decl) > > >>> +{ > > >>> > > >>> can't you use fold_ctor_reference here? > > >> > > >> Yes, but only with an additional enhancement. Char initializers > > >> for flexible array members aren't transformed to STRING_CSTs yet, > > >> so without the size of the initializer specified, the function > > >> returns the initializer for the smallest subobject, or char in > > >> this case. I've enhanced the function to handle them. > > > > > > So at the moment it returns an empty array constructor, correct? > > > Isn't that the more canonical representation? The STRING_CST > > > index type doesn't really handle "empty" strings and I expect code > > > more confused about that than about an empty CTOR? > > > > Yes. Returning an empty CTOR is more general than an empty > > string and enables more optimizations. It requires changing > > the caller(s) that look for a string but I think that's fine. > > Thanks for the hint! > > > > >>> +/* Determine the size of the flexible array FLD from the initializer > > >>> + expression for the struct object DECL in which the meber is declared > > >>> + (possibly recursively). Return the size or zero constant if it isn't > > >>> + initialized. */ > > >>> + > > >>> +static tree > > >>> +get_flexarray_size (tree decl, tree fld) > > >>> +{ > > >>> + if (tree init = DECL_INITIAL (decl)) > > >>> + { > > >>> + init = get_initializer_for (init, fld); > > >>> + if (init) > > >>> + return TYPE_SIZE_UNIT (TREE_TYPE (init)); > > >>> + } > > >>> + > > >>> + return integer_zero_node; > > >>> > > >>> so you're hoping that the (sub-)CONSTRUCTOR get_initializer_for > > >>> returns has a complete type but the initialized object didn't get it > > >>> completed. Isnt that wishful thinking? > > >> > > >> I don't know what you mean. When might a CONSTRUCTOR not have > > >> a complete type, and if/when it doesn't, why would that be > > >> a problem here? TYPE_SIZE_UNIT will evaluate to null meaning > > >> "don't know" and that's fine. Could you try to be more specific > > >> about the problem you're pointing out? > > >> > > >>> And why return integer_zero_node > > >>> rather than NULL_TREE here? > > >> > > >> Because the size of a flexible array member with no initializer > > >> is zero. > > >> > > >>> > > >>> + if (TREE_CODE (dest) == COMPONENT_REF) > > >>> + { > > >>> + *pdecl = TREE_OPERAND (dest, 1); > > >>> + > > >>> + /* If the member has a size return it. Otherwise it's a flexible > > >>> + array member. */ > > >>> + if (tree size = DECL_SIZE_UNIT (*pdecl)) > > >>> + return size; > > >>> > > >>> because here you do. > > >> > > >> Not sure what you mean here either. (This code was also a bit > > > > > > You return NULL_TREE. > > > > > >> out of date WRT to the patch I had tested. Not sure how that > > >> happened. The attached patch is up to date.) > > >> > > >>> > > >>> Also once you have an underlying VAR_DECL you can compute > > >>> the flexarray size by DECL_SIZE (var) - offset-of flexarray member. > > >>> Isn't that way cheaper than walking the initializer (possibly many > > >>> times?) > > >> > > >> It would be nice if it were this easy. Is the value of DECL_SIZE > > >> (var) supposed to include the size of the flexible array member? > > > > > > Yes, DECL_SIZE of the VAR_DECL is the size we use for assembling. > > > It is usually only the types that remain incomplete (or too small) since > > > the FE doesn't create many variants of a struct S { int n; char x[]; } > > > when "instantiating" it via struct S s = { 5, "abcde" }; that then holds > > > true for the DECL_SIZE of the FIELD_DECL of x as well. > > > > > >> I don't see it mentioned in the comments in tree.h and in my tests > > >> it only does in C but not in C++. Is that a bug that in C++ it > > >> doesn't? > > > > > > tree.h dcoumentation isn't complete. And for me the C++ FE for > > > the above simple example has > > > > > > <var_decl 0x7ffff7fedb40 s > > > type <record_type 0x7ffff6d94f18 S cxx-odr-p type_5 type_6 BLK > > > size <integer_cst 0x7ffff6c5e0a8 constant 32> > > > unit-size <integer_cst 0x7ffff6c5e0c0 constant 4> > > > align:32 warn_if_not_align:0 symtab:0 alias-set -1 > > > canonical-type 0x7ffff6d94f18 > > > fields <function_decl 0x7ffff6d98b00 __dt type <method_type > > > 0x7ffff6da8150> > > > public abstract external autoinline decl_3 QI t.ii:1:8 > > > align:16 warn_if_not_align:0 context <record_type 0x7ffff6d94f18 S> > > > full-name "S::~S() noexcept (<uninstantiated>)" > > > not-really-extern chain <function_decl 0x7ffff6d98d00 > > > __dt_base >> context <translation_unit_decl 0x7ffff6c4a168 t.ii> > > > full-name "struct S" > > > X() X(constX&) this=(X&) n_parents=0 use_template=0 interface-unknown > > > pointer_to_this <pointer_type 0x7ffff6da85e8> > > > reference_to_this <reference_type 0x7ffff6da8a80> chain <type_decl > > > 0x7ffff6c6b850 S>> > > > public static tree_1 tree_2 tree_3 BLK t.ii:3:3 size <integer_cst > > > 0x7ffff6c5e0a8 32> unit-size <integer_cst 0x7ffff6c5e0c0 4> > > > align:32 warn_if_not_align:0 context <translation_unit_decl > > > 0x7ffff6c4a168 t.ii> initial <constructor 0x7ffff6da52a0> chain > > > <type_decl 0x7ffff6c6b850 S>> > > > > > > and we assemble it like > > > > > > .globl s > > > .data > > > .align 4 > > > .type s, @object > > > .size s, 4 > > > s: > > > .long 5 > > > .string "abcde" > > > > > > note how we have .size s, 4 here... > > > > > > of course in the end nobody cares about a symbols size ...(?) > > > > > > I expected layout_decl to fixup DECL_SIZE according to the iniitalizer > > > but it looks like the initializer isn't present when it is layouted. > > > > > > I think this is a middle-end flaw we should try to fix that either in > > > the FEs or in the middle-end. The C FE calls add_flexible_array_elts_to_size > > > and complete_flexible_array_elts in its finish_decl routine (layouting happens > > > at build_decl time, too early, but if we want to move this functionality then > > > re-layouting might be posible). I'm not sure about other FEs but I guess > > > the easiest point we could do sanity checking is when assemble_variable > > > outputs the constructor. > > > > I don't know if I followed all this but it sounds like you're > > saying this is a separate issue. > > Yeah, it looks like a separate issue to me but one that would be > nice to have fixed so we can compute the length of the trailing > array without needing to parse the initializer. I'd even say we should > delay that effort if the above is confirmed as a bug and a fix is > possible. Jason, is the above intended behavior? I realize > "trailing" arrays is a GNU extension in C++ but as seen above > it doesn't match GNU C behavior here. Having DECL_SIZE different from GNU C for the same example is definitely not intended. Jason > As said, I'm not sure if a wrong symbol size has an impact on > any target binary format we support. > > > > > > >> Attached is an updated patch that uses fold_ctor_reference as you > > >> suggested. I've also made a few other minor changes to diagnose > > >> a few more invalid strlen calls with out-of-bounds offsets. > > >> (More still remain.) > > > > > > Thanks, but the patch was already large ... it would be nice to strip it > > > down to the bare bones again and do followup adjustments instead. > > > > I'm not sure what you view as the bare bones bits but I took > > a guess and split off the changes for PR 91490 (missing folding > > of constant flexible array members) into a separate patch and > > posted that. Once that's approved I'll update this one and post > > what's left of it. > > Thanks, > Richard. > > > Martin > > > > > > > > Richard. > > > > > >> Martin > > >> > > >>> > > >>> Richard. > > >>> > > >>>> > > >>>> Martin > > >>>> > > >>>> PS I imagine the new get_flexarray_size function will probably > > >>>> need to move somewhere more appropriate so that other warnings > > >>>> (like -Warray-bounds to remove the XFAILs) and optimizations > > >>>> can make use of it. > > >> > >
With the PR 91490 subset committed, attached is what's left of the original fix. It seems simple enough that I decided it's not worth chopping up any further. The meat of the change is in builtins.c. Everything else just adjusts warnings either by making use of it (tree-ssa-strlen.c) or by setting the no-warning bit (tree-vrp.c). Retested on x86_64-linux. Martin
On 8/28/19 9:44 AM, Martin Sebor wrote: > With the PR 91490 subset committed, attached is what's left > of the original fix. It seems simple enough that I decided it's > not worth chopping up any further. The meat of the change is in > builtins.c. Everything else just adjusts warnings either by > making use of it (tree-ssa-strlen.c) or by setting the no-warning > bit (tree-vrp.c). > > Retested on x86_64-linux. > > Martin > > gcc-91457.diff > > PR tree-optimization/91457 - inconsistent warning for writing past the end of an array member > > gcc/ChangeLog: > > PR tree-optimization/91457 > * builtins.c (component_size): New function. > (compute_objsize): Add argument. Handle ARRAY_REF and COMPONENT_REF. > * builtins.h (compute_objsize): Add argument. > * tree-ssa-strlen.c (handle_store): Handle no-warning bit. > * tree-vrp.c (vrp_prop::check_array_ref): Return warning result. > (vrp_prop::check_mem_ref): Same. > (vrp_prop::search_for_addr_array): Set no-warning bit. > (check_array_bounds): Same. > > gcc/testsuite/ChangeLog: > > PR tree-optimization/91457 > * c-c++-common/Wstringop-overflow-2.c: New test. > * g++.dg/warn/Warray-bounds-8.C: New test. > * g++.dg/warn/Wstringop-overflow-3.C: New test. > * gcc.dg/Wstringop-overflow-15.c: New test. OK jeff
PR tree-optimization/91458 - inconsistent warning for writing past the end of an array member gcc/testsuite/ChangeLog: PR tree-optimization/91458 * g++.dg/warn/Warray-bounds-8.C: New test. * g++.dg/warn/Wstringop-overflow-3.C: New test. gcc/ChangeLog: PR tree-optimization/91458 * builtins.c (get_initializer_for, get_flexarray_size): New functions. (compute_objsize): Add argument. Handle ARRAY_REF and COMPONENT_REF. * builtins.h ((compute_objsize): Add argument. * tree-ssa-strlen.c (handle_store): Handle no-warning bit. * tree-vrp.c (vrp_prop::check_array_ref): Return warning result. (vrp_prop::check_mem_ref): Same. (vrp_prop::search_for_addr_array): Set no-warning bit. (check_array_bounds): Same. Index: gcc/builtins.c =================================================================== --- gcc/builtins.c (revision 274541) +++ gcc/builtins.c (working copy) @@ -3560,6 +3560,51 @@ check_access (tree exp, tree, tree, tree dstwrite, return true; } +/* Given the initializer INIT, return the initializer for the field + DECL if it exists, otherwise null. Used to obtain the initializer + for a flexible array member and determine its size. */ + +static tree +get_initializer_for (tree init, tree decl) +{ + STRIP_NOPS (init); + + tree fld, fld_init; + unsigned HOST_WIDE_INT i; + FOR_EACH_CONSTRUCTOR_ELT (CONSTRUCTOR_ELTS (init), i, fld, fld_init) + { + if (decl == fld) + return fld_init; + + if (TREE_CODE (fld) == CONSTRUCTOR) + { + fld_init = get_initializer_for (fld_init, decl); + if (fld_init) + return fld_init; + } + } + + return NULL_TREE; +} + +/* Determine the size of the flexible array FLD from the initializer + expression for the struct object DECL in which the meber is declared + (possibly recursively). Return the size or zero constant if it isn't + initialized. */ + +static tree +get_flexarray_size (tree decl, tree fld) +{ + if (tree init = DECL_INITIAL (decl)) + { + init = get_initializer_for (init, fld); + if (init) + return TYPE_SIZE_UNIT (TREE_TYPE (init)); + } + + return integer_zero_node; +} + /* Helper to compute the size of the object referenced by the DEST expression which must have pointer type, using Object Size type OSTYPE (only the least significant 2 bits are used). Return @@ -3567,12 +3612,18 @@ check_access (tree exp, tree, tree, tree dstwrite, the size cannot be determined. When the referenced object involves a non-constant offset in some range the returned value represents the largest size given the smallest non-negative offset in the - range. The function is intended for diagnostics and should not - be used to influence code generation or optimization. */ + range. If nonnull, set *PDECL to the decl of the referenced + subobject if it can be determined, or to null otherwise. + The function is intended for diagnostics and should not be used + to influence code generation or optimization. */ tree -compute_objsize (tree dest, int ostype) +compute_objsize (tree dest, int ostype, tree *pdecl /* = NULL */) { + tree dummy = NULL_TREE; + if (!pdecl) + pdecl = &dummy; + unsigned HOST_WIDE_INT size; /* Only the two least significant bits are meaningful. */ @@ -3599,7 +3650,7 @@ tree tree off = gimple_assign_rhs2 (stmt); if (TREE_CODE (off) == INTEGER_CST) { - if (tree size = compute_objsize (dest, ostype)) + if (tree size = compute_objsize (dest, ostype, pdecl)) { wide_int wioff = wi::to_wide (off); wide_int wisiz = wi::to_wide (size); @@ -3624,7 +3675,7 @@ tree if (rng == VR_RANGE) { - if (tree size = compute_objsize (dest, ostype)) + if (tree size = compute_objsize (dest, ostype, pdecl)) { wide_int wisiz = wi::to_wide (size); @@ -3652,12 +3703,32 @@ tree if (!ostype) return NULL_TREE; - if (TREE_CODE (dest) == MEM_REF) + if (TREE_CODE (dest) == ARRAY_REF + || TREE_CODE (dest) == MEM_REF) { tree ref = TREE_OPERAND (dest, 0); tree off = TREE_OPERAND (dest, 1); - if (tree size = compute_objsize (ref, ostype)) + if (tree size = compute_objsize (ref, ostype, pdecl)) { + /* If the declaration of the destination object is known to + have zero size, return zero. Otherwise, if the returned + size is zero and the idex/offset is unsigned, also return + zero. */ + if (integer_zerop (size)) + return integer_zero_node; + + if (TREE_CODE (off) != INTEGER_CST) + return NULL_TREE; + + if (TREE_CODE (dest) == ARRAY_REF) + { + tree eltype = TREE_TYPE (dest); + if (tree tpsize = TYPE_SIZE_UNIT (eltype)) + off = fold_build2 (MULT_EXPR, size_type_node, off, tpsize); + else + return NULL_TREE; + } + if (tree_int_cst_lt (off, size)) return fold_build2 (MINUS_EXPR, size_type_node, size, off); return integer_zero_node; @@ -3666,9 +3737,34 @@ tree return NULL_TREE; } + if (TREE_CODE (dest) == COMPONENT_REF) + { + *pdecl = TREE_OPERAND (dest, 1); + + /* If the member has a size return it. Otherwise it's a flexible + array member. */ + if (tree size = DECL_SIZE_UNIT (*pdecl)) + return size; + + /* If the destination is a declared object with zero size try to + determine its size from its initializer. */ + if (tree base = get_base_address (dest)) + if (VAR_P (base)) + return get_flexarray_size (base, *pdecl); + + return NULL_TREE; + } + if (TREE_CODE (dest) != ADDR_EXPR) return NULL_TREE; + tree ref = TREE_OPERAND (dest, 0); + if (DECL_P (ref)) + { + *pdecl = ref; + return DECL_SIZE_UNIT (ref); + } + tree type = TREE_TYPE (dest); if (TREE_CODE (type) == POINTER_TYPE) type = TREE_TYPE (type); @@ -3678,12 +3774,8 @@ tree if (TREE_CODE (type) == ARRAY_TYPE && !array_at_struct_end_p (TREE_OPERAND (dest, 0))) { - /* Return the constant size unless it's zero (that's a zero-length - array likely at the end of a struct). */ - tree size = TYPE_SIZE_UNIT (type); - if (size && TREE_CODE (size) == INTEGER_CST - && !integer_zerop (size)) - return size; + if (tree size = TYPE_SIZE_UNIT (type)) + return TREE_CODE (size) == INTEGER_CST ? size : NULL_TREE; } return NULL_TREE; Index: gcc/builtins.h =================================================================== --- gcc/builtins.h (revision 274541) +++ gcc/builtins.h (working copy) @@ -134,7 +134,7 @@ extern tree fold_call_stmt (gcall *, bool); extern void set_builtin_user_assembler_name (tree decl, const char *asmspec); extern bool is_simple_builtin (tree); extern bool is_inexpensive_builtin (tree); -extern tree compute_objsize (tree, int); +extern tree compute_objsize (tree, int, tree * = NULL); extern bool readonly_data_expr (tree exp); extern bool init_target_chars (void); Index: gcc/testsuite/g++.dg/warn/Warray-bounds-8.C =================================================================== --- gcc/testsuite/g++.dg/warn/Warray-bounds-8.C (nonexistent) +++ gcc/testsuite/g++.dg/warn/Warray-bounds-8.C (working copy) @@ -0,0 +1,61 @@ +/* PR middle-end/91458 - inconsistent warning for writing past the end + of an array member + { dg-do compile } + { dg-options "-O2 -Wall" } */ + +struct A0 +{ + char n, a[0]; + + A0 () { a[0] = 0; } // { dg-warning "\\\[-Warray-bounds" } +}; + +void f (void*); + +void g0 (void) +{ + struct A0 a; + f (&a); +} + + +struct A1 +{ + char n, a[1]; + + A1 () { a[1] = 0; } // { dg-warning "\\\[-Warray-bounds" } +}; + +void g1 (void) +{ + struct A1 a; + f (&a); +} + + +struct A123 +{ + char n, a[123]; + + A123 () { a[123] = 0; } // { dg-warning "\\\[-Warray-bounds" } +}; + +void g123 (void) +{ + struct A123 a; + f (&a); +} + + +struct A234 +{ + char n, a[234]; + + A234 (int i) { a[i] = 0; } // { dg-warning "\\\[-Warray-bounds" } +}; + +void g234 (void) +{ + struct A234 a (234); + f (&a); +} Index: gcc/testsuite/g++.dg/warn/Wstringop-overflow-3.C =================================================================== --- gcc/testsuite/g++.dg/warn/Wstringop-overflow-3.C (nonexistent) +++ gcc/testsuite/g++.dg/warn/Wstringop-overflow-3.C (working copy) @@ -0,0 +1,112 @@ +/* PR middle-end/91458 - inconsistent warning for writing past the end + of an array member + { dg-do compile } + { dg-options "-O2 -Wall -Wno-array-bounds" } */ + +struct Axi +{ + char n; + char a[]; // { dg-message "destination object declared here" } +}; + +Axi ax2 = { 2, { 1, 0 } }; + +void gx2 () +{ + ax2.a[0] = 0; + ax2.a[1] = 0; + ax2.a[2] = 0; // { dg-warning "\\\[-Wstringop-overflow" } +} + +void gxx (Axi *p) +{ + p->a[0] = 0; + p->a[3] = 0; + p->a[9] = 0; +} + +struct Ax +{ + char n; + char a[]; // { dg-message "destination object declared here" } + + // Verify the warning for a constant. + Ax () { a[0] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } + + // And also for a non-constant. Regardless of the subscript, the array + // of the object in function gxi() below has a zero size. + Ax (int i) { a[i] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } +}; + +void f (void*); + +void gx (void) +{ + struct Ax a; + f (&a); +} + +void gxi (int i) +{ + struct Ax a (i); + f (&a); +} + +struct A0 +{ + char n; + char a[0]; // { dg-message "destination object declared here" } + + A0 () { a[0] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } +}; + +void f (void*); + +void g0 (void) +{ + struct A0 a; + f (&a); +} + + +struct A1 +{ + char n; + char a[1]; // { dg-message "destination object declared here" } + + A1 () { a[1] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } +}; + +void g1 (void) +{ + struct A1 a; + f (&a); +} + + +struct A123 +{ + char a[123]; // { dg-message "destination object declared here" } + + A123 () { a[123] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } +}; + +void g123 (void) +{ + struct A123 a; + f (&a); +} + + +struct A234 +{ + char a[234]; // { dg-message "destination object declared here" } + + A234 (int i) { a[i] = 0; } // { dg-warning "\\\[-Wstringop-overflow" } +}; + +void g234 (void) +{ + struct A234 a (234); + f (&a); +} Index: gcc/tree-ssa-strlen.c =================================================================== --- gcc/tree-ssa-strlen.c (revision 274541) +++ gcc/tree-ssa-strlen.c (working copy) @@ -3679,16 +3679,30 @@ handle_store (gimple_stmt_iterator *gsi) rhs_minlen = lenrange[0]; storing_nonzero_p = lenrange[1] > 0; - if (tree dstsize = compute_objsize (lhs, 1)) - if (compare_tree_int (dstsize, lenrange[2]) < 0) - { - location_t loc = gimple_nonartificial_location (stmt); - warning_n (loc, OPT_Wstringop_overflow_, - lenrange[2], - "%Gwriting %u byte into a region of size %E", - "%Gwriting %u bytes into a region of size %E", - stmt, lenrange[2], dstsize); - } + /* Avoid issuing multiple warnings for the same LHS or statement. + For example, -Warray-bounds may have already been issued for + an out-of-bounds subscript. */ + if (!TREE_NO_WARNING (lhs) && !gimple_no_warning_p (stmt)) + { + /* Set to the declaration referenced by LHS (if known). */ + tree decl = NULL_TREE; + if (tree dstsize = compute_objsize (lhs, 1, &decl)) + if (compare_tree_int (dstsize, lenrange[2]) < 0) + { + location_t loc = gimple_nonartificial_location (stmt); + if (warning_n (loc, OPT_Wstringop_overflow_, + lenrange[2], + "%Gwriting %u byte into a region of size %E", + "%Gwriting %u bytes into a region of size %E", + stmt, lenrange[2], dstsize)) + { + if (decl) + inform (DECL_SOURCE_LOCATION (decl), + "destination object declared here"); + gimple_set_no_warning (stmt, true); + } + } + } } else { Index: gcc/tree-vrp.c =================================================================== --- gcc/tree-vrp.c (revision 274541) +++ gcc/tree-vrp.c (working copy) @@ -4354,8 +4354,8 @@ class vrp_prop : public ssa_propagation_engine void vrp_initialize (void); void vrp_finalize (bool); void check_all_array_refs (void); - void check_array_ref (location_t, tree, bool); - void check_mem_ref (location_t, tree, bool); + bool check_array_ref (location_t, tree, bool); + bool check_mem_ref (location_t, tree, bool); void search_for_addr_array (tree, location_t); class vr_values vr_values; @@ -4381,9 +4381,10 @@ class vrp_prop : public ssa_propagation_engine array subscript is a constant, check if it is outside valid range. If the array subscript is a RANGE, warn if it is non-overlapping with valid range. - IGNORE_OFF_BY_ONE is true if the ARRAY_REF is inside a ADDR_EXPR. */ + IGNORE_OFF_BY_ONE is true if the ARRAY_REF is inside a ADDR_EXPR. + Returns true if a warning has been issued. */ -void +bool vrp_prop::check_array_ref (location_t location, tree ref, bool ignore_off_by_one) { @@ -4392,7 +4393,7 @@ vrp_prop::check_array_ref (location_t location, tr tree low_bound, up_bound, up_bound_p1; if (TREE_NO_WARNING (ref)) - return; + return false; low_sub = up_sub = TREE_OPERAND (ref, 1); up_bound = array_ref_up_bound (ref); @@ -4513,6 +4514,8 @@ vrp_prop::check_array_ref (location_t location, tr TREE_NO_WARNING (ref) = 1; } + + return warned; } /* Checks one MEM_REF in REF, located at LOCATION, for out-of-bounds @@ -4522,14 +4525,15 @@ vrp_prop::check_array_ref (location_t location, tr with valid range. IGNORE_OFF_BY_ONE is true if the MEM_REF is inside an ADDR_EXPR (used to allow one-past-the-end indices for code that takes - the address of the just-past-the-end element of an array). */ + the address of the just-past-the-end element of an array). + Returns true if a warning has been issued. */ -void +bool vrp_prop::check_mem_ref (location_t location, tree ref, bool ignore_off_by_one) { if (TREE_NO_WARNING (ref)) - return; + return false; tree arg = TREE_OPERAND (ref, 0); /* The constant and variable offset of the reference. */ @@ -4581,7 +4585,7 @@ vrp_prop::check_mem_ref (location_t location, tree continue; } else - return; + return false; /* VAROFF should always be a SSA_NAME here (and not even INTEGER_CST) but there's no point in taking chances. */ @@ -4643,10 +4647,10 @@ vrp_prop::check_mem_ref (location_t location, tree arg = TREE_OPERAND (arg, 0); if (TREE_CODE (arg) != STRING_CST && TREE_CODE (arg) != VAR_DECL) - return; + return false; } else - return; + return false; /* The type of the object being referred to. It can be an array, string literal, or a non-array type when the MEM_REF represents @@ -4661,7 +4665,7 @@ vrp_prop::check_mem_ref (location_t location, tree || !COMPLETE_TYPE_P (reftype) || TREE_CODE (TYPE_SIZE_UNIT (reftype)) != INTEGER_CST || RECORD_OR_UNION_TYPE_P (reftype)) - return; + return false; offset_int eltsize; if (TREE_CODE (reftype) == ARRAY_TYPE) @@ -4763,11 +4767,11 @@ vrp_prop::check_mem_ref (location_t location, tree if (warned) TREE_NO_WARNING (ref) = 1; - return; + return warned; } if (warn_array_bounds < 2) - return; + return false; /* At level 2 check also intermediate offsets. */ int i = 0; @@ -4778,8 +4782,13 @@ vrp_prop::check_mem_ref (location_t location, tree if (warning_at (location, OPT_Warray_bounds, "intermediate array offset %wi is outside array bounds " "of %qT", tmpidx, reftype)) - TREE_NO_WARNING (ref) = 1; + { + TREE_NO_WARNING (ref) = 1; + return true; + } } + + return false; } /* Searches if the expr T, located at LOCATION computes @@ -4791,11 +4800,15 @@ vrp_prop::search_for_addr_array (tree t, location_ /* Check each ARRAY_REF and MEM_REF in the reference chain. */ do { + bool warned = false; if (TREE_CODE (t) == ARRAY_REF) - check_array_ref (location, t, true /*ignore_off_by_one*/); + warned = check_array_ref (location, t, true /*ignore_off_by_one*/); else if (TREE_CODE (t) == MEM_REF) - check_mem_ref (location, t, true /*ignore_off_by_one*/); + warned = check_mem_ref (location, t, true /*ignore_off_by_one*/); + if (warned) + TREE_NO_WARNING (t) = true; + t = TREE_OPERAND (t, 0); } while (handled_component_p (t) || TREE_CODE (t) == MEM_REF); @@ -4886,16 +4899,20 @@ check_array_bounds (tree *tp, int *walk_subtree, v *walk_subtree = TRUE; + bool warned = false; vrp_prop *vrp_prop = (class vrp_prop *)wi->info; if (TREE_CODE (t) == ARRAY_REF) - vrp_prop->check_array_ref (location, t, false /*ignore_off_by_one*/); + warned = vrp_prop->check_array_ref (location, t, false/*ignore_off_by_one*/); else if (TREE_CODE (t) == MEM_REF) - vrp_prop->check_mem_ref (location, t, false /*ignore_off_by_one*/); + warned = vrp_prop->check_mem_ref (location, t, false /*ignore_off_by_one*/); else if (TREE_CODE (t) == ADDR_EXPR) { vrp_prop->search_for_addr_array (t, location); *walk_subtree = FALSE; } + /* Propagate the no-warning bit to the outer expression. */ + if (warned) + TREE_NO_WARNING (t) = true; return NULL_TREE; }