[C++0x] contiguous bitfields race implementation

This is a "slight" update from the last revision, with your issues 
addressed as I explained in the last email.  However, everything turned 
out to be much tricker than I expected (variable length offsets with 
arrays, bit fields spanning multiple words, surprising padding 
gymnastics by GCC, etc etc).

It turns out that what we need is to know the precise bit region size at 
all times, and adjust it as we rearrange and cut things into pieces 
throughout the RTL bit field machinery.

I enabled the C++ memory model, and forced a boostrap and regression 
test with it.  This brought about many interesting cases, which I was 
able to distill and add to the testsuite.

Of particular interest was the struct-layout-1.exp tests.  Since many of 
the tests set a global bit field, only to later check it against a local 
variable containing the same value, it is the perfect stressor because, 
while globals are restricted under the memory model, locals are not.  So 
we can check that we can interoperate with the less restrictive model, 
and that the patch does not introduce ABI inconsistencies.  After much 
grief, we are now passing all the struct-layout-1.exp tests. 
Eventually, I'd like to force the struct-layout-1.exp tests to run for 
"--param allow-store-data-races=0" as well.  Unfortunately, this will 
increase testing time.

I have (unfortunately) introduced an additional call to 
get_inner_reference(), but only for the field itself (one time).  I 
can't remember the details, but it was something to effect of the bit 
position + padding being impossible to calculate in one variable array 
reference case.  I can dig up the case if you'd like.

I am currently tackling a reload miscompilation failure while building a 
32-bit library.  I am secretly hoping your review will uncover the flaw 
without me having to pick this up.  Otherwise, this is a much more 
comprehensive approach than what is currently in mainline, and we now 
pass all the bitfield tests the GCC testsuite could throw at it.

Fire away.
* machmode.h (get_best_mode): Remove 2 arguments.
	* fold-const.c (optimize_bit_field_compare): Same.
	(fold_truthop): Same.
	* expr.c (store_field): Change argument types in prototype.
	(emit_group_store): Change argument types to store_bit_field call.
	(copy_blkmode_from_reg): Same.
	(write_complex_part): Same.
	(optimize_bitfield_assignment_op): Change argument types.
	Change arguments to get_best_mode.
	(get_bit_range): Rewrite.
	(expand_assignment): Adjust new call to get_bit_range.
	Adjust bitregion_offset when to_rtx is changed.
	Adjust calls to store_field with new argument types.
	(store_field): New argument types.
	Adjust calls to store_bit_field with new arguments.
	* expr.h (store_bit_field): Change argument types.
	* stor-layout.c (get_best_mode): Remove use of bitregion* arguments.
	* expmed.c (store_bit_field_1): Change argument types.
	Do not calculate maxbits.
	Adjust bitregion_maxbits if offset changes.
	(store_bit_field): Change argument types.
	Adjust address taking into account bitregion_offset.
	(store_fixed_bit_field): Change argument types.
	Do not calculate maxbits.
	(store_split_bit_field): Change argument types.
	(extract_bit_field_1): Adjust arguments to get_best_mode.
	(extract_fixed_bit_field): Same.

On Fri, Aug 26, 2011 at 8:54 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
> This is a "slight" update from the last revision, with your issues addressed
> as I explained in the last email.  However, everything turned out to be much
> tricker than I expected (variable length offsets with arrays, bit fields
> spanning multiple words, surprising padding gymnastics by GCC, etc etc).
>
> It turns out that what we need is to know the precise bit region size at all
> times, and adjust it as we rearrange and cut things into pieces throughout
> the RTL bit field machinery.
>
> I enabled the C++ memory model, and forced a boostrap and regression test
> with it.  This brought about many interesting cases, which I was able to
> distill and add to the testsuite.
>
> Of particular interest was the struct-layout-1.exp tests.  Since many of the
> tests set a global bit field, only to later check it against a local
> variable containing the same value, it is the perfect stressor because,
> while globals are restricted under the memory model, locals are not.  So we
> can check that we can interoperate with the less restrictive model, and that
> the patch does not introduce ABI inconsistencies.  After much grief, we are
> now passing all the struct-layout-1.exp tests. Eventually, I'd like to force
> the struct-layout-1.exp tests to run for "--param allow-store-data-races=0"
> as well.  Unfortunately, this will increase testing time.
>
> I have (unfortunately) introduced an additional call to
> get_inner_reference(), but only for the field itself (one time).  I can't
> remember the details, but it was something to effect of the bit position +
> padding being impossible to calculate in one variable array reference case.
>  I can dig up the case if you'd like.
>
> I am currently tackling a reload miscompilation failure while building a
> 32-bit library.  I am secretly hoping your review will uncover the flaw
> without me having to pick this up.  Otherwise, this is a much more
> comprehensive approach than what is currently in mainline, and we now pass
> all the bitfield tests the GCC testsuite could throw at it.
>
> Fire away.

+  /* Be as conservative as possible on variable offsets.  */
+  if (TREE_OPERAND (exp, 2)
+      && !host_integerp (TREE_OPERAND (exp, 2), 1))
+    {
+      *byte_offset = TREE_OPERAND (exp, 2);
+      *maxbits = BITS_PER_UNIT;
+      return;
+    }

shouldn't this be at the very beginning of the function?  Because
you've set *bit_offset to an offset that was _not_ calculated relative
to TREE_OPERAND (exp, 2).  And you'll avoid ICEing

+	  /* bitoff = start_byte * 8 - (fld.byteoff * 8 + fld.bitoff) */
+	  t = fold_build2 (MINUS_EXPR, size_type_node,
+			   fold_build2 (PLUS_EXPR, size_type_node,
+					fold_build2 (MULT_EXPR, size_type_node,
+						     toffset, bits),
+					build_int_cst (integer_type_node,
+						       tbitpos)),
+			   fold_build2 (MULT_EXPR, size_type_node,
+					*byte_offset, bits));
+
+	  *bit_offset = tree_low_cst (t, 1);

here in case t isn't an INTEGER_CST.  The comment before the
tree formula above doesn't match it, please update it.  If
*bit_offset is supposed to be relative to *byte_offset then it should
be easy to calculate it without another get_inner_reference.

Btw, *byte_offset is still not relative to the containing object as
documented, but relative to the base object of the exp reference
tree (thus, to a in a.i.j.k.l instead of to a.i.j.k).  If it were supposed
to be relative to a.i.j.k get_inner_reference would be not needed
either.  Can you clarify what "containing object" means in the
overall comment please?

If it is really relative to the innermost reference of exp you can
"CSE" the offset of TREE_OPERAND (exp, 0) and do relative
adjustments for all the other get_inner_reference calls.  For
example the

+  /* If we found the end of the bit field sequence, include the
+     padding up to the next field...  */
   if (fld)
     {
...
+      /* Calculate bitpos and offset of the next field.  */
+      get_inner_reference (build3 (COMPONENT_REF,
+				   TREE_TYPE (exp),
+				   TREE_OPERAND (exp, 0),
+				   fld, NULL_TREE),
+			   &tbitsize, &end_bitpos, &end_offset,
+			   &tmode, &tunsignedp, &tvolatilep, true);

case is not correct anyway, fld may have variable position
(non-INTEGER_CST DECL_FIELD_OFFSET), you can't
assume

+      *maxbits = TREE_INT_CST_LOW (maxbits_tree);

this thus.

+  /* ...otherwise, this is the last element in the structure.  */
   else
     {
-      /* If this is the last element in the structure, include the padding
-	 at the end of structure.  */
-      *bitend = TREE_INT_CST_LOW (TYPE_SIZE (record_type)) - 1;
+      /* Include the padding at the end of structure.  */
+      *maxbits = TREE_INT_CST_LOW (TYPE_SIZE (record_type))
+	- TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (bitregion_start));
+      /* Round up to the next byte.  */
+      *maxbits = (*maxbits + BITS_PER_UNIT - 1) & ~(BITS_PER_UNIT - 1);
     }

so you wasn't convinced about my worries about tail-padding re-use?
And you blindly assume a constant-size record_type ...
and you don't account for DECL_FIELD_OFFSET of bitregion_start
(shouldn't you simply use (and compute) a byte_offset relative to
the start of the record)?  Well, I still think you cannot incoude the
padding at the end of the structure (if TREE_OPERAND (exp, 0) is
a COMPONENT_REF as well then its DECL_SIZE can be different
than it's TYPE_SIZE).

+	      bitregion_byte_offset = fold_build2 (MINUS_EXPR, integer_type_node,
+					      bitregion_byte_offset,
+					      build_int_cst (integer_type_node,
+							     bitpos / BITS_PER_UNIT));

general remark - you should be using sizetype for byte offsets,
bitsizetype for bit offset trees and size_binop for computations, instead
of fold_build2 (applies everywhere).  And thus pass size_zero_node
to store_field bitregion_byte_offset.

Can you split out the get_best_mode two param removal pieces?  Consider
them pre-approved.

Why do you need to adjust store_bit_field with the extra param - can't
you simply pass an adjusted str_rtx from the single caller that can
have that non-zero?

Thanks,
Richard.

[I'm going to respond to this piece-meal, to make sure I don't drop 
anything.  My apologies for the long thread, but I'm pretty sure it's in 
everybody's kill file by now.]

> +  /* Be as conservative as possible on variable offsets.  */
> +  if (TREE_OPERAND (exp, 2)
> +&&  !host_integerp (TREE_OPERAND (exp, 2), 1))
> +    {
> +      *byte_offset = TREE_OPERAND (exp, 2);
> +      *maxbits = BITS_PER_UNIT;
> +      return;
> +    }
>
> shouldn't this be at the very beginning of the function?  Because
> you've set *bit_offset to an offset that was _not_ calculated relative

Sure.  I assume in this case, *bit_offset would be 0, right?

> *bit_offset is supposed to be relative to *byte_offset then it should
> be easy to calculate it without another get_inner_reference.

Since, as you suggested, we will terminate early on variable length 
offsets, we can assume both DECL_FIELD_OFFSET and DECL_FIELD_BIT_OFFSET 
will be constants by now.  So, I assume we can calculate the bit offset 
like this:

*bit_offset = (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (fld))
	       * BITS_PER_UNIT
	       + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (fld)))
   - (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (bitregion_start))
      * BITS_PER_UNIT
      + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (bitregion_start)));

(Yes, I know we can factor out the BITS_PER_UNIT and only do one 
multiplication, it's just easier to read this way.)

Is this what you had in mind?

> Btw, *byte_offset is still not relative to the containing object as
> documented, but relative to the base object of the exp reference
> tree (thus, to a in a.i.j.k.l instead of to a.i.j.k).  If it were supposed
> to be relative to a.i.j.k get_inner_reference would be not needed
> either.  Can you clarify what "containing object" means in the
> overall comment please?

I'm thoroughly confused here.  Originally I had "inner decl", then we 
changed the nomenclature to "containing object", and now there's this 
"innermost reference".

What I mean to say is the "a" in a.i.j.k.l.  How would you like me to 
call that?  The innermost reference?  The inner decl?  Would this 
comment be acceptable:

    Given a COMPONENT_REF, this function calculates the byte offset
    from the innermost reference ("a" in a.i.j.k.l) to the start of the
    contiguous bit region containing the field in question.

>
> If it is really relative to the innermost reference of exp you can
> "CSE" the offset of TREE_OPERAND (exp, 0) and do relative
> adjustments for all the other get_inner_reference calls.  For
> example the
>
> +  /* If we found the end of the bit field sequence, include the
> +     padding up to the next field...  */
>     if (fld)
>       {
> ...
> +      /* Calculate bitpos and offset of the next field.  */
> +      get_inner_reference (build3 (COMPONENT_REF,
> +				   TREE_TYPE (exp),
> +				   TREE_OPERAND (exp, 0),
> +				   fld, NULL_TREE),
> +			&tbitsize,&end_bitpos,&end_offset,
> +			&tmode,&tunsignedp,&tvolatilep, true);
>
> case is not correct anyway, fld may have variable position
> (non-INTEGER_CST DECL_FIELD_OFFSET), you can't
> assume

Innermost here means "a" in a.i.j.k.l?  If so, this is what we're 
currently doing, *byte_offset is the start of the bit region, and 
*bit_offset is the offset from that.

First, I thought we couldn't get a variable position here because we are 
now handling that case at the beginning of the function with:

   /* Be as conservative as possible on variable offsets.  */
   if (TREE_OPERAND (exp, 2)
       && !host_integerp (TREE_OPERAND (exp, 2), 1))
     {
       *byte_offset = TREE_OPERAND (exp, 2);
       *maxbits = BITS_PER_UNIT;
       *bit_offset = 0;
       return;
     }

And even if we do get a variable position, I have so far being able to 
get away with this...

>
> +      *maxbits = TREE_INT_CST_LOW (maxbits_tree);
>
> this thus.

...because the call to fold_build2 immediately preceding this will fold 
away the variable offset.

Is what you want, that we call get_inner_reference once, and then use 
DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET to calculate any subsequent bit 
offset?  I found this to be quite tricky with padding, and such, but am 
willing to give it a whirl again.

However, could I beg you to reconsider this, and get something working 
first, only later concentrating on removing the get_inner_reference() 
calls, and performing any other tweaks/optimizations?

Aldy

On Tue, Aug 30, 2011 at 5:01 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
> [I'm going to respond to this piece-meal, to make sure I don't drop
> anything.  My apologies for the long thread, but I'm pretty sure it's in
> everybody's kill file by now.]
>
>> +  /* Be as conservative as possible on variable offsets.  */
>> +  if (TREE_OPERAND (exp, 2)
>> +&&  !host_integerp (TREE_OPERAND (exp, 2), 1))
>> +    {
>> +      *byte_offset = TREE_OPERAND (exp, 2);
>> +      *maxbits = BITS_PER_UNIT;
>> +      return;
>> +    }
>>
>> shouldn't this be at the very beginning of the function?  Because
>> you've set *bit_offset to an offset that was _not_ calculated relative
>
> Sure.  I assume in this case, *bit_offset would be 0, right?

It would be DECL_FIELD_BIT_OFFSET of that field.  Oh, and
*byte_offset would be

*byte_offset = size_binop (MULT_EXPR, TREE_OPERAND (exp, 2),
                                       size_int (DECL_OFFSET_ALIGN
(field) / BITS_PER_UNIT));

see expr.c:component_ref_field_offset () (which you conveniently
could use here).

Note that both TREE_OPERAND (exp, 2) and compoment_ref_field_offset
return offsets relative to the immediate containing struct type, not
relative to the base object like get_inner_reference does ...
(where it is still unclear to me what we are supposed to return from this
function ...)

Thus, conservative would be using get_inner_reference here, if the
offset is supposed to be relative to the base object.

Richard.

On Tue, Aug 30, 2011 at 6:15 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>> *bit_offset is supposed to be relative to *byte_offset then it should
>> be easy to calculate it without another get_inner_reference.
>
> Since, as you suggested, we will terminate early on variable length offsets,
> we can assume both DECL_FIELD_OFFSET and DECL_FIELD_BIT_OFFSET will be
> constants by now.

Yes.

>  So, I assume we can calculate the bit offset like this:
>
> *bit_offset = (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (fld))
>               * BITS_PER_UNIT
>               + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (fld)))
>  - (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (bitregion_start))
>     * BITS_PER_UNIT
>     + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (bitregion_start)));
>
> (Yes, I know we can factor out the BITS_PER_UNIT and only do one
> multiplication, it's just easier to read this way.)
>
> Is this what you had in mind?

Yes.  For convenience I'd simply use double_ints for the intermediate
calculations.

Richard.

On Tue, Aug 30, 2011 at 8:13 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>> Btw, *byte_offset is still not relative to the containing object as
>> documented, but relative to the base object of the exp reference
>> tree (thus, to a in a.i.j.k.l instead of to a.i.j.k).  If it were supposed
>> to be relative to a.i.j.k get_inner_reference would be not needed
>> either.  Can you clarify what "containing object" means in the
>> overall comment please?
>
> I'm thoroughly confused here.  Originally I had "inner decl", then we
> changed the nomenclature to "containing object", and now there's this
> "innermost reference".

Well, the nomenclature is not so important once the function only
computes one variant.  Only because it doesn't right now I am
confused with the nomenclature trying to figure out what it is supposed
to be relative to ...

The containing object of a component-ref is TREE_OPERAND (exp, 0)
to me.  The base object would be get_base_object (exp), which is
eventually what we want, right?

> What I mean to say is the "a" in a.i.j.k.l.  How would you like me to call
> that?  The innermost reference?  The inner decl?  Would this comment be
> acceptable:
>
>   Given a COMPONENT_REF, this function calculates the byte offset
>   from the innermost reference ("a" in a.i.j.k.l) to the start of the
>   contiguous bit region containing the field in question.

  from the base object ("a" in a.i.j.k.l) ...

would be fine with me.

>>
>> If it is really relative to the innermost reference of exp you can
>> "CSE" the offset of TREE_OPERAND (exp, 0) and do relative
>> adjustments for all the other get_inner_reference calls.  For
>> example the
>>
>> +  /* If we found the end of the bit field sequence, include the
>> +     padding up to the next field...  */
>>    if (fld)
>>      {
>> ...
>> +      /* Calculate bitpos and offset of the next field.  */
>> +      get_inner_reference (build3 (COMPONENT_REF,
>> +                                  TREE_TYPE (exp),
>> +                                  TREE_OPERAND (exp, 0),
>> +                                  fld, NULL_TREE),
>> +                       &tbitsize,&end_bitpos,&end_offset,
>> +                       &tmode,&tunsignedp,&tvolatilep, true);
>>
>> case is not correct anyway, fld may have variable position
>> (non-INTEGER_CST DECL_FIELD_OFFSET), you can't
>> assume
>
> Innermost here means "a" in a.i.j.k.l?  If so, this is what we're currently
> doing, *byte_offset is the start of the bit region, and *bit_offset is the
> offset from that.
>
> First, I thought we couldn't get a variable position here because we are now
> handling that case at the beginning of the function with:
>
>  /* Be as conservative as possible on variable offsets.  */
>  if (TREE_OPERAND (exp, 2)
>      && !host_integerp (TREE_OPERAND (exp, 2), 1))
>    {
>      *byte_offset = TREE_OPERAND (exp, 2);
>      *maxbits = BITS_PER_UNIT;
>      *bit_offset = 0;
>      return;
>    }
>
> And even if we do get a variable position, I have so far being able to get
> away with this...

Did you test Ada and enable the C++ memory model? ;)

Btw, even if the bitfield we access (and thus the whole region) is at a
constant offset, the field _following_ the bitregion (the one you query
above with get_inner_reference) can be at variable offset.  I suggest
to simply not include any padding in that case (which would be,
TREE_CODE (DECL_FIELD_OFFSET (fld)) != INTEGER_CST).

>>
>> +      *maxbits = TREE_INT_CST_LOW (maxbits_tree);
>>
>> this thus.
>
> ...because the call to fold_build2 immediately preceding this will fold away
> the variable offset.

You hope so ;)

> Is what you want, that we call get_inner_reference once, and then use
> DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET to calculate any subsequent bit
> offset?  I found this to be quite tricky with padding, and such, but am
> willing to give it a whirl again.

Yes.

> However, could I beg you to reconsider this, and get something working
> first, only later concentrating on removing the get_inner_reference() calls,
> and performing any other tweaks/optimizations?

Sure, it's fine to tweak this in a followup.

Thanks,
Richard.

> Aldy
>

On Wed, Aug 31, 2011 at 9:45 AM, Richard Guenther
<richard.guenther@gmail.com> wrote:
> On Tue, Aug 30, 2011 at 5:01 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>> [I'm going to respond to this piece-meal, to make sure I don't drop
>> anything.  My apologies for the long thread, but I'm pretty sure it's in
>> everybody's kill file by now.]
>>
>>> +  /* Be as conservative as possible on variable offsets.  */
>>> +  if (TREE_OPERAND (exp, 2)
>>> +&&  !host_integerp (TREE_OPERAND (exp, 2), 1))
>>> +    {
>>> +      *byte_offset = TREE_OPERAND (exp, 2);
>>> +      *maxbits = BITS_PER_UNIT;
>>> +      return;
>>> +    }
>>>
>>> shouldn't this be at the very beginning of the function?  Because
>>> you've set *bit_offset to an offset that was _not_ calculated relative
>>
>> Sure.  I assume in this case, *bit_offset would be 0, right?
>
> It would be DECL_FIELD_BIT_OFFSET of that field.  Oh, and
> *byte_offset would be
>
> *byte_offset = size_binop (MULT_EXPR, TREE_OPERAND (exp, 2),
>                                       size_int (DECL_OFFSET_ALIGN
> (field) / BITS_PER_UNIT));
>
> see expr.c:component_ref_field_offset () (which you conveniently
> could use here).
>
> Note that both TREE_OPERAND (exp, 2) and compoment_ref_field_offset
> return offsets relative to the immediate containing struct type, not
> relative to the base object like get_inner_reference does ...
> (where it is still unclear to me what we are supposed to return from this
> function ...)
>
> Thus, conservative would be using get_inner_reference here, if the
> offset is supposed to be relative to the base object.

That said, shouldn't *maxbits not at least make sure to cover the field itself?

> Richard.
>

>> *bit_offset = (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (fld))
>>                * BITS_PER_UNIT
>>                + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (fld)))
>>   - (TREE_INT_CST_LOW (DECL_FIELD_OFFSET (bitregion_start))
>>      * BITS_PER_UNIT
>>      + TREE_INT_CST_LOW (DECL_FIELD_BIT_OFFSET (bitregion_start)));
>>
>> (Yes, I know we can factor out the BITS_PER_UNIT and only do one
>> multiplication, it's just easier to read this way.)
>>
>> Is this what you had in mind?
>
> Yes.  For convenience I'd simply use double_ints for the intermediate
> calculations.

Ok, let's leave it like this for now.  I have added a FIXME note, and we 
can optimize this after we get everything working.

>>> Sure.  I assume in this case, *bit_offset would be 0, right?
>>
>> It would be DECL_FIELD_BIT_OFFSET of that field.  Oh, and
>> *byte_offset would be
>>
>> *byte_offset = size_binop (MULT_EXPR, TREE_OPERAND (exp, 2),
>>                                        size_int (DECL_OFFSET_ALIGN
>> (field) / BITS_PER_UNIT));
>>
>> see expr.c:component_ref_field_offset () (which you conveniently
>> could use here).
>>
>> Note that both TREE_OPERAND (exp, 2) and compoment_ref_field_offset
>> return offsets relative to the immediate containing struct type, not
>> relative to the base object like get_inner_reference does ...
>> (where it is still unclear to me what we are supposed to return from this
>> function ...)

Ok, I see where your confusion lies.  The function is supposed to return 
a byte offset from the base object, none of this containing object or 
immediate struct, or whatever.  Base object, as in "a" in a.i.j.k, as in 
what you get back from get_base_address().

Originally everything was calculated with get_inner_reference(), which 
is relative to the base object, but now we have this hodge podge of 
get_inner_reference() calls with ad-hoc calculations and optimizations. 
  Gladly, we've agreed to use get_inner_reference() and optimize at a 
later time.

So... base object throughout, anything else is a mistake on my part.

BTW, this whole variable length offset I still can't trigger.  I know 
you want to cater to Ada, but does it even make sense to enable the C++ 
memory model in Ada?  Who would ever do this?  Be that as it may, I'll 
humor you and handle it.

>> Thus, conservative would be using get_inner_reference here, if the
>> offset is supposed to be relative to the base object.
>
> That said, shouldn't *maxbits not at least make sure to cover the field itself?

Is this what you want?

   /* Be as conservative as possible on variable offsets.  */
   if (TREE_OPERAND (exp, 2)
       && !host_integerp (TREE_OPERAND (exp, 2), 1))
     {
       get_inner_reference (build3 (COMPONENT_REF,
				   TREE_TYPE (exp),
				   TREE_OPERAND (exp, 0),
				   field, NULL_TREE),
			   &tbitsize, &start_bitpos, &start_offset,
			   &tmode, &tunsignedp, &tvolatilep, true);

       *byte_offset = start_offset ? start_offset : size_zero_node;
       *bit_offset = start_bitpos;
       *maxbits = tbitsize;
       return;
     }

> Did you test Ada and enable the C++ memory model? ;)

See my earlier comment on Ada.  Who would ever use the C++ memory model 
on Ada?

> Btw, even if the bitfield we access (and thus the whole region) is at a
> constant offset, the field _following_ the bitregion (the one you query
> above with get_inner_reference) can be at variable offset.  I suggest
> to simply not include any padding in that case (which would be,
> TREE_CODE (DECL_FIELD_OFFSET (fld)) != INTEGER_CST).

I still have not found a place where we get a variable offset here 
(after folding the computation).  How about we put a gcc_assert() along 
with a big fat comment with your above suggestion when we encounter 
this.  Or can you give me an example of this case?

>> Is what you want, that we call get_inner_reference once, and then use
>> DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET to calculate any subsequent bit
>> offset?  I found this to be quite tricky with padding, and such, but am
>> willing to give it a whirl again.
>
> Yes.

I have added a comment to this effect, and will address it along with 
the get_inner_reference() removal you have suggested as a followup.

On Wed, Aug 31, 2011 at 6:53 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>>>> Sure.  I assume in this case, *bit_offset would be 0, right?
>>>
>>> It would be DECL_FIELD_BIT_OFFSET of that field.  Oh, and
>>> *byte_offset would be
>>>
>>> *byte_offset = size_binop (MULT_EXPR, TREE_OPERAND (exp, 2),
>>>                                       size_int (DECL_OFFSET_ALIGN
>>> (field) / BITS_PER_UNIT));
>>>
>>> see expr.c:component_ref_field_offset () (which you conveniently
>>> could use here).
>>>
>>> Note that both TREE_OPERAND (exp, 2) and compoment_ref_field_offset
>>> return offsets relative to the immediate containing struct type, not
>>> relative to the base object like get_inner_reference does ...
>>> (where it is still unclear to me what we are supposed to return from this
>>> function ...)
>
> Ok, I see where your confusion lies.  The function is supposed to return a
> byte offset from the base object, none of this containing object or
> immediate struct, or whatever.  Base object, as in "a" in a.i.j.k, as in
> what you get back from get_base_address().
>
> Originally everything was calculated with get_inner_reference(), which is
> relative to the base object, but now we have this hodge podge of
> get_inner_reference() calls with ad-hoc calculations and optimizations.
>  Gladly, we've agreed to use get_inner_reference() and optimize at a later
> time.
>
> So... base object throughout, anything else is a mistake on my part.
>
> BTW, this whole variable length offset I still can't trigger.  I know you
> want to cater to Ada, but does it even make sense to enable the C++ memory
> model in Ada?  Who would ever do this?  Be that as it may, I'll humor you
> and handle it.
>
>>> Thus, conservative would be using get_inner_reference here, if the
>>> offset is supposed to be relative to the base object.
>>
>> That said, shouldn't *maxbits not at least make sure to cover the field
>> itself?
>
> Is this what you want?
>
>  /* Be as conservative as possible on variable offsets.  */
>  if (TREE_OPERAND (exp, 2)
>      && !host_integerp (TREE_OPERAND (exp, 2), 1))
>    {
>      get_inner_reference (build3 (COMPONENT_REF,
>                                   TREE_TYPE (exp),
>                                   TREE_OPERAND (exp, 0),
>                                   field, NULL_TREE),
>                           &tbitsize, &start_bitpos, &start_offset,
>                           &tmode, &tunsignedp, &tvolatilep, true);
>
>      *byte_offset = start_offset ? start_offset : size_zero_node;
>      *bit_offset = start_bitpos;
>      *maxbits = tbitsize;
>      return;
>    }

Yes, exactly.

Richard.

On Wed, Aug 31, 2011 at 8:09 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>> Did you test Ada and enable the C++ memory model? ;)
>
> See my earlier comment on Ada.  Who would ever use the C++ memory model on
> Ada?

People interoperating Ada with C++.  Our bug triager Zdenek who
figures out the --param?

>> Btw, even if the bitfield we access (and thus the whole region) is at a
>> constant offset, the field _following_ the bitregion (the one you query
>> above with get_inner_reference) can be at variable offset.  I suggest
>> to simply not include any padding in that case (which would be,
>> TREE_CODE (DECL_FIELD_OFFSET (fld)) != INTEGER_CST).
>
> I still have not found a place where we get a variable offset here (after
> folding the computation).  How about we put a gcc_assert() along with a big
> fat comment with your above suggestion when we encounter this.  Or can you
> give me an example of this case?

My point is, the middle-end infrastructure makes it possible for this
case to appear, and it seems to be easy to handle conservatively.
There isn't a need to wait for users to run into an ICE or an assert we put
there IMHO.  If I'd be fluent in Ada I'd write you a testcase, but I ain't.

>>> Is what you want, that we call get_inner_reference once, and then use
>>> DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET to calculate any subsequent bit
>>> offset?  I found this to be quite tricky with padding, and such, but am
>>> willing to give it a whirl again.
>>
>> Yes.
>
> I have added a comment to this effect, and will address it along with the
> get_inner_reference() removal you have suggested as a followup.

Thanks,
Richard.

> >> Did you test Ada and enable the C++ memory model? ;)
> >
> > See my earlier comment on Ada.  Who would ever use the C++ memory model on
> > Ada?
> 
> People interoperating Ada with C++.  Our bug triager Zdenek who
> figures out the --param?

Right, that's one example. There are also actually some similarities between
the C++ memory model and the Ada language, so it's not so unconcievable
that Ada would like to take advantage of some of these capabilities.

Arno

> My point is, the middle-end infrastructure makes it possible for this
> case to appear, and it seems to be easy to handle conservatively.
> There isn't a need to wait for users to run into an ICE or an assert we put
> there IMHO.  If I'd be fluent in Ada I'd write you a testcase, but I ain't.

Ughh, this is getting messier.

Ok, I propose keeping track of the field prior (lastfld), calling 
get_inner_reference() and adding DECL_SIZE (or tbitsize if you prefer) 
to calculate maxbits without the padding.

Notice the comment at the top.  We can get rid of yet another call to 
get_inner_reference later.

Is this what you had in mind?

BTW, we don't need to round up to the next byte here, do we?

Thanks.
Aldy

   /* If we found the end of the bit field sequence, include the
      padding up to the next field...  */
   if (fld)
     {
       tree end_offset, t;
       HOST_WIDE_INT end_bitpos;

       /* FIXME: Only call get_inner_reference once (at the beginning
	 of the bit region), and use
	 DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET throughout to
	 calculate any subsequent bit offset.  */

       /* Even if the bitfield we access (and thus the whole region) is
	 at a constant offset, the field _following_ the bitregion can
	 be at variable offset.  In this case, do not include any
	 padding.  This is mostly for Ada.  */
       if (TREE_CODE (DECL_FIELD_OFFSET (fld)) != INTEGER_CST)
	{
	  get_inner_reference (build3 (COMPONENT_REF,
				       TREE_TYPE (exp),
				       TREE_OPERAND (exp, 0),
				       lastfld, NULL_TREE),
			       &tbitsize, &end_bitpos, &end_offset,
			       &tmode, &tunsignedp, &tvolatilep, true);

	  /* Calculate the size of the bit region up the last
	     bitfield, excluding any subsequent padding.

	     t = (end_byte_off - start_byte_offset) * 8 + end_bit_off */
	  end_offset = end_offset ? end_offset : size_zero_node;
	  t = fold_build2 (PLUS_EXPR, size_type_node,
			   fold_build2 (MULT_EXPR, size_type_node,
					fold_build2 (MINUS_EXPR, size_type_node,
						     end_offset,
						     *byte_offset),
					build_int_cst (size_type_node,
						       BITS_PER_UNIT)),
			   build_int_cst (size_type_node,
					  end_bitpos));
	  /* Add the bitsize of the last field.  */
	  t = fold_build2 (PLUS_EXPR, size_type_node,
			   t, DECL_SIZE (lastfld));

	  *maxbits = tree_low_cst (t, 1);
	  return;
	}
...
...
...

[Jason, can you pontificate on tail-padding and the upcoming C++ 
standard with regards to bitfields?]

> so you wasn't convinced about my worries about tail-padding re-use?

To answer your question, I believe we can't touch past the last field 
(into the padding) if the subsequent record will be packed into the 
first's padding.

struct A {
   int a : 17;
};
struct B : public A {
   char c;
};

So here, if <c> gets packed into the tail-padding of A, we can't touch 
the padding of A when storing into <a>.  These are different structures, 
and I assume would be treated as nested structures, which are distinct 
memory locations.

Is there a way of distinguishing this particular variant (possible 
tail-packing), or will we have to disallow storing into the record tail 
padding altogether?  That would seriously suck.

Aldy

On 09/01/2011 10:52 AM, Aldy Hernandez wrote:
> To answer your question, I believe we can't touch past the last field
> (into the padding) if the subsequent record will be packed into the
> first's padding.

Right.

> struct A {
>   int a : 17;
> };
> struct B : public A {
>   char c;
> };
>
> So here, if <c> gets packed into the tail-padding of A, we can't touch
> the padding of A when storing into <a>.

But that doesn't apply to this testcase because A is a POD class, so we 
don't mess with its tail padding.

> Is there a way of distinguishing this particular variant (possible
> tail-packing), or will we have to disallow storing into the record tail
> padding altogether? That would seriously suck.

Basically you can only touch the size of the CLASSTYPE_AS_BASE variant. 
  For many classes this will be the same as the size of the class itself.

Jason

>> Is there a way of distinguishing this particular variant (possible
>> tail-packing), or will we have to disallow storing into the record tail
>> padding altogether? That would seriously suck.
>
> Basically you can only touch the size of the CLASSTYPE_AS_BASE variant.
> For many classes this will be the same as the size of the class itself.

All this code is in the middle end, so we're language agnostic.

What do we need here, a hook to query the front-end, or is it too late? 
  Or will we have to play it conservative and never touch the padding 
(regardless of language)?

Aldy

On 09/01/2011 11:10 AM, Aldy Hernandez wrote:
>> Basically you can only touch the size of the CLASSTYPE_AS_BASE variant.
>> For many classes this will be the same as the size of the class itself.
>
> All this code is in the middle end, so we're language agnostic.
>
> What do we need here, a hook to query the front-end, or is it too late?
> Or will we have to play it conservative and never touch the padding
> (regardless of language)?

I think it would make sense to expose this information to the back end 
somehow.  A hook would do the trick: call it type_data_size or 
type_min_size or some such, which in the C++ front end would return 
TYPE_SIZE (CLASSTYPE_AS_BASE (t)) for classes or just TYPE_SIZE for 
other types.

Jason

On Thu, Sep 1, 2011 at 4:16 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>> My point is, the middle-end infrastructure makes it possible for this
>> case to appear, and it seems to be easy to handle conservatively.
>> There isn't a need to wait for users to run into an ICE or an assert we
>> put
>> there IMHO.  If I'd be fluent in Ada I'd write you a testcase, but I
>> ain't.
>
> Ughh, this is getting messier.
>
> Ok, I propose keeping track of the field prior (lastfld), calling
> get_inner_reference() and adding DECL_SIZE (or tbitsize if you prefer) to
> calculate maxbits without the padding.
>
> Notice the comment at the top.  We can get rid of yet another call to
> get_inner_reference later.
>
> Is this what you had in mind?

That could work also for the tail-padding re-use case, yes.  Note that
DECL_SIZE of the field is just the last fieds bit-precision, so ..

> BTW, we don't need to round up to the next byte here, do we?

.. rounding up to the next byte cannot hurt (dependent on what the
caller will do with that value).

Note that with all this mess I'll re-iterate some of my initial thoughts.
1) why not do this C++ (or C) specific stuff in the frontends, maybe
at gimplifying/genericization time?  That way you wouldn't need to
worry about middle-end features but you could rely solely on what
C/C++ permit.  It is, after all, C++ _frontend_ semantics that we
enforce here, in the middle-end, which looks out-of-place.
2) all this information we try to re-construct here is sort-of readily
available when we layout the record (thus, from layout_type and
friends).  We should really, really try to preserve it there, rather
than jumping through hoops here (ideally we'd have an
(unused?) FIELD_DECL that covers the whole "bitfield group"
followed by the individual FIELD_DECLS for the bits (yep, they'd
overlap that group FIELD_DECL), and they would refer back to
that group FIELD_DECL)

Is the C++ memory model stuff going to be "ready" for 4.7?

Thanks,
Richard.

> Thanks.
> Aldy
>
>  /* If we found the end of the bit field sequence, include the
>     padding up to the next field...  */
>  if (fld)
>    {
>      tree end_offset, t;
>      HOST_WIDE_INT end_bitpos;
>
>      /* FIXME: Only call get_inner_reference once (at the beginning
>         of the bit region), and use
>         DECL_FIELD_OFFSET+DECL_FIELD_BIT_OFFSET throughout to
>         calculate any subsequent bit offset.  */
>
>      /* Even if the bitfield we access (and thus the whole region) is
>         at a constant offset, the field _following_ the bitregion can
>         be at variable offset.  In this case, do not include any
>         padding.  This is mostly for Ada.  */
>      if (TREE_CODE (DECL_FIELD_OFFSET (fld)) != INTEGER_CST)
>        {
>          get_inner_reference (build3 (COMPONENT_REF,
>                                       TREE_TYPE (exp),
>                                       TREE_OPERAND (exp, 0),
>                                       lastfld, NULL_TREE),
>                               &tbitsize, &end_bitpos, &end_offset,
>                               &tmode, &tunsignedp, &tvolatilep, true);
>
>          /* Calculate the size of the bit region up the last
>             bitfield, excluding any subsequent padding.
>
>             t = (end_byte_off - start_byte_offset) * 8 + end_bit_off */
>          end_offset = end_offset ? end_offset : size_zero_node;
>          t = fold_build2 (PLUS_EXPR, size_type_node,
>                           fold_build2 (MULT_EXPR, size_type_node,
>                                        fold_build2 (MINUS_EXPR,
> size_type_node,
>                                                     end_offset,
>                                                     *byte_offset),
>                                        build_int_cst (size_type_node,
>                                                       BITS_PER_UNIT)),
>                           build_int_cst (size_type_node,
>                                          end_bitpos));
>          /* Add the bitsize of the last field.  */
>          t = fold_build2 (PLUS_EXPR, size_type_node,
>                           t, DECL_SIZE (lastfld));
>
>          *maxbits = tree_low_cst (t, 1);
>          return;
>        }
> ...
> ...
> ...
>

On Thu, Sep 1, 2011 at 5:19 PM, Jason Merrill <jason@redhat.com> wrote:
> On 09/01/2011 11:10 AM, Aldy Hernandez wrote:
>>>
>>> Basically you can only touch the size of the CLASSTYPE_AS_BASE variant.
>>> For many classes this will be the same as the size of the class itself.
>>
>> All this code is in the middle end, so we're language agnostic.
>>
>> What do we need here, a hook to query the front-end, or is it too late?
>> Or will we have to play it conservative and never touch the padding
>> (regardless of language)?
>
> I think it would make sense to expose this information to the back end
> somehow.  A hook would do the trick: call it type_data_size or type_min_size
> or some such, which in the C++ front end would return TYPE_SIZE
> (CLASSTYPE_AS_BASE (t)) for classes or just TYPE_SIZE for other types.

That's too late to work with LTO, you'd need to store that information
permanently
somewhere.

Maybe move this whole C++ specific bitfield handling where it belongs,
namely to the C++ frontend?

I suggest to always not re-use tail padding for now (I believe if your
parent object is a COMPONENT_REF, thus, x.parent.bitfield,
you can use the TYPE_SIZE vs. field-decl DECL_SIZE discrepance
to decide about whether the tail-padding was reused, but please
double-check that ;)))

Richard.

> Jason
>
>

> Note that with all this mess I'll re-iterate some of my initial thoughts.
> 1) why not do this C++ (or C) specific stuff in the frontends, maybe
> at gimplifying/genericization time?  That way you wouldn't need to
> worry about middle-end features but you could rely solely on what
> C/C++ permit.  It is, after all, C++ _frontend_ semantics that we
> enforce here, in the middle-end, which looks out-of-place.

The front-end, really?  After all this going back and forth?  After you 
were all so worried about Ada, and now you're ditching it in favor of 
handling only C++?

> Is the C++ memory model stuff going to be "ready" for 4.7?

No, not if you expect me rewrite things every day.

On Fri, Sep 2, 2011 at 2:49 PM, Aldy Hernandez <aldyh@redhat.com> wrote:
>
>> Note that with all this mess I'll re-iterate some of my initial thoughts.
>> 1) why not do this C++ (or C) specific stuff in the frontends, maybe
>> at gimplifying/genericization time?  That way you wouldn't need to
>> worry about middle-end features but you could rely solely on what
>> C/C++ permit.  It is, after all, C++ _frontend_ semantics that we
>> enforce here, in the middle-end, which looks out-of-place.
>
> The front-end, really?  After all this going back and forth?

Well, I'm fine with handling it in the middle-end if it's correct there.

> After you were
> all so worried about Ada, and now you're ditching it in favor of handling
> only C++?

I'm just showing you a possible solution for where you'd not need to
worry ;)  Consider LTOing an Ada and a C++ module - you need to
enable the C++ memory model at link-time so it is in effect when we
process bit-fields.  That will automatically enable it for the Ada pieces, too.

>> Is the C++ memory model stuff going to be "ready" for 4.7?
>
> No, not if you expect me rewrite things every day.

I don't expect you to rewrite things every day.

Don't read every comment I make as a definite decision and order to
you.  I am a mere mortal, too, and the bitfield thing is, I must admit,
still partially a mystery to myself (which is why I keep asking questions
instead of simply providing you with definite answers).  After all I pushed
back my idea of lowering bitfield accesses somewhere on GIMPLE and
I'm not sure if I get back to it for 4.7.  And I definitely would consider
2) for that work.

Btw, it would be nice if I weren't the only one reading your updated
patches :/  I'm just punching holes where I see them and hope I and
you learn something in that process.

Richard.

On 09/02/2011 04:53 AM, Richard Guenther wrote:
> On Thu, Sep 1, 2011 at 5:19 PM, Jason Merrill<jason@redhat.com>  wrote:
>> I think it would make sense to expose this information to the back end
>> somehow.  A hook would do the trick: call it type_data_size or type_min_size
>> or some such, which in the C++ front end would return TYPE_SIZE
>> (CLASSTYPE_AS_BASE (t)) for classes or just TYPE_SIZE for other types.
>
> That's too late to work with LTO, you'd need to store that information
> permanently somewhere.

OK.

> Maybe move this whole C++ specific bitfield handling where it belongs,
> namely to the C++ frontend?

I don't think that is the way to go; C is adopting the same memory 
model, and this is the only sane thing to do with bit-fields.

> I suggest to always not re-use tail padding for now (I believe if your
> parent object is a COMPONENT_REF, thus, x.parent.bitfield,
> you can use the TYPE_SIZE vs. field-decl DECL_SIZE discrepancy
> to decide about whether the tail-padding was reused, but please
> double-check that ;)))

But you don't always have a COMPONENT_REF; you still need to avoid 
touching the tail padding when you just have a pointer to the type 
because it might be a base sub-object.

I wonder what would break if C++ just set TYPE_SIZE to the as-base size?

Jason

On Fri, Sep 2, 2011 at 4:10 PM, Jason Merrill <jason@redhat.com> wrote:
> On 09/02/2011 04:53 AM, Richard Guenther wrote:
>>
>> On Thu, Sep 1, 2011 at 5:19 PM, Jason Merrill<jason@redhat.com>  wrote:
>>>
>>> I think it would make sense to expose this information to the back end
>>> somehow.  A hook would do the trick: call it type_data_size or
>>> type_min_size
>>> or some such, which in the C++ front end would return TYPE_SIZE
>>> (CLASSTYPE_AS_BASE (t)) for classes or just TYPE_SIZE for other types.
>>
>> That's too late to work with LTO, you'd need to store that information
>> permanently somewhere.
>
> OK.
>
>> Maybe move this whole C++ specific bitfield handling where it belongs,
>> namely to the C++ frontend?
>
> I don't think that is the way to go; C is adopting the same memory model,
> and this is the only sane thing to do with bit-fields.
>
>> I suggest to always not re-use tail padding for now (I believe if your
>> parent object is a COMPONENT_REF, thus, x.parent.bitfield,
>> you can use the TYPE_SIZE vs. field-decl DECL_SIZE discrepancy
>> to decide about whether the tail-padding was reused, but please
>> double-check that ;)))
>
> But you don't always have a COMPONENT_REF; you still need to avoid touching
> the tail padding when you just have a pointer to the type because it might
> be a base sub-object.
>
> I wonder what would break if C++ just set TYPE_SIZE to the as-base size?

Good question.  Probably argument passing, as the as-base size wouldn't
get a proper mode assigned form layout_type then(?) for small structs?

Maybe worth a try ...

Richard.

> Jason
>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/02/11 02:48, Richard Guenther wrote:
> 
> Note that with all this mess I'll re-iterate some of my initial
> thoughts. 1) why not do this C++ (or C) specific stuff in the
> frontends, maybe at gimplifying/genericization time?  That way you
> wouldn't need to worry about middle-end features but you could rely
> solely on what C/C++ permit.  It is, after all, C++ _frontend_
> semantics that we enforce here, in the middle-end, which looks
> out-of-place.
Well, it's worth keeping in mind that fixing the way we handle bitfields
is just one piece of a larger project.  Furthermore, many of the ideas
in the C++ memory model are applicable to other languages.

However, I must admit, I'm somewhat at a loss, I thought we were doing
all this in stor-layout.c at the time we layout the structure's memory
form then just trying to keep the code generator and optimizers from
mucking things up by combining accesses and the like.

Clearly I'm going to need to sit down and review the code as well.
Which means learning about a part of GCC I've largely been able to
ignore...  Sigh...

jeff
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOYT2jAAoJEBRtltQi2kC7BF4IAJWwqzsPUdhsHaodlUfm1LRu
JpMojs04wPVfu12+8zcXaWfitjOhEPJDPcZ5c0AHb74NRPJINJmjDvSBZWfFazaE
CrZE/U9IBz7Ay8s/gw//uMIWDS8lNjjYFxpqn6VMUpY2F/4QSkDZtaTlsTfm8YfU
+IZnwq82Johh8MzGDRuYY0HBKRdAotGS2F+SycdOxBGW6hnW0WR/2pt0BpIxNYgl
ro0dOgSptWoEmOFzhlN9pVsFvImSjXVlbV9GnF4AsDrh9x9FIaFIpvhgZMUU8wc+
Akg2jgLy2hhysQ9JtES0rL9qrptPPcQVqCL8ct/sLB85vYw7oUxwSenj7w8mwmE=
=yjMx
-----END PGP SIGNATURE-----

On 09/02/2011 10:38 AM, Richard Guenther wrote:
> On Fri, Sep 2, 2011 at 4:10 PM, Jason Merrill<jason@redhat.com>  wrote:
>> I wonder what would break if C++ just set TYPE_SIZE to the as-base size?
>
> Good question.  Probably argument passing, as the as-base size wouldn't
> get a proper mode assigned form layout_type then(?) for small structs?

Classes for which the as-base size is different are passed by invisible 
reference, so that wouldn't be an issue.

But layout_decl would get the wrong size for variables and fields of the 
type, so that won't work.

Perhaps it's time to get serious about the change I talked about in 
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22488#c42 ...

Jason