Fix PR preprocessor/58893 access to uninitialized memory

Hi Jeff,

On Mon, 29 Sep 2014 22:40:58, Jeff Law wrote:
>
> On 09/27/14 03:53, Bernd Edlinger wrote:
>>>> Comment before this change. Someone not familiar with this code is
>>>> going to have no idea why these two lines exist.
>>>>
>>>
>>> Ok, I added a comment now, do you like it?
> Yes.
>
>
>>>
>>>> Please try to include a testcase. If you're having trouble reproducing
>>>> on the trunk, you could use MALLOC_PERTURB per c#8 in the bug report.
>>>> If there's a way to set environment variables in our testing framework
>>>> that may be a reasonable way to test (if you need to do that, limit
>>>> testing to linux targets as we'll have a dependency on glibc features).
>>>>
>>>
>>> For whatever reason, the first -include must end with a pragma
>>> as in the PR, and MALLOC_PERTURB_ must be set to something.
>>> Then we get an ICE, otherwise we get an error message without line number.
>>> I tried to make this a valid test case, but that might be less trivial than
>>> it looks at first sight.
>
>>>
>>> I tried to set MALLOC_PERTURB_=123 globally, like this:
>>>
>>> MALLOC_PERTURB_=123 make -k check
>>>
>>> but then this happened:
> Sigh. Yea, I guess if we're hitting the allocator insanely hard,
> scrubbing memory might turn out to slow things down in a significant
> way. Or it may simply be the case that we're using free'd memory in
> some way and with the MALLOC_PERTURB changes we're in an infinite loop
> in the dumping code or something similar.
>

Yeah, that is an interesting thing.
I debugged that, and it turns out, that this is just incredibly slow.
It seems to be in the macro expansion of this construct:

#define t16(x) x x x x x x x x x x x x x x x x
#define M (sizeof (t16(t16(t16(t16(t16(" ")))))) - 1)

libcpp is calling realloc 1.000.000 times for this, resizing
the memory by just one byte at a time. And the worst case of
realloc is O(n), so in the worst case realloc would have
to copy 1/2 * 1.000.000^2 bytes = 500 GB of memory.

With this little change in libcpp, the test suite passed, without any
further regressions:


I will prepare a patch for that later.

Interestingly, if I define MALLOC_CHECK_=3 _and_ MALLOC_PERTURB_
this test passes, even without the above change,
but the test case 
      gfortran.dg/realloc_on_assign_5.f03 fails in this configuration,
which is a known bug: PR 47674. However it passes when only MALLOC_PERTURB_
is defined.

Weird...

>
>>>
>>>
>>> Well, I added a test case, but it does not reliably fail without the
>>> patch, because setting
>>> MALLOC_PERTURB_ causes too much trouble at this time.
>>>
>>> I would propose to set MALLOC_PERTURB_ globally at a later time.
> Sorry, just to be clear, I wasn't suggesting to set it globally, but
> just for the duration of this test as a potentially easier way to
> trigger the failure.
>
> However, it may make sense to do that at some point. I also think that
> Jakub bootstraps and runs the regression suite with valgrind late in the
> release cycle, which would catch this problem if it raises its head again.
>
>>>
>>> Boot-Strapped & Regression-Tested on x86_64-linux-gnu.
>>> Ok for trunk?
> Yes, this is OK for the trunk.
>

Thanks!
Bernd.

> jeff
>

On 09/30/14 03:01, Bernd Edlinger wrote:
>> Sigh. Yea, I guess if we're hitting the allocator insanely hard,
>> scrubbing memory might turn out to slow things down in a significant
>> way. Or it may simply be the case that we're using free'd memory in
>> some way and with the MALLOC_PERTURB changes we're in an infinite loop
>> in the dumping code or something similar.
>>
>
> Yeah, that is an interesting thing.
> I debugged that, and it turns out, that this is just incredibly slow.
> It seems to be in the macro expansion of this construct:
>
> #define t16(x) x x x x x x x x x x x x x x x x
> #define M (sizeof (t16(t16(t16(t16(t16(" ")))))) - 1)
>
> libcpp is calling realloc 1.000.000 times for this, resizing
> the memory by just one byte at a time. And the worst case of
> realloc is O(n), so in the worst case realloc would have
> to copy 1/2 * 1.000.000^2 bytes = 500 GB of memory.
>
> With this little change in libcpp, the test suite passed, without any
> further regressions:
>
> --- libcpp/charset.c.jj    2014-08-19 07:34:31.000000000 +0200
> +++ libcpp/charset.c    2014-09-30 10:45:26.676954120 +0200
> @@ -537,6 +537,7 @@ convert_no_conversion (iconv_t cd ATTRIB
>     if (to->len + flen> to->asize)
>       {
>         to->asize = to->len + flen;
> +      to->asize *= 2;
>         to->text = XRESIZEVEC (uchar, to->text, to->asize);
>       }
>     memcpy (to->text + to->len, from, flen);
>
> I will prepare a patch for that later.
Thanks for digging into this.  We usually try to throttle this growth a 
little.  Something like this would be consistent with other cases in GCC:

to->asize += to->asize / 4;


>
> Interestingly, if I define MALLOC_CHECK_=3 _and_ MALLOC_PERTURB_
> this test passes, even without the above change,
> but the test case
>        gfortran.dg/realloc_on_assign_5.f03 fails in this configuration,
> which is a known bug: PR 47674. However it passes when only MALLOC_PERTURB_
> is defined.
>
> Weird...
Yea, but that's par for the course when dealing with memory errors.

Jeff