Fix compiling large files

On 03/10/2016 08:20 PM, DJ Delorie wrote:
> I'm moving on to Plan C but I put a copy of the file on
> .../dj/foo.c.gz (195Mb) if anyone wants to find out
> why there's a 16Gb limit compiling it...

With just the following, we successfully compile your file.

It takes about 25 minutes and memory use tops out around 40GB.
Which still seems insane for a 1.6GB input file consisting
primarily of data for a static array, but that's a
different problem.

At this point we usually have a PR to go with all stage4
changes.  But a meaningful PR is difficult to create, since
the attachment would be too large.  Perhaps a generator could
be created, but since it wouldn't go in the testsuite it seems
like a waste of time.

Thoughts?


r~


	* line-map.c (new_linemap): Make alloc_size a size_t.

On Tue, Mar 15, 2016 at 03:31:44PM -0700, Richard Henderson wrote:
> On 03/10/2016 08:20 PM, DJ Delorie wrote:
> >I'm moving on to Plan C but I put a copy of the file on
> >.../dj/foo.c.gz (195Mb) if anyone wants to find out
> >why there's a 16Gb limit compiling it...
> 
> With just the following, we successfully compile your file.
> 
> It takes about 25 minutes and memory use tops out around 40GB.
> Which still seems insane for a 1.6GB input file consisting
> primarily of data for a static array, but that's a
> different problem.
> 
> At this point we usually have a PR to go with all stage4
> changes.  But a meaningful PR is difficult to create, since
> the attachment would be too large.  Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
> 
> Thoughts?
> 
> 
> r~
> 
> 
> 	* line-map.c (new_linemap): Make alloc_size a size_t.

Ok for stage4.

> diff --git a/libcpp/line-map.c b/libcpp/line-map.c
> index 1fb634a..80d4e6b 100644
> --- a/libcpp/line-map.c
> +++ b/libcpp/line-map.c
> @@ -376,7 +376,7 @@ new_linemap (struct line_maps *set,
>    if (LINEMAPS_USED (set, macro_map_p) == LINEMAPS_ALLOCATED (set, macro_map_p))
>      {
>        /* We ran out of allocated line maps. Let's allocate more.  */
> -      unsigned alloc_size;
> +      size_t alloc_size;
> 
>        /* Cast away extern "C" from the type of xrealloc.  */
>        line_map_realloc reallocator = (set->reallocator

	Jakub

> At this point we usually have a PR to go with all stage4
> changes.  But a meaningful PR is difficult to create, since
> the attachment would be too large.  Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
> 
> Thoughts?

CPP macros grow exponentially, we could do it there, if we can get it
to preserve (inject?) line breaks.  But I wouldn't want to be the poor
developer on a 4Gb 32-bit system trying to run it...

On 03/15/2016 04:31 PM, Richard Henderson wrote:
> On 03/10/2016 08:20 PM, DJ Delorie wrote:
>> I'm moving on to Plan C but I put a copy of the file on
>> .../dj/foo.c.gz (195Mb) if anyone wants to find out
>> why there's a 16Gb limit compiling it...
>
> With just the following, we successfully compile your file.
>
> It takes about 25 minutes and memory use tops out around 40GB.
> Which still seems insane for a 1.6GB input file consisting
> primarily of data for a static array, but that's a
> different problem.
>
> At this point we usually have a PR to go with all stage4
> changes.  But a meaningful PR is difficult to create, since
> the attachment would be too large.  Perhaps a generator could
> be created, but since it wouldn't go in the testsuite it seems
> like a waste of time.
>
> Thoughts?
>
>
> r~
>
>
>      * line-map.c (new_linemap): Make alloc_size a size_t.
OK.
jeff

Hello.

As mentioned in PR71920, I would like to backport the change to GCC 4.9 and 5.

May I install the patch after proper testing?

Thanks,
Martin

On Tue, Jul 19, 2016 at 1:19 PM, Martin Liška <mliska@suse.cz> wrote:
> Hello.
>
> As mentioned in PR71920, I would like to backport the change to GCC 4.9 and 5.
>
> May I install the patch after proper testing?

Yes, it's pretty obvious.

Richard.

> Thanks,
> Martin