[1/2] Fix LTO bootstrap documentation

On Tue, Aug 31, 2010 at 03:17:56PM +0200, Paolo Bonzini wrote:
> On 08/31/2010 03:14 PM, Andi Kleen wrote:
> >
> >>No, config/bootstrap-lto.mk should be adjusted instead to use the
> >>options that Andi specifies. The doc change should be limited to the
> >>--enable-stage1-languages=c,lto part.
> >
> >The problem is that config/bootstrap-lto.mk doesn't work in the first place
> >(and I don't know how to fix it). It never built for me.
> >That is why I documented an alternative method.
> 
> Have you tried s/-flto/flags-you-proposed-to-use-instead/ in that file?

Hi Paolo,

I reran the test with BUILD_CONFIG and I cannot reproduce the earlier
failures I had (I had multiple earlier, always in the early built
of stage2). I added -frandom-seed=1 and -fuse-linker-plugin.

It might be related to -j* races. I had to drop down my -j
to small values because I found that without using -fwhopr it 
is unusable slow because multiple lto1s will make any reasonable machine swap.

Honza also had some issue with it maybe he can elaborate.

With the old failures not reproducible I guess the documentation
change is obsolete (unless I can reproduce this again) 
and it would be reasonable to simply change 
the build option to this:

Comments?
-Andi

On 09/01/2010 09:57 AM, Andi Kleen wrote:
> With the old failures not reproducible I guess the documentation
> change is obsolete (unless I can reproduce this again)
> and it would be reasonable to simply change
> the build option to this:
>
> diff --git a/config/bootstrap-lto.mk b/config/bootstrap-lto.mk
> index 14099a0..785d15f 100644
> --- a/config/bootstrap-lto.mk
> +++ b/config/bootstrap-lto.mk
> @@ -1,8 +1,8 @@
>   # This option enables LTO for stage2 and stage3.  It requires lto to
>   # be enabled for stage1 with --enable-stage1-languages.
>
> -STAGE2_CFLAGS += -flto
> -STAGE3_CFLAGS += -flto
> +STAGE2_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
> +STAGE3_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
>
>   # Ada fails to build with LTO, turn it off for now.
>   BOOT_ADAFLAGS += -fno-lto

Yes, this is okay when your patch 2 goes in.  Thanks!

Paolo

Paolo Bonzini <bonzini@gnu.org> writes:

> On 09/01/2010 09:57 AM, Andi Kleen wrote:
>> With the old failures not reproducible I guess the documentation
>> change is obsolete (unless I can reproduce this again)
>> and it would be reasonable to simply change
>> the build option to this:
>>
>> diff --git a/config/bootstrap-lto.mk b/config/bootstrap-lto.mk
>> index 14099a0..785d15f 100644
>> --- a/config/bootstrap-lto.mk
>> +++ b/config/bootstrap-lto.mk
>> @@ -1,8 +1,8 @@
>>   # This option enables LTO for stage2 and stage3.  It requires lto to
>>   # be enabled for stage1 with --enable-stage1-languages.
>>
>> -STAGE2_CFLAGS += -flto
>> -STAGE3_CFLAGS += -flto
>> +STAGE2_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
>> +STAGE3_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
>>
>>   # Ada fails to build with LTO, turn it off for now.
>>   BOOT_ADAFLAGS += -fno-lto
>
> Yes, this is okay when your patch 2 goes in.  Thanks!

Patch 2 is already in. I'll commit it then with a suitable changelog
after I reran testing.

Hmm thinking about it should probably fix the documentation
to say that gold is required too.

-Andi

On 09/01/2010 10:07 AM, Andi Kleen wrote:
> Paolo Bonzini<bonzini@gnu.org>  writes:
>
>> On 09/01/2010 09:57 AM, Andi Kleen wrote:
>>> With the old failures not reproducible I guess the documentation
>>> change is obsolete (unless I can reproduce this again)
>>> and it would be reasonable to simply change
>>> the build option to this:
>>>
>>> diff --git a/config/bootstrap-lto.mk b/config/bootstrap-lto.mk
>>> index 14099a0..785d15f 100644
>>> --- a/config/bootstrap-lto.mk
>>> +++ b/config/bootstrap-lto.mk
>>> @@ -1,8 +1,8 @@
>>>    # This option enables LTO for stage2 and stage3.  It requires lto to
>>>    # be enabled for stage1 with --enable-stage1-languages.
>>>
>>> -STAGE2_CFLAGS += -flto
>>> -STAGE3_CFLAGS += -flto
>>> +STAGE2_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
>>> +STAGE3_CFLAGS += -fwhopr=jobserver -fuse-linker-plugin -frandom-seed=1
>>>
>>>    # Ada fails to build with LTO, turn it off for now.
>>>    BOOT_ADAFLAGS += -fno-lto
>>
>> Yes, this is okay when your patch 2 goes in.  Thanks!
>
> Patch 2 is already in. I'll commit it then with a suitable changelog
> after I reran testing.
>
> Hmm thinking about it should probably fix the documentation
> to say that gold is required too.

Is -fuse-linker-plugin required for correctness?  Losing the ability to 
bootstrap-lto on Windows is a bit ugly.  We could use makefile magic to 
disable it when gold is not in use.

Paolo

> Is -fuse-linker-plugin required for correctness?  Losing the ability

Yes it's needed for the ar files (libbackend.a etc.)

collect2 without linker help doesn't handle ar.

> to bootstrap-lto on Windows is a bit ugly.  We could use makefile
> magic to disable it when gold is not in use.

I doubt it will work. That is at least if you really want
LTO code generation.

-Andi

On Wed, Sep 1, 2010 at 10:40 AM, Andi Kleen <andi@firstfloor.org> wrote:
>> Is -fuse-linker-plugin required for correctness?  Losing the ability
>
> Yes it's needed for the ar files (libbackend.a etc.)

But not for correctness.  As long as we do not use -fwhole-program you'll
just not get any link-time optimization for libbackend.a and friends.

> collect2 without linker help doesn't handle ar.
>
>> to bootstrap-lto on Windows is a bit ugly.  We could use makefile
>> magic to disable it when gold is not in use.
>
> I doubt it will work. That is at least if you really want
> LTO code generation.

Disabling libbackend.a generation for LTO bootstrap might be a
working idea.

Richard.

> -Andi
>
> --
> ak@linux.intel.com -- Speaking for myself only.
>

> > collect2 without linker help doesn't handle ar.
> >
> >> to bootstrap-lto on Windows is a bit ugly.  We could use makefile
> >> magic to disable it when gold is not in use.
> >
> > I doubt it will work. That is at least if you really want
> > LTO code generation.
> 
> Disabling libbackend.a generation for LTO bootstrap might be a
> working idea.

There are more: libcpp (although I think that doesn't get
the correct options anyways, PR45227) and libiberty

-Andi

On Wed, Sep 1, 2010 at 10:50 AM, Andi Kleen <andi@firstfloor.org> wrote:
>> > collect2 without linker help doesn't handle ar.
>> >
>> >> to bootstrap-lto on Windows is a bit ugly.  We could use makefile
>> >> magic to disable it when gold is not in use.
>> >
>> > I doubt it will work. That is at least if you really want
>> > LTO code generation.
>>
>> Disabling libbackend.a generation for LTO bootstrap might be a
>> working idea.
>
> There are more: libcpp (although I think that doesn't get
> the correct options anyways, PR45227) and libiberty

But they do not call back into gcc, so using -fwhole-program and linking
against them will just work.

Richard,

> -Andi
> --
> ak@linux.intel.com -- Speaking for myself only.
>