Patchwork [1.3] build: compile translate.o at -O1 optimization

login
register
mail settings
Submitter Paolo Bonzini
Date Nov. 27, 2012, 8:34 a.m.
Message ID <1354005293-17741-1-git-send-email-pbonzini@redhat.com>
Download mbox | patch
Permalink /patch/202120/
State New
Headers show

Comments

Paolo Bonzini - Nov. 27, 2012, 8:34 a.m.
Some versions of GCC require insane (>2GB) amounts of memory
to compile translate.o.  As a countermeasure, compile it
with -O1.  This should fix the buildbot failure for
default_x86_64_fedora16.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Makefile.target | 2 ++
 1 file changed, 2 insertions(+)
Peter Maydell - Nov. 27, 2012, 1:24 p.m.
On 27 November 2012 08:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Some versions of GCC require insane (>2GB) amounts of memory
> to compile translate.o.  As a countermeasure, compile it
> with -O1.  This should fix the buildbot failure for
> default_x86_64_fedora16.

This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
trunk a year ago). Use a newer gcc, or a 64 bit build system
with a reasonable amount of RAM, or as a workaround apply some
suitable compiler flags by passing configure
'--extra-cflags=-fno-var-tracking'. This patch definitely
shouldn't be applied as we shouldn't be hampering the majority
for the benefit of old broken systems.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48190
https://bugs.launchpad.net/gcc-linaro/+bug/714921

-- PMM
Gerd Hoffmann - Nov. 27, 2012, 1:45 p.m.
On 11/27/12 14:24, Peter Maydell wrote:
> On 27 November 2012 08:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Some versions of GCC require insane (>2GB) amounts of memory
>> to compile translate.o.  As a countermeasure, compile it
>> with -O1.  This should fix the buildbot failure for
>> default_x86_64_fedora16.
> 
> This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
> trunk a year ago). Use a newer gcc, or a 64 bit build system
> with a reasonable amount of RAM, or as a workaround apply some
> suitable compiler flags by passing configure
> '--extra-cflags=-fno-var-tracking'. This patch definitely
> shouldn't be applied as we shouldn't be hampering the majority
> for the benefit of old broken systems.

It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
while compiling translate.c

cheers,
  Gerd
Peter Maydell - Nov. 27, 2012, 1:46 p.m.
On 27 November 2012 13:45, Gerd Hoffmann <kraxel@redhat.com> wrote:
> On 11/27/12 14:24, Peter Maydell wrote:
>> This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
>> trunk a year ago). Use a newer gcc, or a 64 bit build system
>> with a reasonable amount of RAM, or as a workaround apply some
>> suitable compiler flags by passing configure
>> '--extra-cflags=-fno-var-tracking'. This patch definitely
>> shouldn't be applied as we shouldn't be hampering the majority
>> for the benefit of old broken systems.
>
> It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
> while compiling translate.c

In that case it is a new (or regressed) gcc bug and we should be
pursuing it with the upstream gcc folk. We still shouldn't be
putting random workarounds in our configure script.

-- PMM
陳韋任 - Nov. 27, 2012, 1:49 p.m.
On Tue, Nov 27, 2012 at 02:45:07PM +0100, Gerd Hoffmann wrote:
> On 11/27/12 14:24, Peter Maydell wrote:
> > On 27 November 2012 08:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >> Some versions of GCC require insane (>2GB) amounts of memory
> >> to compile translate.o.  As a countermeasure, compile it
> >> with -O1.  This should fix the buildbot failure for
> >> default_x86_64_fedora16.
> > 
> > This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
> > trunk a year ago). Use a newer gcc, or a 64 bit build system
> > with a reasonable amount of RAM, or as a workaround apply some
> > suitable compiler flags by passing configure
> > '--extra-cflags=-fno-var-tracking'. This patch definitely
> > shouldn't be applied as we shouldn't be hampering the majority
> > for the benefit of old broken systems.
> 
> It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
> while compiling translate.c

  Even apply Peter's suggestion? Do all gcc 4.7.2 on various platform
have the same problem, or it only happen on Fedora 17.

Regards,
chenwj
Paolo Bonzini - Nov. 27, 2012, 2:10 p.m.
Il 27/11/2012 14:46, Peter Maydell ha scritto:
> On 27 November 2012 13:45, Gerd Hoffmann <kraxel@redhat.com> wrote:
>> On 11/27/12 14:24, Peter Maydell wrote:
>>> This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
>>> trunk a year ago). Use a newer gcc, or a 64 bit build system
>>> with a reasonable amount of RAM, or as a workaround apply some
>>> suitable compiler flags by passing configure
>>> '--extra-cflags=-fno-var-tracking'. This patch definitely
>>> shouldn't be applied as we shouldn't be hampering the majority
>>> for the benefit of old broken systems.
>>
>> It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
>> while compiling translate.c
> 
> In that case it is a new (or regressed) gcc bug and we should be
> pursuing it with the upstream gcc folk. We still shouldn't be
> putting random workarounds in our configure script.

Sure we should, but we're also one week from release.  Pretty sure a lot
of people will compile QEMU on a machine with <2GB and fail; the choice
is between documenting the problem and workaround in the release
information, or doing it for everyone in the Makefile.

Unfortunately, this one is not fixed by settling for worse debug
information, and it happens with GCC 4.7.2 (latest stable).  It is fixed
in GCC trunk, but it is also possible that it is just latent because I
found that several GCC switches fix it:

* -fPIE is what causes it.  Removing it obviously fixes it, but I'm not
sure it is a good idea (does it break linux-user self-virtualization?)

* -fno-gcse turns off the pass that explodes, and fixes it

* -fno-tree-pre turns off a similar but unrelated pass, but also fixes
it (found by chance because I confused it with -fno-gcse :)).

I think -fno-gcse is a valid workaround, and better than this patch.

Paolo
Paolo Bonzini - Nov. 27, 2012, 3:05 p.m.
Il 27/11/2012 14:49, 陳韋任 (Wei-Ren Chen) ha scritto:
>> > It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
>> > while compiling translate.c
>   Even apply Peter's suggestion? Do all gcc 4.7.2 on various platform
> have the same problem, or it only happen on Fedora 17.

All.  It is reproducible with GCC's stable branch from the FSF
repository.  I'm testing a patch, but it will take more than 1 week to
get it to the distros.

Paolo
Avi Kivity - Nov. 27, 2012, 3:53 p.m.
On 11/27/2012 10:34 AM, Paolo Bonzini wrote:
> Some versions of GCC require insane (>2GB) amounts of memory
> to compile translate.o.  As a countermeasure, compile it
> with -O1.  This should fix the buildbot failure for
> default_x86_64_fedora16.
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  Makefile.target | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/Makefile.target b/Makefile.target
> index 8b658c0..3981931 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -143,6 +143,8 @@ GENERATED_HEADERS += hmp-commands.h qmp-commands-old.h
>  
>  endif # CONFIG_SOFTMMU
>  
> +%/translate.o: CFLAGS := $(patsubst -O2,-O1,$(CFLAGS))
> +

This may change some string argument in CFLAGS, for example an argument
to -I.

How about:

  CFLAGS_opt = -O2
  CFLAGS += $(CFLAGS_opt)
  ...
  %/translate.o: CFLAGS_opt = -O1
Paolo Bonzini - Nov. 27, 2012, 4:02 p.m.
Il 27/11/2012 16:53, Avi Kivity ha scritto:
> This may change some string argument in CFLAGS, for example an argument
> to -I.
> 
> How about:
> 
>   CFLAGS_opt = -O2
>   CFLAGS += $(CFLAGS_opt)
>   ...
>   %/translate.o: CFLAGS_opt = -O1

Not possible because CFLAGS comes from configure, but anyway what I'm
going to do is

%/translate.o: QEMU_CFLAGS += -fno-gcse

now that Gerd found the culprit patch.

Paolo
Andreas Färber - Nov. 27, 2012, 4:24 p.m.
Am 27.11.2012 14:49, schrieb 陳韋任 (Wei-Ren Chen):
> On Tue, Nov 27, 2012 at 02:45:07PM +0100, Gerd Hoffmann wrote:
>> On 11/27/12 14:24, Peter Maydell wrote:
>>> On 27 November 2012 08:34, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>> Some versions of GCC require insane (>2GB) amounts of memory
>>>> to compile translate.o.  As a countermeasure, compile it
>>>> with -O1.  This should fix the buildbot failure for
>>>> default_x86_64_fedora16.
>>>
>>> This is a well known bug in old gcc (ie fixed in 4.5, 4.6 and
>>> trunk a year ago). Use a newer gcc, or a 64 bit build system
>>> with a reasonable amount of RAM, or as a workaround apply some
>>> suitable compiler flags by passing configure
>>> '--extra-cflags=-fno-var-tracking'. This patch definitely
>>> shouldn't be applied as we shouldn't be hampering the majority
>>> for the benefit of old broken systems.
>>
>> It isn't that simple.  It's Fedora 17 with gcc 4.7.2 which runs oom
>> while compiling translate.c
> 
>   Even apply Peter's suggestion? Do all gcc 4.7.2 on various platform
> have the same problem, or it only happen on Fedora 17.

I ran into the same problem with various versions of openSUSE in the
openSUSE Build Service where v1.2 built fine.

openSUSE Factory is using gcc 4.7.2, 12.2 4.7.1.

Andreas
Markus Armbruster - Nov. 27, 2012, 4:49 p.m.
Avi Kivity <avi@redhat.com> writes:

> On 11/27/2012 10:34 AM, Paolo Bonzini wrote:
>> Some versions of GCC require insane (>2GB) amounts of memory
>> to compile translate.o.  As a countermeasure, compile it
>> with -O1.  This should fix the buildbot failure for
>> default_x86_64_fedora16.
>> 
>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>> ---
>>  Makefile.target | 2 ++
>>  1 file changed, 2 insertions(+)
>> 
>> diff --git a/Makefile.target b/Makefile.target
>> index 8b658c0..3981931 100644
>> --- a/Makefile.target
>> +++ b/Makefile.target
>> @@ -143,6 +143,8 @@ GENERATED_HEADERS += hmp-commands.h qmp-commands-old.h
>>  
>>  endif # CONFIG_SOFTMMU
>>  
>> +%/translate.o: CFLAGS := $(patsubst -O2,-O1,$(CFLAGS))
>> +
>
> This may change some string argument in CFLAGS, for example an argument
> to -I.
>
> How about:
>
>   CFLAGS_opt = -O2
>   CFLAGS += $(CFLAGS_opt)
>   ...
>   %/translate.o: CFLAGS_opt = -O1

Just append -O1 to CFLAGS; the last -O wins.
Paolo Bonzini - Nov. 27, 2012, 4:55 p.m.
Il 27/11/2012 17:49, Markus Armbruster ha scritto:
>>> +%/translate.o: CFLAGS := $(patsubst -O2,-O1,$(CFLAGS))
>>> +
>>
>> This may change some string argument in CFLAGS, for example an argument
>> to -I.
>>
>> How about:
>>
>>   CFLAGS_opt = -O2
>>   CFLAGS += $(CFLAGS_opt)
>>   ...
>>   %/translate.o: CFLAGS_opt = -O1
> 
> Just append -O1 to CFLAGS; the last -O wins.

But you don't want to override -O0...

Anyhow, v2 was posted and does it in a different, more fine-grained manner.

Paolo

Patch

diff --git a/Makefile.target b/Makefile.target
index 8b658c0..3981931 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -143,6 +143,8 @@  GENERATED_HEADERS += hmp-commands.h qmp-commands-old.h
 
 endif # CONFIG_SOFTMMU
 
+%/translate.o: CFLAGS := $(patsubst -O2,-O1,$(CFLAGS))
+
 nested-vars += obj-y
 
 # This resolves all nested paths, so it must come last