Patchwork [Qemu-trivial,2/2] Makefile: use full path for qapi-generated directory

login
register
mail settings
Submitter Stefano Stabellini
Date Nov. 30, 2011, 1:03 p.m.
Message ID <alpine.DEB.2.00.1111301227570.31179@kaball-desktop>
Download mbox | patch
Permalink /patch/128504/
State New
Headers show

Comments

Stefano Stabellini - Nov. 30, 2011, 1:03 p.m.
On Wed, 30 Nov 2011, Stefan Hajnoczi wrote:
> On Tue, Nov 29, 2011 at 10:47 PM, Michael Roth
> <mdroth@linux.vnet.ibm.com> wrote:
> > Generally $(BUILD_DIR) == $(CURDIR), but that isn't necessarilly the
> > case, so use $(BUILD_DIR)/qapi-generated for generated files to
> > avoid potentionally sticking generating files in odd places outside
> > the build's include paths.
> > ---
> > � Makefile | �  � 2 +-
> > � 1 files changed, 1 insertions(+), 1 deletions(-)
> 
> Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> 
> Not applied to trivial tree yet.  I want to let Stefano, Stefan Weil,
> or Anthony take a look too since we were all discussing these changes
> yesterday.
 
While I think that the change is correct (certainly better than the
original patch), unfortunately still doesn't solve the problem
entirely.
Steps to repro, assuming "qemu" is the source tree and "temp" is the
empty build directory:

- cd qemu; ./configure; make

- cd ../temp; ./configure --source-path=../qemu; make

Result:

Makefile:9: config-devices.mak: No such file or directory

In order to fix the issue, we need to make sure that config-devices.mak
is relative to $(BUILD_DIR) in the Makefile.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Stefan Weil - Nov. 30, 2011, 7:59 p.m.
Am 30.11.2011 14:03, schrieb Stefano Stabellini:
> On Wed, 30 Nov 2011, Stefan Hajnoczi wrote:
>> On Tue, Nov 29, 2011 at 10:47 PM, Michael Roth
>> <mdroth@linux.vnet.ibm.com> wrote:
>>> Generally $(BUILD_DIR) == $(CURDIR), but that isn't necessarilly the
>>> case, so use $(BUILD_DIR)/qapi-generated for generated files to
>>> avoid potentionally sticking generating files in odd places outside
>>> the build's include paths.
>>> ---
>>> � Makefile | � � 2 +-
>>> � 1 files changed, 1 insertions(+), 1 deletions(-)
>>
>> Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>>
>> Not applied to trivial tree yet. I want to let Stefano, Stefan Weil,
>> or Anthony take a look too since we were all discussing these changes
>> yesterday.
>
> While I think that the change is correct (certainly better than the
> original patch), unfortunately still doesn't solve the problem
> entirely.
> Steps to repro, assuming "qemu" is the source tree and "temp" is the
> empty build directory:
>
> - cd qemu; ./configure; make
>
> - cd ../temp; ./configure --source-path=../qemu; make
>
> Result:
>
> Makefile:9: config-devices.mak: No such file or directory
>
> In order to fix the issue, we need to make sure that config-devices.mak
> is relative to $(BUILD_DIR) in the Makefile.
>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>
> diff --git a/Makefile b/Makefile
[snip]

It's common to use either out-of-tree builds or in-tree builds,
but not to mix both variants with a common root directory.
I think QEMU should explicitly forbid that mixed scenario (like
other projects do).

Even with your fix there can remain problems with generated
header files. The mixed scenario creates unnecessary complexity.
Without the mixed scenario, your patch is not needed.

Regards,
Stefan Weil
Stefano Stabellini - Dec. 1, 2011, 3:19 p.m.
On Wed, 30 Nov 2011, Stefan Weil wrote:
> It's common to use either out-of-tree builds or in-tree builds,
> but not to mix both variants with a common root directory.
> I think QEMU should explicitly forbid that mixed scenario (like
> other projects do).
> 
> Even with your fix there can remain problems with generated
> header files.

Really? Can you provide more specific details?

> The mixed scenario creates unnecessary complexity.
> Without the mixed scenario, your patch is not needed.

I agree that supporting the mixed scenario shouldn't be a priority.
However without this last patch a "make clean" on the main tree is not
enough to allow out of tree builds.

Try the following scenario:

- cd qemu; ./configure; make

- make clean

- cd ../temp; ./configure --source-path=../qemu; make

this has to work, right? It does not without the patch to fix the
generation of config-devices.mak.
Michael Roth - Dec. 1, 2011, 4:05 p.m.
On 12/01/2011 09:19 AM, Stefano Stabellini wrote:
> On Wed, 30 Nov 2011, Stefan Weil wrote:
>> It's common to use either out-of-tree builds or in-tree builds,
>> but not to mix both variants with a common root directory.
>> I think QEMU should explicitly forbid that mixed scenario (like
>> other projects do).
>>
>> Even with your fix there can remain problems with generated
>> header files.
>
> Really? Can you provide more specific details?
>
>> The mixed scenario creates unnecessary complexity.
>> Without the mixed scenario, your patch is not needed.
>
> I agree that supporting the mixed scenario shouldn't be a priority.
> However without this last patch a "make clean" on the main tree is not
> enough to allow out of tree builds.
>
> Try the following scenario:
>
> - cd qemu; ./configure; make
>
> - make clean
>
> - cd ../temp; ./configure --source-path=../qemu; make
>
> this has to work, right? It does not without the patch to fix the
> generation of config-devices.mak.
>

To clarify, the first patch I sent fix a bug that's not currently 
triggered due to qapi-dir always ending up being "qapi-generated/", so 
the guardname generated works, albeit with some needlessly verbose names.

The second isn't intended to fix your issue with builds, but avoid 
potential problems that may arise if at some point $(BUILD_DIR) != 
$(CURDIR).

So they don't actually fix anything, just potential bugs that may arise 
in the future.

The issue with doing out of tree builds with a dirty source directory is 
really just an unsupported scenario. To really clean things up, you need 
to run `make distclean` using the same configuration you used to dirty 
the directory in the first place, otherwise even `make distclean` will 
leave cruft laying in target directories not present in your current config.

Alexandre Raymond sent a patch to make it so that `make distclean` 
always left the source directory pristine, but there was some contention 
on what exactly a `make distclean` entails and it wasn't applied:

http://lists.gnu.org/archive/html/qemu-trivial/2011-07/msg00037.html

That's really the only complete fix for these build issues. I'm starting 
to think it'd be worth it to add a `make clean_for_reals` target that 
does what Alexandre's patch does and tell people to run it after a pull 
or switching to out-of-tree builds.
Stefan Weil - Dec. 1, 2011, 6:11 p.m.
Am 01.12.2011 16:19, schrieb Stefano Stabellini:
> On Wed, 30 Nov 2011, Stefan Weil wrote:
>> It's common to use either out-of-tree builds or in-tree builds,
>> but not to mix both variants with a common root directory.
>> I think QEMU should explicitly forbid that mixed scenario (like
>> other projects do).
>>
>> Even with your fix there can remain problems with generated
>> header files.
>
> Really? Can you provide more specific details?

Yes. Suppose different generated header files with the same name
exist in the source directory tree and in the build directory tree.

Then compiler options (the order of -I options) will determine
which of two header files with same name will be used in out-of-tree
builds. If the wrong one (that from the source directory tree) is used,
the compiler might fail or produce wrong code (for example when
macro values changed).
>> The mixed scenario creates unnecessary complexity.
>> Without the mixed scenario, your patch is not needed.
>
> I agree that supporting the mixed scenario shouldn't be a priority.
> However without this last patch a "make clean" on the main tree is not
> enough to allow out of tree builds.
>
> Try the following scenario:
>
> - cd qemu; ./configure; make
>
> - make clean
>
> - cd ../temp; ./configure --source-path=../qemu; make

Which ./configure do you run in an empty temp directory?
This example (like similar ones in your previous mail) won't work.

>
> this has to work, right? It does not without the patch to fix the
> generation of config-devices.mak.

No. You'll need a 'make distclean'. If that does what it should do,
a following out-of-tree build will work.

Regards,
Stefan Weil
Anthony Liguori - Dec. 1, 2011, 6:28 p.m.
On 12/01/2011 12:11 PM, Stefan Weil wrote:
> Am 01.12.2011 16:19, schrieb Stefano Stabellini:
>> On Wed, 30 Nov 2011, Stefan Weil wrote:
>>> It's common to use either out-of-tree builds or in-tree builds,
>>> but not to mix both variants with a common root directory.
>>> I think QEMU should explicitly forbid that mixed scenario (like
>>> other projects do).
>>>
>>> Even with your fix there can remain problems with generated
>>> header files.
>>
>> Really? Can you provide more specific details?
>
> Yes. Suppose different generated header files with the same name
> exist in the source directory tree and in the build directory tree.
>
> Then compiler options (the order of -I options) will determine
> which of two header files with same name will be used in out-of-tree
> builds. If the wrong one (that from the source directory tree) is used,
> the compiler might fail or produce wrong code (for example when
> macro values changed).
>>> The mixed scenario creates unnecessary complexity.
>>> Without the mixed scenario, your patch is not needed.
>>
>> I agree that supporting the mixed scenario shouldn't be a priority.
>> However without this last patch a "make clean" on the main tree is not
>> enough to allow out of tree builds.
>>
>> Try the following scenario:
>>
>> - cd qemu; ./configure; make
>>
>> - make clean
>>
>> - cd ../temp; ./configure --source-path=../qemu; make
>
> Which ./configure do you run in an empty temp directory?
> This example (like similar ones in your previous mail) won't work.

Yes, you must do external builds only from a clean source directory.  This is a 
perfectly normal requirement.  In fact, I can't think of a project that doesn't 
require this.

Regards,

Anthony Liguori

>
>>
>> this has to work, right? It does not without the patch to fix the
>> generation of config-devices.mak.
>
> No. You'll need a 'make distclean'. If that does what it should do,
> a following out-of-tree build will work.
>
> Regards,
> Stefan Weil
>
>
Stefano Stabellini - Dec. 1, 2011, 6:38 p.m.
On Thu, 1 Dec 2011, Stefan Weil wrote:
> Am 01.12.2011 16:19, schrieb Stefano Stabellini:
> > On Wed, 30 Nov 2011, Stefan Weil wrote:
> >> It's common to use either out-of-tree builds or in-tree builds,
> >> but not to mix both variants with a common root directory.
> >> I think QEMU should explicitly forbid that mixed scenario (like
> >> other projects do).
> >>
> >> Even with your fix there can remain problems with generated
> >> header files.
> >
> > Really? Can you provide more specific details?
> 
> Yes. Suppose different generated header files with the same name
> exist in the source directory tree and in the build directory tree.
> 
> Then compiler options (the order of -I options) will determine
> which of two header files with same name will be used in out-of-tree
> builds. If the wrong one (that from the source directory tree) is used,
> the compiler might fail or produce wrong code (for example when
> macro values changed).
> >> The mixed scenario creates unnecessary complexity.
> >> Without the mixed scenario, your patch is not needed.
> >
> > I agree that supporting the mixed scenario shouldn't be a priority.
> > However without this last patch a "make clean" on the main tree is not
> > enough to allow out of tree builds.
> >
> > Try the following scenario:
> >
> > - cd qemu; ./configure; make
> >
> > - make clean
> >
> > - cd ../temp; ./configure --source-path=../qemu; make
> 
> Which ./configure do you run in an empty temp directory?
> This example (like similar ones in your previous mail) won't work.

Sorry, it was supposed to be ../qemu/configure
Stefano Stabellini - Dec. 1, 2011, 6:42 p.m.
On Thu, 1 Dec 2011, Anthony Liguori wrote:
> On 12/01/2011 12:11 PM, Stefan Weil wrote:
> > Am 01.12.2011 16:19, schrieb Stefano Stabellini:
> >> On Wed, 30 Nov 2011, Stefan Weil wrote:
> >>> It's common to use either out-of-tree builds or in-tree builds,
> >>> but not to mix both variants with a common root directory.
> >>> I think QEMU should explicitly forbid that mixed scenario (like
> >>> other projects do).
> >>>
> >>> Even with your fix there can remain problems with generated
> >>> header files.
> >>
> >> Really? Can you provide more specific details?
> >
> > Yes. Suppose different generated header files with the same name
> > exist in the source directory tree and in the build directory tree.
> >
> > Then compiler options (the order of -I options) will determine
> > which of two header files with same name will be used in out-of-tree
> > builds. If the wrong one (that from the source directory tree) is used,
> > the compiler might fail or produce wrong code (for example when
> > macro values changed).
> >>> The mixed scenario creates unnecessary complexity.
> >>> Without the mixed scenario, your patch is not needed.
> >>
> >> I agree that supporting the mixed scenario shouldn't be a priority.
> >> However without this last patch a "make clean" on the main tree is not
> >> enough to allow out of tree builds.
> >>
> >> Try the following scenario:
> >>
> >> - cd qemu; ./configure; make
> >>
> >> - make clean
> >>
> >> - cd ../temp; ./configure --source-path=../qemu; make
> >
> > Which ./configure do you run in an empty temp directory?
> > This example (like similar ones in your previous mail) won't work.
> 
> Yes, you must do external builds only from a clean source directory.  This is a 
> perfectly normal requirement.  In fact, I can't think of a project that doesn't 
> require this.

Fair enough.

In that case, like Michael Roth wrote, the patch "Makefile: use full
path for qapi-generated directory" doesn't actually fix anything, so
probably we don't need it either, but I'll leave it up to you.
Michael Roth - Dec. 1, 2011, 7:12 p.m.
On 12/01/2011 12:42 PM, Stefano Stabellini wrote:
> On Thu, 1 Dec 2011, Anthony Liguori wrote:
>> On 12/01/2011 12:11 PM, Stefan Weil wrote:
>>> Am 01.12.2011 16:19, schrieb Stefano Stabellini:
>>>> On Wed, 30 Nov 2011, Stefan Weil wrote:
>>>>> It's common to use either out-of-tree builds or in-tree builds,
>>>>> but not to mix both variants with a common root directory.
>>>>> I think QEMU should explicitly forbid that mixed scenario (like
>>>>> other projects do).
>>>>>
>>>>> Even with your fix there can remain problems with generated
>>>>> header files.
>>>>
>>>> Really? Can you provide more specific details?
>>>
>>> Yes. Suppose different generated header files with the same name
>>> exist in the source directory tree and in the build directory tree.
>>>
>>> Then compiler options (the order of -I options) will determine
>>> which of two header files with same name will be used in out-of-tree
>>> builds. If the wrong one (that from the source directory tree) is used,
>>> the compiler might fail or produce wrong code (for example when
>>> macro values changed).
>>>>> The mixed scenario creates unnecessary complexity.
>>>>> Without the mixed scenario, your patch is not needed.
>>>>
>>>> I agree that supporting the mixed scenario shouldn't be a priority.
>>>> However without this last patch a "make clean" on the main tree is not
>>>> enough to allow out of tree builds.
>>>>
>>>> Try the following scenario:
>>>>
>>>> - cd qemu; ./configure; make
>>>>
>>>> - make clean
>>>>
>>>> - cd ../temp; ./configure --source-path=../qemu; make
>>>
>>> Which ./configure do you run in an empty temp directory?
>>> This example (like similar ones in your previous mail) won't work.
>>
>> Yes, you must do external builds only from a clean source directory.  This is a
>> perfectly normal requirement.  In fact, I can't think of a project that doesn't
>> require this.
>
> Fair enough.
>
> In that case, like Michael Roth wrote, the patch "Makefile: use full
> path for qapi-generated directory" doesn't actually fix anything, so
> probably we don't need it either, but I'll leave it up to you.
>

Yup, it's just to address some potential issues/suggestions Stefan Weil 
pointed out. I think patch 1 should be applied though.

Patch

diff --git a/Makefile b/Makefile
index 301c75e..b9f8de7 100644
--- a/Makefile
+++ b/Makefile
@@ -43,15 +43,15 @@  DOCS=
 endif
 
 SUBDIR_MAKEFLAGS=$(if $(V),,--no-print-directory) BUILD_DIR=$(BUILD_DIR)
-SUBDIR_DEVICES_MAK=$(patsubst %, %/config-devices.mak, $(TARGET_DIRS))
-SUBDIR_DEVICES_MAK_DEP=$(patsubst %, %/config-devices.mak.d, $(TARGET_DIRS))
+SUBDIR_DEVICES_MAK=$(patsubst %, %/config-devices.mak, $(BUILD_DIR)/$(TARGET_DIRS))
+SUBDIR_DEVICES_MAK_DEP=$(patsubst %, %/config-devices.mak.d, $(BUILD_DIR)/$(TARGET_DIRS))
 
 config-all-devices.mak: $(SUBDIR_DEVICES_MAK)
 	$(call quiet-command,cat $(SUBDIR_DEVICES_MAK) | grep =y | sort -u > $@,"  GEN   $@")
 
 -include $(SUBDIR_DEVICES_MAK_DEP)
 
-%/config-devices.mak: default-configs/%.mak
+$(BUILD_DIR)/%/config-devices.mak: default-configs/%.mak
 	$(call quiet-command,$(SHELL) $(SRC_PATH)/scripts/make_device_config.sh $@ $<, "  GEN   $@")
 	@if test -f $@; then \
 	  if cmp -s $@.old $@; then \