Patchwork [1/2] qemu-system: new host package

login
register
mail settings
Submitter Gustavo Zacarias
Date Dec. 4, 2013, 12:40 a.m.
Message ID <1386117649-7119-2-git-send-email-gustavo@zacarias.com.ar>
Download mbox | patch
Permalink /patch/296367/
State Superseded
Headers show

Comments

Gustavo Zacarias - Dec. 4, 2013, 12:40 a.m.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
 package/Config.in.host             |  1 +
 package/qemu-system/Config.in.host | 30 ++++++++++++++++++
 package/qemu-system/qemu-system.mk | 64 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 95 insertions(+)
 create mode 100644 package/qemu-system/Config.in.host
 create mode 100644 package/qemu-system/qemu-system.mk
Thomas Petazzoni - Dec. 4, 2013, 8:44 a.m.
Dear Gustavo Zacarias,

On Tue,  3 Dec 2013 21:40:48 -0300, Gustavo Zacarias wrote:
> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> ---
>  package/Config.in.host             |  1 +
>  package/qemu-system/Config.in.host | 30 ++++++++++++++++++
>  package/qemu-system/qemu-system.mk | 64 ++++++++++++++++++++++++++++++++++++++
>  3 files changed, 95 insertions(+)
>  create mode 100644 package/qemu-system/Config.in.host
>  create mode 100644 package/qemu-system/qemu-system.mk

The first question that comes to mind is why we have a separate
"qemu-system" package for this, instead of improving the existing
"qemu" package, which already builds Qemu for the host, but only builds
the user emulation mode?

Best regards,

Thomas Petazzoni
Gustavo Zacarias - Dec. 4, 2013, 10:06 a.m.
On 12/04/2013 05:44 AM, Thomas Petazzoni wrote:

> The first question that comes to mind is why we have a separate
> "qemu-system" package for this, instead of improving the existing
> "qemu" package, which already builds Qemu for the host, but only builds
> the user emulation mode?

Version freedom to begin with, for the system emulation you sometimes
need a special version (example: sparc won't work with 1.7.0) that might
not be the latest. You might also want this freedom for user emulation,
but then nobody guarantees you'll match system for a given architecture.
Then i also don't know what to do with the user variant, if it's going
to be used for anything or what (it's an orphan now) - maybe killing
user is an option and replace it by system?
Regards.
Thomas Petazzoni - Dec. 4, 2013, 12:33 p.m.
Dear Gustavo Zacarias,

On Wed, 04 Dec 2013 07:06:50 -0300, Gustavo Zacarias wrote:

> Version freedom to begin with, for the system emulation you sometimes
> need a special version (example: sparc won't work with 1.7.0) that might
> not be the latest. You might also want this freedom for user emulation,
> but then nobody guarantees you'll match system for a given architecture.
> Then i also don't know what to do with the user variant, if it's going
> to be used for anything or what (it's an orphan now) - maybe killing
> user is an option and replace it by system?

Yes, I agree, the user mode emulation should probably be removed. It
simply cannot work properly, because we have no way of guaranteeing
that the kernel headers used for the target toolchain are older than
the kernel running on the system.

And even if we want to keep the user mode emulation, I think there
should be one single package. Nothing prevents from having sub-options
to enable either the user mode or system mode emulation, or both, or to
have a version selection.

Best regards,

Thomas
Gustavo Zacarias - Dec. 4, 2013, 12:48 p.m.
On 12/04/2013 09:33 AM, Thomas Petazzoni wrote:

> Yes, I agree, the user mode emulation should probably be removed. It
> simply cannot work properly, because we have no way of guaranteeing
> that the kernel headers used for the target toolchain are older than
> the kernel running on the system.
> 
> And even if we want to keep the user mode emulation, I think there
> should be one single package. Nothing prevents from having sub-options
> to enable either the user mode or system mode emulation, or both, or to
> have a version selection.

A single package prevents different versions of -user and -system (at
least in a nice way).
Sometimes the latest version of qemu doesn't play nice with some kernel
versions because of newer OpenBIOS/other BIOSes versions breaking with
newer kernels - it's a common scenario for non-x86/x86_64.
And there may be some particular need for -user to be a newer or
different version than system, though i can't come up with any idea of why.
At the moment i don't think -user is useful for anything in particular
because of the limitations you wrote.
Problem is building two different versions in a single package wouldn't
be all that great, for all intents and purposes they would be two
packages anyway.
Regards.
Arnout Vandecappelle - Dec. 5, 2013, 5:49 p.m.
On 04/12/13 13:48, Gustavo Zacarias wrote:
> On 12/04/2013 09:33 AM, Thomas Petazzoni wrote:
>
>> Yes, I agree, the user mode emulation should probably be removed. It
>> simply cannot work properly, because we have no way of guaranteeing
>> that the kernel headers used for the target toolchain are older than
>> the kernel running on the system.

  It's not possible to do it in Config.in, but it's easy to do during the 
build: compare /usr/include/linux/version.h with 
$(STAGING_DIR)/usr/include/linux/version.h

  Or am I missing something? I admit, it assumes that the distro installs 
kernel headers corresponding to the running kernel (but a distro that 
doesn't do that is pretty broken), and it assumes that you'll run qemu on 
the build machine.


>> And even if we want to keep the user mode emulation, I think there
>> should be one single package. Nothing prevents from having sub-options
>> to enable either the user mode or system mode emulation, or both, or to
>> have a version selection.
>
> A single package prevents different versions of -user and -system (at
> least in a nice way).

  We do it for gcc, binutils, ... and it doesn't look that ugly to me. 
It's mostly the Config.in part that looks ugly.


> Sometimes the latest version of qemu doesn't play nice with some kernel
> versions because of newer OpenBIOS/other BIOSes versions breaking with
> newer kernels - it's a common scenario for non-x86/x86_64.
> And there may be some particular need for -user to be a newer or
> different version than system, though i can't come up with any idea of why.
> At the moment i don't think -user is useful for anything in particular
> because of the limitations you wrote.
> Problem is building two different versions in a single package wouldn't
> be all that great, for all intents and purposes they would be two
> packages anyway.

  The two packages would be 90% identical (except for needless 
divergences, like one using generic-package and the other using 
autotools-package). So I don't agree they would be two packages. The host 
and the target build of e.g. python differ more than these two.

  Regards,
  Arnout
Thomas Petazzoni - Dec. 5, 2013, 10:31 p.m.
Dear Arnout Vandecappelle,

On Thu, 05 Dec 2013 18:49:48 +0100, Arnout Vandecappelle wrote:

> >> Yes, I agree, the user mode emulation should probably be removed. It
> >> simply cannot work properly, because we have no way of guaranteeing
> >> that the kernel headers used for the target toolchain are older than
> >> the kernel running on the system.
> 
>   It's not possible to do it in Config.in, but it's easy to do during the 
> build: compare /usr/include/linux/version.h with 
> $(STAGING_DIR)/usr/include/linux/version.h

And then you fail the build if
$(STAGING_DIR)/usr/include/linux/version.h gives a newer version
of /usr/include/linux/version.h ? Could be possible, even if not really
nice. But François is proposing to remove cpanminus, which was the only
reason to have qemu-user support. So do we care?

Thomas
Frank Hunleth - Dec. 6, 2013, 1:14 a.m.
On Thu, Dec 5, 2013 at 5:31 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Arnout Vandecappelle,
>
> On Thu, 05 Dec 2013 18:49:48 +0100, Arnout Vandecappelle wrote:
>
>> >> Yes, I agree, the user mode emulation should probably be removed. It
>> >> simply cannot work properly, because we have no way of guaranteeing
>> >> that the kernel headers used for the target toolchain are older than
>> >> the kernel running on the system.
>>
>>   It's not possible to do it in Config.in, but it's easy to do during the
>> build: compare /usr/include/linux/version.h with
>> $(STAGING_DIR)/usr/include/linux/version.h
>
> And then you fail the build if
> $(STAGING_DIR)/usr/include/linux/version.h gives a newer version
> of /usr/include/linux/version.h ? Could be possible, even if not really
> nice. But François is proposing to remove cpanminus, which was the only
> reason to have qemu-user support. So do we care?

I care a little (for as much as that counts), but not enough to argue
for keeping it if it is becoming a pain. We use qemu-user in a
post-build step to run a battery of regression tests on algorithmic
code. It has been convenient since the regression test data is too
large to run in one pass on the target hardware. It's also faster to
run on our PCs. I don't think that the regression tests exercise any
of the issues above.

Frank
Arnout Vandecappelle - Dec. 6, 2013, 3:58 p.m.
On 06/12/13 02:14, Frank Hunleth wrote:
> On Thu, Dec 5, 2013 at 5:31 PM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Dear Arnout Vandecappelle,
>>
>> On Thu, 05 Dec 2013 18:49:48 +0100, Arnout Vandecappelle wrote:
>>
>>>>> Yes, I agree, the user mode emulation should probably be removed. It
>>>>> simply cannot work properly, because we have no way of guaranteeing
>>>>> that the kernel headers used for the target toolchain are older than
>>>>> the kernel running on the system.
>>>
>>>    It's not possible to do it in Config.in, but it's easy to do during the
>>> build: compare /usr/include/linux/version.h with
>>> $(STAGING_DIR)/usr/include/linux/version.h
>>
>> And then you fail the build if
>> $(STAGING_DIR)/usr/include/linux/version.h gives a newer version
>> of /usr/include/linux/version.h ? Could be possible, even if not really
>> nice. But François is proposing to remove cpanminus, which was the only
>> reason to have qemu-user support. So do we care?
>
> I care a little (for as much as that counts), but not enough to argue
> for keeping it if it is becoming a pain. We use qemu-user in a
> post-build step to run a battery of regression tests on algorithmic
> code. It has been convenient since the regression test data is too
> large to run in one pass on the target hardware. It's also faster to
> run on our PCs. I don't think that the regression tests exercise any
> of the issues above.

  That seems to me like a valuable use case. But you would need a 
Config.in.host for it so it actually becomes visible, right? Care to 
contribute that?

  Regards,
  Arnout
Frank Hunleth - Dec. 9, 2013, 12:15 p.m.
Hi Arnout,

On Fri, Dec 6, 2013 at 10:58 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 06/12/13 02:14, Frank Hunleth wrote:
>> I care a little (for as much as that counts), but not enough to argue
>> for keeping it if it is becoming a pain. We use qemu-user in a
>> post-build step to run a battery of regression tests on algorithmic
>> code. It has been convenient since the regression test data is too
>> large to run in one pass on the target hardware. It's also faster to
>> run on our PCs. I don't think that the regression tests exercise any
>> of the issues above.
>
>
>  That seems to me like a valuable use case. But you would need a
> Config.in.host for it so it actually becomes visible, right? Care to
> contribute that?

You're right. I had completely forgotten that I had made that change
for that project. I'll submit the patch that I'm using shortly.

Also, I'll need some help on any caveats that are desired. I see the
caveat in cpanminus, but submitting a patch that depends on BR2_BROKEN
feels wrong.

Frank
Francois Perrad - Dec. 9, 2013, 12:18 p.m.
2013/12/6 Arnout Vandecappelle <arnout@mind.be>:
> On 06/12/13 02:14, Frank Hunleth wrote:
>>
>> On Thu, Dec 5, 2013 at 5:31 PM, Thomas Petazzoni
>> <thomas.petazzoni@free-electrons.com> wrote:
>>>
>>> Dear Arnout Vandecappelle,
>>>
>>> On Thu, 05 Dec 2013 18:49:48 +0100, Arnout Vandecappelle wrote:
>>>
>>>>>> Yes, I agree, the user mode emulation should probably be removed. It
>>>>>> simply cannot work properly, because we have no way of guaranteeing
>>>>>> that the kernel headers used for the target toolchain are older than
>>>>>> the kernel running on the system.
>>>>
>>>>
>>>>    It's not possible to do it in Config.in, but it's easy to do during
>>>> the
>>>> build: compare /usr/include/linux/version.h with
>>>> $(STAGING_DIR)/usr/include/linux/version.h
>>>
>>>
>>> And then you fail the build if
>>> $(STAGING_DIR)/usr/include/linux/version.h gives a newer version
>>> of /usr/include/linux/version.h ? Could be possible, even if not really
>>> nice. But François is proposing to remove cpanminus, which was the only
>>> reason to have qemu-user support. So do we care?
>>
>>
>> I care a little (for as much as that counts), but not enough to argue
>> for keeping it if it is becoming a pain. We use qemu-user in a
>> post-build step to run a battery of regression tests on algorithmic
>> code. It has been convenient since the regression test data is too
>> large to run in one pass on the target hardware. It's also faster to
>> run on our PCs. I don't think that the regression tests exercise any
>> of the issues above.
>
>
>  That seems to me like a valuable use case. But you would need a
> Config.in.host for it so it actually becomes visible, right? Care to
> contribute that?
>

I did it with http://patchwork.ozlabs.org/patch/293643/

François

>
>  Regards,
>  Arnout
>
>
> --
> Arnout Vandecappelle                          arnout at mind be
> Senior Embedded Software Architect            +32-16-286500
> Essensium/Mind                                http://www.mind.be
> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
> GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

Patch

diff --git a/package/Config.in.host b/package/Config.in.host
index 5fd2570..0f69402 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -10,6 +10,7 @@  source "package/lpc3250loader/Config.in.host"
 source "package/mtools/Config.in.host"
 source "package/omap-u-boot-utils/Config.in.host"
 source "package/openocd/Config.in.host"
+source "package/qemu-system/Config.in.host"
 source "package/sam-ba/Config.in.host"
 source "package/sunxi-tools/Config.in.host"
 source "package/uboot-tools/Config.in.host"
diff --git a/package/qemu-system/Config.in.host b/package/qemu-system/Config.in.host
new file mode 100644
index 0000000..16d1a11
--- /dev/null
+++ b/package/qemu-system/Config.in.host
@@ -0,0 +1,30 @@ 
+config BR2_PACKAGE_HOST_QEMU_SYSTEM
+	bool "host qemu-system"
+	# Supported system target architectures
+	depends on BR2_arm || BR2_i386 || BR2_microblaze || BR2_mips \
+		|| BR2_mips64 || BR2_mips64el || BR2_mipsel || BR2_powerpc \
+		|| BR2_sh4 || BR2_sh4eb || BR2_sparc || BR2_x86_64
+	help
+	  QEMU is a generic and open source machine emulator and virtualizer.
+
+	  This option builds a system emulator for your selected architecture.
+
+	  http://www.qemu.org
+
+if BR2_PACKAGE_HOST_QEMU_SYSTEM
+
+config BR2_PACKAGE_HOST_QEMU_SYSTEM_VERSION
+	string "qemu version"
+	default "1.7.0"
+	help
+	  QEMU version to use.
+	  Sometimes the latest version is broken for some specific
+	  architecture or target machine.
+
+config BR2_PACKAGE_HOST_QEMU_SYSTEM_ARGS
+	string "qemu-system command arguments"
+	help
+	  Arguments to be used for your target host-qemu-system to run
+	  via "make qemu-system-run".
+
+endif
diff --git a/package/qemu-system/qemu-system.mk b/package/qemu-system/qemu-system.mk
new file mode 100644
index 0000000..21f4213
--- /dev/null
+++ b/package/qemu-system/qemu-system.mk
@@ -0,0 +1,64 @@ 
+################################################################################
+#
+# qemu-system
+#
+################################################################################
+
+HOST_QEMU_SYSTEM_VERSION = $(call qstrip,$(BR2_PACKAGE_HOST_QEMU_SYSTEM_VERSION))
+HOST_QEMU_SYSTEM_SOURCE = qemu-$(HOST_QEMU_SYSTEM_VERSION).tar.bz2
+QEMU_SYSTEM_SITE = http://wiki.qemu-project.org/download
+QEMU_SYSTEM_LICENSE = GPLv2 LGPLv2.1 MIT BSD-3c BSD-2c Others/BSD-1c
+QEMU_SYSTEM_LICENSE_FILES = COPYING COPYING.LIB
+# NOTE: there is no top-level license file for non-(L)GPL licenses;
+#       the non-(L)GPL license texts are specified in the affected
+#       individual source files.
+
+HOST_QEMU_SYSTEM_DEPENDENCIES = host-flex host-bison host-libglib2 \
+	host-pixman host-pkgconf host-zlib
+
+HOST_QEMU_SYSTEM_ARCH = $(ARCH)
+ifeq ($(HOST_QEMU_SYSTEM_ARCH),i486)
+	HOST_QEMU_SYSTEM_ARCH = i386
+endif
+ifeq ($(HOST_QEMU_SYSTEM_ARCH),i586)
+	HOST_QEMU_SYSTEM_ARCH = i386
+endif
+ifeq ($(HOST_QEMU_SYSTEM_ARCH),i686)
+	HOST_QEMU_SYSTEM_ARCH = i386
+endif
+ifeq ($(BR2_microblazeel),y)
+	HOST_QEMU_SYSTEM_ARCH = microblazeel
+endif
+ifeq ($(HOST_QEMU_SYSTEM_ARCH),powerpc)
+	HOST_QEMU_SYSTEM_ARCH = ppc
+endif
+HOST_QEMU_SYSTEM_TARGETS=$(HOST_QEMU_SYSTEM_ARCH)-softmmu
+
+define HOST_QEMU_SYSTEM_CONFIGURE_CMDS
+	(cd $(@D); $(HOST_CONFIGURE_OPTS) ./configure \
+		--target-list="$(HOST_QEMU_SYSTEM_TARGETS)" \
+		--prefix="$(HOST_DIR)/usr" \
+		--interp-prefix=$(STAGING_DIR) \
+		--cc="$(HOSTCC)" \
+		--host-cc="$(HOSTCC)" \
+		--extra-cflags="$(HOST_CFLAGS)" \
+		--extra-ldflags="$(HOST_LDFLAGS)" \
+		--with-system-pixman \
+	)
+endef
+
+define HOST_QEMU_SYSTEM_BUILD_CMDS
+	$(HOST_MAKE_ENV) $(MAKE) -C $(@D)
+endef
+
+define HOST_QEMU_SYSTEM_INSTALL_CMDS
+	$(HOST_MAKE_ENV) $(MAKE) -C $(@D) install
+endef
+
+$(eval $(host-generic-package))
+
+# System emulation binary used here or somewhere else
+QEMU_SYSTEM = $(HOST_DIR)/usr/bin/qemu-system-$(HOST_QEMU_SYSTEM_ARCH)
+
+qemu-system-run: all
+	$(QEMU_SYSTEM) $(call qstrip,$(BR2_PACKAGE_HOST_QEMU_SYSTEM_ARGS))