diff mbox

package/kvm-unit-tests: new package

Message ID 20160817230236.8255-1-cyrilbur@gmail.com
State Accepted
Headers show

Commit Message

Cyril Bur Aug. 17, 2016, 11:02 p.m. UTC 
The unit tests are tiny guest operating systems that generally execute
only tens of lines of C and assembler test code in order to obtain its
PASS/FAIL result. Unit tests provide KVM and virt hardware functional
testing by targeting the features through minimal implementations of
their use per the hardware specification. The simplicity of unit tests
make them easy to verify they are correct, easy to maintain, and easy
to use in timing measurements. Unit tests are also often used for
quick and dirty bug reproducers. The reproducers may then be kept as
regression tests. It's strongly encouraged that patches implementing
new KVM features are submitted with accompanying unit tests.
http://www.linux-kvm.org/page/KVM-unit-tests

Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
---
 package/Config.in                        |  1 +
 package/kvm-unit-tests/Config.in         | 17 ++++++++++++
 package/kvm-unit-tests/kvm-unit-tests.mk | 45 ++++++++++++++++++++++++++++++++
 3 files changed, 63 insertions(+)
 create mode 100644 package/kvm-unit-tests/Config.in
 create mode 100644 package/kvm-unit-tests/kvm-unit-tests.mk

Comments

Thomas Petazzoni Sept. 19, 2016, 2:48 p.m. UTC | #1 
Hello,

On Thu, 18 Aug 2016 09:02:36 +1000, Cyril Bur wrote:
> The unit tests are tiny guest operating systems that generally execute
> only tens of lines of C and assembler test code in order to obtain its
> PASS/FAIL result. Unit tests provide KVM and virt hardware functional
> testing by targeting the features through minimal implementations of
> their use per the hardware specification. The simplicity of unit tests
> make them easy to verify they are correct, easy to maintain, and easy
> to use in timing measurements. Unit tests are also often used for
> quick and dirty bug reproducers. The reproducers may then be kept as
> regression tests. It's strongly encouraged that patches implementing
> new KVM features are submitted with accompanying unit tests.
> http://www.linux-kvm.org/page/KVM-unit-tests
> 
> Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
> ---
>  package/Config.in                        |  1 +
>  package/kvm-unit-tests/Config.in         | 17 ++++++++++++
>  package/kvm-unit-tests/kvm-unit-tests.mk | 45 ++++++++++++++++++++++++++++++++
>  3 files changed, 63 insertions(+)
>  create mode 100644 package/kvm-unit-tests/Config.in
>  create mode 100644 package/kvm-unit-tests/kvm-unit-tests.mk

Applied to master with the following changes:

    [Thomas:
     - order architecture dependencies in Config.in alphabetically.
     - rewrap Config.in help text, lines were too long
     - add an empty line between the package description and the upstream
       project URL in the Config.in help text
     - don't make KVM_UNIT_TESTS_ARCH default to $(ARCH). This was not
       correct for i386 for example. Instead, just handle the few
       architectures that the package supports.
     - remove useless double quotes in variable definitions.
     - remove --prefix="$(TARGET_DIR)" from CONF_OPTS. It was installing
       everything in /share/ and not /usr/share/, and setting the prefix to
       TARGET_DIR at configure time is not good. Instead, pass DESTDIR at
       installation time.]

Thomas
Peter Korsgaard Sept. 19, 2016, 9:07 p.m. UTC | #2 
>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 > Hello,
 > On Thu, 18 Aug 2016 09:02:36 +1000, Cyril Bur wrote:
 >> The unit tests are tiny guest operating systems that generally execute
 >> only tens of lines of C and assembler test code in order to obtain its
 >> PASS/FAIL result. Unit tests provide KVM and virt hardware functional
 >> testing by targeting the features through minimal implementations of
 >> their use per the hardware specification. The simplicity of unit tests
 >> make them easy to verify they are correct, easy to maintain, and easy
 >> to use in timing measurements. Unit tests are also often used for
 >> quick and dirty bug reproducers. The reproducers may then be kept as
 >> regression tests. It's strongly encouraged that patches implementing
 >> new KVM features are submitted with accompanying unit tests.
 >> http://www.linux-kvm.org/page/KVM-unit-tests
 >> 
 >> Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
 >> ---
 >> package/Config.in                        |  1 +
 >> package/kvm-unit-tests/Config.in         | 17 ++++++++++++
 >> package/kvm-unit-tests/kvm-unit-tests.mk | 45 ++++++++++++++++++++++++++++++++
 >> 3 files changed, 63 insertions(+)
 >> create mode 100644 package/kvm-unit-tests/Config.in
 >> create mode 100644 package/kvm-unit-tests/kvm-unit-tests.mk

 > Applied to master with the following changes:

 >     [Thomas:
 >      - order architecture dependencies in Config.in alphabetically.
 >      - rewrap Config.in help text, lines were too long
 >      - add an empty line between the package description and the upstream
 >        project URL in the Config.in help text
 >      - don't make KVM_UNIT_TESTS_ARCH default to $(ARCH). This was not
 >        correct for i386 for example. Instead, just handle the few
 >        architectures that the package supports.
 >      - remove useless double quotes in variable definitions.
 >      - remove --prefix="$(TARGET_DIR)" from CONF_OPTS. It was installing
 >        everything in /share/ and not /usr/share/, and setting the prefix to
 >        TARGET_DIR at configure time is not good. Instead, pass DESTDIR at
 >        installation time.]

It unfortunately fails on x86-64:
http://autobuild.buildroot.org/results/445/4453e8001406a658ae4dc58448baaceb6d93cbf1/

As it contains 32bit code and our toolchains (normally) aren't
multilib. I guess we need to build those parts using the host compiler
and depend on BR2_HOSTARCH_NEEDS_IA32_COMPILER like we do for
grub/syslinux.
Cyril Bur Sept. 19, 2016, 11:12 p.m. UTC | #3 
On Mon, 2016-09-19 at 16:48 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 18 Aug 2016 09:02:36 +1000, Cyril Bur wrote:
> > 
> > The unit tests are tiny guest operating systems that generally
> > execute
> > only tens of lines of C and assembler test code in order to obtain
> > its
> > PASS/FAIL result. Unit tests provide KVM and virt hardware
> > functional
> > testing by targeting the features through minimal implementations
> > of
> > their use per the hardware specification. The simplicity of unit
> > tests
> > make them easy to verify they are correct, easy to maintain, and
> > easy
> > to use in timing measurements. Unit tests are also often used for
> > quick and dirty bug reproducers. The reproducers may then be kept
> > as
> > regression tests. It's strongly encouraged that patches
> > implementing
> > new KVM features are submitted with accompanying unit tests.
> > http://www.linux-kvm.org/page/KVM-unit-tests
> > 
> > Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
> > ---
> >  package/Config.in                        |  1 +
> >  package/kvm-unit-tests/Config.in         | 17 ++++++++++++
> >  package/kvm-unit-tests/kvm-unit-tests.mk | 45
> > ++++++++++++++++++++++++++++++++
> >  3 files changed, 63 insertions(+)
> >  create mode 100644 package/kvm-unit-tests/Config.in
> >  create mode 100644 package/kvm-unit-tests/kvm-unit-tests.mk
> 
> Applied to master with the following changes:
> 
>     [Thomas:
>      - order architecture dependencies in Config.in alphabetically.
>      - rewrap Config.in help text, lines were too long
>      - add an empty line between the package description and the
> upstream
>        project URL in the Config.in help text
>      - don't make KVM_UNIT_TESTS_ARCH default to $(ARCH). This was
> not
>        correct for i386 for example. Instead, just handle the few
>        architectures that the package supports.
>      - remove useless double quotes in variable definitions.
>      - remove --prefix="$(TARGET_DIR)" from CONF_OPTS. It was
> installing
>        everything in /share/ and not /usr/share/, and setting the
> prefix to
>        TARGET_DIR at configure time is not good. Instead, pass
> DESTDIR at
>        installation time.]
> 

Thanks for the fixups.

Cyril

> Thomas
Thomas Petazzoni Sept. 21, 2016, 7:24 a.m. UTC | #4 
Hello,

On Thu, 18 Aug 2016 09:02:36 +1000, Cyril Bur wrote:
> The unit tests are tiny guest operating systems that generally execute
> only tens of lines of C and assembler test code in order to obtain its
> PASS/FAIL result. Unit tests provide KVM and virt hardware functional
> testing by targeting the features through minimal implementations of
> their use per the hardware specification. The simplicity of unit tests
> make them easy to verify they are correct, easy to maintain, and easy
> to use in timing measurements. Unit tests are also often used for
> quick and dirty bug reproducers. The reproducers may then be kept as
> regression tests. It's strongly encouraged that patches implementing
> new KVM features are submitted with accompanying unit tests.
> http://www.linux-kvm.org/page/KVM-unit-tests
> 
> Signed-off-by: Cyril Bur <cyrilbur@gmail.com>

This package is causing a good number of build failures. See:

         arm | kvm-unit-tests-682187afcc99... | NOK | http://autobuild.buildroot.net/results/a81d40f1453ee027c7e2a9da01f482e1d09b464d
        i686 | kvm-unit-tests-682187afcc99... | NOK | http://autobuild.buildroot.net/results/20efe1e310fe71a587774bbe8490d6357acc8edd
      x86_64 | kvm-unit-tests-682187afcc99... | NOK | http://autobuild.buildroot.net/results/07deb6183ed23447de5182ee20df6c9c49e5ca49

Could you have a look and provide patches to address these?

Thanks,

Thomas
Cyril Bur Sept. 22, 2016, 12:01 a.m. UTC | #5 
On Wed, 2016-09-21 at 09:24 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 18 Aug 2016 09:02:36 +1000, Cyril Bur wrote:
> > 
> > The unit tests are tiny guest operating systems that generally
> > execute
> > only tens of lines of C and assembler test code in order to obtain
> > its
> > PASS/FAIL result. Unit tests provide KVM and virt hardware
> > functional
> > testing by targeting the features through minimal implementations
> > of
> > their use per the hardware specification. The simplicity of unit
> > tests
> > make them easy to verify they are correct, easy to maintain, and
> > easy
> > to use in timing measurements. Unit tests are also often used for
> > quick and dirty bug reproducers. The reproducers may then be kept
> > as
> > regression tests. It's strongly encouraged that patches
> > implementing
> > new KVM features are submitted with accompanying unit tests.
> > http://www.linux-kvm.org/page/KVM-unit-tests
> > 
> > Signed-off-by: Cyril Bur <cyrilbur@gmail.com>
> 
> This package is causing a good number of build failures. See:
> 
>          arm | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
> d.buildroot.net/results/a81d40f1453ee027c7e2a9da01f482e1d09b464d
>         i686 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
> d.buildroot.net/results/20efe1e310fe71a587774bbe8490d6357acc8edd
>       x86_64 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
> d.buildroot.net/results/07deb6183ed23447de5182ee20df6c9c49e5ca49
> 
> Could you have a look and provide patches to address these?
> 

Sure i'll see what appears to be happening.

> Thanks,
> 
> Thomas
Peter Korsgaard Sept. 22, 2016, 5:18 a.m. UTC | #6 
>>>>> "Cyril" == Cyril Bur <cyrilbur@gmail.com> writes:

Hi,

 >> This package is causing a good number of build failures. See:
 >> 
 >>          arm | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
 >> d.buildroot.net/results/a81d40f1453ee027c7e2a9da01f482e1d09b464d
 >>         i686 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
 >> d.buildroot.net/results/20efe1e310fe71a587774bbe8490d6357acc8edd
 >>       x86_64 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
 >> d.buildroot.net/results/07deb6183ed23447de5182ee20df6c9c49e5ca49
 >> 
 >> Could you have a look and provide patches to address these?
 >> 

 > Sure i'll see what appears to be happening.

Some of the issues are that it expects the x86-64 toolchain to support
-m32 for bootstrap code (which our cross compilers normally don't), so
we need the same workaround as we do for syslinux and grub to build
those with the host compiler.
Thomas Petazzoni Sept. 22, 2016, 9:34 a.m. UTC | #7 
Hello,

On Thu, 22 Sep 2016 07:18:18 +0200, Peter Korsgaard wrote:

>  >> This package is causing a good number of build failures. See:
>  >> 
>  >>          arm | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
>  >> d.buildroot.net/results/a81d40f1453ee027c7e2a9da01f482e1d09b464d
>  >>         i686 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
>  >> d.buildroot.net/results/20efe1e310fe71a587774bbe8490d6357acc8edd
>  >>       x86_64 | kvm-unit-tests-682187afcc99... | NOK | http://autobuil
>  >> d.buildroot.net/results/07deb6183ed23447de5182ee20df6c9c49e5ca49
>  >> 
>  >> Could you have a look and provide patches to address these?
>  >>   
> 
>  > Sure i'll see what appears to be happening.  
> 
> Some of the issues are that it expects the x86-64 toolchain to support
> -m32 for bootstrap code (which our cross compilers normally don't), so
> we need the same workaround as we do for syslinux and grub to build
> those with the host compiler.

I guess this explains the build failure on x86-64, but not the ones on
arm and i686.

Thomas
Peter Korsgaard Sept. 22, 2016, 9:42 a.m. UTC | #8 
>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@free-electrons.com> writes:

 >> Some of the issues are that it expects the x86-64 toolchain to support
 >> -m32 for bootstrap code (which our cross compilers normally don't), so
 >> we need the same workaround as we do for syslinux and grub to build
 >> those with the host compiler.

 > I guess this explains the build failure on x86-64, but not the ones on
 > arm and i686.

Indeed - That's why I wrote 'some' ;)
Cyril Bur Sept. 23, 2016, 6:06 a.m. UTC | #9 
On Thu, 2016-09-22 at 07:18 +0200, Peter Korsgaard wrote:
> > 
> > > 
> > > > 
> > > > > 
> > > > > > 
> > > > > > "Cyril" == Cyril Bur <cyrilbur@gmail.com> writes:
> 
> Hi,
> 
>  >> This package is causing a good number of build failures. See:
>  >> 
>  >>          arm | kvm-unit-tests-682187afcc99... | NOK | http://auto
> buil
>  >> d.buildroot.net/results/a81d40f1453ee027c7e2a9da01f482e1d09b464d
>  >>         i686 | kvm-unit-tests-682187afcc99... | NOK | http://auto
> buil
>  >> d.buildroot.net/results/20efe1e310fe71a587774bbe8490d6357acc8edd
>  >>       x86_64 | kvm-unit-tests-682187afcc99... | NOK | http://auto
> buil
>  >> d.buildroot.net/results/07deb6183ed23447de5182ee20df6c9c49e5ca49
>  >> 
>  >> Could you have a look and provide patches to address these?
>  >> 
> 
>  > Sure i'll see what appears to be happening.
> 

So the i686 problem is actually fixed upstream so I've sent a patch to
bump the version.

I tried select BR2_HOSTARCH_NEEDS_IA32_COMPILER for x86_64 but didn't
seem to solve the problem. I know almost nothing about x86 asm so I'm
at a bit of a loss, it looks like gcc is generating instructions from C
that should be 16bit but then later it realises it messed up?

Still chasing but having BR2_HOSTARCH_NEEDS_IA32_COMPILER=y didn't
solve.

Cyril

> Some of the issues are that it expects the x86-64 toolchain to
> support
> -m32 for bootstrap code (which our cross compilers normally don't),
> so
> we need the same workaround as we do for syslinux and grub to build
> those with the host compiler.
>
Thomas Petazzoni Sept. 23, 2016, 6:12 a.m. UTC | #10 
Hello,

On Fri, 23 Sep 2016 16:06:56 +1000, Cyril Bur wrote:

> So the i686 problem is actually fixed upstream so I've sent a patch to
> bump the version.
> 
> I tried select BR2_HOSTARCH_NEEDS_IA32_COMPILER for x86_64 but didn't
> seem to solve the problem. I know almost nothing about x86 asm so I'm
> at a bit of a loss, it looks like gcc is generating instructions from C
> that should be 16bit but then later it realises it messed up?
> 
> Still chasing but having BR2_HOSTARCH_NEEDS_IA32_COMPILER=y didn't
> solve.

No, you need this *and* use the host compiler when building
kvm-unit-tests on x86_64. By using the host compiler, you use a
compiler that can generate 32 bits code.

See syslinux and grub, like Peter suggested.

Thanks!

Thomas
Cyril Bur Sept. 23, 2016, 6:14 a.m. UTC | #11 
On Fri, 2016-09-23 at 08:12 +0200, Thomas Petazzoni wrote:
> Hello,
> 
> On Fri, 23 Sep 2016 16:06:56 +1000, Cyril Bur wrote:
> 
> > 
> > So the i686 problem is actually fixed upstream so I've sent a patch
> > to
> > bump the version.
> > 
> > I tried select BR2_HOSTARCH_NEEDS_IA32_COMPILER for x86_64 but
> > didn't
> > seem to solve the problem. I know almost nothing about x86 asm so
> > I'm
> > at a bit of a loss, it looks like gcc is generating instructions
> > from C
> > that should be 16bit but then later it realises it messed up?
> > 
> > Still chasing but having BR2_HOSTARCH_NEEDS_IA32_COMPILER=y didn't
> > solve.
> 
> No, you need this *and* use the host compiler when building
> kvm-unit-tests on x86_64. By using the host compiler, you use a
> compiler that can generate 32 bits code.
> 
> See syslinux and grub, like Peter suggested.
> 

Yeah I had a look, sorry, I must have missed that part. Will do.

Thanks,

Cyril

> Thanks!
> 
> Thomas
diff mbox

Patch

diff --git a/package/Config.in b/package/Config.in
index e1ddcea..60f7d6e 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -82,6 +82,7 @@  menu "Debugging, profiling and benchmark"
 	source "package/kexec/Config.in"
 	source "package/kexec-lite/Config.in"
 	source "package/ktap/Config.in"
+	source "package/kvm-unit-tests/Config.in"
 	source "package/kyua/Config.in"
 	source "package/latencytop/Config.in"
 	source "package/lmbench/Config.in"
diff --git a/package/kvm-unit-tests/Config.in b/package/kvm-unit-tests/Config.in
new file mode 100644
index 0000000..cf0da58
--- /dev/null
+++ b/package/kvm-unit-tests/Config.in
@@ -0,0 +1,17 @@ 
+config BR2_PACKAGE_KVM_UNIT_TESTS
+	bool "kvm-unit-tests"
+	depends on BR2_powerpc64 || BR2_powerpc64le || BR2_x86_64 || BR2_i386 || BR2_arm
+	help
+	  kvm-unit-tests is a project as old as KVM. As its name suggests,
+	  it's purpose is to provide unit tests for KVM. The unit tests are tiny
+	  guest operating systems that generally execute only tens of lines of C
+	  and assembler test code in order to obtain its PASS/FAIL result. Unit
+	  tests provide KVM and virt hardware functional testing by targeting
+	  the features through minimal implementations of their use per the
+	  hardware specification. The simplicity of unit tests make them easy to
+	  verify they are correct, easy to maintain, and easy to use in timing
+	  measurements. Unit tests are also often used for quick and dirty bug
+	  reproducers. The reproducers may then be kept as regression tests.
+	  It's strongly encouraged that patches implementing new KVM features
+	  are submitted with accompanying unit tests.
+	  http://www.linux-kvm.org/page/KVM-unit-tests
diff --git a/package/kvm-unit-tests/kvm-unit-tests.mk b/package/kvm-unit-tests/kvm-unit-tests.mk
new file mode 100644
index 0000000..457e656
--- /dev/null
+++ b/package/kvm-unit-tests/kvm-unit-tests.mk
@@ -0,0 +1,45 @@ 
+################################################################################
+#
+# kvm-unit-tests
+#
+################################################################################
+
+KVM_UNIT_TESTS_VERSION = 682187afcc998bc3dd25061e98b43792cc340b33
+KVM_UNIT_TESTS_SITE = $(BR2_KERNEL_MIRROR)/scm/virt/kvm/kvm-unit-tests.git
+KVM_UNIT_TESTS_SITE_METHOD = git
+KVM_UNIT_TESTS_LICENSE = LGPLv2
+KVM_UNIT_TESTS_LICENSE_FILES = COPYRIGHT
+KVM_UNIT_TESTS_ARCH = "$(ARCH)"
+
+ifeq ($(ARCH),powerpc64)
+KVM_UNIT_TESTS_ARCH = "ppc64"
+endif
+ifeq ($(ARCH),powerpc64le)
+KVM_UNIT_TESTS_ARCH = "ppc64"
+endif
+
+ifeq ($(BR2_ENDIAN),"LITTLE")
+KVM_UNIT_TESTS_ENDIAN = "little"
+else
+KVM_UNIT_TESTS_ENDIAN = "big"
+endif
+
+KVM_UNIT_TESTS_CONF_OPTS = --arch="$(KVM_UNIT_TESTS_ARCH)" \
+						   --cross-prefix="$(TARGET_CROSS)" \
+						   --endian="$(KVM_UNIT_TESTS_ENDIAN)" \
+						   --prefix="$(TARGET_DIR)"
+
+define KVM_UNIT_TESTS_CONFIGURE_CMDS
+	cd $(@D) && ./configure $(KVM_UNIT_TESTS_CONF_OPTS)
+endef
+
+define KVM_UNIT_TESTS_BUILD_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) standalone
+endef
+
+define KVM_UNIT_TESTS_INSTALL_TARGET_CMDS
+	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install
+endef
+
+# Does use configure script but not an autotools one
+$(eval $(generic-package))