diff mbox series

tests/boot_linux_console: add a test for riscv64 + virt

Message ID 1562781026-27570-1-git-send-email-chihmin.chao@sifive.com
State New
Headers show
Series tests/boot_linux_console: add a test for riscv64 + virt | expand

Commit Message

Chih-Min Chao July 10, 2019, 5:50 p.m. UTC
Similar to the mips + malta test, it boots a Linux kernel on a virt
board and verify the serial is working.  Also, it relies on the serial
device set by the machine itself.

If riscv64 is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:riscv64" tags.

Alternatively, this test can be run using:

  $ avocado run -t arch:riscv64 tests/acceptance

Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
---
 tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

Comments

Alistair Francis July 10, 2019, 6:50 p.m. UTC | #1
On Wed, Jul 10, 2019 at 10:51 AM Chih-Min Chao <chihmin.chao@sifive.com> wrote:
>
> Similar to the mips + malta test, it boots a Linux kernel on a virt
> board and verify the serial is working.  Also, it relies on the serial
> device set by the machine itself.
>
> If riscv64 is a target being built, "make check-acceptance" will
> automatically include this test by the use of the "arch:riscv64" tags.
>
> Alternatively, this test can be run using:
>
>   $ avocado run -t arch:riscv64 tests/acceptance
>
> Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>

Awesome! Thanks for the test case. This will help a lot with RISC-V
regressions in QEMU.

> ---
>  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 3215950..bbc6b06 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>          self.vm.launch()
>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>          self.wait_for_console_pattern(console_pattern)
> +
> +    def test_riscv64_virt(self):
> +        """
> +        :avocado: tags=arch:riscv64
> +        :avocado: tags=machine:virt
> +        """
> +
> +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'

These images need to be built with a standard build flow. Having them
built from SiFive's custom scripts will make debugging problems in the
future impossible. I'm also a little worried here about GPL
violations, I'm not sure if it's enough to just point to a script SHA
to meet GPL source disclosure. I know companies have huge headaches
meeting GPL requirements so this seems too easy.

> +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> +                      'bbl_w_kernel.gz')

Don't use BBL, most people use OpenSBI now which is what we should be
testing with.

> +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +        kernel_path = self.workdir + "bbl_w_kernel"
> +
> +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> +            with open(kernel_path, 'wb') as f_out:
> +                shutil.copyfileobj(f_in, f_out)
> +
> +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> +                      'riscv64/rootfs.cpio.gz')

Same comment about build tools.


Alistair

> +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> +
> +        self.vm.set_machine('virt')
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +                               + 'console=ttyS0 noreboot')
> +        self.vm.add_args('-kernel', kernel_path,
> +                         '-initrd', initrd_path,
> +                         '-append', kernel_command_line)
> +        self.vm.launch()
> +        self.wait_for_console_pattern('Boot successful.')
> +
> +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> +                                               'isa')
> +        self.exec_command_and_wait_for_pattern('uname -a',
> +                                               'sifive')
> +        self.exec_command_and_wait_for_pattern('reboot',
> +                                               'reboot: Restarting system')
> --
> 2.7.4
>
>
Cleber Rosa July 10, 2019, 8:23 p.m. UTC | #2
On Wed, Jul 10, 2019 at 10:50:23AM -0700, Chih-Min Chao wrote:
> Similar to the mips + malta test, it boots a Linux kernel on a virt
> board and verify the serial is working.  Also, it relies on the serial
> device set by the machine itself.
> 
> If riscv64 is a target being built, "make check-acceptance" will
> automatically include this test by the use of the "arch:riscv64" tags.
> 
> Alternatively, this test can be run using:
> 
>   $ avocado run -t arch:riscv64 tests/acceptance
> 
> Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
> ---
>  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 3215950..bbc6b06 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>          self.vm.launch()
>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>          self.wait_for_console_pattern(console_pattern)
> +
> +    def test_riscv64_virt(self):
> +        """
> +        :avocado: tags=arch:riscv64
> +        :avocado: tags=machine:virt
> +        """
> +
> +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> +                      'bbl_w_kernel.gz')
> +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +        kernel_path = self.workdir + "bbl_w_kernel"
> +
> +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> +            with open(kernel_path, 'wb') as f_out:
> +                shutil.copyfileobj(f_in, f_out)

There are currently two patterns for extracting a gzipped file
in this test.  So, this is not a must, but maybe you'd prefer:

---
diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index bbc6b0683f..9f819e20e1 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -365,12 +365,8 @@ class BootLinuxConsole(Test):
                       '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
                       'bbl_w_kernel.gz')
         kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
-        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
-        kernel_path = self.workdir + "bbl_w_kernel"
-
-        with gzip.open(kernel_path_gz, 'rb') as f_in:
-            with open(kernel_path, 'wb') as f_out:
-                shutil.copyfileobj(f_in, f_out)
+        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+        uncompressed_kernel = archive.uncompress(kernel_path, self.workdir)
 
         initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
                       '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
@@ -382,7 +378,7 @@ class BootLinuxConsole(Test):
         self.vm.set_console()
         kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
                                + 'console=ttyS0 noreboot')
-        self.vm.add_args('-kernel', kernel_path,
+        self.vm.add_args('-kernel', uncompressed_kernel,
                          '-initrd', initrd_path,
                          '-append', kernel_command_line)
         self.vm.launch()
---

> +
> +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> +                      'riscv64/rootfs.cpio.gz')
> +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> +
> +        self.vm.set_machine('virt')
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +                               + 'console=ttyS0 noreboot')
> +        self.vm.add_args('-kernel', kernel_path,
> +                         '-initrd', initrd_path,
> +                         '-append', kernel_command_line)
> +        self.vm.launch()
> +        self.wait_for_console_pattern('Boot successful.')
> +
> +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> +                                               'isa')
> +        self.exec_command_and_wait_for_pattern('uname -a',
> +                                               'sifive')
> +        self.exec_command_and_wait_for_pattern('reboot',
> +                                               'reboot: Restarting system')
> -- 
> 2.7.4
> 
> 

It'd be nice to also add riscv64 to the target list in .travis.yaml
"acceptance tests" job.

Regards and many thanks for this contribution!
- Cleber.
Chih-Min Chao July 11, 2019, 1:54 a.m. UTC | #3
On Thu, Jul 11, 2019 at 2:53 AM Alistair Francis <alistair23@gmail.com>
wrote:

> On Wed, Jul 10, 2019 at 10:51 AM Chih-Min Chao <chihmin.chao@sifive.com>
> wrote:
> >
> > Similar to the mips + malta test, it boots a Linux kernel on a virt
> > board and verify the serial is working.  Also, it relies on the serial
> > device set by the machine itself.
> >
> > If riscv64 is a target being built, "make check-acceptance" will
> > automatically include this test by the use of the "arch:riscv64" tags.
> >
> > Alternatively, this test can be run using:
> >
> >   $ avocado run -t arch:riscv64 tests/acceptance
> >
> > Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
>
> Awesome! Thanks for the test case. This will help a lot with RISC-V
> regressions in QEMU.
>
> > ---
> >  tests/acceptance/boot_linux_console.py | 40
> ++++++++++++++++++++++++++++++++++
> >  1 file changed, 40 insertions(+)
> >
> > diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> > index 3215950..bbc6b06 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
> >          self.vm.launch()
> >          console_pattern = 'Kernel command line: %s' %
> kernel_command_line
> >          self.wait_for_console_pattern(console_pattern)
> > +
> > +    def test_riscv64_virt(self):
> > +        """
> > +        :avocado: tags=arch:riscv64
> > +        :avocado: tags=machine:virt
> > +        """
> > +
> > +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
>
> These images need to be built with a standard build flow. Having them
> built from SiFive's custom scripts will make debugging problems in the
> future impossible. I'm also a little worried here about GPL
> violations, I'm not sure if it's enough to just point to a script SHA
> to meet GPL source disclosure. I know companies have huge headaches
> meeting GPL requirements so this seems too easy.
>

   I am not very familiar with this kind of binary and source license
conflict.
   Is it ok if I write a simple script with BSD license to build  kernel
and image from sifive's linux/buildroot repo and commit the script to my
test-binary repo ?

>
> > +
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> > +                      'bbl_w_kernel.gz')
>
> Don't use BBL, most people use OpenSBI now which is what we should be
> testing with.
>
   I will try to move to OpenSBI in next version. My environment is based
freedom-u-sdk and It still relays  on BBL. So ..:P

>
> > +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> > +        kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> > +        kernel_path = self.workdir + "bbl_w_kernel"
> > +
> > +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> > +            with open(kernel_path, 'wb') as f_out:
> > +                shutil.copyfileobj(f_in, f_out)
> > +
> > +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> > +                      'riscv64/rootfs.cpio.gz')
>
> Same comment about build tools.
>
>
> Alistair
>
>     Got it

> > +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> > +        initrd_path = self.fetch_asset(initrd_url,
> asset_hash=initrd_hash)
> > +
> > +        self.vm.set_machine('virt')
> > +        self.vm.set_console()
> > +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +                               + 'console=ttyS0 noreboot')
> > +        self.vm.add_args('-kernel', kernel_path,
> > +                         '-initrd', initrd_path,
> > +                         '-append', kernel_command_line)
> > +        self.vm.launch()
> > +        self.wait_for_console_pattern('Boot successful.')
> > +
> > +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> > +                                               'isa')
> > +        self.exec_command_and_wait_for_pattern('uname -a',
> > +                                               'sifive')
> > +        self.exec_command_and_wait_for_pattern('reboot',
> > +                                               'reboot: Restarting
> system')
> > --
> > 2.7.4
> >
> >
>
Chih-Min Chao July 11, 2019, 2:01 a.m. UTC | #4
On Thu, Jul 11, 2019 at 4:23 AM Cleber Rosa <crosa@redhat.com> wrote:

> On Wed, Jul 10, 2019 at 10:50:23AM -0700, Chih-Min Chao wrote:
> > Similar to the mips + malta test, it boots a Linux kernel on a virt
> > board and verify the serial is working.  Also, it relies on the serial
> > device set by the machine itself.
> >
> > If riscv64 is a target being built, "make check-acceptance" will
> > automatically include this test by the use of the "arch:riscv64" tags.
> >
> > Alternatively, this test can be run using:
> >
> >   $ avocado run -t arch:riscv64 tests/acceptance
> >
> > Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
> > ---
> >  tests/acceptance/boot_linux_console.py | 40
> ++++++++++++++++++++++++++++++++++
> >  1 file changed, 40 insertions(+)
> >
> > diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> > index 3215950..bbc6b06 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
> >          self.vm.launch()
> >          console_pattern = 'Kernel command line: %s' %
> kernel_command_line
> >          self.wait_for_console_pattern(console_pattern)
> > +
> > +    def test_riscv64_virt(self):
> > +        """
> > +        :avocado: tags=arch:riscv64
> > +        :avocado: tags=machine:virt
> > +        """
> > +
> > +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> > +
> '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> > +                      'bbl_w_kernel.gz')
> > +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> > +        kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> > +        kernel_path = self.workdir + "bbl_w_kernel"
> > +
> > +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> > +            with open(kernel_path, 'wb') as f_out:
> > +                shutil.copyfileobj(f_in, f_out)
>
> There are currently two patterns for extracting a gzipped file
> in this test.  So, this is not a must, but maybe you'd prefer:
>

  The suggestion is good and avocado is an awesome testing framework.

---
> diff --git a/tests/acceptance/boot_linux_console.py
> b/tests/acceptance/boot_linux_console.py
> index bbc6b0683f..9f819e20e1 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -365,12 +365,8 @@ class BootLinuxConsole(Test):
>
>  '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
>                        'bbl_w_kernel.gz')
>          kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> -        kernel_path_gz = self.fetch_asset(kernel_url,
> asset_hash=kernel_hash)
> -        kernel_path = self.workdir + "bbl_w_kernel"
> -
> -        with gzip.open(kernel_path_gz, 'rb') as f_in:
> -            with open(kernel_path, 'wb') as f_out:
> -                shutil.copyfileobj(f_in, f_out)
> +        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +        uncompressed_kernel = archive.uncompress(kernel_path,
> self.workdir)
>
>          initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>                        '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> @@ -382,7 +378,7 @@ class BootLinuxConsole(Test):
>          self.vm.set_console()
>          kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
>                                 + 'console=ttyS0 noreboot')
> -        self.vm.add_args('-kernel', kernel_path,
> +        self.vm.add_args('-kernel', uncompressed_kernel,
>                           '-initrd', initrd_path,
>                           '-append', kernel_command_line)
>          self.vm.launch()
> ---
>
> > +
> > +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> > +                      'riscv64/rootfs.cpio.gz')
> > +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> > +        initrd_path = self.fetch_asset(initrd_url,
> asset_hash=initrd_hash)
> > +
> > +        self.vm.set_machine('virt')
> > +        self.vm.set_console()
> > +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +                               + 'console=ttyS0 noreboot')
> > +        self.vm.add_args('-kernel', kernel_path,
> > +                         '-initrd', initrd_path,
> > +                         '-append', kernel_command_line)
> > +        self.vm.launch()
> > +        self.wait_for_console_pattern('Boot successful.')
> > +
> > +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> > +                                               'isa')
> > +        self.exec_command_and_wait_for_pattern('uname -a',
> > +                                               'sifive')
> > +        self.exec_command_and_wait_for_pattern('reboot',
> > +                                               'reboot: Restarting
> system')
> > --
> > 2.7.4
> >
> >
>
> It'd be nice to also add riscv64 to the target list in .travis.yaml
> "acceptance tests" job.
>
> Regards and many thanks for this contribution!
> - Cleber.
>
Philippe Mathieu-Daudé July 11, 2019, 1:56 p.m. UTC | #5
Hi Chih-Min,

On 7/10/19 7:50 PM, Chih-Min Chao wrote:
> Similar to the mips + malta test, it boots a Linux kernel on a virt
> board and verify the serial is working.  Also, it relies on the serial
> device set by the machine itself.

Good idea!

> 
> If riscv64 is a target being built, "make check-acceptance" will
> automatically include this test by the use of the "arch:riscv64" tags.
> 
> Alternatively, this test can be run using:
> 
>   $ avocado run -t arch:riscv64 tests/acceptance
> 
> Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
> ---
>  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 3215950..bbc6b06 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>          self.vm.launch()
>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>          self.wait_for_console_pattern(console_pattern)
> +
> +    def test_riscv64_virt(self):
> +        """
> +        :avocado: tags=arch:riscv64
> +        :avocado: tags=machine:virt
> +        """
> +
> +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> +                      'bbl_w_kernel.gz')

I recommend you to extract fw_jump.elf from [1] and
vmlinux-4.19.0-5-riscv64 from [2] with the extract_from_deb() helper.

[1] https://packages.debian.org/sid/all/opensbi
[2] https://packages.debian.org/sid/linux-image-4.19.0-5-riscv64

> +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> +        kernel_path = self.workdir + "bbl_w_kernel"
> +
> +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> +            with open(kernel_path, 'wb') as f_out:
> +                shutil.copyfileobj(f_in, f_out)
> +
> +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> +                      'riscv64/rootfs.cpio.gz')
> +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> +
> +        self.vm.set_machine('virt')
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> +                               + 'console=ttyS0 noreboot')
> +        self.vm.add_args('-kernel', kernel_path,
> +                         '-initrd', initrd_path,
> +                         '-append', kernel_command_line)

You might want to use '-no-reboot' here too.

> +        self.vm.launch()
> +        self.wait_for_console_pattern('Boot successful.')
> +
> +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> +                                               'isa')
> +        self.exec_command_and_wait_for_pattern('uname -a',
> +                                               'sifive')
> +        self.exec_command_and_wait_for_pattern('reboot',
> +                                               'reboot: Restarting system')

Regards,

Phil.
Alistair Francis July 11, 2019, 2:41 p.m. UTC | #6
On Wed, Jul 10, 2019 at 6:54 PM Chih-Min Chao <chihmin.chao@sifive.com> wrote:
>
>
> On Thu, Jul 11, 2019 at 2:53 AM Alistair Francis <alistair23@gmail.com> wrote:
>>
>> On Wed, Jul 10, 2019 at 10:51 AM Chih-Min Chao <chihmin.chao@sifive.com> wrote:
>> >
>> > Similar to the mips + malta test, it boots a Linux kernel on a virt
>> > board and verify the serial is working.  Also, it relies on the serial
>> > device set by the machine itself.
>> >
>> > If riscv64 is a target being built, "make check-acceptance" will
>> > automatically include this test by the use of the "arch:riscv64" tags.
>> >
>> > Alternatively, this test can be run using:
>> >
>> >   $ avocado run -t arch:riscv64 tests/acceptance
>> >
>> > Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
>>
>> Awesome! Thanks for the test case. This will help a lot with RISC-V
>> regressions in QEMU.
>>
>> > ---
>> >  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
>> >  1 file changed, 40 insertions(+)
>> >
>> > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
>> > index 3215950..bbc6b06 100644
>> > --- a/tests/acceptance/boot_linux_console.py
>> > +++ b/tests/acceptance/boot_linux_console.py
>> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>> >          self.vm.launch()
>> >          console_pattern = 'Kernel command line: %s' % kernel_command_line
>> >          self.wait_for_console_pattern(console_pattern)
>> > +
>> > +    def test_riscv64_virt(self):
>> > +        """
>> > +        :avocado: tags=arch:riscv64
>> > +        :avocado: tags=machine:virt
>> > +        """
>> > +
>> > +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
>>
>> These images need to be built with a standard build flow. Having them
>> built from SiFive's custom scripts will make debugging problems in the
>> future impossible. I'm also a little worried here about GPL
>> violations, I'm not sure if it's enough to just point to a script SHA
>> to meet GPL source disclosure. I know companies have huge headaches
>> meeting GPL requirements so this seems too easy.
>
>
>    I am not very familiar with this kind of binary and source license conflict.

Thinking about it more you probably are ok with a SHA to a build tool.
I guess companies do all the extra work and vetting just to be extra
sure. So how much you do I guess is up to you ad SiFive.

>    Is it ok if I write a simple script with BSD license to build  kernel and image from sifive's linux/buildroot repo and commit the script to my test-binary repo ?

No, it must be built from an upstream repo. Others need to be able to
reproduce this in the future. You can just use upstream buildroot and
include a config. Ideally as well use a projects release instead of
some random SHA.

>>
>>
>> > +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
>> > +                      'bbl_w_kernel.gz')
>>
>> Don't use BBL, most people use OpenSBI now which is what we should be
>> testing with.
>
>    I will try to move to OpenSBI in next version. My environment is based freedom-u-sdk and It still relays  on BBL. So ..:P

Swap to an upstream distro, I think they all support OpenSBI now.
Buildroot and OE/Yocto both give you development environments and
Fedora and Debian work as well.

Alistair

>>
>>
>> > +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
>> > +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>> > +        kernel_path = self.workdir + "bbl_w_kernel"
>> > +
>> > +        with gzip.open(kernel_path_gz, 'rb') as f_in:
>> > +            with open(kernel_path, 'wb') as f_out:
>> > +                shutil.copyfileobj(f_in, f_out)
>> > +
>> > +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>> > +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
>> > +                      'riscv64/rootfs.cpio.gz')
>>
>> Same comment about build tools.
>>
>>
>> Alistair
>>
>     Got it
>>
>> > +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
>> > +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
>> > +
>> > +        self.vm.set_machine('virt')
>> > +        self.vm.set_console()
>> > +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
>> > +                               + 'console=ttyS0 noreboot')
>> > +        self.vm.add_args('-kernel', kernel_path,
>> > +                         '-initrd', initrd_path,
>> > +                         '-append', kernel_command_line)
>> > +        self.vm.launch()
>> > +        self.wait_for_console_pattern('Boot successful.')
>> > +
>> > +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
>> > +                                               'isa')
>> > +        self.exec_command_and_wait_for_pattern('uname -a',
>> > +                                               'sifive')
>> > +        self.exec_command_and_wait_for_pattern('reboot',
>> > +                                               'reboot: Restarting system')
>> > --
>> > 2.7.4
>> >
>> >
Alistair Francis July 11, 2019, 2:43 p.m. UTC | #7
On Thu, Jul 11, 2019 at 6:56 AM Philippe Mathieu-Daudé
<philmd@redhat.com> wrote:
>
> Hi Chih-Min,
>
> On 7/10/19 7:50 PM, Chih-Min Chao wrote:
> > Similar to the mips + malta test, it boots a Linux kernel on a virt
> > board and verify the serial is working.  Also, it relies on the serial
> > device set by the machine itself.
>
> Good idea!
>
> >
> > If riscv64 is a target being built, "make check-acceptance" will
> > automatically include this test by the use of the "arch:riscv64" tags.
> >
> > Alternatively, this test can be run using:
> >
> >   $ avocado run -t arch:riscv64 tests/acceptance
> >
> > Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
> > ---
> >  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 40 insertions(+)
> >
> > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> > index 3215950..bbc6b06 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
> >          self.vm.launch()
> >          console_pattern = 'Kernel command line: %s' % kernel_command_line
> >          self.wait_for_console_pattern(console_pattern)
> > +
> > +    def test_riscv64_virt(self):
> > +        """
> > +        :avocado: tags=arch:riscv64
> > +        :avocado: tags=machine:virt
> > +        """
> > +
> > +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
> > +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
> > +                      'bbl_w_kernel.gz')
>
> I recommend you to extract fw_jump.elf from [1] and
> vmlinux-4.19.0-5-riscv64 from [2] with the extract_from_deb() helper.
>
> [1] https://packages.debian.org/sid/all/opensbi
> [2] https://packages.debian.org/sid/linux-image-4.19.0-5-riscv64

This will works as well if you don't want to bother building the
images yourself. Ideally we could use a newer kernel then 4.19 but for
an acceptance test it probably doesn't matter much.

This way Debain has done all the licence work for you as well :)

Alistair

>
> > +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
> > +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> > +        kernel_path = self.workdir + "bbl_w_kernel"
> > +
> > +        with gzip.open(kernel_path_gz, 'rb') as f_in:
> > +            with open(kernel_path, 'wb') as f_out:
> > +                shutil.copyfileobj(f_in, f_out)
> > +
> > +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
> > +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
> > +                      'riscv64/rootfs.cpio.gz')
> > +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
> > +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
> > +
> > +        self.vm.set_machine('virt')
> > +        self.vm.set_console()
> > +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
> > +                               + 'console=ttyS0 noreboot')
> > +        self.vm.add_args('-kernel', kernel_path,
> > +                         '-initrd', initrd_path,
> > +                         '-append', kernel_command_line)
>
> You might want to use '-no-reboot' here too.
>
> > +        self.vm.launch()
> > +        self.wait_for_console_pattern('Boot successful.')
> > +
> > +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
> > +                                               'isa')
> > +        self.exec_command_and_wait_for_pattern('uname -a',
> > +                                               'sifive')
> > +        self.exec_command_and_wait_for_pattern('reboot',
> > +                                               'reboot: Restarting system')
>
> Regards,
>
> Phil.
>
Philippe Mathieu-Daudé July 11, 2019, 2:54 p.m. UTC | #8
On 7/11/19 4:43 PM, Alistair Francis wrote:
> On Thu, Jul 11, 2019 at 6:56 AM Philippe Mathieu-Daudé
> <philmd@redhat.com> wrote:
>>
>> Hi Chih-Min,
>>
>> On 7/10/19 7:50 PM, Chih-Min Chao wrote:
>>> Similar to the mips + malta test, it boots a Linux kernel on a virt
>>> board and verify the serial is working.  Also, it relies on the serial
>>> device set by the machine itself.
>>
>> Good idea!
>>
>>>
>>> If riscv64 is a target being built, "make check-acceptance" will
>>> automatically include this test by the use of the "arch:riscv64" tags.
>>>
>>> Alternatively, this test can be run using:
>>>
>>>   $ avocado run -t arch:riscv64 tests/acceptance
>>>
>>> Signed-off-by: Chih-Min Chao <chihmin.chao@sifive.com>
>>> ---
>>>  tests/acceptance/boot_linux_console.py | 40 ++++++++++++++++++++++++++++++++++
>>>  1 file changed, 40 insertions(+)
>>>
>>> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
>>> index 3215950..bbc6b06 100644
>>> --- a/tests/acceptance/boot_linux_console.py
>>> +++ b/tests/acceptance/boot_linux_console.py
>>> @@ -354,3 +354,43 @@ class BootLinuxConsole(Test):
>>>          self.vm.launch()
>>>          console_pattern = 'Kernel command line: %s' % kernel_command_line
>>>          self.wait_for_console_pattern(console_pattern)
>>> +
>>> +    def test_riscv64_virt(self):
>>> +        """
>>> +        :avocado: tags=arch:riscv64
>>> +        :avocado: tags=machine:virt
>>> +        """
>>> +
>>> +        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
>>> +                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
>>> +                      'bbl_w_kernel.gz')
>>
>> I recommend you to extract fw_jump.elf from [1] and
>> vmlinux-4.19.0-5-riscv64 from [2] with the extract_from_deb() helper.
>>
>> [1] https://packages.debian.org/sid/all/opensbi
>> [2] https://packages.debian.org/sid/linux-image-4.19.0-5-riscv64
> 
> This will works as well if you don't want to bother building the
> images yourself. Ideally we could use a newer kernel then 4.19 but for
> an acceptance test it probably doesn't matter much.

This one is available in 'experimental', in case you want to test it
too, but I'd rather use some stable channel for testing:

https://packages.debian.org/experimental/linux-image-5.0.0-trunk-riscv64

> This way Debain has done all the licence work for you as well :)

And all the building work ;)

> 
> Alistair
> 
>>
>>> +        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
>>> +        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
>>> +        kernel_path = self.workdir + "bbl_w_kernel"
>>> +
>>> +        with gzip.open(kernel_path_gz, 'rb') as f_in:
>>> +            with open(kernel_path, 'wb') as f_out:
>>> +                shutil.copyfileobj(f_in, f_out)
>>> +
>>> +        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
>>> +                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
>>> +                      'riscv64/rootfs.cpio.gz')
>>> +        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
>>> +        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
>>> +
>>> +        self.vm.set_machine('virt')
>>> +        self.vm.set_console()
>>> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
>>> +                               + 'console=ttyS0 noreboot')
>>> +        self.vm.add_args('-kernel', kernel_path,
>>> +                         '-initrd', initrd_path,
>>> +                         '-append', kernel_command_line)
>>
>> You might want to use '-no-reboot' here too.
>>
>>> +        self.vm.launch()
>>> +        self.wait_for_console_pattern('Boot successful.')
>>> +
>>> +        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
>>> +                                               'isa')
>>> +        self.exec_command_and_wait_for_pattern('uname -a',
>>> +                                               'sifive')
>>> +        self.exec_command_and_wait_for_pattern('reboot',
>>> +                                               'reboot: Restarting system')
>>
>> Regards,
>>
>> Phil.
>>
diff mbox series

Patch

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 3215950..bbc6b06 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -354,3 +354,43 @@  class BootLinuxConsole(Test):
         self.vm.launch()
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
+
+    def test_riscv64_virt(self):
+        """
+        :avocado: tags=arch:riscv64
+        :avocado: tags=machine:virt
+        """
+
+        kernel_url = ('https://github.com/chihminchao/test-binary/raw/'
+                      '0b7787305d9e40815c05a805266cc74ff356239e/qemu/riscv64/'
+                      'bbl_w_kernel.gz')
+        kernel_hash = 'c7f6cc7967975ad42dc61ee0535db01c9cbd0968'
+        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+        kernel_path = self.workdir + "bbl_w_kernel"
+
+        with gzip.open(kernel_path_gz, 'rb') as f_in:
+            with open(kernel_path, 'wb') as f_out:
+                shutil.copyfileobj(f_in, f_out)
+
+        initrd_url = ('https://github.com/groeck/linux-build-test/raw/'
+                      '8584a59ed9e5eb5ee7ca91f6d74bbb06619205b8/rootfs/'
+                      'riscv64/rootfs.cpio.gz')
+        initrd_hash = 'f4867d263754961b6f626cdcdc0cb334c47e3b49'
+        initrd_path = self.fetch_asset(initrd_url, asset_hash=initrd_hash)
+
+        self.vm.set_machine('virt')
+        self.vm.set_console()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+                               + 'console=ttyS0 noreboot')
+        self.vm.add_args('-kernel', kernel_path,
+                         '-initrd', initrd_path,
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        self.wait_for_console_pattern('Boot successful.')
+
+        self.exec_command_and_wait_for_pattern('cat /proc/cpuinfo',
+                                               'isa')
+        self.exec_command_and_wait_for_pattern('uname -a',
+                                               'sifive')
+        self.exec_command_and_wait_for_pattern('reboot',
+                                               'reboot: Restarting system')