diff mbox series

[5/5] test: unit test for smbios command

Message ID 20231223004429.247301-6-xypron.glpk@gmx.de
State Changes Requested
Delegated to: Tom Rini
Headers show
Series cmd: provide command to display SMBIOS information | expand

Commit Message

Heinrich Schuchardt Dec. 23, 2023, 12:44 a.m. UTC
Provide a unit test for the smbios command.

Provide different test functions for QEMU, sandbox, and other systems.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
---
 test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)
 create mode 100644 test/py/tests/test_smbios.py

--
2.43.0

Comments

Simon Glass Dec. 26, 2023, 9:47 a.m. UTC | #1
Hi Heinrich,

On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> Provide a unit test for the smbios command.
>
> Provide different test functions for QEMU, sandbox, and other systems.
>
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> ---
>  test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>  create mode 100644 test/py/tests/test_smbios.py

Can you please write this test in C?

Regards,
Simon
Heinrich Schuchardt Dec. 26, 2023, 10:11 a.m. UTC | #2
On 12/26/23 10:47, Simon Glass wrote:
> Hi Heinrich,
>
> On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> Provide a unit test for the smbios command.
>>
>> Provide different test functions for QEMU, sandbox, and other systems.
>>
>> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
>> ---
>>   test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
>>   1 file changed, 47 insertions(+)
>>   create mode 100644 test/py/tests/test_smbios.py
>
> Can you please write this test in C?
>
> Regards,
> Simon

According to our documentation we can use either Python or C tests.
We generally use Python tests for testing the CLI.

I cannot see any benefit in trying to write this in C.

Best regards

Heinrich
Simon Glass Dec. 27, 2023, 7:40 a.m. UTC | #3
Hi Heinrich,

On Tue, Dec 26, 2023 at 10:11 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 12/26/23 10:47, Simon Glass wrote:
> > Hi Heinrich,
> >
> > On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> Provide a unit test for the smbios command.
> >>
> >> Provide different test functions for QEMU, sandbox, and other systems.
> >>
> >> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> >> ---
> >>   test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
> >>   1 file changed, 47 insertions(+)
> >>   create mode 100644 test/py/tests/test_smbios.py
> >
> > Can you please write this test in C?
> >
> > Regards,
> > Simon
>
> According to our documentation we can use either Python or C tests.
> We generally use Python tests for testing the CLI.

Not really...there are lots of C tests for the CLI. It is actually
easier to code, in most cases.

>
> I cannot see any benefit in trying to write this in C.

I did go to some effort to document this:

https://docs.u-boot.org/en/latest/develop/tests_writing.html#test-types

Regards,
Simon
Ilias Apalodimas Dec. 27, 2023, 2:15 p.m. UTC | #4
On Sat, Dec 23, 2023 at 01:44:29AM +0100, Heinrich Schuchardt wrote:
> Provide a unit test for the smbios command.
>
> Provide different test functions for QEMU, sandbox, and other systems.
>
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> ---
>  test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>  create mode 100644 test/py/tests/test_smbios.py
>
> diff --git a/test/py/tests/test_smbios.py b/test/py/tests/test_smbios.py
> new file mode 100644
> index 0000000000..86d8d07539
> --- /dev/null
> +++ b/test/py/tests/test_smbios.py
> @@ -0,0 +1,47 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +"""Test smbios command"""
> +
> +import pytest
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.notbuildconfigspec('qfw_smbios')
> +@pytest.mark.notbuildconfigspec('sandbox')
> +def test_cmd_smbios(u_boot_console):
> +    """Run the smbios command"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 0,' in output
> +    assert 'String 1: U-Boot' in output
> +    assert 'DMI type 1,' in output
> +    assert 'DMI type 2,' in output
> +    assert 'DMI type 3,' in output
> +    assert 'DMI type 4,' in output
> +    assert 'DMI type 127,' in output
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.buildconfigspec('qfw_smbios')
> +@pytest.mark.notbuildconfigspec('sandbox')
> +# TODO:
> +# QEMU v8.2.0 lacks SMBIOS support for RISC-V
> +# Once support is available in our Docker image we can remove the constraint.
> +@pytest.mark.notbuildconfigspec('riscv')
> +def test_cmd_smbios_qemu(u_boot_console):
> +    """Run the smbios command on QEMU"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 1,' in output
> +    assert 'Manufacturer: QEMU' in output
> +    assert 'DMI type 127,' in output
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.buildconfigspec('sandbox')
> +def test_cmd_smbios_sandbox(u_boot_console):
> +    """Run the smbios command on the sandbox"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 0,' in output
> +    assert 'String 1: U-Boot' in output
> +    assert 'DMI type 1,' in output
> +    assert 'Manufacturer: sandbox' in output
> +    assert 'DMI type 2,' in output
> +    assert 'DMI type 3,' in output
> +    assert 'DMI type 4,' in output
> +    assert 'DMI type 127,' in output
> --
> 2.43.0
>

Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Heinrich Schuchardt Dec. 28, 2023, 5:06 p.m. UTC | #5
Am 27. Dezember 2023 08:40:40 MEZ schrieb Simon Glass <sjg@chromium.org>:
>Hi Heinrich,
>
>On Tue, Dec 26, 2023 at 10:11 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> On 12/26/23 10:47, Simon Glass wrote:
>> > Hi Heinrich,
>> >
>> > On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>> >>
>> >> Provide a unit test for the smbios command.
>> >>
>> >> Provide different test functions for QEMU, sandbox, and other systems.
>> >>
>> >> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
>> >> ---
>> >>   test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
>> >>   1 file changed, 47 insertions(+)
>> >>   create mode 100644 test/py/tests/test_smbios.py
>> >
>> > Can you please write this test in C?
>> >
>> > Regards,
>> > Simon
>>
>> According to our documentation we can use either Python or C tests.
>> We generally use Python tests for testing the CLI.
>
>Not really...there are lots of C tests for the CLI. It is actually
>easier to code, in most cases.
>
>>
>> I cannot see any benefit in trying to write this in C.
>
>I did go to some effort to document this:
>
>https://docs.u-boot.org/en/latest/develop/tests_writing.html#test-types

As described in that document C is most adequate for testing on the sandbox. I want to run this test on QEMU and real boards with release U-Boot firmware, i.e. without deviating configuration.

Best regards

Heinrich

>
>Regards,
>Simon
Tom Rini Dec. 28, 2023, 5:23 p.m. UTC | #6
On Thu, Dec 28, 2023 at 06:06:23PM +0100, Heinrich Schuchardt wrote:
> 
> 
> Am 27. Dezember 2023 08:40:40 MEZ schrieb Simon Glass <sjg@chromium.org>:
> >Hi Heinrich,
> >
> >On Tue, Dec 26, 2023 at 10:11 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> On 12/26/23 10:47, Simon Glass wrote:
> >> > Hi Heinrich,
> >> >
> >> > On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >> >>
> >> >> Provide a unit test for the smbios command.
> >> >>
> >> >> Provide different test functions for QEMU, sandbox, and other systems.
> >> >>
> >> >> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> >> >> ---
> >> >>   test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
> >> >>   1 file changed, 47 insertions(+)
> >> >>   create mode 100644 test/py/tests/test_smbios.py
> >> >
> >> > Can you please write this test in C?
> >> >
> >> > Regards,
> >> > Simon
> >>
> >> According to our documentation we can use either Python or C tests.
> >> We generally use Python tests for testing the CLI.
> >
> >Not really...there are lots of C tests for the CLI. It is actually
> >easier to code, in most cases.
> >
> >>
> >> I cannot see any benefit in trying to write this in C.
> >
> >I did go to some effort to document this:
> >
> >https://docs.u-boot.org/en/latest/develop/tests_writing.html#test-types
> 
> As described in that document C is most adequate for testing on the sandbox. I want to run this test on QEMU and real boards with release U-Boot firmware, i.e. without deviating configuration.

The important part is that we have a test. And I really don't want to
re-read too much of that doc right now as I feel like it gets things a
bit wrong.
Simon Glass Dec. 28, 2023, 7:48 p.m. UTC | #7
Hi Heinrich,

On Thu, Dec 28, 2023 at 5:11 PM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
>
>
> Am 27. Dezember 2023 08:40:40 MEZ schrieb Simon Glass <sjg@chromium.org>:
> >Hi Heinrich,
> >
> >On Tue, Dec 26, 2023 at 10:11 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >>
> >> On 12/26/23 10:47, Simon Glass wrote:
> >> > Hi Heinrich,
> >> >
> >> > On Sat, Dec 23, 2023 at 12:49 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
> >> >>
> >> >> Provide a unit test for the smbios command.
> >> >>
> >> >> Provide different test functions for QEMU, sandbox, and other systems.
> >> >>
> >> >> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> >> >> ---
> >> >>   test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
> >> >>   1 file changed, 47 insertions(+)
> >> >>   create mode 100644 test/py/tests/test_smbios.py
> >> >
> >> > Can you please write this test in C?
> >> >
> >> > Regards,
> >> > Simon
> >>
> >> According to our documentation we can use either Python or C tests.
> >> We generally use Python tests for testing the CLI.
> >
> >Not really...there are lots of C tests for the CLI. It is actually
> >easier to code, in most cases.
> >
> >>
> >> I cannot see any benefit in trying to write this in C.
> >
> >I did go to some effort to document this:
> >
> >https://docs.u-boot.org/en/latest/develop/tests_writing.html#test-types
>
> As described in that document C is most adequate for testing on the sandbox. I want to run this test on QEMU and real boards with release U-Boot firmware, i.e. without deviating configuration.

Are you checking that the compiler works? Most such tests are a waste
of time, IMO. Anyway, any test is better than no test, as Tom says,
so:

Reviewed-by: Simon Glass <sjg@chromium.org>

Regards,
SImon
Tom Rini Jan. 16, 2024, 10:04 p.m. UTC | #8
On Sat, Dec 23, 2023 at 01:44:29AM +0100, Heinrich Schuchardt wrote:

> Provide a unit test for the smbios command.
> 
> Provide different test functions for QEMU, sandbox, and other systems.
> 
> Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
> Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
> Reviewed-by: Simon Glass <sjg@chromium.org>
> ---
>  test/py/tests/test_smbios.py | 47 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
>  create mode 100644 test/py/tests/test_smbios.py
> 
> --
> 2.43.0
> 
> diff --git a/test/py/tests/test_smbios.py b/test/py/tests/test_smbios.py
> new file mode 100644
> index 0000000000..86d8d07539
> --- /dev/null
> +++ b/test/py/tests/test_smbios.py
> @@ -0,0 +1,47 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +"""Test smbios command"""
> +
> +import pytest
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.notbuildconfigspec('qfw_smbios')
> +@pytest.mark.notbuildconfigspec('sandbox')
> +def test_cmd_smbios(u_boot_console):
> +    """Run the smbios command"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 0,' in output
> +    assert 'String 1: U-Boot' in output
> +    assert 'DMI type 1,' in output
> +    assert 'DMI type 2,' in output
> +    assert 'DMI type 3,' in output
> +    assert 'DMI type 4,' in output
> +    assert 'DMI type 127,' in output
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.buildconfigspec('qfw_smbios')
> +@pytest.mark.notbuildconfigspec('sandbox')
> +# TODO:
> +# QEMU v8.2.0 lacks SMBIOS support for RISC-V
> +# Once support is available in our Docker image we can remove the constraint.
> +@pytest.mark.notbuildconfigspec('riscv')
> +def test_cmd_smbios_qemu(u_boot_console):
> +    """Run the smbios command on QEMU"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 1,' in output
> +    assert 'Manufacturer: QEMU' in output
> +    assert 'DMI type 127,' in output
> +
> +@pytest.mark.buildconfigspec('cmd_smbios')
> +@pytest.mark.buildconfigspec('sandbox')
> +def test_cmd_smbios_sandbox(u_boot_console):
> +    """Run the smbios command on the sandbox"""
> +    output = u_boot_console.run_command('smbios')
> +    assert 'DMI type 0,' in output
> +    assert 'String 1: U-Boot' in output
> +    assert 'DMI type 1,' in output
> +    assert 'Manufacturer: sandbox' in output
> +    assert 'DMI type 2,' in output
> +    assert 'DMI type 3,' in output
> +    assert 'DMI type 4,' in output
> +    assert 'DMI type 127,' in output

This fails on coreboot:
https://source.denx.de/u-boot/u-boot/-/jobs/771654
diff mbox series

Patch

diff --git a/test/py/tests/test_smbios.py b/test/py/tests/test_smbios.py
new file mode 100644
index 0000000000..86d8d07539
--- /dev/null
+++ b/test/py/tests/test_smbios.py
@@ -0,0 +1,47 @@ 
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+"""Test smbios command"""
+
+import pytest
+
+@pytest.mark.buildconfigspec('cmd_smbios')
+@pytest.mark.notbuildconfigspec('qfw_smbios')
+@pytest.mark.notbuildconfigspec('sandbox')
+def test_cmd_smbios(u_boot_console):
+    """Run the smbios command"""
+    output = u_boot_console.run_command('smbios')
+    assert 'DMI type 0,' in output
+    assert 'String 1: U-Boot' in output
+    assert 'DMI type 1,' in output
+    assert 'DMI type 2,' in output
+    assert 'DMI type 3,' in output
+    assert 'DMI type 4,' in output
+    assert 'DMI type 127,' in output
+
+@pytest.mark.buildconfigspec('cmd_smbios')
+@pytest.mark.buildconfigspec('qfw_smbios')
+@pytest.mark.notbuildconfigspec('sandbox')
+# TODO:
+# QEMU v8.2.0 lacks SMBIOS support for RISC-V
+# Once support is available in our Docker image we can remove the constraint.
+@pytest.mark.notbuildconfigspec('riscv')
+def test_cmd_smbios_qemu(u_boot_console):
+    """Run the smbios command on QEMU"""
+    output = u_boot_console.run_command('smbios')
+    assert 'DMI type 1,' in output
+    assert 'Manufacturer: QEMU' in output
+    assert 'DMI type 127,' in output
+
+@pytest.mark.buildconfigspec('cmd_smbios')
+@pytest.mark.buildconfigspec('sandbox')
+def test_cmd_smbios_sandbox(u_boot_console):
+    """Run the smbios command on the sandbox"""
+    output = u_boot_console.run_command('smbios')
+    assert 'DMI type 0,' in output
+    assert 'String 1: U-Boot' in output
+    assert 'DMI type 1,' in output
+    assert 'Manufacturer: sandbox' in output
+    assert 'DMI type 2,' in output
+    assert 'DMI type 3,' in output
+    assert 'DMI type 4,' in output
+    assert 'DMI type 127,' in output