| Message ID | 20201024215200.494209-1-nieklinnenbank@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | tests/acceptance: expand Orange Pi PC SD images for NetBSD, Armbian above nearest power of two | expand |
Hi Niek, On 10/24/20 11:52 PM, Niek Linnenbank wrote: > The acceptance tests for the Orange Pi PC need to expand the SD card images > to a size which is a power of two. As Qemu uses the size of the SD image file > as well for the size of the emulated SD card, this can sometimes give problems > for guests that assume a certain minimum size of the SD card. > > This commit resolves the following acceptance test error for the NetBSD 9.0 test > of the Orange Pi PC by increasing the size of the expanded SD card image to two times > the nearest power of two: > > (5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) > console: DRAM: 1024 MiB > console: Failed to set core voltage! Can't set CPU frequency > /console: Trying to boot from MMC1 > console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) Allwinner Technology > ... > console: Starting kernel ... > console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... > ... > console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 > console: [ 1.3300167] ld0 at sdmmc0: <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> The test has this comment: # This test download a 304MB compressed image and expand it to 2GB Once uncompressed the image is ~1.2GB before rounding to 2GB. > console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, 512 bytes/sect x 2097152 sectors Why the card appears as 1GB?? ^^^^^^^ Can you try reverting commit 6d2d4069c47? ("sd: Correct the maximum size of a Standard Capacity SD Memory Card") > console: [ 1.4102580] ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz > console: [ 2.0674392] WARNING: 4 errors while detecting hardware; check system log. > console: [ 2.0674392] boot device: ld0 > console: [ 2.0775401] root on ld0a dumps on ld0b > console: [ 2.0977679] vfs_mountroot: can't open root device > console: [ 2.0977679] cannot mount root, error = 6 > INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached. > > Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> > Based-on: ("[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC") > --- > tests/acceptance/boot_linux_console.py | 10 +++++----- > 1 file changed, 5 insertions(+), 5 deletions(-)
On 10/25/20 2:31 AM, Philippe Mathieu-Daudé wrote: > Hi Niek, > > On 10/24/20 11:52 PM, Niek Linnenbank wrote: >> The acceptance tests for the Orange Pi PC need to expand the SD card >> images >> to a size which is a power of two. As Qemu uses the size of the SD >> image file >> as well for the size of the emulated SD card, this can sometimes give >> problems >> for guests that assume a certain minimum size of the SD card. >> >> This commit resolves the following acceptance test error for the >> NetBSD 9.0 test >> of the Orange Pi PC by increasing the size of the expanded SD card >> image to two times >> the nearest power of two: >> >> (5/5) >> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: >> |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) >> console: DRAM: 1024 MiB >> console: Failed to set core voltage! Can't set CPU frequency >> /console: Trying to boot from MMC1 >> console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) >> Allwinner Technology >> ... >> console: Starting kernel ... >> console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... >> ... >> console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 >> console: [ 1.3300167] ld0 at sdmmc0: >> <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> > > The test has this comment: > > # This test download a 304MB compressed image and expand it to 2GB > > Once uncompressed the image is ~1.2GB before rounding to 2GB. > >> console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, >> 512 bytes/sect x 2097152 sectors > > Why the card appears as 1GB?? ^^^^^^^ > > Can you try reverting commit 6d2d4069c47? > ("sd: Correct the maximum size of a Standard Capacity SD Memory Card") Now I remember, I hit the similar problem 2 years ago :S See the description of the C_SIZE field in CSD register: "To indicate 2 GByte card, BLOCK_LEN shall be 1024 bytes." This model uses a fixed BLOCK_LEN = 512 bytes. > >> console: [ 1.4102580] ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz >> console: [ 2.0674392] WARNING: 4 errors while detecting hardware; >> check system log. >> console: [ 2.0674392] boot device: ld0 >> console: [ 2.0775401] root on ld0a dumps on ld0b >> console: [ 2.0977679] vfs_mountroot: can't open root device >> console: [ 2.0977679] cannot mount root, error = 6 >> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: >> Timeout reached. >> >> Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> >> Based-on: ("[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from >> Armbian 20.08 on Orange Pi PC") >> --- >> tests/acceptance/boot_linux_console.py | 10 +++++----- >> 1 file changed, 5 insertions(+), 5 deletions(-) >
On Sun, Oct 25, 2020 at 8:13 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: > > On 10/25/20 2:31 AM, Philippe Mathieu-Daudé wrote: > > Hi Niek, > > > > On 10/24/20 11:52 PM, Niek Linnenbank wrote: > >> The acceptance tests for the Orange Pi PC need to expand the SD card > >> images > >> to a size which is a power of two. As Qemu uses the size of the SD > >> image file > >> as well for the size of the emulated SD card, this can sometimes give > >> problems > >> for guests that assume a certain minimum size of the SD card. > >> > >> This commit resolves the following acceptance test error for the > >> NetBSD 9.0 test > >> of the Orange Pi PC by increasing the size of the expanded SD card > >> image to two times > >> the nearest power of two: > >> > >> (5/5) > >> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: > >> |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) > >> console: DRAM: 1024 MiB > >> console: Failed to set core voltage! Can't set CPU frequency > >> /console: Trying to boot from MMC1 > >> console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) > >> Allwinner Technology > >> ... > >> console: Starting kernel ... > >> console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... > >> ... > >> console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 > >> console: [ 1.3300167] ld0 at sdmmc0: > >> <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> > > > > The test has this comment: > > > > # This test download a 304MB compressed image and expand it to 2GB > > > > Once uncompressed the image is ~1.2GB before rounding to 2GB. > > > >> console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, > >> 512 bytes/sect x 2097152 sectors > > > > Why the card appears as 1GB?? ^^^^^^^ > > > > Can you try reverting commit 6d2d4069c47? > > ("sd: Correct the maximum size of a Standard Capacity SD Memory Card") > > Now I remember, I hit the similar problem 2 years ago :S > > See the description of the C_SIZE field in CSD register: > > "To indicate 2 GByte card, BLOCK_LEN shall be 1024 bytes." Ah, yes! 2 GiB is special in the spec. Thanks Philippe! > > This model uses a fixed BLOCK_LEN = 512 bytes. > Niek, could you please try the following patch? http://patchwork.ozlabs.org/project/qemu-devel/patch/20201025152357.11865-1-bmeng.cn@gmail.com/ Regards, Bin
Hi Philippe, I've tested the patch by Bin on the 2G & Sd blocksize issue, see my response there. https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg07318.html So probably we do not need the patch I send here. Regards, Niek On Mon, Oct 26, 2020 at 12:48 AM Niek Linnenbank <nieklinnenbank@gmail.com> wrote: > Hello Philippe, > > On Sun, Oct 25, 2020 at 1:11 PM Philippe Mathieu-Daudé <f4bug@amsat.org> > wrote: > >> On 10/25/20 2:31 AM, Philippe Mathieu-Daudé wrote: >> > Hi Niek, >> > >> > On 10/24/20 11:52 PM, Niek Linnenbank wrote: >> >> The acceptance tests for the Orange Pi PC need to expand the SD card >> >> images >> >> to a size which is a power of two. As Qemu uses the size of the SD >> >> image file >> >> as well for the size of the emulated SD card, this can sometimes give >> >> problems >> >> for guests that assume a certain minimum size of the SD card. >> >> >> >> This commit resolves the following acceptance test error for the >> >> NetBSD 9.0 test >> >> of the Orange Pi PC by increasing the size of the expanded SD card >> >> image to two times >> >> the nearest power of two: >> >> >> >> (5/5) >> >> >> tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: >> >> >> |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) >> >> console: DRAM: 1024 MiB >> >> console: Failed to set core voltage! Can't set CPU frequency >> >> /console: Trying to boot from MMC1 >> >> console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) >> >> Allwinner Technology >> >> ... >> >> console: Starting kernel ... >> >> console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... >> >> ... >> >> console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 >> >> console: [ 1.3300167] ld0 at sdmmc0: >> >> <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> >> > >> > The test has this comment: >> > >> > # This test download a 304MB compressed image and expand it to 2GB >> > >> > Once uncompressed the image is ~1.2GB before rounding to 2GB. >> > >> >> console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, >> >> 512 bytes/sect x 2097152 sectors >> > >> > Why the card appears as 1GB?? ^^^^^^^ >> > > Very well spotted Philippe. Indeed that is strange and is probably also > why we are getting these issues. > > >> > >> > Can you try reverting commit 6d2d4069c47? >> > ("sd: Correct the maximum size of a Standard Capacity SD Memory Card") >> >> > I've tried your suggestion and indeed for the NetBSD 9.0 test, reverting > 6d2d4069c47 resolves the error for NetBSD 9.0. > But the armbian tests fail with this output: > > (4/6) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_19_11: > /console: DRAM: 1024 MiB > console: Failed to set core voltage! Can't set CPU frequency > console: Trying to boot from MMC1 > console: U-Boot 2019.04-armbian (Nov 18 2019 - 23:08:35 +0100) Allwinner > Technology > console: CPU: Allwinner H3 (SUN8I 0000) > ... > console: Autoboot in 1 seconds, press <Space> to stop > /console: switch to partitions #0, OK > console: mmc0 is current device > ... > console: Loading Device Tree to 49757000, end 497c6fff ... OK > console: Starting kernel ... > console: Uncompressing Linux... done, booting the kernel. > /console: [ 149.045498] debugfs: Directory '1c22c00.codec' with parent > 'H3 Audio Codec' already present! > INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout > reached\nOriginal status: ERROR\n > > The 'setenv extraargs' isn't done when 'Autoboot in' appears, which is why > there is much less output from the kernel. > And it looks like that is because 'U-Boot SPL' isn't printed in the output > so the python test is still waiting for that. > > However with the new patch submitted by Bin just now on the list, all > tests are passing again like before, including the new 20.08 armbian test: > https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg07318.html > > (I'll reply to that message shortly) > > Regards, > Niek > > > >> Now I remember, I hit the similar problem 2 years ago :S >> >> See the description of the C_SIZE field in CSD register: >> >> "To indicate 2 GByte card, BLOCK_LEN shall be 1024 bytes." >> >> This model uses a fixed BLOCK_LEN = 512 bytes. >> >> > >> >> console: [ 1.4102580] ld0: 4-bit width, High-Speed/SDR25, 50.000 >> MHz >> >> console: [ 2.0674392] WARNING: 4 errors while detecting hardware; >> >> check system log. >> >> console: [ 2.0674392] boot device: ld0 >> >> console: [ 2.0775401] root on ld0a dumps on ld0b >> >> console: [ 2.0977679] vfs_mountroot: can't open root device >> >> console: [ 2.0977679] cannot mount root, error = 6 >> >> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: >> >> Timeout reached. >> >> >> >> Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> >> >> Based-on: ("[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from >> >> Armbian 20.08 on Orange Pi PC") >> >> --- >> >> tests/acceptance/boot_linux_console.py | 10 +++++----- >> >> 1 file changed, 5 insertions(+), 5 deletions(-) >> > >> > > > -- > Niek Linnenbank > >
Hello Philippe, On Sun, Oct 25, 2020 at 1:11 PM Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: > On 10/25/20 2:31 AM, Philippe Mathieu-Daudé wrote: > > Hi Niek, > > > > On 10/24/20 11:52 PM, Niek Linnenbank wrote: > >> The acceptance tests for the Orange Pi PC need to expand the SD card > >> images > >> to a size which is a power of two. As Qemu uses the size of the SD > >> image file > >> as well for the size of the emulated SD card, this can sometimes give > >> problems > >> for guests that assume a certain minimum size of the SD card. > >> > >> This commit resolves the following acceptance test error for the > >> NetBSD 9.0 test > >> of the Orange Pi PC by increasing the size of the expanded SD card > >> image to two times > >> the nearest power of two: > >> > >> (5/5) > >> > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: > > >> |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) > >> console: DRAM: 1024 MiB > >> console: Failed to set core voltage! Can't set CPU frequency > >> /console: Trying to boot from MMC1 > >> console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) > >> Allwinner Technology > >> ... > >> console: Starting kernel ... > >> console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... > >> ... > >> console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 > >> console: [ 1.3300167] ld0 at sdmmc0: > >> <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> > > > > The test has this comment: > > > > # This test download a 304MB compressed image and expand it to 2GB > > > > Once uncompressed the image is ~1.2GB before rounding to 2GB. > > > >> console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, > >> 512 bytes/sect x 2097152 sectors > > > > Why the card appears as 1GB?? ^^^^^^^ > Very well spotted Philippe. Indeed that is strange and is probably also why we are getting these issues. > > > > Can you try reverting commit 6d2d4069c47? > > ("sd: Correct the maximum size of a Standard Capacity SD Memory Card") > > I've tried your suggestion and indeed for the NetBSD 9.0 test, reverting 6d2d4069c47 resolves the error for NetBSD 9.0. But the armbian tests fail with this output: (4/6) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_19_11: /console: DRAM: 1024 MiB console: Failed to set core voltage! Can't set CPU frequency console: Trying to boot from MMC1 console: U-Boot 2019.04-armbian (Nov 18 2019 - 23:08:35 +0100) Allwinner Technology console: CPU: Allwinner H3 (SUN8I 0000) ... console: Autoboot in 1 seconds, press <Space> to stop /console: switch to partitions #0, OK console: mmc0 is current device ... console: Loading Device Tree to 49757000, end 497c6fff ... OK console: Starting kernel ... console: Uncompressing Linux... done, booting the kernel. /console: [ 149.045498] debugfs: Directory '1c22c00.codec' with parent 'H3 Audio Codec' already present! INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n The 'setenv extraargs' isn't done when 'Autoboot in' appears, which is why there is much less output from the kernel. And it looks like that is because 'U-Boot SPL' isn't printed in the output so the python test is still waiting for that. However with the new patch submitted by Bin just now on the list, all tests are passing again like before, including the new 20.08 armbian test: https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg07318.html (I'll reply to that message shortly) Regards, Niek > Now I remember, I hit the similar problem 2 years ago :S > > See the description of the C_SIZE field in CSD register: > > "To indicate 2 GByte card, BLOCK_LEN shall be 1024 bytes." > > This model uses a fixed BLOCK_LEN = 512 bytes. > > > > >> console: [ 1.4102580] ld0: 4-bit width, High-Speed/SDR25, 50.000 > MHz > >> console: [ 2.0674392] WARNING: 4 errors while detecting hardware; > >> check system log. > >> console: [ 2.0674392] boot device: ld0 > >> console: [ 2.0775401] root on ld0a dumps on ld0b > >> console: [ 2.0977679] vfs_mountroot: can't open root device > >> console: [ 2.0977679] cannot mount root, error = 6 > >> INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: > >> Timeout reached. > >> > >> Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> > >> Based-on: ("[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from > >> Armbian 20.08 on Orange Pi PC") > >> --- > >> tests/acceptance/boot_linux_console.py | 10 +++++----- > >> 1 file changed, 5 insertions(+), 5 deletions(-) > > >
diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py index b604cfe7da..40835904bb 100644 --- a/tests/acceptance/boot_linux_console.py +++ b/tests/acceptance/boot_linux_console.py @@ -35,11 +35,11 @@ def pow2ceil(x): return 1 if x == 0 else 2**(x - 1).bit_length() """ -Expand file size to next power of 2 +Expand file size to next power of 2 or higher """ -def image_pow2ceil_expand(path): +def image_pow2ceil_expand(path, multiply_factor = 1): size = os.path.getsize(path) - size_aligned = pow2ceil(size) + size_aligned = pow2ceil(size) * multiply_factor if size != size_aligned: with open(path, 'ab+') as fd: fd.truncate(size_aligned) @@ -850,7 +850,7 @@ class BootLinuxConsole(LinuxKernelTest): image_path_xz = self.fetch_asset(image_url, asset_hash=image_hash, algorithm='sha256') image_path = archive.extract(image_path_xz, self.workdir) - image_pow2ceil_expand(image_path) + image_pow2ceil_expand(image_path, 2) self.do_test_arm_orangepi_uboot_armbian(image_path) @@ -879,7 +879,7 @@ class BootLinuxConsole(LinuxKernelTest): image_path_gz = self.fetch_asset(image_url, asset_hash=image_hash) image_path = os.path.join(self.workdir, 'armv7.img') archive.gzip_uncompress(image_path_gz, image_path) - image_pow2ceil_expand(image_path) + image_pow2ceil_expand(image_path, 2) image_drive_args = 'if=sd,format=raw,snapshot=on,file=' + image_path # dd if=u-boot-sunxi-with-spl.bin of=armv7.img bs=1K seek=8 conv=notrunc
The acceptance tests for the Orange Pi PC need to expand the SD card images to a size which is a power of two. As Qemu uses the size of the SD image file as well for the size of the emulated SD card, this can sometimes give problems for guests that assume a certain minimum size of the SD card. This commit resolves the following acceptance test error for the NetBSD 9.0 test of the Orange Pi PC by increasing the size of the expanded SD card image to two times the nearest power of two: (5/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9: |console: U-Boot SPL 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) console: DRAM: 1024 MiB console: Failed to set core voltage! Can't set CPU frequency /console: Trying to boot from MMC1 console: U-Boot 2020.01+dfsg-1 (Jan 08 2020 - 08:19:44 +0000) Allwinner Technology ... console: Starting kernel ... console: [ 1.0000000] NetBSD/evbarm (fdt) booting ... ... console: [ 1.3300167] sdmmc0: SD card status: 4-bit, C0 console: [ 1.3300167] ld0 at sdmmc0: <0xaa:0x5859:QEMU!:0x01:0xdeadbeef:0x062> console: [ 1.3430678] ld0: 1024 MB, 1040 cyl, 32 head, 63 sec, 512 bytes/sect x 2097152 sectors console: [ 1.4102580] ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz console: [ 2.0674392] WARNING: 4 errors while detecting hardware; check system log. console: [ 2.0674392] boot device: ld0 console: [ 2.0775401] root on ld0a dumps on ld0b console: [ 2.0977679] vfs_mountroot: can't open root device console: [ 2.0977679] cannot mount root, error = 6 INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached. Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com> Based-on: ("[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC") --- tests/acceptance/boot_linux_console.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)