diff mbox series

[11/11] Nokia RX-51: Add automated test for running RX-51 build in qemu

Message ID 20200331223518.10936-12-pali@kernel.org
State Superseded
Delegated to: Lokesh Vutla
Headers show
Series Fixes for Nokia RX-51 | expand

Commit Message

Pali Rohár March 31, 2020, 10:35 p.m. UTC 
This patch contains a script which automatically download and compile all
needed tools to build a simple MTD images for booting Maemo kernel image by
U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
machine provided by qemu-linaro project.

It can be used to check that U-Boot for Nokia N900 is not broken and can be
successfully booted in emulator.

Script is registered in to .travis.yml so it would be automatically run on
Travi CI service.

Signed-off-by: Pali Rohár <pali@kernel.org>
---
 .travis.yml       |  10 +++
 test/rx51_test.sh | 208 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 218 insertions(+)
 create mode 100755 test/rx51_test.sh

Comments

Pali Rohár April 14, 2020, 10:40 a.m. UTC | #1 
On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> This patch contains a script which automatically download and compile all
> needed tools to build a simple MTD images for booting Maemo kernel image by
> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> machine provided by qemu-linaro project.
> 
> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> successfully booted in emulator.
> 
> Script is registered in to .travis.yml so it would be automatically run on
> Travi CI service.
> 
> Signed-off-by: Pali Rohár <pali@kernel.org>

Tom Rini, in past you have asked me for N900 Travis test. So could you
please review this patch (including fixup at the bottom)?

> ---
>  .travis.yml       |  10 +++
>  test/rx51_test.sh | 208 ++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 218 insertions(+)
>  create mode 100755 test/rx51_test.sh
> 
> diff --git a/.travis.yml b/.travis.yml
> index c59bd7790b..d96811473c 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -40,6 +40,8 @@ addons:
>      - clang-7
>      - srecord
>      - graphviz
> +    - mtools
> +    - mtd-utils
>  
>  install:
>   # Clone uboot-test-hooks
> @@ -116,6 +118,11 @@ script:
>   # Comments must be outside the command strings below, or the Travis parser
>   # will get confused.
>   #
> + # Run tests for Nokia RX-51 (aka N900)
> + - if [[ -n "${NOKIA_RX51}" ]]; then
> +     test/rx51_test.sh
> +     exit $?;
> +   fi
>   # From buildman, exit code 129 means warnings only.  If we've been asked to
>   # use clang only do one configuration.
>   - if [[ "${BUILDMAN}" != "" ]]; then
> @@ -160,6 +167,9 @@ matrix:
>    include:
>    # we need to build by vendor due to 50min time limit for builds
>    # each env setting here is a dedicated build
> +    - name: "nokia rx51"
> +      env:
> +        - NOKIA_RX51=1
>      - name: "buildman arc"
>        env:
>          - BUILDMAN="arc"
> diff --git a/test/rx51_test.sh b/test/rx51_test.sh
> new file mode 100755
> index 0000000000..43ecc07c08
> --- /dev/null
> +++ b/test/rx51_test.sh
> @@ -0,0 +1,208 @@
> +#!/bin/sh -e
> +# SPDX-License-Identifier: GPL-2.0+
> +# (C) 2020 Pali Rohár <pali@kernel.org>
> +
> +# This test script depends on external tools:
> +# wget git truncate tar dpkg dd mcopy (from mtools) mkfs.ubifs (from mtd-utils) ubinize (from mtd-utils)
> +
> +# Download and compile linaro version qemu which has support for n900 machine
> +# Last working commit is 8f8d8e0796efe1a6f34cdd83fb798f3c41217ec1
> +if ! test -f qemu-system-arm; then
> +	test -d qemu-linaro || git clone https://git.linaro.org/qemu/qemu-linaro.git
> +	cd qemu-linaro
> +	git checkout 8f8d8e0796efe1a6f34cdd83fb798f3c41217ec1
> +	./configure --enable-system --target-list=arm-softmmu --disable-sdl --disable-gtk --disable-curses --audio-drv-list=pa,alsa --audio-card-list= --disable-werror --disable-xen --disable-xen-pci-passthrough --disable-brlapi --disable-vnc --disable-curl --disable-slirp --disable-kvm --disable-user --disable-linux-user --disable-bsd-user --disable-guest-base --disable-uuid --disable-vde --disable-linux-aio --disable-cap-ng --disable-attr --disable-blobs --disable-docs --disable-spice --disable-libiscsi --disable-smartcard-nss --disable-usb-redir --disable-guest-agent --disable-seccomp --disable-glusterfs --disable-nptl --disable-fdt
> +	make -j4
> +	cd ..
> +	ln -s qemu-linaro/arm-softmmu/qemu-system-arm .
> +fi
> +
> +# Download and compile dosfstools with mbr support
> +# Currently this support is in open pull request 95
> +if ! test -f mkfs.fat; then
> +	test -d dosfstools || git clone https://github.com/dosfstools/dosfstools.git
> +	cd dosfstools
> +	git fetch origin refs/pull/95/head
> +	git checkout FETCH_HEAD
> +	./autogen.sh
> +	./configure
> +	make -j4
> +	cd ..
> +	ln -s dosfstools/src/mkfs.fat .
> +fi
> +
> +# Download qflasher and nolo images
> +# This is proprietary qemu flasher tool with first stage images, but license allows non-commercial redistribution
> +wget -c http://repository.maemo.org/qemu-n900/qemu-n900.tar.gz
> +tar -xf qemu-n900.tar.gz
> +
> +# Download Maemo script u-boot-gen-combined
> +if ! test -f u-boot-gen-combined; then
> +	test -d u-boot-maemo || git clone https://github.com/pali/u-boot-maemo.git
> +	chmod +x u-boot-maemo/debian/u-boot-gen-combined
> +	ln -s u-boot-maemo/debian/u-boot-gen-combined .
> +fi
> +
> +# Download Maemo fiasco kernel
> +wget -c http://repository.maemo.org/pool/maemo5.0/free/k/kernel/kernel_2.6.28-20103103+0m5_armel.deb
> +dpkg -x kernel_2.6.28-20103103+0m5_armel.deb kernel_2.6.28
> +
> +# Download Maemo libc
> +wget -c http://repository.maemo.org/pool/maemo5.0/free/g/glibc/libc6_2.5.1-1eglibc27+0m5_armel.deb
> +dpkg -x libc6_2.5.1-1eglibc27+0m5_armel.deb libc6_2.5.1
> +
> +# Download Maemo busybox
> +wget -c http://repository.maemo.org/pool/maemo5.0/free/b/busybox/busybox_1.10.2.legal-1osso30+0m5_armel.deb
> +dpkg -x busybox_1.10.2.legal-1osso30+0m5_armel.deb busybox_1.10.2
> +
> +# Generate rootfs directory
> +# WARNING: there is "sudo mknod" call for /dev/console
> +mkdir -p rootfs
> +mkdir -p rootfs/dev/
> +mkdir -p rootfs/bin/
> +mkdir -p rootfs/sbin/
> +mkdir -p rootfs/lib/
> +test -c rootfs/dev/console || sudo mknod rootfs/dev/console c 5 1
> +cp -a busybox_1.10.2/bin/busybox rootfs/bin/
> +cp -a libc6_2.5.1/lib/ld-linux.so.3 rootfs/lib/
> +cp -a libc6_2.5.1/lib/ld-2.5.so rootfs/lib/
> +cp -a libc6_2.5.1/lib/libc.so.6 rootfs/lib/
> +cp -a libc6_2.5.1/lib/libc-2.5.so rootfs/lib/
> +cp -a libc6_2.5.1/lib/libcrypt.so.1 rootfs/lib/
> +cp -a libc6_2.5.1/lib/libcrypt-2.5.so rootfs/lib/
> +test -f rootfs/bin/sh || ln -sf busybox rootfs/bin/sh
> +test -f rootfs/sbin/poweroff || ln -sf ../bin/busybox rootfs/sbin/poweroff
> +cat > rootfs/sbin/preinit << EOF
> +#!/bin/sh
> +echo
> +echo "Successfully booted"
> +echo
> +/sbin/poweroff -f
> +EOF
> +chmod +x rootfs/sbin/preinit
> +
> +# Generate bootmenu for eMMC booting
> +cat > bootmenu_emmc << EOF
> +setenv bootmenu_0 'uImage-2.6.28-omap1 from eMMC=setenv mmcnum 1; setenv mmcpart 1; setenv mmctype fat; setenv bootargs; setenv setup_omap_atag 1; setenv mmckernfile uImage-2.6.28-omap1; run trymmckernboot';
> +setenv bootmenu_1;
> +setenv bootmenu_delay 1;
> +setenv bootdelay 1;
> +EOF
> +./mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n bootmenu -d bootmenu_emmc bootmenu_emmc.scr
> +
> +# Generate bootmenu for OneNAND booting
> +# FIXME: is address, size and offset really correct?
> +cat > bootmenu_nand << EOF
> +setenv bootmenu_0 'uImage-2.6.28-omap1 from OneNAND=mtd read initfs \${kernaddr} 0x800 0x1FF800; setenv bootargs; setenv setup_omap_atag 1; bootm \${kernaddr}';
> +setenv bootmenu_1;
> +setenv bootmenu_delay 1;
> +setenv bootdelay 1;
> +EOF
> +./mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n bootmenu -d bootmenu_nand bootmenu_nand.scr
> +
> +# Generate ubi config file for ubi rootfs image
> +cat > ubi.ini << EOF
> +[rootfs]
> +mode=ubi
> +image=ubifs.img
> +vol_id=0
> +vol_size=160MiB
> +vol_type=dynamic
> +vol_name=rootfs
> +vol_alignment=1
> +vol_flags=autoresize
> +EOF
> +
> +# Generate combined image from u-boot and Maemo fiasco kernel
> +dd if=kernel_2.6.28/boot/zImage-2.6.28-20103103+0m5.fiasco of=zImage-2.6.28-omap1 skip=95 bs=1
> +./mkimage -A arm -O linux -T kernel -C none -a 80008000 -e 80008000 -n zImage-2.6.28-omap1 -d zImage-2.6.28-omap1 uImage-2.6.28-omap1
> +./u-boot-gen-combined u-boot.bin uImage-2.6.28-omap1 combined.bin
> +
> +# Generate combined hack image from u-boot and Maemo fiasco kernel (kernel starts at 2MB offset)
> +cp u-boot.bin combined_hack.bin
> +dd if=uImage-2.6.28-omap1 of=combined_hack.bin bs=1024 seek=2048
> +
> +# Generate ubi rootfs image from rootfs directory
> +/usr/sbin/mkfs.ubifs -m 2048 -e 129024 -c 2047 -r rootfs ubifs.img
> +/usr/sbin/ubinize -o ubi.img -p 128KiB -m 2048 -s 512 ubi.ini
> +
> +# Generate FAT32 eMMC image for eMMC booting
> +truncate -s 50MiB emmc_emmc.img
> +./mkfs.fat --mbr -F32 emmc_emmc.img
> +mcopy uImage-2.6.28-omap1 ::/uImage-2.6.28-omap1 -i emmc_emmc.img
> +mcopy bootmenu_emmc.scr ::/bootmenu.scr -i emmc_emmc.img
> +
> +# Generate FAT32 eMMC image for OneNAND booting
> +truncate -s 50MiB emmc_nand.img
> +./mkfs.fat --mbr -F32 emmc_nand.img
> +mcopy bootmenu_nand.scr ::/bootmenu.scr -i emmc_nand.img
> +
> +# Generate MTD image for RAM booting from bootloader nolo images, compiled image and rootfs image
> +rm -f mtd_ram.img
> +./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k combined.bin -r ubi.img -m rx51 -o mtd_ram.img
> +
> +# Generate MTD image for eMMC booting from bootloader nolo images, u-boot image and rootfs image
> +rm -f mtd_emmc.img
> +./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k u-boot.bin -r ubi.img -m rx51 -o mtd_emmc.img
> +
> +# Generate MTD image for OneNAND booting from bootloader nolo images, combined hacked image and rootfs image
> +# Kernel image is put into initfs area, but qflasher reject to copy kernel image into initfs area because it does not have initfs signature
> +# This is hack to workaround this problem, tell qflasher that kernel area for u-boot is bigger and put big combined hacked image (u-boot + kernel with correct offset)
> +rm -f mtd_nand.img
> +./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k combined_hack.bin -r ubi.img -m rx51 -p k=4094,i=2 -o mtd_nand.img
> +
> +echo
> +echo
> +echo "All images were successfully generated"
> +echo "Now going to run them in qemu"
> +echo
> +echo
> +
> +# Run MTD image in qemu and wait for 300s if kernel from RAM is correctly booted
> +rm -f qemu_ram.log
> +./qemu-system-arm -M n900 -mtdblock mtd_ram.img -serial /dev/stdout -display none > qemu_ram.log &
> +qemu_pid=$!
> +tail -F qemu_ram.log &
> +tail_pid=$!
> +{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
> +sleep_pid=$!
> +wait $qemu_pid || true
> +kill -9 $tail_pid $sleep_pid 2>/dev/null || true
> +
> +# Run MTD image in qemu and wait for 300s if kernel from eMMC is correctly booted
> +rm -f qemu_emmc.log
> +./qemu-system-arm -M n900 -mtdblock mtd_emmc.img -sd emmc_emmc.img -serial /dev/stdout -display none > qemu_emmc.log &
> +qemu_pid=$!
> +tail -F qemu_emmc.log &
> +tail_pid=$!
> +{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
> +sleep_pid=$!
> +wait $qemu_pid || true
> +kill -9 $tail_pid $sleep_pid 2>/dev/null || true
> +
> +# Run MTD image in qemu and wait for 300s if kernel from OneNAND is correctly booted
> +rm -f qemu_nand.log
> +./qemu-system-arm -M n900 -mtdblock mtd_nand.img -sd emmc_nand.img -serial /dev/stdout -display none > qemu_nand.log &
> +qemu_pid=$!
> +tail -F qemu_nand.log &
> +tail_pid=$!
> +{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
> +sleep_pid=$!
> +wait $qemu_pid || true
> +kill -9 $tail_pid $sleep_pid 2>/dev/null || true
> +
> +echo
> +echo
> +if grep -q 'Successfully booted' qemu_ram.log; then echo "Kernel was successfully booted from RAM"; else echo "Failed to boot kernel from RAM"; fi
> +if grep -q 'Successfully booted' qemu_emmc.log; then echo "Kernel was successfully booted from eMMC"; else echo "Failed to boot kernel from eMMC"; fi
> +if grep -q 'Successfully booted' qemu_nand.log; then echo "Kernel was successfully booted from OneNAND"; else echo "Failed to boot kernel from OneNAND"; fi
> +echo
> +echo
> +
> +if grep -q 'Successfully booted' qemu_ram.log && grep -q 'Successfully booted' qemu_emmc.log && grep -q 'Successfully booted' qemu_nand.log; then
> +	echo "All tests passed"
> +	exit 0
> +else
> +	echo "Some tests failed"
> +	exit 1
> +fi
> -- 
> 2.20.1
> 

This change needs following fixup:
https://github.com/u-boot/u-boot/commit/2ace522bba40b70a26950e7d4b033fd10740dad0

and with it N900 test on Travis passed:
https://travis-ci.org/github/u-boot/u-boot/jobs/669672160

From 2ace522bba40b70a26950e7d4b033fd10740dad0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Pali=20Roh=C3=A1r?= <pali@kernel.org>
Date: Wed, 1 Apr 2020 10:04:11 +0200
Subject: [PATCH] fixup! Nokia RX-51: Add automated test for running RX-51
 build in qemu

---
 .travis.yml       | 1 +
 test/rx51_test.sh | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index d96811473cb..cb556e85282 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -42,6 +42,7 @@ addons:
     - graphviz
     - mtools
     - mtd-utils
+    - gcc-arm-linux-gnueabi
 
 install:
  # Clone uboot-test-hooks
diff --git a/test/rx51_test.sh b/test/rx51_test.sh
index 43ecc07c08a..12dc1f28f0c 100755
--- a/test/rx51_test.sh
+++ b/test/rx51_test.sh
@@ -2,6 +2,10 @@
 # SPDX-License-Identifier: GPL-2.0+
 # (C) 2020 Pali Rohár <pali@kernel.org>
 
+make nokia_rx51_config
+make -j4 u-boot.bin ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
+test -f mkimage || ln -s tools/mkimage .
+
 # This test script depends on external tools:
 # wget git truncate tar dpkg dd mcopy (from mtools) mkfs.ubifs (from mtd-utils) ubinize (from mtd-utils)
Lokesh Vutla April 21, 2020, 2:55 p.m. UTC | #2 
Tom,

On 14/04/20 4:10 PM, Pali Rohár wrote:
> On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
>> This patch contains a script which automatically download and compile all
>> needed tools to build a simple MTD images for booting Maemo kernel image by
>> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
>> machine provided by qemu-linaro project.
>>
>> It can be used to check that U-Boot for Nokia N900 is not broken and can be
>> successfully booted in emulator.
>>
>> Script is registered in to .travis.yml so it would be automatically run on
>> Travi CI service.
>>
>> Signed-off-by: Pali Rohár <pali@kernel.org>
> 
> Tom Rini, in past you have asked me for N900 Travis test. So could you
> please review this patch (including fixup at the bottom)?

Can you ack this patch?

Thanks and regards,
Lokesh
Simon Glass April 21, 2020, 5:36 p.m. UTC | #3 
Hi,

On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
>
> Tom,
>
> On 14/04/20 4:10 PM, Pali Rohár wrote:
> > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> >> This patch contains a script which automatically download and compile all
> >> needed tools to build a simple MTD images for booting Maemo kernel image by
> >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> >> machine provided by qemu-linaro project.
> >>
> >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> >> successfully booted in emulator.
> >>
> >> Script is registered in to .travis.yml so it would be automatically run on
> >> Travi CI service.
> >>
> >> Signed-off-by: Pali Rohár <pali@kernel.org>
> >
> > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > please review this patch (including fixup at the bottom)?
>
> Can you ack this patch?

Please use a pytest for this (test/py). We don't use shell scripts anymore.

Regards,
Simon
Tom Rini April 21, 2020, 8:12 p.m. UTC | #4 
On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> Hi,
> 
> On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> >
> > Tom,
> >
> > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > >> This patch contains a script which automatically download and compile all
> > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > >> machine provided by qemu-linaro project.
> > >>
> > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > >> successfully booted in emulator.
> > >>
> > >> Script is registered in to .travis.yml so it would be automatically run on
> > >> Travi CI service.
> > >>
> > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > >
> > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > please review this patch (including fixup at the bottom)?
> >
> > Can you ack this patch?
> 
> Please use a pytest for this (test/py). We don't use shell scripts anymore.

Well, this is where it's tricky and I've been debating with myself on
how to move forward here.

Part of the problem here is that much like a Pi, we could emulate this
board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
there's not a lot of these devices around to test with.  It's not a big
deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
has a Pi and other labs could add one fairly easy.  But adding an N900
to a lab is hard.

Looking over the script to do it, there's a lot of other stuff required
too, for it all to work.  Looking over the script again, there's enough
stuff going on that I wouldn't want it done in a persistent
image/container.

The only changes I would ask for I guess are that it should be put in
.travis.yml in the same areas other non-pytest tests, and put in similar
stanzas in .azure-ci.yml and .gitlab-ci.yml.
Simon Glass April 21, 2020, 8:37 p.m. UTC | #5 
Hi,

On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > Hi,
> >
> > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > >
> > > Tom,
> > >
> > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > >> This patch contains a script which automatically download and compile all
> > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > >> machine provided by qemu-linaro project.
> > > >>
> > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > >> successfully booted in emulator.
> > > >>
> > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > >> Travi CI service.
> > > >>
> > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > >
> > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > please review this patch (including fixup at the bottom)?
> > >
> > > Can you ack this patch?
> >
> > Please use a pytest for this (test/py). We don't use shell scripts anymore.
>
> Well, this is where it's tricky and I've been debating with myself on
> how to move forward here.
>
> Part of the problem here is that much like a Pi, we could emulate this
> board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> there's not a lot of these devices around to test with.  It's not a big
> deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> has a Pi and other labs could add one fairly easy.  But adding an N900
> to a lab is hard.
>
> Looking over the script to do it, there's a lot of other stuff required
> too, for it all to work.  Looking over the script again, there's enough
> stuff going on that I wouldn't want it done in a persistent
> image/container.
>
> The only changes I would ask for I guess are that it should be put in
> .travis.yml in the same areas other non-pytest tests, and put in similar
> stanzas in .azure-ci.yml and .gitlab-ci.yml.

For the existing stuff we use some sort of qemu that is built into the
image, so far as I understand it. Is that right?

Could we do something similar here? I actually don't like that though,
since there is so much setup needed to run things locally (without
docker).

Also, what is to stop me running this script on my machine?

Regards,
Simon
Tom Rini April 21, 2020, 8:46 p.m. UTC | #6 
On Tue, Apr 21, 2020 at 02:37:45PM -0600, Simon Glass wrote:
> Hi,
> 
> On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > Hi,
> > >
> > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > >
> > > > Tom,
> > > >
> > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > >> This patch contains a script which automatically download and compile all
> > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > >> machine provided by qemu-linaro project.
> > > > >>
> > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > >> successfully booted in emulator.
> > > > >>
> > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > >> Travi CI service.
> > > > >>
> > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > >
> > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > please review this patch (including fixup at the bottom)?
> > > >
> > > > Can you ack this patch?
> > >
> > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> >
> > Well, this is where it's tricky and I've been debating with myself on
> > how to move forward here.
> >
> > Part of the problem here is that much like a Pi, we could emulate this
> > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > there's not a lot of these devices around to test with.  It's not a big
> > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > has a Pi and other labs could add one fairly easy.  But adding an N900
> > to a lab is hard.
> >
> > Looking over the script to do it, there's a lot of other stuff required
> > too, for it all to work.  Looking over the script again, there's enough
> > stuff going on that I wouldn't want it done in a persistent
> > image/container.
> >
> > The only changes I would ask for I guess are that it should be put in
> > .travis.yml in the same areas other non-pytest tests, and put in similar
> > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> 
> For the existing stuff we use some sort of qemu that is built into the
> image, so far as I understand it. Is that right?

Right for GitLab/Azure, for Travis we checkout/build/install.

> Could we do something similar here? I actually don't like that though,
> since there is so much setup needed to run things locally (without
> docker).

That's what the script does.  The problem is that we need a specific
(seemingly dead-end, but I'd like to be told I'm wrong!) old commit.  So
we can't replace the QEMU we use for everyone with one that also
supports N900.

> Also, what is to stop me running this script on my machine?

Nothing.  And it does a good job of keeping all of the specific versions
of stuff it needs local to itself.
Simon Glass April 21, 2020, 8:49 p.m. UTC | #7 
Hi Tom,

On Tue, 21 Apr 2020 at 14:46, Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Apr 21, 2020 at 02:37:45PM -0600, Simon Glass wrote:
> > Hi,
> >
> > On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > > Hi,
> > > >
> > > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > > >
> > > > > Tom,
> > > > >
> > > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > > >> This patch contains a script which automatically download and compile all
> > > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > > >> machine provided by qemu-linaro project.
> > > > > >>
> > > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > > >> successfully booted in emulator.
> > > > > >>
> > > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > > >> Travi CI service.
> > > > > >>
> > > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > > >
> > > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > > please review this patch (including fixup at the bottom)?
> > > > >
> > > > > Can you ack this patch?
> > > >
> > > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> > >
> > > Well, this is where it's tricky and I've been debating with myself on
> > > how to move forward here.
> > >
> > > Part of the problem here is that much like a Pi, we could emulate this
> > > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > > there's not a lot of these devices around to test with.  It's not a big
> > > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > > has a Pi and other labs could add one fairly easy.  But adding an N900
> > > to a lab is hard.
> > >
> > > Looking over the script to do it, there's a lot of other stuff required
> > > too, for it all to work.  Looking over the script again, there's enough
> > > stuff going on that I wouldn't want it done in a persistent
> > > image/container.
> > >
> > > The only changes I would ask for I guess are that it should be put in
> > > .travis.yml in the same areas other non-pytest tests, and put in similar
> > > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> >
> > For the existing stuff we use some sort of qemu that is built into the
> > image, so far as I understand it. Is that right?
>
> Right for GitLab/Azure, for Travis we checkout/build/install.
>
> > Could we do something similar here? I actually don't like that though,
> > since there is so much setup needed to run things locally (without
> > docker).
>
> That's what the script does.  The problem is that we need a specific
> (seemingly dead-end, but I'd like to be told I'm wrong!) old commit.  So
> we can't replace the QEMU we use for everyone with one that also
> supports N900.

OK I see.

>
> > Also, what is to stop me running this script on my machine?
>
> Nothing.  And it does a good job of keeping all of the specific versions
> of stuff it needs local to itself.

OK, so you don't think we should add this as a pytest?

Regards,
Simon
Tom Rini April 21, 2020, 8:51 p.m. UTC | #8 
On Tue, Apr 21, 2020 at 02:49:05PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Tue, 21 Apr 2020 at 14:46, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Apr 21, 2020 at 02:37:45PM -0600, Simon Glass wrote:
> > > Hi,
> > >
> > > On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > > > Hi,
> > > > >
> > > > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > > > >
> > > > > > Tom,
> > > > > >
> > > > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > > > >> This patch contains a script which automatically download and compile all
> > > > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > > > >> machine provided by qemu-linaro project.
> > > > > > >>
> > > > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > > > >> successfully booted in emulator.
> > > > > > >>
> > > > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > > > >> Travi CI service.
> > > > > > >>
> > > > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > > > >
> > > > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > > > please review this patch (including fixup at the bottom)?
> > > > > >
> > > > > > Can you ack this patch?
> > > > >
> > > > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> > > >
> > > > Well, this is where it's tricky and I've been debating with myself on
> > > > how to move forward here.
> > > >
> > > > Part of the problem here is that much like a Pi, we could emulate this
> > > > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > > > there's not a lot of these devices around to test with.  It's not a big
> > > > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > > > has a Pi and other labs could add one fairly easy.  But adding an N900
> > > > to a lab is hard.
> > > >
> > > > Looking over the script to do it, there's a lot of other stuff required
> > > > too, for it all to work.  Looking over the script again, there's enough
> > > > stuff going on that I wouldn't want it done in a persistent
> > > > image/container.
> > > >
> > > > The only changes I would ask for I guess are that it should be put in
> > > > .travis.yml in the same areas other non-pytest tests, and put in similar
> > > > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> > >
> > > For the existing stuff we use some sort of qemu that is built into the
> > > image, so far as I understand it. Is that right?
> >
> > Right for GitLab/Azure, for Travis we checkout/build/install.
> >
> > > Could we do something similar here? I actually don't like that though,
> > > since there is so much setup needed to run things locally (without
> > > docker).
> >
> > That's what the script does.  The problem is that we need a specific
> > (seemingly dead-end, but I'd like to be told I'm wrong!) old commit.  So
> > we can't replace the QEMU we use for everyone with one that also
> > supports N900.
> 
> OK I see.
> 
> >
> > > Also, what is to stop me running this script on my machine?
> >
> > Nothing.  And it does a good job of keeping all of the specific versions
> > of stuff it needs local to itself.
> 
> OK, so you don't think we should add this as a pytest?

No, I don't think wrapping this as a pytest would be valuable.  Re-doing
the FS tests is a step towards being able to run them on at least
emulated HW too.
Pali Rohár April 21, 2020, 8:53 p.m. UTC | #9 
On Tuesday 21 April 2020 14:37:45 Simon Glass wrote:
> Hi,
> 
> On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > Hi,
> > >
> > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > >
> > > > Tom,
> > > >
> > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > >> This patch contains a script which automatically download and compile all
> > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > >> machine provided by qemu-linaro project.
> > > > >>
> > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > >> successfully booted in emulator.
> > > > >>
> > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > >> Travi CI service.
> > > > >>
> > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > >
> > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > please review this patch (including fixup at the bottom)?
> > > >
> > > > Can you ack this patch?
> > >
> > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> >
> > Well, this is where it's tricky and I've been debating with myself on
> > how to move forward here.
> >
> > Part of the problem here is that much like a Pi, we could emulate this
> > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > there's not a lot of these devices around to test with.  It's not a big
> > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > has a Pi and other labs could add one fairly easy.  But adding an N900
> > to a lab is hard.
> >
> > Looking over the script to do it, there's a lot of other stuff required
> > too, for it all to work.  Looking over the script again, there's enough
> > stuff going on that I wouldn't want it done in a persistent
> > image/container.
> >
> > The only changes I would ask for I guess are that it should be put in
> > .travis.yml in the same areas other non-pytest tests, and put in similar
> > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> 
> For the existing stuff we use some sort of qemu that is built into the
> image, so far as I understand it. Is that right?
> 
> Could we do something similar here? I actually don't like that though,
> since there is so much setup needed to run things locally (without
> docker).

That script run u-boot in n900 qemu machine and test that u-boot can
correctly boot Maemo kernel (from RAM, eMMC and OneNAND) with simple
rootfs. So it is full u-boot test for n900. And basically simulates
whole bootloader process for Maemo. In this case I'm sure that
everything is fine and I can replace new u-boot binary without
introducing any regressions. 

It downloads all needed stuff to construct images and filesystems.

> Also, what is to stop me running this script on my machine?

In script is "sudo mknod rootfs/dev/console c 5 1" call as in rootfs is
needed /dev/console character device. Otherwise everything is called
under normal user and all stuff is put into current directory.

And because on travis we can use 'sudo' I chosen this solution.

Alternative way to avoid usage of sudo, is to run whole script under
"fakeroot" utility. It use LD_PRELOAD with own library to fake mknod and
stat functions, so process can mknod (in memory) character device and
then mkfs.ubifs which reads (via stat) all files can properly put
character devices into rootfs image -- without any root privilege
escalation.

So if you want to run it on your machine, you needs be aware of that
sudo call.

Or if you want I can change script to use fakeroot utility (e.g Debian
is using it for building any DEB package) and then you can run it safely
under "nobody" user locally on your machine (if you do not like that
sudo call).

> Regards,
> Simon
Pali Rohár April 21, 2020, 9:03 p.m. UTC | #10 
On Tuesday 21 April 2020 16:46:36 Tom Rini wrote:
> That's what the script does.  The problem is that we need a specific
> (seemingly dead-end, but I'd like to be told I'm wrong!) old commit.

Not only specific commit, but specific fork! This is Linaro fork of qemu
(with support for OMAP3) which Linaro never upstreamed and fork is now
dead :-(

> So we can't replace the QEMU we use for everyone with one that also
> supports N900.

You really need that specific version of qemu to run it. Upstream qemu
has no support for N900 nor for OMAP3 HW.

Plus it needs proprietary Nokia's first stage bootloader (which starts
u-boot) with proprietary userspace tool which put this first stage
bootloader and configuration for it into MTD image. In MTD there is one
special partition with configuration data for this first stage
bootloader and that proprietary tool fill it. Tool and first stage
bootloader is under license which allows non-commercial redistribution

Why we need that first stage bootloader? Because on real N900 HW it is
signed and therefore cannot be replaced. Qemu version of that first
stage bootloader is obviously not signed.
Pali Rohár April 21, 2020, 9:34 p.m. UTC | #11 
On Tuesday 21 April 2020 16:51:23 Tom Rini wrote:
> On Tue, Apr 21, 2020 at 02:49:05PM -0600, Simon Glass wrote:
> > Hi Tom,
> > 
> > On Tue, 21 Apr 2020 at 14:46, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Apr 21, 2020 at 02:37:45PM -0600, Simon Glass wrote:
> > > > Hi,
> > > >
> > > > On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > > > > Hi,
> > > > > >
> > > > > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > > > > >
> > > > > > > Tom,
> > > > > > >
> > > > > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > > > > >> This patch contains a script which automatically download and compile all
> > > > > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > > > > >> machine provided by qemu-linaro project.
> > > > > > > >>
> > > > > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > > > > >> successfully booted in emulator.
> > > > > > > >>
> > > > > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > > > > >> Travi CI service.
> > > > > > > >>
> > > > > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > > > > >
> > > > > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > > > > please review this patch (including fixup at the bottom)?
> > > > > > >
> > > > > > > Can you ack this patch?
> > > > > >
> > > > > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> > > > >
> > > > > Well, this is where it's tricky and I've been debating with myself on
> > > > > how to move forward here.
> > > > >
> > > > > Part of the problem here is that much like a Pi, we could emulate this
> > > > > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > > > > there's not a lot of these devices around to test with.  It's not a big
> > > > > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > > > > has a Pi and other labs could add one fairly easy.  But adding an N900
> > > > > to a lab is hard.
> > > > >
> > > > > Looking over the script to do it, there's a lot of other stuff required
> > > > > too, for it all to work.  Looking over the script again, there's enough
> > > > > stuff going on that I wouldn't want it done in a persistent
> > > > > image/container.
> > > > >
> > > > > The only changes I would ask for I guess are that it should be put in
> > > > > .travis.yml in the same areas other non-pytest tests, and put in similar
> > > > > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> > > >
> > > > For the existing stuff we use some sort of qemu that is built into the
> > > > image, so far as I understand it. Is that right?
> > >
> > > Right for GitLab/Azure, for Travis we checkout/build/install.
> > >
> > > > Could we do something similar here? I actually don't like that though,
> > > > since there is so much setup needed to run things locally (without
> > > > docker).
> > >
> > > That's what the script does.  The problem is that we need a specific
> > > (seemingly dead-end, but I'd like to be told I'm wrong!) old commit.  So
> > > we can't replace the QEMU we use for everyone with one that also
> > > supports N900.
> > 
> > OK I see.
> > 
> > >
> > > > Also, what is to stop me running this script on my machine?
> > >
> > > Nothing.  And it does a good job of keeping all of the specific versions
> > > of stuff it needs local to itself.
> > 
> > OK, so you don't think we should add this as a pytest?
> 
> No, I don't think wrapping this as a pytest would be valuable.  Re-doing
> the FS tests is a step towards being able to run them on at least
> emulated HW too.

Ok, so is something needed to do with this patch?
Tom Rini April 21, 2020, 11:24 p.m. UTC | #12 
On Tue, Apr 21, 2020 at 11:34:02PM +0200, Pali Rohár wrote:
> On Tuesday 21 April 2020 16:51:23 Tom Rini wrote:
> > On Tue, Apr 21, 2020 at 02:49:05PM -0600, Simon Glass wrote:
> > > Hi Tom,
> > > 
> > > On Tue, 21 Apr 2020 at 14:46, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Apr 21, 2020 at 02:37:45PM -0600, Simon Glass wrote:
> > > > > Hi,
> > > > >
> > > > > On Tue, 21 Apr 2020 at 14:12, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Tue, Apr 21, 2020 at 11:36:37AM -0600, Simon Glass wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > On Tue, 21 Apr 2020 at 08:56, Lokesh Vutla <lokeshvutla@ti.com> wrote:
> > > > > > > >
> > > > > > > > Tom,
> > > > > > > >
> > > > > > > > On 14/04/20 4:10 PM, Pali Rohár wrote:
> > > > > > > > > On Wednesday 01 April 2020 00:35:18 Pali Rohár wrote:
> > > > > > > > >> This patch contains a script which automatically download and compile all
> > > > > > > > >> needed tools to build a simple MTD images for booting Maemo kernel image by
> > > > > > > > >> U-Boot from RAM, eMMC and OneNAND. MTD images are then run in virtual n900
> > > > > > > > >> machine provided by qemu-linaro project.
> > > > > > > > >>
> > > > > > > > >> It can be used to check that U-Boot for Nokia N900 is not broken and can be
> > > > > > > > >> successfully booted in emulator.
> > > > > > > > >>
> > > > > > > > >> Script is registered in to .travis.yml so it would be automatically run on
> > > > > > > > >> Travi CI service.
> > > > > > > > >>
> > > > > > > > >> Signed-off-by: Pali Rohár <pali@kernel.org>
> > > > > > > > >
> > > > > > > > > Tom Rini, in past you have asked me for N900 Travis test. So could you
> > > > > > > > > please review this patch (including fixup at the bottom)?
> > > > > > > >
> > > > > > > > Can you ack this patch?
> > > > > > >
> > > > > > > Please use a pytest for this (test/py). We don't use shell scripts anymore.
> > > > > >
> > > > > > Well, this is where it's tricky and I've been debating with myself on
> > > > > > how to move forward here.
> > > > > >
> > > > > > Part of the problem here is that much like a Pi, we could emulate this
> > > > > > board in QEMU but would need not-upstream-QEMU to do it.  But unlike Pi,
> > > > > > there's not a lot of these devices around to test with.  It's not a big
> > > > > > deal that Pi isn't tested by CI via QEMU, my lab as a Pi, Simon's lab
> > > > > > has a Pi and other labs could add one fairly easy.  But adding an N900
> > > > > > to a lab is hard.
> > > > > >
> > > > > > Looking over the script to do it, there's a lot of other stuff required
> > > > > > too, for it all to work.  Looking over the script again, there's enough
> > > > > > stuff going on that I wouldn't want it done in a persistent
> > > > > > image/container.
> > > > > >
> > > > > > The only changes I would ask for I guess are that it should be put in
> > > > > > .travis.yml in the same areas other non-pytest tests, and put in similar
> > > > > > stanzas in .azure-ci.yml and .gitlab-ci.yml.
> > > > >
> > > > > For the existing stuff we use some sort of qemu that is built into the
> > > > > image, so far as I understand it. Is that right?
> > > >
> > > > Right for GitLab/Azure, for Travis we checkout/build/install.
> > > >
> > > > > Could we do something similar here? I actually don't like that though,
> > > > > since there is so much setup needed to run things locally (without
> > > > > docker).
> > > >
> > > > That's what the script does.  The problem is that we need a specific
> > > > (seemingly dead-end, but I'd like to be told I'm wrong!) old commit.  So
> > > > we can't replace the QEMU we use for everyone with one that also
> > > > supports N900.
> > > 
> > > OK I see.
> > > 
> > > >
> > > > > Also, what is to stop me running this script on my machine?
> > > >
> > > > Nothing.  And it does a good job of keeping all of the specific versions
> > > > of stuff it needs local to itself.
> > > 
> > > OK, so you don't think we should add this as a pytest?
> > 
> > No, I don't think wrapping this as a pytest would be valuable.  Re-doing
> > the FS tests is a step towards being able to run them on at least
> > emulated HW too.
> 
> Ok, so is something needed to do with this patch?

Yes, re-order where the .travis.yml hunk is and add a similar part to
.gitlab-ci.yml and .azure-ci.yml.
Pali Rohár April 23, 2020, 7:34 a.m. UTC | #13 
On Tuesday 21 April 2020 19:24:57 Tom Rini wrote:
> On Tue, Apr 21, 2020 at 11:34:02PM +0200, Pali Rohár wrote:
> > Ok, so is something needed to do with this patch?
> 
> Yes, re-order where the .travis.yml hunk is and add a similar part to
> .gitlab-ci.yml and .azure-ci.yml.

Hello Tom! I have looked at .travis.yml again, but I do not understand
what do you mean by re-order. I really have no idea where you want to
move that rx51_test.sh hunk. Could you describe it?
Tom Rini April 23, 2020, 12:24 p.m. UTC | #14 
On Thu, Apr 23, 2020 at 09:34:26AM +0200, Pali Rohár wrote:
> On Tuesday 21 April 2020 19:24:57 Tom Rini wrote:
> > On Tue, Apr 21, 2020 at 11:34:02PM +0200, Pali Rohár wrote:
> > > Ok, so is something needed to do with this patch?
> > 
> > Yes, re-order where the .travis.yml hunk is and add a similar part to
> > .gitlab-ci.yml and .azure-ci.yml.
> 
> Hello Tom! I have looked at .travis.yml again, but I do not understand
> what do you mean by re-order. I really have no idea where you want to
> move that rx51_test.sh hunk. Could you describe it?

This should go around where we check tools-only/envtools.  Thanks!
Pali Rohár April 23, 2020, 5:48 p.m. UTC | #15 
On Thursday 23 April 2020 08:24:05 Tom Rini wrote:
> On Thu, Apr 23, 2020 at 09:34:26AM +0200, Pali Rohár wrote:
> > On Tuesday 21 April 2020 19:24:57 Tom Rini wrote:
> > > On Tue, Apr 21, 2020 at 11:34:02PM +0200, Pali Rohár wrote:
> > > > Ok, so is something needed to do with this patch?
> > > 
> > > Yes, re-order where the .travis.yml hunk is and add a similar part to
> > > .gitlab-ci.yml and .azure-ci.yml.
> > 
> > Hello Tom! I have looked at .travis.yml again, but I do not understand
> > what do you mean by re-order. I really have no idea where you want to
> > move that rx51_test.sh hunk. Could you describe it?
> 
> This should go around where we check tools-only/envtools.  Thanks!

Thank you, now I figured out. I will send V2 of this one patch.
diff mbox series

Patch

diff --git a/.travis.yml b/.travis.yml
index c59bd7790b..d96811473c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -40,6 +40,8 @@  addons:
     - clang-7
     - srecord
     - graphviz
+    - mtools
+    - mtd-utils
 
 install:
  # Clone uboot-test-hooks
@@ -116,6 +118,11 @@  script:
  # Comments must be outside the command strings below, or the Travis parser
  # will get confused.
  #
+ # Run tests for Nokia RX-51 (aka N900)
+ - if [[ -n "${NOKIA_RX51}" ]]; then
+     test/rx51_test.sh
+     exit $?;
+   fi
  # From buildman, exit code 129 means warnings only.  If we've been asked to
  # use clang only do one configuration.
  - if [[ "${BUILDMAN}" != "" ]]; then
@@ -160,6 +167,9 @@  matrix:
   include:
   # we need to build by vendor due to 50min time limit for builds
   # each env setting here is a dedicated build
+    - name: "nokia rx51"
+      env:
+        - NOKIA_RX51=1
     - name: "buildman arc"
       env:
         - BUILDMAN="arc"
diff --git a/test/rx51_test.sh b/test/rx51_test.sh
new file mode 100755
index 0000000000..43ecc07c08
--- /dev/null
+++ b/test/rx51_test.sh
@@ -0,0 +1,208 @@ 
+#!/bin/sh -e
+# SPDX-License-Identifier: GPL-2.0+
+# (C) 2020 Pali Rohár <pali@kernel.org>
+
+# This test script depends on external tools:
+# wget git truncate tar dpkg dd mcopy (from mtools) mkfs.ubifs (from mtd-utils) ubinize (from mtd-utils)
+
+# Download and compile linaro version qemu which has support for n900 machine
+# Last working commit is 8f8d8e0796efe1a6f34cdd83fb798f3c41217ec1
+if ! test -f qemu-system-arm; then
+	test -d qemu-linaro || git clone https://git.linaro.org/qemu/qemu-linaro.git
+	cd qemu-linaro
+	git checkout 8f8d8e0796efe1a6f34cdd83fb798f3c41217ec1
+	./configure --enable-system --target-list=arm-softmmu --disable-sdl --disable-gtk --disable-curses --audio-drv-list=pa,alsa --audio-card-list= --disable-werror --disable-xen --disable-xen-pci-passthrough --disable-brlapi --disable-vnc --disable-curl --disable-slirp --disable-kvm --disable-user --disable-linux-user --disable-bsd-user --disable-guest-base --disable-uuid --disable-vde --disable-linux-aio --disable-cap-ng --disable-attr --disable-blobs --disable-docs --disable-spice --disable-libiscsi --disable-smartcard-nss --disable-usb-redir --disable-guest-agent --disable-seccomp --disable-glusterfs --disable-nptl --disable-fdt
+	make -j4
+	cd ..
+	ln -s qemu-linaro/arm-softmmu/qemu-system-arm .
+fi
+
+# Download and compile dosfstools with mbr support
+# Currently this support is in open pull request 95
+if ! test -f mkfs.fat; then
+	test -d dosfstools || git clone https://github.com/dosfstools/dosfstools.git
+	cd dosfstools
+	git fetch origin refs/pull/95/head
+	git checkout FETCH_HEAD
+	./autogen.sh
+	./configure
+	make -j4
+	cd ..
+	ln -s dosfstools/src/mkfs.fat .
+fi
+
+# Download qflasher and nolo images
+# This is proprietary qemu flasher tool with first stage images, but license allows non-commercial redistribution
+wget -c http://repository.maemo.org/qemu-n900/qemu-n900.tar.gz
+tar -xf qemu-n900.tar.gz
+
+# Download Maemo script u-boot-gen-combined
+if ! test -f u-boot-gen-combined; then
+	test -d u-boot-maemo || git clone https://github.com/pali/u-boot-maemo.git
+	chmod +x u-boot-maemo/debian/u-boot-gen-combined
+	ln -s u-boot-maemo/debian/u-boot-gen-combined .
+fi
+
+# Download Maemo fiasco kernel
+wget -c http://repository.maemo.org/pool/maemo5.0/free/k/kernel/kernel_2.6.28-20103103+0m5_armel.deb
+dpkg -x kernel_2.6.28-20103103+0m5_armel.deb kernel_2.6.28
+
+# Download Maemo libc
+wget -c http://repository.maemo.org/pool/maemo5.0/free/g/glibc/libc6_2.5.1-1eglibc27+0m5_armel.deb
+dpkg -x libc6_2.5.1-1eglibc27+0m5_armel.deb libc6_2.5.1
+
+# Download Maemo busybox
+wget -c http://repository.maemo.org/pool/maemo5.0/free/b/busybox/busybox_1.10.2.legal-1osso30+0m5_armel.deb
+dpkg -x busybox_1.10.2.legal-1osso30+0m5_armel.deb busybox_1.10.2
+
+# Generate rootfs directory
+# WARNING: there is "sudo mknod" call for /dev/console
+mkdir -p rootfs
+mkdir -p rootfs/dev/
+mkdir -p rootfs/bin/
+mkdir -p rootfs/sbin/
+mkdir -p rootfs/lib/
+test -c rootfs/dev/console || sudo mknod rootfs/dev/console c 5 1
+cp -a busybox_1.10.2/bin/busybox rootfs/bin/
+cp -a libc6_2.5.1/lib/ld-linux.so.3 rootfs/lib/
+cp -a libc6_2.5.1/lib/ld-2.5.so rootfs/lib/
+cp -a libc6_2.5.1/lib/libc.so.6 rootfs/lib/
+cp -a libc6_2.5.1/lib/libc-2.5.so rootfs/lib/
+cp -a libc6_2.5.1/lib/libcrypt.so.1 rootfs/lib/
+cp -a libc6_2.5.1/lib/libcrypt-2.5.so rootfs/lib/
+test -f rootfs/bin/sh || ln -sf busybox rootfs/bin/sh
+test -f rootfs/sbin/poweroff || ln -sf ../bin/busybox rootfs/sbin/poweroff
+cat > rootfs/sbin/preinit << EOF
+#!/bin/sh
+echo
+echo "Successfully booted"
+echo
+/sbin/poweroff -f
+EOF
+chmod +x rootfs/sbin/preinit
+
+# Generate bootmenu for eMMC booting
+cat > bootmenu_emmc << EOF
+setenv bootmenu_0 'uImage-2.6.28-omap1 from eMMC=setenv mmcnum 1; setenv mmcpart 1; setenv mmctype fat; setenv bootargs; setenv setup_omap_atag 1; setenv mmckernfile uImage-2.6.28-omap1; run trymmckernboot';
+setenv bootmenu_1;
+setenv bootmenu_delay 1;
+setenv bootdelay 1;
+EOF
+./mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n bootmenu -d bootmenu_emmc bootmenu_emmc.scr
+
+# Generate bootmenu for OneNAND booting
+# FIXME: is address, size and offset really correct?
+cat > bootmenu_nand << EOF
+setenv bootmenu_0 'uImage-2.6.28-omap1 from OneNAND=mtd read initfs \${kernaddr} 0x800 0x1FF800; setenv bootargs; setenv setup_omap_atag 1; bootm \${kernaddr}';
+setenv bootmenu_1;
+setenv bootmenu_delay 1;
+setenv bootdelay 1;
+EOF
+./mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n bootmenu -d bootmenu_nand bootmenu_nand.scr
+
+# Generate ubi config file for ubi rootfs image
+cat > ubi.ini << EOF
+[rootfs]
+mode=ubi
+image=ubifs.img
+vol_id=0
+vol_size=160MiB
+vol_type=dynamic
+vol_name=rootfs
+vol_alignment=1
+vol_flags=autoresize
+EOF
+
+# Generate combined image from u-boot and Maemo fiasco kernel
+dd if=kernel_2.6.28/boot/zImage-2.6.28-20103103+0m5.fiasco of=zImage-2.6.28-omap1 skip=95 bs=1
+./mkimage -A arm -O linux -T kernel -C none -a 80008000 -e 80008000 -n zImage-2.6.28-omap1 -d zImage-2.6.28-omap1 uImage-2.6.28-omap1
+./u-boot-gen-combined u-boot.bin uImage-2.6.28-omap1 combined.bin
+
+# Generate combined hack image from u-boot and Maemo fiasco kernel (kernel starts at 2MB offset)
+cp u-boot.bin combined_hack.bin
+dd if=uImage-2.6.28-omap1 of=combined_hack.bin bs=1024 seek=2048
+
+# Generate ubi rootfs image from rootfs directory
+/usr/sbin/mkfs.ubifs -m 2048 -e 129024 -c 2047 -r rootfs ubifs.img
+/usr/sbin/ubinize -o ubi.img -p 128KiB -m 2048 -s 512 ubi.ini
+
+# Generate FAT32 eMMC image for eMMC booting
+truncate -s 50MiB emmc_emmc.img
+./mkfs.fat --mbr -F32 emmc_emmc.img
+mcopy uImage-2.6.28-omap1 ::/uImage-2.6.28-omap1 -i emmc_emmc.img
+mcopy bootmenu_emmc.scr ::/bootmenu.scr -i emmc_emmc.img
+
+# Generate FAT32 eMMC image for OneNAND booting
+truncate -s 50MiB emmc_nand.img
+./mkfs.fat --mbr -F32 emmc_nand.img
+mcopy bootmenu_nand.scr ::/bootmenu.scr -i emmc_nand.img
+
+# Generate MTD image for RAM booting from bootloader nolo images, compiled image and rootfs image
+rm -f mtd_ram.img
+./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k combined.bin -r ubi.img -m rx51 -o mtd_ram.img
+
+# Generate MTD image for eMMC booting from bootloader nolo images, u-boot image and rootfs image
+rm -f mtd_emmc.img
+./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k u-boot.bin -r ubi.img -m rx51 -o mtd_emmc.img
+
+# Generate MTD image for OneNAND booting from bootloader nolo images, combined hacked image and rootfs image
+# Kernel image is put into initfs area, but qflasher reject to copy kernel image into initfs area because it does not have initfs signature
+# This is hack to workaround this problem, tell qflasher that kernel area for u-boot is bigger and put big combined hacked image (u-boot + kernel with correct offset)
+rm -f mtd_nand.img
+./qflasher -v -x xloader-qemu.bin -s secondary-qemu.bin -k combined_hack.bin -r ubi.img -m rx51 -p k=4094,i=2 -o mtd_nand.img
+
+echo
+echo
+echo "All images were successfully generated"
+echo "Now going to run them in qemu"
+echo
+echo
+
+# Run MTD image in qemu and wait for 300s if kernel from RAM is correctly booted
+rm -f qemu_ram.log
+./qemu-system-arm -M n900 -mtdblock mtd_ram.img -serial /dev/stdout -display none > qemu_ram.log &
+qemu_pid=$!
+tail -F qemu_ram.log &
+tail_pid=$!
+{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
+sleep_pid=$!
+wait $qemu_pid || true
+kill -9 $tail_pid $sleep_pid 2>/dev/null || true
+
+# Run MTD image in qemu and wait for 300s if kernel from eMMC is correctly booted
+rm -f qemu_emmc.log
+./qemu-system-arm -M n900 -mtdblock mtd_emmc.img -sd emmc_emmc.img -serial /dev/stdout -display none > qemu_emmc.log &
+qemu_pid=$!
+tail -F qemu_emmc.log &
+tail_pid=$!
+{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
+sleep_pid=$!
+wait $qemu_pid || true
+kill -9 $tail_pid $sleep_pid 2>/dev/null || true
+
+# Run MTD image in qemu and wait for 300s if kernel from OneNAND is correctly booted
+rm -f qemu_nand.log
+./qemu-system-arm -M n900 -mtdblock mtd_nand.img -sd emmc_nand.img -serial /dev/stdout -display none > qemu_nand.log &
+qemu_pid=$!
+tail -F qemu_nand.log &
+tail_pid=$!
+{ sleep 300 || true; kill -9 $qemu_pid $tail_pid 2>/dev/null || true; } &
+sleep_pid=$!
+wait $qemu_pid || true
+kill -9 $tail_pid $sleep_pid 2>/dev/null || true
+
+echo
+echo
+if grep -q 'Successfully booted' qemu_ram.log; then echo "Kernel was successfully booted from RAM"; else echo "Failed to boot kernel from RAM"; fi
+if grep -q 'Successfully booted' qemu_emmc.log; then echo "Kernel was successfully booted from eMMC"; else echo "Failed to boot kernel from eMMC"; fi
+if grep -q 'Successfully booted' qemu_nand.log; then echo "Kernel was successfully booted from OneNAND"; else echo "Failed to boot kernel from OneNAND"; fi
+echo
+echo
+
+if grep -q 'Successfully booted' qemu_ram.log && grep -q 'Successfully booted' qemu_emmc.log && grep -q 'Successfully booted' qemu_nand.log; then
+	echo "All tests passed"
+	exit 0
+else
+	echo "Some tests failed"
+	exit 1
+fi