[v3,for-next,10/11] support/scripts/boot-qemu-image.py: boot Qemu images with Qemu-system.
diff mbox series

Message ID 20191117201909.12900-11-romain.naour@smile.fr
State New
Headers show
  • gitlab Qemu runtime testing
Related show

Commit Message

Romain Naour Nov. 17, 2019, 8:19 p.m. UTC
From: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>

This script is intended to be used by gitlab CI to test
at runtime Qemu images generated by Buildroot's Qemu defconfigs.

This allows to troubleshoot different issues that may be
associated with defective builds by lanching a qemu machine,
sending root password, waiting for login shell and then perform
a shutdown.

This script is inspired by toolchain builder [1] and the Buildroot
testing infrastructure.

Usage: boot-qemu-image.py <qemu_arch_defconfig>

The gitlab CI will call this script for each defconfig build but
only Qemu defconfig will be runtime tested, all others defconfig are
ignored. The script retrieve the patch to the launch.sh script that
contain the Qemu command ligne. The launch.sh script is no executed
directly since we need to tweak some Qemu options (remove serial port
redirection, disable graphic support...).

The script check is the corresponding Qemu binary (qemu-system) is
available, either built by Buildroot or available from the host.
Some Qemu defconfig must be used with a specific Qemu version (fork)
that is not always available. The script doesn't error out when
qemu-system is missing but a message is added in a log.

Finally, the script start Qemu like it's done for the Buildroot
testing infrastructure (using pexpect).

We noticed some timeout issues with pexpect when the Qemu machine is
powered off. That's because Qemu process doesn't stop even if the
system is halted (after "System halted"). So the script doesn't error
out when such timeout occure. The behaviour depends on the architecture
emulated by Qemu.

[1] https://github.com/bootlin/toolchains-builder/blob/master/build.sh

Signed-off-by: Jugurtha BELKALEM <jugurtha.belkalem@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
 support/scripts/boot-qemu-image.py | 109 +++++++++++++++++++++++++++++
 1 file changed, 109 insertions(+)
 create mode 100755 support/scripts/boot-qemu-image.py

diff mbox series

diff --git a/support/scripts/boot-qemu-image.py b/support/scripts/boot-qemu-image.py
new file mode 100755
index 0000000000..14cb3b02d0
--- /dev/null
+++ b/support/scripts/boot-qemu-image.py
@@ -0,0 +1,109 @@ 
+#!/usr/bin/env python3
+import pexpect
+import sys
+import os
+import re
+import shlex
+import shutil
+import subprocess
+argc = len(sys.argv)
+if not (argc == 2):
+    print("Error: incorrect number of arguments")
+    print("""Usage: boot-qemu-image.py <qemu_arch_defconfig>""")
+    sys.exit(1)
+defconfig_filename = sys.argv[1]
+# Ignore non Qemu defconfig
+if defconfig_filename.startswith('qemu_') is False:
+    sys.exit(0)
+# This script expect to run from the Buildroot top directory.
+defconfig_filepath = os.path.join(os.getcwd(), 'configs', defconfig_filename)
+with open(defconfig_filepath, 'r') as defconfig_file:
+    # read only the first line and keep the path to launch.sh script.
+    launch_filepath = defconfig_file.readline().split(":")[1].strip()
+qemu_cmd = ""
+with open(launch_filepath, 'r') as launch_file:
+    for line in launch_file:
+        if re.search("qemu-system", line):
+            qemu_cmd = line
+            break
+if not qemu_cmd:
+    print("Error: No QEMU command line found in " + defconfig_filename)
+    sys.exit(1)
+# Replace bashism
+qemu_cmd = line.replace("${1:-output}", "output")
+# Remove -serial stdio if present
+qemu_cmd = qemu_cmd.replace("-serial stdio", "")
+# Special case for SH4 -display none
+if defconfig_filename.startswith('qemu_sh4') is True:
+    qemu_cmd += "-serial stdio -display none"
+    qemu_cmd += "-nographic"
+# pexpect needs a list, convert a sting to a list and escape quoted substring.
+qemu_cmd = shlex.split(qemu_cmd)
+# Use host Qemu if provided by Buildroot.
+os.environ["PATH"] = os.getcwd() + "/output/host/bin" + os.pathsep + os.environ["PATH"]
+# Ignore when no Qemu emulation is available
+if not shutil.which(qemu_cmd[0]):
+    print("No " + qemu_cmd[0] + " binary available, THIS DEFCONFIG CAN NOT BE TESTED!")
+    sys.exit(0)
+def main():
+    global child
+    try:
+        child.expect(["buildroot login:", pexpect.TIMEOUT], timeout=60)
+    except pexpect.EOF:
+        print("Connection problem, exiting.")
+        sys.exit(1)
+    except pexpect.TIMEOUT:
+        print("System did not boot in time, exiting.")
+        sys.exit(1)
+    child.sendline("root\r")
+    try:
+        child.expect(["# ", pexpect.TIMEOUT], timeout=60)
+    except pexpect.EOF:
+        print("Cannot connect to shell")
+        sys.exit(1)
+    except pexpect.TIMEOUT:
+        print("Timeout while waiting for shell")
+        sys.exit(1)
+    child.sendline("poweroff\r")
+    try:
+        child.expect(["System halted", pexpect.TIMEOUT], timeout=60)
+        child.expect(pexpect.EOF)
+    except pexpect.EOF:
+        sys.exit(0)
+    except pexpect.TIMEOUT:
+        print("Cannot halt machine")
+        # Qemu may not exit properly after "System halted", ignore.
+        sys.exit(0)
+# Log the Qemu version
+subprocess.call([qemu_cmd[0], "--version"])
+child = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:], timeout=5, encoding='utf-8',
+                      env={"QEMU_AUDIO_DRV": "none", 'PATH': os.environ["PATH"]})
+# We want only stdout into the log to avoid double echo
+child.logfile = sys.stdout