From patchwork Sun Jun 9 21:33:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Naour X-Patchwork-Id: 1112759 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=smile.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=smile-fr.20150623.gappssmtp.com header.i=@smile-fr.20150623.gappssmtp.com header.b="arEgEQoT"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45MTz25RLxz9sCJ for ; Mon, 10 Jun 2019 07:33:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id C73028577F; Sun, 9 Jun 2019 21:33:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JBnlOGe5YrhG; Sun, 9 Jun 2019 21:33:26 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id AAA188574A; Sun, 9 Jun 2019 21:33:26 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 0CCEC1BF487 for ; Sun, 9 Jun 2019 21:33:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 026C2853C5 for ; Sun, 9 Jun 2019 21:33:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tL4Q0zHfwVNy for ; Sun, 9 Jun 2019 21:33:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by whitealder.osuosl.org (Postfix) with ESMTPS id 50640811DD for ; Sun, 9 Jun 2019 21:33:23 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id r18so7153641wrm.10 for ; Sun, 09 Jun 2019 14:33:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=pU77gzGhu3J83NwlUYjFE+e5GVbB1FLXe9OU9r1Gj7g=; b=arEgEQoT1/4wV9vU9RAGVwXk+AOLFwbc7wWOngaIAvtYtg3ZOdMt1vBJoSeGhvCvnI xJP0OoF3yb6Q7E9Ihu4gqjYbK+qtPbfHzWnLEFLN6xuAwIuq4TPummZOMwvrmhD9jai/ iroYc7IiDJ/lRw6/P+uRPxUw0ySNeB+C8ROQl5WXpdXB1lkeA3S/3gVGUjmim15m6CDt XyEX+jkGg/dTgoHuZJWMdDui0SnJSqQVI0hcylXIF1hcjWZH9O8FHbQNS/OfNLRb7Evl NQCJspvkRZdJoWfbMc/Uk/O30ry+YKztKM3loAt/79dKTQjapM22Z/x+9l9KoGIqR7+w S6dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=pU77gzGhu3J83NwlUYjFE+e5GVbB1FLXe9OU9r1Gj7g=; b=e1HNBR17pY1r0AEJ1VuvJ/dmu4qS909QfffVboSOvAV2H23JkviXMhs/Z+UT2NsuyF wkPcRipJVs7rRvorr0ulTutDj9hFDX3dtwBRLCxvONzqQHchwiBoQ57WDN+179oneU9J 5YTHmPKRJmv58tWpO2t1rkt0uM6ipdK1Ln9daQtqO1WkXXMO0877qQBcd+RwGSbFCob1 yqaV9KVAyaa5Q3gccyAKXwV5+0cHhItt9NmV8rQnolHvz6WtMa1Vd5jGEbjAmro9eOlO 2hE5tGFd1RKdWLVHSvad02QrHuv00bWeWxjpZYgm4XLmSMqwfSfEpJcA+7MF0F+w6fcE Rw3Q== X-Gm-Message-State: APjAAAWumsfzr3Kbvx6zieTL6e5E6x7YYOcnnLcfEtYAtl2pYO6GNf3U H1jw6EZ1dEPK6gVJgCRfBd8DT4g1kfU= X-Google-Smtp-Source: APXvYqxpzTJqv0MA5qZi2Sa0Cf0ywQYWJLVU3YWHQtRmyRCGna6mqml+ihph+RR+gqZmszeH7vrOxg== X-Received: by 2002:a5d:52cc:: with SMTP id r12mr3186641wrv.132.1560116001735; Sun, 09 Jun 2019 14:33:21 -0700 (PDT) Received: from localhost.localdomain (aorleans-653-1-154-219.w92-160.abo.wanadoo.fr. [92.160.113.219]) by smtp.gmail.com with ESMTPSA id v67sm8865502wme.24.2019.06.09.14.33.20 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 09 Jun 2019 14:33:21 -0700 (PDT) From: Romain Naour To: buildroot@buildroot.org Date: Sun, 9 Jun 2019 23:33:17 +0200 Message-Id: <20190609213317.16444-1-romain.naour@smile.fr> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [Buildroot] [PATCH] support/testing: add python-numpy tests X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour , Asaf Kahlon Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This commit add 4 runtime tests for python-numpy. The current state of python-numpy is not good since several runtime issue has been discovered while working on new python packages that are based on it. First, we have a runtime issue while testing python-numpy with clapack option enabled: (glibc toolchain) python code: import numpy as np list = [1,2,3,4] arr = np.array(list) print arr python Error: ImportError: /usr/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-arm-linux-gnueabi.so: undefined symbol: cblas_sgemm Since python-numpy is not covered by the Buildroot testing infrastructure, two runtime test are initially added for python-numpy for python2 and python3. But the default system use a uClibc-ng toolchain and there is a new runtime issue: python Error: code object from '/usr/lib/python3.7/site-packages/numpy/core/overrides.pyc' /usr/bin/python3.7: symbol 'npy_clear_floatstatus_barrier': can't resolve symbol /usr/bin/python3.7: symbol 'npy_get_floatstatus_barrier': can't resolve symbol This issue seems to be only present on uClibc-ng system (tested on arm and aarch64 with glibc or musl). This may due to some feature depending on __GLIBC__ and glibc >= 2.18. So, two additional test based on Glibc are added. The two first test based on uClibc-ng fail with the same error above. The third test is working since it's based on Glibc toolchain with python-numpy without clapack support. The last test fail since the python-numpy's clapack support is enabled. Signed-off-by: Romain Naour Cc: Asaf Kahlon Cc: Samuel Martin Cc: Yegor Yefremov --- .../tests/package/sample_python_numpy.py | 5 + .../tests/package/test_python_numpy.py | 112 ++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100644 support/testing/tests/package/sample_python_numpy.py create mode 100644 support/testing/tests/package/test_python_numpy.py diff --git a/support/testing/tests/package/sample_python_numpy.py b/support/testing/tests/package/sample_python_numpy.py new file mode 100644 index 0000000000..8e8d36ea4c --- /dev/null +++ b/support/testing/tests/package/sample_python_numpy.py @@ -0,0 +1,5 @@ +# From https://www.tutorialspoint.com/scipy/scipy_basic_functionality.htm +import numpy as np +list = [1,2,3,4] +arr = np.array(list) +print (arr) diff --git a/support/testing/tests/package/test_python_numpy.py b/support/testing/tests/package/test_python_numpy.py new file mode 100644 index 0000000000..64d59cc702 --- /dev/null +++ b/support/testing/tests/package/test_python_numpy.py @@ -0,0 +1,112 @@ +import os + +from tests.package.test_python import TestPythonPackageBase + +# Test with uClibc-ng +class TestPythonPy2uClibcNumpy(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ + """ + BR2_PACKAGE_PYTHON=y + BR2_PACKAGE_PYTHON_INCREMENTAL=y + """ + sample_scripts = ["tests/package/sample_python_numpy.py"] + timeout = 30 + +# Test with uClibc-ng +class TestPythonPy3uClibcNumpy(TestPythonPackageBase): + __test__ = True + config = TestPythonPackageBase.config + \ + """ + BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON_NUMPY=y + """ + sample_scripts = ["tests/package/sample_python_numpy.py"] + timeout = 30 + +class TestPythonNumpy(TestPythonPackageBase): + sample_scripts = ["tests/package/sample_python_numpy.py"] + timeout = 30 + + def login(self): + cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio") + kern = os.path.join(self.builddir, "images", "zImage") + dtb = os.path.join(self.builddir, "images", "versatile-pb.dtb") + + self.emulator.boot(arch="armv5", + kernel=kern, + options=["-append","console=ttyAMA0","-dtb" , dtb, "-initrd", cpio_file, "-M", "versatilepb"]) + self.emulator.login() + +# Test with Glibc and Numpy +class TestPythonPy3GlibcNumpy(TestPythonNumpy): + __test__ = True + config = \ + """ + BR2_arm=y + BR2_arm926t=y + BR2_ARM_CPU_ARMV5=y + BR2_ARM_EABI=y + BR2_ARM_SOFT_FLOAT=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y + BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y + BR2_TOOLCHAIN_EXTERNAL_URL="http://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2018.11-1.tar.bz2" + BR2_TOOLCHAIN_EXTERNAL_GCC_8=y + BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y + # BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set + BR2_TOOLCHAIN_EXTERNAL_HAS_CXX=y + BR2_TOOLCHAIN_EXTERNAL_HAS_SSP=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y + BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/arm-versatile/linux.config" + BR2_LINUX_KERNEL_DTS_SUPPORT=y + BR2_LINUX_KERNEL_INTREE_DTS_NAME="versatile-pb" + BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON_NUMPY=y + BR2_TARGET_ROOTFS_CPIO=y + # BR2_TARGET_ROOTFS_TAR is not set + """ + +# Test with Glibc and Numpy + Clapack +class TestPythonPy3GlibcNumpyWithClapack(TestPythonNumpy): + __test__ = True + config = \ + """ + BR2_arm=y + BR2_arm926t=y + BR2_ARM_CPU_ARMV5=y + BR2_ARM_EABI=y + BR2_ARM_SOFT_FLOAT=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y + BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y + BR2_TOOLCHAIN_EXTERNAL_URL="http://toolchains.bootlin.com/downloads/releases/toolchains/armv5-eabi/tarballs/armv5-eabi--glibc--bleeding-edge-2018.11-1.tar.bz2" + BR2_TOOLCHAIN_EXTERNAL_GCC_8=y + BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_14=y + # BR2_TOOLCHAIN_EXTERNAL_LOCALE is not set + BR2_TOOLCHAIN_EXTERNAL_HAS_CXX=y + BR2_TOOLCHAIN_EXTERNAL_HAS_SSP=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS=y + BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_NPTL=y + BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/arm-versatile/linux.config" + BR2_LINUX_KERNEL_DTS_SUPPORT=y + BR2_LINUX_KERNEL_INTREE_DTS_NAME="versatile-pb" + BR2_PACKAGE_PYTHON3=y + BR2_PACKAGE_PYTHON_NUMPY=y + BR2_PACKAGE_CLAPACK=y + BR2_TARGET_ROOTFS_CPIO=y + # BR2_TARGET_ROOTFS_TAR is not set + """