From patchwork Mon Oct 26 02:38:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1387438 X-Patchwork-Delegate: sjg@chromium.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=Rt6nU1zc; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CKJw65nLxz9sSn for ; Mon, 26 Oct 2020 13:40:06 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 62FCF824D6; Mon, 26 Oct 2020 03:39:07 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="Rt6nU1zc"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 980A2824C4; Mon, 26 Oct 2020 03:38:56 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,SPF_HELO_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-il1-x144.google.com (mail-il1-x144.google.com [IPv6:2607:f8b0:4864:20::144]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id C806582490 for ; Mon, 26 Oct 2020 03:38:53 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x144.google.com with SMTP id v18so1497796ilg.1 for ; Sun, 25 Oct 2020 19:38:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UxUiyxka/4ssipVsBO3G9okyxY8AXtrCshfGK67+Is4=; b=Rt6nU1zcNPBaZ3A27QGDHF1h/gUggYZr8kgvb778uAexYwnvp7NQ41xZsJVW9ViT4C Mv0FXZ8gb8G/PSk0FxWad2sLzgg7jfTpQTxA+F/XBDQmVbloEH95bdZ1H6wUfeIuiU26 6iMH14QxbFnR3bZ24o7NjrM4T6DwrwtHJPnwo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UxUiyxka/4ssipVsBO3G9okyxY8AXtrCshfGK67+Is4=; b=pX+tuEfRUor1B5OHmbRDltC91KASt6Nt37ly3drj/ZN2tzNAyTkCi56f9f/hhm06pk Ez5rVEeststxqQ7zyyc1qgT/wrZZm0j1doZ9d2RzkzAMkFhWazN4i4BfkPe0RVM2lTyo mAJ21Hfnxy/fP7FIV0E5QdglKUm0FpwVC7mWQ0Q5LFRwmGw6xh1bNyHSfSvMobIXHFm5 LNXQHDL53uQR2ctSwxBtAbfK5Hmz+FDyeiehOj90nKAyxDT6i4MNLG0ELzhKa31ar020 O8xQ2ttniOJW3mxpPveQooNNbYVFk7A6D+2Blb69F8n39o9kbObZKt5Fdxv2MIHD/KU3 jQXw== X-Gm-Message-State: AOAM532M8ALGU55Y35AQiEhY6dUe2AoAnlpaDFwHYIJqBWxtWf9yPkBc P/sLZ6SN/tRExEQZwJ9VR5erMwfBpI+iVHC+ X-Google-Smtp-Source: ABdhPJymUUdEQtLe63m/z4WsQqUc0c75lj50x3v4NtCOqwKakmg+5pDeh5Z+MRSX9BNRh3LVd5DcuA== X-Received: by 2002:a05:6e02:14b:: with SMTP id j11mr1847817ilr.109.1603679932579; Sun, 25 Oct 2020 19:38:52 -0700 (PDT) Received: from localhost.localdomain (c-73-14-175-90.hsd1.co.comcast.net. [73.14.175.90]) by smtp.gmail.com with ESMTPSA id y3sm4632838ioq.18.2020.10.25.19.38.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Oct 2020 19:38:52 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Stephen Warren , Stephen Warren , Simon Glass , Cristian Ciocaltea , Heinrich Schuchardt , Marek Vasut Subject: [PATCH v2 06/11] pytest: Collect SPL unit tests Date: Sun, 25 Oct 2020 20:38:31 -0600 Message-Id: <20201026023836.1629465-7-sjg@chromium.org> X-Mailer: git-send-email 2.29.0.rc2.309.g374f81d7ae-goog In-Reply-To: <20201026023836.1629465-1-sjg@chromium.org> References: <20201026023836.1629465-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.3 at phobos.denx.de X-Virus-Status: Clean Add a new test_spl fixture to handle running SPL unit tests. Signed-off-by: Simon Glass Signed-off-by: Simon Glass --- Changes in v2: - Drop debug print() - Restart U-Boot after each SPL test test/py/conftest.py | 13 ++++++++----- test/py/tests/test_spl.py | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+), 5 deletions(-) create mode 100644 test/py/tests/test_spl.py diff --git a/test/py/conftest.py b/test/py/conftest.py index 30920474b36..dc92c0be32e 100644 --- a/test/py/conftest.py +++ b/test/py/conftest.py @@ -227,7 +227,7 @@ def pytest_configure(config): console = u_boot_console_exec_attach.ConsoleExecAttach(log, ubconfig) re_ut_test_list = re.compile(r'_u_boot_list_2_(.*)_test_2_\1_test_(.*)\s*$') -def generate_ut_subtest(metafunc, fixture_name): +def generate_ut_subtest(metafunc, fixture_name, sym_path): """Provide parametrization for a ut_subtest fixture. Determines the set of unit tests built into a U-Boot binary by parsing the @@ -237,12 +237,13 @@ def generate_ut_subtest(metafunc, fixture_name): Args: metafunc: The pytest test function. fixture_name: The fixture name to test. + sym_path: Relative path to the symbol file with preceding '/' + (e.g. '/u-boot.sym') Returns: Nothing. """ - - fn = console.config.build_dir + '/u-boot.sym' + fn = console.config.build_dir + sym_path try: with open(fn, 'rt') as f: lines = f.readlines() @@ -317,10 +318,12 @@ def pytest_generate_tests(metafunc): Returns: Nothing. """ - for fn in metafunc.fixturenames: if fn == 'ut_subtest': - generate_ut_subtest(metafunc, fn) + generate_ut_subtest(metafunc, fn, '/u-boot.sym') + continue + if fn == 'ut_spl_subtest': + generate_ut_subtest(metafunc, fn, '/spl/u-boot-spl.sym') continue generate_config(metafunc, fn) diff --git a/test/py/tests/test_spl.py b/test/py/tests/test_spl.py new file mode 100644 index 00000000000..990cc9b0717 --- /dev/null +++ b/test/py/tests/test_spl.py @@ -0,0 +1,34 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright 2020 Google LLC +# Written by Simon Glass + +import os.path +import pytest + +def test_spl(u_boot_console, ut_spl_subtest): + """Execute a "ut" subtest. + + The subtests are collected in function generate_ut_subtest() from linker + generated lists by applying a regular expression to the lines of file + spl/u-boot-spl.sym. The list entries are created using the C macro + UNIT_TEST(). + + Strict naming conventions have to be followed to match the regular + expression. Use UNIT_TEST(foo_test_bar, _flags, foo_test) for a test bar in + test suite foo that can be executed via command 'ut foo bar' and is + implemented in C function foo_test_bar(). + + Args: + u_boot_console (ConsoleBase): U-Boot console + ut_subtest (str): SPL test to be executed (e.g. 'dm platdata_phandle') + """ + try: + cons = u_boot_console + cons.restart_uboot_with_flags(['-u', ut_spl_subtest]) + output = cons.get_spawn_output().replace('\r', '') + assert 'Failures: 0' in output + finally: + # Restart afterward in case a non-SPL test is run next. This should not + # happen since SPL tests are run in their own invocation of test.py, but + # the cost of doing this is not too great at present. + u_boot_console.restart_uboot()