From patchwork Thu Mar 4 13:50:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1447323 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=gUCTaGR0; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Drsl31LLnz9sVt for ; Fri, 5 Mar 2021 00:53:03 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id CEC81827E1; Thu, 4 Mar 2021 14:52:13 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="gUCTaGR0"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A4F878277D; Thu, 4 Mar 2021 14:51:57 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 1582A825C6 for ; Thu, 4 Mar 2021 14:51:51 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-oi1-x236.google.com with SMTP id f3so30110294oiw.13 for ; Thu, 04 Mar 2021 05:51:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YExSWGzH3tq0e5lYCL8u+HMlRilmt0LPdPwrt4EXiC8=; b=gUCTaGR06yb46Ms4BJKX9InIk1kXEORGAj0/dfrid4KHYFpIGBAo9O5Nhl84X9Y/x5 U3LFFwnYBuMRO/3KDseCZCffYX8wLi/8uoDprxQ/xIDdxrmyWz4ufdHPnnjHbjk3nL1a mVQI6vQRKYTt22GIQCCJmu1mP7vECBkehXX8g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YExSWGzH3tq0e5lYCL8u+HMlRilmt0LPdPwrt4EXiC8=; b=B3jL/egoimq1bXLhy2w+qzoLEGG3tmFco26ZTdfNEJbsZAO1Gr2fZV5rwND71+6gbF CVopIO15ExSQhBVPaG4bnkp3JhMtOWzHPpvsdYEQCmsJG9HMtvibIuz22nGhycBk2cSj L/sHeeaUAzDoNCoByqLu9caigoo1I9k6ISNH+C4bySg8pJVwzpBn++6uPAymVRhtdjOV Bj7pDduREAKQJgA4TRWzzDIN1cuETqP6GJmltFCOODBTEIKl6bkWsCBOf6G+lW61ADdo fD5M+bEGrAOEibZ5hM9ZTCXAlMLIP9WrJZdtTyKz2BWjbZuSlIYw1cPiOicF5D+8QugB QO3w== X-Gm-Message-State: AOAM530zKLsJcjoyqxQUr1moivBvGrdsYbRUQQG4eTZSS2d2gqihRJWM M3BW93j7Qxc+OOeWKl79Eobw6uRQRExX763R X-Google-Smtp-Source: ABdhPJyeRWaqrk+lpSqhXVmlgHonzQ02F2mXtt2KEJXfdrnCTJT/v1sR9/gEKl3pn2rboYyASUEpZA== X-Received: by 2002:aca:1b05:: with SMTP id b5mr3018046oib.96.1614865909590; Thu, 04 Mar 2021 05:51:49 -0800 (PST) Received: from kiwi.bld.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id x10sm4029465otp.19.2021.03.04.05.51.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Mar 2021 05:51:49 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Heinrich Schuchardt , Stephen Warren , Stephen Warren , Simon Glass , Pratyush Yadav Subject: [PATCH v4 07/42] doc: Document how sandbox_spl_tests are run Date: Thu, 4 Mar 2021 06:50:43 -0700 Message-Id: <20210304135118.643277-7-sjg@chromium.org> X-Mailer: git-send-email 2.30.1.766.gb4fecdf3b7-goog In-Reply-To: <20210304135118.643277-1-sjg@chromium.org> References: <20210304135118.643277-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Add a few notes about the sandbox_spl tests, since they are special. Signed-off-by: Simon Glass Acked-by: Pratyush Yadav --- Changes in v4: - Fix 'of-pldata' typo Changes in v3: - Reword the SPL tests section for clarity Changes in v2: - Add a note that SPL tests can in fact be run individualy - Document how to run all C tests with 'ut all' - Fix 'get list' typo doc/develop/testing.rst | 5 +++ doc/develop/tests_sandbox.rst | 82 +++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+) diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst index 87c90eee271..b181c2e2e41 100644 --- a/doc/develop/testing.rst +++ b/doc/develop/testing.rst @@ -58,6 +58,11 @@ also select particular tests with -k:: ./test/py/test.py --bd sandbox_flattree --build -k hello +There are some special tests that run in SPL. For this you need the sandbox_spl +build:: + + ./test/py/test.py --bd sandbox_spl --build -k test_spl + See test/py/README.md for more information about the pytest suite. See :doc:`tests_sandbox` for how to run tests directly (not through pytest). diff --git a/doc/develop/tests_sandbox.rst b/doc/develop/tests_sandbox.rst index 85bbd4f6734..daf87e53a99 100644 --- a/doc/develop/tests_sandbox.rst +++ b/doc/develop/tests_sandbox.rst @@ -77,3 +77,85 @@ You can easily use gdb on these tests, without needing --gdbserver:: You can then single-step and look at variables as needed. + +Running sandbox_spl tests directly +---------------------------------- + +SPL is the phase before U-Boot proper. It is present in the sandbox_spl build, +so you can run SPL like this:: + + ./spl/u-boot-spl + +SPL tests are special in that they run (only in the SPL phase, of course) if the +-u flag is given:: + + ./spl/u-boot-spl -u + + U-Boot SPL 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700) + Running 5 driver model tests + Test: dm_test_of_plat_base: of_platdata.c (flat tree) + Test: dm_test_of_plat_dev: of_platdata.c (flat tree) + Test: dm_test_of_plat_parent: of_platdata.c (flat tree) + Test: dm_test_of_plat_phandle: of_platdata.c (flat tree) + Test: dm_test_of_plat_props: of_platdata.c (flat tree) + Failures: 0 + + + U-Boot 2021.01-00723-g43c77b51be5-dirty (Jan 24 2021 - 16:38:24 -0700) + + DRAM: 128 MiB + ... + +It is not possible to run SPL tests in U-Boot proper, firstly because they are +not built into U-Boot proper and secondly because the environment is very +different, e.g. some SPL tests rely on of-platdata which is only available in +SPL. + +Note that after running, SPL continues to boot into U-Boot proper. You can add +'-c exit' to make U-Boot quit without doing anything further. It is not +currently possible to run SPL tests and then stop, since the pytests require +that U-Boot produces the expected banner. + +You can use the -k flag to select which tests run:: + + ./spl/u-boot-spl -u -k dm_test_of_plat_parent + +Of course you can use gdb with sandbox_spl, just as with sandbox. + + +Running all tests directly +-------------------------- + +A fast way to run all sandbox tests is:: + + ./u-boot -T -c "ut all" + +It typically runs single-thread in 6 seconds on 2021 hardware, with 2s of that +to the delays in the time test. + +This should not be considered a substitute for 'make check', but can be helpful +for git bisect, etc. + + +What tests are built in? +------------------------ + +Whatever sandbox build is used, which tests are present is determined by which +source files are built. For sandbox_spl, the of_platdata tests are built +because of the build rule in test/dm/Makefile:: + + ifeq ($(CONFIG_SPL_BUILD),y) + obj-$(CONFIG_SPL_OF_PLATDATA) += of_platdata.o + else + ...other tests for non-spl + endif + +You can get a list of tests in a U-Boot ELF file by looking for the +linker_list:: + + $ nm /tmp/b/sandbox_spl/spl/u-boot-spl |grep 2_dm_test + 000000000001f200 D _u_boot_list_2_dm_test_2_dm_test_of_plat_base + 000000000001f220 D _u_boot_list_2_dm_test_2_dm_test_of_plat_dev + 000000000001f240 D _u_boot_list_2_dm_test_2_dm_test_of_plat_parent + 000000000001f260 D _u_boot_list_2_dm_test_2_dm_test_of_plat_phandle + 000000000001f280 D _u_boot_list_2_dm_test_2_dm_test_of_plat_props