From patchwork Sun Sep 25 15:02:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1682200 X-Patchwork-Delegate: trini@ti.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=nclE9UaE; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Mb8Q23x8Vz1yq1 for ; Mon, 26 Sep 2022 01:07:38 +1000 (AEST) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1D81984BFB; Sun, 25 Sep 2022 17:04:30 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="nclE9UaE"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6977084C4B; Sun, 25 Sep 2022 17:03:46 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4050184C3A for ; Sun, 25 Sep 2022 17:03:35 +0200 (CEST) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x12c.google.com with SMTP id g6so2419205ild.6 for ; Sun, 25 Sep 2022 08:03:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=88Lcf7CVFvL/gGA2sDogD/0c2wf6hdjn5kMrCcUAt1k=; b=nclE9UaE/LY1qmXInUzqOh2dsz+9IRVheIbZxpXB1NXOS/PVBlXw8KTUhRCN9lcN0a +xmGrwfvU1p/SUdye8aEUZWDCTfhyZ+C13g43WCE+xKohsE9rZUCnD+sFaWk9vEupOGX pdUYrxxEd65wn93utzszqhuixMp1H1sdorEM8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=88Lcf7CVFvL/gGA2sDogD/0c2wf6hdjn5kMrCcUAt1k=; b=8GiIPjpx7/bclsnGnXq7UUNJud2gesExehl5JyOHJ5cliH8AyJ79blKFrGYPAn8ULk zf2SU8QTigCGu/xm7vfiDUkaMQgxEIiaxbk/mxf9rzpPBs+UC0g6qVkZtPxVNyD6hEiZ yN3bM21eA3C+RRmoJhW+Cy6nFCi16dzwL02HLdAfk+qts/aF+9jYyf8Uslg4R8x6I2+w 26t58VYvchphYIqitCoMj66KZb+Z52TzlgcJxi/18FjEgeFv2uKH7Wjr4ykmGEPK1onk ILHoTEqFe6LtX4cyYq07iZmlEGF80IifvKEgDa3GykUHOb0z7Kq4DY9XQs2YFWCSnUzc eyLQ== X-Gm-Message-State: ACrzQf0NcINVIeK2PD7H+WgrtdmuCNNXsW+IKIGA4HoiMWU/D33dlr1P PgHw38j3Z2LvwPGViAb9LTOcg85JRvssdQ== X-Google-Smtp-Source: AMsMyM4c8M8sg29eIUsIq7mznKGQoZ/emcY2AL/peM3DOUjuqQJT70cBJGapk7Mgss39Q5GmS2wUaw== X-Received: by 2002:a05:6e02:180c:b0:2f1:5405:4bba with SMTP id a12-20020a056e02180c00b002f154054bbamr8215705ilv.152.1664118214431; Sun, 25 Sep 2022 08:03:34 -0700 (PDT) Received: from sjg1.roam.corp.google.com (c-67-190-102-125.hsd1.co.comcast.net. [67.190.102.125]) by smtp.gmail.com with ESMTPSA id u19-20020a926013000000b002f19d9838c6sm5360578ilb.25.2022.09.25.08.03.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 08:03:34 -0700 (PDT) From: Simon Glass To: U-Boot Mailing List Cc: Tom Rini , Simon Glass , Heinrich Schuchardt , Stefan Roese Subject: [PATCH 23/45] test: Report skippped tests Date: Sun, 25 Sep 2022 09:02:26 -0600 Message-Id: <20220925150248.2524421-24-sjg@chromium.org> X-Mailer: git-send-email 2.37.3.998.g577e59143f-goog In-Reply-To: <20220925150248.2524421-1-sjg@chromium.org> References: <20220925150248.2524421-1-sjg@chromium.org> MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean At present it is possible for a test to skip itself by returning -EAGAIN but this is not recorded. Keep a track of skipped tests and report the total at the end. Signed-off-by: Simon Glass --- include/test/test.h | 2 ++ test/test-main.c | 23 ++++++++++++++++++++--- 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/include/test/test.h b/include/test/test.h index 25d5bdb201b..011668795b9 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -13,6 +13,7 @@ * struct unit_test_state - Entire state of test system * * @fail_count: Number of tests that failed + * @skip_count: Number of tests that were skipped * @start: Store the starting mallinfo when doing leak test * @of_live: true to use livetree if available, false to use flattree * @of_root: Record of the livetree root node (used for setting up tests) @@ -32,6 +33,7 @@ */ struct unit_test_state { int fail_count; + int skip_count; struct mallinfo start; struct device_node *of_root; bool of_live; diff --git a/test/test-main.c b/test/test-main.c index 3062d7f8d95..8d1a4c4a818 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -365,6 +365,19 @@ static int test_post_run(struct unit_test_state *uts, struct unit_test *test) return 0; } +/** + * skip_test() - Handle skipping a test + * + * @uts: Test state to update + * @return -EAGAIN (always) + */ +static int skip_test(struct unit_test_state *uts) +{ + uts->skip_count++; + + return -EAGAIN; +} + /** * ut_run_test() - Run a single test * @@ -395,11 +408,13 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test, ret = test_pre_run(uts, test); if (ret == -EAGAIN) - return -EAGAIN; + return skip_test(uts); if (ret) return ret; - test->func(uts); + ret = test->func(uts); + if (ret == -EAGAIN) + skip_test(uts); ret = test_post_run(uts, test); if (ret) @@ -433,7 +448,7 @@ static int ut_run_test_live_flat(struct unit_test_state *uts, int runs; if ((test->flags & UT_TESTF_OTHER_FDT) && !IS_ENABLED(CONFIG_SANDBOX)) - return -EAGAIN; + return skip_test(uts); /* Run with the live tree if possible */ runs = 0; @@ -567,6 +582,8 @@ int ut_run_list(const char *category, const char *prefix, os_free(uts.other_fdt); } + if (uts.skip_count) + printf("Skipped: %d, ", uts.skip_count); if (ret == -ENOENT) printf("Test '%s' not found\n", select_name); else