{"id":2222498,"url":"http://patchwork.ozlabs.org/api/1.1/patches/2222498/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/patch/20260412111958.943933-11-sjg@chromium.org/","project":{"id":18,"url":"http://patchwork.ozlabs.org/api/1.1/projects/18/?format=json","name":"U-Boot","link_name":"uboot","list_id":"u-boot.lists.denx.de","list_email":"u-boot@lists.denx.de","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"<20260412111958.943933-11-sjg@chromium.org>","date":"2026-04-12T11:19:47","name":"[v2,10/12] test: Add documentation for the test framework","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"b37c1e131b9a1603fe25e33884b75e18b2b0811a","submitter":{"id":6170,"url":"http://patchwork.ozlabs.org/api/1.1/people/6170/?format=json","name":"Simon Glass","email":"sjg@chromium.org"},"delegate":{"id":3651,"url":"http://patchwork.ozlabs.org/api/1.1/users/3651/?format=json","username":"trini","first_name":"Tom","last_name":"Rini","email":"trini@ti.com"},"mbox":"http://patchwork.ozlabs.org/project/uboot/patch/20260412111958.943933-11-sjg@chromium.org/mbox/","series":[{"id":499596,"url":"http://patchwork.ozlabs.org/api/1.1/series/499596/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/list/?series=499596","date":"2026-04-12T11:19:37","name":"test: Add support for passing arguments to C unit tests","version":2,"mbox":"http://patchwork.ozlabs.org/series/499596/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2222498/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2222498/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256\n header.s=google header.b=h5DmWY8m;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=85.214.62.61; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org\n header.b=\"h5DmWY8m\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=sjg@chromium.org"],"Received":["from phobos.denx.de (phobos.denx.de [85.214.62.61])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4ftp9r50dxz1yCx\n\tfor ; Sun, 12 Apr 2026 21:25:20 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 5E07B8420C;\n\tSun, 12 Apr 2026 13:24:33 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id 2AFB584242; Sun, 12 Apr 2026 13:24:33 +0200 (CEST)","from mail-oo1-xc2f.google.com (mail-oo1-xc2f.google.com\n [IPv6:2607:f8b0:4864:20::c2f])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id C7264839D5\n for ; Sun, 12 Apr 2026 13:24:30 +0200 (CEST)","by mail-oo1-xc2f.google.com with SMTP id\n 006d021491bc7-67e0d3f288aso2342707eaf.0\n for ; Sun, 12 Apr 2026 04:24:30 -0700 (PDT)","from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id\n 586e51a60fabf-423dd395606sm6622071fac.1.2026.04.12.04.24.26\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Sun, 12 Apr 2026 04:24:28 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,\n DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,\n RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham\n autolearn_force=no version=3.4.2","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=chromium.org; s=google; t=1775993069; x=1776597869; darn=lists.denx.de;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=HtC3HYFPdl/00ksMSAHwr/U9a9/nYBnm16wH2uxUpHk=;\n b=h5DmWY8ml9YGwD1E7qJNqV3yOdbX8R4NBnYxrjRzjxLG4AGS1Lnr2x3rw+CDoMhoiK\n o+RgC8FQI7VaLK0gC9gh4gz2BXCCuhbTOa0WEh4QkP0m22LQe81NUoIHhpIXvmzd3i4U\n MEQWwg6Wv96b2DfqCXHAIzxyulUdMBRDPMg14=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1775993069; x=1776597869;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=HtC3HYFPdl/00ksMSAHwr/U9a9/nYBnm16wH2uxUpHk=;\n b=Vw4qp4Zf4aGo79FyQc0hQnOz4u0nrIkWInYl42jPwEM9MRfchO+dFoEwt6TUj84Z74\n GTtZwAVibanLoD0htvZgYn96Vx4d/O3X1vzoftqMN+e4tTmC6FKnXa3RJtcWafOgyK2M\n oC+ewnkqLUB+MUDxswUo64FHpRNgTa0WO7tRoSNj+O7E1+S6TbI0Iv6u4mwiz9puIUzN\n yZS5QvA5/hA98VLMKQsKwFyMF7fzaivP66fO+VER9c74K9al9rVKw0Hn8ufB94lf/pzY\n efw3+nmyJF37ArpAo5F+rBKqzr3+0JZbQqpas97JjEF/Q3p9eGtnYvSirBHNxQMPxrEj\n a5MQ==","X-Gm-Message-State":"AOJu0YyJ6PiAUtXuP72H7g9vuWnBhgD5W/cVvD/i/E7/NRRoX9IZVisA\n q7swiaZAY/2p1x5dmMdSSlXai97draSdwBwimi7d2JPw/W5ssaYFRTML5MaeBbl1lCHk0tUpynK\n 0FBKaNA==","X-Gm-Gg":"AeBDietwyXIzT5ZvB6eC2Qdc1/KTp4vw1+2vaZikp/rUxQIA5kTWQeQZxc+pgKj197q\n u0Yim0A9jQ9BFANmR8m4eOESCii0lEw5ZSimpxaZn8P0zCU7ukWC2UmSev/h+1BKxk8IBh5Vpc4\n NI/2anhZRUpkiMX/57NMqY6vqDnGzBHfuTJIyI6NewQVCGuEgMCDYMOd8J+WX6H5+litRzDnMat\n dd2YxgcGdUMG94BD8FazbPWM6fV1eR9OxEjAgOevhFfgniEQSaKqyb4lxgFOm9m2OXH2ZIC1Rhm\n I8lgd27SDnNcrWPTOMGlY6jArSXdAr//b5t0PEhABq34qOBwAET+01Upi5HaqrS0P63OUYtx1hm\n gKUAEXljcwvD5ObtwHR3a8OVfDgK5LHgN7GTbg8N/5gzZpPAperTt9A3/AhvQDKUlZhSFcepc1u\n v9DeGB5pIJMm1/dHxeXw==","X-Received":"by 2002:a05:6820:5183:b0:682:ecb8:c3c with SMTP id\n 006d021491bc7-68be88efeadmr3016282eaf.49.1775993068733;\n Sun, 12 Apr 2026 04:24:28 -0700 (PDT)","From":"Simon Glass ","To":"u-boot@lists.denx.de","Cc":"Tom Rini , Heinrich Schuchardt ,\n Simon Glass ,\n Ilias Apalodimas ","Subject":"[PATCH v2 10/12] test: Add documentation for the test framework","Date":"Sun, 12 Apr 2026 05:19:47 -0600","Message-ID":"<20260412111958.943933-11-sjg@chromium.org>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20260412111958.943933-1-sjg@chromium.org>","References":"<20260412111958.943933-1-sjg@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"},"content":"Add documentation for test assertions, the private buffer, and test\nparameters. This covers the available ut_assert*() macros, console\noutput checks, memory helpers, and how to pass typed arguments from\nPython to C tests.\n\nSigned-off-by: Simon Glass \n---\n\n(no changes since v1)\n\n doc/develop/tests_writing.rst | 169 ++++++++++++++++++++++++++++++++++\n doc/usage/cmd/ut.rst | 11 ++-\n 2 files changed, 178 insertions(+), 2 deletions(-)","diff":"diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst\nindex 1a020caa411..40192f9db30 100644\n--- a/doc/develop/tests_writing.rst\n+++ b/doc/develop/tests_writing.rst\n@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the\n Python test.\n \n \n+Passing arguments to C tests\n+----------------------------\n+\n+Sometimes a C test needs parameters from Python, such as filenames or expected\n+values that are generated at runtime. The test-argument feature allows this.\n+\n+Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::\n+\n+ static int my_test_norun(struct unit_test_state *uts)\n+ {\n+ const char *filename = ut_str(0);\n+ int count = ut_int(1);\n+\n+ /* test code using filename and count */\n+\n+ return 0;\n+ }\n+ UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,\n+ { \"filename\", UT_ARG_STR },\n+ { \"count\", UT_ARG_INT });\n+\n+Each argument definition specifies a name and type:\n+\n+- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`\n+- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`\n+- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`\n+ (use `1` for true, any other value for false)\n+\n+Arguments are passed on the command line in `name=value` format::\n+\n+ ut -f my_suite my_test_norun filename=/path/to/file count=42\n+\n+From Python, you can call the test like this::\n+\n+ cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'\n+ ubman.run_command(cmd)\n+\n+This approach combines Python's flexibility for setup (creating files,\n+generating values) with C's speed and debuggability for the actual test logic.\n+\n+\n How slow are Python tests?\n --------------------------\n \n@@ -384,6 +425,134 @@ existing suite or creating a new one.\n An example SPL test is spl_test_load().\n \n \n+.. _tests_writing_assertions:\n+\n+Assertions\n+----------\n+\n+The test framework provides various assertion macros, defined in\n+``include/test/ut.h``. All of these return from the test function on failure,\n+unless ``uts->soft_fail`` is set.\n+\n+Basic assertions\n+~~~~~~~~~~~~~~~~\n+\n+ut_assert(cond)\n+ Assert that a condition is non-zero (true)\n+\n+ut_assertf(cond, fmt, args...)\n+ Assert that a condition is non-zero, with a printf() message on failure\n+\n+ut_assertok(cond)\n+ Assert that an operation succeeds (returns 0)\n+\n+ut_reportf(fmt, args...)\n+ Report a failure with a printf() message (always fails)\n+\n+Value comparisons\n+~~~~~~~~~~~~~~~~~\n+\n+ut_asserteq(expr1, expr2)\n+ Assert that two int expressions are equal\n+\n+ut_asserteq_64(expr1, expr2)\n+ Assert that two 64-bit expressions are equal\n+\n+ut_asserteq_str(expr1, expr2)\n+ Assert that two strings are equal\n+\n+ut_asserteq_strn(expr1, expr2)\n+ Assert that two strings are equal, up to the length of the first\n+\n+ut_asserteq_mem(expr1, expr2, len)\n+ Assert that two memory areas are equal\n+\n+ut_asserteq_ptr(expr1, expr2)\n+ Assert that two pointers are equal\n+\n+ut_asserteq_addr(expr1, expr2)\n+ Assert that two addresses (converted from pointers via map_to_sysmem())\n+ are equal\n+\n+ut_asserteq_regex(pattern, str)\n+ Assert that a string matches a regular expression pattern. Uses the SLRE\n+ library for regex matching. Useful when exact matching is fragile, e.g.\n+ when output contains line numbers or variable content.\n+\n+Pointer assertions\n+~~~~~~~~~~~~~~~~~~\n+\n+ut_assertnull(expr)\n+ Assert that a pointer is NULL\n+\n+ut_assertnonnull(expr)\n+ Assert that a pointer is not NULL\n+\n+ut_assertok_ptr(expr)\n+ Assert that a pointer is not an error pointer (checked with IS_ERR())\n+\n+Console output assertions\n+~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+These are used to check console output when ``UTF_CONSOLE`` flag is set.\n+\n+ut_assert_nextline(fmt, args...)\n+ Assert that the next console output line matches the format string\n+\n+ut_assert_nextlinen(fmt, args...)\n+ Assert that the next console output line matches up to the format\n+ string length\n+\n+ut_assert_nextline_regex(pattern)\n+ Assert that the next console output line matches a regex pattern\n+\n+ut_assert_nextline_empty()\n+ Assert that the next console output line is empty\n+\n+ut_assert_skipline()\n+ Assert that there is a next console output line, and skip it\n+\n+ut_assert_skip_to_line(fmt, args...)\n+ Skip console output until a matching line is found\n+\n+ut_assert_skip_to_linen(fmt, args...)\n+ Skip console output until a partial match is found (compares up to the\n+ format-string length)\n+\n+ut_assert_console_end()\n+ Assert that there is no more console output\n+\n+ut_assert_nextlines_are_dump(total_bytes)\n+ Assert that the next lines are a print_buffer() hex dump of the specified\n+ size\n+\n+Memory helpers\n+~~~~~~~~~~~~~~\n+\n+These help check for memory leaks:\n+\n+ut_check_free()\n+ Return the number of bytes free in the malloc() pool\n+\n+ut_check_delta(last)\n+ Return the change in free memory since ``last`` was obtained from\n+ ``ut_check_free()``. A positive value means more memory has been allocated.\n+\n+Private buffer\n+~~~~~~~~~~~~~~\n+\n+Each test has access to a private buffer ``uts->priv`` (256 bytes) for temporary\n+data. This avoids the need to allocate memory or use global variables::\n+\n+ static int my_test(struct unit_test_state *uts)\n+ {\n+ snprintf(uts->priv, sizeof(uts->priv), \"/%s\", filename);\n+ /* use uts->priv as a path string */\n+\n+ return 0;\n+ }\n+\n+\n Writing Python tests\n --------------------\n \ndiff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst\nindex d8c3cbf496c..a26ee6ad7de 100644\n--- a/doc/usage/cmd/ut.rst\n+++ b/doc/usage/cmd/ut.rst\n@@ -11,7 +11,7 @@ Synopsis\n \n ::\n \n- ut [-r] [-f] [-I:] [ | all []] [...]\n+ ut [-r] [-f] [-R] [-I:] [ | all []] [...]\n ut [-s] info\n \n Description\n@@ -37,10 +37,17 @@ test\n causes another test to fail. If the one test fails, testing stops\n immediately.\n \n+-R\n+ Preserve console recording on test failure. Normally when a test fails,\n+ console recording is disabled so error messages go directly to output.\n+ This flag keeps recording enabled, which is useful when testing the test\n+ framework itself.\n+\n args\n Optional arguments to pass to the test, in `name=value` format. These are\n used by tests declared with `UNIT_TEST_ARGS()` which define expected\n- argument names and types.\n+ argument names and types. See :ref:`develop/tests_writing:passing arguments\n+ to c tests` for details.\n \n Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it\n includes a near-complete set of emulators, no code-size limits, many CONFIG\n","prefixes":["v2","10/12"]}