{"id":2222464,"url":"http://patchwork.ozlabs.org/api/1.1/patches/2222464/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/patch/20260412013451.2929001-11-sjg@chromium.org/","project":{"id":18,"url":"http://patchwork.ozlabs.org/api/1.1/projects/18/?format=json","name":"U-Boot","link_name":"uboot","list_id":"u-boot.lists.denx.de","list_email":"u-boot@lists.denx.de","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"<20260412013451.2929001-11-sjg@chromium.org>","date":"2026-04-12T01:34:38","name":"[10/12] test: Add documentation for the test framework","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"b37c1e131b9a1603fe25e33884b75e18b2b0811a","submitter":{"id":6170,"url":"http://patchwork.ozlabs.org/api/1.1/people/6170/?format=json","name":"Simon Glass","email":"sjg@chromium.org"},"delegate":{"id":3184,"url":"http://patchwork.ozlabs.org/api/1.1/users/3184/?format=json","username":"sjg","first_name":"Simon","last_name":"Glass","email":"sjg@chromium.org"},"mbox":"http://patchwork.ozlabs.org/project/uboot/patch/20260412013451.2929001-11-sjg@chromium.org/mbox/","series":[{"id":499582,"url":"http://patchwork.ozlabs.org/api/1.1/series/499582/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/list/?series=499582","date":"2026-04-12T01:34:28","name":"test: Add support for passing arguments to C unit tests","version":1,"mbox":"http://patchwork.ozlabs.org/series/499582/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2222464/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2222464/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256\n header.s=google header.b=HsQIbw/g;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=85.214.62.61; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org\n header.b=\"HsQIbw/g\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=sjg@chromium.org"],"Received":["from phobos.denx.de (phobos.denx.de [85.214.62.61])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4ftY6f2t2wz1xtJ\n\tfor ; Sun, 12 Apr 2026 11:36:42 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 377DD84213;\n\tSun, 12 Apr 2026 03:35:54 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id 1B83984214; Sun, 12 Apr 2026 03:35:50 +0200 (CEST)","from mail-ot1-x330.google.com (mail-ot1-x330.google.com\n [IPv6:2607:f8b0:4864:20::330])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id 05CCB84253\n for ; Sun, 12 Apr 2026 03:35:46 +0200 (CEST)","by mail-ot1-x330.google.com with SMTP id\n 46e09a7af769-7dbf23885dfso2095178a34.3\n for ; Sat, 11 Apr 2026 18:35:46 -0700 (PDT)","from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id\n 46e09a7af769-7dc2d1aca3asm4242826a34.6.2026.04.11.18.35.41\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Sat, 11 Apr 2026 18:35:41 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,\n DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,\n RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham\n autolearn_force=no version=3.4.2","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=chromium.org; s=google; t=1775957743; x=1776562543; darn=lists.denx.de;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=Gbhm8tbOULVyHwBhE5VbchyfnQKWBC04NywKNgZ9ALQ=;\n b=HsQIbw/gLe6A+NvxhQd8mBJ5AWUq6iaD/UsxFgMrNKtfWLFlO0tBtd8x17Sr5Zgb9U\n NGZrkYK2GKbbmCn0vlEUfNia7HZ0NtymwvmJyZVV91hmX+YvUJjxqufxfN4RWEasYes6\n 5yNB0IBtUv04x7BMahfcFFwPer2J0EjLQrnsY=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1775957743; x=1776562543;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from\n :to:cc:subject:date:message-id:reply-to;\n bh=Gbhm8tbOULVyHwBhE5VbchyfnQKWBC04NywKNgZ9ALQ=;\n b=CaNC6aIFxjnnMAG/hNTMA9xdJxjuqpY7g0xXYqok6EN/T01a1WhtLjmNoOI+sNbaBJ\n cESRZuPl2VhcLqqPRVx4DxjqlE13+nXBn3HvC/dWlmIezXvdZQlo4VZ5qxAJPc8mQY9Y\n T3lXHJy07T/vE4KeS5CJGBNMEQBgNPn9E9oOk8g2XVrdB3TBt06u10rkT/fqKNSFEFlJ\n KZxa8NAUFwRiFJoFgmFqMof6hOfwvyC6QG6Z1TRQOr+5oCxu3ZeejKEipfsHLTvplWRK\n 5yS4oEw3exAYbo5tahi6ime+06GUDvREqCQhYij5dipMCyxYl50qKIll7iXdPCyE6QgE\n njHA==","X-Gm-Message-State":"AOJu0YxFY9Erzp/klBEXGSWYIdbzdqvtz/wGMcLfiZk6/Y4V+LKUo0hI\n HCTmFIvctJV28RRUAIcW0W4Q+JyGWZINzTmP4Vqk7ncxK5/AZRLyi7Q1lUFNR/bK71j8ulUetbt\n 0gCP6aw==","X-Gm-Gg":"AeBDieua+a/gHqrpVpZciJrgCHbR8JhtT2qXTxYnQ9aKwpnKDMU1pQCbMdGeP8uxUMA\n QPbeMVawMj3PclIgaOqISAnPT3DYTFL/4SrfGdXA6EeLn1euAZfmeW75xiSdEGIko9YdM2jAh1O\n T8PnRsXoWQL0wWguTUFPwAbtaVB/BuwXPChHwXqLgbwe1axAHzkKMvDTUbSkjbQgt5Ei1pu9sHl\n Zi2S7bUnX+HlVVyigOkamPl9a2wsccJJwboUdDavmfGUVkHtapOKqKuuq0nT9SsWjeQACPGKliw\n NnOHbGlA+Cy9ASsM3//yexd9yiI/7Vo/WzfTqgxDejw1pvKT4EiuYCqD4pIw61ioyPZlijd9x0B\n AN/0WMcdBRpe5TPbNbcchsiReZbUEHWvrJMsZ93DPDhESgj1jTN7Wz/C0HJuYz0hxLvUPml+y36\n FSxoYzLIM+OIKX2KDMNA==","X-Received":"by 2002:a05:6830:6a95:b0:7d7:da1d:4472 with SMTP id\n 46e09a7af769-7dc27e2c361mr5404743a34.8.1775957743520;\n Sat, 11 Apr 2026 18:35:43 -0700 (PDT)","From":"Simon Glass ","To":"u-boot@lists.denx.de","Cc":"Heinrich Schuchardt , Tom Rini ,\n Simon Glass , Simon Glass ,\n Ilias Apalodimas ","Subject":"[PATCH 10/12] test: Add documentation for the test framework","Date":"Sat, 11 Apr 2026 19:34:38 -0600","Message-ID":"<20260412013451.2929001-11-sjg@chromium.org>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20260412013451.2929001-1-sjg@chromium.org>","References":"<20260412013451.2929001-1-sjg@chromium.org>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"},"content":"From: Simon Glass \n\nAdd documentation for test assertions, the private buffer, and test\nparameters. This covers the available ut_assert*() macros, console\noutput checks, memory helpers, and how to pass typed arguments from\nPython to C tests.\n\nSigned-off-by: Simon Glass \nSigned-off-by: Simon Glass \n---\n\n doc/develop/tests_writing.rst | 169 ++++++++++++++++++++++++++++++++++\n doc/usage/cmd/ut.rst | 11 ++-\n 2 files changed, 178 insertions(+), 2 deletions(-)","diff":"diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst\nindex 1a020caa411..40192f9db30 100644\n--- a/doc/develop/tests_writing.rst\n+++ b/doc/develop/tests_writing.rst\n@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the\n Python test.\n \n \n+Passing arguments to C tests\n+----------------------------\n+\n+Sometimes a C test needs parameters from Python, such as filenames or expected\n+values that are generated at runtime. The test-argument feature allows this.\n+\n+Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::\n+\n+ static int my_test_norun(struct unit_test_state *uts)\n+ {\n+ const char *filename = ut_str(0);\n+ int count = ut_int(1);\n+\n+ /* test code using filename and count */\n+\n+ return 0;\n+ }\n+ UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,\n+ { \"filename\", UT_ARG_STR },\n+ { \"count\", UT_ARG_INT });\n+\n+Each argument definition specifies a name and type:\n+\n+- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`\n+- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`\n+- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`\n+ (use `1` for true, any other value for false)\n+\n+Arguments are passed on the command line in `name=value` format::\n+\n+ ut -f my_suite my_test_norun filename=/path/to/file count=42\n+\n+From Python, you can call the test like this::\n+\n+ cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'\n+ ubman.run_command(cmd)\n+\n+This approach combines Python's flexibility for setup (creating files,\n+generating values) with C's speed and debuggability for the actual test logic.\n+\n+\n How slow are Python tests?\n --------------------------\n \n@@ -384,6 +425,134 @@ existing suite or creating a new one.\n An example SPL test is spl_test_load().\n \n \n+.. _tests_writing_assertions:\n+\n+Assertions\n+----------\n+\n+The test framework provides various assertion macros, defined in\n+``include/test/ut.h``. All of these return from the test function on failure,\n+unless ``uts->soft_fail`` is set.\n+\n+Basic assertions\n+~~~~~~~~~~~~~~~~\n+\n+ut_assert(cond)\n+ Assert that a condition is non-zero (true)\n+\n+ut_assertf(cond, fmt, args...)\n+ Assert that a condition is non-zero, with a printf() message on failure\n+\n+ut_assertok(cond)\n+ Assert that an operation succeeds (returns 0)\n+\n+ut_reportf(fmt, args...)\n+ Report a failure with a printf() message (always fails)\n+\n+Value comparisons\n+~~~~~~~~~~~~~~~~~\n+\n+ut_asserteq(expr1, expr2)\n+ Assert that two int expressions are equal\n+\n+ut_asserteq_64(expr1, expr2)\n+ Assert that two 64-bit expressions are equal\n+\n+ut_asserteq_str(expr1, expr2)\n+ Assert that two strings are equal\n+\n+ut_asserteq_strn(expr1, expr2)\n+ Assert that two strings are equal, up to the length of the first\n+\n+ut_asserteq_mem(expr1, expr2, len)\n+ Assert that two memory areas are equal\n+\n+ut_asserteq_ptr(expr1, expr2)\n+ Assert that two pointers are equal\n+\n+ut_asserteq_addr(expr1, expr2)\n+ Assert that two addresses (converted from pointers via map_to_sysmem())\n+ are equal\n+\n+ut_asserteq_regex(pattern, str)\n+ Assert that a string matches a regular expression pattern. Uses the SLRE\n+ library for regex matching. Useful when exact matching is fragile, e.g.\n+ when output contains line numbers or variable content.\n+\n+Pointer assertions\n+~~~~~~~~~~~~~~~~~~\n+\n+ut_assertnull(expr)\n+ Assert that a pointer is NULL\n+\n+ut_assertnonnull(expr)\n+ Assert that a pointer is not NULL\n+\n+ut_assertok_ptr(expr)\n+ Assert that a pointer is not an error pointer (checked with IS_ERR())\n+\n+Console output assertions\n+~~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+These are used to check console output when ``UTF_CONSOLE`` flag is set.\n+\n+ut_assert_nextline(fmt, args...)\n+ Assert that the next console output line matches the format string\n+\n+ut_assert_nextlinen(fmt, args...)\n+ Assert that the next console output line matches up to the format\n+ string length\n+\n+ut_assert_nextline_regex(pattern)\n+ Assert that the next console output line matches a regex pattern\n+\n+ut_assert_nextline_empty()\n+ Assert that the next console output line is empty\n+\n+ut_assert_skipline()\n+ Assert that there is a next console output line, and skip it\n+\n+ut_assert_skip_to_line(fmt, args...)\n+ Skip console output until a matching line is found\n+\n+ut_assert_skip_to_linen(fmt, args...)\n+ Skip console output until a partial match is found (compares up to the\n+ format-string length)\n+\n+ut_assert_console_end()\n+ Assert that there is no more console output\n+\n+ut_assert_nextlines_are_dump(total_bytes)\n+ Assert that the next lines are a print_buffer() hex dump of the specified\n+ size\n+\n+Memory helpers\n+~~~~~~~~~~~~~~\n+\n+These help check for memory leaks:\n+\n+ut_check_free()\n+ Return the number of bytes free in the malloc() pool\n+\n+ut_check_delta(last)\n+ Return the change in free memory since ``last`` was obtained from\n+ ``ut_check_free()``. A positive value means more memory has been allocated.\n+\n+Private buffer\n+~~~~~~~~~~~~~~\n+\n+Each test has access to a private buffer ``uts->priv`` (256 bytes) for temporary\n+data. This avoids the need to allocate memory or use global variables::\n+\n+ static int my_test(struct unit_test_state *uts)\n+ {\n+ snprintf(uts->priv, sizeof(uts->priv), \"/%s\", filename);\n+ /* use uts->priv as a path string */\n+\n+ return 0;\n+ }\n+\n+\n Writing Python tests\n --------------------\n \ndiff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst\nindex d8c3cbf496c..a26ee6ad7de 100644\n--- a/doc/usage/cmd/ut.rst\n+++ b/doc/usage/cmd/ut.rst\n@@ -11,7 +11,7 @@ Synopsis\n \n ::\n \n- ut [-r] [-f] [-I:] [ | all []] [...]\n+ ut [-r] [-f] [-R] [-I:] [ | all []] [...]\n ut [-s] info\n \n Description\n@@ -37,10 +37,17 @@ test\n causes another test to fail. If the one test fails, testing stops\n immediately.\n \n+-R\n+ Preserve console recording on test failure. Normally when a test fails,\n+ console recording is disabled so error messages go directly to output.\n+ This flag keeps recording enabled, which is useful when testing the test\n+ framework itself.\n+\n args\n Optional arguments to pass to the test, in `name=value` format. These are\n used by tests declared with `UNIT_TEST_ARGS()` which define expected\n- argument names and types.\n+ argument names and types. See :ref:`develop/tests_writing:passing arguments\n+ to c tests` for details.\n \n Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it\n includes a near-complete set of emulators, no code-size limits, many CONFIG\n","prefixes":["10/12"]}