{"id":2222488,"url":"http://patchwork.ozlabs.org/api/1.2/covers/2222488/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/cover/20260412111958.943933-1-sjg@chromium.org/","project":{"id":18,"url":"http://patchwork.ozlabs.org/api/1.2/projects/18/?format=json","name":"U-Boot","link_name":"uboot","list_id":"u-boot.lists.denx.de","list_email":"u-boot@lists.denx.de","web_url":null,"scm_url":null,"webscm_url":null,"list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20260412111958.943933-1-sjg@chromium.org>","list_archive_url":null,"date":"2026-04-12T11:19:37","name":"[v2,00/12] test: Add support for passing arguments to C unit tests","submitter":{"id":6170,"url":"http://patchwork.ozlabs.org/api/1.2/people/6170/?format=json","name":"Simon Glass","email":"sjg@chromium.org"},"mbox":"http://patchwork.ozlabs.org/project/uboot/cover/20260412111958.943933-1-sjg@chromium.org/mbox/","series":[{"id":499596,"url":"http://patchwork.ozlabs.org/api/1.2/series/499596/?format=json","web_url":"http://patchwork.ozlabs.org/project/uboot/list/?series=499596","date":"2026-04-12T11:19:37","name":"test: Add support for passing arguments to C unit tests","version":2,"mbox":"http://patchwork.ozlabs.org/series/499596/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/covers/2222488/comments/","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256\n header.s=google header.b=GN9LRA3p;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org\n header.b=\"GN9LRA3p\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=sjg@chromium.org"],"Received":["from phobos.denx.de (phobos.denx.de\n [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4ftp6W6gbXz1xtJ\n\tfor ; Sun, 12 Apr 2026 21:22:27 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 8957C839D5;\n\tSun, 12 Apr 2026 13:22:25 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id 5900583DC9; Sun, 12 Apr 2026 13:22:24 +0200 (CEST)","from mail-oa1-x43.google.com (mail-oa1-x43.google.com\n [IPv6:2001:4860:4864:20::43])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id D98B18341A\n for ; Sun, 12 Apr 2026 13:22:21 +0200 (CEST)","by mail-oa1-x43.google.com with SMTP id\n 586e51a60fabf-40f1a1f77a6so2333843fac.2\n for ; Sun, 12 Apr 2026 04:22:21 -0700 (PDT)","from chromium.org ([73.34.74.121]) by smtp.gmail.com with ESMTPSA id\n 586e51a60fabf-423dd395606sm6622071fac.1.2026.04.12.04.22.16\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Sun, 12 Apr 2026 04:22:17 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,\n DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,\n RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham\n autolearn_force=no version=3.4.2","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=chromium.org; s=google; t=1775992939; x=1776597739; darn=lists.denx.de;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:from:to:cc:subject:date:message-id:reply-to;\n bh=bHlsgf3XvUT/3FwiR/V9/5qtSqMlTiQOgdPR3vheRsM=;\n b=GN9LRA3pV6TOLcMxeTAh6GcQTkg89CgnC+nS9UynSVyLLlOxiFJfaiHfDpex5UPubd\n Isp4vfzsXZcouiTrTTkI+Mjl67NoCnlteDVSwMSss82f4kvRCM379omnnQdZ9tR+qhXr\n 2LhMIkq00hbjIQT5UYRMfMJa4YGgNZx0+oS6w=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1775992939; x=1776597739;\n h=content-transfer-encoding:mime-version:message-id:date:subject:cc\n :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=bHlsgf3XvUT/3FwiR/V9/5qtSqMlTiQOgdPR3vheRsM=;\n b=sO+Re8bNNqG9M65Rb8EOioJyjhJ/BEW0cnd67Dg/ZdA3Vl75n/321WzX5SzxWpEofh\n NPpVdN/kblPVaPsgHZ8BTalk+kXX7B+RjuAMV177BRyoZ6zFdmhcHsC8VikVhCVmlP76\n k8UASE4OPC+r/fuuH/doe40yd/1qV0kS9t8tnBjWJCcmntU8i2uuwJJmi61Dk8Dbtyh1\n r0NZh4YlUwBe7fMQangiW4psGuEaY4nMsFnUmRDJ1p3Nbic/BnvRv4vcdxwZOfA4BKD3\n TPIWMrfrptfRWJgOq8zPNIQVi3aCu6GBXAnfIwrEg34Hup76oG74Fm+79anKydFXXKqf\n fJmw==","X-Gm-Message-State":"AOJu0YwlTh1AqjxuEcP82LEDFRMmCh+SlPv52/QJ1XhIkJJcI5jTQKpK\n ecAZ0ReJ4lTTqsPbAMmUH0yzDvfTDIgAwYkwkkNxdLowkvfhJcG3jaKjfcWMBJdZg2i894qbQVI\n jFF3Vuwqf","X-Gm-Gg":"AeBDiesNvS3XtEiUaoWvkEpijCzB1EVlpI5UHxwBP5/rQc5aKnbwzdng5g6n9SM/im4\n OdVI79+y28qv9xu+ylwlYsTzJerdaLdI5x9r14jkz2u02uUNExNp/q2MeyUKHlscRgBsemhuHoq\n d2dzxcUNVFT6YBxWvGcps6rXdrmQdKSs7Il3xQQxYtB+lwQpcienCj16XbTgIiN5FcjR0lI0uwY\n +Vw0CnEvK7xCh9SlZsMCfX7se82Cl6vZANiv3p7Ny2H2L6TihOa2lW6FSKjacpIwmHqv2YCnvZZ\n Mk0pfeX8xSBLJzQXlBekqDAL7PZ9H1HZ9YE6bnQJcro7WWUhP6cBnIBYRq+Wb2bVHcViTxMAXw3\n hOuA77RnTDGyYcpBsE5rnoJNVYkMU7Zy5L1dhFIyJ1FzzYW2QcDG6q4cIg7z8UgSyNXdGDJqYCS\n fU7R4hjWbi0WwDaX20cg==","X-Received":"by 2002:a05:6871:430b:b0:417:2b13:f2cd with SMTP id\n 586e51a60fabf-423e0e16ed8mr5432358fac.10.1775992939378;\n Sun, 12 Apr 2026 04:22:19 -0700 (PDT)","From":"Simon Glass ","To":"u-boot@lists.denx.de","Cc":"Tom Rini , Heinrich Schuchardt ,\n Simon Glass ,\n Andrew Goodbody ,\n Ilias Apalodimas , Marek Vasut ","Subject":"[PATCH v2 00/12] test: Add support for passing arguments to C unit\n tests","Date":"Sun, 12 Apr 2026 05:19:37 -0600","Message-ID":"<20260412111958.943933-1-sjg@chromium.org>","X-Mailer":"git-send-email 2.43.0","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"},"content":"Some tests have two parts: a Python test which handles setup, such as\ncreating filesystem images or configuring the environment, and a C\ntest which does the bulk of the work including assertions. At present\nthere is no way to pass information between these two parts, so\nthings like paths and device names must be hard-coded in the C test.\n\nThis tends to make people write tests in Python, but this is not ideal\nfor several reasons:\n\n- Python lacks access to things like return codes and has to scan the\n console output to figure out what happened\n- Python lacks access to internal state, so it cannot directly check\n the result of an operation\n- Python is ~100x slower due to console interaction, etc.\n- C tests can be stepped through in gdb, which is much harder with\n Python driving the console\n- C tests can exercise internal APIs directly (e.g. fs_read(),\n fs_size()) rather than going through the command layer, so they\n test more precisely what they intend to\n\nThis series adds infrastructure for passing typed, runtime arguments\nfrom Python to C unit tests, using a name=value format on the ut\ncommand line.\n\nThe series includes:\n\n- Regex assertion helpers for matching variable output in tests\n\n- A private buffer in unit_test_state for test-local temporary data\n\n- Argument-type definitions (string, integer, boolean) with optional\n flags and default values\n\n- A new UNIT_TEST_ARGS() macro for declaring tests with typed\n arguments\n\n- Argument parsing in the test framework, accepting name=value format\n\n- Updates to the ut command to pass arguments through to tests\n\n- Type-checked accessor macros ut_str(), ut_int(), and ut_bool()\n with bounds validation\n\n- Tests for the argument feature covering type checking, optional\n arguments, and argument-parsing failures\n\n- Documentation for the test-parameter feature\n\n- C-based filesystem tests as an example of the hybrid approach,\n with Python wrappers that pass filesystem type, image path, and\n expected MD5 values to the C tests\n\nNote: This series depends on:\n\n https://patchwork.ozlabs.org/project/uboot/list/?series=496972\n\nChanges in v2:\n- Correct double signoff\n\nSimon Glass (12):\n test: Add ut_asserteq_regex() for regex pattern matching\n test: Add a helper to check the next line against a regex\n test: Add a private buffer for tests\n test: Add argument-type definitions\n test: Add a macro to declare unit tests with arguments\n test: Add support for passing arguments to C tests\n test: Enhance the ut command to pass test arguments\n test: Add type-checked argument accessor functions\n test: Add tests for unit-test arguments\n test: Add documentation for the test framework\n test: fs: add C-based filesystem tests\n test: fs: Update Python tests to call C implementations\n\n arch/sandbox/cpu/spl.c | 3 +-\n doc/develop/tests_writing.rst | 169 ++++++++++++\n doc/usage/cmd/ut.rst | 18 +-\n include/test/fs.h | 39 +++\n include/test/test.h | 107 ++++++++\n include/test/ut.h | 123 ++++++++-\n test/Makefile | 1 +\n test/cmd_ut.c | 32 ++-\n test/common/Makefile | 1 +\n test/common/test_args.c | 176 ++++++++++++\n test/fs/Makefile | 3 +\n test/fs/fs_basic.c | 407 ++++++++++++++++++++++++++++\n test/py/tests/test_fs/conftest.py | 4 +-\n test/py/tests/test_fs/test_basic.py | 346 +++++++----------------\n test/test-main.c | 170 +++++++++++-\n test/ut.c | 117 ++++++++\n 16 files changed, 1437 insertions(+), 279 deletions(-)\n create mode 100644 include/test/fs.h\n create mode 100644 test/common/test_args.c\n create mode 100644 test/fs/Makefile\n create mode 100644 test/fs/fs_basic.c"}