[{"id":3676824,"web_url":"http://patchwork.ozlabs.org/comment/3676824/","msgid":"<20260413152954.GW41863@bill-the-cat>","list_archive_url":null,"date":"2026-04-13T15:29:54","subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","submitter":{"id":65875,"url":"http://patchwork.ozlabs.org/api/people/65875/","name":"Tom Rini","email":"trini@konsulko.com"},"content":"On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:\n\n> Some tests have two parts: a Python test which handles setup, such as\n> creating filesystem images or configuring the environment, and a C\n> test which does the bulk of the work including assertions. At present\n> there is no way to pass information between these two parts, so\n> things like paths and device names must be hard-coded in the C test.\n> \n> This tends to make people write tests in Python, but this is not ideal\n> for several reasons:\n> \n> - Python lacks access to things like return codes and has to scan the\n> console output to figure out what happened\n> - Python lacks access to internal state, so it cannot directly check\n> the result of an operation\n> - Python is ~100x slower due to console interaction, etc.\n> - C tests can be stepped through in gdb, which is much harder with\n> Python driving the console\n> - C tests can exercise internal APIs directly (e.g. fs_read(),\n> fs_size()) rather than going through the command layer, so they\n> test more precisely what they intend to\n> \n> This series adds infrastructure for passing typed, runtime arguments\n> from Python to C unit tests, using a name=value format on the ut\n> command line.\n> \n> The series includes:\n> \n> - Regex assertion helpers for matching variable output in tests\n> \n> - A private buffer in unit_test_state for test-local temporary data\n> \n> - Argument-type definitions (string, integer, boolean) with optional\n> flags and default values\n> \n> - A new UNIT_TEST_ARGS() macro for declaring tests with typed\n> arguments\n> \n> - Argument parsing in the test framework, accepting name=value format\n> \n> - Updates to the ut command to pass arguments through to tests\n> \n> - Type-checked accessor macros ut_str(), ut_int(), and ut_bool()\n> with bounds validation\n> \n> - Tests for the argument feature covering type checking, optional\n> arguments, and argument-parsing failures\n> \n> - Documentation for the test-parameter feature\n> \n> - C-based filesystem tests as an example of the hybrid approach,\n> with Python wrappers that pass filesystem type, image path, and\n> expected MD5 values to the C tests\n> \n> Note: This series depends on:\n> \n> https://patchwork.ozlabs.org/project/uboot/list/?series=496972\n> \n> Changes in v2:\n> - Correct double signoff\n\nReposting a series this large, on the weekend, the day after v1, because\nyou made a trivial mistake is bad form. You should just acknowledge that\nin the cover letter and wait for feedback.\n\nAnd since I'm seeing over 100 emails from you this weekend, this\nmatters.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=konsulko.com header.i=@konsulko.com header.a=rsa-sha256\n header.s=google header.b=tEICVqR9;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=konsulko.com","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=konsulko.com header.i=@konsulko.com\n header.b=\"tEICVqR9\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=konsulko.com","phobos.denx.de;\n spf=pass smtp.mailfrom=trini@konsulko.com"],"Received":["from phobos.denx.de (phobos.denx.de\n [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fvWYw224Wz1yDG\n\tfor ; Tue, 14 Apr 2026 01:30:12 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 8959A839D5;\n\tMon, 13 Apr 2026 17:30:02 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id 67319836AC; Mon, 13 Apr 2026 17:30:01 +0200 (CEST)","from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com\n [IPv6:2607:f8b0:4864:20::32a])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id C5742836AC\n for ; Mon, 13 Apr 2026 17:29:58 +0200 (CEST)","by mail-ot1-x32a.google.com with SMTP id\n 46e09a7af769-7dbce74e537so3544394a34.1\n for ; Mon, 13 Apr 2026 08:29:58 -0700 (PDT)","from bill-the-cat (fixed-189-203-106-235.totalplay.net.\n [189.203.106.235]) by smtp.gmail.com with ESMTPSA id\n 006d021491bc7-68bc7dfa174sm5828927eaf.3.2026.04.13.08.29.56\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Mon, 13 Apr 2026 08:29:56 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,\n DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED,\n SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.2","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=konsulko.com; s=google; t=1776094197; x=1776698997; darn=lists.denx.de;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;\n bh=+BeenR5TzBs/pG3bH9LsLqRy7CapMpUwJBMln5DyK/w=;\n b=tEICVqR9qGCmoycJtYt1woWEPR2pWP35OzxgkovFuYzbW4BM+c4yr6xjyRSNlYV2sv\n lYTmASExDaJ0G3vOEwZfbmCpx6Wrh3PGSsX325+ISVtehuTBnovPieCu7B7rSIdJwbyx\n zygZNh6mc7JbcKeC9sBJOsMrS7x2YllG9sk4k=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776094197; x=1776698997;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=+BeenR5TzBs/pG3bH9LsLqRy7CapMpUwJBMln5DyK/w=;\n b=DMGcess+/d6EDdCnUIKS7fOVTXSjZppw4XifzYE35uiELRopSFPH20Uh6A0w8HtJLz\n i3GRPlzm2C6BkUv/sPwf9qKVo43u2wD0iZrOJFOGwD5RyOQEyEYgkXN2U8EMAapCojhb\n f/sct7FqEpeg9morZMldnaCfviE9uXB62+ChhIqDUrgVO3rj64gzcJsM7BDktrlCwoof\n hazLXEMRSvBV4d3xuVzILJFMpIt5jTtSog6vbSYFejOnt+cOATgSnVo+7EiafR/9I/XK\n hE5H9vEggPlk0I+ra11RB2dj4HLVS3ba1WIL13whRQPPbrBEtNrEnDDbCp5Yyb69PFWx\n d/Tw==","X-Gm-Message-State":"AOJu0YyUkcYa0JMoe/RuVHuUFaeKeccZumfKgO21lKwdUkp+bZ5U2RNB\n hFfdCnVx93p5tPQYqbm/bHOvKlvy4NfYR2ATSXYalBBfnaDZYmCH7Pp/5MNBhi+i7j8=","X-Gm-Gg":"AeBDies54R+tJNVwIvuQ0HK5MljLg0/9Ffk+4B6vVlfOvMY+m3ikrpkZyCjOp3vz+OH\n qlRnS472CqXE2Rmz+DovCGUdjt4NjY2RbkeeZd2lole+WBVHLrCbeu2PBTI5gws/sPWCcaUsBRy\n 6DhQ8Mjngk+Z2XxeV+cau/4qIpuYLvKYnya9KpEjnDLBVLQWGSBRsvedhCtzYFFrh/Far5FS3IX\n y5QlFczRMEjfviJRRYt7A5tdTZkCcoAORxASyzTMnpFrOiOx0gpK8snBEW4kj+tvzgl6qOGW58L\n cRVW3YNhGRoCmz830KjIQrQFG0/n54lBzWyTNqpZGHtErnnabX5T1ZzpCPLqOynoeBvNauYCT4H\n 5tTNG+rNTbnqCg9j81MurkwuhtW4dy5KswZooDdey1pa//ooYPy/UCwiCJfD0kUo2jfdIV62FQb\n RwxnkL+toIXXGUe2csiT8/FCTCR4Ij5ccF/qnyOROJSJhaVA/b/VuCG7BaQI98/QJOMYzfxRSCC\n dG+ddnvK+Y2Id5iSXa+pa9cmcjNdZwEMd21jkkHFp5GZdYLVeY=","X-Received":"by 2002:a05:6820:3397:10b0:67d:ccf6:13f1 with SMTP id\n 006d021491bc7-68be7ee39f3mr4494344eaf.37.1776094197334;\n Mon, 13 Apr 2026 08:29:57 -0700 (PDT)","Date":"Mon, 13 Apr 2026 09:29:54 -0600","From":"Tom Rini ","To":"Simon Glass ","Cc":"u-boot@lists.denx.de, Heinrich Schuchardt ,\n Andrew Goodbody ,\n Ilias Apalodimas , Marek Vasut ","Subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","Message-ID":"<20260413152954.GW41863@bill-the-cat>","References":"<20260412111958.943933-1-sjg@chromium.org>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha512;\n protocol=\"application/pgp-signature\"; boundary=\"rZtAOxUt+nr/UNtV\"","Content-Disposition":"inline","In-Reply-To":"<20260412111958.943933-1-sjg@chromium.org>","X-Clacks-Overhead":"GNU Terry Pratchett","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"}},{"id":3676832,"web_url":"http://patchwork.ozlabs.org/comment/3676832/","msgid":"","list_archive_url":null,"date":"2026-04-13T15:40:24","subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","submitter":{"id":6170,"url":"http://patchwork.ozlabs.org/api/people/6170/","name":"Simon Glass","email":"sjg@chromium.org"},"content":"Hi Tom,\n\nOn Mon, 13 Apr 2026 at 09:29, Tom Rini wrote:\n>\n> On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:\n>\n> > Some tests have two parts: a Python test which handles setup, such as\n> > creating filesystem images or configuring the environment, and a C\n> > test which does the bulk of the work including assertions. At present\n> > there is no way to pass information between these two parts, so\n> > things like paths and device names must be hard-coded in the C test.\n> >\n> > This tends to make people write tests in Python, but this is not ideal\n> > for several reasons:\n> >\n> > - Python lacks access to things like return codes and has to scan the\n> > console output to figure out what happened\n> > - Python lacks access to internal state, so it cannot directly check\n> > the result of an operation\n> > - Python is ~100x slower due to console interaction, etc.\n> > - C tests can be stepped through in gdb, which is much harder with\n> > Python driving the console\n> > - C tests can exercise internal APIs directly (e.g. fs_read(),\n> > fs_size()) rather than going through the command layer, so they\n> > test more precisely what they intend to\n> >\n> > This series adds infrastructure for passing typed, runtime arguments\n> > from Python to C unit tests, using a name=value format on the ut\n> > command line.\n> >\n> > The series includes:\n> >\n> > - Regex assertion helpers for matching variable output in tests\n> >\n> > - A private buffer in unit_test_state for test-local temporary data\n> >\n> > - Argument-type definitions (string, integer, boolean) with optional\n> > flags and default values\n> >\n> > - A new UNIT_TEST_ARGS() macro for declaring tests with typed\n> > arguments\n> >\n> > - Argument parsing in the test framework, accepting name=value format\n> >\n> > - Updates to the ut command to pass arguments through to tests\n> >\n> > - Type-checked accessor macros ut_str(), ut_int(), and ut_bool()\n> > with bounds validation\n> >\n> > - Tests for the argument feature covering type checking, optional\n> > arguments, and argument-parsing failures\n> >\n> > - Documentation for the test-parameter feature\n> >\n> > - C-based filesystem tests as an example of the hybrid approach,\n> > with Python wrappers that pass filesystem type, image path, and\n> > expected MD5 values to the C tests\n> >\n> > Note: This series depends on:\n> >\n> > https://patchwork.ozlabs.org/project/uboot/list/?series=496972\n> >\n> > Changes in v2:\n> > - Correct double signoff\n>\n> Reposting a series this large, on the weekend, the day after v1, because\n> you made a trivial mistake is bad form. You should just acknowledge that\n> in the cover letter and wait for feedback.\n\nOK I'll do that if I make this mistake again.\n\n> And since I'm seeing over 100 emails from you this weekend, this\n> matters.\n\nYes, I was trying to catch up on various things.\n\nRegards,\nSimon","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256\n header.s=google header.b=HVP0Gz3t;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=85.214.62.61; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=chromium.org header.i=@chromium.org\n header.b=\"HVP0Gz3t\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=chromium.org","phobos.denx.de;\n spf=pass smtp.mailfrom=sjg@chromium.org"],"Received":["from phobos.denx.de (phobos.denx.de [85.214.62.61])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fvWpB74Nwz1xtJ\n\tfor ; Tue, 14 Apr 2026 01:40:50 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 8F284839D5;\n\tMon, 13 Apr 2026 17:40:47 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id 1C86C839DF; Mon, 13 Apr 2026 17:40:47 +0200 (CEST)","from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com\n [IPv6:2607:f8b0:4864:20::62a])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id 8AB79836AC\n for ; Mon, 13 Apr 2026 17:40:44 +0200 (CEST)","by mail-pl1-x62a.google.com with SMTP id\n d9443c01a7336-2a8fba3f769so20384195ad.2\n for ; Mon, 13 Apr 2026 08:40:44 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,\n DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,\n RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS autolearn=ham\n autolearn_force=no version=3.4.2","ARC-Seal":"i=1; a=rsa-sha256; t=1776094843; cv=none;\n d=google.com; s=arc-20240605;\n b=NQ3fCdtSHF3vEzkwMfRfZu0pacxON73s2eedp9sgTKq3K8k6oMNHrDGN2gZy24OGSV\n AXiCBZlugT5AOcBg2EZQsC53oCAFIBdy7ubUEepM5H2yVYzfgY4aiF5DZ3rGiGmofYPt\n s9nDdL/lpY8LEYy03dTI+BwkPXvREiYOi3AAJ/iOLcTqxSAHw6IWMkaC8iXqQ/54Zig7\n nwk/aT3xBbw/oSvZIIyw3/pbgIALStyoD5OP0ykgoeUH7a4qVSF3Wklqlh+GSbBg2mAp\n AK6fZIKEd7qlfYhFqrRQdyrSlVJ9d7+c3R3b/sPZGGcfXDGodZ2OYrWqxjF29P+uyHAs\n HeCg==","ARC-Message-Signature":"i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\n s=arc-20240605;\n h=cc:to:subject:message-id:date:from:in-reply-to:references\n :mime-version:dkim-signature;\n bh=FXhGz2w9Q39HYftHfmJ6BsbrFstmBBX1vkGDqiinlW4=;\n fh=x8CHnSGD4cRa4Gb5OxW3DEd7ozMoYZsI49tbFC7/xcM=;\n b=ccmwm1j3DJ1jv4CSDXKzoVuSEDFj+z9fbm6/A3LqTIw5P2NpMzf4wucbXaWK0uIctS\n 0XWRAGMmq/ydGCllxuE7rg0D2U3RHp4TTYiX9rbOcfJ7wCV9svD19fRsnjZe3TnkvTy3\n gOtQRrR9+MnoMtK8NTaNF7CMYDsYxIavp2cecc+JkLI8zYNwKII2lB2403xNExJIH4iq\n JQCTKsdFDV/UwH7X94GJAzB6jm5GXsWnA4+ZhLacYCDroH2E9Z+JygflevnuRGn/BGKD\n Udb2s3taeRRvLAlhglTdDXhs5ekdkXLVsxM6pDHLfv7AVtVlxtadm1F3JureV/Y6q7zS\n Q0Kw==; darn=lists.denx.de","ARC-Authentication-Results":"i=1; mx.google.com; arc=none","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=chromium.org; s=google; t=1776094843; x=1776699643; darn=lists.denx.de;\n h=cc:to:subject:message-id:date:from:in-reply-to:references\n :mime-version:from:to:cc:subject:date:message-id:reply-to;\n bh=FXhGz2w9Q39HYftHfmJ6BsbrFstmBBX1vkGDqiinlW4=;\n b=HVP0Gz3td51V8SF32G+up0gQoBv93Lf7RKVtIUkB1Mo0Z2uEY5dGnrc24BmWLXaILA\n 8TGbvoE6FK5SW7zjD4N4cic1xcVQoJBM9CD6EEr1GVW6nB2CEV0pzEI9OJN0OR5oe2uc\n 0nvsqhXvqaYYaJY325seiB/FgCFC+Z9QPmeZ4=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776094843; x=1776699643;\n h=cc:to:subject:message-id:date:from:in-reply-to:references\n :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date\n :message-id:reply-to;\n bh=FXhGz2w9Q39HYftHfmJ6BsbrFstmBBX1vkGDqiinlW4=;\n b=cWYTdE3+ELVxheIP5SN8YVnYqyc8UMF3S55yD8VPGlCIw7hHIOAbcB83YqBnwami7h\n ST5WQt4XRJKets8jzjCSn+sp3MqyBvj/Mk4TQzODAK9TM//fiCQYQmFDiqN8+UsjXNE5\n ORofBKrPd2xsFmSm98fI5LO3JoHOMvBeXDBO8PfRpq9Ata8KDbMpv7ueuweArHU9MppK\n BKA3SKiq+YUNUQuQKEqyXCGi+tcDxBMBTJZ2ri3nUQLAZj/AUJke4SR/4AZ2zt/x44uO\n j/b1IvipYvTPYY3sEahR0aaLMJbiemOX+1gKBNW9XkO5zRm7y3dd0cjipmGrb38Fc5T5\n TqXw==","X-Gm-Message-State":"AOJu0YyEeDiREfP6bRMixHREwuMLOemSboNWzkCxYgK9kZibDA8vAPpE\n 2krsluz5Q5vgDki73sO3cOqoqDrXjpjzms5FYjn0+Jnv25yqIAcEC1/gPoBtBuXriTK8CLqjscd\n aHI+Uk6rUs3YztreoH/AVNVuXLL9DYAzR4XYUFUwP5jLtpjvZcjQ=","X-Gm-Gg":"AeBDiesHJkikDjSHnLgU06BnfueSUsEhdj1OHdhXGCpEjFgMGvowoku6n6xqVF9hB6N\n 9fUbJ1su5HwLoLcusfpunpYnD+ZcRTFF+wj+Qr9dGIXAYXE/g3LTsBxS6xfGKh5CY7Ge+DXyqCC\n 2u1JnGdkY8BkMasCQu6MCRNotgEFWDKHZQacC7K1necMVREGZ+SzsqBMWWLgtGgV5iFKET6VUTn\n rqIr0gRDzsybn3evJF44GXttvrC2Br93+9XAN6TgiHQ2SIzCKjriF+AFCpAfCDLfrWRdhTNQDI6\n 4keEjw==","X-Received":"by 2002:a17:903:1207:b0:2b2:5637:1480 with SMTP id\n d9443c01a7336-2b2d5a95dacmr139959885ad.40.1776094843029; Mon, 13 Apr 2026\n 08:40:43 -0700 (PDT)","MIME-Version":"1.0","References":"<20260412111958.943933-1-sjg@chromium.org>\n <20260413152954.GW41863@bill-the-cat>","In-Reply-To":"<20260413152954.GW41863@bill-the-cat>","From":"Simon Glass ","Date":"Mon, 13 Apr 2026 09:40:24 -0600","X-Gm-Features":"AQROBzAoNH6qaXK9RnHmzmByVFHPDmMgXbXfhxUcUfq6e-U-eG-we96qWpQxa3Y","Message-ID":"\n ","Subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","To":"Tom Rini ","Cc":"u-boot@lists.denx.de, Heinrich Schuchardt ,\n Andrew Goodbody ,\n Ilias Apalodimas ,\n Marek Vasut ","Content-Type":"text/plain; charset=\"UTF-8\"","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"}},{"id":3676909,"web_url":"http://patchwork.ozlabs.org/comment/3676909/","msgid":"<20260413201344.GF41863@bill-the-cat>","list_archive_url":null,"date":"2026-04-13T20:13:44","subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","submitter":{"id":65875,"url":"http://patchwork.ozlabs.org/api/people/65875/","name":"Tom Rini","email":"trini@konsulko.com"},"content":"On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:\n> Some tests have two parts: a Python test which handles setup, such as\n> creating filesystem images or configuring the environment, and a C\n> test which does the bulk of the work including assertions. At present\n> there is no way to pass information between these two parts, so\n> things like paths and device names must be hard-coded in the C test.\n> \n> This tends to make people write tests in Python, but this is not ideal\n> for several reasons:\n\nYou still need to work on explaining your changes in a positive manner.\n\n> - Python lacks access to things like return codes and has to scan the\n> console output to figure out what happened\n\nPro: Python exercises things the way users exercise them and so exposes\ndeficiencies in our user interface as well, when applicable.\n\n> - Python lacks access to internal state, so it cannot directly check\n> the result of an operation\n\nIt's unclear when that is, or is not a problem and when tests that do\ncheck intermediate state are providing value.\n\n> - Python is ~100x slower due to console interaction, etc.\n\nAgain, maybe you should go back to your suggestion of several years ago\nto see about addressing that, as the tests themselves are not slow, it's\nthe pipe? Our brave new AI powered world runs on Python handing stuff\noff and getting it back super quickly.\n\n> - C tests can be stepped through in gdb, which is much harder with\n> Python driving the console\n> - C tests can exercise internal APIs directly (e.g. fs_read(),\n> fs_size()) rather than going through the command layer, so they\n> test more precisely what they intend to\n\nAll things that may or may not be problems for other people, or are\nintentional choices as they again test the system the way it's used.\nUsing gdb attached to u-boot in the other terminal where you're running\nbreaking things sounds like a reasonable debug path to me. And having\nunit tests calling in to functions is why today we have a lot of things\nnot marked as \"static\" and so not being optimized as well as they could\nby the compiler, which is its own set of trade-offs and not an automatic\ngood thing and design win.\n\n[snip]\n> 16 files changed, 1437 insertions(+), 279 deletions(-)\n\nAll of which is to say, you need to explain yourself in a positive, not\nnegative or oppositional way. Otherwise this reads like \"I don't like\nwhat we do now, here's a bunch of change to what I like instead\".","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=konsulko.com header.i=@konsulko.com header.a=rsa-sha256\n header.s=google header.b=P+mFypT8;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de\n (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de;\n envelope-from=u-boot-bounces@lists.denx.de; receiver=patchwork.ozlabs.org)","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=konsulko.com","phobos.denx.de;\n spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de","phobos.denx.de;\n\tdkim=pass (1024-bit key;\n unprotected) header.d=konsulko.com header.i=@konsulko.com\n header.b=\"P+mFypT8\";\n\tdkim-atps=neutral","phobos.denx.de;\n dmarc=pass (p=none dis=none) header.from=konsulko.com","phobos.denx.de;\n spf=pass smtp.mailfrom=trini@konsulko.com"],"Received":["from phobos.denx.de (phobos.denx.de\n [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fvdsQ2FFQz1xtJ\n\tfor ; Tue, 14 Apr 2026 06:14:02 +1000 (AEST)","from h2850616.stratoserver.net (localhost [IPv6:::1])\n\tby phobos.denx.de (Postfix) with ESMTP id 0039483DC9;\n\tMon, 13 Apr 2026 22:13:53 +0200 (CEST)","by phobos.denx.de (Postfix, from userid 109)\n id D902584198; Mon, 13 Apr 2026 22:13:51 +0200 (CEST)","from mail-ot1-x333.google.com (mail-ot1-x333.google.com\n [IPv6:2607:f8b0:4864:20::333])\n (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits))\n (No client certificate requested)\n by phobos.denx.de (Postfix) with ESMTPS id A43A9839DF\n for ; Mon, 13 Apr 2026 22:13:49 +0200 (CEST)","by mail-ot1-x333.google.com with SMTP id\n 46e09a7af769-7d86eb7c854so2387034a34.3\n for ; Mon, 13 Apr 2026 13:13:49 -0700 (PDT)","from bill-the-cat (fixed-189-203-106-235.totalplay.net.\n [189.203.106.235]) by smtp.gmail.com with ESMTPSA id\n 46e09a7af769-7dc26907f82sm9895212a34.17.2026.04.13.13.13.46\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Mon, 13 Apr 2026 13:13:47 -0700 (PDT)"],"X-Spam-Checker-Version":"SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de","X-Spam-Level":"","X-Spam-Status":"No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,\n DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_BLOCKED,\n SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.2","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=konsulko.com; s=google; t=1776111228; x=1776716028; darn=lists.denx.de;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to;\n bh=FWlQ+PCbB7G6Set9qlJlaok8JKGXDb3YlFG9D93GQ9I=;\n b=P+mFypT8bgfeRY5ga93BB+Kjd00MSJutMpYAfuUl274LQeiwHuO4gh1M3seNNgbty6\n 8fYt5WRKzGzxz81VdD3r5QUq+LyUKDwzI2rhVUp8160K7ug/96zD4+aTjjE67FIuxiGm\n foAC1p06QUSnvFni+7AQLuGwBmNarjlatKj2Y=","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776111228; x=1776716028;\n h=in-reply-to:content-disposition:mime-version:references:message-id\n :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=FWlQ+PCbB7G6Set9qlJlaok8JKGXDb3YlFG9D93GQ9I=;\n b=LNQge+PHU9L9dvtILYIcq5J7A7hV2GLTggAnMDX4PEKG9srjDYIq7SW8r4zxwMmEbS\n Xs1YV7IxCFP/aLIr8dQhQ24HL7Qwme9xGDKO7V/5F12rJvhn3VFfL7iy+O8kiaDYCHS4\n MIUlzmi9UK+DSAS/HCAXXKrLSn/Ah76HxwAYADsXvWjTgHBp7i8sWZne2V1tB4P1hozw\n L6c7P8nYl2tklRkWoTdJsNyJtSWeYmLwDklLArryOvh+HfQpxMlP/INHo08I39kbfOJP\n RG2p2Ro0kQLEjnBrkt04Mztj/6QYCMyMe6Vi/5Jc6g9yNxMIyS309iDGlGOGNX3jpk1K\n Te1g==","X-Gm-Message-State":"AOJu0Yxz57ZYtClYh7sy6W4Wzl07tSPweaAuGmvrD9f9CucfEvpy+Fxy\n TxLB+KgIxk94sPUltSiARiuXJvqz8xt+a2t0Qj/rBnliMnJm2ncWxi3ZEDNGmiY60u4=","X-Gm-Gg":"AeBDies25BwUwA7y2wfshbrDpEshMYPuSumYIVYiIqUlySD1KgTWZm5Q4tHO4JF3pzI\n Yaua3s03BY7lws9k5751cCEv6oU0iOCSE6L78UxbLwK8uxvS/MERMAWezL5JTKycHwJJdLCVyk5\n lguVZk0Mn5msMg/btbMle9o7kRb/BX5Nu6JalvmJR0HdNTso0LSgJSmiVXJhDks9mvb+uasvFfi\n G2L3zNsvaHcdEduYDeQJSnR+QBn39V/D0+q1NQO13r7VkZryHXqbPEZNRsxilf14GPPDS/7X1VM\n oyG8DMYDsGt7IsmJ3yG5Jfqq/gEaQ6hZOJ7O08gtWJFgYY+h4HlRkMJI40/lnzg67kulmjI7Zp+\n iSB+ncjGIqdB8H7Az1HF+o5kfsZjYEnbj0A49OCu86XZID97hSDk37wNHh/LFVmdxQFXR3It1ap\n KtpxMyL4VkEs8Rx/HTIHqOE3h6qtM3lWJk9cKqcevmBY8mw0p946cNpIj8hJHuamSckZNa/voNz\n cEpsl+egRXt/1lxa6QST8EFv3M4a3tmRrChbpUH0aKcBWwaMCE=","X-Received":"by 2002:a05:6830:4ac1:b0:7d7:d0bf:7045 with SMTP id\n 46e09a7af769-7dc27c9c987mr9167379a34.8.1776111227996;\n Mon, 13 Apr 2026 13:13:47 -0700 (PDT)","Date":"Mon, 13 Apr 2026 14:13:44 -0600","From":"Tom Rini ","To":"Simon Glass ","Cc":"u-boot@lists.denx.de, Heinrich Schuchardt ,\n Andrew Goodbody ,\n Ilias Apalodimas , Marek Vasut ","Subject":"Re: [PATCH v2 00/12] test: Add support for passing arguments to C\n unit tests","Message-ID":"<20260413201344.GF41863@bill-the-cat>","References":"<20260412111958.943933-1-sjg@chromium.org>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha512;\n protocol=\"application/pgp-signature\"; boundary=\"JJlf/GrFNRJmZbm5\"","Content-Disposition":"inline","In-Reply-To":"<20260412111958.943933-1-sjg@chromium.org>","X-Clacks-Overhead":"GNU Terry Pratchett","X-BeenThere":"u-boot@lists.denx.de","X-Mailman-Version":"2.1.39","Precedence":"list","List-Id":"U-Boot discussion ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"u-boot-bounces@lists.denx.de","Sender":"\"U-Boot\" ","X-Virus-Scanned":"clamav-milter 0.103.8 at phobos.denx.de","X-Virus-Status":"Clean"}}]