From patchwork Tue Jan 17 17:47:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1727610 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.a=rsa-sha256 header.s=google header.b=lZSUoIHy; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NxGb85YXwz23fT for ; Wed, 18 Jan 2023 04:48:36 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 89D8B82FE7; Tue, 17 Jan 2023 18:48:32 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="lZSUoIHy"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id AE83280F98; Tue, 17 Jan 2023 18:48:31 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 Received: from mail-il1-x135.google.com (mail-il1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id A4CF184231 for ; Tue, 17 Jan 2023 18:48:28 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-il1-x135.google.com with SMTP id h26so15737236ila.11 for ; Tue, 17 Jan 2023 09:48:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=pzkXl8esqPV7MJe/JqLt7jWPSXzoPSJCsmjbcMo2V04=; b=lZSUoIHyPfauv9GK9fB9a6QG4T0lcJl5dl3+JgRvWgbBMlXMG2wgNKTFYWvzFWRzxH eeTWK4F4v73zPygmYJ4fChTptBLME0mnMy9DaJ46fLVLFviq5O8MRQI1owLqp48l6BdN WCDDTnRpiooU1lOMwKpptbEUhbGR+V11Ra/3o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pzkXl8esqPV7MJe/JqLt7jWPSXzoPSJCsmjbcMo2V04=; b=71Fc5tU++H99Mn5jxKcKYqy51prIw2p/6JEEN+YCJ3s4VvHf1x0NGlbgl5IbLwBsS2 pmVL9AoSdzlSPcaCeuVVQLwR1OQaQaBGXR7NrCcDIxr0Ullpr5XKwmXGsWY0cy8oQpLC HJaV/GLxATArVyObwdoP54JGi1GZ/ioIyRGvfllswnBlg8ete5q3NKTXOp35U+csl1iW 0kMx9+rIorxxuBRDv7dLROVI7hkH+TtwxaOZprSTn0SVgGhtSWACfq60IVjKosHcgWgP bbtpObGj5GidqTHhJm4o6xd6n3LSw1Gzho+whxAOJh6rdL//wjmxt6ibpqLcPUm7S4RY hiIA== X-Gm-Message-State: AFqh2kqpAUUyYY9nw80M3VDI5JoEyTtvV1a99fZqeoNQR7iF0hoTQIax Ak+GSk+Hbu1/JTzDHlRMLJPDGd+fP2Sy/5mS X-Google-Smtp-Source: AMrXdXt9UAABn99FcA5MVj07l1UnLu9A3Xmntm1SyE7uk1afsRox1RbgoUNRb3On0iYZpWzAOKGxdw== X-Received: by 2002:a92:c5c6:0:b0:30d:8d5c:c030 with SMTP id s6-20020a92c5c6000000b0030d8d5cc030mr2904710ilt.27.1673977706466; Tue, 17 Jan 2023 09:48:26 -0800 (PST) Received: from kea.bld.corp.google.com ([2620:15c:183:200:28d1:224:8617:3704]) by smtp.gmail.com with ESMTPSA id c32-20020a023f60000000b0038a01eba60fsm9631668jaf.69.2023.01.17.09.48.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Jan 2023 09:48:25 -0800 (PST) From: Simon Glass To: u-boot@lists.denx.de Cc: Simon Glass , Jaehoon Chung , Joe Hershberger , Kever Yang , Marek Vasut , Peng Fan , Philipp Tomsich Subject: [PATCH v3 00/70] bootstd: Allow migration from distro_bootcmd scripts Date: Tue, 17 Jan 2023 10:47:10 -0700 Message-Id: <20230117174820.1281477-1-sjg@chromium.org> X-Mailer: git-send-email 2.39.0.246.g2a6d74b583-goog MIME-Version: 1.0 X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.6 at phobos.denx.de X-Virus-Status: Clean So far, standard boot does not replicate all the of the functionality of the distro_bootcmd scripts. In particular it lacks some bootdevs and some of the bootmeths are incomplete. Also there is currently no internal mechanism to enumerate buses in order to discover bootdevs, e.g. with USB. This series addresses these shortcomings: - Adds the concept of a 'bootdev hunter' to enumerate buses, etc. in an effort to find bootdevs of a certain priority - Adds bootdevs for SCSI, IDE, NVMe, virtio, SPI flash - Handles PXE and DHCP properly - Supports reading the device tree with EFI and reading scripts from the network It also tidies up label processing, so it is possible to use: bootflow scan mmc2 to scan just one MMC device (with BOOTSTD_FULL). As before this implementation still relies on CONFIG_CMDLINE being enabled, mostly for the network stack. Further work would be required to disentangle that. Quite a few tests are added but there are some gaps: - SPI flash bootdev - EFI FDT loading Note that SATA works via SCSI (CONFIG_SCSI_AHCI) and does not use driver model. Only pogo_v4 seems to be affected. Probably all thats is needed is to call bootdev_setup_sibling_blk() in the Marvell SATA driver. Also, while it would be possible to init MMC in a bootdev hunter, there is no point since U-Boot always inits MMC on startup, if present. With this series it should be possible to migrate boards to standard boot by removing the inclusion of config_distro_bootcmd.h and instead adding a suitable value for boot_targets to the environment, e.g.: boot_targets=mmc1 mmc0 nvme scsi usb pxe dhcp spi Thus it is possible to boot automatically without scripts and boards can use a text-based environment instead of the config.h files. To demonstrate this, rockpro64-rk3399 is migrated to standard boot in this series. Full migration could probably be automated using a script, similar in concept to moveconfig: - obtain the board environment via 'make u-boot-initial-env' - get the value of "boot_targets" - drop config_distro_bootcmd.h from the config.h file - rebuild again to get the environment without distro scripts - write the environment (adding boot_targets) to board.env - remove CONFIG_EXTRA_ENV_SETTINGS from the config.h file This series is based on top of the boot menu series v3 [1]. The tree is available at u-boot-dm/dis-working [1] https://patchwork.ozlabs.org/project/uboot/list/?series=335364 Changes in v3: - Rebase to -master Changes in v2: - Rebase to -next Simon Glass (70): dm: core: Correct ordering of uclasses IDs dm: core: Support sorting devices with dm tree dm: test: Correct assertion in dm_test_part() lib: Add a function to split a string into substrings bootstd: Remove special-case code for boot_targets bootstd: Simplify locating existing bootdevs test: Fix the help for the ut command test: Drop duplicate restore of DM state sandbox: mmc: Start off with a zeroed file vbe: Avoid a build failure when bloblist is not enabled vbe: sandbox: Drop VBE node in chosen dm: part: Update test to use mmc2 dm: test: Correct ordering of DM setup ide: Drop non-DM code for BLK dm: mmc: Use bootdev_setup_sibling_blk() bootstd: Add a default method to get bootflows sandbox: Allow ethernet to be disabled at runtime sandbox: Allow ethernet bootdevs to be disabled for tests sandbox: Enable the Ethernet bootdev lib: Support printing an error string event: Correct duplicate log message in event_notify() efi: Improve logging in efi_disk bootstd: Add the concept of a bootdev hunter bootstd: Support running bootdev hunters dm: usb: Mark the device name as alloced when binding test: Add a generic function to skip delays bootstd: Add a USB hunter bootstd: Add an MMC hunter net: Add a function to run dhcp bootstd: Add a hunter for ethernet part: Add a function to find the first bootable partition bootstd: Only scan bootable partitions scsi: Correct allocation of block-device name scsi: Remove all children of SCSI devices before rescanning bootstd: Add a SCSI bootdev bootstd: Add an IDE bootdev bootstd: Add an NVMe bootdev virtio: Avoid repeating a long expression virtio: Fix returning -ENODEV virtio: Avoid strange behaviour on removal virtio: Add a block device bootstd: Add a virtio bootdev ata: Don't try to use non-existent ports bootstd: Rename bootdev checkers bootstd: Allow reading an EFI file from the network bootstd: Include the device tree in the bootflow bootstd: Support reading the device tree with EFI bootstd: Set the distro_bootpart env var with scripts bootstd: Update docs on bootmeth_try_file() for sandbox bootstd: Move label parsing into its own function bootstd: Add a new bootmeth method to set the bootflow sandbox: Allow SPI flash bootdevs to be disabled for tests bootstd: Add a SPI flash bootdev bootstd: Support reading a script from network or SPI flash bootstd: Treat DHCP and PXE as bootdev labels bootstd: Use hunters when scanning for bootflows bootstd: Allow hunting for bootdevs of a given priority bootstd: Add a new pre-scan priority for bootdevs bootstd: Allow hunting for a bootdev by label bootstd: Allow iterating to the next label in a list bootstd: Allow iterating to the next bootdev priortiy extension: Refactor to allow non-command usage bootstd: Add a hunter for the extension feature bootstd: Switch bootdev scanning to use labels bootstd: Allow scanning a single bootdev label bootstd: Drop the old bootflow_scan_first() bootstd: Record the bootdevs used during scanning bootstd: Add a little more logging of bootflows bootstd: Update documentation for new features rockchip: Convert rockpro64-rk3399 to use standard boot arch/sandbox/cpu/state.c | 30 ++ arch/sandbox/dts/sandbox.dtsi | 13 - arch/sandbox/dts/test.dts | 6 + arch/sandbox/include/asm/state.h | 2 + arch/sandbox/include/asm/test.h | 30 ++ boot/bootdev-uclass.c | 551 +++++++++++++++++++--------- boot/bootflow.c | 173 ++++++--- boot/bootmeth-uclass.c | 11 + boot/bootmeth_distro.c | 2 +- boot/bootmeth_efi.c | 220 ++++++++++- boot/bootmeth_efi_mgr.c | 2 +- boot/bootmeth_pxe.c | 5 +- boot/bootmeth_script.c | 98 ++++- boot/bootstd-uclass.c | 17 +- boot/vbe_simple_fw.c | 2 +- cmd/bootdev.c | 42 ++- cmd/bootflow.c | 97 +++-- cmd/dm.c | 10 +- cmd/extension_board.c | 43 ++- cmd/net.c | 35 ++ cmd/vbe.c | 7 +- common/event.c | 2 +- configs/sandbox_defconfig | 2 +- configs/sandbox_flattree_defconfig | 10 +- disk/part.c | 16 + doc/develop/bootstd.rst | 221 +++++++---- doc/develop/driver-model/nvme.rst | 2 +- doc/usage/cmd/bootdev.rst | 48 ++- doc/usage/cmd/bootflow.rst | 17 +- doc/usage/cmd/dm.rst | 5 +- drivers/ata/ahci.c | 6 + drivers/block/ide.c | 86 ++--- drivers/core/dump.c | 65 +++- drivers/mmc/mmc-uclass.c | 2 +- drivers/mmc/mmc_bootdev.c | 33 +- drivers/mmc/sandbox_mmc.c | 2 +- drivers/mtd/spi/Kconfig | 8 + drivers/mtd/spi/Makefile | 1 + drivers/mtd/spi/sf-uclass.c | 11 + drivers/mtd/spi/sf_bootdev.c | 82 +++++ drivers/nvme/nvme-uclass.c | 54 +++ drivers/nvme/nvme.c | 5 + drivers/scsi/Makefile | 7 + drivers/scsi/scsi.c | 32 +- drivers/scsi/scsi_bootdev.c | 62 ++++ drivers/usb/host/usb-uclass.c | 2 + drivers/usb/host/usb_bootdev.c | 38 +- drivers/virtio/virtio-uclass.c | 62 +++- drivers/virtio/virtio_sandbox.c | 16 +- include/bootdev.h | 206 ++++++++++- include/bootflow.h | 134 +++++-- include/bootmeth.h | 35 +- include/bootstd.h | 17 +- include/configs/rk3399_common.h | 5 +- include/configs/rockchip-common.h | 2 + include/dm/uclass-id.h | 4 +- include/dm/util.h | 8 +- include/net.h | 31 ++ include/part.h | 8 + include/test/test.h | 72 ++++ include/vsprintf.h | 24 ++ lib/efi_loader/efi_disk.c | 30 +- lib/strto.c | 41 +++ lib/vsprintf.c | 12 + net/eth-uclass.c | 14 +- net/eth_bootdev.c | 68 ++-- net/net.c | 4 + test/boot/bootdev.c | 563 +++++++++++++++++++++++++++-- test/boot/bootflow.c | 138 +++++-- test/boot/bootstd_common.c | 19 + test/boot/bootstd_common.h | 13 + test/cmd_ut.c | 2 +- test/dm/part.c | 37 +- test/dm/virtio_device.c | 3 + test/py/tests/bootstd/mmc1.img.xz | Bin 4448 -> 4480 bytes test/py/tests/test_dm.py | 38 ++ test/py/tests/test_ut.py | 13 +- test/str_ut.c | 82 +++++ test/test-main.c | 24 +- 79 files changed, 3225 insertions(+), 715 deletions(-) create mode 100644 drivers/mtd/spi/sf_bootdev.c create mode 100644 drivers/scsi/scsi_bootdev.c