[v3,30/31] bootstd: doc: Add documentation

Add documentation for this feature, including the commands and full
devicetree bindings.

Signed-off-by: Simon Glass <sjg@chromium.org>
---

Changes in v3:
- Update docs for "bootmeths" and "boot_targets" env vars

 MAINTAINERS                           |   4 +
 doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
 doc/develop/distro.rst                |   3 +
 doc/develop/index.rst                 |   1 +
 doc/device-tree-bindings/bootdev.txt  |  18 +
 doc/device-tree-bindings/bootmeth.txt |  31 ++
 doc/device-tree-bindings/bootstd.txt  |   8 +
 doc/usage/bootdev.rst                 | 135 ++++++
 doc/usage/bootflow.rst                | 427 +++++++++++++++++
 doc/usage/bootmeth.rst                | 108 +++++
 doc/usage/index.rst                   |   3 +
 11 files changed, 1376 insertions(+)
 create mode 100644 doc/develop/bootstd.rst
 create mode 100644 doc/device-tree-bindings/bootmeth.txt
 create mode 100644 doc/usage/bootdev.rst
 create mode 100644 doc/usage/bootflow.rst
 create mode 100644 doc/usage/bootmeth.rst

On 1/19/22 02:43, Simon Glass wrote:
> Add documentation for this feature, including the commands and full
> devicetree bindings.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> 
> Changes in v3:
> - Update docs for "bootmeths" and "boot_targets" env vars
> 
>   MAINTAINERS                           |   4 +
>   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
>   doc/develop/distro.rst                |   3 +
>   doc/develop/index.rst                 |   1 +
>   doc/device-tree-bindings/bootdev.txt  |  18 +
>   doc/device-tree-bindings/bootmeth.txt |  31 ++
>   doc/device-tree-bindings/bootstd.txt  |   8 +
>   doc/usage/bootdev.rst                 | 135 ++++++
>   doc/usage/bootflow.rst                | 427 +++++++++++++++++
>   doc/usage/bootmeth.rst                | 108 +++++
>   doc/usage/index.rst                   |   3 +
>   11 files changed, 1376 insertions(+)
>   create mode 100644 doc/develop/bootstd.rst
>   create mode 100644 doc/device-tree-bindings/bootmeth.txt
>   create mode 100644 doc/usage/bootdev.rst
>   create mode 100644 doc/usage/bootflow.rst
>   create mode 100644 doc/usage/bootmeth.rst
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8ad70d3d968..c2af8ada3c9 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -669,6 +669,10 @@ F:	boot/bootmeth*.c
>   F:	boot/bootstd.c
>   F:	cmd/bootdev.c
>   F:	cmd/bootflow.c
> +F:	doc/develop/bootstd.rst
> +F:	doc/usage/bootdev.rst
> +F:	doc/usage/bootflow.rst
> +F:	doc/usage/bootmeth.rst
>   F:	drivers/mmc/mmc_bootdev.c
>   F:	include/bootdev.h
>   F:	include/bootflow.h
> diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> new file mode 100644
> index 00000000000..1b65a806efb
> --- /dev/null
> +++ b/doc/develop/bootstd.rst
> @@ -0,0 +1,638 @@
> +.. SPDX-License-Identifier: GPL-2.0+:
> +
> +U-Boot Standard Boot
> +====================
> +
> +Introduction
> +------------
> +
> +Standard boot provides a built-in way for U-Boot to automatically boot
> +an Operating System without custom scripting and other customisation. It
> +introduces the following concepts:
> +
> +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> +   - bootflow - a description of how to boot (provided by the distro)
> +
> +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> +for creating a bootflow for each kernel combination that it wants to offer.

This gets it completely wrong. There is one standardized boot flow: 
UEFI. All major distros support this. U-Boot has to offer UEFI booting 
out of the box.

Only when using the legacy entry point of the Linux kernel - which is 
discouraged - you have to think about other boot flows.

Best regards

Heinrich

> +These bootflows are stored on media so they can be discovered by U-Boot. This
> +feature is typically called `distro boot` (see :doc:`distro`) because it is
> +a way for distributions to boot on any hardware.
> +
> +Traditionally U-Boot has relied on scripts to implement this feature. See
> +disto_boodcmd_ for details. This is done because U-Boot has no native support
> +for scanning devices. While the scripts work remarkably well, they can be hard
> +to understand and extend, and the feature does not include tests. They are also
> +making it difficult to move away from ad-hoc CONFIGs, since they are implemented
> +using the environment and a lot of #defines.
> +
> +Standard boot is a generalisation of distro boot. It provides a more built-in
> +way to boot with U-Boot. The feature is extensible to different Operating
> +Systems (such as Chromium OS) and devices (beyond just block and network
> +devices). It supports EFI boot and EFI bootmgr too.
> +
> +
> +Bootflow
> +--------
> +
> +A bootflow is a file that describes how to boot a distro. Conceptually there can
> +be different formats for that file but at present U-Boot only supports the
> +BootLoaderSpec_ format. which looks something like this::
> +
> +   menu autoboot Welcome to Fedora-Workstation-armhfp-31-1.9. Automatic boot in # second{,s}. Press a key for options.
> +   menu title Fedora-Workstation-armhfp-31-1.9 Boot Options.
> +   menu hidden
> +
> +   label Fedora-Workstation-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
> +       kernel /vmlinuz-5.3.7-301.fc31.armv7hl
> +       append ro root=UUID=9732b35b-4cd5-458b-9b91-80f7047e0b8a rhgb quiet LANG=en_US.UTF-8 cma=192MB cma=256MB
> +       fdtdir /dtb-5.3.7-301.fc31.armv7hl/
> +       initrd /initramfs-5.3.7-301.fc31.armv7hl.img
> +
> +As you can see it specifies a kernel, a ramdisk (initrd) and a directory from
> +which to load devicetree files. The details are described in disto_boodcmd_.
> +
> +The bootflow is provided by the distro. It is not part of U-Boot. U-Boot's job
> +is simply to interpret the file and carry out the instructions. This allows
> +distros to boot on essentially any device supported by U-Boot.
> +
> +Typically the first available bootflow is selected and booted. If that fails,
> +then the next one is tried.
> +
> +
> +Bootdev
> +-------
> +
> +Where does U-Boot find the media that holds the operating systems? That is the
> +job of bootdev. A bootdev is simply a layer on top of a media device (such as
> +MMC, NVMe). The bootdev accesses the device, including partitions and
> +filesystems that might contain things related to an operating system.
> +
> +For example, an MMC bootdev provides access to the individual partitions on the
> +MMC device. It scans through these to find filesystems, then provides a list of
> +these for consideration.
> +
> +
> +Bootmeth
> +--------
> +
> +Once the list of filesystems is provided, how does U-Boot find the bootflow
> +files in these filesystems. That is the job of bootmeth. Each boot method has
> +its own way of doing this.
> +
> +For example, the distro bootmeth simply looks through the provided filesystem
> +for a file called `extlinux/extlinux.conf`. This files constitutes a bootflow.
> +If the distro bootmeth is used on multiple partitions it may produce multiple
> +bootflows.
> +
> +Note: it is possible to have a bootmeth that uses a partition or a whole device
> +directly, but it is more common to use a filesystem.
> +
> +
> +Boot process
> +------------
> +
> +U-Boot tries to use the 'lazy init' approach whereever possible and distro boot
> +is no exception. The algorithm is::
> +
> +   while (get next bootdev)
> +      while (get next bootmeth)
> +          while (get next bootflow)
> +              try to boot it
> +
> +So U-Boot works its way through the bootdevs, trying each bootmeth in turn to
> +obtain bootflows, until it either boots or exhausts the available options.
> +
> +Instead of 500 lines of #defines and a 4KB boot script, all that is needed is
> +the following command::
> +
> +   bootflow scan -lb
> +
> +which scans for available bootflows, optionally listing each find it finds (-l)
> +and trying to boot it (-b).
> +
> +
> +Controlling ordering
> +--------------------
> +
> +Several options are available to control the ordering of boot scanning:
> +
> +
> +boot_targets
> +~~~~~~~~~~~~
> +
> +This environment variable can be used to control the list of bootdevs searched
> +and their ordering, for example::
> +
> +   setenv boot_targets "mmc0 mmc1 usb pxe"
> +
> +Entries may be removed or re-ordered in this list to affect the boot order. If
> +the variable is empty, the default ordering is used, based on the priority of
> +bootdevs and their sequence numbers.
> +
> +
> +bootmeths
> +~~~~~~~~~
> +
> +This environment variable can be used to control the list of bootmeths used and
> +their ordering for example::
> +
> +   setenv bootmeths "syslinux efi"
> +
> +Entries may be removed or re-ordered in this list to affect the order the
> +bootmeths are tried on each bootdev. If the variable is empty, the default
> +ordering is used, based on the bootmeth sequence numbers, which can be
> +controlled by aliases.
> +
> +The :ref:`usage/bootmeth:bootmeth command` (`bootmeth order`) operates in the
> +same way as setting this variable.
> +
> +
> +Bootdev uclass
> +--------------
> +
> +The bootdev uclass provides an simple API call to obtain a bootflows from a
> +device::
> +
> +   int bootdev_get_bootflow(struct udevice *dev, struct bootflow_iter *iter,
> +                            struct bootflow *bflow);
> +
> +This takes a iterator which indicates the bootdev, partition and bootmeth to
> +use. It returns a bootflow. This is the core of the bootdev implementation. The
> +bootdev drivers that implement this differ depending on the media they are
> +reading from, but each is responsible for returning a valid bootflow if
> +available.
> +
> +A helper called `bootdev_find_in_blk()` makes it fairly easy to implement this
> +function for each media device uclass, in a few lines of code.
> +
> +
> +Bootdev drivers
> +---------------
> +
> +A bootdev driver is typically fairly simple. Here is one for mmc::
> +
> +    static int mmc_get_bootflow(struct udevice *dev, struct bootflow_iter *iter,
> +                    struct bootflow *bflow)
> +    {
> +        struct udevice *mmc_dev = dev_get_parent(dev);
> +        struct udevice *blk;
> +        int ret;
> +
> +        ret = mmc_get_blk(mmc_dev, &blk);
> +        /*
> +         * If there is no media, indicate that no more partitions should be
> +         * checked
> +         */
> +        if (ret == -EOPNOTSUPP)
> +            ret = -ESHUTDOWN;
> +        if (ret)
> +            return log_msg_ret("blk", ret);
> +        assert(blk);
> +        ret = bootdev_find_in_blk(dev, blk, iter, bflow);
> +        if (ret)
> +            return log_msg_ret("find", ret);
> +
> +        return 0;
> +    }
> +
> +    static int mmc_bootdev_bind(struct udevice *dev)
> +    {
> +        struct bootdev_uc_plat *ucp = dev_get_uclass_plat(dev);
> +
> +        ucp->prio = BOOTDEVP_0_INTERNAL_FAST;
> +
> +        return 0;
> +    }
> +
> +    struct bootdev_ops mmc_bootdev_ops = {
> +        .get_bootflow    = mmc_get_bootflow,
> +    };
> +
> +    static const struct udevice_id mmc_bootdev_ids[] = {
> +        { .compatible = "u-boot,bootdev-mmc" },
> +        { }
> +    };
> +
> +    U_BOOT_DRIVER(mmc_bootdev) = {
> +        .name        = "mmc_bootdev",
> +        .id        = UCLASS_BOOTDEV,
> +        .ops        = &mmc_bootdev_ops,
> +        .bind        = mmc_bootdev_bind,
> +        .of_match    = mmc_bootdev_ids,
> +    };
> +
> +The implementation of the `get_bootflow()` method is simply to obtain the
> +block device and call a bootdev helper function to do the rest. The
> +implementation of `bootdev_find_in_blk()` checks the partition table, and
> +attempts to read a file from a filesystem on the partition number given by the
> +`@iter->part` parameter.
> +
> +Each bootdev has a priority, which indicates the order in which it is used.
> +Faster bootdevs are used first, since they are more likely to be able to boot
> +the device quickly.
> +
> +
> +Device hierarchy
> +----------------
> +
> +A bootdev device is a child of the media device. In this example, you can see
> +that the bootdev is a sibling of the block device and both are children of
> +media device::
> +
> +    mmc           0  [ + ]   bcm2835-sdhost        |   |-- mmc@7e202000
> +    blk           0  [ + ]   mmc_blk               |   |   |-- mmc@7e202000.blk
> +    bootdev       0  [   ]   mmc_bootdev           |   |   `-- mmc@7e202000.bootdev
> +    mmc           1  [ + ]   sdhci-bcm2835         |   |-- sdhci@7e300000
> +    blk           1  [   ]   mmc_blk               |   |   |-- sdhci@7e300000.blk
> +    bootdev       1  [   ]   mmc_bootdev           |   |   `-- sdhci@7e300000.bootdev
> +
> +The bootdev device is typically created automatically in the media uclass'
> +`post_bind()` method by calling `bootdev_setup_for_dev()`. The code typically
> +something like this::
> +
> +    ret = bootdev_setup_for_dev(dev, "eth_bootdev");
> +    if (ret)
> +        return log_msg_ret("bootdev", ret);
> +
> +Here, `eth_bootdev` is the name of the Ethernet bootdev driver and `dev`
> +is the ethernet device. This function is safe to call even if standard boot is
> +not enabled, since it does nothing in that case. It can be added to all uclasses
> +which implement suitable media.
> +
> +
> +The bootstd device
> +------------------
> +
> +Standard boot requires a single instance of the bootstd device to make things
> +work. This includes global information about the state of standard boot. See
> +`struct bootstd_priv` for this structure, accessed with `bootstd_get_priv()`.
> +
> +Within the devicetree, if you add bootmeth devices or a system bootdev, they
> +should be children of the bootstd device. See `arch/sandbox/dts/test.dts` for
> +an example of this.
> +
> +
> +The system bootdev
> +------------------
> +
> +Some bootmeths don't operate on individual bootdevs, but on the whole system.
> +For example, the EFI boot manager does its own device scanning and does not
> +make use of the bootdev devices. Such bootmeths can make use of the system
> +bootdev, typically considered last, after everything else has been tried.
> +
> +
> +.. _`Automatic Devices`:
> +
> +Automatic devices
> +-----------------
> +
> +It is possible to define all the required devices in the devicetree manually,
> +but it is not necessary. The bootstd uclass includes a `dm_scan_other()`
> +function which creates the bootstd device if not found. If no bootmeth devices
> +are found at all, it creates one for each available bootmeth driver as well as a
> +system bootdev.
> +
> +If your devicetree has any bootmeth device it must have all of them that you
> +want to use, as well as the system bootdev if needed, since no bootmeth devices
> +will be created automatically in that case.
> +
> +
> +Using devicetree
> +----------------
> +
> +If a bootdev is complicated or needs configuration information, it can be
> +added to the devicetree as a child of the media device. For example, imagine a
> +bootdev which reads a bootflow from SPI flash. The devicetree fragment might
> +look like this::
> +
> +    spi@0 {
> +        flash@0 {
> +            reg = <0>;
> +            compatible = "spansion,m25p16", "jedec,spi-nor";
> +            spi-max-frequency = <40000000>;
> +
> +            bootdev {
> +                compatible = "u-boot,sf-bootdev";
> +                offset = <0x2000>;
> +                size = <0x1000>;
> +            };
> +        };
> +    };
> +
> +The `sf-bootdev` driver can implement a way to read from the SPI flash, using
> +the offset and size provided, and return that bootflow file back to the caller.
> +When distro boot wants to read the kernel it calls disto_getfile() which must
> +provide a way to read from the SPI flash. See `distro_boot()` at distro_boot_
> +for more details.
> +
> +Of course this is all internal to U-Boot. All the distro sees is another way
> +to boot.
> +
> +
> +Configuration
> +-------------
> +
> +Standard boot is enabled with `CONFIG_BOOTSTD`. Each bootmeth has its own CONFIG
> +option also. For example, `CONFIG_BOOTMETH_DISTRO` enables support for distro
> +boot from a disk.
> +
> +
> +Available bootmeth drivers
> +--------------------------
> +
> +Bootmeth drivers are provided for:
> +
> +   - distro boot from a disk (syslinux)
> +   - distro boot from a network (PXE)
> +   - EFI boot using bootefi
> +   - EFI boot using boot manager
> +
> +
> +Command interface
> +-----------------
> +
> +Three commands are available:
> +
> +`bootdev`
> +    Allows listing of available bootdevs, selecting a particular one and
> +    getting information about it. See :doc:`../usage/bootdev`
> +
> +`bootflow`
> +    Allows scanning one or more bootdevs for bootflows, listing available
> +    bootflows, selecting one, obtaining information about it and booting it.
> +    See :doc:`../usage/bootflow`
> +
> +`bootmeth`
> +    Allow listing of available bootmethds and setting the order in which they
> +    are tried. See :doc:`../usage/bootmeth`
> +
> +.. _BootflowStates:
> +
> +Bootflow states
> +---------------
> +
> +Here is a list of states that a bootflow can be in:
> +
> +=======  =======================================================================
> +State    Meaning
> +=======  =======================================================================
> +base     Starting-out state, indicates that no media/partition was found. For an
> +         SD card socket it may indicate that the card is not inserted.
> +media    Media was found (e.g. SD card is inserted) but no partition information
> +         was found. It might lack a partition table or have a read error.
> +part     Partition was found but a filesystem could not be read. This could be
> +         because the partition does not hold a filesystem or the filesystem is
> +         very corrupted.
> +fs       Filesystem was found but the file could not be read. It could be
> +         missing or in the wrong subdirectory.
> +file     File was found and its size detected, but it could not be read. This
> +         could indicate filesystem corruption.
> +ready    File was loaded and is ready for use. In this state the bootflow is
> +         ready to be booted.
> +=======  =======================================================================
> +
> +
> +Theory of operation
> +-------------------
> +
> +This describes how standard boot progresses through to booting an operating
> +system.
> +
> +To start. all the necessary devices must be bound, including bootstd, which
> +provides the top-level `struct bootstd_priv` containing optional configuration
> +information. The bootstd device is also holds the various lists used while
> +scanning. This step is normally handled automatically by driver model, as
> +described in `Automatic Devices`_.
> +
> +Bootdevs are also required, to provide access to the media to use. These are not
> +useful by themselves: bootmeths are needed to provide the means of scanning
> +those bootdevs. So, all up, we need a single bootstd device, one or more bootdev
> +devices and one or more bootmeth devices.
> +
> +Once these are ready, typically a `bootflow scan` command is issued. This kicks
> +of the iteration process, which involves looking through the bootdevs and their
> +partitions one by one to find bootflows.
> +
> +Iteration is kicked off using `bootflow_scan_first()`, which calls
> +`bootflow_scan_bootdev()`.
> +
> +The iterator is set up with `bootflow_iter_init()`. This simply creates an
> +empty one with the given flags. Flags are used to control whether each
> +iteration is displayed, whether to return iterations even if they did not result
> +in a valid bootflow, whether to iterate through just a single bootdev, etc.
> +
> +Then the ordering of bootdevs is determined, by `bootdev_setup_iter_order()`. By
> +default, the bootdevs are used in the order specified by the `boot_targets`
> +environment variable (e.g. "mmc2 mmc0 usb"). If that is missing then their
> +sequence order is used, as determined by the `/aliases` node, or failing that
> +their order in the devicetree. For BOOTSTD_FULL, if there is a `bootdev-order`
> +property in the bootstd node, then this is used as a final fallback. In any
> +case, the iterator ends up with a `dev_order` array containing the bootdevs that
> +are going to be used, with `num_devs` set to the number of bootdevs and
> +`cur_dev` starting at 0.
> +
> +Next, the ordering of bootdevs is determined, by `bootmeth_setup_iter_order()`.
> +By default the ordering is again by sequence number, i.e. the `/aliases` node,
> +or failing that the order in the devicetree. But the `bootmeth order` command
> +or `bootmeths` environment variable can be used to set up an ordering. If that
> +has been done, the ordering is in `struct bootstd_priv`, so that ordering is
> +simply copied into the iterator. Either way, the `method_order` array it set up,
> +along with `num_methods`. Then `cur_method` is set to 0.
> +
> +At this point the iterator is ready to use, with the first bootdev and bootmeth
> +selected. All the other fields are 0. This means that the current partition is
> +0, which is taken to mean the whole device, since partition numbers start at 1.
> +It also means that `max_part` is 0, i.e. the maximum partition number we know
> +about is 0, meaning that, as far as we know, there is no partition table on this
> +bootdev.
> +
> +With the iterator ready, `bootflow_scan_bootdev()` checks whether the current
> +settings produce a valid bootflow. This is handled by `bootflow_check()`, which
> +either returns 0 (if it got something) or an error if not (more on that later).
> +If the `BOOTFLOWF_ALL` iterator flag is set, even errors are returned as
> +incomplete bootflows, but normally an error results in moving onto the next
> +iteration.
> +
> +The `bootflow_scan_next()` function handles moving onto the next iteration and
> +checking it. In fact it sits in a loop doing that repeatedly until it finds
> +something it wants to return.
> +
> +The actual 'moving on' part is implemented in `iter_incr()`. This is a very
> +simple function. It increments the first counter. If that hits its maximum, it
> +sets it to zero and increments the second counter. You can think of all the
> +counters together as a number with three digits which increment in order, with
> +the least-sigificant digit on the right, counting like this:
> +
> +   ========    =======    =======
> +   bootdev     part       method
> +   ========    =======    =======
> +   0           0          0
> +   0           0          1
> +   0           0          2
> +   0           1          0
> +   0           1          1
> +   0           1          1
> +   1           0          0
> +   1           0          1
> +   ========    =======    =======
> +
> +The maximum value for `method` is `num_methods - 1` so when it exceeds that, it
> +goes back to 0 and the next `part` is considered. The maximum value for that is
> +`max_part`, which is initially zero for all bootdevs. If we find a partition
> +table on that bootdev, `max_part` can be updated during the iteration to a
> +higher value - see `bootdev_find_in_blk()` for that, described later. If that
> +exceeds its maximum, then the next bootdev is used. In this way, iter_incr()
> +works its way through all possibilities, moving forward one each time it is
> +called.
> +
> +There is no expectation that iteration will actually finish. Quite often a
> +valid bootflow is found early on. With `bootflow scan -b`, that causes the
> +bootflow to be immediately booted. Assuming it is successful, the iteration never
> +completes.
> +
> +Also note that the iterator hold the **current** combination being considered.
> +So when `iter_incr()` is called, it increments to the next one and returns it,
> +the new **current** combination.
> +
> +Note also the `err` field in `struct bootflow_iter`. This is normally 0 and has
> +thus has no effect on `iter_inc()`. But if it is non-zero, signalling an error,
> +it indicates to the iterator what it should do when called. It can force moving
> +to the next partition, or bootdev, for example. The special values
> +`BF_NO_MORE_PARTS` and `BF_NO_MORE_DEVICES` handle this. When `iter_incr` sees
> +`BF_NO_MORE_PARTS` it knows that it should immediately move to the next bootdev.
> +When it sees `BF_NO_MORE_DEVICES` it knows that there is nothing more it can do
> +so it should immediately return. The caller of `iter_incr()` is responsible for
> +updating the `err` field, based on the return value it sees.
> +
> +The above describes the iteration process at a high level. It is basically a
> +very simple increment function with a checker called `bootflow_check()` that
> +checks the result of each iteration generated, to determine whether it can
> +produce a bootflow.
> +
> +So what happens inside of `bootflow_check()`? It simply calls the uclass
> +method `bootdev_get_bootflow()` to ask the bootdev to return a bootflow. It
> +passes the iterator to the bootdev method, so that function knows what we are
> +talking about. At first, the bootflow is set up in the state `BOOTFLOWST_BASE`,
> +with just the `method` and `dev` intiialised. But the bootdev may fill in more,
> +e.g. updating the state, depending on what it finds.
> +
> +Based on what the bootdev responds with, `bootflow_check()` either
> +returns a valid bootflow, or a partial one with an error. A partial bootflow
> +is one that has some fields set up, but did not reach the `BOOTFLOWST_READY`
> +state. As noted before, if the `BOOTFLOWF_ALL` iterator flag is set, then all
> +bootflows are returned, even partial ones. This can help with debugging.
> +
> +So at this point you can see that total control over whether a bootflow can
> +be generated from a particular iteration, or not, rests with the bootdev.
> +Each one can adopt its own approach.
> +
> +Going down a level, what does the bootdev do in its `get_bootflow()` method?
> +Let us consider the MMC bootdev. In that case the call to
> +`bootdev_get_bootflow()` ends up in `mmc_get_bootflow()`. It locates the parent
> +device of the bootdev, i.e. the `UCLASS_MMC` device itself, then finds the block
> +device associated with it. It then calls the helper function
> +`bootdev_find_in_blk()` to do all the work. This is common with just about any
> +bootdev that is based on a media device.
> +
> +The `bootdev_find_in_blk()` helper is implemented in the bootdev uclass. It
> +names the bootflow and copies the partition number in from the iterator. Then it
> +calls the bootmeth device to check if it can support this device. This is
> +important since some bootmeths only work with network devices, for example. If
> +that check fails, it stops.
> +
> +Assuming the bootmeth is happy, or at least indicates that it is willing to try
> +(by returning 0 from its `check()` method), the next step is to try the
> +partition. If that works it tries to detect a file system. If that works then it
> +calls the bootmeth device once more, this time to read the bootflow.
> +
> +Note: At present a filesystem is needed for the bootmeth to be called on block
> +devices, simply because we don't have any examples where this is not the case.
> +This feature can be added as needed.
> +
> +If we take the example of the `bootmeth_distro` driver, this call ends up at
> +`distro_read_bootflow()`. It has the filesystem ready, so tries various
> +filenames to try to find the `extlinux.conf` file, reading it if possible. If
> +all goes well the bootflow ends up in the `BOOTFLOWST_READY` state.
> +
> +At this point, we fall back from the bootmeth driver, to
> +`bootdev_find_in_blk()`, then back to `mmc_get_bootflow()`, then to
> +`bootdev_get_bootflow()`, then to `bootflow_check()` and finally to its caller,
> +either `bootflow_scan_bootdev()` or `bootflow_scan_next()`. In either case,
> +the bootflow is returned as the result of this iteration, assuming it made it to
> +the  `BOOTFLOWST_READY` state.
> +
> +That is the basic operation of scanning for bootflows. The process of booting a
> +bootflow is handled by the bootmeth driver for that bootflow. In the case of
> +distro boot, this parses and processes the `extlinux.conf` file that was read.
> +See `distro_boot()` for how that works. The processing may involve reading
> +additional files, which is handled by the `read_file()` method, which is
> +`distro_read_file()` in this case. All bootmethds should support reading files,
> +since the bootflow is typically only the basic instructions and does not include
> +the operating system itself, ramdisk, device tree, etc.
> +
> +The vast majority of the bootstd code is concerned with iterating through
> +partitions on bootdevs and using bootmethds to find bootflows.
> +
> +How about bootdevs which are not block devices? They are handled by the same
> +methods as above, but with a different implementation. For example, the bootmeth
> +for PXE boot (over a network) uses `tftp` to read files rather than `fs_read()`.
> +But other that that it is very similar.
> +
> +
> +Tests
> +-----
> +
> +Tests are located in `test/boot` and cover the core functionality as well as
> +the commands. All tests use sandbox so can be run on a standard Linux computer
> +and in U-Boot's CI.
> +
> +For testing, a DOS-formatted disk image is used with a single FAT partition on
> +it. This is created in `setup_bootflow_image()`, with a canned one from the
> +source tree used if it cannot be created (e.g. in CI).
> +
> +
> +Bootflow internals
> +------------------
> +
> +The bootstd device holds a linked list of scanned bootflows as well as the
> +currently selected bootdev and bootflow (for use by commands). This is in
> +`struct bootstd_priv`.
> +
> +Each bootdev device has its own `struct bootdev_uc_plat` which holds a
> +list of scanned bootflows just for that device.
> +
> +The bootflow itself is documented in bootflow_h_. It includes various bits of
> +information about the bootflow and a buffer to hold the file.
> +
> +
> +Future
> +------
> +
> +Apart from the to-do items below, different types of bootflow files may be
> +implemented in future, e.g. Chromium OS support which is currently only
> +available as a script in chromebook_coral.
> +
> +
> +To do
> +-----
> +
> +Some things that need to be done to completely replace the distro-boot scripts:
> +
> +- add bootdev drivers for dhcp, sata, scsi, ide, virtio
> +- PXE boot for EFI
> +- support for loading U-Boot scripts
> +
> +Other ideas:
> +
> +- `bootflow prep` to load everything preparing for boot, so that `bootflow boot`
> +  can just do the boot.
> +- automatically load kernel, FDT, etc. to suitable addresses so the board does
> +  not need to specify things like `pxefile_addr_r`
> +
> +
> +.. _disto_boodcmd: https://github.com/u-boot/u-boot/blob/master/include/config_distro_bootcmd.h
> +.. _BootLoaderSpec: http://www.freedesktop.org/wiki/Specifications/BootLoaderSpec/
> +.. _distro_boot: https://github.com/u-boot/u-boot/blob/master/boot/distro.c
> +.. _bootflow_h: https://github.com/u-boot/u-boot/blob/master/include/bootflow.h
> diff --git a/doc/develop/distro.rst b/doc/develop/distro.rst
> index c522be69349..7ea84b5793c 100644
> --- a/doc/develop/distro.rst
> +++ b/doc/develop/distro.rst
> @@ -157,6 +157,9 @@ a line with "CONFIG_DISTRO_DEFAULTS=y". If you want to enable this
>   from Kconfig itself, for e.g. all boards using a specific SoC then
>   add a "imply DISTRO_DEFAULTS" to your SoC CONFIG option.
>   
> +
> +TO BE UPDATED:
> +
>   In your board configuration file, include the following::
>   
>       #ifndef CONFIG_SPL_BUILD
> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> index 9592d193fca..7c41589fb92 100644
> --- a/doc/develop/index.rst
> +++ b/doc/develop/index.rst
> @@ -10,6 +10,7 @@ Implementation
>      :maxdepth: 1
>   
>      bloblist
> +   bootstd
>      ci_testing
>      commands
>      config_binding
> diff --git a/doc/device-tree-bindings/bootdev.txt b/doc/device-tree-bindings/bootdev.txt
> index 95b7fec8212..4bb2345a0b9 100644
> --- a/doc/device-tree-bindings/bootdev.txt
> +++ b/doc/device-tree-bindings/bootdev.txt
> @@ -6,3 +6,21 @@ child of the media device (UCLASS_MMC, UCLASS_SPI_FLASH, etc.)
>   
>   The bootdev driver is provided by the media devices. The bindings for each
>   are described in this file (to come).
> +
> +Required properties:
> +
> +compatible:
> +   "u-boot,bootdev-eth" - Ethernet bootdev
> +   "u-boot,bootdev-mmc" - MMC bootdev
> +   "u-boot,bootdev-usb" - USB bootdev
> +
> +
> +Example:
> +
> +	mmc1 {
> +		compatible = "sandbox,mmc";
> +
> +		mmc-bootdev {
> +			compatible = "u-boot,bootdev-eth";
> +		};
> +	};
> diff --git a/doc/device-tree-bindings/bootmeth.txt b/doc/device-tree-bindings/bootmeth.txt
> new file mode 100644
> index 00000000000..cba2dbe871c
> --- /dev/null
> +++ b/doc/device-tree-bindings/bootmeth.txt
> @@ -0,0 +1,31 @@
> +U-Boot standard boot methods (bootmeth)
> +======================================
> +
> +This provides methods (called bootmeths) for locating bootflows on a boot
> +device (bootdev). These are normally created as children of the bootstd device.
> +
> +Required properties:
> +
> +compatible:
> +   "u-boot,distro-syslinux" - distro boot from a block device
> +   "u-boot,distro-pxe" - distro boot from a network device
> +   "u-boot,distro-efi" - EFI boot from a .efi file
> +   "u-boot,efi-bootmgr" - EFI boot using boot manager (bootmgr)
> +
> +
> +Example:
> +
> +	bootstd {
> +		compatible = "u-boot,boot-std";
> +
> +		filename-prefixes = "/", "/boot/";
> +		bootdev-order = "mmc2", "mmc1";
> +
> +		syslinux {
> +			compatible = "u-boot,distro-syslinux";
> +		};
> +
> +		efi {
> +			compatible = "u-boot,distro-efi";
> +		};
> +	};
> diff --git a/doc/device-tree-bindings/bootstd.txt b/doc/device-tree-bindings/bootstd.txt
> index f048b9dd327..8706c5f4993 100644
> --- a/doc/device-tree-bindings/bootstd.txt
> +++ b/doc/device-tree-bindings/bootstd.txt
> @@ -25,4 +25,12 @@ Example:
>   
>   		filename-prefixes = "/", "/boot/";
>   		bootdev-order = "mmc2", "mmc1";
> +
> +		syslinux {
> +			compatible = "u-boot,distro-syslinux";
> +		};
> +
> +		efi {
> +			compatible = "u-boot,distro-efi";
> +		};
>   	};
> diff --git a/doc/usage/bootdev.rst b/doc/usage/bootdev.rst
> new file mode 100644
> index 00000000000..ca5671e6f74
> --- /dev/null
> +++ b/doc/usage/bootdev.rst
> @@ -0,0 +1,135 @@
> +.. SPDX-License-Identifier: GPL-2.0+:
> +
> +bootdev command
> +===============
> +
> +Synopis
> +-------
> +
> +::
> +
> +    bootdev list [-p]      - list all available bootdevs (-p to probe)\n"
> +    bootdev select <bm>    - select a bootdev by name\n"
> +    bootdev info [-p]      - show information about a bootdev";
> +
> +Description
> +-----------
> +
> +The `bootdev` command is used to manage bootdevs. It can list available
> +bootdevs, select one and obtain information about it.
> +
> +See :doc:`../develop/bootstd` for more information about bootdevs in general.
> +
> +
> +bootdev list
> +~~~~~~~~~~~~
> +
> +This lists available bootdevs
> +
> +Scanning with `-p` causes the bootdevs to be probed. This happens automatically
> +when they are used.
> +
> +The list looks something like this:
> +
> +===  ======  ======  ========  =========================
> +Seq  Probed  Status  Uclass    Name
> +===  ======  ======  ========  =========================
> +  0   [ + ]      OK  mmc       mmc@7e202000.bootdev
> +  1   [   ]      OK  mmc       sdhci@7e300000.bootdev
> +  2   [   ]      OK  ethernet  smsc95xx_eth.bootdev
> +===  ======  ======  ========  =========================
> +
> +
> +The fields are as follows:
> +
> +Seq:
> +    Sequence number in the scan, used to reference the bootflow later
> +
> +Probed:
> +    Shows a plus (+) if the device is probed, empty if not.
> +
> +Status:
> +    Shows the status of the device. Typically this is `OK` meaning that there is
> +    no error. If you use -p and an error occurs when probing, then this shows
> +    the error number. You can look up Linux error codes to find the meaning of
> +    the number.
> +
> +Uclass:
> +    Name of the media device's Uclass. This indicates the type of the parent
> +    device (e.g. MMC, Ethernet).
> +
> +Name:
> +    Name of the bootdev. This is generated from the media device appended
> +    with `.bootdev`
> +
> +
> +bootdev select
> +~~~~~~~~~~~~~~~~~
> +
> +Use this to select a particular bootdev. You can select it by the sequence
> +number or name, as shown in `bootdev list`.
> +
> +Once a bootdev is selected, you can use `bootdev info` to look at it or
> +`bootflow scan` to scan it.
> +
> +If no bootdev name or number is provided, then any existing bootdev is
> +unselected.
> +
> +
> +bootdev info
> +~~~~~~~~~~~~~~~
> +
> +This shows information on the current bootdev, with the format looking like
> +this:
> +
> +=========  =======================
> +Name       mmc@7e202000.bootdev
> +Sequence   0
> +Status     Probed
> +Uclass     mmc
> +Bootflows  1 (1 valid)
> +=========  =======================
> +
> +Most of the information is the same as `bootdev list` above. The new fields
> +are:
> +
> +Device
> +    Name of the bootdev
> +
> +Status
> +    Shows `Probed` if the device is probed, `OK` if not. If `-p` is used and the
> +    device fails to probe, an error code is shown.
> +
> +Bootflows
> +    Indicates the number of bootflows attached to the bootdev. This is 0
> +    unless you have used 'bootflow scan' on the bootflow, or on all bootflows.
> +
> +
> +Example
> +-------
> +
> +This example shows listing available bootdev and getting information about
> +one of them::
> +
> +   U-Boot> bootdev list
> +   Seq  Probed  Status  Uclass    Name
> +   ---  ------  ------  --------  ------------------
> +     0   [ + ]      OK  mmc       mmc@7e202000.bootdev
> +     1   [   ]      OK  mmc       sdhci@7e300000.bootdev
> +     2   [   ]      OK  ethernet  smsc95xx_eth.bootdev
> +   ---  ------  ------  --------  ------------------
> +   (3 devices)
> +   U-Boot> bootdev sel 0
> +   U-Boot> bootflow scan
> +   U-Boot> bootdev info
> +   Name:      mmc@7e202000.bootdev
> +   Sequence:  0
> +   Status:    Probed
> +   Uclass:    mmc
> +   Bootflows: 1 (1 valid)
> +
> +
> +Return value
> +------------
> +
> +The return value $? is always 0 (true).
> diff --git a/doc/usage/bootflow.rst b/doc/usage/bootflow.rst
> new file mode 100644
> index 00000000000..347cc181bc5
> --- /dev/null
> +++ b/doc/usage/bootflow.rst
> @@ -0,0 +1,427 @@
> +.. SPDX-License-Identifier: GPL-2.0+:
> +
> +bootflow command
> +================
> +
> +Synopis
> +-------
> +
> +::
> +
> +    bootflow scan [-abel] [bootdev]
> +    bootflow list [-e]
> +    bootflow select [<num|name>]
> +    bootflow info [-d]
> +    bootflow boot
> +
> +
> +Description
> +-----------
> +
> +The `bootflow` command is used to manage bootflows. It can scan bootdevs to
> +locate bootflows, list them and boot them.
> +
> +See :doc:`../develop/bootstd` for more information.
> +
> +
> +bootflow scan
> +~~~~~~~~~~~~~
> +
> +Scans for available bootflows, optionally booting the first valid one it finds.
> +This operates in two modes:
> +
> +- If no bootdev is selected (see `bootdev select`) it scans bootflows one
> +  by one, extracting all the bootdevs from each
> +- If a bootdev is selected, it just scans that one bootflow
> +
> +Flags are:
> +
> +-a
> +    Collect all bootflows, even those that cannot be loaded. Normally if a file
> +    is not where it is expected, then the bootflow fails and so is dropped
> +    during the scan. With this option you can see why each bootflow would be
> +    dropped.
> +
> +-b
> +    Boot each valid bootflow as it is scanned. Typically only the first bootflow
> +    matters, since by then the system boots in the OS and U-Boot is no-longer
> +    running. `bootflow scan -b` is a quick way to boot the first available OS.
> +    A valid bootflow is one that made it all the way to the `loaded` state.
> +
> +-e
> +    Used with -l to also show errors for each bootflow. The shows detailed error
> +    information for each bootflow that failed to make it to the `loaded` state.
> +
> +-l
> +    List bootflows while scanning. This is helpful when you want to see what
> +    is happening during scanning. Use it with the `-b` flag to see which
> +    bootdev and bootflows are being tried.
> +
> +The optional argument specifies a particular bootdev to scan. This can either be
> +the name of a bootdev or its sequence number (both shown with `bootdev list`).
> +Alternatively a convenience label can be used, like `mmc0`, which is the type of
> +device and an optional sequence number. Specifically, the label is the uclass of
> +the bootdev's parent followed by the sequence number of that parent. Sequence
> +numbers are typically set by aliases, so if you have 'mmc0' in your devicetree
> +alias section, then `mmc0` refers to the bootdev attached to that device.
> +
> +
> +bootflow list
> +~~~~~~~~~~~~~
> +
> +Lists the previously scanned bootflows. You must use `bootflow scan` before this
> +to see anything.
> +
> +If you scanned with -a and have bootflows with errors, -e can be used to show
> +those errors.
> +
> +The list looks something like this:
> +
> +===  ======  ======  ========  ====  ===============================   ================
> +Seq  Method  State   Uclass    Part  Name                              Filename
> +===  ======  ======  ========  ====  ===============================   ================
> +  0  distro  ready   mmc          2  mmc\@7e202000.bootdev.part_2      /boot/extlinux/extlinux.conf
> +  1  pxe     ready   ethernet     0  smsc95xx_eth.bootdev.0            rpi.pxe/extlinux/extlinux.conf
> +===  ======  ======  ========  ====  ===============================   ================
> +
> +The fields are as follows:
> +
> +Seq:
> +    Sequence number in the scan, used to reference the bootflow later
> +
> +Method:
> +    The boot method (bootmeth) used to find the bootflow. Several methods are
> +    included in U-Boot.
> +
> +State:
> +    Current state of the bootflow, indicating how far the bootdev got in
> +    obtaining a valid one. See :ref:`BootflowStates` for a list of states.
> +
> +Uclass:
> +    Name of the media device's Uclass. This indicates the type of the parent
> +    device (e.g. MMC, Ethernet).
> +
> +Part:
> +    Partition number being accesseed, numbered from 1. Normally a device will
> +    have a partition table with a small number of partitions. For devices
> +    without partition tables (e.g. network) this field is 0.
> +
> +Name:
> +    Name of the bootflow. This is generated from the bootdev appended with
> +    the partition information
> +
> +Filename:
> +    Name of the bootflow file. This indicates where the file is on the
> +    filesystem or network device.
> +
> +
> +bootflow select
> +~~~~~~~~~~~~~~~
> +
> +Use this to select a particular bootflow. You can select it by the sequence
> +number or name, as shown in `bootflow list`.
> +
> +Once a bootflow is selected, you can use `bootflow info` and `bootflow boot`.
> +
> +If no bootflow name or number is provided, then any existing bootflow is
> +unselected.
> +
> +
> +bootflow info
> +~~~~~~~~~~~~~
> +
> +This shows information on the current bootflow, with the format looking like
> +this:
> +
> +=========  ===============================
> +Name       mmc\@7e202000.bootdev.part_2
> +Device     mmc\@7e202000.bootdev
> +Block dev  mmc\@7e202000.blk
> +Type       distro
> +Method:    syslinux
> +State      ready
> +Partition  2
> +Subdir     (none)
> +Filename   /extlinux/extlinux.conf
> +Buffer     3db7ad48
> +Size       232 (562 bytes)
> +Error      0
> +=========  ===============================
> +
> +Most of the information is the same as `bootflow list` above. The new fields
> +are:
> +
> +Device
> +    Name of the bootdev
> +
> +Block dev
> +    Name of the block device, if any. Network devices don't have a block device.
> +
> +Subdir
> +    Subdirectory used for retrieving files. For network bootdevs this is the
> +    directory of the 'bootfile' parameter passed from DHCP. All file retrievals
> +    when booting are relative to this.
> +
> +Buffer
> +    Buffer containing the bootflow file. You can use the :doc:`md` to look at
> +    it, or dump it with `bootflow info -d`.
> +
> +Size
> +    Size of the bootflow file
> +
> +Error
> +    Error number returned from scanning for the bootflow. This is 0 if the
> +    bootflow is in the 'loaded' state, or a negative error value on error. You
> +    can look up Linux error codes to find the meaning of the number.
> +
> +Use the `-d` flag to dump out the contents of the bootfile file.
> +
> +
> +bootflow boot
> +~~~~~~~~~~~~~
> +
> +This boots the current bootflow.
> +
> +
> +Example
> +-------
> +
> +Here is an example of scanning for bootflows, then listing them::
> +
> +    U-Boot> bootflow scan -l
> +    Scanning for bootflows in all bootdevs
> +    Seq  Type         State   Uclass    Part  Name                      Filename
> +    ---  -----------  ------  --------  ----  ------------------------  ----------------
> +    Scanning bootdev 'mmc@7e202000.bootdev':
> +      0  distro       ready   mmc          2  mmc@7e202000.bootdev.p    /extlinux/extlinux.conf
> +    Scanning bootdev 'sdhci@7e300000.bootdev':
> +    Card did not respond to voltage select! : -110
> +    Scanning bootdev 'smsc95xx_eth.bootdev':
> +    Waiting for Ethernet connection... done.
> +    BOOTP broadcast 1
> +    DHCP client bound to address 192.168.4.30 (4 ms)
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe/'.
> +    Load address: 0x200000
> +    Loading: *
> +    TFTP error: 'Is a directory' (0)
> +    Starting again
> +
> +    missing environment variable: pxeuuid
> +    Retrieving file: rpi.pxe/pxelinux.cfg/01-b8-27-eb-a6-61-e1
> +    Waiting for Ethernet connection... done.
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe/pxelinux.cfg/01-b8-27-eb-a6-61-e1'.
> +    Load address: 0x2500000
> +    Loading: ##################################################  566 Bytes
> +    	 45.9 KiB/s
> +    done
> +    Bytes transferred = 566 (236 hex)
> +      1  distro       ready   ethernet     0  smsc95xx_eth.bootdev.0 rpi.pxe/extlinux/extlinux.conf
> +    No more bootdevs
> +    ---  -----------  ------  --------  ----  ------------------------  ----------------
> +    (2 bootflows, 2 valid)
> +    U-Boot> bootflow l
> +    Showing all bootflows
> +    Seq  Type         State   Uclass    Part  Name                      Filename
> +    ---  -----------  ------  --------  ----  ------------------------  ----------------
> +      0  distro       ready   mmc          2  mmc@7e202000.bootdev.p    /extlinux/extlinux.conf
> +      1  pxe          ready   ethernet     0  smsc95xx_eth.bootdev.0     rpi.pxe/extlinux/extlinux.conf
> +    ---  -----------  ------  --------  ----  ------------------------  ----------------
> +    (2 bootflows, 2 valid)
> +
> +
> +The second one is then selected by name (we could instead use `bootflow sel 0`),
> +displayed and booted::
> +
> +    U-Boot> bootflow info
> +    No bootflow selected
> +    U-Boot> bootflow sel mmc@7e202000.bootdev.part_2
> +    U-Boot> bootflow info
> +    Name:      mmc@7e202000.bootdev.part_2
> +    Device:    mmc@7e202000.bootdev
> +    Block dev: mmc@7e202000.blk
> +    Sequence:  1
> +    Method:    distro
> +    State:     ready
> +    Partition: 2
> +    Subdir:    (none)
> +    Filename:  extlinux/extlinux.conf
> +    Buffer:    3db7ae88
> +    Size:      232 (562 bytes)
> +    Error:     0
> +    U-Boot> bootflow boot
> +    ** Booting bootflow 'smsc95xx_eth.bootdev.0'
> +    Ignoring unknown command: ui
> +    Ignoring malformed menu command:  autoboot
> +    Ignoring malformed menu command:  hidden
> +    Ignoring unknown command: totaltimeout
> +    1:	Fedora-Workstation-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
> +    Retrieving file: rpi.pxe/initramfs-5.3.7-301.fc31.armv7hl.img
> +    get 2700000 rpi.pxe/initramfs-5.3.7-301.fc31.armv7hl.img
> +    Waiting for Ethernet connection... done.
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe/initramfs-5.3.7-301.fc31.armv7hl.img'.
> +    Load address: 0x2700000
> +    Loading: ###################################T ###############  57.7 MiB
> +    	 1.9 MiB/s
> +    done
> +    Bytes transferred = 60498594 (39b22a2 hex)
> +    Retrieving file: rpi.pxe//vmlinuz-5.3.7-301.fc31.armv7hl
> +    get 80000 rpi.pxe//vmlinuz-5.3.7-301.fc31.armv7hl
> +    Waiting for Ethernet connection... done.
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe//vmlinuz-5.3.7-301.fc31.armv7hl'.
> +    Load address: 0x80000
> +    Loading: ##################################################  7.2 MiB
> +    	 2.3 MiB/s
> +    done
> +    Bytes transferred = 7508480 (729200 hex)
> +    append: ro root=UUID=9732b35b-4cd5-458b-9b91-80f7047e0b8a rhgb quiet LANG=en_US.UTF-8 cma=192MB cma=256MB
> +    Retrieving file: rpi.pxe//dtb-5.3.7-301.fc31.armv7hl/bcm2837-rpi-3-b.dtb
> +    get 2600000 rpi.pxe//dtb-5.3.7-301.fc31.armv7hl/bcm2837-rpi-3-b.dtb
> +    Waiting for Ethernet connection... done.
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe//dtb-5.3.7-301.fc31.armv7hl/bcm2837-rpi-3-b.dtb'.
> +    Load address: 0x2600000
> +    Loading: ##################################################  13.8 KiB
> +    	 764.6 KiB/s
> +    done
> +    Bytes transferred = 14102 (3716 hex)
> +    Kernel image @ 0x080000 [ 0x000000 - 0x729200 ]
> +    ## Flattened Device Tree blob at 02600000
> +       Booting using the fdt blob at 0x2600000
> +       Using Device Tree in place at 02600000, end 02606715
> +
> +    Starting kernel ...
> +
> +    [  OK  ] Started Show Plymouth Boot Screen.
> +    [  OK  ] Started Forward Password R…s to Plymouth Directory Watch.
> +    [  OK  ] Reached target Local Encrypted Volumes.
> +    [  OK  ] Reached target Paths.
> +    ....
> +
> +
> +Here we scan for bootflows and boot the first one found::
> +
> +    U-Boot> bootflow scan -bl
> +    Scanning for bootflows in all bootdevs
> +    Seq  Method       State   Uclass    Part  Name                    Filename
> +    ---  -----------  ------  --------  ----  ----------------------  ----------------
> +    Scanning bootdev 'mmc@7e202000.bootdev':
> +      0  distro       ready   mmc          2  mmc@7e202000.bootdev.p  /extlinux/extlinux.conf
> +    ** Booting bootflow 'mmc@7e202000.bootdev.part_2'
> +    Ignoring unknown command: ui
> +    Ignoring malformed menu command:  autoboot
> +    Ignoring malformed menu command:  hidden
> +    Ignoring unknown command: totaltimeout
> +    1:	Fedora-KDE-armhfp-31-1.9 (5.3.7-301.fc31.armv7hl)
> +    Retrieving file: /initramfs-5.3.7-301.fc31.armv7hl.img
> +    getfile 2700000 /initramfs-5.3.7-301.fc31.armv7hl.img
> +    Retrieving file: /vmlinuz-5.3.7-301.fc31.armv7hl
> +    getfile 80000 /vmlinuz-5.3.7-301.fc31.armv7hl
> +    append: ro root=UUID=b8781f09-e2dd-4cb8-979b-7df5eeaaabea rhgb LANG=en_US.UTF-8 cma=192MB console=tty0 console=ttyS1,115200
> +    Retrieving file: /dtb-5.3.7-301.fc31.armv7hl/bcm2837-rpi-3-b.dtb
> +    getfile 2600000 /dtb-5.3.7-301.fc31.armv7hl/bcm2837-rpi-3-b.dtb
> +    Kernel image @ 0x080000 [ 0x000000 - 0x729200 ]
> +    ## Flattened Device Tree blob at 02600000
> +       Booting using the fdt blob at 0x2600000
> +       Using Device Tree in place at 02600000, end 02606715
> +
> +    Starting kernel ...
> +
> +    [    0.000000] Booting Linux on physical CPU 0x0
> +
> +
> +Here is am example using the -e flag to see all errors::
> +
> +    U-Boot> bootflow scan -a
> +    Card did not respond to voltage select! : -110
> +    Waiting for Ethernet connection... done.
> +    BOOTP broadcast 1
> +    DHCP client bound to address 192.168.4.30 (4 ms)
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe/'.
> +    Load address: 0x200000
> +    Loading: *
> +    TFTP error: 'Is a directory' (0)
> +    Starting again
> +
> +    missing environment variable: pxeuuid
> +    Retrieving file: rpi.pxe/pxelinux.cfg/01-b8-27-eb-a6-61-e1
> +    Waiting for Ethernet connection... done.
> +    Using smsc95xx_eth device
> +    TFTP from server 192.168.4.1; our IP address is 192.168.4.30
> +    Filename 'rpi.pxe/pxelinux.cfg/01-b8-27-eb-a6-61-e1'.
> +    Load address: 0x2500000
> +    Loading: ##################################################  566 Bytes
> +    	 49.8 KiB/s
> +    done
> +    Bytes transferred = 566 (236 hex)
> +    U-Boot> bootflow l -e
> +    Showing all bootflows
> +    Seq  Type         State   Uclass    Part  Name                   Filename
> +    ---  -----------  ------  --------  ----  ---------------------  ----------------
> +      0  distro       fs      mmc          1  mmc@7e202000.bootdev.p /extlinux/extlinux.conf
> +         ** File not found, err=-2
> +      1  distro       ready   mmc          2  mmc@7e202000.bootdev.p /extlinux/extlinux.conf
> +      2  distro       fs      mmc          3  mmc@7e202000.bootdev.p /extlinux/extlinux.conf
> +         ** File not found, err=-1
> +      3  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      4  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      5  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      6  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      7  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      8  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      9  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      a  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      b  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      c  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      d  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      e  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +      f  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +     10  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +     11  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +     12  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +     13  distro       media   mmc          0  mmc@7e202000.bootdev.p <NULL>
> +         ** No partition found, err=-2
> +     14  distro       ready   ethernet     0  smsc95xx_eth.bootdev.0 rpi.pxe/extlinux/extlinux.conf
> +    ---  -----------  ------  --------  ----  ---------------------  ----------------
> +    (21 bootflows, 2 valid)
> +    U-Boot>
> +
> +
> +Return value
> +------------
> +
> +On success `bootflow boot` normally boots into the Operating System and does not
> +return to U-Boot. If something about the U-Boot processing fails, then the
> +return value $? is 1. If the boot succeeds but for some reason the Operating
> +System returns, then $? is 0, indicating success.
> +
> +For other subcommands, the return value $? is always 0 (true).
> +
> +
> +.. BootflowStates_:
> diff --git a/doc/usage/bootmeth.rst b/doc/usage/bootmeth.rst
> new file mode 100644
> index 00000000000..1bb2b9d5c09
> --- /dev/null
> +++ b/doc/usage/bootmeth.rst
> @@ -0,0 +1,108 @@
> +.. SPDX-License-Identifier: GPL-2.0+:
> +
> +bootmeth command
> +================
> +
> +Synopis
> +-------
> +
> +::
> +
> +    bootmeth list [-a]          - list selected bootmeths (-a for all)
> +    bootmeth order "[<bm> ...]" - select the order of bootmeths\n"
> +
> +
> +Description
> +-----------
> +
> +The `bootmeth` command is used to manage bootmeths. It can list them and change
> +the order in which they are used.
> +
> +See :doc:`../develop/bootstd` for more information.
> +
> +
> +.. _bootmeth_order:
> +
> +bootmeth order
> +~~~~~~~~~~~~~~
> +
> +Selects which bootmeths to use and the order in which they are invoked. When
> +scanning bootdevs, each bootmeth is tried in turn to see if it can find a valid
> +bootflow. You can use this command to adjust the order or even to omit some
> +boomeths.
> +
> +The argument is a quoted list of bootmeths to use, by name.
> +
> +
> +bootmeth list
> +~~~~~~~~~~~~~
> +
> +This lists the selected bootmeths, or all of them, if the `-a` flag is used.
> +The format looks like this:
> +
> +=====  ===  ==================  =================================
> +Order  Seq  Name                Description
> +=====  ===  ==================  =================================
> +    0    0  distro              Syslinux boot from a block device
> +    1    1  efi                 EFI boot from a .efi file
> +    2    2  pxe                 PXE boot from a network device
> +    3    3  sandbox             Sandbox boot for testing
> +    4    4  efi_mgr             EFI bootmgr flow
> +=====  ===  ==================  =================================
> +
> +The fields are as follows:
> +
> +Order:
> +    The order in which these bootmeths are invoked for each bootdev. If this
> +    shows as a hyphen, then the bootmeth is not in the current ordering.
> +
> +Seq:
> +    The sequence number of the bootmeth, i.e. the normal ordering if none is set
> +
> +Name:
> +    Name of the bootmeth
> +
> +Description:
> +    A friendly description for the bootmeth
> +
> +
> +Example
> +-------
> +
> +This shows listing bootmeths. All are present and in the normal order::
> +
> +    => bootmeth list
> +    Order  Seq  Name                Description
> +    -----  ---  ------------------  ------------------
> +        0    0  distro              Syslinux boot from a block device
> +        1    1  efi                 EFI boot from a .efi file
> +        2    2  pxe                 PXE boot from a network device
> +        3    3  sandbox             Sandbox boot for testing
> +        4    4  efi_mgr             EFI bootmgr flow
> +    -----  ---  ------------------  ------------------
> +    (5 bootmeths)
> +
> +Now the order is changed, to include only two of them::
> +
> +    => bootmeth order "sandbox distro"
> +    => bootmeth list
> +    Order  Seq  Name                Description
> +    -----  ---  ------------------  ------------------
> +        0    3  sandbox             Sandbox boot for testing
> +        1    0  distro              Syslinux boot from a block device
> +    -----  ---  ------------------  ------------------
> +    (2 bootmeths)
> +
> +The -a flag shows all bootmeths so you can clearly see which ones are used and
> +which are not::
> +
> +    => bootmeth list -a
> +    Order  Seq  Name                Description
> +    -----  ---  ------------------  ------------------
> +        1    0  distro              Syslinux boot from a block device
> +        -    1  efi                 EFI boot from a .efi file
> +        -    2  pxe                 PXE boot from a network device
> +        0    3  sandbox             Sandbox boot for testing
> +        -    4  efi_mgr             EFI bootmgr flow
> +    -----  ---  ------------------  ------------------
> +    (5 bootmeths)
> diff --git a/doc/usage/index.rst b/doc/usage/index.rst
> index 33761af96af..3f6903ad76b 100644
> --- a/doc/usage/index.rst
> +++ b/doc/usage/index.rst
> @@ -21,9 +21,12 @@ Shell commands
>      addrmap
>      askenv
>      base
> +   bootdev
>      bootefi
> +   bootflow
>      booti
>      bootmenu
> +   bootmeth
>      button
>      x86/cbsysinfo
>      conitrace

On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> On 1/19/22 02:43, Simon Glass wrote:
> > Add documentation for this feature, including the commands and full
> > devicetree bindings.
> > 
> > Signed-off-by: Simon Glass <sjg@chromium.org>
> > ---
> > 
> > Changes in v3:
> > - Update docs for "bootmeths" and "boot_targets" env vars
> > 
> >   MAINTAINERS                           |   4 +
> >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> >   doc/develop/distro.rst                |   3 +
> >   doc/develop/index.rst                 |   1 +
> >   doc/device-tree-bindings/bootdev.txt  |  18 +
> >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> >   doc/device-tree-bindings/bootstd.txt  |   8 +
> >   doc/usage/bootdev.rst                 | 135 ++++++
> >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> >   doc/usage/bootmeth.rst                | 108 +++++
> >   doc/usage/index.rst                   |   3 +
> >   11 files changed, 1376 insertions(+)
> >   create mode 100644 doc/develop/bootstd.rst
> >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> >   create mode 100644 doc/usage/bootdev.rst
> >   create mode 100644 doc/usage/bootflow.rst
> >   create mode 100644 doc/usage/bootmeth.rst
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 8ad70d3d968..c2af8ada3c9 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -669,6 +669,10 @@ F:	boot/bootmeth*.c
> >   F:	boot/bootstd.c
> >   F:	cmd/bootdev.c
> >   F:	cmd/bootflow.c
> > +F:	doc/develop/bootstd.rst
> > +F:	doc/usage/bootdev.rst
> > +F:	doc/usage/bootflow.rst
> > +F:	doc/usage/bootmeth.rst
> >   F:	drivers/mmc/mmc_bootdev.c
> >   F:	include/bootdev.h
> >   F:	include/bootflow.h
> > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > new file mode 100644
> > index 00000000000..1b65a806efb
> > --- /dev/null
> > +++ b/doc/develop/bootstd.rst
> > @@ -0,0 +1,638 @@
> > +.. SPDX-License-Identifier: GPL-2.0+:
> > +
> > +U-Boot Standard Boot
> > +====================
> > +
> > +Introduction
> > +------------
> > +
> > +Standard boot provides a built-in way for U-Boot to automatically boot
> > +an Operating System without custom scripting and other customisation. It
> > +introduces the following concepts:
> > +
> > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > +   - bootflow - a description of how to boot (provided by the distro)
> > +
> > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > +for creating a bootflow for each kernel combination that it wants to offer.
> 
> This gets it completely wrong. There is one standardized boot flow: UEFI.
> All major distros support this. U-Boot has to offer UEFI booting out of the
> box.

I want to jump up and down and emphasize this part as well.  While I
believe our UEFI bootmgr is still missing the normal scan code, that's
something that has been promised to be implemented.  And that turns the
bootcmd for platforms that just want to support modern off the shelf
distros in to something fairly small.

Hi,

On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > On 1/19/22 02:43, Simon Glass wrote:
> > > Add documentation for this feature, including the commands and full
> > > devicetree bindings.
> > >
> > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > ---
> > >
> > > Changes in v3:
> > > - Update docs for "bootmeths" and "boot_targets" env vars
> > >
> > >   MAINTAINERS                           |   4 +
> > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > >   doc/develop/distro.rst                |   3 +
> > >   doc/develop/index.rst                 |   1 +
> > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > >   doc/usage/bootdev.rst                 | 135 ++++++
> > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > >   doc/usage/bootmeth.rst                | 108 +++++
> > >   doc/usage/index.rst                   |   3 +
> > >   11 files changed, 1376 insertions(+)
> > >   create mode 100644 doc/develop/bootstd.rst
> > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > >   create mode 100644 doc/usage/bootdev.rst
> > >   create mode 100644 doc/usage/bootflow.rst
> > >   create mode 100644 doc/usage/bootmeth.rst
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index 8ad70d3d968..c2af8ada3c9 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > >   F:        boot/bootstd.c
> > >   F:        cmd/bootdev.c
> > >   F:        cmd/bootflow.c
> > > +F: doc/develop/bootstd.rst
> > > +F: doc/usage/bootdev.rst
> > > +F: doc/usage/bootflow.rst
> > > +F: doc/usage/bootmeth.rst
> > >   F:        drivers/mmc/mmc_bootdev.c
> > >   F:        include/bootdev.h
> > >   F:        include/bootflow.h
> > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > new file mode 100644
> > > index 00000000000..1b65a806efb
> > > --- /dev/null
> > > +++ b/doc/develop/bootstd.rst
> > > @@ -0,0 +1,638 @@
> > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > +
> > > +U-Boot Standard Boot
> > > +====================
> > > +
> > > +Introduction
> > > +------------
> > > +
> > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > +an Operating System without custom scripting and other customisation. It
> > > +introduces the following concepts:
> > > +
> > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > +   - bootflow - a description of how to boot (provided by the distro)
> > > +
> > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > +for creating a bootflow for each kernel combination that it wants to offer.
> >
> > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > All major distros support this. U-Boot has to offer UEFI booting out of the
> > box.
>
> I want to jump up and down and emphasize this part as well.  While I
> believe our UEFI bootmgr is still missing the normal scan code, that's
> something that has been promised to be implemented.  And that turns the
> bootcmd for platforms that just want to support modern off the shelf
> distros in to something fairly small.

Sigh...

UEFI is a bootflow in this model, one of many. If we don't support the
others, then U-Boot is not U-Boot anymore, it is just EFI Boot.

If we get EFI bootmgr going, then are you saying you want to disable
everything else?

You say 'major distros' but there are many that don't use it,
particularly in the embedded space. I'll go out on a limb and say that
the vast majority of embedded devices in the world don't use it. Are
you really saying we should drop support for everything else? Even the
distro stuff supports other options.

Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
of the box'. Which bit of this series is in conflict with that? What
exactly is "completely wrong" ?? Is it just the wording that is
confusing?

??

Regards,
Simon

On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> Hi,
> 
> On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > On 1/19/22 02:43, Simon Glass wrote:
[snip]
> > > > +Introduction
> > > > +------------
> > > > +
> > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > +an Operating System without custom scripting and other customisation. It
> > > > +introduces the following concepts:
> > > > +
> > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > +
> > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > >
> > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > box.
> >
> > I want to jump up and down and emphasize this part as well.  While I
> > believe our UEFI bootmgr is still missing the normal scan code, that's
> > something that has been promised to be implemented.  And that turns the
> > bootcmd for platforms that just want to support modern off the shelf
> > distros in to something fairly small.
> 
> Sigh...
> 
> UEFI is a bootflow in this model, one of many. If we don't support the
> others, then U-Boot is not U-Boot anymore, it is just EFI Boot.

No one is talking about removing anything else.  But a major part of
your motivation here seems to be "discovering what to boot where is a
pain" and that's solved (or at least defined, I'm poking Ilias about the
status of that off-list).  And I want to emphasize discover.

> If we get EFI bootmgr going, then are you saying you want to disable
> everything else?

Not at all.

> You say 'major distros' but there are many that don't use it,
> particularly in the embedded space. I'll go out on a limb and say that
> the vast majority of embedded devices in the world don't use it. Are
> you really saying we should drop support for everything else? Even the
> distro stuff supports other options.

I don't know about buildroot off-hand, but I've had OpenEmbedded spit
out UEFI-compatible aarch64 images no problem.  If you're talking about
embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
flow".  Armbian is the biggest distro I know of off-hand that doesn't
do UEFI boot for U-Boot targets and I would love to talk with someone
there and find out why (but I guess it's all the 32bit platforms).

But I'd also say the vast majority of embedded devices don't need the
complexity you're adding here, but DO need the ability to implement A/B
things as easily in U-Boot as they can in grub.  And that in turn is
because it's a pain to modify the default environment in U-Boot and easy
to drop in another script for grub.

Hi Tom,

On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > Hi,
> >
> > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > On 1/19/22 02:43, Simon Glass wrote:
> [snip]
> > > > > +Introduction
> > > > > +------------
> > > > > +
> > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > +an Operating System without custom scripting and other customisation. It
> > > > > +introduces the following concepts:
> > > > > +
> > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > +
> > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > >
> > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > box.
> > >
> > > I want to jump up and down and emphasize this part as well.  While I
> > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > something that has been promised to be implemented.  And that turns the
> > > bootcmd for platforms that just want to support modern off the shelf
> > > distros in to something fairly small.
> >
> > Sigh...
> >
> > UEFI is a bootflow in this model, one of many. If we don't support the
> > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
>
> No one is talking about removing anything else.  But a major part of
> your motivation here seems to be "discovering what to boot where is a
> pain" and that's solved (or at least defined, I'm poking Ilias about the
> status of that off-list).  And I want to emphasize discover.

But only if you use EFI boot manager, right? Even then I'm not sure we
have a deterministic way of listing the available bootdevs, which is
something that this series provides.

>
> > If we get EFI bootmgr going, then are you saying you want to disable
> > everything else?
>
> Not at all.

OK, good.

>
> > You say 'major distros' but there are many that don't use it,
> > particularly in the embedded space. I'll go out on a limb and say that
> > the vast majority of embedded devices in the world don't use it. Are
> > you really saying we should drop support for everything else? Even the
> > distro stuff supports other options.
>
> I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> out UEFI-compatible aarch64 images no problem.  If you're talking about
> embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> flow".  Armbian is the biggest distro I know of off-hand that doesn't
> do UEFI boot for U-Boot targets and I would love to talk with someone
> there and find out why (but I guess it's all the 32bit platforms).
>
> But I'd also say the vast majority of embedded devices don't need the
> complexity you're adding here, but DO need the ability to implement A/B
> things as easily in U-Boot as they can in grub.  And that in turn is
> because it's a pain to modify the default environment in U-Boot and easy
> to drop in another script for grub.

My feeling is the complexity is already there, just in scripts, which
are harder to understand (from personal experience trying to
understand what they do) and don't have tests. They are also very hard
to build on top of (e.g. verified boot).

I can't really say that this series is more complex than EFI bootmgr,
if that is what you are suggesting. I think the bootdev uclass is
well-motivated and will prove useful even for EFI.

Also A/B/recovery is a lot easier to implement in code than in
scripts. I have linked to the proposed design there.

Anyway if we can agree that we are not going to disable the non-EFI
flows, then how about we focus on replacing the distro boot scripts,
dropping the config.h files, and leave EFI bootmgr out of this
discussion?

Regards,
Simon

> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 08:20:17 -0700
> 
> Hi,
> 
> On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > On 1/19/22 02:43, Simon Glass wrote:
> > > > Add documentation for this feature, including the commands and full
> > > > devicetree bindings.
> > > >
> > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > ---
> > > >
> > > > Changes in v3:
> > > > - Update docs for "bootmeths" and "boot_targets" env vars
> > > >
> > > >   MAINTAINERS                           |   4 +
> > > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > > >   doc/develop/distro.rst                |   3 +
> > > >   doc/develop/index.rst                 |   1 +
> > > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > > >   doc/usage/bootdev.rst                 | 135 ++++++
> > > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > > >   doc/usage/bootmeth.rst                | 108 +++++
> > > >   doc/usage/index.rst                   |   3 +
> > > >   11 files changed, 1376 insertions(+)
> > > >   create mode 100644 doc/develop/bootstd.rst
> > > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > > >   create mode 100644 doc/usage/bootdev.rst
> > > >   create mode 100644 doc/usage/bootflow.rst
> > > >   create mode 100644 doc/usage/bootmeth.rst
> > > >
> > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > index 8ad70d3d968..c2af8ada3c9 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > > >   F:        boot/bootstd.c
> > > >   F:        cmd/bootdev.c
> > > >   F:        cmd/bootflow.c
> > > > +F: doc/develop/bootstd.rst
> > > > +F: doc/usage/bootdev.rst
> > > > +F: doc/usage/bootflow.rst
> > > > +F: doc/usage/bootmeth.rst
> > > >   F:        drivers/mmc/mmc_bootdev.c
> > > >   F:        include/bootdev.h
> > > >   F:        include/bootflow.h
> > > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > > new file mode 100644
> > > > index 00000000000..1b65a806efb
> > > > --- /dev/null
> > > > +++ b/doc/develop/bootstd.rst
> > > > @@ -0,0 +1,638 @@
> > > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > > +
> > > > +U-Boot Standard Boot
> > > > +====================
> > > > +
> > > > +Introduction
> > > > +------------
> > > > +
> > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > +an Operating System without custom scripting and other customisation. It
> > > > +introduces the following concepts:
> > > > +
> > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > +
> > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > >
> > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > box.
> >
> > I want to jump up and down and emphasize this part as well.  While I
> > believe our UEFI bootmgr is still missing the normal scan code, that's
> > something that has been promised to be implemented.  And that turns the
> > bootcmd for platforms that just want to support modern off the shelf
> > distros in to something fairly small.
> 
> Sigh...
> 
> UEFI is a bootflow in this model, one of many. If we don't support the
> others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> 
> If we get EFI bootmgr going, then are you saying you want to disable
> everything else?
> 
> You say 'major distros' but there are many that don't use it,
> particularly in the embedded space. I'll go out on a limb and say that
> the vast majority of embedded devices in the world don't use it. Are
> you really saying we should drop support for everything else? Even the
> distro stuff supports other options.

And U-Boot supports a wide variety of CPUs and some of those don't
even have official UEFI support.

However, on arm64 (and possibly riscv64) even the embedded folks
should seriously consider using the UEFI bootflow.  Linux now supports
physical address randomization when loaded via the UEFI stub, which is
something that can't really be implemented using the legacy boot path.
Note that you don't have to use a separate UEFI bootloader as U-Boot
can directly boot kernels with the UEFI stub.

> Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
> of the box'. Which bit of this series is in conflict with that? What
> exactly is "completely wrong" ?? Is it just the wording that is
> confusing?

Possibly.  The documentation seems to suggest that OSes have to
specify a bootflow for U-Boot.  Whereas one of the main advantages of
the UEFI bootflow is that this allows OSes not to care whether we're
booted by U-Boot, EDK2 or a closed source firmware implementation.  I
think the docs should say that the bootflow can be customized by an
OS, but that in general this isn't necessary.

Hi Mark,

On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 08:20:17 -0700
> >
> > Hi,
> >
> > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > Add documentation for this feature, including the commands and full
> > > > > devicetree bindings.
> > > > >
> > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > ---
> > > > >
> > > > > Changes in v3:
> > > > > - Update docs for "bootmeths" and "boot_targets" env vars
> > > > >
> > > > >   MAINTAINERS                           |   4 +
> > > > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > > > >   doc/develop/distro.rst                |   3 +
> > > > >   doc/develop/index.rst                 |   1 +
> > > > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > > > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > > > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > > > >   doc/usage/bootdev.rst                 | 135 ++++++
> > > > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > > > >   doc/usage/bootmeth.rst                | 108 +++++
> > > > >   doc/usage/index.rst                   |   3 +
> > > > >   11 files changed, 1376 insertions(+)
> > > > >   create mode 100644 doc/develop/bootstd.rst
> > > > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > > > >   create mode 100644 doc/usage/bootdev.rst
> > > > >   create mode 100644 doc/usage/bootflow.rst
> > > > >   create mode 100644 doc/usage/bootmeth.rst
> > > > >
> > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > index 8ad70d3d968..c2af8ada3c9 100644
> > > > > --- a/MAINTAINERS
> > > > > +++ b/MAINTAINERS
> > > > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > > > >   F:        boot/bootstd.c
> > > > >   F:        cmd/bootdev.c
> > > > >   F:        cmd/bootflow.c
> > > > > +F: doc/develop/bootstd.rst
> > > > > +F: doc/usage/bootdev.rst
> > > > > +F: doc/usage/bootflow.rst
> > > > > +F: doc/usage/bootmeth.rst
> > > > >   F:        drivers/mmc/mmc_bootdev.c
> > > > >   F:        include/bootdev.h
> > > > >   F:        include/bootflow.h
> > > > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > > > new file mode 100644
> > > > > index 00000000000..1b65a806efb
> > > > > --- /dev/null
> > > > > +++ b/doc/develop/bootstd.rst
> > > > > @@ -0,0 +1,638 @@
> > > > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > > > +
> > > > > +U-Boot Standard Boot
> > > > > +====================
> > > > > +
> > > > > +Introduction
> > > > > +------------
> > > > > +
> > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > +an Operating System without custom scripting and other customisation. It
> > > > > +introduces the following concepts:
> > > > > +
> > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > +
> > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > >
> > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > box.
> > >
> > > I want to jump up and down and emphasize this part as well.  While I
> > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > something that has been promised to be implemented.  And that turns the
> > > bootcmd for platforms that just want to support modern off the shelf
> > > distros in to something fairly small.
> >
> > Sigh...
> >
> > UEFI is a bootflow in this model, one of many. If we don't support the
> > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> >
> > If we get EFI bootmgr going, then are you saying you want to disable
> > everything else?
> >
> > You say 'major distros' but there are many that don't use it,
> > particularly in the embedded space. I'll go out on a limb and say that
> > the vast majority of embedded devices in the world don't use it. Are
> > you really saying we should drop support for everything else? Even the
> > distro stuff supports other options.
>
> And U-Boot supports a wide variety of CPUs and some of those don't
> even have official UEFI support.
>
> However, on arm64 (and possibly riscv64) even the embedded folks
> should seriously consider using the UEFI bootflow.  Linux now supports
> physical address randomization when loaded via the UEFI stub, which is
> something that can't really be implemented using the legacy boot path.
> Note that you don't have to use a separate UEFI bootloader as U-Boot
> can directly boot kernels with the UEFI stub.

'legacy'? Isn't it just a case of relocating the kernel to a random
address? I'm pretty sure U-Boot can do that :-)

Re direct boot, the issue seems to me that distros really want to use
grub. I think a lot of people talk about direct boot, but it doesn't
seem to be happening?

>
> > Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
> > of the box'. Which bit of this series is in conflict with that? What
> > exactly is "completely wrong" ?? Is it just the wording that is
> > confusing?
>
> Possibly.  The documentation seems to suggest that OSes have to
> specify a bootflow for U-Boot.  Whereas one of the main advantages of
> the UEFI bootflow is that this allows OSes not to care whether we're
> booted by U-Boot, EDK2 or a closed source firmware implementation.  I
> think the docs should say that the bootflow can be customized by an
> OS, but that in general this isn't necessary.

The definition of a bootflow is pretty broad. In the case of grub, it
isn't even visible to U-Boot so there is a bootflow ('bootmeth_efi' in
this series) but no actual file (grub.cfg) is visible to U-Boot other
than the grub.efi that it boots. But if grub is not used, then the
bootflow may be a file.

We could perhaps use the next U-Boot contributor call to discuss it.

Regards,
Simon

On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > Hi,
> > >
> > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > On 1/19/22 02:43, Simon Glass wrote:
> > [snip]
> > > > > > +Introduction
> > > > > > +------------
> > > > > > +
> > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > +introduces the following concepts:
> > > > > > +
> > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > +
> > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > >
> > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > box.
> > > >
> > > > I want to jump up and down and emphasize this part as well.  While I
> > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > something that has been promised to be implemented.  And that turns the
> > > > bootcmd for platforms that just want to support modern off the shelf
> > > > distros in to something fairly small.
> > >
> > > Sigh...
> > >
> > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> >
> > No one is talking about removing anything else.  But a major part of
> > your motivation here seems to be "discovering what to boot where is a
> > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > status of that off-list).  And I want to emphasize discover.
> 
> But only if you use EFI boot manager, right? Even then I'm not sure we
> have a deterministic way of listing the available bootdevs, which is
> something that this series provides.

I'll let someone else that knows the EFI boot manager code / design
speak to this.  But yes, for the discoverable case I'm not seeing where
"use EFI boot manager" isn't the reasonable answer.

> > > If we get EFI bootmgr going, then are you saying you want to disable
> > > everything else?
> >
> > Not at all.
> 
> OK, good.
> 
> >
> > > You say 'major distros' but there are many that don't use it,
> > > particularly in the embedded space. I'll go out on a limb and say that
> > > the vast majority of embedded devices in the world don't use it. Are
> > > you really saying we should drop support for everything else? Even the
> > > distro stuff supports other options.
> >
> > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > do UEFI boot for U-Boot targets and I would love to talk with someone
> > there and find out why (but I guess it's all the 32bit platforms).
> >
> > But I'd also say the vast majority of embedded devices don't need the
> > complexity you're adding here, but DO need the ability to implement A/B
> > things as easily in U-Boot as they can in grub.  And that in turn is
> > because it's a pain to modify the default environment in U-Boot and easy
> > to drop in another script for grub.
> 
> My feeling is the complexity is already there, just in scripts, which
> are harder to understand (from personal experience trying to
> understand what they do) and don't have tests. They are also very hard
> to build on top of (e.g. verified boot).

Yes, the scripts that live in the environment based boot flow is
complex.  It's also been a huge step forward from what we had before and
has been a great help.

> I can't really say that this series is more complex than EFI bootmgr,
> if that is what you are suggesting. I think the bootdev uclass is
> well-motivated and will prove useful even for EFI.

No, what I'm suggesting is that I see this as "current boot script stuff
is too complex, lets replace it".  And I also see the big part of the
complexity there being the discover where to boot from side of things.

> Also A/B/recovery is a lot easier to implement in code than in
> scripts. I have linked to the proposed design there.

Show me what implementing Mender or RAUC or swupdate looks like with
your proposal.  Those are some of the real A/B use cases today and have
had to take various approaches to dealing with our environment, and also
supporting x86 and so started dealing with grub.

> Anyway if we can agree that we are not going to disable the non-EFI
> flows, then how about we focus on replacing the distro boot scripts,
> dropping the config.h files, and leave EFI bootmgr out of this
> discussion?

The primary use case for the distro boot work is EFI boot, and the "make
this logic clearer" solution is "use EFI bootmgr".  That's where we get
stuck in a loop here.  There's no "the distro must create .." because
the distro is already creating what's needed.

> From: Simon Glass <sjg@chromium.org>
> Date: Fri, 21 Jan 2022 09:53:37 -0700
> 
> Hi Mark,
> 
> On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >
> > > From: Simon Glass <sjg@chromium.org>
> > > Date: Fri, 21 Jan 2022 08:20:17 -0700
> > >
> > > Hi,
> > >
> > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > Add documentation for this feature, including the commands and full
> > > > > > devicetree bindings.
> > > > > >
> > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > ---
> > > > > >
> > > > > > Changes in v3:
> > > > > > - Update docs for "bootmeths" and "boot_targets" env vars
> > > > > >
> > > > > >   MAINTAINERS                           |   4 +
> > > > > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > > > > >   doc/develop/distro.rst                |   3 +
> > > > > >   doc/develop/index.rst                 |   1 +
> > > > > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > > > > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > > > > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > > > > >   doc/usage/bootdev.rst                 | 135 ++++++
> > > > > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > > > > >   doc/usage/bootmeth.rst                | 108 +++++
> > > > > >   doc/usage/index.rst                   |   3 +
> > > > > >   11 files changed, 1376 insertions(+)
> > > > > >   create mode 100644 doc/develop/bootstd.rst
> > > > > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > > > > >   create mode 100644 doc/usage/bootdev.rst
> > > > > >   create mode 100644 doc/usage/bootflow.rst
> > > > > >   create mode 100644 doc/usage/bootmeth.rst
> > > > > >
> > > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > index 8ad70d3d968..c2af8ada3c9 100644
> > > > > > --- a/MAINTAINERS
> > > > > > +++ b/MAINTAINERS
> > > > > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > > > > >   F:        boot/bootstd.c
> > > > > >   F:        cmd/bootdev.c
> > > > > >   F:        cmd/bootflow.c
> > > > > > +F: doc/develop/bootstd.rst
> > > > > > +F: doc/usage/bootdev.rst
> > > > > > +F: doc/usage/bootflow.rst
> > > > > > +F: doc/usage/bootmeth.rst
> > > > > >   F:        drivers/mmc/mmc_bootdev.c
> > > > > >   F:        include/bootdev.h
> > > > > >   F:        include/bootflow.h
> > > > > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > > > > new file mode 100644
> > > > > > index 00000000000..1b65a806efb
> > > > > > --- /dev/null
> > > > > > +++ b/doc/develop/bootstd.rst
> > > > > > @@ -0,0 +1,638 @@
> > > > > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > > > > +
> > > > > > +U-Boot Standard Boot
> > > > > > +====================
> > > > > > +
> > > > > > +Introduction
> > > > > > +------------
> > > > > > +
> > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > +introduces the following concepts:
> > > > > > +
> > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > +
> > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > >
> > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > box.
> > > >
> > > > I want to jump up and down and emphasize this part as well.  While I
> > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > something that has been promised to be implemented.  And that turns the
> > > > bootcmd for platforms that just want to support modern off the shelf
> > > > distros in to something fairly small.
> > >
> > > Sigh...
> > >
> > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > >
> > > If we get EFI bootmgr going, then are you saying you want to disable
> > > everything else?
> > >
> > > You say 'major distros' but there are many that don't use it,
> > > particularly in the embedded space. I'll go out on a limb and say that
> > > the vast majority of embedded devices in the world don't use it. Are
> > > you really saying we should drop support for everything else? Even the
> > > distro stuff supports other options.
> >
> > And U-Boot supports a wide variety of CPUs and some of those don't
> > even have official UEFI support.
> >
> > However, on arm64 (and possibly riscv64) even the embedded folks
> > should seriously consider using the UEFI bootflow.  Linux now supports
> > physical address randomization when loaded via the UEFI stub, which is
> > something that can't really be implemented using the legacy boot path.
> > Note that you don't have to use a separate UEFI bootloader as U-Boot
> > can directly boot kernels with the UEFI stub.
> 
> 'legacy'? Isn't it just a case of relocating the kernel to a random
> address? I'm pretty sure U-Boot can do that :-)

The problem is that the legacy boot protocol for the Linux arm64
kernel requires a 2MB aligned kernel base, which reduces the number of
randomized bits.  That also means that virtual addresses are not fully
randomized as the kernel uses large mappings to map itself.  My
understanding is that the UEFI stub can relocate the kernel to any 64K
aligned address.  I suppose it is possible to add code to achieve the
same thing for the legacy boot path, but I don't think the arm64
maintainers are really interested in doing that.

But yes, U-Boot should certainly try to load arm64 kernels at a random
address instead hardcoding the load address ;)

> Re direct boot, the issue seems to me that distros really want to use
> grub. I think a lot of people talk about direct boot, but it doesn't
> seem to be happening?

I don't think direct boot makes sense for distros.  If you want to
support all variations of UEFI firmware you'll need to install your
kernel on a FAT filesystem.  And that doesn't work well if you let
your package manager manage the kernel.  Using grub is attractive
because x86 users are familliar with it and it offers an interface to
boot different kernels.

I think direct boot is targeted more at the embedded world or perhaps
virtualized environments.

> > > Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
> > > of the box'. Which bit of this series is in conflict with that? What
> > > exactly is "completely wrong" ?? Is it just the wording that is
> > > confusing?
> >
> > Possibly.  The documentation seems to suggest that OSes have to
> > specify a bootflow for U-Boot.  Whereas one of the main advantages of
> > the UEFI bootflow is that this allows OSes not to care whether we're
> > booted by U-Boot, EDK2 or a closed source firmware implementation.  I
> > think the docs should say that the bootflow can be customized by an
> > OS, but that in general this isn't necessary.
> 
> The definition of a bootflow is pretty broad. In the case of grub, it
> isn't even visible to U-Boot so there is a bootflow ('bootmeth_efi' in
> this series) but no actual file (grub.cfg) is visible to U-Boot other
> than the grub.efi that it boots. But if grub is not used, then the
> bootflow may be a file.
> 
> We could perhaps use the next U-Boot contributor call to discuss it.
> 
> Regards,
> Simon
>

On Fri, Jan 21, 2022 at 07:22:55PM +0100, Mark Kettenis wrote:
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 09:53:37 -0700
> > 
> > Hi Mark,
> > 
> > On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> > >
> > > > From: Simon Glass <sjg@chromium.org>
> > > > Date: Fri, 21 Jan 2022 08:20:17 -0700
> > > >
> > > > Hi,
> > > >
> > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > > Add documentation for this feature, including the commands and full
> > > > > > > devicetree bindings.
> > > > > > >
> > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > ---
> > > > > > >
> > > > > > > Changes in v3:
> > > > > > > - Update docs for "bootmeths" and "boot_targets" env vars
> > > > > > >
> > > > > > >   MAINTAINERS                           |   4 +
> > > > > > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > > > > > >   doc/develop/distro.rst                |   3 +
> > > > > > >   doc/develop/index.rst                 |   1 +
> > > > > > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > > > > > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > > > > > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > > > > > >   doc/usage/bootdev.rst                 | 135 ++++++
> > > > > > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > > > > > >   doc/usage/bootmeth.rst                | 108 +++++
> > > > > > >   doc/usage/index.rst                   |   3 +
> > > > > > >   11 files changed, 1376 insertions(+)
> > > > > > >   create mode 100644 doc/develop/bootstd.rst
> > > > > > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > > > > > >   create mode 100644 doc/usage/bootdev.rst
> > > > > > >   create mode 100644 doc/usage/bootflow.rst
> > > > > > >   create mode 100644 doc/usage/bootmeth.rst
> > > > > > >
> > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > index 8ad70d3d968..c2af8ada3c9 100644
> > > > > > > --- a/MAINTAINERS
> > > > > > > +++ b/MAINTAINERS
> > > > > > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > > > > > >   F:        boot/bootstd.c
> > > > > > >   F:        cmd/bootdev.c
> > > > > > >   F:        cmd/bootflow.c
> > > > > > > +F: doc/develop/bootstd.rst
> > > > > > > +F: doc/usage/bootdev.rst
> > > > > > > +F: doc/usage/bootflow.rst
> > > > > > > +F: doc/usage/bootmeth.rst
> > > > > > >   F:        drivers/mmc/mmc_bootdev.c
> > > > > > >   F:        include/bootdev.h
> > > > > > >   F:        include/bootflow.h
> > > > > > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > > > > > new file mode 100644
> > > > > > > index 00000000000..1b65a806efb
> > > > > > > --- /dev/null
> > > > > > > +++ b/doc/develop/bootstd.rst
> > > > > > > @@ -0,0 +1,638 @@
> > > > > > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > > > > > +
> > > > > > > +U-Boot Standard Boot
> > > > > > > +====================
> > > > > > > +
> > > > > > > +Introduction
> > > > > > > +------------
> > > > > > > +
> > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > +introduces the following concepts:
> > > > > > > +
> > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > +
> > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > >
> > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > box.
> > > > >
> > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > something that has been promised to be implemented.  And that turns the
> > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > distros in to something fairly small.
> > > >
> > > > Sigh...
> > > >
> > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > >
> > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > everything else?
> > > >
> > > > You say 'major distros' but there are many that don't use it,
> > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > the vast majority of embedded devices in the world don't use it. Are
> > > > you really saying we should drop support for everything else? Even the
> > > > distro stuff supports other options.
> > >
> > > And U-Boot supports a wide variety of CPUs and some of those don't
> > > even have official UEFI support.
> > >
> > > However, on arm64 (and possibly riscv64) even the embedded folks
> > > should seriously consider using the UEFI bootflow.  Linux now supports
> > > physical address randomization when loaded via the UEFI stub, which is
> > > something that can't really be implemented using the legacy boot path.
> > > Note that you don't have to use a separate UEFI bootloader as U-Boot
> > > can directly boot kernels with the UEFI stub.
> > 
> > 'legacy'? Isn't it just a case of relocating the kernel to a random
> > address? I'm pretty sure U-Boot can do that :-)
> 
> The problem is that the legacy boot protocol for the Linux arm64
> kernel requires a 2MB aligned kernel base, which reduces the number of
> randomized bits.  That also means that virtual addresses are not fully
> randomized as the kernel uses large mappings to map itself.  My
> understanding is that the UEFI stub can relocate the kernel to any 64K
> aligned address.  I suppose it is possible to add code to achieve the
> same thing for the legacy boot path, but I don't think the arm64
> maintainers are really interested in doing that.
> 
> But yes, U-Boot should certainly try to load arm64 kernels at a random
> address instead hardcoding the load address ;)

Well, it sounds like "bootefi" should be used over "booti" directly,
these days on aarch64 kernels.

Hi Tom,

On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > Hi,
> > > >
> > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > [snip]
> > > > > > > +Introduction
> > > > > > > +------------
> > > > > > > +
> > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > +introduces the following concepts:
> > > > > > > +
> > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > +
> > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > >
> > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > box.
> > > > >
> > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > something that has been promised to be implemented.  And that turns the
> > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > distros in to something fairly small.
> > > >
> > > > Sigh...
> > > >
> > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > >
> > > No one is talking about removing anything else.  But a major part of
> > > your motivation here seems to be "discovering what to boot where is a
> > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > status of that off-list).  And I want to emphasize discover.
> >
> > But only if you use EFI boot manager, right? Even then I'm not sure we
> > have a deterministic way of listing the available bootdevs, which is
> > something that this series provides.
>
> I'll let someone else that knows the EFI boot manager code / design
> speak to this.  But yes, for the discoverable case I'm not seeing where
> "use EFI boot manager" isn't the reasonable answer.

With bootdev you have a proper driver and device tree node where
configuration can be provided. The default ordering for bootdevs can
be provided. We can deal with the quirks of particular subsystems,
like MMC. I think there will be other benefits apparent as this all
matures.

But let's see. Perhaps it doesn't really matter anyway, since it seems
that EFI boot manager is doing its own thing for now.

>
> > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > everything else?
> > >
> > > Not at all.
> >
> > OK, good.
> >
> > >
> > > > You say 'major distros' but there are many that don't use it,
> > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > the vast majority of embedded devices in the world don't use it. Are
> > > > you really saying we should drop support for everything else? Even the
> > > > distro stuff supports other options.
> > >
> > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > there and find out why (but I guess it's all the 32bit platforms).
> > >
> > > But I'd also say the vast majority of embedded devices don't need the
> > > complexity you're adding here, but DO need the ability to implement A/B
> > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > because it's a pain to modify the default environment in U-Boot and easy
> > > to drop in another script for grub.
> >
> > My feeling is the complexity is already there, just in scripts, which
> > are harder to understand (from personal experience trying to
> > understand what they do) and don't have tests. They are also very hard
> > to build on top of (e.g. verified boot).
>
> Yes, the scripts that live in the environment based boot flow is
> complex.  It's also been a huge step forward from what we had before and
> has been a great help.
>
> > I can't really say that this series is more complex than EFI bootmgr,
> > if that is what you are suggesting. I think the bootdev uclass is
> > well-motivated and will prove useful even for EFI.
>
> No, what I'm suggesting is that I see this as "current boot script stuff
> is too complex, lets replace it".  And I also see the big part of the
> complexity there being the discover where to boot from side of things.

OK, so shall we replace the scripts, or not?

>
> > Also A/B/recovery is a lot easier to implement in code than in
> > scripts. I have linked to the proposed design there.
>
> Show me what implementing Mender or RAUC or swupdate looks like with
> your proposal.  Those are some of the real A/B use cases today and have
> had to take various approaches to dealing with our environment, and also
> supporting x86 and so started dealing with grub.

That sounds like an interesting project. For mendor I found this:

https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch

More scripts...

>
> > Anyway if we can agree that we are not going to disable the non-EFI
> > flows, then how about we focus on replacing the distro boot scripts,
> > dropping the config.h files, and leave EFI bootmgr out of this
> > discussion?
>
> The primary use case for the distro boot work is EFI boot, and the "make
> this logic clearer" solution is "use EFI bootmgr".  That's where we get
> stuck in a loop here.  There's no "the distro must create .." because
> the distro is already creating what's needed.

So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
support? I thought you said no.

Are you saying you want to keep the environment boot scripts, or not?

Regards,
Simon

Hi Mark,

On Fri, 21 Jan 2022 at 11:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Simon Glass <sjg@chromium.org>
> > Date: Fri, 21 Jan 2022 09:53:37 -0700
> >
> > Hi Mark,
> >
> > On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> > >
> > > > From: Simon Glass <sjg@chromium.org>
> > > > Date: Fri, 21 Jan 2022 08:20:17 -0700
> > > >
> > > > Hi,
> > > >
> > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > > Add documentation for this feature, including the commands and full
> > > > > > > devicetree bindings.
> > > > > > >
> > > > > > > Signed-off-by: Simon Glass <sjg@chromium.org>
> > > > > > > ---
> > > > > > >
> > > > > > > Changes in v3:
> > > > > > > - Update docs for "bootmeths" and "boot_targets" env vars
> > > > > > >
> > > > > > >   MAINTAINERS                           |   4 +
> > > > > > >   doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> > > > > > >   doc/develop/distro.rst                |   3 +
> > > > > > >   doc/develop/index.rst                 |   1 +
> > > > > > >   doc/device-tree-bindings/bootdev.txt  |  18 +
> > > > > > >   doc/device-tree-bindings/bootmeth.txt |  31 ++
> > > > > > >   doc/device-tree-bindings/bootstd.txt  |   8 +
> > > > > > >   doc/usage/bootdev.rst                 | 135 ++++++
> > > > > > >   doc/usage/bootflow.rst                | 427 +++++++++++++++++
> > > > > > >   doc/usage/bootmeth.rst                | 108 +++++
> > > > > > >   doc/usage/index.rst                   |   3 +
> > > > > > >   11 files changed, 1376 insertions(+)
> > > > > > >   create mode 100644 doc/develop/bootstd.rst
> > > > > > >   create mode 100644 doc/device-tree-bindings/bootmeth.txt
> > > > > > >   create mode 100644 doc/usage/bootdev.rst
> > > > > > >   create mode 100644 doc/usage/bootflow.rst
> > > > > > >   create mode 100644 doc/usage/bootmeth.rst
> > > > > > >
> > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > > > > index 8ad70d3d968..c2af8ada3c9 100644
> > > > > > > --- a/MAINTAINERS
> > > > > > > +++ b/MAINTAINERS
> > > > > > > @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> > > > > > >   F:        boot/bootstd.c
> > > > > > >   F:        cmd/bootdev.c
> > > > > > >   F:        cmd/bootflow.c
> > > > > > > +F: doc/develop/bootstd.rst
> > > > > > > +F: doc/usage/bootdev.rst
> > > > > > > +F: doc/usage/bootflow.rst
> > > > > > > +F: doc/usage/bootmeth.rst
> > > > > > >   F:        drivers/mmc/mmc_bootdev.c
> > > > > > >   F:        include/bootdev.h
> > > > > > >   F:        include/bootflow.h
> > > > > > > diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> > > > > > > new file mode 100644
> > > > > > > index 00000000000..1b65a806efb
> > > > > > > --- /dev/null
> > > > > > > +++ b/doc/develop/bootstd.rst
> > > > > > > @@ -0,0 +1,638 @@
> > > > > > > +.. SPDX-License-Identifier: GPL-2.0+:
> > > > > > > +
> > > > > > > +U-Boot Standard Boot
> > > > > > > +====================
> > > > > > > +
> > > > > > > +Introduction
> > > > > > > +------------
> > > > > > > +
> > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > +introduces the following concepts:
> > > > > > > +
> > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > +
> > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > >
> > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > box.
> > > > >
> > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > something that has been promised to be implemented.  And that turns the
> > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > distros in to something fairly small.
> > > >
> > > > Sigh...
> > > >
> > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > >
> > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > everything else?
> > > >
> > > > You say 'major distros' but there are many that don't use it,
> > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > the vast majority of embedded devices in the world don't use it. Are
> > > > you really saying we should drop support for everything else? Even the
> > > > distro stuff supports other options.
> > >
> > > And U-Boot supports a wide variety of CPUs and some of those don't
> > > even have official UEFI support.
> > >
> > > However, on arm64 (and possibly riscv64) even the embedded folks
> > > should seriously consider using the UEFI bootflow.  Linux now supports
> > > physical address randomization when loaded via the UEFI stub, which is
> > > something that can't really be implemented using the legacy boot path.
> > > Note that you don't have to use a separate UEFI bootloader as U-Boot
> > > can directly boot kernels with the UEFI stub.
> >
> > 'legacy'? Isn't it just a case of relocating the kernel to a random
> > address? I'm pretty sure U-Boot can do that :-)
>
> The problem is that the legacy boot protocol for the Linux arm64
> kernel requires a 2MB aligned kernel base, which reduces the number of
> randomized bits.  That also means that virtual addresses are not fully
> randomized as the kernel uses large mappings to map itself.  My
> understanding is that the UEFI stub can relocate the kernel to any 64K
> aligned address.  I suppose it is possible to add code to achieve the
> same thing for the legacy boot path, but I don't think the arm64
> maintainers are really interested in doing that.
>
> But yes, U-Boot should certainly try to load arm64 kernels at a random
> address instead hardcoding the load address ;)

This is simply a design decision. Linux wants to be able to do
everything in its own tree, hence the decompression code, ALSR, etc.
I'm not suggesting we change it, just that it could be done if people
wanted it.

Another example is x86 kernels. U-Boot supports putting those in a FIT
(even an uncompressed 64-bit kernel) which is helpful for signing /
verified boot.

>
> > Re direct boot, the issue seems to me that distros really want to use
> > grub. I think a lot of people talk about direct boot, but it doesn't
> > seem to be happening?
>
> I don't think direct boot makes sense for distros.  If you want to
> support all variations of UEFI firmware you'll need to install your
> kernel on a FAT filesystem.  And that doesn't work well if you let
> your package manager manage the kernel.  Using grub is attractive
> because x86 users are familliar with it and it offers an interface to
> boot different kernels.
>
> I think direct boot is targeted more at the embedded world or perhaps
> virtualized environments.

Sounds right. But I keep asking if people have just given up on
embedded/custom flows and want U-Boot to just be a UEFI runner?

>
> > > > Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
> > > > of the box'. Which bit of this series is in conflict with that? What
> > > > exactly is "completely wrong" ?? Is it just the wording that is
> > > > confusing?
> > >
> > > Possibly.  The documentation seems to suggest that OSes have to
> > > specify a bootflow for U-Boot.  Whereas one of the main advantages of
> > > the UEFI bootflow is that this allows OSes not to care whether we're
> > > booted by U-Boot, EDK2 or a closed source firmware implementation.  I
> > > think the docs should say that the bootflow can be customized by an
> > > OS, but that in general this isn't necessary.
> >
> > The definition of a bootflow is pretty broad. In the case of grub, it
> > isn't even visible to U-Boot so there is a bootflow ('bootmeth_efi' in
> > this series) but no actual file (grub.cfg) is visible to U-Boot other
> > than the grub.efi that it boots. But if grub is not used, then the
> > bootflow may be a file.
> >
> > We could perhaps use the next U-Boot contributor call to discuss it.

Regards,
SImon

On Fri, Jan 21, 2022 at 12:14:22PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > > Hi,
> > > > >
> > > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > [snip]
> > > > > > > > +Introduction
> > > > > > > > +------------
> > > > > > > > +
> > > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > > +introduces the following concepts:
> > > > > > > > +
> > > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > > +
> > > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > > >
> > > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > > box.
> > > > > >
> > > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > > something that has been promised to be implemented.  And that turns the
> > > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > > distros in to something fairly small.
> > > > >
> > > > > Sigh...
> > > > >
> > > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > >
> > > > No one is talking about removing anything else.  But a major part of
> > > > your motivation here seems to be "discovering what to boot where is a
> > > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > > status of that off-list).  And I want to emphasize discover.
> > >
> > > But only if you use EFI boot manager, right? Even then I'm not sure we
> > > have a deterministic way of listing the available bootdevs, which is
> > > something that this series provides.
> >
> > I'll let someone else that knows the EFI boot manager code / design
> > speak to this.  But yes, for the discoverable case I'm not seeing where
> > "use EFI boot manager" isn't the reasonable answer.
> 
> With bootdev you have a proper driver and device tree node where
> configuration can be provided. The default ordering for bootdevs can
> be provided. We can deal with the quirks of particular subsystems,
> like MMC. I think there will be other benefits apparent as this all
> matures.
> 
> But let's see. Perhaps it doesn't really matter anyway, since it seems
> that EFI boot manager is doing its own thing for now.
> 
> >
> > > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > > everything else?
> > > >
> > > > Not at all.
> > >
> > > OK, good.
> > >
> > > >
> > > > > You say 'major distros' but there are many that don't use it,
> > > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > > the vast majority of embedded devices in the world don't use it. Are
> > > > > you really saying we should drop support for everything else? Even the
> > > > > distro stuff supports other options.
> > > >
> > > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > > there and find out why (but I guess it's all the 32bit platforms).
> > > >
> > > > But I'd also say the vast majority of embedded devices don't need the
> > > > complexity you're adding here, but DO need the ability to implement A/B
> > > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > > because it's a pain to modify the default environment in U-Boot and easy
> > > > to drop in another script for grub.
> > >
> > > My feeling is the complexity is already there, just in scripts, which
> > > are harder to understand (from personal experience trying to
> > > understand what they do) and don't have tests. They are also very hard
> > > to build on top of (e.g. verified boot).
> >
> > Yes, the scripts that live in the environment based boot flow is
> > complex.  It's also been a huge step forward from what we had before and
> > has been a great help.
> >
> > > I can't really say that this series is more complex than EFI bootmgr,
> > > if that is what you are suggesting. I think the bootdev uclass is
> > > well-motivated and will prove useful even for EFI.
> >
> > No, what I'm suggesting is that I see this as "current boot script stuff
> > is too complex, lets replace it".  And I also see the big part of the
> > complexity there being the discover where to boot from side of things.
> 
> OK, so shall we replace the scripts, or not?

I'm still looking to be convinced that something is better than the
scripts, or that the problem is the scripts and not the pain involved
with modifying the U-Boot environment.

> > > Also A/B/recovery is a lot easier to implement in code than in
> > > scripts. I have linked to the proposed design there.
> >
> > Show me what implementing Mender or RAUC or swupdate looks like with
> > your proposal.  Those are some of the real A/B use cases today and have
> > had to take various approaches to dealing with our environment, and also
> > supporting x86 and so started dealing with grub.
> 
> That sounds like an interesting project. For mendor I found this:
> 
> https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch
> 
> More scripts...

Yes.  Mender, RAUC and swupdate (iirc) all have some form of environment
based bootcmd to Do The Right Thing and boot A or B or detect failure
and fall back.  Show me how what you're doing improves integration for
that case.  That's a case that's not covered by "use UEFI boot manager"
directly.  I know for Mender a huge pain point for integration is "drop
_this_ script in to the default environment".  RAUC is similar.  Show me
something that makes that much easier to integrate, like it is with
grub.

> > > Anyway if we can agree that we are not going to disable the non-EFI
> > > flows, then how about we focus on replacing the distro boot scripts,
> > > dropping the config.h files, and leave EFI bootmgr out of this
> > > discussion?
> >
> > The primary use case for the distro boot work is EFI boot, and the "make
> > this logic clearer" solution is "use EFI bootmgr".  That's where we get
> > stuck in a loop here.  There's no "the distro must create .." because
> > the distro is already creating what's needed.
> 
> So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
> support? I thought you said no.

Maybe we're talking at cross purposes.  I'm not going to drop
booti/bootm/bootz/bootelf.  I'm not going to drop setting bootcmd to
whatever the user / board developer knows is best for them.

> Are you saying you want to keep the environment boot scripts, or not?

As I said in the previous iteration on this series, I'm not convinced
this is a win over other clean-ups that can be done with what we have
today, or that it solves the problems that I often see popping up.

Hi Tom,

On Fri, 21 Jan 2022 at 12:23, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 21, 2022 at 12:14:22PM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > > > Hi,
> > > > > >
> > > > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > [snip]
> > > > > > > > > +Introduction
> > > > > > > > > +------------
> > > > > > > > > +
> > > > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > > > +introduces the following concepts:
> > > > > > > > > +
> > > > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > > > +
> > > > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > > > >
> > > > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > > > box.
> > > > > > >
> > > > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > > > something that has been promised to be implemented.  And that turns the
> > > > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > > > distros in to something fairly small.
> > > > > >
> > > > > > Sigh...
> > > > > >
> > > > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > > >
> > > > > No one is talking about removing anything else.  But a major part of
> > > > > your motivation here seems to be "discovering what to boot where is a
> > > > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > > > status of that off-list).  And I want to emphasize discover.
> > > >
> > > > But only if you use EFI boot manager, right? Even then I'm not sure we
> > > > have a deterministic way of listing the available bootdevs, which is
> > > > something that this series provides.
> > >
> > > I'll let someone else that knows the EFI boot manager code / design
> > > speak to this.  But yes, for the discoverable case I'm not seeing where
> > > "use EFI boot manager" isn't the reasonable answer.
> >
> > With bootdev you have a proper driver and device tree node where
> > configuration can be provided. The default ordering for bootdevs can
> > be provided. We can deal with the quirks of particular subsystems,
> > like MMC. I think there will be other benefits apparent as this all
> > matures.
> >
> > But let's see. Perhaps it doesn't really matter anyway, since it seems
> > that EFI boot manager is doing its own thing for now.
> >
> > >
> > > > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > > > everything else?
> > > > >
> > > > > Not at all.
> > > >
> > > > OK, good.
> > > >
> > > > >
> > > > > > You say 'major distros' but there are many that don't use it,
> > > > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > > > the vast majority of embedded devices in the world don't use it. Are
> > > > > > you really saying we should drop support for everything else? Even the
> > > > > > distro stuff supports other options.
> > > > >
> > > > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > > > there and find out why (but I guess it's all the 32bit platforms).
> > > > >
> > > > > But I'd also say the vast majority of embedded devices don't need the
> > > > > complexity you're adding here, but DO need the ability to implement A/B
> > > > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > > > because it's a pain to modify the default environment in U-Boot and easy
> > > > > to drop in another script for grub.
> > > >
> > > > My feeling is the complexity is already there, just in scripts, which
> > > > are harder to understand (from personal experience trying to
> > > > understand what they do) and don't have tests. They are also very hard
> > > > to build on top of (e.g. verified boot).
> > >
> > > Yes, the scripts that live in the environment based boot flow is
> > > complex.  It's also been a huge step forward from what we had before and
> > > has been a great help.
> > >
> > > > I can't really say that this series is more complex than EFI bootmgr,
> > > > if that is what you are suggesting. I think the bootdev uclass is
> > > > well-motivated and will prove useful even for EFI.
> > >
> > > No, what I'm suggesting is that I see this as "current boot script stuff
> > > is too complex, lets replace it".  And I also see the big part of the
> > > complexity there being the discover where to boot from side of things.
> >
> > OK, so shall we replace the scripts, or not?
>
> I'm still looking to be convinced that something is better than the
> scripts, or that the problem is the scripts and not the pain involved
> with modifying the U-Boot environment.

Well luckily, someone has gone to all the trouble of providing an
alternative that we can try :-)

>
> > > > Also A/B/recovery is a lot easier to implement in code than in
> > > > scripts. I have linked to the proposed design there.
> > >
> > > Show me what implementing Mender or RAUC or swupdate looks like with
> > > your proposal.  Those are some of the real A/B use cases today and have
> > > had to take various approaches to dealing with our environment, and also
> > > supporting x86 and so started dealing with grub.
> >
> > That sounds like an interesting project. For mendor I found this:
> >
> > https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch
> >
> > More scripts...
>
> Yes.  Mender, RAUC and swupdate (iirc) all have some form of environment
> based bootcmd to Do The Right Thing and boot A or B or detect failure
> and fall back.  Show me how what you're doing improves integration for
> that case.  That's a case that's not covered by "use UEFI boot manager"
> directly.  I know for Mender a huge pain point for integration is "drop
> _this_ script in to the default environment".  RAUC is similar.  Show me
> something that makes that much easier to integrate, like it is with
> grub.

The obvious answer would be to create a boot method for Mender. In
that you can do anything, including using the standard bootmethods to
obtain things to boot, etc. Also perhaps a 'recovery' bootdev which is
invoked last, when other things have failed, or first if recovery mode
is selected. Both could be done without any scripting, so could be
enabled on any board with just enabling CONFIG_MENDER, I suspect.

>
> > > > Anyway if we can agree that we are not going to disable the non-EFI
> > > > flows, then how about we focus on replacing the distro boot scripts,
> > > > dropping the config.h files, and leave EFI bootmgr out of this
> > > > discussion?
> > >
> > > The primary use case for the distro boot work is EFI boot, and the "make
> > > this logic clearer" solution is "use EFI bootmgr".  That's where we get
> > > stuck in a loop here.  There's no "the distro must create .." because
> > > the distro is already creating what's needed.
> >
> > So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
> > support? I thought you said no.
>
> Maybe we're talking at cross purposes.  I'm not going to drop
> booti/bootm/bootz/bootelf.  I'm not going to drop setting bootcmd to
> whatever the user / board developer knows is best for them.

Maybe. My confusion is that EFI seems to be blocking progress in the
general booting domain. When innovation is suggested, the answer is
'well EFI does this so why bother'. When the boot scripts are
mentioned, we can apparently limp along with them and iterate there,
since EFI is the only thing that matters anyway.

Perhaps the real difference here is that I am not focused on EFI as
the solution to all things. I know that is where many distros are and
I believe you feel I am wasting my time. But that is where I am. From
my vantage point I see a lot of improvements we can make with booting,
independent of EFI.

>
> > Are you saying you want to keep the environment boot scripts, or not?
>
> As I said in the previous iteration on this series, I'm not convinced
> this is a win over other clean-ups that can be done with what we have
> today, or that it solves the problems that I often see popping up.

Previously you were worried about the size growth. I am pretty
confident this will make a lot of boot-related things more structured
and easier over the medium term. But we do need something to build on.
I believe the concepts are sound, and applicable to various boot
scenarios. In fact I think the lack of a 'bootdev' is something we
should have thought to create a while back.

I certainly think it is better than building more and more on the boot scripts.

Regards,
Simon

On Fri, Jan 21, 2022 at 02:15:24PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 21 Jan 2022 at 12:23, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Fri, Jan 21, 2022 at 12:14:22PM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > [snip]
> > > > > > > > > > +Introduction
> > > > > > > > > > +------------
> > > > > > > > > > +
> > > > > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > > > > +introduces the following concepts:
> > > > > > > > > > +
> > > > > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > > > > +
> > > > > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > > > > >
> > > > > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > > > > box.
> > > > > > > >
> > > > > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > > > > something that has been promised to be implemented.  And that turns the
> > > > > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > > > > distros in to something fairly small.
> > > > > > >
> > > > > > > Sigh...
> > > > > > >
> > > > > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > > > >
> > > > > > No one is talking about removing anything else.  But a major part of
> > > > > > your motivation here seems to be "discovering what to boot where is a
> > > > > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > > > > status of that off-list).  And I want to emphasize discover.
> > > > >
> > > > > But only if you use EFI boot manager, right? Even then I'm not sure we
> > > > > have a deterministic way of listing the available bootdevs, which is
> > > > > something that this series provides.
> > > >
> > > > I'll let someone else that knows the EFI boot manager code / design
> > > > speak to this.  But yes, for the discoverable case I'm not seeing where
> > > > "use EFI boot manager" isn't the reasonable answer.
> > >
> > > With bootdev you have a proper driver and device tree node where
> > > configuration can be provided. The default ordering for bootdevs can
> > > be provided. We can deal with the quirks of particular subsystems,
> > > like MMC. I think there will be other benefits apparent as this all
> > > matures.
> > >
> > > But let's see. Perhaps it doesn't really matter anyway, since it seems
> > > that EFI boot manager is doing its own thing for now.
> > >
> > > >
> > > > > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > > > > everything else?
> > > > > >
> > > > > > Not at all.
> > > > >
> > > > > OK, good.
> > > > >
> > > > > >
> > > > > > > You say 'major distros' but there are many that don't use it,
> > > > > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > > > > the vast majority of embedded devices in the world don't use it. Are
> > > > > > > you really saying we should drop support for everything else? Even the
> > > > > > > distro stuff supports other options.
> > > > > >
> > > > > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > > > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > > > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > > > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > > > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > > > > there and find out why (but I guess it's all the 32bit platforms).
> > > > > >
> > > > > > But I'd also say the vast majority of embedded devices don't need the
> > > > > > complexity you're adding here, but DO need the ability to implement A/B
> > > > > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > > > > because it's a pain to modify the default environment in U-Boot and easy
> > > > > > to drop in another script for grub.
> > > > >
> > > > > My feeling is the complexity is already there, just in scripts, which
> > > > > are harder to understand (from personal experience trying to
> > > > > understand what they do) and don't have tests. They are also very hard
> > > > > to build on top of (e.g. verified boot).
> > > >
> > > > Yes, the scripts that live in the environment based boot flow is
> > > > complex.  It's also been a huge step forward from what we had before and
> > > > has been a great help.
> > > >
> > > > > I can't really say that this series is more complex than EFI bootmgr,
> > > > > if that is what you are suggesting. I think the bootdev uclass is
> > > > > well-motivated and will prove useful even for EFI.
> > > >
> > > > No, what I'm suggesting is that I see this as "current boot script stuff
> > > > is too complex, lets replace it".  And I also see the big part of the
> > > > complexity there being the discover where to boot from side of things.
> > >
> > > OK, so shall we replace the scripts, or not?
> >
> > I'm still looking to be convinced that something is better than the
> > scripts, or that the problem is the scripts and not the pain involved
> > with modifying the U-Boot environment.
> 
> Well luckily, someone has gone to all the trouble of providing an
> alternative that we can try :-)
> 
> >
> > > > > Also A/B/recovery is a lot easier to implement in code than in
> > > > > scripts. I have linked to the proposed design there.
> > > >
> > > > Show me what implementing Mender or RAUC or swupdate looks like with
> > > > your proposal.  Those are some of the real A/B use cases today and have
> > > > had to take various approaches to dealing with our environment, and also
> > > > supporting x86 and so started dealing with grub.
> > >
> > > That sounds like an interesting project. For mendor I found this:
> > >
> > > https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch
> > >
> > > More scripts...
> >
> > Yes.  Mender, RAUC and swupdate (iirc) all have some form of environment
> > based bootcmd to Do The Right Thing and boot A or B or detect failure
> > and fall back.  Show me how what you're doing improves integration for
> > that case.  That's a case that's not covered by "use UEFI boot manager"
> > directly.  I know for Mender a huge pain point for integration is "drop
> > _this_ script in to the default environment".  RAUC is similar.  Show me
> > something that makes that much easier to integrate, like it is with
> > grub.
> 
> The obvious answer would be to create a boot method for Mender. In
> that you can do anything, including using the standard bootmethods to
> obtain things to boot, etc. Also perhaps a 'recovery' bootdev which is
> invoked last, when other things have failed, or first if recovery mode
> is selected. Both could be done without any scripting, so could be
> enabled on any board with just enabling CONFIG_MENDER, I suspect.

OK.  So please show that this will improve things for this case, rather
than speculate.

> > > > > Anyway if we can agree that we are not going to disable the non-EFI
> > > > > flows, then how about we focus on replacing the distro boot scripts,
> > > > > dropping the config.h files, and leave EFI bootmgr out of this
> > > > > discussion?
> > > >
> > > > The primary use case for the distro boot work is EFI boot, and the "make
> > > > this logic clearer" solution is "use EFI bootmgr".  That's where we get
> > > > stuck in a loop here.  There's no "the distro must create .." because
> > > > the distro is already creating what's needed.
> > >
> > > So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
> > > support? I thought you said no.
> >
> > Maybe we're talking at cross purposes.  I'm not going to drop
> > booti/bootm/bootz/bootelf.  I'm not going to drop setting bootcmd to
> > whatever the user / board developer knows is best for them.
> 
> Maybe. My confusion is that EFI seems to be blocking progress in the
> general booting domain. When innovation is suggested, the answer is
> 'well EFI does this so why bother'. When the boot scripts are
> mentioned, we can apparently limp along with them and iterate there,
> since EFI is the only thing that matters anyway.
> 
> Perhaps the real difference here is that I am not focused on EFI as
> the solution to all things. I know that is where many distros are and
> I believe you feel I am wasting my time. But that is where I am. From
> my vantage point I see a lot of improvements we can make with booting,
> independent of EFI.

My high level concern is that yes, I don't see this as improving
anything yet.  This isn't an improvement over what EFI has for booting
generic distributions.  And this one of the areas where the boot scripts
complex, but can be simplified.  Another area where they could be
simplified is for any of the existing A/B style update mechanisms.
That's an interesting case that could show value here.

> > > Are you saying you want to keep the environment boot scripts, or not?
> >
> > As I said in the previous iteration on this series, I'm not convinced
> > this is a win over other clean-ups that can be done with what we have
> > today, or that it solves the problems that I often see popping up.
> 
> Previously you were worried about the size growth. I am pretty
> confident this will make a lot of boot-related things more structured
> and easier over the medium term. But we do need something to build on.
> I believe the concepts are sound, and applicable to various boot
> scenarios. In fact I think the lack of a 'bootdev' is something we
> should have thought to create a while back.
> 
> I certainly think it is better than building more and more on the boot scripts.

Alright, so rpi_3_32b grew as:
       arm: (for 1/1 boards) all +6908.0 data +920.0 rodata -2664.0 text +8652.0
            rpi_3_32b      : all +6908 data +920 rodata -2664 text +8652
               u-boot: add: 101/0, grow: 3/-2 bytes: 8662/-160 (8502)
with this series, and I corrected the minor problems due to '@return'
becoming "Return:".  That's a problem.  Environment space is cheap (it's
a file, or it needs to have some pretty big these days alignment
requirements to have physical redundancy, if you need that).  The
biggest problem I see with boot scripts is that defining them in a
header file where we have to deal with escaping stuff in a header, and
you resolved that with the environment cleanup series you did before.
The series (that I keep failing to find the time to reply to, but was
happy to see!) that updates hush to a modern copy is another good step
in the right direction.  Being able to write the script but more as
plain text rather than escaped strings in C is I suspect why Armbian
does what it does.

On 1/21/22 20:17, Simon Glass wrote:
> Hi Mark,
> 
> On Fri, 21 Jan 2022 at 11:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>>
>>> From: Simon Glass <sjg@chromium.org>
>>> Date: Fri, 21 Jan 2022 09:53:37 -0700
>>>
>>> Hi Mark,
>>>
>>> On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>>>>
>>>>> From: Simon Glass <sjg@chromium.org>
>>>>> Date: Fri, 21 Jan 2022 08:20:17 -0700
>>>>>
>>>>> Hi,
>>>>>
>>>>> On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
>>>>>>
>>>>>> On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
>>>>>>> On 1/19/22 02:43, Simon Glass wrote:
>>>>>>>> Add documentation for this feature, including the commands and full
>>>>>>>> devicetree bindings.
>>>>>>>>
>>>>>>>> Signed-off-by: Simon Glass <sjg@chromium.org>
>>>>>>>> ---
>>>>>>>>
>>>>>>>> Changes in v3:
>>>>>>>> - Update docs for "bootmeths" and "boot_targets" env vars
>>>>>>>>
>>>>>>>>    MAINTAINERS                           |   4 +
>>>>>>>>    doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
>>>>>>>>    doc/develop/distro.rst                |   3 +
>>>>>>>>    doc/develop/index.rst                 |   1 +
>>>>>>>>    doc/device-tree-bindings/bootdev.txt  |  18 +
>>>>>>>>    doc/device-tree-bindings/bootmeth.txt |  31 ++
>>>>>>>>    doc/device-tree-bindings/bootstd.txt  |   8 +
>>>>>>>>    doc/usage/bootdev.rst                 | 135 ++++++
>>>>>>>>    doc/usage/bootflow.rst                | 427 +++++++++++++++++
>>>>>>>>    doc/usage/bootmeth.rst                | 108 +++++
>>>>>>>>    doc/usage/index.rst                   |   3 +
>>>>>>>>    11 files changed, 1376 insertions(+)
>>>>>>>>    create mode 100644 doc/develop/bootstd.rst
>>>>>>>>    create mode 100644 doc/device-tree-bindings/bootmeth.txt
>>>>>>>>    create mode 100644 doc/usage/bootdev.rst
>>>>>>>>    create mode 100644 doc/usage/bootflow.rst
>>>>>>>>    create mode 100644 doc/usage/bootmeth.rst
>>>>>>>>
>>>>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>>>>> index 8ad70d3d968..c2af8ada3c9 100644
>>>>>>>> --- a/MAINTAINERS
>>>>>>>> +++ b/MAINTAINERS
>>>>>>>> @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
>>>>>>>>    F:        boot/bootstd.c
>>>>>>>>    F:        cmd/bootdev.c
>>>>>>>>    F:        cmd/bootflow.c
>>>>>>>> +F: doc/develop/bootstd.rst
>>>>>>>> +F: doc/usage/bootdev.rst
>>>>>>>> +F: doc/usage/bootflow.rst
>>>>>>>> +F: doc/usage/bootmeth.rst
>>>>>>>>    F:        drivers/mmc/mmc_bootdev.c
>>>>>>>>    F:        include/bootdev.h
>>>>>>>>    F:        include/bootflow.h
>>>>>>>> diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
>>>>>>>> new file mode 100644
>>>>>>>> index 00000000000..1b65a806efb
>>>>>>>> --- /dev/null
>>>>>>>> +++ b/doc/develop/bootstd.rst
>>>>>>>> @@ -0,0 +1,638 @@
>>>>>>>> +.. SPDX-License-Identifier: GPL-2.0+:
>>>>>>>> +
>>>>>>>> +U-Boot Standard Boot
>>>>>>>> +====================
>>>>>>>> +
>>>>>>>> +Introduction
>>>>>>>> +------------
>>>>>>>> +
>>>>>>>> +Standard boot provides a built-in way for U-Boot to automatically boot
>>>>>>>> +an Operating System without custom scripting and other customisation. It
>>>>>>>> +introduces the following concepts:
>>>>>>>> +
>>>>>>>> +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
>>>>>>>> +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
>>>>>>>> +   - bootflow - a description of how to boot (provided by the distro)
>>>>>>>> +
>>>>>>>> +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
>>>>>>>> +for creating a bootflow for each kernel combination that it wants to offer.
>>>>>>>
>>>>>>> This gets it completely wrong. There is one standardized boot flow: UEFI.
>>>>>>> All major distros support this. U-Boot has to offer UEFI booting out of the
>>>>>>> box.
>>>>>>
>>>>>> I want to jump up and down and emphasize this part as well.  While I
>>>>>> believe our UEFI bootmgr is still missing the normal scan code, that's
>>>>>> something that has been promised to be implemented.  And that turns the
>>>>>> bootcmd for platforms that just want to support modern off the shelf
>>>>>> distros in to something fairly small.
>>>>>
>>>>> Sigh...
>>>>>
>>>>> UEFI is a bootflow in this model, one of many. If we don't support the
>>>>> others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
>>>>>
>>>>> If we get EFI bootmgr going, then are you saying you want to disable
>>>>> everything else?
>>>>>
>>>>> You say 'major distros' but there are many that don't use it,
>>>>> particularly in the embedded space. I'll go out on a limb and say that
>>>>> the vast majority of embedded devices in the world don't use it. Are
>>>>> you really saying we should drop support for everything else? Even the
>>>>> distro stuff supports other options.
>>>>
>>>> And U-Boot supports a wide variety of CPUs and some of those don't
>>>> even have official UEFI support.
>>>>
>>>> However, on arm64 (and possibly riscv64) even the embedded folks
>>>> should seriously consider using the UEFI bootflow.  Linux now supports
>>>> physical address randomization when loaded via the UEFI stub, which is
>>>> something that can't really be implemented using the legacy boot path.
>>>> Note that you don't have to use a separate UEFI bootloader as U-Boot
>>>> can directly boot kernels with the UEFI stub.


You could set kaslr-seed in the device-tree using one of our hardware 
RNG drivers. This would allow address randomization when booting via the 
legacy entry point.


>>>
>>> 'legacy'? Isn't it just a case of relocating the kernel to a random
>>> address? I'm pretty sure U-Boot can do that :-)


Instead of 'legacy' you could call it vendor lock in.
If a firmware does not support UEFI you cannot boot other operating 
systems than Linux, e.g. BSD.


>>
>> The problem is that the legacy boot protocol for the Linux arm64
>> kernel requires a 2MB aligned kernel base, which reduces the number of
>> randomized bits.  That also means that virtual addresses are not fully
>> randomized as the kernel uses large mappings to map itself.  My
>> understanding is that the UEFI stub can relocate the kernel to any 64K
>> aligned address.  I suppose it is possible to add code to achieve the
>> same thing for the legacy boot path, but I don't think the arm64
>> maintainers are really interested in doing that.
>>
>> But yes, U-Boot should certainly try to load arm64 kernels at a random
>> address instead hardcoding the load address ;)
> 
> This is simply a design decision. Linux wants to be able to do
> everything in its own tree, hence the decompression code, ALSR, etc.
> I'm not suggesting we change it, just that it could be done if people
> wanted it.
> 
> Another example is x86 kernels. U-Boot supports putting those in a FIT
> (even an uncompressed 64-bit kernel) which is helpful for signing /
> verified boot.
> 
>>
>>> Re direct boot, the issue seems to me that distros really want to use
>>> grub. I think a lot of people talk about direct boot, but it doesn't
>>> seem to be happening?
>>
>> I don't think direct boot makes sense for distros.  If you want to
>> support all variations of UEFI firmware you'll need to install your
>> kernel on a FAT filesystem.  And that doesn't work well if you let
>> your package manager manage the kernel.  Using grub is attractive
>> because x86 users are familliar with it and it offers an interface to
>> boot different kernels.

Ilias has added the possibility to add the intird path to the UEFI boot 
option. The gap is that U-Boot does not support SetVariables() and hence 
you cannot update the boot option from the OS.


>>
>> I think direct boot is targeted more at the embedded world or perhaps
>> virtualized environments.
> 
> Sounds right. But I keep asking if people have just given up on
> embedded/custom flows and want U-Boot to just be a UEFI runner?
> 
>>
>>>>> Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
>>>>> of the box'. Which bit of this series is in conflict with that? What
>>>>> exactly is "completely wrong" ?? Is it just the wording that is
>>>>> confusing?


I didn't comment your code. I disagreed to the specific sentence in the 
documentation regarding the work of distros.

There should be no distro specific work needed to get UEFI boot enabled. 
It should simply work out of the box in the default configuration like 
it does today.

Best regards

Heinrich


>>>>
>>>> Possibly.  The documentation seems to suggest that OSes have to
>>>> specify a bootflow for U-Boot.  Whereas one of the main advantages of
>>>> the UEFI bootflow is that this allows OSes not to care whether we're
>>>> booted by U-Boot, EDK2 or a closed source firmware implementation.  I
>>>> think the docs should say that the bootflow can be customized by an
>>>> OS, but that in general this isn't necessary.
>>>
>>> The definition of a bootflow is pretty broad. In the case of grub, it
>>> isn't even visible to U-Boot so there is a bootflow ('bootmeth_efi' in
>>> this series) but no actual file (grub.cfg) is visible to U-Boot other
>>> than the grub.efi that it boots. But if grub is not used, then the
>>> bootflow may be a file.
>>>
>>> We could perhaps use the next U-Boot contributor call to discuss it.
> 
> Regards,
> SImon

Hi Heinrich,

On Fri, 21 Jan 2022 at 15:05, Heinrich Schuchardt
<heinrich.schuchardt@canonical.com> wrote:
>
> On 1/21/22 20:17, Simon Glass wrote:
> > Hi Mark,
> >
> > On Fri, 21 Jan 2022 at 11:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >>
> >>> From: Simon Glass <sjg@chromium.org>
> >>> Date: Fri, 21 Jan 2022 09:53:37 -0700
> >>>
> >>> Hi Mark,
> >>>
> >>> On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >>>>
> >>>>> From: Simon Glass <sjg@chromium.org>
> >>>>> Date: Fri, 21 Jan 2022 08:20:17 -0700
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> >>>>>>
> >>>>>> On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> >>>>>>> On 1/19/22 02:43, Simon Glass wrote:
> >>>>>>>> Add documentation for this feature, including the commands and full
> >>>>>>>> devicetree bindings.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Simon Glass <sjg@chromium.org>
> >>>>>>>> ---
> >>>>>>>>
> >>>>>>>> Changes in v3:
> >>>>>>>> - Update docs for "bootmeths" and "boot_targets" env vars
> >>>>>>>>
> >>>>>>>>    MAINTAINERS                           |   4 +
> >>>>>>>>    doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> >>>>>>>>    doc/develop/distro.rst                |   3 +
> >>>>>>>>    doc/develop/index.rst                 |   1 +
> >>>>>>>>    doc/device-tree-bindings/bootdev.txt  |  18 +
> >>>>>>>>    doc/device-tree-bindings/bootmeth.txt |  31 ++
> >>>>>>>>    doc/device-tree-bindings/bootstd.txt  |   8 +
> >>>>>>>>    doc/usage/bootdev.rst                 | 135 ++++++
> >>>>>>>>    doc/usage/bootflow.rst                | 427 +++++++++++++++++
> >>>>>>>>    doc/usage/bootmeth.rst                | 108 +++++
> >>>>>>>>    doc/usage/index.rst                   |   3 +
> >>>>>>>>    11 files changed, 1376 insertions(+)
> >>>>>>>>    create mode 100644 doc/develop/bootstd.rst
> >>>>>>>>    create mode 100644 doc/device-tree-bindings/bootmeth.txt
> >>>>>>>>    create mode 100644 doc/usage/bootdev.rst
> >>>>>>>>    create mode 100644 doc/usage/bootflow.rst
> >>>>>>>>    create mode 100644 doc/usage/bootmeth.rst
> >>>>>>>>
> >>>>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
> >>>>>>>> index 8ad70d3d968..c2af8ada3c9 100644
> >>>>>>>> --- a/MAINTAINERS
> >>>>>>>> +++ b/MAINTAINERS
> >>>>>>>> @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> >>>>>>>>    F:        boot/bootstd.c
> >>>>>>>>    F:        cmd/bootdev.c
> >>>>>>>>    F:        cmd/bootflow.c
> >>>>>>>> +F: doc/develop/bootstd.rst
> >>>>>>>> +F: doc/usage/bootdev.rst
> >>>>>>>> +F: doc/usage/bootflow.rst
> >>>>>>>> +F: doc/usage/bootmeth.rst
> >>>>>>>>    F:        drivers/mmc/mmc_bootdev.c
> >>>>>>>>    F:        include/bootdev.h
> >>>>>>>>    F:        include/bootflow.h
> >>>>>>>> diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> >>>>>>>> new file mode 100644
> >>>>>>>> index 00000000000..1b65a806efb
> >>>>>>>> --- /dev/null
> >>>>>>>> +++ b/doc/develop/bootstd.rst
> >>>>>>>> @@ -0,0 +1,638 @@
> >>>>>>>> +.. SPDX-License-Identifier: GPL-2.0+:
> >>>>>>>> +
> >>>>>>>> +U-Boot Standard Boot
> >>>>>>>> +====================
> >>>>>>>> +
> >>>>>>>> +Introduction
> >>>>>>>> +------------
> >>>>>>>> +
> >>>>>>>> +Standard boot provides a built-in way for U-Boot to automatically boot
> >>>>>>>> +an Operating System without custom scripting and other customisation. It
> >>>>>>>> +introduces the following concepts:
> >>>>>>>> +
> >>>>>>>> +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> >>>>>>>> +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> >>>>>>>> +   - bootflow - a description of how to boot (provided by the distro)
> >>>>>>>> +
> >>>>>>>> +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> >>>>>>>> +for creating a bootflow for each kernel combination that it wants to offer.
> >>>>>>>
> >>>>>>> This gets it completely wrong. There is one standardized boot flow: UEFI.
> >>>>>>> All major distros support this. U-Boot has to offer UEFI booting out of the
> >>>>>>> box.
> >>>>>>
> >>>>>> I want to jump up and down and emphasize this part as well.  While I
> >>>>>> believe our UEFI bootmgr is still missing the normal scan code, that's
> >>>>>> something that has been promised to be implemented.  And that turns the
> >>>>>> bootcmd for platforms that just want to support modern off the shelf
> >>>>>> distros in to something fairly small.
> >>>>>
> >>>>> Sigh...
> >>>>>
> >>>>> UEFI is a bootflow in this model, one of many. If we don't support the
> >>>>> others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> >>>>>
> >>>>> If we get EFI bootmgr going, then are you saying you want to disable
> >>>>> everything else?
> >>>>>
> >>>>> You say 'major distros' but there are many that don't use it,
> >>>>> particularly in the embedded space. I'll go out on a limb and say that
> >>>>> the vast majority of embedded devices in the world don't use it. Are
> >>>>> you really saying we should drop support for everything else? Even the
> >>>>> distro stuff supports other options.
> >>>>
> >>>> And U-Boot supports a wide variety of CPUs and some of those don't
> >>>> even have official UEFI support.
> >>>>
> >>>> However, on arm64 (and possibly riscv64) even the embedded folks
> >>>> should seriously consider using the UEFI bootflow.  Linux now supports
> >>>> physical address randomization when loaded via the UEFI stub, which is
> >>>> something that can't really be implemented using the legacy boot path.
> >>>> Note that you don't have to use a separate UEFI bootloader as U-Boot
> >>>> can directly boot kernels with the UEFI stub.
>
>
> You could set kaslr-seed in the device-tree using one of our hardware
> RNG drivers. This would allow address randomization when booting via the
> legacy entry point.
>
>
> >>>
> >>> 'legacy'? Isn't it just a case of relocating the kernel to a random
> >>> address? I'm pretty sure U-Boot can do that :-)
>
>
> Instead of 'legacy' you could call it vendor lock in.
> If a firmware does not support UEFI you cannot boot other operating
> systems than Linux, e.g. BSD.
>
>
> >>
> >> The problem is that the legacy boot protocol for the Linux arm64
> >> kernel requires a 2MB aligned kernel base, which reduces the number of
> >> randomized bits.  That also means that virtual addresses are not fully
> >> randomized as the kernel uses large mappings to map itself.  My
> >> understanding is that the UEFI stub can relocate the kernel to any 64K
> >> aligned address.  I suppose it is possible to add code to achieve the
> >> same thing for the legacy boot path, but I don't think the arm64
> >> maintainers are really interested in doing that.
> >>
> >> But yes, U-Boot should certainly try to load arm64 kernels at a random
> >> address instead hardcoding the load address ;)
> >
> > This is simply a design decision. Linux wants to be able to do
> > everything in its own tree, hence the decompression code, ALSR, etc.
> > I'm not suggesting we change it, just that it could be done if people
> > wanted it.
> >
> > Another example is x86 kernels. U-Boot supports putting those in a FIT
> > (even an uncompressed 64-bit kernel) which is helpful for signing /
> > verified boot.
> >
> >>
> >>> Re direct boot, the issue seems to me that distros really want to use
> >>> grub. I think a lot of people talk about direct boot, but it doesn't
> >>> seem to be happening?
> >>
> >> I don't think direct boot makes sense for distros.  If you want to
> >> support all variations of UEFI firmware you'll need to install your
> >> kernel on a FAT filesystem.  And that doesn't work well if you let
> >> your package manager manage the kernel.  Using grub is attractive
> >> because x86 users are familliar with it and it offers an interface to
> >> boot different kernels.
>
> Ilias has added the possibility to add the intird path to the UEFI boot
> option. The gap is that U-Boot does not support SetVariables() and hence
> you cannot update the boot option from the OS.
>
>
> >>
> >> I think direct boot is targeted more at the embedded world or perhaps
> >> virtualized environments.
> >
> > Sounds right. But I keep asking if people have just given up on
> > embedded/custom flows and want U-Boot to just be a UEFI runner?
> >
> >>
> >>>>> Also Heinrich your comment says 'U-Boot has to offer UEFI booting out
> >>>>> of the box'. Which bit of this series is in conflict with that? What
> >>>>> exactly is "completely wrong" ?? Is it just the wording that is
> >>>>> confusing?
>
>
> I didn't comment your code. I disagreed to the specific sentence in the
> documentation regarding the work of distros.
>
> There should be no distro specific work needed to get UEFI boot enabled.
> It should simply work out of the box in the default configuration like
> it does today.

...which is what this series does, to the extent that things are
implemented. What are you getting at? There is no special file to
write or anything different from what is there today.

Regards,
Simon

Hi Tom,

On Fri, 21 Jan 2022 at 14:46, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 21, 2022 at 02:15:24PM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Fri, 21 Jan 2022 at 12:23, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Fri, Jan 21, 2022 at 12:14:22PM -0700, Simon Glass wrote:
> > > > Hi Tom,
> > > >
> > > > On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > > > > > Hi Tom,
> > > > > >
> > > > > > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > > > > > > > >
> > > > > > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > > [snip]
> > > > > > > > > > > +Introduction
> > > > > > > > > > > +------------
> > > > > > > > > > > +
> > > > > > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > > > > > +introduces the following concepts:
> > > > > > > > > > > +
> > > > > > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > > > > > +
> > > > > > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > > > > > >
> > > > > > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > > > > > box.
> > > > > > > > >
> > > > > > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > > > > > something that has been promised to be implemented.  And that turns the
> > > > > > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > > > > > distros in to something fairly small.
> > > > > > > >
> > > > > > > > Sigh...
> > > > > > > >
> > > > > > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > > > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > > > > >
> > > > > > > No one is talking about removing anything else.  But a major part of
> > > > > > > your motivation here seems to be "discovering what to boot where is a
> > > > > > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > > > > > status of that off-list).  And I want to emphasize discover.
> > > > > >
> > > > > > But only if you use EFI boot manager, right? Even then I'm not sure we
> > > > > > have a deterministic way of listing the available bootdevs, which is
> > > > > > something that this series provides.
> > > > >
> > > > > I'll let someone else that knows the EFI boot manager code / design
> > > > > speak to this.  But yes, for the discoverable case I'm not seeing where
> > > > > "use EFI boot manager" isn't the reasonable answer.
> > > >
> > > > With bootdev you have a proper driver and device tree node where
> > > > configuration can be provided. The default ordering for bootdevs can
> > > > be provided. We can deal with the quirks of particular subsystems,
> > > > like MMC. I think there will be other benefits apparent as this all
> > > > matures.
> > > >
> > > > But let's see. Perhaps it doesn't really matter anyway, since it seems
> > > > that EFI boot manager is doing its own thing for now.
> > > >
> > > > >
> > > > > > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > > > > > everything else?
> > > > > > >
> > > > > > > Not at all.
> > > > > >
> > > > > > OK, good.
> > > > > >
> > > > > > >
> > > > > > > > You say 'major distros' but there are many that don't use it,
> > > > > > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > > > > > the vast majority of embedded devices in the world don't use it. Are
> > > > > > > > you really saying we should drop support for everything else? Even the
> > > > > > > > distro stuff supports other options.
> > > > > > >
> > > > > > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > > > > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > > > > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > > > > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > > > > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > > > > > there and find out why (but I guess it's all the 32bit platforms).
> > > > > > >
> > > > > > > But I'd also say the vast majority of embedded devices don't need the
> > > > > > > complexity you're adding here, but DO need the ability to implement A/B
> > > > > > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > > > > > because it's a pain to modify the default environment in U-Boot and easy
> > > > > > > to drop in another script for grub.
> > > > > >
> > > > > > My feeling is the complexity is already there, just in scripts, which
> > > > > > are harder to understand (from personal experience trying to
> > > > > > understand what they do) and don't have tests. They are also very hard
> > > > > > to build on top of (e.g. verified boot).
> > > > >
> > > > > Yes, the scripts that live in the environment based boot flow is
> > > > > complex.  It's also been a huge step forward from what we had before and
> > > > > has been a great help.
> > > > >
> > > > > > I can't really say that this series is more complex than EFI bootmgr,
> > > > > > if that is what you are suggesting. I think the bootdev uclass is
> > > > > > well-motivated and will prove useful even for EFI.
> > > > >
> > > > > No, what I'm suggesting is that I see this as "current boot script stuff
> > > > > is too complex, lets replace it".  And I also see the big part of the
> > > > > complexity there being the discover where to boot from side of things.
> > > >
> > > > OK, so shall we replace the scripts, or not?
> > >
> > > I'm still looking to be convinced that something is better than the
> > > scripts, or that the problem is the scripts and not the pain involved
> > > with modifying the U-Boot environment.
> >
> > Well luckily, someone has gone to all the trouble of providing an
> > alternative that we can try :-)
> >
> > >
> > > > > > Also A/B/recovery is a lot easier to implement in code than in
> > > > > > scripts. I have linked to the proposed design there.
> > > > >
> > > > > Show me what implementing Mender or RAUC or swupdate looks like with
> > > > > your proposal.  Those are some of the real A/B use cases today and have
> > > > > had to take various approaches to dealing with our environment, and also
> > > > > supporting x86 and so started dealing with grub.
> > > >
> > > > That sounds like an interesting project. For mendor I found this:
> > > >
> > > > https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch
> > > >
> > > > More scripts...
> > >
> > > Yes.  Mender, RAUC and swupdate (iirc) all have some form of environment
> > > based bootcmd to Do The Right Thing and boot A or B or detect failure
> > > and fall back.  Show me how what you're doing improves integration for
> > > that case.  That's a case that's not covered by "use UEFI boot manager"
> > > directly.  I know for Mender a huge pain point for integration is "drop
> > > _this_ script in to the default environment".  RAUC is similar.  Show me
> > > something that makes that much easier to integrate, like it is with
> > > grub.
> >
> > The obvious answer would be to create a boot method for Mender. In
> > that you can do anything, including using the standard bootmethods to
> > obtain things to boot, etc. Also perhaps a 'recovery' bootdev which is
> > invoked last, when other things have failed, or first if recovery mode
> > is selected. Both could be done without any scripting, so could be
> > enabled on any board with just enabling CONFIG_MENDER, I suspect.
>
> OK.  So please show that this will improve things for this case, rather
> than speculate.

Can you define improvement? You want me to implement Mender before
this series can go in?

>
> > > > > > Anyway if we can agree that we are not going to disable the non-EFI
> > > > > > flows, then how about we focus on replacing the distro boot scripts,
> > > > > > dropping the config.h files, and leave EFI bootmgr out of this
> > > > > > discussion?
> > > > >
> > > > > The primary use case for the distro boot work is EFI boot, and the "make
> > > > > this logic clearer" solution is "use EFI bootmgr".  That's where we get
> > > > > stuck in a loop here.  There's no "the distro must create .." because
> > > > > the distro is already creating what's needed.
> > > >
> > > > So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
> > > > support? I thought you said no.
> > >
> > > Maybe we're talking at cross purposes.  I'm not going to drop
> > > booti/bootm/bootz/bootelf.  I'm not going to drop setting bootcmd to
> > > whatever the user / board developer knows is best for them.
> >
> > Maybe. My confusion is that EFI seems to be blocking progress in the
> > general booting domain. When innovation is suggested, the answer is
> > 'well EFI does this so why bother'. When the boot scripts are
> > mentioned, we can apparently limp along with them and iterate there,
> > since EFI is the only thing that matters anyway.
> >
> > Perhaps the real difference here is that I am not focused on EFI as
> > the solution to all things. I know that is where many distros are and
> > I believe you feel I am wasting my time. But that is where I am. From
> > my vantage point I see a lot of improvements we can make with booting,
> > independent of EFI.
>
> My high level concern is that yes, I don't see this as improving
> anything yet.  This isn't an improvement over what EFI has for booting
> generic distributions.  And this one of the areas where the boot scripts
> complex, but can be simplified.  Another area where they could be
> simplified is for any of the existing A/B style update mechanisms.
> That's an interesting case that could show value here.

Hmm.

>
> > > > Are you saying you want to keep the environment boot scripts, or not?
> > >
> > > As I said in the previous iteration on this series, I'm not convinced
> > > this is a win over other clean-ups that can be done with what we have
> > > today, or that it solves the problems that I often see popping up.
> >
> > Previously you were worried about the size growth. I am pretty
> > confident this will make a lot of boot-related things more structured
> > and easier over the medium term. But we do need something to build on.
> > I believe the concepts are sound, and applicable to various boot
> > scenarios. In fact I think the lack of a 'bootdev' is something we
> > should have thought to create a while back.
> >
> > I certainly think it is better than building more and more on the boot scripts.
>
> Alright, so rpi_3_32b grew as:
>        arm: (for 1/1 boards) all +6908.0 data +920.0 rodata -2664.0 text +8652.0
>             rpi_3_32b      : all +6908 data +920 rodata -2664 text +8652
>                u-boot: add: 101/0, grow: 3/-2 bytes: 8662/-160 (8502)
> with this series, and I corrected the minor problems due to '@return'
> becoming "Return:".  That's a problem.  Environment space is cheap (it's

Do you mean the 7KB overall growth is a problem? If so, that seems
unfair. Just EFI_LOADER has added that much each year since 2019 (as
far back as I checked). I actually thought 7KB was a great result and
was very pleased with it.

> a file, or it needs to have some pretty big these days alignment
> requirements to have physical redundancy, if you need that).  The
> biggest problem I see with boot scripts is that defining them in a
> header file where we have to deal with escaping stuff in a header, and
> you resolved that with the environment cleanup series you did before.

I shudder to think how one would do distro_bootcmd in that, if that is
what you are suggesting.

> The series (that I keep failing to find the time to reply to, but was
> happy to see!) that updates hush to a modern copy is another good step
> in the right direction.  Being able to write the script but more as
> plain text rather than escaped strings in C is I suspect why Armbian
> does what it does.

So, the scripts are preferable to this series?

Regards,
Simon

> Date: Fri, 21 Jan 2022 23:05:34 +0100
> From: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
> 
> On 1/21/22 20:17, Simon Glass wrote:
> > Hi Mark,
> > 
> > On Fri, 21 Jan 2022 at 11:23, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >>
> >>> From: Simon Glass <sjg@chromium.org>
> >>> Date: Fri, 21 Jan 2022 09:53:37 -0700
> >>>
> >>> Hi Mark,
> >>>
> >>> On Fri, 21 Jan 2022 at 09:03, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
> >>>>
> >>>>> From: Simon Glass <sjg@chromium.org>
> >>>>> Date: Fri, 21 Jan 2022 08:20:17 -0700
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> >>>>>>
> >>>>>> On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> >>>>>>> On 1/19/22 02:43, Simon Glass wrote:
> >>>>>>>> Add documentation for this feature, including the commands and full
> >>>>>>>> devicetree bindings.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Simon Glass <sjg@chromium.org>
> >>>>>>>> ---
> >>>>>>>>
> >>>>>>>> Changes in v3:
> >>>>>>>> - Update docs for "bootmeths" and "boot_targets" env vars
> >>>>>>>>
> >>>>>>>>    MAINTAINERS                           |   4 +
> >>>>>>>>    doc/develop/bootstd.rst               | 638 ++++++++++++++++++++++++++
> >>>>>>>>    doc/develop/distro.rst                |   3 +
> >>>>>>>>    doc/develop/index.rst                 |   1 +
> >>>>>>>>    doc/device-tree-bindings/bootdev.txt  |  18 +
> >>>>>>>>    doc/device-tree-bindings/bootmeth.txt |  31 ++
> >>>>>>>>    doc/device-tree-bindings/bootstd.txt  |   8 +
> >>>>>>>>    doc/usage/bootdev.rst                 | 135 ++++++
> >>>>>>>>    doc/usage/bootflow.rst                | 427 +++++++++++++++++
> >>>>>>>>    doc/usage/bootmeth.rst                | 108 +++++
> >>>>>>>>    doc/usage/index.rst                   |   3 +
> >>>>>>>>    11 files changed, 1376 insertions(+)
> >>>>>>>>    create mode 100644 doc/develop/bootstd.rst
> >>>>>>>>    create mode 100644 doc/device-tree-bindings/bootmeth.txt
> >>>>>>>>    create mode 100644 doc/usage/bootdev.rst
> >>>>>>>>    create mode 100644 doc/usage/bootflow.rst
> >>>>>>>>    create mode 100644 doc/usage/bootmeth.rst
> >>>>>>>>
> >>>>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
> >>>>>>>> index 8ad70d3d968..c2af8ada3c9 100644
> >>>>>>>> --- a/MAINTAINERS
> >>>>>>>> +++ b/MAINTAINERS
> >>>>>>>> @@ -669,6 +669,10 @@ F:     boot/bootmeth*.c
> >>>>>>>>    F:        boot/bootstd.c
> >>>>>>>>    F:        cmd/bootdev.c
> >>>>>>>>    F:        cmd/bootflow.c
> >>>>>>>> +F: doc/develop/bootstd.rst
> >>>>>>>> +F: doc/usage/bootdev.rst
> >>>>>>>> +F: doc/usage/bootflow.rst
> >>>>>>>> +F: doc/usage/bootmeth.rst
> >>>>>>>>    F:        drivers/mmc/mmc_bootdev.c
> >>>>>>>>    F:        include/bootdev.h
> >>>>>>>>    F:        include/bootflow.h
> >>>>>>>> diff --git a/doc/develop/bootstd.rst b/doc/develop/bootstd.rst
> >>>>>>>> new file mode 100644
> >>>>>>>> index 00000000000..1b65a806efb
> >>>>>>>> --- /dev/null
> >>>>>>>> +++ b/doc/develop/bootstd.rst
> >>>>>>>> @@ -0,0 +1,638 @@
> >>>>>>>> +.. SPDX-License-Identifier: GPL-2.0+:
> >>>>>>>> +
> >>>>>>>> +U-Boot Standard Boot
> >>>>>>>> +====================
> >>>>>>>> +
> >>>>>>>> +Introduction
> >>>>>>>> +------------
> >>>>>>>> +
> >>>>>>>> +Standard boot provides a built-in way for U-Boot to automatically boot
> >>>>>>>> +an Operating System without custom scripting and other customisation. It
> >>>>>>>> +introduces the following concepts:
> >>>>>>>> +
> >>>>>>>> +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> >>>>>>>> +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> >>>>>>>> +   - bootflow - a description of how to boot (provided by the distro)
> >>>>>>>> +
> >>>>>>>> +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> >>>>>>>> +for creating a bootflow for each kernel combination that it wants to offer.
> >>>>>>>
> >>>>>>> This gets it completely wrong. There is one standardized boot flow: UEFI.
> >>>>>>> All major distros support this. U-Boot has to offer UEFI booting out of the
> >>>>>>> box.
> >>>>>>
> >>>>>> I want to jump up and down and emphasize this part as well.  While I
> >>>>>> believe our UEFI bootmgr is still missing the normal scan code, that's
> >>>>>> something that has been promised to be implemented.  And that turns the
> >>>>>> bootcmd for platforms that just want to support modern off the shelf
> >>>>>> distros in to something fairly small.
> >>>>>
> >>>>> Sigh...
> >>>>>
> >>>>> UEFI is a bootflow in this model, one of many. If we don't support the
> >>>>> others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> >>>>>
> >>>>> If we get EFI bootmgr going, then are you saying you want to disable
> >>>>> everything else?
> >>>>>
> >>>>> You say 'major distros' but there are many that don't use it,
> >>>>> particularly in the embedded space. I'll go out on a limb and say that
> >>>>> the vast majority of embedded devices in the world don't use it. Are
> >>>>> you really saying we should drop support for everything else? Even the
> >>>>> distro stuff supports other options.
> >>>>
> >>>> And U-Boot supports a wide variety of CPUs and some of those don't
> >>>> even have official UEFI support.
> >>>>
> >>>> However, on arm64 (and possibly riscv64) even the embedded folks
> >>>> should seriously consider using the UEFI bootflow.  Linux now supports
> >>>> physical address randomization when loaded via the UEFI stub, which is
> >>>> something that can't really be implemented using the legacy boot path.
> >>>> Note that you don't have to use a separate UEFI bootloader as U-Boot
> >>>> can directly boot kernels with the UEFI stub.
> 
> You could set kaslr-seed in the device-tree using one of our hardware 
> RNG drivers. This would allow address randomization when booting via the 
> legacy entry point.

No quite.  Setting kaslr-seed will only do randomization of virtual
addresses.  But because the bottom 21 bits of the virtual address have
to match the bottom 21 bits of the physical address, which isn't
randomized, the number of randomized bits in your virtual addresses
will be somewhat limited.  With EFISTUB booting, physical addresses
are also randomized giving you more randomized bits in your virtual
addresses.  This is what ardb pointed out when we were discussing
Ilias's patch to remove kaslr-seed from the device tree.

This points out another downside of legacy booting.  As far as I can
tell you have to run the kaslrseed command to set kaslr-seed in the
device tree.  Unless it is already set by a prior boot stage of
course.  But none of the boards in the U-Boot tree enable that
command.  Whereas support for EFI_RNG_PROTOCOL is enabled
automatically if DM_RNG is enabled.

> >>>
> >>> 'legacy'? Isn't it just a case of relocating the kernel to a random
> >>> address? I'm pretty sure U-Boot can do that :-)
> 
> Instead of 'legacy' you could call it vendor lock in.
> If a firmware does not support UEFI you cannot boot other operating 
> systems than Linux, e.g. BSD.

Well yes.  That is why I keep fighting for enabling EFI_LOADER support
for as many arm/arm64/riscv64 boards as possible. ;)

> >> The problem is that the legacy boot protocol for the Linux arm64
> >> kernel requires a 2MB aligned kernel base, which reduces the number of
> >> randomized bits.  That also means that virtual addresses are not fully
> >> randomized as the kernel uses large mappings to map itself.  My
> >> understanding is that the UEFI stub can relocate the kernel to any 64K
> >> aligned address.  I suppose it is possible to add code to achieve the
> >> same thing for the legacy boot path, but I don't think the arm64
> >> maintainers are really interested in doing that.
> >>
> >> But yes, U-Boot should certainly try to load arm64 kernels at a random
> >> address instead hardcoding the load address ;)

Really late to the party but...

On Fri, 21 Jan 2022 at 17:31, Tom Rini <trini@konsulko.com> wrote:
>
> On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > Hi,
> >
> > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > On 1/19/22 02:43, Simon Glass wrote:
> [snip]
> > > > > +Introduction
> > > > > +------------
> > > > > +
> > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > +an Operating System without custom scripting and other customisation. It
> > > > > +introduces the following concepts:
> > > > > +
> > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > +
> > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > >
> > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > box.
> > >
> > > I want to jump up and down and emphasize this part as well.  While I
> > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > something that has been promised to be implemented.  And that turns the
> > > bootcmd for platforms that just want to support modern off the shelf
> > > distros in to something fairly small.
> >
> > Sigh...
> >
> > UEFI is a bootflow in this model, one of many. If we don't support the
> > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
>
> No one is talking about removing anything else.  But a major part of
> your motivation here seems to be "discovering what to boot where is a
> pain" and that's solved (or at least defined, I'm poking Ilias about the
> status of that off-list).  And I want to emphasize discover.

There's this [1] which I am happy to either ping Akashi-san or pick it
myself if it makes anyone life easier. Most of the concerns in that
thread are valid imho, but they also are not to hard to solve (famous
last words).

[...]

[1] https://lore.kernel.org/u-boot/20211109013233.72902-3-takahiro.akashi@linaro.org/#t

Thanks and apologies for the *really* delayed reply.
/Ilias