[v4,0/7] Fix Rockchip RK3399 bootstd migration

Hey all,

I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
and it changed too much for everything else. I took about half of that
series and then reworked a few things. Now only rk3399 platforms change
at all and aside from bootcmd changes, the only thing is they now
disable true/test/sysboot/showvar/false/exit commands as those were
being pulled in from distro and now we don't set that flag. I think the
way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
perform more SoC migrations.

Hi Tom,

On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
>
> Hey all,
>
> I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> and it changed too much for everything else. I took about half of that
> series and then reworked a few things. Now only rk3399 platforms change
> at all and aside from bootcmd changes, the only thing is they now
> disable true/test/sysboot/showvar/false/exit commands as those were
> being pulled in from distro and now we don't set that flag. I think the
> way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> perform more SoC migrations.

Thanks for digging into this. I haven't seen any comments on the rpi
conversion, so perhaps people could test that?

Regards,
Simon

On Mon, Mar 27, 2023 at 05:00:31PM +1300, Simon Glass wrote:
> Hi Tom,
> 
> On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> >
> > Hey all,
> >
> > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > and it changed too much for everything else. I took about half of that
> > series and then reworked a few things. Now only rk3399 platforms change
> > at all and aside from bootcmd changes, the only thing is they now
> > disable true/test/sysboot/showvar/false/exit commands as those were
> > being pulled in from distro and now we don't set that flag. I think the
> > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > perform more SoC migrations.
> 
> Thanks for digging into this. I haven't seen any comments on the rpi
> conversion, so perhaps people could test that?

Well, it needs to be rebased on top of this series (which will be merged
in the next -rc today).

On Fri, 24 Mar 2023 16:58:09 -0400, Tom Rini wrote:

> Hey all,
> 
> I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> and it changed too much for everything else. I took about half of that
> series and then reworked a few things. Now only rk3399 platforms change
> at all and aside from bootcmd changes, the only thing is they now
> disable true/test/sysboot/showvar/false/exit commands as those were
> being pulled in from distro and now we don't set that flag. I think the
> way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> perform more SoC migrations.
> 
> [...]

Applied to u-boot/master, thanks!

On Mon, Mar 27, 2023 at 5:02 AM Simon Glass <sjg@chromium.org> wrote:
>
> Hi Tom,
>
> On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> >
> > Hey all,
> >
> > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > and it changed too much for everything else. I took about half of that
> > series and then reworked a few things. Now only rk3399 platforms change
> > at all and aside from bootcmd changes, the only thing is they now
> > disable true/test/sysboot/showvar/false/exit commands as those were
> > being pulled in from distro and now we don't set that flag. I think the
> > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > perform more SoC migrations.
>
> Thanks for digging into this. I haven't seen any comments on the rpi
> conversion, so perhaps people could test that?

I was planning on looking at that once 2023.04 was out but TBH I have
wasted so much time over the last few cycles dealing with regressions
through a bunch of these series that I now have so little time for
enhancements I now shy away. I know a lot of these series should
improve things in the future but they don't feel like when there's
unnecessary changes for things that are clearly untested.

There's also a lot of change for changes sake, for example the
rockchips ATF binaries needed is called bl31.elf by the default output
of the ATF build process, for others it's bl31.bin, binman for what
ever reason has changed that to be atf-bl31, now I have to change the
entire build process to be able to work out what is what on a board by
board basis to be able to set the required variable to be able to
specify the ATF where  previously it "just worked (tm)"..... I suppose
there is some perceived goal and improvement here but with both my
"U-Boot device maintainer" and "distro maintainer" hats on, both of
which I do in my own spare time, I currently fail to see it and I end
up.

All that said, thank you Tom for picking up the pieces for something
which should have been actually working when it landed.

Tested-by: Peter Robinson <pbrobinson@gmail.com>

Now I get to go and work out all of the rest of the mess!

Regards,
Peter

Hi Peter,

On Tue, 28 Mar 2023 at 06:50, Peter Robinson <pbrobinson@gmail.com> wrote:
>
> On Mon, Mar 27, 2023 at 5:02 AM Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > Hey all,
> > >
> > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > and it changed too much for everything else. I took about half of that
> > > series and then reworked a few things. Now only rk3399 platforms change
> > > at all and aside from bootcmd changes, the only thing is they now
> > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > being pulled in from distro and now we don't set that flag. I think the
> > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > perform more SoC migrations.
> >
> > Thanks for digging into this. I haven't seen any comments on the rpi
> > conversion, so perhaps people could test that?
>
> I was planning on looking at that once 2023.04 was out but TBH I have
> wasted so much time over the last few cycles dealing with regressions
> through a bunch of these series that I now have so little time for
> enhancements I now shy away. I know a lot of these series should
> improve things in the future but they don't feel like when there's
> unnecessary changes for things that are clearly untested.
>
> There's also a lot of change for changes sake, for example the
> rockchips ATF binaries needed is called bl31.elf by the default output
> of the ATF build process, for others it's bl31.bin, binman for what
> ever reason has changed that to be atf-bl31, now I have to change the
> entire build process to be able to work out what is what on a board by
> board basis to be able to set the required variable to be able to
> specify the ATF where  previously it "just worked (tm)"..... I suppose

For this point, you could use the BL31 environment variable, which
would allow the old file to be used. That is in the instructions for
some boards.

Also, the change to atf-bl31 is because no particular filename is
provided as a default, so we end up using the entry type. I suppose
the problem is that there are two names in common use (bl31.bin and
bl31.elf) and if we use the wrong one it won't boot. That is an
unfortunate result of how things work with ATF. But in any case this
is a decision for the SoC maintainer, who can provide a default
filename if desired, in the binman description for that SoC.

> there is some perceived goal and improvement here but with both my
> "U-Boot device maintainer" and "distro maintainer" hats on, both of
> which I do in my own spare time, I currently fail to see it and I end
> up.

Basically the benefit is that U-Boot is not full of loads of strange
shell / Python scripts for each SoC type, it is possible (at least in
principle) to figure out how to build an image without scanning the
web for vendor-specific instructions, tools which are needed to build
images are registered (binman tool -l) and can be fetched. Overall,
having a data-driven approach to firmware packaging is vastly superior
to a code-based approach, particularly as firmware fragments more and
more. You can find more here [1]

>
> All that said, thank you Tom for picking up the pieces for something
> which should have been actually working when it landed.
>
> Tested-by: Peter Robinson <pbrobinson@gmail.com>
>
> Now I get to go and work out all of the rest of the mess!

Yes, thank you Tom.

Thank you also Peter for your comments. I suspect a lot of people feel
the same way.

From my perspective, these migrations can be exhausting, particularly
when drawn out over a long period of time. Removing SPL_FIT_GENERATOR
started almost 3 years ago[2]. People continued adding new boards to
it even until recently.

It is also often difficult to predict how things will turn out, even
for people with many years of experience in software development. So
we sometimes make wrong turns.

I would love to see more attention from SoC maintainers, to sit down
with a coffee once a month and take a hard look at the state of the
code they look after, what migrations are outstanding, etc.

The firmware world is changing rapidly. We need to be able to keep on
top of the increasing complexity with new tools and techniques.
Perhaps it will settle down at some point, but for now it is quite a
daunting task and we all need to pitch in.

Regards,
Simon

[1] https://u-boot.readthedocs.io/en/latest/develop/package/binman.html#motivation
[2] https://patchwork.ozlabs.org/project/uboot/patch/20200613205717.v2.42.I2428dcb9b077364f9517f2c291db63b6bac1e992@changeid/

On Mon, Mar 27, 2023 at 06:50:41PM +0100, Peter Robinson wrote:
> On Mon, Mar 27, 2023 at 5:02 AM Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > Hey all,
> > >
> > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > and it changed too much for everything else. I took about half of that
> > > series and then reworked a few things. Now only rk3399 platforms change
> > > at all and aside from bootcmd changes, the only thing is they now
> > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > being pulled in from distro and now we don't set that flag. I think the
> > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > perform more SoC migrations.
> >
> > Thanks for digging into this. I haven't seen any comments on the rpi
> > conversion, so perhaps people could test that?
> 
> I was planning on looking at that once 2023.04 was out but TBH I have
> wasted so much time over the last few cycles dealing with regressions
> through a bunch of these series that I now have so little time for
> enhancements I now shy away. I know a lot of these series should
> improve things in the future but they don't feel like when there's
> unnecessary changes for things that are clearly untested.

I too am unhappy with how some of these have gone. The _intent_ here is
that getting the current "boot generic distro" framework is complex /
error prone, and we can do better. Unfortunately the first set of
platforms to switch to this are Rockchip and I think there was overlap
there with platforms that got broken at the end of the v2023.01 cycle to
fix other platforms, and then those sets of platforms flipped early in
v2023.04 and took until -rc2? to get resolved.  Which was less than
ideal.

> There's also a lot of change for changes sake, for example the
> rockchips ATF binaries needed is called bl31.elf by the default output
> of the ATF build process, for others it's bl31.bin, binman for what
> ever reason has changed that to be atf-bl31, now I have to change the
> entire build process to be able to work out what is what on a board by
> board basis to be able to set the required variable to be able to
> specify the ATF where  previously it "just worked (tm)"..... I suppose
> there is some perceived goal and improvement here but with both my
> "U-Boot device maintainer" and "distro maintainer" hats on, both of
> which I do in my own spare time, I currently fail to see it and I end
> up.

I wish I knew where to talk to with ATF / TF-A to get some agreed upon
naming scheme going as one of the things that is very frustrating is
getting the names and combinations of everything else that's required
Just Right for every chip. And feedback that things aren't working is
appreciated, since we do need to make things easier.

On Wed, Mar 29, 2023 at 3:54 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Mon, Mar 27, 2023 at 06:50:41PM +0100, Peter Robinson wrote:
> > On Mon, Mar 27, 2023 at 5:02 AM Simon Glass <sjg@chromium.org> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > Hey all,
> > > >
> > > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > > and it changed too much for everything else. I took about half of that
> > > > series and then reworked a few things. Now only rk3399 platforms change
> > > > at all and aside from bootcmd changes, the only thing is they now
> > > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > > being pulled in from distro and now we don't set that flag. I think the
> > > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > > perform more SoC migrations.
> > >
> > > Thanks for digging into this. I haven't seen any comments on the rpi
> > > conversion, so perhaps people could test that?
> >
> > I was planning on looking at that once 2023.04 was out but TBH I have
> > wasted so much time over the last few cycles dealing with regressions
> > through a bunch of these series that I now have so little time for
> > enhancements I now shy away. I know a lot of these series should
> > improve things in the future but they don't feel like when there's
> > unnecessary changes for things that are clearly untested.
>
> I too am unhappy with how some of these have gone. The _intent_ here is
> that getting the current "boot generic distro" framework is complex /
> error prone, and we can do better. Unfortunately the first set of
> platforms to switch to this are Rockchip and I think there was overlap
> there with platforms that got broken at the end of the v2023.01 cycle to
> fix other platforms, and then those sets of platforms flipped early in
> v2023.04 and took until -rc2? to get resolved.  Which was less than
> ideal.
>
> > There's also a lot of change for changes sake, for example the
> > rockchips ATF binaries needed is called bl31.elf by the default output
> > of the ATF build process, for others it's bl31.bin, binman for what
> > ever reason has changed that to be atf-bl31, now I have to change the
> > entire build process to be able to work out what is what on a board by
> > board basis to be able to set the required variable to be able to
> > specify the ATF where  previously it "just worked (tm)"..... I suppose
> > there is some perceived goal and improvement here but with both my
> > "U-Boot device maintainer" and "distro maintainer" hats on, both of
> > which I do in my own spare time, I currently fail to see it and I end
> > up.
>
> I wish I knew where to talk to with ATF / TF-A to get some agreed upon
> naming scheme going as one of the things that is very frustrating is
> getting the names and combinations of everything else that's required
> Just Right for every chip. And feedback that things aren't working is
> appreciated, since we do need to make things easier.

In all of the various make_fit_atf.py the various vendors specified
them, this is the case for the rockchip one [1]. This is the case for
the Allwinner boards [2] but the rockchip ports have missed this so it
also should be fixed for GA.

A side point is that binman should not be storing firmware build
specifics in the device tree which is a means of describing the
hardware, This really needs to be fixed as it really isn't the right
place for that sort of things. I suspect a file in arch/arm/mach-<SOC>
is likely a better location, or if it's board specific in the board/
sub directory.

Peter

[1] https://source.denx.de/u-boot/u-boot/-/blob/v2023.01/arch/arm/mach-rockchip/make_fit_atf.py#L227
[2] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/dts/sunxi-u-boot.dtsi#L64

Hi Peter,

On Thu, 30 Mar 2023 at 19:56, Peter Robinson <pbrobinson@gmail.com> wrote:
>
> On Wed, Mar 29, 2023 at 3:54 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Mon, Mar 27, 2023 at 06:50:41PM +0100, Peter Robinson wrote:
> > > On Mon, Mar 27, 2023 at 5:02 AM Simon Glass <sjg@chromium.org> wrote:
> > > >
> > > > Hi Tom,
> > > >
> > > > On Sat, 25 Mar 2023 at 09:58, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > Hey all,
> > > > >
> > > > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > > > and it changed too much for everything else. I took about half of that
> > > > > series and then reworked a few things. Now only rk3399 platforms change
> > > > > at all and aside from bootcmd changes, the only thing is they now
> > > > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > > > being pulled in from distro and now we don't set that flag. I think the
> > > > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > > > perform more SoC migrations.
> > > >
> > > > Thanks for digging into this. I haven't seen any comments on the rpi
> > > > conversion, so perhaps people could test that?
> > >
> > > I was planning on looking at that once 2023.04 was out but TBH I have
> > > wasted so much time over the last few cycles dealing with regressions
> > > through a bunch of these series that I now have so little time for
> > > enhancements I now shy away. I know a lot of these series should
> > > improve things in the future but they don't feel like when there's
> > > unnecessary changes for things that are clearly untested.
> >
> > I too am unhappy with how some of these have gone. The _intent_ here is
> > that getting the current "boot generic distro" framework is complex /
> > error prone, and we can do better. Unfortunately the first set of
> > platforms to switch to this are Rockchip and I think there was overlap
> > there with platforms that got broken at the end of the v2023.01 cycle to
> > fix other platforms, and then those sets of platforms flipped early in
> > v2023.04 and took until -rc2? to get resolved.  Which was less than
> > ideal.
> >
> > > There's also a lot of change for changes sake, for example the
> > > rockchips ATF binaries needed is called bl31.elf by the default output
> > > of the ATF build process, for others it's bl31.bin, binman for what
> > > ever reason has changed that to be atf-bl31, now I have to change the
> > > entire build process to be able to work out what is what on a board by
> > > board basis to be able to set the required variable to be able to
> > > specify the ATF where  previously it "just worked (tm)"..... I suppose
> > > there is some perceived goal and improvement here but with both my
> > > "U-Boot device maintainer" and "distro maintainer" hats on, both of
> > > which I do in my own spare time, I currently fail to see it and I end
> > > up.
> >
> > I wish I knew where to talk to with ATF / TF-A to get some agreed upon
> > naming scheme going as one of the things that is very frustrating is
> > getting the names and combinations of everything else that's required
> > Just Right for every chip. And feedback that things aren't working is
> > appreciated, since we do need to make things easier.
>
> In all of the various make_fit_atf.py the various vendors specified
> them, this is the case for the rockchip one [1]. This is the case for
> the Allwinner boards [2] but the rockchip ports have missed this so it
> also should be fixed for GA.
>
> A side point is that binman should not be storing firmware build
> specifics in the device tree which is a means of describing the
> hardware, This really needs to be fixed as it really isn't the right
> place for that sort of things. I suspect a file in arch/arm/mach-<SOC>
> is likely a better location, or if it's board specific in the board/
> sub directory.

Sorry, I don't agree with that at all. We store configuration
information in devicetree in firmware as this seems to be best format
for it, particularly with the growing number of firmware components
that need to share this information at runtime. The layout of firmware
is an important part of the system. We are still figuring out the
flows though. Also I have not attempted to upstream the binman
binding. I am very open to ideas on how best to do that.

Regards,
Simon


>
> Peter
>
> [1] https://source.denx.de/u-boot/u-boot/-/blob/v2023.01/arch/arm/mach-rockchip/make_fit_atf.py#L227
> [2] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/dts/sunxi-u-boot.dtsi#L64

Hi Simon,

> > > > > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > > > > and it changed too much for everything else. I took about half of that
> > > > > > series and then reworked a few things. Now only rk3399 platforms change
> > > > > > at all and aside from bootcmd changes, the only thing is they now
> > > > > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > > > > being pulled in from distro and now we don't set that flag. I think the
> > > > > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > > > > perform more SoC migrations.
> > > > >
> > > > > Thanks for digging into this. I haven't seen any comments on the rpi
> > > > > conversion, so perhaps people could test that?
> > > >
> > > > I was planning on looking at that once 2023.04 was out but TBH I have
> > > > wasted so much time over the last few cycles dealing with regressions
> > > > through a bunch of these series that I now have so little time for
> > > > enhancements I now shy away. I know a lot of these series should
> > > > improve things in the future but they don't feel like when there's
> > > > unnecessary changes for things that are clearly untested.
> > >
> > > I too am unhappy with how some of these have gone. The _intent_ here is
> > > that getting the current "boot generic distro" framework is complex /
> > > error prone, and we can do better. Unfortunately the first set of
> > > platforms to switch to this are Rockchip and I think there was overlap
> > > there with platforms that got broken at the end of the v2023.01 cycle to
> > > fix other platforms, and then those sets of platforms flipped early in
> > > v2023.04 and took until -rc2? to get resolved.  Which was less than
> > > ideal.
> > >
> > > > There's also a lot of change for changes sake, for example the
> > > > rockchips ATF binaries needed is called bl31.elf by the default output
> > > > of the ATF build process, for others it's bl31.bin, binman for what
> > > > ever reason has changed that to be atf-bl31, now I have to change the
> > > > entire build process to be able to work out what is what on a board by
> > > > board basis to be able to set the required variable to be able to
> > > > specify the ATF where  previously it "just worked (tm)"..... I suppose
> > > > there is some perceived goal and improvement here but with both my
> > > > "U-Boot device maintainer" and "distro maintainer" hats on, both of
> > > > which I do in my own spare time, I currently fail to see it and I end
> > > > up.
> > >
> > > I wish I knew where to talk to with ATF / TF-A to get some agreed upon
> > > naming scheme going as one of the things that is very frustrating is
> > > getting the names and combinations of everything else that's required
> > > Just Right for every chip. And feedback that things aren't working is
> > > appreciated, since we do need to make things easier.
> >
> > In all of the various make_fit_atf.py the various vendors specified
> > them, this is the case for the rockchip one [1]. This is the case for
> > the Allwinner boards [2] but the rockchip ports have missed this so it
> > also should be fixed for GA.

Can you do a patch to fix this regression please and then specify the
correct pieces in the binman section then?

> > A side point is that binman should not be storing firmware build
> > specifics in the device tree which is a means of describing the
> > hardware, This really needs to be fixed as it really isn't the right
> > place for that sort of things. I suspect a file in arch/arm/mach-<SOC>
> > is likely a better location, or if it's board specific in the board/
> > sub directory.
>
> Sorry, I don't agree with that at all. We store configuration
> information in devicetree in firmware as this seems to be best format
> for it, particularly with the growing number of firmware components
> that need to share this information at runtime. The layout of firmware
> is an important part of the system. We are still figuring out the
> flows though. Also I have not attempted to upstream the binman
> binding. I am very open to ideas on how best to do that.

Rob what's your thoughts on the binman firmware build pieces being in
device tree and the process on upstreaming the bindings?

Regards,
Peter

> > [1] https://source.denx.de/u-boot/u-boot/-/blob/v2023.01/arch/arm/mach-rockchip/make_fit_atf.py#L227
> > [2] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/dts/sunxi-u-boot.dtsi#L64

On Fri, Mar 31, 2023 at 09:39:53AM +0100, Peter Robinson wrote:
> Hi Simon,
> 
> > > > > > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > > > > > and it changed too much for everything else. I took about half of that
> > > > > > > series and then reworked a few things. Now only rk3399 platforms change
> > > > > > > at all and aside from bootcmd changes, the only thing is they now
> > > > > > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > > > > > being pulled in from distro and now we don't set that flag. I think the
> > > > > > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > > > > > perform more SoC migrations.
> > > > > >
> > > > > > Thanks for digging into this. I haven't seen any comments on the rpi
> > > > > > conversion, so perhaps people could test that?
> > > > >
> > > > > I was planning on looking at that once 2023.04 was out but TBH I have
> > > > > wasted so much time over the last few cycles dealing with regressions
> > > > > through a bunch of these series that I now have so little time for
> > > > > enhancements I now shy away. I know a lot of these series should
> > > > > improve things in the future but they don't feel like when there's
> > > > > unnecessary changes for things that are clearly untested.
> > > >
> > > > I too am unhappy with how some of these have gone. The _intent_ here is
> > > > that getting the current "boot generic distro" framework is complex /
> > > > error prone, and we can do better. Unfortunately the first set of
> > > > platforms to switch to this are Rockchip and I think there was overlap
> > > > there with platforms that got broken at the end of the v2023.01 cycle to
> > > > fix other platforms, and then those sets of platforms flipped early in
> > > > v2023.04 and took until -rc2? to get resolved.  Which was less than
> > > > ideal.
> > > >
> > > > > There's also a lot of change for changes sake, for example the
> > > > > rockchips ATF binaries needed is called bl31.elf by the default output
> > > > > of the ATF build process, for others it's bl31.bin, binman for what
> > > > > ever reason has changed that to be atf-bl31, now I have to change the
> > > > > entire build process to be able to work out what is what on a board by
> > > > > board basis to be able to set the required variable to be able to
> > > > > specify the ATF where  previously it "just worked (tm)"..... I suppose
> > > > > there is some perceived goal and improvement here but with both my
> > > > > "U-Boot device maintainer" and "distro maintainer" hats on, both of
> > > > > which I do in my own spare time, I currently fail to see it and I end
> > > > > up.
> > > >
> > > > I wish I knew where to talk to with ATF / TF-A to get some agreed upon
> > > > naming scheme going as one of the things that is very frustrating is
> > > > getting the names and combinations of everything else that's required
> > > > Just Right for every chip. And feedback that things aren't working is
> > > > appreciated, since we do need to make things easier.
> > >
> > > In all of the various make_fit_atf.py the various vendors specified
> > > them, this is the case for the rockchip one [1]. This is the case for
> > > the Allwinner boards [2] but the rockchip ports have missed this so it
> > > also should be fixed for GA.
> 
> Can you do a patch to fix this regression please and then specify the
> correct pieces in the binman section then?
> 
> > > A side point is that binman should not be storing firmware build
> > > specifics in the device tree which is a means of describing the
> > > hardware, This really needs to be fixed as it really isn't the right
> > > place for that sort of things. I suspect a file in arch/arm/mach-<SOC>
> > > is likely a better location, or if it's board specific in the board/
> > > sub directory.
> >
> > Sorry, I don't agree with that at all. We store configuration
> > information in devicetree in firmware as this seems to be best format
> > for it, particularly with the growing number of firmware components
> > that need to share this information at runtime. The layout of firmware
> > is an important part of the system. We are still figuring out the
> > flows though. Also I have not attempted to upstream the binman
> > binding. I am very open to ideas on how best to do that.
> 
> Rob what's your thoughts on the binman firmware build pieces being in
> device tree and the process on upstreaming the bindings?

One question I have is, with your distribution person hat on, what do
you think the right answer here is? It's not that U-Boot requires TF-A
(and also other blobs depending on SoC) but rather that modern ARM SoCs
are like x86-64 SoCs have been for ages and require multiple parts to
get assembled "just so", to result in something that initializes enough
of the system to get to loading something that can load the OS, and also
provide assorted run time services. You have one set of fixups for
amlogic SoCs, another for sunxi, yet another for imx8* and so on. And so
long as the software does the job, some of those parts can be swapped
for different ones.  U-Boot has binman *stuff* because it's really
useful for us to be able to take an output and test our part of the
world. And since it's gotten more and more complex over the years, we've
moved from handfuls of one-off tools to binman with the intention of
making it easier, not harder, for the case of "I care about multiple
vastly different SoCs".

Hi Peter,

On Fri, 31 Mar 2023 at 21:40, Peter Robinson <pbrobinson@gmail.com> wrote:
>
> Hi Simon,
>
> > > > > > > I took a look at Simon's v3 series to fix the rk3399 bootstd migration,
> > > > > > > and it changed too much for everything else. I took about half of that
> > > > > > > series and then reworked a few things. Now only rk3399 platforms change
> > > > > > > at all and aside from bootcmd changes, the only thing is they now
> > > > > > > disable true/test/sysboot/showvar/false/exit commands as those were
> > > > > > > being pulled in from distro and now we don't set that flag. I think the
> > > > > > > way I changed how we enable BOOTSTD_DEFAULTS should make it easier to
> > > > > > > perform more SoC migrations.
> > > > > >
> > > > > > Thanks for digging into this. I haven't seen any comments on the rpi
> > > > > > conversion, so perhaps people could test that?
> > > > >
> > > > > I was planning on looking at that once 2023.04 was out but TBH I have
> > > > > wasted so much time over the last few cycles dealing with regressions
> > > > > through a bunch of these series that I now have so little time for
> > > > > enhancements I now shy away. I know a lot of these series should
> > > > > improve things in the future but they don't feel like when there's
> > > > > unnecessary changes for things that are clearly untested.
> > > >
> > > > I too am unhappy with how some of these have gone. The _intent_ here is
> > > > that getting the current "boot generic distro" framework is complex /
> > > > error prone, and we can do better. Unfortunately the first set of
> > > > platforms to switch to this are Rockchip and I think there was overlap
> > > > there with platforms that got broken at the end of the v2023.01 cycle to
> > > > fix other platforms, and then those sets of platforms flipped early in
> > > > v2023.04 and took until -rc2? to get resolved.  Which was less than
> > > > ideal.
> > > >
> > > > > There's also a lot of change for changes sake, for example the
> > > > > rockchips ATF binaries needed is called bl31.elf by the default output
> > > > > of the ATF build process, for others it's bl31.bin, binman for what
> > > > > ever reason has changed that to be atf-bl31, now I have to change the
> > > > > entire build process to be able to work out what is what on a board by
> > > > > board basis to be able to set the required variable to be able to
> > > > > specify the ATF where  previously it "just worked (tm)"..... I suppose
> > > > > there is some perceived goal and improvement here but with both my
> > > > > "U-Boot device maintainer" and "distro maintainer" hats on, both of
> > > > > which I do in my own spare time, I currently fail to see it and I end
> > > > > up.
> > > >
> > > > I wish I knew where to talk to with ATF / TF-A to get some agreed upon
> > > > naming scheme going as one of the things that is very frustrating is
> > > > getting the names and combinations of everything else that's required
> > > > Just Right for every chip. And feedback that things aren't working is
> > > > appreciated, since we do need to make things easier.
> > >
> > > In all of the various make_fit_atf.py the various vendors specified
> > > them, this is the case for the rockchip one [1]. This is the case for
> > > the Allwinner boards [2] but the rockchip ports have missed this so it
> > > also should be fixed for GA.
>
> Can you do a patch to fix this regression please and then specify the
> correct pieces in the binman section then?

Yes I think this should be fixed.

We don't have any Rockchip maintainers / contributors on this thread.
Would you like to start a new one, or add them to this thread?

>
> > > A side point is that binman should not be storing firmware build
> > > specifics in the device tree which is a means of describing the
> > > hardware, This really needs to be fixed as it really isn't the right
> > > place for that sort of things. I suspect a file in arch/arm/mach-<SOC>
> > > is likely a better location, or if it's board specific in the board/
> > > sub directory.
> >
> > Sorry, I don't agree with that at all. We store configuration
> > information in devicetree in firmware as this seems to be best format
> > for it, particularly with the growing number of firmware components
> > that need to share this information at runtime. The layout of firmware
> > is an important part of the system. We are still figuring out the
> > flows though. Also I have not attempted to upstream the binman
> > binding. I am very open to ideas on how best to do that.
>
> Rob what's your thoughts on the binman firmware build pieces being in
> device tree and the process on upstreaming the bindings?

It might be easier for Rob to comment on an actual proposal, which I
have not done. It is on my radar though.

Regards,
Simon


>
> Regards,
> Peter
>
> > > [1] https://source.denx.de/u-boot/u-boot/-/blob/v2023.01/arch/arm/mach-rockchip/make_fit_atf.py#L227
> > > [2] https://source.denx.de/u-boot/u-boot/-/blob/master/arch/arm/dts/sunxi-u-boot.dtsi#L64