[RFC,RFT] imagebuilder: introduce Device Tree Blob rebuild option

From: Tomasz Maciej Nowak <tmn505@gmail.com>

Various advanced users have added or modified hardware for OpenWrt
supported boards. Naturally those mods can't be accepted in the
repository, so users of those modifications need to do full buildroot
build. This means almost every addition of package needs another rebuild
and flashing of new firmware complicating already running setup.
Fortunately ImageBuilder has almost all necessary resources to rebuild
Device Tree Blob (DTB), which almost all targets rely on to describe the
hardware. Using ImageBuilder to build modified firmware cuts necessary
time to few minutes. Another benefit is ability to use official package
repository, esspecialy for kmods, which otherwise would conflict with
kernel package hash. Thus I'm introducing this patch applicable directly
on 24.10.x ImageBuilder for ease of testing and comments. This patch is
"would be" result of apllying paches from this tree [1], except
_get_bindings target which downloads SDK to copy matching dt-bindings.
Some devices need kernel+dtb wrapped in some kind of loader, so
REBUILD_DTB option downloads matching toolchain, which alows to recompile
the loader. The REBUILD_DTB option is limited to release artifacts, since
SNAPSHOT(s) artifacts are ephemeral, so using outdated ImageBuilder from
snapshots directory won't cause unnecessary strain on OpenWrt human
resources.

Please test by modifying ImageBuilder dts in (depending on target)

target/linux/(target)/dts
or
build_dir/target-*/linux-(target)/linux-(version)/arch/*/boot/dts

and don't shy commenting on the issues.

I've used these changes for a long time on ath79 or ramips, and because
questions about modifying firmware, which has still full benefit of
official repo packages, pop up frequently, I've decided to share this
and prepare patches for inclusion in official repo after initial test
and comment round.

1. https://github.com/tmn505/openwrt/tree/rebuild-dtb

Signed-off-by: Tomasz Maciej Nowak <tmn505@gmail.com>
---
 Makefile         | 77 ++++++++++++++++++++++++++++++++++++++++++++++--
 include/image.mk | 11 ++-----
 2 files changed, 77 insertions(+), 11 deletions(-)

Hi Tomasz!

Thanks for contributing to this topic!

Something like this was brought up in 2021 by Joe (jwmullally)
with subject "Custom DTS / DTB building with ImageBuilder" [1]
and was successfully exercised for older devices like [2], [3] and [4].

Do you have some real world example using this rebuild option?


[1]: https://lists.openwrt.org/pipermail/openwrt-devel/2021-March/034239.html
[2]: https://github.com/jwmullally/openwrt_wpa8630p_v2_fullmem
[3]: https://github.com/jwmullally/openwrt_wpa8630pv2_patched_firmware
[4]: https://github.com/xabolcs/openwrt_hootoo_ht-tm02_oldstable


Thanks,
Szabolcs

W dniu 29.05.2025 o 15:43, Szabolcs Hubai pisze:
> Hi Tomasz!

Hi

> Thanks for contributing to this topic!
> 
> Something like this was brought up in 2021 by Joe (jwmullally)
> with subject "Custom DTS / DTB building with ImageBuilder" [1]
> and was successfully exercised for older devices like [2], [3] and [4].

Hmm... I don't know how I've missed that, since I've been doing that around
same time. Issue then was that OpenWrt didn't publish separate toolchain
archive, so workaround was downloading SDK and extracting toolchain and
dt-bindings from it. Now the toolchain archives are provided, so if the images
that need some kind of wrapping, download and extract toolchain into
appropriate dir and You're able to rebuild firmware. Some of use cases
require building additional packages not present in official repo but that
can be achieved with SDK and later providing that package to IB
(ImageBuilder).

> Do you have some real world example using this rebuild option?

This is not exactly what is written this rebuild option does, but I patch IB
and use SDK to build 24.10 firmware for Dell ACi for which patches [5] and [6]
are sitting in queue waiting for inclusion.
Another case would be simple enable of USB ports like in this topic [7].
Or enabling of SPI flash memory on my OrangePi Zero. Or removing console on
my Dell ACi to attach RS485 converter. Along the way there were also requests
from users which replaced RAM chips or SPI flash with bigger ones.
With new enough bootloader one can patch the DTB from there but not every
target has it.
All of it can be done in fraction of time with benefit of compatibility
with official packages repository.

> [1]: https://lists.openwrt.org/pipermail/openwrt-devel/2021-March/034239.html
> [2]: https://github.com/jwmullally/openwrt_wpa8630p_v2_fullmem
> [3]: https://github.com/jwmullally/openwrt_wpa8630pv2_patched_firmware
> [4]: https://github.com/xabolcs/openwrt_hootoo_ht-tm02_oldstable

5. https://patchwork.ozlabs.org/project/openwrt/patch/20250202151028.84276-1-tmn505@terefe.re
6. https://patchwork.ozlabs.org/project/openwrt/patch/20250202151028.84276-2-tmn505@terefe.re
7. https://forum.openwrt.org/t/nanopi-neo-enable-usb1-riser-cards/230948/1

> Thanks,
> Szabolcs
Regards

Il giorno gio 29 mag 2025 alle ore 17:35 Tomasz Maciej Nowak
<tmn505@gmail.com> ha scritto:
>
> W dniu 29.05.2025 o 15:43, Szabolcs Hubai pisze:
> > Hi Tomasz!
>
> Hi
>
> > Thanks for contributing to this topic!
> >
> > Something like this was brought up in 2021 by Joe (jwmullally)
> > with subject "Custom DTS / DTB building with ImageBuilder" [1]
> > and was successfully exercised for older devices like [2], [3] and [4].
>
> Hmm... I don't know how I've missed that, since I've been doing that around
> same time. Issue then was that OpenWrt didn't publish separate toolchain
> archive, so workaround was downloading SDK and extracting toolchain and
> dt-bindings from it. Now the toolchain archives are provided, so if the images
> that need some kind of wrapping, download and extract toolchain into
> appropriate dir and You're able to rebuild firmware. Some of use cases
> require building additional packages not present in official repo but that
> can be achieved with SDK and later providing that package to IB
> (ImageBuilder).
>
> > Do you have some real world example using this rebuild option?
>
> This is not exactly what is written this rebuild option does, but I patch IB
> and use SDK to build 24.10 firmware for Dell ACi for which patches [5] and [6]
> are sitting in queue waiting for inclusion.
> Another case would be simple enable of USB ports like in this topic [7].
> Or enabling of SPI flash memory on my OrangePi Zero. Or removing console on
> my Dell ACi to attach RS485 converter. Along the way there were also requests
> from users which replaced RAM chips or SPI flash with bigger ones.
> With new enough bootloader one can patch the DTB from there but not every
> target has it.
> All of it can be done in fraction of time with benefit of compatibility
> with official packages repository.
>

There is also an idea of introducing CI test for DTS so introducing some kind of
easy system to compile DTS might be interesting.
Some kind of logic to pack all the required dependency for all the supported DTS
so that they can be externally built.

W dniu 29.05.2025 o 17:38, Christian Marangi (Ansuel) pisze:
> Il giorno gio 29 mag 2025 alle ore 17:35 Tomasz Maciej Nowak
> <tmn505@gmail.com> ha scritto:
>>
>> W dniu 29.05.2025 o 15:43, Szabolcs Hubai pisze:
>>> Hi Tomasz!
>>
>> Hi
>>
>>> Thanks for contributing to this topic!
>>>
>>> Something like this was brought up in 2021 by Joe (jwmullally)
>>> with subject "Custom DTS / DTB building with ImageBuilder" [1]
>>> and was successfully exercised for older devices like [2], [3] and [4].
>>
>> Hmm... I don't know how I've missed that, since I've been doing that around
>> same time. Issue then was that OpenWrt didn't publish separate toolchain
>> archive, so workaround was downloading SDK and extracting toolchain and
>> dt-bindings from it. Now the toolchain archives are provided, so if the images
>> that need some kind of wrapping, download and extract toolchain into
>> appropriate dir and You're able to rebuild firmware. Some of use cases
>> require building additional packages not present in official repo but that
>> can be achieved with SDK and later providing that package to IB
>> (ImageBuilder).
>>
>>> Do you have some real world example using this rebuild option?
>>
>> This is not exactly what is written this rebuild option does, but I patch IB
>> and use SDK to build 24.10 firmware for Dell ACi for which patches [5] and [6]
>> are sitting in queue waiting for inclusion.
>> Another case would be simple enable of USB ports like in this topic [7].
>> Or enabling of SPI flash memory on my OrangePi Zero. Or removing console on
>> my Dell ACi to attach RS485 converter. Along the way there were also requests
>> from users which replaced RAM chips or SPI flash with bigger ones.
>> With new enough bootloader one can patch the DTB from there but not every
>> target has it.
>> All of it can be done in fraction of time with benefit of compatibility
>> with official packages repository.
>>
> 
> There is also an idea of introducing CI test for DTS so introducing some kind of
> easy system to compile DTS might be interesting.
> Some kind of logic to pack all the required dependency for all the supported DTS
> so that they can be externally built.

IB has almost all dependencies to build DTB except dt-bindings [1]. With [2]
and [3] You've got almost all guards dropped, the last thing would be allowing
to invoke compile-dtb in IB nad creating target in IB top makefile.

Given that, I don't know if IB is good place to test DTS compilation.
The dt-bindings would need to be patched in some cases, also another hurdle
is difference in where the DTS are sourced from across different targets. Some
are compiled directly from target/linux/(target)/dts dir, while others are
overlay on top of kernel source in form of patches or
target/linux/(target)/files tree, which end up in build_dir $(LINUX_DIR) only.
The firs case is the easiest one, since we can simply apply the contributed
patch to the DTS in IB, while the second one is much more tricky.

Rebased patches are in: https://github.com/tmn505/openwrt/commits/rebuild-dtb

1. https://patchwork.ozlabs.org/project/openwrt/patch/20240227181238.114931-1-tmn505@terefe.re
2. https://patchwork.ozlabs.org/project/openwrt/patch/20240227181321.114954-1-tmn505@terefe.re
3. https://patchwork.ozlabs.org/project/openwrt/patch/20240227181346.114970-1-tmn505@terefe.re

Regards