Message ID | 20210917131851.31717-1-dev@open-music-kontrollers.ch |
---|---|
State | Rejected |
Headers | show |
Series | [1/1] linux: add compressions for appended initramfs | expand |
Hanspeter, All, On 2021-09-17 15:18 +0200, Hanspeter Portner spake thusly: > This changes take effect in the case of BR2_TARGET_ROOTFS_INITRAMFS=y > and one of BR2_LINUX_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ}=y. > > Instead of using the kernel's default compression (gzip) for appended initramfs, > this changes set initramfs compression explicitely to the same compression as > the kernel itself, which seems a sensible thing to do, imho. Hmmm... But isn't the initranfs compressed as part of the kernel already? See help for CONFIG_INITRAMFS_COMPRESSION_NONE: Do not compress the built-in initramfs at all. This may sound wasteful in space, but, you should be aware that the built-in initramfs will be compressed at a later stage anyways along with the rest of the kernel, on those architectures that support this. However, not compressing the initramfs may lead to slightly higher memory consumption during a short time at boot, while both the cpio image and the unpacked filesystem image will be present in memory simultaneously Additionally, compressing the initramfs separately, means that it will be copmopressed twice at build time, thus increasing the build time (but who cares? ;-) ), but also means it will be decompressed twice, thus increasing the boot time, every time the device boots. So, forcibly setting the compression to that of the kernel seems totally superfluous, _as kernel image size matters_; indeed, simply disabling initramfs compression (actually, cpio compression) would seem like the only correct thing to do. The only reason when we would want to actually compress the initramfs, as hinted in the help text I quoted above, is for machines that are tight on RAM, so tight indeed that they can't easily fit the whole uncompressed cpio and the extracted initramfs at the same time at boot (the uncompressed cpio is eventually garbage-collected once the initramfs is fully populated). Nowadays, that probably means machine with less than a very few MiB of RAM, something around 4MiB. Those kind of machines do exist, but they are often better served without using an initramfs (i.e. a root on a flash or something), or are going to require a very tiny initramfs anyway. So, I wonder if we should not simply disable compression of the initramfs altogether... Could look into that, and look at actual numbers for the resulting kernel image when the initramfs is compressed or not? Regards, Yann E. MORIN. > Apart from these kernel config parameters ... > > * CONFIG_KERNEL_GZIP > * CONFIG_KERNEL_LZ4 > * CONFIG_KERNEL_LZMA > * CONFIG_KERNEL_LZO > * CONFIG_KERNEL_XZ > * CONFIG_KERNEL_ZSTD > * CONFIG_KERNEL_UNCOMPRESSED > > ... we now also set these new kernel config parameters, as without those, > the appended initramfs seems to always be compressed with gzip: > > * CONFIG_INITRAMFS_COMPRESSION_GZIP > * CONFIG_INITRAMFS_COMPRESSION_LZ4 > * CONFIG_INITRAMFS_COMPRESSION_LZMA > * CONFIG_INITRAMFS_COMPRESSION_LZO > * CONFIG_INITRAMFS_COMPRESSION_XZ > * CONFIG_INITRAMFS_COMPRESSION_NONE > > The former are automatically set depending on whether one of the > following buildroot config parameters is set analogously to how it is done > for CONFIG_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ,ZSTD,UNCOMPRESSED}: > > * BR2_LINUX_KERNEL_GZIP > * BR2_LINUX_KERNEL_LZ4 > * BR2_LINUX_KERNEL_LZMA > * BR2_LINUX_KERNEL_LZO > * BR2_LINUX_KERNEL_XZ > * BR2_LINUX_KERNEL_UNCOMPRESSED > > Signed-off-by: Hanspeter Portner <dev@open-music-kontrollers.ch> > --- > linux/linux.mk | 13 +++++++++++++ > 1 file changed, 13 insertions(+) > > diff --git a/linux/linux.mk b/linux/linux.mk > index 61fdc0c76c..f192341a32 100644 > --- a/linux/linux.mk > +++ b/linux/linux.mk > @@ -106,6 +106,15 @@ LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_KERNEL_XZ > LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_KERNEL_ZSTD > LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_KERNEL_UNCOMPRESSED > > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_GZIP) += CONFIG_INITRAMFS_COMPRESSION_GZIP > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZ4) += CONFIG_INITRAMFS_COMPRESSION_LZ4 > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZMA) += CONFIG_INITRAMFS_COMPRESSION_LZMA > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZO) += CONFIG_INITRAMFS_COMPRESSION_LZO > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_INITRAMFS_COMPRESSION_XZ > +# zstd compression seems not supported, yet > +#INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_INITRAMFS_COMPRESSION_ZSTD > +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_INITRAMFS_COMPRESSION_NONE > + > ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) > LINUX_DEPENDENCIES += host-openssl > endif > @@ -352,6 +361,10 @@ define LINUX_KCONFIG_FIXUP_CMDS > $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_SOURCE,"$${BR_BINARIES_DIR}/rootfs.cpio") > $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_UID,0) > $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_GID,0)) > + $(call KCONFIG_ENABLE_OPT,$(strip $(INITRAMFS_COMPRESSION_OPT_y))) > + $(foreach opt, $(INITRAMFS_COMPRESSION_OPT_), > + $(call KCONFIG_DISABLE_OPT,$(opt)) > + ) > $(if $(BR2_ROOTFS_DEVICE_CREATION_STATIC),, > $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS) > $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS_MOUNT)) > -- > 2.33.0 > > _______________________________________________ > buildroot mailing list > buildroot@lists.buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
On 18/09/2021 09:33, Yann E. MORIN wrote: > Hanspeter, All, > > On 2021-09-17 15:18 +0200, Hanspeter Portner spake thusly: >> This changes take effect in the case of BR2_TARGET_ROOTFS_INITRAMFS=y >> and one of BR2_LINUX_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ}=y. >> >> Instead of using the kernel's default compression (gzip) for appended initramfs, >> this changes set initramfs compression explicitely to the same compression as >> the kernel itself, which seems a sensible thing to do, imho. > > Hmmm... But isn't the initranfs compressed as part of the kernel > already? See help for CONFIG_INITRAMFS_COMPRESSION_NONE: > > Do not compress the built-in initramfs at all. This may sound > wasteful in space, but, you should be aware that the built-in > initramfs will be compressed at a later stage anyways along > with the rest of the kernel, on those architectures that support > this. However, not compressing the initramfs may lead to slightly > higher memory consumption during a short time at boot, while both > the cpio image and the unpacked filesystem image will be present > in memory simultaneously > > Additionally, compressing the initramfs separately, means that it will > be copmopressed twice at build time, thus increasing the build time (but > who cares? ;-) ), but also means it will be decompressed twice, thus > increasing the boot time, every time the device boots. > > So, forcibly setting the compression to that of the kernel seems totally > superfluous, _as kernel image size matters_; indeed, simply disabling > initramfs compression (actually, cpio compression) would seem like the > only correct thing to do. > > The only reason when we would want to actually compress the initramfs, > as hinted in the help text I quoted above, is for machines that are > tight on RAM, so tight indeed that they can't easily fit the whole > uncompressed cpio and the extracted initramfs at the same time at boot > (the uncompressed cpio is eventually garbage-collected once the > initramfs is fully populated). > > Nowadays, that probably means machine with less than a very few MiB of > RAM, something around 4MiB. Those kind of machines do exist, but they > are often better served without using an initramfs (i.e. a root on a > flash or something), or are going to require a very tiny initramfs > anyway. > > So, I wonder if we should not simply disable compression of the > initramfs altogether... The current situation allows the user to choose through the kernel config, so sounds like the optimal approach. Unfortunately, the kernel apparently does default to gzip instead of uncompressed, so there's something to be said for forcing it to uncompressed... Anyway, setting it to the same as the kernel compression is useless I think. Therefore, I marked this patch as Rejected. Regards, Arnout > > Could look into that, and look at actual numbers for the resulting > kernel image when the initramfs is compressed or not? > > Regards, > Yann E. MORIN. > >> Apart from these kernel config parameters ... >> >> * CONFIG_KERNEL_GZIP >> * CONFIG_KERNEL_LZ4 >> * CONFIG_KERNEL_LZMA >> * CONFIG_KERNEL_LZO >> * CONFIG_KERNEL_XZ >> * CONFIG_KERNEL_ZSTD >> * CONFIG_KERNEL_UNCOMPRESSED >> >> ... we now also set these new kernel config parameters, as without those, >> the appended initramfs seems to always be compressed with gzip: >> >> * CONFIG_INITRAMFS_COMPRESSION_GZIP >> * CONFIG_INITRAMFS_COMPRESSION_LZ4 >> * CONFIG_INITRAMFS_COMPRESSION_LZMA >> * CONFIG_INITRAMFS_COMPRESSION_LZO >> * CONFIG_INITRAMFS_COMPRESSION_XZ >> * CONFIG_INITRAMFS_COMPRESSION_NONE >> >> The former are automatically set depending on whether one of the >> following buildroot config parameters is set analogously to how it is done >> for CONFIG_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ,ZSTD,UNCOMPRESSED}: >> >> * BR2_LINUX_KERNEL_GZIP >> * BR2_LINUX_KERNEL_LZ4 >> * BR2_LINUX_KERNEL_LZMA >> * BR2_LINUX_KERNEL_LZO >> * BR2_LINUX_KERNEL_XZ >> * BR2_LINUX_KERNEL_UNCOMPRESSED >> >> Signed-off-by: Hanspeter Portner <dev@open-music-kontrollers.ch> >> --- >> linux/linux.mk | 13 +++++++++++++ >> 1 file changed, 13 insertions(+) >> >> diff --git a/linux/linux.mk b/linux/linux.mk >> index 61fdc0c76c..f192341a32 100644 >> --- a/linux/linux.mk >> +++ b/linux/linux.mk >> @@ -106,6 +106,15 @@ LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_KERNEL_XZ >> LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_KERNEL_ZSTD >> LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_KERNEL_UNCOMPRESSED >> >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_GZIP) += CONFIG_INITRAMFS_COMPRESSION_GZIP >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZ4) += CONFIG_INITRAMFS_COMPRESSION_LZ4 >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZMA) += CONFIG_INITRAMFS_COMPRESSION_LZMA >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZO) += CONFIG_INITRAMFS_COMPRESSION_LZO >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_INITRAMFS_COMPRESSION_XZ >> +# zstd compression seems not supported, yet >> +#INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_INITRAMFS_COMPRESSION_ZSTD >> +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_INITRAMFS_COMPRESSION_NONE >> + >> ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) >> LINUX_DEPENDENCIES += host-openssl >> endif >> @@ -352,6 +361,10 @@ define LINUX_KCONFIG_FIXUP_CMDS >> $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_SOURCE,"$${BR_BINARIES_DIR}/rootfs.cpio") >> $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_UID,0) >> $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_GID,0)) >> + $(call KCONFIG_ENABLE_OPT,$(strip $(INITRAMFS_COMPRESSION_OPT_y))) >> + $(foreach opt, $(INITRAMFS_COMPRESSION_OPT_), >> + $(call KCONFIG_DISABLE_OPT,$(opt)) >> + ) >> $(if $(BR2_ROOTFS_DEVICE_CREATION_STATIC),, >> $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS) >> $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS_MOUNT)) >> -- >> 2.33.0 >> >> _______________________________________________ >> buildroot mailing list >> buildroot@lists.buildroot.org >> https://lists.buildroot.org/mailman/listinfo/buildroot >
diff --git a/linux/linux.mk b/linux/linux.mk index 61fdc0c76c..f192341a32 100644 --- a/linux/linux.mk +++ b/linux/linux.mk @@ -106,6 +106,15 @@ LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_KERNEL_XZ LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_KERNEL_ZSTD LINUX_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_KERNEL_UNCOMPRESSED +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_GZIP) += CONFIG_INITRAMFS_COMPRESSION_GZIP +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZ4) += CONFIG_INITRAMFS_COMPRESSION_LZ4 +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZMA) += CONFIG_INITRAMFS_COMPRESSION_LZMA +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_LZO) += CONFIG_INITRAMFS_COMPRESSION_LZO +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_XZ) += CONFIG_INITRAMFS_COMPRESSION_XZ +# zstd compression seems not supported, yet +#INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_ZSTD) += CONFIG_INITRAMFS_COMPRESSION_ZSTD +INITRAMFS_COMPRESSION_OPT_$(BR2_LINUX_KERNEL_UNCOMPRESSED) += CONFIG_INITRAMFS_COMPRESSION_NONE + ifeq ($(BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL),y) LINUX_DEPENDENCIES += host-openssl endif @@ -352,6 +361,10 @@ define LINUX_KCONFIG_FIXUP_CMDS $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_SOURCE,"$${BR_BINARIES_DIR}/rootfs.cpio") $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_UID,0) $(call KCONFIG_SET_OPT,CONFIG_INITRAMFS_ROOT_GID,0)) + $(call KCONFIG_ENABLE_OPT,$(strip $(INITRAMFS_COMPRESSION_OPT_y))) + $(foreach opt, $(INITRAMFS_COMPRESSION_OPT_), + $(call KCONFIG_DISABLE_OPT,$(opt)) + ) $(if $(BR2_ROOTFS_DEVICE_CREATION_STATIC),, $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS) $(call KCONFIG_ENABLE_OPT,CONFIG_DEVTMPFS_MOUNT))
This changes take effect in the case of BR2_TARGET_ROOTFS_INITRAMFS=y and one of BR2_LINUX_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ}=y. Instead of using the kernel's default compression (gzip) for appended initramfs, this changes set initramfs compression explicitely to the same compression as the kernel itself, which seems a sensible thing to do, imho. Apart from these kernel config parameters ... * CONFIG_KERNEL_GZIP * CONFIG_KERNEL_LZ4 * CONFIG_KERNEL_LZMA * CONFIG_KERNEL_LZO * CONFIG_KERNEL_XZ * CONFIG_KERNEL_ZSTD * CONFIG_KERNEL_UNCOMPRESSED ... we now also set these new kernel config parameters, as without those, the appended initramfs seems to always be compressed with gzip: * CONFIG_INITRAMFS_COMPRESSION_GZIP * CONFIG_INITRAMFS_COMPRESSION_LZ4 * CONFIG_INITRAMFS_COMPRESSION_LZMA * CONFIG_INITRAMFS_COMPRESSION_LZO * CONFIG_INITRAMFS_COMPRESSION_XZ * CONFIG_INITRAMFS_COMPRESSION_NONE The former are automatically set depending on whether one of the following buildroot config parameters is set analogously to how it is done for CONFIG_KERNEL_{GZIP,LZ4,LZMA,LZO,XZ,ZSTD,UNCOMPRESSED}: * BR2_LINUX_KERNEL_GZIP * BR2_LINUX_KERNEL_LZ4 * BR2_LINUX_KERNEL_LZMA * BR2_LINUX_KERNEL_LZO * BR2_LINUX_KERNEL_XZ * BR2_LINUX_KERNEL_UNCOMPRESSED Signed-off-by: Hanspeter Portner <dev@open-music-kontrollers.ch> --- linux/linux.mk | 13 +++++++++++++ 1 file changed, 13 insertions(+)