support: Allow compression of sdcard.img

Message ID 20171218135144.4000-1-didin@synopsys.com
State Rejected
Headers show
Series
  • support: Allow compression of sdcard.img
Related show

Commit Message

Evgeniy Didin Dec. 18, 2017, 1:51 p.m.
In image files, generated by genimage.sh, there
are tens of megabytes most of which might be filled
with zeros as partitions are made a bit larger than
needed for minimalictic rootfs (which allows users to
add more libs and aps later on without messign with
partition sizes).

So adding an option for genimage.sh script, setting which
we gzip image file and greatly reduce it size.

Signed-off-by: Evgeniy Didin <didin@synopsys.com>
Cc: Alexey Brodkin <abrodkin@synopsys.com>
Cc: arc-buildroot@synopsys.com
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 support/scripts/genimage.sh | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

Comments

Yann E. MORIN Jan. 7, 2018, 11 p.m. | #1
Evgeniy, All,

On 2017-12-18 16:51 +0300, Evgeniy Didin spake thusly:
> In image files, generated by genimage.sh, there
> are tens of megabytes most of which might be filled
> with zeros as partitions are made a bit larger than
> needed for minimalictic rootfs (which allows users to
> add more libs and aps later on without messign with
> partition sizes).
> 
> So adding an option for genimage.sh script, setting which
> we gzip image file and greatly reduce it size.

Sorry, I am not in favour if this.

Today you're adding gzip compression (and by the way, -g is really
unusual for a compresion option, it usually is called -z), but tomorrow
someone will want xz compression, then another one will want lzo or lzip
or whatever...

And then someone will want to add gpg signature and encryption or
whatnot...

So, in such situation, I would argue that one should provide their
post-image script, that does whatever last-minute tweaks on the
generated image(s).

And by the way, this patch compreses the image, but does not remove the
original, so this is actually a loss of space.

So I am not in favour of this patch, sorry.

Regards,
Yann E. MORIN.

> Signed-off-by: Evgeniy Didin <didin@synopsys.com>
> Cc: Alexey Brodkin <abrodkin@synopsys.com>
> Cc: arc-buildroot@synopsys.com
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  support/scripts/genimage.sh | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/support/scripts/genimage.sh b/support/scripts/genimage.sh
> index 039b3fef1d..fa00e12855 100755
> --- a/support/scripts/genimage.sh
> +++ b/support/scripts/genimage.sh
> @@ -1,16 +1,17 @@
>  #!/usr/bin/env bash
> +# -g option gzips IMAGE_FILE to reduce it size
>  
>  die() {
>    cat <<EOF >&2
>  Error: $@
>  
> -Usage: ${0} -c GENIMAGE_CONFIG_FILE
> +Usage: ${0} -c GENIMAGE_CONFIG_FILE -g IMAGE_FILE
>  EOF
>    exit 1
>  }
>  
>  # Parse arguments and put into argument list of the script
> -opts="$(getopt -n "${0##*/}" -o c: -- "$@")" || exit $?
> +opts="$(getopt -n "${0##*/}" -o c:g: -- "$@")" || exit $?
>  eval set -- "$opts"
>  
>  GENIMAGE_TMP="${BUILD_DIR}/genimage.tmp"
> @@ -23,6 +24,9 @@ while true ; do
>  	--) # Discard all non-option parameters
>  	  shift 1;
>  	  break ;;
> +        -g)
> +	  GENIMAGE_NAME="${2}";
> +          shift 2;;
>  	*)
>  	  die "unknown option '${1}'" ;;
>  	esac
> @@ -38,3 +42,8 @@ genimage \
>  	--inputpath "${BINARIES_DIR}"  \
>  	--outputpath "${BINARIES_DIR}" \
>  	--config "${GENIMAGE_CFG}"
> +if [ ${GENIMAGE_NAME} ]
> +then
> +echo ${GENIMAGE_NAME}
> +gzip < ${BINARIES_DIR}/${GENIMAGE_NAME} > ${BINARIES_DIR}/${GENIMAGE_NAME}.gz
> +fi
> -- 
> 2.11.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
Alexey Brodkin Jan. 10, 2018, 6:35 p.m. | #2
Hi Yann,

On Mon, 2018-01-08 at 00:00 +0100, Yann E. MORIN wrote:
> Evgeniy, All,
> 
> On 2017-12-18 16:51 +0300, Evgeniy Didin spake thusly:
> > In image files, generated by genimage.sh, there
> > are tens of megabytes most of which might be filled
> > with zeros as partitions are made a bit larger than
> > needed for minimalictic rootfs (which allows users to
> > add more libs and aps later on without messign with
> > partition sizes).
> > 
> > So adding an option for genimage.sh script, setting which
> > we gzip image file and greatly reduce it size.
> 
> Sorry, I am not in favour if this.
> 
> Today you're adding gzip compression (and by the way, -g is really
> unusual for a compresion option, it usually is called -z), but tomorrow
> someone will want xz compression, then another one will want lzo or lzip
> or whatever...

Well I'm not sure if the idea behind that move was clearly explained.
The point was to generate larger first partition so users may squeeze more
stuff there (like splashscreens/logos for U-Boot, larger uImage/zImage or
a couple of them even) while keeping smaller footprint of the resulting artifact
(sdcard.img.xxx). We were inspired by OpenWrt guys who compress images,
see https://downloads.lede-project.org/releases/17.01.4/targets/brcm2708/bcm2710/lede-17.01.4-brcm2708-bcm2710-rpi-3-ext4-sdcard.img.gz
as an example.

> And then someone will want to add gpg signature and encryption or
> whatnot...

That's for sure :)

> So, in such situation, I would argue that one should provide their
> post-image script, that does whatever last-minute tweaks on the
> generated image(s).

Ok it's OK by me as well.

> And by the way, this patch compreses the image, but does not remove the
> original, so this is actually a loss of space.

Sure, that should be addressed.

> So I am not in favour of this patch, sorry.

Anyways, thanks for the review!

-Alexey

Patch

diff --git a/support/scripts/genimage.sh b/support/scripts/genimage.sh
index 039b3fef1d..fa00e12855 100755
--- a/support/scripts/genimage.sh
+++ b/support/scripts/genimage.sh
@@ -1,16 +1,17 @@ 
 #!/usr/bin/env bash
+# -g option gzips IMAGE_FILE to reduce it size
 
 die() {
   cat <<EOF >&2
 Error: $@
 
-Usage: ${0} -c GENIMAGE_CONFIG_FILE
+Usage: ${0} -c GENIMAGE_CONFIG_FILE -g IMAGE_FILE
 EOF
   exit 1
 }
 
 # Parse arguments and put into argument list of the script
-opts="$(getopt -n "${0##*/}" -o c: -- "$@")" || exit $?
+opts="$(getopt -n "${0##*/}" -o c:g: -- "$@")" || exit $?
 eval set -- "$opts"
 
 GENIMAGE_TMP="${BUILD_DIR}/genimage.tmp"
@@ -23,6 +24,9 @@  while true ; do
 	--) # Discard all non-option parameters
 	  shift 1;
 	  break ;;
+        -g)
+	  GENIMAGE_NAME="${2}";
+          shift 2;;
 	*)
 	  die "unknown option '${1}'" ;;
 	esac
@@ -38,3 +42,8 @@  genimage \
 	--inputpath "${BINARIES_DIR}"  \
 	--outputpath "${BINARIES_DIR}" \
 	--config "${GENIMAGE_CFG}"
+if [ ${GENIMAGE_NAME} ]
+then
+echo ${GENIMAGE_NAME}
+gzip < ${BINARIES_DIR}/${GENIMAGE_NAME} > ${BINARIES_DIR}/${GENIMAGE_NAME}.gz
+fi