diff mbox series

package/lzma-alone: new package

Message ID 20201026213433.4309-1-zajec5@gmail.com
State New
Headers show
Series package/lzma-alone: new package | expand

Commit Message

Rafał Miłecki Oct. 26, 2020, 9:34 p.m. UTC
From: Rafał Miłecki <rafal@milecki.pl>

This new package provides "lzma_alone" host binary based on the original
LZMA SDK. It provides few extra options when compared to the LZMA Utils
project "lzma" binary (already packaged as the "lzma").

This tool is required for preparing firmware images for some home
routers with picky bootloaders. It's main advantage is support for
specifying dictionary size and lc/lp/pb LZMA values.

Broadcom CFE bootloader for MIPS BCM47xx devices is known to fail to
boot images using LZMA compression with dictionary.
Broadcom cferam bootloader for ARM64 BCM4908 devices is known to fail to
boot images without uncompressed size specified.
Broadcom CFE for BCM63xx DSL devices has even different requirements.

Packaging LZMA SDK "lzma" as "lzma_alone" is how it's handled in the
Arch, Debian and Ubuntu.

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
---
 package/lzma-alone/lzma-alone.hash |  1 +
 package/lzma-alone/lzma-alone.mk   | 23 +++++++++++++++++++++++
 2 files changed, 24 insertions(+)
 create mode 100644 package/lzma-alone/lzma-alone.hash
 create mode 100644 package/lzma-alone/lzma-alone.mk

Comments

Thomas Petazzoni Oct. 27, 2020, 9:18 a.m. UTC | #1
Hello Rafał,

On Mon, 26 Oct 2020 22:34:33 +0100
Rafał Miłecki <zajec5@gmail.com> wrote:

> From: Rafał Miłecki <rafal@milecki.pl>
> 
> This new package provides "lzma_alone" host binary based on the original
> LZMA SDK. It provides few extra options when compared to the LZMA Utils
> project "lzma" binary (already packaged as the "lzma").
> 
> This tool is required for preparing firmware images for some home
> routers with picky bootloaders. It's main advantage is support for
> specifying dictionary size and lc/lp/pb LZMA values.
> 
> Broadcom CFE bootloader for MIPS BCM47xx devices is known to fail to
> boot images using LZMA compression with dictionary.
> Broadcom cferam bootloader for ARM64 BCM4908 devices is known to fail to
> boot images without uncompressed size specified.
> Broadcom CFE for BCM63xx DSL devices has even different requirements.
> 
> Packaging LZMA SDK "lzma" as "lzma_alone" is how it's handled in the
> Arch, Debian and Ubuntu.
> 
> Signed-off-by: Rafał Miłecki <rafal@milecki.pl>

I am a bit confused by all the different lzma projects out there.

The Debian source package lzma (https://tracker.debian.org/pkg/lzma)
produces the lzma, lzma-alone and lzma-dev packages, and its source
package is apparently referring to what you suggest to use here.

The Buildroot package lzma points to https://tukaani.org/lzma/, which
says "Users of LZMA Utils should move to XZ Utils." and where the
latest release is from 2008.

And then, there is the LZMA SDK at https://www.7-zip.org/sdk.html,
where the latest release is 19.00, from 2019, at
https://www.7-zip.org/a/lzma1900.7z. This seems to be the same files
provided at
https://sourceforge.net/projects/sevenzip/files/LZMA%20SDK/, but you're
using the lzma922 version.

Do you have some clarifications about this (apparent) mess ?

>  package/lzma-alone/lzma-alone.hash |  1 +
>  package/lzma-alone/lzma-alone.mk   | 23 +++++++++++++++++++++++
>  2 files changed, 24 insertions(+)
>  create mode 100644 package/lzma-alone/lzma-alone.hash
>  create mode 100644 package/lzma-alone/lzma-alone.mk

We probably want a Config.in.host entry to this package.

> +define HOST_LZMA_ALONE_INSTALL_CMDS
> +	$(INSTALL) -d -m 0755 $(HOST_DIR)/bin

Not needed, the following $(INSTALL) -D will create the destination
folder for you.

> +	$(INSTALL) -D -m 0755 $(@D)/CPP/7zip/Bundles/LzmaCon/lzma $(HOST_DIR)/bin/lzma_alone
> +endef
> +
> +$(eval $(host-generic-package))

Thanks!

Thomas
Rafał Miłecki Oct. 27, 2020, 2:33 p.m. UTC | #2
On 27.10.2020 10:18, Thomas Petazzoni wrote:
> On Mon, 26 Oct 2020 22:34:33 +0100
> Rafał Miłecki <zajec5@gmail.com> wrote:
> 
>> From: Rafał Miłecki <rafal@milecki.pl>
>>
>> This new package provides "lzma_alone" host binary based on the original
>> LZMA SDK. It provides few extra options when compared to the LZMA Utils
>> project "lzma" binary (already packaged as the "lzma").
>>
>> This tool is required for preparing firmware images for some home
>> routers with picky bootloaders. It's main advantage is support for
>> specifying dictionary size and lc/lp/pb LZMA values.
>>
>> Broadcom CFE bootloader for MIPS BCM47xx devices is known to fail to
>> boot images using LZMA compression with dictionary.
>> Broadcom cferam bootloader for ARM64 BCM4908 devices is known to fail to
>> boot images without uncompressed size specified.
>> Broadcom CFE for BCM63xx DSL devices has even different requirements.
>>
>> Packaging LZMA SDK "lzma" as "lzma_alone" is how it's handled in the
>> Arch, Debian and Ubuntu.
>>
>> Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
> 
> I am a bit confused by all the different lzma projects out there.
> 
> The Debian source package lzma (https://tracker.debian.org/pkg/lzma)
> produces the lzma, lzma-alone and lzma-dev packages, and its source
> package is apparently referring to what you suggest to use here.
> 
> The Buildroot package lzma points to https://tukaani.org/lzma/, which
> says "Users of LZMA Utils should move to XZ Utils." and where the
> latest release is from 2008.
> 
> And then, there is the LZMA SDK at https://www.7-zip.org/sdk.html,
> where the latest release is 19.00, from 2019, at
> https://www.7-zip.org/a/lzma1900.7z. This seems to be the same files
> provided at
> https://sourceforge.net/projects/sevenzip/files/LZMA%20SDK/, but you're
> using the lzma922 version.
> 
> Do you have some clarifications about this (apparent) mess ?

I wasn't fully aware of Debian solution. So it seems that Debian
developers:
1. Switched from LZMA Utils to the LZMA SDK
2. Kept "lzma" CLI from the LZMA Utils project

They did that by patching LZMA SDK sources (see
lzma_9.22-2.1.debian.tar-1.xz and its "patches"). They ported lzmp.cpp
from the LZMA Utils.

Advantages:
1. Single code base
2. Using the latest LZMA core code

Disadvantages:
1. Some missing tools like lzmainfo

To make things worse, there is also a XZ Utils project that also
provides binaries lzma (with CLI like in LZMA Utils), unlzma & lzmainfo.


What strategy should buildroot follow in this case?


As for version choise I used 9.22 as that is what Debian uses and it's
the latest release with sources available as tar.bz2 (newer use 7z).


>>   package/lzma-alone/lzma-alone.hash |  1 +
>>   package/lzma-alone/lzma-alone.mk   | 23 +++++++++++++++++++++++
>>   2 files changed, 24 insertions(+)
>>   create mode 100644 package/lzma-alone/lzma-alone.hash
>>   create mode 100644 package/lzma-alone/lzma-alone.mk
> 
> We probably want a Config.in.host entry to this package.

What extra options should I put in Config.in.host?


>> +define HOST_LZMA_ALONE_INSTALL_CMDS
>> +	$(INSTALL) -d -m 0755 $(HOST_DIR)/bin
> 
> Not needed, the following $(INSTALL) -D will create the destination
> folder for you.

Thanks!
Rafał Miłecki Nov. 5, 2020, 6:26 a.m. UTC | #3
Hi Thomas,

On 27.10.2020 15:33, Rafał Miłecki wrote:
> On 27.10.2020 10:18, Thomas Petazzoni wrote:
>> On Mon, 26 Oct 2020 22:34:33 +0100
>> Rafał Miłecki <zajec5@gmail.com> wrote:
>>
>>> From: Rafał Miłecki <rafal@milecki.pl>
>>>
>>> This new package provides "lzma_alone" host binary based on the original
>>> LZMA SDK. It provides few extra options when compared to the LZMA Utils
>>> project "lzma" binary (already packaged as the "lzma").
>>>
>>> This tool is required for preparing firmware images for some home
>>> routers with picky bootloaders. It's main advantage is support for
>>> specifying dictionary size and lc/lp/pb LZMA values.
>>>
>>> Broadcom CFE bootloader for MIPS BCM47xx devices is known to fail to
>>> boot images using LZMA compression with dictionary.
>>> Broadcom cferam bootloader for ARM64 BCM4908 devices is known to fail to
>>> boot images without uncompressed size specified.
>>> Broadcom CFE for BCM63xx DSL devices has even different requirements.
>>>
>>> Packaging LZMA SDK "lzma" as "lzma_alone" is how it's handled in the
>>> Arch, Debian and Ubuntu.
>>>
>>> Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
>>
>> I am a bit confused by all the different lzma projects out there.
>>
>> The Debian source package lzma (https://tracker.debian.org/pkg/lzma)
>> produces the lzma, lzma-alone and lzma-dev packages, and its source
>> package is apparently referring to what you suggest to use here.
>>
>> The Buildroot package lzma points to https://tukaani.org/lzma/, which
>> says "Users of LZMA Utils should move to XZ Utils." and where the
>> latest release is from 2008.
>>
>> And then, there is the LZMA SDK at https://www.7-zip.org/sdk.html,
>> where the latest release is 19.00, from 2019, at
>> https://www.7-zip.org/a/lzma1900.7z. This seems to be the same files
>> provided at
>> https://sourceforge.net/projects/sevenzip/files/LZMA%20SDK/, but you're
>> using the lzma922 version.
>>
>> Do you have some clarifications about this (apparent) mess ?
> 
> I wasn't fully aware of Debian solution. So it seems that Debian
> developers:
> 1. Switched from LZMA Utils to the LZMA SDK
> 2. Kept "lzma" CLI from the LZMA Utils project
> 
> They did that by patching LZMA SDK sources (see
> lzma_9.22-2.1.debian.tar-1.xz and its "patches"). They ported lzmp.cpp
> from the LZMA Utils.
> 
> Advantages:
> 1. Single code base
> 2. Using the latest LZMA core code
> 
> Disadvantages:
> 1. Some missing tools like lzmainfo
> 
> To make things worse, there is also a XZ Utils project that also
> provides binaries lzma (with CLI like in LZMA Utils), unlzma & lzmainfo.
> 
> 
> What strategy should buildroot follow in this case?

Any opinion on that?
diff mbox series

Patch

diff --git a/package/lzma-alone/lzma-alone.hash b/package/lzma-alone/lzma-alone.hash
new file mode 100644
index 0000000000..39759c4e3b
--- /dev/null
+++ b/package/lzma-alone/lzma-alone.hash
@@ -0,0 +1 @@ 
+sha256	9aade84f229fb25f7aef39d8866b375fe6d35a9e18098d7cd86a99e294902944	lzma922.tar.bz2
diff --git a/package/lzma-alone/lzma-alone.mk b/package/lzma-alone/lzma-alone.mk
new file mode 100644
index 0000000000..6a1d8dd3c0
--- /dev/null
+++ b/package/lzma-alone/lzma-alone.mk
@@ -0,0 +1,23 @@ 
+################################################################################
+#
+# lzma-alone
+#
+################################################################################
+
+LZMA_ALONE_VERSION = 9.22
+LZMA_ALONE_SITE = https://sourceforge.net/projects/sevenzip/files/LZMA%20SDK/
+LZMA_ALONE_SOURCE = lzma922.tar.bz2
+LZMA_ALONE_STRIP_COMPONENTS = 0
+LZMA_ALONE_LICENSE = LGPL-2.1-or-later, BSD-3-Clause
+
+define HOST_LZMA_ALONE_BUILD_CMDS
+	$(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) -C $(@D)/C/Util/Lzma -f makefile.gcc
+	$(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) -C $(@D)/CPP/7zip/Bundles/LzmaCon -f makefile.gcc
+endef
+
+define HOST_LZMA_ALONE_INSTALL_CMDS
+	$(INSTALL) -d -m 0755 $(HOST_DIR)/bin
+	$(INSTALL) -D -m 0755 $(@D)/CPP/7zip/Bundles/LzmaCon/lzma $(HOST_DIR)/bin/lzma_alone
+endef
+
+$(eval $(host-generic-package))