| Message ID | 20200721092259.26916-4-joaomarcos.costa@bootlin.com |
|---|---|
| State | Superseded |
| Delegated to: | Tom Rini |
| Headers | show |
| Series | Add support for the SquashFS filesystem | expand |
On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa wrote: > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on > this function to decompress data from a raw disk image. The actual > support for zlib into SquashFS sources will be added in a follow-up > commit. > > Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> > --- > Changes in v3: > - No changes since v2. > Changes in v2: > - Changed commit title and message, which were wrong and/or misleading > in v1. > include/u-boot/zlib.h | 32 ++++++++++++++ > lib/zlib/uncompr.c | 97 +++++++++++++++++++++++++++++++++++++++++++ > lib/zlib/zlib.c | 1 + > 3 files changed, 130 insertions(+) > create mode 100644 lib/zlib/uncompr.c Can you confirm that on a platform where we aren't enabling squashfs the new zlib functionality ends up being discarded at link time? Thanks!
On Wed, 29 Jul 2020 11:27:45 -0400 Tom Rini <trini@konsulko.com> wrote: > On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa wrote: > > > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on > > this function to decompress data from a raw disk image. The actual > > support for zlib into SquashFS sources will be added in a follow-up > > commit. > > > > Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> > > --- > > Changes in v3: > > - No changes since v2. > > Changes in v2: > > - Changed commit title and message, which were wrong and/or > > misleading in v1. > > include/u-boot/zlib.h | 32 ++++++++++++++ > > lib/zlib/uncompr.c | 97 > > +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c > > | 1 + 3 files changed, 130 insertions(+) > > create mode 100644 lib/zlib/uncompr.c > > Can you confirm that on a platform where we aren't enabling squashfs > the new zlib functionality ends up being discarded at link time? > Thanks! > I dumped lib/zlib/zlib.o with readelf and the new functionality is not discarded, even though the SquashFS support is not enabled, but this issue will be fixed in the next version (v4). Thank you!
On Wed, Jul 29, 2020 at 06:00:57PM +0200, Joao Marcos Costa wrote: > On Wed, 29 Jul 2020 11:27:45 -0400 > Tom Rini <trini@konsulko.com> wrote: > > > On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa wrote: > > > > > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on > > > this function to decompress data from a raw disk image. The actual > > > support for zlib into SquashFS sources will be added in a follow-up > > > commit. > > > > > > Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> > > > --- > > > Changes in v3: > > > - No changes since v2. > > > Changes in v2: > > > - Changed commit title and message, which were wrong and/or > > > misleading in v1. > > > include/u-boot/zlib.h | 32 ++++++++++++++ > > > lib/zlib/uncompr.c | 97 > > > +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c > > > | 1 + 3 files changed, 130 insertions(+) > > > create mode 100644 lib/zlib/uncompr.c > > > > Can you confirm that on a platform where we aren't enabling squashfs > > the new zlib functionality ends up being discarded at link time? > > Thanks! > > > > I dumped lib/zlib/zlib.o with readelf and the new functionality is not > discarded, even though the SquashFS support is not enabled, but this > issue will be fixed in the next version (v4). Thank you! Well, don't dump lib/zlib/zlib.o, that won't say. Check u-boot.map :)
On Wed, 29 Jul 2020 12:54:23 -0400 Tom Rini <trini@konsulko.com> wrote: > On Wed, Jul 29, 2020 at 06:00:57PM +0200, Joao Marcos Costa wrote: > > On Wed, 29 Jul 2020 11:27:45 -0400 > > Tom Rini <trini@konsulko.com> wrote: > > > > > On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa wrote: > > > > > > > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS > > > > depends on this function to decompress data from a raw disk > > > > image. The actual support for zlib into SquashFS sources will > > > > be added in a follow-up commit. > > > > > > > > Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> > > > > --- > > > > Changes in v3: > > > > - No changes since v2. > > > > Changes in v2: > > > > - Changed commit title and message, which were wrong > > > > and/or misleading in v1. > > > > include/u-boot/zlib.h | 32 ++++++++++++++ > > > > lib/zlib/uncompr.c | 97 > > > > +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c > > > > | 1 + 3 files changed, 130 insertions(+) > > > > create mode 100644 lib/zlib/uncompr.c > > > > > > Can you confirm that on a platform where we aren't enabling > > > squashfs the new zlib functionality ends up being discarded at > > > link time? Thanks! > > > > > > > I dumped lib/zlib/zlib.o with readelf and the new functionality is > > not discarded, even though the SquashFS support is not enabled, but > > this issue will be fixed in the next version (v4). Thank you! > > Well, don't dump lib/zlib/zlib.o, that won't say. Check u-boot.map :) > Thanks for the tip! Now I can confirm that the functionality is discarded when squashfs is not enabled. I just would like to know if I should send the tests in the same patch as the SquashFS commands, or should it be into a separate patch?
On Thu, Jul 30, 2020 at 02:10:03PM +0200, Joao Marcos Costa wrote: > On Wed, 29 Jul 2020 12:54:23 -0400 > Tom Rini <trini@konsulko.com> wrote: > > > On Wed, Jul 29, 2020 at 06:00:57PM +0200, Joao Marcos Costa wrote: > > > On Wed, 29 Jul 2020 11:27:45 -0400 > > > Tom Rini <trini@konsulko.com> wrote: > > > > > > > On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa wrote: > > > > > > > > > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS > > > > > depends on this function to decompress data from a raw disk > > > > > image. The actual support for zlib into SquashFS sources will > > > > > be added in a follow-up commit. > > > > > > > > > > Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> > > > > > --- > > > > > Changes in v3: > > > > > - No changes since v2. > > > > > Changes in v2: > > > > > - Changed commit title and message, which were wrong > > > > > and/or misleading in v1. > > > > > include/u-boot/zlib.h | 32 ++++++++++++++ > > > > > lib/zlib/uncompr.c | 97 > > > > > +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c > > > > > | 1 + 3 files changed, 130 insertions(+) > > > > > create mode 100644 lib/zlib/uncompr.c > > > > > > > > Can you confirm that on a platform where we aren't enabling > > > > squashfs the new zlib functionality ends up being discarded at > > > > link time? Thanks! > > > > > > > > > > I dumped lib/zlib/zlib.o with readelf and the new functionality is > > > not discarded, even though the SquashFS support is not enabled, but > > > this issue will be fixed in the next version (v4). Thank you! > > > > Well, don't dump lib/zlib/zlib.o, that won't say. Check u-boot.map :) > > > > Thanks for the tip! Now I can confirm that the functionality is > discarded when squashfs is not enabled. I just would like to know if I > should send the tests in the same patch as the SquashFS commands, or > should it be into a separate patch? A separate patch, and are you able to run them on sandbox then? So it should also enable squashfs on that target. Thanks!
On Thu, 30 Jul 2020 08:17:59 -0400 Tom Rini <trini@konsulko.com> wrote: > On Thu, Jul 30, 2020 at 02:10:03PM +0200, Joao Marcos Costa wrote: > > On Wed, 29 Jul 2020 12:54:23 -0400 > > Tom Rini <trini@konsulko.com> wrote: > > > > > On Wed, Jul 29, 2020 at 06:00:57PM +0200, Joao Marcos Costa > > > wrote: > > > > On Wed, 29 Jul 2020 11:27:45 -0400 > > > > Tom Rini <trini@konsulko.com> wrote: > > > > > > > > > On Tue, Jul 21, 2020 at 11:22:57AM +0200, Joao Marcos Costa > > > > > wrote: > > > > > > Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS > > > > > > depends on this function to decompress data from a raw disk > > > > > > image. The actual support for zlib into SquashFS sources > > > > > > will be added in a follow-up commit. > > > > > > > > > > > > Signed-off-by: Joao Marcos Costa > > > > > > <joaomarcos.costa@bootlin.com> --- > > > > > > Changes in v3: > > > > > > - No changes since v2. > > > > > > Changes in v2: > > > > > > - Changed commit title and message, which were wrong > > > > > > and/or misleading in v1. > > > > > > include/u-boot/zlib.h | 32 ++++++++++++++ > > > > > > lib/zlib/uncompr.c | 97 > > > > > > +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c > > > > > > | 1 + 3 files changed, 130 insertions(+) > > > > > > create mode 100644 lib/zlib/uncompr.c > > > > > > > > > > Can you confirm that on a platform where we aren't enabling > > > > > squashfs the new zlib functionality ends up being discarded at > > > > > link time? Thanks! > > > > > > > > > > > > > I dumped lib/zlib/zlib.o with readelf and the new functionality > > > > is not discarded, even though the SquashFS support is not > > > > enabled, but this issue will be fixed in the next version (v4). > > > > Thank you! > > > > > > Well, don't dump lib/zlib/zlib.o, that won't say. Check > > > u-boot.map :) > > > > Thanks for the tip! Now I can confirm that the functionality is > > discarded when squashfs is not enabled. I just would like to know > > if I should send the tests in the same patch as the SquashFS > > commands, or should it be into a separate patch? > > A separate patch, and are you able to run them on sandbox then? So it > should also enable squashfs on that target. Thanks! > This will be addressed in v4. Thanks!
diff --git a/include/u-boot/zlib.h b/include/u-boot/zlib.h index e23ceb50ca..a33cc8780d 100644 --- a/include/u-boot/zlib.h +++ b/include/u-boot/zlib.h @@ -110,6 +110,12 @@ extern "C" { # define voidp z_voidp #endif +#if defined(ZLIB_CONST) && !defined(z_const) +# define z_const const +#else +# define z_const +#endif + #if defined(__MSDOS__) && !defined(MSDOS) # define MSDOS #endif @@ -710,6 +716,32 @@ ZEXTERN uInt ZEXPORT crc32 OF((uInt crc, const Bytef *buf, uInt len)); if (crc != original_crc) error(); */ +ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong sourceLen)); +/* + Decompresses the source buffer into the destination buffer. sourceLen is + the byte length of the source buffer. Upon entry, destLen is the total size + of the destination buffer, which must be large enough to hold the entire + uncompressed data. (The size of the uncompressed data must have been saved + previously by the compressor and transmitted to the decompressor by some + mechanism outside the scope of this compression library.) Upon exit, destLen + is the actual size of the uncompressed data. + + uncompress returns Z_OK if success, Z_MEM_ERROR if there was not + enough memory, Z_BUF_ERROR if there was not enough room in the output + buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete. In + the case where there is not enough room, uncompress() will fill the output + buffer with the uncompressed data up to that point. +*/ + +ZEXTERN int ZEXPORT uncompress2 OF((Bytef *dest, uLongf *destLen, + const Bytef *source, uLong *sourceLen)); +/* + Same as uncompress, except that sourceLen is a pointer, where the + length of the source is *sourceLen. On return, *sourceLen is the number of + source bytes consumed. +*/ + ZEXTERN int ZEXPORT inflateInit2_ OF((z_streamp strm, int windowBits, const char *version, int stream_size)); #define inflateInit(strm) \ diff --git a/lib/zlib/uncompr.c b/lib/zlib/uncompr.c new file mode 100644 index 0000000000..21e93933b2 --- /dev/null +++ b/lib/zlib/uncompr.c @@ -0,0 +1,97 @@ +/* uncompr.c -- decompress a memory buffer + * Copyright (C) 1995-2003, 2010, 2014, 2016 Jean-loup Gailly, Mark Adler + * For conditions of distribution and use, see copyright notice in zlib.h + */ + +/* @(#) $Id$ */ + +#define ZLIB_INTERNAL +#include "zlib.h" + +/* =========================================================================== + Decompresses the source buffer into the destination buffer. *sourceLen is + the byte length of the source buffer. Upon entry, *destLen is the total size + of the destination buffer, which must be large enough to hold the entire + uncompressed data. (The size of the uncompressed data must have been saved + previously by the compressor and transmitted to the decompressor by some + mechanism outside the scope of this compression library.) Upon exit, + *destLen is the size of the decompressed data and *sourceLen is the number + of source bytes consumed. Upon return, source + *sourceLen points to the + first unused input byte. + + uncompress returns Z_OK if success, Z_MEM_ERROR if there was not enough + memory, Z_BUF_ERROR if there was not enough room in the output buffer, or + Z_DATA_ERROR if the input data was corrupted, including if the input data is + an incomplete zlib stream. +*/ +int ZEXPORT uncompress2(dest, destLen, source, sourceLen) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong *sourceLen; + +{ + z_stream stream; + int err; + const uInt max = (uInt)-1; + uLong len, left; + /* for detection of incomplete stream when *destLen == 0 */ + Byte buf[1]; + + len = *sourceLen; + if (*destLen) { + left = *destLen; + *destLen = 0; + } else { + left = 1; + dest = buf; + } + + stream.next_in = (z_const Bytef *)source; + stream.avail_in = 0; + stream.zalloc = (alloc_func)0; + stream.zfree = (free_func)0; + stream.opaque = (voidpf)0; + + err = inflateInit(&stream); + if (err != Z_OK) + return err; + + stream.next_out = dest; + stream.avail_out = 0; + + do { + if (stream.avail_out == 0) { + stream.avail_out = left > (uLong)max ? max : (uInt)left; + left -= stream.avail_out; + } + + if (stream.avail_in == 0) { + stream.avail_in = len > (uLong)max ? max : (uInt)len; + len -= stream.avail_in; + } + + err = inflate(&stream, Z_NO_FLUSH); + } while (err == Z_OK); + + *sourceLen -= len + stream.avail_in; + if (dest != buf) + *destLen = stream.total_out; + else if (stream.total_out && err == Z_BUF_ERROR) + left = 1; + + inflateEnd(&stream); + return err == Z_STREAM_END ? Z_OK : + err == Z_NEED_DICT ? Z_DATA_ERROR : + err == Z_BUF_ERROR && left + stream.avail_out ? Z_DATA_ERROR : + err; +} + +int ZEXPORT uncompress(dest, destLen, source, sourceLen) + Bytef *dest; + uLongf *destLen; + const Bytef *source; + uLong sourceLen; +{ + return uncompress2(dest, destLen, source, &sourceLen); +} diff --git a/lib/zlib/zlib.c b/lib/zlib/zlib.c index 7e1570292c..05a2734086 100644 --- a/lib/zlib/zlib.c +++ b/lib/zlib/zlib.c @@ -30,3 +30,4 @@ #include "inflate.c" #include "zutil.c" #include "adler32.c" +#include "uncompr.c"
Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on this function to decompress data from a raw disk image. The actual support for zlib into SquashFS sources will be added in a follow-up commit. Signed-off-by: Joao Marcos Costa <joaomarcos.costa@bootlin.com> --- Changes in v3: - No changes since v2. Changes in v2: - Changed commit title and message, which were wrong and/or misleading in v1. include/u-boot/zlib.h | 32 ++++++++++++++ lib/zlib/uncompr.c | 97 +++++++++++++++++++++++++++++++++++++++++++ lib/zlib/zlib.c | 1 + 3 files changed, 130 insertions(+) create mode 100644 lib/zlib/uncompr.c