diff mbox

fs/squashfs: honour the number of parallel jobs

Message ID 20170802212239.27167-1-yann.morin.1998@free.fr
State Changes Requested
Headers show

Commit Message

Yann E. MORIN Aug. 2, 2017, 9:22 p.m. UTC
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
---
 fs/squashfs/squashfs.mk | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Comments

Thomas Petazzoni Aug. 3, 2017, 7:25 a.m. UTC | #1
Hello,

On Wed,  2 Aug 2017 23:22:39 +0200, Yann E. MORIN wrote:
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>

This would benefit from an extended commit log. Why do we do this? Is
this fixing something? Is this reducing the duration of mksquashfs
because it leverages multiple cores when doing the compression? In the
latter case, a few numbers would be nice to have.

Thanks,

Thomas
Yann E. MORIN Aug. 6, 2017, 3:40 p.m. UTC | #2
Thomas, All,

On 2017-08-03 09:25 +0200, Thomas Petazzoni spake thusly:
> On Wed,  2 Aug 2017 23:22:39 +0200, Yann E. MORIN wrote:
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
> 
> This would benefit from an extended commit log. Why do we do this? Is
> this fixing something? Is this reducing the duration of mksquashfs
> because it leverages multiple cores when doing the compression? In the
> latter case, a few numbers would be nice to have.

By default, mksquashfs uses all the CPUs available (e.g. like us with
BR2_JLEVEL=0).

However, a user may well set BR2_JLEVEL to a specific value, to
voluntarily limit the number of CPUs to use (e.g. like us on the
autobuilders).

So, this patch just propagates that BR2_JLEVEL setting to mksquashfs,
to use the same number of jobs as the rest of the build process.

So, there is not speed improvement to expect, quite the opposite in
fact. And this is why this patch can be seen as a mitigation (but not a
fix) for #10141, as it allows to reduce the parallelism on systems with
a huge number of CPUs, which is where #10141 occurs.

I'll add a bit more explanations to this patch and respin...

Regards,
Yann E. MORIN.
diff mbox

Patch

diff --git a/fs/squashfs/squashfs.mk b/fs/squashfs/squashfs.mk
index 342c7a8a37..7de7f51af1 100644
--- a/fs/squashfs/squashfs.mk
+++ b/fs/squashfs/squashfs.mk
@@ -6,6 +6,8 @@ 
 
 ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs
 
+ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+
 ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZ4),y)
 ROOTFS_SQUASHFS_ARGS += -comp lz4 -Xhc
 else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZO),y)
@@ -19,8 +21,7 @@  ROOTFS_SQUASHFS_ARGS += -comp gzip
 endif
 
 define ROOTFS_SQUASHFS_CMD
-	$(HOST_DIR)/bin/mksquashfs $(TARGET_DIR) $@ -noappend \
-		$(ROOTFS_SQUASHFS_ARGS)
+	$(HOST_DIR)/bin/mksquashfs $(TARGET_DIR) $@ $(ROOTFS_SQUASHFS_ARGS)
 endef
 
 $(eval $(call ROOTFS_TARGET,squashfs))