From patchwork Tue May 17 03:24:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus X-Patchwork-Id: 1632003 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=cosmos-ink.net header.i=@cosmos-ink.net header.a=rsa-sha256 header.s=dkim header.b=WZEiwVnh; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=buildroot.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=buildroot-bounces@buildroot.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2M2V0bGTz9s1l for ; Tue, 17 May 2022 13:25:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 271196113B; Tue, 17 May 2022 03:25:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L8ASr2hncxIK; Tue, 17 May 2022 03:25:35 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id 27BA0610D8; Tue, 17 May 2022 03:25:34 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id AD7AD1BF322 for ; Tue, 17 May 2022 03:25:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 9900E40B95 for ; Tue, 17 May 2022 03:25:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=cosmos-ink.net Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F0aEC9fftxt6 for ; Tue, 17 May 2022 03:25:31 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from mail.cosmos-ink.net (mail.cosmos-ink.net [IPv6:2a01:4f9:4a:25aa::13]) by smtp2.osuosl.org (Postfix) with ESMTPS id 52CB540496 for ; Tue, 17 May 2022 03:25:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 6BDE71C666DE; Tue, 17 May 2022 05:25:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmos-ink.net; s=dkim; t=1652757928; h=from:subject:date:message-id:to:cc:mime-version: content-transfer-encoding; bh=WUSBvQhxlgFSeXrCMaHYN9+dLbVepMQAzoQCTmE6taU=; b=WZEiwVnhnEaTnuvVojEh5Kry7m0HffcfcdANGOf8wP0C3jzvg+B9GMnTDIrP3E1ChnLHho yVmRdd5e/d/jjPvwr6FriDiXKcuG0az2pVNWDeOP3mOwa6Q7KRjo2/r72f8gcxulwR5I5k cKxcOM4pIL2ZCpRHyRMMVLta703BSqC/GcEkF+ZLwbuqq6pGuf9Z9QB59XCspGUvmGnZSO MB6piBj/negYgrk7nju3ihkoK4CyHcJ9VnXN31sWinvRNqUHPlpZd+kO3a0xF6Nh0lglFm 9z8qRepEQiPyuaPraXWkgRu3+SMAunK1cykKgV3eT0NPVtvvdfl7f4SVNVc52Q== To: buildroot@buildroot.org Date: Tue, 17 May 2022 03:24:58 +0000 Message-Id: <20220517032459.3400-1-linus@cosmos-ink.net> MIME-Version: 1.0 X-Last-TLS-Session-Version: TLSv1.3 Subject: [Buildroot] [PATCH 1/2] fs/squashfs: Add block size option X-BeenThere: buildroot@buildroot.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Linus Kaschulla via buildroot From: Linus Reply-To: Linus Kaschulla Cc: Linus Kaschulla , "Yann E . MORIN" Errors-To: buildroot-bounces@buildroot.org Sender: "buildroot" One advantage of squashfs over similar technologies is the support for bigger block sizes. However the default size is not a lot bigger (typically 128k if no `-b` flag specified). This patch adds the ability to select from common block sizes which for example can aid in improving compression ratio. Signed-off-by: Linus Kaschulla --- fs/squashfs/Config.in | 50 +++++++++++++++++++++++++++++++++++++++++ fs/squashfs/squashfs.mk | 14 +++++++++++- 2 files changed, 63 insertions(+), 1 deletion(-) diff --git a/fs/squashfs/Config.in b/fs/squashfs/Config.in index 70d4a20cf0..341e50a08c 100644 --- a/fs/squashfs/Config.in +++ b/fs/squashfs/Config.in @@ -5,6 +5,56 @@ config BR2_TARGET_ROOTFS_SQUASHFS if BR2_TARGET_ROOTFS_SQUASHFS +choice + prompt "block size" + default BR2_TARGET_ROOTFS_SQUASHFS_BS_128K + help + Data block size. Bigger values can improve + compression ratio. + + If unsure, leave at 128k (default). + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_4K + bool "4k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_8K + bool "8k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_16K + bool "16k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_32K + bool "32k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_64K + bool "64k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_128K + bool "128k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_256K + bool "256k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_512K + bool "512k" + +config BR2_TARGET_ROOTFS_SQUASHFS_BS_1024K + bool "1024k" + +endchoice + +config BR2_TARGET_ROOTFS_SQUASHFS_BS + string + default "4K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_4K + default "8K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_84K + default "16K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_16K + default "32K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_32K + default "64K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_64K + default "128K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_128K + default "256K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_256K + default "512K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_512K + default "1024K" if BR2_TARGET_ROOTFS_SQUASHFS_BS_1024K + config BR2_TARGET_ROOTFS_SQUASHFS_PAD bool "pad to a 4K boundary" default y # legacy was always ON diff --git a/fs/squashfs/squashfs.mk b/fs/squashfs/squashfs.mk index 7a5e3e313e..1790773f7e 100644 --- a/fs/squashfs/squashfs.mk +++ b/fs/squashfs/squashfs.mk @@ -6,7 +6,19 @@ ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs -ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS) +ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS) -b $(BR2_TARGET_ROOTFS_SQUASHFS_BS) + +ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_BS_64K),y) +ROOTFS_SQUASHFS_ARGS += -b 64K +else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_BS_128K),y) +ROOTFS_SQUASHFS_ARGS += -b 128K +else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_BS_256K),y) +ROOTFS_SQUASHFS_ARGS += -b 256K +else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_BS_512K),y) +ROOTFS_SQUASHFS_ARGS += -b 512K +else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_BS_1024K),y) +ROOTFS_SQUASHFS_ARGS += -b 1024K +endif ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS_PAD),) ROOTFS_SQUASHFS_ARGS += -nopad From patchwork Tue May 17 03:24:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus X-Patchwork-Id: 1632004 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=cosmos-ink.net header.i=@cosmos-ink.net header.a=rsa-sha256 header.s=dkim header.b=L8GtP3SS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=buildroot.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=buildroot-bounces@buildroot.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4L2M2l0k9tz9s1l for ; Tue, 17 May 2022 13:25:51 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id DD91A834AE; Tue, 17 May 2022 03:25:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oDu6TeWHJu21; Tue, 17 May 2022 03:25:48 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id DB02983490; Tue, 17 May 2022 03:25:46 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 8E31E1BF322 for ; Tue, 17 May 2022 03:25:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7C82C41956 for ; Tue, 17 May 2022 03:25:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=cosmos-ink.net Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 501uMlhB9oiO for ; Tue, 17 May 2022 03:25:34 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from mail.cosmos-ink.net (mail.cosmos-ink.net [IPv6:2a01:4f9:4a:25aa::13]) by smtp4.osuosl.org (Postfix) with ESMTPS id 7ADB64194D for ; Tue, 17 May 2022 03:25:34 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 1A2681C666E9; Tue, 17 May 2022 05:25:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cosmos-ink.net; s=dkim; t=1652757932; h=from:subject:date:message-id:to:cc:mime-version: content-transfer-encoding:in-reply-to:references; bh=w4E+rfy3EPAWgOcXQb580m5u10WfhO2uqq/QZ0M9VAE=; b=L8GtP3SSWvqw6tLxqdX0PH/SMWOpEm57K0KFkR0hZX98NuoxGR0R1d/EWvvE9k/0JV/6eK ssuw+bELGLp+YAHG5wy2uRWXYtd6cfewirJz8OmeyhTNinmsBmE2E0GZvi7+ndeQiCRaii wbLFhEsDtlX9amcw0VigSmug16lGAlAMDZyPDfZ+RQeq3kNVYyCWMGzHeuDvxr/yjmnP40 cpPGvFuStUSLOE7jekzR/1UCzz+agZwkf/S9wX83lxzH0biYBkhRO6/KQ2U0dW3Hk7XEYd leyLv35m5kUKXOy2s6etuF2BvoyUs12vRYTfFhQrksdON9tjgtnSciKhY0VXzg== To: buildroot@buildroot.org Date: Tue, 17 May 2022 03:24:59 +0000 Message-Id: <20220517032459.3400-2-linus@cosmos-ink.net> In-Reply-To: <20220517032459.3400-1-linus@cosmos-ink.net> References: <20220517032459.3400-1-linus@cosmos-ink.net> MIME-Version: 1.0 X-Last-TLS-Session-Version: TLSv1.3 Subject: [Buildroot] [PATCH 2/2] fs/squashfs: Add option to compress with extreme options X-BeenThere: buildroot@buildroot.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Linus Kaschulla via buildroot From: Linus Reply-To: Linus Kaschulla Cc: Linus Kaschulla , "Yann E . MORIN" Errors-To: buildroot-bounces@buildroot.org Sender: "buildroot" mksquashfs allows to tweak many compressions parameters. Currently they can't be changed from kmenu. Leaving out potential space savings. This adds the option to enable a set of predetermined compression options. This option is enabled by default for lz4 since lz4 currently implicitly added the extreme to it in the makefile. So this aids in keeping backward compatibility. Signed-off-by: Linus Kaschulla --- fs/squashfs/Config.in | 26 ++++++++++++++++++++++++++ fs/squashfs/squashfs.mk | 6 +++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/fs/squashfs/Config.in b/fs/squashfs/Config.in index 341e50a08c..41c782105d 100644 --- a/fs/squashfs/Config.in +++ b/fs/squashfs/Config.in @@ -89,4 +89,30 @@ config BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD endchoice +config BR2_TARGET_ROOTFS_SQUASHFS_EXTREME_COMP + bool "extreme compression if possible" + default BR2_TARGET_ROOTFS_SQUASHFS4_LZ4 + help + Use options to increase compression ration as much as + possible, like using architecture-specific options, at + the cost of time when assembling the filesystem image. + + For example: + - with gzip and lzo, use -Xcompression-level 9 + - with xz use a architecture specific bjc (branch-call-jump) filter + - with zstd use -Xcompression-level 22 + - and more + +config BR2_TARGET_ROOTFS_SQUASHFS_COMP_OPTS + string + depends on BR2_TARGET_ROOTFS_SQUASHFS_EXTREME_COMP + default "-Xcompression-level 9" if BR2_TARGET_ROOTFS_SQUASHFS4_GZIP + default "-Xcompression-level 9" if BR2_TARGET_ROOTFS_SQUASHFS4_LZO + default "-Xhc" if BR2_TARGET_ROOTFS_SQUASHFS4_LZ4 + default "-Xbcj arm,armthumb" if BR2_TARGET_ROOTFS_SQUASHFS4_XZ && (BR2_arm || BR_aarch64) + default "-Xbcj powerpc" if BR2_TARGET_ROOTFS_SQUASHFS4_XZ && (BR2_powerpc || BR2_powerpc64) + default "-Xbcj sparc" if BR2_TARGET_ROOTFS_SQUASHFS4_XZ && (BR2_sparc || BR2_sparc64) + default "-Xbcj x86" if BR2_TARGET_ROOTFS_SQUASHFS4_XZ && (BR2_i386 || BR2_x86_64) + default "-Xcompression-level 22" if BR2_TARGET_ROOTFS_SQUASHFS4_ZSTD + endif diff --git a/fs/squashfs/squashfs.mk b/fs/squashfs/squashfs.mk index 1790773f7e..51f8d0d7c9 100644 --- a/fs/squashfs/squashfs.mk +++ b/fs/squashfs/squashfs.mk @@ -25,7 +25,7 @@ ROOTFS_SQUASHFS_ARGS += -nopad endif ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZ4),y) -ROOTFS_SQUASHFS_ARGS += -comp lz4 -Xhc +ROOTFS_SQUASHFS_ARGS += -comp lz4 else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZO),y) ROOTFS_SQUASHFS_ARGS += -comp lzo else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZMA),y) @@ -38,6 +38,10 @@ else ROOTFS_SQUASHFS_ARGS += -comp gzip endif +ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS_EXTREME_COMP),y) +ROOTFS_SQUASHFS_ARGS += $(call qstrip,$(BR2_TARGET_ROOTFS_SQUASHFS_COMP_OPTS)) +endif + define ROOTFS_SQUASHFS_CMD $(HOST_DIR)/bin/mksquashfs $(TARGET_DIR) $@ $(ROOTFS_SQUASHFS_ARGS) endef