From patchwork Fri Aug 5 10:24:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dominik Michael Rauh X-Patchwork-Id: 1664027 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=buildroot.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=buildroot-bounces@buildroot.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LzhYX42rZz9s1l for ; Fri, 5 Aug 2022 20:25:02 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BA2A360DB2; Fri, 5 Aug 2022 10:25:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BA2A360DB2 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hEpKqaX3dqXE; Fri, 5 Aug 2022 10:24:59 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id 90F876F93E; Fri, 5 Aug 2022 10:24:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 90F876F93E X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id E95151BF387 for ; Fri, 5 Aug 2022 10:24:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CFC1B41C8A for ; Fri, 5 Aug 2022 10:24:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org CFC1B41C8A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2wOHiVT9viiZ for ; Fri, 5 Aug 2022 10:24:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B9B0041C71 Received: from mout02.posteo.de (mout02.posteo.de [185.67.36.66]) by smtp4.osuosl.org (Postfix) with ESMTPS id B9B0041C71 for ; Fri, 5 Aug 2022 10:24:53 +0000 (UTC) Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id E4E76240107 for ; Fri, 5 Aug 2022 12:24:50 +0200 (CEST) Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4LzhYF4D88z9rxN; Fri, 5 Aug 2022 12:24:49 +0200 (CEST) From: Dominik Michael Rauh To: buildroot@buildroot.org Date: Fri, 5 Aug 2022 10:24:42 +0000 Message-Id: <20220805102442.8590-1-dmrauh@posteo.de> In-Reply-To: <20220804132939.17903-1-dmrauh@posteo.de> References: <20220804132939.17903-1-dmrauh@posteo.de> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1659695090; bh=S/j2vhxwI5oHdm34EC7jFg+pKtg9YxJY8Sm7cKgujrg=; h=From:To:Cc:Subject:Date:From; b=Tn1gVv8ibf1E/9UaqwIcyjzMZiOD7xaqgptApaszNJB4kQYvjyfJmGPtDoRRzxATr RnYtNOZdnfVrGMzMpQlvgSjh6wJWQPVScQyAW3ox+p7Zb/MDUUgHt8UtFbdblpwQcX IxxR1yhyiO9vj0Ok23E6y77kOdoGbT1O9aX5vSuweuFrRNXc7GdbQpuPWjK9a4t3jf mwAS/ueJmTp7qRwZ6EGH/12c64XTnW4GPfZLVkN9tv6NxLOrOO3DcVkJTadzroqbh8 xKmpUW24chzP7pZ06/Wi9+hTRyvKQ6AeFjApOI0nwBKNXEiDlu8M93YZZN/w2dYwVn wjBYeKyxQ57Pg== X-Mailman-Original-Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key) header.d=posteo.de header.i=@posteo.de header.a=rsa-sha256 header.s=2017 header.b=Tn1gVv8i Subject: [Buildroot] [PATCH v2] package/ccache: bump version to 4.6.1 X-BeenThere: buildroot@buildroot.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Anders_F_Bj=C3=B6rklund?= , Andrey Smirnov , Samuel Martin , Dominik Michael Rauh , Thomas Petazzoni , Fabrice Fontaine Errors-To: buildroot-bounces@buildroot.org Sender: "buildroot" Since version 4.0, the following major changes have been made to ccache, which affect its integration with Buildroot: - Changed the default cache directory location to follow the XDG base directory specification. - Changed compression algorithm from Deflate (zlib) to Zstandard, enabled by default. - CMake is now used instead of Autoconf for configuration and building. Additionally with version 4.4, an optional Redis backend for secondary storage on any server that supports the Redis protocol (by means of hiredis) has been added. In contrast to ccache versions < 4, that came bundled with zlib, now zstd can merely be downloaded at compile time. Thus using the version that comes with Buildroot is the more sensible option. The same applies to hiredis. Due to this, all of ccache's new dependencies (host-cmake, host-hiredis, host-pkgconf and host-zstd) have to be built before ccache. Co-authored-by: Anders F Björklund Signed-off-by: Dominik Michael Rauh --- Changes v1 -> v2: - Improve commmit message - Disable building host-ccache's docs Config.in | 8 ++++ ...cache_dir-through-CCACHE_DIR-env-var.patch | 42 +++++++++++++++++ package/ccache/ccache.hash | 6 +-- package/ccache/ccache.mk | 45 ++++++++++--------- package/cmake/cmake.mk | 9 +++- package/hiredis/hiredis.mk | 6 +++ package/pkg-generic.mk | 2 +- package/pkgconf/pkgconf.mk | 5 +++ package/zstd/zstd.mk | 5 +++ 9 files changed, 103 insertions(+), 25 deletions(-) create mode 100644 package/ccache/0001-Allow-to-patch-cache_dir-through-CCACHE_DIR-env-var.patch diff --git a/Config.in b/Config.in index 3c57c591a8..5afd06faa8 100644 --- a/Config.in +++ b/Config.in @@ -376,6 +376,14 @@ config BR2_CCACHE_USE_BASEDIR the section "COMPILING IN DIFFERENT DIRECTORIES" in the ccache manual for more information. +config BR2_CCACHE_REDIS_BACKEND + bool "Enable Redis storage backend" + default n + select BR2_PACKAGE_HIREDIS + help + Enable ccache to use the Redis protocol to communicate with a + server for the purpose of sharing cache results with others. + endif config BR2_ENABLE_DEBUG diff --git a/package/ccache/0001-Allow-to-patch-cache_dir-through-CCACHE_DIR-env-var.patch b/package/ccache/0001-Allow-to-patch-cache_dir-through-CCACHE_DIR-env-var.patch new file mode 100644 index 0000000000..dc2c86ddf8 --- /dev/null +++ b/package/ccache/0001-Allow-to-patch-cache_dir-through-CCACHE_DIR-env-var.patch @@ -0,0 +1,42 @@ +From 09231764c72bad287dd94c0dd67dff07f288b559 Mon Sep 17 00:00:00 2001 +From: Dominik Michael Rauh +Date: Thu, 4 Aug 2022 14:46:16 +0200 +Subject: [PATCH] Allow to patch "cache_dir" through "CCACHE_DIR" env var +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +Co-authored-by: Anders F Björklund +Signed-off-by: Dominik Michael Rauh +[Upstream: not applicable due to only being needed by Buildroot] +--- + src/Config.cpp | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +diff --git a/src/Config.cpp b/src/Config.cpp +index ee4cd9f7..8b40f960 100644 +--- a/src/Config.cpp ++++ b/src/Config.cpp +@@ -162,7 +162,6 @@ const std::unordered_map k_env_variable_table = { + {"DEBUG", "debug"}, + {"DEBUGDIR", "debug_dir"}, + {"DEPEND", "depend_mode"}, +- {"DIR", "cache_dir"}, + {"DIRECT", "direct_mode"}, + {"DISABLE", "disable"}, + {"EXTENSION", "cpp_extension"}, +@@ -613,6 +612,11 @@ Config::update_from_environment() + throw core::Error("CCACHE_{}{}: {}", negate ? "NO" : "", key, e.what()); + } + } ++ ++ const char* const env_ccache_dir = getenv("CCACHE_DIR"); ++ if (env_ccache_dir && *env_ccache_dir) { ++ set_item("cache_dir", env_ccache_dir, "DIR", false, "environment"); ++ } + } + + std::string +-- +2.30.2 + diff --git a/package/ccache/ccache.hash b/package/ccache/ccache.hash index 16cfad602c..980ac46d77 100644 --- a/package/ccache/ccache.hash +++ b/package/ccache/ccache.hash @@ -1,4 +1,4 @@ # sha256 computed locally -sha256 a02f4e8360dc6618bc494ca35b0ae21cea080f804a4898eab1ad3fcd108eb400 ccache-3.7.12.tar.xz -sha256 ec6b1a326ff93b2cc21df88a697ae470ff6927a55b8929e7e491b315e1563361 GPL-3.0.txt -sha256 4f63223bcdee822d55a6768cc8399ffe06401d7a03cfe79e1dc305f50dc22c59 LICENSE.adoc +sha256 e5d47bd3cbb504ada864124690e7c0d28ecb1f9aeac22a9976025aed9633f3d1 ccache-4.6.1.tar.xz +sha256 80b5112739a423dfac7bed1ca8a1df3cccda3d794425441997d4462b83db4dd5 GPL-3.0.txt +sha256 c5b8e7e1c59184fd9c0e823e6f019ec261d3ee8e2860dd5e3ad98842b730994a LICENSE.adoc diff --git a/package/ccache/ccache.mk b/package/ccache/ccache.mk index 97aa8b1c75..51b3c2ecba 100644 --- a/package/ccache/ccache.mk +++ b/package/ccache/ccache.mk @@ -4,27 +4,28 @@ # ################################################################################ -CCACHE_VERSION = 3.7.12 +CCACHE_VERSION = 4.6.1 CCACHE_SITE = https://github.com/ccache/ccache/releases/download/v$(CCACHE_VERSION) CCACHE_SOURCE = ccache-$(CCACHE_VERSION).tar.xz CCACHE_LICENSE = GPL-3.0+, others CCACHE_LICENSE_FILES = LICENSE.adoc GPL-3.0.txt +HOST_CCACHE_DEPENDENCIES = host-zstd -# Force ccache to use its internal zlib. The problem is that without -# this, ccache would link against the zlib of the build system, but we -# might build and install a different version of zlib in $(O)/host -# afterwards, which ccache will pick up. This might break if there is -# a version mismatch. A solution would be to add host-zlib has a -# dependency of ccache, but it would require tuning the zlib .mk file -# to use HOSTCC_NOCCACHE as the compiler. Instead, we take the easy -# path: tell ccache to use its internal copy of zlib, so that ccache -# has zero dependency besides the C library. -HOST_CCACHE_CONF_OPTS += --with-bundled-zlib +HOST_CCACHE_CONF_OPTS += \ + -DENABLE_DOCUMENTATION=OFF \ + -DENABLE_TESTING=OFF # We are ccache, so we can't use ccache -HOST_CCACHE_CONF_ENV = \ - CC="$(HOSTCC_NOCCACHE)" \ - CXX="$(HOSTCXX_NOCCACHE)" +HOST_CCACHE_CONF_OPTS += \ + -UCMAKE_C_COMPILER_LAUNCHER \ + -UCMAKE_CXX_COMPILER_LAUNCHER + +ifeq ($(BR2_PACKAGE_HIREDIS),y) +HOST_CCACHE_DEPENDENCIES += host-hiredis +HOST_CCACHE_CONF_OPTS += -DREDIS_STORAGE_BACKEND=ON +else +HOST_CCACHE_CONF_OPTS += -DREDIS_STORAGE_BACKEND=OFF +endif # Patch host-ccache as follows: # - Use BR_CACHE_DIR instead of CCACHE_DIR, because CCACHE_DIR @@ -33,13 +34,17 @@ HOST_CCACHE_CONF_ENV = \ # BR2_CCACHE_DIR. # - Change hard-coded last-ditch default to match path in .config, to avoid # the need to specify BR_CACHE_DIR when invoking ccache directly. -# CCache replaces "%s" with the home directory of the current user, -# So rewrite BR_CACHE_DIR to take that into consideration for SDK purpose -HOST_CCACHE_DEFAULT_CCACHE_DIR = $(patsubst $(HOME)/%,\%s/%,$(BR_CACHE_DIR)) +# CCache replaces "home_dir" with the home directory of the current user, +# so rewrite BR_CACHE_DIR to take that into consideration for SDK purpose +HOST_CCACHE_DEFAULT_CCACHE_DIR = $(patsubst \"$(HOME)/%,home_dir + \"/%,\"$(BR_CACHE_DIR)\") define HOST_CCACHE_PATCH_CONFIGURATION - sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/src/ccache.c - sed -i 's,"%s/.ccache","$(HOST_CCACHE_DEFAULT_CCACHE_DIR)",' $(@D)/src/conf.c + sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/src/Config.cpp + sed -i 's,home_dir + "/.ccache",$(HOST_CCACHE_DEFAULT_CCACHE_DIR),' $(@D)/src/Config.cpp + sed -i 's,getenv("XDG_CACHE_HOME"),nullptr,' $(@D)/src/Config.cpp + sed -i 's,home_dir + "/.cache/ccache",$(HOST_CCACHE_DEFAULT_CCACHE_DIR),' $(@D)/src/Config.cpp + sed -i 's,getenv("XDG_CONFIG_HOME"),nullptr,' $(@D)/src/Config.cpp + sed -i 's,home_dir + "/.config/ccache",$(HOST_CCACHE_DEFAULT_CCACHE_DIR),' $(@D)/src/Config.cpp endef HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION @@ -62,7 +67,7 @@ endef HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_DO_INITIAL_SETUP endif -$(eval $(host-autotools-package)) +$(eval $(host-cmake-package)) ifeq ($(BR2_CCACHE),y) ccache-stats: host-ccache diff --git a/package/cmake/cmake.mk b/package/cmake/cmake.mk index 053658fad6..eae12c1465 100644 --- a/package/cmake/cmake.mk +++ b/package/cmake/cmake.mk @@ -40,9 +40,16 @@ CMAKE_CONF_OPTS = \ HOST_CMAKE_CFLAGS = $(shell echo $(HOST_CFLAGS) | sed -r "s%$(HOST_CPPFLAGS)%%") HOST_CMAKE_CXXFLAGS = $(shell echo $(HOST_CXXFLAGS) | sed -r "s%$(HOST_CPPFLAGS)%%") +# we are built before ccache +HOST_CMAKE_CONFIGURE_OPTS = \ + $(HOST_CONFIGURE_OPTS) \ + CC="$(HOSTCC_NOCCACHE)" \ + GCC="$(HOSTCC_NOCCACHE)" \ + CXX="$(HOSTCXX_NOCCACHE)" + define HOST_CMAKE_CONFIGURE_CMDS (cd $(@D); \ - $(HOST_CONFIGURE_OPTS) \ + $(HOST_CMAKE_CONFIGURE_OPTS) \ CFLAGS="$(HOST_CMAKE_CFLAGS)" \ ./bootstrap --prefix=$(HOST_DIR) \ --parallel=$(PARALLEL_JOBS) -- \ diff --git a/package/hiredis/hiredis.mk b/package/hiredis/hiredis.mk index a571951fa3..2cb9991029 100644 --- a/package/hiredis/hiredis.mk +++ b/package/hiredis/hiredis.mk @@ -20,4 +20,10 @@ else HIREDIS_CONF_OPTS += -DENABLE_SSL=OFF endif +# we are built before ccache +HOST_HIREDIS_CONF_OPTS += \ + -UCMAKE_C_COMPILER_LAUNCHER \ + -UCMAKE_CXX_COMPILER_LAUNCHER + $(eval $(cmake-package)) +$(eval $(host-cmake-package)) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index b233b07548..cba94052a0 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -792,7 +792,7 @@ $(2)_EXTRACT_DEPENDENCIES += \ endif ifeq ($$(BR2_CCACHE),y) -ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache,$(1)),) +ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache host-cmake host-hiredis host-pkgconf host-zstd,$(1)),) $(2)_DEPENDENCIES += host-ccache endif endif diff --git a/package/pkgconf/pkgconf.mk b/package/pkgconf/pkgconf.mk index 5d65f69c10..6b187b0238 100644 --- a/package/pkgconf/pkgconf.mk +++ b/package/pkgconf/pkgconf.mk @@ -32,6 +32,11 @@ define HOST_PKGCONF_SHARED $(SED) 's,@STATIC@,,' $(HOST_DIR)/bin/pkg-config endef +# we are built before ccache +HOST_PKGCONF_CONF_ENV = \ + CC="$(HOSTCC_NOCCACHE)" \ + CXX="$(HOSTCXX_NOCCACHE)" + PKGCONF_POST_INSTALL_TARGET_HOOKS += PKGCONF_LINK_PKGCONFIG HOST_PKGCONF_POST_INSTALL_HOOKS += HOST_PKGCONF_INSTALL_WRAPPER diff --git a/package/zstd/zstd.mk b/package/zstd/zstd.mk index e8f6315222..edc4f1c624 100644 --- a/package/zstd/zstd.mk +++ b/package/zstd/zstd.mk @@ -97,6 +97,11 @@ endef HOST_ZSTD_OPTS += PREFIX=$(HOST_DIR) +# we are built before ccache +HOST_ZSTD_OPTS += \ + CC="$(HOSTCC_NOCCACHE)" \ + CXX="$(HOSTCXX_NOCCACHE)" + define HOST_ZSTD_BUILD_CMDS $(HOST_MAKE_ENV) $(HOST_CONFIGURE_OPTS) $(MAKE) $(HOST_ZSTD_OPTS) \ -C $(@D) zstd-release lib-release