| Message ID | 20171201205352.24287-9-thomas.petazzoni@free-electrons.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Per-package host/target directory support | expand |
Thomas, All, On 2017-12-01 21:53 +0100, Thomas Petazzoni spake thusly: > This moves the host-xz dependency handling from > DEPENDENCY_HOST_PREREQ to an extract dependency. > > To achieve that, check-host-xz.mk fills in the > BR2_XZ_HOST_DEPENDENCY with host-tar if building a host-tar is > needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it > matches the name BR2_CMAKE_HOST_DEPENDENCY already used in > check-host-cmake.mk. > > The BR2_XZ_HOST_DEPENDENCY is added to all packages, except: > > - host-xz, because we would otherwise depend on ourself. > > - host-tar, because xz itself is delivered as a tarball, so we need > to have host-xz depend on host-tar, and not host-tar depend on > host-xz > > - host-skeleton, because we need to have host-xz depend on > host-skeleton, and not the opposite. > > In addition, we modify xz.mk to explicitly build host-xz without > ccache. We generally took the approach of building host-ccache *after* > all the extractors have been built. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > --- > Changes since v2: > - New patch > --- > package/pkg-generic.mk | 4 ++++ > package/xz/xz.mk | 5 +++++ > support/dependencies/check-host-xzcat.mk | 2 +- > 3 files changed, 10 insertions(+), 1 deletion(-) > > diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk > index 86dd2561d4..dfd4719d29 100644 > --- a/package/pkg-generic.mk > +++ b/package/pkg-generic.mk > @@ -591,6 +591,10 @@ ifeq ($(filter host-tar host-skeleton,$(1)),) > $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY) > endif > > +ifeq ($(filter host-tar host-skeleton host-xz,$(1)),) > +$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY) > +endif So, all packages will now depend on this, even those that are not compressed with xz? Currently, filesystems that want to compress with xz all depend on host-xz unconditioanlly. Do we want to change that as well, so they only depend on it if needed? Regards, Yann E. MORIN. > # Eliminate duplicates in dependencies > $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES)) > $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES)) > diff --git a/package/xz/xz.mk b/package/xz/xz.mk > index e8116f24ae..bcdac13ee3 100644 > --- a/package/xz/xz.mk > +++ b/package/xz/xz.mk > @@ -18,5 +18,10 @@ else > XZ_CONF_OPTS = --disable-threads > endif > > +# we are built before ccache > +HOST_XZ_CONF_ENV = \ > + CC="$(HOSTCC_NOCCACHE)" \ > + CXX="$(HOSTCXX_NOCCACHE)" > + > $(eval $(autotools-package)) > $(eval $(host-autotools-package)) > diff --git a/support/dependencies/check-host-xzcat.mk b/support/dependencies/check-host-xzcat.mk > index 9be75c7311..e5d72f95dd 100644 > --- a/support/dependencies/check-host-xzcat.mk > +++ b/support/dependencies/check-host-xzcat.mk > @@ -2,7 +2,7 @@ > # If it is not present, build our own host-xzcat > > ifeq (,$(call suitable-host-package,xzcat,$(XZCAT))) > -DEPENDENCIES_HOST_PREREQ += host-xz > +BR2_XZCAT_HOST_DEPENDENCY = host-xz > EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .xz .lzma > XZCAT = $(HOST_DIR)/bin/xzcat > endif > -- > 2.13.6 >
Hello, On Sat, 2 Dec 2017 16:08:43 +0100, Yann E. MORIN wrote: > > +ifeq ($(filter host-tar host-skeleton host-xz,$(1)),) > > +$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY) > > +endif > > So, all packages will now depend on this, even those that are not > compressed with xz? Yes, if there is no xzcat available on the system. This is exactly what the current behavior is with DEPENDENCIES_HOST_PREREQ. My series does not intend to solve this problem, which is a separate one. I do believe however that by moving those dependencies (host-xz and host-lzip) as proper package dependencies, it will be easier to have follow-up patches that add the dependency only if lzip or xz are needed. But again, my series does not intend to fix that, it's a separate problem. > Currently, filesystems that want to compress with xz all depend on > host-xz unconditioanlly. Do we want to change that as well, so they only > depend on it if needed? I don't understand this part, because filesystems do not depend on host-xz unconditionally. They only depend on it if xz compression of the filesystem image is requested. From fs/common.mk: ifeq ($$(BR2_TARGET_ROOTFS_$(2)_XZ),y) ROOTFS_$(2)_DEPENDENCIES += host-xz ROOTFS_$(2)_COMPRESS_EXT = .xz ROOTFS_$(2)_COMPRESS_CMD = xz -9 -C crc32 -c endif Best regards, Thomas Petazzoni
On 2017-12-02 21:16 +0100, Thomas Petazzoni spake thusly: > Hello, > > On Sat, 2 Dec 2017 16:08:43 +0100, Yann E. MORIN wrote: > > > > +ifeq ($(filter host-tar host-skeleton host-xz,$(1)),) > > > +$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY) > > > +endif > > > > So, all packages will now depend on this, even those that are not > > compressed with xz? > > Yes, if there is no xzcat available on the system. This is exactly what > the current behavior is with DEPENDENCIES_HOST_PREREQ. My series does > not intend to solve this problem, which is a separate one. I somehow interpreted your patches in my head as "only build them when needed". ;-) > I do believe however that by moving those dependencies (host-xz and > host-lzip) as proper package dependencies, it will be easier to have > follow-up patches that add the dependency only if lzip or xz are > needed. But again, my series does not intend to fix that, it's a > separate problem. OK, agreed. > > Currently, filesystems that want to compress with xz all depend on > > host-xz unconditioanlly. Do we want to change that as well, so they only > > depend on it if needed? > > I don't understand this part, because filesystems do not depend on > host-xz unconditionally. They only depend on it if xz compression of > the filesystem image is requested. From fs/common.mk: > > ifeq ($$(BR2_TARGET_ROOTFS_$(2)_XZ),y) > ROOTFS_$(2)_DEPENDENCIES += host-xz > ROOTFS_$(2)_COMPRESS_EXT = .xz > ROOTFS_$(2)_COMPRESS_CMD = xz -9 -C crc32 -c > endif Well, first, let's assume that when the host has a suitable xzcat, it also has the xz command as well; I think it's not too far-fetched an assumption. Then, we have BR2_XZCAT_HOST_DEPENDENCY empty, because we do have a suitable xzcat, and thus a suitable xz. Yet, when the user requires their filesystem to be xz-compressed, we still forcefully depend on host-xz, even though we do have a suitable xz command. What I mean is that, probably, we will want to make it: ROOTFS_$(2)_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY) Regards, Yann E. MORIN. > Best regards, > > Thomas Petazzoni > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com
Hello, On Sun, 3 Dec 2017 09:32:56 +0100, Yann E. MORIN wrote: > > Yes, if there is no xzcat available on the system. This is exactly what > > the current behavior is with DEPENDENCIES_HOST_PREREQ. My series does > > not intend to solve this problem, which is a separate one. > > I somehow interpreted your patches in my head as "only build them when > needed". ;-) Nope, I'm not fixing that (separate) problem. I still want to fix this problem as well, but not as part of the per-package SDK stuff. > > > Currently, filesystems that want to compress with xz all depend on > > > host-xz unconditioanlly. Do we want to change that as well, so they only > > > depend on it if needed? > > > > I don't understand this part, because filesystems do not depend on > > host-xz unconditionally. They only depend on it if xz compression of > > the filesystem image is requested. From fs/common.mk: > > > > ifeq ($$(BR2_TARGET_ROOTFS_$(2)_XZ),y) > > ROOTFS_$(2)_DEPENDENCIES += host-xz > > ROOTFS_$(2)_COMPRESS_EXT = .xz > > ROOTFS_$(2)_COMPRESS_CMD = xz -9 -C crc32 -c > > endif > > Well, first, let's assume that when the host has a suitable xzcat, it > also has the xz command as well; I think it's not too far-fetched an > assumption. > > Then, we have BR2_XZCAT_HOST_DEPENDENCY empty, because we do have a > suitable xzcat, and thus a suitable xz. Yet, when the user requires > their filesystem to be xz-compressed, we still forcefully depend on > host-xz, even though we do have a suitable xz command. > > What I mean is that, probably, we will want to make it: > > ROOTFS_$(2)_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY) Yes, I thought about this too. But again, it's a separate issue: the filesystem code already depends on host-xz, regardless of whether xz is already available on the system. Thomas
Thomas, All, On 2017-12-03 10:29 +0100, Thomas Petazzoni spake thusly: [--SNIP--] > > > > Currently, filesystems that want to compress with xz all depend on > > > > host-xz unconditioanlly. Do we want to change that as well, so they only > > > > depend on it if needed? > > > > > > I don't understand this part, because filesystems do not depend on > > > host-xz unconditionally. They only depend on it if xz compression of > > > the filesystem image is requested. From fs/common.mk: > > > > > > ifeq ($$(BR2_TARGET_ROOTFS_$(2)_XZ),y) > > > ROOTFS_$(2)_DEPENDENCIES += host-xz > > > ROOTFS_$(2)_COMPRESS_EXT = .xz > > > ROOTFS_$(2)_COMPRESS_CMD = xz -9 -C crc32 -c > > > endif [--SNIP--] > > What I mean is that, probably, we will want to make it: > > > > ROOTFS_$(2)_DEPENDENCIES += $$(BR2_XZCAT_HOST_DEPENDENCY) > > Yes, I thought about this too. But again, it's a separate issue: the > filesystem code already depends on host-xz, regardless of whether xz is > already available on the system. Yes, sure. I was just merely saying that when you eventually make those dependencies really when-needed, you should also have a look at the filesystem stuff. ;-) Regards, Yann E. MORIN.
Thomas, All, On 2017-12-01 21:53 +0100, Thomas Petazzoni spake thusly: > This moves the host-xz dependency handling from > DEPENDENCY_HOST_PREREQ to an extract dependency. > > To achieve that, check-host-xz.mk fills in the > BR2_XZ_HOST_DEPENDENCY with host-tar if building a host-tar is > needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it > matches the name BR2_CMAKE_HOST_DEPENDENCY already used in > check-host-cmake.mk. > > The BR2_XZ_HOST_DEPENDENCY is added to all packages, except: > > - host-xz, because we would otherwise depend on ourself. > > - host-tar, because xz itself is delivered as a tarball, so we need > to have host-xz depend on host-tar, and not host-tar depend on > host-xz > > - host-skeleton, because we need to have host-xz depend on > host-skeleton, and not the opposite. > > In addition, we modify xz.mk to explicitly build host-xz without > ccache. We generally took the approach of building host-ccache *after* > all the extractors have been built. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> My prior questions have been answered, so: Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Regards, Yann E. MORIN. > --- > Changes since v2: > - New patch > --- > package/pkg-generic.mk | 4 ++++ > package/xz/xz.mk | 5 +++++ > support/dependencies/check-host-xzcat.mk | 2 +- > 3 files changed, 10 insertions(+), 1 deletion(-) > > diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk > index 86dd2561d4..dfd4719d29 100644 > --- a/package/pkg-generic.mk > +++ b/package/pkg-generic.mk > @@ -591,6 +591,10 @@ ifeq ($(filter host-tar host-skeleton,$(1)),) > $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY) > endif > > +ifeq ($(filter host-tar host-skeleton host-xz,$(1)),) > +$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY) > +endif > + > # Eliminate duplicates in dependencies > $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES)) > $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES)) > diff --git a/package/xz/xz.mk b/package/xz/xz.mk > index e8116f24ae..bcdac13ee3 100644 > --- a/package/xz/xz.mk > +++ b/package/xz/xz.mk > @@ -18,5 +18,10 @@ else > XZ_CONF_OPTS = --disable-threads > endif > > +# we are built before ccache > +HOST_XZ_CONF_ENV = \ > + CC="$(HOSTCC_NOCCACHE)" \ > + CXX="$(HOSTCXX_NOCCACHE)" > + > $(eval $(autotools-package)) > $(eval $(host-autotools-package)) > diff --git a/support/dependencies/check-host-xzcat.mk b/support/dependencies/check-host-xzcat.mk > index 9be75c7311..e5d72f95dd 100644 > --- a/support/dependencies/check-host-xzcat.mk > +++ b/support/dependencies/check-host-xzcat.mk > @@ -2,7 +2,7 @@ > # If it is not present, build our own host-xzcat > > ifeq (,$(call suitable-host-package,xzcat,$(XZCAT))) > -DEPENDENCIES_HOST_PREREQ += host-xz > +BR2_XZCAT_HOST_DEPENDENCY = host-xz > EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .xz .lzma > XZCAT = $(HOST_DIR)/bin/xzcat > endif > -- > 2.13.6 >
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 86dd2561d4..dfd4719d29 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -591,6 +591,10 @@ ifeq ($(filter host-tar host-skeleton,$(1)),) $(2)_EXTRACT_DEPENDENCIES += $(BR2_TAR_HOST_DEPENDENCY) endif +ifeq ($(filter host-tar host-skeleton host-xz,$(1)),) +$(2)_EXTRACT_DEPENDENCIES += $(BR2_XZCAT_HOST_DEPENDENCY) +endif + # Eliminate duplicates in dependencies $(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES)) $(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES)) diff --git a/package/xz/xz.mk b/package/xz/xz.mk index e8116f24ae..bcdac13ee3 100644 --- a/package/xz/xz.mk +++ b/package/xz/xz.mk @@ -18,5 +18,10 @@ else XZ_CONF_OPTS = --disable-threads endif +# we are built before ccache +HOST_XZ_CONF_ENV = \ + CC="$(HOSTCC_NOCCACHE)" \ + CXX="$(HOSTCXX_NOCCACHE)" + $(eval $(autotools-package)) $(eval $(host-autotools-package)) diff --git a/support/dependencies/check-host-xzcat.mk b/support/dependencies/check-host-xzcat.mk index 9be75c7311..e5d72f95dd 100644 --- a/support/dependencies/check-host-xzcat.mk +++ b/support/dependencies/check-host-xzcat.mk @@ -2,7 +2,7 @@ # If it is not present, build our own host-xzcat ifeq (,$(call suitable-host-package,xzcat,$(XZCAT))) -DEPENDENCIES_HOST_PREREQ += host-xz +BR2_XZCAT_HOST_DEPENDENCY = host-xz EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS += .xz .lzma XZCAT = $(HOST_DIR)/bin/xzcat endif
This moves the host-xz dependency handling from DEPENDENCY_HOST_PREREQ to an extract dependency. To achieve that, check-host-xz.mk fills in the BR2_XZ_HOST_DEPENDENCY with host-tar if building a host-tar is needed. The name BR2_XZ_HOST_DEPENDENCY has been chosen because it matches the name BR2_CMAKE_HOST_DEPENDENCY already used in check-host-cmake.mk. The BR2_XZ_HOST_DEPENDENCY is added to all packages, except: - host-xz, because we would otherwise depend on ourself. - host-tar, because xz itself is delivered as a tarball, so we need to have host-xz depend on host-tar, and not host-tar depend on host-xz - host-skeleton, because we need to have host-xz depend on host-skeleton, and not the opposite. In addition, we modify xz.mk to explicitly build host-xz without ccache. We generally took the approach of building host-ccache *after* all the extractors have been built. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- Changes since v2: - New patch --- package/pkg-generic.mk | 4 ++++ package/xz/xz.mk | 5 +++++ support/dependencies/check-host-xzcat.mk | 2 +- 3 files changed, 10 insertions(+), 1 deletion(-)