| Message ID | 6556713514fd3c751384c5ea655420dba5e8876a.1477843328.git.yann.morin.1998@free.fr |
|---|---|
| State | Accepted |
| Headers | show |
Yann, All, Le 30/10/2016 à 17:02, Yann E. MORIN a écrit : > This new waf-package infrastructure simplifies writing waf-based > packages. It can be used by our six current such packages, plus a > later-incoming one by Romain. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Romain Naour <romain.naour@openwide.fr> Reviewed-by: Romain Naour <romain.naour@gmail.com> [Tested with aubio package which will be added latter] Tested-by: Romain Naour <romain.naour@gmail.com> > --- > package/Makefile.in | 1 + > package/pkg-waf.mk | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 120 insertions(+) > create mode 100644 package/pkg-waf.mk > > diff --git a/package/Makefile.in b/package/Makefile.in > index 5d591e9..dd18d2b 100644 > --- a/package/Makefile.in > +++ b/package/Makefile.in > @@ -398,3 +398,4 @@ include package/pkg-generic.mk > include package/pkg-kconfig.mk > include package/pkg-rebar.mk > include package/pkg-kernel-module.mk > +include package/pkg-waf.mk > diff --git a/package/pkg-waf.mk b/package/pkg-waf.mk > new file mode 100644 > index 0000000..b36f47a > --- /dev/null > +++ b/package/pkg-waf.mk > @@ -0,0 +1,119 @@ > +################################################################################ > +# WAF package infrastructure > +# > +# This file implements an infrastructure that eases development of package > +# .mk files for WAF packages. It should be used for all packages that use > +# WAF as their build system. > +# > +# See the Buildroot documentation for details on the usage of this > +# infrastructure > +# > +# In terms of implementation, this WAF infrastructure requires the .mk file > +# to only specify metadata information about the package: name, version, > +# download URL, etc. > +# > +# We still allow the package .mk file to override what the different steps > +# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined, > +# it is used as the list of commands to perform to build the package, > +# instead of the default WAF behaviour. The package can also define some > +# post operation hooks. > +# > +################################################################################ > + > +################################################################################ > +# inner-waf-package -- defines how the configuration, compilation and > +# installation of a waf package should be done, implements a few hooks > +# to tune the build process for waf specifities and calls the generic > +# package infrastructure to generate the necessary make targets > +# > +# argument 1 is the lowercase package name > +# argument 2 is the uppercase package name, including a HOST_ prefix > +# for host packages > +# argument 3 is the uppercase package name, without the HOST_ prefix > +# for host packages > +# argument 4 is the type (target or host) > +################################################################################ > + > +define inner-waf-package > + > +# If the package does not have its own waf, use our own. > +ifeq ($$($(2)_BUNDLED_WAF),NO) > +# Dependency on host-python is done by host-waf > +$(2)_DEPENDENCIES += host-waf > +$(2)_WAF = $(HOST_DIR)/usr/bin/waf > +else > +# We need host-python to run the package's waf > +$(2)_DEPENDENCIES += host-python > +$(2)_WAF = ./waf > +endif > + > +ifndef $(2)_MAKE > + ifdef $(3)_MAKE > + $(2)_MAKE = $$($(3)_MAKE) > + else > + $(2)_MAKE ?= $$(MAKE) > + endif > +endif > + > +# > +# Configure step. Only define it if not already defined by the package > +# .mk file. > +# > +ifndef $(2)_CONFIGURE_CMDS > +define $(2)_CONFIGURE_CMDS > + cd $$(@D) && \ > + $$(TARGET_CONFIGURE_OPTS) \ > + $$($(2)_CONF_ENV) \ > + $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) configure \ > + --prefix=/usr \ > + --libdir=/usr/lib \ > + $$($(2)_CONF_OPTS) > +endef > +endif > + > +# > +# Build step. Only define it if not already defined by the package .mk > +# file. > +# > +ifndef $(2)_BUILD_CMDS > +define $(2)_BUILD_CMDS > + cd $$(@D) && \ > + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) build -j $$(PARALLEL_JOBS) > +endef > +endif > + > +# > +# Staging installation step. Only define it if not already defined by > +# the package .mk file. > +# > +ifndef $(2)_INSTALL_STAGING_CMDS > +define $(2)_INSTALL_STAGING_CMDS > + cd $$(@D) && \ > + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) \ > + install --destdir=$$(STAGING_DIR) > +endef > +endif > + > +# > +# Target installation step. Only define it if not already defined by > +# the package .mk file. > +# > +ifndef $(2)_INSTALL_TARGET_CMDS > +define $(2)_INSTALL_TARGET_CMDS > + cd $$(@D) && \ > + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) \ > + install --destdir=$$(TARGET_DIR) > +endef > +endif > + > +# Call the generic package infrastructure to generate the necessary > +# make targets > +$(call inner-generic-package,$(1),$(2),$(3),$(4)) > + > +endef > + > +################################################################################ > +# waf-package -- the target generator macro for WAF packages > +################################################################################ > + > +waf-package = $(call inner-waf-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target) >
Hello, On Sun, 30 Oct 2016 17:02:13 +0100, Yann E. MORIN wrote: > This new waf-package infrastructure simplifies writing waf-based > packages. It can be used by our six current such packages, plus a > later-incoming one by Romain. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Romain Naour <romain.naour@openwide.fr> I am generally happy with the whole series, I just have a few comments on this specific patch, which are mainly questions. We discussed them on IRC, but I'd like to raise the questions here as well, in case people have some opinions. If nobody gives some feedback, then I'm going to apply the series. > +# If the package does not have its own waf, use our own. > +ifeq ($$($(2)_BUNDLED_WAF),NO) Where is the default value of this variable defined? It seems it's not defined anywhere, while it should be set to "YES". However, what I dislike a bit is the slightly "negative" logic of this variable. By default, we assume packages have their bundled version of waf, so this variable is by default assumed to be set to "YES" (even though in fact it's empty). Only packages that do *NOT* have a bundled version of waf can set <pkg>_BUNDLED_WAF = NO, to tell the infrastructure to add a dependency on host-waf. To me it feels a bit weird to: 1. Have a boolean that defaults to YES and that can be overridden to NO. I generally expects the opposite. 2. Have a boolean that when set to "NO" actually asks the infrastructure to do more things. I generally expects the opposite. So, ideally, I'd like to invert this variable, but I can't really find a good name for it. I propose <pkg>_NEEDS_WAF on IRC, but I'm not convinced because all packages need waf, either bundled or external. <pkg>_NEEDS_EXTERNAL_WAF ? Anyone has some other proposal ? > +# Dependency on host-python is done by host-waf > +$(2)_DEPENDENCIES += host-waf > +$(2)_WAF = $(HOST_DIR)/usr/bin/waf > +else > +# We need host-python to run the package's waf > +$(2)_DEPENDENCIES += host-python > +$(2)_WAF = ./waf > +endif Here another point where I'm hesitating. host-waf doesn't really need to build-depend on host-python. It's *running* host-waf that requires host-python. So I'm wondering if we shouldn't: 1. Remove the host-python dependency from host-waf 2. In the first case above (no bundled waf), add to the package a dependency on host-waf *and* host-python. On IRC, you suggested that packages that depend on host-waf should not have to know host-python is needed to run it, which is fair point. So I'm opening the question to others. > +ifndef $(2)_MAKE > + ifdef $(3)_MAKE > + $(2)_MAKE = $$($(3)_MAKE) > + else > + $(2)_MAKE ?= $$(MAKE) > + endif > +endif Is $(2)_MAKE used anywhere in this infrastructure? Thomas
Thomas, All, On 2016-11-17 00:02 +0100, Thomas Petazzoni spake thusly: > On Sun, 30 Oct 2016 17:02:13 +0100, Yann E. MORIN wrote: > > This new waf-package infrastructure simplifies writing waf-based > > packages. It can be used by our six current such packages, plus a > > later-incoming one by Romain. > > > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Cc: Romain Naour <romain.naour@openwide.fr> > > I am generally happy with the whole series, I just have a few comments > on this specific patch, which are mainly questions. We discussed them > on IRC, but I'd like to raise the questions here as well, in case > people have some opinions. My answers below... > If nobody gives some feedback, then I'm going to apply the series. :-) > > +# If the package does not have its own waf, use our own. > > +ifeq ($$($(2)_BUNDLED_WAF),NO) > > Where is the default value of this variable defined? It seems it's not > defined anywhere, while it should be set to "YES". We don't really need to set a default, because we're not testing the default. We're only testing if it was set by the package. > However, what I dislike a bit is the slightly "negative" logic of this > variable. By default, we assume packages have their bundled version of > waf, so this variable is by default assumed to be set to "YES" (even > though in fact it's empty). > > Only packages that do *NOT* have a bundled version of waf can set > <pkg>_BUNDLED_WAF = NO, to tell the infrastructure to add a dependency > on host-waf. > > To me it feels a bit weird to: > > 1. Have a boolean that defaults to YES and that can be overridden to > NO. I generally expects the opposite. > > 2. Have a boolean that when set to "NO" actually asks the > infrastructure to do more things. I generally expects the opposite. > > So, ideally, I'd like to invert this variable, but I can't really find > a good name for it. > > I propose <pkg>_NEEDS_WAF on IRC, but I'm not convinced because all > packages need waf, either bundled or external. > <pkg>_NEEDS_EXTERNAL_WAF ? That last one is OK for me. > Anyone has some other proposal ? > > > +# Dependency on host-python is done by host-waf > > +$(2)_DEPENDENCIES += host-waf > > +$(2)_WAF = $(HOST_DIR)/usr/bin/waf > > +else > > +# We need host-python to run the package's waf > > +$(2)_DEPENDENCIES += host-python > > +$(2)_WAF = ./waf > > +endif > > Here another point where I'm hesitating. host-waf doesn't really need > to build-depend on host-python. It's *running* host-waf that requires > host-python. So I'm wondering if we shouldn't: > > 1. Remove the host-python dependency from host-waf > > 2. In the first case above (no bundled waf), add to the package a > dependency on host-waf *and* host-python. > > On IRC, you suggested that packages that depend on host-waf should not > have to know host-python is needed to run it, which is fair point. Yes, I followed the same as we do with AUTORECONF: we automatically add host-automake, host-autoconf, host-libtool and, if needed, host-gettext. Packages need not add them as they are dependencies of their build systems (autotools), not their own dependencies. The only exceptions being packages that do need libtool for themselves (for libltdl). Same goes for host-python: it is a dependency of the build system, not of the package, so it should go in the infra. > So I'm opening the question to others. > > > +ifndef $(2)_MAKE > > + ifdef $(3)_MAKE > > + $(2)_MAKE = $$($(3)_MAKE) > > + else > > + $(2)_MAKE ?= $$(MAKE) > > + endif > > +endif > > Is $(2)_MAKE used anywhere in this infrastructure? Not used. I initially added it because I did contemplate converting samba4 as well, but it does not make sense to have such support for a single package, so I backed off. I just forgot to remove this hunk. > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com
On 17-11-16 00:02, Thomas Petazzoni wrote: > Hello, > > On Sun, 30 Oct 2016 17:02:13 +0100, Yann E. MORIN wrote: [snip] >> +# If the package does not have its own waf, use our own. >> +ifeq ($$($(2)_BUNDLED_WAF),NO) > > Where is the default value of this variable defined? It seems it's not > defined anywhere, while it should be set to "YES". > > However, what I dislike a bit is the slightly "negative" logic of this > variable. By default, we assume packages have their bundled version of > waf, so this variable is by default assumed to be set to "YES" (even > though in fact it's empty). > > Only packages that do *NOT* have a bundled version of waf can set > <pkg>_BUNDLED_WAF = NO, to tell the infrastructure to add a dependency > on host-waf. > > To me it feels a bit weird to: > > 1. Have a boolean that defaults to YES and that can be overridden to > NO. I generally expects the opposite. > > 2. Have a boolean that when set to "NO" actually asks the > infrastructure to do more things. I generally expects the opposite. We have a bunch of these already: _INSTALL_TARGET, _LIBTOOL_PATCH, _REDISTRIBUTE, .... Actually, most of them default to YES and are overridden as NO. It's just that the most popular one (_INSTALL_STAGING) defaults to NO so you'd think that that one is the norm. The only unusual thing here is that the condition checks for NO, while usually the condition checks for YES. But turning it around we make it an ifneq which is also a negative. So I'd keep it like this. > So, ideally, I'd like to invert this variable, but I can't really find > a good name for it. > > I propose <pkg>_NEEDS_WAF on IRC, but I'm not convinced because all > packages need waf, either bundled or external. > <pkg>_NEEDS_EXTERNAL_WAF ? > > Anyone has some other proposal ? > >> +# Dependency on host-python is done by host-waf >> +$(2)_DEPENDENCIES += host-waf >> +$(2)_WAF = $(HOST_DIR)/usr/bin/waf >> +else >> +# We need host-python to run the package's waf >> +$(2)_DEPENDENCIES += host-python >> +$(2)_WAF = ./waf >> +endif > > Here another point where I'm hesitating. host-waf doesn't really need > to build-depend on host-python. It's *running* host-waf that requires > host-python. So I'm wondering if we shouldn't: > > 1. Remove the host-python dependency from host-waf > > 2. In the first case above (no bundled waf), add to the package a > dependency on host-waf *and* host-python. > > On IRC, you suggested that packages that depend on host-waf should not > have to know host-python is needed to run it, which is fair point. Well, it makes sense to me that a package infra does something like that, but less so that the waf package itself would do it. And it's just simpler if the dependency is only in the infra. However, I wonder if we need this dependency at all. We already require a python (although admittedly we don't check the version). The few packages that don't support python3 can add the dependency themselves. Building host-python really eats into the build time... Regards, Arnout > > So I'm opening the question to others. > >> +ifndef $(2)_MAKE >> + ifdef $(3)_MAKE >> + $(2)_MAKE = $$($(3)_MAKE) >> + else >> + $(2)_MAKE ?= $$(MAKE) >> + endif >> +endif > > Is $(2)_MAKE used anywhere in this infrastructure? > > Thomas >
On 17-11-16 21:44, Yann E. MORIN wrote: > Thomas, All, > > On 2016-11-17 00:02 +0100, Thomas Petazzoni spake thusly: >> On Sun, 30 Oct 2016 17:02:13 +0100, Yann E. MORIN wrote: [snip] >>> +# If the package does not have its own waf, use our own. >>> +ifeq ($$($(2)_BUNDLED_WAF),NO) >> >> Where is the default value of this variable defined? It seems it's not >> defined anywhere, while it should be set to "YES". > > We don't really need to set a default, because we're not testing the > default. We're only testing if it was set by the package. We do set a default for all other YES/NO variables. Regards, Arnout [snip]
Hello, On Fri, 18 Nov 2016 00:52:58 +0100, Arnout Vandecappelle wrote: > Well, it makes sense to me that a package infra does something like that, but > less so that the waf package itself would do it. And it's just simpler if the > dependency is only in the infra. > > However, I wonder if we need this dependency at all. We already require a > python (although admittedly we don't check the version). The few packages that > don't support python3 can add the dependency themselves. Building host-python > really eats into the build time... Is waf python2/python3 compliant? Thomas
Hi all, On Fri, Nov 18, 2016 at 9:45 AM, Thomas Petazzoni < thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Fri, 18 Nov 2016 00:52:58 +0100, Arnout Vandecappelle wrote: > > > Well, it makes sense to me that a package infra does something like > that, but > > less so that the waf package itself would do it. And it's just simpler > if the > > dependency is only in the infra. > > > > However, I wonder if we need this dependency at all. We already require > a > > python (although admittedly we don't check the version). The few > packages that > > don't support python3 can add the dependency themselves. Building > host-python > > really eats into the build time... > > Is waf python2/python3 compliant? > Yes, see here : https://waf.io/book/#_how_to_run_waf AFAIK, there is an internal wrapper to make code compatible with both versions (https://github.com/waf-project/waf/blob/master/waflib/fixpy2.py). > > Thomas > -- > Thomas Petazzoni, CTO, Free Electrons > Embedded Linux and Kernel engineering > http://free-electrons.com > _______________________________________________ > buildroot mailing list > buildroot@busybox.net > http://lists.busybox.net/mailman/listinfo/buildroot > Kind regards, Lionel
Hello, On Sun, 30 Oct 2016 17:02:13 +0100, Yann E. MORIN wrote: > +# If the package does not have its own waf, use our own. > +ifeq ($$($(2)_BUNDLED_WAF),NO) I've renamed this variable to _NEEDS_EXTERNAL_WAF, which inverts its logic. It's explicitly defined to NO by default, and when set to YES, a dependency on host-waf is needed. > +# Dependency on host-python is done by host-waf > +$(2)_DEPENDENCIES += host-waf > +$(2)_WAF = $(HOST_DIR)/usr/bin/waf > +else > +# We need host-python to run the package's waf > +$(2)_DEPENDENCIES += host-python This line has been moved out of the condition, since it's now needed for both the "bundled waf" and "external waf" cases. > +ifndef $(2)_MAKE > + ifdef $(3)_MAKE > + $(2)_MAKE = $$($(3)_MAKE) > + else > + $(2)_MAKE ?= $$(MAKE) > + endif > +endif I've dropped this as it wasn't needed. Thanks! Thomas
diff --git a/package/Makefile.in b/package/Makefile.in index 5d591e9..dd18d2b 100644 --- a/package/Makefile.in +++ b/package/Makefile.in @@ -398,3 +398,4 @@ include package/pkg-generic.mk include package/pkg-kconfig.mk include package/pkg-rebar.mk include package/pkg-kernel-module.mk +include package/pkg-waf.mk diff --git a/package/pkg-waf.mk b/package/pkg-waf.mk new file mode 100644 index 0000000..b36f47a --- /dev/null +++ b/package/pkg-waf.mk @@ -0,0 +1,119 @@ +################################################################################ +# WAF package infrastructure +# +# This file implements an infrastructure that eases development of package +# .mk files for WAF packages. It should be used for all packages that use +# WAF as their build system. +# +# See the Buildroot documentation for details on the usage of this +# infrastructure +# +# In terms of implementation, this WAF infrastructure requires the .mk file +# to only specify metadata information about the package: name, version, +# download URL, etc. +# +# We still allow the package .mk file to override what the different steps +# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined, +# it is used as the list of commands to perform to build the package, +# instead of the default WAF behaviour. The package can also define some +# post operation hooks. +# +################################################################################ + +################################################################################ +# inner-waf-package -- defines how the configuration, compilation and +# installation of a waf package should be done, implements a few hooks +# to tune the build process for waf specifities and calls the generic +# package infrastructure to generate the necessary make targets +# +# argument 1 is the lowercase package name +# argument 2 is the uppercase package name, including a HOST_ prefix +# for host packages +# argument 3 is the uppercase package name, without the HOST_ prefix +# for host packages +# argument 4 is the type (target or host) +################################################################################ + +define inner-waf-package + +# If the package does not have its own waf, use our own. +ifeq ($$($(2)_BUNDLED_WAF),NO) +# Dependency on host-python is done by host-waf +$(2)_DEPENDENCIES += host-waf +$(2)_WAF = $(HOST_DIR)/usr/bin/waf +else +# We need host-python to run the package's waf +$(2)_DEPENDENCIES += host-python +$(2)_WAF = ./waf +endif + +ifndef $(2)_MAKE + ifdef $(3)_MAKE + $(2)_MAKE = $$($(3)_MAKE) + else + $(2)_MAKE ?= $$(MAKE) + endif +endif + +# +# Configure step. Only define it if not already defined by the package +# .mk file. +# +ifndef $(2)_CONFIGURE_CMDS +define $(2)_CONFIGURE_CMDS + cd $$(@D) && \ + $$(TARGET_CONFIGURE_OPTS) \ + $$($(2)_CONF_ENV) \ + $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) configure \ + --prefix=/usr \ + --libdir=/usr/lib \ + $$($(2)_CONF_OPTS) +endef +endif + +# +# Build step. Only define it if not already defined by the package .mk +# file. +# +ifndef $(2)_BUILD_CMDS +define $(2)_BUILD_CMDS + cd $$(@D) && \ + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) build -j $$(PARALLEL_JOBS) +endef +endif + +# +# Staging installation step. Only define it if not already defined by +# the package .mk file. +# +ifndef $(2)_INSTALL_STAGING_CMDS +define $(2)_INSTALL_STAGING_CMDS + cd $$(@D) && \ + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) \ + install --destdir=$$(STAGING_DIR) +endef +endif + +# +# Target installation step. Only define it if not already defined by +# the package .mk file. +# +ifndef $(2)_INSTALL_TARGET_CMDS +define $(2)_INSTALL_TARGET_CMDS + cd $$(@D) && \ + $$(TARGET_MAKE_ENV) $$(HOST_DIR)/usr/bin/python2 $$($(2)_WAF) \ + install --destdir=$$(TARGET_DIR) +endef +endif + +# Call the generic package infrastructure to generate the necessary +# make targets +$(call inner-generic-package,$(1),$(2),$(3),$(4)) + +endef + +################################################################################ +# waf-package -- the target generator macro for WAF packages +################################################################################ + +waf-package = $(call inner-waf-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
This new waf-package infrastructure simplifies writing waf-based packages. It can be used by our six current such packages, plus a later-incoming one by Romain. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Romain Naour <romain.naour@openwide.fr> --- package/Makefile.in | 1 + package/pkg-waf.mk | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 120 insertions(+) create mode 100644 package/pkg-waf.mk