| Message ID | 20191016111927.14208-3-nolange79@gmail.com |
|---|---|
| State | Rejected |
| Headers | show |
| Series | [1/4] package/pkg-meson: move crosscompilation support out of package | expand |
Hello Norbert On Wed, 16 Oct 2019 13:19:25 +0200 Norbert Lange <nolange79@gmail.com> wrote: > Automatically check for an available meson tool, > and use it as long as the version is fitting. > > Using a system provided meson is not an official supported > option, hence this is currently disabled. > use of the system provided meson can be enforced with: > > make ... BR2_MESON_VERSION_MIN=0 > > Signed-off-by: Norbert Lange <nolange79@gmail.com> Thanks for this, however I don't think we will ever merge this until a system-provided meson can effectively be used. What about working on merging 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream Meson, in this form or another form ? This would definitely pave the way to using the system-provided Meson. Another (more useful ?) thing to look at: is it possible to use the system-provided Python for Meson and Ninja, when python3 is provided by the system ? I think Meson and Ninja by themselves are not long at all to build, and it would be a much more useful direction for this patch series. Thanks! Thomas
Am Mi., 16. Okt. 2019 um 15:48 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Hello Norbert > > On Wed, 16 Oct 2019 13:19:25 +0200 > Norbert Lange <nolange79@gmail.com> wrote: > > > Automatically check for an available meson tool, > > and use it as long as the version is fitting. > > > > Using a system provided meson is not an official supported > > option, hence this is currently disabled. > > use of the system provided meson can be enforced with: > > > > make ... BR2_MESON_VERSION_MIN=0 > > > > Signed-off-by: Norbert Lange <nolange79@gmail.com> > > Thanks for this, however I don't think we will ever merge this until a > system-provided meson can effectively be used. Well I do effectively use it, meson and its dependencies multiply my build-time, builds systemd and my libfuse3 package fine. Patch #1 is a freestanding improvement, I hope there arent issues with merging at least that one? > What about working on merging > 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream > Meson, in this form or another form ? This would definitely pave the > way to using the system-provided Meson. I'm in no way involved with meson, AFAIK this was proposed for upstream already? > Another (more useful ?) thing to look at: is it possible to use the > system-provided Python for Meson and Ninja, when python3 is provided by > the system ? I think Meson and Ninja by themselves are not long at all > to build, and it would be a much more useful direction for this patch > series. Define "useful". The patches work correctly for me (tm), and I dont use more than 1000 included packages that are useless for me (tm). Even if meson is not ready (again: it is for me), then improving buildroot ahead of it causes no harm? If I were cynical, Id say I prefer spending time porting packages to CMake rather than trying to fix meson. regards Norbert.
Thomas, Norbert, All, On 2019-10-16 15:48 +0200, Thomas Petazzoni spake thusly: > On Wed, 16 Oct 2019 13:19:25 +0200 > Norbert Lange <nolange79@gmail.com> wrote: > > Automatically check for an available meson tool, > > and use it as long as the version is fitting. > > Thanks for this, however I don't think we will ever merge this until a > system-provided meson can effectively be used. I do agreee with that. However, I guess that what Norbert is trying to do, is to build their own meson that they patched with our patch, thus achieving the "system-provided meson is capable" status. But if we're going that way for meson, why not do the same for all the other host tools we do build? That would be totally unfeasible. So, I am still not convinced by this feature, until we can at least detect that a system-provided meson is capable, and that we can decide by actually asking that meson (e.g. by checking the version, or dumping its help for options, or whatevs...). > What about working on merging > 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream > Meson, in this form or another form ? This would definitely pave the > way to using the system-provided Meson. > > Another (more useful ?) thing to look at: is it possible to use the > system-provided Python for Meson and Ninja, when python3 is provided by > the system ? I think Meson and Ninja by themselves are not long at all > to build, and it would be a much more useful direction for this patch > series. But then, it should be done very carefully: if the hot-python3 must otherwise be built (because something else forcibly depends on it), then our host-meson must also be using that host-python3 too, otherwise, we could end up with a host-meson built with the system-provided python3 but run with our host-python3. Regards, Yann E. MORIN. > Thanks! > > Thomas > -- > Thomas Petazzoni, CTO, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com
Am Mi., 16. Okt. 2019 um 18:47 Uhr schrieb Yann E. MORIN <yann.morin.1998@free.fr>: > > Thomas, Norbert, All, > > On 2019-10-16 15:48 +0200, Thomas Petazzoni spake thusly: > > On Wed, 16 Oct 2019 13:19:25 +0200 > > Norbert Lange <nolange79@gmail.com> wrote: > > > Automatically check for an available meson tool, > > > and use it as long as the version is fitting. > > > > Thanks for this, however I don't think we will ever merge this until a > > system-provided meson can effectively be used. > > I do agreee with that. > > However, I guess that what Norbert is trying to do, is to build their > own meson that they patched with our patch, thus achieving the > "system-provided meson is capable" status. Actually I use meson from debian testing. > But if we're going that way for meson, why not do the same for all the > other host tools we do build? That would be totally unfeasible. For those complicated or long to build its useful. You already do that for CMake, several compressors and a few others. > So, I am still not convinced by this feature, until we can at least > detect that a system-provided meson is capable, and that we can decide > by actually asking that meson (e.g. by checking the version, or dumping > its help for options, or whatevs...). Again, patch #1 is a freestanding cleanup, hope atleast that is clear for merging? And the patches never automatically use meson/ninja, but are helpful if a user would decide to cut buildtime (in a scenario where the system tools suffice). What about if you just want to build a toolchain? I can use this painless from CMake, as the toolchain.cmake gets installed even if I dont build CMake or have CMake installed. I would want the same for meson. > > What about working on merging > > 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream > > Meson, in this form or another form ? This would definitely pave the > > way to using the system-provided Meson. > > > > Another (more useful ?) thing to look at: is it possible to use the > > system-provided Python for Meson and Ninja, when python3 is provided by > > the system ? I think Meson and Ninja by themselves are not long at all > > to build, and it would be a much more useful direction for this patch > > series. > > But then, it should be done very carefully: if the hot-python3 must > otherwise be built (because something else forcibly depends on it), then > our host-meson must also be using that host-python3 too, otherwise, we > could end up with a host-meson built with the system-provided python3 > but run with our host-python3. Hmm, dont get why using system python is anything better than using cmake, ninja or (a fixed/usable) meson from the system. regards, Norbert
Hello, On Wed, 16 Oct 2019 18:28:02 +0200 Norbert Lange <nolange79@gmail.com> wrote: > > Thanks for this, however I don't think we will ever merge this until a > > system-provided meson can effectively be used. > > Well I do effectively use it, meson and its dependencies multiply my build-time, > builds systemd and my libfuse3 package fine. > > Patch #1 is a freestanding improvement, I hope there arent issues with merging > at least that one? Yes, of course PATCH 1/4 is totally independent from this discussion, and can/will be considered separately. I looked at it briefly, I haven't made up my mind yet, but I agree it is completely separate. > > What about working on merging > > 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream > > Meson, in this form or another form ? This would definitely pave the > > way to using the system-provided Meson. > > I'm in no way involved with meson, AFAIK this was proposed for upstream already? You don't have to be involved with Meson to work on it. Many of us regularly contribute patches to upstream projects without being involved in any way with those projects. > > Another (more useful ?) thing to look at: is it possible to use the > > system-provided Python for Meson and Ninja, when python3 is provided by > > the system ? I think Meson and Ninja by themselves are not long at all > > to build, and it would be a much more useful direction for this patch > > series. > > Define "useful". The patches work correctly for me (tm), and I dont > use more than 1000 included packages that are useless for me (tm). > Even if meson is not ready (again: it is for me), then improving > buildroot ahead of it causes no harm? Because today meson requires a Buildroot patch, and in fact what really adds to the build time is not building host-meson or host-ninja, but really the fact that it requires the build of host-python3. And Python 3.x is now widely available on most build machines. So the idea would be, instead of relying on system-provided meson/ninja, to still build our own meson/ninja, but really the system-provided python3. > If I were cynical, Id say I prefer spending time porting packages to > CMake rather than trying to fix meson. I don't think the point of this discussion is to decide whether cmake, meson or autotools, or whatever is the best build system. Thomas
Norbert, Thomas, All, On 2019-10-16 21:01 +0200, Thomas Petazzoni spake thusly: > On Wed, 16 Oct 2019 18:28:02 +0200 > Norbert Lange <nolange79@gmail.com> wrote: > > Well I do effectively use it, meson and its dependencies multiply my build-time, > > builds systemd and my libfuse3 package fine. So this had me wonder: why do you report that it works for you (and I am ready to believe you), although we do have a patch that you are not using? Put in other words, I was wondering that, maybe, your report was an indication that our patch was in fact no longer needed, in which case we could drop it and would reconsider your proposal. So I had to dig a bit on what this patch was doing (it is a relatively simple patch), and why it was needed. It turns out that the prupose of that patch is to ensure that a host-meson-package that also uses a library installed by one of our other host packages, keeps an RPATH to find that library (instead of not finding it, or of finding the system-provided one). This is nicely laid out in the git log for our patch: commit 5c939246a802c0ad9704dac1505105037542a1d3 Author: Eric Le Bihan <eric.le.bihan.dev@free.fr> Date: Sat Oct 27 19:59:22 2018 +0200 meson: re-add patch for skipping RPATH fixing The patch to skip RPATH fixing performed by Meson was removed in commit a03f46ca6e9f43028003aedc92f1a1204ae7480f, as the script support/scripts/check-host-rpath was not complaining anymore. But without it, the problem still occurs for host packages [1]. So, restore this patch to fix build of host packages with Meson. [1] http://lists.busybox.net/pipermail/buildroot/2018-October/232956.html Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> So yes, ther are situations where using a system-provided meson _may_ work, but the generic case is that our patch is still required to ensure that the situatiuon dexribed above still works. Note that the only package in that situation in upstream Buildroot, is libglib2, so if you don't have that, then you do not have the problem this patch is solving. So, with those explanations, I hope it is now clearer why we can't rely on the system-provided meson. Anyway, your proposal was interesting, if at least because it really made us reconsider the usefulness of our patch. > > > Another (more useful ?) thing to look at: is it possible to use the > > > system-provided Python for Meson and Ninja, when python3 is provided by > > > the system ? I think Meson and Ninja by themselves are not long at all > > > to build, and it would be a much more useful direction for this patch > > > series. > > Define "useful". The patches work correctly for me (tm), and I dont > > use more than 1000 included packages that are useless for me (tm). > > Even if meson is not ready (again: it is for me), then improving > > buildroot ahead of it causes no harm? The case "works for me" is not a valid reason, because as demonstrated above, there are cases where it won't work. Regards, Yann E. MORIN.
Am Mi., 16. Okt. 2019 um 21:01 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Hello, > > On Wed, 16 Oct 2019 18:28:02 +0200 > Norbert Lange <nolange79@gmail.com> wrote: > > > > Thanks for this, however I don't think we will ever merge this until a > > > system-provided meson can effectively be used. > > > > Well I do effectively use it, meson and its dependencies multiply my build-time, > > builds systemd and my libfuse3 package fine. > > > > Patch #1 is a freestanding improvement, I hope there arent issues with merging > > at least that one? > > Yes, of course PATCH 1/4 is totally independent from this discussion, > and can/will be considered separately. I looked at it briefly, I > haven't made up my mind yet, but I agree it is completely separate. > > > > What about working on merging > > > 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch to upstream > > > Meson, in this form or another form ? This would definitely pave the > > > way to using the system-provided Meson. > > > > I'm in no way involved with meson, AFAIK this was proposed for upstream already? > > You don't have to be involved with Meson to work on it. Many of us > regularly contribute patches to upstream projects without being > involved in any way with those projects. I don't even know what the issue is, as I use the system meson without troubles, I can however do the other part: prepare BR for using system meson (in the hope that those issues will be sorted out sometime). > > > Another (more useful ?) thing to look at: is it possible to use the > > > system-provided Python for Meson and Ninja, when python3 is provided by > > > the system ? I think Meson and Ninja by themselves are not long at all > > > to build, and it would be a much more useful direction for this patch > > > series. > > > > Define "useful". The patches work correctly for me (tm), and I dont > > use more than 1000 included packages that are useless for me (tm). > > Even if meson is not ready (again: it is for me), then improving > > buildroot ahead of it causes no harm? > > Because today meson requires a Buildroot patch, and in fact what really > adds to the build time is not building host-meson or host-ninja, but > really the fact that it requires the build of host-python3. And Python > 3.x is now widely available on most build machines. So the idea would > be, instead of relying on system-provided meson/ninja, to still build > our own meson/ninja, but really the system-provided python3. I get that you prefer to build meson as only supported option, but why exactly is there a big issue of preparing a feature (that needs some arcane Variable overrides to enable). I build my meson packages without any trouble (and got a email to rebase my patch, so I don't think I am the only one). With ninja its even worse as it only needs python for a bootstrap *during build*, ninja is actually stable and broadly available, means any newer version will work. Of course meson is currently the only realistic reason for pulling in host-ninja. All I am saying is that the decision to what tools can be picked from the system seems arbitrary to me. Nowadays you would just use a container with the tools prepared (could even use a patched meson). > > If I were cynical, Id say I prefer spending time porting packages to > > CMake rather than trying to fix meson. > > I don't think the point of this discussion is to decide whether cmake, > meson or autotools, or whatever is the best build system. Nope, but only one of these needs to be rebuilt every time in buildroot. Regards, Norbert
Am Mi., 16. Okt. 2019 um 22:16 Uhr schrieb Yann E. MORIN <yann.morin.1998@free.fr>: > > Norbert, Thomas, All, > > On 2019-10-16 21:01 +0200, Thomas Petazzoni spake thusly: > > On Wed, 16 Oct 2019 18:28:02 +0200 > > Norbert Lange <nolange79@gmail.com> wrote: > > > Well I do effectively use it, meson and its dependencies multiply my build-time, > > > builds systemd and my libfuse3 package fine. > > So this had me wonder: why do you report that it works for you (and I am > ready to believe you), although we do have a patch that you are not > using? > > Put in other words, I was wondering that, maybe, your report was an > indication that our patch was in fact no longer needed, in which case we > could drop it and would reconsider your proposal. > > So I had to dig a bit on what this patch was doing (it is a relatively > simple patch), and why it was needed. > > It turns out that the prupose of that patch is to ensure that a > host-meson-package that also uses a library installed by one of our > other host packages, keeps an RPATH to find that library (instead of not > finding it, or of finding the system-provided one). > > This is nicely laid out in the git log for our patch: > > commit 5c939246a802c0ad9704dac1505105037542a1d3 > Author: Eric Le Bihan <eric.le.bihan.dev@free.fr> > Date: Sat Oct 27 19:59:22 2018 +0200 > > meson: re-add patch for skipping RPATH fixing > > The patch to skip RPATH fixing performed by Meson was removed in commit > a03f46ca6e9f43028003aedc92f1a1204ae7480f, as the script > support/scripts/check-host-rpath was not complaining anymore. > > But without it, the problem still occurs for host packages [1]. > > So, restore this patch to fix build of host packages with Meson. > > [1] http://lists.busybox.net/pipermail/buildroot/2018-October/232956.html > > Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > > So yes, ther are situations where using a system-provided meson _may_ > work, but the generic case is that our patch is still required to ensure > that the situatiuon dexribed above still works. Yes, you want to be able to build all packages in buildroot without issues. My patch would not change that. You would need to set an variable override to pick up the system meson/ninja. If upstream is fixed and you are confident nothing breaks, just set that variable in check-host-meson.mk. Work done now, tested by users at their own peril now, flipping the switch later. Sounds not so outrageous for me? > Note that the only package in that situation in upstream Buildroot, is > libglib2, so if you don't have that, then you do not have the problem > this patch is solving. > > So, with those explanations, I hope it is now clearer why we can't rely > on the system-provided meson. Not rely, but add an option that the user can pick up the system meson if he dares to do so. > Anyway, your proposal was interesting, if at least because it really > made us reconsider the usefulness of our patch. > > > > > Another (more useful ?) thing to look at: is it possible to use the > > > > system-provided Python for Meson and Ninja, when python3 is provided by > > > > the system ? I think Meson and Ninja by themselves are not long at all > > > > to build, and it would be a much more useful direction for this patch > > > > series. > > > Define "useful". The patches work correctly for me (tm), and I dont > > > use more than 1000 included packages that are useless for me (tm). > > > Even if meson is not ready (again: it is for me), then improving > > > buildroot ahead of it causes no harm? > > The case "works for me" is not a valid reason, because as demonstrated > above, there are cases where it won't work. As reiterated above, unless you explicitly enable it, those patches will NOT pick up system meson. "Works for me" means there is a valid reason for having the option. Regards, Norbert
Hello Norbert, On Wed, 16 Oct 2019 22:31:24 +0200 Norbert Lange <nolange79@gmail.com> wrote: > > So yes, ther are situations where using a system-provided meson _may_ > > work, but the generic case is that our patch is still required to ensure > > that the situatiuon dexribed above still works. > > Yes, you want to be able to build all packages in buildroot without issues. > My patch would not change that. You would need to set an variable > override to pick up the system meson/ninja. We don't want to have this code that is basically "dead" (because it can only be enabled by tweaking those variables in the environment). Best regards, Thomas
Am Mi., 16. Okt. 2019 um 23:08 Uhr schrieb Thomas Petazzoni <thomas.petazzoni@bootlin.com>: > > Hello Norbert, > > On Wed, 16 Oct 2019 22:31:24 +0200 > Norbert Lange <nolange79@gmail.com> wrote: > > > > So yes, ther are situations where using a system-provided meson _may_ > > > work, but the generic case is that our patch is still required to ensure > > > that the situatiuon dexribed above still works. > > > > Yes, you want to be able to build all packages in buildroot without issues. > > My patch would not change that. You would need to set an variable > > override to pick up the system meson/ninja. > > We don't want to have this code that is basically "dead" (because it can > only be enabled by tweaking those variables in the environment). Sure then, please look at the other patches and I will respin those you deem useful regards, Norbert
diff --git a/package/pkg-meson.mk b/package/pkg-meson.mk index 2756fb7856..3a66ab8f5a 100644 --- a/package/pkg-meson.mk +++ b/package/pkg-meson.mk @@ -25,7 +25,7 @@ # $(HOST_DIR)/bin/python3 will not look for Meson modules in # $HOME/.local/lib/python3.x/site-packages # -MESON = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson +MESON = PYTHONNOUSERSITE=y $(BR2_MESON) NINJA = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja NINJA_OPTS = $(if $(VERBOSE),-v) -j$(PARALLEL_JOBS) @@ -99,7 +99,7 @@ endef endif endif -$(2)_DEPENDENCIES += host-meson +$(2)_DEPENDENCIES += $(BR2_MESON_HOST_DEPENDENCY) # # Build step. Only define it if not already defined by the package .mk diff --git a/support/dependencies/check-host-meson.mk b/support/dependencies/check-host-meson.mk new file mode 100644 index 0000000000..fbe0b7d7aa --- /dev/null +++ b/support/dependencies/check-host-meson.mk @@ -0,0 +1,15 @@ +# Set this to a valid version like 0.49, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. +# An empty version will never match a system executable +# +# BR2_MESON_VERSION_MIN = 0.49 + +BR2_MESON_CANDIDATES ?= meson +BR2_MESON ?= $(call suitable-host-package,meson,\ + $(BR2_MESON_VERSION_MIN) $(BR2_MESON_CANDIDATES)) +ifeq ($(BR2_MESON),) +BR2_MESON = $(HOST_DIR)/bin/meson +BR2_MESON_HOST_DEPENDENCY = host-meson host-ninja +else +BR2_MESON_HOST_DEPENDENCY = host-ninja +endif diff --git a/support/dependencies/check-host-meson.sh b/support/dependencies/check-host-meson.sh new file mode 100755 index 0000000000..62746822cc --- /dev/null +++ b/support/dependencies/check-host-meson.sh @@ -0,0 +1,45 @@ +#!/bin/sh + +# prevent shift error +[ $# -ge 2 ] && [ -n "$1" ] || exit 1 + +split_version() { + local VARPREFIX + local NUMBERS + local major + local minor + + VARPREFIX=$1 + NUMBERS=$2 + + major=${NUMBERS%%\.*} + NUMBERS=${NUMBERS#$major}; NUMBERS=${NUMBERS#\.} + minor=${NUMBERS%%\.*} + NUMBERS=${NUMBERS#$minor}; NUMBERS=${NUMBERS#\.} + + # ensure that missing values are 0 + eval "${VARPREFIX}_major=\$major; ${VARPREFIX}_minor=\$((minor + 0)); ${VARPREFIX}_bugfix=\$((NUMBERS + 0));" +} + +split_version req "$1" + +shift + +for candidate; do + + # Try to locate the candidate. Discard it if not located. + tool=$(which "${candidate}" 2>/dev/null) + [ -n "${tool}" ] || continue + + split_version cur "$("${tool}" --version)" + + [ -n "${cur_major}" -a "${cur_major}" -ge "${req_major}" ] || continue + [ "${cur_major}" -gt "${req_major}" ] || [ "${cur_minor}" -ge "${req_minor}" ] || continue + [ "${cur_minor}" -gt "${req_minor}" ] || [ "${cur_bugfix}" -ge "${req_bugfix}" ] || continue + + echo "${tool}" + exit +done + +# echo nothing: no suitable tool found +exit 1
Automatically check for an available meson tool, and use it as long as the version is fitting. Using a system provided meson is not an official supported option, hence this is currently disabled. use of the system provided meson can be enforced with: make ... BR2_MESON_VERSION_MIN=0 Signed-off-by: Norbert Lange <nolange79@gmail.com> --- package/pkg-meson.mk | 4 +-- support/dependencies/check-host-meson.mk | 15 ++++++++ support/dependencies/check-host-meson.sh | 45 ++++++++++++++++++++++++ 3 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 support/dependencies/check-host-meson.mk create mode 100755 support/dependencies/check-host-meson.sh -- 2.23.0