| Message ID | c85b3d3994261356bedfa3daeaddbc4329836fe9.1533327111.git.yann.morin.1998@free.fr |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [1/3,v3] core/pkg-kconfig: allow dependencies before configurators | expand |
On pátek 3. srpna 2018 22:16:55 CEST, Yann E. MORIN wrote: > Take this opportunity to also drop the append-assignment as the first > assignment to LINUX_DEPENDENCIES. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > > --- > Changes v2 -> v3: > - also fix the append-assignment > --- > linux/linux.mk | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/linux/linux.mk b/linux/linux.mk > index 7527b11673..60dad3a249 100644 > --- a/linux/linux.mk > +++ b/linux/linux.mk > @@ -62,7 +62,11 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) > LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES)) > > LINUX_INSTALL_IMAGES = YES > -LINUX_DEPENDENCIES += host-bison host-flex host-kmod > +LINUX_DEPENDENCIES = host-kmod > + > +# Starting with 4.16, the generated kconfig code is no longer > +# shipped with the kernel sources, so we need flex and bison. > +LINUX_KCONFIG_DEPENDENCIES = host-bison host-flex > > # host tools needed for kernel compression > ifeq ($(BR2_LINUX_KERNEL_LZ4),y) Tested-by: Jan Kundrát <jan.kundrat@cesnet.cz>
Hello, On Fri, 3 Aug 2018 22:16:55 +0200, Yann E. MORIN wrote: > Take this opportunity to also drop the append-assignment as the first > assignment to LINUX_DEPENDENCIES. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> I am a bit confused. In this series, you suggest to make host-flex/host-bison dependencies of the configuration step of Linux. But in another series (http://patchwork.ozlabs.org/project/buildroot/list/?series=59260), you suggest to make flex/bison hard requirements of Buildroot, so that we don't have to build them. Could you clarify in which direction you would like to go ? Also, if we make bison/flex hard requirements of Buildroot, I'd like to have a clear rule on whether we keep or not our host-bison/host-flex packages, and if we keep them, when they should be used vs. when the system bison/flex should be used. Thanks! Thomas
Thomas, All, On 2018-08-14 16:21 +0200, Thomas Petazzoni spake thusly: > On Fri, 3 Aug 2018 22:16:55 +0200, Yann E. MORIN wrote: > > Take this opportunity to also drop the append-assignment as the first > > assignment to LINUX_DEPENDENCIES. > > > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Cc: Arnout Vandecappelle <arnout@mind.be> > > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > > I am a bit confused. In this series, you suggest to make > host-flex/host-bison dependencies of the configuration step of Linux. > But in another series > (http://patchwork.ozlabs.org/project/buildroot/list/?series=59260), you > suggest to make flex/bison hard requirements of Buildroot, so that we > don't have to build them. > > Could you clarify in which direction you would like to go ? This very series is about solving a technical problem, that kconfig-based packages may have dependencies required even before running the configurators. On the other hand, that other series is about changing our requirements. If we decide to require flex+bison, so in this series, only patch 2 can be dropped. > Also, if we > make bison/flex hard requirements of Buildroot, I'd like to have a > clear rule on whether we keep or not our host-bison/host-flex packages, > and if we keep them, when they should be used vs. when the system > bison/flex should be used. My position is that we will always want to have our host flex and bison to build packages that install things in host/, in staging/ or in target/. For the linux kernel and other kconfig-based packages, we don't care which flex/bison are used, because the resulting binaries are not installed, unless those packages also generate code eventually installed in host/, staging/ or target/ Regards, Yann E. MORIN.
Hello, On Tue, 14 Aug 2018 17:27:43 +0200, Yann E. MORIN wrote: > > Could you clarify in which direction you would like to go ? > > This very series is about solving a technical problem, that > kconfig-based packages may have dependencies required even before > running the configurators. > > On the other hand, that other series is about changing our requirements. > > If we decide to require flex+bison, so in this series, only patch 2 can > be dropped. OK. > > Also, if we > > make bison/flex hard requirements of Buildroot, I'd like to have a > > clear rule on whether we keep or not our host-bison/host-flex packages, > > and if we keep them, when they should be used vs. when the system > > bison/flex should be used. > > My position is that we will always want to have our host flex and bison > to build packages that install things in host/, in staging/ or in > target/. > > For the linux kernel and other kconfig-based packages, we don't care > which flex/bison are used, because the resulting binaries are not > installed, unless those packages also generate code eventually installed > in host/, staging/ or target/ So the system-wide bison/flex would only be used for the kconfig programs built by Buildroot itself (if we move to a newer kconfig version that doesn't have the pre-generated files) or by Linux/U-Boot/Busybox/Barebox. Any other use of flex/bison would use host-flex and host-bison. That's your proposal ? Best regards, Thomas
Thomas, All, On 2018-08-14 21:53 +0200, Thomas Petazzoni spake thusly: > On Tue, 14 Aug 2018 17:27:43 +0200, Yann E. MORIN wrote: > > > Also, if we > > > make bison/flex hard requirements of Buildroot, I'd like to have a > > > clear rule on whether we keep or not our host-bison/host-flex packages, > > > and if we keep them, when they should be used vs. when the system > > > bison/flex should be used. > > > > My position is that we will always want to have our host flex and bison > > to build packages that install things in host/, in staging/ or in > > target/. > > > > For the linux kernel and other kconfig-based packages, we don't care > > which flex/bison are used, because the resulting binaries are not > > installed, unless those packages also generate code eventually installed > > in host/, staging/ or target/ > > So the system-wide bison/flex would only be used for the kconfig > programs built by Buildroot itself (if we move to a newer kconfig > version that doesn't have the pre-generated files) or by > Linux/U-Boot/Busybox/Barebox. Any other use of flex/bison would use > host-flex and host-bison. That's your proposal ? Yes, that is my proposal. Regards, Yann E. MORIN.
Hello, On Tue, 14 Aug 2018 23:03:49 +0200, Yann E. MORIN wrote: > > So the system-wide bison/flex would only be used for the kconfig > > programs built by Buildroot itself (if we move to a newer kconfig > > version that doesn't have the pre-generated files) or by > > Linux/U-Boot/Busybox/Barebox. Any other use of flex/bison would use > > host-flex and host-bison. That's your proposal ? > > Yes, that is my proposal. OK. Let's see what Arnout/Peter have to say about this. If they don't express any opinion by Thursday, I'll go with your proposal which seems reasonable to me. Thanks, Thomas
On 14-08-18 17:27, Yann E. MORIN wrote: > Thomas, All, > > On 2018-08-14 16:21 +0200, Thomas Petazzoni spake thusly: >> On Fri, 3 Aug 2018 22:16:55 +0200, Yann E. MORIN wrote: >>> Take this opportunity to also drop the append-assignment as the first >>> assignment to LINUX_DEPENDENCIES. >>> >>> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> >>> Cc: Arnout Vandecappelle <arnout@mind.be> >>> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> >> >> I am a bit confused. In this series, you suggest to make >> host-flex/host-bison dependencies of the configuration step of Linux. >> But in another series >> (http://patchwork.ozlabs.org/project/buildroot/list/?series=59260), you >> suggest to make flex/bison hard requirements of Buildroot, so that we >> don't have to build them. >> >> Could you clarify in which direction you would like to go ? > > This very series is about solving a technical problem, that > kconfig-based packages may have dependencies required even before > running the configurators. > > On the other hand, that other series is about changing our requirements. > > If we decide to require flex+bison, so in this series, only patch 2 can > be dropped. > >> Also, if we >> make bison/flex hard requirements of Buildroot, I'd like to have a >> clear rule on whether we keep or not our host-bison/host-flex packages, >> and if we keep them, when they should be used vs. when the system >> bison/flex should be used. > > My position is that we will always want to have our host flex and bison > to build packages that install things in host/, in staging/ or in > target/. My position is that we do indeed need our host flex and bison for everything installed in target/ (which implies everything installed in staging/), but not for things installed in host/. We definitely need it for BR2_REPRODUCIBLE. It would also be nice if dependencies.sh could do a version check and accept the system flex/bison if it is the right version, like we do for cmake and tar. But both of these wishes make things more complicated, so I'm OK with just keeping all current host-flex/bison dependencies. > For the linux kernel and other kconfig-based packages, we don't care > which flex/bison are used, because the resulting binaries are not > installed, unless those packages also generate code eventually installed > in host/, staging/ or target/ Note that this means you'll still need host-flex/bison for the kernel, because the kernel may build dtc and that gets installed in host... Regards, Arnout > > Regards, > Yann E. MORIN. >
Hello, On Wed, 15 Aug 2018 00:50:42 +0200, Arnout Vandecappelle wrote: > My position is that we do indeed need our host flex and bison for everything > installed in target/ (which implies everything installed in staging/), but not > for things installed in host/. We definitely need it for BR2_REPRODUCIBLE. > > It would also be nice if dependencies.sh could do a version check and accept > the system flex/bison if it is the right version, like we do for cmake and tar. > > But both of these wishes make things more complicated, so I'm OK with just > keeping all current host-flex/bison dependencies. In fact, I think adding flex and bison as mandatory dependencies of the system is going to cause too much disruption. All the autobuilders would start complaining, all the Travis/Gitlab CI setups would have to be adjusted, and everyone building in containers/minimal environments will be affected. So instead, I'd like to detect if flex/bison are available on the system. If they are available, then we use them for the Linux kernel configuration process. If they are not available, we build them. Instead of: LINUX_KCONFIG_DEPENDENCIES = host-bison host-flex I'd prefer to see: LINUX_KCONFIG_DEPENDENCIES = $(BR2_FLEX_HOST_DEPENDENCY) $(BR2_BISON_HOST_DEPENDENCY) And the usual logic in support/dependencies/ to check for bison/flex. This will avoid disrupting all existing setups, and still provide the speed-up that we don't need to build flex/bison when doing "make linux-menuconfig". > > For the linux kernel and other kconfig-based packages, we don't care > > which flex/bison are used, because the resulting binaries are not > > installed, unless those packages also generate code eventually installed > > in host/, staging/ or target/ > > Note that this means you'll still need host-flex/bison for the kernel, because > the kernel may build dtc and that gets installed in host... Are you sure flex/bison are used when building the DTC copy in the Linux kernel ? Indeed we only added host-bison/host-flex as dependencies of linux in commit 1b9faedf32be26d9c983c573ccd98f57fc6f6569, when kconfig stopped shipping its pre-generated files. How could DTC be built before 1b9faedf32be26d9c983c573ccd98f57fc6f6569 if it required bison/flex ? Best regards, Thomas
Thomas, All, On 2018-08-15 14:18 +0200, Thomas Petazzoni spake thusly: > On Wed, 15 Aug 2018 00:50:42 +0200, Arnout Vandecappelle wrote: > > My position is that we do indeed need our host flex and bison for everything > > installed in target/ (which implies everything installed in staging/), but not > > for things installed in host/. We definitely need it for BR2_REPRODUCIBLE. > > > > It would also be nice if dependencies.sh could do a version check and accept > > the system flex/bison if it is the right version, like we do for cmake and tar. > > > > But both of these wishes make things more complicated, so I'm OK with just > > keeping all current host-flex/bison dependencies. > > In fact, I think adding flex and bison as mandatory dependencies of the > system is going to cause too much disruption. All the autobuilders > would start complaining, all the Travis/Gitlab CI setups would have to > be adjusted, and everyone building in containers/minimal environments > will be affected. Yes, they will be affected. If not today, then later, because at some point we will also bump to require new versions of componentns. Oh, wait, we already bumped to gcc-4.4 (even if that is mostly transparent). So I don't think we should refrain from updating, or adding to, our requirements just because some people will be affected... > So instead, I'd like to detect if flex/bison are available on the > system. If they are available, then we use them for the Linux kernel > configuration process. If they are not available, we build them. > > Instead of: > LINUX_KCONFIG_DEPENDENCIES = host-bison host-flex > I'd prefer to see: > LINUX_KCONFIG_DEPENDENCIES = $(BR2_FLEX_HOST_DEPENDENCY) $(BR2_BISON_HOST_DEPENDENCY) Yeah, why not. That is easy enough. > And the usual logic in support/dependencies/ to check for bison/flex. > This will avoid disrupting all existing setups, and still provide the > speed-up that we don't need to build flex/bison when doing "make > linux-menuconfig". Except we will soon to have to build the toolchain to be able to run linux-menuconfig anyway... So, the flex/bison build time will not be noticeable, except for external toolchains. > > > For the linux kernel and other kconfig-based packages, we don't care > > > which flex/bison are used, because the resulting binaries are not > > > installed, unless those packages also generate code eventually installed > > > in host/, staging/ or target/ > > > > Note that this means you'll still need host-flex/bison for the kernel, because > > the kernel may build dtc and that gets installed in host... But they would not be a LINUX_KCONFIG_DEPENDENCIES, just a plain LINUX_DEPENDENCIES. Regards, Yann E. MORIN. > Are you sure flex/bison are used when building the DTC copy in the > Linux kernel ? Indeed we only added host-bison/host-flex as > dependencies of linux in commit > 1b9faedf32be26d9c983c573ccd98f57fc6f6569, when kconfig stopped shipping > its pre-generated files. How could DTC be built before > 1b9faedf32be26d9c983c573ccd98f57fc6f6569 if it required bison/flex ? > > Best regards, > > Thomas > -- > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) > Embedded Linux and Kernel engineering > https://bootlin.com
On 15/08/2018 14:18, Thomas Petazzoni wrote: > Hello, > > On Wed, 15 Aug 2018 00:50:42 +0200, Arnout Vandecappelle wrote: [snip] > I'd prefer to see: > > LINUX_KCONFIG_DEPENDENCIES = $(BR2_FLEX_HOST_DEPENDENCY) $(BR2_BISON_HOST_DEPENDENCY) > > And the usual logic in support/dependencies/ to check for bison/flex. > This will avoid disrupting all existing setups, and still provide the > speed-up that we don't need to build flex/bison when doing "make > linux-menuconfig". +1. It also makes it easy to add the feature "build flex is system version is not OK" later. >>> For the linux kernel and other kconfig-based packages, we don't care >>> which flex/bison are used, because the resulting binaries are not >>> installed, unless those packages also generate code eventually installed >>> in host/, staging/ or target/ >> >> Note that this means you'll still need host-flex/bison for the kernel, because >> the kernel may build dtc and that gets installed in host... > > Are you sure flex/bison are used when building the DTC copy in the > Linux kernel ? Indeed we only added host-bison/host-flex as > dependencies of linux in commit > 1b9faedf32be26d9c983c573ccd98f57fc6f6569, when kconfig stopped shipping > its pre-generated files. How could DTC be built before > 1b9faedf32be26d9c983c573ccd98f57fc6f6569 if it required bison/flex ? The shipped files for DTC were removed in e039139be8c25145b103ab365ff1bd4a540066a3 which is v4.17-rc1~106^2~8^2~2 (while the kconfig stuff was removed in 29c833061c1d8c2d1d23a62e7061561eadd76cdb which is v4.16-rc1~105^2~26). So we already had the dependencies when the shipped DTC files were removed from the kernel. And as Yann said, they're not needed for kconfig itself, only for building. Regards, Arnout > > Best regards, > > Thomas >
diff --git a/linux/linux.mk b/linux/linux.mk index 7527b11673..60dad3a249 100644 --- a/linux/linux.mk +++ b/linux/linux.mk @@ -62,7 +62,11 @@ LINUX_PATCHES = $(call qstrip,$(BR2_LINUX_KERNEL_PATCH)) LINUX_PATCH = $(filter ftp://% http://% https://%,$(LINUX_PATCHES)) LINUX_INSTALL_IMAGES = YES -LINUX_DEPENDENCIES += host-bison host-flex host-kmod +LINUX_DEPENDENCIES = host-kmod + +# Starting with 4.16, the generated kconfig code is no longer +# shipped with the kernel sources, so we need flex and bison. +LINUX_KCONFIG_DEPENDENCIES = host-bison host-flex # host tools needed for kernel compression ifeq ($(BR2_LINUX_KERNEL_LZ4),y)
Take this opportunity to also drop the append-assignment as the first assignment to LINUX_DEPENDENCIES. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> --- Changes v2 -> v3: - also fix the append-assignment --- linux/linux.mk | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)