| Message ID | 20170902212938.23712-1-thomas.petazzoni@free-electrons.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | utils/genrandconfig: use --no-check-certificate in wget by default | expand |
Hello, On Sat, 2 Sep 2017 23:29:38 +0200, Thomas Petazzoni wrote: > A number of autobuilder failures are due to the fact that autobuilder > instances use old distributions, with old SSL certificates, and > therefore wget aborts with an error "The certificate of `xyz.org' is > not trusted.". > > In order to avoid such failures that are not very interesting in the > context of the autobuilders, we pass --no-check-certificate to > wget. The integrity of the downloaded files is anyway verified by the > hashes, and this is only meant to be used in the context of > testing/CI, not in production. > > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Would it be possible to get some feedback on this patch? We recently bump dbus to 1.10.24, and look how the autobuilders are "polluted" by this certificate issue: http://autobuild.buildroot.net/?reason=dbus-1.10.24. We've got two options: either we do it in utils/genrandconfig as proposed, or we do it in the autobuild-run script that runs on the autobuilder slaves. Comments ? Thanks, Thomas
On 10-10-17 22:25, Thomas Petazzoni wrote: > Hello, > > On Sat, 2 Sep 2017 23:29:38 +0200, Thomas Petazzoni wrote: >> A number of autobuilder failures are due to the fact that autobuilder >> instances use old distributions, with old SSL certificates, and >> therefore wget aborts with an error "The certificate of `xyz.org' is >> not trusted.". I don't understand how this can happen. The autobuilders should all be running supported (not EOL) distros, right? And in a supported distro, I'd expect ca-certificates get updated. Otherwise the distro becomes next to useless. But I seem to remember that you have a CentOS 5 autobuilder running, and CentOS 5 went EOL on March 31, 2017. So perhaps it's time to switch to CentOS 6? >> In order to avoid such failures that are not very interesting in the >> context of the autobuilders I think they *are* interesting (not very, but still interesting), because actual users *will* hit these problems. >> , we pass --no-check-certificate to >> wget. The integrity of the downloaded files is anyway verified by the >> hashes, and this is only meant to be used in the context of >> testing/CI, not in production. >> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > > Would it be possible to get some feedback on this patch? > > We recently bump dbus to 1.10.24, and look how the autobuilders are > "polluted" by this certificate issue: > http://autobuild.buildroot.net/?reason=dbus-1.10.24. But once Peter updates sources.buildroot.org that should be OK again, no? Regards, Arnout > > We've got two options: either we do it in utils/genrandconfig as > proposed, or we do it in the autobuild-run script that runs on the > autobuilder slaves. > > Comments ? > > Thanks, > > Thomas >
Hello, On Tue, 10 Oct 2017 23:25:03 +0200, Arnout Vandecappelle wrote: > I don't understand how this can happen. The autobuilders should all be running > supported (not EOL) distros, right? Absolutely not. I'm intentionally running an old Debian in order to catch build issues related to old systems. Running non-supported (i.e EOL distros) is very common in enterprise environments, so I want to keep testing this. Just to be clear: I don't personally run EOL distros, and we don't use EOL distros at Free Electrons so it's not something that I personally care about. But I believe that a lot of companies do use such old distros on build machines, and therefore it makes sense to keep testing with a fairly old system. And therefore the certificates are often out of date. > But I seem to remember that you have a CentOS 5 autobuilder running, and CentOS > 5 went EOL on March 31, 2017. So perhaps it's time to switch to CentOS 6? Me running CentOS ? Crazy you. I'm running a Debian system for the autobuilders. > >> In order to avoid such failures that are not very interesting in the > >> context of the autobuilders > > I think they *are* interesting (not very, but still interesting), because > actual users *will* hit these problems. Not really: by the time we make a release, sources.buildroot.net will have fetched the tarball, and therefore our users will simply see nothing, except that their download gracefully falls back to sources.buildroot.net. So those build failures only pollute the autobuild.buildroot.net results in the time window between a package being bumped, and the tarball being grabbed by sources.buildroot.net. If you look today at autobuild.buildroot.net, there are no more dbus-1.10.24 build failures, because the "old" autobuilder instances download the tarball from sources.b.n. > > We recently bump dbus to 1.10.24, and look how the autobuilders are > > "polluted" by this certificate issue: > > http://autobuild.buildroot.net/?reason=dbus-1.10.24. > > But once Peter updates sources.buildroot.org that should be OK again, no? Yes, but this pollution happens again and again and again every time we bump a package that is fetched from https://, with a certificate too recent for those old systems. This creates useless noise in the autobuilders. And this noise is useless because we are really not going to do anything about this. But perhaps it's an issue more unique to my autobuilder instance, and therefore something I should fix locally? Perhaps just an autobuild-run feature that allows to append a custom config option to the config file being built, so that people can locally add some tweaks? Best regards, Thomas
On 11-10-17 08:53, Thomas Petazzoni wrote: > Hello, > > On Tue, 10 Oct 2017 23:25:03 +0200, Arnout Vandecappelle wrote: [snip] >>> We recently bump dbus to 1.10.24, and look how the autobuilders are >>> "polluted" by this certificate issue: >>> http://autobuild.buildroot.net/?reason=dbus-1.10.24. >> >> But once Peter updates sources.buildroot.org that should be OK again, no? > > Yes, but this pollution happens again and again and again every time we > bump a package that is fetched from https://, with a certificate too > recent for those old systems. This creates useless noise in the > autobuilders. And this noise is useless because we are really not going > to do anything about this. Even though this problem hasn't occurred again in the last half year AFAIK, your explanation does make sense so I've applied to master, thanks. Regards, Arnout > But perhaps it's an issue more unique to my autobuilder instance, and > therefore something I should fix locally? Perhaps just an autobuild-run > feature that allows to append a custom config option to the config file > being built, so that people can locally add some tweaks?
diff --git a/utils/genrandconfig b/utils/genrandconfig index a67d46fad9..5ce6015765 100755 --- a/utils/genrandconfig +++ b/utils/genrandconfig @@ -345,6 +345,9 @@ def gen_config(args): with open(minimalconfigfile) as minimalf: configlines += minimalf.readlines() + # Allow hosts with old certificates to download over https + configlines.append("BR2_WGET=\"wget --passive-ftp -nd -t 3 --no-check-certificate\"") + # Amend the configuration with a few things. if randint(0, 20) == 0: configlines.append("BR2_ENABLE_DEBUG=y\n")
A number of autobuilder failures are due to the fact that autobuilder instances use old distributions, with old SSL certificates, and therefore wget aborts with an error "The certificate of `xyz.org' is not trusted.". In order to avoid such failures that are not very interesting in the context of the autobuilders, we pass --no-check-certificate to wget. The integrity of the downloaded files is anyway verified by the hashes, and this is only meant to be used in the context of testing/CI, not in production. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- utils/genrandconfig | 3 +++ 1 file changed, 3 insertions(+)