| Message ID | 20171130154202.437-1-johan.oudinet@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [1/1] erlang: bump to version 20.0 | expand |
Hi Johan, On Thu, Nov 30, 2017 at 10:42 AM, Johan Oudinet <johan.oudinet@gmail.com> wrote: > > This change bumps the Erlang version from 19.3 to 20.0. As a new major > release, there are a few incompatibilities. Details are here: > http://www.erlang.org/news/114 Thanks for doing this. I had been holding on to a nearly identical patch, but it breaks several erlang packages currently in Buildroot. It didn't look easy to fix, and then I got busy. As I'm sure that you're aware, ejabberd builds fine. I don't think there are many of us Erlang/Buildroot users, so if this still breaks erlang packages that you don't use, I'm fine with removing them. Two comments: 1. Do you want to bump to Erlang 20.1 since that's the latest? 2. Should we remove support for non-SMP builds now that it's deprecated? I've actually never used the option due to it disabling NIF features used by some libraries. Having said that, I'd prefer this patch in than not since it moves Buildroot forward, so Reviewed-by: Frank Hunleth <fhunleth@troodon-software.com> Thanks! Frank > > This should fix builds on ppc64le host machines: > http://autobuild.buildroot.net/results/407af9800313d4540643d7625e85acc0c2366892 > http://autobuild.buildroot.net/results/2aca54ad48ecfd72242d97499e21a46c1b384b10 > > Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com> > --- > package/erlang/erlang.hash | 4 ++-- > package/erlang/erlang.mk | 4 ++-- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/package/erlang/erlang.hash b/package/erlang/erlang.hash > index dc12ccd062..cf820cec1b 100644 > --- a/package/erlang/erlang.hash > +++ b/package/erlang/erlang.hash > @@ -1,3 +1,3 @@ > # md5 from http://www.erlang.org/download/MD5, sha256 locally computed > -md5 a8c259ec47bf84e77510673e1b76b6db otp_src_19.3.tar.gz > -sha256 fe4a00651db39b8542b04530a48d24b2f2e7e0b77cbe93d728c9f05325bdfe83 otp_src_19.3.tar.gz > +md5 2faed2c3519353e6bc2501ed4d8e6ae7 otp_src_20.0.tar.gz > +sha256 fe80e1e14a2772901be717694bb30ac4e9a07eee0cc7a28988724cbd21476811 otp_src_20.0.tar.gz > diff --git a/package/erlang/erlang.mk b/package/erlang/erlang.mk > index f3684d0c91..733c1d5786 100644 > --- a/package/erlang/erlang.mk > +++ b/package/erlang/erlang.mk > @@ -5,7 +5,7 @@ > ################################################################################ > > # See note below when updating Erlang > -ERLANG_VERSION = 19.3 > +ERLANG_VERSION = 20.0 > ERLANG_SITE = http://www.erlang.org/download > ERLANG_SOURCE = otp_src_$(ERLANG_VERSION).tar.gz > ERLANG_DEPENDENCIES = host-erlang > @@ -19,7 +19,7 @@ ERLANG_AUTORECONF = YES > > # Whenever updating Erlang, this value should be updated as well, to the > # value of EI_VSN in the file lib/erl_interface/vsn.mk > -ERLANG_EI_VSN = 3.9.3 > +ERLANG_EI_VSN = 3.10 > > # The configure checks for these functions fail incorrectly > ERLANG_CONF_ENV = ac_cv_func_isnan=yes ac_cv_func_isinf=yes > -- > 2.14.1 >
Hello, On Thu, 30 Nov 2017 11:31:53 -0500, Frank Hunleth wrote: > On Thu, Nov 30, 2017 at 10:42 AM, Johan Oudinet <johan.oudinet@gmail.com> wrote: > > > > This change bumps the Erlang version from 19.3 to 20.0. As a new major > > release, there are a few incompatibilities. Details are here: > > http://www.erlang.org/news/114 > > Thanks for doing this. I had been holding on to a nearly identical > patch, but it breaks several erlang packages currently in Buildroot. What are those packages ? > It didn't look easy to fix, and then I got busy. As I'm sure that > you're aware, ejabberd builds fine. I don't think there are many of us > Erlang/Buildroot users, so if this still breaks erlang packages that > you don't use, I'm fine with removing them. We definitely need to at least identify those packages. Is this a build time breakage ? Run-time breakage ? Best regards, Thomas
Frank, All, On Thu, Nov 30, 2017 at 5:31 PM, Frank Hunleth <fhunleth@troodon-software.com> wrote: > > On Thu, Nov 30, 2017 at 10:42 AM, Johan Oudinet <johan.oudinet@gmail.com> wrote: >> >> This change bumps the Erlang version from 19.3 to 20.0. As a new major >> release, there are a few incompatibilities. Details are here: >> http://www.erlang.org/news/114 > > Thanks for doing this. I had been holding on to a nearly identical > patch, but it breaks several erlang packages currently in Buildroot. > It didn't look easy to fix, and then I got busy. As I'm sure that > you're aware, ejabberd builds fine. I don't think there are many of us > Erlang/Buildroot users, so if this still breaks erlang packages that > you don't use, I'm fine with removing them. > > Two comments: > > 1. Do you want to bump to Erlang 20.1 since that's the latest? No, I tried it but it has a compiler bug that breaks some Erlang packages in Buildroot. For example, I got the following error in erlang-p1-xmpp package: https://bugs.erlang.org/browse/ERL-493 I think it's better to skip OTP 20.1 for now. I'm not aware of Erlang packages that are broken by OTP 20.0. So same question as Thomas's one, what are those packages? > 2. Should we remove support for non-SMP builds now that it's > deprecated? I've actually never used the option due to it disabling > NIF features used by some libraries. Excellent question. I haven't thought about it and I've never used this option neither. Since it is now deprecated, I guess we can remove it. Should I remove it in another patch or revise this patch in a v2 that does both a bump to version 20.0 and a commit to remove the option to disable compilation of the SMP emulator? Best,
Johan, On Fri, Dec 1, 2017 at 10:16 AM, Johan Oudinet <johan.oudinet@gmail.com> wrote: > Frank, All, > > On Thu, Nov 30, 2017 at 5:31 PM, Frank Hunleth > <fhunleth@troodon-software.com> wrote: >> >> On Thu, Nov 30, 2017 at 10:42 AM, Johan Oudinet <johan.oudinet@gmail.com> wrote: >>> >>> This change bumps the Erlang version from 19.3 to 20.0. As a new major >>> release, there are a few incompatibilities. Details are here: >>> http://www.erlang.org/news/114 >> >> Thanks for doing this. I had been holding on to a nearly identical >> patch, but it breaks several erlang packages currently in Buildroot. >> It didn't look easy to fix, and then I got busy. As I'm sure that >> you're aware, ejabberd builds fine. I don't think there are many of us >> Erlang/Buildroot users, so if this still breaks erlang packages that >> you don't use, I'm fine with removing them. >> >> Two comments: >> >> 1. Do you want to bump to Erlang 20.1 since that's the latest? > > No, I tried it but it has a compiler bug that breaks some Erlang > packages in Buildroot. > For example, I got the following error in erlang-p1-xmpp package: > https://bugs.erlang.org/browse/ERL-493 > > I think it's better to skip OTP 20.1 for now. > > I'm not aware of Erlang packages that are broken by OTP 20.0. So same > question as Thomas's one, what are those packages? My mistake. I ran into the compiler bug too, but looking back at my experiment, it was OTP 20.1 and not OTP 20.0. I'd have to rerun everything to get the list of errors from 20.1. I agree with you about sticking with OTP 20.0 for now. Frank > >> 2. Should we remove support for non-SMP builds now that it's >> deprecated? I've actually never used the option due to it disabling >> NIF features used by some libraries. > > Excellent question. I haven't thought about it and I've never used > this option neither. > Since it is now deprecated, I guess we can remove it. > Should I remove it in another patch or revise this patch in a v2 that > does both a bump to version 20.0 and a commit to remove the option to > disable compilation of the SMP emulator?
Hello, On Thu, 30 Nov 2017 16:42:02 +0100, Johan Oudinet wrote: > This change bumps the Erlang version from 19.3 to 20.0. As a new major > release, there are a few incompatibilities. Details are here: > http://www.erlang.org/news/114 > > This should fix builds on ppc64le host machines: > http://autobuild.buildroot.net/results/407af9800313d4540643d7625e85acc0c2366892 > http://autobuild.buildroot.net/results/2aca54ad48ecfd72242d97499e21a46c1b384b10 > > Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com> > --- > package/erlang/erlang.hash | 4 ++-- > package/erlang/erlang.mk | 4 ++-- > 2 files changed, 4 insertions(+), 4 deletions(-) Applied to master, thanks. Thomas
Hello, On Thu, 30 Nov 2017 16:42:02 +0100, Johan Oudinet wrote: > This change bumps the Erlang version from 19.3 to 20.0. As a new major > release, there are a few incompatibilities. Details are here: > http://www.erlang.org/news/114 > > This should fix builds on ppc64le host machines: > http://autobuild.buildroot.net/results/407af9800313d4540643d7625e85acc0c2366892 > http://autobuild.buildroot.net/results/2aca54ad48ecfd72242d97499e21a46c1b384b10 > > Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com> > --- > package/erlang/erlang.hash | 4 ++-- > package/erlang/erlang.mk | 4 ++-- > 2 files changed, 4 insertions(+), 4 deletions(-) This causes a bunch of build failures: http://autobuild.buildroot.net/?reason=host-erlang-20.0 Seems like host-zlib is needed. Could you have a look ? Thanks! Thomas
Hi, On Sun, Dec 3, 2017 at 10:59 AM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > Hello, > > On Thu, 30 Nov 2017 16:42:02 +0100, Johan Oudinet wrote: >> This change bumps the Erlang version from 19.3 to 20.0. As a new major >> release, there are a few incompatibilities. Details are here: >> http://www.erlang.org/news/114 >> >> This should fix builds on ppc64le host machines: >> http://autobuild.buildroot.net/results/407af9800313d4540643d7625e85acc0c2366892 >> http://autobuild.buildroot.net/results/2aca54ad48ecfd72242d97499e21a46c1b384b10 >> >> Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com> >> --- >> package/erlang/erlang.hash | 4 ++-- >> package/erlang/erlang.mk | 4 ++-- >> 2 files changed, 4 insertions(+), 4 deletions(-) > > This causes a bunch of build failures: > > http://autobuild.buildroot.net/?reason=host-erlang-20.0 > > Seems like host-zlib is needed. > > Could you have a look ? When trying to reproduce this build failure on my machine, I discovered an unrelated error in flann package: ERROR: unsafe header/library path used in cross-compilation '/usr/include/hdf5/serial' I skipped this error by adding this to FLANN_CONF_OPTS: -DCMAKE_DISABLE_FIND_PACKAGE_HDF5=TRUE As far as I understand it, this HDF5 dependency from flann is only here for testing purpose and can be disabled in buildroot build. Should I send a patch? Unfortunately, I failed to reproduce the build failure on erlang but when looking at the log, it indeed failed to find libz even though the configure above seems to find it: checking for zlib 1.2.5 or higher... yes checking for zlib inflateGetDictionary presence... checking for library containing inflateGetDictionary... -lz What bother me is that I can't find which changes from OTP 19.3 to OTP 20.0 could lead to this error. Anyway, should I simply add host-zlib as an erlang dependency and see if the next build would pass? Best,
Am Sun, 03 Dec 2017 10:59:05 +0100 schrieb Thomas Petazzoni:
> Seems like host-zlib is needed.
Hi,
host-erlang depends on host-openssl which depends on host-zlib so this
should not be the problem. Anyway I could reproduce the build error on a
minimal Debian system which lacks a zlib-dev package, host-zlib was built
by buildroot.
More investigation is needed on my part, but for today I have no time
left for this, sorry.
Regards, Bernd
Hi, On Mon, Dec 4, 2017 at 6:49 PM, Bernd Kuhls <bernd.kuhls@t-online.de> wrote: > Am Sun, 03 Dec 2017 10:59:05 +0100 schrieb Thomas Petazzoni: > >> Seems like host-zlib is needed. > > host-erlang depends on host-openssl which depends on host-zlib so this > should not be the problem. Anyway I could reproduce the build error on a > minimal Debian system which lacks a zlib-dev package, host-zlib was built > by buildroot. > > More investigation is needed on my part, but for today I have no time > left for this, sorry. > Thanks, I did the same (i.e., run it from a fresh debian-testing-amd64 chroot) and manage to reproduce the -lz error: /usr/bin/gcc -c -o ../priv/obj/x86_64-pc-linux-gnu/trace_ip_drv.o -DHAVE_CONFIG_H -Werror=undef -Werror=implicit -Werror=return-type -Wall -Wstrict-prototypes -Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -D_POSIX_THREAD_SAFE_FUNCTIONS -O2 -I/path/to/host/include -I/path/to/build/host-erlang-20.0/erts/x86_64-pc-linux-gnu -fno-tree-copyrename -I/path/to/host/include -D_GNU_SOURCE -fPIC -I./ -I/path/to/build/host-erlang-20.0/erts/emulator/beam -I/path/to/build/host-erlang-20.0/erts/include -I/path/to/build/host-erlang-20.0/erts/include/x86_64-pc-linux-gnu -I/path/to/build/host-erlang-20.0/erts/include/internal -I/path/to/build/host-erlang-20.0/erts/include/internal/x86_64-pc-linux-gnu -I/path/to/build/host-erlang-20.0/erts/emulator/sys/unix -I/path/to/build/host-erlang-20.0/erts/emulator/sys/common -I../priv/obj/x86_64-pc-linux-gnu -I/path/to/build/host-erlang-20.0/erts/emulator/x86_64-pc-linux-gnu trace_ip_drv.c /usr/bin/gcc -shared -Wl,-Bsymbolic -o ../priv/lib/x86_64-pc-linux-gnu/dyntrace.so ../priv/obj/x86_64-pc-linux-gnu/dyntrace.o -lz -lutil -ldl -lm /usr/bin/ld: cannot find -lz collect2: error: ld returned 1 exit status It does have libz.so in the host so I think the issue is in one of Erlang's makefiles. Their build architecture is a real mess and I'm not sure how to fix it properly, but it clearly miss the HOST_LDFLAGS when linking lib/runtime_tools/c_src/dyntrace.c I try to add LDFLAGS="$(HOST_LDFLAGS)" to HOST_ERLANG_MAKE_OPTS and it does solve this issue but fails later on: /usr/bin/gcc -fno-PIE -Werror=undef -Werror=implicit -Werror=return-type -O3 -I/path/to/host/include -I/path/to/build/host-erlang-20.0/erts/x86_64-pc-linux-gnu -fno-tree-copyrename -I/path/to/host/include -D_GNU_SOURCE -DERTS_SMP -DHAVE_CONFIG_H -Wall -Wstrict-prototypes -Wmissing-prototypes -Wdeclaration-after-statement -DUSE_THREADS -D_THREAD_SAFE -D_REENTRANT -DPOSIX_THREADS -D_POSIX_THREAD_SAFE_FUNCTIONS -DERTS_DIRTY_SCHEDULERS -Ix86_64-pc-linux-gnu/opt/smp -Ibeam -Isys/unix -Isys/common -Ix86_64-pc-linux-gnu -Ipcre -Ihipe -I../include -I../include/x86_64-pc-linux-gnu -I../include/internal -I../include/internal/x86_64-pc-linux-gnu -c hipe/hipe_mkliterals.c -o obj/x86_64-pc-linux-gnu/opt/smp/hipe_mkliterals.o /usr/bin/gcc -L/path/to/host/lib -Wl,-rpath,/path/to/host/lib -o /path/to/build/host-erlang-20.0/bin/x86_64-pc-linux-gnu/hipe_mkliterals.smp obj/x86_64-pc-linux-gnu/opt/smp/hipe_mkliterals.o -lz -lutil -ldl -lm /usr/bin/ld: obj/x86_64-pc-linux-gnu/opt/smp/hipe_mkliterals.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC /usr/bin/ld: final link failed: Nonrepresentable section on output collect2: error: ld returned 1 exit status I stop here for today.
Am Mon, 04 Dec 2017 17:32:31 +0100 schrieb Johan Oudinet: > What bother me is that I can't find which changes from OTP 19.3 to OTP > 20.0 could lead to this error. Hi, I think I found the commit, added with version 20.0, which causes the error: https://github.com/erlang/otp/ commit/640c988fd41f9709b494554b2e5ef1f06f06957e The autoconf function AC_SEARCH_LIBS adds "-llibrary to LIBS for the first library found to contain function" according to the autoconf manual: https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/ Libraries.html And this is exactly what happens: host-erlang-20.0/erts/config.status contains S["LIBS"]="-lz -lutil -ldl -lm " while host-erlang-19.3/erts/config.status lacks -lz: S["LIBS"]="-lutil -ldl -lm " The content of this variable is used to build dyntrace.o with this Makefile: lib/runtime_tools/c_src/Makefile.in, line 98. Unfortunately I do not have more time to look for a fix today. Regards, Bernd
On Tue, Dec 5, 2017 at 8:34 PM, Bernd Kuhls <bernd.kuhls@t-online.de> wrote: > Am Mon, 04 Dec 2017 17:32:31 +0100 schrieb Johan Oudinet: > > I think I found the commit, added with version 20.0, which causes the > error: https://github.com/erlang/otp/ > commit/640c988fd41f9709b494554b2e5ef1f06f06957e > > The autoconf function AC_SEARCH_LIBS adds "-llibrary to LIBS for the > first library found to contain function" according to the autoconf manual: > https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/ > Libraries.html > > And this is exactly what happens: > > host-erlang-20.0/erts/config.status contains > > S["LIBS"]="-lz -lutil -ldl -lm " > > while host-erlang-19.3/erts/config.status lacks -lz: > > S["LIBS"]="-lutil -ldl -lm " > > The content of this variable is used to build dyntrace.o with this > Makefile: lib/runtime_tools/c_src/Makefile.in, line 98. > Good catch. I tried the workaround of adding --enable-builtin-zlib in HOST_ERLANG_CONF_OPTS and it works. So this is definitely an issue with how their use of AC_SEARCH_LIBS in erts/configure.in leaks to how dyntrace.o is linked in lib/runtime_tools/c_src/Makefile.in. I don't think it is acceptable to use --enable-builtin-zlib in buildroot packaging of Erlang but we're making progress.
Hello, On Mon, 4 Dec 2017 17:32:31 +0100, Johan Oudinet wrote: > When trying to reproduce this build failure on my machine, I > discovered an unrelated error in flann package: > ERROR: unsafe header/library path used in cross-compilation > '/usr/include/hdf5/serial' > > I skipped this error by adding this to FLANN_CONF_OPTS: > -DCMAKE_DISABLE_FIND_PACKAGE_HDF5=TRUE > > As far as I understand it, this HDF5 dependency from flann is only > here for testing purpose and can be disabled in buildroot build. > Should I send a patch? Yes, please :) > Unfortunately, I failed to reproduce the build failure on erlang but > when looking at the log, it indeed failed to find libz even though the > configure above seems to find it: > checking for zlib 1.2.5 or higher... yes > checking for zlib inflateGetDictionary presence... checking for > library containing inflateGetDictionary... -lz Perhaps it's testing with the 64-bit version of the library, but then using the 32-bit version or the opposite. Can you build with V=1 (or whatever is needed in Erlang to get verbose details), and see what is the exact command that fails to link with zlib ? Best regards, Thomas
diff --git a/package/erlang/erlang.hash b/package/erlang/erlang.hash index dc12ccd062..cf820cec1b 100644 --- a/package/erlang/erlang.hash +++ b/package/erlang/erlang.hash @@ -1,3 +1,3 @@ # md5 from http://www.erlang.org/download/MD5, sha256 locally computed -md5 a8c259ec47bf84e77510673e1b76b6db otp_src_19.3.tar.gz -sha256 fe4a00651db39b8542b04530a48d24b2f2e7e0b77cbe93d728c9f05325bdfe83 otp_src_19.3.tar.gz +md5 2faed2c3519353e6bc2501ed4d8e6ae7 otp_src_20.0.tar.gz +sha256 fe80e1e14a2772901be717694bb30ac4e9a07eee0cc7a28988724cbd21476811 otp_src_20.0.tar.gz diff --git a/package/erlang/erlang.mk b/package/erlang/erlang.mk index f3684d0c91..733c1d5786 100644 --- a/package/erlang/erlang.mk +++ b/package/erlang/erlang.mk @@ -5,7 +5,7 @@ ################################################################################ # See note below when updating Erlang -ERLANG_VERSION = 19.3 +ERLANG_VERSION = 20.0 ERLANG_SITE = http://www.erlang.org/download ERLANG_SOURCE = otp_src_$(ERLANG_VERSION).tar.gz ERLANG_DEPENDENCIES = host-erlang @@ -19,7 +19,7 @@ ERLANG_AUTORECONF = YES # Whenever updating Erlang, this value should be updated as well, to the # value of EI_VSN in the file lib/erl_interface/vsn.mk -ERLANG_EI_VSN = 3.9.3 +ERLANG_EI_VSN = 3.10 # The configure checks for these functions fail incorrectly ERLANG_CONF_ENV = ac_cv_func_isnan=yes ac_cv_func_isinf=yes
This change bumps the Erlang version from 19.3 to 20.0. As a new major release, there are a few incompatibilities. Details are here: http://www.erlang.org/news/114 This should fix builds on ppc64le host machines: http://autobuild.buildroot.net/results/407af9800313d4540643d7625e85acc0c2366892 http://autobuild.buildroot.net/results/2aca54ad48ecfd72242d97499e21a46c1b384b10 Signed-off-by: Johan Oudinet <johan.oudinet@gmail.com> --- package/erlang/erlang.hash | 4 ++-- package/erlang/erlang.mk | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)