| Message ID | 1448654529-27021-1-git-send-email-yann.morin.1998@free.fr |
|---|---|
| State | Changes Requested |
| Headers | show |
On 27-11-15 21:02, Yann E. MORIN wrote: > When 'file' is missing on the host, libtool fails miserably. Packages > that use libtool will suddenly FTBFS, with cryptic error messages, like > missing libraries on the linker invocation. I've taken a look at an ltmain.sh and a libtool, but the only two calls to 'file' I could find were for darwin and for win32. > > We could ensure that autotools based packages now all depend on > host-file. But It itself is an autotools package, so it's again a > chicken-n-egg issue. And even non-autotools package may use libtool. > > So, just require that 'file' is present on the host. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > --- > support/dependencies/dependencies.sh | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh > index 3146401..f14fb79 100755 > --- a/support/dependencies/dependencies.sh > +++ b/support/dependencies/dependencies.sh > @@ -69,6 +69,12 @@ check_prog_host "which" > # Verify that sed is installed > check_prog_host "sed" > > +# 'file' must be present, otherwise libtool fails in incomprehensible > +# ways. For example, bandwidthd would fail to link with -lz, even bandwidthd doesn't even have a libtool... Could you give a concrete example of where libtool uses file? Regards, Arnout > +# though the library is there (and it's not a static/shared issue). > +# Adding the 'file' program on the system miraculously fixes the issue. > +check_prog_host "file" > + > # Check make > MAKE=$(which make 2> /dev/null) > if [ -z "$MAKE" ] ; then >
Arnout, All, On 2015-11-27 22:19 +0100, Arnout Vandecappelle spake thusly: > On 27-11-15 21:02, Yann E. MORIN wrote: > > When 'file' is missing on the host, libtool fails miserably. Packages > > that use libtool will suddenly FTBFS, with cryptic error messages, like > > missing libraries on the linker invocation. > > I've taken a look at an ltmain.sh and a libtool, but the only two calls to > 'file' I could find were for darwin and for win32. > > > > > We could ensure that autotools based packages now all depend on > > host-file. But It itself is an autotools package, so it's again a > > chicken-n-egg issue. And even non-autotools package may use libtool. > > > > So, just require that 'file' is present on the host. > > > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > > --- > > support/dependencies/dependencies.sh | 6 ++++++ > > 1 file changed, 6 insertions(+) > > > > diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh > > index 3146401..f14fb79 100755 > > --- a/support/dependencies/dependencies.sh > > +++ b/support/dependencies/dependencies.sh > > @@ -69,6 +69,12 @@ check_prog_host "which" > > # Verify that sed is installed > > check_prog_host "sed" > > > > +# 'file' must be present, otherwise libtool fails in incomprehensible > > +# ways. For example, bandwidthd would fail to link with -lz, even > > bandwidthd doesn't even have a libtool... > > Could you give a concrete example of where libtool uses file? Damn... It seems I confused the netsnmp and the bandwidthd cases. If you look at netsnmp, they have all this blurb about file being required for libtool otherwise it breaks, in configure.d/config_os_progs And indeed, my minimalistic chroot does not have file installed. And then you have bandwidthd that also fails miserably in that chroot, even though the same configuration works on my machine. So I installed 'file' and then both now compile without any issue in that minimalistic chroot. And then I did the commit log for this patch, and only accounted for the reasons explained in netsnmp (without trying to validat etheir point, I admit), but referenced bandwidthd. Sigh. Anyway, here's my minimalistic chroot: debootstrap --arch=amd64 --variant buildd trusty $(pwd)/chroot and the list of installed packages is in attachment. Regards, Yann E. MORIN.
On 28-11-15 00:04, Yann E. MORIN wrote: > Arnout, All, > > On 2015-11-27 22:19 +0100, Arnout Vandecappelle spake thusly: >> On 27-11-15 21:02, Yann E. MORIN wrote: >>> When 'file' is missing on the host, libtool fails miserably. Packages >>> that use libtool will suddenly FTBFS, with cryptic error messages, like >>> missing libraries on the linker invocation. >> >> I've taken a look at an ltmain.sh and a libtool, but the only two calls to >> 'file' I could find were for darwin and for win32. >> >>> >>> We could ensure that autotools based packages now all depend on >>> host-file. But It itself is an autotools package, so it's again a >>> chicken-n-egg issue. And even non-autotools package may use libtool. >>> >>> So, just require that 'file' is present on the host. >>> >>> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> >>> --- >>> support/dependencies/dependencies.sh | 6 ++++++ >>> 1 file changed, 6 insertions(+) >>> >>> diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh >>> index 3146401..f14fb79 100755 >>> --- a/support/dependencies/dependencies.sh >>> +++ b/support/dependencies/dependencies.sh >>> @@ -69,6 +69,12 @@ check_prog_host "which" >>> # Verify that sed is installed >>> check_prog_host "sed" >>> >>> +# 'file' must be present, otherwise libtool fails in incomprehensible >>> +# ways. For example, bandwidthd would fail to link with -lz, even >> >> bandwidthd doesn't even have a libtool... >> >> Could you give a concrete example of where libtool uses file? > > Damn... It seems I confused the netsnmp and the bandwidthd cases. > > If you look at netsnmp, they have all this blurb about file being > required for libtool otherwise it breaks, in configure.d/config_os_progs > And indeed, my minimalistic chroot does not have file installed. Well, I ran strace -f -e trace=execve make netsnmp and the only call to file was in the configure script... That said, adding file as a global dependency is probably the easiest way of dealing with it. In practice, file is anyway installed in default distro. Regards, Arnout > > And then you have bandwidthd that also fails miserably in that chroot, > even though the same configuration works on my machine. > > So I installed 'file' and then both now compile without any issue in > that minimalistic chroot. > > And then I did the commit log for this patch, and only accounted for the > reasons explained in netsnmp (without trying to validat etheir point, I > admit), but referenced bandwidthd. > > Sigh. > > Anyway, here's my minimalistic chroot: > debootstrap --arch=amd64 --variant buildd trusty $(pwd)/chroot > > and the list of installed packages is in attachment. > > Regards, > Yann E. MORIN. >
Hi Yann, On Fri, Nov 27, 2015 at 09:02:09PM +0100, Yann E. MORIN wrote: > When 'file' is missing on the host, libtool fails miserably. Packages > that use libtool will suddenly FTBFS, with cryptic error messages, like > missing libraries on the linker invocation. > > We could ensure that autotools based packages now all depend on > host-file. But It itself is an autotools package, so it's again a > chicken-n-egg issue. And even non-autotools package may use libtool. > > So, just require that 'file' is present on the host. The "Mandatory packages" section in docs/manual/prerequisite.txt should mention that. baruch
Baruch, All, On 2015-11-28 22:30 +0200, Baruch Siach spake thusly: > On Fri, Nov 27, 2015 at 09:02:09PM +0100, Yann E. MORIN wrote: > > When 'file' is missing on the host, libtool fails miserably. Packages > > that use libtool will suddenly FTBFS, with cryptic error messages, like > > missing libraries on the linker invocation. > > > > We could ensure that autotools based packages now all depend on > > host-file. But It itself is an autotools package, so it's again a > > chicken-n-egg issue. And even non-autotools package may use libtool. > > > > So, just require that 'file' is present on the host. > > The "Mandatory packages" section in docs/manual/prerequisite.txt should > mention that. Yes, sure. Thanks! :-) Regards, Yann E. MORIN.
Arnout, All, On 2015-11-28 00:34 +0100, Arnout Vandecappelle spake thusly: > On 28-11-15 00:04, Yann E. MORIN wrote: > > On 2015-11-27 22:19 +0100, Arnout Vandecappelle spake thusly: > >> On 27-11-15 21:02, Yann E. MORIN wrote: > >>> When 'file' is missing on the host, libtool fails miserably. Packages > >>> that use libtool will suddenly FTBFS, with cryptic error messages, like > >>> missing libraries on the linker invocation. [--SNIP--] > >> bandwidthd doesn't even have a libtool... > >> > >> Could you give a concrete example of where libtool uses file? > > > > Damn... It seems I confused the netsnmp and the bandwidthd cases. > > > > If you look at netsnmp, they have all this blurb about file being > > required for libtool otherwise it breaks, in configure.d/config_os_progs > > And indeed, my minimalistic chroot does not have file installed. > > Well, I ran > > strace -f -e trace=execve make netsnmp OK, so I have some more info on this. Not yet completely clear-cut, but here it goes... Target: mips64el Libraries: shared only Toolchain: http://autobuild.buildroot.org/toolchains/tarballs/br-mips64r6-n64-el-hf-2015.11-rc1-71-g90d1299.tar.bz2 The failing package is still bandwidthd, with messages like: /home/buildroot/autobuild/reproduce/O/host/usr/mips64el-buildroot-linux-gnu/sysroot/usr/lib/libpng16.a(pngrutil.o): In function `.L78': pngrutil.c:(.text+0x55c): undefined reference to `inflate' pngrutil.c:(.text+0x56c): undefined reference to `inflate' So, what strikes me odd here, is that it is trying to link to the static variant of libpng. And indeed, the shared variant is not installed. And it does not try to link with -lz, which is a dependency of libpng, not badnwidthd, so it is expected. In shared case, libz would be pulled because it is a NEEDED of libpng. (See not [0], too) Weird, considering that static are disabled, and we are supposed to only build shared libs. OK, let's see why libpng does not install a shared library: checking whether the /home/buildroot/autobuild/reproduce/O/host/usr/bin/mips64el-linux-gcc linker (/home/buildroot/autobuild/reproduce/O/host/usr/bin/mips64el-linux-ld -m elf) supports shared libraries... /home/buildroot/autobuild/reproduce/O/host/usr/bin/mips64el-linux-ld: unrecognised emulation mode: elf Supported emulations: elf32ltsmipn32 elf32btsmipn32 elf32ltsmip elf32btsmip elf64ltsmip elf64btsmip no Uh-oh... Weird. Ah, what's a few line above, then? ./configure: line 6838: /usr/bin/file: No such file or directory ./configure: line 6846: /usr/bin/file: No such file or directory ./configure: line 6854: /usr/bin/file: No such file or directory Haha... So, where do these come from? Looking at the output of ./configure, and at configure.ac, the 'file' errors are right between: AC_PROG_MAKE_SET and AC_MSG_CHECKING([that AWK works]) And guess what's in there? Hint: I already spoke about it... LT_INIT([win32-dll]) LT_PREREQ([2.4.2]) Ah... libtool... But then, what's really failing, that libtool requires 'file' for? case $host in [...] mips64*-*linux*) [...] (eval $ac_compile) 2>&5 [...] case `/usr/bin/file conftest.$ac_objext` in *32-bit*) emul="${emul}32" ;; *64-bit*) emul="${emul}64" ;; esac (the two other failures are similar) So, libtool injects code that checks the output of a compilation to see what type of file was generated. And of course, it fails miserably... So, the failing package, bandwidthd, is not the culprit. It is jut the trigger. The real failure is earlier in the dependency chain. Damn... I hope this is enough information to warant the commit log that libtool is foobared when 'file' is not present? Yes, I can add some more details now, but I will certainly not duplicate all that message in the commit log. [0] Now I wonder if we would have a similar build failure in a real static-only scenario... Afterall, libpng install a .la file with -lz in there... But of course, in our failing test, that .la file is not used, because we're not even supposed to attempt a static link, but a shared link. Regards, Yann E. MORIN.
On 27-11-15 21:02, Yann E. MORIN wrote: > When 'file' is missing on the host, libtool fails miserably. Packages > that use libtool will suddenly FTBFS, with cryptic error messages, like > missing libraries on the linker invocation. > > We could ensure that autotools based packages now all depend on > host-file. But It itself is an autotools package, so it's again a > chicken-n-egg issue. And even non-autotools package may use libtool. > > So, just require that 'file' is present on the host. Improved commit message: support/dependencies: ensure we have 'file' on the host Recently, the libtool.m4 autoconf macros that are bundled with the libtool package started using '/usr/bin/file' to determine the type of library that is generated by the toolchain. Packages that use this macro will fail in a rather dramatic way when /usr/bin/file is not present on the host: the package will still build but no shared library is generated, which in turn may cause build failures in other packages that link with it. For example, libpng's configure determines that it is not possible to build a shared library on MIPS64 because the expected output from 'file' is not present. Therefore, only a static libpng.a is built. Later, bandwithd links with -lpng but it doesn't use the pkg-config's Private-Libs (because it's not linking statically) and it doesn't have access to the NEEDED reference from the shared library. Therefore, it doesn't link with zlib and fails with pngrutil.c:(.text+0x55c): undefined reference to `inflate' Since we can expect more packages to behave like this in the future, just require that '/usr/bin/file' is present on the host unconditionally. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > --- > support/dependencies/dependencies.sh | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh > index 3146401..f14fb79 100755 > --- a/support/dependencies/dependencies.sh > +++ b/support/dependencies/dependencies.sh > @@ -69,6 +69,12 @@ check_prog_host "which" > # Verify that sed is installed > check_prog_host "sed" > > +# 'file' must be present, otherwise libtool fails in incomprehensible > +# ways. For example, bandwidthd would fail to link with -lz, even > +# though the library is there (and it's not a static/shared issue). > +# Adding the 'file' program on the system miraculously fixes the issue. > +check_prog_host "file" This is actually not good enough: it really has to be /usr/bin/file (so adding host-file would in fact not help at all). > + > # Check make > MAKE=$(which make 2> /dev/null) > if [ -z "$MAKE" ] ; then >
diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh index 3146401..f14fb79 100755 --- a/support/dependencies/dependencies.sh +++ b/support/dependencies/dependencies.sh @@ -69,6 +69,12 @@ check_prog_host "which" # Verify that sed is installed check_prog_host "sed" +# 'file' must be present, otherwise libtool fails in incomprehensible +# ways. For example, bandwidthd would fail to link with -lz, even +# though the library is there (and it's not a static/shared issue). +# Adding the 'file' program on the system miraculously fixes the issue. +check_prog_host "file" + # Check make MAKE=$(which make 2> /dev/null) if [ -z "$MAKE" ] ; then
When 'file' is missing on the host, libtool fails miserably. Packages that use libtool will suddenly FTBFS, with cryptic error messages, like missing libraries on the linker invocation. We could ensure that autotools based packages now all depend on host-file. But It itself is an autotools package, so it's again a chicken-n-egg issue. And even non-autotools package may use libtool. So, just require that 'file' is present on the host. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> --- support/dependencies/dependencies.sh | 6 ++++++ 1 file changed, 6 insertions(+)