| Message ID | 20240408205742.1422545-1-yann.morin.1998@free.fr |
|---|---|
| State | Accepted |
| Headers | show |
| Series | toolchain/external: remove libcrypt from glibc | expand |
Hello Yann On Mon, 8 Apr 2024 22:57:42 +0200 "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: > Note: this commit only deals with glibc and its internal libcrypt (or > lack thereof); other C libraries, musl and uClibc-NG, are not considered. [snip] This all looks reasonable to me. One comment below, though. > ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y) > TOOLCHAIN_EXTERNAL_LIBS += libnss_files.so.* libnss_dns.so.* libmvec.so.* libanl.so.* > +# Note: explicitly do copy libcrypt*.so.* even for glibc: it is not the same Are you sure you wanted to say libcrypt*.so.* here? Didn't you want to say libcrypt.so.1 ? If you wanted to say libcrypt.so.1, then the patch is fine for me and you can add my Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Best regards, Thomas
Thomas, All, On 2024-04-09 21:51 +0200, Thomas Petazzoni spake thusly: > On Mon, 8 Apr 2024 22:57:42 +0200 > "Yann E. MORIN" <yann.morin.1998@free.fr> wrote: > > Note: this commit only deals with glibc and its internal libcrypt (or > > lack thereof); other C libraries, musl and uClibc-NG, are not considered. [--SNIP--] > > ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y) > > TOOLCHAIN_EXTERNAL_LIBS += libnss_files.so.* libnss_dns.so.* libmvec.so.* libanl.so.* > > +# Note: explicitly do copy libcrypt*.so.* even for glibc: it is not the same > Are you sure you wanted to say libcrypt*.so.* here? Didn't you want to > say libcrypt.so.1 ? Damned, I missed that. Yes, this is about "libcrypt.so.1" explicitly. > If you wanted to say libcrypt.so.1, then the patch is fine for me and > you can add my > > Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Good, I'll respin. Regards, Yann E. MORIN. > Best regards, > > Thomas > -- > Thomas Petazzoni, co-owner and CEO, Bootlin > Embedded Linux and Kernel engineering and training > https://bootlin.com
All, On 2024-04-08 22:57 +0200, Yann E. MORIN spake thusly: > Note: this commit only deals with glibc and its internal libcrypt (or > lack thereof); other C libraries, musl and uClibc-NG, are not considered. [--SNIP--] > So, we remove from staging whatever could be used to compile and link > with libcrypt. We however keep the SO_NAME file, if it exists, and we > also install it in target/, for those pre-built binaries that may be > linked with it [1]. The glibc SO_NAME has always been libcrypt.so.1, so > this is what we copy exactly, to avoid copying the libxcrypt one, which > is libcrypt.so.2. > > [0] that could happen if a toolchain provider tried to be helpful and > suplies a toolchain with libxcrypt to be trasnparent to users, in which > case that would conflict with ours... > > [1] if such a prebuilt binary (executable or library) is used with a > glibc 2.39 or later toolchain, it will obviously not work at all. > libxcrypt is supposed to be a drop-in replacement for glibc's libcrypt, > so we could look into symlinking libcrypt.so.1 to libcrypt.so.2. In a > later patch, maybe... > > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> > Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com> Applied to master, with the fixes pointed out by Thomas, thanks. Regards, Yann E. MORIN. > --- > .../pkg-toolchain-external.mk | 18 +++++++++++++++++- > 1 file changed, 17 insertions(+), 1 deletion(-) > > diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk > index ddf1d70ab4..9b0652c4e5 100644 > --- a/toolchain/toolchain-external/pkg-toolchain-external.mk > +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk > @@ -120,7 +120,7 @@ TOOLCHAIN_EXTERNAL_LIBS += libssp.so.* > endif > > ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y) > -TOOLCHAIN_EXTERNAL_LIBS += libc.so.* libcrypt.so.* libdl.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.* > +TOOLCHAIN_EXTERNAL_LIBS += libc.so.* libdl.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.* > ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y) > TOOLCHAIN_EXTERNAL_LIBS += libpthread.so.* > ifneq ($(BR2_PACKAGE_GDB)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),) > @@ -129,8 +129,23 @@ endif # gdbserver > endif # ! no threads > endif > > +ifeq ($(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y) > +# uClibc, though mono-lib, still has a separate libcrypt as a stub > +TOOLCHAIN_EXTERNAL_LIBS += libcrypt.so.* > +endif > + > ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y) > TOOLCHAIN_EXTERNAL_LIBS += libnss_files.so.* libnss_dns.so.* libmvec.so.* libanl.so.* > +# Note: explicitly do copy libcrypt*.so.* even for glibc: it is not the same > +# SONAME as the one from libxcrypt, so no conflict, but some prebuilt binaries > +# may have it in their DT_NEEDED. However, do remove the headers, static lib, > +# and symlink to avoid conflict, as they are not needed at buildtime. > +TOOLCHAIN_EXTERNAL_LIBS += libcrypt.so.1 > +define TOOLCHAIN_EXTERNAL_GLIBC_NO_LIBCRYPT > + rm -f $(STAGING_DIR)/usr/include/crypt.h \ > + $(STAGING_DIR)/usr/lib/libcrypt.so \ > + $(STAGING_DIR)/usr/lib/libcrypt.a > +endef > endif > > ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y) > @@ -599,6 +614,7 @@ define $(2)_INSTALL_STAGING_CMDS > $$(TOOLCHAIN_WRAPPER_INSTALL) > $$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK) > $$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS) > + $$(TOOLCHAIN_EXTERNAL_GLIBC_NO_LIBCRYPT) > $$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER) > $$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT) > $$(TOOLCHAIN_EXTERNAL_FIXUP_PRETTY_PRINTER_LOADER) > -- > 2.44.0 > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk index ddf1d70ab4..9b0652c4e5 100644 --- a/toolchain/toolchain-external/pkg-toolchain-external.mk +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk @@ -120,7 +120,7 @@ TOOLCHAIN_EXTERNAL_LIBS += libssp.so.* endif ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC)$(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y) -TOOLCHAIN_EXTERNAL_LIBS += libc.so.* libcrypt.so.* libdl.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.* +TOOLCHAIN_EXTERNAL_LIBS += libc.so.* libdl.so.* libm.so.* libnsl.so.* libresolv.so.* librt.so.* libutil.so.* ifeq ($(BR2_TOOLCHAIN_HAS_THREADS),y) TOOLCHAIN_EXTERNAL_LIBS += libpthread.so.* ifneq ($(BR2_PACKAGE_GDB)$(BR2_TOOLCHAIN_EXTERNAL_GDB_SERVER_COPY),) @@ -129,8 +129,23 @@ endif # gdbserver endif # ! no threads endif +ifeq ($(BR2_TOOLCHAIN_EXTERNAL_UCLIBC),y) +# uClibc, though mono-lib, still has a separate libcrypt as a stub +TOOLCHAIN_EXTERNAL_LIBS += libcrypt.so.* +endif + ifeq ($(BR2_TOOLCHAIN_EXTERNAL_GLIBC),y) TOOLCHAIN_EXTERNAL_LIBS += libnss_files.so.* libnss_dns.so.* libmvec.so.* libanl.so.* +# Note: explicitly do copy libcrypt*.so.* even for glibc: it is not the same +# SONAME as the one from libxcrypt, so no conflict, but some prebuilt binaries +# may have it in their DT_NEEDED. However, do remove the headers, static lib, +# and symlink to avoid conflict, as they are not needed at buildtime. +TOOLCHAIN_EXTERNAL_LIBS += libcrypt.so.1 +define TOOLCHAIN_EXTERNAL_GLIBC_NO_LIBCRYPT + rm -f $(STAGING_DIR)/usr/include/crypt.h \ + $(STAGING_DIR)/usr/lib/libcrypt.so \ + $(STAGING_DIR)/usr/lib/libcrypt.a +endef endif ifeq ($(BR2_TOOLCHAIN_EXTERNAL_MUSL),y) @@ -599,6 +614,7 @@ define $(2)_INSTALL_STAGING_CMDS $$(TOOLCHAIN_WRAPPER_INSTALL) $$(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK) $$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS) + $$(TOOLCHAIN_EXTERNAL_GLIBC_NO_LIBCRYPT) $$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER) $$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT) $$(TOOLCHAIN_EXTERNAL_FIXUP_PRETTY_PRINTER_LOADER)
Note: this commit only deals with glibc and its internal libcrypt (or lack thereof); other C libraries, musl and uClibc-NG, are not considered. libcrypt from glibc has been deprecated for a long time, and it has now been entirely dropped with glibc 2.39. Now, packages that need crypt(3) features need to explicitly depend on the libxcrypt pacakge. However, the set of files installed both by glibc and libxcrypt is not empty: glibc libxcrypt /usr/include/crypt.h /usr/include/crypt.h /usr/lib/libcrypt.a /usr/lib/libcrypt.a /usr/lib/libcrypt.so /usr/lib/libcrypt.so /lib/libcrypt.so.1 /lib/libcrypt-2.23.so /usr/lib/libcrypt.so.2 The two libraries have different SO_NAME, so they do not conflict on the library filename. However, the .so synlink is present in both, and thus conflicts. The header and the static library also conflict. So, the situation is that, with a glibc 2.39 or later, packages have to use libxcrypt, which is a drop-in replacement. With glibc 2.38 or earlier, they can use either. Since we already bumped to glibc 2.39 for the internal toolchain, we have already converted quite a few packages to use libxcrypt. That works well with an internl toolchain, because glibc does not install the conflicting files. However, for external toolchains, we may very well end up in three situations: - a glibc 2.39 or later, without libcrypt - a glibc 2.39 or later, without libcrypt, but with libxcrypt [0] - a glibc 2.38 or earlier with libcrypt In the first case, all is OK and we are in a situation similar to the internal toolchain, but in the latter two cases, we end up with a conflict. We could introduce BR2_TOOLCHAIN_EXTERNAL_HAS_LIBCRYPT os something along those lines, but this is going to be a bit complex on packages, which would have to select LIBXCRYPT if GLIBC && !_HAS_LIBCRYPT. So, to simplify things, we want to get the external toolchains into a situation similar to the internal one, where libcrypt is not provided by the toolchain; packages have to select libxcrypt for glibc toolchains, without having to care whether this is an internal or external toolchain or some more complex conditions. So, we remove from staging whatever could be used to compile and link with libcrypt. We however keep the SO_NAME file, if it exists, and we also install it in target/, for those pre-built binaries that may be linked with it [1]. The glibc SO_NAME has always been libcrypt.so.1, so this is what we copy exactly, to avoid copying the libxcrypt one, which is libcrypt.so.2. [0] that could happen if a toolchain provider tried to be helpful and suplies a toolchain with libxcrypt to be trasnparent to users, in which case that would conflict with ours... [1] if such a prebuilt binary (executable or library) is used with a glibc 2.39 or later toolchain, it will obviously not work at all. libxcrypt is supposed to be a drop-in replacement for glibc's libcrypt, so we could look into symlinking libcrypt.so.1 to libcrypt.so.2. In a later patch, maybe... Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr> Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com> --- .../pkg-toolchain-external.mk | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-)