| Message ID | 1436949545-12487-1-git-send-email-patrickdepinguin@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
On Wed, Jul 15, 2015 at 10:39 AM, Thomas De Schampheleire <patrickdepinguin@gmail.com> wrote: > From: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> > > Currently, following symbolic links are created in both target and > staging directories: > - lib(32|64) --> lib > - usr/lib(32|64) --> lib > > The decision for lib32 or lib64 is based on the target architecture > configuration in buildroot (BR2_ARCH_IS_64). > > In at least one case this is not correct: when building for a Cavium Octeon > III processor using the toolchain from the Cavium Networks SDK, and > specifying -march=octeon3 in BR2_TARGET_OPTIMIZATION, libraries are expected > in directory 'lib32-fp' rather than 'lib32' (likewise for lib64-fp). > > More generally, for external toolchains, the correct symbolic link is > from (usr/)${ARCH_LIB_DIR} to lib. For internal toolchains, current > toolchains always use either lib32 or lib64. > > Fix the problem as follows: > - create a symlink creation helper in toolchain/helpers.mk > - for external toolchains, call these helpers based on ARCH_LIB_DIR > - for internal toolchains, call these helpers based on the existing > fixed lib32/lib64 logic, moved from Makefile > > Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> > --- > Makefile | 10 ---------- > package/skeleton/skeleton.mk | 3 --- > toolchain/helpers.mk | 13 +++++++++++++ > toolchain/toolchain-buildroot/toolchain-buildroot.mk | 18 ++++++++++++++++++ > toolchain/toolchain-external/toolchain-external.mk | 12 ++++++++++++ > 5 files changed, 43 insertions(+), 13 deletions(-) > > Note: in output/staging/usr/ there would still be more directories than I > think are really necessary. This behavior is not changed by this patch, it > was already present before. > For example, with the mentioned Octeon III toolchain, output/staging/usr/ > contains: > bin bin32 bin32-fp bin64-fp, > lib lib32-fp lib64-fp > libexec libexec32 libexec32-fp libexec64-fp > sbin sbin32 sbin32-fp sbin64-fp > > where bin/lib/libexec/sbin seem to be the 64-bit equivalents of > bin32/lib32/libexec32/sbin32. > This is related to the behavior of copy_toolchain_sysroot in > toolchain/helpers.mk. It already attempts to filter out lib32 and lib64, but > does not care about any bin/sbin/libexec directories, nor about such names > as lib32-fp. > As the behavior is not changed by this patch, and as I'm not fully aware > about which directories are really needed in all cases, I am not touching > this area. Sorry, forgot to add the subject flag v4.
Thomas, On Wed, 15 Jul 2015 10:39:05 +0200, Thomas De Schampheleire wrote: > diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk > index b30cc33..30c50d6 100644 > --- a/toolchain/toolchain-buildroot/toolchain-buildroot.mk > +++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk > @@ -14,4 +14,22 @@ TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final > > TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO > > +# The creation of lib32/lib64 symlinks into target and staging directories > +# needs to be done before the C library is installed. Hooking into the libc > +# hooks directly is tricky because there are multiple C libraries supported. > +# Instead, hook into the install step of host-gcc-initial. > +# > +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. > +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) > +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 > +else > +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 > +endif > + > +define TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK > + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(STAGING_DIR)) > + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(TARGET_DIR)) > +endef > +HOST_GCC_INITIAL_POST_INSTALL_HOOKS += TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK If it's a host-gcc-initial hook, then it should go in the gcc/gcc-initial/gcc-initial.mk file. Thomas
On Wed, Jul 15, 2015 at 10:55 AM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: > Thomas, > > On Wed, 15 Jul 2015 10:39:05 +0200, Thomas De Schampheleire wrote: > >> diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> index b30cc33..30c50d6 100644 >> --- a/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> +++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> @@ -14,4 +14,22 @@ TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final >> >> TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO >> >> +# The creation of lib32/lib64 symlinks into target and staging directories >> +# needs to be done before the C library is installed. Hooking into the libc >> +# hooks directly is tricky because there are multiple C libraries supported. >> +# Instead, hook into the install step of host-gcc-initial. >> +# >> +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. >> +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) >> +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 >> +else >> +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 >> +endif >> + >> +define TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK >> + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(STAGING_DIR)) >> + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(TARGET_DIR)) >> +endef >> +HOST_GCC_INITIAL_POST_INSTALL_HOOKS += TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK > > If it's a host-gcc-initial hook, then it should go in the > gcc/gcc-initial/gcc-initial.mk file. > Well that is part of the discussion: conceptually I believe that the symlink creation is part of the toolchain as a whole. It needs to be done before libc, but actually all remaining toolchain components will probably rely on it, and also the packages. By placing it in gcc-initial I believe it is pretty hidden. Moreover, suppose we add support for the llvm/clang compiler. For such toolchains, we'd also need the symbolic links, so that code would have to be duplicated... /Thomas
On 07/15/15 10:55, Thomas Petazzoni wrote: > Thomas, > > On Wed, 15 Jul 2015 10:39:05 +0200, Thomas De Schampheleire wrote: > >> diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> index b30cc33..30c50d6 100644 >> --- a/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> +++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk >> @@ -14,4 +14,22 @@ TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final >> >> TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO >> >> +# The creation of lib32/lib64 symlinks into target and staging directories >> +# needs to be done before the C library is installed. Hooking into the libc >> +# hooks directly is tricky because there are multiple C libraries supported. >> +# Instead, hook into the install step of host-gcc-initial. >> +# >> +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. >> +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) >> +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 >> +else >> +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 >> +endif >> + >> +define TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK >> + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(STAGING_DIR)) >> + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(TARGET_DIR)) >> +endef >> +HOST_GCC_INITIAL_POST_INSTALL_HOOKS += TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK > > If it's a host-gcc-initial hook, then it should go in the > gcc/gcc-initial/gcc-initial.mk file. Neither is really perfect. It is not really related to gcc-initial, so putting it in gcc-initial.mk is not correct either. It really _is_ related to the buildroot toolchain as a whole, so in that sense it fits here. But on the other hand, indeed it is not nice that when you look at the gcc-initial.mk, you don't see the complete picture. Bottom line: either way works for me. Regards, Arnout
Hello, On Wed, 15 Jul 2015 10:57:52 +0200, Thomas De Schampheleire wrote: > Well that is part of the discussion: conceptually I believe that the > symlink creation is part of the toolchain as a whole. It needs to be > done before libc, but actually all remaining toolchain components will > probably rely on it, and also the packages. > By placing it in gcc-initial I believe it is pretty hidden. It is also pretty hidden to put the hook for a given package outside of this package .mk file. > Moreover, suppose we add support for the llvm/clang compiler. For such > toolchains, we'd also need the symbolic links, so that code would have > to be duplicated... I don't think trying to take llvm/clang into account really makes sense here. It's probably going to be so much different than what we have today that lots of things would have to be reworked anyway. Thomas
On 07/15/15 10:59, Thomas Petazzoni wrote: > Hello, > > On Wed, 15 Jul 2015 10:57:52 +0200, Thomas De Schampheleire wrote: > >> Well that is part of the discussion: conceptually I believe that the >> symlink creation is part of the toolchain as a whole. It needs to be >> done before libc, but actually all remaining toolchain components will >> probably rely on it, and also the packages. >> By placing it in gcc-initial I believe it is pretty hidden. > > It is also pretty hidden to put the hook for a given package outside of > this package .mk file. > >> Moreover, suppose we add support for the llvm/clang compiler. For such >> toolchains, we'd also need the symbolic links, so that code would have >> to be duplicated... > > I don't think trying to take llvm/clang into account really makes sense > here. It's probably going to be so much different than what we have > today that lots of things would have to be reworked anyway. Just to be clear: for me this is a bikeshedding discussion. Neither is right, so just go for one. BTW, yesterday I did ask everyone in Toulouse if it would be OK have add to the gcc-final hook from toolchain-buildroot.mk, and nobody really opposed (though I don't think you really said anything, ThomasP). Regards, Arnout
On Wed, Jul 15, 2015 at 11:05 AM, Arnout Vandecappelle <arnout@mind.be> wrote: > On 07/15/15 10:59, Thomas Petazzoni wrote: >> Hello, >> >> On Wed, 15 Jul 2015 10:57:52 +0200, Thomas De Schampheleire wrote: >> >>> Well that is part of the discussion: conceptually I believe that the >>> symlink creation is part of the toolchain as a whole. It needs to be >>> done before libc, but actually all remaining toolchain components will >>> probably rely on it, and also the packages. >>> By placing it in gcc-initial I believe it is pretty hidden. >> >> It is also pretty hidden to put the hook for a given package outside of >> this package .mk file. >> >>> Moreover, suppose we add support for the llvm/clang compiler. For such >>> toolchains, we'd also need the symbolic links, so that code would have >>> to be duplicated... >> >> I don't think trying to take llvm/clang into account really makes sense >> here. It's probably going to be so much different than what we have >> today that lots of things would have to be reworked anyway. > > Just to be clear: for me this is a bikeshedding discussion. Neither is right, > so just go for one. > > BTW, yesterday I did ask everyone in Toulouse if it would be OK have add to the > gcc-final hook from toolchain-buildroot.mk, and nobody really opposed (though I > don't think you really said anything, ThomasP). > Suppose we do add the hook in gcc-initial, where would the link target be defined? So this code: +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 +else +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 +endif Do you think that belongs also in gcc-initial? Or should that stay in buildroot-toolchain? /Thomas
On 07/15/15 12:08, Thomas De Schampheleire wrote: > On Wed, Jul 15, 2015 at 11:05 AM, Arnout Vandecappelle <arnout@mind.be> wrote: >> On 07/15/15 10:59, Thomas Petazzoni wrote: >>> Hello, >>> >>> On Wed, 15 Jul 2015 10:57:52 +0200, Thomas De Schampheleire wrote: >>> >>>> Well that is part of the discussion: conceptually I believe that the >>>> symlink creation is part of the toolchain as a whole. It needs to be >>>> done before libc, but actually all remaining toolchain components will >>>> probably rely on it, and also the packages. >>>> By placing it in gcc-initial I believe it is pretty hidden. >>> >>> It is also pretty hidden to put the hook for a given package outside of >>> this package .mk file. >>> >>>> Moreover, suppose we add support for the llvm/clang compiler. For such >>>> toolchains, we'd also need the symbolic links, so that code would have >>>> to be duplicated... >>> >>> I don't think trying to take llvm/clang into account really makes sense >>> here. It's probably going to be so much different than what we have >>> today that lots of things would have to be reworked anyway. >> >> Just to be clear: for me this is a bikeshedding discussion. Neither is right, >> so just go for one. >> >> BTW, yesterday I did ask everyone in Toulouse if it would be OK have add to the >> gcc-final hook from toolchain-buildroot.mk, and nobody really opposed (though I >> don't think you really said anything, ThomasP). >> > > Suppose we do add the hook in gcc-initial, where would the link target > be defined? So this code: > > +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. > +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) > +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 > +else > +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 > +endif > > > Do you think that belongs also in gcc-initial? Or should that stay in > buildroot-toolchain? IMHO it should certainly stay together with the piece of code that actually uses it - which is the point of putting it into gcc-initial to begin with (otherwise the adding of the hook is not in the same file as the rest of gcc-initial). And of course, it should be called GCC_INITIAL_* then. Regards, Arnout
diff --git a/Makefile b/Makefile index 531ac5d..c1fec47 100644 --- a/Makefile +++ b/Makefile @@ -444,20 +444,10 @@ world: target-post-image $(BUILD_DIR) $(TARGET_DIR) $(HOST_DIR) $(BINARIES_DIR) $(LEGAL_INFO_DIR) $(REDIST_SOURCES_DIR_TARGET) $(REDIST_SOURCES_DIR_HOST): @mkdir -p $@ -# We make a symlink lib32->lib or lib64->lib as appropriate -# MIPS64/n32 requires lib32 even though it's a 64-bit arch. -ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) -LIB_SYMLINK = lib64 -else -LIB_SYMLINK = lib32 -endif - $(STAGING_DIR): @mkdir -p $(STAGING_DIR)/bin @mkdir -p $(STAGING_DIR)/lib - @ln -snf lib $(STAGING_DIR)/$(LIB_SYMLINK) @mkdir -p $(STAGING_DIR)/usr/lib - @ln -snf lib $(STAGING_DIR)/usr/$(LIB_SYMLINK) @mkdir -p $(STAGING_DIR)/usr/include @mkdir -p $(STAGING_DIR)/usr/bin @ln -snf $(STAGING_DIR) $(BASE_DIR)/staging diff --git a/package/skeleton/skeleton.mk b/package/skeleton/skeleton.mk index 48e7085..2481bd1 100644 --- a/package/skeleton/skeleton.mk +++ b/package/skeleton/skeleton.mk @@ -25,9 +25,6 @@ define SKELETON_INSTALL_TARGET_CMDS $(SKELETON_PATH)/ $(TARGET_DIR)/ $(INSTALL) -m 0644 support/misc/target-dir-warning.txt \ $(TARGET_DIR_WARNING_FILE) - ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK) - mkdir -p $(TARGET_DIR)/usr - ln -snf lib $(TARGET_DIR)/usr/$(LIB_SYMLINK) endef SKELETON_TARGET_GENERIC_HOSTNAME = $(call qstrip,$(BR2_TARGET_GENERIC_HOSTNAME)) diff --git a/toolchain/helpers.mk b/toolchain/helpers.mk index 895f3f1..292a865 100644 --- a/toolchain/helpers.mk +++ b/toolchain/helpers.mk @@ -1,5 +1,18 @@ # This Makefile fragment declares toolchain related helper functions. +# Create the necessary symlink from (usr/)lib32|lib64|lib32-fp|... to lib +# In general, for external toolchains, the correct link name is $ARCH_LIB_DIR. +# $1: link name +# $2: destination directory (TARGET_DIR / STAGING_DIR) +create_lib_symlinks = \ + LIB_SYMLINK="$(strip $1)" ; \ + DESTDIR="$(strip $2)" ; \ + if [ "$${LIB_SYMLINK}" != "lib" ]; then \ + mkdir -p "$${DESTDIR}/usr" ; \ + ln -snf lib "$${DESTDIR}/$${LIB_SYMLINK}" ; \ + ln -snf lib "$${DESTDIR}/usr/$${LIB_SYMLINK}" ; \ + fi + # The copy_toolchain_lib_root function copies a toolchain library and # its symbolic links from the sysroot directory to the target # directory. Note that this function is used both by the external diff --git a/toolchain/toolchain-buildroot/toolchain-buildroot.mk b/toolchain/toolchain-buildroot/toolchain-buildroot.mk index b30cc33..30c50d6 100644 --- a/toolchain/toolchain-buildroot/toolchain-buildroot.mk +++ b/toolchain/toolchain-buildroot/toolchain-buildroot.mk @@ -14,4 +14,22 @@ TOOLCHAIN_BUILDROOT_DEPENDENCIES = host-gcc-final TOOLCHAIN_BUILDROOT_ADD_TOOLCHAIN_DEPENDENCY = NO +# The creation of lib32/lib64 symlinks into target and staging directories +# needs to be done before the C library is installed. Hooking into the libc +# hooks directly is tricky because there are multiple C libraries supported. +# Instead, hook into the install step of host-gcc-initial. +# +# MIPS64/n32 requires lib32 even though it's a 64-bit arch. +ifeq ($(BR2_ARCH_IS_64)$(BR2_MIPS_NABI32),y) +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib64 +else +TOOLCHAIN_BUILDROOT_LIB_SYMLINK = lib32 +endif + +define TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(STAGING_DIR)) + $(call create_lib_symlinks,$(TOOLCHAIN_BUILDROOT_LIB_SYMLINK),$(TARGET_DIR)) +endef +HOST_GCC_INITIAL_POST_INSTALL_HOOKS += TOOLCHAIN_BUILDROOT_CREATE_STAGING_TARGET_SYMLINK + $(eval $(virtual-package)) diff --git a/toolchain/toolchain-external/toolchain-external.mk b/toolchain/toolchain-external/toolchain-external.mk index fcb033c..348e90f 100644 --- a/toolchain/toolchain-external/toolchain-external.mk +++ b/toolchain/toolchain-external/toolchain-external.mk @@ -615,6 +615,16 @@ define TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS $(call copy_toolchain_sysroot,$${SYSROOT_DIR},$${ARCH_SYSROOT_DIR},$${ARCH_SUBDIR},$${ARCH_LIB_DIR},$${SUPPORT_LIB_DIR}) endef +define TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK + $(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \ + $(call create_lib_symlinks,$${ARCH_LIB_DIR},$(STAGING_DIR)) +endef + +define TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK + $(Q)ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \ + $(call create_lib_symlinks,$${ARCH_LIB_DIR},$(TARGET_DIR)) +endef + # Special installation target used on the Blackfin architecture when # FDPIC is not the primary binary format being used, but the user has # nonetheless requested the installation of the FDPIC libraries to the @@ -733,6 +743,7 @@ define TOOLCHAIN_EXTERNAL_FIXUP_UCLIBCNG_LDSO endef define TOOLCHAIN_EXTERNAL_INSTALL_STAGING_CMDS + $(TOOLCHAIN_EXTERNAL_CREATE_STAGING_LIB_SYMLINK) $(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS) $(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER) $(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT) @@ -742,6 +753,7 @@ endef # and the target directory, we do everything within the # install-staging step, arbitrarily. define TOOLCHAIN_EXTERNAL_INSTALL_TARGET_CMDS + $(TOOLCHAIN_EXTERNAL_CREATE_TARGET_LIB_SYMLINK) $(TOOLCHAIN_EXTERNAL_INSTALL_TARGET_LIBS) $(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FDPIC) $(TOOLCHAIN_EXTERNAL_INSTALL_BFIN_FLAT)