[v2,10/12] core: allow to use Clang as cross-compiler

Add this new experimental option into "Advanced" menu since
we are expecting some build and runtime issues for number of
packages (bootloaders, kernel).

Add host-clang as dependency in toolchain virtual package
in order to be sure that the clang cross-compiler is built
before generating Buildroot.cmake file from an
TOOLCHAIN_CMAKE_INSTALL_FILES hook (defined in cmake package).
Buildroot.cmake can be used by a cmake package as soon as
the toolchain package is installed.

Install the GCC external toolchain before executing
realpath command by adding an explicit dependency on
toolchain-external. Without this dependency, host-clang
may be build before toolchain-external when
BR2_USER_HOST_CLANG_AS_CROSS_COMPILER=y, so the realpath
fail to set GCC_INSTALL_PREFIX leading to
"crtbegin.o no such file or directory" error while linking
applications.

Only Glibc toolchain has been tested.

Signed-off-by: Romain Naour <romain.naour@smile.fr>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Valentin Korenblit <valentinkorenblit@gmail.com>
---
v2: fix typo TOOLCHAIN_{BUILDROOT,EXTERNAL}_DEPENDENCIES in toolchain-external.mk
    Install the GCC external toolchain before executing realpath command.
    Move host-clang dependency to toolchain virtual package since host-clang now
    depends on external-toolchain.
    Add Glibc toolchain dependency
---
 Config.in                        | 23 +++++++++++++++++++++++
 package/Makefile.in              | 13 ++++++++++---
 package/clang/clang.mk           |  7 +++++++
 toolchain/toolchain/toolchain.mk |  7 +++++++
 4 files changed, 47 insertions(+), 3 deletions(-)

Romain,

On Sat, Sep 7, 2019 at 4:40 AM Romain Naour <romain.naour@smile.fr> wrote:
>
> Add this new experimental option into "Advanced" menu since
> we are expecting some build and runtime issues for number of
> packages (bootloaders, kernel).
>
> Add host-clang as dependency in toolchain virtual package
> in order to be sure that the clang cross-compiler is built
> before generating Buildroot.cmake file from an
> TOOLCHAIN_CMAKE_INSTALL_FILES hook (defined in cmake package).
> Buildroot.cmake can be used by a cmake package as soon as
> the toolchain package is installed.
>
> Install the GCC external toolchain before executing
> realpath command by adding an explicit dependency on
> toolchain-external. Without this dependency, host-clang
> may be build before toolchain-external when
> BR2_USER_HOST_CLANG_AS_CROSS_COMPILER=y, so the realpath
> fail to set GCC_INSTALL_PREFIX leading to
> "crtbegin.o no such file or directory" error while linking
> applications.
>
> Only Glibc toolchain has been tested.
>
> Signed-off-by: Romain Naour <romain.naour@smile.fr>
> Cc: Matt Weber <matthew.weber@rockwellcollins.com>
> Cc: Valentin Korenblit <valentinkorenblit@gmail.com>

Did a build time test using the qemu aarch64 configuration, selecting
the prebuilt external toolchain and enabling the Clang as
cross-compiler option.

Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>

So, this is the one I don't like...

On 07/09/2019 11:40, Romain Naour wrote:
> Add this new experimental option into "Advanced" menu since
> we are expecting some build and runtime issues for number of
> packages (bootloaders, kernel).
> 
> Add host-clang as dependency in toolchain virtual package
> in order to be sure that the clang cross-compiler is built
> before generating Buildroot.cmake file from an
> TOOLCHAIN_CMAKE_INSTALL_FILES hook (defined in cmake package).
> Buildroot.cmake can be used by a cmake package as soon as
> the toolchain package is installed.
> 
> Install the GCC external toolchain before executing
> realpath command by adding an explicit dependency on
> toolchain-external. Without this dependency, host-clang
> may be build before toolchain-external when
> BR2_USER_HOST_CLANG_AS_CROSS_COMPILER=y, so the realpath
> fail to set GCC_INSTALL_PREFIX leading to
> "crtbegin.o no such file or directory" error while linking
> applications.
> 
> Only Glibc toolchain has been tested.

 As far as I understand, the *only* reason we still need GCC is for libgcc and
libstdc++ (and for building glibc). The intention is to build all packages with
clang instead of gcc.

 Therefore, I think it would be better to really treat clang as part of the
internal toolchain. That means:

- In the Config.in, we put it as an option of the internal toolchain. Before the
GCC version, you select clang or GCC. If clang is chosen, we choose a fixed
version of GCC (e.g. our current default).

- In the build, we replace host-gcc-final with host-clang as the
toolchain-buildroot dependency.

 I think that that approach, combined with completely replacing the gcc symlinks
with links to clang, will end up being simpler and more elegant than this patch.
And of course, more user friendly.

 There are two caveats:
- we no longer have an "escape clause" that allows us to build packages with GCC
if they really don't work with clang (except for glibc, but that's part of the
toolchain);
- it might be impossible to build libstdc++ as part of host-gcc-initial, so we
may still need a host-gcc-final stage (or maybe a specific host-gcc-libstdc++)
to build only libstdc++. But I do believe it's possible to build only libstdc++
by simply setting _SUBDIR to libstdc++-v3.

 So it would mean that to use clang as a cross-compiler, you always have to use
an internal toolchain. Later, of course, we should extend this with supporting
clang as an external toolchain as well.

 Also (but again, can be done later), we should revisit the
BR2_TOOLCHAIN_GCC_AT_LEAST_ options. That's actually the toughest one... *Most*
of them are related to the C++ support. Again in most cases, this is provided by
libstdc++ (most language extensions are to the standard library and not to the
core language), but in some cases it may actually depend on the compiler itself.
For the case where it's libstdc++ that is the limitation, we can simply set the
appropriate BR2_TOOLCHAIN_GCC_AT_LEAST_ from clang. For the other cases, we'll
have to add a BR2_TOOLCHAIN_CLANG option, and modify the packages to
BR2_TOOLCHAIN_GCC_AT_LEAST_... || BR2_TOOLCHAIN_CLANG. Except of course that
nobody will notice that it actually works with clang and an older libstdc++, so
that will probably never happen :-). More importantly, though, we can set
!BR2_TOOLCHAIN_CLANG for packages that really don't want clang. And I do prefer
that over falling back to GCC for those packages.

> 
> Signed-off-by: Romain Naour <romain.naour@smile.fr>
> Cc: Matt Weber <matthew.weber@rockwellcollins.com>
> Cc: Valentin Korenblit <valentinkorenblit@gmail.com>
> ---
> v2: fix typo TOOLCHAIN_{BUILDROOT,EXTERNAL}_DEPENDENCIES in toolchain-external.mk
>     Install the GCC external toolchain before executing realpath command.
>     Move host-clang dependency to toolchain virtual package since host-clang now
>     depends on external-toolchain.
>     Add Glibc toolchain dependency
> ---
>  Config.in                        | 23 +++++++++++++++++++++++
>  package/Makefile.in              | 13 ++++++++++---
>  package/clang/clang.mk           |  7 +++++++
>  toolchain/toolchain/toolchain.mk |  7 +++++++
>  4 files changed, 47 insertions(+), 3 deletions(-)
> 
> diff --git a/Config.in b/Config.in
> index a6e280b885..dc2e5f7320 100644
> --- a/Config.in
> +++ b/Config.in
> @@ -711,6 +711,29 @@ config BR2_REPRODUCIBLE
>  	  This is labeled as an experimental feature, as not all
>  	  packages behave properly to ensure reproducibility.
>  
> +config BR2_USER_HOST_CLANG_AS_CROSS_COMPILER
> +	bool "Use Clang as cross-compiler (experimental)"
> +	depends on BR2_TOOLCHAIN_USES_GLIBC
> +	depends on BR2_PACKAGE_LLVM_ARCH_SUPPORTS
> +	depends on BR2_HOST_GCC_AT_LEAST_4_8
> +	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8
> +	depends on BR2_TOOLCHAIN_HAS_THREADS
> +	depends on BR2_INSTALL_LIBSTDCPP
> +	depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_64735 # std::shared_future
> +	depends on !BR2_STATIC_LIBS
> +	depends on BR2_USE_WCHAR # std::wstring

 Again, I doubt that most of these limitations apply.

> +	select BR2_PACKAGE_HOST_CLANG
> +	help
> +	  This option will force Buildroot to use Clang as cross-compiler
> +	  instead of GCC. A GCC toolchain (internal or external) is still
> +	  needed to build with Clang.
> +
> +	  This is labeled as an experimental feature, since we are
> +	  expecting some build and runtime issues for number of packages
> +	  (bootloaders, kernel).
> +
> +	  Note: Only Glibc toolchain has been tested.
> +
>  endmenu
>  
>  comment "Security Hardening Options"
> diff --git a/package/Makefile.in b/package/Makefile.in
> index 0a7899c852..aebf4edd1b 100644
> --- a/package/Makefile.in
> +++ b/package/Makefile.in
> @@ -200,9 +200,6 @@ endif
>  # Define TARGET_xx variables for all common binutils/gcc
>  TARGET_AR       = $(TARGET_CROSS)ar
>  TARGET_AS       = $(TARGET_CROSS)as
> -TARGET_CC       = $(TARGET_CROSS)gcc
> -TARGET_CPP      = $(TARGET_CROSS)cpp
> -TARGET_CXX      = $(TARGET_CROSS)g++
>  TARGET_FC       = $(TARGET_CROSS)gfortran
>  TARGET_LD       = $(TARGET_CROSS)ld
>  TARGET_NM       = $(TARGET_CROSS)nm
> @@ -211,6 +208,16 @@ TARGET_READELF  = $(TARGET_CROSS)readelf
>  TARGET_OBJCOPY  = $(TARGET_CROSS)objcopy
>  TARGET_OBJDUMP  = $(TARGET_CROSS)objdump
>  
> +ifeq ($(BR2_USER_HOST_CLANG_AS_CROSS_COMPILER),y)
> +TARGET_CC       = $(HOST_DIR)/bin/clang
> +TARGET_CPP      = $(HOST_DIR)/bin/clang-cpp
> +TARGET_CXX      = $(HOST_DIR)/bin/clang++
> +else
> +TARGET_CC       = $(TARGET_CROSS)gcc
> +TARGET_CPP      = $(TARGET_CROSS)cpp
> +TARGET_CXX      = $(TARGET_CROSS)g++
> +endif

 This stuff is also not needed if we simply overwrite gcc, cpp, g++ and other
gcc-installed symlinks.

> +
>  ifeq ($(BR2_STRIP_strip),y)
>  STRIP_STRIP_DEBUG := --strip-debug
>  TARGET_STRIP = $(TARGET_CROSS)strip
> diff --git a/package/clang/clang.mk b/package/clang/clang.mk
> index 148df8e611..36703f5d8a 100644
> --- a/package/clang/clang.mk
> +++ b/package/clang/clang.mk
> @@ -14,6 +14,11 @@ CLANG_LICENSE_FILES = LICENSE.TXT
>  CLANG_SUPPORTS_IN_SOURCE_BUILD = NO
>  CLANG_INSTALL_STAGING = YES
>  
> +# Allow host-clang to be build as part of the toolchain
> +ifeq ($(BR2_USER_HOST_CLANG_AS_CROSS_COMPILER),y)
> +HOST_CLANG_ADD_TOOLCHAIN_DEPENDENCY = NO

 Since this is a host package, a toolchain dependency won't be added anyway.
Note how host-gcc-{initial,final} don't have it either.


 Regards,
 Arnout

> +endif
> +
>  HOST_CLANG_DEPENDENCIES = host-llvm host-libxml2
>  CLANG_DEPENDENCIES = llvm host-clang
>  
> @@ -106,7 +111,9 @@ CLANG_CONF_OPTS += -DLLVM_DYLIB_COMPONENTS=all
>  # Help host-clang to find our external toolchain, use a relative path from the clang
>  # installation directory to the external toolchain installation directory in order to
>  # not hardcode the toolchain absolute path.
> +# Install the GCC external toolchain before executing realpath command.
>  ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
> +HOST_CLANG_DEPENDENCIES += toolchain-external
>  HOST_CLANG_CONF_OPTS += -DGCC_INSTALL_PREFIX:PATH=`realpath --relative-to=$(HOST_DIR)/bin/ $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)`
>  endif
>  
> diff --git a/toolchain/toolchain/toolchain.mk b/toolchain/toolchain/toolchain.mk
> index 17fb62147e..d945e27aac 100644
> --- a/toolchain/toolchain/toolchain.mk
> +++ b/toolchain/toolchain/toolchain.mk
> @@ -10,6 +10,13 @@ else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)
>  TOOLCHAIN_DEPENDENCIES += toolchain-external
>  endif
>  
> +# Make sure the clang cross-compiler is built before
> +# generating Buildroot.cmake file from an
> +# TOOLCHAIN_CMAKE_INSTALL_FILES hook.
> +ifeq ($(BR2_USER_HOST_CLANG_AS_CROSS_COMPILER),y)
> +TOOLCHAIN_DEPENDENCIES += host-clang
> +endif
> +
>  TOOLCHAIN_ADD_TOOLCHAIN_DEPENDENCY = NO
>  
>  # Apply a hack that Rick Felker suggested[1] to avoid conflicts between libc
>