[v4,08/11] package/clang: enable target variant

This patch provides Clang for the target, tools and libraries.

In this package we have to deal with the same kind of problem we encountered
when building mesa3d with llvm support: llvm-config (host-version) had to be
copied to STAGING_DIR/usr/bin. In this case, we need llvm-tblgen from host
to be installed in STAGING_DIR/usr/bin for cross-compiling clang.

We need libclang for the target because it is used by all OpenCL implementations.
In this series, Clover is provided.

Signed-off-by: Valentin Korenblit <valentin.korenblit@smile.fr>
---
 package/Config.in       |  1 +
 package/clang/Config.in |  8 ++++++++
 package/clang/clang.mk  | 30 ++++++++++++++++++++++++++++++
 3 files changed, 39 insertions(+)
 create mode 100644 package/clang/Config.in

Hello,

On Thu, 29 Mar 2018 13:33:43 +0200, Valentin Korenblit wrote:
> This patch provides Clang for the target, tools and libraries.
> 
> In this package we have to deal with the same kind of problem we encountered
> when building mesa3d with llvm support: llvm-config (host-version) had to be
> copied to STAGING_DIR/usr/bin. In this case, we need llvm-tblgen from host
> to be installed in STAGING_DIR/usr/bin for cross-compiling clang.
> 
> We need libclang for the target because it is used by all OpenCL implementations.
> In this series, Clover is provided.

I'm not sure to understand why you need Clang on the target, and what
Clover is. Maybe I missed it, but I don't see where the target Clang is
used in the rest of your series.

In addition, we don't support gcc on the target, so having clang on the
target raises a red flag for me.

Could you explain a bit more why this is needed/useful ?

Thanks!

Thomas

Hello Thomas,

On 30/03/2018 11:07, Thomas Petazzoni wrote:
> Hello,
>
> On Thu, 29 Mar 2018 13:33:43 +0200, Valentin Korenblit wrote:
>> This patch provides Clang for the target, tools and libraries.
>>
>> In this package we have to deal with the same kind of problem we encountered
>> when building mesa3d with llvm support: llvm-config (host-version) had to be
>> copied to STAGING_DIR/usr/bin. In this case, we need llvm-tblgen from host
>> to be installed in STAGING_DIR/usr/bin for cross-compiling clang.
>>
>> We need libclang for the target because it is used by all OpenCL implementations.
>> In this series, Clover is provided.
> I'm not sure to understand why you need Clang on the target, and what
> Clover is. Maybe I missed it, but I don't see where the target Clang is
> used in the rest of your series.

What we need is libclang, we don't need the binaries. But as libclang is a tool,
I need to set CLANG_BUILD_TOOLS to ON, what will build everything.

Clover is the OpenCL implementation provided with Mesa. It uses libclang as a frontend
for OpenCL C and libLLVM for code generation.

> In addition, we don't support gcc on the target, so having clang on the
> target raises a red flag for me.
>
> Could you explain a bit more why this is needed/useful ?

As explained above, we don't need it. I could add a hook to erase all unnecessary
binaries from the target if that is ok.

>
> Thanks!
>
> Thomas

Best regards,

Valentin

Hello,

On Fri, 30 Mar 2018 14:16:00 +0200, Valentin Korenblit wrote:

> What we need is libclang, we don't need the binaries. But as libclang is a tool,
> I need to set CLANG_BUILD_TOOLS to ON, what will build everything.
> 
> Clover is the OpenCL implementation provided with Mesa. It uses libclang as a frontend
> for OpenCL C and libLLVM for code generation.

OK.

> > In addition, we don't support gcc on the target, so having clang on the
> > target raises a red flag for me.
> >
> > Could you explain a bit more why this is needed/useful ?  
> 
> As explained above, we don't need it. I could add a hook to erase all unnecessary
> binaries from the target if that is ok.

That's indeed a possibility.

I was a bit confused, because I missed which patch was actually using
the target clang package, but I didn't read PATCH 10/11 carefully
enough, and it does use the target clang, so it makes perfect sense.

Adjusting the target clang package to remove stuff that is not used is
indeed a good idea.

Thanks!

Thomas

Valentin,

On Thu, Mar 29, 2018 at 6:33 AM, Valentin Korenblit
<valentin.korenblit@smile.fr> wrote:
> This patch provides Clang for the target, tools and libraries.
>
> In this package we have to deal with the same kind of problem we encountered
> when building mesa3d with llvm support: llvm-config (host-version) had to be
> copied to STAGING_DIR/usr/bin. In this case, we need llvm-tblgen from host
> to be installed in STAGING_DIR/usr/bin for cross-compiling clang.
>
> We need libclang for the target because it is used by all OpenCL implementations.
> In this series, Clover is provided.
>


I'm building a basic x86 system  (I can work on reproducing this
failure in a generic QEMU build if you need that).  My build just
enabled a host and target versions of llvm and clang (no sub options).

2018-03-30T12:39:37 /bin/sh: clang-tblgen: command not found
2018-03-30T12:39:37 make[4]: *** [lib/Headers/arm_neon.h.tmp] Error 127
2018-03-30T12:39:37 make[4]: *** Waiting for unfinished jobs....
2018-03-30T12:39:37 make[3]: ***
[lib/Headers/CMakeFiles/clang-headers.dir/all] Error 2
2018-03-30T12:39:37 make[2]: *** [all] Error 2
2018-03-30T12:39:37 make[1]: ***
[/accts/mlweber1//target_build/qemu_x86_defconfig/build/clang-5.0.1/.stamp_built]
Error 2
2018-03-30T12:39:37 make: *** [_all] Error 2

Hello,

On Thu, 29 Mar 2018 13:33:43 +0200, Valentin Korenblit wrote:
> This patch provides Clang for the target, tools and libraries.
> 
> In this package we have to deal with the same kind of problem we encountered
> when building mesa3d with llvm support: llvm-config (host-version) had to be
> copied to STAGING_DIR/usr/bin.

I don't see why it's a problem in mesa3d and in this package. Having
the <foo>-config stuff in $(STAGING_DIR) is the standard thing in
Buildroot, so there's nothing crazy or unusual about it.

> In this case, we need llvm-tblgen from host
> to be installed in STAGING_DIR/usr/bin for cross-compiling clang.

This is however a bit unusual, and perhaps we should keep llvm-tlbgen
only in $(HOST_DIR).

> We need libclang for the target because it is used by all OpenCL implementations.

I find this assertion a bit affirmative. Do *all* OpenCL
implementations in the world need clang ? Probably not.

> In this series, Clover is provided.

Perhaps a better wording is needed:

We need libclang to build a number of OpenCL implementations. A later
patch in this series will enable Clover, the OpenCL implementation part
of Mesa3D, which requires libclang.

> diff --git a/package/clang/clang.mk b/package/clang/clang.mk
> index 5ea0bb05f9..c2b83a8e60 100644
> --- a/package/clang/clang.mk
> +++ b/package/clang/clang.mk
> @@ -10,31 +10,61 @@ CLANG_SOURCE = cfe-$(CLANG_VERSION).src.tar.xz
>  CLANG_LICENSE = NCSA
>  CLANG_LICENSE_FILES = LICENSE.TXT
>  CLANG_SUPPORTS_IN_SOURCE_BUILD = NO
> +CLANG_INSTALL_STAGING = YES
>  
>  HOST_CLANG_DEPENDENCIES = host-llvm host-libxml2
> +CLANG_DEPENDENCIES = host-clang llvm
>  
>  # Use "Unix Makefiles" generator for generating make-compatible parallel makefiles.
>  HOST_CLANG_CONF_OPTS += -G "Unix Makefiles"
> +CLANG_CONF_OPTS += -G "Unix Makefiles"
> +
> +# This option is needed. Otherwise multiple shared libs (libclangAST.so,
> +# libclangBasic.so, libclangFrontend.so, etc.) will generated. As a final

will *be* generated

> +# shared lib containing all these components (libclang.so) is also generated, this
> +# resulted in the following error when trying to use tools that use libclang:
> +# $ CommandLine Error: Option 'track-memory' registered more than once!
> +# $ LLVM ERROR: inconsistency in registered CommandLine options
> +# By setting BUILD_SHARED_LIBS to OFF, we generate multiple static libraries
> +# (the same way as host's clang build) and finally libclang.so to be installed on the
> +# target.
> +CLANG_CONF_OPTS += -DBUILD_SHARED_LIBS=OFF
> +
> +# Copy llvm-tblgen (host variant) to STAGING_DIR
> +define CLANG_COPY_LLVM_TBLGEN_TO_STAGING_DIR
> +	$(INSTALL) -D -m 0755 $(HOST_DIR)/bin/llvm-tblgen $(STAGING_DIR)/usr/bin/llvm-tblgen
> +endef
> +CLANG_PRE_CONFIGURE_HOOKS = CLANG_COPY_LLVM_TBLGEN_TO_STAGING_DIR

Why does llvm-tblgen *has* to be in STAGING_DIR ?

>  # Select Release build
>  HOST_CLANG_CONF_OPTS += -DCMAKE_BUILD_TYPE=Release
> +CLANG_CONF_OPTS += -DCMAKE_BUILD_TYPE=Release

Not needed, it's passed by the package infrastructure.

> +
> +# Make it explicit that we are cross-compiling
> +CLANG_CONF_OPTS += -DCMAKE_CROSSCOMPILING=1
>  
>  # Build tools enabled
>  HOST_CLANG_CONF_OPTS += -DCLANG_BUILD_TOOLS=ON
> +CLANG_CONF_OPTS += -DCLANG_BUILD_TOOLS=ON
>  
>  # Don't build examples
>  HOST_CLANG_CONF_OPTS += -DCLANG_BUILD_EXAMPLES=OFF
> +CLANG_CONF_OPTS += -DCLANG_BUILD_EXAMPLES=OFF
>  
>  # Don't build doc
>  HOST_CLANG_CONF_OPTS += -DCLANG_INCLUDE_DOCS=OFF
> +CLANG_CONF_OPTS += -DCLANG_INCLUDE_DOCS=OFF
>  
>  # Don't build tests
>  HOST_CLANG_CONF_OPTS += -DCLANG_INCLUDE_TESTS=OFF
> +CLANG_CONF_OPTS += -DCLANG_INCLUDE_TESTS=OFF

This is also too verbose.

Thanks,

Thomas