| Message ID | 20180228173852.66069-1-charles.hardin@storagecraft.com |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [1/1] rust: make sure the cxx compiler is also set for the target | expand |
Hello Charles, Eric: could you review the patch below ? On Wed, 28 Feb 2018 09:38:52 -0800, charles.hardin@storagecraft.com wrote: > From: Charles Hardin <charles.hardin@storagecraft.com> > > Noticed that building the LLVM build -DCMAKE_CXX_COMPILER=c++ > during the rust build which the C compiler was set to the > expected buildroot compiler. I am sorry, but I don't understand this sentence. Could you explain in more details what is going on ? > diff --git a/package/rust/rust.mk b/package/rust/rust.mk > index ff2634b996..9ea4c79af5 100644 > --- a/package/rust/rust.mk > +++ b/package/rust/rust.mk > @@ -62,7 +62,10 @@ define HOST_RUST_CONFIGURE_CMDS > echo '[rust]'; \ > echo 'use-jemalloc = $(HOST_RUST_JEMALLOC_ENABLED)'; \ > echo '[target.$(RUSTC_TARGET_NAME)]'; \ > - echo 'cc = "$(TARGET_CROSS)gcc"'; \ > + if [ "x$(RUSTC_HOST_NAME)" != "x$(RUSTC_TARGET_NAME)" ]; then \ > + echo 'cc = "$(TARGET_CROSS)gcc"'; \ > + echo 'cxx = "$(TARGET_CROSS)g++"'; \ > + fi; \ This is doing two things: 1. Defining cxx, like the commit title says 2. Only defining cc and cxx when RUSTC_HOST_NAME and RUSTC_TARGET_NAME, which is not explained anywhere. Could you clarify what this patch does, and why it is needed, with a better commit log ? Thanks! Thomas
We have a problem where we build on a HOST with x86_64 and RUST doesn’t realize the difference between the host tool chain and the buildroot toolchain. In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu” target that would be recognized, but the latest patches removed that around rust 1.9 or so. This was literally just looking at the logs and seeing the “cc” was the buildroot cc and the the cxx was “c++”, so it is just what we hacked together to get things to work. But - definitely need to set “cxx” in some appropriate fashion for rust to get compiled correctly. > On Apr 1, 2018, at 3:48 PM, Thomas Petazzoni <thomas.petazzoni@bootlin.com> wrote: > > Hello Charles, > > Eric: could you review the patch below ? > > On Wed, 28 Feb 2018 09:38:52 -0800, charles.hardin@storagecraft.com > wrote: >> From: Charles Hardin <charles.hardin@storagecraft.com> >> >> Noticed that building the LLVM build -DCMAKE_CXX_COMPILER=c++ >> during the rust build which the C compiler was set to the >> expected buildroot compiler. > > I am sorry, but I don't understand this sentence. Could you explain in > more details what is going on ? > >> diff --git a/package/rust/rust.mk b/package/rust/rust.mk >> index ff2634b996..9ea4c79af5 100644 >> --- a/package/rust/rust.mk >> +++ b/package/rust/rust.mk >> @@ -62,7 +62,10 @@ define HOST_RUST_CONFIGURE_CMDS >> echo '[rust]'; \ >> echo 'use-jemalloc = $(HOST_RUST_JEMALLOC_ENABLED)'; \ >> echo '[target.$(RUSTC_TARGET_NAME)]'; \ >> - echo 'cc = "$(TARGET_CROSS)gcc"'; \ >> + if [ "x$(RUSTC_HOST_NAME)" != "x$(RUSTC_TARGET_NAME)" ]; then \ >> + echo 'cc = "$(TARGET_CROSS)gcc"'; \ >> + echo 'cxx = "$(TARGET_CROSS)g++"'; \ >> + fi; \ > > This is doing two things: > > 1. Defining cxx, like the commit title says > > 2. Only defining cc and cxx when RUSTC_HOST_NAME and > RUSTC_TARGET_NAME, which is not explained anywhere. > > Could you clarify what this patch does, and why it is needed, with a > better commit log ? > > Thanks! > > Thomas > -- > Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) > Embedded Linux and Kernel engineering > https://bootlin.com >
Hello, Thanks for the quick feedback! On Sun, 1 Apr 2018 22:52:42 +0000, Charles Hardin wrote: > We have a problem where we build on a HOST with x86_64 and RUST > doesn’t realize the difference between the host tool chain and the buildroot > toolchain. > > In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu” > target that would be recognized, but the latest patches removed that around > rust 1.9 or so. > > This was literally just looking at the logs and seeing the “cc” was the buildroot > cc and the the cxx was “c++”, so it is just what we hacked together to get > things to work. > > But - definitely need to set “cxx” in some appropriate fashion for rust to get > compiled correctly. Setting cxx looks definitely OK in your patch. It's the RUSTC_HOST_NAME != RUSTC_TARGET_NAME that looks wrong. Could you provide some specific examples (with defconfigs to reproduce) for the two problems: - Lack of cxx variable - Case of RUSTC_HOST_NAME == RUSTC_TARGET_NAME Thanks! Thomas
Yeah - for “upstream” drop the HOST - TARGET thing that is a problem for us - I think… The def config was basically this… but, it doesn’t show an error - i just read the logs of the build. BR2_x86_64=y BR2_x86_core_avx2=y BR2_TOOLCHAIN_BUILDROOT_CXX=y BR2_TOOLCHAIN_BUILDROOT_GLIBC=y BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_9=y BR2_GCC_VERSION_7_X=y BR2_GCC_ENABLE_LTO=y BR2_GCC_ENABLE_LIBMUDFLAP=y BR2_ENABLE_LOCALE_PURGE=y BR2_LINUX_KERNEL=n BR2_KERNEL_HEADERS_4_9=y BR2_INIT_NONE=y BR2_PACKAGE_HOST_GO=y BR2_PACKAGE_HOST_RUST=y BR2_PACKAGE_HOST_RUSTC=y BR2_PACKAGE_HOST_CARGO=y To basically just build a toolchain. On Apr 1, 2018, at 3:57 PM, Thomas Petazzoni <thomas.petazzoni@bootlin.com<mailto:thomas.petazzoni@bootlin.com>> wrote: Hello, Thanks for the quick feedback! On Sun, 1 Apr 2018 22:52:42 +0000, Charles Hardin wrote: We have a problem where we build on a HOST with x86_64 and RUST doesn’t realize the difference between the host tool chain and the buildroot toolchain. In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu” target that would be recognized, but the latest patches removed that around rust 1.9 or so. This was literally just looking at the logs and seeing the “cc” was the buildroot cc and the the cxx was “c++”, so it is just what we hacked together to get things to work. But - definitely need to set “cxx” in some appropriate fashion for rust to get compiled correctly. Setting cxx looks definitely OK in your patch. It's the RUSTC_HOST_NAME != RUSTC_TARGET_NAME that looks wrong. Could you provide some specific examples (with defconfigs to reproduce) for the two problems: - Lack of cxx variable - Case of RUSTC_HOST_NAME == RUSTC_TARGET_NAME Thanks! Thomas -- Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering https://bootlin.com <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""> Yeah - for “upstream” drop the HOST - TARGET thing that is a problem for <div class="">us - I think…</div> <div class=""><br class=""> </div> <div class="">The def config was basically this… but, it doesn’t show an error - i just read</div> <div class="">the logs of the build.</div> <div class=""><br class=""> </div> <div class=""> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_x86_64=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_x86_core_avx2=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_BUILDROOT_CXX=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_BUILDROOT_GLIBC=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_9=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_VERSION_7_X=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_ENABLE_LTO=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_ENABLE_LIBMUDFLAP=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_ENABLE_LOCALE_PURGE=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_LINUX_KERNEL=n</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_KERNEL_HEADERS_4_9=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_INIT_NONE=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_GO=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_RUST=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_RUSTC=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_CARGO=y</span></div> <div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""> </span></div> <div class="">To basically just build a toolchain.</div> <div><br class=""> <blockquote type="cite" class=""> <div class="">On Apr 1, 2018, at 3:57 PM, Thomas Petazzoni <<a href="mailto:thomas.petazzoni@bootlin.com" class="">thomas.petazzoni@bootlin.com</a>> wrote:</div> <br class="Apple-interchange-newline"> <div class=""> <div class="">Hello,<br class=""> <br class=""> Thanks for the quick feedback!<br class=""> <br class=""> On Sun, 1 Apr 2018 22:52:42 +0000, Charles Hardin wrote:<br class=""> <blockquote type="cite" class="">We have a problem where we build on a HOST with x86_64 and RUST<br class=""> doesn’t realize the difference between the host tool chain and the buildroot<br class=""> toolchain.<br class=""> <br class=""> In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu”<br class=""> target that would be recognized, but the latest patches removed that around<br class=""> rust 1.9 or so.<br class=""> <br class=""> This was literally just looking at the logs and seeing the “cc” was the buildroot<br class=""> cc and the the cxx was “c++”, so it is just what we hacked together to get<br class=""> things to work.<br class=""> <br class=""> But - definitely need to set “cxx” in some appropriate fashion for rust to get<br class=""> compiled correctly.<br class=""> </blockquote> <br class=""> Setting cxx looks definitely OK in your patch. It's the<br class=""> RUSTC_HOST_NAME != RUSTC_TARGET_NAME that looks wrong.<br class=""> <br class=""> Could you provide some specific examples (with defconfigs to reproduce)<br class=""> for the two problems:<br class=""> <br class=""> - Lack of cxx variable<br class=""> <br class=""> - Case of RUSTC_HOST_NAME == RUSTC_TARGET_NAME<br class=""> <br class=""> Thanks!<br class=""> <br class=""> Thomas<br class=""> -- <br class=""> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)<br class=""> Embedded Linux and Kernel engineering<br class=""> <a href="https://bootlin.com" class="">https://bootlin.com</a><br class=""> <br class=""> </div> </div> </blockquote> </div> <br class=""> </div> </body> </html>
Oh, I think I might remember the problem on TARGET - HOST, notice the core_avx2 is our target and our buildslaves don’t have avx2 and rust has some stage where it built a local tool and it didn’t run on the buildslave because it used the cross compiler again because it doesn’t distinguish the buildroot targets from generic targets. The cxx thing is all that most people will care about getting in upstream. On Apr 1, 2018, at 4:00 PM, Charles Hardin <charles.hardin@storagecraft.com<mailto:charles.hardin@storagecraft.com>> wrote: Yeah - for “upstream” drop the HOST - TARGET thing that is a problem for us - I think… The def config was basically this… but, it doesn’t show an error - i just read the logs of the build. BR2_x86_64=y BR2_x86_core_avx2=y BR2_TOOLCHAIN_BUILDROOT_CXX=y BR2_TOOLCHAIN_BUILDROOT_GLIBC=y BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_9=y BR2_GCC_VERSION_7_X=y BR2_GCC_ENABLE_LTO=y BR2_GCC_ENABLE_LIBMUDFLAP=y BR2_ENABLE_LOCALE_PURGE=y BR2_LINUX_KERNEL=n BR2_KERNEL_HEADERS_4_9=y BR2_INIT_NONE=y BR2_PACKAGE_HOST_GO=y BR2_PACKAGE_HOST_RUST=y BR2_PACKAGE_HOST_RUSTC=y BR2_PACKAGE_HOST_CARGO=y To basically just build a toolchain. On Apr 1, 2018, at 3:57 PM, Thomas Petazzoni <thomas.petazzoni@bootlin.com<mailto:thomas.petazzoni@bootlin.com>> wrote: Hello, Thanks for the quick feedback! On Sun, 1 Apr 2018 22:52:42 +0000, Charles Hardin wrote: We have a problem where we build on a HOST with x86_64 and RUST doesn’t realize the difference between the host tool chain and the buildroot toolchain. In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu” target that would be recognized, but the latest patches removed that around rust 1.9 or so. This was literally just looking at the logs and seeing the “cc” was the buildroot cc and the the cxx was “c++”, so it is just what we hacked together to get things to work. But - definitely need to set “cxx” in some appropriate fashion for rust to get compiled correctly. Setting cxx looks definitely OK in your patch. It's the RUSTC_HOST_NAME != RUSTC_TARGET_NAME that looks wrong. Could you provide some specific examples (with defconfigs to reproduce) for the two problems: - Lack of cxx variable - Case of RUSTC_HOST_NAME == RUSTC_TARGET_NAME Thanks! Thomas -- Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering https://bootlin.com<https://bootlin.com/> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""> Oh, I think I might remember the problem on TARGET - HOST, notice the core_avx2 <div class="">is our target and our buildslaves don’t have avx2 and rust has some stage where it</div> <div class="">built a local tool and it didn’t run on the buildslave because it used the cross compiler</div> <div class="">again because it doesn’t distinguish the buildroot targets from generic targets.</div> <div class=""><br class=""> </div> <div class="">The cxx thing is all that most people will care about getting in upstream.</div> <div class=""> <div><br class=""> <blockquote type="cite" class=""> <div class="">On Apr 1, 2018, at 4:00 PM, Charles Hardin <<a href="mailto:charles.hardin@storagecraft.com" class="">charles.hardin@storagecraft.com</a>> wrote:</div> <br class="Apple-interchange-newline"> <div class=""> <div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""> Yeah - for “upstream” drop the HOST - TARGET thing that is a problem for <div class="">us - I think…</div> <div class=""><br class=""> </div> <div class="">The def config was basically this… but, it doesn’t show an error - i just read</div> <div class="">the logs of the build.</div> <div class=""><br class=""> </div> <div class=""> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_x86_64=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_x86_core_avx2=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_BUILDROOT_CXX=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_BUILDROOT_GLIBC=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_TOOLCHAIN_HEADERS_AT_LEAST_4_9=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_VERSION_7_X=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_ENABLE_LTO=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_GCC_ENABLE_LIBMUDFLAP=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_ENABLE_LOCALE_PURGE=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_LINUX_KERNEL=n</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_KERNEL_HEADERS_4_9=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_INIT_NONE=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_GO=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_RUST=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_RUSTC=y</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">BR2_PACKAGE_HOST_CARGO=y</span></div> <div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""> </span></div> <div class="">To basically just build a toolchain.</div> <div class=""><br class=""> <blockquote type="cite" class=""> <div class="">On Apr 1, 2018, at 3:57 PM, Thomas Petazzoni <<a href="mailto:thomas.petazzoni@bootlin.com" class="">thomas.petazzoni@bootlin.com</a>> wrote:</div> <br class="Apple-interchange-newline"> <div class=""> <div class="">Hello,<br class=""> <br class=""> Thanks for the quick feedback!<br class=""> <br class=""> On Sun, 1 Apr 2018 22:52:42 +0000, Charles Hardin wrote:<br class=""> <blockquote type="cite" class="">We have a problem where we build on a HOST with x86_64 and RUST<br class=""> doesn’t realize the difference between the host tool chain and the buildroot<br class=""> toolchain.<br class=""> <br class=""> In the old days - we had to patch rust to make a “x86_64-buildroot-linux-gnu”<br class=""> target that would be recognized, but the latest patches removed that around<br class=""> rust 1.9 or so.<br class=""> <br class=""> This was literally just looking at the logs and seeing the “cc” was the buildroot<br class=""> cc and the the cxx was “c++”, so it is just what we hacked together to get<br class=""> things to work.<br class=""> <br class=""> But - definitely need to set “cxx” in some appropriate fashion for rust to get<br class=""> compiled correctly.<br class=""> </blockquote> <br class=""> Setting cxx looks definitely OK in your patch. It's the<br class=""> RUSTC_HOST_NAME != RUSTC_TARGET_NAME that looks wrong.<br class=""> <br class=""> Could you provide some specific examples (with defconfigs to reproduce)<br class=""> for the two problems:<br class=""> <br class=""> - Lack of cxx variable<br class=""> <br class=""> - Case of RUSTC_HOST_NAME == RUSTC_TARGET_NAME<br class=""> <br class=""> Thanks!<br class=""> <br class=""> Thomas<br class=""> -- <br class=""> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)<br class=""> Embedded Linux and Kernel engineering<br class=""> <a href="https://bootlin.com/" class="">https://bootlin.com</a><br class=""> <br class=""> </div> </div> </blockquote> </div> <br class=""> </div> </div> </div> </blockquote> </div> <br class=""> </div> </body> </html>
Hello, [Side note: it is customary on open-source mailing list to avoid top-posting, and instead reply below the text you are replying to. Thanks!] On Sun, 1 Apr 2018 23:05:11 +0000, Charles Hardin wrote: > Oh, I think I might remember the problem on TARGET - HOST, notice the core_avx2 > is our target and our buildslaves don’t have avx2 and rust has some stage where it > built a local tool and it didn’t run on the buildslave because it used the cross compiler Well, that's a severe bug. If Rust uses the cross-compiler to build a program that it then runs on the host, this is not going to work on a number of situations. Your defconfig uses glibc for the target, and your host most likely uses glibc as well, so there are some chances that a tool built with the cross-compiler will also run on the host. But if the CPU optimization options are different (like you had with AVX2) or the C library is different than glibc, then the tool built with the cross-compiler will definitely not run on the host. However, we do have a x86-64 uClibc configuration in our autobuilders, which is built on a x86-64 machine, and I don't think we have seen any issue. So, I'm a bit puzzled. Which specific internal tool was causing the problem ? Thanks, Thomas
so… hopefully the following makes sense... On Apr 1, 2018, at 11:38 PM, Thomas Petazzoni <thomas.petazzoni@bootlin.com<mailto:thomas.petazzoni@bootlin.com>> wrote: Hello, [Side note: it is customary on open-source mailing list to avoid top-posting, and instead reply below the text you are replying to. Thanks!] On Sun, 1 Apr 2018 23:05:11 +0000, Charles Hardin wrote: Oh, I think I might remember the problem on TARGET - HOST, notice the core_avx2 is our target and our buildslaves don’t have avx2 and rust has some stage where it built a local tool and it didn’t run on the buildslave because it used the cross compiler Well, that's a severe bug. If Rust uses the cross-compiler to build a program that it then runs on the host, this is not going to work on a number of situations. Your defconfig uses glibc for the target, and your host most likely uses glibc as well, so there are some chances that a tool built with the cross-compiler will also run on the host. But if the CPU optimization options are different (like you had with AVX2) or the C library is different than glibc, then the tool built with the cross-compiler will definitely not run on the host. However, we do have a x86-64 uClibc configuration in our autobuilders, which is built on a x86-64 machine, and I don't think we have seen any issue. So, I'm a bit puzzled. When you fix the CXX linking you will see this in the stage0 - notice the target is unknown - not buildroot, and so what happens is the tools use the “same” compiler as the target for the host side, which means that I ended up getting “target” compiled with avx2 on a host that doesn’t support it Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) … snip snip … Building LLVM for x86_64-unknown-linux-gnu running: "cmake" "/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=2" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_LINK_LLVM_DYLIB=ON" "-DCMAKE_C_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc" "-DCMAKE_CXX_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_AR=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-ar" "-DCMAKE_INSTALL_PREFIX=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_BUILD_TYPE=Release" -- The C compiler identification is GNU 7.3.0 -- The CXX compiler identification is GNU 7.3.0 -- The ASM compiler identification is GNU -- Found assembler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Warning at CMakeLists.txt:140 (message): Job pooling is only available with Ninja generators. leads to building this “host” tool [ 2%] Built target LLVMTableGen which cann’t run on that host because the compiler it used was the “buildroot” compiler [ 19%] Linking CXX static library ../../libLLVMDebugInfoPDB.a [ 19%] Built target LLVMDebugInfoPDB Makefile:151: recipe for target 'all' failed make[3]: *** [all] Error 2 thread 'main' panicked at ' command did not execute successfully, got: exit code: 2 build script failed, must exit now', src/vendor/cmake/src/lib.rs:631<http://lib.rs:631>:4 note: Run with `RUST_BACKTRACE=1` for a backtrace. finished in 267.033 failed to run: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/bootstrap/debug/bootstrap build Build completed unsuccessfully in 0:06:22 package/pkg-generic.mk:247: recipe for target '/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built' failed make[2]: *** [/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built] Error 1 Makefile:79: recipe for target '_all' failed make[1]: *** [_all] Error 2 make[1]: Leaving directory '/home/vagrant/oneblox-buildroot/repos/buildroot' Makefile:93: recipe for target 'br-x86_64-core_avx2-glibc-tmp.tar.bz2' failed make: *** [br-x86_64-core_avx2-glibc-tmp.tar.bz2] Error 2 Which specific internal tool was causing the problem ? Thanks, Thomas -- Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering https://bootlin.com <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""> so… hopefully the following makes sense...<br class=""> <div><br class=""> <blockquote type="cite" class=""> <div class="">On Apr 1, 2018, at 11:38 PM, Thomas Petazzoni <<a href="mailto:thomas.petazzoni@bootlin.com" class="">thomas.petazzoni@bootlin.com</a>> wrote:</div> <br class="Apple-interchange-newline"> <div class=""> <div class="">Hello,<br class=""> <br class=""> [Side note: it is customary on open-source mailing list to avoid<br class=""> top-posting, and instead reply below the text you are replying to.<br class=""> Thanks!]<br class=""> <br class=""> On Sun, 1 Apr 2018 23:05:11 +0000, Charles Hardin wrote:<br class=""> <br class=""> <blockquote type="cite" class="">Oh, I think I might remember the problem on TARGET - HOST, notice the core_avx2<br class=""> is our target and our buildslaves don’t have avx2 and rust has some stage where it<br class=""> built a local tool and it didn’t run on the buildslave because it used the cross compiler<br class=""> </blockquote> <br class=""> Well, that's a severe bug. If Rust uses the cross-compiler to build a<br class=""> program that it then runs on the host, this is not going to work on a<br class=""> number of situations. Your defconfig uses glibc for the target, and<br class=""> your host most likely uses glibc as well, so there are some chances<br class=""> that a tool built with the cross-compiler will also run on the host.<br class=""> But if the CPU optimization options are different (like you had with<br class=""> AVX2) or the C library is different than glibc, then the tool built<br class=""> with the cross-compiler will definitely not run on the host.<br class=""> <br class=""> However, we do have a x86-64 uClibc configuration in our autobuilders,<br class=""> which is built on a x86-64 machine, and I don't think we have seen any<br class=""> issue. So, I'm a bit puzzled.<br class=""> <br class=""> </div> </div> </blockquote> <div><br class=""> </div> <div>When you fix the CXX linking you will see this in the stage0 - notice the target is unknown - not buildroot, and</div> <div>so what happens is the tools use the “same” compiler as the target for the host side, which means that I ended</div> <div>up getting “target” compiled with avx2 on a host that doesn’t support it</div> <div><br class=""> </div> <div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)</span></div> <div class=""><br class=""> </div> </div> <div><span class="Apple-tab-span" style="white-space:pre"></span>… snip snip …</div> <div><br class=""> </div> <div><span style="font-family: Menlo; font-size: 11px; background-color: rgb(255, 255, 255);" class="">Building LLVM for x86_64-unknown-linux-gnu</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">running: "cmake" "/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=2" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_LINK_LLVM_DYLIB=ON" "-DCMAKE_C_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc" "-DCMAKE_CXX_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_AR=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-ar" "-DCMAKE_INSTALL_PREFIX=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_BUILD_TYPE=Release"</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- The C compiler identification is GNU 7.3.0</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- The CXX compiler identification is GNU 7.3.0</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- The ASM compiler identification is GNU</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Found assembler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- works</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting C compiler ABI info</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting C compiler ABI info - done</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting C compile features</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting C compile features - done</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ -- works</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting CXX compiler ABI info</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting CXX compiler ABI info - done</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting CXX compile features</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">-- Detecting CXX compile features - done</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">CMake Warning at CMakeLists.txt:140 (message):</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class=""> Job pooling is only available with Ninja generators.</span></div> <div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""> </span></div> <div class="">leads to building this “host” tool</div> <div class=""><br class=""> </div> <div class=""> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">[ 2%] Built target LLVMTableGen</span></div> </div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""> </span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">which cann’t run on that host because the compiler it used was the “buildroot” compiler</span></div> <div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""> </span></div> <div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; color: rgb(52, 188, 38); background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures; color: #000000" class="">[ 19%] </span><span style="font-variant-ligatures: no-common-ligatures" class=""><b class="">Linking CXX static library ../../libLLVMDebugInfoPDB.a</b></span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">[ 19%] Built target LLVMDebugInfoPDB</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Makefile:151: recipe for target 'all' failed</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">make[3]: *** [all] Error 2</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">thread 'main' panicked at '</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">command did not execute successfully, got: exit code: 2</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 13px;" class=""> <span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""> </div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">build script failed, must exit now', src/vendor/cmake/src/<a href="http://lib.rs:631" class="">lib.rs:631</a>:4</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">note: Run with `RUST_BACKTRACE=1` for a backtrace.</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class=""><span class="Apple-tab-span" style="white-space:pre"></span>finished in 267.033</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">failed to run: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/bootstrap/debug/bootstrap build</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Build completed unsuccessfully in 0:06:22</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">package/pkg-generic.mk:247: recipe for target '/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built' failed</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">make[2]: *** [/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built] Error 1</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Makefile:79: recipe for target '_all' failed</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">make[1]: *** [_all] Error 2</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">make[1]: Leaving directory '/home/vagrant/oneblox-buildroot/repos/buildroot'</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">Makefile:93: recipe for target 'br-x86_64-core_avx2-glibc-tmp.tar.bz2' failed</span></div> <div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="font-variant-ligatures: no-common-ligatures" class="">make: *** [br-x86_64-core_avx2-glibc-tmp.tar.bz2] Error 2</span></div> <div class=""><br class=""> </div> </div> <br class=""> <blockquote type="cite" class=""> <div class=""> <div class="">Which specific internal tool was causing the problem ?<br class=""> <br class=""> Thanks,<br class=""> <br class=""> Thomas<br class=""> -- <br class=""> Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)<br class=""> Embedded Linux and Kernel engineering<br class=""> <a href="https://bootlin.com" class="">https://bootlin.com</a><br class=""> <br class=""> </div> </div> </blockquote> </div> <br class=""> </body> </html>
Hi! On 2018-04-02 19:54, Charles Hardin wrote: > so… hopefully the following makes sense... > > When you fix the CXX linking you will see this in the stage0 - notice the target is unknown - not buildroot, and > so what happens is the tools use the “same” compiler as the target for the host side, which means that I ended > up getting “target” compiled with avx2 on a host that doesn’t support it > > Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) > Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) > > … snip snip … > > Building LLVM for x86_64-unknown-linux-gnu > running: "cmake" "/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=2" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_LINK_LLVM_DYLIB=ON" "-DCMAKE_C_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc" "-DCMAKE_CXX_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_AR=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-ar" "-DCMAKE_INSTALL_PREFIX=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_BUILD_TYPE=Release" > -- The C compiler identification is GNU 7.3.0 > -- The CXX compiler identification is GNU 7.3.0 > -- The ASM compiler identification is GNU > -- Found assembler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc > -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc > -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- works > -- Detecting C compiler ABI info > -- Detecting C compiler ABI info - done > -- Detecting C compile features > -- Detecting C compile features - done > -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ > -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ -- works > -- Detecting CXX compiler ABI info > -- Detecting CXX compiler ABI info - done > -- Detecting CXX compile features > -- Detecting CXX compile features - done > CMake Warning at CMakeLists.txt:140 (message): > Job pooling is only available with Ninja generators. > > leads to building this “host” tool > > [ 2%] Built target LLVMTableGen > > which cann’t run on that host because the compiler it used was the “buildroot” compiler > > [ 19%] Linking CXX static library ../../libLLVMDebugInfoPDB.a > [ 19%] Built target LLVMDebugInfoPDB > Makefile:151: recipe for target 'all' failed > make[3]: *** [all] Error 2 > thread 'main' panicked at ' > command did not execute successfully, got: exit code: 2 > > build script failed, must exit now', src/vendor/cmake/src/lib.rs:631<http://lib.rs:631>:4 > note: Run with `RUST_BACKTRACE=1` for a backtrace. > finished in 267.033 > failed to run: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/bootstrap/debug/bootstrap build > Build completed unsuccessfully in 0:06:22 > package/pkg-generic.mk:247: recipe for target '/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built' failed > make[2]: *** [/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built] Error 1 > Makefile:79: recipe for target '_all' failed > make[1]: *** [_all] Error 2 > make[1]: Leaving directory '/home/vagrant/oneblox-buildroot/repos/buildroot' > Makefile:93: recipe for target 'br-x86_64-core_avx2-glibc-tmp.tar.bz2' failed > make: *** [br-x86_64-core_avx2-glibc-tmp.tar.bz2] Error 2 Building Rust for x86_64 on a x86_64 is indeed tricky! As seen in the build section of src/bootstrap/config.toml.example, Rust manages "build", "host" and "target" parameters, which all default to "x86_64-unknown-linux-gnu". When building, the compiler is built for the host and the standard library for the host and the target. So when the target is the same as the host, only the compiler and the standard library for the host should be built. As the compiler is built on LLVM, coded in C++, a C++ compiler for the host is indeed required. It happens that in the build of LLVM, a host tool named FileCheck is built and run for test. IIUC, the C compiler is only used for building the standard library (this is what I've seen when cross-compiling for ARM). So when the target is the same as the host, it might be sensible to avoid defining c and cxx in the target configuration file, as suggested by your patch. I'll poke upstream for clarifications on this special case, though. The first versions of the patch series did try to work with new targets using triples with "buildroot" instead of "unknown", but defining new targets instead of using the available ones turned out to be difficult/cumbersome. But maybe it would have avoided the issue raised here where the target is the same as the host. Out of curiosity, does using the pre-built toolchain may be of use in your situation? Regards, -- ELB
Eric, Good to talk to you again… > On Apr 3, 2018, at 3:01 PM, Eric Le Bihan <eric.le.bihan.dev@free.fr> wrote: > > Hi! > > On 2018-04-02 19:54, Charles Hardin wrote: >> so… hopefully the following makes sense... >> >> When you fix the CXX linking you will see this in the stage0 - notice the target is unknown - not buildroot, and >> so what happens is the tools use the “same” compiler as the target for the host side, which means that I ended >> up getting “target” compiled with avx2 on a host that doesn’t support it >> >> Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu) >> Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu) >> >> … snip snip … >> >> Building LLVM for x86_64-unknown-linux-gnu >> running: "cmake" "/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;JSBackend;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=2" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_LINK_LLVM_DYLIB=ON" "-DCMAKE_C_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc" "-DCMAKE_CXX_COMPILER=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_AR=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-ar" "-DCMAKE_INSTALL_PREFIX=/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_BUILD_TYPE=Release" >> -- The C compiler identification is GNU 7.3.0 >> -- The CXX compiler identification is GNU 7.3.0 >> -- The ASM compiler identification is GNU >> -- Found assembler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc >> -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc >> -- Check for working C compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-gcc -- works >> -- Detecting C compiler ABI info >> -- Detecting C compiler ABI info - done >> -- Detecting C compile features >> -- Detecting C compile features - done >> -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ >> -- Check for working CXX compiler: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/host/bin/x86_64-buildroot-linux-gnu-g++ -- works >> -- Detecting CXX compiler ABI info >> -- Detecting CXX compiler ABI info - done >> -- Detecting CXX compile features >> -- Detecting CXX compile features - done >> CMake Warning at CMakeLists.txt:140 (message): >> Job pooling is only available with Ninja generators. >> >> leads to building this “host” tool >> >> [ 2%] Built target LLVMTableGen >> >> which cann’t run on that host because the compiler it used was the “buildroot” compiler >> >> [ 19%] Linking CXX static library ../../libLLVMDebugInfoPDB.a >> [ 19%] Built target LLVMDebugInfoPDB >> Makefile:151: recipe for target 'all' failed >> make[3]: *** [all] Error 2 >> thread 'main' panicked at ' >> command did not execute successfully, got: exit code: 2 >> >> build script failed, must exit now', src/vendor/cmake/src/lib.rs:631<http://lib.rs:631>:4 >> note: Run with `RUST_BACKTRACE=1` for a backtrace. >> finished in 267.033 >> failed to run: /home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/build/bootstrap/debug/bootstrap build >> Build completed unsuccessfully in 0:06:22 >> package/pkg-generic.mk:247: recipe for target '/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built' failed >> make[2]: *** [/home/vagrant/oneblox-buildroot/toolchains/br-x86_64-core_avx2-glibc_toolchains.build/build/host-rust-1.23.0/.stamp_built] Error 1 >> Makefile:79: recipe for target '_all' failed >> make[1]: *** [_all] Error 2 >> make[1]: Leaving directory '/home/vagrant/oneblox-buildroot/repos/buildroot' >> Makefile:93: recipe for target 'br-x86_64-core_avx2-glibc-tmp.tar.bz2' failed >> make: *** [br-x86_64-core_avx2-glibc-tmp.tar.bz2] Error 2 > > Building Rust for x86_64 on a x86_64 is indeed tricky! > > As seen in the build section of src/bootstrap/config.toml.example, Rust > manages "build", "host" and "target" parameters, which all default to > "x86_64-unknown-linux-gnu". > > When building, the compiler is built for the host and the standard > library for the host and the target. > > So when the target is the same as the host, only the compiler and the > standard library for the host should be built. > > As the compiler is built on LLVM, coded in C++, a C++ compiler for the > host is indeed required. It happens that in the build of LLVM, a host > tool named FileCheck is built and run for test. > > IIUC, the C compiler is only used for building the standard library > (this is what I've seen when cross-compiling for ARM). > > So when the target is the same as the host, it might be sensible to > avoid defining c and cxx in the target configuration file, as suggested > by your patch. > > I'll poke upstream for clarifications on this special case, though. > > The first versions of the patch series did try to work with new targets > using triples with "buildroot" instead of "unknown", but defining new > targets instead of using the available ones turned out to be > difficult/cumbersome. > > But maybe it would have avoided the issue raised here where the target > is the same as the host. > > Out of curiosity, does using the pre-built toolchain may be of use in > your situation? > Yes and no, on some targets we use the pre-built and on some (ie. mips64) we have to build for now at least. So, this isn’t blocking us - I have it working “good enough” but this was just sending it upstream incase anyone else hit it. I am not inclined to know the fix or prefer one way over another. > Regards, > > -- > ELB >
Hello Eric, Sorry for the very slow feedback. On Wed, 4 Apr 2018 00:01:32 +0200, Eric Le Bihan wrote: > Building Rust for x86_64 on a x86_64 is indeed tricky! > > As seen in the build section of src/bootstrap/config.toml.example, Rust > manages "build", "host" and "target" parameters, which all default to > "x86_64-unknown-linux-gnu". > > When building, the compiler is built for the host and the standard > library for the host and the target. > > So when the target is the same as the host, only the compiler and the > standard library for the host should be built. When you are talking about "target" and "host", are you talking just about the CPU architecture, or about the full target and host definition, which includes the C library ? If you build on x86-64, targeting a x86-64 platform, you have host arch == target arch, but you still can't use the cross-compiler to build programs and expect them to run on the host: - The C library might be different: uClibc or musl for the target, glibc available on the host - The kernel headers version may be different: the C library for the target may be built to use very recent/modern kernel headers and therefore meant to run on a system that uses a recent kernel, while the host machine may be running an older kernel. So basically, using the cross-compiler to build something that will be executed on the host machine is always wrong. So the initial patch from Charles, that does this: + if [ "x$(RUSTC_HOST_NAME)" != "x$(RUSTC_TARGET_NAME)" ]; then \ + echo 'cc = "$(TARGET_CROSS)gcc"'; \ + echo 'cxx = "$(TARGET_CROSS)g++"'; \ + fi; \ is clearly not good, because: RUSTC_HOST_NAME = $(RUSTC_HOST_ARCH)-unknown-linux-gnu RUSTC_TARGET_NAME = $(RUSTC_ARCH)-unknown-linux-gnu$(RUSTC_ABI) So when you target x86-64 and build on x86-64, RUSTC_HOST_NAME == RUSTC_TARGET_NAME, and therefore it will continue to use the target compiler to build programs meant to run on the host, which is very wrong. > As the compiler is built on LLVM, coded in C++, a C++ compiler for the > host is indeed required. It happens that in the build of LLVM, a host > tool named FileCheck is built and run for test. > > IIUC, the C compiler is only used for building the standard library > (this is what I've seen when cross-compiling for ARM). > > So when the target is the same as the host, it might be sensible to > avoid defining c and cxx in the target configuration file, as suggested > by your patch. > > I'll poke upstream for clarifications on this special case, though. Did you get some feedback ? I'll mark the patch as Changes Requested. We need at least the following things to be changed: - Split into two patches the addition of "cxx" vs. the target != host condition - A detailed and documented commit log Charles, Éric, could you work on a new version of this patch ? Thanks, Thomas
diff --git a/package/rust/rust.mk b/package/rust/rust.mk index ff2634b996..9ea4c79af5 100644 --- a/package/rust/rust.mk +++ b/package/rust/rust.mk @@ -62,7 +62,10 @@ define HOST_RUST_CONFIGURE_CMDS echo '[rust]'; \ echo 'use-jemalloc = $(HOST_RUST_JEMALLOC_ENABLED)'; \ echo '[target.$(RUSTC_TARGET_NAME)]'; \ - echo 'cc = "$(TARGET_CROSS)gcc"'; \ + if [ "x$(RUSTC_HOST_NAME)" != "x$(RUSTC_TARGET_NAME)" ]; then \ + echo 'cc = "$(TARGET_CROSS)gcc"'; \ + echo 'cxx = "$(TARGET_CROSS)g++"'; \ + fi; \ echo $(HOST_RUST_JEMALLOC_CONF); \ ) > $(@D)/config.toml endef