| Message ID | 20210711022824.29915-6-jrtc27@jrtc27.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | Fully support standalone Clang/LLVM toolchains | expand |
On Sun, Jul 11, 2021 at 10:29 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: > > This is intended to mirror the Linux kernel. Building with CC=clang will > use Clang as the compiler but default to using the existing binutils. > Building with LLVM=1 will default to using Clang and LLVM binutils. > > Whilst GCC will accept the -N linker option and forward it on to the > linker, Clang will not, and so in order to support both compilers we > must use -Wl, to forward it to the linker as is required for most other > linker options. > > Note that there is currently a bug when using Clang as the compiler and > riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this > appears to be a bug in GNU binutils, but this could also be Clang or > OpenSBI at fault in some subtle way. Thus, for now, advise that this > combination be avoided. > > Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> > --- > Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ > README.md | 48 ++++++++++++++++++++++++++++++++++++++-- > 2 files changed, 105 insertions(+), 9 deletions(-) > > diff --git a/Makefile b/Makefile > index ba06313..5c188d5 100644 > --- a/Makefile > +++ b/Makefile > @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb > OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) > > # Setup compilation commands > +ifneq ($(LLVM),) > +CC = clang > +AR = llvm-ar > +LD = ld.lld > +OBJCOPY = llvm-objcopy > +else > ifdef CROSS_COMPILE > CC = $(CROSS_COMPILE)gcc > -CPP = $(CROSS_COMPILE)cpp > AR = $(CROSS_COMPILE)ar > LD = $(CROSS_COMPILE)ld > OBJCOPY = $(CROSS_COMPILE)objcopy > else > CC ?= gcc > -CPP ?= cpp > AR ?= ar > LD ?= ld > OBJCOPY ?= objcopy > endif > +endif > +CPP = $(CC) -E > AS = $(CC) > DTC = dtc > > -# Guess the compillers xlen > -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) > +CC_IS_CLANG = y > +else > +CC_IS_CLANG = n > +endif > + > +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) > +LD_IS_LLD = y > +else > +LD_IS_LLD = n > +endif > + > +ifeq ($(CC_IS_CLANG),y) > +ifneq ($(CROSS_COMPILE),) > +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) > +endif > +endif > + > +# Guess the compiler's XLEN > +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > + > +# Guess the compiler's ABI and ISA > +ifneq ($(CC_IS_CLANG),y) > OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > +endif > > # Setup platform XLEN > ifndef PLATFORM_RISCV_XLEN > @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN > endif > endif > > +ifeq ($(CC_IS_CLANG),y) > +ifeq ($(CROSS_COMPILE),) > +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf > +endif > +endif > + > +ifeq ($(LD_IS_LLD),y) > +RELAX_FLAG = -mno-relax > +USE_LD_FLAG = -fuse-ld=lld > +else > +USE_LD_FLAG = -fuse-ld=bfd > +endif > + > # Check whether the linker supports creating PIEs > -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) It looks like this combination does not work? Both riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be located. $ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c /dev/null -o /dev/null 2>&1 collect2: fatal error: cannot find 'ld' compilation terminated. So this will result in a PDE image, even though LLD supports PIE. > > # Setup list of objects.mk files > ifdef PLATFORM > @@ -197,7 +238,11 @@ else > endif > > # Setup compilation commands flags > -GENFLAGS = -I$(platform_src_dir)/include > +ifeq ($(CC_IS_CLANG),y) > +GENFLAGS += $(CLANG_TARGET) > +GENFLAGS += -Wno-unused-command-line-argument > +endif > +GENFLAGS += -I$(platform_src_dir)/include > GENFLAGS += -I$(include_dir) > ifneq ($(OPENSBI_VERSION_GIT),) > GENFLAGS += -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\"" > @@ -211,6 +256,7 @@ CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls > CFLAGS += -mno-save-restore -mstrict-align > CFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) > CFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) > +CFLAGS += $(RELAX_FLAG) > CFLAGS += $(GENFLAGS) > CFLAGS += $(platform-cflags-y) > CFLAGS += -fno-pie -no-pie > @@ -225,18 +271,24 @@ ASFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls > ASFLAGS += -mno-save-restore -mstrict-align > ASFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) > ASFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) > +ASFLAGS += $(RELAX_FLAG) > ASFLAGS += $(GENFLAGS) > ASFLAGS += $(platform-asflags-y) > ASFLAGS += $(firmware-asflags-y) > > ARFLAGS = rcs > > -ELFFLAGS += -Wl,--build-id=none -N -static-libgcc -lgcc > +ELFFLAGS += $(USE_LD_FLAG) > +ELFFLAGS += -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc > ELFFLAGS += $(platform-ldflags-y) > ELFFLAGS += $(firmware-ldflags-y) > > MERGEFLAGS += -r > +ifeq ($(LD_IS_LLD),y) > +MERGEFLAGS += -b elf > +else > MERGEFLAGS += -b elf$(PLATFORM_RISCV_XLEN)-littleriscv > +endif > MERGEFLAGS += -m elf$(PLATFORM_RISCV_XLEN)lriscv > > DTSCPPFLAGS = $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp > diff --git a/README.md b/README.md > index 03c02fb..b296f00 100644 > --- a/README.md > +++ b/README.md > @@ -96,8 +96,13 @@ Required Toolchain > ------------------ > > OpenSBI can be compiled natively or cross-compiled on a x86 host. For > -cross-compilation, you can build your own toolchain or just download > -a prebuilt one from the [Bootlin toolchain repository]. > +cross-compilation, you can build your own toolchain, download a prebuilt one > +from the [Bootlin toolchain repository] or install a distribution-provided > +toolchain; if you opt to use LLVM/Clang, most distribution toolchains will > +support cross-compiling for RISC-V using the same toolchain as your native > +LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the > +same binary, so is often an easy way to obtain a working cross-compilation > +toolchain. > > Please note that only a 64-bit version of the toolchain is available in > the Bootlin toolchain repository for now. > @@ -202,6 +207,45 @@ export PLATFORM_RISCV_XLEN=32 > > will generate 32-bit OpenSBI images. And vice vesa. > > +Building with Clang/LLVM > +------------------------ > + > +OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep > +the default binutils (which will still use the *CROSS_COMPILE* prefix if > +defined), override the *CC* make variable with: > +``` > +make CC=clang > +``` > + > +To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM* > +option with: > +``` > +make LLVM=1 > +``` > + > +When using Clang, *CROSS_COMPILE* often does not need to be defined unless > +using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be > +used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN* > +itself defaults to an undesired value then prefer setting that rather than the > +full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined, > +rather than being used as a prefix for the executable name, it will instead be > +passed via the `--target` option with the trailing `-` removed, so must be a > +valid triple. > + > +These can also be mixed; for example using a GCC cross-compiler but LLVM > +binutils would be: > +``` > +make CC=riscv64-unknown-elf-gcc LLVM=1 > +``` > + > +These variables must be passed for all the make invocations described in this > +document. > + > +NOTE: Using Clang with a `riscv*-linux-gnu` GNU binutils linker has been seen > +to produce broken binaries with missing relocations; it is therefore currently > +recommended that this combination be avoided or *FW_PIC=n* be used to disable > +building OpenSBI as a position-independent binary. > + > Contributing to OpenSBI > ----------------------- Regards, Bin
On 11 Jul 2021, at 15:26, Bin Meng <bmeng.cn@gmail.com> wrote: > > On Sun, Jul 11, 2021 at 10:29 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: >> >> This is intended to mirror the Linux kernel. Building with CC=clang will >> use Clang as the compiler but default to using the existing binutils. >> Building with LLVM=1 will default to using Clang and LLVM binutils. >> >> Whilst GCC will accept the -N linker option and forward it on to the >> linker, Clang will not, and so in order to support both compilers we >> must use -Wl, to forward it to the linker as is required for most other >> linker options. >> >> Note that there is currently a bug when using Clang as the compiler and >> riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this >> appears to be a bug in GNU binutils, but this could also be Clang or >> OpenSBI at fault in some subtle way. Thus, for now, advise that this >> combination be avoided. >> >> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> >> --- >> Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ >> README.md | 48 ++++++++++++++++++++++++++++++++++++++-- >> 2 files changed, 105 insertions(+), 9 deletions(-) >> >> diff --git a/Makefile b/Makefile >> index ba06313..5c188d5 100644 >> --- a/Makefile >> +++ b/Makefile >> @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb >> OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) >> >> # Setup compilation commands >> +ifneq ($(LLVM),) >> +CC = clang >> +AR = llvm-ar >> +LD = ld.lld >> +OBJCOPY = llvm-objcopy >> +else >> ifdef CROSS_COMPILE >> CC = $(CROSS_COMPILE)gcc >> -CPP = $(CROSS_COMPILE)cpp >> AR = $(CROSS_COMPILE)ar >> LD = $(CROSS_COMPILE)ld >> OBJCOPY = $(CROSS_COMPILE)objcopy >> else >> CC ?= gcc >> -CPP ?= cpp >> AR ?= ar >> LD ?= ld >> OBJCOPY ?= objcopy >> endif >> +endif >> +CPP = $(CC) -E >> AS = $(CC) >> DTC = dtc >> >> -# Guess the compillers xlen >> -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) >> +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) >> +CC_IS_CLANG = y >> +else >> +CC_IS_CLANG = n >> +endif >> + >> +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) >> +LD_IS_LLD = y >> +else >> +LD_IS_LLD = n >> +endif >> + >> +ifeq ($(CC_IS_CLANG),y) >> +ifneq ($(CROSS_COMPILE),) >> +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) >> +endif >> +endif >> + >> +# Guess the compiler's XLEN >> +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) >> + >> +# Guess the compiler's ABI and ISA >> +ifneq ($(CC_IS_CLANG),y) >> OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) >> OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) >> +endif >> >> # Setup platform XLEN >> ifndef PLATFORM_RISCV_XLEN >> @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN >> endif >> endif >> >> +ifeq ($(CC_IS_CLANG),y) >> +ifeq ($(CROSS_COMPILE),) >> +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf >> +endif >> +endif >> + >> +ifeq ($(LD_IS_LLD),y) >> +RELAX_FLAG = -mno-relax >> +USE_LD_FLAG = -fuse-ld=lld >> +else >> +USE_LD_FLAG = -fuse-ld=bfd >> +endif >> + >> # Check whether the linker supports creating PIEs >> -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) >> +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > > It looks like this combination does not work? Both > riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be > located. > > $ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c > /dev/null -o /dev/null 2>&1 > collect2: fatal error: cannot find 'ld' > compilation terminated. > > So this will result in a PDE image, even though LLD supports PIE. Isn’t this the same issue as you reported elsewhere, and will ultimately result in the same error, this time not suppressed, at link time too? Using GCC as a driver for LLD has always been a painful combination, in part because historically there’s been very little interest in supporting it from the GNU side; it wasn’t until a few years ago (I think late 2018 or early 2019?) that -fuse-ld=lld was even accepted as an option, and that was just a minimal patch that reused all the logic for BFD and gold despite the fact that that’s unnecessarily restrictive for LLD and doesn’t support cases like this properly. Jess
On Sun, Jul 11, 2021 at 10:46 PM Jessica Clarke <jrtc27@jrtc27.com> wrote: > > On 11 Jul 2021, at 15:26, Bin Meng <bmeng.cn@gmail.com> wrote: > > > > On Sun, Jul 11, 2021 at 10:29 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: > >> > >> This is intended to mirror the Linux kernel. Building with CC=clang will > >> use Clang as the compiler but default to using the existing binutils. > >> Building with LLVM=1 will default to using Clang and LLVM binutils. > >> > >> Whilst GCC will accept the -N linker option and forward it on to the > >> linker, Clang will not, and so in order to support both compilers we > >> must use -Wl, to forward it to the linker as is required for most other > >> linker options. > >> > >> Note that there is currently a bug when using Clang as the compiler and > >> riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this > >> appears to be a bug in GNU binutils, but this could also be Clang or > >> OpenSBI at fault in some subtle way. Thus, for now, advise that this > >> combination be avoided. > >> > >> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> > >> --- > >> Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ > >> README.md | 48 ++++++++++++++++++++++++++++++++++++++-- > >> 2 files changed, 105 insertions(+), 9 deletions(-) > >> > >> diff --git a/Makefile b/Makefile > >> index ba06313..5c188d5 100644 > >> --- a/Makefile > >> +++ b/Makefile > >> @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb > >> OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) > >> > >> # Setup compilation commands > >> +ifneq ($(LLVM),) > >> +CC = clang > >> +AR = llvm-ar > >> +LD = ld.lld > >> +OBJCOPY = llvm-objcopy > >> +else > >> ifdef CROSS_COMPILE > >> CC = $(CROSS_COMPILE)gcc > >> -CPP = $(CROSS_COMPILE)cpp > >> AR = $(CROSS_COMPILE)ar > >> LD = $(CROSS_COMPILE)ld > >> OBJCOPY = $(CROSS_COMPILE)objcopy > >> else > >> CC ?= gcc > >> -CPP ?= cpp > >> AR ?= ar > >> LD ?= ld > >> OBJCOPY ?= objcopy > >> endif > >> +endif > >> +CPP = $(CC) -E > >> AS = $(CC) > >> DTC = dtc > >> > >> -# Guess the compillers xlen > >> -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > >> +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) > >> +CC_IS_CLANG = y > >> +else > >> +CC_IS_CLANG = n > >> +endif > >> + > >> +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) > >> +LD_IS_LLD = y > >> +else > >> +LD_IS_LLD = n > >> +endif > >> + > >> +ifeq ($(CC_IS_CLANG),y) > >> +ifneq ($(CROSS_COMPILE),) > >> +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) > >> +endif > >> +endif > >> + > >> +# Guess the compiler's XLEN > >> +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > >> + > >> +# Guess the compiler's ABI and ISA > >> +ifneq ($(CC_IS_CLANG),y) > >> OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > >> OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > >> +endif > >> > >> # Setup platform XLEN > >> ifndef PLATFORM_RISCV_XLEN > >> @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN > >> endif > >> endif > >> > >> +ifeq ($(CC_IS_CLANG),y) > >> +ifeq ($(CROSS_COMPILE),) > >> +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf > >> +endif > >> +endif > >> + > >> +ifeq ($(LD_IS_LLD),y) > >> +RELAX_FLAG = -mno-relax > >> +USE_LD_FLAG = -fuse-ld=lld > >> +else > >> +USE_LD_FLAG = -fuse-ld=bfd > >> +endif > >> + > >> # Check whether the linker supports creating PIEs > >> -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > >> +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > > > > It looks like this combination does not work? Both > > riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be > > located. > > > > $ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c > > /dev/null -o /dev/null 2>&1 > > collect2: fatal error: cannot find 'ld' > > compilation terminated. > > > > So this will result in a PDE image, even though LLD supports PIE. > > Isn’t this the same issue as you reported elsewhere, and will > ultimately result in the same error, this time not suppressed, at link > time too? Yeah, the same issue I reported in the cover letter. Sorry :) So I followed your advice in that thread, that I created a symbolic link in my cross-compile GCC toolchain bin directory, riscv64-unknown-elf-ld.lld to point to the actual LLVM ld.lld. Then this 'collect2' error is gone. I suggest we can document this in the doc for the mixed usage (GCC to compile + LLD to link). But still the build failed in the end: $ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1 mkdir -p `dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o`; echo " AS platform/generic/firmware/payloads/test_head.o"; riscv64-unknown-elf-gcc -g -Wall -nostdlib -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 -fpic -I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_head -c /home/test/git/opensbi/firmware/payloads/test_head.S -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o AS platform/generic/firmware/payloads/test_head.o mkdir -p `dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o`; echo " CC platform/generic/firmware/payloads/test_main.o"; riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 -fno-pie -no-pie -fPIE -pie -I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_main -c /home/test/git/opensbi/firmware/payloads/test_main.c -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o CC platform/generic/firmware/payloads/test_main.o mkdir -p `dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o`; echo " MERGE platform/generic/firmware/payloads/test.o"; ld.lld -r -b elf -m elf64lriscv /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o MERGE platform/generic/firmware/payloads/test.o mkdir -p `dirname /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf`; echo " ELF platform/generic/firmware/payloads/test.elf"; riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-save-restore -mstrict-align -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax -I/home/test/git/opensbi/platform/generic/include -I/home/test/git/opensbi/include -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 -DFW_JUMP_FDT_ADDR=0x82200000 -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 -fno-pie -no-pie -fPIE -pie /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a -fuse-ld=lld -Wl,--build-id=none -Wl,-N -Wl,--no-dynamic-linker -Wl,-pie -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf ELF platform/generic/firmware/payloads/test.elf riscv64-unknown-elf-ld.lld: error: test_main.c:(.entry+0x0): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax riscv64-unknown-elf-ld.lld: error: /home/test/git/opensbi/firmware/payloads/test_head.S:38:(.entry+0x84): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax riscv64-unknown-elf-ld.lld: error: /home/test/git/opensbi/firmware/payloads/test_head.S:39:(.entry+0x90): relocation R_RISCV_ALIGN requires unimplemented linker relaxation; recompile with -mno-relax collect2: error: ld returned 1 exit status So far the only fully working one is the complete LLVM build. The other 2 mixed usages are broken. It seems the toolchains guys created some sort of flexibility but opened lots of possibilities to break :) > Using GCC as a driver for LLD has always been a painful > combination, in part because historically there’s been very little > interest in supporting it from the GNU side; it wasn’t until a few > years ago (I think late 2018 or early 2019?) that -fuse-ld=lld was even > accepted as an option, and that was just a minimal patch that reused > all the logic for BFD and gold despite the fact that that’s > unnecessarily restrictive for LLD and doesn’t support cases like this > properly. Regards, Bin
On Sun, Jul 11, 2021 at 7:58 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: > > This is intended to mirror the Linux kernel. Building with CC=clang will > use Clang as the compiler but default to using the existing binutils. > Building with LLVM=1 will default to using Clang and LLVM binutils. > > Whilst GCC will accept the -N linker option and forward it on to the > linker, Clang will not, and so in order to support both compilers we > must use -Wl, to forward it to the linker as is required for most other > linker options. > > Note that there is currently a bug when using Clang as the compiler and > riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this > appears to be a bug in GNU binutils, but this could also be Clang or > OpenSBI at fault in some subtle way. Thus, for now, advise that this > combination be avoided. > > Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> I tried this patch using GCC linux toolchain and U-Boot SPL which worked fine for me. I don't have bare-metal GCC toolchain and LLVM but I assume LLVM is tested by Jessica. Reviewed-by: Anup Patel <anup.patel@wdc.com> Tested-by: Anup Patel <anup.patel@wdc.com> If there are other compile issues then please send fix as a separate patch. I insist more people to play around with LLVM and GCC bare-metal so that we catch issues early. Applied this patch to the riscv/opensbi repo. Thanks, Anup > --- > Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ > README.md | 48 ++++++++++++++++++++++++++++++++++++++-- > 2 files changed, 105 insertions(+), 9 deletions(-) > > diff --git a/Makefile b/Makefile > index ba06313..5c188d5 100644 > --- a/Makefile > +++ b/Makefile > @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb > OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) > > # Setup compilation commands > +ifneq ($(LLVM),) > +CC = clang > +AR = llvm-ar > +LD = ld.lld > +OBJCOPY = llvm-objcopy > +else > ifdef CROSS_COMPILE > CC = $(CROSS_COMPILE)gcc > -CPP = $(CROSS_COMPILE)cpp > AR = $(CROSS_COMPILE)ar > LD = $(CROSS_COMPILE)ld > OBJCOPY = $(CROSS_COMPILE)objcopy > else > CC ?= gcc > -CPP ?= cpp > AR ?= ar > LD ?= ld > OBJCOPY ?= objcopy > endif > +endif > +CPP = $(CC) -E > AS = $(CC) > DTC = dtc > > -# Guess the compillers xlen > -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) > +CC_IS_CLANG = y > +else > +CC_IS_CLANG = n > +endif > + > +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) > +LD_IS_LLD = y > +else > +LD_IS_LLD = n > +endif > + > +ifeq ($(CC_IS_CLANG),y) > +ifneq ($(CROSS_COMPILE),) > +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) > +endif > +endif > + > +# Guess the compiler's XLEN > +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) > + > +# Guess the compiler's ABI and ISA > +ifneq ($(CC_IS_CLANG),y) > OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) > +endif > > # Setup platform XLEN > ifndef PLATFORM_RISCV_XLEN > @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN > endif > endif > > +ifeq ($(CC_IS_CLANG),y) > +ifeq ($(CROSS_COMPILE),) > +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf > +endif > +endif > + > +ifeq ($(LD_IS_LLD),y) > +RELAX_FLAG = -mno-relax > +USE_LD_FLAG = -fuse-ld=lld > +else > +USE_LD_FLAG = -fuse-ld=bfd > +endif > + > # Check whether the linker supports creating PIEs > -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) > > # Setup list of objects.mk files > ifdef PLATFORM > @@ -197,7 +238,11 @@ else > endif > > # Setup compilation commands flags > -GENFLAGS = -I$(platform_src_dir)/include > +ifeq ($(CC_IS_CLANG),y) > +GENFLAGS += $(CLANG_TARGET) > +GENFLAGS += -Wno-unused-command-line-argument > +endif > +GENFLAGS += -I$(platform_src_dir)/include > GENFLAGS += -I$(include_dir) > ifneq ($(OPENSBI_VERSION_GIT),) > GENFLAGS += -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\"" > @@ -211,6 +256,7 @@ CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls > CFLAGS += -mno-save-restore -mstrict-align > CFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) > CFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) > +CFLAGS += $(RELAX_FLAG) > CFLAGS += $(GENFLAGS) > CFLAGS += $(platform-cflags-y) > CFLAGS += -fno-pie -no-pie > @@ -225,18 +271,24 @@ ASFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls > ASFLAGS += -mno-save-restore -mstrict-align > ASFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) > ASFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) > +ASFLAGS += $(RELAX_FLAG) > ASFLAGS += $(GENFLAGS) > ASFLAGS += $(platform-asflags-y) > ASFLAGS += $(firmware-asflags-y) > > ARFLAGS = rcs > > -ELFFLAGS += -Wl,--build-id=none -N -static-libgcc -lgcc > +ELFFLAGS += $(USE_LD_FLAG) > +ELFFLAGS += -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc > ELFFLAGS += $(platform-ldflags-y) > ELFFLAGS += $(firmware-ldflags-y) > > MERGEFLAGS += -r > +ifeq ($(LD_IS_LLD),y) > +MERGEFLAGS += -b elf > +else > MERGEFLAGS += -b elf$(PLATFORM_RISCV_XLEN)-littleriscv > +endif > MERGEFLAGS += -m elf$(PLATFORM_RISCV_XLEN)lriscv > > DTSCPPFLAGS = $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp > diff --git a/README.md b/README.md > index 03c02fb..b296f00 100644 > --- a/README.md > +++ b/README.md > @@ -96,8 +96,13 @@ Required Toolchain > ------------------ > > OpenSBI can be compiled natively or cross-compiled on a x86 host. For > -cross-compilation, you can build your own toolchain or just download > -a prebuilt one from the [Bootlin toolchain repository]. > +cross-compilation, you can build your own toolchain, download a prebuilt one > +from the [Bootlin toolchain repository] or install a distribution-provided > +toolchain; if you opt to use LLVM/Clang, most distribution toolchains will > +support cross-compiling for RISC-V using the same toolchain as your native > +LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the > +same binary, so is often an easy way to obtain a working cross-compilation > +toolchain. > > Please note that only a 64-bit version of the toolchain is available in > the Bootlin toolchain repository for now. > @@ -202,6 +207,45 @@ export PLATFORM_RISCV_XLEN=32 > > will generate 32-bit OpenSBI images. And vice vesa. > > +Building with Clang/LLVM > +------------------------ > + > +OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep > +the default binutils (which will still use the *CROSS_COMPILE* prefix if > +defined), override the *CC* make variable with: > +``` > +make CC=clang > +``` > + > +To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM* > +option with: > +``` > +make LLVM=1 > +``` > + > +When using Clang, *CROSS_COMPILE* often does not need to be defined unless > +using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be > +used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN* > +itself defaults to an undesired value then prefer setting that rather than the > +full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined, > +rather than being used as a prefix for the executable name, it will instead be > +passed via the `--target` option with the trailing `-` removed, so must be a > +valid triple. > + > +These can also be mixed; for example using a GCC cross-compiler but LLVM > +binutils would be: > +``` > +make CC=riscv64-unknown-elf-gcc LLVM=1 > +``` > + > +These variables must be passed for all the make invocations described in this > +document. > + > +NOTE: Using Clang with a `riscv*-linux-gnu` GNU binutils linker has been seen > +to produce broken binaries with missing relocations; it is therefore currently > +recommended that this combination be avoided or *FW_PIC=n* be used to disable > +building OpenSBI as a position-independent binary. > + > Contributing to OpenSBI > ----------------------- > > -- > 2.31.0 > > > -- > opensbi mailing list > opensbi@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/opensbi
On 11 Jul 2021, at 16:03, Bin Meng <bmeng.cn@gmail.com> wrote: > > On Sun, Jul 11, 2021 at 10:46 PM Jessica Clarke <jrtc27@jrtc27.com> wrote: >> >> On 11 Jul 2021, at 15:26, Bin Meng <bmeng.cn@gmail.com> wrote: >>> >>> On Sun, Jul 11, 2021 at 10:29 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: >>>> >>>> This is intended to mirror the Linux kernel. Building with CC=clang will >>>> use Clang as the compiler but default to using the existing binutils. >>>> Building with LLVM=1 will default to using Clang and LLVM binutils. >>>> >>>> Whilst GCC will accept the -N linker option and forward it on to the >>>> linker, Clang will not, and so in order to support both compilers we >>>> must use -Wl, to forward it to the linker as is required for most other >>>> linker options. >>>> >>>> Note that there is currently a bug when using Clang as the compiler and >>>> riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this >>>> appears to be a bug in GNU binutils, but this could also be Clang or >>>> OpenSBI at fault in some subtle way. Thus, for now, advise that this >>>> combination be avoided. >>>> >>>> Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> >>>> --- >>>> Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ >>>> README.md | 48 ++++++++++++++++++++++++++++++++++++++-- >>>> 2 files changed, 105 insertions(+), 9 deletions(-) >>>> >>>> diff --git a/Makefile b/Makefile >>>> index ba06313..5c188d5 100644 >>>> --- a/Makefile >>>> +++ b/Makefile >>>> @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb >>>> OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) >>>> >>>> # Setup compilation commands >>>> +ifneq ($(LLVM),) >>>> +CC = clang >>>> +AR = llvm-ar >>>> +LD = ld.lld >>>> +OBJCOPY = llvm-objcopy >>>> +else >>>> ifdef CROSS_COMPILE >>>> CC = $(CROSS_COMPILE)gcc >>>> -CPP = $(CROSS_COMPILE)cpp >>>> AR = $(CROSS_COMPILE)ar >>>> LD = $(CROSS_COMPILE)ld >>>> OBJCOPY = $(CROSS_COMPILE)objcopy >>>> else >>>> CC ?= gcc >>>> -CPP ?= cpp >>>> AR ?= ar >>>> LD ?= ld >>>> OBJCOPY ?= objcopy >>>> endif >>>> +endif >>>> +CPP = $(CC) -E >>>> AS = $(CC) >>>> DTC = dtc >>>> >>>> -# Guess the compillers xlen >>>> -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) >>>> +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) >>>> +CC_IS_CLANG = y >>>> +else >>>> +CC_IS_CLANG = n >>>> +endif >>>> + >>>> +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) >>>> +LD_IS_LLD = y >>>> +else >>>> +LD_IS_LLD = n >>>> +endif >>>> + >>>> +ifeq ($(CC_IS_CLANG),y) >>>> +ifneq ($(CROSS_COMPILE),) >>>> +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) >>>> +endif >>>> +endif >>>> + >>>> +# Guess the compiler's XLEN >>>> +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) >>>> + >>>> +# Guess the compiler's ABI and ISA >>>> +ifneq ($(CC_IS_CLANG),y) >>>> OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) >>>> OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) >>>> +endif >>>> >>>> # Setup platform XLEN >>>> ifndef PLATFORM_RISCV_XLEN >>>> @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN >>>> endif >>>> endif >>>> >>>> +ifeq ($(CC_IS_CLANG),y) >>>> +ifeq ($(CROSS_COMPILE),) >>>> +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf >>>> +endif >>>> +endif >>>> + >>>> +ifeq ($(LD_IS_LLD),y) >>>> +RELAX_FLAG = -mno-relax >>>> +USE_LD_FLAG = -fuse-ld=lld >>>> +else >>>> +USE_LD_FLAG = -fuse-ld=bfd >>>> +endif >>>> + >>>> # Check whether the linker supports creating PIEs >>>> -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) >>>> +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) >>> >>> It looks like this combination does not work? Both >>> riscv64-unknown-elf-gcc and LLVM are in my $PATH, but ld.lld cannot be >>> located. >>> >>> $ riscv64-unknown-elf-gcc -fPIE -fuse-ld=lld -nostdlib -Wl,-pie -x c >>> /dev/null -o /dev/null 2>&1 >>> collect2: fatal error: cannot find 'ld' >>> compilation terminated. >>> >>> So this will result in a PDE image, even though LLD supports PIE. >> >> Isn’t this the same issue as you reported elsewhere, and will >> ultimately result in the same error, this time not suppressed, at link >> time too? > > Yeah, the same issue I reported in the cover letter. Sorry :) > > So I followed your advice in that thread, that I created a symbolic > link in my cross-compile GCC toolchain bin directory, > riscv64-unknown-elf-ld.lld to point to the actual LLVM ld.lld. Then > this 'collect2' error is gone. I suggest we can document this in the > doc for the mixed usage (GCC to compile + LLD to link). > > But still the build failed in the end: > > $ make CC=riscv64-unknown-elf-gcc LLVM=1 PLATFORM=generic V=1 > > mkdir -p `dirname > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o`; > echo " AS platform/generic/firmware/payloads/test_head.o"; > riscv64-unknown-elf-gcc -g -Wall -nostdlib -fno-omit-frame-pointer > -fno-optimize-sibling-calls -mno-save-restore -mstrict-align > -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax > -I/home/test/git/opensbi/platform/generic/include > -I/home/test/git/opensbi/include > -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" > -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC > -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 > -DFW_JUMP_FDT_ADDR=0x82200000 > -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" > -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 -fpic > -I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_head -c > /home/test/git/opensbi/firmware/payloads/test_head.S -o > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o > AS platform/generic/firmware/payloads/test_head.o > mkdir -p `dirname > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o`; > echo " CC platform/generic/firmware/payloads/test_main.o"; > riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib > -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer > -fno-optimize-sibling-calls -mno-save-restore -mstrict-align > -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax > -I/home/test/git/opensbi/platform/generic/include > -I/home/test/git/opensbi/include > -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" > -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC > -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 > -DFW_JUMP_FDT_ADDR=0x82200000 > -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" > -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 > -fno-pie -no-pie -fPIE -pie > -I/home/test/git/opensbi/firmware/payloads -D__OBJNAME__=test_main -c > /home/test/git/opensbi/firmware/payloads/test_main.c -o > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o > CC platform/generic/firmware/payloads/test_main.o > mkdir -p `dirname > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o`; > echo " MERGE platform/generic/firmware/payloads/test.o"; ld.lld -r > -b elf -m elf64lriscv > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_head.o > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test_main.o > -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o > MERGE platform/generic/firmware/payloads/test.o > mkdir -p `dirname > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf`; > echo " ELF platform/generic/firmware/payloads/test.elf"; > riscv64-unknown-elf-gcc -g -Wall -Werror -ffreestanding -nostdlib > -fno-stack-protector -fno-strict-aliasing -O2 -fno-omit-frame-pointer > -fno-optimize-sibling-calls -mno-save-restore -mstrict-align > -mabi=lp64 -march=rv64imafdc -mcmodel=medany -mno-relax > -I/home/test/git/opensbi/platform/generic/include > -I/home/test/git/opensbi/include > -DOPENSBI_VERSION_GIT="\"v0.9-111-g401461d\"" > -I/home/test/git/opensbi/lib/utils/libfdt/ -DFW_PIC > -DFW_TEXT_START=0x80000000 -DFW_JUMP_ADDR=0x80200000 > -DFW_JUMP_FDT_ADDR=0x82200000 > -DFW_PAYLOAD_PATH=\"/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.bin\" > -DFW_PAYLOAD_OFFSET=0x200000 -DFW_PAYLOAD_FDT_ADDR=0x82200000 > -fno-pie -no-pie -fPIE -pie > /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.o > /home/test/git/opensbi/build/platform/generic/lib/libplatsbi.a > -fuse-ld=lld -Wl,--build-id=none -Wl,-N -Wl,--no-dynamic-linker > -Wl,-pie -Wl,-T/home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf.ld > -o /home/test/git/opensbi/build/platform/generic/firmware/payloads/test.elf > ELF platform/generic/firmware/payloads/test.elf > riscv64-unknown-elf-ld.lld: error: test_main.c:(.entry+0x0): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-relax > riscv64-unknown-elf-ld.lld: error: > /home/test/git/opensbi/firmware/payloads/test_head.S:38:(.entry+0x84): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-relax > riscv64-unknown-elf-ld.lld: error: > /home/test/git/opensbi/firmware/payloads/test_head.S:39:(.entry+0x90): > relocation R_RISCV_ALIGN requires unimplemented linker relaxation; > recompile with -mno-relax > collect2: error: ld returned 1 exit status > > So far the only fully working one is the complete LLVM build. The > other 2 mixed usages are broken. It seems the toolchains guys created > some sort of flexibility but opened lots of possibilities to break :) Ah, right, yes, that old GCC bug that only got fixed a couple of months ago[1]. When generating code with -mno-relax, GCC puts .option norelax in the generated assembly, and so doesn’t bother passing on -mno-relax to the assembler (I don’t think that flag even existed originally, only the assembly directive, but then people realised it was a good idea to be able to have one assembly file that can be used for both). This has the unfortunate effect that, when using GCC to assemble hand-written assembly, -mno-relax does nothing, and you have to pass -Wa,-mno-relax to manually forward it to the assembler. I’ll send a follow-up to add -Wa,-mno-relax to ASFLAGS for GCC+LLD. Jess [1] https://github.com/gcc-mirror/gcc/commit/3b0a7d624e64eeb81e4d5e8c62c46d86ef521857
On Sun, Jul 11, 2021 at 11:11 PM Anup Patel <anup@brainfault.org> wrote: > > On Sun, Jul 11, 2021 at 7:58 AM Jessica Clarke <jrtc27@jrtc27.com> wrote: > > > > This is intended to mirror the Linux kernel. Building with CC=clang will > > use Clang as the compiler but default to using the existing binutils. > > Building with LLVM=1 will default to using Clang and LLVM binutils. > > > > Whilst GCC will accept the -N linker option and forward it on to the > > linker, Clang will not, and so in order to support both compilers we > > must use -Wl, to forward it to the linker as is required for most other > > linker options. > > > > Note that there is currently a bug when using Clang as the compiler and > > riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this > > appears to be a bug in GNU binutils, but this could also be Clang or > > OpenSBI at fault in some subtle way. Thus, for now, advise that this > > combination be avoided. > > > > Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> > > I tried this patch using GCC linux toolchain and U-Boot SPL which > worked fine for me. I don't have bare-metal GCC toolchain and LLVM > but I assume LLVM is tested by Jessica. It's nothing related to bare-metal, but a mixed usage of GCC and LLVM (the 3rd combination in the README) > > Reviewed-by: Anup Patel <anup.patel@wdc.com> > Tested-by: Anup Patel <anup.patel@wdc.com> > > If there are other compile issues then please send fix as a separate > patch. I insist more people to play around with LLVM and GCC > bare-metal so that we catch issues early. > > Applied this patch to the riscv/opensbi repo. Oops, I was about to give my RB / TB tags, but that's not a big deal :) Regards, Bin
diff --git a/Makefile b/Makefile index ba06313..5c188d5 100644 --- a/Makefile +++ b/Makefile @@ -76,26 +76,54 @@ OPENSBI_VERSION_MINOR=`grep "define OPENSBI_VERSION_MINOR" $(include_dir)/sbi/sb OPENSBI_VERSION_GIT=$(shell if [ -d $(src_dir)/.git ]; then git describe 2> /dev/null; fi) # Setup compilation commands +ifneq ($(LLVM),) +CC = clang +AR = llvm-ar +LD = ld.lld +OBJCOPY = llvm-objcopy +else ifdef CROSS_COMPILE CC = $(CROSS_COMPILE)gcc -CPP = $(CROSS_COMPILE)cpp AR = $(CROSS_COMPILE)ar LD = $(CROSS_COMPILE)ld OBJCOPY = $(CROSS_COMPILE)objcopy else CC ?= gcc -CPP ?= cpp AR ?= ar LD ?= ld OBJCOPY ?= objcopy endif +endif +CPP = $(CC) -E AS = $(CC) DTC = dtc -# Guess the compillers xlen -OPENSBI_CC_XLEN := $(shell TMP=`$(CC) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) +ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),) +CC_IS_CLANG = y +else +CC_IS_CLANG = n +endif + +ifneq ($(shell $(LD) --version 2>&1 | head -n 1 | grep LLD),) +LD_IS_LLD = y +else +LD_IS_LLD = n +endif + +ifeq ($(CC_IS_CLANG),y) +ifneq ($(CROSS_COMPILE),) +CLANG_TARGET = --target=$(notdir $(CROSS_COMPILE:%-=%)) +endif +endif + +# Guess the compiler's XLEN +OPENSBI_CC_XLEN := $(shell TMP=`$(CC) $(CLANG_TARGET) -dumpmachine | sed 's/riscv\([0-9][0-9]\).*/\1/'`; echo $${TMP}) + +# Guess the compiler's ABI and ISA +ifneq ($(CC_IS_CLANG),y) OPENSBI_CC_ABI := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-abi=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) OPENSBI_CC_ISA := $(shell TMP=`$(CC) -v 2>&1 | sed -n 's/.*\(with\-arch=\([a-zA-Z0-9]*\)\).*/\2/p'`; echo $${TMP}) +endif # Setup platform XLEN ifndef PLATFORM_RISCV_XLEN @@ -106,8 +134,21 @@ ifndef PLATFORM_RISCV_XLEN endif endif +ifeq ($(CC_IS_CLANG),y) +ifeq ($(CROSS_COMPILE),) +CLANG_TARGET = --target=riscv$(PLATFORM_RISCV_XLEN)-unknown-elf +endif +endif + +ifeq ($(LD_IS_LLD),y) +RELAX_FLAG = -mno-relax +USE_LD_FLAG = -fuse-ld=lld +else +USE_LD_FLAG = -fuse-ld=bfd +endif + # Check whether the linker supports creating PIEs -OPENSBI_LD_PIE := $(shell $(CC) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) +OPENSBI_LD_PIE := $(shell $(CC) $(CLANG_TARGET) $(RELAX_FLAG) $(USE_LD_FLAG) -fPIE -nostdlib -Wl,-pie -x c /dev/null -o /dev/null >/dev/null 2>&1 && echo y || echo n) # Setup list of objects.mk files ifdef PLATFORM @@ -197,7 +238,11 @@ else endif # Setup compilation commands flags -GENFLAGS = -I$(platform_src_dir)/include +ifeq ($(CC_IS_CLANG),y) +GENFLAGS += $(CLANG_TARGET) +GENFLAGS += -Wno-unused-command-line-argument +endif +GENFLAGS += -I$(platform_src_dir)/include GENFLAGS += -I$(include_dir) ifneq ($(OPENSBI_VERSION_GIT),) GENFLAGS += -DOPENSBI_VERSION_GIT="\"$(OPENSBI_VERSION_GIT)\"" @@ -211,6 +256,7 @@ CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls CFLAGS += -mno-save-restore -mstrict-align CFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) CFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) +CFLAGS += $(RELAX_FLAG) CFLAGS += $(GENFLAGS) CFLAGS += $(platform-cflags-y) CFLAGS += -fno-pie -no-pie @@ -225,18 +271,24 @@ ASFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls ASFLAGS += -mno-save-restore -mstrict-align ASFLAGS += -mabi=$(PLATFORM_RISCV_ABI) -march=$(PLATFORM_RISCV_ISA) ASFLAGS += -mcmodel=$(PLATFORM_RISCV_CODE_MODEL) +ASFLAGS += $(RELAX_FLAG) ASFLAGS += $(GENFLAGS) ASFLAGS += $(platform-asflags-y) ASFLAGS += $(firmware-asflags-y) ARFLAGS = rcs -ELFFLAGS += -Wl,--build-id=none -N -static-libgcc -lgcc +ELFFLAGS += $(USE_LD_FLAG) +ELFFLAGS += -Wl,--build-id=none -Wl,-N -static-libgcc -lgcc ELFFLAGS += $(platform-ldflags-y) ELFFLAGS += $(firmware-ldflags-y) MERGEFLAGS += -r +ifeq ($(LD_IS_LLD),y) +MERGEFLAGS += -b elf +else MERGEFLAGS += -b elf$(PLATFORM_RISCV_XLEN)-littleriscv +endif MERGEFLAGS += -m elf$(PLATFORM_RISCV_XLEN)lriscv DTSCPPFLAGS = $(CPPFLAGS) -nostdinc -nostdlib -fno-builtin -D__DTS__ -x assembler-with-cpp diff --git a/README.md b/README.md index 03c02fb..b296f00 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,13 @@ Required Toolchain ------------------ OpenSBI can be compiled natively or cross-compiled on a x86 host. For -cross-compilation, you can build your own toolchain or just download -a prebuilt one from the [Bootlin toolchain repository]. +cross-compilation, you can build your own toolchain, download a prebuilt one +from the [Bootlin toolchain repository] or install a distribution-provided +toolchain; if you opt to use LLVM/Clang, most distribution toolchains will +support cross-compiling for RISC-V using the same toolchain as your native +LLVM/Clang toolchain due to LLVM's ability to support multiple backends in the +same binary, so is often an easy way to obtain a working cross-compilation +toolchain. Please note that only a 64-bit version of the toolchain is available in the Bootlin toolchain repository for now. @@ -202,6 +207,45 @@ export PLATFORM_RISCV_XLEN=32 will generate 32-bit OpenSBI images. And vice vesa. +Building with Clang/LLVM +------------------------ + +OpenSBI can also be built with Clang/LLVM. To build with just Clang but keep +the default binutils (which will still use the *CROSS_COMPILE* prefix if +defined), override the *CC* make variable with: +``` +make CC=clang +``` + +To build with a full LLVM-based toolchain, not just Clang, enable the *LLVM* +option with: +``` +make LLVM=1 +``` + +When using Clang, *CROSS_COMPILE* often does not need to be defined unless +using GNU binutils with prefixed binary names. *PLATFORM_RISCV_XLEN* will be +used to infer a default triple to pass to Clang, so if *PLATFORM_RISCV_XLEN* +itself defaults to an undesired value then prefer setting that rather than the +full triple via *CROSS_COMPILE*. If *CROSS_COMPILE* is nonetheless defined, +rather than being used as a prefix for the executable name, it will instead be +passed via the `--target` option with the trailing `-` removed, so must be a +valid triple. + +These can also be mixed; for example using a GCC cross-compiler but LLVM +binutils would be: +``` +make CC=riscv64-unknown-elf-gcc LLVM=1 +``` + +These variables must be passed for all the make invocations described in this +document. + +NOTE: Using Clang with a `riscv*-linux-gnu` GNU binutils linker has been seen +to produce broken binaries with missing relocations; it is therefore currently +recommended that this combination be avoided or *FW_PIC=n* be used to disable +building OpenSBI as a position-independent binary. + Contributing to OpenSBI -----------------------
This is intended to mirror the Linux kernel. Building with CC=clang will use Clang as the compiler but default to using the existing binutils. Building with LLVM=1 will default to using Clang and LLVM binutils. Whilst GCC will accept the -N linker option and forward it on to the linker, Clang will not, and so in order to support both compilers we must use -Wl, to forward it to the linker as is required for most other linker options. Note that there is currently a bug when using Clang as the compiler and riscv64-linux-gnu-ld as the linker for FW_PIC=y. At first glance this appears to be a bug in GNU binutils, but this could also be Clang or OpenSBI at fault in some subtle way. Thus, for now, advise that this combination be avoided. Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com> --- Makefile | 66 +++++++++++++++++++++++++++++++++++++++++++++++++------ README.md | 48 ++++++++++++++++++++++++++++++++++++++-- 2 files changed, 105 insertions(+), 9 deletions(-)