| Message ID | 1612763186-18161-36-git-send-email-tsimpson@quicinc.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v8,01/35] Hexagon Update MAINTAINERS file | expand |
Hi Alessandro! On 2/8/21 6:46 AM, Taylor Simpson wrote: > From: Alessandro Di Federico <ale@rev.ng> > > Signed-off-by: Alessandro Di Federico <ale@rev.ng> > Tested-by: Taylor Simpson <tsimpson@quicinc.com> > --- > .../debian-hexagon-cross-build-local.docker | 18 +++ > .../debian-hexagon-cross.build-toolchain.sh | 141 +++++++++++++++++++++ > .../docker/dockerfiles/debian-hexagon-cross.docker | 1 + > tests/tcg/configure.sh | 4 + > 4 files changed, 164 insertions(+) > create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker > create mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh > create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross.docker > > diff --git a/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker b/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker > new file mode 100644 > index 0000000..355adbf > --- /dev/null > +++ b/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker > @@ -0,0 +1,18 @@ > +FROM debian:10 > + > +RUN apt update && \ > + DEBIAN_FRONTEND=noninteractive \ > + apt install -y --no-install-recommends \ > + ca-certificates build-essential cmake clang lld ninja-build \ > + flex bison pkg-config wget python3 rsync > + > +ENV TOOLCHAIN_INSTALL /usr > +ENV ROOTFS /usr > + > +ENV LLVM_URL https://github.com/llvm/llvm-project/archive/3d8149c2a1228609fd7d7c91a04681304a2f0ca9.tar.gz > +ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz > +ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz > + > +COPY debian-hexagon-cross.build-toolchain.sh /root/hexagon-toolchain/ I had to add this to build your image with QEMU build system: -- >8 -- diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include index 93b29ad823e..ba73fb4b1f0 100644 --- a/tests/docker/Makefile.include +++ b/tests/docker/Makefile.include @@ -146,6 +146,8 @@ docker-image-debian-riscv64-cross: docker-image-debian10 docker-image-debian-s390x-cross: docker-image-debian10 docker-image-debian-sh4-cross: docker-image-debian10 docker-image-debian-sparc64-cross: docker-image-debian10 +docker-image-debian-hexagon-cross-build-local: \ + EXTRA_FILES:=$(SRC_PATH)/tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh docker-image-travis: NOUSER=1 --- > +++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker > @@ -0,0 +1 @@ > +FROM revng/qemu:debian-hexagon-cross Once this patch gets merged, the image will be available on qemu/qemu:debian-hexagon-cross. I suppose for the final merge patch we want: - rm debian-hexagon-cross.docker - mv debian-hexagon-cross-build-local.docker debian-hexagon-cross.docker Is that correct? If so, can you amend that detail after the '---' separator of the commit description so we don't merge this patch as it? Also, can you add a job to build your image in .gitlab-ci.d/containers.yml? Meanwhile: Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Regards, Phil.
On 2/17/21 6:23 PM, Alessandro Di Federico via wrote: > On Sun, 14 Feb 2021 19:50:41 +0100 > Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: > >> Hi Alessandro! > > Hi Philippe! > >> I had to add this to build your image with QEMU build system: >> >> -- >8 -- >> >> diff --git a/tests/docker/Makefile.include >> b/tests/docker/Makefile.include index 93b29ad823e..ba73fb4b1f0 100644 >> --- a/tests/docker/Makefile.include >> +++ b/tests/docker/Makefile.include >> @@ -146,6 +146,8 @@ docker-image-debian-riscv64-cross: >> docker-image-debian10 docker-image-debian-s390x-cross: >> docker-image-debian10 docker-image-debian-sh4-cross: >> docker-image-debian10 docker-image-debian-sparc64-cross: >> docker-image-debian10 >> +docker-image-debian-hexagon-cross-build-local: >> \ >> + >> EXTRA_FILES:=$(SRC_PATH)/tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh >> >> docker-image-travis: NOUSER=1 >> >> --- > > OK, I'll drop `-local` (see below). > >>> +++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker >>> @@ -0,0 +1 @@ >>> +FROM revng/qemu:debian-hexagon-cross >> >> Once this patch gets merged, the image will be available on >> qemu/qemu:debian-hexagon-cross. >> >> I suppose for the final merge patch we want: >> - rm debian-hexagon-cross.docker >> - mv debian-hexagon-cross-build-local.docker >> debian-hexagon-cross.docker >> >> Is that correct? > > The `revng/qemu:debian-hexagon-cross` image was there only to let > people quickly try out a toolchain without building it from source. OK, not obvious from the commit description ;) > If building from source is OK with you, that works for us. > In any case, people will be able to use > `qemu/qemu:debian-hexagon-cross`, right? If you add a job to build it (see .gitlab-ci.d/containers.yml) it will then be pushed to the Gitlab QEMU namespace. So Yes. When we need a Docker image to build things, we first try to pull from QEMU namespace, then build locally if not available (see tests/docker/Makefile.include). >> If so, can you amend that detail after the '---' >> separator of the commit description so we don't merge this patch >> as it? > > I'm a bit confused, do you want me to send a patch corrected by hand > here? > I'll send a new patch tomorrow. I basically meant, "If this is the case and you want to keep it as it, please describe this better, so this patch doesn't get merged". >> Also, can you add a job to build your image in >> .gitlab-ci.d/containers.yml? > > OK, is there a quick way for me to test changes I make to that file? Open a Gitlab account, clone QEMU, push your change :) https://wiki.qemu.org/Testing/CI/GitLabCI Regards, Phil.
On 2/8/21 6:46 AM, Taylor Simpson wrote: > From: Alessandro Di Federico <ale@rev.ng> > > Signed-off-by: Alessandro Di Federico <ale@rev.ng> > Tested-by: Taylor Simpson <tsimpson@quicinc.com> > --- > .../debian-hexagon-cross-build-local.docker | 18 +++ > .../debian-hexagon-cross.build-toolchain.sh | 141 +++++++++++++++++++++ > .../docker/dockerfiles/debian-hexagon-cross.docker | 1 + > tests/tcg/configure.sh | 4 + > 4 files changed, 164 insertions(+) > create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker > create mode 100755 tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh > create mode 100644 tests/docker/dockerfiles/debian-hexagon-cross.docker ... > +build_llvm_clang() { > + fetch "$BASE/llvm-project" "$LLVM_URL" > + cdp "$BASE/build-llvm" > + > + cmake -G Ninja \ > + -DCMAKE_BUILD_TYPE=Release \ > + -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \ > + -DLLVM_ENABLE_LLD=ON \ > + -DLLVM_TARGETS_TO_BUILD="X86;Hexagon" \ Is the X86 target really required? > + -DLLVM_ENABLE_PROJECTS="clang;lld" \ > + "$BASE/llvm-project/llvm" > + ninja all install > + cd ${TOOLCHAIN_BIN} > + ln -sf clang hexagon-unknown-linux-musl-clang > + ln -sf clang++ hexagon-unknown-linux-musl-clang++ > + ln -sf llvm-ar hexagon-unknown-linux-musl-ar > + ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump > + ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy > + ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf > + ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib > + > + # workaround for now: > + cat <<EOF > hexagon-unknown-linux-musl.cfg > +-G0 --sysroot=${HEX_SYSROOT} > +EOF > +}
> -----Original Message----- > From: Philippe Mathieu-Daudé <philippe.mathieu.daude@gmail.com> On > Behalf Of Philippe Mathieu-Daudé > Sent: Saturday, February 27, 2021 8:10 AM > To: Taylor Simpson <tsimpson@quicinc.com>; qemu-devel@nongnu.org > Cc: richard.henderson@linaro.org; alex.bennee@linaro.org; > laurent@vivier.eu; ale@rev.ng; Brian Cain <bcain@quicinc.com>; Fam Zheng > <fam@euphon.net> > Subject: [EXT] Re: [PATCH v8 35/35] Add Dockerfile for hexagon > > On 2/8/21 6:46 AM, Taylor Simpson wrote: > > From: Alessandro Di Federico <ale@rev.ng> > > > > Signed-off-by: Alessandro Di Federico <ale@rev.ng> > > Tested-by: Taylor Simpson <tsimpson@quicinc.com> > > --- > > .../debian-hexagon-cross-build-local.docker | 18 +++ > > .../debian-hexagon-cross.build-toolchain.sh | 141 > +++++++++++++++++++++ > > .../docker/dockerfiles/debian-hexagon-cross.docker | 1 + > > tests/tcg/configure.sh | 4 + > > 4 files changed, 164 insertions(+) > > create mode 100644 > > tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker > > create mode 100755 > > tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh > > create mode 100644 > > tests/docker/dockerfiles/debian-hexagon-cross.docker > ... > > > +build_llvm_clang() { > > + fetch "$BASE/llvm-project" "$LLVM_URL" > > + cdp "$BASE/build-llvm" > > + > > + cmake -G Ninja \ > > + -DCMAKE_BUILD_TYPE=Release \ > > + -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \ > > + -DLLVM_ENABLE_LLD=ON \ > > + -DLLVM_TARGETS_TO_BUILD="X86;Hexagon" \ > > Is the X86 target really required? No, it isn't. When I was developing this recipe it was useful to have as another backend to compare against. Also many of the tests of target-independent functionality require the x86 backend, so it was useful during experimentation to check for regressions. > > + -DLLVM_ENABLE_PROJECTS="clang;lld" \ > > + "$BASE/llvm-project/llvm" > > + ninja all install > > + cd ${TOOLCHAIN_BIN} > > + ln -sf clang hexagon-unknown-linux-musl-clang ln -sf clang++ > > + hexagon-unknown-linux-musl-clang++ > > + ln -sf llvm-ar hexagon-unknown-linux-musl-ar ln -sf llvm-objdump > > + hexagon-unknown-linux-musl-objdump > > + ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy > > + ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf > > + ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib > > + > > + # workaround for now: > > + cat <<EOF > hexagon-unknown-linux-musl.cfg > > +-G0 --sysroot=${HEX_SYSROOT} > > +EOF > > +}
diff --git a/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker b/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker new file mode 100644 index 0000000..355adbf --- /dev/null +++ b/tests/docker/dockerfiles/debian-hexagon-cross-build-local.docker @@ -0,0 +1,18 @@ +FROM debian:10 + +RUN apt update && \ + DEBIAN_FRONTEND=noninteractive \ + apt install -y --no-install-recommends \ + ca-certificates build-essential cmake clang lld ninja-build \ + flex bison pkg-config wget python3 rsync + +ENV TOOLCHAIN_INSTALL /usr +ENV ROOTFS /usr + +ENV LLVM_URL https://github.com/llvm/llvm-project/archive/3d8149c2a1228609fd7d7c91a04681304a2f0ca9.tar.gz +ENV MUSL_URL https://github.com/quic/musl/archive/aff74b395fbf59cd7e93b3691905aa1af6c0778c.tar.gz +ENV LINUX_URL https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.6.18.tar.xz + +COPY debian-hexagon-cross.build-toolchain.sh /root/hexagon-toolchain/ + +RUN cd /root/hexagon-toolchain/ && ./debian-hexagon-cross.build-toolchain.sh && cd / && rm -rf /root/hexagon-toolchain/ diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh b/tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh new file mode 100755 index 0000000..a08c6cd --- /dev/null +++ b/tests/docker/dockerfiles/debian-hexagon-cross.build-toolchain.sh @@ -0,0 +1,141 @@ +#!/bin/bash + +set -e + +BASE=$(readlink -f ${PWD}) + +TOOLCHAIN_INSTALL=$(readlink -f "$TOOLCHAIN_INSTALL") +ROOTFS=$(readlink -f "$ROOTFS") + +TOOLCHAIN_BIN=${TOOLCHAIN_INSTALL}/bin +HEX_SYSROOT=${TOOLCHAIN_INSTALL}/hexagon-unknown-linux-musl +HEX_TOOLS_TARGET_BASE=${HEX_SYSROOT}/usr + +function cdp() { + DIR="$1" + mkdir -p "$DIR" + cd "$DIR" +} + +function fetch() { + DIR="$1" + URL="$2" + TEMP="$(readlink -f "$PWD/tmp.tar.gz")" + wget --quiet "$URL" -O "$TEMP" + cdp "$DIR" + tar xaf "$TEMP" --strip-components=1 + rm "$TEMP" + cd - +} + +build_llvm_clang() { + fetch "$BASE/llvm-project" "$LLVM_URL" + cdp "$BASE/build-llvm" + + cmake -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=${TOOLCHAIN_INSTALL} \ + -DLLVM_ENABLE_LLD=ON \ + -DLLVM_TARGETS_TO_BUILD="X86;Hexagon" \ + -DLLVM_ENABLE_PROJECTS="clang;lld" \ + "$BASE/llvm-project/llvm" + ninja all install + cd ${TOOLCHAIN_BIN} + ln -sf clang hexagon-unknown-linux-musl-clang + ln -sf clang++ hexagon-unknown-linux-musl-clang++ + ln -sf llvm-ar hexagon-unknown-linux-musl-ar + ln -sf llvm-objdump hexagon-unknown-linux-musl-objdump + ln -sf llvm-objcopy hexagon-unknown-linux-musl-objcopy + ln -sf llvm-readelf hexagon-unknown-linux-musl-readelf + ln -sf llvm-ranlib hexagon-unknown-linux-musl-ranlib + + # workaround for now: + cat <<EOF > hexagon-unknown-linux-musl.cfg +-G0 --sysroot=${HEX_SYSROOT} +EOF +} + +build_clang_rt() { + cdp "$BASE/build-clang_rt" + cmake -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_CONFIG_PATH="$BASE/build-llvm/bin/llvm-config" \ + -DCMAKE_ASM_FLAGS="-G0 -mlong-calls -fno-pic --target=hexagon-unknown-linux-musl " \ + -DCMAKE_SYSTEM_NAME=Linux \ + -DCMAKE_C_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \ + -DCMAKE_ASM_COMPILER="${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang" \ + -DCMAKE_INSTALL_PREFIX=${HEX_TOOLS_TARGET_BASE} \ + -DCMAKE_CROSSCOMPILING=ON \ + -DCMAKE_C_COMPILER_FORCED=ON \ + -DCMAKE_CXX_COMPILER_FORCED=ON \ + -DCOMPILER_RT_BUILD_BUILTINS=ON \ + -DCOMPILER_RT_BUILTINS_ENABLE_PIC=OFF \ + -DCMAKE_SIZEOF_VOID_P=4 \ + -DCOMPILER_RT_OS_DIR= \ + -DCAN_TARGET_hexagon=1 \ + -DCAN_TARGET_x86_64=0 \ + -DCOMPILER_RT_SUPPORTED_ARCH=hexagon \ + -DLLVM_ENABLE_PROJECTS="compiler-rt" \ + "$BASE/llvm-project/compiler-rt" + ninja install-compiler-rt +} + +build_musl_headers() { + fetch "$BASE/musl" "$MUSL_URL" + cd "$BASE/musl" + make clean + CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \ + CROSS_COMPILE=hexagon-unknown-linux-musl \ + LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \ + CROSS_CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \ + ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE} + PATH=${TOOLCHAIN_BIN}:$PATH make CROSS_COMPILE= install-headers + + cd ${HEX_SYSROOT}/.. + ln -sf hexagon-unknown-linux-musl hexagon +} + +build_kernel_headers() { + fetch "$BASE/linux" "$LINUX_URL" + mkdir -p "$BASE/build-linux" + cd "$BASE/linux" + make O=../build-linux ARCH=hexagon \ + KBUILD_CFLAGS_KERNEL="-mlong-calls" \ + CC=${TOOLCHAIN_BIN}/hexagon-unknown-linux-musl-clang \ + LD=${TOOLCHAIN_BIN}/ld.lld \ + KBUILD_VERBOSE=1 comet_defconfig + make mrproper + + cd "$BASE/build-linux" + make \ + ARCH=hexagon \ + CC=${TOOLCHAIN_BIN}/clang \ + INSTALL_HDR_PATH=${HEX_TOOLS_TARGET_BASE} \ + V=1 \ + headers_install +} + +build_musl() { + cd "$BASE/musl" + make clean + CROSS_COMPILE=hexagon-unknown-linux-musl- \ + AR=llvm-ar \ + RANLIB=llvm-ranlib \ + STRIP=llvm-strip \ + CC=clang \ + LIBCC=${HEX_TOOLS_TARGET_BASE}/lib/libclang_rt.builtins-hexagon.a \ + CFLAGS="-G0 -O0 -mv65 -fno-builtin -fno-rounding-math --target=hexagon-unknown-linux-musl" \ + ./configure --target=hexagon --prefix=${HEX_TOOLS_TARGET_BASE} + PATH=${TOOLCHAIN_BIN}/:$PATH make -j CROSS_COMPILE= install + cd ${HEX_TOOLS_TARGET_BASE}/lib + ln -sf libc.so ld-musl-hexagon.so + ln -sf ld-musl-hexagon.so ld-musl-hexagon.so.1 + cdp ${HEX_TOOLS_TARGET_BASE}/../lib + ln -sf ../usr/lib/ld-musl-hexagon.so.1 +} + +build_llvm_clang +build_kernel_headers +build_musl_headers +build_clang_rt +build_musl diff --git a/tests/docker/dockerfiles/debian-hexagon-cross.docker b/tests/docker/dockerfiles/debian-hexagon-cross.docker new file mode 100644 index 0000000..92c586e --- /dev/null +++ b/tests/docker/dockerfiles/debian-hexagon-cross.docker @@ -0,0 +1 @@ +FROM revng/qemu:debian-hexagon-cross diff --git a/tests/tcg/configure.sh b/tests/tcg/configure.sh index 35f1a77..4efb3f7 100755 --- a/tests/tcg/configure.sh +++ b/tests/tcg/configure.sh @@ -124,6 +124,10 @@ for target in $target_list; do container_image=fedora-cris-cross container_cross_cc=cris-linux-gnu-gcc ;; + hexagon-*) + container_image=debian-hexagon-cross + container_cross_cc=hexagon-unknown-linux-musl-clang + ;; hppa-*) container_image=debian-hppa-cross container_cross_cc=hppa-linux-gnu-gcc