From patchwork Sun Aug 4 07:45:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Naour X-Patchwork-Id: 1141682 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=smile.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=smile-fr.20150623.gappssmtp.com header.i=@smile-fr.20150623.gappssmtp.com header.b="HVjqZeGM"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 461XyN2HMHz9sMr for ; Sun, 4 Aug 2019 17:46:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 8985F20002; Sun, 4 Aug 2019 07:46:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SZ1ajEpRTPkF; Sun, 4 Aug 2019 07:45:56 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 5784B20027; Sun, 4 Aug 2019 07:45:56 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 173511BF3C0 for ; Sun, 4 Aug 2019 07:45:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 1461586AF0 for ; Sun, 4 Aug 2019 07:45:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r0gIlgJFg2Bu for ; Sun, 4 Aug 2019 07:45:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by whitealder.osuosl.org (Postfix) with ESMTPS id 924B086AFE for ; Sun, 4 Aug 2019 07:45:53 +0000 (UTC) Received: by mail-wm1-f65.google.com with SMTP id a15so71919064wmj.5 for ; Sun, 04 Aug 2019 00:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=z2cs5+SYXkJINcF/8cYIo+Gs1fi9iFIkTCVG3vY1BtQ=; b=HVjqZeGMBqqIH+4jV8ua7ciW3o6BGksmPG59qhd11o4goFZ69senEPL/VwdBY8KLbR 2rNsLX4nOdMT9Jq+D9V3DPqZhXlgIZNN9iqnZH/1MauiBJa6ZafHRcxhpsGCx7NWIZia JEUXIjCo81AuWM0pBJGOs36y3VojdWYiu144rom0/Sx5p+WvXP2q6/fqldKADXNH5UZf mapmIjj62RMsvnoK7T9s0q6e+ivY8tMAw50c8sDvJVVyrmaUDo0m20FTvT3q11V66H0f QfAJk57C3M0au4NsmjJIV1wGP752KOPDp/oVp0atf0U6IMhE/BvHKigBnkv2lbKK/Q9x cckg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=z2cs5+SYXkJINcF/8cYIo+Gs1fi9iFIkTCVG3vY1BtQ=; b=l92rPLFE6ymXXkEoUQmnpsnVH1Z2bN8K0jqiIUILLKiw6hM3tCFBiieQHWHrLSdrG2 j3/wDmC/Yomw9apiU4mh1sAivbPIokAcIGAvsNeek7CkQt4SL2WJdMw7tSx273sxVuV2 DNFSr8na7IQd6VzioiH7hzxoxtMMnDlhpdc/sYKXzi/XDFKqnf8IGE1Pr43xlQDqpBNS EQ0EPGuPDj99eUMeYAInYcJ5ub7PRLbtucw0mk10VmyGkIUNc7O3q7RokPvwwX0nShZr J6ZetAO/yfF9o0czCMBjzrMF+S0Y1e2Mh/QOEvZF90YaxrG4JChvgvokdjHIddHEujix whew== X-Gm-Message-State: APjAAAVKESDlaQxW0yM20hEciYKKk1ocVKUaDzX30lG/SykOVKUIMahG 5zLBYMi2FdPFXeAEjVWsH7LWukL+1Q4= X-Google-Smtp-Source: APXvYqx3VUhObirZSggZAOIIoFfoeZhYjLA5cpWAEqmIgqSl55bk4yXQBpHR0nMKZymdaNfO3xICiA== X-Received: by 2002:a1c:e409:: with SMTP id b9mr12181248wmh.110.1564904750955; Sun, 04 Aug 2019 00:45:50 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb19:8acf:5600:26a7:892e:5434:faac]) by smtp.gmail.com with ESMTPSA id o6sm174848081wra.27.2019.08.04.00.45.50 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 04 Aug 2019 00:45:50 -0700 (PDT) From: Romain Naour To: buildroot@buildroot.org Date: Sun, 4 Aug 2019 09:45:44 +0200 Message-Id: <20190804074545.28841-1-romain.naour@smile.fr> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [Buildroot] [WIP 1/2] toolchain/toolchain-external: copy gcc files needed to run clang on the host X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour , Thomas Petazzoni Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" To build libfuzzer package Matthew Weber noticed that (host) clang doesn't run on the host without "-B $(HOST_DIR)/opt/ext-toolchain" option. This option add a new search path for binaries and object files used implicitly. Without -B clang fail to link due to missing crtbeging.o file and libgcc: output/host/bin/aarch64-linux-gnu-ld: cannot find crtbegin.o: No such file or directory output/host/bin/aarch64-linux-gnu-ld: cannot find -lgcc Indeed, clang search path doesn't include the dafault cross-gcc's search paths: $ output/host/bin/clang -print-search-dirs programs: = output/host/bin:output/host/bin:/..//bin libraries: = output/host/lib/clang/8.0.0: output/host/bin/../lib64: /lib/../lib64: /usr/lib/../lib64: output/host/bin/../lib: /lib:/usr/lib Here is the same command for cross-gcc: $ output/host/bin/aarch64-linux-gnu-gcc -print-search-dirs install: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/ programs: = output/host/opt/ext-toolchain/bin/../libexec/gcc/aarch64-linux-gnu/8.3.0/: output/host/opt/ext-toolchain/bin/../libexec/gcc/: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/../../../../aarch64-linux-gnu/bin/aarch64-linux-gnu/8.3.0/: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/../../../../aarch64-linux-gnu/bin/ libraries: = output/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/: output/host/opt/ext-toolchain/bin/../lib/gcc/: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/../../../../aarch64-linux-gnu/lib/aarch64-linux-gnu/8.3.0/: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/../../../../aarch64-linux-gnu/lib/../lib64/: output/host/aarch64-buildroot-linux-gnu/sysroot/lib/aarch64-linux-gnu/8.3.0/: output/host/aarch64-buildroot-linux-gnu/sysroot/lib/../lib64/: output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/aarch64-linux-gnu/8.3.0/: output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/../lib64/: output/host/opt/ext-toolchain/bin/../lib/gcc/aarch64-linux-gnu/8.3.0/../../../../aarch64-linux-gnu/lib/: output/host/aarch64-buildroot-linux-gnu/sysroot/lib/: output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib/ We can see that gcc default search path contains "output/host/opt/ext-toolchain" directory where the external toolchain has been extracted. Since we want to use clang without additional option like -B, add some hook to copy crt*.o files in the clang default patch (output/host/lib/clang/8.0.0) and libgcc.a to STAGING_DIR. Doing so allow to build fuzzme program without "-B $(HOST_DIR)/opt/ext-toolchain" We fix this issue only the external toolchain backend, similar hooks are needed for internal toolchain backend. Signed-off-by: Romain Naour Cc: Matthew Weber Cc: Thomas Petazzoni --- This patch duplicate some functions from helpers.mk, but this allow to show what we need to do to fix this issue. The compiler-rt series from Matt needs to be applied first: http://patchwork.ozlabs.org/patch/1093806/ http://patchwork.ozlabs.org/patch/1093808/ http://patchwork.ozlabs.org/patch/1093805/ --- .../pkg-toolchain-external.mk | 42 ++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/toolchain/toolchain-external/pkg-toolchain-external.mk b/toolchain/toolchain-external/pkg-toolchain-external.mk index c3ddff263f..33c732c9ae 100644 --- a/toolchain/toolchain-external/pkg-toolchain-external.mk +++ b/toolchain/toolchain-external/pkg-toolchain-external.mk @@ -150,6 +150,45 @@ endif TOOLCHAIN_EXTERNAL_LIBS += $(call qstrip,$(BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS)) +# host clang needs "crt*.o" objects in it's default search path. +define TOOLCHAIN_EXTERNAL_INSTALL_GCC_OBJS_FOR_CLANG + DESTDIR="lib/clang/$(HOST_CLANG_VERSION)/" ; \ + mkdir -p $(HOST_DIR)/$${DESTDIR} ; \ + LIBPATHS=`find $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/lib/gcc/ -name "crt*.o" 2>/dev/null` ; \ + for LIBPATH in $${LIBPATHS} ; do \ + while true ; do \ + LIBNAME=`basename $${LIBPATH}`; \ + rm -fr $(HOST_DIR)/$${DESTDIR}/$${LIBNAME}; \ + if test -f $${LIBPATH}; then \ + $(INSTALL) -D -m0755 $${LIBPATH} $(HOST_DIR)/$${DESTDIR}/$${LIBNAME}; \ + break ; \ + else \ + exit -1; \ + fi; \ + done; \ + done +endef + +# host clang needs libgcc.a in STAGING_DIR +# opt/ext-toolchain/lib/gcc/aarch64-linux-gnu/8.3.0/libgcc.a +define TOOLCHAIN_EXTERNAL_INSTALL_GCC_LIBS_FOR_CLANG + LIBPATHS=`find $(TOOLCHAIN_EXTERNAL_INSTALL_DIR)/lib/gcc/ -name "libgcc.a" 2>/dev/null` ; \ + ARCH_LIB_DIR="$(call toolchain_find_libdir,$(TOOLCHAIN_EXTERNAL_CC) $(TOOLCHAIN_EXTERNAL_CFLAGS))" ; \ + for LIBPATH in $${LIBPATHS} ; do \ + while true ; do \ + LIBNAME=`basename $${LIBPATH}`; \ + DESTDIR=$${ARCH_LIB_DIR} ; \ + mkdir -p $(STAGING_DIR)/$${DESTDIR}; \ + rm -fr $(STAGING_DIR)/$${DESTDIR}/$${LIBNAME}; \ + if test -f $${LIBPATH}; then \ + $(INSTALL) -D -m0755 $${LIBPATH} $(STAGING_DIR)/$${DESTDIR}/$${LIBNAME}; \ + break ; \ + else \ + exit -1; \ + fi; \ + done; \ + done +endef # # Definition of the CFLAGS to use with the external toolchain, as well as the @@ -490,7 +529,6 @@ define TOOLCHAIN_EXTERNAL_FIXUP_UCLIBCNG_LDSO fi endef - ################################################################################ # inner-toolchain-external-package -- defines the generic installation rules # for external toolchain packages @@ -569,6 +607,8 @@ define $(2)_INSTALL_STAGING_CMDS $$(TOOLCHAIN_EXTERNAL_INSTALL_SYSROOT_LIBS) $$(TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER) $$(TOOLCHAIN_EXTERNAL_INSTALL_GDBINIT) + $$(TOOLCHAIN_EXTERNAL_INSTALL_GCC_OBJS_FOR_CLANG) + $$(TOOLCHAIN_EXTERNAL_INSTALL_GCC_LIBS_FOR_CLANG) endef # Even though we're installing things in both the staging, the host From patchwork Sun Aug 4 07:45:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Romain Naour X-Patchwork-Id: 1141681 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=smile.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=smile-fr.20150623.gappssmtp.com header.i=@smile-fr.20150623.gappssmtp.com header.b="friwbde2"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 461XyL1PT9z9s7T for ; Sun, 4 Aug 2019 17:46:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6DEB386B3B; Sun, 4 Aug 2019 07:45:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S4Y5ErWdxxNZ; Sun, 4 Aug 2019 07:45:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 71E1886AFE; Sun, 4 Aug 2019 07:45:55 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id D06CA1BF3C0 for ; Sun, 4 Aug 2019 07:45:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CD73186B3B for ; Sun, 4 Aug 2019 07:45:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ze7GZ2uZ1UON for ; Sun, 4 Aug 2019 07:45:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) by whitealder.osuosl.org (Postfix) with ESMTPS id 7AD5286AF0 for ; Sun, 4 Aug 2019 07:45:53 +0000 (UTC) Received: by mail-wr1-f51.google.com with SMTP id y4so81292665wrm.2 for ; Sun, 04 Aug 2019 00:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smile-fr.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WHGtRdQGzU7PLB58rQ6rqVUnb6AfHbErVpQevHjwEio=; b=friwbde2SRxOiuSHcWGmM5eEo6CQ/g1ZbEwycF32mOQlCqDMuZgoMmfvh2lD9YzEEL Y5kZI2+pGTQ+a5osaNRQx3D45BhpZ5Ds90hvzaE59f9wTEM9k+CRresxO9PurX0vl2Yy /li61+OBElCkDKETAgf1tFutekRoPsxpLrW3jCjm+eG5fkZxJzY6HHFSW3AVo/G/Gyuj tRZprZeT2xvxJ9QVTjDcbbVi0uAvwEwVAoBG9wmvrWDm7TgZO4G7W98hvHnnFMrAa90+ CzOEAfFI0MLfPQ7/nKeO73w6WYJpM36SYR2MZhkqfB66Sh4KovaKCGCgoRioptQvZLb3 HguA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WHGtRdQGzU7PLB58rQ6rqVUnb6AfHbErVpQevHjwEio=; b=ESyh7aM1GNWt+DgZtd5uI5nTL0HIBaRoJdtndIxVJHfR+PIIepufzafqnfbtznS3m3 PDRtRhpEvS+O7qn8cOLu46ksC+Sk+p4VUKCkLDk+UhvepTT7EaPoOsOCAUUEq8CP5zGF w7FDrdlYE6jZSgYEljZ3yv0R5PYOXXIyNhxzghcIaZCx1Df9eyVysPiKPPN//EaFLfBv 0fW55DTQjyMbLHqxl9wZeahzZt81r2YVrgk3pKBWQiXq12Q33psnB29BRkHDQREbOD/V 9sVbNXjy1ZZVJymmv/c3DgE5qat2o81h0+JhUR7IlennveVa4hmlo53ibA7BdvHqGMP3 7+9g== X-Gm-Message-State: APjAAAVM888+IkE3UORS38w5U5CxSmJ2VksJgb4mocwA7MsdElssjqG3 alj9VtTE3/IAanZAgLeSvdgioDGFhYs= X-Google-Smtp-Source: APXvYqzLP00YeTFjKuWG4XeCez7XaJCKd3uh56IcDzxVXwYYXRGHwTFAsJzFgp9NNa/qUDlfHSQkpA== X-Received: by 2002:adf:80e6:: with SMTP id 93mr117283038wrl.298.1564904751841; Sun, 04 Aug 2019 00:45:51 -0700 (PDT) Received: from localhost.localdomain ([2a01:cb19:8acf:5600:26a7:892e:5434:faac]) by smtp.gmail.com with ESMTPSA id o6sm174848081wra.27.2019.08.04.00.45.51 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 04 Aug 2019 00:45:51 -0700 (PDT) From: Romain Naour To: buildroot@buildroot.org Date: Sun, 4 Aug 2019 09:45:45 +0200 Message-Id: <20190804074545.28841-2-romain.naour@smile.fr> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190804074545.28841-1-romain.naour@smile.fr> References: <20190804074545.28841-1-romain.naour@smile.fr> MIME-Version: 1.0 Subject: [Buildroot] [WIP 2/2] testing/package/libfuzzer: remove -B option X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Since the previous patch, clang can be used without -B option. Signed-off-by: Romain Naour Cc: Matthew Weber --- .../clang-compiler-rt/package/libfuzzer/libfuzzer.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk index 7d6f83b4d3..3a9902b3e4 100644 --- a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk @@ -12,7 +12,7 @@ LIBFUZZER_DEPENDENCIES = compiler-rt define LIBFUZZER_BUILD_CMDS $(HOST_DIR)/bin/clang++ -mcpu=cortex-a53 --sysroot=$(STAGING_DIR) \ - -B $(HOST_DIR)/opt/ext-toolchain -fsanitize=address,fuzzer \ + -fsanitize=address,fuzzer \ $(@D)/tutorial/fuzz_me.cc \ -o $(@D)/fuzz_me endef