From patchwork Mon Nov 19 02:37:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 999625 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ytL92xHxz9s9m for ; Mon, 19 Nov 2018 13:38:09 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 638CB8708D; Mon, 19 Nov 2018 02:38:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZNFT7qFe2FU9; Mon, 19 Nov 2018 02:38:02 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id BFDED86F81; Mon, 19 Nov 2018 02:38:01 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id C6F401C2DC1 for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id C4A288647B for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1ZMYV8sboOcU for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs04.rockwellcollins.com (ch3vs04.rockwellcollins.com [205.175.226.52]) by hemlock.osuosl.org (Postfix) with ESMTPS id 87AD386DCF for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO ciulimr01.rockwellcollins.com) ([205.175.226.14]) by ch3vs04.rockwellcollins.com with ESMTP; 18 Nov 2018 20:37:50 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr01.rockwellcollins.com (Postfix) with ESMTP id 738C560174; Sun, 18 Nov 2018 20:37:49 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Sun, 18 Nov 2018 20:37:42 -0600 Message-Id: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 Subject: [Buildroot] [PATCH v3 1/6] package/compiler-rt: new package X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Romain Naour , Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This patch adds support for the compiler-rt (CLANG runtime) library. It builds a set of static libraries and installs them into the CLANG/LLVM toolchain resource folder. Signed-off-by: Matthew Weber Cc: Romain Naour Cc: Ricardo Martincoski Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com> --- Changes v1 -> v2 [Romain - Removed unnecessary host-cmake dependency v2 -> v3 [Romain - Update to use COMPILER_RT_INSTALL_PATH for library install [Ricardo - Fixed check-package extra line - Spelling in commit message --- DEVELOPERS | 1 + package/Config.in | 1 + package/compiler-rt/Config.in | 12 ++++++++++++ package/compiler-rt/compiler-rt.hash | 3 +++ package/compiler-rt/compiler-rt.mk | 33 +++++++++++++++++++++++++++++++++ 5 files changed, 50 insertions(+) create mode 100644 package/compiler-rt/Config.in create mode 100644 package/compiler-rt/compiler-rt.hash create mode 100644 package/compiler-rt/compiler-rt.mk diff --git a/DEVELOPERS b/DEVELOPERS index 53467da..e78d649 100644 --- a/DEVELOPERS +++ b/DEVELOPERS @@ -1360,6 +1360,7 @@ F: package/bridge-utils/ F: package/checkpolicy/ F: package/checksec/ F: package/cgroupfs-mount/ +F: package/compiler-rt/ F: package/crda/ F: package/devmem2/ F: package/dnsmasq/ diff --git a/package/Config.in b/package/Config.in index b60e770..73ddc2d 100644 --- a/package/Config.in +++ b/package/Config.in @@ -1602,6 +1602,7 @@ menu "Other" source "package/clapack/Config.in" source "package/classpath/Config.in" source "package/cmocka/Config.in" + source "package/compiler-rt/Config.in" source "package/cppcms/Config.in" source "package/cracklib/Config.in" source "package/dawgdic/Config.in" diff --git a/package/compiler-rt/Config.in b/package/compiler-rt/Config.in new file mode 100644 index 0000000..e15d3aa --- /dev/null +++ b/package/compiler-rt/Config.in @@ -0,0 +1,12 @@ +config BR2_PACKAGE_COMPILER_RT + bool "compiler-rt" + depends on BR2_PACKAGE_LLVM + help + A collection of runtime libraries primarily used by clang and + llvm to provide builtins, sanitizer runtimes, and profiling + at runtime. + + https://compiler-rt.llvm.org/ + +comment "compiler-rt requires llvm to be enabled" + depends on !BR2_PACKAGE_LLVM diff --git a/package/compiler-rt/compiler-rt.hash b/package/compiler-rt/compiler-rt.hash new file mode 100644 index 0000000..df6ed19 --- /dev/null +++ b/package/compiler-rt/compiler-rt.hash @@ -0,0 +1,3 @@ +# Locally computed: +sha256 bdec7fe3cf2c85f55656c07dfb0bd93ae46f2b3dd8f33ff3ad6e7586f4c670d6 compiler-rt-7.0.0.src.tar.xz +sha256 417541d990edb3f96327ac03cb67e52eac80fc5c3e7afc69213cd04d7c3b9b27 LICENSE.TXT diff --git a/package/compiler-rt/compiler-rt.mk b/package/compiler-rt/compiler-rt.mk new file mode 100644 index 0000000..fc64707 --- /dev/null +++ b/package/compiler-rt/compiler-rt.mk @@ -0,0 +1,33 @@ +################################################################################ +# +# compiler-rt +# +################################################################################ + +# Compiler-RT should be bumped together with LLVM and Clang as the run-time is +# tied to the version of those tools +COMPILER_RT_VERSION = 7.0.0 +COMPILER_RT_SOURCE = compiler-rt-$(COMPILER_RT_VERSION).src.tar.xz +COMPILER_RT_SITE = http://llvm.org/releases/$(COMPILER_RT_VERSION) +COMPILER_RT_LICENSE = NCSA MIT +COMPILER_RT_LICENSE_FILES = LICENSE.TXT +COMPILER_RT_DEPENDENCIES = host-clang llvm + +COMPILER_RT_INSTALL_STAGING = YES +COMPILER_RT_INSTALL_TARGET = NO + +# The installation of the target runtime libraries is currently into the +# host-{clang,llvm} resources directory. The "resources" directory is loosely +# document and this location will probably need revisited when making the +# llvm/clang tools SDK relocatable. +COMPILER_RT_CONF_OPTS=-DCOMPILER_RT_INCLUDE_TESTS=ON \ + -DCOMPILER_RT_STANDALONE_BUILD=ON \ + -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE=$(GNU_TARGET_NAME) \ + -DLLVM_CONFIG_PATH=$(HOST_DIR)/usr/bin/llvm-config \ + -DCOMPILER_RT_INSTALL_PATH=$(HOST_DIR)/lib/clang/$(HOST_CLANG_VERSION)/ + +define COMPILER_RT_INSTALL_STAGING_CMDS + $(TARGET_MAKE_ENV) $(MAKE) -C $(@D) install/fast +endef + +$(eval $(cmake-package)) From patchwork Mon Nov 19 02:37:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 999623 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ytL72LLRz9s9m for ; Mon, 19 Nov 2018 13:38:07 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 7DE4886D91; Mon, 19 Nov 2018 02:38:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uIrnj4IePe+5; Mon, 19 Nov 2018 02:38:00 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id EE87C86889; Mon, 19 Nov 2018 02:37:59 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 15AF71BF2FD for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 12772865D4 for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BpsvIDW5GMoa for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs03.rockwellcollins.com (da1vs03.rockwellcollins.com [205.175.227.47]) by whitealder.osuosl.org (Postfix) with ESMTPS id 8C336865ED for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO ciulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs03.rockwellcollins.com with ESMTP; 18 Nov 2018 20:37:50 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr01.rockwellcollins.com (Postfix) with ESMTP id 91FA060207; Sun, 18 Nov 2018 20:37:49 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Sun, 18 Nov 2018 20:37:44 -0600 Message-Id: <1542595067-45972-3-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v3 3/6] testing/infra/builder: build with target and environment X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Ricardo Martincoski Make the builder able to call 'VAR1=1 make VAR2=2 target'. Allow to send extra parameters to be added to the end of make command line. It can be used for these purposes: - to configure a br2-external, by passing 'BR2_EXTERNAL="dir"'; - to specify a make target, such as 'foo-source'. Allow to add variables to the environment in which make runs. It can be used to override values from environment, such as 'BR2_DL_DIR="dl"'. This change will be needed when adding support for br2-externals to the test infra. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Signed-off-by: Matt Weber --- Changes v2 - New from: http://patchwork.ozlabs.org/patch/912351/ v2 -> v3 [Ricardo - Updated patch commit description context. --- support/testing/infra/builder.py | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-) diff --git a/support/testing/infra/builder.py b/support/testing/infra/builder.py index fc318fe..4d9c512 100644 --- a/support/testing/infra/builder.py +++ b/support/testing/infra/builder.py @@ -12,7 +12,18 @@ class Builder(object): self.builddir = builddir self.logfile = infra.open_log_file(builddir, "build", logtofile) - def configure(self): + def configure(self, make_extra_opts=[], make_extra_env={}): + """ + Configure the build. + + make_extra_opts: a list of arguments to be passed to the make + command. + e.g. make_extra_opts=["BR2_EXTERNAL=/path"] + + make_extra_env: a dict of variables to be appended (or replaced) + in the environment that calls make. + e.g. make_extra_env={"BR2_DL_DIR": "/path"} + """ if not os.path.isdir(self.builddir): os.makedirs(self.builddir) @@ -25,22 +36,41 @@ class Builder(object): self.logfile.flush() env = {"PATH": os.environ["PATH"]} + env.update(make_extra_env) + cmd = ["make", - "O={}".format(self.builddir), - "olddefconfig"] + "O={}".format(self.builddir)] + cmd += make_extra_opts + cmd += ["olddefconfig"] + ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile, env=env) if ret != 0: raise SystemError("Cannot olddefconfig") - def build(self): + def build(self, make_extra_opts=[], make_extra_env={}): + """ + Perform the build. + + make_extra_opts: a list of arguments to be passed to the make + command. It can include a make target. + e.g. make_extra_opts=["foo-source"] + + make_extra_env: a dict of variables to be appended (or replaced) + in the environment that calls make. + e.g. make_extra_env={"BR2_DL_DIR": "/path"} + """ env = {"PATH": os.environ["PATH"]} if "http_proxy" in os.environ: self.logfile.write("Using system proxy: " + os.environ["http_proxy"] + "\n") env['http_proxy'] = os.environ["http_proxy"] env['https_proxy'] = os.environ["http_proxy"] + env.update(make_extra_env) + cmd = ["make", "-C", self.builddir] + cmd += make_extra_opts + ret = subprocess.call(cmd, stdout=self.logfile, stderr=self.logfile, env=env) if ret != 0: From patchwork Mon Nov 19 02:37:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 999621 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ytKx3sbLz9sBQ for ; Mon, 19 Nov 2018 13:37:57 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 9BA9085BD3; Mon, 19 Nov 2018 02:37:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SSyS6Zabcrs0; Mon, 19 Nov 2018 02:37:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 1D2F985BF0; Mon, 19 Nov 2018 02:37:55 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 394181C2DC8 for ; Mon, 19 Nov 2018 02:37:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 35A6E865F8 for ; Mon, 19 Nov 2018 02:37:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KsaDjRKmW2h8 for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from da1vs03.rockwellcollins.com (da1vs03.rockwellcollins.com [205.175.227.47]) by whitealder.osuosl.org (Postfix) with ESMTPS id CB9F086625 for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) Received: from ofwda1n02.rockwellcollins.com (HELO ciulimr01.rockwellcollins.com) ([205.175.227.14]) by da1vs03.rockwellcollins.com with ESMTP; 18 Nov 2018 20:37:50 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr01.rockwellcollins.com (Postfix) with ESMTP id A358A6020D; Sun, 18 Nov 2018 20:37:49 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Sun, 18 Nov 2018 20:37:45 -0600 Message-Id: <1542595067-45972-4-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v3 4/6] testing/infra/basetest: support br2-external X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Ricardo Martincoski Some upcoming test cases can use one or more br2-external trees as fixtures that provide packages used only in runtime tests. Add support for br2-external into the BRTest class. Any test case can then provide a list of paths for be used as br2-external trees during the build of the image to be tested. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle Signed-off-by: Matt Weber --- Changes v2 - New from: http://patchwork.ozlabs.org/patch/912354/ v2 -> v3 [Ricardo - Updated patch commit description context. --- support/testing/infra/basetest.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/support/testing/infra/basetest.py b/support/testing/infra/basetest.py index 5014fef..8413943 100644 --- a/support/testing/infra/basetest.py +++ b/support/testing/infra/basetest.py @@ -30,6 +30,7 @@ MINIMAL_CONFIG = \ class BRTest(unittest.TestCase): config = None + br2_external = list() downloaddir = None outputdir = None logtofile = True @@ -58,7 +59,7 @@ class BRTest(unittest.TestCase): if not self.b.is_finished(): self.show_msg("Building") - self.b.configure() + self.b.configure(["BR2_EXTERNAL={}".format(":".join(self.br2_external))]) self.b.build() self.show_msg("Building done") From patchwork Mon Nov 19 02:37:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 999624 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ytL74jWlz9sBQ for ; Mon, 19 Nov 2018 13:38:07 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 32D2A86DCF; Mon, 19 Nov 2018 02:38:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XFZnrCfb1Iq2; Mon, 19 Nov 2018 02:38:00 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 9F50786F47; Mon, 19 Nov 2018 02:38:00 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 27E2F1C2DC1 for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2597185BF2 for ; Mon, 19 Nov 2018 02:37:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lTxOh9x3Igri for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs02.rockwellcollins.com (secvs02.rockwellcollins.com [205.175.225.241]) by fraxinus.osuosl.org (Postfix) with ESMTPS id A7DAA85699 for ; Mon, 19 Nov 2018 02:37:50 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO ciulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs02.rockwellcollins.com with ESMTP; 18 Nov 2018 20:37:49 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr01.rockwellcollins.com (Postfix) with ESMTP id B4C1F6020F; Sun, 18 Nov 2018 20:37:49 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Sun, 18 Nov 2018 20:37:46 -0600 Message-Id: <1542595067-45972-5-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v3 5/6] testing/tests: CLANG compiler-rt runtime test X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This patch adds a test case that 1) Builds the complete LLVM and CLANG set of host tools 2) Cross-compiles the compiler-rt runtime using CLANG 3) Builds a cross-compiled application using CLANG and the libfuzzer compiler-rt library. 4) Executes the fuzz application (part of the libfuzzer package) on target and checks expected output Signed-off-by: Matthew Weber Cc: Ricardo Martincoski Reviewed-by: Ricardo Martincoski [with the entire series applied on next branch: https://gitlab.com/RicardoMartincoski/buildroot/-/jobs/121908178 NOTE: this test case takes longer than test_rust, ~2 hours in my host machine with all tarballs previously downloaded] Tested-by: Ricardo Martincoski --- Changes v1 -> v2 [Ricardo - updated yml with test case - moved emulator launch cmd to test case from infra - defconfig in the order provided by savedefconfig - indent defconfig - add full package for test code to the test br2-external - consolidated to one class only v2 -> v3 - Added Review/tested by --- .gitlab-ci.yml | 1 + .../br2-external/clang-compiler-rt/Config.in | 1 + .../br2-external/clang-compiler-rt/external.desc | 1 + .../br2-external/clang-compiler-rt/external.mk | 1 + .../clang-compiler-rt/package/libfuzzer/Config.in | 7 ++++ .../package/libfuzzer/libfuzzer.hash | 2 + .../package/libfuzzer/libfuzzer.mk | 24 +++++++++++ support/testing/tests/package/test_clang.py | 46 ++++++++++++++++++++++ 8 files changed, 83 insertions(+) create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/Config.in create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.desc create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/external.mk create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash create mode 100644 support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk create mode 100644 support/testing/tests/package/test_clang.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c78f2ea..2f2125f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -309,6 +309,7 @@ tests.init.test_systemd.TestInitSystemSystemdRoNetworkd: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwFull: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwIfupdown: *runtime_test tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test +tests.package.test_clang.TestClangCompilerRT: *runtime_test tests.package.test_dropbear.TestDropbear: *runtime_test tests.package.test_ipython.TestIPythonPy2: *runtime_test tests.package.test_ipython.TestIPythonPy3: *runtime_test diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in new file mode 100644 index 0000000..e1f9f8c --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/Config.in @@ -0,0 +1 @@ +source "$BR2_EXTERNAL_CLANG_COMPILER_RT_PATH/package/libfuzzer/Config.in" diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc new file mode 100644 index 0000000..92df859 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.desc @@ -0,0 +1 @@ +name: CLANG_COMPILER_RT diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk new file mode 100644 index 0000000..6fa55c1 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/external.mk @@ -0,0 +1 @@ +include $(sort $(wildcard $(BR2_EXTERNAL_CLANG_COMPILER_RT_PATH)/package/*/*.mk)) diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in new file mode 100644 index 0000000..5af5d1b --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/Config.in @@ -0,0 +1,7 @@ +config BR2_PACKAGE_LIBFUZZER + bool "libfuzzer" + help + This is a set of tests (benchmarks) for fuzzing + engines (fuzzers). + + https://github.com/google/fuzzer-test-suite diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash new file mode 100644 index 0000000..6baf576 --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.hash @@ -0,0 +1,2 @@ +sha256 c0addb4d7f0447fc9fd7c80e5721fafe4c137f29a8ebd94c5fef7e1d6a2c944c libfuzzer-64e4d9aa19a8d33b61882154addbf8419d7416e1.tar.gz +sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 LICENSE diff --git a/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk new file mode 100644 index 0000000..7d6f83b --- /dev/null +++ b/support/testing/tests/package/br2-external/clang-compiler-rt/package/libfuzzer/libfuzzer.mk @@ -0,0 +1,24 @@ +################################################################################ +# +# libfuzzer +# +################################################################################ + +LIBFUZZER_VERSION = 64e4d9aa19a8d33b61882154addbf8419d7416e1 +LIBFUZZER_SITE = $(call github,google,fuzzer-test-suite,$(LIBFUZZER_VERSION)) +LIBFUZZER_LICENSE = Apache-2.0 +LIBFUZZER_LICENSE_FILES = LICENSE +LIBFUZZER_DEPENDENCIES = compiler-rt + +define LIBFUZZER_BUILD_CMDS + $(HOST_DIR)/bin/clang++ -mcpu=cortex-a53 --sysroot=$(STAGING_DIR) \ + -B $(HOST_DIR)/opt/ext-toolchain -fsanitize=address,fuzzer \ + $(@D)/tutorial/fuzz_me.cc \ + -o $(@D)/fuzz_me +endef + +define LIBFUZZER_INSTALL_TARGET_CMDS + $(INSTALL) -D -m 755 $(@D)/fuzz_me $(TARGET_DIR)/usr/bin/fuzz_me +endef + +$(eval $(generic-package)) diff --git a/support/testing/tests/package/test_clang.py b/support/testing/tests/package/test_clang.py new file mode 100644 index 0000000..84a2528 --- /dev/null +++ b/support/testing/tests/package/test_clang.py @@ -0,0 +1,46 @@ +import os + +import infra.basetest + +FUZZ_TIMEOUT = 120 + + +class TestClangCompilerRT(infra.basetest.BRTest): + br2_external = [infra.filepath("tests/package/br2-external/clang-compiler-rt")] + config = \ + """ + BR2_aarch64=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.16.7" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config" + BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y + BR2_PACKAGE_COMPILER_RT=y + BR2_PACKAGE_LLVM=y + BR2_TARGET_ROOTFS_CPIO=y + BR2_TARGET_ROOTFS_CPIO_GZIP=y + # BR2_TARGET_ROOTFS_TAR is not set + BR2_PACKAGE_LIBFUZZER=y + """ + + def login(self): + img = os.path.join(self.builddir, "images", "rootfs.cpio.gz") + kern = os.path.join(self.builddir, "images", "Image") + # Sanitizers overallocate memory and the minimum that seemed to work was 512MB + self.emulator.boot(arch="aarch64", + kernel=kern, + kernel_cmdline=["console=ttyAMA0"], + options=["-M", "virt", "-cpu", "cortex-a53", "-m", "512", "-initrd", img]) + self.emulator.login() + + def test_run(self): + self.login() + + # The test case verifies both that the application executes and that + # the symbolizer is working to decode the stack trace + cmd = "fuzz_me 2>&1 | grep _M_replace" + _, exit_code = self.emulator.run(cmd, FUZZ_TIMEOUT) + self.assertEqual(exit_code, 0) From patchwork Mon Nov 19 02:37:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 999622 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42ytL200F9z9sBQ for ; Mon, 19 Nov 2018 13:38:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id A453422639; Mon, 19 Nov 2018 02:37:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P6Kr-eA6eO25; Mon, 19 Nov 2018 02:37:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 16E1E220FA; Mon, 19 Nov 2018 02:37:54 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 2F9571BF2FD for ; Mon, 19 Nov 2018 02:37:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2D1D885C08 for ; Mon, 19 Nov 2018 02:37:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i04obglz4_VM for ; Mon, 19 Nov 2018 02:37:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from secvs02.rockwellcollins.com (secvs02.rockwellcollins.com [205.175.225.241]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 18ACD85BF0 for ; Mon, 19 Nov 2018 02:37:51 +0000 (UTC) Received: from ofwgwc03.rockwellcollins.com (HELO ciulimr01.rockwellcollins.com) ([205.175.225.12]) by secvs02.rockwellcollins.com with ESMTP; 18 Nov 2018 20:37:49 -0600 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr01.rockwellcollins.com (Postfix) with ESMTP id C223960210; Sun, 18 Nov 2018 20:37:49 -0600 (CST) From: Matt Weber To: buildroot@buildroot.org Date: Sun, 18 Nov 2018 20:37:47 -0600 Message-Id: <1542595067-45972-6-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1542595067-45972-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v3 6/6] llvm/clang: add note about version bumping together X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Matthew Weber Acked-by: Romain Naour Cc: Ricardo Martincoski --- Changes v1 -> v2 - None v2 -> v3 [Ricardo - Added compiler-rt to list of items to bump. I originaly had kept this patch independent of the compiler-rt series but since compiler-rt is first in the series, this one may as well include it in the statements --- package/clang/clang.mk | 1 + package/llvm/llvm.mk | 1 + 2 files changed, 2 insertions(+) diff --git a/package/clang/clang.mk b/package/clang/clang.mk index f9d4ae3..461d5f9 100644 --- a/package/clang/clang.mk +++ b/package/clang/clang.mk @@ -4,6 +4,7 @@ # ################################################################################ +# LLVM, Clang and Compiler-RT should be version bumped together CLANG_VERSION = 7.0.0 CLANG_SITE = http://llvm.org/releases/$(CLANG_VERSION) CLANG_SOURCE = cfe-$(CLANG_VERSION).src.tar.xz diff --git a/package/llvm/llvm.mk b/package/llvm/llvm.mk index 79477f0..14f317d 100644 --- a/package/llvm/llvm.mk +++ b/package/llvm/llvm.mk @@ -4,6 +4,7 @@ # ################################################################################ +# LLVM, Clang and Compiler-RT should be version bumped together LLVM_VERSION = 7.0.0 LLVM_SITE = http://llvm.org/releases/$(LLVM_VERSION) LLVM_SOURCE = llvm-$(LLVM_VERSION).src.tar.xz