From patchwork Mon Sep 17 21:21:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matt Weber X-Patchwork-Id: 970830 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=rockwellcollins.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42DfG81jyYz9sCD for ; Tue, 18 Sep 2018 07:22:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 5775786B9A; Mon, 17 Sep 2018 21:22:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s9yoMIpA690B; Mon, 17 Sep 2018 21:22:03 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 39B8286B76; Mon, 17 Sep 2018 21:22:03 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 0BFE81C24C5 for ; Mon, 17 Sep 2018 21:21:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 080C8876C5 for ; Mon, 17 Sep 2018 21:21:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TbQ4JOQKiByT for ; Mon, 17 Sep 2018 21:21:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from ch3vs04.rockwellcollins.com (ch3vs04.rockwellcollins.com [205.175.226.52]) by whitealder.osuosl.org (Postfix) with ESMTPS id A2278875FB for ; Mon, 17 Sep 2018 21:21:55 +0000 (UTC) Received: from ofwch3n02.rockwellcollins.com (HELO ciulimr02.rockwellcollins.com) ([205.175.226.14]) by ch3vs04.rockwellcollins.com with ESMTP; 17 Sep 2018 16:21:54 -0500 X-Received: from largo.rockwellcollins.com (unknown [192.168.140.76]) by ciulimr02.rockwellcollins.com (Postfix) with ESMTP id 238292009F; Mon, 17 Sep 2018 16:21:54 -0500 (CDT) From: Matt Weber To: buildroot@buildroot.org Date: Mon, 17 Sep 2018 16:21:52 -0500 Message-Id: <1537219312-59962-5-git-send-email-matthew.weber@rockwellcollins.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1537219312-59962-1-git-send-email-matthew.weber@rockwellcollins.com> References: <1537219312-59962-1-git-send-email-matthew.weber@rockwellcollins.com> Subject: [Buildroot] [PATCH v7 4/4] support/testing/tests/core: SSP & hardening flags X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Catch the commonly used options of SSP, Relro, and fortify. Using the package targets of busybox and lighttpd. This can easily be expanded to a larger list. Signed-off-by: Matthew Weber --- Changes v1 -> v7 (link wrapper patchset) - None Original v4 -> v1 of link wrapper patchset - Added back in busybox test case v3 -> v4 - Removed commented out lines I missed when I removed busybox - Removed duplicate fortify assertion test v2 -> v3 [Matt - Removed the busybox target as without the link time wrapper/specfile being merged the build will fail. Link time conflict between use of 'r' and pie. [Thomas - Add clarificaion of what checksec can test - Reworked using inheritance - Relocated json load (removed duplication) v1 -> v2 [Ricardo - Fix flake8 warnings - Added missing busyfox pie assertions - Updated the yml to include new test cases --- .gitlab-ci.yml | 6 ++ support/testing/tests/core/test_hardening.py | 110 +++++++++++++++++++++++++++ 2 files changed, 116 insertions(+) create mode 100644 support/testing/tests/core/test_hardening.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index d3d544b..8f325cd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -263,6 +263,12 @@ tests.boot.test_atf.TestATFAllwinner: *runtime_test tests.boot.test_atf.TestATFMarvell: *runtime_test tests.boot.test_atf.TestATFVexpress: *runtime_test tests.core.test_file_capabilities.TestFileCapabilities: *runtime_test +tests.core.test_hardening.TestFortifyConserv: *runtime_test +tests.core.test_hardening.TestFortifyNone: *runtime_test +tests.core.test_hardening.TestRelro: *runtime_test +tests.core.test_hardening.TestRelroPartial: *runtime_test +tests.core.test_hardening.TestSspNone: *runtime_test +tests.core.test_hardening.TestSspStrong: *runtime_test tests.core.test_post_scripts.TestPostScripts: *runtime_test tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test diff --git a/support/testing/tests/core/test_hardening.py b/support/testing/tests/core/test_hardening.py new file mode 100644 index 0000000..9f26962 --- /dev/null +++ b/support/testing/tests/core/test_hardening.py @@ -0,0 +1,110 @@ +import os +import subprocess +import json + +import infra.basetest + + +class TestHardeningBase(infra.basetest.BRTest): + config = \ + """ + BR2_powerpc64=y + BR2_powerpc_e5500=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y + BR2_TOOLCHAIN_EXTERNAL_URL="https://toolchains.bootlin.com/downloads/releases/toolchains/powerpc64-e5500/tarballs/powerpc64-e5500--glibc--stable-2018.02-2.tar.bz2" + BR2_TOOLCHAIN_EXTERNAL_GCC_6=y + BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y + BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y + BR2_TOOLCHAIN_EXTERNAL_CXX=y + BR2_PACKAGE_LIGHTTPD=y + BR2_PACKAGE_HOST_CHECKSEC=y + # BR2_TARGET_ROOTFS_TAR is not set + """ + + checksec_files = ["usr/sbin/lighttpd","bin/busybox"] + + def checksec_run(self, target_file): + filepath = os.path.join(self.builddir, "target", target_file) + cmd = ["host/bin/checksec", "--output", "json", "--file", filepath] + # Checksec is being used for elf file analysis only. There are no + # assumptions of target/run-time checks as part of this testing. + ret = subprocess.check_output(cmd, + stderr=open(os.devnull, "w"), + cwd=self.builddir, + env={"LANG": "C"}) + return json.loads(ret) + + +class TestRelro(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_RELRO_FULL=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertEqual(out["file"]["relro"], "full") + self.assertEqual(out["file"]["pie"], "yes") + + +class TestRelroPartial(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_RELRO_PARTIAL=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertEqual(out["file"]["relro"], "partial") + self.assertEqual(out["file"]["pie"], "no") + + +class TestSspNone(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_SSP_NONE=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertEqual(out["file"]["canary"], "no") + + +class TestSspStrong(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_SSP_STRONG=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertEqual(out["file"]["canary"], "yes") + + +class TestFortifyNone(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_FORTIFY_SOURCE_NONE=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertEqual(out["file"]["fortified"], "0") + + +class TestFortifyConserv(TestHardeningBase): + config = TestHardeningBase.config + \ + """ + BR2_FORTIFY_SOURCE_1=y + """ + + def test_run(self): + for f in self.checksec_files: + out = self.checksec_run(f) + self.assertNotEqual(out["file"]["fortified"], "0")