From patchwork Tue Feb 5 09:46:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas De Schampheleire X-Patchwork-Id: 1036572 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="hXLLRrSJ"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43v0862xLNz9sMM for ; Tue, 5 Feb 2019 20:46:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id CFB5386D33; Tue, 5 Feb 2019 09:46:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jXeOzuctM75N; Tue, 5 Feb 2019 09:46:12 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 059EF86433; Tue, 5 Feb 2019 09:46:12 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id E78361BF398 for ; Tue, 5 Feb 2019 09:46:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id E47F086433 for ; Tue, 5 Feb 2019 09:46:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3G2eTkk5cNuV for ; Tue, 5 Feb 2019 09:46:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f41.google.com (mail-wr1-f41.google.com [209.85.221.41]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 3B41986427 for ; Tue, 5 Feb 2019 09:46:06 +0000 (UTC) Received: by mail-wr1-f41.google.com with SMTP id l9so2784586wrt.13 for ; Tue, 05 Feb 2019 01:46:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5CyqQuZc1uLl8ACDakVgvIrKd3GpJi3kULiiF1IOorg=; b=hXLLRrSJlQGmps7fygE8qYZzLUddH6VX1e1F5uxKLFNwCrq/aNGriafvhOD0UnHs0X +3gAXKNJn87f3JG7zvfQLTennD5xPu2BvpNGXLCHG1k7RBdf6s+QUnbwU4/bobUZpt3m xtHeEk/qOaSGaEhPPxy5cVl0lifuf7zd6UosJIz3Dif93SBIcg8wcCH6SDhJb2YH9fUB EV3ZyFwN1/JyT32MTHFCARTpovwmaOPsR1zGKk49G2vBPcA1IXvrLhqMWLfHJzhocQ9P EDUB8+yktbd4jwbzXSP1d3/nxRyZciNfJPQucpqfst/r2x4ZBaUanotT1ULInPjec9bu XvAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5CyqQuZc1uLl8ACDakVgvIrKd3GpJi3kULiiF1IOorg=; b=EU7erewfDZZJjdWbUm88tn3I/R6QXlgmrgkn8dEOMpg9Czx7fRz9qf9AnCSTdZpaX9 D0/cLms2i6PNYW7bKPilvA4mGkO122OfAgQBNKhouSI/b2ud7jlaWKsV9yGPc/TPINs9 IZI48DpRQz+8xTPtdsBasxiumjic8xdELhzrvymEU0GTR4sXzLoJncB6oakd/wWok/el p5WcHNElXWPLRGBWjWyfwp89o+lau7IS93sl0OMb6ajUlUAr1hwHMcDVmKrqu/+MNr4J a06Pac5+Qq2H6dhRWKRCK9/FUeI6XpxZFzoLDhGDENxUjLIObtLlJKFr8Q+KmRWEh4MS UqzQ== X-Gm-Message-State: AHQUAuYuXnseaShxSeMtpqfAhh79cUIBpj1jwlxdls1e4Z7GtJTvYCTT Ducqx3/hs0bvRx2QSFlf0vZ93Xwc0FA= X-Google-Smtp-Source: AHgI3IaMlv0lXpmZHoOGhKrygsvINEBGpjICfRHf1szB9IxgG0BLrmFadWk5ImYlsZjVlZBEP7ANbw== X-Received: by 2002:a5d:4acb:: with SMTP id y11mr2809192wrs.281.1549359964244; Tue, 05 Feb 2019 01:46:04 -0800 (PST) Received: from localhost.localdomain ([2620:0:1055:fd00:57a8:6082:7724:fd53]) by smtp.gmail.com with ESMTPSA id r200sm28644167wmb.36.2019.02.05.01.46.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Feb 2019 01:46:03 -0800 (PST) From: Thomas De Schampheleire To: buildroot@buildroot.org Date: Tue, 5 Feb 2019 10:46:03 +0100 Message-Id: <20190205094603.28989-1-patrickdepinguin@gmail.com> X-Mailer: git-send-email 2.19.2 MIME-Version: 1.0 Subject: [Buildroot] [WIP] support/testing: add test to verify 'scp' download via BR2_PRIMARY_SITE X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski , Thomas De Schampheleire Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas De Schampheleire Recently it was found that the scp download infrastructure was broken. To avoid future failures, create a test that verifies that the scp command receives the expected arguments. Signed-off-by: Thomas De Schampheleire --- .../tests/download/br2-external/scp/Config.in | 0 .../download/br2-external/scp/external.desc | 1 + .../download/br2-external/scp/external.mk | 1 + .../br2-external/scp/package/nohash/nohash.mk | 10 +++ support/testing/tests/download/scp-wrapper | 62 +++++++++++++++++++ support/testing/tests/download/test_scp.py | 45 ++++++++++++++ 6 files changed, 119 insertions(+) create mode 100644 support/testing/tests/download/br2-external/scp/Config.in create mode 100644 support/testing/tests/download/br2-external/scp/external.desc create mode 100644 support/testing/tests/download/br2-external/scp/external.mk create mode 100644 support/testing/tests/download/br2-external/scp/package/nohash/nohash.mk create mode 100755 support/testing/tests/download/scp-wrapper create mode 100644 support/testing/tests/download/test_scp.py diff --git a/support/testing/tests/download/br2-external/scp/Config.in b/support/testing/tests/download/br2-external/scp/Config.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/support/testing/tests/download/br2-external/scp/external.desc b/support/testing/tests/download/br2-external/scp/external.desc new file mode 100644 index 0000000000..0ca0389a32 --- /dev/null +++ b/support/testing/tests/download/br2-external/scp/external.desc @@ -0,0 +1 @@ +name: SCP diff --git a/support/testing/tests/download/br2-external/scp/external.mk b/support/testing/tests/download/br2-external/scp/external.mk new file mode 100644 index 0000000000..2636c7da24 --- /dev/null +++ b/support/testing/tests/download/br2-external/scp/external.mk @@ -0,0 +1 @@ +include $(sort $(wildcard $(BR2_EXTERNAL_SCP_PATH)/package/*/*.mk)) diff --git a/support/testing/tests/download/br2-external/scp/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/scp/package/nohash/nohash.mk new file mode 100644 index 0000000000..cfd49370cd --- /dev/null +++ b/support/testing/tests/download/br2-external/scp/package/nohash/nohash.mk @@ -0,0 +1,10 @@ +################################################################################ +# +# nohash +# +################################################################################ + +NOHASH_VERSION = 123 +NOHASH_SITE = http://realsite.example.org/foo + +$(eval $(generic-package)) diff --git a/support/testing/tests/download/scp-wrapper b/support/testing/tests/download/scp-wrapper new file mode 100755 index 0000000000..3d09ebb121 --- /dev/null +++ b/support/testing/tests/download/scp-wrapper @@ -0,0 +1,62 @@ +#!/usr/bin/env python +import argparse +import os +import sys + +# expected command-line is: +# scp [options] +# +# environment variables set up by test suite: +# - SCP_WRAPPER_EXPECTED_HOST +# - SCP_WRAPPER_EXPECTED_BASEPATH +# - SCP_WRAPPER_EXPECTED_FILEPATH + +def main(): + parser = argparse.ArgumentParser(description='Scp wrapper for Buildroot tests') + parser.add_argument('source', help='source path') + parser.add_argument('target', help='target path') + + args = parser.parse_args() + + error = False + + expected_source_host_basepath = '%s:%s' % ( + os.environ['SCP_WRAPPER_EXPECTED_HOST'], + os.environ['SCP_WRAPPER_EXPECTED_BASEPATH']) + if not args.source.startswith(expected_source_host_basepath): + print("") + print("ERROR: unexpected source host and/or base path.") + print("Got : '%s'" % args.source) + print("Expected: '%s...'" % expected_source_host_basepath) + print("") + error = True + + expected_source_filepath = '%s' % (os.environ['SCP_WRAPPER_EXPECTED_FILEPATH']) + if not args.source.endswith(expected_source_filepath): + print("") + print("ERROR: unexpected source file path.") + print("Got : '%s'" % args.source) + print("Expected: '...%s'" % expected_source_filepath) + print("") + error = True + + # Really make sure that the source is a tarball. + # The test is using .tar.gz only so we don't check other extensions. + if not args.source.endswith('.tar.gz'): + print("") + print("ERROR: the source path does not seem like a tarball.") + print("Got : '%s'" % args.source) + print("Expected: '*.tar.gz'") + print("") + error = True + + if not error: + # create a dummy file to let the build succeed + open(args.target, 'a').close() + + return error + +if __name__ == "__main__": + error = main() + if error: + sys.exit(1) diff --git a/support/testing/tests/download/test_scp.py b/support/testing/tests/download/test_scp.py new file mode 100644 index 0000000000..95f76b718e --- /dev/null +++ b/support/testing/tests/download/test_scp.py @@ -0,0 +1,45 @@ +import infra +import os + +class TestScpPrimarySite(infra.basetest.BRTest): + host = 'user@server.example.org' + basepath = 'some/directory' + scp_wrapper = infra.filepath("tests/download/scp-wrapper") + br2_external = [infra.filepath("tests/download/br2-external/scp")] + + def __init__(self, names): + self.config = \ + """ + BR2_PRIMARY_SITE="scp://%s:%s" + BR2_PRIMARY_SITE_ONLY=y + BR2_BACKUP_SITE="" + BR2_SCP="%s" + """ % (self.host, self.basepath, self.scp_wrapper) + + super(TestScpPrimarySite, self).__init__(names) + + def tearDown(self): + self.show_msg("Cleaning up") + if self.b and not self.keepbuilds: + self.b.delete() + + def test_download(self): + package = 'nohash' + # store downloaded tarball inside the output dir so the test infra + # cleans it up at the end + env = { + "BR2_DL_DIR": os.path.join(self.builddir, "dl"), + "SCP_WRAPPER_EXPECTED_HOST": self.host, + "SCP_WRAPPER_EXPECTED_BASEPATH": self.basepath, + "SCP_WRAPPER_EXPECTED_FILEPATH": '%s-123.tar.gz' % package + } + try: + self.b.build(["{}-dirclean".format(package), + "{}-source".format(package)], + env) + except SystemError as e: # FIXME: introduce specific Exception classes + if str(e) == 'Build failed': + self.assertFalse('Download error, search for ERROR in the log') + else: + # an unexpected error + raise