From patchwork Sat May 12 02:58:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Martincoski X-Patchwork-Id: 912355 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="auMGrA+3"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40jWrh41XBz9s19 for ; Sat, 12 May 2018 12:59:16 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 040DF885CA; Sat, 12 May 2018 02:59:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8q1TlF-u6Uom; Sat, 12 May 2018 02:59:10 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id BC241848A1; Sat, 12 May 2018 02:59:10 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 92EC61C29B1 for ; Sat, 12 May 2018 02:59:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 8F67E207AE for ; Sat, 12 May 2018 02:59:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KG9rmreY7Vax for ; Sat, 12 May 2018 02:59:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-qt0-f172.google.com (mail-qt0-f172.google.com [209.85.216.172]) by silver.osuosl.org (Postfix) with ESMTPS id D1A081FEED for ; Sat, 12 May 2018 02:59:05 +0000 (UTC) Received: by mail-qt0-f172.google.com with SMTP id f13-v6so9578523qtp.10 for ; Fri, 11 May 2018 19:59:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=/eHn1C3SMz/ddhNSgChoOCdk/F54TBb1XOxHeY3L5P0=; b=auMGrA+3ku5agkmRFgg014KVaHWpwXg4KAw00Ju7djTAqbWUtr7TnywT8YNRmMe859 nqFpV+Z7yCZVED3CQxIQagu17wP/HLHD+81MK4MpLwfRfEIrWJIINg27hs6fFAe91H9I jxBaR/2As7FVuB7UWPmStEFX5buTxSKFaorDURjxT+YnxJ/dw3+F7joEk0DwNaPt89Dx 1GL23PottrZ5ONvceyBR+vvTDJc62SrbsCAFA8M4xU7iBqwvaGuIVPM88x/9B49p+zbd dH0zY0qjOnAjSZK0c1IlhspGpo454uPxW4In6X0LObk+QvEiUpgi9o19/EQ7KHBsCru/ zr6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=/eHn1C3SMz/ddhNSgChoOCdk/F54TBb1XOxHeY3L5P0=; b=XwfGCEFZbTEluhWAJPUrrfbn0f5BV/a/6jxRWsLUC4BA5U2fDDJV6pQZdk8f+AS65z YjlQ1sGsB1FerpNG7T0dnzv75yo7Ttm5/SJRyJ7PdXlx9QgvD8f2Ah7Ws+CTjJQFuIYj 8RXe4ielt60cEunN/LKqwUnA//sajNEQcNWNLTXoG20Y909gd9YtKLacf0iglwvIapA7 wXQ+UjtolPoQAKf4sz/ECrnIFo0jO4a4folnP40W0CHHDfZJDv9eXlRdH1KlZkRVZkDO JK/A8+3akxQ2KZoss1kROvScJmp3JJ2HTBZYBYv2GPYVl8cZmLw0W9UXB5a7J63OzMgT JXfw== X-Gm-Message-State: ALKqPwd2hIxAW2QIHwVDAHbSlWir6CBq8lHVuYol4ISg60SP4jO5JCoS OJ+KfSnIhiwG7mK01mMW/Op9oA== X-Google-Smtp-Source: AB8JxZpIAEuG/pc6uiXT2PHaqRKUolfokmo4CEl1cJUoH0S8vATBFKKLj27VYJa5DscoZ3MVVYxTww== X-Received: by 2002:a0c:9944:: with SMTP id i4-v6mr901220qvd.192.1526093944603; Fri, 11 May 2018 19:59:04 -0700 (PDT) Received: from ultri5.home ([189.114.97.193]) by smtp.gmail.com with ESMTPSA id l38-v6sm3832914qta.86.2018.05.11.19.59.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 May 2018 19:59:04 -0700 (PDT) From: Ricardo Martincoski To: buildroot@buildroot.org Date: Fri, 11 May 2018 23:58:27 -0300 Message-Id: <20180512025833.22998-5-ricardo.martincoski@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180512025833.22998-1-ricardo.martincoski@gmail.com> References: <20180512025833.22998-1-ricardo.martincoski@gmail.com> In-Reply-To: <20180429143349.24010-1-ricardo.martincoski@gmail.com> References: <20180429143349.24010-1-ricardo.martincoski@gmail.com> Subject: [Buildroot] [PATCH v5 04/10] testing/tests/download: add git hash test X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Martincoski , Thomas Petazzoni , Luca Ceresoli , "Yann E . MORIN" , Ricardo Martincoski MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Ricardo Martincoski Add one test case to ensure the hash is checked for git packages: - correct hash; - wrong hash; - no hash file. Add required infra: - a GitRemote class, that can start a git server in the host machine to emulate a remote git server under the control of the test; - a new base class, called GitTestBase, that inherits from BRTest and must be subclassed by all git test cases. Its setUp() method takes care of configuring the build with a br2-external, avoiding to hit http://sources.buildroot.net by using an empty BR2_BACKUP_SITE. It also avoids downloading not pre-installed dependencies (i.e. lzip) every time by calling 'make dependencies' using the common dl directory, and it instantiates the GitRemote object. Besides the Python scripts, add some fixtures used during the tests: - a br2-external (git-hash) with one package for each part of the test case; - a static git bare repo (repo.git) to be served using GitRemote class. Neither the br2-external nor the check hash functionalities are the subject of these tests per se, so for simplicity limit the check to the error codes and don't look for the messages in the log. Thanks to Arnout for the hint about how to add a bare repo to test. Signed-off-by: Ricardo Martincoski Cc: Arnout Vandecappelle --- Layout of static repo with the series applied until this patch: $ git -C support/testing/tests/download/git-remote/repo.git \ log --all --decorate --graph --oneline --decorate * a238b1d (HEAD -> master) initial Changes v4 -> v5: - no changes Changes v3 -> v4: - remove call to 'make dependencies'; - merge generic git base file to the main test case as the first was only used by the second; - use variable naming similar to builder; - move useful methods to the base class; - rename logfile to -gitremote (it will become handy when adding the http server on a later patch); - use more columns (our limit is now 132); - merge start to __init__ as it was called only there; - remove temp variable used only once (port_arg); - use a more future-proof string format for the port arg (it will become handy when adding the http server on a later patch); - use simpler code to raise exception if there is no free port; - fix the range for ports to use; - br2_external is now a list; Changes v2 -> v3: - renumber 5 -> 6; - rebase after removing patch 3 "allow to override logfile" from the series and adding new patches 4 "split runtime test from BRTest" and 5 "support br2-external"; - merge part of old patch 4 (very minimal GitTestBase) to old patch 5 (Arnout Vandecappelle); - merge and tweak part of commit log from old patch 4; - move GITREMOTE_PORT_NUMBER to external.mk (Arnout Vandecappelle); - use dosctrings; - start git remote from __init__ (Arnout Vandecappelle), but I kept a separate method and call start(); - remove dead code (Arnout Vandecappelle); - use the same git repo for all tests (Arnout Vandecappelle); - rename the file to test_git and merge the 3 test cases in one, since the feature "check hash for git packages" is only OK if all work well and also because the test does not take too much time; - update commit log accordingly. Changes v1 -> v2 (patch 4): - new patch Changes v1 -> v2 (patch 5): - use git daemon + git:// instead of ssh (Arnout); - remove __main__ handling since the test infra already does that; - use the logging support from test infra; - this patch is part of series 1/3 of a new version of http://patchwork.ozlabs.org/patch/690097/ --- .gitlab-ci.yml | 1 + support/testing/tests/download/__init__.py | 0 .../download/br2-external/git-hash/Config.in | 0 .../br2-external/git-hash/external.desc | 1 + .../br2-external/git-hash/external.mk | 4 ++ .../git-hash/package/bad/bad.hash | 1 + .../br2-external/git-hash/package/bad/bad.mk | 10 ++++ .../git-hash/package/good/good.hash | 1 + .../git-hash/package/good/good.mk | 10 ++++ .../git-hash/package/nohash/nohash.mk | 10 ++++ .../git-remote/repo.git/.gitattributes | 1 + .../tests/download/git-remote/repo.git/HEAD | 1 + .../tests/download/git-remote/repo.git/config | 4 ++ .../99/f2e3e1cb15f9b52fa29f66d380dda061d917ab | Bin 0 -> 49 bytes .../a2/38b1dfcd825d47d834af3c5223417c8411d90d | Bin 0 -> 152 bytes .../e7/9c5e8f964493290a409888d5413a737e8e5dd5 | Bin 0 -> 23 bytes .../git-remote/repo.git/refs/heads/master | 1 + support/testing/tests/download/gitremote.py | 44 ++++++++++++++++++ support/testing/tests/download/test_git.py | 43 +++++++++++++++++ 19 files changed, 132 insertions(+) create mode 100644 support/testing/tests/download/__init__.py create mode 100644 support/testing/tests/download/br2-external/git-hash/Config.in create mode 100644 support/testing/tests/download/br2-external/git-hash/external.desc create mode 100644 support/testing/tests/download/br2-external/git-hash/external.mk create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash create mode 100644 support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.hash create mode 100644 support/testing/tests/download/br2-external/git-hash/package/good/good.mk create mode 100644 support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk create mode 100644 support/testing/tests/download/git-remote/repo.git/.gitattributes create mode 100644 support/testing/tests/download/git-remote/repo.git/HEAD create mode 100644 support/testing/tests/download/git-remote/repo.git/config create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/a2/38b1dfcd825d47d834af3c5223417c8411d90d create mode 100644 support/testing/tests/download/git-remote/repo.git/objects/e7/9c5e8f964493290a409888d5413a737e8e5dd5 create mode 100644 support/testing/tests/download/git-remote/repo.git/refs/heads/master create mode 100644 support/testing/tests/download/gitremote.py create mode 100644 support/testing/tests/download/test_git.py diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e80491cdde..27b0255343 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -263,6 +263,7 @@ tests.core.test_rootfs_overlay.TestRootfsOverlay: *runtime_test tests.core.test_timezone.TestGlibcAllTimezone: *runtime_test tests.core.test_timezone.TestGlibcNonDefaultLimitedTimezone: *runtime_test tests.core.test_timezone.TestNoTimezone: *runtime_test +tests.download.test_git.TestGitHash: *runtime_test tests.fs.test_ext.TestExt2: *runtime_test tests.fs.test_ext.TestExt2r1: *runtime_test tests.fs.test_ext.TestExt3: *runtime_test diff --git a/support/testing/tests/download/__init__.py b/support/testing/tests/download/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/support/testing/tests/download/br2-external/git-hash/Config.in b/support/testing/tests/download/br2-external/git-hash/Config.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/support/testing/tests/download/br2-external/git-hash/external.desc b/support/testing/tests/download/br2-external/git-hash/external.desc new file mode 100644 index 0000000000..41316c8b25 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/external.desc @@ -0,0 +1 @@ +name: GIT_HASH diff --git a/support/testing/tests/download/br2-external/git-hash/external.mk b/support/testing/tests/download/br2-external/git-hash/external.mk new file mode 100644 index 0000000000..4646dfe2b0 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/external.mk @@ -0,0 +1,4 @@ +include $(sort $(wildcard $(BR2_EXTERNAL_GIT_HASH_PATH)/package/*/*.mk)) + +# Get the git server port number from the test infra +GITREMOTE_PORT_NUMBER ?= 9418 diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash new file mode 100644 index 0000000000..b9e1baec84 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.hash @@ -0,0 +1 @@ +sha256 0000000000000000000000000000000000000000000000000000000000000000 bad-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz diff --git a/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk new file mode 100644 index 0000000000..5497bd6bfe --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/package/bad/bad.mk @@ -0,0 +1,10 @@ +################################################################################ +# +# bad +# +################################################################################ + +BAD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d +BAD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git + +$(eval $(generic-package)) diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.hash b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash new file mode 100644 index 0000000000..9e92ab8ab9 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.hash @@ -0,0 +1 @@ +sha256 d00ae598e9e770d607621a86766030b42eaa58156cb8d482b043969da7963c23 good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz diff --git a/support/testing/tests/download/br2-external/git-hash/package/good/good.mk b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk new file mode 100644 index 0000000000..0f0eefd944 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/package/good/good.mk @@ -0,0 +1,10 @@ +################################################################################ +# +# good +# +################################################################################ + +GOOD_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d +GOOD_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git + +$(eval $(generic-package)) diff --git a/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk new file mode 100644 index 0000000000..1da19d88c6 --- /dev/null +++ b/support/testing/tests/download/br2-external/git-hash/package/nohash/nohash.mk @@ -0,0 +1,10 @@ +################################################################################ +# +# nohash +# +################################################################################ + +NOHASH_VERSION = a238b1dfcd825d47d834af3c5223417c8411d90d +NOHASH_SITE = git://localhost:$(GITREMOTE_PORT_NUMBER)/repo.git + +$(eval $(generic-package)) diff --git a/support/testing/tests/download/git-remote/repo.git/.gitattributes b/support/testing/tests/download/git-remote/repo.git/.gitattributes new file mode 100644 index 0000000000..eb50c64a21 --- /dev/null +++ b/support/testing/tests/download/git-remote/repo.git/.gitattributes @@ -0,0 +1 @@ +objects/*/* binary diff --git a/support/testing/tests/download/git-remote/repo.git/HEAD b/support/testing/tests/download/git-remote/repo.git/HEAD new file mode 100644 index 0000000000..cb089cd89a --- /dev/null +++ b/support/testing/tests/download/git-remote/repo.git/HEAD @@ -0,0 +1 @@ +ref: refs/heads/master diff --git a/support/testing/tests/download/git-remote/repo.git/config b/support/testing/tests/download/git-remote/repo.git/config new file mode 100644 index 0000000000..07d359d07c --- /dev/null +++ b/support/testing/tests/download/git-remote/repo.git/config @@ -0,0 +1,4 @@ +[core] + repositoryformatversion = 0 + filemode = true + bare = true diff --git a/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab b/support/testing/tests/download/git-remote/repo.git/objects/99/f2e3e1cb15f9b52fa29f66d380dda061d917ab new file mode 100644 index 0000000000000000000000000000000000000000..9db72668cf9374b0b85a25a19f30084fd460072d GIT binary patch literal 49 zcmb starting git remote with '{}'\n".format(" ".join(cmd))) + self.daemon = pexpect.spawn(cmd[0], cmd[1:], logfile=self.logfile) + ret = self.daemon.expect(["Ready to rumble", + "Address already in use"]) + if ret == 0: + self.port = port + return + raise SystemError("Could not find a free port to run git remote") + + def stop(self): + if self.daemon is None: + return + self.daemon.terminate(force=True) diff --git a/support/testing/tests/download/test_git.py b/support/testing/tests/download/test_git.py new file mode 100644 index 0000000000..14fc8e4da3 --- /dev/null +++ b/support/testing/tests/download/test_git.py @@ -0,0 +1,43 @@ +import os + +import infra +from gitremote import GitRemote + + +class GitTestBase(infra.basetest.BRTest): + config = \ + """ + BR2_BACKUP_SITE="" + """ + gitremotedir = infra.filepath("tests/download/git-remote") + gitremote = None + + def setUp(self): + super(GitTestBase, self).setUp() + self.gitremote = GitRemote(self.builddir, self.gitremotedir, self.logtofile) + + def tearDown(self): + self.show_msg("Cleaning up") + if self.gitremote: + self.gitremote.stop() + if self.b and not self.keepbuilds: + self.b.delete() + + def check_hash(self, package): + # store downloaded tarball inside the output dir so the test infra + # cleans it up at the end + env = {"BR2_DL_DIR": os.path.join(self.builddir, "dl"), + "GITREMOTE_PORT_NUMBER": str(self.gitremote.port)} + self.b.build(["{}-dirclean".format(package), + "{}-source".format(package)], + env) + + +class TestGitHash(GitTestBase): + br2_external = [infra.filepath("tests/download/br2-external/git-hash")] + + def test_run(self): + with self.assertRaises(SystemError): + self.check_hash("bad") + self.check_hash("good") + self.check_hash("nohash")