From patchwork Sun Feb 19 22:17:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Martincoski X-Patchwork-Id: 729667 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vRLqp30x6z9s7R for ; Mon, 20 Feb 2017 09:23:02 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="J0RhZd7y"; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id D1A948756A; Sun, 19 Feb 2017 22:23:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5UqKqa5S9FkK; Sun, 19 Feb 2017 22:23:00 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0E93887D4B; Sun, 19 Feb 2017 22:23:00 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id D6E161C1650 for ; Sun, 19 Feb 2017 22:22:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D20EF89C96 for ; Sun, 19 Feb 2017 22:22:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ab4spnXyb0gw for ; Sun, 19 Feb 2017 22:22:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3731189885 for ; Sun, 19 Feb 2017 22:22:58 +0000 (UTC) Received: by mail-qk0-f195.google.com with SMTP id s186so6168566qkb.1 for ; Sun, 19 Feb 2017 14:22:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=2FJmLkxThVCp6xRHb+X4qTb7580i6OLM1dokUjEarUo=; b=J0RhZd7ymaq/ZB1HJx+XIplVjBXvae9qKgt6AmQBeCzV2Gha7SlqLVNPqy6v6f2eHT rNwSnYHEzfaJhI9YB6shQ0XeLxuZyFBR4jaXqr97YkqZxyU/w81E6Kf+DkHdu+SdrgXv uY8tjWpVaEOrdAeQgUUQ1MwDjscnycppS4GQP61oFXz/xcI+9LP93JFLkc9CC7ml4cjo 8JBxybIyQiMW2h+9RXcjFrluDNN5GyGkvrkHn96VbpXHyQ0d1efflvrCVc+8KicARqpm mfJvpirAb5uiAUEfLVEBJHHB0qkhnzy3xq6ais8ivC6QPW9RdNzig2rT9UiS+MjSMGnl pP7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=2FJmLkxThVCp6xRHb+X4qTb7580i6OLM1dokUjEarUo=; b=Uh8XMxp7JW9oO70OIN54AZ7L6Cc0ENZ25slOR4wGqvkS0uyK9rFaOCRXFrQKGvVnBo FMg+qVuLHskPPgBaS7yrAO+f/Jksf44ogX5M8RYsL/wyZVVirapDsQjUb1yRWLR2Zrtc iM1gZxIbrAwD01i7z+WYK0SEwae/TVEaXitY44XYiK2ghTtMwTX0EnoSxgEsn0WB8y/s /BgelMtmdl/wXuUdtwtI2j/n6rnS+oA2sAeRtjHgX9iywkpvgm0+aGw6F/VfHmU4o86P 915ARfqOTYc2o7/3ktqbRXwjX3+Q0Tl0RaZ4Rg2mfjySS53/hx+VfZ39ZAiuQvJmxOXF RTtw== X-Gm-Message-State: AMke39m8e3XXSZ+YEpnP71Uwm5TtRsgzsAZqRucat8tkNka1zRpOOz2XbJmYjtr2OqwQJw== X-Received: by 10.55.69.80 with SMTP id s77mr18409398qka.159.1487542977314; Sun, 19 Feb 2017 14:22:57 -0800 (PST) Received: from ultri3.home ([201.86.219.185]) by smtp.gmail.com with ESMTPSA id d127sm10970245qka.30.2017.02.19.14.22.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 19 Feb 2017 14:22:56 -0800 (PST) From: Ricardo Martincoski To: buildroot@buildroot.org Date: Sun, 19 Feb 2017 19:17:19 -0300 Message-Id: <20170219221724.27298-5-ricardo.martincoski@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170219221724.27298-1-ricardo.martincoski@gmail.com> References: <20170219221724.27298-1-ricardo.martincoski@gmail.com> In-Reply-To: <20161231032110.11573-1-ricardo.martincoski@gmail.com> References: <20161231032110.11573-1-ricardo.martincoski@gmail.com> Cc: Thomas De Schampheleire , Ricardo Martincoski Subject: [Buildroot] [PATCH v2 4/9] check-package: check *.hash files X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Check each hash entry (see [1]) and warn when: - it does not have three fields; - its type is unknown; - its length does not match its type; - the name of the file contains a directory component. [1] http://nightly.buildroot.org/#adding-packages-hash Signed-off-by: Ricardo Martincoski Cc: Thomas De Schampheleire --- Changes v1 -> v2: - do not use regex when a simple 'in' is enough (Thomas DS); - use classes instead of functions to declare each check (Thomas DS); --- Notes: $ time support/scripts/check-package $(find package -type f) >/dev/null 2>/dev/null real 0m0.684s user 0m0.656s sys 0m0.024s HashFilename: support/scripts/check-package --include-only HashFilename \ $(find package -name '*.hash') 2>/dev/null | wc -l 0 (cd support/scripts/check-package-example && \ ../check-package --include-only HashFilename -vv package/*/*) package/package1/package1.hash:5: use filename without directory component (http://nightly.buildroot.org/#adding-packages-hash) sha256< tab >12345678901234567890123456789012345678901234567890123456789012345< tab >dl/package1-1.0.tar.gz 180 lines processed 1 warnings generated HashNumberOfFields: support/scripts/check-package --include-only HashNumberOfFields \ $(find package -name '*.hash') 2>/dev/null | wc -l 0 (cd support/scripts/check-package-example && \ ../check-package --include-only HashNumberOfFields -vv package/*/*) package/package1/package1.hash:6: expected three fields (http://nightly.buildroot.org/#adding-packages-hash) sha256 1234567890123456789012345678901234567890123456789012345678901234 180 lines processed 1 warnings generated HashType: support/scripts/check-package --include-only HashType \ $(find package -name '*.hash') 2>/dev/null | wc -l 1 (cd support/scripts/check-package-example && \ ../check-package --include-only HashType -vv package/*/*) package/package1/package1.hash:3: hash size does not match type (http://nightly.buildroot.org/#adding-packages-hash) sha256 123456789 package1-1.0.tar.gz expected 64 hex digits package/package1/package1.hash:4: unexpected type of hash (http://nightly.buildroot.org/#adding-packages-hash) crc16< tab >123456789< tab >package1-1.0.tar.gz package/package1/package1.hash:5: hash size does not match type (http://nightly.buildroot.org/#adding-packages-hash) sha256< tab >12345678901234567890123456789012345678901234567890123456789012345< tab >dl/package1-1.0.tar.gz expected 64 hex digits 180 lines processed 3 warnings generated support/scripts/checkpackagelib_hash.py | 62 +++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/support/scripts/checkpackagelib_hash.py b/support/scripts/checkpackagelib_hash.py index 1f268838f..47fe18756 100644 --- a/support/scripts/checkpackagelib_hash.py +++ b/support/scripts/checkpackagelib_hash.py @@ -3,8 +3,70 @@ # functions don't need to check for things already checked by running # "make package-dirclean package-source". +import re + +from checkpackagebase import _CheckFunction # Notice: ignore 'imported but unused' from pyflakes for check functions. from checkpackagelib import ConsecutiveEmptyLines from checkpackagelib import EmptyLastLine from checkpackagelib import NewlineAtEof from checkpackagelib import TrailingSpace + + +def _empty_line_or_comment(text): + return text.strip() == "" or text.startswith("#") + + +class HashFilename(_CheckFunction): + def check_line(self, lineno, text): + if _empty_line_or_comment(text): + return + + fields = text.split() + if len(fields) < 3: + return + + if '/' in fields[2]: + return ["{}:{}: use filename without directory component" + " ({}#adding-packages-hash)" + .format(self.filename, lineno, self.url_to_manual), + text] + + +class HashNumberOfFields(_CheckFunction): + def check_line(self, lineno, text): + if _empty_line_or_comment(text): + return + + fields = text.split() + if len(fields) != 3: + return ["{}:{}: expected three fields ({}#adding-packages-hash)" + .format(self.filename, lineno, self.url_to_manual), + text] + + +class HashType(_CheckFunction): + len_of_hash = {"md5": 32, "sha1": 40, "sha224": 56, "sha256": 64, + "sha384": 96, "sha512": 128} + + def check_line(self, lineno, text): + if _empty_line_or_comment(text): + return + + fields = text.split() + if len(fields) < 2: + return + + htype, hexa = fields[:2] + if htype == "none": + return + if htype not in self.len_of_hash.keys(): + return ["{}:{}: unexpected type of hash ({}#adding-packages-hash)" + .format(self.filename, lineno, self.url_to_manual), + text] + if not re.match("^[0-9A-Fa-f]{%s}$" % self.len_of_hash[htype], hexa): + return ["{}:{}: hash size does not match type " + "({}#adding-packages-hash)" + .format(self.filename, lineno, self.url_to_manual), + text, + "expected {} hex digits".format(self.len_of_hash[htype])]