From patchwork Mon Jan 15 10:04:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yegor Yefremov X-Patchwork-Id: 860714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zKpr435Cvz9s9Y for ; Mon, 15 Jan 2018 21:05:08 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id ADE692F4F7; Mon, 15 Jan 2018 10:05:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tHIa7Vi+k8oi; Mon, 15 Jan 2018 10:05:03 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 30BB02E5B0; Mon, 15 Jan 2018 10:05:03 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 264B31C2D4A for ; Mon, 15 Jan 2018 10:05:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 221872E5B0 for ; Mon, 15 Jan 2018 10:05:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TaVClwBmYKfw for ; Mon, 15 Jan 2018 10:04:59 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail.visionsystems.de (mail.visionsystems.de [213.209.99.202]) by silver.osuosl.org (Postfix) with ESMTP id 9EE4E257B1 for ; Mon, 15 Jan 2018 10:04:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.visionsystems.de (Postfix) with ESMTP id 3BE6136421E; Mon, 15 Jan 2018 11:04:59 +0100 (CET) Received: from mail.visionsystems.de ([127.0.0.1]) by localhost (mail.visionsystems.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 14519-10; Mon, 15 Jan 2018 11:04:59 +0100 (CET) Received: from visionsystems.de (kallisto.visionsystems.local [192.168.1.3]) by mail.visionsystems.de (Postfix) with ESMTP id 0659836420A; Mon, 15 Jan 2018 11:04:58 +0100 (CET) Received: from development1.visionsystems.de ([192.168.1.36]) by visionsystems.de with Microsoft SMTPSVC(6.0.3790.4675); Mon, 15 Jan 2018 11:04:57 +0100 From: yegorslists@googlemail.com To: buildroot@buildroot.org Date: Mon, 15 Jan 2018 11:04:50 +0100 Message-Id: <1516010690-7361-1-git-send-email-yegorslists@googlemail.com> X-Mailer: git-send-email 2.1.4 X-OriginalArrivalTime: 15 Jan 2018 10:04:57.0730 (UTC) FILETIME=[4C620E20:01D38DE8] X-Virus-Scanned: amavisd-new at visionsystems.de Subject: [Buildroot] [PATCH] scanpypi: add support for licence files checksums X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Yegor Yefremov Store the list of detected licence files in the main object and automatically add their sha256 checksums when creating *.hash file. Bonus: fix wrong indentation. Signed-off-by: Yegor Yefremov --- utils/scanpypi | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/utils/scanpypi b/utils/scanpypi index d3b0d2f..765bfdc 100755 --- a/utils/scanpypi +++ b/utils/scanpypi @@ -24,6 +24,8 @@ import tempfile import imp from functools import wraps +BUF_SIZE = 65536 + try: import spdx_lookup as liclookup except ImportError: @@ -132,6 +134,7 @@ class BuildrootPackage(): self.filename = None self.url = None self.version = None + self.license_files = [] def fetch_package_info(self): """ @@ -439,12 +442,12 @@ class BuildrootPackage(): filenames = ['LICENCE', 'LICENSE', 'LICENSE.RST', 'LICENSE.TXT', 'COPYING', 'COPYING.TXT'] - license_files = list(find_file_upper_case(filenames, self.tmp_extract)) + self.license_files = list(find_file_upper_case(filenames, self.tmp_extract)) - lines.append(self.__get_license_names(license_files)) + lines.append(self.__get_license_names(self.license_files)) license_files = [license.replace(self.tmp_extract, '')[1:] - for license in license_files] + for license in self.license_files] if len(license_files) > 0: if len(license_files) > 1: print('More than one file found for license:', @@ -508,7 +511,7 @@ class BuildrootPackage(): lines = [] if self.used_url['md5_digest']: md5_comment = '# md5 from {url}, sha256 locally computed\n'.format( - url=self.metadata_url) + url=self.metadata_url) lines.append(md5_comment) hash_line = '{method}\t{digest} {filename}\n'.format( method='md5', @@ -522,6 +525,20 @@ class BuildrootPackage(): filename=self.filename) lines.append(hash_line) + for license_file in self.license_files: + sha256 = hashlib.sha256() + with open(license_file, 'rb') as lic_f: + while True: + data = lic_f.read(BUF_SIZE) + if not data: + break + sha256.update(data) + hash_line = '{method}\t{digest} {filename}\n'.format( + method='sha256', + digest=sha256.hexdigest(), + filename=os.path.basename(license_file)) + lines.append(hash_line) + with open(path_to_hash, 'w') as hash_file: hash_file.writelines(lines)