From patchwork Sat May 12 07:03:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Francois Perrad X-Patchwork-Id: 912381 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="cTAImcjI"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40jdGL16cWz9s19 for ; Sat, 12 May 2018 17:03:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 13D342316E; Sat, 12 May 2018 07:03:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jGjgdoXFxv+L; Sat, 12 May 2018 07:03:18 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id B4341230A4; Sat, 12 May 2018 07:03:18 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id EA47F1C146E for ; Sat, 12 May 2018 07:03:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id E5D1B88D49 for ; Sat, 12 May 2018 07:03:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id We8CNIO1rCBU for ; Sat, 12 May 2018 07:03:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr0-f193.google.com (mail-wr0-f193.google.com [209.85.128.193]) by whitealder.osuosl.org (Postfix) with ESMTPS id 0748688D1D for ; Sat, 12 May 2018 07:03:17 +0000 (UTC) Received: by mail-wr0-f193.google.com with SMTP id g21-v6so7250110wrb.8 for ; Sat, 12 May 2018 00:03:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=6WYxMkgLq9be2NwW2tF5J6G4fIgVkzuhDBPz8u2c8Y8=; b=cTAImcjI+LvdomkOQ5Xg9W6XmwmDXbZLXG/SkdO194YbfBa1O9Ncu2zmXWeZVFxC/T rdzl3h3PgZ/G1peKnWsU9hZhC4ZNTkoB9d8HsNtNDtwMOaBHwdOQWxZ98fp7ZgGlmIoK 8l0NW3QjgRy4b54geNFVFxrNAnwiH4/Kgpzpsf7Ls+L702YRHKp1eO7139lnqBPqVY6W 0GlA13/vFE6RoimHLGaR9u06b2muWyPo6EGo+UGqeCxZUpYtMCzJ9OGmCWmoD77bfT6i 5VuqDHa/R/NltllKgq2ymkdZLAxnUGR+xFsBVgTgh1ixofPHszUdhYUZVh/FpgHkWXQc 6/nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=6WYxMkgLq9be2NwW2tF5J6G4fIgVkzuhDBPz8u2c8Y8=; b=TkTyrMpeg7r1Dyh3OTYDYufIsp6WB6qbDnkzGweEED59wNxkXEJARvlHNi97EzqeZF 9fCslecXBRLj7HZOb9ZIUcTHToBtOMsQHICmR8iWZ6NFVMIXA8vNEUcC6E1bu80PygR3 9eOmFGhCDHqF9h+d3eTG+UmOUXmG45vzKnw5tDnztcvu8d/74VwuftVMj5xuSco9ojuk gEB6F3GboXqlYyUj/qaCObEU5jxtcvLfMm9ZGO21LEQEN8w1lSTfxm8RJ4RnPW6XFmK5 NGcpPLy1q8TuG0Yhvi0v/rCefS7jiLVddE9ZG697rMSxcprjcEdwwWQTQyOl3Ltom9C7 G7Lg== X-Gm-Message-State: ALKqPwfeMlJKoN2WnVL28t/p9C8/4+dFNv6jdlXN1trRqd9xRI1D7T1b sLUK8sJfbwHw5psIEZqZZGvzYw== X-Google-Smtp-Source: AB8JxZqobwICg8IJNMT1fsSY1sXu1tgxS3mZgCdOm41uV1o+aPxtU7Gcqf1i2vumeUg9LAcGYHHyNg== X-Received: by 2002:adf:dfcc:: with SMTP id q12-v6mr1304855wrn.68.1526108595222; Sat, 12 May 2018 00:03:15 -0700 (PDT) Received: from localhost.localdomain (254.7.19.109.rev.sfr.net. [109.19.7.254]) by smtp.gmail.com with ESMTPSA id a14-v6sm8091658wra.84.2018.05.12.00.03.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 12 May 2018 00:03:14 -0700 (PDT) From: Francois Perrad X-Google-Original-From: Francois Perrad To: buildroot@busybox.net Date: Sat, 12 May 2018 09:03:00 +0200 Message-Id: <20180512070300.128153-1-francois.perrad@gadz.org> X-Mailer: git-send-email 2.17.0 Subject: [Buildroot] [PATCH] utils/scancpan: generates hashes of license files X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Signed-off-by: Francois Perrad --- utils/scancpan | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/utils/scancpan b/utils/scancpan index 6d1cdc57a..da8e7b91a 100755 --- a/utils/scancpan +++ b/utils/scancpan @@ -483,6 +483,7 @@ use Module::CoreList; use HTTP::Tiny; use Safe; use MetaCPAN::API::Tiny; +use Digest::SHA qw(sha256_hex); # Below, 5.026 should be aligned with the version of perl actually # bundled in Buildroot: @@ -519,7 +520,7 @@ my %need_dlopen; # name -> 1 if requires dynamic library my %deps_build; # name -> list of host dependencies my %deps_runtime; # name -> list of target dependencies my %deps_optional; # name -> list of optional target dependencies -my %license_files; # name -> list of license files +my %license_files; # name -> hash of license files my %checksum; # author -> list of checksum my $mirror = 'http://cpan.metacpan.org'; # a CPAN mirror my $mcpan = MetaCPAN::API::Tiny->new(base_url => 'http://fastapi.metacpan.org/v1'); @@ -556,7 +557,7 @@ sub find_license_files { if (scalar @license_files == 0 && $manifest =~ m/(README)[\n\s]/i) { @license_files = ($1); } - return \@license_files; + return @license_files; } sub fetch { @@ -567,16 +568,19 @@ sub fetch { say qq{fetch ${name}} unless $quiet; my $result = $mcpan->release( distribution => $name ); $dist{$name} = $result; + $license_files{$name} = {}; eval { - my $manifest = $mcpan->source( author => $result->{author}, - release => $name . q{-} . $result->{version}, - path => 'MANIFEST' ); + my $author = $result->{author}; + my $release = $name . q{-} . $result->{version}; + my $manifest = $mcpan->source( author => $author, release => $release, path => 'MANIFEST' ); $need_dlopen{$name} = is_xs( $manifest ); - $license_files{$name} = find_license_files( $manifest ); + foreach my $fname (find_license_files( $manifest )) { + my $license = $mcpan->source( author => $author, release => $release, path => $fname ); + $license_files{$name}->{$fname} = sha256_hex( $license ); + } }; if ($@) { warn $@; - $license_files{$name} = []; } my %build = (); my %runtime = (); @@ -692,7 +696,7 @@ while (my ($distname, $dist) = each %dist) { $license =~ s|mit|MIT|; $license =~ s|openssl|OpenSSL|; $license =~ s|perl_5|Artistic or GPL-1.0+|; - my $license_files = join q{ }, @{$license_files{$distname}}; + my $license_files = join q{ }, keys %{$license_files{$distname}}; say qq{write ${mkname}} unless $quiet; open my $fh, q{>}, $mkname; say {$fh} qq{################################################################################}; @@ -731,6 +735,13 @@ while (my ($distname, $dist) = each %dist) { say {$fh} qq{# retrieved by scancpan from ${mirror}/}; say {$fh} qq{md5 ${md5} ${filename}}; say {$fh} qq{sha256 ${sha256} ${filename}}; + if (scalar keys %{$license_files{$distname}}) { + say {$fh} q{}; + say {$fh} qq{# computed by scancpan}; + while (my ($license, $digest) = each %{$license_files{$distname}}) { + say {$fh} qq{sha256 ${digest} ${license}}; + } + } close $fh; } } @@ -819,7 +830,6 @@ Perl/CPAN distributions required by the specified distnames. The dependencies and metadata are fetched from https://metacpan.org/. After running this script, it is necessary to check the generated files. -You have to manually add the license files (PERL_FOO_LICENSE_FILES variable). For distributions that link against a target library, you have to add the buildroot package name for that library to the DEPENDENCIES variable. @@ -831,7 +841,7 @@ in order to work with the right CoreList data. =head1 LICENSE -Copyright (C) 2013-2017 by Francois Perrad +Copyright (C) 2013-2018 by Francois Perrad This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by