From patchwork Fri Jul 19 13:06:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Victor Huesca X-Patchwork-Id: 1134047 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45qrqp5c3lz9sBZ for ; Fri, 19 Jul 2019 23:06:46 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 59C16838D9; Fri, 19 Jul 2019 13:06:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PxF-gUFzxOBg; Fri, 19 Jul 2019 13:06:42 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 16FE987594; Fri, 19 Jul 2019 13:06:42 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id E31351BF9BF for ; Fri, 19 Jul 2019 13:06:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E07E0857BC for ; Fri, 19 Jul 2019 13:06:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9a3qYAiltgpy for ; Fri, 19 Jul 2019 13:06:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by hemlock.osuosl.org (Postfix) with ESMTPS id 40B1B85803 for ; Fri, 19 Jul 2019 13:06:37 +0000 (UTC) X-Originating-IP: 86.250.200.211 Received: from localhost.localdomain (lfbn-1-17395-211.w86-250.abo.wanadoo.fr [86.250.200.211]) (Authenticated sender: victor.huesca@bootlin.com) by relay1-d.mail.gandi.net (Postfix) with ESMTPSA id 8E9C6240005; Fri, 19 Jul 2019 13:06:35 +0000 (UTC) From: Victor Huesca To: buildroot@buildroot.org Date: Fri, 19 Jul 2019 15:06:29 +0200 Message-Id: <20190719130630.9967-3-victor.huesca@bootlin.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190719130630.9967-1-victor.huesca@bootlin.com> References: <20190719130630.9967-1-victor.huesca@bootlin.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH v3 2/3] support/scripts/pkg-stats: add support for json output X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Victor Huesca , thomas.petazzoni@bootlin.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Pkg-stats is a great script that get a lot of interesting info from buildroot packages. Unfortunately it is currently designed to output a static HTML page only. While this is great to include on the buildroot's website, the HTML is not designed to be easily parsable and thus it is difficult to reuse it in other scripts. This patch provide a new option to output a JSON file in addition to the HTML one. The old 'output' option has been renamed to 'html' to distinguish from the new 'json' option. Signed-off-by: Victor Huesca --- Changes v2 --> v3: - Use better names for the argparse group - Comment the tricky steps in dump_json --- support/scripts/pkg-stats | 48 +++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats index d65f609d57..478f98fbc2 100755 --- a/support/scripts/pkg-stats +++ b/support/scripts/pkg-stats @@ -695,16 +695,52 @@ def dump_html(packages, stats, output): f.write(html_footer) +def dump_json(packages, stats, output): + # Format packages as a dictionnary instead of a list + # Exclude local field that does not contains real date + excluded_fields = ['url_worker', 'name'] + pkgs = { + pkg.name: { + k: v + for k, v in pkg.__dict__.items() + if k not in excluded_fields + } for pkg in packages + } + # Aggregate infrastructures into a single dict entry + statistics = { + k: v + for k, v in stats.items() + if not k.startswith('infra-') + } + statistics['infra'] = {k[6:]: v for k, v in stats.items() if k.startswith('infra-')} + # The actual structure to dump, add commit and date to it + o = subprocess.check_output(["git", "log", "master", "-n", "1", "--pretty=format:%H"]) + final = {'packages': pkgs, + 'stats': statistics, + 'commit': o.splitlines()[0], + 'date': str(datetime.datetime.utcnow())} + + with open(output, 'w') as f: + json.dump(final, f, indent=2, separators=(',', ': ')) + f.write('\n') + + def parse_args(): parser = argparse.ArgumentParser() - parser.add_argument('-o', dest='output', action='store', required=True, + output = parser.add_argument_group('output', 'Output file(s)') + output.add_argument('--html', dest='html', action='store', help='HTML output file') + output.add_argument('--json', dest='json', action='store', + help='JSON output file') packages = parser.add_mutually_exclusive_group() packages.add_argument('-n', dest='npackages', type=int, action='store', help='Number of packages') packages.add_argument('-p', dest='packages', action='store', help='List of packages (comma separated)') - return parser.parse_args() + args = parser.parse_args() + if not args.html and not args.json: + parser.error('at least one of --html or --json (or both) is required') + return args def __main__(): @@ -732,8 +768,12 @@ def __main__(): check_package_latest_version(packages) print("Calculate stats") stats = calculate_stats(packages) - print("Write HTML") - dump_html(packages, stats, args.output) + if args.html: + print("Write HTML") + dump_html(packages, stats, args.html) + if args.json: + print("Write JSON") + dump_json(packages, stats, args.json) __main__()