From patchwork Sun Apr 14 20:17:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1085381 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ihdmXuxM"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44j2xb5d0Pz9s47 for ; Mon, 15 Apr 2019 06:17:55 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 3396C861D4; Sun, 14 Apr 2019 20:17:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XfpUZ7xOPmwC; Sun, 14 Apr 2019 20:17:47 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 83150865C4; Sun, 14 Apr 2019 20:17:45 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 93AA11BF82F for ; Sun, 14 Apr 2019 20:17:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 90EAF8565B for ; Sun, 14 Apr 2019 20:17:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HZaPBRy4NRzt for ; Sun, 14 Apr 2019 20:17:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by fraxinus.osuosl.org (Postfix) with ESMTPS id B44DA85653 for ; Sun, 14 Apr 2019 20:17:37 +0000 (UTC) Received: by mail-wr1-f68.google.com with SMTP id q1so19107566wrp.0 for ; Sun, 14 Apr 2019 13:17:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=teEwwAQW1UyfpCD/Xaxww/8G2dywLd1nhBu97UGxkc4=; b=ihdmXuxMqcxa6YvtrEidlL3yXWAj4VjFOMzyfPhy4EE06pNvR9yevPkkBxStAkBYM+ EpubhoL5xUXLPR0j038ZUJM69LAt95+ETi/9wJi5QuO2ugEjV6wo/96XD/AfcTH++OkO iZCKNdfYyIhOY7gal/hQuYzh/GreuRAMJjnwMMaLklujQNkVG6+rQqwjFp0GlB/8QVmI EgXVJ7yTkAOTLi8raMrRUKiBXKbKigixfzWGibkcXxRfd6iirEjQztdjMz1nSzsttiFv 6q5UZHgExbtFB4tbYQP6QZCM9O0cl0haUEDNDjGBRyumiRQjdLfN09FxC7YE62RZ6LQX xNMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=teEwwAQW1UyfpCD/Xaxww/8G2dywLd1nhBu97UGxkc4=; b=g2Ly3UGm5/Pvt8RD6W2i1LYeGVidg4gIzDi6t+fUlWWl9Jz/CYWdVuIDsHd8xvpd3b EufEnZJl4CBQToLjj9pyATAaNizpy1idER3QVnqSckZDPMAZ3DsZlXJpD8OOIzjSBydA 7mnaASFINbAoJaKmLQZOhRDz27k0aSY7BOX7xtQZCIB+fd6J/DcS+nRs11fkxfNd2Agc 7U4V79hSAYc8Ea3Ukxn/HFLXkCpZkQCOefduN2jj2RUJjuMmaQn3n2lfQOp8rq/E7bnt HIp0EzePw6qncgxsmtgr8+tpmFdOFajgamI3D8jhTfeXKjSO6p5sWZ5WTJOctxeca6I5 kQ7Q== X-Gm-Message-State: APjAAAWa5VsO2GaRnN3QucNZS3UCaFlVpPM2S9FdFQzXYzfCiwxUw7gM /A+M4D4ltq/KHdQSJeHar/2C3Pwm X-Google-Smtp-Source: APXvYqy9KE20UvTaLRAjsAKhRpWCb1p1zNd4zOvmSQBNw5x3XgNAom+MNmM82xcXW9diki54fpgQoQ== X-Received: by 2002:adf:f147:: with SMTP id y7mr43764797wro.102.1555273056158; Sun, 14 Apr 2019 13:17:36 -0700 (PDT) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id o4sm23395507wmo.20.2019.04.14.13.17.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Apr 2019 13:17:35 -0700 (PDT) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 14 Apr 2019 22:17:20 +0200 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 09/12 v2] core: introduce new global show-info X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Users are increasingly trying to extract information about packages. For example, they might need to get the list of URIs, or the dependencies of a package. Although we do have a bunch of rules to generate some of that, this is done in ad-hoc way, with most of the output formats just ad-hoc, raw, unformatted blurbs, mostly internal data dumped as-is. Introduce a new rule, show-info, that provides a properly formatted output of all the meta-information about packages: name, type, version, licenses, dependencies... We choose to use JSON as the output format, because it is pretty versatile, has parsers in virtually all languages, has tools to parse from the shell (jq). It also closely matches Python data structure, which makes it easy to use with our own internal tools as well. Finally, JSON being a key-value store, allows for easy expanding the output without requiring existing consumers to be updated; new, unknown keys are simply ignored by those (as long as they are true JSON parsers). The complex part of this change was the conditional output of parts of the data: virtual packages have no source, version, license or downloads, unlike non-virtual packages. Same goes for filesystems. We use a wrapper macro, show-info, that de-multiplexes unto either the package-related- or filesystem-related macros, and for packages, we also use a detailed macro for non-virtual packages. It is non-trivial to properly output correct JSON blurbs, especially when trying to output an array of objects, like so, where the last item shall not be followed by a comma: [ { ... }, { ... } ] So, we use a trick (as sugegsted by Arnout), to $(subst) any pair of ",}" or ", }" or ",]" or ", ]" with only the respective closing symbol, "}" or "]". The whole stuff is $(strip)ed to make it a somewhat-minified JSON blurb that fits on a single line with all spaces squashed (but still with spaces, as it is not possible to differentiate spaces between JSON elements from spaces inside JSON strings). Reported-by: Thomas De Schampheleire Signed-off-by: "Yann E. MORIN" Cc: Thomas De Schampheleire Cc: Thomas Petazzoni Cc: Arnout Vandecappelle --- Changes v1 -> v2: - make it a macro to be called (Arnout) - make it a top-level rule (Arnout) --- Makefile | 15 ++++++++++++ package/pkg-utils.mk | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) diff --git a/Makefile b/Makefile index 60bf7d7d08..33215f91e5 100644 --- a/Makefile +++ b/Makefile @@ -903,6 +903,21 @@ check-dependencies: @cd "$(CONFIG_DIR)"; \ $(TOPDIR)/support/scripts/graph-depends -C +.PHONY: show-info +show-info: + @: + $(info $(call clean-json, \ + $(foreach p, \ + $(sort $(foreach i,$(PACKAGES) $(TARGETS_ROOTFS), \ + $(i) \ + $($(call UPPERCASE,$(i))_FINAL_RECURSIVE_DEPENDENCIES) \ + ) \ + ), \ + $(call show-info,$(call UPPERCASE,$(p)))$(comma) \ + ) \ + ) \ + ) + else # ifeq ($(BR2_HAVE_DOT_CONFIG),y) # Some subdirectories are also package names. To avoid that "make linux" diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk index bffd79dfb0..00d1d6bcac 100644 --- a/package/pkg-utils.mk +++ b/package/pkg-utils.mk @@ -62,6 +62,74 @@ $$(error Package error: use $(2) instead of $(1). Please fix your .mk file) endif endef +# show-info -- return package or filesystem metadata formatted as an entry +# of a JSON dictionnary +# $(1): upper-case package or filesystem name +define show-info + "$($(1)_NAME)": { + "type": "$($(1)_TYPE)", + $(if $(filter rootfs,$($(1)_TYPE)), \ + $(call _show-info-fs,$(1)), \ + $(call _show-info-pkg,$(1)), \ + ) + } +endef + +# _show-info-pkg, _show-info-pkg-details, _show-info-fs: private helpers +# for show-info, above +define _show-info-pkg + $(if $($(1)_IS_VIRTUAL), \ + "virtual": true$(comma), + $(call _show-info-pkg-details,$(1)) \ + ) + "dependencies": [ + $(call make-comma-list,$(sort $($(1)_FINAL_ALL_DEPENDENCIES))) + ], + "reverse_dependencies": [ + $(call make-comma-list,$(sort $($(1)_RDEPENDENCIES))) + ] +endef +define _show-info-pkg-details + "virtual": false, + "version": "$($(1)_DL_VERSION)", + "licenses": "$($(1)_LICENSE)", + "downloads": [ + $(foreach dl,$(sort $($(1)_ALL_DOWNLOADS)), + { + "source": "$(notdir $(dl))", + "URIs": [ + $(call make-comma-list, + $(subst \|,|, + $(call DOWNLOAD_URIS,$(dl),$(1)) + ) + ) + ] + }, + ) + ], +endef +define _show-info-fs + "dependencies": [ + $(call make-comma-list,$(sort $($(1)_DEPENDENCIES))) + ] +endef + +# clean-json -- cleanup pseudo-json into clean json: +# - adds opening { and closing } +# - remove commas before closing ] and } +# - minify with $(strip) +clean-json = $(strip \ + $(subst $(comma)},, \ + $(subst $(comma)$(space)},$(space)}, \ + $(subst $(comma)],, \ + $(subst $(comma)$(space)],$(space)], \ + { $(strip $(1)) } \ + ) \ + ) \ + ) \ + ) \ +) + # # legal-info helper functions #