From patchwork Sun Dec 2 09:04:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1006455 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="sIcVjwYG"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4372Jp70FSz9s3Z for ; Sun, 2 Dec 2018 20:05:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 7624788097; Sun, 2 Dec 2018 09:05:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WgzeGmRwuiaC; Sun, 2 Dec 2018 09:05:08 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id AB84088039; Sun, 2 Dec 2018 09:05:08 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 0B3511C2E88 for ; Sun, 2 Dec 2018 09:05:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id F240E23C93 for ; Sun, 2 Dec 2018 09:05:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1x9Lf7Foj17B for ; Sun, 2 Dec 2018 09:05:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by silver.osuosl.org (Postfix) with ESMTPS id 03802241A8 for ; Sun, 2 Dec 2018 09:05:03 +0000 (UTC) Received: by mail-wr1-f67.google.com with SMTP id l9so9110646wrt.13 for ; Sun, 02 Dec 2018 01:05:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=4iP5YIc4hZNTPNulCgBzNbousyxyiG8D7fIw9GEGWPA=; b=sIcVjwYGiZVIFE4l4NjSOxt7xtcmDZLRQeCbTJZHjpeXGKjFph7K0YvbE3X2PtZOC7 cCzX2YxLqUW8PrLMOvpEssf4qc0hF/Qng5M3BBv4yYfaBMXj1xsZAMkhUg3Kl2m/hb83 F0hNuMcIa7FvQSlUlhTc28sqKEwZcapQt3wMIWVttaVbc1ulAk6/ypVGTzlNsLm7Q2w+ AKv9oBiAe7x+VcFLK59BDEZ7kZCCKzhn8jdP7/AIBtIm28AWHZkGp8Rwe8rofXz4urLu ZYPkcBETXDy5q+TxE+4AUWO74f1MYGi9x/npmlCcZUMGMcYiHdWP3lNVUQ978qdpS8w6 yDPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=4iP5YIc4hZNTPNulCgBzNbousyxyiG8D7fIw9GEGWPA=; b=KmVBdl7zDSR0w1XkeaNQaZfyszdsZhViEPipXpVSHub070Q1j/uf650WpZvCru+2EB Ez8QGhFgknFSnb+e7m7bFCT+ERPtdWRC/hGDJgyMi8rlXim783ZYhWNfeVuwcfGJVAu/ 4KBg8Awam2BzLoh1p+/F5ti+JF49fCzTnR9QD7ElY5n/l++KTqLhHwHC7GLf2e4V5zWn DndTGE7hgb9Owdsz+a/pw5NxMeh852f1mWqN1eeOTucVgNPqR7aseeVPuQfo5Gkbfwsq mQ4gfXabjkORrlkq+vcgjxq0+ScQI9Xp4NvC7FM64qQD86jVegTJ8HavHpribAEgHv4K dusQ== X-Gm-Message-State: AA+aEWarcJjAOrfJrehW9wPr4TUK1WFzdcdrm9VH22/e7yPPVFBLDlmN G8OZehjnOg7ujiKf20eUZXza2JlK X-Google-Smtp-Source: AFSGD/W2jCtbadbTJAnIQUhJ+/ioN5dAnk/zk8VRgeOGprxOvEpG/83FXa3f1u0g7Rdw3eAQu2E1YQ== X-Received: by 2002:adf:b190:: with SMTP id q16mr10275128wra.95.1543741496413; Sun, 02 Dec 2018 01:04:56 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id o8sm7834985wrx.15.2018.12.02.01.04.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Dec 2018 01:04:55 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 2 Dec 2018 10:04:42 +0100 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 10/11] core: add make-based full-dependency list X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Currently, when we need to build the full dependency graph, we call make to show the list of packages (make show-targets), and then call it again and again iteratively while it returns new packages. Since calling make will parse the whole set of our Makefiles, this takes quite a bit of time (~4s each here), and the total can get pretty long. However, make being make, already builds the whole dependency tree information, so we can just ask for it. Add a new top-level rule 'show-dependency-tree' that displays the whole set of dependencies for all packages. For each package, its name, type and version is displayed, then all the direct, first-level dependencies are dumped. We choose a format that is not unlike the dot-graph format, because it is both easy to read as a human, and easy to parse as a machine: foo: target 1.2.3 foo -> bar host-meh bar: target virtual bar -> buz buz: target 2.3.4 buz -> host-meh: host virtual host-meh -> host-bleark host-bleark: host 3.4.5 host-bleark -> rootfs-meh: host rootfs-meh -> host-bleark To be noted: rootfs are currently reported as if they were 'host' packages, to stay aligned with how graph-depends currently treats them. Ideally, graph-depends could be enhanced to recognise them separately, but that is another story. For just plain defconfig, which is about the smallest config we can have with an internal toolchain, we already have a seven-fold improvement (with the graph-depends rule modified to not run the pdf generation, to be able to just compare the tree generation): $ time make graph-depends real 0m27.344s $ time make show-dependency-tree real 0m3.848s From defconfig, C++, wchar, locales, ssp, and allyespackageconfig, tweaked for even more packages (qt5 not qt4, luajit to avoid multi providers, etc...), the timings are (graph-depends still modified to not generate the pdf): $ time make graph-depends real 1m56.459s $ time make show-dependency-tree real 0m5.748s There. I don't think those numbers need any explanation whatsoever; they do speak on their own. OK, for maths sake, the ratio is about twenty-fold. So, "yeah", I guess... ;-) Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Thomas De Schampheleire --- Makefile | 4 ++++ fs/common.mk | 8 ++++++++ package/pkg-generic.mk | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/Makefile b/Makefile index 37df98520e..7a50c19de9 100644 --- a/Makefile +++ b/Makefile @@ -846,6 +846,10 @@ graph-depends-requirements: @dot -? >/dev/null 2>&1 || \ { echo "ERROR: The 'dot' program from Graphviz is needed for graph-depends" >&2; exit 1; } +.PHONY: show-dependency-tree +show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(PACKAGES) $(TARGETS_ROOTFS)) + @: + .PHONY: graph-depends graph-depends: graph-depends-requirements @$(INSTALL) -d $(GRAPHS_DIR) diff --git a/fs/common.mk b/fs/common.mk index 1625b65d0e..ba651fd4b7 100644 --- a/fs/common.mk +++ b/fs/common.mk @@ -45,6 +45,10 @@ ROOTFS_COMMON_DEPENDENCIES = \ $(BR2_TAR_HOST_DEPENDENCY) \ $(if $(PACKAGES_USERS)$(ROOTFS_USERS_TABLES),host-mkpasswd) +rootfs-common-show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(ROOTFS_COMMON_DEPENDENCIES)) + $(info rootfs-common: host) + $(info rootfs-common -> $(foreach d,$(ROOTFS_COMMON_DEPENDENCIES),$(d))) + .PHONY: rootfs-common rootfs-common: $(ROOTFS_COMMON_DEPENDENCIES) target-finalize @$(call MESSAGE,"Generating root filesystems common tables") @@ -77,6 +81,10 @@ ROOTFS_$(2)_TARGET_DIR = $$(ROOTFS_$(2)_DIR)/target ROOTFS_$(2)_DEPENDENCIES += rootfs-common +rootfs-$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$(ROOTFS_$(2)_DEPENDENCIES)) + $$(info rootfs-$(1): host) + $$(info rootfs-$(1) -> $$(foreach d,$$(ROOTFS_$(2)_DEPENDENCIES),$$(d))) + ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y) ROOTFS_$(2)_COMPRESS_EXT = .gz ROOTFS_$(2)_COMPRESS_CMD = gzip -9 -c diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index ae4bac509f..4b3100115d 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -827,6 +827,10 @@ $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPEND @: $$(info $(1)) +$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$($(2)_FINAL_ALL_DEPENDENCIES)) + $$(info $(1): $(4) $$(if $$($(2)_IS_VIRTUAL),virtual,$$($(2)_DL_VERSION))) + $$(info $(1) -> $$(foreach d,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(d))) + $(1)-graph-depends: graph-depends-requirements $(call pkg-graph-depends,$(1),--direct)