From patchwork Sun Mar 3 10:16:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050840 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="RaEP8SIM"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbN2SNmz9s4Y for ; Sun, 3 Mar 2019 21:16:47 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 567BD8637D; Sun, 3 Mar 2019 10:16:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8GdbngGklNQb; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 7156386138; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 112F41BF5E6 for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 0E512207A2 for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f0ZwWnODd7DH for ; Sun, 3 Mar 2019 10:16:40 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) by silver.osuosl.org (Postfix) with ESMTPS id 2D06620791 for ; Sun, 3 Mar 2019 10:16:40 +0000 (UTC) Received: by mail-wm1-f51.google.com with SMTP id a62so1916587wmh.4 for ; Sun, 03 Mar 2019 02:16:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=2H8jcdWvMXYRCN1GdqLFgwvofPBJ/yZG2lLGUduUSag=; b=RaEP8SIM3FTMEjUYkMZqg7rZj7TpbmBicJhkBNEziGU5+TzRJ9yjECpIv4NQj+sGrZ N9kwzc0YOamzyILLvu+Km8INTD2gARYad2q+EUrBwrnXaCs1AzFu7pnftEFz2pDkJjgq F6+c4ZlCMpW3XTKB0F356naTRaDjIwR98mEDr5KVfFYbAMcHtA+QGqwwSA+4GjjKqilg IvBKRyCBwSE4Fdz/bH3G6SJIQpNtRBK1BOwono31er2thC9/Nq4/zWAloYzfU3sNuIHG +CHcnzdG5IS5QojIS727SKezpk8s+/Zf2T4Sg+/KP0brZ10gzFOQAvd6pBesPcLv7bYG s1rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=2H8jcdWvMXYRCN1GdqLFgwvofPBJ/yZG2lLGUduUSag=; b=W/OLsyq1K0CGbWoSnm3GOfhcKT4v9Ys7lTMGVMWgNpY/od+xvFpGC2R90ZhGOEQZQC aC4yDr94CpOQb3OSTBXu2bDYof0ZZ8UXQsNSw0is+rT1c3jRPPUV8sIlj2WjA4pWNtoD dm+fCuiAQzCKl58uFv6YDRkIBWlqQx8QOWF4S8/Bch5sibJpxjeL9aGnDbSrwIOhQqlN /+/fmlWaGxr1FxAwRUgMTo7N7IOD7s2gxClQsCrEnl/CyiEcPB13NURTcs1EHlY7sdBM fbSmn0+CKtxou+4isG1KSTzlYX3dZy/A0L/cJPToLvtsHZmz1mVomG+I/oXsBJDGzMHY /0iw== X-Gm-Message-State: APjAAAUsAm+dKDY6vDNnmZe6jNEzt/Mn7xxpFai/AsaenVMk7W9TSDFJ DsGF93I/rO8oxL453EQle/r7m4Uk X-Google-Smtp-Source: AHgI3Ia7JeZFnu9WuikLlcgVk8aPZz/rWRynsCi6Fq9y220H31suYWrLivBcAPE7Nv6ks/qy9I40Ug== X-Received: by 2002:a1c:9c04:: with SMTP id f4mr8068011wme.149.1551608198480; Sun, 03 Mar 2019 02:16:38 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:37 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:29 +0100 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 1/8 v2] support/graph-depends: make sure mandatory deps are displayed X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: Thomas Petazzoni The current graph-depends implementation filters out a number of "mandatory" dependencies that all packages have: dependency on "toolchain" and dependency on "skeleton". Despite this filtering, in full graph dependencies, "toolchain" and "skeleton" are still shown, because they are target packages, and therefore appear in the result of "make show-targets". Thanks to this, they will be visible as dependencies of the "ALL" node, which is the root of the dependency tree. However, as we are going to introduce host-skeleton as a "mandatory dependency" to be filtered out, this is no longer going to work. This commit adjusts the remove_extra_deps() function to ensure that when a mandatory dependency is removed, this dependency exists between the root of the dependency tree and the mandatory dependency. This issue was noticed by Yann E. Morin, and this commit provides a different implementation than what Yann proposed in https://patchwork.ozlabs.org/patch/910453/. Signed-off-by: Thomas Petazzoni [yann.morin.1998@free.fr: - list mandatory deps before removing them - fix flake8 warnings ] Signed-off-by: "Yann E. MORIN" --- support/scripts/graph-depends | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index d2b100f385..5a6f6930e9 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -181,6 +181,12 @@ def remove_mandatory_deps(pkg, deps): return [p for p in deps[pkg] if p not in MANDATORY_DEPS] +# This function returns all dependencies of pkg that are part of the +# mandatory dependencies: +def get_mandatory_deps(pkg, deps): + return [p for p in deps[pkg] if p in MANDATORY_DEPS] + + # This function will check that there is no loop in the dependency chain # As a side effect, it builds up the dependency cache. def check_circular_deps(deps): @@ -213,6 +219,9 @@ def check_circular_deps(deps): def remove_extra_deps(deps, rootpkg, transitive): for pkg in list(deps.keys()): if not pkg == rootpkg: + for d in get_mandatory_deps(pkg, deps): + if d not in deps[rootpkg]: + deps[rootpkg].append(d) deps[pkg] = remove_mandatory_deps(pkg, deps) for pkg in list(deps.keys()): if not transitive or pkg == rootpkg: From patchwork Sun Mar 3 10:16:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050839 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="qL1P7B/f"; dkim-atps=neutral Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbM3WlCz9s47 for ; Sun, 3 Mar 2019 21:16:47 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 355CA20791; Sun, 3 Mar 2019 10:16:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OqSOLNXb1TQP; Sun, 3 Mar 2019 10:16:43 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 599B6207A2; Sun, 3 Mar 2019 10:16:43 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id EF1B31BF5E6 for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id EBFB087592 for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C0CDrYlHMm1g for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by hemlock.osuosl.org (Postfix) with ESMTPS id 3DDB986CDC for ; Sun, 3 Mar 2019 10:16:41 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id w17so2278296wrn.12 for ; Sun, 03 Mar 2019 02:16:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=Qi66UejYbPKj76VIDV+lwhe7BKXwsR0OftnevwNktAQ=; b=qL1P7B/fOBbKTmqC2QbYUBha2R4tlmUfRpmx8p/Q4fMcv8IzZrBBMIY5V/JL91j0Da cyRBGHFi8vCRPZCZ+LBFCIiC4lrLEpEf1+J4pmyVT1OwcWgYEWRXiGBM8NZ3oB314qhg 3ck8M+mf6fDbqky96E/9GaCL/y1shEJ1UiEKXCf2ipPDixp2XC7NgeTK0VEDkGEt/NRA SPxX2qrayfGQqsT+jUFTU3c1CNka9zoIK85HzPnOm7JDJa+64aPgIxf4HL00gdEZ81cY OoMUXf7JkFjj6wn1RUmFl85AylpnfvNjY5gH9kKFhObOD1dQkEF8SAvjfCCialSuaH0H 8E2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=Qi66UejYbPKj76VIDV+lwhe7BKXwsR0OftnevwNktAQ=; b=GCtrA0ENgtrVnTlxiCf1roLijIdfkK78Km0eGt/J4CCW7KJ0yvo/JX7yiWuX+H53B+ kOFUSoPmE9Nj37gRp+clmY36UI9Um2c0n3J92TRBye+4KUlZrS/h9ph984MEVQdG1Px3 e8zKF7Uj70RABl9m1RMR2bukvVxKxEKnhMKmmBMmn1c0+rwzmGx9DENS5e6H190XaCrQ HQxUpLDi+9N0wmx4OCMLkSHdSAPLZNJmSV4R93FjaxyekvVUmgKC7nzTSZY9sJ1nS/if d/ZliZfo/p0aI7q41V85YRRpwp3zNzkgZ2+NFDOKSbgxr0t5mVb3hHvWxDqO4VUK8Ti2 AAmQ== X-Gm-Message-State: APjAAAWMl9rv4WPAk83Hp/iNCeBJaNI9OUUENx6qwM7NaC5Hll0MoXjg YY7Jo74kh7tRvv+RIcXph75ywjRQ X-Google-Smtp-Source: APXvYqwcFX36Zctf+Ac/FQkj1JqspzbSxvBxbuuRYoRQoJlNV3elMeXK6V4Vtabj3EwiDNjMjD8lYg== X-Received: by 2002:adf:ee01:: with SMTP id y1mr8834229wrn.268.1551608199549; Sun, 03 Mar 2019 02:16:39 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:39 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:30 +0100 Message-Id: <6e0fb4d5b76d1188367a039f5b9102086d4ac8f8.1551608168.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 2/8 v2] support/graph-depends: add option to exclude mandatory deps X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Some times, multiple dependency graphs for a set of packages (mostly the application-level packages for the project) are included in reports (e.g. delivery notes). Repeating the mandatory dependencies on all those graphs is useless and clutters the important dependencies. When we had only two such mandatory dependencies (toolchain, skeleton), it was manageable to list them as manual exclusions: -x toolchain -x skeleton But we now have quite a few such dependencies, and it becomes a bit more cumbersome to manage, not counting the ones we may add in the future. Add an option to exclude all those mandatory dependencies, to generate neat graphs. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni --- support/scripts/graph-depends | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index 5a6f6930e9..5c5de7dd0b 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -311,6 +311,8 @@ def parse_args(): "'host' to stop on host packages.") parser.add_argument("--exclude", "-x", metavar="PACKAGE", dest="exclude_list", action="append", help="Like --stop-on, but do not add PACKAGE to the graph.") + parser.add_argument("--exclude-mandatory", "-X", action="store_true", + help="Like if -x was passed for all mandatory dependencies.") parser.add_argument("--colors", "-c", metavar="COLOR_LIST", dest="colors", default="lightblue,grey,gainsboro", help="Comma-separated list of the three colors to use" + @@ -364,6 +366,9 @@ def main(): else: exclude_list = args.exclude_list + if args.exclude_mandatory: + exclude_list += MANDATORY_DEPS + if args.direct: get_depends_func = brpkgutil.get_depends arrow_dir = "forward" From patchwork Sun Mar 3 10:16:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050841 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Yw+ccj0o"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbP6nY0z9s47 for ; Sun, 3 Mar 2019 21:16:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0502A85BEE; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K7g6bqaPfZUG; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0BE0C85A78; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id ED9171BF5E6 for ; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id E8D7485A78 for ; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OUOpPTlWUA9k for ; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 2AB2B858C6 for ; Sun, 3 Mar 2019 10:16:42 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id n19so1917068wmi.1 for ; Sun, 03 Mar 2019 02:16:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=lnKTnFZBeHkbKENlqcHlFUUAOduj7xuLYhsEjnA8/a8=; b=Yw+ccj0odaOEm8ZlWb32dMitlhIDoiuAEUZszbyIB0ABzRaHLN0Z8V+PyW0pjGP9Aa FwWH52nHWba7H3otjK2elLeJHUUEGoWnPS/mM0Kr2suzZt2X81hD1lye+TBlb6/4cwtS LxLlmUethXnOzL/SUvESvBaGTyg8LRcH9eEiHbakQiW+Ba4t87hlQGnppMYQlOV+Pc4x aSUkgoisSCXMq7WIYY615AbzDJZAW2nnUDMasMDaTnOa+/eyqDnzTtbz/yK82rSQB6gW ydi8PrJPkm46KR+hLBU53f4oUgxPagq3/cxcs3c1HQJ6X1j8GjaIhFFCbhrn4uBGTg+l vckA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=lnKTnFZBeHkbKENlqcHlFUUAOduj7xuLYhsEjnA8/a8=; b=B8lypgtBI5oWsw7m6nQQUUySevv4a4gfb15P5ux9u/5YrgapPkUq5iEG4J4JVszWS0 WxpEotpl8WHkDADdrdA/psS272aXjHxnEgtIKhfUoaqiTUv4aUKs+V4BJIrRlt70VVVO sLsBgvSbalkcZ1XX+vhVGvWR+WJJhAh+tUPT6M9DZBrnjCzMJm1lRNHf0Lf/H7n/682i j/Aah3uLw26M4NdK6M0Li/6B2oJTMt9xhVT1moqs5QAAefuoT2WVXUOgtwheq1oPw4DP VM4mJ4vxqsJkNSIJ22OGt2bsj1YiZAWNm5f+uxCXzSFBRKl5pQyCYEms7D/Vn8c9C6py FNEQ== X-Gm-Message-State: AHQUAuY4X9bPF/hxArd2uFsblXOVKd+K7nK1ppfe38q9e/jgcFjeVOOt +DCQwMa9uuU68YlNxAx+gd4NddbS X-Google-Smtp-Source: APXvYqyAD2zNxrBcag3dh5a5hrNKTtJeRGl5eHY6lq1TXgRPbJ2UJcUxV1GHWJEG30gKgCeFHbzANA== X-Received: by 2002:a1c:2082:: with SMTP id g124mr7708636wmg.59.1551608200413; Sun, 03 Mar 2019 02:16:40 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:39 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:31 +0100 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 3/8 v2] support/graph-depends: also cut on host-skeleton X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" host-skeleton is a dependency of almost all packages, except a very few. As such, it clutters the dependency graph uselessly. Do with it as we do for the skeleton: cut the dependency chains. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni --- support/scripts/graph-depends | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index 5c5de7dd0b..c3e5d83ee8 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -172,7 +172,7 @@ def remove_transitive_deps(pkg, deps): # List of dependencies that all/many packages have, and that we want # to trim when generating the dependency graph. -MANDATORY_DEPS = ['toolchain', 'skeleton'] +MANDATORY_DEPS = ['toolchain', 'skeleton', 'host-skeleton'] # This function removes the dependency on some 'mandatory' package, like the From patchwork Sun Mar 3 10:16:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050843 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="LM1rzmP2"; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbQ1wBRz9s5R for ; Sun, 3 Mar 2019 21:16:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A843C876C3; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zmpa31x4Ghgw; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id DFC8687592; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id DB08E1BF5E6 for ; Sun, 3 Mar 2019 10:16:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 5923F8613C for ; Sun, 3 Mar 2019 10:16:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P28CAbbG0EgP for ; Sun, 3 Mar 2019 10:16:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f67.google.com (mail-wm1-f67.google.com [209.85.128.67]) by whitealder.osuosl.org (Postfix) with ESMTPS id 765328618E for ; Sun, 3 Mar 2019 10:16:43 +0000 (UTC) Received: by mail-wm1-f67.google.com with SMTP id g20so1930382wmh.5 for ; Sun, 03 Mar 2019 02:16:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=NYDPx2DGLVKE4941Tl31V34xFnvVOhCzOlJk8L6L8H8=; b=LM1rzmP281RCh5w1crWEUst+yO4NeN02SfVfwNWHW+GN+RJ+5G+ryAPwFClM7oewlo AQ/gnYpEpsCzm4P5jAOHkbvc1g8Re9v31+gOLLPD9OV9H5VEgF+Zx76+i/ZKC5l5BVid hf7aixULOYsJJ1HQl4WoLD9fHGF0YIDC46DXG6OfwhzHaHasrCURE6H0Xlq3Yuil4OPr w+qYGa+EBC0l+Y/JwMF2NFwZIQHCPOXZBINKxMNgBMQo7QOJcg6IKSWKY59fIFHH5Y6L GKf17pg6Dmc0oHXfe5DA5mbu9A+q7AyGSnIPFzGwNKFqFhL+E7CdAk9aufbY4GBNHmSq VxkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=NYDPx2DGLVKE4941Tl31V34xFnvVOhCzOlJk8L6L8H8=; b=ZCFO+Et2oYG4d3V+US7w5461Lo3R+zWAmSO38DxvrPid08M+N0J+6PeLR1C1gfHehF I40k9ivjv6NNlaWkRjJlHPv//3nJ9cr0+78RC/4lh3IFwoHfGEsVL55L62KiyQvJ7Xcl LteyyqgDwmFejVw+0wQNUuicw6VQCfhwSBr/QD3X9OaBw3DNwKTyD6qHoTTX5Jp41dkc nzdIUXHbk41fv/ZLM/3yvAt1oM0GxorRh7ScRdM1WqLlv2rryibOG4TBDwGQ+qDOiBNE VwlnKRe6RET7N2AfTpOgZEPNRKQUREnGkE3DOlAvQfnxXoqZKY4OMz12skgGj39zr+45 4nyg== X-Gm-Message-State: AHQUAuYsD6zrURlZresMQIoYxoYInGARzf/OI5nHJ9uvEQWWzR7AnnGe s0cr/J+TEWaJg+7Sb9VjAy04SkkD X-Google-Smtp-Source: AHgI3Ib9JQSHvZqRPsDCzLwuCSqmnWGUXA7SjsgY1l3NfkKPI8SwA7/A5InMuFVOWjNwZV8kBzK1sA== X-Received: by 2002:a1c:cf43:: with SMTP id f64mr7658554wmg.61.1551608201791; Sun, 03 Mar 2019 02:16:41 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:41 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:32 +0100 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 4/8 v2] support/graph-depends: also cut on host-tar X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" When host-tar is needed, it is a mandatory dependency of all packages. As such, drawing the dependency lines toward host-tar would uselessly clutter the graph. So, like for the skeleton and host-skeleton, we cut the dependency chains toward host-tar. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni --- support/scripts/graph-depends | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index c3e5d83ee8..3143c61cc4 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -172,7 +172,7 @@ def remove_transitive_deps(pkg, deps): # List of dependencies that all/many packages have, and that we want # to trim when generating the dependency graph. -MANDATORY_DEPS = ['toolchain', 'skeleton', 'host-skeleton'] +MANDATORY_DEPS = ['toolchain', 'skeleton', 'host-skeleton', 'host-tar'] # This function removes the dependency on some 'mandatory' package, like the From patchwork Sun Mar 3 10:16:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050842 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="V25xHKy+"; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbQ0wdVz9s4Y for ; Sun, 3 Mar 2019 21:16:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 6D4CD8613C; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qL-0GKVg47FT; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 9C95B86180; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 01C8E1BF5E6 for ; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id EF1FC2150B for ; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cr+VSH7RwsDM for ; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by silver.osuosl.org (Postfix) with ESMTPS id 8D63B22618 for ; Sun, 3 Mar 2019 10:16:44 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id a62so1916688wmh.4 for ; Sun, 03 Mar 2019 02:16:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=6TRYf41uMne1IFp6pAnPxIMF5dx5TnxIGsTTpYae/KU=; b=V25xHKy+f4u9qgR6TvSgaNzPOFD2OuTmXZ/2g3UsS71rx6KxRqiAZ4hOg0YfhTDKs0 lVSfUuZsJKbjk6KNsKCI09Qvko4KOaKJHd+ZgPPHTZX37J54hj+UprlQf04qEmFkVphD p6xnMEUq7SifCeMy63f6XD7yYtkniKVVdpk3i3Np2Sz5HWpTJ7ud7puD7VKbkonD9pNq ZvDBLXf1XSY8LtH+QI3HeBlZjHfaR4P9FBhRQjtDdTI8DygtlN6v+Q71FNc+yVj3rqAj U8f5ytdvh7b3aYaj+M745zarsX2tOoChECVn0iUojbAjd5iCrrd3zIjaqw4caFoaNe3l Vavg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=6TRYf41uMne1IFp6pAnPxIMF5dx5TnxIGsTTpYae/KU=; b=tChKLS+B6au0kMpui6KaYTx563DhUjbIxer9MgbJ+0Cu5vQuPyKbYb6mTEBAgD6e4b MPw0dxKD1kqVTAhz/AIdcvQHemgZtrBEXcoOK48KDb/I6pmk2UP/lKYx7UBEtvZdTpM7 wN0mqmReez3bc87t540SfXuVTrKiN3srZCELJG5zGSnmsavvpYTVG4po+FDZ2Oi5T6/P C8O0We+yVzUfmngxpFJrJQMEZJfsRp77BEF1JkrkIuGGGZ2EVr0GH4CGpI3Lz2cfyXND hBEyuEu3xL6BiFNJLl4maBlVFQz/Z4Zl2zqs7XYsEuC0eE+d6QFopW7bLSCV3IKWAzwy p4jg== X-Gm-Message-State: APjAAAXr5HkcTDtj1bFZLHpWU/4Sx92WcPEjMT03OcCztapN5yIEJqza Xrw768TTKgOu4HPsCQymRy9tnchi X-Google-Smtp-Source: AHgI3IZZgoH3mieGOag14dj/x6ufSfhyMOxjBg+GfX+fKJc4sI6xL257Xcnwsy0fL2oJFGLNy+HVtQ== X-Received: by 2002:a1c:27c6:: with SMTP id n189mr7946847wmn.108.1551608202831; Sun, 03 Mar 2019 02:16:42 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:42 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:33 +0100 Message-Id: <54a5230dcc7d087c93c86805a89a3652b4fca09c.1551608168.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 5/8 v2] support/graph-depends: also cut on host-gzip X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" When host-gzip is needed, it is a mandatory dependency of all packages. As such, drawing the dependency lines toward host-gzip would uselessly clutter the graph. So, like for the skeleton, host-skeleton, and host-tar, we cut the dependency chains toward host-gzip. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni --- support/scripts/graph-depends | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index 3143c61cc4..29134c8237 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -172,7 +172,7 @@ def remove_transitive_deps(pkg, deps): # List of dependencies that all/many packages have, and that we want # to trim when generating the dependency graph. -MANDATORY_DEPS = ['toolchain', 'skeleton', 'host-skeleton', 'host-tar'] +MANDATORY_DEPS = ['toolchain', 'skeleton', 'host-skeleton', 'host-tar', 'host-gzip'] # This function removes the dependency on some 'mandatory' package, like the From patchwork Sun Mar 3 10:16:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050844 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="jpLLXwKh"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbS5bBLz9s47 for ; Sun, 3 Mar 2019 21:16:52 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 1D3C885C63; Sun, 3 Mar 2019 10:16:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xMRUOXN4BcrK; Sun, 3 Mar 2019 10:16:49 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id AEF7185BCA; Sun, 3 Mar 2019 10:16:49 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 7A18B1BF5E6 for ; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 71C8985BCA for ; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7R4sud9sIXec for ; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 91CEB85BCF for ; Sun, 3 Mar 2019 10:16:46 +0000 (UTC) Received: by mail-wm1-f66.google.com with SMTP id n19so1917170wmi.1 for ; Sun, 03 Mar 2019 02:16:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ALZZ5NCvDwA6UZBP6dtU628W4eNDqrtqSYEalz+vS4s=; b=jpLLXwKhMb1JEeRjczY9es2BtpzUaiI3Ry5Jf5KkFMNFVfHK7zzKzYQGVC5ndEEHc5 FpxnU8LD3V3MyhLLQheBCFie9F4iN+sBwpC1pFl8LRUl5LLW1o/LD2iOtnuydWf7g8eZ HQ+QofsL9HdqLgaybD2xR1H8r8mPBiELm0ApCk7zQhjtH0ia2/e2WX2zPtadlFooGAhU 95Ge9yxmPpoz5tqI6IX15p0saU0D0OaMWX4A4EMhifCrsEnspOZhDnImvLtnwU3uamAK rjqG44ROXSdMu/4cQIFOnjbSP9p02KlVLD5tmrjrxz7VtJhX1/oq9zA4YyrDbJoYUDkT R8tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=ALZZ5NCvDwA6UZBP6dtU628W4eNDqrtqSYEalz+vS4s=; b=mdTmMPwooYwKrFzJsEWoj6g+fLFRuww0OJervDO7UqDc+TOhd2jhFn4RBBKDTcG68m noie1Brwk6EDsDB0QP4R97QVmgyVO/t3O6D0TvuDAETXU+WX5flh4dy7f5Umgt7a7/qF SHSnt1ARQL9fdfg7qYj3FEBk4OtWtVZQ/hK6D56nXb08VUuCx8n8MCBzaBowwkh5xt64 OClxZarbJMCXy/2IylkcPtLjf/IXK15o+Q+ghbvzi1gqeKvHd6c/AXjcX6d9XEoFN2Dr Ey4tm3gd5gSRkFzDmpwi1o7WSoK7yuxiM5ZomzdD17OAJgmAfmC5JQfdmsz5miknYw90 dbQA== X-Gm-Message-State: APjAAAX+bLzLrZQfvgQ6ptaZi2s9BOZz5BaovN8MveD0VUZZPYWrRBiC YvAPTFetsAT8hnPocQjvfsck1leZ X-Google-Smtp-Source: APXvYqwGHVjdMMBvhrsgVxc5CtkFp3RKYqaR/Zx/5sIMEbfxfzw0P9dH73/PTJzV7LlTBZybySFs/A== X-Received: by 2002:a1c:7dd7:: with SMTP id y206mr7704198wmc.123.1551608204832; Sun, 03 Mar 2019 02:16:44 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:44 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:34 +0100 Message-Id: <7db07f60e79a87fbfd832aa5c6287641aeb78cf3.1551608168.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: MIME-Version: 1.0 Subject: [Buildroot] [PATCH 6/8 v2] infra/pkg-generic: use pure Makefile-based recursive dependencies X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , Joseph Kogut , "Yann E. MORIN" , Thomas Petazzoni , Trent Piepho Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Calling to the graph-depends script is very costly, as it calls back to 'make' a lot of time. It turns out that we already have the list of recursive dependencies, so we can just print that. As for the recursive reverse dependencies, we use the same memoisation technique to cut-down on the expansion cost, which would otherwise be on the order of 𝑶(𝑛²) (with 𝑛 enabled packages). From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and allyespackageconfig (tweaked to avoid multi providers, etc...), the timings for X-show-recursive-rdepends are: before after speedup #rdeps libnss 0m22.932s 0m5.775s 3.97x 3 qt5base 0m41.176s 0m5.781s 7.12x 67 libjpeg 0m56.185s 0m5.749s 9.71x 228 libxml2 0m54.964s 0m5.795s 9.48x 271 freetype 0m46.754s 0m5.819s 8.07x 287 libpng 0m53.577s 0m5.760s 9.30x 303 sqlite 1m15.222s 0m5.807s 12.95x 801 libopenssl 1m25.471s 0m5.844s 14.63x 931 readline 1m13.805s 0m5.775s 12.78x 958 libzlib 1m11.807s 0m5.820s 12.34x 1039 toolchain 1m23.712s 0m6.080s 13.77x 2107 skeleton 1m27.839s 0m6.293s 13.96x 2111 (+1) host-skeleton 1m27.405s 0m6.350s 13.76x 2172 (+2) - speedup: ratio before/after - #rdeps: number of recursive reverse dependencies, with the extra dependencies returned with this patch, see below for the reason. So, for a low-level package with a lot of reverse dependencies, like libzlibz, libopenssl or readline are, the timings are already very much in favour of the change. This is less impressive with packages that have few dependencies (libnss), but still much faster. Also, remember that the config tested has as much packages enabled as possible, so is in itself a degenerate case. With simpler and more realistic configurations, the gains would probably be a bit lower than reported above, but various tests still report good improvements overall (note: coming up with a 'realistic' configuration is pretty hard, as everyone and their dog have their notion of what is realistic in their context, so nothing displayed here; timings are left as an exercise for the interested parties to report aggravation in their cases should they notice some regression). Note that, more recursive reverse dependencies may be displayed now, since we do not apply the exceptions applied in graph-depends. For example, host-skeleton gains two new recursive reverse dependencies: skeleton and toolchain, which are both exceptions in graph-depends. As for direct (not reverse) dependencies: the gain is not as fantastic as for reverse ones, but it is still noticeable, especially thanks to a21212fb7cf (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES); just a few examples for %-show-recursive-depends: before after speedup #deps libzlib 0m46.864s 0m5.902s 7.94x 17 qt5base 0m57.590s 0m5.848s 9.85x 190 sqlite 0m46.601s 0m5.816s 8.01x 24 Basically, displaying recursive dependencies, direct or reverse, is almost a constant now: it only slightly varies by about 10% depending on the complexity of the dependency chain, with the parsing of the Makefiles still accounting for the large majority of the time. (PS. Thanks to Joseph for suggesting a list of interesting packages to test, and thanks to Trent for his example of memoisation!) Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Thomas De Schampheleire Cc: Joseph Kogut Cc: Trent Piepho --- Changes v1 -> v2: - use memoisation (Thanks Trent!) - redo the timings - drop the comments about skeleton and host-skeleton, that are no longer degenerate cases --- package/pkg-generic.mk | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 4353bd3868..0d17e62a70 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -649,6 +649,18 @@ $(2)_FINAL_RECURSIVE_DEPENDENCIES = $$(sort \ ) \ $$($(2)_FINAL_RECURSIVE_DEPENDENCIES__X)) +$(2)_FINAL_RECURSIVE_RDEPENDENCIES = $$(sort \ + $$(if $$(filter undefined,$$(origin $(2)_FINAL_RECURSIVE_RDEPENDENCIES__X)), \ + $$(eval $(2)_FINAL_RECURSIVE_RDEPENDENCIES__X := \ + $$(foreach p, \ + $$($(2)_RDEPENDENCIES), \ + $$(p) \ + $$($$(call UPPERCASE,$$(p))_FINAL_RECURSIVE_RDEPENDENCIES) \ + ) \ + ) \ + ) \ + $$($(2)_FINAL_RECURSIVE_RDEPENDENCIES__X)) + $(2)_INSTALL_STAGING ?= NO $(2)_INSTALL_IMAGES ?= NO $(2)_INSTALL_TARGET ?= YES @@ -826,15 +838,13 @@ $(1)-show-depends: @echo $$($(2)_FINAL_ALL_DEPENDENCIES) $(1)-show-recursive-depends: - @cd "$$(CONFIG_DIR)" && \ - $$(TOPDIR)/support/scripts/graph-depends -p $(1) -f -q + @echo $$($(2)_FINAL_RECURSIVE_DEPENDENCIES) $(1)-show-rdepends: @echo $$($(2)_RDEPENDENCIES) $(1)-show-recursive-rdepends: - @cd "$$(CONFIG_DIR)" && \ - $$(TOPDIR)/support/scripts/graph-depends -p $(1) --reverse -f -q + @echo $$($(2)_FINAL_RECURSIVE_RDEPENDENCIES) $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPENDENCIES)) @: From patchwork Sun Mar 3 10:16:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050845 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lfHKhCRx"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44BzbW3MBSz9s4Y for ; Sun, 3 Mar 2019 21:16:55 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A5EB085CD8; Sun, 3 Mar 2019 10:16:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1xfFmA2n3UI6; Sun, 3 Mar 2019 10:16:52 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8F03085C4B; Sun, 3 Mar 2019 10:16:50 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id CD7581BF5E6 for ; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id C4CEC85BCA for ; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U_rzL1DARpHy for ; Sun, 3 Mar 2019 10:16:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) by fraxinus.osuosl.org (Postfix) with ESMTPS id DCC25858C6 for ; Sun, 3 Mar 2019 10:16:47 +0000 (UTC) Received: by mail-wr1-f49.google.com with SMTP id i16so2272997wrs.13 for ; Sun, 03 Mar 2019 02:16:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=jiI1UP6AC1dZgasMgxMr6zDrj6bv8Cn9MxZ5LohxOsk=; b=lfHKhCRxI+JdoelhvNDOsr7glukNcBYmSjUlE8kOC2ywKGLXQRQK5UxQDvX8g2rScS 2AGpOrT6m5CkGtW4HHGb5pU6aZ7Lp+amMnBdQbkAP4WiFfOpQdasxdn3DjzPGoUFnO9s 0v3aCIZ0bDzoXeicUyMg8spRwZaom4yBxKwfSj5MpFDCk+CPW5DCdAzn40UWL0TlCnob +gBfZ+/dT3gjbe1WlecDvTmUBP/XeY8tMt24ulQW2kbt4gM1nmYEdlN4NNxOSRUaMCZj ZmwtEfsTRloqo665lN7hXa7a/dHV9owwBWCMLnCfHH/1+Mu6ml72/3B9FYc7ve8mdO6D VwJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=jiI1UP6AC1dZgasMgxMr6zDrj6bv8Cn9MxZ5LohxOsk=; b=FnWVFWI3qwGeNtZWtUbkyDKcIsDFNtR01hPjyg1QT6rnpJOA2ZH6Ea+L6znclO2aAv B1AJmORL2M+z0zw+0ehJq2CkFqm82SeOxrGnjWbd/7ZMV2e4bntk5lyOw1XXp/ls+gcW DKjqzjRMHE9r05GanjLL09TuHXo53ruHMk7E4ZBdNVbo6qmgwP1xjT34O5Yw5Nz/+44q 1PegsbTf9dhBOR5Ciit50b2GSQgn+yQOfyZIIkI1SY1UOuVEzXxxQDkl074Q6mQlIrL8 9B4ogXLIoqrqUSJgFIS6BaAe8FVCL+FU+g+AbOKn11fsdYtmT4ilJWpbRr9EYw3C/zq2 d3ew== X-Gm-Message-State: APjAAAXYzMhqUb0NTYYIoz3nU0Pgyc6ppA5JKbVwYWjgG7pzOtau4Fzv BDCo6++2Khy6D8i7s/7oJWJbcWgB X-Google-Smtp-Source: APXvYqzV2bZXTFACdS/JQmhKnNrcjn/stprirPy3riEOecUcuGKqU9PcEF5oowBNQh87CbfYgvyHQg== X-Received: by 2002:adf:e3d1:: with SMTP id k17mr8980300wrm.143.1551608206156; Sun, 03 Mar 2019 02:16:46 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:45 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:35 +0100 Message-Id: <441a286ab4c051146d250a65115a40431727faac.1551608168.git.yann.morin.1998@free.fr> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 7/8 v2] core: add make-based full-dependency list X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Currently, when we need to build the full dependency graph, we call make to show the list of packages (make show-targets), and then call it again and again iteratively while it returns new packages. Since calling make will parse the whole set of our Makefiles, this takes quite a bit of time (~4s each here), and the total can get pretty long. However, make being make, already builds the whole dependency tree information, so we can just ask for it. Add a new top-level rule 'show-dependency-tree' that displays the whole set of dependencies for all packages. For each package, its name, type and version is displayed, then all the direct, first-level dependencies are dumped. We choose a format that is not unlike the dot-graph format, because it is both easy to read as a human, and easy to parse as a machine: foo: target 1.2.3 foo -> bar host-meh bar: target virtual bar -> buz buz: target 2.3.4 buz -> host-meh: host virtual host-meh -> host-bleark host-bleark: host 3.4.5 host-bleark -> rootfs-meh: host rootfs-meh -> host-bleark To be noted: rootfs are currently reported as if they were 'host' packages, to stay aligned with how graph-depends currently treats them. Ideally, graph-depends could be enhanced to recognise them separately, but that is another story. For just plain defconfig, which is about the smallest config we can have with an internal toolchain, we already have a seven-fold improvement (with the graph-depends rule modified to not run the pdf generation, to be able to just compare the tree generation): $ time make graph-depends real 0m27.344s $ time make show-dependency-tree real 0m3.848s From defconfig, C++, wchar, locales, ssp, and allyespackageconfig, tweaked for even more packages (qt5 not qt4, luajit to avoid multi providers, etc...), the timings are (graph-depends still modified to not generate the pdf): $ time make graph-depends real 1m56.459s $ time make show-dependency-tree real 0m5.748s There. I don't think those numbers need any explanation whatsoever; they do speak on their own. OK, for maths sake, the ratio is about twenty-fold. So, "yeah", I guess... ;-) Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Thomas De Schampheleire --- Makefile | 4 ++++ fs/common.mk | 8 ++++++++ package/pkg-generic.mk | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/Makefile b/Makefile index 3d95d066fb..7d127e3799 100644 --- a/Makefile +++ b/Makefile @@ -876,6 +876,10 @@ graph-depends-requirements: @dot -? >/dev/null 2>&1 || \ { echo "ERROR: The 'dot' program from Graphviz is needed for graph-depends" >&2; exit 1; } +.PHONY: show-dependency-tree +show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(PACKAGES) $(TARGETS_ROOTFS)) + @: + .PHONY: graph-depends graph-depends: graph-depends-requirements @$(INSTALL) -d $(GRAPHS_DIR) diff --git a/fs/common.mk b/fs/common.mk index a560417c6c..daa43efd75 100644 --- a/fs/common.mk +++ b/fs/common.mk @@ -47,6 +47,10 @@ ROOTFS_COMMON_DEPENDENCIES = \ $(BR2_TAR_HOST_DEPENDENCY) \ $(if $(PACKAGES_USERS)$(ROOTFS_USERS_TABLES),host-mkpasswd) +rootfs-common-show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(ROOTFS_COMMON_DEPENDENCIES)) + $(info rootfs-common: host) + $(info rootfs-common -> $(foreach d,$(ROOTFS_COMMON_DEPENDENCIES),$(d))) + .PHONY: rootfs-common rootfs-common: $(ROOTFS_COMMON_DEPENDENCIES) target-finalize @$(call MESSAGE,"Generating root filesystems common tables") @@ -80,6 +84,10 @@ ROOTFS_$(2)_TARGET_DIR = $$(ROOTFS_$(2)_DIR)/target ROOTFS_$(2)_DEPENDENCIES += rootfs-common +rootfs-$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$(ROOTFS_$(2)_DEPENDENCIES)) + $$(info rootfs-$(1): host) + $$(info rootfs-$(1) -> $$(foreach d,$$(ROOTFS_$(2)_DEPENDENCIES),$$(d))) + ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y) ROOTFS_$(2)_COMPRESS_EXT = .gz ROOTFS_$(2)_COMPRESS_CMD = gzip -9 -c diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 0d17e62a70..62c2e221f7 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -850,6 +850,10 @@ $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPEND @: $$(info $(1)) +$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$($(2)_FINAL_ALL_DEPENDENCIES)) + $$(info $(1): $(4) $$(if $$($(2)_IS_VIRTUAL),virtual,$$($(2)_DL_VERSION))) + $$(info $(1) -> $$(foreach d,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(d))) + $(1)-graph-depends: graph-depends-requirements $(call pkg-graph-depends,$(1),--direct) From patchwork Sun Mar 3 10:16:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 1050846 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=free.fr Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="bcVmE+th"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44Bzbc5Ylpz9s47 for ; Sun, 3 Mar 2019 21:17:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0B3B485BD3; Sun, 3 Mar 2019 10:16:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jjenGNDGXxSQ; Sun, 3 Mar 2019 10:16:54 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 72A7A85C9F; Sun, 3 Mar 2019 10:16:53 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 31A851BF5E6 for ; Sun, 3 Mar 2019 10:16:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2E220858B3 for ; Sun, 3 Mar 2019 10:16:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MOXYZOUJasw7 for ; Sun, 3 Mar 2019 10:16:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f44.google.com (mail-wr1-f44.google.com [209.85.221.44]) by fraxinus.osuosl.org (Postfix) with ESMTPS id D25D985BD0 for ; Sun, 3 Mar 2019 10:16:49 +0000 (UTC) Received: by mail-wr1-f44.google.com with SMTP id w17so2278471wrn.12 for ; Sun, 03 Mar 2019 02:16:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references; bh=bnXhgtTA6xht8AOSyBxce62yLU+o5INPxV7iCWib3Mg=; b=bcVmE+thfcwoUpkrG/l9cV98CNvu05KyttJejeoiYuj/N91pMgRCwFk8Ga1rNH0XXw WT/daXwNxHrMFxkSL9uuiozFDd64wy+gaG3giAaQysnhyovZwWNNDFyTHuQcEDM3Q1PO cZa+l4TlUS3A7axUnkIGe8jtJ1vbeNHg3eZXcKawvhdhxCu73Osi4ENs5Qpb+ALER6Ke c2X5ilATcRwIn++DLoqB+4Ccromb3kAcr2qU0hkFrPjUQ72ZO61gi3cxF4Fz8aSanL+U 0tIxe6Agh3zhGX4JW4c4PrA1+fCKhCYCF3RhlLSzIIbRv0hBTEsusdEE7A7ntG69eEcp XWjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=bnXhgtTA6xht8AOSyBxce62yLU+o5INPxV7iCWib3Mg=; b=D4lO2OYhU8tliaC6H25QIQJvxpaTfir9R9pb1wYdT5SRH88uI2lHjaYyGBWeY6qbW5 4FsWVj+69QpLL5sHESunMDwkP4HAtOcLv5kC+ZuthAhGQwyuAzOOssmbKZV5R/EZWhtB HgSaWDuZDVNDQfj1qoMtWq/IMTrkvkyU7/jTi3NAVGa1DVE4nYJZ8EZvlnBa98m1K46q gK2QH/Hw9QBXCAi3qpQeaExNlv3KJh5agdGDPxhtTzrN3gIaytnqPjZPyHEkqGINiHlF ZLfwdceFv9p0cHtHMxQmAVr/g9XtbZHhgRUq5Ef5rkzIihsRTpXBnyAFeMF+mquC23wh RN6Q== X-Gm-Message-State: APjAAAUQhQiwvf4ijir9p4b8zT2H8GdsYzTiGUEGgNK/J7vRQKBNk2x+ XEeOr2D2GiAXXztzSrrxcSP/3aHs X-Google-Smtp-Source: APXvYqwIICdms/Ptz+O0dOgMHtiQB01Rbjk0/nKm1MOgkYhFOCIksToJ3jNDloEW9qQPnfTUg4eKkw== X-Received: by 2002:a5d:654d:: with SMTP id z13mr8784239wrv.270.1551608207918; Sun, 03 Mar 2019 02:16:47 -0800 (PST) Received: from scaer.home ([2a01:cb19:829a:2800:68e8:7a61:9bb9:12a]) by smtp.gmail.com with ESMTPSA id d9sm3031500wrn.72.2019.03.03.02.16.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Mar 2019 02:16:47 -0800 (PST) From: "Yann E. MORIN" To: buildroot@buildroot.org Date: Sun, 3 Mar 2019 11:16:36 +0100 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: Subject: [Buildroot] [PATCH 8/8 v2] support/graph-depends: use the new make-based dependency tree X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas De Schampheleire , "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Now that we can get the whole dependency tree from make, use it to speed up things considerably. So far, we had three functions to get the dependencies information: get_depends(), get_rdepends(), and, somehow unrelated, get_version(). Because of the way %-show-{,r}depends works, getting the dependency tree was expensive, the three functions all took a set of packages for which to get the dependencies, in an attempt to limit the time it took to get that tree. Now, getting the tree is much, much less costly, and we can get the whole tree as cheaply as we previously got only the first-level dependencies. Furthemore, we can now also get the version information at the same time, and that also brings in whether the package is virtual or not, target or host. So, we drop all three helper functions, and replace them with a single one that returns all that information in one go: full dependency tree, per-package type, and per-package version. Note: since commit 2d29fd96a (pkg-virtual: remove VERSION/SOURCE), virtual packages are no longer reported as having a 'virtual' version, so have since been displayed as regular packages in the graphs. Although noone complained, this patch incidentally restores the initial behaviour, and virtual packages are now correctly displayed as such again. Signed-off-by: "Yann E. MORIN" Cc: Thomas Petazzoni Cc: Thomas De Schampheleire --- Note: as I rewrote brpkgutil.py almost entirely now, and as I also substantially modified graph-depends, I added my (C) to them. We don't usually have copyright information in our files (no .mk or no Config.in and such have that), so I think it would be OK to just drop the (C) in there as well, especially since the Berne convention does not require it for a work to be protected and recognised anyway: https://en.wikipedia.org/wiki/Copyright_symbol Beside, the authorship information is already present in the git log, and it is much more accurate in there than it is in the files themselves. --- support/scripts/brpkgutil.py | 107 ++++++++++++++++++-------------------- support/scripts/graph-depends | 117 +++++------------------------------------- 2 files changed, 64 insertions(+), 160 deletions(-) diff --git a/support/scripts/brpkgutil.py b/support/scripts/brpkgutil.py index e70d525353..b5ea467bf9 100644 --- a/support/scripts/brpkgutil.py +++ b/support/scripts/brpkgutil.py @@ -1,67 +1,60 @@ # Copyright (C) 2010-2013 Thomas Petazzoni +# Copyright (C) 2018 Yann E. MORIN import logging -import sys +import os import subprocess -# Execute the "make -show-version" command to get the version of a given -# list of packages, and return the version formatted as a Python dictionary. -def get_version(pkgs): - logging.info("Getting version for %s" % pkgs) - cmd = ["make", "-s", "--no-print-directory"] - for pkg in pkgs: - cmd.append("%s-show-version" % pkg) - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) - output = p.communicate()[0] - if p.returncode != 0: - logging.error("Error getting version %s" % pkgs) - sys.exit(1) - output = output.split("\n") - if len(output) != len(pkgs) + 1: - logging.error("Error getting version") - sys.exit(1) - version = {} - for i in range(0, len(pkgs)): - pkg = pkgs[i] - version[pkg] = output[i] - return version +# This function returns a tuple of three dictionaries, all using package +# names as keys: +# - a dictionary which values are the lists of packages that are the +# dependencies of the package used as key; +# - a dictionary which values are the type of the package used as key; +# - a dictionary which values are the version of the package used as key, +# 'virtual' for a virtual package, or the empty string for a rootfs. +# +# 'direction' can be either 'direct' or 'forward' to get the direct (aka +# forward) dependencies, or 'back' or 'reverse' to get the reverse (aka +# backward) dependencies. +def get_dependency_tree(direction="direct"): + logging.info("Getting dependency tree...") - -def _get_depends(pkgs, rule): - logging.info("Getting dependencies for %s" % pkgs) - cmd = ["make", "-s", "--no-print-directory"] - for pkg in pkgs: - cmd.append("%s-%s" % (pkg, rule)) - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) - output = p.communicate()[0] - if p.returncode != 0: - logging.error("Error getting dependencies %s\n" % pkgs) - sys.exit(1) - output = output.split("\n") - if len(output) != len(pkgs) + 1: - logging.error("Error getting dependencies") - sys.exit(1) deps = {} - for i in range(0, len(pkgs)): - pkg = pkgs[i] - pkg_deps = output[i].split(" ") - if pkg_deps == ['']: - deps[pkg] = [] + types = {} + versions = {} + + # Special case for the 'all' top-level fake package + deps['all'] = [] + types['all'] = 'target' + versions['all'] = '' + + cmd = ["make", "-s", "--no-print-directory", "show-dependency-tree"] + with open(os.devnull, 'wb') as devnull: + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=devnull, universal_newlines=True) + output = p.communicate()[0] + + for l in output.splitlines(): + if " -> " in l: + pkg = l.split(" -> ")[0] + if direction == "forward" or direction == "direct": + try: + deps[pkg] += l.split(" -> ")[1].split() + except KeyError: + deps[pkg] = l.split(" -> ")[1].split() + elif direction == "back" or direction == "reverse": + for p in l.split(" -> ")[1].split(): + try: + deps[p].append(pkg) + except KeyError: + deps[p] = [pkg] + else: + raise ValueError('direction must be one of: direct, forward, reverse, back') else: - deps[pkg] = pkg_deps - return deps + pkg, type_version = l.split(": ", 1) + t, v = "{} -".format(type_version).split(None, 2)[:2] + deps['all'].append(pkg) + types[pkg] = t + versions[pkg] = v - -# Execute the "make -show-depends" command to get the list of -# dependencies of a given list of packages, and return the list of -# dependencies formatted as a Python dictionary. -def get_depends(pkgs): - return _get_depends(pkgs, 'show-depends') - - -# Execute the "make -show-rdepends" command to get the list of -# reverse dependencies of a given list of packages, and return the -# list of dependencies formatted as a Python dictionary. -def get_rdepends(pkgs): - return _get_depends(pkgs, 'show-rdepends') + return (deps, types, versions) diff --git a/support/scripts/graph-depends b/support/scripts/graph-depends index 29134c8237..139a5dee80 100755 --- a/support/scripts/graph-depends +++ b/support/scripts/graph-depends @@ -20,10 +20,10 @@ # configuration. # # Copyright (C) 2010-2013 Thomas Petazzoni +# Copyright (C) 2018 Yann E. MORIN import logging import sys -import subprocess import argparse from fnmatch import fnmatch @@ -36,63 +36,6 @@ MODE_PKG = 2 # draw dependency graph for a given package allpkgs = [] -# Execute the "make show-targets" command to get the list of the main -# Buildroot PACKAGES and return it formatted as a Python list. This -# list is used as the starting point for full dependency graphs -def get_targets(): - logging.info("Getting targets") - cmd = ["make", "-s", "--no-print-directory", "show-targets"] - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) - output = p.communicate()[0].strip() - if p.returncode != 0: - return None - if output == '': - return [] - return output.split(' ') - - -# Recursive function that builds the tree of dependencies for a given -# list of packages. The dependencies are built in a list called -# 'dependencies', which contains tuples of the form (pkg1 -> -# pkg2_on_which_pkg1_depends, pkg3 -> pkg4_on_which_pkg3_depends) and -# the function finally returns this list. -def get_all_depends(pkgs, get_depends_func): - dependencies = [] - - # Filter the packages for which we already have the dependencies - filtered_pkgs = [] - for pkg in pkgs: - if pkg in allpkgs: - continue - filtered_pkgs.append(pkg) - allpkgs.append(pkg) - - if len(filtered_pkgs) == 0: - return [] - - depends = get_depends_func(filtered_pkgs) - - deps = set() - for pkg in filtered_pkgs: - pkg_deps = depends[pkg] - - # This package has no dependency. - if pkg_deps == []: - continue - - # Add dependencies to the list of dependencies - for dep in pkg_deps: - dependencies.append((pkg, dep)) - deps.add(dep) - - if len(deps) != 0: - newdeps = get_all_depends(deps, get_depends_func) - if newdeps is not None: - dependencies += newdeps - - return dependencies - - # The Graphviz "dot" utility doesn't like dashes in node names. So for # node names, we strip all dashes. Also, nodes can't start with a number, # so we prepend an underscore. @@ -230,7 +173,7 @@ def remove_extra_deps(deps, rootpkg, transitive): # Print the attributes of a node: label and fill-color -def print_attrs(outfile, pkg, version, depth, colors): +def print_attrs(outfile, pkg, pkg_type, pkg_version, depth, colors): name = pkg_node_name(pkg) if pkg == 'all': label = 'ALL' @@ -239,13 +182,11 @@ def print_attrs(outfile, pkg, version, depth, colors): if depth == 0: color = colors[0] else: - if pkg.startswith('host') \ - or pkg.startswith('toolchain') \ - or pkg.startswith('rootfs'): + if pkg_type == "host": color = colors[2] else: color = colors[1] - if version == "virtual": + if pkg_version == "virtual": outfile.write("%s [label = <%s>]\n" % (name, label)) else: outfile.write("%s [label = \"%s\"]\n" % (name, label)) @@ -256,13 +197,13 @@ done_deps = [] # Print the dependency graph of a package -def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, +def print_pkg_deps(outfile, dict_deps, dict_types, dict_versions, stop_list, exclude_list, arrow_dir, draw_graph, depth, max_depth, pkg, colors): if pkg in done_deps: return done_deps.append(pkg) if draw_graph: - print_attrs(outfile, pkg, dict_version.get(pkg), depth, colors) + print_attrs(outfile, pkg, dict_types[pkg], dict_versions[pkg], depth, colors) elif depth != 0: outfile.write("%s " % pkg) if pkg not in dict_deps: @@ -270,17 +211,15 @@ def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, for p in stop_list: if fnmatch(pkg, p): return - if dict_version.get(pkg) == "virtual" and "virtual" in stop_list: + if dict_versions[pkg] == "virtual" and "virtual" in stop_list: return - if pkg.startswith("host-") and "host" in stop_list: + if dict_types[pkg] == "host" and "host" in stop_list: return if max_depth == 0 or depth < max_depth: for d in dict_deps[pkg]: - if dict_version.get(d) == "virtual" \ - and "virtual" in exclude_list: + if dict_versions[d] == "virtual" and "virtual" in exclude_list: continue - if d.startswith("host-") \ - and "host" in exclude_list: + if dict_types[d] == "host" and "host" in exclude_list: continue add = True for p in exclude_list: @@ -290,7 +229,7 @@ def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, if add: if draw_graph: outfile.write("%s -> %s [dir=%s]\n" % (pkg_node_name(pkg), pkg_node_name(d), arrow_dir)) - print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, + print_pkg_deps(outfile, dict_deps, dict_types, dict_versions, stop_list, exclude_list, arrow_dir, draw_graph, depth + 1, max_depth, d, colors) @@ -352,6 +291,7 @@ def main(): if args.package is None: mode = MODE_FULL + rootpkg = 'all' else: mode = MODE_PKG rootpkg = args.package @@ -370,13 +310,11 @@ def main(): exclude_list += MANDATORY_DEPS if args.direct: - get_depends_func = brpkgutil.get_depends arrow_dir = "forward" else: if mode == MODE_FULL: logging.error("--reverse needs a package") sys.exit(1) - get_depends_func = brpkgutil.get_rdepends arrow_dir = "back" draw_graph = not args.flat_list @@ -389,46 +327,19 @@ def main(): logging.error("Error: incorrect color list '%s'" % args.colors) sys.exit(1) - # In full mode, start with the result of get_targets() to get the main - # targets and then use get_all_depends() for all targets - if mode == MODE_FULL: - targets = get_targets() - dependencies = [] - allpkgs.append('all') - filtered_targets = [] - for tg in targets: - dependencies.append(('all', tg)) - filtered_targets.append(tg) - deps = get_all_depends(filtered_targets, get_depends_func) - if deps is not None: - dependencies += deps - rootpkg = 'all' - - # In pkg mode, start directly with get_all_depends() on the requested - # package - elif mode == MODE_PKG: - dependencies = get_all_depends([rootpkg], get_depends_func) - - # Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] } - dict_deps = {} - for dep in dependencies: - if dep[0] not in dict_deps: - dict_deps[dep[0]] = [] - dict_deps[dep[0]].append(dep[1]) + dict_deps, dict_types, dict_versions = brpkgutil.get_dependency_tree(arrow_dir) check_circular_deps(dict_deps) if check_only: sys.exit(0) dict_deps = remove_extra_deps(dict_deps, rootpkg, args.transitive) - dict_version = brpkgutil.get_version([pkg for pkg in allpkgs - if pkg != "all" and not pkg.startswith("root")]) # Start printing the graph data if draw_graph: outfile.write("digraph G {\n") - print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, + print_pkg_deps(outfile, dict_deps, dict_types, dict_versions, stop_list, exclude_list, arrow_dir, draw_graph, 0, args.depth, rootpkg, colors) if draw_graph: