From patchwork Fri Sep 21 13:31:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Petazzoni X-Patchwork-Id: 973215 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=bootlin.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gvdn14xPz9sBW for ; Fri, 21 Sep 2018 23:31:56 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 0F84030ACE; Fri, 21 Sep 2018 13:31:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XVETc+5xYgZ9; Fri, 21 Sep 2018 13:31:50 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id EC71C23251; Fri, 21 Sep 2018 13:31:49 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 9882D1C1492 for ; Fri, 21 Sep 2018 13:31:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 94915884E9 for ; Fri, 21 Sep 2018 13:31:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KzJKnHB+jNeX for ; Fri, 21 Sep 2018 13:31:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail.bootlin.com (mail.bootlin.com [62.4.15.54]) by whitealder.osuosl.org (Postfix) with ESMTP id C13238846E for ; Fri, 21 Sep 2018 13:31:47 +0000 (UTC) Received: by mail.bootlin.com (Postfix, from userid 110) id 36AE8208D9; Fri, 21 Sep 2018 15:31:46 +0200 (CEST) Received: from localhost (AAubervilliers-681-1-42-80.w90-88.abo.wanadoo.fr [90.88.160.80]) by mail.bootlin.com (Postfix) with ESMTPSA id 869A4207E5; Fri, 21 Sep 2018 15:31:26 +0200 (CEST) From: Thomas Petazzoni To: Buildroot List Date: Fri, 21 Sep 2018 15:31:17 +0200 Message-Id: <20180921133117.9300-3-thomas.petazzoni@bootlin.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180921133117.9300-1-thomas.petazzoni@bootlin.com> References: <20180921133117.9300-1-thomas.petazzoni@bootlin.com> Subject: [Buildroot] [PATCH 2/2] support/scripts/graph-build-time: add support for timeline graphing X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Yann E. MORIN" , Thomas Petazzoni MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This commit adds support for a new type of graph, showing the timeline of a build. It shows, with one line per package, when each of this package steps started/ended, and therefore allows to see the sequencing of the package builds. For a fully serialized build like we have today, this is not super useful (except to show that everything is serialized), but it becomes much more useful in the context of top-level parallel build. The graph-build make target is extended to also generate this new timeline graph. Signed-off-by: Thomas Petazzoni --- Makefile | 2 ++ support/scripts/graph-build-time | 71 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 72 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 18afa36b28..49f1106213 100644 --- a/Makefile +++ b/Makefile @@ -831,6 +831,8 @@ graph-build: $(O)/build/build-time.log --type=pie-$(t) --input=$(<) \ --output=$(GRAPHS_DIR)/build.pie-$(t).$(BR_GRAPH_OUT) \ $(if $(BR2_GRAPH_ALT),--alternate-colors)$(sep)) + ./support/scripts/graph-build-time --type=timeline --input=$(<) \ + --output=$(GRAPHS_DIR)/build.timeline.$(BR_GRAPH_OUT) .PHONY: graph-depends-requirements graph-depends-requirements: diff --git a/support/scripts/graph-build-time b/support/scripts/graph-build-time index 892e08bf07..c4aa066050 100755 --- a/support/scripts/graph-build-time +++ b/support/scripts/graph-build-time @@ -240,6 +240,73 @@ def pkg_pie_time_per_step(data, output): plt.title('Build time per step') plt.savefig(output) +step_colors = { + 'download': 'blue', + 'extract': 'red', + 'patch': 'green', + 'configure': 'black', + 'build': 'cyan', + 'install-target': 'orange', + 'install-staging': 'yellow', + 'install-images': 'purple', + 'install-host': 'grey' +} + +def pkg_timeline(data, output): + start = 0 + end = 0 + + # Find the first timestamp and the last timestamp + for p in data: + for k, v in p.steps_start.iteritems(): + if start == 0 or v < start: + start = v + if end < v: + end = v + + # Readjust all timestamps so that 0 is the start of the build + # instead of being Epoch + for p in data: + for k, v in p.steps_start.iteritems(): + p.steps_start[k] = v - start + for k, v in p.steps_end.iteritems(): + p.steps_end[k] = v - start + + plt.figure() + + i = 0 + labels_names = [] + labels_coords = [] + # reversing the list, because the packages build first appear last in the list + for p in reversed(data): + durations = [] + colors = [] + for step in steps: + if not step in p.steps_start or not step in p.steps_end: + continue + durations.append((p.steps_start[step], + p.steps_end[step] - p.steps_start[step])) + colors.append(step_colors[step]) + plt.broken_barh(durations, (i, 6), facecolors=colors) + labels_coords.append(i + 3) + labels_names.append(p.name) + i += 10 + + axes = plt.gcf().gca() + + axes.set_ylim(0, i + 10) + axes.set_xlim(0, end - start) + axes.set_xlabel('seconds since start') + axes.set_yticks(labels_coords) + axes.set_yticklabels(labels_names) + axes.set_axisbelow(True) + axes.grid(True, linewidth=0.2, zorder=-1) + + plt.gcf().subplots_adjust(left=0.2) + + plt.tick_params(axis='y', which='both', labelsize=6) + plt.title('Timeline') + plt.savefig(output, dpi=300) # Parses the csv file passed on standard input and returns a list of # Package objects, filed with the duration of each step and the total @@ -277,7 +344,7 @@ def read_data(input_file): parser = argparse.ArgumentParser(description='Draw build time graphs') parser.add_argument("--type", '-t', metavar="GRAPH_TYPE", - help="Type of graph (histogram, pie-packages, pie-steps)") + help="Type of graph (histogram, pie-packages, pie-steps, timeline)") parser.add_argument("--order", '-O', metavar="GRAPH_ORDER", help="Ordering of packages: build or duration (for histogram only)") parser.add_argument("--alternate-colors", '-c', action="store_true", @@ -307,6 +374,8 @@ elif args.type == "pie-packages": pkg_pie_time_per_package(d, args.output) elif args.type == "pie-steps": pkg_pie_time_per_step(d, args.output) +elif args.type == "timeline": + pkg_timeline(d, args.output) else: sys.stderr.write("Unknown type: %s\n" % args.type) exit(1)