From patchwork Tue May 2 09:23:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775766 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=RpN1DcXw; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZS06qZxz1ydX for ; Tue, 2 May 2023 19:25:20 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmEl-0004zd-Oe; Tue, 02 May 2023 05:23:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEj-0004yr-SE for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:53 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEi-0003zC-2R for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019428; bh=gyQnCVNzCEHVqVEoI58x5xyPhDVaVkRYebDl9/4EbWs=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=RpN1DcXwcWhaZcIGcAC+x+28ysyroenfcYW5S1hJbYWbbkzMRyI3xIy8hXLIVjs+e xslc1+Jc/asfFmS8Oq+Yu+EpR6+eDcqzwyFalo3s+zJ3S0K3mb3Y548lynPVlZb4Jf pNKhYXSzd/ryn8hgRXAG69RmEMVHhipm0dTtsfN4NVy0DKqcP9xw+TD306R+GjLbYd Rh0+3a38aFLO5YRG3Ws/Odo3sbXhta1629L0ry0NScRQQyT83uWnFNRH7r17mQeDIN kxgMq5Qs3k5pmYN/paZai5PIqHHRMq6avn6Z6aJJoaoxwjlg4BNvmSJjRwzPGxtIka AxB1LeIa2frRg== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 67A1F26339A7; Tue, 2 May 2023 09:23:46 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 01/12] simpletrace: Improve parsing of sys.argv; fix files never closed. Date: Tue, 2 May 2023 11:23:28 +0200 Message-Id: <20230502092339.27341-2-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-GUID: HRRXwbabQLfOZHV_s2sE6rYYAuvvTOAv X-Proofpoint-ORIG-GUID: HRRXwbabQLfOZHV_s2sE6rYYAuvvTOAv X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E425=2C18=2E0?= =?utf-8?q?=2E816=2C17=2E11=2E62=2E513=2E0000000_definitions=3D2022-01-18=5F?= =?utf-8?q?01=3A2022-01-14=5F01=2C2022-01-18=5F01=2C2021-12-02=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1030 mlxlogscore=631 bulkscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2305020080 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal The arguments extracted from `sys.argv` named and unpacked to make it clear what the arguments are and what they're used for. The two input files were opened, but never explicitly closed. File usage changed to use `with` statement to take care of this. At the same time, ownership of the file-object is moved up to `run` function. Secondary `open` inside `process` removed so there's only one place to handle `open`. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 1f6d1ae1f3..9211caaec1 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -9,6 +9,7 @@ # # For help see docs/devel/tracing.rst +import sys import struct import inspect from tracetool import read_events, Event @@ -44,7 +45,6 @@ def get_record(edict, idtoname, rechdr, fobj): try: event = edict[name] except KeyError as e: - import sys sys.stderr.write('%s event is logged but is not declared ' \ 'in the trace events file, try using ' \ 'trace-events-all instead.\n' % str(e)) @@ -166,11 +166,6 @@ def end(self): def process(events, log, analyzer, read_header=True): """Invoke an analyzer on each event in a log.""" - if isinstance(events, str): - events = read_events(open(events, 'r'), events) - if isinstance(log, str): - log = open(log, 'rb') - if read_header: read_trace_header(log) @@ -223,19 +218,18 @@ def run(analyzer): This function is useful as a driver for simple analysis scripts. More advanced scripts will want to call process() instead.""" - import sys - - read_header = True - if len(sys.argv) == 4 and sys.argv[1] == '--no-header': - read_header = False - del sys.argv[1] - elif len(sys.argv) != 3: - sys.stderr.write('usage: %s [--no-header] ' \ - '\n' % sys.argv[0]) + + try: + # NOTE: See built-in `argparse` module for a more robust cli interface + *no_header, trace_event_path, trace_file_path = sys.argv[1:] + assert no_header == [] or no_header == ['--no-header'], 'Invalid no-header argument' + except (AssertionError, ValueError): + sys.stderr.write(f'usage: {sys.argv[0]} [--no-header] \n') sys.exit(1) - events = read_events(open(sys.argv[1], 'r'), sys.argv[1]) - process(events, sys.argv[2], analyzer, read_header=read_header) + with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: + events = read_events(events_fobj, trace_event_path) + process(events, log_fobj, analyzer, read_header=not no_header) if __name__ == '__main__': class Formatter(Analyzer): From patchwork Tue May 2 09:23:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775773 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=HxooxGhA; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZTB6YnDz1ydX for ; Tue, 2 May 2023 19:26:22 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmEm-0004zw-Ef; Tue, 02 May 2023 05:23:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEk-0004z2-2m for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:54 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEi-0003zM-JL for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019431; bh=bQQvCx31M4vekj2rknUKPAqUmbQz4GRIzBFReFyfIzI=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=HxooxGhAkIY7cGYiBmJsUqtPn6TIO9yvZWf/WPsU8LEQ85P51gxgOE9xZC930G3Z7 OlsQTf6VlQbU0uAeRIVgomOp9aDNNaJ/CCxmLcll7UI0wGG45ddPfnPqq5q5yb0n1v Z/qQQdBSUXpHTPEv2pqEwhtjHZclAk765TC0m96ei90j7mPXxXAjd9WAmwr95aiN9+ UNf5Ua2UukbLzgxHbE5ig1YrWegPfq1484e3ULCb1Ged6wRs9ajG9F8Co1aJ/ld9F+ csdMfTVNUAA1gdSIL3i0xacidriUM8ZQgFyZjTwuj8Za0MCwfwJs/jaklpXUacW4uE fUJJV86FXAQxw== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 3B4F526337E1; Tue, 2 May 2023 09:23:48 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 02/12] simpletrace: Annotate magic constants from QEMU code Date: Tue, 2 May 2023 11:23:29 +0200 Message-Id: <20230502092339.27341-3-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-GUID: 5ye3mee2UTaR4haVwc9YY3PFB5RqNemA X-Proofpoint-ORIG-GUID: 5ye3mee2UTaR4haVwc9YY3PFB5RqNemA X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E425=2C18=2E0?= =?utf-8?q?=2E816=2C17=2E11=2E62=2E513=2E0000000_definitions=3D2022-01-18=5F?= =?utf-8?q?01=3A2022-01-14=5F01=2C2022-01-18=5F01=2C2021-12-02=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1030 mlxlogscore=613 bulkscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2305020080 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal It wasn't clear where the constants and structs came from, so I added comments to help. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 9211caaec1..7ba805443d 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -15,15 +15,15 @@ from tracetool import read_events, Event from tracetool.backend.simple import is_string -header_event_id = 0xffffffffffffffff -header_magic = 0xf2b177cb0aa429b4 -dropped_event_id = 0xfffffffffffffffe +header_event_id = 0xffffffffffffffff # trace/simple.c::HEADER_EVENT_ID +header_magic = 0xf2b177cb0aa429b4 # trace/simple.c::HEADER_MAGIC +dropped_event_id = 0xfffffffffffffffe # trace/simple.c::DROPPED_EVENT_ID -record_type_mapping = 0 -record_type_event = 1 +record_type_mapping = 0 # trace/simple.c::TRACE_RECORD_TYPE_MAPPING +record_type_event = 1 # trace/simple.c::TRACE_RECORD_TYPE_EVENT -log_header_fmt = '=QQQ' -rec_header_fmt = '=QQII' +log_header_fmt = '=QQQ' # trace/simple.c::TraceLogHeader +rec_header_fmt = '=QQII' # trace/simple.c::TraceRecord def read_header(fobj, hfmt): '''Read a trace record header''' From patchwork Tue May 2 09:23:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=bO+aeN5q; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZRc0Fv4z1ymp for ; Tue, 2 May 2023 19:24:59 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmEr-00050Y-2C; Tue, 02 May 2023 05:24:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEq-00050A-0u for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:00 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEk-0003zq-VA for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019433; bh=/KmlytnpwbgD49q4DS/EcNt6l8csM0KFrKLmm68kPao=; h=From:To:Subject:Date:Message-Id:MIME-Version:Content-Type; b=bO+aeN5qy0KHH3pY5T2G5P0BTwBzY+tZkdfmjb4uZmREzRMPDqpH1MSbNKgTquiic L8p9l6Ys+JURfK0f8CMqOOi9z8/FiDdgGaNHUxkRsi8aY3m9bVjcnSCO2ltPyfQfXr gxVBp0y6L4ht2LQWGumm6Il0G1lXdsNvD+ggToIiuVwmBZ48zZur1UFNzeeVY8xCMY x5EsQVlVXPrzKdKfRT+JBMaA1dCaeZSR3fmOOxCcOGsxp3jDNkQBEQdsbGo7jhyEQq PtbTaOGj3I9+EdMI/wDvIYpygL75WncQ0eagjR+DUMYpzWyohXFDpALesPuVeLGRC9 wUETA/0WA1q4w== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id A4B50263363D; Tue, 2 May 2023 09:23:51 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 03/12] simpletrace: changed naming of edict and idtoname to improve readability Date: Tue, 2 May 2023 11:23:30 +0200 Message-Id: <20230502092339.27341-4-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-GUID: PQTYx7dpOPA7u02HiOmSus87arTRD6vD X-Proofpoint-ORIG-GUID: PQTYx7dpOPA7u02HiOmSus87arTRD6vD X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E425=2C18=2E0?= =?utf-8?q?=2E816=2C17=2E11=2E62=2E513=2E0000000_definitions=3D2022-01-18=5F?= =?utf-8?q?01=3A2022-01-14=5F01=2C2022-01-18=5F01=2C2021-12-02=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1030 mlxlogscore=747 bulkscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2305020080 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Readability is subjective, but I've expanded the naming of the variables and arguments, to help with understanding for new eyes on the code. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 7ba805443d..9981699630 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -33,17 +33,17 @@ def read_header(fobj, hfmt): return None return struct.unpack(hfmt, hdr) -def get_record(edict, idtoname, rechdr, fobj): +def get_record(event_mapping, event_id_to_name, rechdr, fobj): """Deserialize a trace record from a file into a tuple (name, timestamp, pid, arg1, ..., arg6).""" if rechdr is None: return None if rechdr[0] != dropped_event_id: event_id = rechdr[0] - name = idtoname[event_id] + name = event_id_to_name[event_id] rec = (name, rechdr[1], rechdr[3]) try: - event = edict[name] + event = event_mapping[name] except KeyError as e: sys.stderr.write('%s event is logged but is not declared ' \ 'in the trace events file, try using ' \ @@ -72,10 +72,10 @@ def get_mapping(fobj): return (event_id, name) -def read_record(edict, idtoname, fobj): +def read_record(event_mapping, event_id_to_name, fobj): """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, arg1, ..., arg6).""" rechdr = read_header(fobj, rec_header_fmt) - return get_record(edict, idtoname, rechdr, fobj) + return get_record(event_mapping, event_id_to_name, rechdr, fobj) def read_trace_header(fobj): """Read and verify trace file header""" @@ -96,14 +96,14 @@ def read_trace_header(fobj): raise ValueError('Log format %d not supported with this QEMU release!' % log_version) -def read_trace_records(edict, idtoname, fobj): +def read_trace_records(event_mapping, event_id_to_name, fobj): """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). - Note that `idtoname` is modified if the file contains mapping records. + Note that `event_id_to_name` is modified if the file contains mapping records. Args: - edict (str -> Event): events dict, indexed by name - idtoname (int -> str): event names dict, indexed by event ID + event_mapping (str -> Event): events dict, indexed by name + event_id_to_name (int -> str): event names dict, indexed by event ID fobj (file): input file """ @@ -115,9 +115,9 @@ def read_trace_records(edict, idtoname, fobj): (rectype, ) = struct.unpack('=Q', t) if rectype == record_type_mapping: event_id, name = get_mapping(fobj) - idtoname[event_id] = name + event_id_to_name[event_id] = name else: - rec = read_record(edict, idtoname, fobj) + rec = read_record(event_mapping, event_id_to_name, fobj) yield rec @@ -172,16 +172,16 @@ def process(events, log, analyzer, read_header=True): frameinfo = inspect.getframeinfo(inspect.currentframe()) dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", frameinfo.lineno + 1, frameinfo.filename) - edict = {"dropped": dropped_event} - idtoname = {dropped_event_id: "dropped"} + event_mapping = {"dropped": dropped_event} + event_id_to_name = {dropped_event_id: "dropped"} for event in events: - edict[event.name] = event + event_mapping[event.name] = event # If there is no header assume event ID mapping matches events list if not read_header: for event_id, event in enumerate(events): - idtoname[event_id] = event.name + event_id_to_name[event_id] = event.name def build_fn(analyzer, event): if isinstance(event, str): @@ -205,9 +205,9 @@ def build_fn(analyzer, event): analyzer.begin() fn_cache = {} - for rec in read_trace_records(edict, idtoname, log): + for rec in read_trace_records(event_mapping, event_id_to_name, log): event_num = rec[0] - event = edict[event_num] + event = event_mapping[event_num] if event_num not in fn_cache: fn_cache[event_num] = build_fn(analyzer, event) fn_cache[event_num](event, rec) From patchwork Tue May 2 09:23:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775769 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=OUU36y49; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSM1743z1ydX for ; Tue, 2 May 2023 19:25:39 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmEq-00050B-4F; Tue, 02 May 2023 05:24:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEp-000502-1R for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:59 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEn-000403-I7 for qemu-devel@nongnu.org; Tue, 02 May 2023 05:23:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019436; bh=rtmjEodPVMUzwYYrkpmLQm9O2of3D9dbaQl2KB0rulo=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=OUU36y49TVqU4eSED4ZfCtTTMEdhfly2QVJN74o2YvkgMKAyo5EpWhI256b1hbnfC m+JzdR/ltonzPa2BDw/eY+0uhPuZPpSlcBSrcBxz+BlY1848CFunOuFweyek0V+Pk2 J2Jc9Xdn4bbCjGpJjmlcytw2el3J/8vLfwdqnppuBC0IYijzEWSXH26LwVegnKaeeY VyRY/iKKC8vifsNDOAFb1IZ1Di7kOBW3QWkzydxH+Nd+ELKsi6kMT4Gkv9VWsxgvnM Jg25kRpEUzvTMrhrGalpELqZvHxe0Ku0N2x8yyBQcNOVm5p7UAUgWcrgmfm+soWYMx +v0TZeB/KqBoA== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 768FA26338B2; Tue, 2 May 2023 09:23:54 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 04/12] simpletrace: update code for Python 3.11 Date: Tue, 2 May 2023 11:23:31 +0200 Message-Id: <20230502092339.27341-5-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-GUID: CSTnMekkgSNKL_y5iZtYNEOu3hNu8vnw X-Proofpoint-ORIG-GUID: CSTnMekkgSNKL_y5iZtYNEOu3hNu8vnw X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E425=2C18=2E0?= =?utf-8?q?=2E816=2C17=2E11=2E62=2E513=2E0000000_definitions=3D2022-01-18=5F?= =?utf-8?q?01=3A2022-01-14=5F01=2C2022-01-18=5F01=2C2021-12-02=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1030 mlxlogscore=724 bulkscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2305020080 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal The call to `getargspec` was deprecated and in Python 3.11 it has been removed in favor of `getfullargspec`. Signed-off-by: Mads Ynddal Reviewed-by: Stefan Hajnoczi --- scripts/simpletrace.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 9981699630..7444a6e090 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -192,7 +192,7 @@ def build_fn(analyzer, event): return analyzer.catchall event_argcount = len(event.args) - fn_argcount = len(inspect.getargspec(fn)[0]) - 1 + fn_argcount = len(inspect.getfullargspec(fn)[0]) - 1 if fn_argcount == event_argcount + 1: # Include timestamp as first argument return lambda _, rec: fn(*(rec[1:2] + rec[3:3 + event_argcount])) From patchwork Tue May 2 09:23:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775768 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=NXL1RNRv; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSK2PtRz20fj for ; Tue, 2 May 2023 19:25:37 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmEs-00051F-M9; Tue, 02 May 2023 05:24:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEr-00050q-DF for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:01 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmEp-00040B-Kc for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019438; bh=truZGcf/836/sOv1LjrGTCJdJmzGnXZDDsKkr3nCFe0=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=NXL1RNRvrxVYmIEtLSg8MnGCVLLXdGdIaEyfki4JKEZK0+CjATmwOd/8Ty+juEK6g hB7LOZZtAdQ4MjqYNALCgfmu5QlEjccLTj8Jd8ubqy9ny+3/pe9ep/q6+kiCPmo+7e wmuS/MTjT8eWKskg5+d1txZdoYR63Xh00V3PyhG0BONlqQ/3FQNaTQAzY8rdONzd7C zt0o0nFrJ323pjp5Qvbdo8hpPxsDffiEcmbGXG/AWIlOtPr5kESTXROoFzyPGLPhSv fFx7Il5MYHM4MKzxg+EZZk/YzZ1KPoBZLCUKdrVU3ZfFAgPqlz2bLLfCPTdmArRIPD IUKqFHsTcLnFg== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id DE1FB2633960; Tue, 2 May 2023 09:23:56 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 05/12] simpletrace: Changed Analyzer class to become context-manager Date: Tue, 2 May 2023 11:23:32 +0200 Message-Id: <20230502092339.27341-6-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 X-Proofpoint-GUID: NInfHy1XcRfCJWMxyUwC2BYIS5pibZQU X-Proofpoint-ORIG-GUID: NInfHy1XcRfCJWMxyUwC2BYIS5pibZQU X-Proofpoint-Virus-Version: =?utf-8?q?vendor=3Dfsecure_engine=3D1=2E1=2E170-?= =?utf-8?q?22c6f66c430a71ce266a39bfe25bc2903e8d5c8f=3A6=2E0=2E425=2C18=2E0?= =?utf-8?q?=2E816=2C17=2E11=2E62=2E513=2E0000000_definitions=3D2022-01-18=5F?= =?utf-8?q?01=3A2022-01-14=5F01=2C2022-01-18=5F01=2C2021-12-02=5F01_signatur?= =?utf-8?q?es=3D0?= X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 clxscore=1030 mlxlogscore=999 bulkscore=0 malwarescore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2305020080 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Instead of explicitly calling `begin` and `end`, we can change the class to use the context-manager paradigm. This is mostly a styling choice, used in modern Python code. But it also allows for more advanced analyzers to handle exceptions gracefully in the `__exit__` method (not demonstrated here). Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 40 +++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 7444a6e090..10ca093046 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -121,12 +121,12 @@ def read_trace_records(event_mapping, event_id_to_name, fobj): yield rec -class Analyzer(object): +class Analyzer: """A trace file analyzer which processes trace records. - An analyzer can be passed to run() or process(). The begin() method is - invoked, then each trace record is processed, and finally the end() method - is invoked. + An analyzer can be passed to run() or process(). The __enter__() method is + invoked when opening the analyzer using the `with` statement, then each trace + record is processed, and finally the __exit__() method is invoked. If a method matching a trace event name exists, it is invoked to process that trace record. Otherwise the catchall() method is invoked. @@ -152,19 +152,25 @@ def runstate_set(self, timestamp, pid, new_state): ... """ - def begin(self): + def __enter__(self): """Called at the start of the trace.""" - pass + return self def catchall(self, event, rec): """Called if no specific method for processing a trace event has been found.""" pass - def end(self): + def __exit__(self, _type, value, traceback): """Called at the end of the trace.""" pass -def process(events, log, analyzer, read_header=True): + def __call__(self): + """Fix for legacy use without context manager. + We call the provided object in `process` regardless of it being the object-type or instance. + With this function, it will work in both cases.""" + return self + +def process(events, log, analyzer_class, read_header=True): """Invoke an analyzer on each event in a log.""" if read_header: read_trace_header(log) @@ -203,15 +209,15 @@ def build_fn(analyzer, event): # Just arguments, no timestamp or pid return lambda _, rec: fn(*rec[3:3 + event_argcount]) - analyzer.begin() - fn_cache = {} - for rec in read_trace_records(event_mapping, event_id_to_name, log): - event_num = rec[0] - event = event_mapping[event_num] - if event_num not in fn_cache: - fn_cache[event_num] = build_fn(analyzer, event) - fn_cache[event_num](event, rec) - analyzer.end() + with analyzer_class() as analyzer: + fn_cache = {} + for rec in read_trace_records(event_mapping, event_id_to_name, log): + event_num = rec[0] + event = event_mapping[event_num] + if event_num not in fn_cache: + fn_cache[event_num] = build_fn(analyzer, event) + fn_cache[event_num](event, rec) + def run(analyzer): """Execute an analyzer on a trace file given on the command-line. From patchwork Tue May 2 09:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775772 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=Bti6yar0; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSs15f2z1ydX for ; Tue, 2 May 2023 19:26:05 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFO-00054C-Qp; Tue, 02 May 2023 05:24:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFN-000541-Ma for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:33 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFL-00043G-Ge for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019470; bh=H1YUNZrIgSCfpd1j0RXperPCMm9oMS4OPMj6tkEAW4k=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=Bti6yar0baBXEj5skF52jL7tfqBjHiajrAWx02apRaDO/IIn02jI5LP0Rih7WcIhl fbCXJt/eipgti7w5NLoCXCqPtbAfkXpjbJDlk1k3AkrM5DdR8xF8ow1Nu8WDM0se0p Qr3l6Pmv9rg9HN1z0Hm20v0U4dxgoOmEM1Q5aacbaceKxGobpVhRKCAQtYCUHH+L9u nDOd7dqVcb/ktHQOrPJeCCBEd4kyfhhkSCX7mSJ4zEjFwiuIZzs3u4U3NM+pEVm8H7 MeKC50u/fcRKzeU58A4NRyCwPYn9leAycvf5NocC3y4mYMW5EKMEHPRwAPbaoTQDqN nBbiFzCzHQU0w== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 022EF2633830; Tue, 2 May 2023 09:23:58 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 06/12] simpletrace: Simplify construction of tracing methods Date: Tue, 2 May 2023 11:23:33 +0200 Message-Id: <20230502092339.27341-7-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal By moving the dynamic argument construction to keyword-arguments, we can remove all of the specialized handling, and streamline it. If a tracing method wants to access these, they can define the kwargs, or ignore it be placing `**kwargs` at the end of the function's arguments list. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 84 ++++++++++++++++-------------------------- 1 file changed, 32 insertions(+), 52 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 10ca093046..f6b40d56f6 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -131,16 +131,25 @@ class Analyzer: If a method matching a trace event name exists, it is invoked to process that trace record. Otherwise the catchall() method is invoked. + The methods are called with a set of keyword-arguments. These can be ignored + using `**kwargs` or defined like any keyword-argument. + + The following keyword-arguments are available: + event: Event object of current trace + event_id: The id of the event in the current trace file + timestamp_ns: The timestamp in nanoseconds of the trace + pid: The process id recorded for the given trace + Example: The following method handles the runstate_set(int new_state) trace event:: - def runstate_set(self, new_state): + def runstate_set(self, new_state, **kwargs): ... - The method can also take a timestamp argument before the trace event - arguments:: + The method can also explicitly take a timestamp keyword-argument with the + trace event arguments:: - def runstate_set(self, timestamp, new_state): + def runstate_set(self, new_state, *, timestamp, **kwargs): ... Timestamps have the uint64_t type and are in nanoseconds. @@ -148,7 +157,7 @@ def runstate_set(self, timestamp, new_state): The pid can be included in addition to the timestamp and is useful when dealing with traces from multiple processes:: - def runstate_set(self, timestamp, pid, new_state): + def runstate_set(self, new_state, *, timestamp, pid, **kwargs): ... """ @@ -156,7 +165,7 @@ def __enter__(self): """Called at the start of the trace.""" return self - def catchall(self, event, rec): + def catchall(self, *rec_args, event, timestamp_ns, pid, event_id): """Called if no specific method for processing a trace event has been found.""" pass @@ -189,34 +198,11 @@ def process(events, log, analyzer_class, read_header=True): for event_id, event in enumerate(events): event_id_to_name[event_id] = event.name - def build_fn(analyzer, event): - if isinstance(event, str): - return analyzer.catchall - - fn = getattr(analyzer, event.name, None) - if fn is None: - return analyzer.catchall - - event_argcount = len(event.args) - fn_argcount = len(inspect.getfullargspec(fn)[0]) - 1 - if fn_argcount == event_argcount + 1: - # Include timestamp as first argument - return lambda _, rec: fn(*(rec[1:2] + rec[3:3 + event_argcount])) - elif fn_argcount == event_argcount + 2: - # Include timestamp and pid - return lambda _, rec: fn(*rec[1:3 + event_argcount]) - else: - # Just arguments, no timestamp or pid - return lambda _, rec: fn(*rec[3:3 + event_argcount]) - with analyzer_class() as analyzer: - fn_cache = {} - for rec in read_trace_records(event_mapping, event_id_to_name, log): - event_num = rec[0] - event = event_mapping[event_num] - if event_num not in fn_cache: - fn_cache[event_num] = build_fn(analyzer, event) - fn_cache[event_num](event, rec) + for event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(event_mapping, event_id_to_name, log): + event = event_mapping[event_id] + fn = getattr(analyzer, event.name, analyzer.catchall) + fn(*rec_args, event=event, event_id=event_id, timestamp_ns=timestamp_ns, pid=record_pid) def run(analyzer): @@ -240,24 +226,18 @@ def run(analyzer): if __name__ == '__main__': class Formatter(Analyzer): def __init__(self): - self.last_timestamp = None - - def catchall(self, event, rec): - timestamp = rec[1] - if self.last_timestamp is None: - self.last_timestamp = timestamp - delta_ns = timestamp - self.last_timestamp - self.last_timestamp = timestamp - - fields = [event.name, '%0.3f' % (delta_ns / 1000.0), - 'pid=%d' % rec[2]] - i = 3 - for type, name in event.args: - if is_string(type): - fields.append('%s=%s' % (name, rec[i])) - else: - fields.append('%s=0x%x' % (name, rec[i])) - i += 1 - print(' '.join(fields)) + self.last_timestamp_ns = None + + def catchall(self, *rec_args, event, timestamp_ns, pid, event_id): + if self.last_timestamp_ns is None: + self.last_timestamp_ns = timestamp_ns + delta_ns = timestamp_ns - self.last_timestamp_ns + self.last_timestamp_ns = timestamp_ns + + fields = [ + f'{name}={r}' if is_string(type) else f'{name}=0x{r:x}' + for r, (type, name) in zip(rec_args, event.args) + ] + print(f'{event.name} {delta_ns / 1000:0.3f} {pid=} ' + ' '.join(fields)) run(Formatter()) From patchwork Tue May 2 09:23:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775775 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=FTAl+tPK; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZTR0qrhz1ydX for ; Tue, 2 May 2023 19:26:35 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFR-00054v-BH; Tue, 02 May 2023 05:24:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFP-00054E-Kb for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:35 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFN-00043S-R0 for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019472; bh=sCL7u+UhJwxQZ2a/xWumax2lo1QYXCfoz8z/0fdBaZ8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=FTAl+tPK4P0ramwQFZwbusQwXhPoxk4grDje5/HI9eIRPihpNc9mFk68j4hWTjUHE Ho89FOv3ro06x1fOGQOfJeLHYGmt/Qb2+cYjgxXLKWhtYxokqhZnEgViKBiXaIQo9h Xsnzr6cSKiXG0Wa0e/FB+ll5dxTfIoxDiKypkP8N2/6yvLpNMTOxXmCvX5oawAYy/g bpQhuowQ4i5G8sn2dt4QEuOtcAR2DTmb0MSPWhPE2QUPm9Y3acIAA5h4GdK3BKXRvu y6r50bUK5gSvma/g36djKWVuLljSFXZHx/B2o74Vx9Jvgk7iT4OiBA0stlYLxodd2F 01jvbUGoiC7uw== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 7237E26337D1; Tue, 2 May 2023 09:24:31 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 07/12] simpletrace: Improved error handling on struct unpack Date: Tue, 2 May 2023 11:23:34 +0200 Message-Id: <20230502092339.27341-8-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal A failed call to `read_header` wouldn't be handled the same for the two different code paths (one path would try to use `None` as a list). Changed to raise exception to be handled centrally. This also allows for easier unpacking, as errors has been filtered out. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index f6b40d56f6..e77cb2469c 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -30,26 +30,24 @@ def read_header(fobj, hfmt): hlen = struct.calcsize(hfmt) hdr = fobj.read(hlen) if len(hdr) != hlen: - return None + raise ValueError('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) def get_record(event_mapping, event_id_to_name, rechdr, fobj): """Deserialize a trace record from a file into a tuple (name, timestamp, pid, arg1, ..., arg6).""" - if rechdr is None: - return None - if rechdr[0] != dropped_event_id: - event_id = rechdr[0] + event_id, timestamp_ns, length, pid = rechdr + if event_id != dropped_event_id: name = event_id_to_name[event_id] - rec = (name, rechdr[1], rechdr[3]) try: event = event_mapping[name] except KeyError as e: - sys.stderr.write('%s event is logged but is not declared ' \ + sys.stderr.write(f'{e} event is logged but is not declared ' \ 'in the trace events file, try using ' \ - 'trace-events-all instead.\n' % str(e)) + 'trace-events-all instead.\n') sys.exit(1) + rec = (name, timestamp_ns, pid) for type, name in event.args: if is_string(type): l = fobj.read(4) @@ -60,9 +58,8 @@ def get_record(event_mapping, event_id_to_name, rechdr, fobj): (value,) = struct.unpack('=Q', fobj.read(8)) rec = rec + (value,) else: - rec = ("dropped", rechdr[1], rechdr[3]) - (value,) = struct.unpack('=Q', fobj.read(8)) - rec = rec + (value,) + (dropped_count,) = struct.unpack('=Q', fobj.read(8)) + rec = ("dropped", timestamp_ns, pid, dropped_count) return rec def get_mapping(fobj): @@ -79,22 +76,16 @@ def read_record(event_mapping, event_id_to_name, fobj): def read_trace_header(fobj): """Read and verify trace file header""" - header = read_header(fobj, log_header_fmt) - if header is None: - raise ValueError('Not a valid trace file!') - if header[0] != header_event_id: - raise ValueError('Not a valid trace file, header id %d != %d' % - (header[0], header_event_id)) - if header[1] != header_magic: - raise ValueError('Not a valid trace file, header magic %d != %d' % - (header[1], header_magic)) - - log_version = header[2] + _header_event_id, _header_magic, log_version = read_header(fobj, log_header_fmt) + if _header_event_id != header_event_id: + raise ValueError(f'Not a valid trace file, header id {_header_event_id} != {header_event_id}') + if _header_magic != header_magic: + raise ValueError(f'Not a valid trace file, header magic {_header_magic} != {header_magic}') + if log_version not in [0, 2, 3, 4]: - raise ValueError('Unknown version of tracelog format!') + raise ValueError(f'Unknown version {log_version} of tracelog format!') if log_version != 4: - raise ValueError('Log format %d not supported with this QEMU release!' - % log_version) + raise ValueError(f'Log format {log_version} not supported with this QEMU release!') def read_trace_records(event_mapping, event_id_to_name, fobj): """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). From patchwork Tue May 2 09:23:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775767 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=fNRSw7YF; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSK1q6fz20fh for ; Tue, 2 May 2023 19:25:37 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFS-000556-PM; Tue, 02 May 2023 05:24:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFR-00054x-Q6 for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:37 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFQ-00043d-9I for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019475; bh=i6DKa/m8dKL3A4IVuxqhXIHlzicfl8iFYODS5hxoS4w=; h=From:To:Subject:Date:Message-Id:MIME-Version:Content-Type; b=fNRSw7YFqKtfERksteKgXJ8WzeQkvyRRLpb47LcCLyeiTynV6PwWeSBl+i0zV3HTy I/LMkPtwtDaKH02KUneMKmVofMl9+RvGaW5Is6M+6DgHG6rdsU0en3bR9x6TdqgNKw W1zEgzw4CdcAq5fJV+LggiurOct/ZptWTidbN1oOkDhP6zucDULI9vM/grfOv9hKUE lxqoQY/4oIcAM5Qi9vvay2HeFJDhGPy04i0fRtOpWLRTtcMBHf2uLB6dw3wq6oz3cS eQ6E8p5KKqNqm5mxTv1HyXvrFiVMt2GzF/uoUAUzIrV9CCc4Jk/a7+Fr3YFR6XTI80 1XgvZ69Qq7wYA== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 528E826339F0; Tue, 2 May 2023 09:24:33 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 08/12] simpletrace: define exception and add handling Date: Tue, 2 May 2023 11:23:35 +0200 Message-Id: <20230502092339.27341-9-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Define `SimpleException` to differentiate our exceptions from generic exceptions (IOError, etc.). Adapted simpletrace to support this and output to stderr. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index e77cb2469c..3cf34103e4 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -25,12 +25,15 @@ log_header_fmt = '=QQQ' # trace/simple.c::TraceLogHeader rec_header_fmt = '=QQII' # trace/simple.c::TraceRecord +class SimpleException(Exception): + pass + def read_header(fobj, hfmt): '''Read a trace record header''' hlen = struct.calcsize(hfmt) hdr = fobj.read(hlen) if len(hdr) != hlen: - raise ValueError('Error reading header. Wrong filetype provided?') + raise SimpleException('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) def get_record(event_mapping, event_id_to_name, rechdr, fobj): @@ -42,10 +45,10 @@ def get_record(event_mapping, event_id_to_name, rechdr, fobj): try: event = event_mapping[name] except KeyError as e: - sys.stderr.write(f'{e} event is logged but is not declared ' \ - 'in the trace events file, try using ' \ - 'trace-events-all instead.\n') - sys.exit(1) + raise SimpleException( + f'{e} event is logged but is not declared in the trace events' + 'file, try using trace-events-all instead.' + ) rec = (name, timestamp_ns, pid) for type, name in event.args: @@ -207,8 +210,7 @@ def run(analyzer): *no_header, trace_event_path, trace_file_path = sys.argv[1:] assert no_header == [] or no_header == ['--no-header'], 'Invalid no-header argument' except (AssertionError, ValueError): - sys.stderr.write(f'usage: {sys.argv[0]} [--no-header] \n') - sys.exit(1) + raise SimpleException(f'usage: {sys.argv[0]} [--no-header] \n') with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: events = read_events(events_fobj, trace_event_path) @@ -231,4 +233,8 @@ def catchall(self, *rec_args, event, timestamp_ns, pid, event_id): ] print(f'{event.name} {delta_ns / 1000:0.3f} {pid=} ' + ' '.join(fields)) - run(Formatter()) + try: + run(Formatter()) + except SimpleException as e: + sys.stderr.write(e + "\n") + sys.exit(1) From patchwork Tue May 2 09:23:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775770 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=hUxyt5M0; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSN2zL7z1ydX for ; Tue, 2 May 2023 19:25:40 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFX-00056R-A7; Tue, 02 May 2023 05:24:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFV-00055e-3z for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:41 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFT-00043s-4W for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019478; bh=7Ut4rJeVaGTYYjAuxVN37gMwYEe4oWe3wtuhlH37QOc=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=hUxyt5M0HQKvtsixz5zRPTjObToTV9TVMYMnMgPCGnm9Bl1/2zgmulKHMaaaHzQKf ZxKMFzsziAoKSLFlMhz1/Xn3BkezlZK/C6Ty0f8eVsq2eVfu5ZIvHwxtUJ7FFK0ik2 LcP2NVYZ0H2GgNrCpl2hezWySlBu9147eA5kmOPqzzixmRkdsAxadPYl//TbaFlEJH jM89QTk35Zv5ThDk4bLZTjQs9eTO05Qym33KQippQiaMYztO6R6h7WQrAsS9lJtPut oUMk3gIjaoNFrC1bOqRssY7rkxhLLQ4iyPmVWEujKN2UbwXLx5hEbyNevGqwnrGH/r EW842jKhi2IbA== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 5D7332633AE0; Tue, 2 May 2023 09:24:35 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 09/12] simpletrace: Refactor to separate responsibilities Date: Tue, 2 May 2023 11:23:36 +0200 Message-Id: <20230502092339.27341-10-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal NOTE: `process` changes function signature Moved event_mapping and event_id_to_name down one level in the function call-stack to keep variable instantiation and usage closer (`process` and `run` has no use of the variables; `read_trace_records` does). Instead of passing event_mapping and event_id_to_name to the bottom of the call-stack, we move their use to `read_trace_records`. This separates responsibility and ownership of the information. `read_record` now just reads the arguments from the file-object by knowning the total number of bytes. Parsing it to specific arguments is moved up to `read_trace_records`. Special handling of dropped events removed, as they can be handled by the general code. Signed-off-by: Mads Ynddal --- scripts/simpletrace.py | 130 ++++++++++++++++++++--------------------- 1 file changed, 63 insertions(+), 67 deletions(-) diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py index 3cf34103e4..2fcbcb77d0 100755 --- a/scripts/simpletrace.py +++ b/scripts/simpletrace.py @@ -24,6 +24,7 @@ log_header_fmt = '=QQQ' # trace/simple.c::TraceLogHeader rec_header_fmt = '=QQII' # trace/simple.c::TraceRecord +rec_header_fmt_len = struct.calcsize(rec_header_fmt) class SimpleException(Exception): pass @@ -36,35 +37,6 @@ def read_header(fobj, hfmt): raise SimpleException('Error reading header. Wrong filetype provided?') return struct.unpack(hfmt, hdr) -def get_record(event_mapping, event_id_to_name, rechdr, fobj): - """Deserialize a trace record from a file into a tuple - (name, timestamp, pid, arg1, ..., arg6).""" - event_id, timestamp_ns, length, pid = rechdr - if event_id != dropped_event_id: - name = event_id_to_name[event_id] - try: - event = event_mapping[name] - except KeyError as e: - raise SimpleException( - f'{e} event is logged but is not declared in the trace events' - 'file, try using trace-events-all instead.' - ) - - rec = (name, timestamp_ns, pid) - for type, name in event.args: - if is_string(type): - l = fobj.read(4) - (len,) = struct.unpack('=L', l) - s = fobj.read(len) - rec = rec + (s,) - else: - (value,) = struct.unpack('=Q', fobj.read(8)) - rec = rec + (value,) - else: - (dropped_count,) = struct.unpack('=Q', fobj.read(8)) - rec = ("dropped", timestamp_ns, pid, dropped_count) - return rec - def get_mapping(fobj): (event_id, ) = struct.unpack('=Q', fobj.read(8)) (len, ) = struct.unpack('=L', fobj.read(4)) @@ -72,10 +44,11 @@ def get_mapping(fobj): return (event_id, name) -def read_record(event_mapping, event_id_to_name, fobj): - """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, arg1, ..., arg6).""" - rechdr = read_header(fobj, rec_header_fmt) - return get_record(event_mapping, event_id_to_name, rechdr, fobj) +def read_record(fobj): + """Deserialize a trace record from a file into a tuple (event_num, timestamp, pid, args).""" + event_id, timestamp_ns, record_length, record_pid = read_header(fobj, rec_header_fmt) + args_payload = fobj.read(record_length - rec_header_fmt_len) + return (event_id, timestamp_ns, record_pid, args_payload) def read_trace_header(fobj): """Read and verify trace file header""" @@ -90,30 +63,60 @@ def read_trace_header(fobj): if log_version != 4: raise ValueError(f'Log format {log_version} not supported with this QEMU release!') -def read_trace_records(event_mapping, event_id_to_name, fobj): - """Deserialize trace records from a file, yielding record tuples (event_num, timestamp, pid, arg1, ..., arg6). - - Note that `event_id_to_name` is modified if the file contains mapping records. +def read_trace_records(events, fobj, read_header): + """Deserialize trace records from a file, yielding record tuples (event, event_num, timestamp, pid, arg1, ..., arg6). Args: event_mapping (str -> Event): events dict, indexed by name - event_id_to_name (int -> str): event names dict, indexed by event ID fobj (file): input file + read_header (bool): whether headers were read from fobj """ - while True: - t = fobj.read(8) - if len(t) == 0: - break + frameinfo = inspect.getframeinfo(inspect.currentframe()) + dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", + frameinfo.lineno + 1, frameinfo.filename) + + event_mapping = {e.name: e for e in events} + event_mapping["dropped"] = dropped_event + event_id_to_name = {dropped_event_id: "dropped"} + # If there is no header assume event ID mapping matches events list + if not read_header: + for event_id, event in enumerate(events): + event_id_to_name[event_id] = event.name + + while t := fobj.read(8): (rectype, ) = struct.unpack('=Q', t) if rectype == record_type_mapping: - event_id, name = get_mapping(fobj) - event_id_to_name[event_id] = name + event_id, event_name = get_mapping(fobj) + event_id_to_name[event_id] = event_name else: - rec = read_record(event_mapping, event_id_to_name, fobj) - - yield rec + event_id, timestamp_ns, pid, args_payload = read_record(fobj) + event_name = event_id_to_name[event_id] + + try: + event = event_mapping[event_name] + except KeyError as e: + raise SimpleException( + f'{e} event is logged but is not declared in the trace events' + 'file, try using trace-events-all instead.' + ) + + offset = 0 + args = [] + for type, _ in event.args: + if is_string(type): + (len,) = struct.unpack_from('=L', args_payload, offset=offset) + offset += 4 + s = args_payload[offset:offset+len] + offset += len + args.append(s) + else: + (value,) = struct.unpack_from('=Q', args_payload, offset=offset) + offset += 8 + args.append(value) + + yield (event_mapping[event_name], event_name, timestamp_ns, pid) + tuple(args) class Analyzer: """A trace file analyzer which processes trace records. @@ -173,28 +176,22 @@ def __call__(self): With this function, it will work in both cases.""" return self -def process(events, log, analyzer_class, read_header=True): - """Invoke an analyzer on each event in a log.""" - if read_header: - read_trace_header(log) +def process(events_fobj, log_fobj, analyzer_class, read_header=True): + """Invoke an analyzer on each event in a log. - frameinfo = inspect.getframeinfo(inspect.currentframe()) - dropped_event = Event.build("Dropped_Event(uint64_t num_events_dropped)", - frameinfo.lineno + 1, frameinfo.filename) - event_mapping = {"dropped": dropped_event} - event_id_to_name = {dropped_event_id: "dropped"} - - for event in events: - event_mapping[event.name] = event + Args: + events_fobj (file): file-object to read event data from + log_fobj (file): file-object to read log data from + analyzer_class (Analyzer): the Analyzer to interpret the event data + read_header (bool, optional): Whether to read header data from the log data. Defaults to True. + """ + if read_header: + read_trace_header(log_fobj) - # If there is no header assume event ID mapping matches events list - if not read_header: - for event_id, event in enumerate(events): - event_id_to_name[event_id] = event.name + events = read_events(events_fobj, events_fobj.name) with analyzer_class() as analyzer: - for event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(event_mapping, event_id_to_name, log): - event = event_mapping[event_id] + for event, event_id, timestamp_ns, record_pid, *rec_args in read_trace_records(events, log_fobj, read_header): fn = getattr(analyzer, event.name, analyzer.catchall) fn(*rec_args, event=event, event_id=event_id, timestamp_ns=timestamp_ns, pid=record_pid) @@ -213,8 +210,7 @@ def run(analyzer): raise SimpleException(f'usage: {sys.argv[0]} [--no-header] \n') with open(trace_event_path, 'r') as events_fobj, open(trace_file_path, 'rb') as log_fobj: - events = read_events(events_fobj, trace_event_path) - process(events, log_fobj, analyzer, read_header=not no_header) + process(events_fobj, log_fobj, analyzer, read_header=not no_header) if __name__ == '__main__': class Formatter(Analyzer): From patchwork Tue May 2 09:23:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775774 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=FTfc3tK3; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZTF5L7pz20fh for ; Tue, 2 May 2023 19:26:25 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFZ-0005Jf-JE; Tue, 02 May 2023 05:24:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFX-00056k-9Q for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:43 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFV-00044D-M2 for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019480; bh=geucgPRYPav+0JfmPTY3isc1cSilmnjEYSHJMnbwUxk=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=FTfc3tK3M3Hjp+6nb4CUNXC7i7jKZpuZDETtLjg7JuWFa6dON5NCJ+e3oYBBo76ev Uoit0YVUzWbuJlgdlK9RFVn06LbFXHlIp1P0+t/MWICryN496Mdp8mv0pigOpO18sC EbQO6J1cg0nHBk5dMbjJtXyNesL2yIsMGUbrrUvyWkUW84zLUuYdTXupXvoGzEiAi6 RbPeommmStm21Y5ApXK++xQRUdi5VLSRTGqPEmBtm5G0o1U3JfaJprXsCu7CJLBGgW bmBu3O/JJf1EK/ekPxk8rY6esEaTZg8JPn8VXKEOBSap2znnI4l0IkBkhOHd8RqZC4 sZ+emO1o1uMww== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 9C8AC2633A68; Tue, 2 May 2023 09:24:38 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 10/12] MAINTAINERS: add maintainer of simpletrace.py Date: Tue, 2 May 2023 11:23:37 +0200 Message-Id: <20230502092339.27341-11-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal In my work to refactor simpletrace.py, I noticed that there's no maintainer of it, and has the status of "odd fixes". I'm using it from time to time, so I'd like to maintain the script. I've added myself as reviewer under "Tracing" to be informed of changes that might affect simpletrace.py. Signed-off-by: Mads Ynddal --- MAINTAINERS | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/MAINTAINERS b/MAINTAINERS index 5340de0515..19edec5dda 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3035,7 +3035,7 @@ R: Paolo Bonzini R: Bandan Das R: Stefan Hajnoczi R: Thomas Huth -R: Darren Kenny +R: Darren Kenny R: Qiuhao Li S: Maintained F: tests/qtest/fuzz/ @@ -3076,6 +3076,7 @@ F: stubs/ Tracing M: Stefan Hajnoczi +R: Mads Ynddal S: Maintained F: trace/ F: trace-events @@ -3088,6 +3089,11 @@ F: docs/tools/qemu-trace-stap.rst F: docs/devel/tracing.rst T: git https://github.com/stefanha/qemu.git tracing +Simpletrace +M: Mads Ynddal +S: Maintained +F: scripts/simpletrace.py + TPM M: Stefan Berger S: Maintained From patchwork Tue May 2 09:23:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775771 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=ey5TFeHA; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZSj2SrSz1ydX for ; Tue, 2 May 2023 19:25:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFb-0005OR-2F; Tue, 02 May 2023 05:24:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFZ-0005Iq-5J for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:45 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFX-00044U-IK for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019482; bh=0TM9Xz67OYngDEuTQ62kPsInUQpx3kaf2lzF824GJ1Y=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=ey5TFeHAWdB3RBwJDadJ62+sNvJ8ckchSdq5ZyRQNITYg34lnaNrMn5qa+CDFMt9M JbT3TVZXtL85kb77USZR0FEsvk3/07alzkFRKvOlXePNY3OTzeNOYswg84efzqdErc uEx+5gCNQIMG+bsWnDJX6OxCCuVL/aB/u3FyqEMKXWPN7Zm0TzGmUR9heE5UsNffVL z9kCA/fQ38eVcD9uD2SLWq1Xxt181EpWMLpNbqqowEpMvzYYikcKTtKsgQrhWoXMjv ep00v1mvmNOSJ3ZQu+wY2yKhmepnq90hBn1JuI0aWa6H2p58CZt7HgLe3L7xWzRxti giOv8FqV/XzkA== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 128CE2633AF2; Tue, 2 May 2023 09:24:40 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 11/12] scripts/analyse-locks-simpletrace.py: changed iteritems() to items() Date: Tue, 2 May 2023 11:23:38 +0200 Message-Id: <20230502092339.27341-12-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal Python 3 removed `dict.iteritems()` in favor of `dict.items()`. This means the script curerntly doesn't work on Python 3. Signed-off-by: Mads Ynddal --- scripts/analyse-locks-simpletrace.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py index 63c11f4fce..d650dd7140 100755 --- a/scripts/analyse-locks-simpletrace.py +++ b/scripts/analyse-locks-simpletrace.py @@ -75,7 +75,7 @@ def get_args(): (analyser.locks, analyser.locked, analyser.unlocks)) # Now dump the individual lock stats - for key, val in sorted(analyser.mutex_records.iteritems(), + for key, val in sorted(analyser.mutex_records.items(), key=lambda k_v: k_v[1]["locks"]): print ("Lock: %#x locks: %d, locked: %d, unlocked: %d" % (key, val["locks"], val["locked"], val["unlocked"])) From patchwork Tue May 2 09:23:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mads Ynddal X-Patchwork-Id: 1775765 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ynddal.dk header.i=@ynddal.dk header.a=rsa-sha256 header.s=sig1 header.b=I4VW+Wc6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q9ZRf23R5z1ymp for ; Tue, 2 May 2023 19:25:02 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ptmFd-0005Qj-T4; Tue, 02 May 2023 05:24:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFb-0005PV-8Z for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:47 -0400 Received: from mr85p00im-ztdg06021701.me.com ([17.58.23.196]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ptmFZ-000462-OG for qemu-devel@nongnu.org; Tue, 02 May 2023 05:24:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ynddal.dk; s=sig1; t=1683019484; bh=cL7THDsuZqmjFuNeLwSxxRPGd6LzrqdPHNp18DKJuT0=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=I4VW+Wc6uKDEBZCUFGDKfDhQ6u3tEJQDk4R6tGVOX/G/qMbu8whxlNxHNdWUGKWE2 XXd7IZSQH3zEVcX05cVlqc8wEMMErIOk4qHYQkWn0EL8hNSPf9FUTLH/a710t7BE5h 5ysRto+L5LiwH6jvj1WJ2YULrYy4Vg6F8/Q2KxIz3T9NjYkX20FK/m6ovJax0Bt7C8 yAd95s7dPhrCQ/ipq1sSmt/697fuaVaXNJLnQDcFmFrsZcz6CNo1MHnzlCsmwcKu2x zi4qTMaYrQFrpvBZqChjgsSlT4GdmajBF5d11Bwc5SnVM07rh6hAt5Yc5OLtq2L/ig aJ0Se0b7cZrxg== Received: from localhost.localdomain (mr38p00im-dlb-asmtp-mailmevip.me.com [17.57.152.18]) by mr85p00im-ztdg06021701.me.com (Postfix) with ESMTPSA id 2BB482633AA1; Tue, 2 May 2023 09:24:43 +0000 (UTC) From: Mads Ynddal To: qemu-devel@nongnu.org Cc: Cleber Rosa , Stefan Hajnoczi , John Snow , Mads Ynddal , Mads Ynddal Subject: [PATCH v2 12/12] scripts/analyse-locks-simpletrace.py: reflect changes to process in simpletrace.py Date: Tue, 2 May 2023 11:23:39 +0200 Message-Id: <20230502092339.27341-13-mads@ynddal.dk> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20230502092339.27341-1-mads@ynddal.dk> References: <20230502092339.27341-1-mads@ynddal.dk> MIME-Version: 1.0 Received-SPF: pass client-ip=17.58.23.196; envelope-from=mads@ynddal.dk; helo=mr85p00im-ztdg06021701.me.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Mads Ynddal The signature of `process` in simpletrace.py has changed to not take filepaths as the two first arguments, but rather a file-like object. We change the code here to reflect that. Signed-off-by: Mads Ynddal --- scripts/analyse-locks-simpletrace.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py index d650dd7140..235fae2ba2 100755 --- a/scripts/analyse-locks-simpletrace.py +++ b/scripts/analyse-locks-simpletrace.py @@ -69,7 +69,8 @@ def get_args(): # Gather data from the trace analyser = MutexAnalyser() - simpletrace.process(args.events, args.tracefile, analyser) + with open(args.events, 'r') as events, open(args.tracefile, 'rb') as tracefile: + simpletrace.process(events, tracefile, analyser) print ("Total locks: %d, locked: %d, unlocked: %d" % (analyser.locks, analyser.locked, analyser.unlocks))