From patchwork Thu Jun 4 20:22:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303729 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=C7uDJuE3; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHQb6WdFz9sT8 for ; Fri, 5 Jun 2020 06:27:51 +1000 (AEST) Received: from localhost ([::1]:40162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwSr-0004g9-Hm for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:27:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49586) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwON-00055Q-7w for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:23:11 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:30767 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwOM-0005gD-Fj for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:23:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AffFvAJ982E8rMg3USDtpzSTkHD0X9KrviRMhx0uy5A=; b=C7uDJuE3fnLVFZJmUc4ylcBUREqwbcxl2vwDmIo1TiaWwom0li1d5eIoW/DXXw9Otmfw7b C2Ox6Bi9HNjel9xVJxGKM2DNe5YM1nk5nx7sa1TG02WkKRCteTn01eNe6VzO/g06dYUz4y w5edO/TPCcofUgfYbOb0I/DanJ0QiQY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-161-atLO5bGLPSilopgRjUHX3Q-1; Thu, 04 Jun 2020 16:22:39 -0400 X-MC-Unique: atLO5bGLPSilopgRjUHX3Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 969AB81F0A0; Thu, 4 Jun 2020 20:22:38 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id B7C5F5D9D3; Thu, 4 Jun 2020 20:22:37 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 01/16] python/qmp.py: Define common types Date: Thu, 4 Jun 2020 16:22:21 -0400 Message-Id: <20200604202236.25039-2-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 13:58:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Define some common types that we'll need to annotate a lot of other functions going forward. Signed-off-by: John Snow --- python/qemu/qmp.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index e64b6b5faa7..8388c7b6030 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -12,13 +12,31 @@ import socket import logging from typing import ( + Any, + Dict, Optional, TextIO, Type, + Tuple, + Union, ) from types import TracebackType +# QMPMessage is a QMP Message of any kind. +# e.g. {'yee': 'haw'} +# +# QMPReturnValue is the inner value of return values only. +# {'return': {}} is the QMPMessage, +# {} is the QMPReturnValue. +QMPMessage = Dict[str, Any] +QMPReturnValue = Dict[str, Any] + +InternetAddrT = Tuple[str, str] +UnixAddrT = str +SocketAddrT = Union[InternetAddrT, UnixAddrT] + + class QMPError(Exception): """ QMP base exception From patchwork Thu Jun 4 20:22:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303730 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=LDObFp+T; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHSN71mbz9sSd for ; Fri, 5 Jun 2020 06:29:24 +1000 (AEST) Received: from localhost ([::1]:46906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwUM-0007Wk-Ll for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:29:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49446) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO0-000437-8s for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:48 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:54897 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-0005VI-E9 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1ZHtmt1m1ZkVFBw/uJkXhMkYPR+P+eLoSIG2usa9Dcg=; b=LDObFp+TdC+WxtGmJTtDdPNZgscFX2p6/uZm5MKy+/ArOUlu83vj8h/mpI1u4tX33BCxpT zKc0fXDV6sux75BukR0qcFJuKiufRNqfGxmNu1fjUb8HuZRoEIYI/whzmRa/C0whOhYQnF N9hpQhcqb6OciulcGTd1M/3uw5qT9D8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-445-Hnce5f6-OoqszN2XEaJUIA-1; Thu, 04 Jun 2020 16:22:40 -0400 X-MC-Unique: Hnce5f6-OoqszN2XEaJUIA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 94C69835B41; Thu, 4 Jun 2020 20:22:39 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAC1C5D9D3; Thu, 4 Jun 2020 20:22:38 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 02/16] iotests.py: use qemu.qmp type aliases Date: Thu, 4 Jun 2020 16:22:22 -0400 Message-Id: <20200604202236.25039-3-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 13:58:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" iotests.py should use the type definitions from qmp.py instead of its own. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index f20d90f9698..7c1773bba37 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -35,13 +35,10 @@ # pylint: disable=import-error, wrong-import-position sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qtest +from qemu.qmp import QMPMessage assert sys.version_info >= (3, 6) -# Type Aliases -QMPResponse = Dict[str, Any] - - # Use this logger for logging messages directly from the iotests module logger = logging.getLogger('qemu.iotests') logger.addHandler(logging.NullHandler()) @@ -554,7 +551,7 @@ def add_incoming(self, addr): self._args.append(addr) return self - def hmp(self, command_line: str, use_log: bool = False) -> QMPResponse: + def hmp(self, command_line: str, use_log: bool = False) -> QMPMessage: cmd = 'human-monitor-command' kwargs = {'command-line': command_line} if use_log: @@ -575,7 +572,7 @@ def resume_drive(self, drive: str) -> None: self.hmp(f'qemu-io {drive} "remove_break bp_{drive}"') def hmp_qemu_io(self, drive: str, cmd: str, - use_log: bool = False) -> QMPResponse: + use_log: bool = False) -> QMPMessage: """Write to a given drive using an HMP command""" return self.hmp(f'qemu-io {drive} "{cmd}"', use_log=use_log) From patchwork Thu Jun 4 20:22:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303727 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eU/EERKZ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHPr5t76z9sSd for ; Fri, 5 Jun 2020 06:27:12 +1000 (AEST) Received: from localhost ([::1]:37684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwSE-0003fo-GD for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:27:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49430) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-0003zj-3L for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:49926 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNx-0005Ug-Rv for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302165; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sZl/vaoLFiELtwTjN3/2xmM+dnn6CVXPx8SRSXh/bks=; b=eU/EERKZFHz6/BGv3Mwxmeq9F72L7TzcZnpBxpOtce9BQKwRlGLwwgwcixSHmZMEJN3vmT k28P51HzWdSHvgq1QpoopIfh/2SRy6YYewZVi1yvjs9xLLhe/nZZn4Pn+vlkan0NfAxnVV PulnmV1/54iDl8HS8SjuspNnQ7D/Exc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-wYe5ZUjdMsi72rzAtx_F4Q-1; Thu, 04 Jun 2020 16:22:41 -0400 X-MC-Unique: wYe5ZUjdMsi72rzAtx_F4Q-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BDDDB19200C1; Thu, 4 Jun 2020 20:22:40 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC3855D9D3; Thu, 4 Jun 2020 20:22:39 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 03/16] python/qmp.py: re-absorb MonitorResponseError Date: Thu, 4 Jun 2020 16:22:23 -0400 Message-Id: <20200604202236.25039-4-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:08:38 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When I initially split this out, I considered this more of a machine error than a QMP protocol error, but I think that's misguided. Move this back to qmp.py and name it QMPResponseError. Convert qmp.command() to use this exception type. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 15 +-------------- python/qemu/qmp.py | 17 +++++++++++++++-- scripts/render_block_graph.py | 7 +++++-- 3 files changed, 21 insertions(+), 18 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 99bcb499878..8ea027566e4 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -49,19 +49,6 @@ class QEMUMachineAddDeviceError(QEMUMachineError): """ -class MonitorResponseError(qmp.QMPError): - """ - Represents erroneous QMP monitor reply - """ - def __init__(self, reply): - try: - desc = reply["error"]["desc"] - except KeyError: - desc = reply - super().__init__(desc) - self.reply = reply - - class QEMUMachine: """ A QEMU VM @@ -469,7 +456,7 @@ def command(self, cmd, conv_keys=True, **args): if reply is None: raise qmp.QMPError("Monitor is closed") if "error" in reply: - raise MonitorResponseError(reply) + raise qmp.QMPResponseError(reply) return reply["return"] def get_qmp_event(self, wait=False): diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 8388c7b6030..aa8a666b8ab 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -61,6 +61,19 @@ class QMPTimeoutError(QMPError): """ +class QMPResponseError(QMPError): + """ + Represents erroneous QMP monitor reply + """ + def __init__(self, reply: QMPMessage): + try: + desc = reply['error']['desc'] + except KeyError: + desc = reply + super().__init__(desc) + self.reply = reply + + class QEMUMonitorProtocol: """ Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then @@ -251,8 +264,8 @@ def command(self, cmd, **kwds): Build and send a QMP command to the monitor, report errors if any """ ret = self.cmd(cmd, kwds) - if "error" in ret: - raise Exception(ret['error']['desc']) + if 'error' in ret: + raise QMPResponseError(ret) return ret['return'] def pull_event(self, wait=False): diff --git a/scripts/render_block_graph.py b/scripts/render_block_graph.py index 409b4321f2e..da6acf050d1 100755 --- a/scripts/render_block_graph.py +++ b/scripts/render_block_graph.py @@ -25,7 +25,10 @@ from graphviz import Digraph sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'python')) -from qemu.machine import MonitorResponseError +from qemu.qmp import ( + QEMUMonitorProtocol, + QMPResponseError, +) def perm(arr): @@ -102,7 +105,7 @@ def command(self, cmd): reply = json.loads(subprocess.check_output(ar)) if 'error' in reply: - raise MonitorResponseError(reply) + raise QMPResponseError(reply) return reply['return'] From patchwork Thu Jun 4 20:22:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303724 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Nmx2bxa9; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHMT6z9nz9sSd for ; Fri, 5 Jun 2020 06:25:09 +1000 (AEST) Received: from localhost ([::1]:56718 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwQF-0008Dy-EY for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:25:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwNy-0003xQ-20 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:46 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:23057 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNx-0005UP-7g for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302164; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V6vPM6AYq3oZLyd83BJvWKOnCz6g7DCIxzAnkyqnH8c=; b=Nmx2bxa9lCwgsordRA1fiJ/0h7cg3Lh9IcpCfWtfEuarzpS7K7Ktbl93RYT0Nq3NtmIc+a 4e6A9PcCzHRStExHi+J+D9hJEgjFnqx4up8QP0JBTqDkbjzyJ9J7+HupSBiO+vEv6t9RPT ZrlkMgUXtgGZDMPUAQ9Ay3jWHqw0Mpc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-1Efehg0oOQy6hQh09yPcGQ-1; Thu, 04 Jun 2020 16:22:42 -0400 X-MC-Unique: 1Efehg0oOQy6hQh09yPcGQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B6E05835B40; Thu, 4 Jun 2020 20:22:41 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id E1CB75D9D3; Thu, 4 Jun 2020 20:22:40 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 04/16] python/qmp.py: Do not return None from cmd_obj Date: Thu, 4 Jun 2020 16:22:24 -0400 Message-Id: <20200604202236.25039-5-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 12:22:43 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This makes typing the qmp library difficult, as it necessitates wrapping Optional[] around the type for every return type up the stack. At some point, it becomes difficult to discern or remember why it's None instead of the expected object. Use the python exception system to tell us exactly why we didn't get an object. Remove this special-cased return. Signed-off-by: John Snow --- python/qemu/qmp.py | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index aa8a666b8ab..ef3c919b76c 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -225,22 +225,18 @@ def accept(self, timeout=15.0): self.__sockfile = self.__sock.makefile(mode='r') return self.__negotiate_capabilities() - def cmd_obj(self, qmp_cmd): + def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: """ Send a QMP command to the QMP Monitor. @param qmp_cmd: QMP command to be sent as a Python dict - @return QMP response as a Python dict or None if the connection has - been closed + @return QMP response as a Python dict """ self.logger.debug(">>> %s", qmp_cmd) - try: - self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) - except OSError as err: - if err.errno == errno.EPIPE: - return None - raise err + self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8')) resp = self.__json_read() + if resp is None: + raise QMPConnectError("Unexpected empty reply from server") self.logger.debug("<<< %s", resp) return resp From patchwork Thu Jun 4 20:22:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303720 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gqYdZZkq; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHKS076Pz9sSf for ; Fri, 5 Jun 2020 06:23:24 +1000 (AEST) Received: from localhost ([::1]:49036 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwOX-0004Ny-Mb for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:23:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49436) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-00040m-DJ for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:51978 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNy-0005Un-Ml for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302165; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4fk+MelxzBtEUkDkwDpmLT6Ud7EYaoQG87DV6PqOep4=; b=gqYdZZkqNkwnM+zp5GuDug9Kxfqv6CHKog2Rb+ba6BBf+Gsy4lqL1fmO+lXM4ehvb9DGc9 i8ny4ifXBdzNA/KaJ9FMrm9FE8pUMY1pqWDNUSkZ3HhWjDwS9IyxOef3UE0PhwD9tTUr7I Iu/BLPkpEv3FA7wrxYlGKaOmImSrWrA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-393-C7KrUWulMtatFYRnzwd7CQ-1; Thu, 04 Jun 2020 16:22:43 -0400 X-MC-Unique: C7KrUWulMtatFYRnzwd7CQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B360519200C1; Thu, 4 Jun 2020 20:22:42 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC5455D9D3; Thu, 4 Jun 2020 20:22:41 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 05/16] python/qmp.py: add casts to JSON deserialization Date: Thu, 4 Jun 2020 16:22:25 -0400 Message-Id: <20200604202236.25039-6-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" mypy and python type hints are not powerful enough to properly describe JSON messages in Python 3.6. The best we can do, generally, is describe them as Dict[str, Any]. Add casts to coerce this type for static analysis; but do NOT enforce this type at runtime in any way. Note: Python 3.8 adds a TypedDict construct which allows for the description of more arbitrary Dictionary shapes. There is a third-party module, "Pydantic", which is compatible with 3.6 that can be used instead of the JSON library that parses JSON messages to fully-typed Python objects, and may be preferable in some cases. (That is well beyond the scope of this commit or series.) Signed-off-by: John Snow --- python/qemu/qmp.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index ef3c919b76c..1ae36050a42 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -13,6 +13,7 @@ import logging from typing import ( Any, + cast, Dict, Optional, TextIO, @@ -130,7 +131,10 @@ def __json_read(self, only_event=False): data = self.__sockfile.readline() if not data: return None - resp = json.loads(data) + # By definition, any JSON received from QMP is a QMPMessage, + # and we are asserting only at static analysis time that it + # has a particular shape. + resp: QMPMessage = json.loads(data) if 'event' in resp: self.logger.debug("<<< %s", resp) self.__events.append(resp) @@ -262,7 +266,7 @@ def command(self, cmd, **kwds): ret = self.cmd(cmd, kwds) if 'error' in ret: raise QMPResponseError(ret) - return ret['return'] + return cast(QMPReturnValue, ret['return']) def pull_event(self, wait=False): """ From patchwork Thu Jun 4 20:22:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303725 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Cmns5x+Z; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHMl3xmvz9sSd for ; Fri, 5 Jun 2020 06:25:23 +1000 (AEST) Received: from localhost ([::1]:57502 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwQT-00005Y-9L for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:25:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49444) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-00042P-SK for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:56501 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-0005Uz-3H for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=550l0y6cMuVFZ9yDHqjsncXPIXXkomnBdKzUAnASITk=; b=Cmns5x+ZNm5rncOKgjQajHChaR4rjzXHCdk0ry1EBQiuizIqpfmmEtbiuvx0gvr/uc9opY SmSzGR80R+efhzaunmOv1Au9s8+aarIPI4Cob1tPYKMhqdovP1n+NHYRLs3PT/zYDqlBHy HW+VAM/FcZxy0mdoQfb8YM5v3v68pHI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-440-0lAQmdoJOAWEyA_irvyV4g-1; Thu, 04 Jun 2020 16:22:44 -0400 X-MC-Unique: 0lAQmdoJOAWEyA_irvyV4g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AE061464; Thu, 4 Jun 2020 20:22:43 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id DA0445D9D3; Thu, 4 Jun 2020 20:22:42 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 06/16] python/qmp.py: add QMPProtocolError Date: Thu, 4 Jun 2020 16:22:26 -0400 Message-Id: <20200604202236.25039-7-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 12:22:43 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" In the case that we receive a reply but are unable to understand it, use this exception name to indicate that case. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé --- python/qemu/qmp.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 1ae36050a42..7935dababbf 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -62,6 +62,12 @@ class QMPTimeoutError(QMPError): """ +class QMPProtocolError(QMPError): + """ + QMP protocol error; unexpected response + """ + + class QMPResponseError(QMPError): """ Represents erroneous QMP monitor reply @@ -266,6 +272,10 @@ def command(self, cmd, **kwds): ret = self.cmd(cmd, kwds) if 'error' in ret: raise QMPResponseError(ret) + if 'return' not in ret: + raise QMPProtocolError( + "'return' key not found in QMP response '{}'".format(str(ret)) + ) return cast(QMPReturnValue, ret['return']) def pull_event(self, wait=False): From patchwork Thu Jun 4 20:22:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303723 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FRxz03Qy; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHM147G1z9sSd for ; Fri, 5 Jun 2020 06:24:45 +1000 (AEST) Received: from localhost ([::1]:54388 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwPr-0006ze-67 for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:24:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49450) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO0-00044Q-RG for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:48 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:37327 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwNz-0005VQ-MH for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302167; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H+DGaqyA97Ux+PtYIEWCCuskpSnBbmRVkLHvH5mYhjA=; b=FRxz03QyG6ocCQxH0GRwx9V5DHEKoTC3KEWYGNm8NhXKCMqZrhvY/c9ARDDn9foeltfP1c fSwH/S732MiIZ/hB0NuIzcF0Dz+Gw1fmgiUeH6Ca6382JTZwIpzTvHLuuyxOCRBt2pnOn4 vcLdQgVS0wwOPoLN+XJH9k1PcrqR0KM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-119-JE7zdOWGPrmGdaHB4Y3eFA-1; Thu, 04 Jun 2020 16:22:45 -0400 X-MC-Unique: JE7zdOWGPrmGdaHB4Y3eFA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A6151107ACF8; Thu, 4 Jun 2020 20:22:44 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id D09ED5D9D3; Thu, 4 Jun 2020 20:22:43 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 07/16] python/machine.py: Fix monitor address typing Date: Thu, 4 Jun 2020 16:22:27 -0400 Message-Id: <20200604202236.25039-8-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Prior to this, it's difficult for mypy to intuit what the concrete type of the monitor address is; it has difficulty inferring the type across two variables. Create _monitor_address as a property that always returns a valid address to simply static type analysis. To preserve our ability to clean up, use a simple boolean to indicate whether or not we should try to clean up the sock file after execution. Signed-off-by: John Snow --- python/qemu/machine.py | 45 +++++++++++++++++++++++++++--------------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 8ea027566e4..34713ba9b15 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -25,10 +25,14 @@ import signal import socket import tempfile -from typing import Optional, Type +from typing import ( + Optional, + Type, +) from types import TracebackType from . import qmp +from .qmp import SocketAddrT LOG = logging.getLogger(__name__) @@ -62,7 +66,8 @@ class QEMUMachine: """ def __init__(self, binary, args=None, wrapper=None, name=None, - test_dir="/var/tmp", monitor_address=None, + test_dir="/var/tmp", + monitor_address: Optional[SocketAddrT] = None, socket_scm_helper=None, sock_dir=None): ''' Initialize a QEMUMachine @@ -85,8 +90,14 @@ def __init__(self, binary, args=None, wrapper=None, name=None, if sock_dir is None: sock_dir = test_dir self._name = name - self._monitor_address = monitor_address - self._vm_monitor = None + if monitor_address is not None: + self._monitor_address = monitor_address + self._remove_monitor_sockfile = False + else: + self._monitor_address = os.path.join( + sock_dir, f"{name}-monitor.sock" + ) + self._remove_monitor_sockfile = True self._qemu_log_path = None self._qemu_log_file = None self._popen = None @@ -225,15 +236,17 @@ def _load_io_log(self): def _base_args(self): args = ['-display', 'none', '-vga', 'none'] + if self._qmp_set: if isinstance(self._monitor_address, tuple): - moncdev = "socket,id=mon,host=%s,port=%s" % ( - self._monitor_address[0], - self._monitor_address[1]) + moncdev = "socket,id=mon,host={},port={}".format( + *self._monitor_address + ) else: - moncdev = 'socket,id=mon,path=%s' % self._vm_monitor + moncdev = f"socket,id=mon,path={self._monitor_address}" args.extend(['-chardev', moncdev, '-mon', 'chardev=mon,mode=control']) + if self._machine is not None: args.extend(['-machine', self._machine]) for _ in range(self._console_index): @@ -258,14 +271,14 @@ def _pre_launch(self): self._qemu_log_file = open(self._qemu_log_path, 'wb') if self._qmp_set: - if self._monitor_address is not None: - self._vm_monitor = self._monitor_address - else: - self._vm_monitor = os.path.join(self._sock_dir, - self._name + "-monitor.sock") - self._remove_files.append(self._vm_monitor) - self._qmp = qmp.QEMUMonitorProtocol(self._vm_monitor, server=True, - nickname=self._name) + if self._remove_monitor_sockfile: + assert isinstance(self._monitor_address, str) + self._remove_files.append(self._monitor_address) + self._qmp = qmp.QEMUMonitorProtocol( + self._monitor_address, + server=True, + nickname=self._name + ) def _post_launch(self): if self._qmp: From patchwork Thu Jun 4 20:22:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303728 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WuRju15Y; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHPr75Jvz9sSf for ; Fri, 5 Jun 2020 06:27:12 +1000 (AEST) Received: from localhost ([::1]:37608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwSE-0003e2-Ln for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:27:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49460) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO2-00048e-83 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:50 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:53464 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO1-0005Vh-2N for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tWWU9rRF1e11V/KCF+pDTzNBygwdzFey+awcvVrFGiM=; b=WuRju15YRU22yeyMikfV8ZmnBy0bEJlkhOg0l5wvIYqqvscmt9bjxZAspZ8v/v4cTCuNpD HliS/7xblcTch/9izrS7BA1LXgWtby193WOFCDLrlSLyg/oM5o171A7PTYOTJSPp4roO4u /lah5BCmCeQy0KjWD8Zz/WEfHgbgPTA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-182-pLSWptBmOhWCxXfBYk4ndg-1; Thu, 04 Jun 2020 16:22:46 -0400 X-MC-Unique: pLSWptBmOhWCxXfBYk4ndg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2AEC461; Thu, 4 Jun 2020 20:22:45 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8F465D9D3; Thu, 4 Jun 2020 20:22:44 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 08/16] python/machine.py: reorder __init__ Date: Thu, 4 Jun 2020 16:22:28 -0400 Message-Id: <20200604202236.25039-9-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:08:38 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Put the init arg handling all at the top, and mostly in order (deviating when one is dependent on another), and put what is effectively runtime state declaration at the bottom. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé --- python/qemu/machine.py | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 34713ba9b15..aae78e3bc87 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -81,38 +81,43 @@ def __init__(self, binary, args=None, wrapper=None, name=None, @param socket_scm_helper: helper program, required for send_fd_scm() @note: Qemu process is not started until launch() is used. ''' + # Direct user configuration + + self._binary = binary + if args is None: args = [] + # Copy mutable input: we will be modifying our copy + self._args = list(args) + if wrapper is None: wrapper = [] - if name is None: - name = "qemu-%d" % os.getpid() - if sock_dir is None: - sock_dir = test_dir - self._name = name + self._wrapper = wrapper + + self._name = name or "qemu-%d" % os.getpid() + self._test_dir = test_dir + self._sock_dir = sock_dir or self._test_dir + self._socket_scm_helper = socket_scm_helper + if monitor_address is not None: self._monitor_address = monitor_address self._remove_monitor_sockfile = False else: self._monitor_address = os.path.join( - sock_dir, f"{name}-monitor.sock" + self._sock_dir, f"{self._name}-monitor.sock" ) self._remove_monitor_sockfile = True + + # Runstate self._qemu_log_path = None self._qemu_log_file = None self._popen = None - self._binary = binary - self._args = list(args) # Force copy args in case we modify them - self._wrapper = wrapper self._events = [] self._iolog = None - self._socket_scm_helper = socket_scm_helper self._qmp_set = True # Enable QMP monitor by default. self._qmp = None self._qemu_full_args = None - self._test_dir = test_dir self._temp_dir = None - self._sock_dir = sock_dir self._launched = False self._machine = None self._console_index = 0 From patchwork Thu Jun 4 20:22:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303721 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AIMC3Ool; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHKV6cf6z9sSd for ; Fri, 5 Jun 2020 06:23:26 +1000 (AEST) Received: from localhost ([::1]:49320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwOa-0004WS-I1 for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:23:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO2-0004An-Vm for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:51 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:22970 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO1-0005Vs-R5 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302169; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ks+3SlCqyogCFxYtEM7IfT04KsZ9o5SIK1xwtRQ3Sf0=; b=AIMC3Ool0eEmVozQuODtR3NhZYv8Cmj49fpZGdBjeW6cuoviOjlXMNwJsJgEbSKpK8fz5Q d1blWtTaXlLeTBQtKp7YQtQ7CmtJLuKQJ8y/sJpsSDPtyGH1szqRua+Ls5o8KuFhhFpH4C 2x8WR5+f/tXASlPo8CBi+CZ5SNdEUlg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-219-JMQ_Ui43NWm98Mzcr1YaPw-1; Thu, 04 Jun 2020 16:22:47 -0400 X-MC-Unique: JMQ_Ui43NWm98Mzcr1YaPw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ACFB619200D1; Thu, 4 Jun 2020 20:22:46 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id C63DC5D9D3; Thu, 4 Jun 2020 20:22:45 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 09/16] python/machine.py: Don't modify state in _base_args() Date: Thu, 4 Jun 2020 16:22:29 -0400 Message-Id: <20200604202236.25039-10-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 13:58:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Don't append to the _remove_files list during _base_args; instead do so during _launch. Rework _base_args as a @property to help facilitate this impression. This has the additional benefit of making the type of _console_address easier to analyze statically. Signed-off-by: John Snow --- python/qemu/machine.py | 16 ++++++++++------ python/qemu/qtest.py | 11 ++++++++--- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index aae78e3bc87..92528a44097 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,7 @@ import socket import tempfile from typing import ( + List, Optional, Type, ) @@ -123,7 +124,9 @@ def __init__(self, binary, args=None, wrapper=None, name=None, self._console_index = 0 self._console_set = False self._console_device_type = None - self._console_address = None + self._console_address = os.path.join( + self._sock_dir, f"{self._name}-console.sock" + ) self._console_socket = None self._remove_files = [] self._killed = False @@ -239,7 +242,8 @@ def _load_io_log(self): with open(self._qemu_log_path, "r") as iolog: self._iolog = iolog.read() - def _base_args(self): + @property + def _base_args(self) -> List[str]: args = ['-display', 'none', '-vga', 'none'] if self._qmp_set: @@ -257,9 +261,6 @@ def _base_args(self): for _ in range(self._console_index): args.extend(['-serial', 'null']) if self._console_set: - self._console_address = os.path.join(self._sock_dir, - self._name + "-console.sock") - self._remove_files.append(self._console_address) chardev = ('socket,id=console,path=%s,server,nowait' % self._console_address) args.extend(['-chardev', chardev]) @@ -275,6 +276,9 @@ def _pre_launch(self): self._qemu_log_path = os.path.join(self._temp_dir, self._name + ".log") self._qemu_log_file = open(self._qemu_log_path, 'wb') + if self._console_set: + self._remove_files.append(self._console_address) + if self._qmp_set: if self._remove_monitor_sockfile: assert isinstance(self._monitor_address, str) @@ -349,7 +353,7 @@ def _launch(self): devnull = open(os.path.devnull, 'rb') self._pre_launch() self._qemu_full_args = (self._wrapper + [self._binary] + - self._base_args() + self._args) + self._base_args + self._args) LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args)) self._popen = subprocess.Popen(self._qemu_full_args, stdin=devnull, diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index 888c8bd2f60..05c63a1d583 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -19,7 +19,11 @@ import socket import os -from typing import Optional, TextIO +from typing import ( + List, + Optional, + TextIO, +) from .machine import QEMUMachine @@ -111,8 +115,9 @@ def __init__(self, binary, args=None, name=None, test_dir="/var/tmp", self._qtest = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") - def _base_args(self): - args = super()._base_args() + @property + def _base_args(self) -> List[str]: + args = super()._base_args args.extend(['-qtest', 'unix:path=' + self._qtest_path, '-accel', 'qtest']) return args From patchwork Thu Jun 4 20:22:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303734 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ewmvnt5Z; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHXR1prYz9sSd for ; Fri, 5 Jun 2020 06:32:53 +1000 (AEST) Received: from localhost ([::1]:57334 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwXi-0003bW-Lo for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:32:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49500) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO6-0004Kf-Kk for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:54 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:53657 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO3-0005WX-OQ for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KLpOdLFRQ8+uXdCSQXpzvYPo1r1s8rRuu7oCr7CpY+0=; b=Ewmvnt5ZYjKlmXXpZ5ggr7+ia3bCh7+2a174K7RSl25Q4uLGUEKXI4BKewLX0B4WTPQPfE 49Oz+jryobsycpw8STRnAsxVwo0SDtvAZ7am+Jyiiy9MNZpRREWSLErpCEm2Ka2csLXjQE gpXJLgI/xLjK5gv9bNN/rbh1D7KiWAk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-300-YG4j5GiXPdey1Urh2tqS0g-1; Thu, 04 Jun 2020 16:22:48 -0400 X-MC-Unique: YG4j5GiXPdey1Urh2tqS0g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AE1A719200C1; Thu, 4 Jun 2020 20:22:47 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id D492B5D9D7; Thu, 4 Jun 2020 20:22:46 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 10/16] python/machine.py: Handle None events in events_wait Date: Thu, 4 Jun 2020 16:22:30 -0400 Message-Id: <20200604202236.25039-11-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 12:22:43 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" If the timeout is 0, we can get None back. Handle this explicitly. Signed-off-by: John Snow --- python/qemu/machine.py | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 92528a44097..4afd67a9351 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -33,7 +33,7 @@ from types import TracebackType from . import qmp -from .qmp import SocketAddrT +from .qmp import SocketAddrT, QMPMessage LOG = logging.getLogger(__name__) @@ -544,13 +544,20 @@ def event_wait(self, name, timeout=60.0, match=None): def events_wait(self, events, timeout=60.0): """ - events_wait waits for and returns a named event - from QMP with a timeout. + events_wait waits for and returns a single named event from QMP. + In the case of multiple qualifying events, this function returns the + first one. - events: a sequence of (name, match_criteria) tuples. - The match criteria are optional and may be None. - See event_match for details. - timeout: QEMUMonitorProtocol.pull_event timeout parameter. + :param events: A sequence of (name, match_criteria) tuples. + The match criteria are optional and may be None. + See event_match for details. + :param timeout: Optional timeout, in seconds. + See QEMUMonitorProtocol.pull_event. + + :raise QMPTimeoutError: If timeout was non-zero and no matching events + were found. + :return: A QMP event matching the filter criteria. + If timeout was 0 and no event matched, None. """ def _match(event): for name, match in events: @@ -558,6 +565,8 @@ def _match(event): return True return False + event: Optional[QMPMessage] + # Search cached events for event in self._events: if _match(event): @@ -567,6 +576,10 @@ def _match(event): # Poll for new events while True: event = self._qmp.pull_event(wait=timeout) + if event is None: + # NB: None is only returned when timeout is false-ish. + # Timeouts raise QMPTimeoutError instead! + break if _match(event): return event self._events.append(event) From patchwork Thu Jun 4 20:22:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303726 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=STr1fKwW; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHMz11N9z9sT6 for ; Fri, 5 Jun 2020 06:25:35 +1000 (AEST) Received: from localhost ([::1]:58604 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwQe-0000YG-QQ for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:25:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO8-0004Ps-T8 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:56 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:56890 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO7-0005Xe-LD for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302174; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P+VgpIzy6wtm1IOWvJcS8BgnjzBJJMuKrYPJF2xyE3g=; b=STr1fKwWs8pHJ4hhaa3scrzI+D2DQsgjtBQLnf1MJwmFFXhoWYgVpW2A9JWZosTm1Xm645 ScU+Gbz332xdDSMfWpnqcS/7V41jhBnnhHc3iVJuMuHeRtxodJLvv1QaFN0I9Fc7j+FxHk XwbCkbhCGKM5p3NNY4GxAWSc0+udMkY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-146-M95O3XBkP0aV0p9YjvZe1w-1; Thu, 04 Jun 2020 16:22:51 -0400 X-MC-Unique: M95O3XBkP0aV0p9YjvZe1w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A7D9981F0A9; Thu, 4 Jun 2020 20:22:48 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id D13BE5D9CD; Thu, 4 Jun 2020 20:22:47 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 11/16] python/machine.py: use qmp.command Date: Thu, 4 Jun 2020 16:22:31 -0400 Message-Id: <20200604202236.25039-12-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:08:38 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" machine.py and qmp.py both do the same thing here; refactor machine.py to use qmp.py's functionality more directly. Signed-off-by: John Snow --- python/qemu/machine.py | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index 4afd67a9351..d8289936816 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -26,6 +26,8 @@ import socket import tempfile from typing import ( + Any, + Dict, List, Optional, Type, @@ -455,17 +457,23 @@ def set_qmp_monitor(self, enabled=True): self._qmp_set = False self._qmp = None - def qmp(self, cmd, conv_keys=True, **args): - """ - Invoke a QMP command and return the response dict - """ + @classmethod + def _qmp_args(cls, _conv_keys: bool = True, **args: Any) -> Dict[str, Any]: qmp_args = dict() for key, value in args.items(): - if conv_keys: + if _conv_keys: qmp_args[key.replace('_', '-')] = value else: qmp_args[key] = value + return qmp_args + def qmp(self, cmd: str, + conv_keys: bool = True, + **args: Any) -> QMPMessage: + """ + Invoke a QMP command and return the response dict + """ + qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.cmd(cmd, args=qmp_args) def command(self, cmd, conv_keys=True, **args): @@ -474,12 +482,8 @@ def command(self, cmd, conv_keys=True, **args): On success return the response dict. On failure raise an exception. """ - reply = self.qmp(cmd, conv_keys, **args) - if reply is None: - raise qmp.QMPError("Monitor is closed") - if "error" in reply: - raise qmp.QMPResponseError(reply) - return reply["return"] + qmp_args = self._qmp_args(conv_keys, **args) + return self._qmp.command(cmd, **qmp_args) def get_qmp_event(self, wait=False): """ From patchwork Thu Jun 4 20:22:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303732 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Vlty2AZZ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHVQ4DkPz9sT5 for ; Fri, 5 Jun 2020 06:31:10 +1000 (AEST) Received: from localhost ([::1]:53058 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwW4-0001ey-Ad for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:31:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO5-0004It-Ux for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:53 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:48845 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO4-0005Wv-Uk for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302172; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vJ6BhU9KIFdT7vmlanlNOtR+XXUXXpb3sxgWYK1kMtM=; b=Vlty2AZZKki6a0Y/cVb8egzMA50tdGQ3W7QEu/NEv7xxbypn4Nh8aW+byFbNtDYzg24UtY hUSJHUYHnW1kF0W8+cF9WBmkAA/5Pa9fwQpr4mS8LvcjpyzEiD8XE04ftKqAnLYCucTGdN HIPEHHcsb/U+ppnOe7bbF9ELeOvG/X0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-dPegOdCyMDyDxkJA7Dze_g-1; Thu, 04 Jun 2020 16:22:50 -0400 X-MC-Unique: dPegOdCyMDyDxkJA7Dze_g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A04EB81F0A5; Thu, 4 Jun 2020 20:22:49 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB12F5D9CD; Thu, 4 Jun 2020 20:22:48 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 12/16] python/machine.py: Add _qmp access shim Date: Thu, 4 Jun 2020 16:22:32 -0400 Message-Id: <20200604202236.25039-13-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.81; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:31:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Like many other Optional[] types, it's not always a given that this object will be set. Wrap it in a type-shim that raises a meaningful error and will always return a concrete type. Signed-off-by: John Snow --- python/qemu/machine.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index d8289936816..a451f9000d6 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -118,7 +118,7 @@ def __init__(self, binary, args=None, wrapper=None, name=None, self._events = [] self._iolog = None self._qmp_set = True # Enable QMP monitor by default. - self._qmp = None + self._qmp_connection: Optional[qmp.QEMUMonitorProtocol] = None self._qemu_full_args = None self._temp_dir = None self._launched = False @@ -285,7 +285,7 @@ def _pre_launch(self): if self._remove_monitor_sockfile: assert isinstance(self._monitor_address, str) self._remove_files.append(self._monitor_address) - self._qmp = qmp.QEMUMonitorProtocol( + self._qmp_connection = qmp.QEMUMonitorProtocol( self._monitor_address, server=True, nickname=self._name @@ -455,7 +455,13 @@ def set_qmp_monitor(self, enabled=True): self._qmp_set = True else: self._qmp_set = False - self._qmp = None + self._qmp_connection = None + + @property + def _qmp(self) -> qmp.QEMUMonitorProtocol: + if self._qmp_connection is None: + raise QEMUMachineError("Attempt to access QMP with no connection") + return self._qmp_connection @classmethod def _qmp_args(cls, _conv_keys: bool = True, **args: Any) -> Dict[str, Any]: From patchwork Thu Jun 4 20:22:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303731 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ISqS2WJl; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHSZ1v2tz9sSd for ; Fri, 5 Jun 2020 06:29:34 +1000 (AEST) Received: from localhost ([::1]:48148 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwUV-00081j-Ug for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:29:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49530) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO9-0004R9-Ao for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:57 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:26515 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO8-0005Xu-9R for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z2F0M0Kxr4yi1TdKeym0zcdSZAKGBslQruOlqOtpQp4=; b=ISqS2WJleS9Rd2+6QM17ImzRSMW6UkHnkZz4KEImxs4IplNuaJClP4LFnpmrXFA3zbn/cN OMHwhR5QjfBWChqsikYPJRaLmMOLzkpovKhcNCaX6JOzJfIcAEUojQlz5gw2/9ItACHyxh dvkYIeIBtYQMN7CWO3JRh0FyqDp2bZc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-319-hpI4ArjONpyo_OdJevTvgQ-1; Thu, 04 Jun 2020 16:22:51 -0400 X-MC-Unique: hpI4ArjONpyo_OdJevTvgQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F2BD8835B41; Thu, 4 Jun 2020 20:22:50 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 243F45D9D3; Thu, 4 Jun 2020 20:22:50 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 13/16] python/machine.py: fix _popen access Date: Thu, 4 Jun 2020 16:22:33 -0400 Message-Id: <20200604202236.25039-14-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 13:58:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" As always, Optional[T] causes problems with unchecked access. Add a helper that asserts the pipe is present before we attempt to talk with it. Signed-off-by: John Snow --- python/qemu/machine.py | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index a451f9000d6..a81a422b60e 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -114,7 +114,7 @@ def __init__(self, binary, args=None, wrapper=None, name=None, # Runstate self._qemu_log_path = None self._qemu_log_file = None - self._popen = None + self._popen: Optional['subprocess.Popen[bytes]'] = None self._events = [] self._iolog = None self._qmp_set = True # Enable QMP monitor by default. @@ -227,6 +227,12 @@ def is_running(self): """Returns true if the VM is running.""" return self._popen is not None and self._popen.poll() is None + @property + def _subp(self) -> 'subprocess.Popen[bytes]': + if self._popen is None: + raise QEMUMachineError('Subprocess pipe not present') + return self._popen + def exitcode(self): """Returns the exit code if possible, or None.""" if self._popen is None: @@ -237,7 +243,7 @@ def get_pid(self): """Returns the PID of the running process, or None.""" if not self.is_running(): return None - return self._popen.pid + return self._subp.pid def _load_io_log(self): if self._qemu_log_path is not None: @@ -369,7 +375,7 @@ def wait(self): """ Wait for the VM to power off """ - self._popen.wait() + self._subp.wait() if self._qmp: self._qmp.close() self._post_shutdown() @@ -381,8 +387,8 @@ def _hard_shutdown(self) -> None: if not self.is_running(): return - self._popen.kill() - self._popen.wait(timeout=60) + self._subp.kill() + self._subp.wait(timeout=60) def _soft_shutdown(self, has_quit: bool = False, timeout: int = 3) -> None: """ @@ -399,7 +405,7 @@ def _soft_shutdown(self, has_quit: bool = False, timeout: int = 3) -> None: self._qmp.cmd('quit') self._qmp.close() - self._popen.wait(timeout=timeout) + self._subp.wait(timeout=timeout) def _do_shutdown(self, has_quit: bool = False, timeout: int = 3) -> None: """ From patchwork Thu Jun 4 20:22:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303733 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=cj4rw95S; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHVT3RF5z9sSf for ; Fri, 5 Jun 2020 06:31:13 +1000 (AEST) Received: from localhost ([::1]:53218 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwW7-0001iz-8w for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:31:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49532) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwO9-0004S2-My for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:57 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:47414 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO7-0005XY-DU for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302174; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wAmEEniZCBwTLOySCJcZgcJZZKs84Tp6/2vVhFoLwCU=; b=cj4rw95SSUkFSCiaAC2kMy0dEtGI9apNWLplH+muZTRYG/Ghcm8yGS7ucxGEXT+Pe79tGj pYMjKHFuxrfLT+kJNHolySxh7CmpZCp6ALvPpYZtEoP833XHrc+7HA9n04r2xkySQdWTAy QPS+JSNd3aTaqP13uZMA1YsgCdTu354= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-338-hlnFUaUYOWGbQGpQyyC3ow-1; Thu, 04 Jun 2020 16:22:52 -0400 X-MC-Unique: hlnFUaUYOWGbQGpQyyC3ow-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EEE4281F0A4; Thu, 4 Jun 2020 20:22:51 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 23DE75D9D3; Thu, 4 Jun 2020 20:22:51 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 14/16] python/qemu: make 'args' style arguments immutable Date: Thu, 4 Jun 2020 16:22:34 -0400 Message-Id: <20200604202236.25039-15-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 12:22:43 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" These arguments don't need to be mutable and aren't really used as such. Clarify their types as immutable and adjust code to match where necessary. In general, It's probably best not to accept a user-defined mutable object and store it as internal object state unless there's a strong justification for doing so. Instead, try to use generic types as input with empty tuples as the default, and coerce to list where necessary. Signed-off-by: John Snow --- python/qemu/machine.py | 30 +++++++++++++++++------------- python/qemu/qtest.py | 16 ++++++++++++---- 2 files changed, 29 insertions(+), 17 deletions(-) diff --git a/python/qemu/machine.py b/python/qemu/machine.py index a81a422b60e..c053c946401 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -18,6 +18,7 @@ # import errno +from itertools import chain import logging import os import subprocess @@ -30,6 +31,8 @@ Dict, List, Optional, + Sequence, + Tuple, Type, ) from types import TracebackType @@ -68,8 +71,12 @@ class QEMUMachine: # vm is guaranteed to be shut down here """ - def __init__(self, binary, args=None, wrapper=None, name=None, - test_dir="/var/tmp", + def __init__(self, + binary: str, + args: Sequence[str] = (), + wrapper: Sequence[str] = (), + name: Optional[str] = None, + test_dir: str = "/var/tmp", monitor_address: Optional[SocketAddrT] = None, socket_scm_helper=None, sock_dir=None): ''' @@ -87,14 +94,7 @@ def __init__(self, binary, args=None, wrapper=None, name=None, # Direct user configuration self._binary = binary - - if args is None: - args = [] - # Copy mutable input: we will be modifying our copy self._args = list(args) - - if wrapper is None: - wrapper = [] self._wrapper = wrapper self._name = name or "qemu-%d" % os.getpid() @@ -119,7 +119,7 @@ def __init__(self, binary, args=None, wrapper=None, name=None, self._iolog = None self._qmp_set = True # Enable QMP monitor by default. self._qmp_connection: Optional[qmp.QEMUMonitorProtocol] = None - self._qemu_full_args = None + self._qemu_full_args: Tuple[str, ...] = () self._temp_dir = None self._launched = False self._machine = None @@ -340,7 +340,7 @@ def launch(self): raise QEMUMachineError('VM already launched') self._iolog = None - self._qemu_full_args = None + self._qemu_full_args = () try: self._launch() self._launched = True @@ -360,8 +360,12 @@ def _launch(self): """ devnull = open(os.path.devnull, 'rb') self._pre_launch() - self._qemu_full_args = (self._wrapper + [self._binary] + - self._base_args + self._args) + self._qemu_full_args = tuple( + chain(self._wrapper, + [self._binary], + self._base_args, + self._args) + ) LOG.debug('VM launch command: %r', ' '.join(self._qemu_full_args)) self._popen = subprocess.Popen(self._qemu_full_args, stdin=devnull, diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index 05c63a1d583..ae4661d4d3e 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -22,6 +22,7 @@ from typing import ( List, Optional, + Sequence, TextIO, ) @@ -103,8 +104,13 @@ class QEMUQtestMachine(QEMUMachine): A QEMU VM, with a qtest socket available. """ - def __init__(self, binary, args=None, name=None, test_dir="/var/tmp", - socket_scm_helper=None, sock_dir=None): + def __init__(self, + binary: str, + args: Sequence[str] = (), + name: Optional[str] = None, + test_dir: str = "/var/tmp", + socket_scm_helper: Optional[str] = None, + sock_dir: Optional[str] = None): if name is None: name = "qemu-%d" % os.getpid() if sock_dir is None: @@ -118,8 +124,10 @@ def __init__(self, binary, args=None, name=None, test_dir="/var/tmp", @property def _base_args(self) -> List[str]: args = super()._base_args - args.extend(['-qtest', 'unix:path=' + self._qtest_path, - '-accel', 'qtest']) + args.extend([ + '-qtest', f"unix:path={self._qtest_path}", + '-accel', 'qtest' + ]) return args def _pre_launch(self): From patchwork Thu Jun 4 20:22:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303735 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FnRJ+vyb; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHZ80bCnz9sSf for ; Fri, 5 Jun 2020 06:34:24 +1000 (AEST) Received: from localhost ([::1]:33434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwZB-0005Sp-Pv for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:34:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49542) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwOA-0004T5-1y for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:58 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:39418 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO8-0005Y2-H0 for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0iGSpQJVJCdKjrWxqDYE2XmRQvkmLt6rW5uM+LnGK5c=; b=FnRJ+vybhmWobQJ7otHwZqAcMV5FuolziuYFOe5m6VG8BAnzGThLbaDkWEnsyS6IVqPvGw 5JPZF+Kvd9QkZJ7jLGs0LGqCJpAdgj8EFYdJLEZg9GUYr8mFwxGbsMI4vQj+0xA14fcTUO jY2Zp4utGAJ8MfFEJrjfTAM7/GnD/pQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-45-y3VyyvH4MaCgf2mWNT6KgQ-1; Thu, 04 Jun 2020 16:22:54 -0400 X-MC-Unique: y3VyyvH4MaCgf2mWNT6KgQ-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 06308A0BD7; Thu, 4 Jun 2020 20:22:53 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1E5C85D9D5; Thu, 4 Jun 2020 20:22:52 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 15/16] iotests.py: Adjust HMP kwargs typing Date: Thu, 4 Jun 2020 16:22:35 -0400 Message-Id: <20200604202236.25039-16-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=207.211.31.120; envelope-from=jsnow@redhat.com; helo=us-smtp-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 13:58:37 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" mypy wants to ensure there's consistency between the kwargs arguments types and any unspecified keyword arguments. In this case, conv_keys is a bool, but the remaining keys are Any type. Mypy (correctly) infers the **kwargs type to be **Dict[str, str], which is not compatible with conv_keys: bool. Because QMP typing is a little fraught right now anyway, re-type kwargs to Dict[str, Any] which has the benefit of silencing this check right now. A future re-design might type these more aggressively, but this will give us a baseline to work from with minimal disruption. (Thanks Kevin Wolf for the debugging assist here) Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 7c1773bba37..19b22bee618 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -553,7 +553,7 @@ def add_incoming(self, addr): def hmp(self, command_line: str, use_log: bool = False) -> QMPMessage: cmd = 'human-monitor-command' - kwargs = {'command-line': command_line} + kwargs: Dict[str, Any] = {'command-line': command_line} if use_log: return self.qmp_log(cmd, **kwargs) else: From patchwork Thu Jun 4 20:22:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1303738 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=hP2vVAUk; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49dHbN64tQz9sSf for ; Fri, 5 Jun 2020 06:35:28 +1000 (AEST) Received: from localhost ([::1]:36858 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgwaE-00074m-Jm for incoming@patchwork.ozlabs.org; Thu, 04 Jun 2020 16:35:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgwOB-0004YG-QE for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:59 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:46439 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jgwO9-0005YO-MF for qemu-devel@nongnu.org; Thu, 04 Jun 2020 16:22:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591302176; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K/N7zFS0sfDyagMr1aPHZhu9dDd/uMAQtie5yWZMVa8=; b=hP2vVAUkcB3aYsPKUWrtj+s6rGAZXpdO8PYE76/Gv7Jw9hAQZVN+LQTvMXF8ZaPdOiExrL psEQqrsrdX0UBJjjWhCM3XuCLmPwdlSVrzcRVBIbP9HnWv27EiLlckIEb913Ub9wPoyPy0 V/gy443qK3MZkk7Q7LTMudd718ETaV0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-TBhhnbVYM0uCHNAzvT5fvg-1; Thu, 04 Jun 2020 16:22:55 -0400 X-MC-Unique: TBhhnbVYM0uCHNAzvT5fvg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1C75581F0A4; Thu, 4 Jun 2020 20:22:54 +0000 (UTC) Received: from probe.redhat.com (ovpn-117-188.rdu2.redhat.com [10.10.117.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 283435D9D3; Thu, 4 Jun 2020 20:22:53 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v3 16/16] python/qemu: Add mypy type annotations Date: Thu, 4 Jun 2020 16:22:36 -0400 Message-Id: <20200604202236.25039-17-jsnow@redhat.com> In-Reply-To: <20200604202236.25039-1-jsnow@redhat.com> References: <20200604202236.25039-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=205.139.110.61; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/04 01:08:38 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, John Snow , Max Reitz , Cleber Rosa , philmd@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" These should all be purely annotations with no changes in behavior at all. You need to be in the python folder, but you should be able to confirm that these annotations are correct (or at least self-consistent) by running `mypy --strict qemu`. Signed-off-by: John Snow --- python/qemu/accel.py | 8 ++-- python/qemu/machine.py | 94 ++++++++++++++++++++++++------------------ python/qemu/qmp.py | 44 +++++++++++--------- python/qemu/qtest.py | 26 +++++++----- 4 files changed, 98 insertions(+), 74 deletions(-) diff --git a/python/qemu/accel.py b/python/qemu/accel.py index 7fabe629208..4325114e51f 100644 --- a/python/qemu/accel.py +++ b/python/qemu/accel.py @@ -17,6 +17,7 @@ import logging import os import subprocess +from typing import List, Optional LOG = logging.getLogger(__name__) @@ -29,7 +30,7 @@ } -def list_accel(qemu_bin): +def list_accel(qemu_bin: str) -> List[str]: """ List accelerators enabled in the QEMU binary. @@ -49,7 +50,8 @@ def list_accel(qemu_bin): return [acc.strip() for acc in out.splitlines()[1:]] -def kvm_available(target_arch=None, qemu_bin=None): +def kvm_available(target_arch: Optional[str] = None, + qemu_bin: Optional[str] = None) -> bool: """ Check if KVM is available using the following heuristic: - Kernel module is present in the host; @@ -72,7 +74,7 @@ def kvm_available(target_arch=None, qemu_bin=None): return True -def tcg_available(qemu_bin): +def tcg_available(qemu_bin: str) -> bool: """ Check if TCG is available. diff --git a/python/qemu/machine.py b/python/qemu/machine.py index c053c946401..d4da4e8ce15 100644 --- a/python/qemu/machine.py +++ b/python/qemu/machine.py @@ -28,6 +28,7 @@ import tempfile from typing import ( Any, + BinaryIO, Dict, List, Optional, @@ -38,7 +39,7 @@ from types import TracebackType from . import qmp -from .qmp import SocketAddrT, QMPMessage +from .qmp import QMPMessage, QMPReturnValue, SocketAddrT LOG = logging.getLogger(__name__) @@ -61,7 +62,7 @@ class QEMUMachineAddDeviceError(QEMUMachineError): class QEMUMachine: """ - A QEMU VM + A QEMU VM. Use this object as a context manager to ensure the QEMU process terminates:: @@ -78,7 +79,8 @@ def __init__(self, name: Optional[str] = None, test_dir: str = "/var/tmp", monitor_address: Optional[SocketAddrT] = None, - socket_scm_helper=None, sock_dir=None): + socket_scm_helper: Optional[str] = None, + sock_dir: Optional[str] = None): ''' Initialize a QEMUMachine @@ -112,28 +114,28 @@ def __init__(self, self._remove_monitor_sockfile = True # Runstate - self._qemu_log_path = None - self._qemu_log_file = None + self._qemu_log_path: Optional[str] = None + self._qemu_log_file: Optional[BinaryIO] = None self._popen: Optional['subprocess.Popen[bytes]'] = None - self._events = [] - self._iolog = None + self._events: List[QMPMessage] = [] + self._iolog: Optional[str] = None self._qmp_set = True # Enable QMP monitor by default. self._qmp_connection: Optional[qmp.QEMUMonitorProtocol] = None self._qemu_full_args: Tuple[str, ...] = () - self._temp_dir = None + self._temp_dir: Optional[str] = None self._launched = False - self._machine = None + self._machine: Optional[str] = None self._console_index = 0 self._console_set = False - self._console_device_type = None + self._console_device_type: Optional[str] = None self._console_address = os.path.join( self._sock_dir, f"{self._name}-console.sock" ) - self._console_socket = None - self._remove_files = [] + self._console_socket: Optional[socket.socket] = None + self._remove_files: List[str] = [] self._killed = False - def __enter__(self): + def __enter__(self) -> 'QEMUMachine': return self def __exit__(self, @@ -142,14 +144,15 @@ def __exit__(self, exc_tb: Optional[TracebackType]) -> None: self.shutdown() - def add_monitor_null(self): + def add_monitor_null(self) -> None: """ This can be used to add an unused monitor instance. """ self._args.append('-monitor') self._args.append('null') - def add_fd(self, fd, fdset, opaque, opts=''): + def add_fd(self, fd: int, fdset: int, + opaque: str, opts: str = '') -> 'QEMUMachine': """ Pass a file descriptor to the VM """ @@ -168,7 +171,8 @@ def add_fd(self, fd, fdset, opaque, opts=''): self._args.append(','.join(options)) return self - def send_fd_scm(self, fd=None, file_path=None): + def send_fd_scm(self, fd: Optional[int] = None, + file_path: Optional[str] = None) -> int: """ Send an fd or file_path to socket_scm_helper. @@ -212,7 +216,7 @@ def send_fd_scm(self, fd=None, file_path=None): return proc.returncode @staticmethod - def _remove_if_exists(path): + def _remove_if_exists(path: str) -> None: """ Remove file object at path if it exists """ @@ -223,7 +227,7 @@ def _remove_if_exists(path): return raise - def is_running(self): + def is_running(self) -> bool: """Returns true if the VM is running.""" return self._popen is not None and self._popen.poll() is None @@ -233,19 +237,19 @@ def _subp(self) -> 'subprocess.Popen[bytes]': raise QEMUMachineError('Subprocess pipe not present') return self._popen - def exitcode(self): + def exitcode(self) -> Optional[int]: """Returns the exit code if possible, or None.""" if self._popen is None: return None return self._popen.poll() - def get_pid(self): + def get_pid(self) -> Optional[int]: """Returns the PID of the running process, or None.""" if not self.is_running(): return None return self._subp.pid - def _load_io_log(self): + def _load_io_log(self) -> None: if self._qemu_log_path is not None: with open(self._qemu_log_path, "r") as iolog: self._iolog = iolog.read() @@ -279,7 +283,7 @@ def _base_args(self) -> List[str]: args.extend(['-device', device]) return args - def _pre_launch(self): + def _pre_launch(self) -> None: self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) self._qemu_log_path = os.path.join(self._temp_dir, self._name + ".log") self._qemu_log_file = open(self._qemu_log_path, 'wb') @@ -297,7 +301,7 @@ def _pre_launch(self): nickname=self._name ) - def _post_launch(self): + def _post_launch(self) -> None: if self._qmp: self._qmp.accept() @@ -330,7 +334,7 @@ def _post_shutdown(self) -> None: self._killed = False self._launched = False - def launch(self): + def launch(self) -> None: """ Launch the VM and make sure we cleanup and expose the command line/output in case of exception @@ -354,7 +358,7 @@ def launch(self): LOG.debug('Output: %r', self._iolog) raise - def _launch(self): + def _launch(self) -> None: """ Launch the VM and establish a QMP connection """ @@ -375,7 +379,7 @@ def _launch(self): close_fds=False) self._post_launch() - def wait(self): + def wait(self) -> None: """ Wait for the VM to power off """ @@ -446,13 +450,13 @@ def shutdown(self, has_quit: bool = False, hard: bool = False) -> None: finally: self._post_shutdown() - def kill(self): + def kill(self) -> None: """ Terminate the VM forcefully and perform cleanup. """ self.shutdown(hard=True) - def set_qmp_monitor(self, enabled=True): + def set_qmp_monitor(self, enabled: bool = True) -> None: """ Set the QMP monitor. @@ -492,7 +496,9 @@ def qmp(self, cmd: str, qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.cmd(cmd, args=qmp_args) - def command(self, cmd, conv_keys=True, **args): + def command(self, cmd: str, + conv_keys: bool = True, + **args: Any) -> QMPReturnValue: """ Invoke a QMP command. On success return the response dict. @@ -501,7 +507,7 @@ def command(self, cmd, conv_keys=True, **args): qmp_args = self._qmp_args(conv_keys, **args) return self._qmp.command(cmd, **qmp_args) - def get_qmp_event(self, wait=False): + def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: """ Poll for one queued QMP events and return it """ @@ -509,7 +515,7 @@ def get_qmp_event(self, wait=False): return self._events.pop(0) return self._qmp.pull_event(wait=wait) - def get_qmp_events(self, wait=False): + def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: """ Poll for queued QMP events and return a list of dicts """ @@ -520,7 +526,7 @@ def get_qmp_events(self, wait=False): return events @staticmethod - def event_match(event, match=None): + def event_match(event: Any, match: Optional[Any]) -> bool: """ Check if an event matches optional match criteria. @@ -550,9 +556,11 @@ def event_match(event, match=None): return True except TypeError: # either match or event wasn't iterable (not a dict) - return match == event + return bool(match == event) - def event_wait(self, name, timeout=60.0, match=None): + def event_wait(self, name: str, + timeout: float = 60.0, + match: Optional[QMPMessage] = None) -> Optional[QMPMessage]: """ event_wait waits for and returns a named event from QMP with a timeout. @@ -562,7 +570,9 @@ def event_wait(self, name, timeout=60.0, match=None): """ return self.events_wait([(name, match)], timeout) - def events_wait(self, events, timeout=60.0): + def events_wait(self, + events: Sequence[Tuple[str, Any]], + timeout: float = 60.0) -> Optional[QMPMessage]: """ events_wait waits for and returns a single named event from QMP. In the case of multiple qualifying events, this function returns the @@ -579,7 +589,7 @@ def events_wait(self, events, timeout=60.0): :return: A QMP event matching the filter criteria. If timeout was 0 and no event matched, None. """ - def _match(event): + def _match(event: QMPMessage) -> bool: for name, match in events: if event['event'] == name and self.event_match(event, match): return True @@ -606,20 +616,20 @@ def _match(event): return None - def get_log(self): + def get_log(self) -> Optional[str]: """ After self.shutdown or failed qemu execution, this returns the output of the qemu process. """ return self._iolog - def add_args(self, *args): + def add_args(self, *args: str) -> None: """ Adds to the list of extra arguments to be given to the QEMU binary """ self._args.extend(args) - def set_machine(self, machine_type): + def set_machine(self, machine_type: str) -> None: """ Sets the machine type @@ -628,7 +638,9 @@ def set_machine(self, machine_type): """ self._machine = machine_type - def set_console(self, device_type=None, console_index=0): + def set_console(self, + device_type: Optional[str] = None, + console_index: int = 0) -> None: """ Sets the device type for a console device @@ -659,7 +671,7 @@ def set_console(self, device_type=None, console_index=0): self._console_index = console_index @property - def console_socket(self): + def console_socket(self) -> socket.socket: """ Returns a socket connected to the console """ diff --git a/python/qemu/qmp.py b/python/qemu/qmp.py index 7935dababbf..303e82ee6b4 100644 --- a/python/qemu/qmp.py +++ b/python/qemu/qmp.py @@ -15,6 +15,7 @@ Any, cast, Dict, + List, Optional, TextIO, Type, @@ -90,7 +91,9 @@ class QEMUMonitorProtocol: #: Logger object for debugging messages logger = logging.getLogger('QMP') - def __init__(self, address, server=False, nickname=None): + def __init__(self, address: SocketAddrT, + server: bool = False, + nickname: Optional[str] = None): """ Create a QEMUMonitorProtocol class. @@ -102,7 +105,7 @@ def __init__(self, address, server=False, nickname=None): @note No connection is established, this is done by the connect() or accept() methods """ - self.__events = [] + self.__events: List[QMPMessage] = [] self.__address = address self.__sock = self.__get_sock() self.__sockfile: Optional[TextIO] = None @@ -114,14 +117,14 @@ def __init__(self, address, server=False, nickname=None): self.__sock.bind(self.__address) self.__sock.listen(1) - def __get_sock(self): + def __get_sock(self) -> socket.socket: if isinstance(self.__address, tuple): family = socket.AF_INET else: family = socket.AF_UNIX return socket.socket(family, socket.SOCK_STREAM) - def __negotiate_capabilities(self): + def __negotiate_capabilities(self) -> QMPMessage: greeting = self.__json_read() if greeting is None or "QMP" not in greeting: raise QMPConnectError @@ -131,7 +134,7 @@ def __negotiate_capabilities(self): return greeting raise QMPCapabilitiesError - def __json_read(self, only_event=False): + def __json_read(self, only_event: bool = False) -> Optional[QMPMessage]: assert self.__sockfile is not None while True: data = self.__sockfile.readline() @@ -148,7 +151,7 @@ def __json_read(self, only_event=False): continue return resp - def __get_events(self, wait=False): + def __get_events(self, wait: Union[bool, float] = False) -> None: """ Check for new events in the stream and cache them in __events. @@ -186,7 +189,7 @@ def __get_events(self, wait=False): raise QMPConnectError("Error while reading from socket") self.__sock.settimeout(None) - def __enter__(self): + def __enter__(self) -> 'QEMUMonitorProtocol': # Implement context manager enter function. return self @@ -199,7 +202,7 @@ def __exit__(self, # Implement context manager exit function. self.close() - def connect(self, negotiate=True): + def connect(self, negotiate: bool = True) -> Optional[QMPMessage]: """ Connect to the QMP Monitor and perform capabilities negotiation. @@ -214,7 +217,7 @@ def connect(self, negotiate=True): return self.__negotiate_capabilities() return None - def accept(self, timeout=15.0): + def accept(self, timeout: float = 15.0) -> QMPMessage: """ Await connection from QMP Monitor and perform capabilities negotiation. @@ -250,7 +253,9 @@ def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: self.logger.debug("<<< %s", resp) return resp - def cmd(self, name, args=None, cmd_id=None): + def cmd(self, name: str, + args: Optional[Dict[str, Any]] = None, + cmd_id: Optional[Any] = None) -> QMPMessage: """ Build a QMP command and send it to the QMP Monitor. @@ -258,14 +263,14 @@ def cmd(self, name, args=None, cmd_id=None): @param args: command arguments (dict) @param cmd_id: command id (dict, list, string or int) """ - qmp_cmd = {'execute': name} + qmp_cmd: QMPMessage = {'execute': name} if args: qmp_cmd['arguments'] = args if cmd_id: qmp_cmd['id'] = cmd_id return self.cmd_obj(qmp_cmd) - def command(self, cmd, **kwds): + def command(self, cmd: str, **kwds: Any) -> QMPReturnValue: """ Build and send a QMP command to the monitor, report errors if any """ @@ -278,7 +283,8 @@ def command(self, cmd, **kwds): ) return cast(QMPReturnValue, ret['return']) - def pull_event(self, wait=False): + def pull_event(self, + wait: Union[bool, float] = False) -> Optional[QMPMessage]: """ Pulls a single event. @@ -298,7 +304,7 @@ def pull_event(self, wait=False): return self.__events.pop(0) return None - def get_events(self, wait=False): + def get_events(self, wait: bool = False) -> List[QMPMessage]: """ Get a list of available QMP events. @@ -315,13 +321,13 @@ def get_events(self, wait=False): self.__get_events(wait) return self.__events - def clear_events(self): + def clear_events(self) -> None: """ Clear current list of pending events. """ self.__events = [] - def close(self): + def close(self) -> None: """ Close the socket and socket file. """ @@ -330,7 +336,7 @@ def close(self): if self.__sockfile: self.__sockfile.close() - def settimeout(self, timeout): + def settimeout(self, timeout: float) -> None: """ Set the socket timeout. @@ -339,7 +345,7 @@ def settimeout(self, timeout): """ self.__sock.settimeout(timeout) - def get_sock_fd(self): + def get_sock_fd(self) -> int: """ Get the socket file descriptor. @@ -347,7 +353,7 @@ def get_sock_fd(self): """ return self.__sock.fileno() - def is_scm_available(self): + def is_scm_available(self) -> bool: """ Check if the socket allows for SCM_RIGHTS. diff --git a/python/qemu/qtest.py b/python/qemu/qtest.py index ae4661d4d3e..49c04bc98d9 100644 --- a/python/qemu/qtest.py +++ b/python/qemu/qtest.py @@ -27,6 +27,7 @@ ) from .machine import QEMUMachine +from .qmp import SocketAddrT class QEMUQtestProtocol: @@ -43,7 +44,8 @@ class QEMUQtestProtocol: No conection is estabalished by __init__(), this is done by the connect() or accept() methods. """ - def __init__(self, address, server=False): + def __init__(self, address: SocketAddrT, + server: bool = False): self._address = address self._sock = self._get_sock() self._sockfile: Optional[TextIO] = None @@ -51,14 +53,14 @@ def __init__(self, address, server=False): self._sock.bind(self._address) self._sock.listen(1) - def _get_sock(self): + def _get_sock(self) -> socket.socket: if isinstance(self._address, tuple): family = socket.AF_INET else: family = socket.AF_UNIX return socket.socket(family, socket.SOCK_STREAM) - def connect(self): + def connect(self) -> None: """ Connect to the qtest socket. @@ -67,7 +69,7 @@ def connect(self): self._sock.connect(self._address) self._sockfile = self._sock.makefile(mode='r') - def accept(self): + def accept(self) -> None: """ Await connection from QEMU. @@ -76,7 +78,7 @@ def accept(self): self._sock, _ = self._sock.accept() self._sockfile = self._sock.makefile(mode='r') - def cmd(self, qtest_cmd): + def cmd(self, qtest_cmd: str) -> str: """ Send a qtest command on the wire. @@ -87,14 +89,16 @@ def cmd(self, qtest_cmd): resp = self._sockfile.readline() return resp - def close(self): - """Close this socket.""" + def close(self) -> None: + """ + Close this socket. + """ self._sock.close() if self._sockfile: self._sockfile.close() self._sockfile = None - def settimeout(self, timeout): + def settimeout(self, timeout: Optional[float]) -> None: """Set a timeout, in seconds.""" self._sock.settimeout(timeout) @@ -118,7 +122,7 @@ def __init__(self, super().__init__(binary, args, name=name, test_dir=test_dir, socket_scm_helper=socket_scm_helper, sock_dir=sock_dir) - self._qtest = None + self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_path = os.path.join(sock_dir, name + "-qtest.sock") @property @@ -130,7 +134,7 @@ def _base_args(self) -> List[str]: ]) return args - def _pre_launch(self): + def _pre_launch(self) -> None: super()._pre_launch() self._qtest = QEMUQtestProtocol(self._qtest_path, server=True) @@ -139,7 +143,7 @@ def _post_launch(self) -> None: super()._post_launch() self._qtest.accept() - def _post_shutdown(self): + def _post_shutdown(self) -> None: super()._post_shutdown() self._remove_if_exists(self._qtest_path)