From patchwork Tue May 11 22:05:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477327 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=aBLGSQ+R; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsTK0xQyz9sWC for ; Wed, 12 May 2021 08:06:43 +1000 (AEST) Received: from localhost ([::1]:40676 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaWV-0007MK-Tz for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:06:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55580) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW4-0007Li-6u for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:38307) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW0-0006My-Ei for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770767; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jwxvOtk9fMBMeDS783vKAy8mU+8EtKGYID2z597kIyU=; b=aBLGSQ+RMTe8Su+8gZufAjDr5fyIpigcpOTANbkiln1MpC7Xpsd9/UIWY+dabpb8NlZ9xo oYEBv32wP4aMVdutxVyS47O5ZsZVs/0lazIRu3Cr4A1lX0ZrotX1cACoIMLW/HVxPva63N nepHaM0bzNUbw/4596xRq6ljulWwITs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-251-rdiSu9DiNDiVbp1j0sgn_g-1; Tue, 11 May 2021 18:06:04 -0400 X-MC-Unique: rdiSu9DiNDiVbp1j0sgn_g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 59E42800D55; Tue, 11 May 2021 22:06:03 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0327BA6F; Tue, 11 May 2021 22:06:02 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 01/21] qapi/parser: Don't try to handle file errors Date: Tue, 11 May 2021 18:05:41 -0400 Message-Id: <20210511220601.2110055-2-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Remove the try/except block that handles file-opening errors in QAPISchemaParser.__init__() and add one each to QAPISchemaParser._include() and QAPISchema.__init__() respectively. The short-ish version of what motivates this patch is: - It's hard to write a good error message in the init method, because we need to determine the context of our caller to do so. It's easier to just let the caller write the message. - We don't want to allow QAPISourceInfo(None, None, None) to exist. - Errors made using such an object are currently incorrect. - It's not technically a semantic error if we cannot open the schema - There are various typing constraints that make mixing these two cases undesirable for a single special case. Other considerations: - open() is moved to a 'with' block to ensure file pointers are cleaned up deterministically. - Python 3.3 deprecated IOError and made it a synonym for OSError. Avoid the misleading perception these exception handlers are narrower than they really are. - Not all QAPIError objects have an 'info' field, so remove that stanza from test-qapi. Don't bother to replace the early exit on purpose so that we can test its output in the next commit. The long version: The error message string here is incorrect (since 52a474180a): > python3 qapi-gen.py 'fake.json' qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory In pursuing it, we find that QAPISourceInfo has a special accommodation for when there's no filename. Meanwhile, the intended typing of QAPISourceInfo was (non-optional) 'str'. To remove this, I'd want to avoid having a "fake" QAPISourceInfo object. We also don't want to explicitly begin accommodating QAPISourceInfo itself being None, because we actually want to eventually prove that this can never happen -- We don't want to confuse "The file isn't open yet" with "This error stems from a definition that wasn't defined in any file". (An earlier series tried to create a dummy info object, but it was tough to prove in review that it worked correctly without creating new regressions. This patch avoids that distraction. We would like to first prove that we never raise QAPISemError for any built-in object before we add "special" info objects. We aren't ready to do that yet.) So, which way out of the labyrinth? Here's one way: Don't try to handle errors at a level with "mixed" semantic contexts; i.e. don't mix inclusion errors (should report a source line where the include was triggered) and command line errors (where we specified a file we couldn't read). Remove the error handling from the initializer of the parser. Pythonic! Now it's the caller's job to figure out what to do about it. Handle the error in QAPISchemaParser._include() instead, where we can write a targeted error message where we are guaranteed to have an 'info' context to report with. The root level error can similarly move to QAPISchema.__init__(), where we know we'll never have an info context to report with, so we use a more abstract error type. Now the error looks sensible again: > python3 qapi-gen.py 'fake.json' qapi-gen.py: can't read schema file 'fake.json': No such file or directory With these error cases separated, QAPISourceInfo can be solidified as never having placeholder arguments that violate our desired types. Clean up test-qapi along similar lines. Fixes: 52a474180a Signed-off-by: John Snow --- scripts/qapi/parser.py | 18 +++++++++--------- scripts/qapi/schema.py | 11 +++++++++-- scripts/qapi/source.py | 3 --- tests/qapi-schema/test-qapi.py | 3 --- 4 files changed, 18 insertions(+), 17 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ca5e8e18e00..a53b735e7de 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -40,15 +40,9 @@ def __init__(self, fname, previously_included=None, incl_info=None): previously_included = previously_included or set() previously_included.add(os.path.abspath(fname)) - try: - fp = open(fname, 'r', encoding='utf-8') + # May raise OSError; allow the caller to handle it. + with open(fname, 'r', encoding='utf-8') as fp: self.src = fp.read() - except IOError as e: - raise QAPISemError(incl_info or QAPISourceInfo(None, None, None), - "can't read %s file '%s': %s" - % ("include" if incl_info else "schema", - fname, - e.strerror)) if self.src == '' or self.src[-1] != '\n': self.src += '\n' @@ -129,7 +123,13 @@ def _include(self, include, info, incl_fname, previously_included): if incl_abs_fname in previously_included: return None - return QAPISchemaParser(incl_fname, previously_included, info) + try: + return QAPISchemaParser(incl_fname, previously_included, info) + except OSError as err: + raise QAPISemError( + info, + f"can't read include file '{incl_fname}': {err.strerror}" + ) from err def _check_pragma_list_of_str(self, name, value, info): if (not isinstance(value, list) diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py index 3a4172fb749..d1d27ff7ee8 100644 --- a/scripts/qapi/schema.py +++ b/scripts/qapi/schema.py @@ -20,7 +20,7 @@ from typing import Optional from .common import POINTER_SUFFIX, c_name -from .error import QAPISemError, QAPISourceError +from .error import QAPIError, QAPISemError, QAPISourceError from .expr import check_exprs from .parser import QAPISchemaParser @@ -849,7 +849,14 @@ def visit(self, visitor): class QAPISchema: def __init__(self, fname): self.fname = fname - parser = QAPISchemaParser(fname) + + try: + parser = QAPISchemaParser(fname) + except OSError as err: + raise QAPIError( + f"can't read schema file '{fname}': {err.strerror}" + ) from err + exprs = check_exprs(parser.exprs) self.docs = parser.docs self._entity_list = [] diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py index 03b6ede0828..1ade864d7b9 100644 --- a/scripts/qapi/source.py +++ b/scripts/qapi/source.py @@ -10,7 +10,6 @@ # See the COPYING file in the top-level directory. import copy -import sys from typing import List, Optional, TypeVar @@ -53,8 +52,6 @@ def next_line(self: T) -> T: return info def loc(self) -> str: - if self.fname is None: - return sys.argv[0] ret = self.fname if self.line is not None: ret += ':%d' % self.line diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py index e8db9d09d91..f1c4deb9a51 100755 --- a/tests/qapi-schema/test-qapi.py +++ b/tests/qapi-schema/test-qapi.py @@ -128,9 +128,6 @@ def test_and_diff(test_name, dir_name, update): try: test_frontend(os.path.join(dir_name, test_name + '.json')) except QAPIError as err: - if err.info.fname is None: - print("%s" % err, file=sys.stderr) - return 2 errstr = str(err) + '\n' if dir_name: errstr = errstr.replace(dir_name + '/', '') From patchwork Tue May 11 22:05:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477332 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O0ksdT6x; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsXG4770z9sPf for ; Wed, 12 May 2021 08:09:18 +1000 (AEST) Received: from localhost ([::1]:49346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaZ2-0004s9-FT for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:09:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55658) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW5-0007NK-UK for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:51549) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW1-0006Nd-RB for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770768; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Zw/1IFeRgebH6yrdkTw5SelFOgt1TKw/afQTI5DhMA=; b=O0ksdT6xIVR9kpF+w/Nip09o9oaalyjN/GD+ajUtqGOpKf4apXH+6QK2O8MqE0U7qoJV29 2XE4taLUNlhvbu7p8X4MOVNcyosBxDqUOJ3lwD5m8PO1GNafxOGSXtedgsf7PoNnkY+FWC 685KvdbAlLDEKNAiBsO8NCvSiIBEqY4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-571-amMImJB_Nf2-hMW46Km4hA-1; Tue, 11 May 2021 18:06:05 -0400 X-MC-Unique: amMImJB_Nf2-hMW46Km4hA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 26187101371D; Tue, 11 May 2021 22:06:04 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D46360CC9; Tue, 11 May 2021 22:06:03 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 02/21] qapi: Add test for nonexistent schema file Date: Tue, 11 May 2021 18:05:42 -0400 Message-Id: <20210511220601.2110055-3-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This tests the error-return pathway introduced in the previous commit. (Thanks to Paolo for the help with the Meson magic.) Signed-off-by: John Snow --- This went after the previous patch instead of before because prior to removing the sys.argv[0] bit from QAPISourceInfo, I can't filter the test to pass the diff. Instead of writing something new to get a better patch ordering, just add the test after. Signed-off-by: John Snow --- tests/qapi-schema/meson.build | 7 ++++++- tests/qapi-schema/missing-schema.err | 1 + tests/qapi-schema/missing-schema.out | 0 3 files changed, 7 insertions(+), 1 deletion(-) create mode 100644 tests/qapi-schema/missing-schema.err create mode 100644 tests/qapi-schema/missing-schema.out diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build index d7163e6601c..dc448e8f74d 100644 --- a/tests/qapi-schema/meson.build +++ b/tests/qapi-schema/meson.build @@ -199,11 +199,16 @@ schemas = [ 'unknown-escape.json', 'unknown-expr-key.json', ] +schemas = files(schemas) + +# Intentionally missing schema file test -- not passed through files(): +schemas += [meson.current_source_dir() / 'missing-schema.json'] # Because people may want to use test-qapi.py from the command line, we # are not using the "#! /usr/bin/env python3" trick here. See # docs/devel/build-system.txt -test('QAPI schema regression tests', python, args: files('test-qapi.py', schemas), +test('QAPI schema regression tests', python, + args: files('test-qapi.py') + schemas, env: test_env, suite: ['qapi-schema', 'qapi-frontend']) diff = find_program('diff') diff --git a/tests/qapi-schema/missing-schema.err b/tests/qapi-schema/missing-schema.err new file mode 100644 index 00000000000..b4d9ff1fb2b --- /dev/null +++ b/tests/qapi-schema/missing-schema.err @@ -0,0 +1 @@ +can't read schema file 'missing-schema.json': No such file or directory diff --git a/tests/qapi-schema/missing-schema.out b/tests/qapi-schema/missing-schema.out new file mode 100644 index 00000000000..e69de29bb2d From patchwork Tue May 11 22:05:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KoW1IDw1; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsXF3btMz9sWC for ; Wed, 12 May 2021 08:09:17 +1000 (AEST) Received: from localhost ([::1]:49288 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaZ1-0004pb-Gu for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:09:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55664) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW6-0007Nw-8q for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48259) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW1-0006O1-Uk for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770769; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XcLCBJp2GifzvxOzMbM4f59L5eLSU4faCfjRnldxOV4=; b=KoW1IDw1ulf81Cj9X3a/fdfPRSGFXFgkPXoEhpp2npbOlQrWLIpjyPnTW+xum3zzDR4ALd chGe8RSbR5NjgxU1ddewi5pvfyJqjrmZSEG7t0IELLTKeKgd+b2ziaybPTRuoFXRI4u74t FcQeHZzwUCwAFvFSuMhUzH9g7o9yFpw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-439-F9KTpqmpOzOl8ESNw8s3nA-1; Tue, 11 May 2021 18:06:05 -0400 X-MC-Unique: F9KTpqmpOzOl8ESNw8s3nA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F40D81854E21; Tue, 11 May 2021 22:06:04 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A33ABA6F; Tue, 11 May 2021 22:06:04 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 03/21] qapi/source: Remove line number from QAPISourceInfo initializer Date: Tue, 11 May 2021 18:05:43 -0400 Message-Id: <20210511220601.2110055-4-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" With the QAPISourceInfo(None, None, None) construct gone, there's no longer any reason to have to specify that a file starts on the first line. Remove it from the initializer and default it to 1. Remove the last vestiges where we check for 'line' being unset, that can't happen, now. Signed-off-by: John Snow --- scripts/qapi/parser.py | 2 +- scripts/qapi/source.py | 10 +++------- 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index a53b735e7de..39dbcc4eacc 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -47,7 +47,7 @@ def __init__(self, fname, previously_included=None, incl_info=None): if self.src == '' or self.src[-1] != '\n': self.src += '\n' self.cursor = 0 - self.info = QAPISourceInfo(fname, 1, incl_info) + self.info = QAPISourceInfo(fname, incl_info) self.line_pos = 0 self.exprs = [] self.docs = [] diff --git a/scripts/qapi/source.py b/scripts/qapi/source.py index 1ade864d7b9..04193cc9643 100644 --- a/scripts/qapi/source.py +++ b/scripts/qapi/source.py @@ -31,10 +31,9 @@ def __init__(self) -> None: class QAPISourceInfo: T = TypeVar('T', bound='QAPISourceInfo') - def __init__(self, fname: str, line: int, - parent: Optional['QAPISourceInfo']): + def __init__(self, fname: str, parent: Optional['QAPISourceInfo']): self.fname = fname - self.line = line + self.line = 1 self.parent = parent self.pragma: QAPISchemaPragma = ( parent.pragma if parent else QAPISchemaPragma() @@ -52,10 +51,7 @@ def next_line(self: T) -> T: return info def loc(self) -> str: - ret = self.fname - if self.line is not None: - ret += ':%d' % self.line - return ret + return f"{self.fname}:{self.line}" def in_defn(self) -> str: if self.defn_name: From patchwork Tue May 11 22:05:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477329 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NIe1lizN; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsTL1nVFz9sWB for ; Wed, 12 May 2021 08:06:46 +1000 (AEST) Received: from localhost ([::1]:40936 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaWa-0007Xw-6M for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:06:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55632) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW4-0007M6-QG for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54166) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW2-0006O3-5f for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770769; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nAUt9ep2U9+NRGrcfgncJ14WoGsv9dOEug/5liuMNFs=; b=NIe1lizNChlWw37Zc26FBL4zetaM/tCoX0OkKIoBK1+1iDVh0/vgvQdNovV15OhMbFqtZn lRCXpDk4onziZmdTvviPPq4f9dfQH77WZMNIGB0x62J5ZJReLfBj+O36qPF/qqwnKXaG8D i7YhxvwLNQk1IIyLFOWhpPitweQuNBE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-372-PpaUFrZ2M62VCTA3t9dhIw-1; Tue, 11 May 2021 18:06:06 -0400 X-MC-Unique: PpaUFrZ2M62VCTA3t9dhIw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BC626107ACE3; Tue, 11 May 2021 22:06:05 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20FDB60CC9; Tue, 11 May 2021 22:06:05 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 04/21] qapi/parser: factor parsing routine into method Date: Tue, 11 May 2021 18:05:44 -0400 Message-Id: <20210511220601.2110055-5-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For the sake of keeping __init__ smaller (and treating it more like a gallery of what state variables we can expect to see), put the actual parsing action into a parse method. It remains invoked from the init method to reduce churn. To accomplish this, 'previously_included' becomes the private data member '_included', and the filename is stashed as _fname. Add any missing declarations to the init method, and group them by function so they can be understood quickly at a glance. Signed-off-by: John Snow --- scripts/qapi/parser.py | 40 ++++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 39dbcc4eacc..d620706fffb 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -37,23 +37,39 @@ def __init__(self, parser, msg): class QAPISchemaParser: def __init__(self, fname, previously_included=None, incl_info=None): - previously_included = previously_included or set() - previously_included.add(os.path.abspath(fname)) + self._fname = fname + self._included = previously_included or set() + self._included.add(os.path.abspath(self._fname)) + self.src = '' - # May raise OSError; allow the caller to handle it. - with open(fname, 'r', encoding='utf-8') as fp: - self.src = fp.read() - - if self.src == '' or self.src[-1] != '\n': - self.src += '\n' + # Lexer state (see `accept` for details): + self.info = QAPISourceInfo(self._fname, incl_info) + self.tok = None + self.pos = 0 self.cursor = 0 - self.info = QAPISourceInfo(fname, incl_info) + self.val = None self.line_pos = 0 + + # Parser output: self.exprs = [] self.docs = [] - self.accept() + + # Showtime! + self._parse() + + def _parse(self): cur_doc = None + # May raise OSError; allow the caller to handle it. + with open(self._fname, 'r', encoding='utf-8') as fp: + self.src = fp.read() + if self.src == '' or self.src[-1] != '\n': + self.src += '\n' + + # Prime the lexer: + self.accept() + + # Parse until done: while self.tok is not None: info = self.info if self.tok == '#': @@ -71,12 +87,12 @@ def __init__(self, fname, previously_included=None, incl_info=None): if not isinstance(include, str): raise QAPISemError(info, "value of 'include' must be a string") - incl_fname = os.path.join(os.path.dirname(fname), + incl_fname = os.path.join(os.path.dirname(self._fname), include) self.exprs.append({'expr': {'include': incl_fname}, 'info': info}) exprs_include = self._include(include, info, incl_fname, - previously_included) + self._included) if exprs_include: self.exprs.extend(exprs_include.exprs) self.docs.extend(exprs_include.docs) From patchwork Tue May 11 22:05:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477330 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OZsVGFD6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsXF0tx6z9sPf for ; Wed, 12 May 2021 08:09:16 +1000 (AEST) Received: from localhost ([::1]:49184 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaZ0-0004lg-SF for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:09:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55628) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW4-0007M4-Lc for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:55019) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW1-0006Ni-Rv for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770769; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z4guPBVDf8grv06qt3sxGquVrp6UojkYqZbZmCbkODc=; b=OZsVGFD6tItbziVucvj9/soLvXrhdV4BFxjw4q7GMGOF681bfXA+rXCu1uDwihezipDm7m q+9bKubcJ9GEwBSL6j/Mrrx8KBV1UBDeLqSK6TVN6XNQs91LpRWippS7PkKVNhzDkjoVfQ Hz8seyTM7ELBS+dZQiWGEEk9P+77kdE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-143-TSdUsqT9MMK540AvssFdVw-1; Tue, 11 May 2021 18:06:07 -0400 X-MC-Unique: TSdUsqT9MMK540AvssFdVw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A481D800FF0; Tue, 11 May 2021 22:06:06 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE29560CC9; Tue, 11 May 2021 22:06:05 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 05/21] qapi/parser: Assert lexer value is a string Date: Tue, 11 May 2021 18:05:45 -0400 Message-Id: <20210511220601.2110055-6-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The type checker can't narrow the type of the token value to string, because it's only loosely correlated with the return token. We know that a token of '#' should always have a "str" value. Add an assertion. Signed-off-by: John Snow --- scripts/qapi/parser.py | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index d620706fffb..ba17f1357ad 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -304,6 +304,7 @@ def get_doc(self, info): cur_doc = QAPIDoc(self, info) self.accept(False) while self.tok == '#': + assert isinstance(self.val, str) # comment token MUST have str val if self.val.startswith('##'): # End of doc comment if self.val != '##': From patchwork Tue May 11 22:05:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477326 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=O7kCd/gK; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsTK0VnBz9sWB for ; Wed, 12 May 2021 08:06:43 +1000 (AEST) Received: from localhost ([::1]:40722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaWX-0007P5-3q for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:06:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55642) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW5-0007MJ-41 for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW3-0006PB-EA for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770770; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MDjVSiuSSGwztDqXk3c2sk74qecBq7DNE6dMOBfqomI=; b=O7kCd/gKNNyB7B9GQuLxXkO+lxDzX+slwMXSxPPtALrplF2+UyBtcX10vLFEKXahL2E++O 4/ctW8dbXz+VxP0T6vStqaO7jNz4Ws37tRPp3a7ySkCmjvMJU9LSpI0LdcMaxPXVRqp9/H rqefpw2swBXUiJzD8Q/9A6TQ1MZSy7w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-233-MQtRNZ9dOOWm7RfGomsZnQ-1; Tue, 11 May 2021 18:06:08 -0400 X-MC-Unique: MQtRNZ9dOOWm7RfGomsZnQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D83E2FC80; Tue, 11 May 2021 22:06:07 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id E46AABA6F; Tue, 11 May 2021 22:06:06 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 06/21] qapi/parser: enforce all top-level expressions must be dict in _parse() Date: Tue, 11 May 2021 18:05:46 -0400 Message-Id: <20210511220601.2110055-7-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Instead of using get_expr nested=False, allow get_expr to always return any expression. In exchange, add a new error message to the top-level parser that explains the semantic error: Top-level expressions must always be JSON objects. This helps mypy understand the rest of this function which assumes that get_expr did indeed return a dict. The exception type changes from QAPIParseError to QAPISemError as a result, and the error message in two tests now changes. Signed-off-by: John Snow --- Thanks Markus, I like this quite a bit better. I think I swung the pendulum back too far away from "try not to change anything". This is cleaner. Signed-off-by: John Snow --- scripts/qapi/parser.py | 14 ++++++++------ tests/qapi-schema/non-objects.err | 2 +- tests/qapi-schema/quoted-structural-chars.err | 2 +- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ba17f1357ad..d554b5485a6 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -78,7 +78,11 @@ def _parse(self): self.docs.append(cur_doc) continue - expr = self.get_expr(False) + expr = self.get_expr() + if not isinstance(expr, dict): + raise QAPISemError( + info, "top-level expression must be an object") + if 'include' in expr: self.reject_expr_doc(cur_doc) if len(expr) != 1: @@ -251,7 +255,7 @@ def get_members(self): self.accept() if key in expr: raise QAPIParseError(self, "duplicate key '%s'" % key) - expr[key] = self.get_expr(True) + expr[key] = self.get_expr() if self.tok == '}': self.accept() return expr @@ -270,7 +274,7 @@ def get_values(self): raise QAPIParseError( self, "expected '{', '[', ']', string, or boolean") while True: - expr.append(self.get_expr(True)) + expr.append(self.get_expr()) if self.tok == ']': self.accept() return expr @@ -278,9 +282,7 @@ def get_values(self): raise QAPIParseError(self, "expected ',' or ']'") self.accept() - def get_expr(self, nested): - if self.tok != '{' and not nested: - raise QAPIParseError(self, "expected '{'") + def get_expr(self): if self.tok == '{': self.accept() expr = self.get_members() diff --git a/tests/qapi-schema/non-objects.err b/tests/qapi-schema/non-objects.err index 3a4ea36966e..23bdb69c711 100644 --- a/tests/qapi-schema/non-objects.err +++ b/tests/qapi-schema/non-objects.err @@ -1 +1 @@ -non-objects.json:1:1: expected '{' +non-objects.json:1: top-level expression must be an object diff --git a/tests/qapi-schema/quoted-structural-chars.err b/tests/qapi-schema/quoted-structural-chars.err index 07d1561d1f7..af6c1e173db 100644 --- a/tests/qapi-schema/quoted-structural-chars.err +++ b/tests/qapi-schema/quoted-structural-chars.err @@ -1 +1 @@ -quoted-structural-chars.json:1:1: expected '{' +quoted-structural-chars.json:1: top-level expression must be an object From patchwork Tue May 11 22:05:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477334 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gfst/u7C; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsZl2gwlz9sPf for ; Wed, 12 May 2021 08:11:27 +1000 (AEST) Received: from localhost ([::1]:57798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgab7-0002JL-BL for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:11:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW7-0007QV-8V for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28350) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW3-0006PL-Ul for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aMjGy3T5CZh1IFB/T9rfdu3J+isxuMjRFjNguqzECBs=; b=gfst/u7CLaEfWr4qKtqVHBvf+bBTNlcXbOQYrVPpZQo0WcR2Tlm6VdTTvSOx50GAr1TbqA vD5lFC1rvwwc4ZaYy2oCD9X0UfExKFvvQPeS45v1f68bhoQfq9WlQpM/7J0q6BxWicIGN2 mVYXrTxZ8niRWp4cpU6O1SOraF2fcFg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-598-SvUJ4OnYNQS9D_-1wYv0fQ-1; Tue, 11 May 2021 18:06:09 -0400 X-MC-Unique: SvUJ4OnYNQS9D_-1wYv0fQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id ED4AB803620; Tue, 11 May 2021 22:06:08 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 07F6DBA6F; Tue, 11 May 2021 22:06:07 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 07/21] qapi/parser: assert object keys are strings Date: Tue, 11 May 2021 18:05:47 -0400 Message-Id: <20210511220601.2110055-8-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The single quote token implies the value is a string. Assert this to be the case, to allow us to write an accurate return type for get_members. Signed-off-by: John Snow --- scripts/qapi/parser.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index d554b5485a6..6d774df6d0a 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -249,6 +249,8 @@ def get_members(self): raise QAPIParseError(self, "expected string or '}'") while True: key = self.val + assert isinstance(key, str) # Guaranteed by tok == "'" + self.accept() if self.tok != ':': raise QAPIParseError(self, "expected ':'") From patchwork Tue May 11 22:05:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477339 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Xeub8yi7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Ffsdb6TBxz9sVt for ; Wed, 12 May 2021 08:13:55 +1000 (AEST) Received: from localhost ([::1]:38308 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgadV-0008L0-Vb for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:13:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55726) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWA-0007bN-Qy for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32273) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW7-0006Rj-4o for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+JIoWxNMsAa0ulLHBM/p3bpvL1JlQ8H8BCE1dJY31+o=; b=Xeub8yi78ygHpIwO9/rjTphaUD7Lqhum9qMv73da8ffmTHli2Tm0l1R012B9BOvo9bl7ws euQG199GO2hW4OUaVTS2VSUaiwRvfMVOas47ttEl7dY5xiMhEFqJU7fLe1Uwzvc6kz5qms iXodu/EJHWII9+IO3oKyKnpJfYn9zzI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-560-KTTw9FRRMHOtoR5Jk2y09A-1; Tue, 11 May 2021 18:06:11 -0400 X-MC-Unique: KTTw9FRRMHOtoR5Jk2y09A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50041101371C; Tue, 11 May 2021 22:06:10 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 19E32BA6F; Tue, 11 May 2021 22:06:09 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 08/21] qapi/parser: Use @staticmethod where appropriate Date: Tue, 11 May 2021 18:05:48 -0400 Message-Id: <20210511220601.2110055-9-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" No self, no thank you! (Quiets pylint warnings.) Signed-off-by: John Snow --- scripts/qapi/parser.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 6d774df6d0a..5ef1b8935e6 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -130,7 +130,8 @@ def reject_expr_doc(doc): "documentation for '%s' is not followed by the definition" % doc.symbol) - def _include(self, include, info, incl_fname, previously_included): + @staticmethod + def _include(include, info, incl_fname, previously_included): incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle inf = info @@ -151,7 +152,8 @@ def _include(self, include, info, incl_fname, previously_included): f"can't read include file '{incl_fname}': {err.strerror}" ) from err - def _check_pragma_list_of_str(self, name, value, info): + @staticmethod + def _check_pragma_list_of_str(name, value, info): if (not isinstance(value, list) or any([not isinstance(elt, str) for elt in value])): raise QAPISemError( From patchwork Tue May 11 22:05:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477336 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=b6h4Jv6z; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsZn6Qx7z9sWC for ; Wed, 12 May 2021 08:11:29 +1000 (AEST) Received: from localhost ([::1]:58002 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgab9-0002Ru-SV for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:11:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55702) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW8-0007Uz-UD for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44088) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW6-0006Rd-Ob for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nOQsNt8acyCnBU2ptkWNXXNpOfhla1lorwI4QOI0Pc=; b=b6h4Jv6zFb/tmP+UCa7ZOqb3Vp5jRGyVqPBqK6w/6hGr2S4pxyREKSbSbb7Joef+BOZDKa kuFghMEB8eN1ZfoSX+tpTmUMpm3loVl8LsozQHsMBvZTqCmehH8OsbVaXEIs0A5DEtQeaA Jc7dK/r3+rHaJZWcEzDlHhCAe3oe8/0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-172-7nbL9VY_P4qLoGJU-ooZRw-1; Tue, 11 May 2021 18:06:12 -0400 X-MC-Unique: 7nbL9VY_P4qLoGJU-ooZRw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B1553101371B; Tue, 11 May 2021 22:06:11 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 94BD960CC9; Tue, 11 May 2021 22:06:10 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 09/21] qapi: add must_match helper Date: Tue, 11 May 2021 18:05:49 -0400 Message-Id: <20210511220601.2110055-10-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Mypy cannot generally understand that these regex functions cannot possibly fail. Add a "must_match" helper that makes this clear for mypy. Signed-off-by: John Snow --- scripts/qapi/common.py | 8 +++++++- scripts/qapi/main.py | 6 ++---- scripts/qapi/parser.py | 13 +++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py index cbd3fd81d36..6ad1eeb61d4 100644 --- a/scripts/qapi/common.py +++ b/scripts/qapi/common.py @@ -12,7 +12,7 @@ # See the COPYING file in the top-level directory. import re -from typing import Optional, Sequence +from typing import Match, Optional, Sequence #: Magic string that gets removed along with all space to its right. @@ -210,3 +210,9 @@ def gen_endif(ifcond: Sequence[str]) -> str: #endif /* %(cond)s */ ''', cond=ifc) return ret + + +def must_match(pattern: str, string: str) -> Match[str]: + match = re.match(pattern, string) + assert match is not None + return match diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py index 703e7ed1ed5..f2ea6e0ce4a 100644 --- a/scripts/qapi/main.py +++ b/scripts/qapi/main.py @@ -8,11 +8,11 @@ """ import argparse -import re import sys from typing import Optional from .commands import gen_commands +from .common import must_match from .error import QAPIError from .events import gen_events from .introspect import gen_introspect @@ -22,9 +22,7 @@ def invalid_prefix_char(prefix: str) -> Optional[str]: - match = re.match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix) - # match cannot be None, but mypy cannot infer that. - assert match is not None + match = must_match(r'([A-Za-z_.-][A-Za-z0-9_.-]*)?', prefix) if match.end() != len(prefix): return prefix[match.end()] return None diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 5ef1b8935e6..39de24785ac 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -18,6 +18,7 @@ import os import re +from .common import must_match from .error import QAPISemError, QAPISourceError from .source import QAPISourceInfo @@ -238,8 +239,8 @@ def accept(self, skip_comment=True): elif not self.tok.isspace(): # Show up to next structural, whitespace or quote # character - match = re.match('[^[\\]{}:,\\s\'"]+', - self.src[self.cursor-1:]) + match = must_match('[^[\\]{}:,\\s\'"]+', + self.src[self.cursor-1:]) raise QAPIParseError(self, "stray '%s'" % match.group(0)) def get_members(self): @@ -369,7 +370,7 @@ def append(self, line): # Strip leading spaces corresponding to the expected indent level # Blank lines are always OK. if line: - indent = re.match(r'\s*', line).end() + indent = must_match(r'\s*', line).end() if indent < self._indent: raise QAPIParseError( self._parser, @@ -505,7 +506,7 @@ def _append_args_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent = re.match(r'@\S*:\s*', line).end() + indent = must_match(r'@\S*:\s*', line).end() line = line[indent:] if not line: # Line was just the "@arg:" header; following lines @@ -540,7 +541,7 @@ def _append_features_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent = re.match(r'@\S*:\s*', line).end() + indent = must_match(r'@\S*:\s*', line).end() line = line[indent:] if not line: # Line was just the "@arg:" header; following lines @@ -586,7 +587,7 @@ def _append_various_line(self, line): # from line and replace it with spaces so that 'f' has the # same index as it did in the original line and can be # handled the same way we will handle following lines. - indent = re.match(r'\S*:\s*', line).end() + indent = must_match(r'\S*:\s*', line).end() line = line[indent:] if not line: # Line was just the "Section:" header; following lines From patchwork Tue May 11 22:05:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Dn1M8LI1; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsYW0q2Hz9sPf for ; Wed, 12 May 2021 08:10:23 +1000 (AEST) Received: from localhost ([::1]:52924 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaa5-0007L8-53 for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:10:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWA-0007Z4-6U for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:42498) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW8-0006SS-5s for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770775; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rIOtzahELHFPo7dqKUXPYk1QumpQPPrSmIDVsNa1JFI=; b=Dn1M8LI1xt2MjMPEK44Z2/nbeiIDuE08nRPkaF3OqaDEnk43cN8SbQ7aWXD3MD3dlwsSiW L6+gWKG4YSc7qa24pZJ9MyJ7prIM+4Pn0kBEkupCzTNwfjFn+VIm9bsfxihz0RcjrPKauT UzqcV4lKT5vFr5ytog1EHsO1+tYM7/w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-455-jilDcniJPN6gg22fxyFniw-1; Tue, 11 May 2021 18:06:13 -0400 X-MC-Unique: jilDcniJPN6gg22fxyFniw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7B199107ACE4; Tue, 11 May 2021 22:06:12 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4B8D60CC9; Tue, 11 May 2021 22:06:11 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 10/21] qapi/parser: Fix token membership tests when token can be None Date: Tue, 11 May 2021 18:05:50 -0400 Message-Id: <20210511220601.2110055-11-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When the token can be None (EOF), we can't use 'x in "abc"' style membership tests to group types of tokens together, because 'None in "abc"' is a TypeError. Easy enough to fix. (Use a tuple: It's neither a static typing error nor a runtime error to check for None in Tuple[str, ...]) Add tests to prevent a regression. (Note: they cannot be added prior to this fix, as the unhandled stack trace will not match test output in the CI system.) Signed-off-by: John Snow --- scripts/qapi/parser.py | 5 +++-- tests/qapi-schema/meson.build | 2 ++ tests/qapi-schema/missing-array-rsqb.err | 1 + tests/qapi-schema/missing-array-rsqb.json | 1 + tests/qapi-schema/missing-array-rsqb.out | 0 tests/qapi-schema/missing-object-member-element.err | 1 + tests/qapi-schema/missing-object-member-element.json | 1 + tests/qapi-schema/missing-object-member-element.out | 0 8 files changed, 9 insertions(+), 2 deletions(-) create mode 100644 tests/qapi-schema/missing-array-rsqb.err create mode 100644 tests/qapi-schema/missing-array-rsqb.json create mode 100644 tests/qapi-schema/missing-array-rsqb.out create mode 100644 tests/qapi-schema/missing-object-member-element.err create mode 100644 tests/qapi-schema/missing-object-member-element.json create mode 100644 tests/qapi-schema/missing-object-member-element.out diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 39de24785ac..959214b7042 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -275,7 +275,7 @@ def get_values(self): if self.tok == ']': self.accept() return expr - if self.tok not in "{['tf": + if self.tok not in tuple("{['tf"): raise QAPIParseError( self, "expected '{', '[', ']', string, or boolean") while True: @@ -294,7 +294,8 @@ def get_expr(self): elif self.tok == '[': self.accept() expr = self.get_values() - elif self.tok in "'tf": + elif self.tok in tuple("'tf"): + assert isinstance(self.val, (str, bool)) expr = self.val self.accept() else: diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build index dc448e8f74d..9e8f658ce38 100644 --- a/tests/qapi-schema/meson.build +++ b/tests/qapi-schema/meson.build @@ -134,9 +134,11 @@ schemas = [ 'indented-expr.json', 'leading-comma-list.json', 'leading-comma-object.json', + 'missing-array-rsqb.json', 'missing-colon.json', 'missing-comma-list.json', 'missing-comma-object.json', + 'missing-object-member-element.json', 'missing-type.json', 'nested-struct-data.json', 'nested-struct-data-invalid-dict.json', diff --git a/tests/qapi-schema/missing-array-rsqb.err b/tests/qapi-schema/missing-array-rsqb.err new file mode 100644 index 00000000000..b5f58b8c12a --- /dev/null +++ b/tests/qapi-schema/missing-array-rsqb.err @@ -0,0 +1 @@ +missing-array-rsqb.json:1:44: expected '{', '[', string, or boolean diff --git a/tests/qapi-schema/missing-array-rsqb.json b/tests/qapi-schema/missing-array-rsqb.json new file mode 100644 index 00000000000..7fca1df923c --- /dev/null +++ b/tests/qapi-schema/missing-array-rsqb.json @@ -0,0 +1 @@ +['Daisy,', 'Daisy,', 'Give me your answer', diff --git a/tests/qapi-schema/missing-array-rsqb.out b/tests/qapi-schema/missing-array-rsqb.out new file mode 100644 index 00000000000..e69de29bb2d diff --git a/tests/qapi-schema/missing-object-member-element.err b/tests/qapi-schema/missing-object-member-element.err new file mode 100644 index 00000000000..c08a3dc307f --- /dev/null +++ b/tests/qapi-schema/missing-object-member-element.err @@ -0,0 +1 @@ +missing-object-member-element.json:1:8: expected '{', '[', string, or boolean diff --git a/tests/qapi-schema/missing-object-member-element.json b/tests/qapi-schema/missing-object-member-element.json new file mode 100644 index 00000000000..f52d0106f31 --- /dev/null +++ b/tests/qapi-schema/missing-object-member-element.json @@ -0,0 +1 @@ +{'key': diff --git a/tests/qapi-schema/missing-object-member-element.out b/tests/qapi-schema/missing-object-member-element.out new file mode 100644 index 00000000000..e69de29bb2d From patchwork Tue May 11 22:05:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477335 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=W13HcbCu; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsZn4jH7z9sPf for ; Wed, 12 May 2021 08:11:29 +1000 (AEST) Received: from localhost ([::1]:57926 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgab9-0002OU-D0 for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:11:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWB-0007cI-3E for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25283) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaW9-0006Sw-50 for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5jkc+pwAbDAwS7ehd+FT7G+egBnKuVjGtJdssp+n+1s=; b=W13HcbCuDUqqU4cED3Rg6XLEAulc5hcLK0RYMh1cVC09OaYVxlYPUplJzDJYUsEFJidKVg 1MAUGEoTtT7ouTodq5SXtH7odQSKgwxxZma9wgzyQzxPruVWqJ3151aCJK/l1t224FyXce lzlQzFAYzIxZxhuyEyf0TrFJJSqGEIY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-522-4Q5GWR33OhS4YMvYRLzTpA-1; Tue, 11 May 2021 18:06:14 -0400 X-MC-Unique: 4Q5GWR33OhS4YMvYRLzTpA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1966B800FF0; Tue, 11 May 2021 22:06:14 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id E2247BA6F; Tue, 11 May 2021 22:06:12 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 11/21] qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard Date: Tue, 11 May 2021 18:05:51 -0400 Message-Id: <20210511220601.2110055-12-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" TypeGuards wont exist in Python proper until 3.10. Ah well. We can hack up our own by declaring this function to return the type we claim it checks for and using this to safely downcast object -> List[str]. In so doing, I bring this function under _pragma so it can use the 'info' object in its closure. Having done this, _pragma also now no longer needs to take a 'self' parameter, so drop it. To help with line-length, and with the context evident from its new scope, rename the function to the shorter check_list_str(). Signed-off-by: John Snow --- I left (name, value) as args to avoid creating a fully magic "macro", because this looked simply too weird: info.pragma.foobar = check_list_str() and it looked more reasonable as: info.pragma.foobar = check_list_str(name, value) Signed-off-by: John Snow --- scripts/qapi/parser.py | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 959214b7042..336959cbbb1 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -17,6 +17,7 @@ from collections import OrderedDict import os import re +from typing import List from .common import must_match from .error import QAPISemError, QAPISourceError @@ -154,28 +155,29 @@ def _include(include, info, incl_fname, previously_included): ) from err @staticmethod - def _check_pragma_list_of_str(name, value, info): - if (not isinstance(value, list) - or any([not isinstance(elt, str) for elt in value])): - raise QAPISemError( - info, - "pragma %s must be a list of strings" % name) + def _pragma(name, value, info): + + def check_list_str(name, value) -> List[str]: + if (not isinstance(value, list) or + any([not isinstance(elt, str) for elt in value])): + raise QAPISemError( + info, + "pragma %s must be a list of strings" % name) + return value + + pragma = info.pragma - def _pragma(self, name, value, info): if name == 'doc-required': if not isinstance(value, bool): raise QAPISemError(info, "pragma 'doc-required' must be boolean") - info.pragma.doc_required = value + pragma.doc_required = value elif name == 'command-name-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.command_name_exceptions = value + pragma.command_name_exceptions = check_list_str(name, value) elif name == 'command-returns-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.command_returns_exceptions = value + pragma.command_returns_exceptions = check_list_str(name, value) elif name == 'member-name-exceptions': - self._check_pragma_list_of_str(name, value, info) - info.pragma.member_name_exceptions = value + pragma.member_name_exceptions = check_list_str(name, value) else: raise QAPISemError(info, "unknown pragma '%s'" % name) From patchwork Tue May 11 22:05:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477340 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YiTsk8oO; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Ffsdj2PPhz9sVt for ; Wed, 12 May 2021 08:14:01 +1000 (AEST) Received: from localhost ([::1]:38320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgadb-0008LT-Bx for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:13:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWC-0007k3-Na for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60665) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWA-0006Ty-L3 for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pkV0GAvbu7qcYa+mswlYzA+gNGbOjdKlXPcM1T77R9U=; b=YiTsk8oOvR6c3UN0rH+Ks+gMq+iufdeB6KHd2WkyzhIdPaqYE/TeXwfLi/+h38BcsoWubn VsEyoVvZf6WSLR8C+YIRIrv/hvfsIZMZOsppV3JgUwIJk6Rt8narsFE/pLkXeSp5DCP9oE 3L7V9EJlc9t4+fLqefVY1z+26gGz8fY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-549-TLy6snXVM1CzTl25ca-WYg-1; Tue, 11 May 2021 18:06:15 -0400 X-MC-Unique: TLy6snXVM1CzTl25ca-WYg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB0DC101371B; Tue, 11 May 2021 22:06:14 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BDA860CC9; Tue, 11 May 2021 22:06:14 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 12/21] qapi/parser: add type hint annotations Date: Tue, 11 May 2021 18:05:52 -0400 Message-Id: <20210511220601.2110055-13-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Annotations do not change runtime behavior. This commit *only* adds annotations. (Annotations for QAPIDoc are in a forthcoming commit.) Signed-off-by: John Snow --- scripts/qapi/parser.py | 58 +++++++++++++++++++++++++++--------------- 1 file changed, 38 insertions(+), 20 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 336959cbbb1..631863bac14 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -17,16 +17,26 @@ from collections import OrderedDict import os import re -from typing import List +from typing import ( + Dict, + List, + Optional, + Set, + Union, +) from .common import must_match from .error import QAPISemError, QAPISourceError from .source import QAPISourceInfo +# Return value alias for get_expr(). +_ExprValue = Union[List[object], Dict[str, object], str, bool] + + class QAPIParseError(QAPISourceError): """Error class for all QAPI schema parsing errors.""" - def __init__(self, parser, msg): + def __init__(self, parser: 'QAPISchemaParser', msg: str): col = 1 for ch in parser.src[parser.line_pos:parser.pos]: if ch == '\t': @@ -38,7 +48,10 @@ def __init__(self, parser, msg): class QAPISchemaParser: - def __init__(self, fname, previously_included=None, incl_info=None): + def __init__(self, + fname: str, + previously_included: Optional[Set[str]] = None, + incl_info: Optional[QAPISourceInfo] = None): self._fname = fname self._included = previously_included or set() self._included.add(os.path.abspath(self._fname)) @@ -46,20 +59,20 @@ def __init__(self, fname, previously_included=None, incl_info=None): # Lexer state (see `accept` for details): self.info = QAPISourceInfo(self._fname, incl_info) - self.tok = None + self.tok: Union[None, str] = None self.pos = 0 self.cursor = 0 - self.val = None + self.val: Optional[Union[bool, str]] = None self.line_pos = 0 # Parser output: - self.exprs = [] - self.docs = [] + self.exprs: List[Dict[str, object]] = [] + self.docs: List[QAPIDoc] = [] # Showtime! self._parse() - def _parse(self): + def _parse(self) -> None: cur_doc = None # May raise OSError; allow the caller to handle it. @@ -125,7 +138,7 @@ def _parse(self): self.reject_expr_doc(cur_doc) @staticmethod - def reject_expr_doc(doc): + def reject_expr_doc(doc: Optional['QAPIDoc']) -> None: if doc and doc.symbol: raise QAPISemError( doc.info, @@ -133,10 +146,14 @@ def reject_expr_doc(doc): % doc.symbol) @staticmethod - def _include(include, info, incl_fname, previously_included): + def _include(include: str, + info: QAPISourceInfo, + incl_fname: str, + previously_included: Set[str] + ) -> Optional['QAPISchemaParser']: incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle - inf = info + inf: Optional[QAPISourceInfo] = info while inf: if incl_abs_fname == os.path.abspath(inf.fname): raise QAPISemError(info, "inclusion loop for %s" % include) @@ -155,9 +172,9 @@ def _include(include, info, incl_fname, previously_included): ) from err @staticmethod - def _pragma(name, value, info): + def _pragma(name: str, value: object, info: QAPISourceInfo) -> None: - def check_list_str(name, value) -> List[str]: + def check_list_str(name: str, value: object) -> List[str]: if (not isinstance(value, list) or any([not isinstance(elt, str) for elt in value])): raise QAPISemError( @@ -181,7 +198,7 @@ def check_list_str(name, value) -> List[str]: else: raise QAPISemError(info, "unknown pragma '%s'" % name) - def accept(self, skip_comment=True): + def accept(self, skip_comment: bool = True) -> None: while True: self.tok = self.src[self.cursor] self.pos = self.cursor @@ -245,8 +262,8 @@ def accept(self, skip_comment=True): self.src[self.cursor-1:]) raise QAPIParseError(self, "stray '%s'" % match.group(0)) - def get_members(self): - expr = OrderedDict() + def get_members(self) -> Dict[str, object]: + expr: Dict[str, object] = OrderedDict() if self.tok == '}': self.accept() return expr @@ -272,8 +289,8 @@ def get_members(self): if self.tok != "'": raise QAPIParseError(self, "expected string") - def get_values(self): - expr = [] + def get_values(self) -> List[object]: + expr: List[object] = [] if self.tok == ']': self.accept() return expr @@ -289,7 +306,8 @@ def get_values(self): raise QAPIParseError(self, "expected ',' or ']'") self.accept() - def get_expr(self): + def get_expr(self) -> _ExprValue: + expr: _ExprValue if self.tok == '{': self.accept() expr = self.get_members() @@ -305,7 +323,7 @@ def get_expr(self): self, "expected '{', '[', string, or boolean") return expr - def get_doc(self, info): + def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: if self.val != '##': raise QAPIParseError( self, "junk after '##' at start of documentation comment") From patchwork Tue May 11 22:05:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477337 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=f9nfSG2F; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Ffsbc2LfWz9sXF for ; Wed, 12 May 2021 08:12:12 +1000 (AEST) Received: from localhost ([::1]:33376 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgabq-00050N-93 for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:12:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55766) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWE-0007si-Vr for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWD-0006VS-CW for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hyKKweyV9uhzgTyt4K3hHElvjyVLocPjrgz8In90YuE=; b=f9nfSG2FXL5EA0nwRfmIn7gdo1JP9ffNyP/4N3bD9z1lSDdyap84gJmlgsHOSvmGp2NAH8 1AGRAAbMOfhHxlLYlPuXdQV/+oewysC2tkZjImYf9Lnt6uU9Tzjs05oxhOKudtkRMS6rBI /H83kUXlTaqGXuiTURUoospapiQ44uo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-553-2VhjmAzWO82OKYIf_R5KBg-1; Tue, 11 May 2021 18:06:16 -0400 X-MC-Unique: 2VhjmAzWO82OKYIf_R5KBg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AF64B101371D; Tue, 11 May 2021 22:06:15 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0771260CC9; Tue, 11 May 2021 22:06:14 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 13/21] qapi/parser: Remove superfluous list comprehension Date: Tue, 11 May 2021 18:05:53 -0400 Message-Id: <20210511220601.2110055-14-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" A generator suffices (and quiets a pylint warning). Signed-off-by: John Snow --- scripts/qapi/parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 631863bac14..e80e0a7d965 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -176,7 +176,7 @@ def _pragma(name: str, value: object, info: QAPISourceInfo) -> None: def check_list_str(name: str, value: object) -> List[str]: if (not isinstance(value, list) or - any([not isinstance(elt, str) for elt in value])): + any(not isinstance(elt, str) for elt in value)): raise QAPISemError( info, "pragma %s must be a list of strings" % name) From patchwork Tue May 11 22:05:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477345 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=IwmSOo1Q; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsnP5xNBz9sVt for ; Wed, 12 May 2021 08:20:41 +1000 (AEST) Received: from localhost ([::1]:55492 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgak3-0003Bi-LG for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:20:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWG-0007zg-Hj for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59513) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWE-0006WG-Jh for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770782; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8QEcYwPpiAdzGnQtGtm+YvWw/XHUAr6+X1gfe0OtI4w=; b=IwmSOo1QolWAOrZCpp84/nxR4Ijpa+X+ADRmL6ZWfg24AxtrZ1hpy/M6vQdWxaqG/ifptr iRuDdwYCEUzIEQFwx0NJDJIS6dZtVv9P6ZZhwU3QKZPRpZ3KxH2msvb8Z/c4KQPdXdy0na 8kJ3buNhFlbCpAlbQ0TKeKaA1o7giLU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-289-jLarfbmVNumLYa3bihIZ2w-1; Tue, 11 May 2021 18:06:17 -0400 X-MC-Unique: jLarfbmVNumLYa3bihIZ2w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AAF9F101371E; Tue, 11 May 2021 22:06:16 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2A2EBA6F; Tue, 11 May 2021 22:06:15 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 14/21] qapi/parser: allow 'ch' variable name Date: Tue, 11 May 2021 18:05:54 -0400 Message-Id: <20210511220601.2110055-15-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We can have a two-letter variable name, as a treat. Signed-off-by: John Snow --- I don't want to use 'chr' or 'char', and in context 'ch' works well enough. I will assume that any possible future uses will also be obvious enough. Signed-off-by: John Snow --- scripts/qapi/pylintrc | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc index 88efbf71cb2..c5275d5f59b 100644 --- a/scripts/qapi/pylintrc +++ b/scripts/qapi/pylintrc @@ -43,6 +43,7 @@ good-names=i, _, fp, # fp = open(...) fd, # fd = os.open(...) + ch, [VARIABLES] From patchwork Tue May 11 22:05:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477342 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JtOz8nz7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsjQ6THlz9sVt for ; Wed, 12 May 2021 08:17:13 +1000 (AEST) Received: from localhost ([::1]:46852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgagf-0005ki-JO for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:17:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55770) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWF-0007u0-8N for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47260) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWD-0006VW-Cb for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/bKqAsIFYdKeQjls6RofjHnFqB2hrk6lkmg1CBgA8TA=; b=JtOz8nz7GM7iyBFm5UZg2jk2bUpN0Ko9gMlvJSRUJKwugYrGaEGkWEppOPd+A7wu3XbsU7 MZQMRyW/JeV1B/nAJDER6VYc8VOwXU0jFEmAs/mPR2T64iPKkAQyHe0oAz5+X0Nw43P7+O QLtFM8Onh5yiUHPf6pHX+B3fl2v8QRg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-gdBELCJ5NFC7c6ZH5eYUQw-1; Tue, 11 May 2021 18:06:18 -0400 X-MC-Unique: gdBELCJ5NFC7c6ZH5eYUQw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D43CAFC81; Tue, 11 May 2021 22:06:17 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id DE52DBA6F; Tue, 11 May 2021 22:06:16 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 15/21] qapi/parser: add docstrings Date: Tue, 11 May 2021 18:05:55 -0400 Message-Id: <20210511220601.2110055-16-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: John Snow --- scripts/qapi/parser.py | 68 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index e80e0a7d965..ed543a2b7a4 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -47,7 +47,27 @@ def __init__(self, parser: 'QAPISchemaParser', msg: str): class QAPISchemaParser: + """ + Performs syntactic parsing of a QAPI schema source file. + Parses a JSON-esque schema file, See qapi-code-gen.txt section + "Schema Syntax" for more information. Grammatical validation + is handled later by `expr.check_exprs()`. + + :param fname: Source filename. + :param previously_included: + The absolute pathnames of previously included source files, + if being invoked from another parser. + :param incl_info: + `QAPISourceInfo` belonging to the parent module. + ``None`` implies this is the root module. + + :ivar exprs: Resulting parsed expressions. + :ivar docs: Resulting parsed documentation blocks. + + :raise OSError: For problems opening the root schema document. + :raise QAPIError: For syntactic or semantic parsing errors. + """ def __init__(self, fname: str, previously_included: Optional[Set[str]] = None, @@ -73,6 +93,11 @@ def __init__(self, self._parse() def _parse(self) -> None: + """ + Parse the QAPI schema document. + + :return: None. Results are stored in ``.exprs`` and ``.docs``. + """ cur_doc = None # May raise OSError; allow the caller to handle it. @@ -199,6 +224,49 @@ def check_list_str(name: str, value: object) -> List[str]: raise QAPISemError(info, "unknown pragma '%s'" % name) def accept(self, skip_comment: bool = True) -> None: + """Read and store the next token. + + :param skip_comment: + When false, return COMMENT tokens ("#"). + This is used when reading documentation blocks. + + :return: + None. Several instance attributes are updated instead: + + - ``.tok`` represents the token type. See below for values. + - ``.info`` describes the token's source location. + - ``.val`` is the token's value, if any. See below. + - ``.pos`` is the buffer index of the first character of + the token. + + * Single-character tokens: + + These are "{", "}", ":", ",", "[", and "]". ``.tok`` holds + the single character and ``.val`` is None. + + * Multi-character tokens: + + * COMMENT: + + This token is not normally returned by the lexer, but it can + be when ``skip_comment`` is False. ``.tok`` is "#", and + ``.val`` is a string including all chars until end-of-line, + including the "#" itself. + + * STRING: + + ``.tok`` is "'", the single quote. ``.val`` contains the + string, excluding the surrounding quotes. + + * TRUE and FALSE: + + ``.tok`` is either "t" or "f", ``.val`` will be the + corresponding bool value. + + * EOF: + + ``.tok`` and ``.val`` will both be None at EOF. + """ while True: self.tok = self.src[self.cursor] self.pos = self.cursor From patchwork Tue May 11 22:05:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477344 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=B9V+WYRz; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfslX1HTWz9sVt for ; Wed, 12 May 2021 08:19:04 +1000 (AEST) Received: from localhost ([::1]:52844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaiU-0001M0-7S for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:19:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWG-0007zY-Hv for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35083) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWE-0006WB-E4 for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770781; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZEo10Eg0ZqkzmOrOO561Y1EeBbIUSFERGFxAWWmJjtY=; b=B9V+WYRzFnF/JhX+puQLOc1rj5+V7a1IBo7m3p1gJDC9h0wyGZR+QgKUthfkL+osyJv7KC kWvQIQCAu4Pc3Qaz7eT0HiBCA72GB6KbZQ+We9dF0pJx9TaHfFRQnSfvs9PG200XafHYU+ r12gFghoGLqDUN863QmCRzDLwnDGc1U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-589-EK-mT9bDPP6c18YT5MaWQQ-1; Tue, 11 May 2021 18:06:19 -0400 X-MC-Unique: EK-mT9bDPP6c18YT5MaWQQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A2656101371B; Tue, 11 May 2021 22:06:18 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 037CBBA6F; Tue, 11 May 2021 22:06:17 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 16/21] CHECKPOINT Date: Tue, 11 May 2021 18:05:56 -0400 Message-Id: <20210511220601.2110055-17-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" As of here, parser is actually fully typed, and QAPIDoc is not. Below, there are a few extra patches that "prove" this, but they are not necessarily meant for inclusion. They could theoretically be included anyway, but a few of them would need to be squashed together to ensure our "no intermediate breakages" rule, and a few things would need to be re-ordered. Consider them [RFC], but optional and completely safe to drop. Signed-off-by: John Snow --- scripts/qapi/mypy.ini | 1 + 1 file changed, 1 insertion(+) diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index 54ca4483d6d..d7bbb2dc9c7 100644 --- a/scripts/qapi/mypy.ini +++ b/scripts/qapi/mypy.ini @@ -4,6 +4,7 @@ disallow_untyped_calls = False python_version = 3.6 [mypy-qapi.parser] +# QAPISchemaParser is done, I promise! disallow_untyped_defs = False disallow_incomplete_defs = False check_untyped_defs = False From patchwork Tue May 11 22:05:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477343 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ds4Kz3Rv; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsjQ6pb4z9sWC for ; Wed, 12 May 2021 08:17:13 +1000 (AEST) Received: from localhost ([::1]:46910 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgagh-0005nF-GB for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:17:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55932) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWd-00013E-7A for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:23060) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWY-0006oa-PJ for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cfbQmIiSzWSfTu5pZq4MPmX+jeTNTqinJ5hSicstPoY=; b=Ds4Kz3RvVyLQYPEfaxQiREWCxdNKr37ASPuQgKuXqj7XSJzn6e3QxGeEZQ1lInmxKQkW0E caYpVJvUmJkd8mo6k3YzRg/3lv30+1AdK1/YCeq5JBIO7Wum/xsJ7mq0iMunJQnSG+3GRF /LXWmLbKySzYonRMs6JDwTowcbKpu5w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-256-NR9f4jY7P4qN5acuOLfN6w-1; Tue, 11 May 2021 18:06:20 -0400 X-MC-Unique: NR9f4jY7P4qN5acuOLfN6w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AE3FE107ACC7; Tue, 11 May 2021 22:06:19 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id C72EA60CC9; Tue, 11 May 2021 22:06:18 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 17/21] qapi: [WIP] Rip QAPIDoc out of parser.py Date: Tue, 11 May 2021 18:05:57 -0400 Message-Id: <20210511220601.2110055-18-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This (rather unglamorously) rips QAPIDoc out of parser.py. It does not leave a working solution in its place, opting instead just for code movement. Signed-off-by: John Snow --- scripts/qapi/parser.py | 342 ------------------------------------- scripts/qapi/qapidoc.py | 362 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 362 insertions(+), 342 deletions(-) create mode 100644 scripts/qapi/qapidoc.py diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index ed543a2b7a4..54df1bfd499 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -16,7 +16,6 @@ from collections import OrderedDict import os -import re from typing import ( Dict, List, @@ -424,344 +423,3 @@ def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: self.accept(False) raise QAPIParseError(self, "documentation comment must end with '##'") - - -class QAPIDoc: - """ - A documentation comment block, either definition or free-form - - Definition documentation blocks consist of - - * a body section: one line naming the definition, followed by an - overview (any number of lines) - - * argument sections: a description of each argument (for commands - and events) or member (for structs, unions and alternates) - - * features sections: a description of each feature flag - - * additional (non-argument) sections, possibly tagged - - Free-form documentation blocks consist only of a body section. - """ - - class Section: - def __init__(self, parser, name=None, indent=0): - # parser, for error messages about indentation - self._parser = parser - # optional section name (argument/member or section name) - self.name = name - self.text = '' - # the expected indent level of the text of this section - self._indent = indent - - def append(self, line): - # Strip leading spaces corresponding to the expected indent level - # Blank lines are always OK. - if line: - indent = must_match(r'\s*', line).end() - if indent < self._indent: - raise QAPIParseError( - self._parser, - "unexpected de-indent (expected at least %d spaces)" % - self._indent) - line = line[self._indent:] - - self.text += line.rstrip() + '\n' - - class ArgSection(Section): - def __init__(self, parser, name, indent=0): - super().__init__(parser, name, indent) - self.member = None - - def connect(self, member): - self.member = member - - def __init__(self, parser, info): - # self._parser is used to report errors with QAPIParseError. The - # resulting error position depends on the state of the parser. - # It happens to be the beginning of the comment. More or less - # servicable, but action at a distance. - self._parser = parser - self.info = info - self.symbol = None - self.body = QAPIDoc.Section(parser) - # dict mapping parameter name to ArgSection - self.args = OrderedDict() - self.features = OrderedDict() - # a list of Section - self.sections = [] - # the current section - self._section = self.body - self._append_line = self._append_body_line - - def has_section(self, name): - """Return True if we have a section with this name.""" - for i in self.sections: - if i.name == name: - return True - return False - - def append(self, line): - """ - Parse a comment line and add it to the documentation. - - The way that the line is dealt with depends on which part of - the documentation we're parsing right now: - * The body section: ._append_line is ._append_body_line - * An argument section: ._append_line is ._append_args_line - * A features section: ._append_line is ._append_features_line - * An additional section: ._append_line is ._append_various_line - """ - line = line[1:] - if not line: - self._append_freeform(line) - return - - if line[0] != ' ': - raise QAPIParseError(self._parser, "missing space after #") - line = line[1:] - self._append_line(line) - - def end_comment(self): - self._end_section() - - @staticmethod - def _is_section_tag(name): - return name in ('Returns:', 'Since:', - # those are often singular or plural - 'Note:', 'Notes:', - 'Example:', 'Examples:', - 'TODO:') - - def _append_body_line(self, line): - """ - Process a line of documentation text in the body section. - - If this a symbol line and it is the section's first line, this - is a definition documentation block for that symbol. - - If it's a definition documentation block, another symbol line - begins the argument section for the argument named by it, and - a section tag begins an additional section. Start that - section and append the line to it. - - Else, append the line to the current section. - """ - name = line.split(' ', 1)[0] - # FIXME not nice: things like '# @foo:' and '# @foo: ' aren't - # recognized, and get silently treated as ordinary text - if not self.symbol and not self.body.text and line.startswith('@'): - if not line.endswith(':'): - raise QAPIParseError(self._parser, "line should end with ':'") - self.symbol = line[1:-1] - # FIXME invalid names other than the empty string aren't flagged - if not self.symbol: - raise QAPIParseError(self._parser, "invalid name") - elif self.symbol: - # This is a definition documentation block - if name.startswith('@') and name.endswith(':'): - self._append_line = self._append_args_line - self._append_args_line(line) - elif line == 'Features:': - self._append_line = self._append_features_line - elif self._is_section_tag(name): - self._append_line = self._append_various_line - self._append_various_line(line) - else: - self._append_freeform(line) - else: - # This is a free-form documentation block - self._append_freeform(line) - - def _append_args_line(self, line): - """ - Process a line of documentation text in an argument section. - - A symbol line begins the next argument section, a section tag - section or a non-indented line after a blank line begins an - additional section. Start that section and append the line to - it. - - Else, append the line to the current section. - - """ - name = line.split(' ', 1)[0] - - if name.startswith('@') and name.endswith(':'): - # If line is "@arg: first line of description", find - # the index of 'f', which is the indent we expect for any - # following lines. We then remove the leading "@arg:" - # from line and replace it with spaces so that 'f' has the - # same index as it did in the original line and can be - # handled the same way we will handle following lines. - indent = must_match(r'@\S*:\s*', line).end() - line = line[indent:] - if not line: - # Line was just the "@arg:" header; following lines - # are not indented - indent = 0 - else: - line = ' ' * indent + line - self._start_args_section(name[1:-1], indent) - elif self._is_section_tag(name): - self._append_line = self._append_various_line - self._append_various_line(line) - return - elif (self._section.text.endswith('\n\n') - and line and not line[0].isspace()): - if line == 'Features:': - self._append_line = self._append_features_line - else: - self._start_section() - self._append_line = self._append_various_line - self._append_various_line(line) - return - - self._append_freeform(line) - - def _append_features_line(self, line): - name = line.split(' ', 1)[0] - - if name.startswith('@') and name.endswith(':'): - # If line is "@arg: first line of description", find - # the index of 'f', which is the indent we expect for any - # following lines. We then remove the leading "@arg:" - # from line and replace it with spaces so that 'f' has the - # same index as it did in the original line and can be - # handled the same way we will handle following lines. - indent = must_match(r'@\S*:\s*', line).end() - line = line[indent:] - if not line: - # Line was just the "@arg:" header; following lines - # are not indented - indent = 0 - else: - line = ' ' * indent + line - self._start_features_section(name[1:-1], indent) - elif self._is_section_tag(name): - self._append_line = self._append_various_line - self._append_various_line(line) - return - elif (self._section.text.endswith('\n\n') - and line and not line[0].isspace()): - self._start_section() - self._append_line = self._append_various_line - self._append_various_line(line) - return - - self._append_freeform(line) - - def _append_various_line(self, line): - """ - Process a line of documentation text in an additional section. - - A symbol line is an error. - - A section tag begins an additional section. Start that - section and append the line to it. - - Else, append the line to the current section. - """ - name = line.split(' ', 1)[0] - - if name.startswith('@') and name.endswith(':'): - raise QAPIParseError(self._parser, - "'%s' can't follow '%s' section" - % (name, self.sections[0].name)) - if self._is_section_tag(name): - # If line is "Section: first line of description", find - # the index of 'f', which is the indent we expect for any - # following lines. We then remove the leading "Section:" - # from line and replace it with spaces so that 'f' has the - # same index as it did in the original line and can be - # handled the same way we will handle following lines. - indent = must_match(r'\S*:\s*', line).end() - line = line[indent:] - if not line: - # Line was just the "Section:" header; following lines - # are not indented - indent = 0 - else: - line = ' ' * indent + line - self._start_section(name[:-1], indent) - - self._append_freeform(line) - - def _start_symbol_section(self, symbols_dict, name, indent): - # FIXME invalid names other than the empty string aren't flagged - if not name: - raise QAPIParseError(self._parser, "invalid parameter name") - if name in symbols_dict: - raise QAPIParseError(self._parser, - "'%s' parameter name duplicated" % name) - assert not self.sections - self._end_section() - self._section = QAPIDoc.ArgSection(self._parser, name, indent) - symbols_dict[name] = self._section - - def _start_args_section(self, name, indent): - self._start_symbol_section(self.args, name, indent) - - def _start_features_section(self, name, indent): - self._start_symbol_section(self.features, name, indent) - - def _start_section(self, name=None, indent=0): - if name in ('Returns', 'Since') and self.has_section(name): - raise QAPIParseError(self._parser, - "duplicated '%s' section" % name) - self._end_section() - self._section = QAPIDoc.Section(self._parser, name, indent) - self.sections.append(self._section) - - def _end_section(self): - if self._section: - text = self._section.text = self._section.text.strip() - if self._section.name and (not text or text.isspace()): - raise QAPIParseError( - self._parser, - "empty doc section '%s'" % self._section.name) - self._section = None - - def _append_freeform(self, line): - match = re.match(r'(@\S+:)', line) - if match: - raise QAPIParseError(self._parser, - "'%s' not allowed in free-form documentation" - % match.group(1)) - self._section.append(line) - - def connect_member(self, member): - if member.name not in self.args: - # Undocumented TODO outlaw - self.args[member.name] = QAPIDoc.ArgSection(self._parser, - member.name) - self.args[member.name].connect(member) - - def connect_feature(self, feature): - if feature.name not in self.features: - raise QAPISemError(feature.info, - "feature '%s' lacks documentation" - % feature.name) - self.features[feature.name].connect(feature) - - def check_expr(self, expr): - if self.has_section('Returns') and 'command' not in expr: - raise QAPISemError(self.info, - "'Returns:' is only valid for commands") - - def check(self): - - def check_args_section(args, info, what): - bogus = [name for name, section in args.items() - if not section.member] - if bogus: - raise QAPISemError( - self.info, - "documented member%s '%s' %s not exist" - % ("s" if len(bogus) > 1 else "", - "', '".join(bogus), - "do" if len(bogus) > 1 else "does")) - - check_args_section(self.args, self.info, 'members') - check_args_section(self.features, self.info, 'features') diff --git a/scripts/qapi/qapidoc.py b/scripts/qapi/qapidoc.py new file mode 100644 index 00000000000..4985d9565b8 --- /dev/null +++ b/scripts/qapi/qapidoc.py @@ -0,0 +1,362 @@ +# -*- coding: utf-8 -*- +# +# QAPI schema (doc) parser +# +# Copyright IBM, Corp. 2011 +# Copyright (c) 2013-2019 Red Hat Inc. +# +# Authors: +# Anthony Liguori +# Markus Armbruster +# Marc-André Lureau +# Kevin Wolf +# +# This work is licensed under the terms of the GNU GPL, version 2. +# See the COPYING file in the top-level directory. + +from collections import OrderedDict +import re + +from .common import must_match +from .error import QAPISemError + + +class QAPIDoc: + """ + A documentation comment block, either definition or free-form + + Definition documentation blocks consist of + + * a body section: one line naming the definition, followed by an + overview (any number of lines) + + * argument sections: a description of each argument (for commands + and events) or member (for structs, unions and alternates) + + * features sections: a description of each feature flag + + * additional (non-argument) sections, possibly tagged + + Free-form documentation blocks consist only of a body section. + """ + + class Section: + def __init__(self, parser, name=None, indent=0): + # parser, for error messages about indentation + self._parser = parser + # optional section name (argument/member or section name) + self.name = name + self.text = '' + # the expected indent level of the text of this section + self._indent = indent + + def append(self, line): + # Strip leading spaces corresponding to the expected indent level + # Blank lines are always OK. + if line: + indent = must_match(r'\s*', line).end() + if indent < self._indent: + raise QAPIParseError( + self._parser, + "unexpected de-indent (expected at least %d spaces)" % + self._indent) + line = line[self._indent:] + + self.text += line.rstrip() + '\n' + + class ArgSection(Section): + def __init__(self, parser, name, indent=0): + super().__init__(parser, name, indent) + self.member = None + + def connect(self, member): + self.member = member + + def __init__(self, parser, info): + # self._parser is used to report errors with QAPIParseError. The + # resulting error position depends on the state of the parser. + # It happens to be the beginning of the comment. More or less + # servicable, but action at a distance. + self._parser = parser + self.info = info + self.symbol = None + self.body = QAPIDoc.Section(parser) + # dict mapping parameter name to ArgSection + self.args = OrderedDict() + self.features = OrderedDict() + # a list of Section + self.sections = [] + # the current section + self._section = self.body + self._append_line = self._append_body_line + + def has_section(self, name): + """Return True if we have a section with this name.""" + for i in self.sections: + if i.name == name: + return True + return False + + def append(self, line): + """ + Parse a comment line and add it to the documentation. + + The way that the line is dealt with depends on which part of + the documentation we're parsing right now: + * The body section: ._append_line is ._append_body_line + * An argument section: ._append_line is ._append_args_line + * A features section: ._append_line is ._append_features_line + * An additional section: ._append_line is ._append_various_line + """ + line = line[1:] + if not line: + self._append_freeform(line) + return + + if line[0] != ' ': + raise QAPIParseError(self._parser, "missing space after #") + line = line[1:] + self._append_line(line) + + def end_comment(self): + self._end_section() + + @staticmethod + def _is_section_tag(name): + return name in ('Returns:', 'Since:', + # those are often singular or plural + 'Note:', 'Notes:', + 'Example:', 'Examples:', + 'TODO:') + + def _append_body_line(self, line): + """ + Process a line of documentation text in the body section. + + If this a symbol line and it is the section's first line, this + is a definition documentation block for that symbol. + + If it's a definition documentation block, another symbol line + begins the argument section for the argument named by it, and + a section tag begins an additional section. Start that + section and append the line to it. + + Else, append the line to the current section. + """ + name = line.split(' ', 1)[0] + # FIXME not nice: things like '# @foo:' and '# @foo: ' aren't + # recognized, and get silently treated as ordinary text + if not self.symbol and not self.body.text and line.startswith('@'): + if not line.endswith(':'): + raise QAPIParseError(self._parser, "line should end with ':'") + self.symbol = line[1:-1] + # FIXME invalid names other than the empty string aren't flagged + if not self.symbol: + raise QAPIParseError(self._parser, "invalid name") + elif self.symbol: + # This is a definition documentation block + if name.startswith('@') and name.endswith(':'): + self._append_line = self._append_args_line + self._append_args_line(line) + elif line == 'Features:': + self._append_line = self._append_features_line + elif self._is_section_tag(name): + self._append_line = self._append_various_line + self._append_various_line(line) + else: + self._append_freeform(line) + else: + # This is a free-form documentation block + self._append_freeform(line) + + def _append_args_line(self, line): + """ + Process a line of documentation text in an argument section. + + A symbol line begins the next argument section, a section tag + section or a non-indented line after a blank line begins an + additional section. Start that section and append the line to + it. + + Else, append the line to the current section. + + """ + name = line.split(' ', 1)[0] + + if name.startswith('@') and name.endswith(':'): + # If line is "@arg: first line of description", find + # the index of 'f', which is the indent we expect for any + # following lines. We then remove the leading "@arg:" + # from line and replace it with spaces so that 'f' has the + # same index as it did in the original line and can be + # handled the same way we will handle following lines. + indent = must_match(r'@\S*:\s*', line).end() + line = line[indent:] + if not line: + # Line was just the "@arg:" header; following lines + # are not indented + indent = 0 + else: + line = ' ' * indent + line + self._start_args_section(name[1:-1], indent) + elif self._is_section_tag(name): + self._append_line = self._append_various_line + self._append_various_line(line) + return + elif (self._section.text.endswith('\n\n') + and line and not line[0].isspace()): + if line == 'Features:': + self._append_line = self._append_features_line + else: + self._start_section() + self._append_line = self._append_various_line + self._append_various_line(line) + return + + self._append_freeform(line) + + def _append_features_line(self, line): + name = line.split(' ', 1)[0] + + if name.startswith('@') and name.endswith(':'): + # If line is "@arg: first line of description", find + # the index of 'f', which is the indent we expect for any + # following lines. We then remove the leading "@arg:" + # from line and replace it with spaces so that 'f' has the + # same index as it did in the original line and can be + # handled the same way we will handle following lines. + indent = must_match(r'@\S*:\s*', line).end() + line = line[indent:] + if not line: + # Line was just the "@arg:" header; following lines + # are not indented + indent = 0 + else: + line = ' ' * indent + line + self._start_features_section(name[1:-1], indent) + elif self._is_section_tag(name): + self._append_line = self._append_various_line + self._append_various_line(line) + return + elif (self._section.text.endswith('\n\n') + and line and not line[0].isspace()): + self._start_section() + self._append_line = self._append_various_line + self._append_various_line(line) + return + + self._append_freeform(line) + + def _append_various_line(self, line): + """ + Process a line of documentation text in an additional section. + + A symbol line is an error. + + A section tag begins an additional section. Start that + section and append the line to it. + + Else, append the line to the current section. + """ + name = line.split(' ', 1)[0] + + if name.startswith('@') and name.endswith(':'): + raise QAPIParseError(self._parser, + "'%s' can't follow '%s' section" + % (name, self.sections[0].name)) + if self._is_section_tag(name): + # If line is "Section: first line of description", find + # the index of 'f', which is the indent we expect for any + # following lines. We then remove the leading "Section:" + # from line and replace it with spaces so that 'f' has the + # same index as it did in the original line and can be + # handled the same way we will handle following lines. + indent = must_match(r'\S*:\s*', line).end() + line = line[indent:] + if not line: + # Line was just the "Section:" header; following lines + # are not indented + indent = 0 + else: + line = ' ' * indent + line + self._start_section(name[:-1], indent) + + self._append_freeform(line) + + def _start_symbol_section(self, symbols_dict, name, indent): + # FIXME invalid names other than the empty string aren't flagged + if not name: + raise QAPIParseError(self._parser, "invalid parameter name") + if name in symbols_dict: + raise QAPIParseError(self._parser, + "'%s' parameter name duplicated" % name) + assert not self.sections + self._end_section() + self._section = QAPIDoc.ArgSection(self._parser, name, indent) + symbols_dict[name] = self._section + + def _start_args_section(self, name, indent): + self._start_symbol_section(self.args, name, indent) + + def _start_features_section(self, name, indent): + self._start_symbol_section(self.features, name, indent) + + def _start_section(self, name=None, indent=0): + if name in ('Returns', 'Since') and self.has_section(name): + raise QAPIParseError(self._parser, + "duplicated '%s' section" % name) + self._end_section() + self._section = QAPIDoc.Section(self._parser, name, indent) + self.sections.append(self._section) + + def _end_section(self): + if self._section: + text = self._section.text = self._section.text.strip() + if self._section.name and (not text or text.isspace()): + raise QAPIParseError( + self._parser, + "empty doc section '%s'" % self._section.name) + self._section = None + + def _append_freeform(self, line): + match = re.match(r'(@\S+:)', line) + if match: + raise QAPIParseError(self._parser, + "'%s' not allowed in free-form documentation" + % match.group(1)) + self._section.append(line) + + def connect_member(self, member): + if member.name not in self.args: + # Undocumented TODO outlaw + self.args[member.name] = QAPIDoc.ArgSection(self._parser, + member.name) + self.args[member.name].connect(member) + + def connect_feature(self, feature): + if feature.name not in self.features: + raise QAPISemError(feature.info, + "feature '%s' lacks documentation" + % feature.name) + self.features[feature.name].connect(feature) + + def check_expr(self, expr): + if self.has_section('Returns') and 'command' not in expr: + raise QAPISemError(self.info, + "'Returns:' is only valid for commands") + + def check(self): + + def check_args_section(args, info, what): + bogus = [name for name, section in args.items() + if not section.member] + if bogus: + raise QAPISemError( + self.info, + "documented member%s '%s' %s not exist" + % ("s" if len(bogus) > 1 else "", + "', '".join(bogus), + "do" if len(bogus) > 1 else "does")) + + check_args_section(self.args, self.info, 'members') + check_args_section(self.features, self.info, 'features') From patchwork Tue May 11 22:05:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477338 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RKSAzO1O; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsdS3gGJz9sVt for ; Wed, 12 May 2021 08:13:48 +1000 (AEST) Received: from localhost ([::1]:38058 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgadO-00089b-IQ for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:13:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWM-0008NI-BC for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20676) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWK-0006aT-L1 for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HtIlONdYBHxGa8pNAI0MHyifBRlo8+Wum1TSrjIcQSQ=; b=RKSAzO1O1TggwF1ll6O5M3D07ZvDVnaCMk9f4eAP92b+uHllCtYvHLnlCCVnZdrvogdD0X /cv+nSLbgoqXzuPqra9QokjbmOkuI4asdZbdzv0crHbrR4Tb2N67/w5gfKHlsrheXFLkeh Jn2XxERdwGMyWO2ynGT1AKRvODrWUTM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-593-UJBMnefpPqqk-13c6HPpYA-1; Tue, 11 May 2021 18:06:25 -0400 X-MC-Unique: UJBMnefpPqqk-13c6HPpYA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1F85C107ACE4; Tue, 11 May 2021 22:06:24 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF1C5BA6F; Tue, 11 May 2021 22:06:19 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 18/21] qapi: [WIP] Add type ignores for qapidoc.py Date: Tue, 11 May 2021 18:05:58 -0400 Message-Id: <20210511220601.2110055-19-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: John Snow --- scripts/qapi/mypy.ini | 5 +++++ scripts/qapi/pylintrc | 1 + 2 files changed, 6 insertions(+) diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index d7bbb2dc9c7..1a72be2c788 100644 --- a/scripts/qapi/mypy.ini +++ b/scripts/qapi/mypy.ini @@ -9,6 +9,11 @@ disallow_untyped_defs = False disallow_incomplete_defs = False check_untyped_defs = False +[mypy-qapi.qapidoc] +disallow_untyped_defs = False +disallow_incomplete_defs = False +check_untyped_defs = False + [mypy-qapi.schema] disallow_untyped_defs = False disallow_incomplete_defs = False diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc index c5275d5f59b..ec7605edade 100644 --- a/scripts/qapi/pylintrc +++ b/scripts/qapi/pylintrc @@ -3,6 +3,7 @@ # Add files or directories matching the regex patterns to the ignore list. # The regex matches against base names, not paths. ignore-patterns=parser.py, + qapidoc.py, schema.py, From patchwork Tue May 11 22:05:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477346 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=atSpsBIA; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Ffspy2J67z9sVt for ; Wed, 12 May 2021 08:22:02 +1000 (AEST) Received: from localhost ([::1]:58222 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgalM-00059M-BY for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:22:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55810) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWL-0008LK-SH for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:29 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:44975) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWK-0006aA-4I for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770787; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tneFAuEEdfRYrPTC2/pL/p2ptFZFTWRvmzSGabmBn7A=; b=atSpsBIAvysKnDWlQM4//h/BECnAibvCLIz054ikbDNWuafmu3deVD0g5pur04L/4serfq Y8x6a7qztZ3wwIg34oIBKnyGWDfnWLTtzWnM/X9f7yi4tUVWpARWYG9iCReVdUkVD7qrmX JMvDYg0/HGXT15b/Jqqt3i/ejFMK3zE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-240-JKeWmSpONAaxc7V5k_QWpQ-1; Tue, 11 May 2021 18:06:25 -0400 X-MC-Unique: JKeWmSpONAaxc7V5k_QWpQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E0F06101371B; Tue, 11 May 2021 22:06:24 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4228ABA6F; Tue, 11 May 2021 22:06:24 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 19/21] qapi: [WIP] Import QAPIDoc from qapidoc Signed-off-by: John Snow Date: Tue, 11 May 2021 18:05:59 -0400 Message-Id: <20210511220601.2110055-20-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: John Snow --- scripts/qapi/expr.py | 2 +- scripts/qapi/parser.py | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py index 496f7e0333e..7616646e43d 100644 --- a/scripts/qapi/expr.py +++ b/scripts/qapi/expr.py @@ -44,7 +44,7 @@ from .common import c_name from .error import QAPISemError -from .parser import QAPIDoc +from .qapidoc import QAPIDoc from .source import QAPISourceInfo diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 54df1bfd499..940a427b1ae 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -26,6 +26,7 @@ from .common import must_match from .error import QAPISemError, QAPISourceError +from .qapidoc import QAPIDoc from .source import QAPISourceInfo From patchwork Tue May 11 22:06:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477347 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hn28Wjh7; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Ffsr33yH8z9sVt for ; Wed, 12 May 2021 08:22:59 +1000 (AEST) Received: from localhost ([::1]:60356 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgamH-0006bT-J1 for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:22:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWO-00006f-VV for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47478) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWM-0006eP-Sr for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770790; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L22ohg11je14HGZTcuKIEY1RXGdhUyC06NJGTwse8Ms=; b=Hn28Wjh7Wf9vUo8IsDJi0887XsJ8NFEtXqrirAg2JQxPHXvKI3dVmv5/JBAEwLDZLMZFym yukeDOtb1t2X0+U+e398wIQoAfI/q0craa9qnPnDsZerVZmmNqbSq4vi0urnC4PpHK6upy GYTFFJuGZLmqrjSsecEqbbgThUQlwwk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-6VMxeuXFMzKw7ImoLPUb6g-1; Tue, 11 May 2021 18:06:26 -0400 X-MC-Unique: 6VMxeuXFMzKw7ImoLPUb6g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id AFEBA107ACCA; Tue, 11 May 2021 22:06:25 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EC69BA6F; Tue, 11 May 2021 22:06:24 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 20/21] qapi: [WIP] Add QAPIDocError Date: Tue, 11 May 2021 18:06:00 -0400 Message-Id: <20210511220601.2110055-21-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Raise this error instead of QAPIParseError and delegate the context up to the parent parser. In a chat off-list, we discussed how this design forces us to continue having less accurate error context information. Still, it's useful for an extremely simple split without a lot of fuss. Signed-off-by: John Snow --- scripts/qapi/parser.py | 12 ++++++++++-- scripts/qapi/qapidoc.py | 36 +++++++++++++++++------------------- 2 files changed, 27 insertions(+), 21 deletions(-) diff --git a/scripts/qapi/parser.py b/scripts/qapi/parser.py index 940a427b1ae..cde9782c5b0 100644 --- a/scripts/qapi/parser.py +++ b/scripts/qapi/parser.py @@ -26,7 +26,7 @@ from .common import must_match from .error import QAPISemError, QAPISourceError -from .qapidoc import QAPIDoc +from .qapidoc import QAPIDoc, QAPIDocError from .source import QAPISourceInfo @@ -391,7 +391,7 @@ def get_expr(self) -> _ExprValue: self, "expected '{', '[', string, or boolean") return expr - def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: + def _get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: if self.val != '##': raise QAPIParseError( self, "junk after '##' at start of documentation comment") @@ -424,3 +424,11 @@ def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: self.accept(False) raise QAPIParseError(self, "documentation comment must end with '##'") + + def get_doc(self, info: QAPISourceInfo) -> List['QAPIDoc']: + # Tie QAPIDocError exceptions to the current parser state, + # re-raise as QAPIParseError. + try: + return self._get_doc(info) + except QAPIDocError as err: + raise QAPIParseError(self, str(err)) from err diff --git a/scripts/qapi/qapidoc.py b/scripts/qapi/qapidoc.py index 4985d9565b8..938e2009dcd 100644 --- a/scripts/qapi/qapidoc.py +++ b/scripts/qapi/qapidoc.py @@ -18,7 +18,11 @@ import re from .common import must_match -from .error import QAPISemError +from .error import QAPIError, QAPISemError + + +class QAPIDocError(QAPIError): + """QAPIDoc parsing errors.""" class QAPIDoc: @@ -56,8 +60,7 @@ def append(self, line): if line: indent = must_match(r'\s*', line).end() if indent < self._indent: - raise QAPIParseError( - self._parser, + raise QAPIDocError( "unexpected de-indent (expected at least %d spaces)" % self._indent) line = line[self._indent:] @@ -114,7 +117,7 @@ def append(self, line): return if line[0] != ' ': - raise QAPIParseError(self._parser, "missing space after #") + raise QAPIDocError("missing space after #") line = line[1:] self._append_line(line) @@ -148,11 +151,11 @@ def _append_body_line(self, line): # recognized, and get silently treated as ordinary text if not self.symbol and not self.body.text and line.startswith('@'): if not line.endswith(':'): - raise QAPIParseError(self._parser, "line should end with ':'") + raise QAPIDocError("line should end with ':'") self.symbol = line[1:-1] # FIXME invalid names other than the empty string aren't flagged if not self.symbol: - raise QAPIParseError(self._parser, "invalid name") + raise QAPIDocError("invalid name") elif self.symbol: # This is a definition documentation block if name.startswith('@') and name.endswith(':'): @@ -261,9 +264,8 @@ def _append_various_line(self, line): name = line.split(' ', 1)[0] if name.startswith('@') and name.endswith(':'): - raise QAPIParseError(self._parser, - "'%s' can't follow '%s' section" - % (name, self.sections[0].name)) + raise QAPIDocError("'%s' can't follow '%s' section" + % (name, self.sections[0].name)) if self._is_section_tag(name): # If line is "Section: first line of description", find # the index of 'f', which is the indent we expect for any @@ -286,10 +288,9 @@ def _append_various_line(self, line): def _start_symbol_section(self, symbols_dict, name, indent): # FIXME invalid names other than the empty string aren't flagged if not name: - raise QAPIParseError(self._parser, "invalid parameter name") + raise QAPIDocError("invalid parameter name") if name in symbols_dict: - raise QAPIParseError(self._parser, - "'%s' parameter name duplicated" % name) + raise QAPIDocError("'%s' parameter name duplicated" % name) assert not self.sections self._end_section() self._section = QAPIDoc.ArgSection(self._parser, name, indent) @@ -303,8 +304,7 @@ def _start_features_section(self, name, indent): def _start_section(self, name=None, indent=0): if name in ('Returns', 'Since') and self.has_section(name): - raise QAPIParseError(self._parser, - "duplicated '%s' section" % name) + raise QAPIDocError("duplicated '%s' section" % name) self._end_section() self._section = QAPIDoc.Section(self._parser, name, indent) self.sections.append(self._section) @@ -313,17 +313,15 @@ def _end_section(self): if self._section: text = self._section.text = self._section.text.strip() if self._section.name and (not text or text.isspace()): - raise QAPIParseError( - self._parser, + raise QAPIDocError( "empty doc section '%s'" % self._section.name) self._section = None def _append_freeform(self, line): match = re.match(r'(@\S+:)', line) if match: - raise QAPIParseError(self._parser, - "'%s' not allowed in free-form documentation" - % match.group(1)) + raise QAPIDocError("'%s' not allowed in free-form documentation" + % match.group(1)) self._section.append(line) def connect_member(self, member): From patchwork Tue May 11 22:06:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1477341 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CLrZWUJa; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfsgK2D2Tz9sVt for ; Wed, 12 May 2021 08:15:25 +1000 (AEST) Received: from localhost ([::1]:41882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lgaew-0002JB-0y for incoming@patchwork.ozlabs.org; Tue, 11 May 2021 18:15:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:55834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWN-0008TV-MT for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45883) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lgaWL-0006cI-Qd for qemu-devel@nongnu.org; Tue, 11 May 2021 18:06:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620770789; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9rkUphiFWgxIY4ugyKiKF34Jvg+Dnahj0kYjwUJ7AMI=; b=CLrZWUJad3uVN/7uDwDNODv9KDmKn0KXKlhp6G0OESBEacw/yHT58/qML9FoxPRgl2GTXm JUgdpMXu3afY4J2qS76oC/UTEZCNoX8TFFqdy19EKtvgIxsfCSUlSnGCvXrjaj9Xqte0O4 ikHvx/TYJEgdZqLeAFESHk3G7srBXvc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-328-v0MybDeKPZyQ6BvuKdw_3g-1; Tue, 11 May 2021 18:06:27 -0400 X-MC-Unique: v0MybDeKPZyQ6BvuKdw_3g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B7250107ACCD; Tue, 11 May 2021 22:06:26 +0000 (UTC) Received: from scv.redhat.com (ovpn-117-64.rdu2.redhat.com [10.10.117.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4017BA6F; Tue, 11 May 2021 22:06:25 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH v2 21/21] qapi: [WIP] Enable linters on parser.py Date: Tue, 11 May 2021 18:06:01 -0400 Message-Id: <20210511220601.2110055-22-jsnow@redhat.com> In-Reply-To: <20210511220601.2110055-1-jsnow@redhat.com> References: <20210511220601.2110055-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -34 X-Spam_score: -3.5 X-Spam_bar: --- X-Spam_report: (-3.5 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.699, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Roth , John Snow , Eduardo Habkost , Markus Armbruster , Cleber Rosa Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" (Only works after we excise QAPIDoc, of course.) Signed-off-by: John Snow --- scripts/qapi/mypy.ini | 6 ------ scripts/qapi/pylintrc | 3 +-- 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/scripts/qapi/mypy.ini b/scripts/qapi/mypy.ini index 1a72be2c788..56c0f306c5e 100644 --- a/scripts/qapi/mypy.ini +++ b/scripts/qapi/mypy.ini @@ -3,12 +3,6 @@ strict = True disallow_untyped_calls = False python_version = 3.6 -[mypy-qapi.parser] -# QAPISchemaParser is done, I promise! -disallow_untyped_defs = False -disallow_incomplete_defs = False -check_untyped_defs = False - [mypy-qapi.qapidoc] disallow_untyped_defs = False disallow_incomplete_defs = False diff --git a/scripts/qapi/pylintrc b/scripts/qapi/pylintrc index ec7605edade..b41a26c1ceb 100644 --- a/scripts/qapi/pylintrc +++ b/scripts/qapi/pylintrc @@ -2,8 +2,7 @@ # Add files or directories matching the regex patterns to the ignore list. # The regex matches against base names, not paths. -ignore-patterns=parser.py, - qapidoc.py, +ignore-patterns=qapidoc.py, schema.py,