From patchwork Mon Oct 4 21:04:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536364 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PVrCKGKW; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYJR146hz9t0Y for ; Tue, 5 Oct 2021 08:09:59 +1100 (AEDT) Received: from localhost ([::1]:38140 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVDg-00072B-DK for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:09:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54532) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9H-0008RP-To for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:22907) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9G-0007Kb-3a for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381521; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3384ENIQ49sidOdTSlZdwEwbLtdeY4hE6hRe0kcr8eg=; b=PVrCKGKW57/bTrZ8p8ZW57eqLLNaV1LuIcnPY2DEzKbZ/zUfIA2VsMa9M0uvc9V9sVHVvN aG81kDFy8reVHMukPOwBQfAfKBv/spWrBLavsnEPJSdN6gbNX5mpE6G3ROLR2M4Tkhbpst PbNl8S5nEIc6eaJqj9kUvj7czWM9x7s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-503-Am12gvx9M1qKpzKVmUAWZQ-1; Mon, 04 Oct 2021 17:05:18 -0400 X-MC-Unique: Am12gvx9M1qKpzKVmUAWZQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5A9091084683; Mon, 4 Oct 2021 21:05:17 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B48919C59; Mon, 4 Oct 2021 21:05:16 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 01/13] iotests/297: Move pylint config into pylintrc Date: Mon, 4 Oct 2021 17:04:51 -0400 Message-Id: <20211004210503.1455391-2-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 4 +--- tests/qemu-iotests/pylintrc | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 91ec34d9521..bc3a0ceb2aa 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -65,10 +65,8 @@ def run_linters(): print('=== pylint ===') sys.stdout.flush() - # Todo notes are fine, but fixme's or xxx's should probably just be - # fixed (in tests, at least) env = os.environ.copy() - subprocess.run(('pylint-3', '--score=n', '--notes=FIXME,XXX', *files), + subprocess.run(('pylint-3', *files), env=env, check=False) print('=== mypy ===') diff --git a/tests/qemu-iotests/pylintrc b/tests/qemu-iotests/pylintrc index 8cb4e1d6a6d..32ab77b8bb9 100644 --- a/tests/qemu-iotests/pylintrc +++ b/tests/qemu-iotests/pylintrc @@ -31,6 +31,22 @@ disable=invalid-name, too-many-statements, consider-using-f-string, + +[REPORTS] + +# Activate the evaluation score. +score=no + + +[MISCELLANEOUS] + +# List of note tags to take in consideration, separated by a comma. +# TODO notes are fine, but FIXMEs or XXXs should probably just be +# fixed (in tests, at least). +notes=FIXME, + XXX, + + [FORMAT] # Maximum number of characters on a single line. From patchwork Mon Oct 4 21:04:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536360 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=USMpLfPf; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYDn5crbz9t0G for ; Tue, 5 Oct 2021 08:06:49 +1100 (AEDT) Received: from localhost ([::1]:57730 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVAd-00011n-1S for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:06:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9H-0008QP-Et for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20398) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9F-0007Jz-71 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381520; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e9svPZTEZXeprp0K3ghNiPLY2v5vyIdmsQFVVqUATgI=; b=USMpLfPfhvGLjMRsWJoE8wcqOsirk8wDr7vCJLWh1JBF6H0m+sUGC7ALUIuDGLgf7QqsBN 43ZmlNVEyPk2ZZqGgL1Z0e93tBsQryavZrOkyFms1M3XTPa496qO5JWqU4nqBkmJi16A1Y xxoAmh2++Vf5RJp4x3oCdIrubjBvSsA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-181-Be6MVSPUOG-1zWpGZmLG8Q-1; Mon, 04 Oct 2021 17:05:19 -0400 X-MC-Unique: Be6MVSPUOG-1zWpGZmLG8Q-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8F23C1808304; Mon, 4 Oct 2021 21:05:18 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 831A419C59; Mon, 4 Oct 2021 21:05:17 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 02/13] iotests/297: Split mypy configuration out into mypy.ini Date: Mon, 4 Oct 2021 17:04:52 -0400 Message-Id: <20211004210503.1455391-3-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -9 X-Spam_score: -1.0 X-Spam_bar: - X-Spam_report: (-1.0 / 5.0 requ) DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" More separation of code and configuration. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 14 +------------- tests/qemu-iotests/mypy.ini | 12 ++++++++++++ 2 files changed, 13 insertions(+), 13 deletions(-) create mode 100644 tests/qemu-iotests/mypy.ini diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index bc3a0ceb2aa..b8101e6024a 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -73,19 +73,7 @@ def run_linters(): sys.stdout.flush() env['MYPYPATH'] = env['PYTHONPATH'] - p = subprocess.run(('mypy', - '--warn-unused-configs', - '--disallow-subclassing-any', - '--disallow-any-generics', - '--disallow-incomplete-defs', - '--disallow-untyped-decorators', - '--no-implicit-optional', - '--warn-redundant-casts', - '--warn-unused-ignores', - '--no-implicit-reexport', - '--namespace-packages', - '--scripts-are-modules', - *files), + p = subprocess.run(('mypy', *files), env=env, check=False, stdout=subprocess.PIPE, diff --git a/tests/qemu-iotests/mypy.ini b/tests/qemu-iotests/mypy.ini new file mode 100644 index 00000000000..4c0339f5589 --- /dev/null +++ b/tests/qemu-iotests/mypy.ini @@ -0,0 +1,12 @@ +[mypy] +disallow_any_generics = True +disallow_incomplete_defs = True +disallow_subclassing_any = True +disallow_untyped_decorators = True +implicit_reexport = False +namespace_packages = True +no_implicit_optional = True +scripts_are_modules = True +warn_redundant_casts = True +warn_unused_configs = True +warn_unused_ignores = True From patchwork Mon Oct 4 21:04:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536363 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=dJWcN6px; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYJQ3s5Nz9t0G for ; Tue, 5 Oct 2021 08:09:58 +1100 (AEDT) Received: from localhost ([::1]:38034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVDe-0006uD-M7 for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:09:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54562) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9J-0008Sq-1P for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:25 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:21565) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9G-0007Ks-DA for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381521; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OxBoClwLGmtvfw4YoPUgfCxtAPRRDdd0OWbSFRevYlY=; b=dJWcN6pxtEpZRFQF8/fWDQO9gRCLkCrQDDYlL4CwhmcqRtnOYr6r9TQTzDQfveDi9g6HFw jWhaEOx8lldctSU2PdEHKZ232nCxrrKvWfoQ2vKUM+DH9mkP4ZpXsBzQ+pj96OsTQRiiwA iIw40QhfNrJiWWqqvtJMsKMGsuuqWQI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-458-PchRNNreMrWLDEeNniNWkA-1; Mon, 04 Oct 2021 17:05:20 -0400 X-MC-Unique: PchRNNreMrWLDEeNniNWkA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C2B79100C66E; Mon, 4 Oct 2021 21:05:19 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC84319C59; Mon, 4 Oct 2021 21:05:18 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 03/13] iotests/297: Add get_files() function Date: Mon, 4 Oct 2021 17:04:53 -0400 Message-Id: <20211004210503.1455391-4-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -9 X-Spam_score: -1.0 X-Spam_bar: - X-Spam_report: (-1.0 / 5.0 requ) DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index b8101e6024a..15b54594c11 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,6 +21,7 @@ import re import shutil import subprocess import sys +from typing import List import iotests @@ -54,10 +55,14 @@ def is_python_file(filename): return False -def run_linters(): +def get_test_files() -> List[str]: named_tests = [f'tests/{entry}' for entry in os.listdir('tests')] check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES) - files = [filename for filename in check_tests if is_python_file(filename)] + return list(filter(is_python_file, check_tests)) + + +def run_linters(): + files = get_test_files() iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) From patchwork Mon Oct 4 21:04:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536361 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HlAV8Ihw; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYDw4W20z9t0G for ; Tue, 5 Oct 2021 08:06:56 +1100 (AEDT) Received: from localhost ([::1]:58130 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVAj-0001IK-Sh for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:06:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54596) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9O-0008V8-6k for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22747) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9M-0007SJ-K0 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381528; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OlAQ+TxXMwW6h5k/qwDi1B2jR4wj7YOWFlYlb0set7g=; b=HlAV8IhwLGLen1nl70DlY+fh+E6lHoXLhb/i8MtA870Kh3YAXb1mRSe4ZMVLQx1dqT64s2 tlc45JWgF+P8KUZAO6q7piypp3MmAS6NkV8Ws/uNLyvSQai6gncm9rR35FNaX8H/eI4zjI qVAQaRTxI1CguRJXPQYr3NDLg/RRZN8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-212-Vc0sjflCPsiTLJ9-Jk6GuQ-1; Mon, 04 Oct 2021 17:05:25 -0400 X-MC-Unique: Vc0sjflCPsiTLJ9-Jk6GuQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 04ECD9126B; Mon, 4 Oct 2021 21:05:24 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 183D619C59; Mon, 4 Oct 2021 21:05:19 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 04/13] iotests/297: Don't rely on distro-specific linter binaries Date: Mon, 4 Oct 2021 17:04:54 -0400 Message-Id: <20211004210503.1455391-5-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, =?utf-8?q?Phi?= =?utf-8?q?lippe_Mathieu-Daud=C3=A9?= , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" 'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m mypy" to access these scripts instead. This style of invocation will prefer the "correct" tool when run in a virtual environment. Note that we still check for "pylint-3" before the test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 15b54594c11..65b1e7058c2 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -71,14 +71,14 @@ def run_linters(): sys.stdout.flush() env = os.environ.copy() - subprocess.run(('pylint-3', *files), + subprocess.run(('python3', '-m', 'pylint', *files), env=env, check=False) print('=== mypy ===') sys.stdout.flush() env['MYPYPATH'] = env['PYTHONPATH'] - p = subprocess.run(('mypy', *files), + p = subprocess.run((('python3', '-m', 'mypy', *files), env=env, check=False, stdout=subprocess.PIPE, From patchwork Mon Oct 4 21:04:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536365 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FpRYt8bo; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYJS4JqZz9t0G for ; Tue, 5 Oct 2021 08:10:00 +1100 (AEDT) Received: from localhost ([::1]:38320 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVDh-00079t-Go for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:09:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54628) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9P-0008Vm-A3 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:50533) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9N-0007TF-3r for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381528; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m9ID7D1gHySIgtuzkpEuGv444ZZtkg9aOm+C1oYsvTU=; b=FpRYt8boTvyDibPPx8gvWE5JFuxsfqyzfstzfWQCTn37lS9xHm4O3j0ur67nIFVvDCoAeM oIpf6+X7ZlBnEUcu6nunguwgRcdvstyruTCj3iECxlu2JF7aoFDt4z1DP9JEglV+59l4Y1 XD/TroCFJtBpXp7ejDI9AhVQSRpfaKI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-427-Sa75VHZcPCaeIAXRCIp5hQ-1; Mon, 04 Oct 2021 17:05:27 -0400 X-MC-Unique: Sa75VHZcPCaeIAXRCIp5hQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A6514835DE0; Mon, 4 Oct 2021 21:05:26 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0BBC19C59; Mon, 4 Oct 2021 21:05:24 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 05/13] iotests/297: Create main() function Date: Mon, 4 Oct 2021 17:04:55 -0400 Message-Id: <20211004210503.1455391-6-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Vladimir Sementsov-Ogievskiy , Eduardo Habkost , qemu-block@nongnu.org, =?utf-8?q?Phi?= =?utf-8?q?lippe_Mathieu-Daud=C3=A9?= , Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 65b1e7058c2..f9fcb039e27 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -89,8 +89,12 @@ def run_linters(): print(p.stdout) -for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') +def main() -> None: + for linter in ('pylint-3', 'mypy'): + if shutil.which(linter) is None: + iotests.notrun(f'{linter} not found') -iotests.script_main(run_linters) + run_linters() + + +iotests.script_main(main) From patchwork Mon Oct 4 21:04:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536369 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=C4uxuwYz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYMF4GPHz9t0G for ; Tue, 5 Oct 2021 08:12:25 +1100 (AEDT) Received: from localhost ([::1]:47224 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVG2-0004vY-Q6 for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:12:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9Q-00006O-Ao for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35331) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9O-0007Uu-GT for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381529; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6c5KJqUBPGjOMB5qC981y/85tPXg6c6GV27yYeRj0DM=; b=C4uxuwYz0D7zgD+CyL+rh96hpDiuUd7GIanjiQJRbkFT4sQ36OAFLvevzUzjhvP8Ihfl8f dvqU515BRWYhIwu9NcuAiLEqwpa3nSb/4EuBCB/zH+5btMoa7acc1F2r/8Kx6J5JY5i/0X EzUBBpfEwxX9JoCPpe1NwUgZcP36Pxg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-530-_5G-ySmUMmWuhFh9pIogmg-1; Mon, 04 Oct 2021 17:05:28 -0400 X-MC-Unique: _5G-ySmUMmWuhFh9pIogmg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DDA0A802947; Mon, 4 Oct 2021 21:05:27 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id D379F19C59; Mon, 4 Oct 2021 21:05:26 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 06/13] iotests/297: Separate environment setup from test execution Date: Mon, 4 Oct 2021 17:04:56 -0400 Message-Id: <20211004210503.1455391-7-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Move environment setup into main(), leaving pure test execution behind in run_linters(). Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index f9fcb039e27..fcbab0631be 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -21,7 +21,7 @@ import re import shutil import subprocess import sys -from typing import List +from typing import List, Mapping, Optional import iotests @@ -61,23 +61,20 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) -def run_linters(): - files = get_test_files() - - iotests.logger.debug('Files to be checked:') - iotests.logger.debug(', '.join(sorted(files))) +def run_linters( + files: List[str], + env: Optional[Mapping[str, str]] = None, +) -> None: print('=== pylint ===') sys.stdout.flush() - env = os.environ.copy() subprocess.run(('python3', '-m', 'pylint', *files), env=env, check=False) print('=== mypy ===') sys.stdout.flush() - env['MYPYPATH'] = env['PYTHONPATH'] p = subprocess.run((('python3', '-m', 'mypy', *files), env=env, check=False, @@ -94,7 +91,15 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') - run_linters() + files = get_test_files() + + iotests.logger.debug('Files to be checked:') + iotests.logger.debug(', '.join(sorted(files))) + + env = os.environ.copy() + env['MYPYPATH'] = env['PYTHONPATH'] + + run_linters(files, env=env) iotests.script_main(main) From patchwork Mon Oct 4 21:04:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536368 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=G9JzGE4b; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYM62Pb9z9t0G for ; Tue, 5 Oct 2021 08:12:17 +1100 (AEDT) Received: from localhost ([::1]:46582 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVFt-0004Tk-D5 for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:12:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9R-00008I-PS for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55122) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9Q-0007Vp-29 for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381531; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YzdHDfL7FHfkEvQau1bQvqvNCryY6Q6DkvfYCYWYzOc=; b=G9JzGE4bT5LeTaoOon6yADk4gdA6bqdByAMWUKPthQUJNnYNMkSRkTIb2y3LvOAh0o0WTi A0m3ozO/mq6D4b3qyLZInqhSZa7Hm1Q10oQ2bIa6pQVWxueljzA3hvoYU1aXk912bFe0Dt aEq/UEro5OTZ2hTR40TuQAAffqbncKo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-542-JIJf9JuSPGuTQs4rJLerKA-1; Mon, 04 Oct 2021 17:05:30 -0400 X-MC-Unique: JIJf9JuSPGuTQs4rJLerKA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7B596835DE3; Mon, 4 Oct 2021 21:05:29 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2D50C19C59; Mon, 4 Oct 2021 21:05:27 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 07/13] iotests/297: Split run_linters apart into run_pylint and run_mypy Date: Mon, 4 Oct 2021 17:04:57 -0400 Message-Id: <20211004210503.1455391-8-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- Note, this patch really ought to be squashed with the next one, but I am performing a move known as "Hedging my bets." It's easier to squash than de-squash :) Signed-off-by: John Snow --- tests/qemu-iotests/297 | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index fcbab0631be..91029dbb34e 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -61,20 +61,19 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) -def run_linters( +def run_pylint( files: List[str], env: Optional[Mapping[str, str]] = None, ) -> None: - print('=== pylint ===') - sys.stdout.flush() - subprocess.run(('python3', '-m', 'pylint', *files), env=env, check=False) - print('=== mypy ===') - sys.stdout.flush() +def run_mypy( + files: List[str], + env: Optional[Mapping[str, str]] = None, +) -> None: p = subprocess.run((('python3', '-m', 'mypy', *files), env=env, check=False, @@ -99,7 +98,13 @@ def main() -> None: env = os.environ.copy() env['MYPYPATH'] = env['PYTHONPATH'] - run_linters(files, env=env) + print('=== pylint ===') + sys.stdout.flush() + run_pylint(files, env=env) + + print('=== mypy ===') + sys.stdout.flush() + run_mypy(files, env=env) iotests.script_main(main) From patchwork Mon Oct 4 21:04:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536362 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WrqX3pRk; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYFJ5mDCz9t0G for ; Tue, 5 Oct 2021 08:07:15 +1100 (AEDT) Received: from localhost ([::1]:58774 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVB1-0001m7-Sv for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:07:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9U-0000At-OH for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32219) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9S-0007Xn-Mh for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381534; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8CCof3xvdXh8fTaWrxwQRK+4yOvLtnzkx91JygqAU0=; b=WrqX3pRkSCEhA3FvoM3lPPuNTlFIcr6g2Lz3CdlGjNc/HicTGcg3pwiTAjCKrJ6DCMaNfT xQcHyVWH+Ue0TyxRafKOGNi3J+Y5n7ewWAwafBzttEF8COTFuB8QKUZ6DsDFCfRXVLR4+9 adyBcKRVZ354/2z1rn6m5PZgkdDcP8w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-364-Yia0LGDbMaefzyOzyxXF_g-1; Mon, 04 Oct 2021 17:05:31 -0400 X-MC-Unique: Yia0LGDbMaefzyOzyxXF_g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D6D859126B; Mon, 4 Oct 2021 21:05:30 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id AD87719C59; Mon, 4 Oct 2021 21:05:29 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 08/13] iotests/297: refactor run_[mypy|pylint] as generic execution shim Date: Mon, 4 Oct 2021 17:04:58 -0400 Message-Id: <20211004210503.1455391-9-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow Acked-by: Hanna Reitz --- tests/qemu-iotests/297 | 46 +++++++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 21 deletions(-) diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 91029dbb34e..4c54dd39b46 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -61,29 +61,33 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) -def run_pylint( - files: List[str], - env: Optional[Mapping[str, str]] = None, -) -> None: +def run_linter( + tool: str, + args: List[str], + env: Optional[Mapping[str, str]] = None, + suppress_output: bool = False, +) -> int: + """ + Run a python-based linting tool. - subprocess.run(('python3', '-m', 'pylint', *files), - env=env, check=False) + If suppress_output is True, capture stdout/stderr of the child + process and only print that information back to stdout if the child + process's return code was non-zero. + """ + p = subprocess.run( + ('python3', '-m', tool, *args), + env=env, + check=False, + stdout=subprocess.PIPE if suppress_output else None, + stderr=subprocess.STDOUT if suppress_output else None, + universal_newlines=True, + ) - -def run_mypy( - files: List[str], - env: Optional[Mapping[str, str]] = None, -) -> None: - p = subprocess.run((('python3', '-m', 'mypy', *files), - env=env, - check=False, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - universal_newlines=True) - - if p.returncode != 0: + if suppress_output and p.returncode != 0: print(p.stdout) + return p.returncode + def main() -> None: for linter in ('pylint-3', 'mypy'): @@ -100,11 +104,11 @@ def main() -> None: print('=== pylint ===') sys.stdout.flush() - run_pylint(files, env=env) + run_linter('pylint', files, env=env) print('=== mypy ===') sys.stdout.flush() - run_mypy(files, env=env) + run_linter('mypy', files, env=env, suppress_output=True) iotests.script_main(main) From patchwork Mon Oct 4 21:04:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536366 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CfR0cR3h; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYJn15d4z9t0G for ; Tue, 5 Oct 2021 08:10:17 +1100 (AEDT) Received: from localhost ([::1]:39524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVDy-000817-Dv for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:10:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9W-0000Bk-Qm for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:36205) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9U-0007ZJ-Rq for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381536; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=D2RwsNnhmW4kg5zkhgDbXUp23lnGDTKQUVioF8J6DsM=; b=CfR0cR3hBRee+yKthckg4V1hkKFzUkj8/gkHsu9DZzv1zjfZG5ylrsO+80FN+GvML7maaF m/vmtPFu287OTt5Su0pJQEKCMf+0lIrs6MSjmLPqE1LDv/JRyzWfcMVPq/mysnM91UKlYU N7Xsh4BSs3l1Ymku+JDSGCJA0NMBH0I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-5-IEIOxMRFOxWCt71KMP5NXg-1; Mon, 04 Oct 2021 17:05:33 -0400 X-MC-Unique: IEIOxMRFOxWCt71KMP5NXg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8F522802935; Mon, 4 Oct 2021 21:05:32 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2B00719C59; Mon, 4 Oct 2021 21:05:31 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 09/13] iotests: split linters.py out from 297 Date: Mon, 4 Oct 2021 17:04:59 -0400 Message-Id: <20211004210503.1455391-10-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Now, 297 is just the iotests-specific incantations and linters.py is as minimal as I can think to make it. The only remaining element in here that ought to be configuration and not code is the list of skip files, but they're still numerous enough that repeating them for mypy and pylint configurations both would be ... a hassle. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/297 | 72 +++--------------------------- tests/qemu-iotests/linters.py | 83 +++++++++++++++++++++++++++++++++++ 2 files changed, 88 insertions(+), 67 deletions(-) create mode 100644 tests/qemu-iotests/linters.py diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 index 4c54dd39b46..f79c80216bf 100755 --- a/tests/qemu-iotests/297 +++ b/tests/qemu-iotests/297 @@ -17,76 +17,14 @@ # along with this program. If not, see . import os -import re import shutil -import subprocess import sys -from typing import List, Mapping, Optional import iotests +import linters -# TODO: Empty this list! -SKIP_FILES = ( - '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', - '096', '118', '124', '132', '136', '139', '147', '148', '149', - '151', '152', '155', '163', '165', '194', '196', '202', - '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', - '218', '219', '224', '228', '234', '235', '236', '237', '238', - '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', - '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', - '299', '302', '303', '304', '307', - 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' -) - - -def is_python_file(filename): - if not os.path.isfile(filename): - return False - - if filename.endswith('.py'): - return True - - with open(filename, encoding='utf-8') as f: - try: - first_line = f.readline() - return re.match('^#!.*python', first_line) is not None - except UnicodeDecodeError: # Ignore binary files - return False - - -def get_test_files() -> List[str]: - named_tests = [f'tests/{entry}' for entry in os.listdir('tests')] - check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES) - return list(filter(is_python_file, check_tests)) - - -def run_linter( - tool: str, - args: List[str], - env: Optional[Mapping[str, str]] = None, - suppress_output: bool = False, -) -> int: - """ - Run a python-based linting tool. - - If suppress_output is True, capture stdout/stderr of the child - process and only print that information back to stdout if the child - process's return code was non-zero. - """ - p = subprocess.run( - ('python3', '-m', tool, *args), - env=env, - check=False, - stdout=subprocess.PIPE if suppress_output else None, - stderr=subprocess.STDOUT if suppress_output else None, - universal_newlines=True, - ) - - if suppress_output and p.returncode != 0: - print(p.stdout) - - return p.returncode +# Looking for the list of files to exclude from linting? See linters.py. def main() -> None: @@ -94,7 +32,7 @@ def main() -> None: if shutil.which(linter) is None: iotests.notrun(f'{linter} not found') - files = get_test_files() + files = linters.get_test_files() iotests.logger.debug('Files to be checked:') iotests.logger.debug(', '.join(sorted(files))) @@ -104,11 +42,11 @@ def main() -> None: print('=== pylint ===') sys.stdout.flush() - run_linter('pylint', files, env=env) + linters.run_linter('pylint', files, env=env) print('=== mypy ===') sys.stdout.flush() - run_linter('mypy', files, env=env, suppress_output=True) + linters.run_linter('mypy', files, env=env, suppress_output=True) iotests.script_main(main) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py new file mode 100644 index 00000000000..f6a2dc139fd --- /dev/null +++ b/tests/qemu-iotests/linters.py @@ -0,0 +1,83 @@ +# Copyright (C) 2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import re +import subprocess +from typing import List, Mapping, Optional + + +# TODO: Empty this list! +SKIP_FILES = ( + '030', '040', '041', '044', '045', '055', '056', '057', '065', '093', + '096', '118', '124', '132', '136', '139', '147', '148', '149', + '151', '152', '155', '163', '165', '194', '196', '202', + '203', '205', '206', '207', '208', '210', '211', '212', '213', '216', + '218', '219', '224', '228', '234', '235', '236', '237', '238', + '240', '242', '245', '246', '248', '255', '256', '257', '258', '260', + '262', '264', '266', '274', '277', '280', '281', '295', '296', '298', + '299', '302', '303', '304', '307', + 'nbd-fault-injector.py', 'qcow2.py', 'qcow2_format.py', 'qed.py' +) + + +def is_python_file(filename): + if not os.path.isfile(filename): + return False + + if filename.endswith('.py'): + return True + + with open(filename, encoding='utf-8') as f: + try: + first_line = f.readline() + return re.match('^#!.*python', first_line) is not None + except UnicodeDecodeError: # Ignore binary files + return False + + +def get_test_files() -> List[str]: + named_tests = [f'tests/{entry}' for entry in os.listdir('tests')] + check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES) + return list(filter(is_python_file, check_tests)) + + +def run_linter( + tool: str, + args: List[str], + env: Optional[Mapping[str, str]] = None, + suppress_output: bool = False, +) -> int: + """ + Run a python-based linting tool. + + If suppress_output is True, capture stdout/stderr of the child + process and only print that information back to stdout if the child + process's return code was non-zero. + """ + p = subprocess.run( + ('python3', '-m', tool, *args), + env=env, + check=False, + stdout=subprocess.PIPE if suppress_output else None, + stderr=subprocess.STDOUT if suppress_output else None, + universal_newlines=True, + ) + + if suppress_output and p.returncode != 0: + print(p.stdout) + + return p.returncode + From patchwork Mon Oct 4 21:05:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536372 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Z71dB6Pi; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYXM1VFGz9t0G for ; Tue, 5 Oct 2021 08:20:19 +1100 (AEDT) Received: from localhost ([::1]:37472 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVNg-0000gM-Du for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:20:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54816) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9d-0000Kv-Pl for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:25108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9X-0007aq-0E for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381538; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tzywCHfezvyILqdnEd3Yo5bNTFUUpTRqOuOoMjthsKI=; b=Z71dB6PiO8kXTs5259pSsO0LE4bKyKz5zZ6LhMjLvk9HBl4Ia+LZYYLm9fZnSWH9Fv8dgF QHcdW2dSnn2rBRTHxYvmNY4rCnmsYCZKiwfFIkmB7RtkLzV/8iVKELlVcY4IzcPWLzk9XB xQamfjPc1IumxwSSug1aZoVNEDzTmcQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-604-NWd1f3tzOsiUlSDphSdBxg-1; Mon, 04 Oct 2021 17:05:35 -0400 X-MC-Unique: NWd1f3tzOsiUlSDphSdBxg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70C5891270; Mon, 4 Oct 2021 21:05:34 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id D340C19C59; Mon, 4 Oct 2021 21:05:32 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 10/13] iotests/linters: Add entry point for linting via Python CI Date: Mon, 4 Oct 2021 17:05:00 -0400 Message-Id: <20211004210503.1455391-11-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We need at least a tiny little shim here to join test file discovery with test invocation. This logic could conceivably be hosted somewhere in python/, but I felt it was strictly the least-rude thing to keep the test logic here in iotests/, even if this small function isn't itself an iotest. Note that we don't actually even need the executable bit here, we'll be relying on the ability to run this module as a script using Python CLI arguments. No chance it gets misunderstood as an actual iotest that way. (It's named, not in tests/, doesn't have the execute bit, and doesn't have an execution shebang.) Signed-off-by: John Snow --- (1) I think that the test file discovery logic and skip list belong together, and that those items belong in iotests/. I think they also belong in whichever directory pylintrc and mypy.ini are in, also in iotests/. (2) Moving this logic into python/tests/ is challenging because I'd have to import iotests code from elsewhere in the source tree, which just inverts an existing problem I have been trying to rid us of -- needing to muck around with PYTHONPATH or sys.path hacking in python scripts. I'm keen to avoid this. (3) If we moved all python tests into tests/ and gave them *.py extensions, we wouldn't even need the test discovery functions anymore, and all of linters.py could be removed entirely, including this execution shim. We could rely on mypy/pylint's own file discovery mechanisms at that point. More work than I'm up for with just this series, but I could be coaxed into doing it if there was some promise of not rejecting all that busywork ;) Signed-off-by: John Snow --- tests/qemu-iotests/linters.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index f6a2dc139fd..191df173064 100644 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -16,6 +16,7 @@ import os import re import subprocess +import sys from typing import List, Mapping, Optional @@ -81,3 +82,20 @@ def run_linter( return p.returncode + +def main() -> int: + """ + Used by the Python CI system as an entry point to run these linters. + """ + files = get_test_files() + + if sys.argv[1] == '--pylint': + return run_linter('pylint', files) + elif sys.argv[1] == '--mypy': + return run_linter('mypy', files) + + raise ValueError(f"Unrecognized argument(s): '{sys.argv[1:]}'") + + +if __name__ == '__main__': + sys.exit(main()) From patchwork Mon Oct 4 21:05:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536370 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TLXhcGxR; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYMP2jFHz9t0G for ; Tue, 5 Oct 2021 08:12:33 +1100 (AEDT) Received: from localhost ([::1]:47958 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVGA-0005Qi-HZ for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:12:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9e-0000Mb-SM for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48396) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9b-0007f9-Le for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BuNG23n/ROlnrCZj1iAPbGo5ccVoOtxrMacAee7pCck=; b=TLXhcGxROAUQOJA+OXzH/6qUNZB5DGsuvLChn98vj8vsEXwSXjEKOviOKbpw1mXjcMg9hG Thu59QQf7gUA7gUUHc5Os3E1oizyI8JY85AWitz9mt5U3o2isAHesvomU/t/hh6VYlY+NJ e21znh61YcPn8wPTyyk6kBTVK4ybIYc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-90-MvTQs5jLN8eFjjHUQiNQrg-1; Mon, 04 Oct 2021 17:05:36 -0400 X-MC-Unique: MvTQs5jLN8eFjjHUQiNQrg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8FC8D100C661; Mon, 4 Oct 2021 21:05:35 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BC3719C59; Mon, 4 Oct 2021 21:05:34 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 11/13] iotests/linters: Add workaround for mypy bug #9852 Date: Mon, 4 Oct 2021 17:05:01 -0400 Message-Id: <20211004210503.1455391-12-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This one is insidious: if you write an import as "from {namespace} import {subpackage}" as mirror-top-perms (now) does, mypy will fail on every-other invocation *if* the package being imported is a typed, installed, namespace-scoped package. Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in the context of Python CI tests. Now, I could just edit mirror-top-perms to avoid this invocation, but since I tripped on a landmine, I might as well head it off at the pass and make sure nobody else trips on that same landmine. It seems to have something to do with the order in which files are checked as well, meaning the random order in which set(os.listdir()) produces the list of files to test will cause problems intermittently and not just strictly "every other run". This will be fixed in mypy >= 0.920, which is not released yet. The workaround for now is to disable incremental checking, which avoids the issue. Note: This workaround is not applied when running iotest 297 directly, because the bug does not surface there! Given the nature of CI jobs not starting with any stale cache to begin with, this really only has a half-second impact on manual runs of the Python test suite when executed directly by a developer on their local machine. The workaround may be removed when the Python package requirements can stipulate mypy 0.920 or higher, which can happen as soon as it is released. (Barring any unforseen compatibility issues that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- tests/qemu-iotests/linters.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 191df173064..83fcc5a960c 100644 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -92,7 +92,9 @@ def main() -> int: if sys.argv[1] == '--pylint': return run_linter('pylint', files) elif sys.argv[1] == '--mypy': - return run_linter('mypy', files) + # mypy bug #9852; disable incremental checking as a workaround. + args = ['--no-incremental'] + files + return run_linter('mypy', args) raise ValueError(f"Unrecognized argument(s): '{sys.argv[1:]}'") From patchwork Mon Oct 4 21:05:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536376 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=IyGQ9Enq; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYcc5T5Nz9t0G for ; Tue, 5 Oct 2021 08:24:00 +1100 (AEDT) Received: from localhost ([::1]:45864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVRG-0006X6-1N for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:23:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9e-0000LJ-9C for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:48545) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9Z-0007ek-Of for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gX3Ufif356uAxVygxGFa+tHu0BBKNkhvwv+7X5BR1eU=; b=IyGQ9Enqo45BMCWpgbkQuMMXOFx+n/VoQ0h0kkODObsK9NPWk9Pj9ykU2vZlUtpU0qxEgZ x4UPekArLvOVAacZdfCo/OtNXJYsELqoMTngyB58ezYDh0jBpLtCYZmlPjh6cgqKp71iv8 Mvs1atG9GUadIaunYLq1bI6ZCqcoj1Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-583-Lvqru-DuMLijeSovx6eadA-1; Mon, 04 Oct 2021 17:05:37 -0400 X-MC-Unique: Lvqru-DuMLijeSovx6eadA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C6D5A1808304; Mon, 4 Oct 2021 21:05:36 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id D39D819723; Mon, 4 Oct 2021 21:05:35 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 12/13] python: Add iotest linters to test suite Date: Mon, 4 Oct 2021 17:05:02 -0400 Message-Id: <20211004210503.1455391-13-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Run mypy and pylint on the iotests files directly from the Python CI test infrastructure. This ensures that any accidental breakages to the qemu.[qmp|aqmp|machine|utils] packages will be caught by that test suite. It also ensures that these linters are run with well-known versions and test against a wide variety of python versions, which helps to find accidental cross-version python compatibility issues. Signed-off-by: John Snow Reviewed-by: Hanna Reitz --- python/tests/iotests-mypy.sh | 4 ++++ python/tests/iotests-pylint.sh | 4 ++++ 2 files changed, 8 insertions(+) create mode 100755 python/tests/iotests-mypy.sh create mode 100755 python/tests/iotests-pylint.sh diff --git a/python/tests/iotests-mypy.sh b/python/tests/iotests-mypy.sh new file mode 100755 index 00000000000..ee764708199 --- /dev/null +++ b/python/tests/iotests-mypy.sh @@ -0,0 +1,4 @@ +#!/bin/sh -e + +cd ../tests/qemu-iotests/ +python3 -m linters --mypy diff --git a/python/tests/iotests-pylint.sh b/python/tests/iotests-pylint.sh new file mode 100755 index 00000000000..4cae03424b4 --- /dev/null +++ b/python/tests/iotests-pylint.sh @@ -0,0 +1,4 @@ +#!/bin/sh -e + +cd ../tests/qemu-iotests/ +python3 -m linters --pylint From patchwork Mon Oct 4 21:05:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Snow X-Patchwork-Id: 1536371 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PF08+lm1; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HNYSz0R2Dz9t0G for ; Tue, 5 Oct 2021 08:17:21 +1100 (AEDT) Received: from localhost ([::1]:57078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mXVKn-0003G7-3n for incoming@patchwork.ozlabs.org; Mon, 04 Oct 2021 17:17:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54794) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9c-0000Jl-Tq for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22840) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mXV9Z-0007e1-Np for qemu-devel@nongnu.org; Mon, 04 Oct 2021 17:05:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633381540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c2URwBssOV4F5yamcvI+h90YNpy4zMea5kx+WDfRvN8=; b=PF08+lm1ldVo7xGmkeXDnjIZgbrWQ+S63XXMQ39mhLoR/RSdAyRlFV+at65jWGb/PATvbV Sc1MTuQyZeZGU23Kj3HS+92u2HoLGvGciGIiDk5nzm5BBaLEiEWVTEyKo6qfvLnJUc739Z VE8Xr0PRVjGMkxiwOP0LFedsebH9RXk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-346-QSophn58NmSpvHsssbe8Eg-1; Mon, 04 Oct 2021 17:05:39 -0400 X-MC-Unique: QSophn58NmSpvHsssbe8Eg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2572F835DE0; Mon, 4 Oct 2021 21:05:38 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.9.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3B9319723; Mon, 4 Oct 2021 21:05:36 +0000 (UTC) From: John Snow To: qemu-devel@nongnu.org Subject: [PATCH 13/13] iotests: [RFC] drop iotest 297 Date: Mon, 4 Oct 2021 17:05:03 -0400 Message-Id: <20211004210503.1455391-14-jsnow@redhat.com> In-Reply-To: <20211004210503.1455391-1-jsnow@redhat.com> References: <20211004210503.1455391-1-jsnow@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Eduardo Habkost , qemu-block@nongnu.org, Hanna Reitz , Cleber Rosa , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" (This is highlighting a what-if, which might make it clear why any special infrastructure is still required at all. It's not intended to actually be merged at this step -- running JUST the iotest linters from e.g. 'make check' is not yet accommodated, so there's no suitable replacement for 297 for block test authors.) Drop 297. As a consequence, we no longer need to pass an environment variable to the mypy/pylint invocations, so that can be dropped. We also now no longer need to hide output-except-on-error, so that can be dropped as well. The only thing that necessitates any special running logic anymore is the skip list and the python-test-detection code. Without those, we could easily codify the tests as simply: [pylint|mypy] *.py tests/*.py ... and drop this entire file. We're not quite there yet, though. Signed-off-by: John Snow --- tests/qemu-iotests/297 | 52 ----------------------------------- tests/qemu-iotests/297.out | 2 -- tests/qemu-iotests/linters.py | 20 ++------------ 3 files changed, 2 insertions(+), 72 deletions(-) delete mode 100755 tests/qemu-iotests/297 delete mode 100644 tests/qemu-iotests/297.out diff --git a/tests/qemu-iotests/297 b/tests/qemu-iotests/297 deleted file mode 100755 index f79c80216bf..00000000000 --- a/tests/qemu-iotests/297 +++ /dev/null @@ -1,52 +0,0 @@ -#!/usr/bin/env python3 -# group: meta -# -# Copyright (C) 2020 Red Hat, Inc. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import os -import shutil -import sys - -import iotests -import linters - - -# Looking for the list of files to exclude from linting? See linters.py. - - -def main() -> None: - for linter in ('pylint-3', 'mypy'): - if shutil.which(linter) is None: - iotests.notrun(f'{linter} not found') - - files = linters.get_test_files() - - iotests.logger.debug('Files to be checked:') - iotests.logger.debug(', '.join(sorted(files))) - - env = os.environ.copy() - env['MYPYPATH'] = env['PYTHONPATH'] - - print('=== pylint ===') - sys.stdout.flush() - linters.run_linter('pylint', files, env=env) - - print('=== mypy ===') - sys.stdout.flush() - linters.run_linter('mypy', files, env=env, suppress_output=True) - - -iotests.script_main(main) diff --git a/tests/qemu-iotests/297.out b/tests/qemu-iotests/297.out deleted file mode 100644 index f2e1314d104..00000000000 --- a/tests/qemu-iotests/297.out +++ /dev/null @@ -1,2 +0,0 @@ -=== pylint === -=== mypy === diff --git a/tests/qemu-iotests/linters.py b/tests/qemu-iotests/linters.py index 83fcc5a960c..ca90604d8d9 100644 --- a/tests/qemu-iotests/linters.py +++ b/tests/qemu-iotests/linters.py @@ -17,7 +17,7 @@ import re import subprocess import sys -from typing import List, Mapping, Optional +from typing import List # TODO: Empty this list! @@ -55,31 +55,15 @@ def get_test_files() -> List[str]: return list(filter(is_python_file, check_tests)) -def run_linter( - tool: str, - args: List[str], - env: Optional[Mapping[str, str]] = None, - suppress_output: bool = False, -) -> int: +def run_linter(tool: str, args: List[str]) -> int: """ Run a python-based linting tool. - - If suppress_output is True, capture stdout/stderr of the child - process and only print that information back to stdout if the child - process's return code was non-zero. """ p = subprocess.run( ('python3', '-m', tool, *args), - env=env, check=False, - stdout=subprocess.PIPE if suppress_output else None, - stderr=subprocess.STDOUT if suppress_output else None, - universal_newlines=True, ) - if suppress_output and p.returncode != 0: - print(p.stdout) - return p.returncode