From patchwork Fri Mar 3 16:07:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751508 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XUCL8ZNU; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStDd24WMz23j7 for ; Sat, 4 Mar 2023 03:08:17 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wb-0007eF-VL; Fri, 03 Mar 2023 11:07:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wY-0007aT-5x for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wW-0005j9-IG for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ReqqbPShs0jLGuOzTHqYnwBo7T64XEGqJtInWVwIvRk=; b=XUCL8ZNURW4LFxfgLcOu4eapTML/lFXBeiNtZzHbHhnyg7wltZcHDuJirdImiDXkToq1PM lbOGHyqFu+JocT5/ohH5myaay0hGXZMs2fDmEDOeXF1N19Y6mg00yJrVH7P9amryORZOEj zwK125uYDJrZqIEEk7lHyQn087ECW84= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-330-m1SGgVzHML6hrTVQU7BmXw-1; Fri, 03 Mar 2023 11:07:32 -0500 X-MC-Unique: m1SGgVzHML6hrTVQU7BmXw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 06FC61C06ECD; Fri, 3 Mar 2023 16:07:32 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id B1ECE492C14; Fri, 3 Mar 2023 16:07:30 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 1/8] iotests: explicitly pass source/build dir to 'check' command Date: Fri, 3 Mar 2023 16:07:20 +0000 Message-Id: <20230303160727.3977246-2-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The 'check' script has some rather dubious logic whereby it assumes that if invoked as a symlink, then it is running from a separate source tree and build tree, otherwise it assumes the current working directory is a combined source and build tree. This doesn't work if you want to invoke the 'check' script using its full source tree path while still using a split source and build tree layout. This would be a typical situation with meson if you ask it to find the 'check' script path using files('check'). Rather than trying to make the logic more magical, add support for explicitly passing the dirs using --source-dir and --build-dir. If either is omitted the current logic is maintained. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qemu-iotests/check | 25 +++++++++++++++++++++++-- tests/qemu-iotests/testenv.py | 13 ++++--------- 2 files changed, 27 insertions(+), 11 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 9bdda1394e..da7e8a87fe 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -26,9 +26,23 @@ from findtests import TestFinder from testenv import TestEnv from testrunner import TestRunner +def get_default_path(follow_link=False): + """ + Try to automagically figure out the path we are running from. + """ + # called from the build tree? + if os.path.islink(sys.argv[0]): + if follow_link: + return os.path.dirname(os.readlink(sys.argv[0])) + else: + return os.path.dirname(os.path.abspath(sys.argv[0])) + else: # or source tree? + return os.getcwd() def make_argparser() -> argparse.ArgumentParser: - p = argparse.ArgumentParser(description="Test run options") + p = argparse.ArgumentParser( + description="Test run options", + formatter_class=argparse.ArgumentDefaultsHelpFormatter) p.add_argument('-n', '--dry-run', action='store_true', help='show me, do not run tests') @@ -113,6 +127,11 @@ def make_argparser() -> argparse.ArgumentParser: 'middle of the process.') g_sel.add_argument('tests', metavar='TEST_FILES', nargs='*', help='tests to run, or "--" followed by a command') + g_sel.add_argument('--build-dir', default=get_default_path(), + help='Path to iotests build directory') + g_sel.add_argument('--source-dir', + default=get_default_path(follow_link=True), + help='Path to iotests build directory') return p @@ -120,7 +139,9 @@ def make_argparser() -> argparse.ArgumentParser: if __name__ == '__main__': args = make_argparser().parse_args() - env = TestEnv(imgfmt=args.imgfmt, imgproto=args.imgproto, + env = TestEnv(source_dir=args.source_dir, + build_dir=args.build_dir, + imgfmt=args.imgfmt, imgproto=args.imgproto, aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index a864c74b12..aa9d735414 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -170,7 +170,8 @@ def root(*names: str) -> str: if not isxfile(b): sys.exit('Not executable: ' + b) - def __init__(self, imgfmt: str, imgproto: str, aiomode: str, + def __init__(self, source_dir: str, build_dir: str, + imgfmt: str, imgproto: str, aiomode: str, cachemode: Optional[str] = None, imgopts: Optional[str] = None, misalign: bool = False, @@ -211,14 +212,8 @@ def __init__(self, imgfmt: str, imgproto: str, aiomode: str, # which are needed to initialize some environment variables. They are # used by init_*() functions as well. - if os.path.islink(sys.argv[0]): - # called from the build tree - self.source_iotests = os.path.dirname(os.readlink(sys.argv[0])) - self.build_iotests = os.path.dirname(os.path.abspath(sys.argv[0])) - else: - # called from the source tree - self.source_iotests = os.getcwd() - self.build_iotests = self.source_iotests + self.source_iotests = source_dir + self.build_iotests = build_dir self.build_root = os.path.join(self.build_iotests, '..', '..') From patchwork Fri Mar 3 16:07:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751514 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NUecyBu0; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStFc5zlqz23j7 for ; Sat, 4 Mar 2023 03:09:08 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wi-0007lw-9c; Fri, 03 Mar 2023 11:07:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wZ-0007dL-GP for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wY-0005jX-1Z for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859657; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PdcDICZq/dCtnaFXWoLRhtoDbEWsAWhIpKZoGOSU984=; b=NUecyBu0Ozzizq5VazaNfFoafp4etYDD0m8nnVQoXo03fRwW2c8rmGO/UuAfKk/3NUlXWX YRiAWH1rpHF69nQaeE/bp2ldocprS9ElvntHtoNDqiraUqsCbacqzfwXxcEpBP/PLL7Nf+ YLsuoz94mzLzeHe6GRMnAF1SEm/t8zs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-444-FUd80qgTMqy-07o2GIoHOQ-1; Fri, 03 Mar 2023 11:07:34 -0500 X-MC-Unique: FUd80qgTMqy-07o2GIoHOQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C876D380610B; Fri, 3 Mar 2023 16:07:33 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 456F6492C14; Fri, 3 Mar 2023 16:07:32 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 2/8] iotests: allow test discovery before building Date: Fri, 3 Mar 2023 16:07:21 +0000 Message-Id: <20230303160727.3977246-3-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The 'check' script can be invoked in "dry run" mode, in which case it merely does test discovery and prints out all their names. Despite only doing test discovery it still validates that the various QEMU binaries can be found. This makes it impossible todo test discovery prior to building QEMU. This is a desirable feature to support, because it will let meson discover tests. Fortunately the code in the TestEnv constructor is ordered in a way that makes this fairly trivial to achieve. We can just short circuit the constructor after the basic directory paths have been set. Reviewed-by: Thomas Huth Reviewed-by: Alex Bennée Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/check | 3 ++- tests/qemu-iotests/testenv.py | 7 ++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index da7e8a87fe..bb294ef556 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -145,7 +145,8 @@ if __name__ == '__main__': aiomode=args.aiomode, cachemode=args.cachemode, imgopts=args.imgopts, misalign=args.misalign, debug=args.debug, valgrind=args.valgrind, - gdb=args.gdb, qprint=args.print) + gdb=args.gdb, qprint=args.print, + dry_run=args.dry_run) if len(sys.argv) > 1 and sys.argv[-len(args.tests)-1] == '--': if not args.tests: diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index aa9d735414..9a37ad9152 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -178,7 +178,8 @@ def __init__(self, source_dir: str, build_dir: str, debug: bool = False, valgrind: bool = False, gdb: bool = False, - qprint: bool = False) -> None: + qprint: bool = False, + dry_run: bool = False) -> None: self.imgfmt = imgfmt self.imgproto = imgproto self.aiomode = aiomode @@ -218,6 +219,10 @@ def __init__(self, source_dir: str, build_dir: str, self.build_root = os.path.join(self.build_iotests, '..', '..') self.init_directories() + + if dry_run: + return + self.init_binaries() self.malloc_perturb_ = os.getenv('MALLOC_PERTURB_', From patchwork Fri Mar 3 16:07:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751510 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=S1XRBAdd; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStF444Rsz23j7 for ; Sat, 4 Mar 2023 03:08:40 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wf-0007fV-Em; Fri, 03 Mar 2023 11:07:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wZ-0007dK-09 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wX-0005jT-L7 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859657; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=43TfmiRzPU1in48uLgR11fNNvns7/DBXgMw5OmgLsVs=; b=S1XRBAddledF2o8h+vwQ96zWhtWcIbphU/fcYMURAZLIHhGM2G7wZxdKb/X0BrGIc2BiPN KoLYjt9GQZjkajrnnqSIroRhsmlk1tyOfDnsvxG5jqLLIv0iz7maQlJQbphqWcNSXaH6mO hM5G97g2EmvDYVD/aLYhyEuTRibGj0k= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-428-y7EB1vjmNSO6btrdNTbdpQ-1; Fri, 03 Mar 2023 11:07:35 -0500 X-MC-Unique: y7EB1vjmNSO6btrdNTbdpQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 610433806105; Fri, 3 Mar 2023 16:07:35 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 117B3492C14; Fri, 3 Mar 2023 16:07:33 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 3/8] iotests: strip subdir path when listing tests Date: Fri, 3 Mar 2023 16:07:22 +0000 Message-Id: <20230303160727.3977246-4-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When asking 'check' to list individual tests by invoking it in dry run mode, it prints the paths to the tests relative to the base of the I/O test directory. When asking 'check' to run an individual test, however, it mandates that only the unqualified test name is given, without any path prefix. This inconsistency makes it harder to ask for a list of tests and then invoke each one. Thus the test listing code is change to flatten the test names, by printing only the base name, which can be directly invoked. Reviewed-by: Alex Bennée Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/check | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index bb294ef556..f2e9d27dcf 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -184,7 +184,7 @@ if __name__ == '__main__': sys.exit(str(e)) if args.dry_run: - print('\n'.join(tests)) + print('\n'.join([os.path.basename(t) for t in tests])) else: with TestRunner(env, tap=args.tap, color=args.color) as tr: From patchwork Fri Mar 3 16:07:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751513 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Q3skgv/z; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStFL1gGMz23j7 for ; Sat, 4 Mar 2023 03:08:54 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wi-0007mc-Sb; Fri, 03 Mar 2023 11:07:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wa-0007dp-R3 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:41 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wZ-0005k4-Es for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:40 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RevuAZO5F7B+a4EaTUYU4hCzGIlXik703DRwedfc8xQ=; b=Q3skgv/zm//lmaMAFMdLFwQXpFFKlszSFKH9AL4797/srZqmGeh0KBubWMsQdd2nxuZczK nm2h3KdVHWZjQcaxIeTDdn47kQObYBp7bedhUIWb/RuvHY/EXP7aZKxSBNby+gGHakUva6 f6fg0zotwXnaBbJrzEDcbl8472joCCg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-178-XT095Vc6Pl2BZCwHN_J_Wg-1; Fri, 03 Mar 2023 11:07:37 -0500 X-MC-Unique: XT095Vc6Pl2BZCwHN_J_Wg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EAC05858F0E; Fri, 3 Mar 2023 16:07:36 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9FDCB492C14; Fri, 3 Mar 2023 16:07:35 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 4/8] iotests: print TAP protocol version when reporting tests Date: Fri, 3 Mar 2023 16:07:23 +0000 Message-Id: <20230303160727.3977246-5-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Recently meson started complaining that TAP test reports don't include the TAP protocol version. While this warning is bogus and has since been removed from Meson, it looks like good practice to include this header going forward. The GLib library test harness has started unconditionally printing the version, so this brings the I/O tests into line. Reviewed-by: Thomas Huth Reviewed-by: Alex Bennée Signed-off-by: Daniel P. Berrangé --- tests/qemu-iotests/testrunner.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 5a771da86e..e734800b3d 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -391,6 +391,7 @@ def run_tests(self, tests: List[str], jobs: int = 1) -> bool: casenotrun = [] if self.tap: + print('TAP version 13') self.env.print_env('# ') print('1..%d' % len(tests)) else: From patchwork Fri Mar 3 16:07:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751512 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JMQB2qGT; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStFK3HrVz23j7 for ; Sat, 4 Mar 2023 03:08:53 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wg-0007ha-Km; Fri, 03 Mar 2023 11:07:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wc-0007eq-Uv for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:44 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wa-0005kP-U0 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859660; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p++4zIgGZrX/Z/sDx3hiWSacNj22/J8kQgnF2nrA63w=; b=JMQB2qGTlOSazfDC+3aJFyqDLe6h6Jrw7YksbwMMsz4w/itOzT5a/Q3tIEJYEvfk8sHUbQ rT0VIBEMKMNVxbScPDgj2u73pK+4F13xnfwvLVdXCXps4I4PxdMtNP/qtybjMsmK8Bawnn jIGhKY0XAlI/XEtmMN/33uwY56I7xKg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-278-VaGGxrC8PHKLho1OZdmSwg-1; Fri, 03 Mar 2023 11:07:39 -0500 X-MC-Unique: VaGGxrC8PHKLho1OZdmSwg-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AFC26101A52E; Fri, 3 Mar 2023 16:07:38 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 37D4E492C14; Fri, 3 Mar 2023 16:07:37 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 5/8] iotests: connect stdin to /dev/null when running tests Date: Fri, 3 Mar 2023 16:07:24 +0000 Message-Id: <20230303160727.3977246-6-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently the tests have their stdin inherited from the test harness, meaning they are connected to a TTY. The QEMU processes spawned by certain tests, however, modify TTY settings and if the test exits abnormally the settings might not be restored. The python test harness thus has some logic which will capture the initial TTY settings and restore them once all tests are finished. This does not, however, take into account the possibility of many copies of the 'check' program running in parallel. With parallel execution, a later invokation may save the TTY state that QEMU has already modified, and thus restore bad state leaving the TTY non-functional. None of the I/O tests shnould actually be interactive requiring user input and so they should not require a TTY at all. To avoid this while TTY save/restore complexity we can connect the test stdin to /dev/null instead. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qemu-iotests/testrunner.py | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index e734800b3d..81519ed6e2 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -24,12 +24,10 @@ import subprocess import contextlib import json -import termios import shutil import sys from multiprocessing import Pool -from contextlib import contextmanager -from typing import List, Optional, Iterator, Any, Sequence, Dict, \ +from typing import List, Optional, Any, Sequence, Dict, \ ContextManager from testenv import TestEnv @@ -56,22 +54,6 @@ def file_diff(file1: str, file2: str) -> List[str]: return res -# We want to save current tty settings during test run, -# since an aborting qemu call may leave things screwed up. -@contextmanager -def savetty() -> Iterator[None]: - isterm = sys.stdin.isatty() - if isterm: - fd = sys.stdin.fileno() - attr = termios.tcgetattr(fd) - - try: - yield - finally: - if isterm: - termios.tcsetattr(fd, termios.TCSADRAIN, attr) - - class LastElapsedTime(ContextManager['LastElapsedTime']): """ Cache for elapsed time for tests, to show it during new test run @@ -169,7 +151,6 @@ def __enter__(self) -> 'TestRunner': self._stack = contextlib.ExitStack() self._stack.enter_context(self.env) self._stack.enter_context(self.last_elapsed) - self._stack.enter_context(savetty()) return self def __exit__(self, exc_type: Any, exc_value: Any, traceback: Any) -> None: @@ -294,6 +275,7 @@ def do_run_test(self, test: str, mp: bool) -> TestResult: t0 = time.time() with f_bad.open('w', encoding="utf-8") as f: with subprocess.Popen(args, cwd=str(f_test.parent), env=env, + stdin=subprocess.DEVNULL, stdout=f, stderr=subprocess.STDOUT) as proc: try: proc.wait() From patchwork Fri Mar 3 16:07:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751509 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NCwmwSbp; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStF10vj8z23j7 for ; Sat, 4 Mar 2023 03:08:37 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wk-0007pp-9t; Fri, 03 Mar 2023 11:07:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wh-0007jI-A5 for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wf-0005lU-QF for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g081gxg4TZ4oy1nAGXBSfCdUr8j/qtgE119mN5AvAgA=; b=NCwmwSbp9cYkBjnx7J36vJuyjcW6T9YX/9ugati4nQhDK5xpYuVMLiY9BElsq+XISznurN bBuUk77x5nbHV6G8UfkM8VVxwO3CUAXxm31mfJlA7xypEnlk5+KM77tHib5l6Ng09HrOdJ iZflZ+4IfKGFNwNlfNdbfjCxqqZ3tHY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-261-TMbQ7gkFMsi4XRQ9z902dw-1; Fri, 03 Mar 2023 11:07:40 -0500 X-MC-Unique: TMbQ7gkFMsi4XRQ9z902dw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 57328811E6E; Fri, 3 Mar 2023 16:07:40 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id F281A492C18; Fri, 3 Mar 2023 16:07:38 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 6/8] iotests: always use a unique sub-directory per test Date: Fri, 3 Mar 2023 16:07:25 +0000 Message-Id: <20230303160727.3977246-7-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The current test runner is only safe against parallel execution within a single instance of the 'check' process, and only if -j is given a value greater than 2. This prevents running multiple copies of the 'check' process for different test scenarios. This change switches the output / socket directories to always include the test name, image format and image protocol. This should allow full parallelism of all distinct test scenarios. eg running both qcow2 and raw tests at the same time, or both file and nbd tests at the same time. It would be possible to allow for parallelism of the same test scenario by including the pid, but that would potentially let many directories accumulate over time on failures, so is not done. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qemu-iotests/testrunner.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 81519ed6e2..7b322272e9 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -228,13 +228,11 @@ def find_reference(self, test: str) -> str: return f'{test}.out' - def do_run_test(self, test: str, mp: bool) -> TestResult: + def do_run_test(self, test: str) -> TestResult: """ Run one test :param test: test file path - :param mp: if true, we are in a multiprocessing environment, use - personal subdirectories for test run Note: this method may be called from subprocess, so it does not change ``self`` object in any way! @@ -257,12 +255,14 @@ def do_run_test(self, test: str, mp: bool) -> TestResult: args = [str(f_test.resolve())] env = self.env.prepare_subprocess(args) - if mp: - # Split test directories, so that tests running in parallel don't - # break each other. - for d in ['TEST_DIR', 'SOCK_DIR']: - env[d] = os.path.join(env[d], f_test.name) - Path(env[d]).mkdir(parents=True, exist_ok=True) + + # Split test directories, so that tests running in parallel don't + # break each other. + for d in ['TEST_DIR', 'SOCK_DIR']: + env[d] = os.path.join( + env[d], + f"{self.env.imgfmt}-{self.env.imgproto}-{f_test.name}") + Path(env[d]).mkdir(parents=True, exist_ok=True) test_dir = env['TEST_DIR'] f_bad = Path(test_dir, f_test.name + '.out.bad') @@ -347,7 +347,7 @@ def run_test(self, test: str, testname = os.path.basename(test) print(f'# running {self.env.imgfmt} {testname}') - res = self.do_run_test(test, mp) + res = self.do_run_test(test) end = datetime.datetime.now().strftime('%H:%M:%S') self.test_print_one_line(test=test, From patchwork Fri Mar 3 16:07:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751515 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ahKAD1Y9; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStFk6Kt4z23j7 for ; Sat, 4 Mar 2023 03:09:14 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wj-0007nZ-FC; Fri, 03 Mar 2023 11:07:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wh-0007lP-SF for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wg-0005la-2L for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a1bKkjmVZRsM4WdYPvIOEvaC+vfwnALk1lioaQDxSfw=; b=ahKAD1Y9JjMZgb6LLY2NKrB/uIvFq3UCjPm0FwSZmTbN8qq9ZP2KGRnHxHJbScA9Z5yn1D 8NGQHG40dyQdpD/5n9uDTsh2IN27BZQX87qcnKvak3vPmKtPxWKCSO5O+/47N98p5rmMKO tVUJ0jAndnp1epLlNP+2O1Xy4quH8Gw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-4rUEFYJCNiSvlnF849urzA-1; Fri, 03 Mar 2023 11:07:42 -0500 X-MC-Unique: 4rUEFYJCNiSvlnF849urzA-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DE569882820; Fri, 3 Mar 2023 16:07:41 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 91EFB492C14; Fri, 3 Mar 2023 16:07:40 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 7/8] iotests: register each I/O test separately with meson Date: Fri, 3 Mar 2023 16:07:26 +0000 Message-Id: <20230303160727.3977246-8-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.129.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently meson registers a single test that invokes an entire group of I/O tests, hiding the test granularity from meson. There are various downsides of doing this * You cannot ask 'meson test' to invoke a single I/O test * The meson test timeout can't be applied to the individual tests * Meson only gets a pass/fail for the overall I/O test group not individual tests * If a CI job gets killed by the GitLab timeout, we don't get visibility into how far through the I/O tests execution got. This switches meson to perform test discovery by invoking 'check' in dry-run mode. It then registers one meson test case for each I/O test. Parallel execution remains disabled since the I/O tests do not use self contained execution environments and thus conflict with each other. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/qemu-iotests/meson.build | 35 ++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index 323a4acb6a..a162f683ef 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -32,16 +32,39 @@ foreach k, v : emulators endif endforeach +qemu_iotests_check_cmd = files('check') + foreach format, speed: qemu_iotests_formats if speed == 'quick' suites = 'block' else suites = ['block-' + speed, speed] endif - test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format], - depends: qemu_iotests_binaries, env: qemu_iotests_env, - protocol: 'tap', - suite: suites, - timeout: 0, - is_parallel: false) + + args = ['-tap', '-' + format] + if speed == 'quick' + args += ['-g', 'auto'] + endif + + rc = run_command( + [qemu_iotests_check_cmd] + args + ['-n'], + check: true, + ) + + foreach item: rc.stdout().strip().split() + args = ['-tap', '-' + format, item, + '--source-dir', meson.current_source_dir(), + '--build-dir', meson.current_build_dir()] + # Some individual tests take as long as 45 seconds + # Bump the timeout to 3 minutes for some headroom + # on slow machines to minimize spurious failures + test('io-' + format + '-' + item, + qemu_iotests_check_cmd, + args: args, + depends: qemu_iotests_binaries, + env: qemu_iotests_env, + protocol: 'tap', + timeout: 180, + suite: suites) + endforeach endforeach From patchwork Fri Mar 3 16:07:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= X-Patchwork-Id: 1751511 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Gc72IU7L; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PStF71bCNz23j7 for ; Sat, 4 Mar 2023 03:08:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY7wk-0007qM-KA; Fri, 03 Mar 2023 11:07:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wi-0007m6-GA for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY7wg-0005lq-Vs for qemu-devel@nongnu.org; Fri, 03 Mar 2023 11:07:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677859666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xXP+QZ5btQMDSVmov/iZLeOsE7tEli1ClvUh6mgqdto=; b=Gc72IU7LuROYz92UglUkk1yM8/uG9kqldbIE2Zxu1dIyCBNhIrCj/9Kx36a/LibBB0X1FU Z0GP9eJw+wCWx8kyfXGxOSS0c73e+aUgOcWYMBZRtPz3IZnEjpaHD/vLbJbvejDjvoQUki iO/aJcGbdq1aKhM3MPJtuP8fsu0q48U= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-516-iKesmzJRP_Owt-7mqfEPAQ-1; Fri, 03 Mar 2023 11:07:43 -0500 X-MC-Unique: iKesmzJRP_Owt-7mqfEPAQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 896E4811E9C; Fri, 3 Mar 2023 16:07:43 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.33.36.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2BC57492C18; Fri, 3 Mar 2023 16:07:42 +0000 (UTC) From: =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Paolo Bonzini , Kevin Wolf , qemu-block@nongnu.org, Thomas Huth , Hanna Reitz , =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_P?= =?utf-8?q?=2E_Berrang=C3=A9?= Subject: [PATCH v2 8/8] iotests: remove the check-block.sh script Date: Fri, 3 Mar 2023 16:07:27 +0000 Message-Id: <20230303160727.3977246-9-berrange@redhat.com> In-Reply-To: <20230303160727.3977246-1-berrange@redhat.com> References: <20230303160727.3977246-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Now that meson directly invokes the individual I/O tests, the check-block.sh wrapper script is no longer required. Signed-off-by: Daniel P. Berrangé Reviewed-by: Thomas Huth --- tests/check-block.sh | 43 ------------------------------------------- 1 file changed, 43 deletions(-) delete mode 100755 tests/check-block.sh diff --git a/tests/check-block.sh b/tests/check-block.sh deleted file mode 100755 index 5de2c1ba0b..0000000000 --- a/tests/check-block.sh +++ /dev/null @@ -1,43 +0,0 @@ -#!/bin/sh - -if [ "$#" -eq 0 ]; then - echo "Usage: $0 fmt..." >&2 - exit 99 -fi - -# Honor the SPEED environment variable, just like we do it for "meson test" -format_list="$@" -if [ "$SPEED" = "slow" ] || [ "$SPEED" = "thorough" ]; then - group= -else - group="-g auto" -fi - -skip() { - echo "1..0 #SKIP $*" - exit 0 -} - -if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then - skip "No qemu-system binary available ==> Not running the qemu-iotests." -fi - -cd tests/qemu-iotests - -# QEMU_CHECK_BLOCK_AUTO is used to disable some unstable sub-tests -export QEMU_CHECK_BLOCK_AUTO=1 -export PYTHONUTF8=1 -# If make was called with -jN we want to call ./check with -j N. Extract the -# flag from MAKEFLAGS, so that if it absent (or MAKEFLAGS is not defined), JOBS -# would be an empty line otherwise JOBS is prepared string of flag with value: -# "-j N" -# Note, that the following works even if make was called with "-j N" or even -# "--jobs N", as all these variants becomes simply "-jN" in MAKEFLAGS variable. -JOBS=$(echo "$MAKEFLAGS" | sed -n 's/\(^\|.* \)-j\([0-9]\+\)\( .*\|$\)/-j \2/p') - -ret=0 -for fmt in $format_list ; do - ${PYTHON} ./check $JOBS -tap -$fmt $group || ret=1 -done - -exit $ret