{"id":1882103,"url":"http://patchwork.ozlabs.org/api/patches/1882103/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20240103173349.398526-21-alex.bennee@linaro.org/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20240103173349.398526-21-alex.bennee@linaro.org>","list_archive_url":null,"date":"2024-01-03T17:33:26","name":"[v2,20/43] mtest2make: stop disabling meson test timeouts","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"1cb029b9ee58191879af354c038598da3ec1334f","submitter":{"id":39532,"url":"http://patchwork.ozlabs.org/api/people/39532/?format=json","name":"Alex Bennée","email":"alex.bennee@linaro.org"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20240103173349.398526-21-alex.bennee@linaro.org/mbox/","series":[{"id":388742,"url":"http://patchwork.ozlabs.org/api/series/388742/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=388742","date":"2024-01-03T17:33:08","name":"testing and plugin updates for 9.0 (pre-PR)","version":2,"mbox":"http://patchwork.ozlabs.org/series/388742/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/1882103/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/1882103/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@legolas.ozlabs.org","Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256\n header.s=google header.b=gxIsOFzW;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=patchwork.ozlabs.org)"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4T4xp72KxXz1ydb\n\tfor ; Thu, 4 Jan 2024 04:40:47 +1100 (AEDT)","from localhost ([::1] helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1rL5DG-0006v9-LY; Wed, 03 Jan 2024 12:39:30 -0500","from eggs.gnu.org ([2001:470:142:3::10])\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1rL5D1-00068q-Vo\n for qemu-devel@nongnu.org; Wed, 03 Jan 2024 12:39:15 -0500","from mail-wm1-x32d.google.com ([2a00:1450:4864:20::32d])\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from )\n id 1rL5Cu-0000c2-Oq\n for qemu-devel@nongnu.org; Wed, 03 Jan 2024 12:39:15 -0500","by mail-wm1-x32d.google.com with SMTP id\n 5b1f17b1804b1-40d604b4b30so4368485e9.1\n for ; Wed, 03 Jan 2024 09:39:08 -0800 (PST)","from draig.lan ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id\n dr16-20020a5d5f90000000b003373ef060d5sm8567186wrb.113.2024.01.03.09.39.05\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Wed, 03 Jan 2024 09:39:05 -0800 (PST)","from draig.lan (localhost [IPv6:::1])\n by draig.lan (Postfix) with ESMTP id 298A25F950;\n Wed, 3 Jan 2024 17:33:51 +0000 (GMT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=linaro.org; s=google; t=1704303546; x=1704908346; darn=nongnu.org;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:from:to:cc:subject:date\n :message-id:reply-to;\n bh=0sLk9Y6V09YT5mxoKGySQqDe7AUl7PrL/RJQ+Y1wnbw=;\n b=gxIsOFzWh6zn0hez+uEbsJ5bUiUqIo6ErnyzmnBKsBWFd4K9p1Azi4Lr2Qru3tc7C5\n u5RvlrXh3WukJLnbTTUORACUx4ZsZ9zKjiB4BBT6AEVLmb4g7bOz8+COgSmLAMO0ZhYg\n 7zrpCmH75KpJlyU0rbKP9s15GXsPJmCVlrEXjO1f6d6YXc0nxTlrmIeFnjDkNWOUgTQD\n ybkNmgu9LvLFEjHTub642lxMvOREptjiC0wkDBSalCaDSQxxnSdS99Ui8AUi9uwkQ3kD\n eH1HB/n4sapgFGrgNXItj6By6ZyLsY8rgnwYvNRap+Nciq8PRbD/INszUj2ejGsGnVYf\n /6yQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20230601; t=1704303546; x=1704908346;\n h=content-transfer-encoding:mime-version:references:in-reply-to\n :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n :subject:date:message-id:reply-to;\n bh=0sLk9Y6V09YT5mxoKGySQqDe7AUl7PrL/RJQ+Y1wnbw=;\n b=ZefWpLhGzVbOylYWidl1NUDavT8PSfybdafHznQCp3jTR+fx+hKR4fJhm30GK2a2q3\n QvVLzmp7aRW98I6zHoT2NcjvyJs1wlMVMEBIdbQZl7BH1PQbnFbspVe5JgV0eR7NTk83\n MT2iKbOgJ6rcsRGv0b5zqwie5XU/z/dZp5JMTWhgFgVhtq1w5Fzd99NeZcVCS1u/Ul4c\n JDNxEVg28TMFFXH9KiMZIjypXNeYJmdJnP57mCuk2DuxIZ67xbKBdSt5tHBpVzgXjjVX\n SJpeLBpRbUO5K8WUpjl9+6av1hCR+3O66wZDZAo+/dVKls4yJP6XS2soKLsWshPkKeOZ\n T3IQ==","X-Gm-Message-State":"AOJu0YxihnlolCQZwGmWcHusmwQowDXy6Du9u0aI2NUOsgndSRRu1G+c\n +f42+BtJvBkEUhlJYYJH5MaXzhcKL8kbiA==","X-Google-Smtp-Source":"\n AGHT+IGR8Qb9ZKRyeNjw27aD8Jf1B/xnmHbcoluCD13YEs3zMnsmdYYJyBx3xpWJIG1auhT+vGCdLA==","X-Received":"by 2002:a05:600c:4fc7:b0:40d:3f30:a09e with SMTP id\n o7-20020a05600c4fc700b0040d3f30a09emr804540wmq.84.1704303546006;\n Wed, 03 Jan 2024 09:39:06 -0800 (PST)","From":"=?utf-8?q?Alex_Benn=C3=A9e?= ","To":"qemu-devel@nongnu.org","Cc":"qemu-s390x@nongnu.org, qemu-ppc@nongnu.org,\n Richard Henderson ,\n Song Gao ,\n =?utf-8?q?Marc-Andr=C3=A9_Lureau?= ,\n David Hildenbrand , Aurelien Jarno ,\n Yoshinori Sato ,\n Yanan Wang , Bin Meng ,\n Laurent Vivier , Michael Rolnik ,\n Alexandre Iooss , David Woodhouse ,\n Laurent Vivier , Paolo Bonzini ,\n Brian Cain ,\n Daniel Henrique Barboza ,\n Beraldo Leal , Paul Durrant ,\n Mahmoud Mandour , Thomas Huth ,\n Liu Zhiwei , Cleber Rosa ,\n kvm@vger.kernel.org, Peter Maydell ,\n Wainer dos Santos Moschetta , =?utf-8?q?Alex_Benn?=\n\t=?utf-8?q?=C3=A9e?= , qemu-arm@nongnu.org,\n Weiwei Li ,\n =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= ,\n John Snow ,\n Daniel Henrique Barboza ,\n Nicholas Piggin , Palmer Dabbelt ,\n Marcel Apfelbaum ,\n Ilya Leoshkevich ,\n =?utf-8?q?C=C3=A9dric_Le_Goater?= ,\n \"Edgar E. Iglesias\" ,\n Eduardo Habkost ,\n Pierrick Bouvier , qemu-riscv@nongnu.org,\n Alistair Francis , =?utf-8?q?Daniel_P=2E_Berrang?=\n\t=?utf-8?q?=C3=A9?= ","Subject":"[PATCH v2 20/43] mtest2make: stop disabling meson test timeouts","Date":"Wed, 3 Jan 2024 17:33:26 +0000","Message-Id":"<20240103173349.398526-21-alex.bennee@linaro.org>","X-Mailer":"git-send-email 2.39.2","In-Reply-To":"<20240103173349.398526-1-alex.bennee@linaro.org>","References":"<20240103173349.398526-1-alex.bennee@linaro.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=UTF-8","Content-Transfer-Encoding":"8bit","Received-SPF":"pass client-ip=2a00:1450:4864:20::32d;\n envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32d.google.com","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,\n RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001,\n T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org"},"content":"From: Daniel P. Berrangé \n\nThe mtest2make.py script passes the arg '-t 0' to 'meson test' which\ndisables all test timeouts. This is a major source of pain when running\nin GitLab CI and a test gets stuck. It will stall until GitLab kills the\nCI job. This leaves us with little easily consumable information about\nthe stalled test. The TAP format doesn't show the test name until it is\ncompleted, and TAP output from multiple tests it interleaved. So we\nhave to analyse the log to figure out what tests had un-finished TAP\noutput present and thus infer which test case caused the hang. This is\nvery time consuming and error prone.\n\nBy allowing meson to kill stalled tests, we get a direct display of what\ntest program got stuck, which lets us more directly focus in on what\nspecific test case within the test program hung.\n\nThe other issue with disabling meson test timeouts by default is that it\nmakes it more likely that maintainers inadvertantly introduce slowdowns.\nFor example the recent-ish change that accidentally made migrate-test\ntake 15-20 minutes instead of around 1 minute.\n\nThe main risk of this change is that the individual test timeouts might\nbe too short to allow completion in high load scenarios. Thus, there is\nlikely to be some short term pain where we have to bump the timeouts for\ncertain tests to make them reliable enough. The preceeding few patches\nraised the timeouts for all failures that were immediately apparent\nin GitLab CI.\n\nEven with the possible short term instability, this should still be a\nnet win for debuggability of failed CI pipelines over the long term.\n\nSigned-off-by: Daniel P. Berrangé \nReviewed-by: Philippe Mathieu-Daudé \nMessage-ID: <20230717182859.707658-13-berrange@redhat.com>\nSigned-off-by: Thomas Huth \nMessage-Id: <20231215070357.10888-17-thuth@redhat.com>\nSigned-off-by: Alex Bennée \n---\n scripts/mtest2make.py | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)","diff":"diff --git a/scripts/mtest2make.py b/scripts/mtest2make.py\nindex 179dd548718..eb01a05ddbd 100644\n--- a/scripts/mtest2make.py\n+++ b/scripts/mtest2make.py\n@@ -27,7 +27,8 @@ def names(self, base):\n .speed.slow = $(foreach s,$(sort $(filter-out %-thorough, $1)), --suite $s)\n .speed.thorough = $(foreach s,$(sort $1), --suite $s)\n \n-.mtestargs = --no-rebuild -t 0\n+TIMEOUT_MULTIPLIER = 1\n+.mtestargs = --no-rebuild -t $(TIMEOUT_MULTIPLIER)\n ifneq ($(SPEED), quick)\n .mtestargs += --setup $(SPEED)\n endif\n","prefixes":["v2","20/43"]}