From patchwork Fri Nov 27 16:13:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Wakely X-Patchwork-Id: 1407315 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gcc.gnu.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=hpcGncDw; dkim-atps=neutral Received: from sourceware.org (unknown [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CjKS316z3z9s0b for ; Sat, 28 Nov 2020 03:13:39 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5C7A039730D7; Fri, 27 Nov 2020 16:13:37 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5C7A039730D7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1606493617; bh=VgaV2odNt0kyozbLATCPcpZzvbrMoUfwUCVEbkMXdmw=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=hpcGncDwfWAxGQfwGn1rVgaqvCOVOcOI4yugafSnrIvOZcEdeTyPPTdQZ5j7z3cyK i7F05z5IJ2IDmEA69eXQpf0oUS+Yc76AopMne+1hdWfXF/2eBQI1wHIp9VGFi7md1q TFlwShGtlVk23wjb81rrCjdIqjxlKxEa7sFY9rv4= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by sourceware.org (Postfix) with ESMTP id 65C193973053 for ; Fri, 27 Nov 2020 16:13:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 65C193973053 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-418-MN1AYToEPg2gKfvsp4wmhA-1; Fri, 27 Nov 2020 11:13:32 -0500 X-MC-Unique: MN1AYToEPg2gKfvsp4wmhA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 586D5107AFAE; Fri, 27 Nov 2020 16:13:31 +0000 (UTC) Received: from localhost (unknown [10.33.37.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0900C18993; Fri, 27 Nov 2020 16:13:30 +0000 (UTC) Date: Fri, 27 Nov 2020 16:13:30 +0000 To: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: [RFC] Decrease default timeout for libstdc++ tests to 6 minutes Message-ID: <20201127161330.GT1312820@redhat.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-Spam-Status: No, score=-8.0 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Jonathan Wakely via Gcc-patches From: Jonathan Wakely Reply-To: Jonathan Wakely Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" The default for the GCC testsuite is 300, i.e. 5 minutes, which is the same as the DejaGnu default. Libstdc++ overrides this to 600, i.e. 10 minutes. This seems ridiculously long. If any test takes that long on modern hardware, something is wrong. We've seen this a lot recently with buggy tests, and waiting for them to FAIL is tedious. I've already made libstdc++.exp respect the user's setting in ~/.dejagnurc or the global site.exp file. This means anybody testing on slow simulators or old hardware can choose their own timeout. I've added dg-timeout-factor to the slowest std::regex tests and have a patch to do it for the PSTL tests, which both take far too long to compile. That means you can choose a sensible timeout appropriate for most tests (e.g. 60 seconds) and not get spurious failures from the few dozen tests which are just very slow. I'd like to change the default to 6 minutes. If that goes well, I'd like to lower it even further. The main benefit of this will be that buggy tests which hang will get killed sooner, so we waste less time waiting for the inevitable timeout. Here's the proposed patch for completeness: --- a/libstdc++-v3/testsuite/lib/libstdc++.exp +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp @@ -248,7 +248,7 @@ proc libstdc++_init { testfile } { # Set the default timeout for v3 tests. # You can override this in ~/.dejagnurc or a .exp file named by $DEJAGNU. if {![info exists tool_timeout]} { - set tool_timeout 600 + set tool_timeout 360 } # Default settings.