From patchwork Fri Mar 17 01:09:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Myers X-Patchwork-Id: 740081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vknLJ4kjsz9ryQ for ; Fri, 17 Mar 2017 12:09:28 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="Jkw/QQ3B"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; q=dns; s=default; b=mJpbaym06G7Hn7w+039howu69Gnyg 7u9UIltKeTJ1P/PQD0vrXz/mZWrEiQOOD6E28yLEzOvBqwzuYHOJYSamHLzzZCO9 wvVa87zm+EmbM4cVWgGtJXc+qdDXKArUSC90m+CfcZBPOsoWiqWJq1LJS9Wk6iqT HbUzbOyMHnt9tY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:date:from:to:subject:message-id:mime-version :content-type; s=default; bh=6aZFCrUks9qzJYCHpWZanbcSqpM=; b=Jkw /QQ3BKqlt9nL/9vtxzeodUi6LCa2p63OiCLsYE8gcm8qlCTiBUJpY6eRzcG1prVn 5vvTzEVbNH8BGHBvS3tvT/WURdjLTBnbhCM4ZC7nE6AO+z3DPxxG6sMXm3v7gUwo 0SJJeJj4vho4FuuwlGMNAL2tEeAk0DhT7r4rYQK8= Received: (qmail 68295 invoked by alias); 17 Mar 2017 01:09:18 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 68276 invoked by uid 89); 17 Mar 2017 01:09:17 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS, URIBL_RED autolearn=ham version=3.3.2 spammy= X-HELO: relay1.mentorg.com Date: Fri, 17 Mar 2017 01:09:06 +0000 From: Joseph Myers To: Subject: conformtest: Support system-specific XFAILs [committed] Message-ID: User-Agent: Alpine 2.20 (DEB 67 2015-01-07) MIME-Version: 1.0 X-ClientProxiedBy: svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) To svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) conformtest has an internal XFAIL mechanism to allow failures of individual expectations to be ignored, so that known hard-to-fix failures (e.g. those affecting ABIs or requiring kernel changes) do not cause the overall tests to FAIL and so hide other failures from the same (header, standard) pair. Various such bugs are system-specific, so this patch adds a mechanism to allow system-specific XFAILs. A system-independent XFAIL is achieved by putting "xfail-" at the start of the relevant expectation in the *-data files. A system-specific XFAIL instead uses "xfail[cond]-", where "cond" is a condition listed in conformtest-xfail-conds in a sysdeps makefile (so one for x32 might set conformtest-xfail-conds = x86_64-x32-linux, for example, and then an expectation for tv_nsec's type could use xfail[x86_64-x32-linux]-). The actual names are arbitrary, just needing to match between the makefiles and the expectations, and if necessary you can use "xfail[cond1|cond2]-" for a test that is expected to fail under multiple conditions. As with system-independent XFAILs, I think system-specific ones should have a bug filed in Bugzilla and a comment referencing that bug. Tested for x86_64, including with test expectations and makefiles changed to use the new facility. Committed. 2017-03-17 Joseph Myers * conform/conformtest.pl ($xfail_str): New variable. (--xfail=): New command-line option. (top level): Handle expectations starting xfail[cond]-. * conform/Makefile (conformtest-xfail): New variable. ($(conformtest-header-tests)): Pass $(conformtest-xfail) to conformtest.pl. diff --git a/conform/Makefile b/conform/Makefile index b393641..c07cc1d 100644 --- a/conform/Makefile +++ b/conform/Makefile @@ -192,13 +192,18 @@ test-xfail-XOPEN2K8/signal.h/conform = yes test-xfail-XOPEN2K8/sys/wait.h/conform = yes conformtest-cc-flags = -I../include $(+sysdep-includes) $(sysincludes) -I.. +# conformtest-xfail-conds may be set by a sysdeps Makefile fragment to +# a list of conditions that are considered to be true when encountered +# in xfail[cond]- lines in test expectations. +conformtest-xfail = $(if $(conformtest-xfail-conds),\ + --xfail='$(conformtest-xfail-conds)') $(conformtest-header-tests): $(objpfx)%/conform.out: \ conformtest.pl $(conformtest-headers-data) (set -e; std_hdr=$*; std=$${std_hdr%%/*}; hdr=$${std_hdr#*/}; \ mkdir -p $(@D)/scratch; \ $(PERL) -I. conformtest.pl --tmpdir=$(@D)/scratch --cc='$(CC)' \ --flags='$(conformtest-cc-flags)' --standard=$$std \ - --headers=$$hdr > $@); \ + --headers=$$hdr $(conformtest-xfail) > $@); \ $(evaluate-test) $(linknamespace-symlists-tests): $(objpfx)symlist-%: list-header-symbols.pl diff --git a/conform/conformtest.pl b/conform/conformtest.pl index 54b38f8..41682e9 100644 --- a/conform/conformtest.pl +++ b/conform/conformtest.pl @@ -7,8 +7,10 @@ use POSIX; $standard = "XOPEN2K8"; $CC = "gcc"; $tmpdir = "/tmp"; +$xfail_str = ""; GetOptions ('headers=s' => \@headers, 'standard=s' => \$standard, - 'flags=s' => \$flags, 'cc=s' => \$CC, 'tmpdir=s' => \$tmpdir); + 'flags=s' => \$flags, 'cc=s' => \$CC, 'tmpdir=s' => \$tmpdir, + 'xfail=s' => \$xfail_str); @headers = split(/,/,join(',',@headers)); # List of the headers we are testing. @@ -347,6 +349,15 @@ while ($#headers >= 0) { if (/^xfail-/) { s/^xfail-//; $xfail = 1; + } elsif (/^xfail\[([^\]]*)\]-/) { + my($xfail_cond) = $1; + s/^xfail\[([^\]]*)\]-//; + # "xfail[cond]-" or "xfail[cond1|cond2|...]-" means a failure of + # the test is allowed if any of the listed conditions are in the + # --xfail command-line option argument. + if ($xfail_str =~ /\b($xfail_cond)\b/) { + $xfail = 1; + } } my($optional) = 0; if (/^optional-/) {