From patchwork Mon Aug 13 18:03:06 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Diego Novillo X-Patchwork-Id: 177015 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 2B6972C0086 for ; Tue, 14 Aug 2012 04:03:31 +1000 (EST) Comment: DKIM? See http://www.dkim.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=gcc.gnu.org; s=default; x=1345485812; h=Comment: DomainKey-Signature:Received:Received:Received:Received:Received: Received:Received:Received:Received:Date:From:To:Subject: Message-ID:MIME-Version:Content-Type:Content-Disposition: User-Agent:Mailing-List:Precedence:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:Sender:Delivered-To; bh=1yivnoO 3XtWgw7RzT2q0L9QvuKA=; b=dPE4ldQSm/I4idZXXoZ1kiXy22fCnZwA2Ua4lXh mXd6q4lNKsJMBZ98SlRC/dHuggQDh7ryRkUqo10AzSQqc0tg4nDZJqvMoT2qpiKR h0dFIK9sq3egnatDc2v1rF29ZHeGwatEiCLIXBailfdzxlROvcuQY7Kbh+Y1lmln IPhc= Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=gcc.gnu.org; h=Received:Received:X-SWARE-Spam-Status:X-Spam-Check-By:Received:Received:X-Google-DKIM-Signature:Received:Received:Received:Received:Received:Date:From:To:Subject:Message-ID:MIME-Version:Content-Type:Content-Disposition:User-Agent:X-Gm-Message-State:X-IsSubscribed:Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:Sender:Delivered-To; b=rhzVTFCd+2VOH/rDNPEJs2nWhCWZhIIPh+NKpj+qrZs3XEaMvfwmlULB5ZcwA5 N8xzI0fTKfI/EwAGkR4BxynoniemXzlRkjbLXe3hxhrIw5al1q4ZA7yIIYQrL4u8 w5lcgp4aPMYF3QkeVsJ6xfxIcaaAwTF0B6Tx43nAHBej0=; Received: (qmail 22352 invoked by alias); 13 Aug 2012 18:03:24 -0000 Received: (qmail 22342 invoked by uid 22791); 13 Aug 2012 18:03:22 -0000 X-SWARE-Spam-Status: No, hits=-4.2 required=5.0 tests=AWL, BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, KHOP_RCVD_TRUST, RCVD_IN_DNSWL_LOW, RCVD_IN_HOSTKARMA_YE, T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from mail-ob0-f201.google.com (HELO mail-ob0-f201.google.com) (209.85.214.201) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Mon, 13 Aug 2012 18:03:08 +0000 Received: by obbun3 with SMTP id un3so2918266obb.2 for ; Mon, 13 Aug 2012 11:03:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-disposition:organization:user-agent:x-gm-message-state; bh=WD8SMmZMVsxK/gH7h05pJroYtfQqRSrJgm570No4tRM=; b=enpQ7gfJjfnBjOmozMOVZRBMVmEGVpkHCV2wpDzLNxwZ/1B9LhJ5a0TiUSjNgLlJVH Z+TVcxVNWFTF5KRb3hF9wUs9xMp8GYZfsTqj53qYYUnvGxRjx255rxwPYR5dcDJJiINY AB3NQNiiRlJmsMNmmkkv6gDNVDBomR0wS6SiAnlfe3P0q44x+OxhArLUjEgr2cInp/fj bVJLCNZ8SAbBeXYb5d5RxBXnPeM4psFf++v5UDE2sLHhhvVGm5roPIBGJvovI9XGQwfi vy7k0YDcKeaRuBIp77eobP1zuNitwZwBDEuQPYdltmcpfQuCspLIJzVcqldWYU72SQOO XZlw== Received: by 10.43.131.7 with SMTP id ho7mr3564940icc.5.1344880987838; Mon, 13 Aug 2012 11:03:07 -0700 (PDT) Received: by 10.43.131.7 with SMTP id ho7mr3564936icc.5.1344880987785; Mon, 13 Aug 2012 11:03:07 -0700 (PDT) Received: from wpzn4.hot.corp.google.com (216-239-44-65.google.com [216.239.44.65]) by gmr-mx.google.com with ESMTPS id mb9si1880278igc.1.2012.08.13.11.03.07 (version=TLSv1/SSLv3 cipher=AES128-SHA); Mon, 13 Aug 2012 11:03:07 -0700 (PDT) Received: from torture.tor.corp.google.com (torture.tor.corp.google.com [172.29.41.4]) by wpzn4.hot.corp.google.com (Postfix) with ESMTP id 45B311E004A; Mon, 13 Aug 2012 11:03:07 -0700 (PDT) Received: by torture.tor.corp.google.com (Postfix, from userid 54752) id 72470C0E7A; Mon, 13 Aug 2012 14:03:06 -0400 (EDT) Date: Mon, 13 Aug 2012 14:03:06 -0400 From: Diego Novillo To: gcc-patches@gcc.gnu.org, simonb@google.com Subject: [contrib] Add expiration support for validate_failures.py Message-ID: <20120813180306.GA24501@google.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Gm-Message-State: ALoCoQnVPU4k1OeHdEYvlFA1SJY7kIBQfySViPaq1vP0Fk5b9htiKjPXglSm9MmRAb0CVlChzcbIG2fCzhz+4lFr0o/rpMQwJa9bgDFaYy0WJUXHswV5qjIJnx+GnPXtCVu+HeeCSpk6fgB7jLuznZ56sZL1X1lY3akCr6+2Dqvjw//JSPkoTS4C86j3cJ17JlVCvA2Toz1v5lj81XL0HV4tZXUnq/ld142lDPYYe0fI3c7lxsBtUy8= X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org I noticed recently that while the validator was accepting the 'expire=YYYYMMDD' attribute, it was not actually doing anything with it. This patch fixes the oversight. Simon, I will be backporting the patch to google/gcc-4_7. Committed to trunk. 2012-08-13 Diego Novillo * testsuite-management/validate_failures.py: Import datetime. (TestResult.ExpirationDate): New. (TestResult.HasExpired): New. (ParseSummary): Call it. If it returns True, warn that the expected failure has expired and do not add it to the set of expected results. (GetResults): Clarify documentation. diff --git a/contrib/testsuite-management/validate_failures.py b/contrib/testsuite-management/validate_failures.py index ef01938..0ac9b15 100755 --- a/contrib/testsuite-management/validate_failures.py +++ b/contrib/testsuite-management/validate_failures.py @@ -46,6 +46,7 @@ executed it will: with exit code 0. Otherwise, it exits with error code 1. """ +import datetime import optparse import os import re @@ -135,6 +136,26 @@ class TestResult(object): attrs = '%s | ' % self.attrs return '%s%s: %s %s' % (attrs, self.state, self.name, self.description) + def ExpirationDate(self): + # Return a datetime.date object with the expiration date for this + # test result expires. Return None, if no expiration # has been set. + if re.search(r'expire=', self.attrs): + expiration = re.search(r'expire=(\d\d\d\d)(\d\d)(\d\d)', self.attrs) + if not expiration: + Error('Invalid expire= format in "%s". Must be of the form ' + '"expire=YYYYMMDD"' % self) + return datetime.date(int(expiration.group(1)), + int(expiration.group(2)), + int(expiration.group(3))) + return None + + def HasExpired(self): + # Return True if the expiration date of this result has passed. + expiration_date = self.ExpirationDate() + if expiration_date: + now = datetime.date.today() + return now > expiration_date + def GetMakefileValue(makefile_name, value_name): if os.path.exists(makefile_name): @@ -178,7 +199,13 @@ def ParseSummary(sum_fname): sum_file = open(sum_fname) for line in sum_file: if IsInterestingResult(line): - result_set.add(TestResult(line)) + result = TestResult(line) + if result.HasExpired(): + # Tests that had an expiration set are not added to the + # set of expected results. + print 'WARNING: Expected failure "%s" has expired.' % line.strip() + continue + result_set.add(result) sum_file.close() return result_set @@ -220,16 +247,20 @@ def GetResults(sum_files): def CompareResults(manifest, actual): """Compare sets of results and return two lists: - - List of results present in MANIFEST but missing from ACTUAL. - List of results present in ACTUAL but missing from MANIFEST. + - List of results present in MANIFEST but missing from ACTUAL. """ - # Report all the actual results not present in the manifest. + # Collect all the actual results not present in the manifest. + # Results in this set will be reported as errors. actual_vs_manifest = set() for actual_result in actual: if actual_result not in manifest: actual_vs_manifest.add(actual_result) - # Simlarly for all the tests in the manifest. + # Collect all the tests in the manifest that were not found + # in the actual results. + # Results in this set will be reported as warnings (since + # they are expected failures that are not failing anymore). manifest_vs_actual = set() for expected_result in manifest: # Ignore tests marked flaky.