From patchwork Tue Sep 20 09:26:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lyon X-Patchwork-Id: 672173 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3sdcpX5Fmnz9sxS for ; Tue, 20 Sep 2016 19:26:59 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b=Tkb9HAqA; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; q= dns; s=default; b=seGZVJWObMQDou8duaOx2AaBl65w2X7UrLglYxmnpyXFET OFyfTJrhl7/EKmne9kTbvTxvBuQyy+oT6fGf6E72sHCCx0Rr3nzMSr9pRDt2MiCw j0pWdKd7SOKyG+Z+Dvn2/FZB0u/SwQ6aPw9VSKs87Ywu3zigY6ehOg8blKE6Q= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; s= default; bh=cwVtBLVHJVy08m3Rh33URDDZWr8=; b=Tkb9HAqAJ0BdzL/aH4wF +pUp1f5kOfRrhl/ILhevCitteupMLgJaHlYYQzv92aXeKKdLJuG7TrfK8TsiSk9r bWUKSm9TqzBsUeISONXA0vQ+dTqPxNCf4N4HuvKnL1R/TLXCsovj3XBlQCmPFDMr Z9jlReVIUywmdb3LjtI3Gzo= Received: (qmail 42448 invoked by alias); 20 Sep 2016 09:26:49 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 42400 invoked by uid 89); 20 Sep 2016 09:26:48 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy=Running, H*c:sk:001a113, Take, 2016-09-20 X-HELO: mail-qt0-f173.google.com Received: from mail-qt0-f173.google.com (HELO mail-qt0-f173.google.com) (209.85.216.173) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 20 Sep 2016 09:26:38 +0000 Received: by mail-qt0-f173.google.com with SMTP id 93so4613514qtg.2 for ; Tue, 20 Sep 2016 02:26:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=a1MwLf6z3vPEYiF2XLk2jEUEQpRT39gDbwf1UJBGqYA=; b=ArYkk+tSfzTwGiGhqS5de2dfvENhQ9vJe6JcNMOrR+0kmcLwPJsBmf8+FJ38a5x49q BySi7b4XsTlktHCrfQ5Oe6LXXDbXJi9hlhXf9HqBA/F3/R5a+L+mRIvMVDf5dSgQj3Q6 qkmSWnRiG4NhEZ0d70YG+sN8K/6fRQBRF5qZOrtjV27N6/Ny1AEXSyRr259etFBEaIYK 1zdXC1Z4BENFyxfxgl12+cJP3lCN1ZoDspdRZC/TnDygFQC20S5pGjvQByen6rsb/6vv a68mfEjWWuns6ffVc9a+gttjWdc+89CzjM8mo7urP/qzp548kLcyOBc7ijrGAfdZSt58 hGiQ== X-Gm-Message-State: AE9vXwNBJAJwsQZcN1jcjsf5YvT4RYEmK09beatjuidMe815fWPO8lCVqJysERxBcfCANNV9no88uue+/heDSvH9 X-Received: by 10.200.47.3 with SMTP id j3mr33062066qta.27.1474363596727; Tue, 20 Sep 2016 02:26:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.21.80 with HTTP; Tue, 20 Sep 2016 02:26:35 -0700 (PDT) From: Christophe Lyon Date: Tue, 20 Sep 2016 11:26:35 +0200 Message-ID: Subject: Report DejaGnu ERROR messages in compare_tests To: "gcc-patches@gcc.gnu.org" X-IsSubscribed: yes Hello, We recently faced a problem where a DejaGnu error went un-noticed (https://gcc.gnu.org/ml/gcc-patches/2016-08/msg01879.html). To help identify these problems earlier, here is a patch for compare_tests that will report such cases as: -------------------------------- New tests that FAIL: arm-sim: (DejaGnu) proc "scan-dump-tree-not fail_test optimized" does not exist. -------------------------------- OK? Thanks, Christophe contrib/ChangeLog: 2016-09-20 Christophe Lyon * compare_tests: Take ERROR messages into account when comparing. diff --git a/contrib/compare_tests b/contrib/compare_tests index 8ecf824..d16e7e9 100755 --- a/contrib/compare_tests +++ b/contrib/compare_tests @@ -107,8 +107,8 @@ elif [ -d "$1" -o -d "$2" ] ; then usage "Must specify either two directories or two files" fi -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 before=$tmp1 now=$tmp2