From patchwork Tue Sep 20 09:26:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lyon X-Patchwork-Id: 672174 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3sdcpn5sSfz9sxS for ; Tue, 20 Sep 2016 19:27:13 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b=kPRuURpW; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; q= dns; s=default; b=DYDBPMFnoxJbHhHvBanw9bk5XrdZNQCfkQuAj8O8gOuNDZ sC8qyCcJuXCBpNjzDdvKqSlP7lvGKvd20SzLux44/JkydTyIBJZokAeixtgtb/sA ZDGviRR4xu5qLx3+2uSxpeYtWrjatGJyxeHPwqoPxXAMEeOrBHjqYtJ16K8+c= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; s= default; bh=LcvwQD75j0yLILnOO1z3DbTd4yI=; b=kPRuURpWwMhJ1SKTe6sq zulyYn9LGGeCuhzb01Kh550N8Gay2fGTJbpb4eUxrv16jCHABP+5Uu7vRikF7vj4 PhVXFhCBKXmLZWI+cL4eQuxy95Hy+Xec6Fl54pW3Pfse39bA3PAEZ65+PThBO+Ci qU9JHHnWnYMJjvSzAr7uv3Y= Received: (qmail 42569 invoked by alias); 20 Sep 2016 09:26:50 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 42456 invoked by uid 89); 20 Sep 2016 09:26:50 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy=Running, Segment, 3917, 3906 X-HELO: mail-qk0-f177.google.com Received: from mail-qk0-f177.google.com (HELO mail-qk0-f177.google.com) (209.85.220.177) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 20 Sep 2016 09:26:40 +0000 Received: by mail-qk0-f177.google.com with SMTP id n185so9368696qke.1 for ; Tue, 20 Sep 2016 02:26:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=7mGxJUZsN7RUQJDyb9rkYSbKoCYO7NNH+7lwls3nzTM=; b=LYuI53Sn/E2tdzsaRxm+u0c/bk3K/Di1aKyEdGYEXTzZYreOsaYMNZ6Kdr4YTZxBMe E5bUJCLMdeF6oRCJ+qlzI8Hi1cMUFLGS7Qgta5YoYlLMn32jtYBsyNNBXpTGJvYg51XP TBcrDw98jCU9UGKg+k7cg5BP1UEVWgy0TusSS+oDoo12rqOnPjPM4PiSDM//I7a1vKBf sQJOn+Il6Jsi4B0CSUzwacyx3pUz79L9YN5HQ59wQSaaXufblNJuI4r7uPv2j+pI/Bh/ QpMsyOUr1wYe9CesHceq9xju7wMC/tAjG5TSzyOq940KqfhvZNDDb58LrYBDh/6By02c l/+g== X-Gm-Message-State: AE9vXwOA7WeOrTaBHjSu5T+RTTp2mUnUQkBwpZgNN2IKuFXX9GYkz6MHF4dmj1+Lcy+BBa4MqtDzxuIaeIkqVlmm X-Received: by 10.55.187.194 with SMTP id l185mr32724481qkf.82.1474363598016; Tue, 20 Sep 2016 02:26:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.21.80 with HTTP; Tue, 20 Sep 2016 02:26:37 -0700 (PDT) From: Christophe Lyon Date: Tue, 20 Sep 2016 11:26:37 +0200 Message-ID: Subject: Report DejaGnu ERROR messages in dg-extract-results To: "gcc-patches@gcc.gnu.org" X-IsSubscribed: yes Hello, We recently faced a problem where a DejaGnu error went un-noticed (https://gcc.gnu.org/ml/gcc-patches/2016-08/msg01879.html). To help identify these problems earlier, here is a patch for dg-extract-results that makes such error obvious in the final summary: === gcc Summary === # of DejaGnu errors 1 # of expected passes 104885 # of unexpected failures 43 # of unexpected successes 2 # of expected failures 276 # of unsupported tests 2284 OK? Thanks, Christophe contrib/ChangeLog: 2016-09-20 Christophe Lyon * dg-extract-results.py: Report DejaGnu error in the final summary. * dg-extract-results.sh: Likewise. diff --git a/contrib/dg-extract-results.py b/contrib/dg-extract-results.py index 7db5e64..4b02a5b 100644 --- a/contrib/dg-extract-results.py +++ b/contrib/dg-extract-results.py @@ -134,6 +134,7 @@ class Prog: self.end_line = None # Known summary types. self.count_names = [ + '# of DejaGnu errors\t\t', '# of expected passes\t\t', '# of unexpected failures\t', '# of unexpected successes\t', @@ -245,6 +246,10 @@ class Prog: segment = Segment (filename, file.tell()) variation.header = segment + # Parse the rest of the summary (the '# of ' lines). + if len (variation.counts) == 0: + variation.counts = self.zero_counts() + # Parse up until the first line of the summary. if num_variations == 1: end = '\t\t=== ' + tool.name + ' Summary ===\n' @@ -291,6 +296,11 @@ class Prog: harness.results.append ((key, line)) if not first_key and sort_logs: first_key = key + if line.startswith ('ERROR: (DejaGnu)'): + for i in range (len (self.count_names)): + if 'DejaGnu errors' in self.count_names[i]: + variation.counts[i] += 1 + break # 'Using ...' lines are only interesting in a header. Splitting # the test up into parallel runs leads to more 'Using ...' lines @@ -309,9 +319,6 @@ class Prog: segment.lines -= final_using harness.add_segment (first_key, segment) - # Parse the rest of the summary (the '# of ' lines). - if len (variation.counts) == 0: - variation.counts = self.zero_counts() while True: before = file.tell() line = file.readline() diff --git a/contrib/dg-extract-results.sh b/contrib/dg-extract-results.sh index 5a8e67e..519d49c 100755 --- a/contrib/dg-extract-results.sh +++ b/contrib/dg-extract-results.sh @@ -369,10 +369,11 @@ EOF BEGIN { variant="$VAR" tool="$TOOL" - passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kpasscnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0; + passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kpasscnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0; dgerrorcnt=0; curvar=""; insummary=0 } /^Running target / { curvar = \$3; next } +/^ERROR: \(DejaGnu\)/ { if (variant == curvar) dgerrorcnt += 1 } /^# of / { if (variant == curvar) insummary = 1 } /^# of expected passes/ { if (insummary == 1) passcnt += \$5; next; } /^# of unexpected successes/ { if (insummary == 1) xpasscnt += \$5; next; } @@ -390,6 +391,7 @@ BEGIN { { next } END { printf ("\t\t=== %s Summary for %s ===\n\n", tool, variant) + if (dgerrorcnt != 0) printf ("# of DejaGnu errors\t\t%d\n", dgerrorcnt) if (passcnt != 0) printf ("# of expected passes\t\t%d\n", passcnt) if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt) if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt) @@ -419,8 +421,9 @@ TOTAL_AWK=${TMP}/total.awk cat << EOF > $TOTAL_AWK BEGIN { tool="$TOOL" - passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0 + passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0; dgerrorcnt=0 } +/^# of DejaGnu errors/ { dgerrorcnt += \$5 } /^# of expected passes/ { passcnt += \$5 } /^# of unexpected failures/ { failcnt += \$5 } /^# of unexpected successes/ { xpasscnt += \$5 } @@ -431,7 +434,8 @@ BEGIN { /^# of unresolved testcases/ { unrescnt += \$5 } /^# of unsupported tests/ { unsupcnt += \$5 } END { - printf ("\n\t\t=== %s Summary ===\n\n", tool) + printf ("\n\t\t=== %s MySummary ===\n\n", tool) + if (dgerrorcnt != 0) printf ("# of DejaGnu errors\t\t%d\n", dgerrorcnt) if (passcnt != 0) printf ("# of expected passes\t\t%d\n", passcnt) if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt) if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt)