diff mbox

[RFA] dg-extract-results.sh: Handle KFAILs.

Message ID 20120314172158.04E892461B2@ruffy.mtv.corp.google.com
State New
Headers show

Commit Message

Doug Evans March 14, 2012, 5:21 p.m. UTC 
The results of running the testsuite in parallel should match the
results when run serially.  This patch adds KFAIL counts so that happens.
[There's still a nit that the order of the results don't precisely match,
but that's a separate issue.]

I will check this into the gdb tree if there are no objections.
Any reason not to apply it to the gcc tree as well?

2012-03-14  Doug Evans  <dje@google.com>

	* dg-extract-results.sh: Handle KFAILs.

Comments

Mike Stump March 14, 2012, 8:15 p.m. UTC | #1 
On Mar 14, 2012, at 10:21 AM, Doug Evans wrote:
> The results of running the testsuite in parallel should match the
> results when run serially.  This patch adds KFAIL counts so that happens.
> [There's still a nit that the order of the results don't precisely match,
> but that's a separate issue.]
> 
> I will check this into the gdb tree if there are no objections.
> Any reason not to apply it to the gcc tree as well?

I don't know that the gcc tree has any of the known stuff, though, that's not on purpose or by design, just no one has done it, I think.
Pedro Alves March 15, 2012, 6:09 p.m. UTC | #2 
On 03/14/2012 08:15 PM, Mike Stump wrote:

> On Mar 14, 2012, at 10:21 AM, Doug Evans wrote:
>> The results of running the testsuite in parallel should match the
>> results when run serially.  This patch adds KFAIL counts so that happens.
>> [There's still a nit that the order of the results don't precisely match,
>> but that's a separate issue.]
>>
>> I will check this into the gdb tree if there are no objections.
>> Any reason not to apply it to the gcc tree as well?
> 
> I don't know that the gcc tree has any of the known stuff, though, that's not on purpose or by design, just no one has done it, I think.


Still, kfail is standard DejaGnu, not a GDB invention.  It'd be nice not to
need to fork the script for this.
Mike Stump March 15, 2012, 6:54 p.m. UTC | #3 
On Mar 15, 2012, at 11:09 AM, Pedro Alves wrote:
> Still, kfail is standard DejaGnu, not a GDB invention.  It'd be nice not to
> need to fork the script for this.

The change is fine for the gcc tree.
diff mbox

Patch

Index: dg-extract-results.sh
===================================================================
RCS file: /cvs/src/src/gdb/testsuite/dg-extract-results.sh,v
retrieving revision 1.4
diff -u -p -r1.4 dg-extract-results.sh
--- dg-extract-results.sh	4 Jan 2012 08:17:26 -0000	1.4
+++ dg-extract-results.sh	14 Mar 2012 17:15:07 -0000
@@ -345,7 +345,7 @@  EOF
 BEGIN {
   variant="$VAR"
   tool="$TOOL"
-  passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; unsupcnt=0; unrescnt=0;
+  passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0;
   curvar=""; insummary=0
 }
 /^Running target /		{ curvar = \$3; next }
@@ -354,6 +354,7 @@  BEGIN {
 /^# of unexpected successes/	{ if (insummary == 1) xpasscnt += \$5; next; }
 /^# of unexpected failures/	{ if (insummary == 1) failcnt += \$5; next; }
 /^# of expected failures/	{ if (insummary == 1) xfailcnt += \$5; next; }
+/^# of known failures/		{ if (insummary == 1) kfailcnt += \$5; next; }
 /^# of untested testcases/	{ if (insummary == 1) untstcnt += \$5; next; }
 /^# of unresolved testcases/	{ if (insummary == 1) unrescnt += \$5; next; }
 /^# of unsupported tests/	{ if (insummary == 1) unsupcnt += \$5; next; }
@@ -368,6 +369,7 @@  END {
   if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt)
   if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt)
   if (xfailcnt != 0) printf ("# of expected failures\t\t%d\n", xfailcnt)
+  if (kfailcnt != 0) printf ("# of known failures\t\t%d\n", kfailcnt)
   if (untstcnt != 0) printf ("# of untested testcases\t\t%d\n", untstcnt)
   if (unrescnt != 0) printf ("# of unresolved testcases\t%d\n", unrescnt)
   if (unsupcnt != 0) printf ("# of unsupported tests\t\t%d\n", unsupcnt)
@@ -391,12 +393,13 @@  TOTAL_AWK=${TMP}/total.awk
 cat << EOF > $TOTAL_AWK
 BEGIN {
   tool="$TOOL"
-  passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; unsupcnt=0; unrescnt=0
+  passcnt=0; failcnt=0; untstcnt=0; xpasscnt=0; xfailcnt=0; kfailcnt=0; unsupcnt=0; unrescnt=0
 }
 /^# of expected passes/		{ passcnt += \$5 }
 /^# of unexpected failures/	{ failcnt += \$5 }
 /^# of unexpected successes/	{ xpasscnt += \$5 }
 /^# of expected failures/	{ xfailcnt += \$5 }
+/^# of known failures/		{ kfailcnt += \$5 }
 /^# of untested testcases/	{ untstcnt += \$5 }
 /^# of unresolved testcases/	{ unrescnt += \$5 }
 /^# of unsupported tests/	{ unsupcnt += \$5 }
@@ -406,6 +409,7 @@  END {
   if (failcnt != 0) printf ("# of unexpected failures\t%d\n", failcnt)
   if (xpasscnt != 0) printf ("# of unexpected successes\t%d\n", xpasscnt)
   if (xfailcnt != 0) printf ("# of expected failures\t\t%d\n", xfailcnt)
+  if (kfailcnt != 0) printf ("# of known failures\t\t%d\n", kfailcnt)
   if (untstcnt != 0) printf ("# of untested testcases\t\t%d\n", untstcnt)
   if (unrescnt != 0) printf ("# of unresolved testcases\t%d\n", unrescnt)
   if (unsupcnt != 0) printf ("# of unsupported tests\t\t%d\n", unsupcnt)