{"id":2224743,"url":"http://patchwork.ozlabs.org/api/1.1/patches/2224743/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/patch/orik9owzwj.fsf@lxoliva.fsfla.org/","project":{"id":17,"url":"http://patchwork.ozlabs.org/api/1.1/projects/17/?format=json","name":"GNU Compiler Collection","link_name":"gcc","list_id":"gcc-patches.gcc.gnu.org","list_email":"gcc-patches@gcc.gnu.org","web_url":null,"scm_url":null,"webscm_url":null},"msgid":"","date":"2026-04-18T06:22:52","name":"testsuite: flag failure to detect failures","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"b5221e51c803669a5b653cae27b032f148f9b4db","submitter":{"id":74937,"url":"http://patchwork.ozlabs.org/api/1.1/people/74937/?format=json","name":"Alexandre Oliva","email":"oliva@adacore.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/gcc/patch/orik9owzwj.fsf@lxoliva.fsfla.org/mbox/","series":[{"id":500419,"url":"http://patchwork.ozlabs.org/api/1.1/series/500419/?format=json","web_url":"http://patchwork.ozlabs.org/project/gcc/list/?series=500419","date":"2026-04-18T06:22:52","name":"testsuite: flag failure to detect failures","version":1,"mbox":"http://patchwork.ozlabs.org/series/500419/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/2224743/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/2224743/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":["incoming@patchwork.ozlabs.org","gcc-patches@gcc.gnu.org"],"Delivered-To":["patchwork-incoming@legolas.ozlabs.org","gcc-patches@gcc.gnu.org"],"Authentication-Results":["legolas.ozlabs.org;\n\tdkim=pass (2048-bit key;\n secure) header.d=adacore.com header.i=@adacore.com header.a=rsa-sha256\n header.s=google header.b=QQsKS9LO;\n\tdkim-atps=neutral","legolas.ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org\n (client-ip=2620:52:6:3111::32; helo=vm01.sourceware.org;\n envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org;\n receiver=patchwork.ozlabs.org)","sourceware.org;\n\tdkim=pass (2048-bit key,\n secure) header.d=adacore.com header.i=@adacore.com header.a=rsa-sha256\n header.s=google header.b=QQsKS9LO","sourceware.org; dmarc=pass (p=quarantine dis=none)\n header.from=adacore.com","sourceware.org; spf=pass smtp.mailfrom=adacore.com","server2.sourceware.org;\n arc=none smtp.remote-ip=74.125.82.169"],"Received":["from vm01.sourceware.org (vm01.sourceware.org\n [IPv6:2620:52:6:3111::32])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange x25519 server-signature ECDSA (secp384r1) server-digest SHA384)\n\t(No client certificate requested)\n\tby legolas.ozlabs.org (Postfix) with ESMTPS id 4fyMCD4PJqz1yDF\n\tfor ; Sat, 18 Apr 2026 16:23:48 +1000 (AEST)","from vm01.sourceware.org (localhost [127.0.0.1])\n\tby sourceware.org (Postfix) with ESMTP id BDBA64D108C8\n\tfor ; Sat, 18 Apr 2026 06:23:40 +0000 (GMT)","from mail-dy1-f169.google.com (mail-dy1-f169.google.com\n [74.125.82.169])\n by sourceware.org (Postfix) with ESMTPS id EBBF14AA51F8\n for ; Sat, 18 Apr 2026 06:23:12 +0000 (GMT)","by mail-dy1-f169.google.com with SMTP id\n 5a478bee46e88-2d891442388so1016433eec.0\n for ; Fri, 17 Apr 2026 23:23:12 -0700 (PDT)","from free.home ([2804:14c:4d1:41a6::2000])\n by smtp.gmail.com with ESMTPSA id\n 5a478bee46e88-2e53d8b3dd9sm5381959eec.27.2026.04.17.23.23.10\n (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n Fri, 17 Apr 2026 23:23:10 -0700 (PDT)","from livre (livre.home [172.31.160.2])\n by free.home (8.15.2/8.15.2) with ESMTPS id 63I6MqKw540511\n (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT);\n Sat, 18 Apr 2026 03:22:53 -0300"],"DKIM-Filter":["OpenDKIM Filter v2.11.0 sourceware.org BDBA64D108C8","OpenDKIM Filter v2.11.0 sourceware.org EBBF14AA51F8"],"DMARC-Filter":"OpenDMARC Filter v1.4.2 sourceware.org EBBF14AA51F8","ARC-Filter":"OpenARC Filter v1.0.0 sourceware.org EBBF14AA51F8","ARC-Seal":"i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1776493393; cv=none;\n b=XdbpWrkB3aYxy6MXE0d5AYVd4/Q4//1jTWfuz0SMuIcDtRvEFUXgBtwZIx7qwrUbiL2az1cVt3WrjcHL/7mmCZ1FzsBXkctSAsPxO5GkA/V+5JYN0mDZoSrdvsv26ygY0ypWb23DpEKSetANYIn4hhtYyzSGzbA99LB6eGa5ExE=","ARC-Message-Signature":"i=1; a=rsa-sha256; d=sourceware.org; s=key;\n t=1776493393; c=relaxed/simple;\n bh=0tq28GiCu1pRsEErFt1E7p8e1BIHxEwFu5gJo7SR+lE=;\n h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version;\n b=rzO5ZfKmpPITsIIR/uAOB3Xpa8ftfCpcnTY7eYxedUaB/hAjMIG8yPg5YD/n6518jhD33s7ZbrnvsaeDfAodZi5OdtXxSaTExWvxMhN1OyJ/m8CU46f19MQYZ1VvXESYYVUH0LGRHELQL8REcNWemRsrmvbZNvEsCX80ZN9JFfs=","ARC-Authentication-Results":"i=1; server2.sourceware.org","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=adacore.com; s=google; t=1776493392; x=1777098192; darn=gcc.gnu.org;\n h=mime-version:user-agent:message-id:date:organization:subject:cc:to\n :from:from:to:cc:subject:date:message-id:reply-to;\n bh=RxVSsHEc3xx44i69bs/aDLxCtWlIjnIT+RJWJDf55zk=;\n b=QQsKS9LOXIfC4sbw1e9d3bHC2lKfv+ffHdI9WdNSQR3h+kQb46WODWszmF3ioiJj0v\n 9gncmSez/20BHmdIxbM32W08ROHETlmkrydoNgVYkwchfJBQTKodY8B3OSEBfILv6kut\n mAQSjmM5wKtySltMu0sTXzkKVyL910boMVqpqzY4gBAaKZW0/4K3FIfj8pZoT0dnzd7S\n Nho4IjtBdDOGlEmTDKZglcmbfyKwIK1FWLfUg1LWRkxLtFkPym20/ltD/dtY0Y3QF6dd\n 2OFWOl7PX6qqit+xJ2rxZ5y0m1uxYvZIQYm0RT/WYfLwvKac0x/KgTkx306RNx202wG/\n CrRQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n d=1e100.net; s=20251104; t=1776493392; x=1777098192;\n h=mime-version:user-agent:message-id:date:organization:subject:cc:to\n :from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id\n :reply-to;\n bh=RxVSsHEc3xx44i69bs/aDLxCtWlIjnIT+RJWJDf55zk=;\n b=qZ/Op/DuQb6TqLESogPcff/qB/Ft0SsrOfdPtBEN3aL7ixSlC7FHLa8S9Hi+8FUBS2\n +hFy1s47yurcEMzaefwo47/lA/4Gf1pIQQn48rCF/VjTP016wbOThzyC58oy+P651RpR\n fhwF+c00od+nc2v5ZK4W8HtLCGUPQtU+Z0BDHhhX3ASQ7cTSU3gaTfq5e9mlYzqiR8Sx\n 5/E9VFySExO/eUsic2kBYCSCcJVi1Pg0fvu1BS5CinFINS0q4xmvY+LCNXPx2UipPyeE\n wCVpaR5ZiR6Jyu5NS6eNDG6Nyzwo4fbyUGBF8xchfJo1pdhIviaJ87jFd/oN78yQ4Wab\n 2ylA==","X-Gm-Message-State":"AOJu0Yxp7/a112m3372lZ1lHel5GNytrerIqCAqiQkgQeb8TPTSoo1H5\n GApGJE7/7zpe+r9VTjdgiHjJe6DqjYL3YfyAxNoa+XTuTfaaCGIP3+D/NI7Yu6wKmQfkOf3576b\n 2/mmWdw==","X-Gm-Gg":"AeBDievtMKCHUfgASW5cBI3sa7dgcOzjIVYmdELuvXvoRQE79Yr1PDXydLZ0MUGHNQp\n TSLthRIyZmE1ZHz1IyNLxid6b14XnufKjr0EkS1LmcxmLdUOmTFiacVhTCXlzb/lnOFoQP/lHFU\n +dIoWWq1HFicJE8LqGsH7HxIfaTOX9rrMbWLc4vUigeXQvprxNKT0bZoF1AnD6RfDvQPsTnKMVo\n Xb0D5Hhj57myZFKdjYYNnm6wHn9haPUz82P+Hb/NxWB5Utjc0dKrMvjSJBYHi34AyXSuUgNJ2fx\n hWFcYHYFkyvMoz/7zHwXFK9yVZqUc4JgUkHSIjc7tp3V9+kon660FPeZ4jXC4fSppNyIkAGZ2Kf\n mccxLtm9SrqpsLzMggAK9bWuqetI2MGYW3lx0iXcFVE3avmg6dOE15qEwpbeI0IRjyf43ae9Xko\n 3X7aYkJDzd8g==","X-Received":"by 2002:a05:693c:2b04:b0:2be:6f30:f2f9 with SMTP id\n 5a478bee46e88-2e479215120mr2987183eec.26.1776493391444;\n Fri, 17 Apr 2026 23:23:11 -0700 (PDT)","From":"Alexandre Oliva ","To":"gcc-patches@gcc.gnu.org","Cc":"Rainer Orth , Mike Stump\n ","Subject":"[PATCH] testsuite: flag failure to detect failures","Organization":"Free thinker, does not speak for AdaCore","Date":"Sat, 18 Apr 2026 03:22:52 -0300","Message-ID":"","User-Agent":"Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)","MIME-Version":"1.0","Content-Type":"text/plain","X-Scanned-By":"MIMEDefang 2.84","X-BeenThere":"gcc-patches@gcc.gnu.org","X-Mailman-Version":"2.1.30","Precedence":"list","List-Id":"Gcc-patches mailing list ","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Errors-To":"gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org"},"content":"There are various ways for tests to report an execution failure.\n\nMake sure the testing infrastructure can detect all of them, so that\nfailures don't go silent.\n\nRegstrapping on x86_64-linux-gnu... Already tested on a target where we\nfailed to detect a negative exit status as a failure, only noticed\nbecause g++.dg/contracts/contracts-ignore2.C's execution failure was not\ndetected as such. We'd better have stronger defenses against failure to\ndetect failures. Ok to install?\n\n\nfor gcc/testsuite/ChangeLog\n\n\t* gcc.dg/shouldfail-abort.c: New.\n\t* gcc.dg/shouldfail-exit-neg.c: New.\n\t* gcc.dg/shouldfail-exit-pos.c: New.\n\t* gcc.dg/shouldfail-return-neg.c: New.\n\t* gcc.dg/shouldfail-return-pos.c: New.\n---\n gcc/testsuite/gcc.dg/shouldfail-abort.c | 10 ++++++++++\n gcc/testsuite/gcc.dg/shouldfail-exit-neg.c | 10 ++++++++++\n gcc/testsuite/gcc.dg/shouldfail-exit-pos.c | 10 ++++++++++\n gcc/testsuite/gcc.dg/shouldfail-return-neg.c | 10 ++++++++++\n gcc/testsuite/gcc.dg/shouldfail-return-pos.c | 10 ++++++++++\n 5 files changed, 50 insertions(+)\n create mode 100644 gcc/testsuite/gcc.dg/shouldfail-abort.c\n create mode 100644 gcc/testsuite/gcc.dg/shouldfail-exit-neg.c\n create mode 100644 gcc/testsuite/gcc.dg/shouldfail-exit-pos.c\n create mode 100644 gcc/testsuite/gcc.dg/shouldfail-return-neg.c\n create mode 100644 gcc/testsuite/gcc.dg/shouldfail-return-pos.c","diff":"diff --git a/gcc/testsuite/gcc.dg/shouldfail-abort.c b/gcc/testsuite/gcc.dg/shouldfail-abort.c\nnew file mode 100644\nindex 0000000000000..a00c483a56ff0\n--- /dev/null\n+++ b/gcc/testsuite/gcc.dg/shouldfail-abort.c\n@@ -0,0 +1,10 @@\n+/* { dg-do run } */\n+/* { dg-shouldfail \"\" } */\n+\n+/* Raise red flags if the way programs are meant to flag failures aren't\n+ detected as such. This is unlikely to indicate a compiler problem, but\n+ maybe a runtime or test infrastructure problem. */\n+\n+int main() {\n+ __builtin_abort();\n+}\ndiff --git a/gcc/testsuite/gcc.dg/shouldfail-exit-neg.c b/gcc/testsuite/gcc.dg/shouldfail-exit-neg.c\nnew file mode 100644\nindex 0000000000000..57d66814120a6\n--- /dev/null\n+++ b/gcc/testsuite/gcc.dg/shouldfail-exit-neg.c\n@@ -0,0 +1,10 @@\n+/* { dg-do run } */\n+/* { dg-shouldfail \"\" } */\n+\n+/* Raise red flags if the way programs are meant to flag failures aren't\n+ detected as such. This is unlikely to indicate a compiler problem, but\n+ maybe a runtime or test infrastructure problem. */\n+\n+int main() {\n+ __builtin_exit(-1);\n+}\ndiff --git a/gcc/testsuite/gcc.dg/shouldfail-exit-pos.c b/gcc/testsuite/gcc.dg/shouldfail-exit-pos.c\nnew file mode 100644\nindex 0000000000000..2ae16fa2879ff\n--- /dev/null\n+++ b/gcc/testsuite/gcc.dg/shouldfail-exit-pos.c\n@@ -0,0 +1,10 @@\n+/* { dg-do run } */\n+/* { dg-shouldfail \"\" } */\n+\n+/* Raise red flags if the way programs are meant to flag failures aren't\n+ detected as such. This is unlikely to indicate a compiler problem, but\n+ maybe a runtime or test infrastructure problem. */\n+\n+int main() {\n+ __builtin_exit(1);\n+}\ndiff --git a/gcc/testsuite/gcc.dg/shouldfail-return-neg.c b/gcc/testsuite/gcc.dg/shouldfail-return-neg.c\nnew file mode 100644\nindex 0000000000000..6d32d260ad5c4\n--- /dev/null\n+++ b/gcc/testsuite/gcc.dg/shouldfail-return-neg.c\n@@ -0,0 +1,10 @@\n+/* { dg-do run } */\n+/* { dg-shouldfail \"\" } */\n+\n+/* Raise red flags if the way programs are meant to flag failures aren't\n+ detected as such. This is unlikely to indicate a compiler problem, but\n+ maybe a runtime or test infrastructure problem. */\n+\n+int main() {\n+ return -1;\n+}\ndiff --git a/gcc/testsuite/gcc.dg/shouldfail-return-pos.c b/gcc/testsuite/gcc.dg/shouldfail-return-pos.c\nnew file mode 100644\nindex 0000000000000..ceb4f4cf95a75\n--- /dev/null\n+++ b/gcc/testsuite/gcc.dg/shouldfail-return-pos.c\n@@ -0,0 +1,10 @@\n+/* { dg-do run } */\n+/* { dg-shouldfail \"\" } */\n+\n+/* Raise red flags if the way programs are meant to flag failures aren't\n+ detected as such. This is unlikely to indicate a compiler problem, but\n+ maybe a runtime or test infrastructure problem. */\n+\n+int main() {\n+ return 1;\n+}\n","prefixes":[]}