From patchwork Tue May 11 18:55:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Myers X-Patchwork-Id: 1477281 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FfnF82sSgz9srX for ; Wed, 12 May 2021 04:55:55 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A5854388E818; Tue, 11 May 2021 18:55:52 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from esa2.mentor.iphmx.com (esa2.mentor.iphmx.com [68.232.141.98]) by sourceware.org (Postfix) with ESMTPS id CEA90385DC3D for ; Tue, 11 May 2021 18:55:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org CEA90385DC3D Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=joseph_myers@mentor.com IronPort-SDR: wOV+Kniq3J13wPNOnI/202/IwWK4KdgHLpkdRutXSbHncftqir+g+l1Qmw71Iwm4llJhfk07SU GX3JuAdAWsIKfnfvRtXXY62mRbBDmos6bC+Ukm8aWflWmi8uaYdtIpr+jdPfEfiLleuVuOrEpj MBnSgrc/934/+CrmX52wbWWuK+qRlR6hS+DvSPHHPZ3fYI2tp1GRf9YwNycR1iQqKN2ZtE41LR +JR0/1AQWXe6AzyjF7ByhxmjgLu+c54k8lyjSrBX59KWHEr0B5BOQ47gKxRUVvDguWiJUE9A15 y6g= X-IronPort-AV: E=Sophos;i="5.82,291,1613462400"; d="scan'208";a="61079180" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa2.mentor.iphmx.com with ESMTP; 11 May 2021 10:55:47 -0800 IronPort-SDR: /CADWJcPk3PyeJaet+B7E0m2cg3cnex3KIOmQY0jKvBjeCu/05pBHaj3EoFtmcelPUwlXXx9j8 Jkbl7YJHuF56NNHVDJSNaRvmNDYfsYW2RPjOY5y7cS2CaNyKdoSGcBKEWlVwIkDZPQbS95ZE9Q gOoN3FL7FwywLK60jJvljIcgBpChyoVTiqC1gd4OH2D+nmLVnvAm1NGggATsqI8v/4j7nDx6cz WSA3azO2CfFE42ZT9ip1sLuF6Gu33Z1C+nMtrlg0LREnK3qXrse2pr/mOpDexbfZA2ufDhWzij SwE= Date: Tue, 11 May 2021 18:55:42 +0000 From: Joseph Myers X-X-Sender: jsm28@digraph.polyomino.org.uk To: Subject: [committed] preprocessor: Fix cpp_avoid_paste for digit separators Message-ID: User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: SVR-IES-MBX-08.mgc.mentorg.com (139.181.222.8) To svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) X-Spam-Status: No, score=-3127.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" The libcpp function cpp_avoid_paste is used to insert whitespace in preprocessed output where needed to avoid two consecutive preprocessing tokens, that logically (e.g. when stringized) do not have whitespace between them, from being incorrectly lexed as one when the preprocessed input is reread by a compiler. This fails to allow for digit separators, so meaning that invalid code, that has a CPP_NUMBER (from a macro expansion) followed by a character literal, can result in preprocessed output with a valid use of digit separators, so that required syntax errors do not occur when compiling with -save-temps. Fix this by handling that case in cpp_avoid_paste (as with other cases in cpp_avoid_paste, this doesn't try to check whether the language version in use supports digit separators; it's always OK to have unnecessary whitespace in preprocessed output). Note: there are other cases, with various kinds of wide character or string literal following a CPP_NUMBER, where spurious pasting of preprocessing tokens can occur but the sequence of tokens remains invalid both before and after that pasting. Maybe cpp_avoid_paste should also handle those cases (and similar cases after a CPP_NAME), to ensure the sequence of preprocessing tokens in preprocessed output is exactly right, whether or not it affects whether syntax errors occur. This patch only addresses the case with digit separators where invalid code can fail to be diagnosed without the space inserted. Bootstrapped with no regressions for x86_64-pc-linux-gnu. Applied to mainline. libcpp/ * lex.c (cpp_avoid_paste): Do not allow pasting CPP_NUMBER with CPP_CHAR. gcc/testsuite/ * g++.dg/cpp1y/digit-sep-paste.C, gcc.dg/c2x-digit-separators-3.c: New tests. diff --git a/gcc/testsuite/g++.dg/cpp1y/digit-sep-paste.C b/gcc/testsuite/g++.dg/cpp1y/digit-sep-paste.C new file mode 100644 index 00000000000..41fb967ef8d --- /dev/null +++ b/gcc/testsuite/g++.dg/cpp1y/digit-sep-paste.C @@ -0,0 +1,11 @@ +// Test token pasting with digit separators avoided for preprocessed output. +// { dg-do compile { target c++14 } } +// { dg-options "-save-temps" } + +#define ZERO 0 + +int +f () +{ + return ZERO'0'0; /* { dg-error "expected" } */ +} diff --git a/gcc/testsuite/gcc.dg/c2x-digit-separators-3.c b/gcc/testsuite/gcc.dg/c2x-digit-separators-3.c new file mode 100644 index 00000000000..cddb88fa880 --- /dev/null +++ b/gcc/testsuite/gcc.dg/c2x-digit-separators-3.c @@ -0,0 +1,12 @@ +/* Test C2x digit separators. Test token pasting avoided for preprocessed + output. */ +/* { dg-do compile } */ +/* { dg-options "-std=c2x -save-temps" } */ + +#define ZERO 0 + +int +f (void) +{ + return ZERO'0'0; /* { dg-error "expected" } */ +} diff --git a/libcpp/lex.c b/libcpp/lex.c index b7ce85a0331..36cd2e30630 100644 --- a/libcpp/lex.c +++ b/libcpp/lex.c @@ -3725,6 +3725,7 @@ cpp_avoid_paste (cpp_reader *pfile, const cpp_token *token1, || b == CPP_NAME || b == CPP_CHAR || b == CPP_STRING); /* L */ case CPP_NUMBER: return (b == CPP_NUMBER || b == CPP_NAME + || b == CPP_CHAR || c == '.' || c == '+' || c == '-'); /* UCNs */ case CPP_OTHER: return ((token1->val.str.text[0] == '\\'