From patchwork Tue Nov 3 21:14:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393418 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=X3up9hP7; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjG51fcCz9sXb for ; Wed, 4 Nov 2020 08:14:21 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 221AC398745C; Tue, 3 Nov 2020 21:14:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) by sourceware.org (Postfix) with ESMTPS id AD77A398745C for ; Tue, 3 Nov 2020 21:14:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org AD77A398745C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72c.google.com with SMTP id 140so16627322qko.2 for ; Tue, 03 Nov 2020 13:14:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=coUIVx0ZOE5hwK7wB67l4ZbcILGH+IrRcU/HOYLOzSI=; b=X3up9hP7RfUnipgircIQ/YT0Ql/2AXSrjp2GTJA+fmPDpv/hp2Co+WgRgzsyhiJita wUjziH6FroMrLnUorzgR4r+vsO0Yj/kUzu5HG9wH75LWFhut9/hkFOhzBTE+aGvXm2BE Og3jwIBDQWLWrBXZnh7wGgdc0q/RCOhcZ+o+77DS4PQ1BmH8zJzO3NGqe9bzkv93thUg V+5anyaFsEOl2JQBRfDxMmPJcrHb1xkvO+5ZErK/G8ptb0kqLzILgnBJljFVX5ZKgt/4 QciO18TbY7RFI9ZEqqbVqdao9DgXB4P7ka9FDDs+bcEnDSw9RmM4RI3PKpgmC7tbOWPa 1FaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=coUIVx0ZOE5hwK7wB67l4ZbcILGH+IrRcU/HOYLOzSI=; b=i8S6pGTnBOqKPg/rzKjcC3mlWvcsjskykQamKpYEKNt7zfueGrqL9QGY0JRZQljAic nrU0Fw4lC+zqdVPB3omRpynMC8ch8DCbsTXP82RkSDiDuyaP4dTtSsdPjPB1s7u9ZFBC 1fdnFnOS27PggnCtSHe/5T6542z6IsU6M2TciTvM9M0jH0+aR0qDMnvA+5GrmZY9msPV Xy9+J1fKTE51S9FvGKMKBzLRCGUvF+wPoOxaquxLx9enhJcWsAybYXeCn7Td+CQyW4jg ci6fFUaUfxAeQiwplxYc8BuGdsdQBMpUCtv/QwedxgHrytqJHKrrS5VkKlmLnm1aINMz imWw== X-Gm-Message-State: AOAM533a4M3+RFynJV2iRXTp5rOHtcwxrTAm91xEbnoNtMatzDTlXaOA CcutNl23f6Xg3wqrxmKuY4xXsg5fwZe9ew== X-Google-Smtp-Source: ABdhPJxqTJBYWiu+gzzReC/nerG4gpO2M5OFKN76ODNbERsuwdqezMg5bqC1rtKfEAW52MgV8VG7kg== X-Received: by 2002:a37:7d43:: with SMTP id y64mr20199779qkc.225.1604438054778; Tue, 03 Nov 2020 13:14:14 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id k4sm11996034qkj.46.2020.11.03.13.14.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:13 -0800 (PST) From: Nathan Sidwell Subject: [06/32] cpp macros To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:14:11 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Header units can provide macros to an importer. For efficiency that is done lazily. When we import a header unit we mark the identifers it defines as significant. It's only when we need the macro definition (including in #ifdef &| defined (X) processing) that we resolve the macro. We already had lazy (builtin) macros. This extends it to deferred macros. Each deferred macro has an index the resolver can use. on an LP64 host that index does not inlarge the node structure. As a macro can be undefined by this mechanism, the users of this deferred interface must be prepared to handle 'not a macro after all' diff --git c/libcpp/directives.c w/libcpp/directives.c index d7b59aae901..899ebbf023a 100644 --- c/libcpp/directives.c +++ w/libcpp/directives.c @@ -667,7 +667,8 @@ do_undef (cpp_reader *pfile) pfile->directive_line, 0, "undefining \"%s\"", NODE_NAME (node)); - if (CPP_OPTION (pfile, warn_unused_macros)) + if (node->value.macro + && CPP_OPTION (pfile, warn_unused_macros)) _cpp_warn_if_unused_macro (pfile, node, NULL); _cpp_free_definition (node); @@ -1981,8 +1983,10 @@ do_ifdef (cpp_reader *pfile) if (node) { skip = !_cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line)) + /* It wasn't a macro after all. */ + skip = true; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line); if (pfile->cb.used) pfile->cb.used (pfile, pfile->directive_line, node); check_eol (pfile, false); @@ -2006,8 +2010,10 @@ do_ifndef (cpp_reader *pfile) if (node) { skip = _cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line)) + /* It wasn't a macro after all. */ + skip = false; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line); if (pfile->cb.used) pfile->cb.used (pfile, pfile->directive_line, node); check_eol (pfile, false); diff --git c/libcpp/expr.c w/libcpp/expr.c index e01a47a8c34..894e8515c90 100644 --- c/libcpp/expr.c +++ w/libcpp/expr.c @@ -1061,6 +1061,7 @@ parse_defined (cpp_reader *pfile) } } + bool is_defined = false; if (node) { if ((pfile->context != initial_context @@ -1068,9 +1069,11 @@ parse_defined (cpp_reader *pfile) && CPP_OPTION (pfile, warn_expansion_to_defined)) cpp_pedwarning (pfile, CPP_W_EXPANSION_TO_DEFINED, "this use of \"defined\" may not be portable"); - + is_defined = _cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, token->src_loc)) + /* It wasn't a macro after all. */ + is_defined = false; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, token->src_loc); /* A possible controlling macro of the form #if !defined (). _cpp_parse_expr checks there was no other junk on the line. */ @@ -1086,7 +1089,7 @@ parse_defined (cpp_reader *pfile) result.unsignedp = false; result.high = 0; result.overflow = false; - result.low = node && _cpp_defined_macro_p (node); + result.low = is_defined; return result; } diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -308,6 +308,13 @@ enum cpp_normalize_level { @@ -801,7 +824,10 @@ struct GTY(()) cpp_macro { tokens. */ unsigned int extra_tokens : 1; - /* 1 bits spare (32-bit). 33 on 64-bit target. */ + /* Imported (from a legacy header module). */ + unsigned int imported : 1; + + /* 0 bits spare (32-bit). 32 on 64-bit target. */ union cpp_exp_u { @@ -874,7 +901,7 @@ enum cpp_builtin_type union GTY(()) _cpp_hashnode_value { /* Assert (maybe NULL) */ cpp_macro * GTY((tag ("NT_VOID"))) answers; - /* Macro (never NULL) */ + /* Macro (maybe NULL) */ cpp_macro * GTY((tag ("NT_USER_MACRO"))) macro; /* Code for a builtin macro. */ enum cpp_builtin_type GTY ((tag ("NT_BUILTIN_MACRO"))) builtin; @@ -888,5 +915,9 @@ struct GTY(()) cpp_hashnode { /* 6 bits spare (plus another 32 on 64-bit hosts). */ + + /* On a 64-bit system there would be 32-bits of padding to the value + field. So placing the deferred index here is not costly. */ + unsigned deferred; /* Deferred index, (unless zero). */ union _cpp_hashnode_value GTY ((desc ("%1.type"))) value; }; @@ -1027,6 +1066,18 @@ inline bool cpp_macro_p (const cpp_hashnode *node) { return node->type & NT_MACRO_MASK; } +inline cpp_macro *cpp_set_deferred_macro (cpp_hashnode *node, + cpp_macro *forced = NULL) +{ + cpp_macro *old = node->value.macro; + + node->value.macro = forced; + node->type = NT_USER_MACRO; + node->flags &= ~NODE_USED; + + return old; +} +cpp_macro *cpp_get_deferred_macro (cpp_reader *, cpp_hashnode *, location_t); /* Returns true if NODE is a function-like user macro. */ inline bool cpp_fun_like_macro_p (cpp_hashnode *node) @@ -1034,11 +1085,13 @@ inline bool cpp_fun_like_macro_p (cpp_hashnode *node) return cpp_user_macro_p (node) && node->value.macro->fun_like; } -extern const unsigned char *cpp_macro_definition (cpp_reader *, - cpp_hashnode *); +extern const unsigned char *cpp_macro_definition (cpp_reader *, cpp_hashnode *); +extern const unsigned char *cpp_macro_definition (cpp_reader *, cpp_hashnode *, + const cpp_macro *); inline location_t cpp_macro_definition_location (cpp_hashnode *node) { - return node->value.macro->line; + const cpp_macro *macro = node->value.macro; + return macro ? macro->line : 0; } extern void _cpp_backup_tokens (cpp_reader *, unsigned int); extern const cpp_token *cpp_peek_token (cpp_reader *, int); @@ -1219,6 +1272,8 @@ extern int cpp_ideq (const cpp_token *, const char *); extern void cpp_output_line (cpp_reader *, FILE *); extern unsigned char *cpp_output_line_to_string (cpp_reader *, const unsigned char *); +extern const unsigned char *cpp_alloc_token_string + (cpp_reader *, const unsigned char *, unsigned); extern void cpp_output_token (const cpp_token *, FILE *); extern const char *cpp_type2name (enum cpp_ttype, unsigned char flags); /* Returns the value of an escape sequence, truncated to the correct @@ -1274,6 +1329,8 @@ extern void cpp_scan_nooutput (cpp_reader *); extern int cpp_sys_macro_p (cpp_reader *); extern unsigned char *cpp_quote_string (unsigned char *, const unsigned char *, unsigned int); +extern bool cpp_compare_macros (const cpp_macro *macro1, + const cpp_macro *macro2); /* In files.c */ extern bool cpp_included (cpp_reader *, const char *); diff --git c/libcpp/internal.h w/libcpp/internal.h index 4759961a33a..a92abf281c0 100644 --- c/libcpp/internal.h +++ w/libcpp/internal.h @@ -649,13 +665,14 @@ inline bool _cpp_defined_macro_p (cpp_hashnode *node) } /* In macro.c */ -extern void _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, - location_t loc); -inline void _cpp_maybe_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, +extern bool _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, + location_t); +inline bool _cpp_maybe_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, location_t loc) { if (!(node->flags & NODE_USED)) - _cpp_notify_macro_use (pfile, node, loc); + return _cpp_notify_macro_use (pfile, node, loc); + return true; } extern cpp_macro *_cpp_new_macro (cpp_reader *, cpp_macro_kind, void *); extern void _cpp_free_definition (cpp_hashnode *); @@ -869,29 +886,7 @@ ufputs (const unsigned char *s, FILE *f) return fputs ((const char *)s, f); } - /* In line-map.c. */ - -/* Create a macro map. A macro map encodes source locations of tokens - that are part of a macro replacement-list, at a macro expansion - point. See the extensive comments of struct line_map and struct - line_map_macro, in line-map.h. - - This map shall be created when the macro is expanded. The map - encodes the source location of the expansion point of the macro as - well as the "original" source location of each token that is part - of the macro replacement-list. If a macro is defined but never - expanded, it has no macro map. SET is the set of maps the macro - map should be part of. MACRO_NODE is the macro which the new macro - map should encode source locations for. EXPANSION is the location - of the expansion point of MACRO. For function-like macros - invocations, it's best to make it point to the closing parenthesis - of the macro, rather than the the location of the first character - of the macro. NUM_TOKENS is the number of tokens that are part of - the replacement-list of MACRO. */ -const line_map_macro *linemap_enter_macro (class line_maps *, - struct cpp_hashnode*, - location_t, - unsigned int); +/* In line-map.c. */ /* Create and return a virtual location for a token that is part of a macro expansion-list at a macro expansion point. See the comment diff --git c/libcpp/lex.c w/libcpp/lex.c index fb222924c8c..b3498f195bf 100644 --- c/libcpp/lex.c +++ w/libcpp/lex.c @@ -1577,13 +1577,20 @@ static void create_literal (cpp_reader *pfile, cpp_token *token, const uchar *base, unsigned int len, enum cpp_ttype type) { - uchar *dest = _cpp_unaligned_alloc (pfile, len + 1); - - memcpy (dest, base, len); - dest[len] = '\0'; token->type = type; token->val.str.len = len; - token->val.str.text = dest; + token->val.str.text = cpp_alloc_token_string (pfile, base, len); +} + +const uchar * +cpp_alloc_token_string (cpp_reader *pfile, + const unsigned char *ptr, unsigned len) +{ + uchar *dest = _cpp_unaligned_alloc (pfile, len + 1); + + dest[len] = 0; + memcpy (dest, ptr, len); + return dest; } /* A pair of raw buffer pointers. The currently open one is [1], the diff --git c/libcpp/macro.c w/libcpp/macro.c index e304f67c2e0..f5f280dfdc7 100644 --- c/libcpp/macro.c +++ w/libcpp/macro.c @@ -268,6 +268,8 @@ class vaopt_state { /* Macro expansion. */ +static cpp_macro *get_deferred_or_lazy_macro (cpp_reader *, cpp_hashnode *, + location_t); static int enter_macro_context (cpp_reader *, cpp_hashnode *, const cpp_token *, location_t); static int builtin_macro (cpp_reader *, cpp_hashnode *, @@ -338,10 +340,6 @@ static cpp_macro *create_iso_definition (cpp_reader *); /* #define directive parsing and handling. */ static cpp_macro *lex_expansion_token (cpp_reader *, cpp_macro *); -static bool warn_of_redefinition (cpp_reader *, cpp_hashnode *, - const cpp_macro *); -static bool compare_macros (const cpp_macro *, const cpp_macro *); - static bool parse_params (cpp_reader *, unsigned *, bool *); static void check_trad_stringification (cpp_reader *, const cpp_macro *, const cpp_string *); @@ -353,8 +351,6 @@ static const cpp_token* cpp_get_token_1 (cpp_reader *, location_t *); static cpp_hashnode* macro_of_context (cpp_context *context); -static bool in_macro_expansion_p (cpp_reader *pfile); - /* Statistical counter tracking the number of macros that got expanded. */ unsigned num_expanded_macros_counter = 0; @@ -2845,6 +2841,12 @@ cpp_get_token_1 (cpp_reader *pfile, location_t *location) if (node->type == NT_VOID || (result->flags & NO_EXPAND)) break; + if (!(node->flags & NODE_USED) + && node->type == NT_USER_MACRO + && !node->value.macro + && !cpp_get_deferred_macro (pfile, node, result->src_loc)) + break; + if (!(node->flags & NODE_DISABLED)) { int ret = 0; @@ -3104,22 +3185,15 @@ warn_of_redefinition (cpp_reader *pfile, cpp_hashnode *node, if (node->flags & NODE_CONDITIONAL) return false; - cpp_macro *macro1 = node->value.macro; - if (macro1->lazy) - { - /* We don't want to mark MACRO as used, but do need to finalize - its laziness. */ - pfile->cb.user_lazy_macro (pfile, macro1, macro1->lazy - 1); - macro1->lazy = 0; - } - - return compare_macros (macro1, macro2); + if (cpp_macro *macro1 = get_deferred_or_lazy_macro (pfile, node, macro2->line)) + return cpp_compare_macros (macro1, macro2); + return false; } /* Return TRUE if MACRO1 and MACRO2 differ. */ -static bool -compare_macros (const cpp_macro *macro1, const cpp_macro *macro2) +bool +cpp_compare_macros (const cpp_macro *macro1, const cpp_macro *macro2) { /* Redefinition of a macro is allowed if and only if the old and new definitions are the same. (6.10.3 paragraph 2). */ @@ -3601,6 +3675,7 @@ _cpp_new_macro (cpp_reader *pfile, cpp_macro_kind kind, void *placement) macro->used = !CPP_OPTION (pfile, warn_unused_macros); macro->count = 0; macro->fun_like = 0; + macro->imported = false; macro->extra_tokens = 0; /* To suppress some diagnostics. */ macro->syshdr = pfile->buffer && pfile->buffer->sysp != 0; @@ -3678,11 +3753,46 @@ cpp_define_lazily (cpp_reader *pfile, cpp_hashnode *node, unsigned num) macro->lazy = num + 1; } +/* NODE is a deferred macro, resolve it, returning the definition + (which may be NULL). */ +cpp_macro * +cpp_get_deferred_macro (cpp_reader *pfile, cpp_hashnode *node, + location_t loc) +{ + node->value.macro = pfile->cb.user_deferred_macro (pfile, loc, node); + + if (!node->value.macro) + node->type = NT_VOID; + + return node->value.macro; +} + +static cpp_macro * +get_deferred_or_lazy_macro (cpp_reader *pfile, cpp_hashnode *node, + location_t loc) +{ + cpp_macro *macro = node->value.macro; + if (!macro) + { + macro = cpp_get_deferred_macro (pfile, node, loc); + if (!macro) + return NULL; + } + + if (macro->lazy) + { + pfile->cb.user_lazy_macro (pfile, macro, macro->lazy - 1); + macro->lazy = 0; + } + + return macro; +} + /* Notify the use of NODE in a macro-aware context (i.e. expanding it, or testing its existance). Also applies any lazy definition. Return FALSE if the macro isn't really there. */ -extern void +extern bool _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, location_t loc) { @@ -3690,14 +3800,8 @@ _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, switch (node->type) { case NT_USER_MACRO: - { - cpp_macro *macro = node->value.macro; - if (macro->lazy) - { - pfile->cb.user_lazy_macro (pfile, macro, macro->lazy - 1); - macro->lazy = 0; - } - } + if (!get_deferred_or_lazy_macro (pfile, node, loc)) + return false; /* FALLTHROUGH. */ case NT_BUILTIN_MACRO: @@ -3713,6 +3817,8 @@ _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, default: abort (); } + + return true; } /* Warn if a token in STRING matches one of a function-like MACRO's @@ -3765,12 +3871,19 @@ check_trad_stringification (cpp_reader *pfile, const cpp_macro *macro, const unsigned char * cpp_macro_definition (cpp_reader *pfile, cpp_hashnode *node) { - unsigned int i, len; - unsigned char *buffer; - gcc_checking_assert (cpp_user_macro_p (node)); - const cpp_macro *macro = node->value.macro; + if (const cpp_macro *macro = get_deferred_or_lazy_macro (pfile, node, 0)) + return cpp_macro_definition (pfile, node, macro); + return NULL; +} + +const unsigned char * +cpp_macro_definition (cpp_reader *pfile, cpp_hashnode *node, + const cpp_macro *macro) +{ + unsigned int i, len; + unsigned char *buffer; /* Calculate length. */ len = NODE_LEN (node) * 10 + 2; /* ' ' and NUL. */