From patchwork Tue Nov 3 21:13:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393413 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=DYVGehJP; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjDy3fTyzB3t3 for ; Wed, 4 Nov 2020 08:13:22 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9BC443861001; Tue, 3 Nov 2020 21:13:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72b.google.com (mail-qk1-x72b.google.com [IPv6:2607:f8b0:4864:20::72b]) by sourceware.org (Postfix) with ESMTPS id 037E03858020 for ; Tue, 3 Nov 2020 21:13:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 037E03858020 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72b.google.com with SMTP id x20so16630635qkn.1 for ; Tue, 03 Nov 2020 13:13:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=FONEYCiXE8omMilknesw6gES7diy5WdTUqsFlOQzuik=; b=DYVGehJPPQqGAXkrKTmuog/rLtnSVzx8gZzdC4Y3XSwmL7h0mqD2VCt5RhDH2AwDy1 5z3C+DOgyRwXyFBi1cmW9IsOHH1wmOK56AogIKpZfKot4+iyYVjnCIWKGzgg00gf53c/ LnF6CK9Qj8VLeZ7WPfVrA8ZE6s31ktEaft+4aUvOvmo2snliEfDBNEiq2AYb3xRdA2WO LUhSV+LbiCNEmyAFm9GFAiYYB2wvD0qNPPpdyuW8Z9I75ZYc+6YSi5EKJi/SGTR75tpM S+ILaMutzkEtJrXIkOX+b19C0VVCnY54VTKKI94nRPxyNip+xC8JWeMBRIa21rCTW9Ug +0CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=FONEYCiXE8omMilknesw6gES7diy5WdTUqsFlOQzuik=; b=duUpAzXHxEzcqnc1B+ugtrKxHCmE0FAqF+TOz4hqWA0gWKshPsSBusPWiaOB3A5358 x/Stx5tcBUTCEQgOGQ2nKZlYazyIO99KEtUNbR+2q2bOO2wLfYiEARM+UI2p6bmP8frF JP+10t6D8PU/BvdQfJVPlmx5Lce31vIf0DoiLdSnbR1LRsN1VWdCS41b7h6U96NyCzZY xBtjI6IcNjKD4Xl1J62V8Eds/AiYBrsKxZAmMj3S+UG5aJpMgI+Er+4GnI6sjO85uWBb +ekkVjtMMXn8CJsyyDgeEYZ5CVzV3GlJRxWoAFSLcZ2vz6V1RFnUbNh3EPgLnzXgVHrz CrUg== X-Gm-Message-State: AOAM53323i0fhk0y6Dz5wy7P1Lp0UTS1xoyqXMpNLjxTVZSKxYiVXUuQ jCz8p+1xUyLFfQdUSd4pxPaEJjQ0+9qOlQ== X-Google-Smtp-Source: ABdhPJwPG3IM3WV0iYyQ9dQF82EHRj5mJHQlSS1lHrNNseuP43ia2z/IB7krlm1M5pzDgBCyyZRhjQ== X-Received: by 2002:a05:620a:a99:: with SMTP id v25mr12462702qkg.406.1604437995320; Tue, 03 Nov 2020 13:13:15 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id y68sm12057518qkb.38.2020.11.03.13.13.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:13:14 -0800 (PST) From: Nathan Sidwell Subject: [01/32] langhooks To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> Message-ID: <1310e4bd-0a39-2245-4360-d4546cf4eb14@acm.org> Date: Tue, 3 Nov 2020 16:13:13 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" I needed a set of hook interfacing the preprocessor to the language. they get called from pieces in c-family. preprocess_main_file: we need to know when any forced headers have been parsed in order to deal with linemaps and macro visibility preprocess_options: A way for the language to adjust any preprocessor options and alter direct callbacks preprocess_undef: We need visibility of #undefs preprocess_deferred_macro: macros from header-units are instantiated lazily. This is the hook for the preprocessor to get that done. preprocess_token: Even in -E processing, we need to observe the token stream in order to load up the macro tables of header units. c-family's c-lex.c, c-opts.c & c-ppoutput.c get to call these hooks in various cases diff --git c/gcc/c-family/c-lex.c w/gcc/c-family/c-lex.c index e81e16ddc26..44575473719 100644 --- c/gcc/c-family/c-lex.c +++ w/gcc/c-family/c-lex.c @@ -28,7 +28,7 @@ along with GCC; see the file COPYING3. If not see #include "c-pragma.h" #include "debug.h" #include "file-prefix-map.h" /* remap_macro_filename() */ - +#include "langhooks.h" #include "attribs.h" /* We may keep statistics about how long which files took to compile. */ @@ -274,9 +274,11 @@ cb_define (cpp_reader *pfile, location_t loc, cpp_hashnode *node) /* #undef callback for DWARF and DWARF2 debug info. */ static void -cb_undef (cpp_reader * ARG_UNUSED (pfile), location_t loc, - cpp_hashnode *node) +cb_undef (cpp_reader *pfile, location_t loc, cpp_hashnode *node) { + if (lang_hooks.preprocess_undef) + lang_hooks.preprocess_undef (pfile, loc, node); + const struct line_map *map = linemap_lookup (line_table, loc); (*debug_hooks->undef) (SOURCE_LINE (linemap_check_ordinary (map), loc), (const char *) NODE_NAME (node)); diff --git c/gcc/c-family/c-opts.c w/gcc/c-family/c-opts.c index 120f4489f6c..c8f08d9e014 100644 --- c/gcc/c-family/c-opts.c +++ w/gcc/c-family/c-opts.c @@ -1105,9 +1116,11 @@ c_common_post_options (const char **pfilename) input_location = UNKNOWN_LOCATION; } - struct cpp_callbacks *cb = cpp_get_callbacks (parse_in); + struct cpp_callbacks *cb = cpp_get_callbacks (parse_in); cb->file_change = cb_file_change; cb->dir_change = cb_dir_change; + if (lang_hooks.preprocess_options) + lang_hooks.preprocess_options (parse_in); cpp_post_options (parse_in); init_global_opts_from_cpp (&global_options, cpp_get_options (parse_in)); @@ -1169,6 +1182,11 @@ c_common_init (void) /* Has to wait until now so that cpplib has its hash table. */ init_pragma (); + struct cpp_callbacks *cb = cpp_get_callbacks (parse_in); + cb->user_deferred_macro = lang_hooks.preprocess_deferred_macro; + if (!cb->undef) + cb->undef = lang_hooks.preprocess_undef; + if (flag_preprocess_only) { c_finish_options (); @@ -1550,7 +1568,13 @@ push_command_line_include (void) cpp_opts->warn_unused_macros = cpp_warn_unused_macros; /* Restore the line map back to the main file. */ if (!cpp_opts->preprocessed) - cpp_change_file (parse_in, LC_RENAME, this_input_filename); + { + cpp_change_file (parse_in, LC_RENAME, this_input_filename); + if (lang_hooks.preprocess_main_file) + /* We're starting the main file. Inform the FE of that. */ + lang_hooks.preprocess_main_file + (parse_in, line_table, LINEMAPS_LAST_ORDINARY_MAP (line_table)); + } /* Set this here so the client can change the option if it wishes, and after stacking the main file so we don't trace the main file. */ @@ -1560,14 +1584,19 @@ push_command_line_include (void) /* File change callback. Has to handle -include files. */ static void -cb_file_change (cpp_reader * ARG_UNUSED (pfile), - const line_map_ordinary *new_map) +cb_file_change (cpp_reader *reader, const line_map_ordinary *new_map) { if (flag_preprocess_only) pp_file_change (new_map); else fe_file_change (new_map); + if (new_map && cpp_opts->preprocessed + && lang_hooks.preprocess_main_file && MAIN_FILE_P (new_map) + && ORDINARY_MAP_STARTING_LINE_NUMBER (new_map)) + /* We're starting the main file. Inform the FE of that. */ + lang_hooks.preprocess_main_file (reader, line_table, new_map); + if (new_map && (new_map->reason == LC_ENTER || new_map->reason == LC_RENAME)) { diff --git c/gcc/c-family/c-ppoutput.c w/gcc/c-family/c-ppoutput.c index 44c6f30e06b..e3e0e59fcc7 100644 --- c/gcc/c-family/c-ppoutput.c +++ w/gcc/c-family/c-ppoutput.c @@ -21,6 +21,7 @@ #include "coretypes.h" #include "c-common.h" /* For flags. */ #include "../libcpp/internal.h" +#include "langhooks.h" #include "c-pragma.h" /* For parse_in. */ #include "file-prefix-map.h" /* remap_macro_filename() */ @@ -301,125 +302,48 @@ token_streamer::stream (cpp_reader *pfile, const cpp_token *token, /* Writes out the preprocessed file, handling spacing and paste avoidance issues. */ + static void scan_translation_unit (cpp_reader *pfile) { - bool avoid_paste = false; - bool do_line_adjustments - = cpp_get_options (parse_in)->lang != CLK_ASM - && !flag_no_line_commands; - bool in_pragma = false; - bool line_marker_emitted = false; + token_streamer streamer (pfile); + uintptr_t filter = 0; + + if (lang_hooks.preprocess_token) + filter = lang_hooks.preprocess_token (pfile, NULL, filter); print.source = NULL; for (;;) { - location_t loc; - const cpp_token *token = cpp_get_token_with_location (pfile, &loc); + location_t spelling_loc; + const cpp_token *token + = cpp_get_token_with_location (pfile, &spelling_loc); - if (token->type == CPP_PADDING) + streamer.stream (pfile, token, spelling_loc); + if (filter) { - avoid_paste = true; - if (print.source == NULL - || (!(print.source->flags & PREV_WHITE) - && token->val.source == NULL)) - print.source = token->val.source; - continue; + unsigned flags = lang_hooks.preprocess_token (pfile, token, filter); + if (flags & lang_hooks::PT_begin_pragma) + streamer.begin_pragma (); } - if (token->type == CPP_EOF) break; + } - /* Subtle logic to output a space if and only if necessary. */ - if (avoid_paste) - { - int src_line = LOCATION_LINE (loc); - - if (print.source == NULL) - print.source = token; - - if (src_line != print.src_line - && do_line_adjustments - && !in_pragma) - { - line_marker_emitted = do_line_change (pfile, token, loc, false); - putc (' ', print.outf); - print.printed = true; - } - else if (print.source->flags & PREV_WHITE - || (print.prev - && cpp_avoid_paste (pfile, print.prev, token)) - || (print.prev == NULL && token->type == CPP_HASH)) - { - putc (' ', print.outf); - print.printed = true; - } - } - else if (token->flags & PREV_WHITE) - { - int src_line = LOCATION_LINE (loc); - - if (src_line != print.src_line - && do_line_adjustments - && !in_pragma) - line_marker_emitted = do_line_change (pfile, token, loc, false); - putc (' ', print.outf); - print.printed = true; - } + if (filter) + lang_hooks.preprocess_token (pfile, NULL, filter); +} - avoid_paste = false; - print.source = NULL; - print.prev = token; - if (token->type == CPP_PRAGMA) - { - const char *space; - const char *name; - - line_marker_emitted = maybe_print_line (token->src_loc); - fputs ("#pragma ", print.outf); - c_pp_lookup_pragma (token->val.pragma, &space, &name); - if (space) - fprintf (print.outf, "%s %s", space, name); - else - fprintf (print.outf, "%s", name); - print.printed = true; - in_pragma = true; - } - else if (token->type == CPP_PRAGMA_EOL) - { - maybe_print_line (token->src_loc); - in_pragma = false; - } - else - { - if (cpp_get_options (parse_in)->debug) - linemap_dump_location (line_table, token->src_loc, print.outf); - - if (do_line_adjustments - && !in_pragma - && !line_marker_emitted - && print.prev_was_system_token != !!in_system_header_at (loc) - && !is_location_from_builtin_token (loc)) - /* The system-ness of this token is different from the one - of the previous token. Let's emit a line change to - mark the new system-ness before we emit the token. */ - { - do_line_change (pfile, token, loc, false); - print.prev_was_system_token = !!in_system_header_at (loc); - } - cpp_output_token (token, print.outf); - line_marker_emitted = false; - print.printed = true; - } +class do_streamer : public token_streamer +{ + public: + uintptr_t filter; - /* CPP_COMMENT tokens and raw-string literal tokens can - have embedded new-line characters. Rather than enumerating - all the possible token types just check if token uses - val.str union member. */ - if (cpp_token_val_index (token) == CPP_TOKEN_FLD_STR) - account_for_newlines (token->val.str.text, token->val.str.len); + do_streamer (cpp_reader *pfile, uintptr_t filter) + :token_streamer (pfile), filter (filter) + { } -} +}; static void directives_only_cb (cpp_reader *pfile, CPP_DO_task task, void *data_, ...) @@ -427,7 +351,7 @@ directives_only_cb (cpp_reader *pfile, CPP_DO_task task, void *data_, ...) va_list args; va_start (args, data_); - token_streamer *streamer = reinterpret_cast (data_); + do_streamer *streamer = reinterpret_cast (data_); switch (task) { default: @@ -452,6 +376,13 @@ directives_only_cb (cpp_reader *pfile, CPP_DO_task task, void *data_, ...) const cpp_token *token = va_arg (args, const cpp_token *); location_t spelling_loc = va_arg (args, location_t); streamer->stream (pfile, token, spelling_loc); + if (streamer->filter) + { + unsigned flags = lang_hooks.preprocess_token + (pfile, token, streamer->filter); + if (flags & lang_hooks::PT_begin_pragma) + streamer->begin_pragma (); + } } break; } @@ -464,8 +395,13 @@ directives_only_cb (cpp_reader *pfile, CPP_DO_task task, void *data_, ...) static void scan_translation_unit_directives_only (cpp_reader *pfile) { - token_streamer streamer (pfile); + uintptr_t filter = 0; + if (lang_hooks.preprocess_token) + filter = lang_hooks.preprocess_token (pfile, NULL, filter); + do_streamer streamer (pfile, filter); cpp_directive_only_process (pfile, &streamer, directives_only_cb); + if (streamer.filter) + lang_hooks.preprocess_token (pfile, NULL, streamer.filter); } /* Adjust print.src_line for newlines embedded in output. */ @@ -564,15 +500,16 @@ print_line_1 (location_t src_loc, const char *special_flags, FILE *stream) unsigned char *to_file_quoted = (unsigned char *) alloca (to_file_len * 4 + 1); - print.src_line = LOCATION_LINE (src_loc); - print.src_file = file_path; - /* cpp_quote_string does not nul-terminate, so we have to do it ourselves. */ unsigned char *p = cpp_quote_string (to_file_quoted, (const unsigned char *) file_path, to_file_len); *p = '\0'; + + print.src_line = LOCATION_LINE (src_loc); + print.src_file = file_path; + fprintf (stream, "# %u \"%s\"%s", print.src_line, to_file_quoted, special_flags); @@ -678,9 +615,10 @@ cb_define (cpp_reader *pfile, location_t line, cpp_hashnode *node) } static void -cb_undef (cpp_reader *pfile ATTRIBUTE_UNUSED, location_t line, - cpp_hashnode *node) +cb_undef (cpp_reader *pfile, location_t line, cpp_hashnode *node) { + if (lang_hooks.preprocess_undef) + lang_hooks.preprocess_undef (pfile, line, node); maybe_print_line (line); fprintf (print.outf, "#undef %s\n", NODE_NAME (node)); print.src_line++; diff --git c/gcc/langhooks-def.h w/gcc/langhooks-def.h index a909915d018..1285d548050 100644 --- c/gcc/langhooks-def.h +++ w/gcc/langhooks-def.h @@ -103,6 +103,11 @@ extern void lhd_finalize_early_debug (void); #define LANG_HOOKS_INIT_OPTIONS_STRUCT hook_void_gcc_optionsp #define LANG_HOOKS_INIT_OPTIONS lhd_init_options #define LANG_HOOKS_INITIALIZE_DIAGNOSTICS lhd_initialize_diagnostics +#define LANG_HOOKS_PREPROCESS_MAIN_FILE NULL +#define LANG_HOOKS_PREPROCESS_OPTIONS NULL +#define LANG_HOOKS_PREPROCESS_UNDEF NULL +#define LANG_HOOKS_PREPROCESS_DEFERRED_MACRO NULL +#define LANG_HOOKS_PREPROCESS_TOKEN NULL #define LANG_HOOKS_REGISTER_DUMPS lhd_register_dumps #define LANG_HOOKS_COMPLAIN_WRONG_LANG_P lhd_complain_wrong_lang_p #define LANG_HOOKS_HANDLE_OPTION lhd_handle_option @@ -317,6 +322,11 @@ extern void lhd_end_section (void); LANG_HOOKS_INIT_OPTIONS_STRUCT, \ LANG_HOOKS_INIT_OPTIONS, \ LANG_HOOKS_INITIALIZE_DIAGNOSTICS, \ + LANG_HOOKS_PREPROCESS_MAIN_FILE, \ + LANG_HOOKS_PREPROCESS_OPTIONS, \ + LANG_HOOKS_PREPROCESS_UNDEF, \ + LANG_HOOKS_PREPROCESS_DEFERRED_MACRO, \ + LANG_HOOKS_PREPROCESS_TOKEN, \ LANG_HOOKS_REGISTER_DUMPS, \ LANG_HOOKS_COMPLAIN_WRONG_LANG_P, \ LANG_HOOKS_HANDLE_OPTION, \ diff --git c/gcc/langhooks.h w/gcc/langhooks.h index a35cf21b673..c70c2317c32 100644 --- c/gcc/langhooks.h +++ w/gcc/langhooks.h @@ -356,6 +356,29 @@ struct lang_hooks global diagnostic context structure. */ void (*initialize_diagnostics) (diagnostic_context *); + /* Beginning the main source file. */ + void (*preprocess_main_file) (cpp_reader *, line_maps *, + const line_map_ordinary *); + + /* Adjust libcpp options and callbacks. */ + void (*preprocess_options) (cpp_reader *); + + /* Undefining a macro. */ + void (*preprocess_undef) (cpp_reader *, location_t, cpp_hashnode *); + + /* Define a deferred macro. */ + struct cpp_macro *(*preprocess_deferred_macro) (cpp_reader *, location_t, + cpp_hashnode *); + + /* Observer for preprocessing stream. */ + uintptr_t (*preprocess_token) (cpp_reader *, const cpp_token *, uintptr_t); + /* Various flags it can return about the token. */ + enum PT_flags + { + PT_begin_pragma = 1 << 0 + }; + + /* Register language-specific dumps. */ void (*register_dumps) (gcc::dump_manager *); From patchwork Tue Nov 3 21:13:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393414 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=YhS62GpJ; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjFF1cw4z9sWp for ; Wed, 4 Nov 2020 08:13:36 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1ACEC3842417; Tue, 3 Nov 2020 21:13:34 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by sourceware.org (Postfix) with ESMTPS id 83F153858020 for ; Tue, 3 Nov 2020 21:13:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 83F153858020 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x733.google.com with SMTP id a65so14371460qkg.13 for ; Tue, 03 Nov 2020 13:13:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=ZYTKuLE24mgJU7/ozMD7xzKALrjP09qW6RLln/L2dZs=; b=YhS62GpJsv/nFDMCrm6V9uOpvImJDi9hOt+qx9Kj86BCx6k2tqYZ/vZ9a9wy9MLiEA aNr7rUli24u50ipllWw+HZ5a+i0q2G2sjBFDV8ybezJTGCptdLlIh5MadRTiXbtDhsx/ BXLhwE0BMvFitNg8KW9G7Av+TFwqRPMDOg3yhrskZqu8SebKtlMv0mrZAoLN3wgW1Okd ZI7mQs+TeLjztWNbE6hicpSExT8EfiOEJODxUtj43bA4TSbnFgEgYfrqjPNiRQZKvT7P bub24DepGFfOdkGOw55tLZmMK9UMXUHqvUC+pe//9TkLIMGS73Ct7ADoRxxk6fE413wJ QChg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=ZYTKuLE24mgJU7/ozMD7xzKALrjP09qW6RLln/L2dZs=; b=KzvNTaN/mzxthkmT2MDxHJKreu3nm4VtVMnHFfNKZ1L5V8XbbAB2/V3/OEAEV3PbrP tqwJKHNYbIGalwRB4MWpLDjfnB/FfGvEuF4GnRafwPihN/Keihybz40nrN9yynUXMi9S xHoRmqt23PbzwsKp8J4yVugsn7vqCmDsykSm8vt8EI5GVE7DVq+yo+6nM99BqmzgHOxr jZWkiW/WB1aZZc6K9TjJ4+6DbY8U6qkPKVQnlUwEsOeAf6eTbl5GkSudFYae1POTjD/D TAFzGRQEF97EIN1BHQdiTWL7wT/pvZE0NvGyW/RHI202QBrIt2LAz9zsHLeAIHa/0pfQ Fu7Q== X-Gm-Message-State: AOAM533mY5QnxuyB0BLLptjv1usBOdzZ/eBjvQwnuSH60V+8fN1cKua8 4JipOpcVg70W73W3jqrRAp0= X-Google-Smtp-Source: ABdhPJy2npFQS+lo3pKkCOag87Ab0hxiiYkdwkFh6xooiSH5nR6kt7b0qt5Irf5P66usEB5e0EekXg== X-Received: by 2002:a37:b785:: with SMTP id h127mr21189239qkf.77.1604438010767; Tue, 03 Nov 2020 13:13:30 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id d73sm12006138qkb.8.2020.11.03.13.13.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:13:29 -0800 (PST) From: Nathan Sidwell Subject: [02/32] linemaps To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> Message-ID: <8b0f8161-8df6-3a55-e24c-5859bae0112f@acm.org> Date: Tue, 3 Nov 2020 16:13:28 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Location handling needs to add 2 things 1) a new kind of inclusion -- namely a module. We add LC_MODULE as a map kind, 2) the ability to allocate blocks of line-maps for both ordinary locations and macro locations, that are then filled in by the module loader. diff --git c/libcpp/include/line-map.h w/libcpp/include/line-map.h index 44008be5c08..50b2e4ff91a 100644 --- c/libcpp/include/line-map.h +++ w/libcpp/include/line-map.h @@ -72,6 +72,7 @@ enum lc_reason LC_RENAME, /* Other reason for name change. */ LC_RENAME_VERBATIM, /* Likewise, but "" != stdin. */ LC_ENTER_MACRO, /* Begin macro expansion. */ + LC_MODULE, /* A (C++) Module. */ /* FIXME: add support for stringize and paste. */ LC_HWM /* High Water Mark. */ }; @@ -439,7 +440,8 @@ struct GTY((tag ("1"))) line_map_ordinary : public line_map { /* Location from whence this line map was included. For regular #includes, this location will be the last location of a map. For - outermost file, this is 0. */ + outermost file, this is 0. For modules it could be anywhere + within a map. */ location_t included_from; /* Size is 20 or 24 bytes, no padding */ @@ -662,6 +664,15 @@ ORDINARY_MAP_IN_SYSTEM_HEADER_P (const line_map_ordinary *ord_map) return ord_map->sysp; } +/* TRUE if this line map is for a module (not a source file). */ + +inline bool +MAP_MODULE_P (const line_map *map) +{ + return (MAP_ORDINARY_P (map) + && linemap_check_ordinary (map)->reason == LC_MODULE); +} + /* Get the filename of ordinary map MAP. */ inline const char * @@ -1076,6 +1087,9 @@ extern void linemap_check_files_exited (class line_maps *); extern location_t linemap_line_start (class line_maps *set, linenum_type to_line, unsigned int max_column_hint); +/* Allocate a raw block of line maps, zero initialized. */ +extern line_map *line_map_new_raw (line_maps *, bool, unsigned); + /* Add a mapping of logical source line to physical source file and line number. This function creates an "ordinary map", which is a map that records locations of tokens that are not part of macro @@ -1093,6 +1107,39 @@ extern const line_map *linemap_add (class line_maps *, enum lc_reason, unsigned int sysp, const char *to_file, linenum_type to_line); +/* Create a macro map. A macro map encodes source locations of tokens + that are part of a macro replacement-list, at a macro expansion + point. See the extensive comments of struct line_map and struct + line_map_macro, in line-map.h. + + This map shall be created when the macro is expanded. The map + encodes the source location of the expansion point of the macro as + well as the "original" source location of each token that is part + of the macro replacement-list. If a macro is defined but never + expanded, it has no macro map. SET is the set of maps the macro + map should be part of. MACRO_NODE is the macro which the new macro + map should encode source locations for. EXPANSION is the location + of the expansion point of MACRO. For function-like macros + invocations, it's best to make it point to the closing parenthesis + of the macro, rather than the the location of the first character + of the macro. NUM_TOKENS is the number of tokens that are part of + the replacement-list of MACRO. */ +const line_map_macro *linemap_enter_macro (line_maps *, cpp_hashnode *, + location_t, unsigned int); + +/* Create a source location for a module. The creator must either do + this after the TU is tokenized, or deal with saving and restoring + map state. */ + +extern location_t linemap_module_loc + (line_maps *, location_t from, const char *name); +extern void linemap_module_reparent + (line_maps *, location_t loc, location_t new_parent); + +/* Restore the linemap state such that the map at LWM-1 continues. */ +extern void linemap_module_restore + (line_maps *, unsigned lwm); + /* Given a logical source location, returns the map which the corresponding (source file, line, column) triplet can be deduced from. Since the set is built chronologically, the logical lines are @@ -1102,6 +1149,8 @@ extern const line_map *linemap_add extern const line_map *linemap_lookup (const line_maps *, location_t); +unsigned linemap_lookup_macro_index (const line_maps *, location_t); + /* Returns TRUE if the line table set tracks token locations across macro expansion, FALSE otherwise. */ bool linemap_tracks_macro_expansion_locs_p (class line_maps *); diff --git c/libcpp/line-map.c w/libcpp/line-map.c index 5a74174579f..4b2b7ef4394 100644 --- c/libcpp/line-map.c +++ w/libcpp/line-map.c @@ -378,23 +378,21 @@ linemap_check_files_exited (line_maps *set) ORDINARY_MAP_FILE_NAME (map)); } -/* Create a new line map in the line map set SET, and return it. - REASON is the reason of creating the map. It determines the type - of map created (ordinary or macro map). Note that ordinary maps and - macro maps are allocated in different memory location. */ +/* Create NUM zero-initialized maps of type MACRO_P. */ -static struct line_map * -new_linemap (line_maps *set, location_t start_location) +line_map * +line_map_new_raw (line_maps *set, bool macro_p, unsigned num) { - bool macro_p = start_location >= LINE_MAP_MAX_LOCATION; unsigned num_maps_allocated = LINEMAPS_ALLOCATED (set, macro_p); unsigned num_maps_used = LINEMAPS_USED (set, macro_p); - - if (num_maps_used == num_maps_allocated) + + if (num > num_maps_allocated - num_maps_used) { /* We need more space! */ if (!num_maps_allocated) num_maps_allocated = 128; + if (num_maps_allocated < num_maps_used + num) + num_maps_allocated = num_maps_used + num; num_maps_allocated *= 2; size_t size_of_a_map; @@ -429,6 +427,7 @@ new_linemap (line_maps *set, location_t start_location) (num_maps - num_maps_used) * size_of_a_map); if (macro_p) set->info_macro.maps = (line_map_macro *)buffer; + else set->info_ordinary.maps = (line_map_ordinary *)buffer; LINEMAPS_ALLOCATED (set, macro_p) = num_maps; @@ -436,13 +435,39 @@ new_linemap (line_maps *set, location_t start_location) line_map *result = (macro_p ? (line_map *)&set->info_macro.maps[num_maps_used] : (line_map *)&set->info_ordinary.maps[num_maps_used]); - LINEMAPS_USED (set, macro_p)++; + LINEMAPS_USED (set, macro_p) += num; + + return result; +} + +/* Create a new line map in the line map set SET, and return it. + REASON is the reason of creating the map. It determines the type + of map created (ordinary or macro map). Note that ordinary maps and + macro maps are allocated in different memory location. */ + +static struct line_map * +new_linemap (line_maps *set, location_t start_location) +{ + line_map *result = line_map_new_raw (set, + start_location >= LINE_MAP_MAX_LOCATION, + 1); result->start_location = start_location; return result; } +/* Return the location of the last source line within an ordinary + map. */ +inline location_t +LAST_SOURCE_LINE_LOCATION (const line_map_ordinary *map) +{ + return (((map[1].start_location - 1 + - map->start_location) + & ~((1 << map->m_column_and_range_bits) - 1)) + + map->start_location); +} + /* Add a mapping of logical source line to physical source file and line number. @@ -570,6 +595,51 @@ linemap_add (line_maps *set, enum lc_reason reason, return map; } +/* Create a location for a module NAME imported at FROM. */ + +location_t +linemap_module_loc (line_maps *set, location_t from, const char *name) +{ + const line_map_ordinary *map + = linemap_check_ordinary (linemap_add (set, LC_MODULE, false, name, 0)); + const_cast (map)->included_from = from; + + location_t loc = linemap_line_start (set, 0, 0); + + return loc; +} + +void +linemap_module_reparent (line_maps *set, location_t loc, location_t adoptor) +{ + const line_map_ordinary *map = linemap_ordinary_map_lookup (set, loc); + const_cast (map)->included_from = adoptor; +} + +/* A linemap at LWM-1 was interrupted to insert module locations & imports. + Append a new map, continuing the interrupted one. */ + +void +linemap_module_restore (line_maps *set, unsigned lwm) +{ + if (lwm && lwm != LINEMAPS_USED (set, false)) + { + const line_map_ordinary *pre_map + = linemap_check_ordinary (LINEMAPS_MAP_AT (set, false, lwm - 1)); + unsigned src_line = SOURCE_LINE (pre_map, + LAST_SOURCE_LINE_LOCATION (pre_map)); + location_t inc_at = pre_map->included_from; + if (const line_map_ordinary *post_map + = (linemap_check_ordinary + (linemap_add (set, LC_RENAME_VERBATIM, + ORDINARY_MAP_IN_SYSTEM_HEADER_P (pre_map), + ORDINARY_MAP_FILE_NAME (pre_map), src_line)))) + /* linemap_add will think we were included from the same as + the preceeding map. */ + const_cast (post_map)->included_from = inc_at; + } +} + /* Returns TRUE if the line table set tracks token locations across macro expansion, FALSE otherwise. */ @@ -1003,14 +1073,25 @@ linemap_macro_map_lookup (const line_maps *set, location_t line) if (set == NULL) return NULL; + unsigned ix = linemap_lookup_macro_index (set, line); + const struct line_map_macro *result = LINEMAPS_MACRO_MAP_AT (set, ix); + linemap_assert (MAP_START_LOCATION (result) <= line); + + return result; +} + +unsigned +linemap_lookup_macro_index (const line_maps *set, location_t line) +{ unsigned mn = LINEMAPS_MACRO_CACHE (set); unsigned mx = LINEMAPS_MACRO_USED (set); const struct line_map_macro *cached = LINEMAPS_MACRO_MAP_AT (set, mn); if (line >= MAP_START_LOCATION (cached)) { - if (mn == 0 || line < MAP_START_LOCATION (&cached[-1])) - return cached; + if (line < (MAP_START_LOCATION (cached) + + MACRO_MAP_NUM_MACRO_TOKENS (cached))) + return mn; mx = mn - 1; mn = 0; } @@ -1025,10 +1106,7 @@ linemap_macro_map_lookup (const line_maps *set, location_t line) } LINEMAPS_MACRO_CACHE (set) = mx; - const struct line_map_macro *result = LINEMAPS_MACRO_MAP_AT (set, mx); - linemap_assert (MAP_START_LOCATION (result) <= line); - - return result; + return mx; } /* Return TRUE if MAP encodes locations coming from a macro @@ -1747,7 +1825,7 @@ linemap_dump (FILE *stream, class line_maps *set, unsigned ix, bool is_macro) { const char *const lc_reasons_v[LC_HWM] = { "LC_ENTER", "LC_LEAVE", "LC_RENAME", "LC_RENAME_VERBATIM", - "LC_ENTER_MACRO" }; + "LC_ENTER_MACRO", "LC_MODULE" }; const line_map *map; unsigned reason; From patchwork Tue Nov 3 21:13:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393415 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=trC+NiMF; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjFQ75WdzB3t3 for ; Wed, 4 Nov 2020 08:13:46 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 049DE3986824; Tue, 3 Nov 2020 21:13:45 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com [IPv6:2607:f8b0:4864:20::f32]) by sourceware.org (Postfix) with ESMTPS id D1C69386F464 for ; Tue, 3 Nov 2020 21:13:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D1C69386F464 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf32.google.com with SMTP id b11so8662743qvr.9 for ; Tue, 03 Nov 2020 13:13:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=5RCxO9b341fEdoyLtmpZMUzlYigGN+eh6qpjDroxqk4=; b=trC+NiMFhaugp9QXdXOjl0+VlY40MGRV7rqhDwDcsONnbbP7ZFjMF9PQEhKbtcfVrj Fw4w0MWVNmgFHjkRnufeThn8AczW658X2FCsEws/enpa6xRn7UZDiR6wvUIPDPNUvr85 ZBY/gZY3ODLKHiRNPfqp2nVcIkd+XcaEvcidQLb2Lq+/vbVEpRvXM4G7E6m/3pi4fGRi pz6BB5F1rt+t5tU69eNEJsKiwVzZvnqrIcW3sh7F7NXFRkTKio/Eso4e0BqyTCpQB5Ih 36XJdQ1sidUVlJ2U6uW8cuec100641AsdxCEN93DUNn2eQBN60+DPPy3sz+VWZboHZ4R 70uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=5RCxO9b341fEdoyLtmpZMUzlYigGN+eh6qpjDroxqk4=; b=UruWq8YQZcBBvk4D1Lb8kU5cOwB79Trl0b1XPIjEl0AnHuh3+hy0cUgyu+3gKAh4OC TltD33pU+Mp5xE8NPC4pUE+hQqbJ/W792S4C+GvJl+ElqRox05bQB54VhdoKKtNZqTic T3cvWWVnJ5JY7TEmN8g2kTRBa4WF4tk5bduVireXUjGMawMnr5GdDEVW384rD3I36XGv KOVDTy7UcXQm4966xqo3FZ1EcWn8ZpOkfIVNXzP8KYwJ7bxa8i/mUYeNURguOig1HPfK xrPdoRIRRHWKa+z+6yI6OppAXapRAVbcGZ9qxt9e4Zxe0aLn9kW/7ViFLCDKs0HEe010 r0tw== X-Gm-Message-State: AOAM531tmXPctwhV554t+BtfKMg0FMCwj8n6C6cDVH55Fz1v8xlGv2Ql O5o534b0S3H2mnLd8PQl6KY= X-Google-Smtp-Source: ABdhPJx8gzmqb1lPyAerhjyY2nxUH158yKhhYYOmZnDHHPF0CGh8MmB0/1eL3Dlyox0K58NWSfCwfg== X-Received: by 2002:a0c:efd2:: with SMTP id a18mr7645186qvt.18.1604438021070; Tue, 03 Nov 2020 13:13:41 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 71sm12047332qko.55.2020.11.03.13.13.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:13:40 -0800 (PST) From: Nathan Sidwell Subject: [03/32] cpp-callbacks To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:13:38 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Here are the callbacks in the preprocessor itself. a) one to handle deferred macros b) one to handle include translation. For every '#include
', there is the possibility of replacing that with 'import
'. This hook determines if that happens. diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -680,6 +695,9 @@ struct cpp_callbacks /* Callback that can change a user lazy into normal macro. */ void (*user_lazy_macro) (cpp_reader *, cpp_macro *, unsigned); + /* Callback to handle deferred cpp_macros. */ + cpp_macro *(*user_deferred_macro) (cpp_reader *, location_t, cpp_hashnode *); + /* Callback to parse SOURCE_DATE_EPOCH from environment. */ time_t (*get_source_date_epoch) (cpp_reader *); @@ -698,6 +716,11 @@ struct cpp_callbacks /* Callback for filename remapping in __FILE__ and __BASE_FILE__ macro expansions. */ const char *(*remap_filename) (const char*); + + /* Maybe translate a #include into something else. Return a + cpp_buffer containing the translation if translating. */ + char *(*translate_include) (cpp_reader *, line_maps *, location_t, + const char *path); }; #ifdef VMS From patchwork Tue Nov 3 21:13:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393416 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=K6kPYiZf; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjFg61rnz9sVM for ; Wed, 4 Nov 2020 08:13:59 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C2FF6398743E; Tue, 3 Nov 2020 21:13:57 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by sourceware.org (Postfix) with ESMTPS id 4EFE7398743E for ; Tue, 3 Nov 2020 21:13:54 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 4EFE7398743E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72d.google.com with SMTP id s14so16578873qkg.11 for ; Tue, 03 Nov 2020 13:13:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=grKX5psdx78bm9rKO9VTtb72RJX9dMdUx0AV+NIc8Lw=; b=K6kPYiZfEiEE8MWdCcbFE3+ipSwaG9i2OvbybkOWsanYrh0zbxy+W+KScAyrIDahVc IoeU7p2n0HtyVyQVtJbplg/wC4YLzAcUdxRK/rbCDwt9N6hc71fPyjnxMEXzn7rVH+KM fNI1rkqhbTmSlux1Xxlk1I4H4nVXd+6y5scrtg0qyC1+C9Q6AKzSMrlv+dS1d5RznCkU IO7laAE4h+kP9USX7sysWOz3+edVXGKxOoLzWRpNt+Hs3sJyf3a44zWpHkYUIML626Ai 7YWcXU7Lil7MujcbDBr0ZXBFBnfPjzLMPwD1jzDjkRUBVTdbEc+56SipOWlRafvtLEV4 qxVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=grKX5psdx78bm9rKO9VTtb72RJX9dMdUx0AV+NIc8Lw=; b=hJFAw9Mw351sc7gKvp1/t/AgsPFdV3J+BV/xsJfA8zMfyCGAvjr0D5vYZ6UzEA2gIH TWZ1l4yWwQfobbOVawiR7wmL4QcSZwuVekmnoVH0/YxJSeiHWtJ8FA0iedck9lms3HqL gpve/mXyq9JZFMDqeTqVHW1ClGale9vS/Vy9snREtmuEFBd36galUkZ1BGknjPp/Gq9N yMSMImXWYnZDAz2PXYI7Nt/Sz61Dcv7v2V0X0fU4SiF1KAgv+BhXrody2XY2/+tmPP7T e2TpRgAWcPEjI6opyoEStHAgJdRmWlRy6y0s4mmuPK6PCtg1tnJbnSNZPqEkM2I+qSiK XAkQ== X-Gm-Message-State: AOAM531UBpojftfiA6f8TCz4M/GxgxX7NKkBjeTcIuGLShzsDTtxXs6n Xb6SpOl96rfWnj5dbJoFGWQ= X-Google-Smtp-Source: ABdhPJyLbjkzHxiQvDUx512HpSqxlgY6lynST9vD+vp9ol9ymgKZo++W2nilAg0XLMxuP4I+yZ3DiA== X-Received: by 2002:a37:4f0b:: with SMTP id d11mr22552046qkb.74.1604438033753; Tue, 03 Nov 2020 13:13:53 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id t184sm11898649qka.19.2020.11.03.13.13.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:13:52 -0800 (PST) From: Nathan Sidwell Subject: [04/32] cpp lexer To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:13:51 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" c++ modules creates 2 new kinds of preprocessor lines [export] module ... [export] import ... To all intents and purposes these are cppdirectives spelt without a leading '#'. module and import are context-sensitive keywords. Thus preprocessor tokenizing needs a bit of token peeking. This is that peeking. We have a new node flag 'NODE_MODULE', which marks whether an identifier is significant to this peeking. When we see such an identifier at the beginning of a logical line, we need to peek further and figure out whether these are those keywords. When successfully peeked, we replace the identifiers with internal-only tokens that the c++ parser recognizes. diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -487,6 +494,9 @@ struct cpp_options /* Nonzero for the '::' token. */ unsigned char scope; + /* Nonzero means tokenize C++20 module directives. */ + unsigned char module_directives; + /* Holds the name of the target (execution) character set. */ const char *narrow_charset; @@ -831,6 +857,7 @@ struct GTY(()) cpp_macro { #define NODE_USED (1 << 5) /* Dumped with -dU. */ #define NODE_CONDITIONAL (1 << 6) /* Conditional macro */ #define NODE_WARN_OPERATOR (1 << 7) /* Warn about C++ named operator. */ +#define NODE_MODULE (1 << 8) /* C++-20 module-related name. */ /* Different flavors of hash node. */ enum node_type @@ -888,9 +915,9 @@ struct GTY(()) cpp_hashnode { unsigned int directive_index : 7; /* If is_directive, then index into directive table. Otherwise, a NODE_OPERATOR. */ - unsigned char rid_code; /* Rid code - for front ends. */ + unsigned int rid_code : 8; /* Rid code - for front ends. */ + unsigned int flags : 9; /* CPP flags. */ ENUM_BITFIELD(node_type) type : 2; /* CPP node type. */ - unsigned int flags : 8; /* CPP flags. */ /* 6 bits spare (plus another 32 on 64-bit hosts). */ diff --git c/libcpp/lex.c w/libcpp/lex.c index fb222924c8c..b3498f195bf 100644 --- c/libcpp/lex.c +++ w/libcpp/lex.c @@ -2606,6 +2622,131 @@ _cpp_temp_token (cpp_reader *pfile) return result; } +/* RESULT is a CPP_NAME with NODE_MODULE set. See if we should enter + deferred_pragma mode to tokenize the rest of the line. */ + +static void +cpp_maybe_module_directive (cpp_reader *pfile, cpp_token *result) +{ + unsigned backup = 0; /* Tokens we peeked. */ + cpp_hashnode *node = result->val.node.node; + cpp_token *peek = result; + cpp_token *keyword = peek; + cpp_hashnode *(&n_modules)[spec_nodes::M_HWM][2] = pfile->spec_nodes.n_modules; + int header_count = 0; + + /* Enter directives mode for the peeking. */ + pfile->state.in_deferred_pragma = true; + pfile->state.pragma_allow_expansion = true; + pfile->state.save_comments = 0; + pfile->directive_line = result->src_loc; + + if (node == n_modules[spec_nodes::M_EXPORT][0]) + { + peek = _cpp_lex_direct (pfile); + keyword = peek; + backup++; + if (keyword->type != CPP_NAME) + goto not_module; + node = keyword->val.node.node; + if (!(node->flags & NODE_MODULE)) + goto not_module; + } + + if (__builtin_expect (node == n_modules[spec_nodes::M__IMPORT][0], false)) + /* __import */ + header_count = backup + 2 + 16; + else if (__builtin_expect (node == n_modules[spec_nodes::M_IMPORT][0], false)) + /* import */ + header_count = backup + 2 + (CPP_OPTION (pfile, preprocessed) ? 16 : 0); + else if (__builtin_expect (node == n_modules[spec_nodes::M_MODULE][0], false)) + ; /* module */ + else + goto not_module; + + /* We've seen [export] {module|import|__import}. Check the next token. */ + if (header_count) + /* After '{,__}import' a header name may appear. */ + pfile->state.angled_headers = true; + peek = _cpp_lex_direct (pfile); + backup++; + + /* ... import followed by identifier, ':', '<' or + header-name preprocessing tokens, or module + followed by cpp-identifier, ':' or ';' preprocessing + tokens. C++ keywords are not yet relevant. */ + if (peek->type == CPP_NAME + || peek->type == CPP_COLON + || (header_count + ? (peek->type == CPP_LESS + || (peek->type == CPP_STRING && peek->val.str.text[0] != 'R') + || peek->type == CPP_HEADER_NAME) + : peek->type == CPP_SEMICOLON)) + { + pfile->state.pragma_allow_expansion = !CPP_OPTION (pfile, preprocessed); + if (!pfile->state.pragma_allow_expansion) + pfile->state.prevent_expansion++; + + if (!header_count && linemap_included_from + (LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table))) + cpp_error_with_line (pfile, CPP_DL_ERROR, keyword->src_loc, 0, + "module control-line cannot be in included file"); + + /* The first one or two tokens cannot be macro names. */ + for (int ix = backup; ix--;) + { + cpp_token *tok = ix ? keyword : result; + cpp_hashnode *node = tok->val.node.node; + + /* Don't attempt to expand the token. */ + tok->flags |= NO_EXPAND; + if (_cpp_defined_macro_p (node) + && _cpp_maybe_notify_macro_use (pfile, node, tok->src_loc) + && !cpp_fun_like_macro_p (node)) + cpp_error_with_line (pfile, CPP_DL_ERROR, tok->src_loc, 0, + "module control-line \"%s\" cannot be" + " an object-like macro", + NODE_NAME (node)); + } + + /* Map to underbar variants. */ + keyword->val.node.node = n_modules[header_count + ? spec_nodes::M_IMPORT + : spec_nodes::M_MODULE][1]; + if (backup != 1) + result->val.node.node = n_modules[spec_nodes::M_EXPORT][1]; + + /* Maybe tell the tokenizer we expect a header-name down the + road. */ + pfile->state.directive_file_token = header_count; + } + else + { + not_module: + /* Drop out of directive mode. */ + pfile->state.save_comments + = !CPP_OPTION (pfile, discard_comments); + pfile->state.in_deferred_pragma = false; + pfile->state.angled_headers = false; + } + + /* In either case we want to backup the peeked tokens. */ + if (backup) + { + /* If we saw EOL, we should drop it, because this isn't a module + control-line after all. */ + bool eol = peek->type == CPP_PRAGMA_EOL; + if (!eol || backup > 1) + { + /* Put put the peeked tokens back */ + _cpp_backup_tokens_direct (pfile, backup); + /* But if the last one was an EOL, forget it. */ + if (eol) + pfile->lookaheads--; + } + } +} + /* Lex a token into RESULT (external interface). Takes care of issues like directive handling, token lookahead, multiple include optimization and skipping. */ @@ -2654,6 +2795,22 @@ _cpp_lex_token (cpp_reader *pfile) } else if (pfile->state.in_deferred_pragma) result = &pfile->directive_result; + else if (result->type == CPP_NAME + && __builtin_expect + (result->val.node.node->flags & NODE_MODULE, 0) + && !pfile->state.skipping + /* Unlike regular directives, we do not deal with + tokenizing module directives as macro arguments. + That's not permitted. */ + && !pfile->state.parsing_args) + { + /* P1857. Before macro expansion, At start of logical + line ... */ + /* We don't have to consider lookaheads at this point. */ + gcc_checking_assert (!pfile->lookaheads); + + cpp_maybe_module_directive (pfile, result); + } if (pfile->cb.line_change && !pfile->state.skipping) pfile->cb.line_change (pfile, result, pfile->state.parsing_args); @@ -3446,7 +3609,11 @@ cpp_output_token (const cpp_token *token, FILE *fp) break; case SPELL_LITERAL: + if (token->type == CPP_HEADER_NAME) + fputc ('"', fp); fwrite (token->val.str.text, 1, token->val.str.len, fp); + if (token->type == CPP_HEADER_NAME) + fputc ('"', fp); break; case SPELL_NONE: @@ -3932,6 +4099,188 @@ do_peek_prev (const unsigned char *peek, const unsigned char *bound) return peek; } +/* If PEEK[-1] is identifier MATCH, scan past it and trailing white + space. Otherwise return NULL. */ + +static const unsigned char * +do_peek_ident (const char *match, const unsigned char *peek, + const unsigned char *limit) +{ + for (; *++match; peek++) + if (*peek != *match) + { + peek = do_peek_next (peek, limit); + if (*peek != *match) + return NULL; + } + + /* Must now not be looking at an identifier char. */ + peek = do_peek_next (peek, limit); + if (ISIDNUM (*peek)) + return NULL; + + /* Skip control-line whitespace. */ + ws: + while (*peek == ' ' || *peek == '\t') + peek++; + if (__builtin_expect (*peek == '\\', false)) + { + peek = do_peek_backslash (peek, limit); + if (*peek != '\\') + goto ws; + } + + return peek; +} + +/* Are we looking at a module control line starting as PEEK - 1? */ + +static bool +do_peek_module (cpp_reader *pfile, unsigned char c, + const unsigned char *peek, const unsigned char *limit) +{ + bool import = false; + + if (__builtin_expect (c == 'e', false)) + { + if (!((peek[0] == 'x' || peek[0] == '\\') + && (peek = do_peek_ident ("export", peek, limit)))) + return false; + + /* export, peek for import or module. No need to peek __import + here. */ + if (peek[0] == 'i') + { + if (!((peek[1] == 'm' || peek[1] == '\\') + && (peek = do_peek_ident ("import", peek + 1, limit)))) + return false; + import = true; + } + else if (peek[0] == 'm') + { + if (!((peek[1] == 'o' || peek[1] == '\\') + && (peek = do_peek_ident ("module", peek + 1, limit)))) + return false; + } + else + return false; + } + else if (__builtin_expect (c == 'i', false)) + { + if (!((peek[0] == 'm' || peek[0] == '\\') + && (peek = do_peek_ident ("import", peek, limit)))) + return false; + import = true; + } + else if (__builtin_expect (c == '_', false)) + { + /* Needed for translated includes. */ + if (!((peek[0] == '_' || peek[0] == '\\') + && (peek = do_peek_ident ("__import", peek, limit)))) + return false; + import = true; + } + else if (__builtin_expect (c == 'm', false)) + { + if (!((peek[0] == 'o' || peek[0] == '\\') + && (peek = do_peek_ident ("module", peek, limit)))) + return false; + } + else + return false; + + /* Peek the next character to see if it's good enough. We'll be at + the first non-whitespace char, including skipping an escaped + newline. */ + /* ... import followed by identifier, ':', '<' or header-name + preprocessing tokens, or module followed by identifier, ':' or + ';' preprocessing tokens. */ + unsigned char p = *peek++; + + /* A character literal is ... single quotes, ... optionally preceded + by u8, u, U, or L */ + /* A string-literal is a ... double quotes, optionally prefixed by + R, u8, u8R, u, uR, U, UR, L, or LR */ + if (p == 'u') + { + peek = do_peek_next (peek, limit); + if (*peek == '8') + { + peek++; + goto peek_u8; + } + goto peek_u; + } + else if (p == 'U' || p == 'L') + { + peek_u8: + peek = do_peek_next (peek, limit); + peek_u: + if (*peek == '\"' || *peek == '\'') + return false; + + if (*peek == 'R') + goto peek_R; + /* Identifier. Ok. */ + } + else if (p == 'R') + { + peek_R: + if (CPP_OPTION (pfile, rliterals)) + { + peek = do_peek_next (peek, limit); + if (*peek == '\"') + return false; + } + /* Identifier. Ok. */ + } + else if ('Z' - 'A' == 25 + ? ((p >= 'A' && p <= 'Z') || (p >= 'a' && p <= 'z') || p == '_') + : ISIDST (p)) + { + /* Identifier. Ok. */ + } + else if (p == '<') + { + /* Maybe angle header, ok for import. Reject + '<=', '<<' digraph:'<:'. */ + if (!import) + return false; + peek = do_peek_next (peek, limit); + if (*peek == '=' || *peek == '<' + || (*peek == ':' && CPP_OPTION (pfile, digraphs))) + return false; + } + else if (p == ';') + { + /* SEMICOLON, ok for module. */ + if (import) + return false; + } + else if (p == '"') + { + /* STRING, ok for import. */ + if (!import) + return false; + } + else if (p == ':') + { + /* Maybe COLON, ok. Reject '::', digraph:':>'. */ + peek = do_peek_next (peek, limit); + if (*peek == ':' || (*peek == '>' && CPP_OPTION (pfile, digraphs))) + return false; + } + else + /* FIXME: Detect a unicode character, excluding those not + permitted as the initial character. [lex.name]/1. I presume + we need to check the \[uU] spellings, and directly using + Unicode in say UTF8 form? Or perhaps we do the phase-1 + conversion of UTF8 to universal-character-names? */ + return false; + + return true; +} + /* Directives-only scanning. Somewhat more relaxed than correct parsing -- some ill-formed programs will not be rejected. */ @@ -3940,6 +4289,8 @@ cpp_directive_only_process (cpp_reader *pfile, void *data, void (*cb) (cpp_reader *, CPP_DO_task, void *, ...)) { + bool module_p = CPP_OPTION (pfile, module_directives); + do { restart: @@ -4332,6 +4683,51 @@ cpp_directive_only_process (cpp_reader *pfile, } goto dflt; + case '_': + case 'e': + case 'i': + case 'm': + if (bol && module_p && !pfile->state.skipping + && do_peek_module (pfile, c, pos, limit)) + { + /* We've seen the start of a module control line. + Start up the tokenizer. */ + pos--; /* Backup over the first character. */ + + /* Backup over whitespace to start of line. */ + while (pos > line_start + && (pos[-1] == ' ' || pos[-1] == '\t')) + pos--; + + if (pos > base) + cb (pfile, CPP_DO_print, data, line_count, base, pos - base); + + /* Prep things for directive handling. */ + buffer->next_line = pos; + buffer->need_line = true; + + /* Now get tokens until the PRAGMA_EOL. */ + do + { + location_t spelling; + const cpp_token *tok + = cpp_get_token_with_location (pfile, &spelling); + + gcc_assert (pfile->state.in_deferred_pragma + || tok->type == CPP_PRAGMA_EOL); + cb (pfile, CPP_DO_token, data, tok, spelling); + } + while (pfile->state.in_deferred_pragma); + + if (pfile->buffer->next_line < pfile->buffer->rlimit) + cb (pfile, CPP_DO_location, data, + pfile->line_table->highest_line); + + pfile->mi_valid = false; + goto restart; + } + goto dflt; + default: dflt: bol = false; diff --git c/libcpp/macro.c w/libcpp/macro.c index e304f67c2e0..f5f280dfdc7 100644 --- c/libcpp/macro.c +++ w/libcpp/macro.c @@ -2930,6 +2932,85 @@ cpp_get_token_1 (cpp_reader *pfile, location_t *location) } pfile->about_to_expand_macro_p = saved_about_to_expand_macro; + + if (pfile->state.directive_file_token + && !pfile->state.parsing_args + && !(result->type == CPP_PADDING || result->type == CPP_COMMENT) + && !(15 & --pfile->state.directive_file_token)) + { + /* Do header-name frobbery. Concatenate < ... > as approprate. + Do header search if needed, and finally drop the outer <> or + "". */ + pfile->state.angled_headers = false; + + /* Do angle-header reconstitution. Then do include searching. + We'll always end up with a ""-quoted header-name in that + case. If searching finds nothing, we emit a diagnostic and + an empty string. */ + size_t len = 0; + char *fname = NULL; + + cpp_token *tmp = _cpp_temp_token (pfile); + *tmp = *result; + + tmp->type = CPP_HEADER_NAME; + bool need_search = !pfile->state.directive_file_token; + pfile->state.directive_file_token = 0; + + bool angle = result->type != CPP_STRING; + if (result->type == CPP_HEADER_NAME + || (result->type == CPP_STRING && result->val.str.text[0] != 'R')) + { + len = result->val.str.len - 2; + fname = XNEWVEC (char, len + 1); + memcpy (fname, result->val.str.text + 1, len); + fname[len] = 0; + } + else if (result->type == CPP_LESS) + fname = _cpp_bracket_include (pfile); + + if (fname) + { + /* We have a header-name. Look it up. This will emit an + unfound diagnostic. Canonicalize the found name. */ + const char *found = fname; + + if (need_search) + { + found = cpp_find_header_unit (pfile, fname, angle, tmp->src_loc); + if (!found) + found = ""; + len = strlen (found); + } + /* Force a leading './' if it's not absolute. */ + bool dotme = (found[0] == '.' ? !IS_DIR_SEPARATOR (found[1]) + : found[0] && !IS_ABSOLUTE_PATH (found)); + + if (BUFF_ROOM (pfile->u_buff) < len + 1 + dotme * 2) + _cpp_extend_buff (pfile, &pfile->u_buff, len + 1 + dotme * 2); + unsigned char *buf = BUFF_FRONT (pfile->u_buff); + size_t pos = 0; + + if (dotme) + { + buf[pos++] = '.'; + /* Apparently '/' is unconditional. */ + buf[pos++] = '/'; + } + memcpy (&buf[pos], found, len); + pos += len; + buf[pos] = 0; + + tmp->val.str.len = pos; + tmp->val.str.text = buf; + + tmp->type = CPP_HEADER_NAME; + XDELETEVEC (fname); + + result = tmp; + } + } + return result; } diff --git c/gcc/c-family/c-lex.c w/gcc/c-family/c-lex.c index e81e16ddc26..44575473719 100644 --- c/gcc/c-family/c-lex.c +++ w/gcc/c-family/c-lex.c @@ -654,8 +656,11 @@ c_lex_with_flags (tree *value, location_t *loc, unsigned char *cpp_flags, *value = build_int_cst (integer_type_node, tok->val.pragma); break; - /* These tokens should not be visible outside cpplib. */ case CPP_HEADER_NAME: + *value = build_string (tok->val.str.len, (const char *)tok->val.str.text); + break; + + /* These tokens should not be visible outside cpplib. */ case CPP_MACRO_ARG: gcc_unreachable (); diff --git c/libcpp/init.c w/libcpp/init.c index 6c52f50de39..96ade569457 100644 --- c/libcpp/init.c +++ w/libcpp/init.c @@ -840,4 +855,27 @@ post_options (cpp_reader *pfile) CPP_OPTION (pfile, trigraphs) = 0; CPP_OPTION (pfile, warn_trigraphs) = 0; } + + if (CPP_OPTION (pfile, module_directives)) + { + /* These unspellable tokens have a leading space. */ + const char *const inits[spec_nodes::M_HWM] + = {"export ", "module ", "import ", "__import"}; + + for (int ix = 0; ix != spec_nodes::M_HWM; ix++) + { + cpp_hashnode *node = cpp_lookup (pfile, UC (inits[ix]), + strlen (inits[ix])); + + /* Token we pass to the compiler. */ + pfile->spec_nodes.n_modules[ix][1] = node; + + if (ix != spec_nodes::M__IMPORT) + /* Token we recognize when lexing, drop the trailing ' '. */ + node = cpp_lookup (pfile, NODE_NAME (node), NODE_LEN (node) - 1); + + node->flags |= NODE_MODULE; + pfile->spec_nodes.n_modules[ix][0] = node; + } + } } diff --git c/libcpp/internal.h w/libcpp/internal.h index 4759961a33a..17b65601b66 100644 --- c/libcpp/internal.h +++ w/libcpp/internal.h @@ -280,6 +280,9 @@ struct lexer_state /* Nonzero when tokenizing a deferred pragma. */ unsigned char in_deferred_pragma; + /* Count to token that is a header-name. */ + unsigned char directive_file_token; + /* Nonzero if the deferred pragma being handled allows macro expansion. */ unsigned char pragma_allow_expansion; }; @@ -292,6 +295,12 @@ struct spec_nodes cpp_hashnode *n_false; /* C++ keyword false */ cpp_hashnode *n__VA_ARGS__; /* C99 vararg macros */ cpp_hashnode *n__VA_OPT__; /* C++ vararg macros */ + + enum {M_EXPORT, M_MODULE, M_IMPORT, M__IMPORT, M_HWM}; + + /* C++20 modules, only set when module_directives is in effect. + incoming variants [0], outgoing ones [1] */ + cpp_hashnode *n_modules[M_HWM][2]; }; typedef struct _cpp_line_note _cpp_line_note; From patchwork Tue Nov 3 21:14:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393417 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=gH5kJz7W; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjFs5lfNz9sVW for ; Wed, 4 Nov 2020 08:14:09 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9CA4D3987428; Tue, 3 Nov 2020 21:14:07 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) by sourceware.org (Postfix) with ESMTPS id 0886F3987426 for ; Tue, 3 Nov 2020 21:14:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 0886F3987426 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72a.google.com with SMTP id k9so16616672qki.6 for ; Tue, 03 Nov 2020 13:14:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=JGpw3uY8XASPH/QZWK9JqofWOo6OVjZyMGwPtG/35Jg=; b=gH5kJz7WPVQ7MRGMLhSKvNPx69XRRP+GRcS+nIJS7jmFMKc/9HJrIfdm7FSLEt4MIn JKrOmzDf5eveOi/mZh53kEWhm5QRGwIh5gsCUrMDSFMIGAIUzoARBmUjqcYIR+ZH6OO1 f8lF3/DaChRF/x6wCwZnt0ui1zimbUmqVSKl3dhllE6mh67wh2/RHmd/BogqExSg3JOt YVbld0cYwB8QTJ/jyhWxNYo2cKJlZqvRqgLhQRdbM+rejPcXRgaW6UveBMobrfGAzCcD NmsDiAu8uAyHSkteg/fSHAtxAAjDVcnLTJY6xH4lb6Qod0n8Teuu+yclnc0omgz0GSCB VP2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=JGpw3uY8XASPH/QZWK9JqofWOo6OVjZyMGwPtG/35Jg=; b=cO3UAD1k2NxMnYObZDhO4p9vwX5UCQmwlaPTuSw161HGgImGjrmhOLS3QCkspl1gxt DUtVSzwaTzpfS0FOvkZauWN7in0HwdPtyLy1mfp29yvH2CYwntT+YIn8FfioPdnV4frS YjxHKI4r8C4XWe/E+QLD6zHcA1jSXp1UvWQOyv2oIFFxfWoR60QhHzJiZrsiLQGcHpwM ZLGL6gRIJAA6o9dDFH4Dz+MiQeg7jEVpC1Vc1iU0vOrPDMJsfvCEkR6Woz5Ej/QVGKzX B+GQgXnWx3zadqpSL3vn5QBTO4fTgzsEN9Vg36+gcVrgOzwDM34dmIxaSSkOFELth7kV KAMg== X-Gm-Message-State: AOAM532zvhxgZ8Y/r9AAgLzGRCq8vWTvBaShd/FHF2BivIi47uoFfxQ+ yUi9xTAaSJ9yKlRl6fyM7/E= X-Google-Smtp-Source: ABdhPJxnxHcTHT8QvFYJwb4hlhinJ0MMwgmPOoObLN+YsUr4Q4BfCgDRd04id0KIWkW8Mt7MptzaSw== X-Received: by 2002:a05:620a:1353:: with SMTP id c19mr16301727qkl.496.1604438044462; Tue, 03 Nov 2020 13:14:04 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id n199sm12064430qkn.77.2020.11.03.13.14.02 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:03 -0800 (PST) From: Nathan Sidwell Subject: [05/32] cpp files To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> Message-ID: <95b23354-fe6b-8c3b-6da0-2551fbc6d0fe@acm.org> Date: Tue, 3 Nov 2020 16:14:02 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" As I mentioned in patch 03, include translation is a thing. This amends the file handling to determine that, and for resolving explicit imports of header-units The logic for locating a header unit is the same as locating a header-file. Except there's a final step of mapping the header-file name to a compiled module interface. That latter step is handled elsewhere. Also, as one needs an explicit compile step to build a header-unit, we need a way of telling the preprocessor that the main file is itself a header, and it should locate the appropriate place on the INCLUDE path so INCLUDE_NEXT works. nathan diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -971,6 +1002,9 @@ extern cpp_callbacks *cpp_get_callbacks (cpp_reader *) ATTRIBUTE_PURE; extern void cpp_set_callbacks (cpp_reader *, cpp_callbacks *); extern class mkdeps *cpp_get_deps (cpp_reader *) ATTRIBUTE_PURE; +extern const char *cpp_find_header_unit (cpp_reader *, const char *file, + bool angle_p, location_t); + /* This function reads the file, but does not start preprocessing. It returns the name of the original file; this is the same as the input file, except for preprocessed input. This will generate at diff --git c/libcpp/files.c w/libcpp/files.c index 5af41364d0a..c23cd18ef2a 100644 --- c/libcpp/files.c +++ w/libcpp/files.c @@ -111,6 +111,9 @@ struct _cpp_file /* If this file is implicitly preincluded. */ bool implicit_preinclude : 1; + + /* Is a C++ Module header unit. */ + int header_unit : 2; }; /* A singly-linked list for all searches for a given file name, with @@ -891,9 +894,9 @@ has_unique_contents (cpp_reader *pfile, _cpp_file *file, bool import, } /* Place the file referenced by FILE into a new buffer on the buffer - stack if possible. IMPORT is true if this stacking attempt is - because of a #import directive. Returns true if a buffer is - stacked. Use LOC for any diagnostics. */ + stack if possible. Returns true if a buffer is stacked. Use LOC + for any diagnostics. */ + bool _cpp_stack_file (cpp_reader *pfile, _cpp_file *file, include_type type, location_t loc) @@ -901,39 +904,75 @@ _cpp_stack_file (cpp_reader *pfile, _cpp_file *file, include_type type, if (is_known_idempotent_file (pfile, file, type == IT_IMPORT)) return false; - if (!read_file (pfile, file, loc)) - return false; - - if (!has_unique_contents (pfile, file, type == IT_IMPORT, loc)) - return false; - int sysp = 0; - if (pfile->buffer && file->dir) - sysp = MAX (pfile->buffer->sysp, file->dir->sysp); - - /* Add the file to the dependencies on its first inclusion. */ - if (CPP_OPTION (pfile, deps.style) > (sysp != 0) - && !file->stack_count - && file->path[0] - && !(file->main_file && CPP_OPTION (pfile, deps.ignore_main_file))) - deps_add_dep (pfile->deps, file->path); - - /* Clear buffer_valid since _cpp_clean_line messes it up. */ - file->buffer_valid = false; - file->stack_count++; - - /* Stack the buffer. */ - cpp_buffer *buffer - = cpp_push_buffer (pfile, file->buffer, file->st.st_size, - CPP_OPTION (pfile, preprocessed) - && !CPP_OPTION (pfile, directives_only)); - buffer->file = file; - buffer->sysp = sysp; - buffer->to_free = file->buffer_start; + char *buf = nullptr; - /* Initialize controlling macro state. */ - pfile->mi_valid = true; - pfile->mi_cmacro = 0; + /* Check C++ module include translation. */ + if (!file->header_unit && type < IT_HEADER_HWM + /* Do not include translate include-next. */ + && type != IT_INCLUDE_NEXT + && pfile->cb.translate_include) + buf = (pfile->cb.translate_include + (pfile, pfile->line_table, loc, file->path)); + + if (buf) + { + /* We don't increment the line number at the end of a buffer, + because we don't usually need that location (we're popping an + include file). However in this case we do want to do the + increment. So push a writable buffer of two newlines to acheive + that. */ + static uchar newlines[] = "\n\n"; + cpp_push_buffer (pfile, newlines, 2, true); + + cpp_buffer *buffer + = cpp_push_buffer (pfile, reinterpret_cast (buf), + strlen (buf), true); + buffer->to_free = buffer->buf; + + file->header_unit = +1; + _cpp_mark_file_once_only (pfile, file); + } + else + { + /* Not a header unit, and we know it. */ + file->header_unit = -1; + + if (!read_file (pfile, file, loc)) + return false; + + if (!has_unique_contents (pfile, file, type == IT_IMPORT, loc)) + return false; + + if (pfile->buffer && file->dir) + sysp = MAX (pfile->buffer->sysp, file->dir->sysp); + + /* Add the file to the dependencies on its first inclusion. */ + if (CPP_OPTION (pfile, deps.style) > (sysp != 0) + && !file->stack_count + && file->path[0] + && !(file->main_file && CPP_OPTION (pfile, deps.ignore_main_file))) + deps_add_dep (pfile->deps, file->path); + + /* Clear buffer_valid since _cpp_clean_line messes it up. */ + file->buffer_valid = false; + file->stack_count++; + + /* Stack the buffer. */ + cpp_buffer *buffer + = cpp_push_buffer (pfile, file->buffer, file->st.st_size, + CPP_OPTION (pfile, preprocessed) + && !CPP_OPTION (pfile, directives_only)); + buffer->file = file; + buffer->sysp = sysp; + buffer->main_file = (type >= IT_HEADER_HWM + && !CPP_OPTION (pfile, main_search)); + buffer->to_free = file->buffer_start; + + /* Initialize controlling macro state. */ + pfile->mi_valid = true; + pfile->mi_cmacro = 0; + } /* In the case of a normal #include, we're now at the start of the line *following* the #include. A separate location_t for this @@ -941,19 +980,30 @@ _cpp_stack_file (cpp_reader *pfile, _cpp_file *file, include_type type, This does not apply if we found a PCH file, we're not a regular include, or we ran out of locations. */ - if (file->pchname == NULL - && type < IT_DIRECTIVE_HWM - && pfile->line_table->highest_location != LINE_MAP_MAX_LOCATION - 1) + bool decrement = (file->pchname == NULL + && type < IT_DIRECTIVE_HWM + && (pfile->line_table->highest_location + != LINE_MAP_MAX_LOCATION - 1)); + if (decrement) pfile->line_table->highest_location--; - /* Add line map and do callbacks. */ - _cpp_do_file_change (pfile, LC_ENTER, file->path, + if (file->header_unit <= 0) + /* Add line map and do callbacks. */ + _cpp_do_file_change (pfile, LC_ENTER, file->path, /* With preamble injection, start on line zero, so the preamble doesn't appear to have been included from line 1. Likewise when starting preprocessed, we expect an initial locating line. */ - type == IT_PRE_MAIN ? 0 : 1, sysp); + type == IT_PRE_MAIN ? 0 : 1, sysp); + else if (decrement) + { + /* Adjust the line back one so we appear on the #include line itself. */ + const line_map_ordinary *map + = LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table); + linenum_type line = SOURCE_LINE (map, pfile->line_table->highest_line); + linemap_line_start (pfile->line_table, line - 1, 0); + } return true; } @@ -1058,6 +1108,65 @@ _cpp_stack_include (cpp_reader *pfile, const char *fname, int angle_brackets, return _cpp_stack_file (pfile, file, type, loc); } +/* NAME is a header file name, find the path we'll use to open it. */ + +const char * +cpp_find_header_unit (cpp_reader *pfile, const char *name, bool angle, + location_t loc) +{ + cpp_dir *dir = search_path_head (pfile, name, angle, IT_INCLUDE); + if (!dir) + return NULL; + + _cpp_file *file = _cpp_find_file (pfile, name, dir, angle, + _cpp_FFK_NORMAL, loc); + if (!file) + return NULL; + + if (file->fd > 0) + { + /* Don't leave it open. */ + close (file->fd); + file->fd = 0; + } + + file->header_unit = +1; + _cpp_mark_file_once_only (pfile, file); + return file->path; +} + +/* Retrofit the just-entered main file asif it was an include. This + will permit correct include_next use, and mark it as a system + header if that's where it resides. We use filesystem-appropriate + prefix matching of the include path to locate the main file. */ +void +cpp_retrofit_as_include (cpp_reader *pfile) +{ + /* We should be the outermost. */ + gcc_assert (!pfile->buffer->prev); + + if (const char *name = pfile->main_file->name) + { + /* Locate name on the include dir path, using a prefix match. */ + size_t name_len = strlen (name); + for (cpp_dir *dir = pfile->quote_include; dir; dir = dir->next) + if (dir->len < name_len + && IS_DIR_SEPARATOR (name[dir->len]) + && !filename_ncmp (name, dir->name, dir->len)) + { + pfile->main_file->dir = dir; + pfile->buffer->main_file = false; + if (dir->sysp) + cpp_make_system_header (pfile, 1, 0); + break; + } + } + + /* Initialize controlling macro state. */ + pfile->mi_valid = true; + pfile->mi_cmacro = 0; +} + /* Could not open FILE. The complication is dependency output. */ static void open_file_failed (cpp_reader *pfile, _cpp_file *file, int angle_brackets, From patchwork Tue Nov 3 21:14:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393418 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=X3up9hP7; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjG51fcCz9sXb for ; Wed, 4 Nov 2020 08:14:21 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 221AC398745C; Tue, 3 Nov 2020 21:14:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72c.google.com (mail-qk1-x72c.google.com [IPv6:2607:f8b0:4864:20::72c]) by sourceware.org (Postfix) with ESMTPS id AD77A398745C for ; Tue, 3 Nov 2020 21:14:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org AD77A398745C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72c.google.com with SMTP id 140so16627322qko.2 for ; Tue, 03 Nov 2020 13:14:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=coUIVx0ZOE5hwK7wB67l4ZbcILGH+IrRcU/HOYLOzSI=; b=X3up9hP7RfUnipgircIQ/YT0Ql/2AXSrjp2GTJA+fmPDpv/hp2Co+WgRgzsyhiJita wUjziH6FroMrLnUorzgR4r+vsO0Yj/kUzu5HG9wH75LWFhut9/hkFOhzBTE+aGvXm2BE Og3jwIBDQWLWrBXZnh7wGgdc0q/RCOhcZ+o+77DS4PQ1BmH8zJzO3NGqe9bzkv93thUg V+5anyaFsEOl2JQBRfDxMmPJcrHb1xkvO+5ZErK/G8ptb0kqLzILgnBJljFVX5ZKgt/4 QciO18TbY7RFI9ZEqqbVqdao9DgXB4P7ka9FDDs+bcEnDSw9RmM4RI3PKpgmC7tbOWPa 1FaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=coUIVx0ZOE5hwK7wB67l4ZbcILGH+IrRcU/HOYLOzSI=; b=i8S6pGTnBOqKPg/rzKjcC3mlWvcsjskykQamKpYEKNt7zfueGrqL9QGY0JRZQljAic nrU0Fw4lC+zqdVPB3omRpynMC8ch8DCbsTXP82RkSDiDuyaP4dTtSsdPjPB1s7u9ZFBC 1fdnFnOS27PggnCtSHe/5T6542z6IsU6M2TciTvM9M0jH0+aR0qDMnvA+5GrmZY9msPV Xy9+J1fKTE51S9FvGKMKBzLRCGUvF+wPoOxaquxLx9enhJcWsAybYXeCn7Td+CQyW4jg ci6fFUaUfxAeQiwplxYc8BuGdsdQBMpUCtv/QwedxgHrytqJHKrrS5VkKlmLnm1aINMz imWw== X-Gm-Message-State: AOAM533a4M3+RFynJV2iRXTp5rOHtcwxrTAm91xEbnoNtMatzDTlXaOA CcutNl23f6Xg3wqrxmKuY4xXsg5fwZe9ew== X-Google-Smtp-Source: ABdhPJxqTJBYWiu+gzzReC/nerG4gpO2M5OFKN76ODNbERsuwdqezMg5bqC1rtKfEAW52MgV8VG7kg== X-Received: by 2002:a37:7d43:: with SMTP id y64mr20199779qkc.225.1604438054778; Tue, 03 Nov 2020 13:14:14 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id k4sm11996034qkj.46.2020.11.03.13.14.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:13 -0800 (PST) From: Nathan Sidwell Subject: [06/32] cpp macros To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:14:11 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Header units can provide macros to an importer. For efficiency that is done lazily. When we import a header unit we mark the identifers it defines as significant. It's only when we need the macro definition (including in #ifdef &| defined (X) processing) that we resolve the macro. We already had lazy (builtin) macros. This extends it to deferred macros. Each deferred macro has an index the resolver can use. on an LP64 host that index does not inlarge the node structure. As a macro can be undefined by this mechanism, the users of this deferred interface must be prepared to handle 'not a macro after all' diff --git c/libcpp/directives.c w/libcpp/directives.c index d7b59aae901..899ebbf023a 100644 --- c/libcpp/directives.c +++ w/libcpp/directives.c @@ -667,7 +667,8 @@ do_undef (cpp_reader *pfile) pfile->directive_line, 0, "undefining \"%s\"", NODE_NAME (node)); - if (CPP_OPTION (pfile, warn_unused_macros)) + if (node->value.macro + && CPP_OPTION (pfile, warn_unused_macros)) _cpp_warn_if_unused_macro (pfile, node, NULL); _cpp_free_definition (node); @@ -1981,8 +1983,10 @@ do_ifdef (cpp_reader *pfile) if (node) { skip = !_cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line)) + /* It wasn't a macro after all. */ + skip = true; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line); if (pfile->cb.used) pfile->cb.used (pfile, pfile->directive_line, node); check_eol (pfile, false); @@ -2006,8 +2010,10 @@ do_ifndef (cpp_reader *pfile) if (node) { skip = _cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line)) + /* It wasn't a macro after all. */ + skip = false; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, pfile->directive_line); if (pfile->cb.used) pfile->cb.used (pfile, pfile->directive_line, node); check_eol (pfile, false); diff --git c/libcpp/expr.c w/libcpp/expr.c index e01a47a8c34..894e8515c90 100644 --- c/libcpp/expr.c +++ w/libcpp/expr.c @@ -1061,6 +1061,7 @@ parse_defined (cpp_reader *pfile) } } + bool is_defined = false; if (node) { if ((pfile->context != initial_context @@ -1068,9 +1069,11 @@ parse_defined (cpp_reader *pfile) && CPP_OPTION (pfile, warn_expansion_to_defined)) cpp_pedwarning (pfile, CPP_W_EXPANSION_TO_DEFINED, "this use of \"defined\" may not be portable"); - + is_defined = _cpp_defined_macro_p (node); + if (!_cpp_maybe_notify_macro_use (pfile, node, token->src_loc)) + /* It wasn't a macro after all. */ + is_defined = false; _cpp_mark_macro_used (node); - _cpp_maybe_notify_macro_use (pfile, node, token->src_loc); /* A possible controlling macro of the form #if !defined (). _cpp_parse_expr checks there was no other junk on the line. */ @@ -1086,7 +1089,7 @@ parse_defined (cpp_reader *pfile) result.unsignedp = false; result.high = 0; result.overflow = false; - result.low = node && _cpp_defined_macro_p (node); + result.low = is_defined; return result; } diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -308,6 +308,13 @@ enum cpp_normalize_level { @@ -801,7 +824,10 @@ struct GTY(()) cpp_macro { tokens. */ unsigned int extra_tokens : 1; - /* 1 bits spare (32-bit). 33 on 64-bit target. */ + /* Imported (from a legacy header module). */ + unsigned int imported : 1; + + /* 0 bits spare (32-bit). 32 on 64-bit target. */ union cpp_exp_u { @@ -874,7 +901,7 @@ enum cpp_builtin_type union GTY(()) _cpp_hashnode_value { /* Assert (maybe NULL) */ cpp_macro * GTY((tag ("NT_VOID"))) answers; - /* Macro (never NULL) */ + /* Macro (maybe NULL) */ cpp_macro * GTY((tag ("NT_USER_MACRO"))) macro; /* Code for a builtin macro. */ enum cpp_builtin_type GTY ((tag ("NT_BUILTIN_MACRO"))) builtin; @@ -888,5 +915,9 @@ struct GTY(()) cpp_hashnode { /* 6 bits spare (plus another 32 on 64-bit hosts). */ + + /* On a 64-bit system there would be 32-bits of padding to the value + field. So placing the deferred index here is not costly. */ + unsigned deferred; /* Deferred index, (unless zero). */ union _cpp_hashnode_value GTY ((desc ("%1.type"))) value; }; @@ -1027,6 +1066,18 @@ inline bool cpp_macro_p (const cpp_hashnode *node) { return node->type & NT_MACRO_MASK; } +inline cpp_macro *cpp_set_deferred_macro (cpp_hashnode *node, + cpp_macro *forced = NULL) +{ + cpp_macro *old = node->value.macro; + + node->value.macro = forced; + node->type = NT_USER_MACRO; + node->flags &= ~NODE_USED; + + return old; +} +cpp_macro *cpp_get_deferred_macro (cpp_reader *, cpp_hashnode *, location_t); /* Returns true if NODE is a function-like user macro. */ inline bool cpp_fun_like_macro_p (cpp_hashnode *node) @@ -1034,11 +1085,13 @@ inline bool cpp_fun_like_macro_p (cpp_hashnode *node) return cpp_user_macro_p (node) && node->value.macro->fun_like; } -extern const unsigned char *cpp_macro_definition (cpp_reader *, - cpp_hashnode *); +extern const unsigned char *cpp_macro_definition (cpp_reader *, cpp_hashnode *); +extern const unsigned char *cpp_macro_definition (cpp_reader *, cpp_hashnode *, + const cpp_macro *); inline location_t cpp_macro_definition_location (cpp_hashnode *node) { - return node->value.macro->line; + const cpp_macro *macro = node->value.macro; + return macro ? macro->line : 0; } extern void _cpp_backup_tokens (cpp_reader *, unsigned int); extern const cpp_token *cpp_peek_token (cpp_reader *, int); @@ -1219,6 +1272,8 @@ extern int cpp_ideq (const cpp_token *, const char *); extern void cpp_output_line (cpp_reader *, FILE *); extern unsigned char *cpp_output_line_to_string (cpp_reader *, const unsigned char *); +extern const unsigned char *cpp_alloc_token_string + (cpp_reader *, const unsigned char *, unsigned); extern void cpp_output_token (const cpp_token *, FILE *); extern const char *cpp_type2name (enum cpp_ttype, unsigned char flags); /* Returns the value of an escape sequence, truncated to the correct @@ -1274,6 +1329,8 @@ extern void cpp_scan_nooutput (cpp_reader *); extern int cpp_sys_macro_p (cpp_reader *); extern unsigned char *cpp_quote_string (unsigned char *, const unsigned char *, unsigned int); +extern bool cpp_compare_macros (const cpp_macro *macro1, + const cpp_macro *macro2); /* In files.c */ extern bool cpp_included (cpp_reader *, const char *); diff --git c/libcpp/internal.h w/libcpp/internal.h index 4759961a33a..a92abf281c0 100644 --- c/libcpp/internal.h +++ w/libcpp/internal.h @@ -649,13 +665,14 @@ inline bool _cpp_defined_macro_p (cpp_hashnode *node) } /* In macro.c */ -extern void _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, - location_t loc); -inline void _cpp_maybe_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, +extern bool _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, + location_t); +inline bool _cpp_maybe_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, location_t loc) { if (!(node->flags & NODE_USED)) - _cpp_notify_macro_use (pfile, node, loc); + return _cpp_notify_macro_use (pfile, node, loc); + return true; } extern cpp_macro *_cpp_new_macro (cpp_reader *, cpp_macro_kind, void *); extern void _cpp_free_definition (cpp_hashnode *); @@ -869,29 +886,7 @@ ufputs (const unsigned char *s, FILE *f) return fputs ((const char *)s, f); } - /* In line-map.c. */ - -/* Create a macro map. A macro map encodes source locations of tokens - that are part of a macro replacement-list, at a macro expansion - point. See the extensive comments of struct line_map and struct - line_map_macro, in line-map.h. - - This map shall be created when the macro is expanded. The map - encodes the source location of the expansion point of the macro as - well as the "original" source location of each token that is part - of the macro replacement-list. If a macro is defined but never - expanded, it has no macro map. SET is the set of maps the macro - map should be part of. MACRO_NODE is the macro which the new macro - map should encode source locations for. EXPANSION is the location - of the expansion point of MACRO. For function-like macros - invocations, it's best to make it point to the closing parenthesis - of the macro, rather than the the location of the first character - of the macro. NUM_TOKENS is the number of tokens that are part of - the replacement-list of MACRO. */ -const line_map_macro *linemap_enter_macro (class line_maps *, - struct cpp_hashnode*, - location_t, - unsigned int); +/* In line-map.c. */ /* Create and return a virtual location for a token that is part of a macro expansion-list at a macro expansion point. See the comment diff --git c/libcpp/lex.c w/libcpp/lex.c index fb222924c8c..b3498f195bf 100644 --- c/libcpp/lex.c +++ w/libcpp/lex.c @@ -1577,13 +1577,20 @@ static void create_literal (cpp_reader *pfile, cpp_token *token, const uchar *base, unsigned int len, enum cpp_ttype type) { - uchar *dest = _cpp_unaligned_alloc (pfile, len + 1); - - memcpy (dest, base, len); - dest[len] = '\0'; token->type = type; token->val.str.len = len; - token->val.str.text = dest; + token->val.str.text = cpp_alloc_token_string (pfile, base, len); +} + +const uchar * +cpp_alloc_token_string (cpp_reader *pfile, + const unsigned char *ptr, unsigned len) +{ + uchar *dest = _cpp_unaligned_alloc (pfile, len + 1); + + dest[len] = 0; + memcpy (dest, ptr, len); + return dest; } /* A pair of raw buffer pointers. The currently open one is [1], the diff --git c/libcpp/macro.c w/libcpp/macro.c index e304f67c2e0..f5f280dfdc7 100644 --- c/libcpp/macro.c +++ w/libcpp/macro.c @@ -268,6 +268,8 @@ class vaopt_state { /* Macro expansion. */ +static cpp_macro *get_deferred_or_lazy_macro (cpp_reader *, cpp_hashnode *, + location_t); static int enter_macro_context (cpp_reader *, cpp_hashnode *, const cpp_token *, location_t); static int builtin_macro (cpp_reader *, cpp_hashnode *, @@ -338,10 +340,6 @@ static cpp_macro *create_iso_definition (cpp_reader *); /* #define directive parsing and handling. */ static cpp_macro *lex_expansion_token (cpp_reader *, cpp_macro *); -static bool warn_of_redefinition (cpp_reader *, cpp_hashnode *, - const cpp_macro *); -static bool compare_macros (const cpp_macro *, const cpp_macro *); - static bool parse_params (cpp_reader *, unsigned *, bool *); static void check_trad_stringification (cpp_reader *, const cpp_macro *, const cpp_string *); @@ -353,8 +351,6 @@ static const cpp_token* cpp_get_token_1 (cpp_reader *, location_t *); static cpp_hashnode* macro_of_context (cpp_context *context); -static bool in_macro_expansion_p (cpp_reader *pfile); - /* Statistical counter tracking the number of macros that got expanded. */ unsigned num_expanded_macros_counter = 0; @@ -2845,6 +2841,12 @@ cpp_get_token_1 (cpp_reader *pfile, location_t *location) if (node->type == NT_VOID || (result->flags & NO_EXPAND)) break; + if (!(node->flags & NODE_USED) + && node->type == NT_USER_MACRO + && !node->value.macro + && !cpp_get_deferred_macro (pfile, node, result->src_loc)) + break; + if (!(node->flags & NODE_DISABLED)) { int ret = 0; @@ -3104,22 +3185,15 @@ warn_of_redefinition (cpp_reader *pfile, cpp_hashnode *node, if (node->flags & NODE_CONDITIONAL) return false; - cpp_macro *macro1 = node->value.macro; - if (macro1->lazy) - { - /* We don't want to mark MACRO as used, but do need to finalize - its laziness. */ - pfile->cb.user_lazy_macro (pfile, macro1, macro1->lazy - 1); - macro1->lazy = 0; - } - - return compare_macros (macro1, macro2); + if (cpp_macro *macro1 = get_deferred_or_lazy_macro (pfile, node, macro2->line)) + return cpp_compare_macros (macro1, macro2); + return false; } /* Return TRUE if MACRO1 and MACRO2 differ. */ -static bool -compare_macros (const cpp_macro *macro1, const cpp_macro *macro2) +bool +cpp_compare_macros (const cpp_macro *macro1, const cpp_macro *macro2) { /* Redefinition of a macro is allowed if and only if the old and new definitions are the same. (6.10.3 paragraph 2). */ @@ -3601,6 +3675,7 @@ _cpp_new_macro (cpp_reader *pfile, cpp_macro_kind kind, void *placement) macro->used = !CPP_OPTION (pfile, warn_unused_macros); macro->count = 0; macro->fun_like = 0; + macro->imported = false; macro->extra_tokens = 0; /* To suppress some diagnostics. */ macro->syshdr = pfile->buffer && pfile->buffer->sysp != 0; @@ -3678,11 +3753,46 @@ cpp_define_lazily (cpp_reader *pfile, cpp_hashnode *node, unsigned num) macro->lazy = num + 1; } +/* NODE is a deferred macro, resolve it, returning the definition + (which may be NULL). */ +cpp_macro * +cpp_get_deferred_macro (cpp_reader *pfile, cpp_hashnode *node, + location_t loc) +{ + node->value.macro = pfile->cb.user_deferred_macro (pfile, loc, node); + + if (!node->value.macro) + node->type = NT_VOID; + + return node->value.macro; +} + +static cpp_macro * +get_deferred_or_lazy_macro (cpp_reader *pfile, cpp_hashnode *node, + location_t loc) +{ + cpp_macro *macro = node->value.macro; + if (!macro) + { + macro = cpp_get_deferred_macro (pfile, node, loc); + if (!macro) + return NULL; + } + + if (macro->lazy) + { + pfile->cb.user_lazy_macro (pfile, macro, macro->lazy - 1); + macro->lazy = 0; + } + + return macro; +} + /* Notify the use of NODE in a macro-aware context (i.e. expanding it, or testing its existance). Also applies any lazy definition. Return FALSE if the macro isn't really there. */ -extern void +extern bool _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, location_t loc) { @@ -3690,14 +3800,8 @@ _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, switch (node->type) { case NT_USER_MACRO: - { - cpp_macro *macro = node->value.macro; - if (macro->lazy) - { - pfile->cb.user_lazy_macro (pfile, macro, macro->lazy - 1); - macro->lazy = 0; - } - } + if (!get_deferred_or_lazy_macro (pfile, node, loc)) + return false; /* FALLTHROUGH. */ case NT_BUILTIN_MACRO: @@ -3713,6 +3817,8 @@ _cpp_notify_macro_use (cpp_reader *pfile, cpp_hashnode *node, default: abort (); } + + return true; } /* Warn if a token in STRING matches one of a function-like MACRO's @@ -3765,12 +3871,19 @@ check_trad_stringification (cpp_reader *pfile, const cpp_macro *macro, const unsigned char * cpp_macro_definition (cpp_reader *pfile, cpp_hashnode *node) { - unsigned int i, len; - unsigned char *buffer; - gcc_checking_assert (cpp_user_macro_p (node)); - const cpp_macro *macro = node->value.macro; + if (const cpp_macro *macro = get_deferred_or_lazy_macro (pfile, node, 0)) + return cpp_macro_definition (pfile, node, macro); + return NULL; +} + +const unsigned char * +cpp_macro_definition (cpp_reader *pfile, cpp_hashnode *node, + const cpp_macro *macro) +{ + unsigned int i, len; + unsigned char *buffer; /* Calculate length. */ len = NODE_LEN (node) * 10 + 2; /* ' ' and NUL. */ From patchwork Tue Nov 3 21:14:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393419 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=qhHEwAaR; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjGJ0Tttz9sXk for ; Wed, 4 Nov 2020 08:14:32 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C39F0398746B; Tue, 3 Nov 2020 21:14:29 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by sourceware.org (Postfix) with ESMTPS id DEE74398746E for ; Tue, 3 Nov 2020 21:14:26 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org DEE74398746E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72f.google.com with SMTP id z6so16649687qkz.4 for ; Tue, 03 Nov 2020 13:14:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=yLi6UyreNQ++Z5zhh3DdvnZKRuD7wgPzUrlfq07lxt0=; b=qhHEwAaREVt+DHPOR/4xJO4rmaWcQpcxjNgzwyn27PEv9PimpBtZeO6seGS5vSWdpD m2aU8O9VA4NFGrmEa9MVBYtlCkY+wrQ3pkMpYwqH6M2KLFSrd7GjvmGbEkI8ifCMGQgn NeawQ9ri+lyiweh/wwEa+l42gl/cdmvPDXYYK3IEhQ31TPuTGjUE+EPK4Xxxey2b29y0 WwQEPMSRjlDgLnE5oTupP16qkDznZdOZ9PRGfmWfbumuR1wweW3BiHjiK82EWbjk/KaY h5Q4ysne5CvVqR5c2LR3GIoK66rJ1Au/yt/cWD7uWLlf8QDC9B64eBBTJXHgF4ivgY4s Z+GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=yLi6UyreNQ++Z5zhh3DdvnZKRuD7wgPzUrlfq07lxt0=; b=YRotu8BQSF8jDzXgrs8l756AGDIsI1LhLIJrS4Z9pirEmYzdwLeIzxHwnJoG7YQv9r MlAuZ3DiG9wG03pVGcOXY3kcznN6v6dlAq/U9UyW1fl6osGsPLZ7JF+L46iQiO34UI4p 2tO05UDWjvpdLqsJ7lw49kxJGYkem4Vf+u9LW8p5up9ehWnFAlu7KG/WS3P890gkT16E InwPdH+/60JzAZOPCxYG0apQsua6e5hS44SYR6yyktWoyZ8vm7pL5qRiERKsrTO9MKI8 SB1+dQeWJAm/pm70ARsoKzOqu/MkXvekqKy257r54aIuDBfpd0g4xTNy7j2dumvd2zU5 QdtA== X-Gm-Message-State: AOAM533n7S6tgMF/U0Wss4kSZs945HtHQTTpJ3pVXNkbYWpoLohXxQBn NhBBDoYOackaB7r2fnrlwSU= X-Google-Smtp-Source: ABdhPJzH01J/mdgVJfOWTyvbBgSYT56yfky6Z6eHJ5q7XgbjNuWoKWtyoD2t8xUIkmvPaXlgQRi7eg== X-Received: by 2002:a37:4855:: with SMTP id v82mr22102163qka.393.1604438066350; Tue, 03 Nov 2020 13:14:26 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id n81sm5043218qke.99.2020.11.03.13.14.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:25 -0800 (PST) From: Nathan Sidwell Subject: [07/32] cpp main To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> Message-ID: <400dae8f-6ad3-5266-455c-fce13eb89282@acm.org> Date: Tue, 3 Nov 2020 16:14:24 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <4d14b230-3263-9a13-3159-c4853f282761@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Here's the changes to starting the main file. I have added the ability to search the user or system include paths for the main file. That's real helpful to users attempting to build header-units. I'll discuss the CLI with the options patch. Also recording the location at which the main file starts is helpful for whole-file diagnostics. diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -308,6 +308,13 @@ enum cpp_normalize_level { normalized_none }; +enum cpp_main_search +{ + CMS_none, + CMS_user, + CMS_system +}; + /* This structure is nested inside struct cpp_reader, and carries all the options visible to the command line. */ struct cpp_options @@ -560,6 +573,8 @@ struct cpp_options /* The maximum depth of the nested #include. */ unsigned int max_include_depth; + + cpp_main_search main_search : 8; }; /* Diagnostic levels. To get a diagnostic without associating a @@ -978,6 +1012,10 @@ extern class mkdeps *cpp_get_deps (cpp_reader *) ATTRIBUTE_PURE; too. If there was an error opening the file, it returns NULL. */ extern const char *cpp_read_main_file (cpp_reader *, const char *, bool injecting = false); +extern location_t cpp_main_loc (const cpp_reader *); + +/* Adjust for the main file to be an include. */ +extern void cpp_retrofit_as_include (cpp_reader *); /* Set up built-ins with special behavior. Use cpp_init_builtins() instead unless your know what you are doing. */ diff --git c/libcpp/init.c w/libcpp/init.c index 6c52f50de39..96ade569457 100644 --- c/libcpp/init.c +++ w/libcpp/init.c @@ -672,8 +672,14 @@ cpp_read_main_file (cpp_reader *pfile, const char *fname, bool injecting) deps_add_default_target (deps, fname); pfile->main_file - = _cpp_find_file (pfile, fname, &pfile->no_search_path, /*angle=*/0, - _cpp_FFK_NORMAL, 0); + = _cpp_find_file (pfile, fname, + CPP_OPTION (pfile, preprocessed) ? &pfile->no_search_path + : CPP_OPTION (pfile, main_search) == CMS_user + ? pfile->quote_include + : CPP_OPTION (pfile, main_search) == CMS_system + ? pfile->bracket_include : &pfile->no_search_path, + /*angle=*/0, _cpp_FFK_NORMAL, 0); + if (_cpp_find_failed (pfile->main_file)) return NULL; @@ -695,7 +701,16 @@ cpp_read_main_file (cpp_reader *pfile, const char *fname, bool injecting) LINEMAP_LINE (last), LINEMAP_SYSP (last)); } - return ORDINARY_MAP_FILE_NAME (LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table)); + auto *map = LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table); + pfile->main_loc = MAP_START_LOCATION (map); + + return ORDINARY_MAP_FILE_NAME (map); +} + +location_t +cpp_main_loc (const cpp_reader *pfile) +{ + return pfile->main_loc; } /* For preprocessed files, if the very first characters are diff --git c/libcpp/internal.h w/libcpp/internal.h index 4759961a33a..17b65601b66 100644 --- c/libcpp/internal.h +++ w/libcpp/internal.h @@ -357,6 +366,9 @@ struct cpp_buffer token from the enclosing buffer is returned. */ bool return_at_eof : 1; + /* Is from main file. */ + bool main_file : 1; + /* One for a system header, two for a C system header file that therefore needs to be extern "C" protected in C++, and zero otherwise. */ unsigned char sysp; @@ -583,6 +595,10 @@ struct cpp_reader /* If non-zero, the lexer will use this location for the next token instead of getting a location from the linemap. */ location_t forced_token_location; + + /* Location identifying the main source file -- intended to be line + zero of said file. */ + location_t main_loc; }; /* Character classes. Based on the more primitive macros in safe-ctype.h. @@ -635,7 +651,7 @@ static inline int cpp_in_primary_file (cpp_reader *); static inline int cpp_in_primary_file (cpp_reader *pfile) { - return pfile->line_table->depth == 1; + return pfile->buffer->main_file; } /* True if NODE is a macro for the purposes of ifdef, defined etc. */ From patchwork Tue Nov 3 21:14:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393420 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Xlkh3gp1; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjGX2h0wz9sRK for ; Wed, 4 Nov 2020 08:14:44 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 3ADAD3987472; Tue, 3 Nov 2020 21:14:42 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf35.google.com (mail-qv1-xf35.google.com [IPv6:2607:f8b0:4864:20::f35]) by sourceware.org (Postfix) with ESMTPS id 288FE3987469 for ; Tue, 3 Nov 2020 21:14:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 288FE3987469 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf35.google.com with SMTP id da2so6465575qvb.0 for ; Tue, 03 Nov 2020 13:14:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=G7viNOa7klWcaH07PVtjRB9Ml90YmEGA51HB0vcXzJ8=; b=Xlkh3gp1R7kvokTgjPhnyiskroe+Unr07Vaux2vzYCF3JVF41ek4PgbBcVmPEtWUzp ekE2euxAov+EiVZop568A1/ZE9wAhFIBoEdh3bkv4DgT2/q2m+WNCmr+lUnuVumMnhSK SMx8sxrhlog+UAqD5drp45tyWgLpvQYEZNmbHsfYxBBhOxjlKB4gf7XeWrzIrup5tLCf SVSYcSaO6LZO6aBj8e7qs9Z/D3+7Xj4NC5rGuIOrK33CSEJV0goNUvnRY3Pp8+QvjZSg w3xvM4mO72LJuxM+Z1085tOlO9LcEy8IgBUElx7J/J/PcI0YHEeviyfCffJCP243Ng5h gB0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=G7viNOa7klWcaH07PVtjRB9Ml90YmEGA51HB0vcXzJ8=; b=ef7UIv4/DmnJP7FoihJuzwc2/00oQcomb78WSwODs6tcej9Wt203zvGy6Zy//rlV6c 1LiOKNhjFqN3kI3XdGctWXppSWYQaQ+XDDZZtBjrAYCjWCzXfdTLeizzrosits9Nj1Dk qyDN7g2Hf9shZA/7rwFo5h0s7R4xsW1aut5d1opspPD16jF7BX+z9EZ4yPaWLcIbgE2c p6A2KGORmKQfwJ6C+0XJ1dJewZ+2S193f7kXBB4aM9oULBTv//qEweCBVTWpp8SVfaJe SSaTtLTHERU7NwG1DL6ZgIr0KnYJBWDxZ1Llnj+8XafET0pYgUVdWWqxT+9yJQwQPG0r b7LA== X-Gm-Message-State: AOAM533xeRKsIxLQ5+f0Jvq0Yju9s8q63wnGADZXD0qoC5UOr3rs1H+l MdO26wC5LGRSVkhFti0NfA8= X-Google-Smtp-Source: ABdhPJxvsLGxe4bWy6RusqAvksaqyPDQ869ya7UMxucpWnwAtvWhsN8fgFWGeNzxgFxz0x4nLUCGkw== X-Received: by 2002:ad4:42ae:: with SMTP id e14mr10316891qvr.34.1604438078500; Tue, 03 Nov 2020 13:14:38 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id z134sm11985714qka.21.2020.11.03.13.14.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:37 -0800 (PST) From: Nathan Sidwell Subject: [08/32] cpp mkdeps To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:14:35 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <688bd28f-5998-0def-8c40-03b817832d63@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" The final bit of preprocessor change is that to mkdeps. We have a new kind of thing to be dependent upon -- a compiled module interface. That's also an additional target. This adds the logic to mkdeps to keep those dependencies separate from include file dependencies. I made use of this when demonstrating a module-aware Make POC. diff --git c/libcpp/include/mkdeps.h w/libcpp/include/mkdeps.h index 6d05351cb4a..1d6dd9562a5 100644 --- c/libcpp/include/mkdeps.h +++ w/libcpp/include/mkdeps.h @@ -55,9 +57,12 @@ extern void deps_add_default_target (class mkdeps *, const char *); dependency entered should be the primary source file. */ extern void deps_add_dep (class mkdeps *, const char *); -/* Write out a deps buffer to a specified file. The third argument +extern void deps_add_module (struct mkdeps *, const char *, + const char * = NULL, bool = false); + +/* Write out a deps buffer to a specified file. The last argument is the number of columns to word-wrap at (0 means don't wrap). */ -extern void deps_write (const class mkdeps *, FILE *, bool, unsigned int); +extern void deps_write (const cpp_reader *, FILE *, unsigned int); /* Write out a deps buffer to a file, in a form that can be read back with deps_restore. Returns nonzero on error, in which case the diff --git c/libcpp/mkdeps.c w/libcpp/mkdeps.c index ea5f060c380..0e70b0b633b 100644 --- c/libcpp/mkdeps.c +++ w/libcpp/mkdeps.c @@ -81,7 +81,7 @@ public: }; mkdeps () - : quote_lwm (0) + : module_name (NULL), cmi_name (NULL), is_header_unit (false), quote_lwm (0) { } ~mkdeps () @@ -94,14 +94,22 @@ public: free (const_cast (deps[i])); for (i = vpath.size (); i--;) XDELETEVEC (vpath[i].str); + for (i = modules.size (); i--;) + XDELETEVEC (modules[i]); + XDELETEVEC (module_name); + free (const_cast (cmi_name)); } public: vec targets; vec deps; vec vpath; + vec modules; public: + const char *module_name; + const char *cmi_name; + bool is_header_unit; unsigned short quote_lwm; }; @@ -323,6 +331,27 @@ deps_add_vpath (class mkdeps *d, const char *vpath) } } +/* Add a new module dependency. M is the module name, with P being + any partition name thereof (might be NULL). If CMI is NULL, this + is an import dependency. Otherwise, this is an output dependency + specifying CMI as the output file, of type IS_HEADER_UNIT. */ + +void +deps_add_module (struct mkdeps *d, const char *m, + const char *cmi, bool is_header_unit) +{ + m = xstrdup (m); + + if (cmi) + { + d->module_name = m; + d->is_header_unit = is_header_unit; + d->cmi_name = xstrdup (cmi); + } + else + d->modules.push (m); +} + /* Write NAME, with a leading space to FP, a Makefile. Advance COL as appropriate, wrap at COLMAX, returning new column number. Iff QUOTE apply quoting. Append TRAIL. */ @@ -379,6 +408,8 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) if (d->deps.size ()) { column = make_write_vec (d->targets, fp, 0, colmax, d->quote_lwm); + if (CPP_OPTION (pfile, deps.modules) && d->cmi_name) + column = make_write_name (d->cmi_name, fp, column, colmax); fputs (":", fp); column++; make_write_vec (d->deps, fp, column, colmax); @@ -387,6 +418,59 @@ make_write (const cpp_reader *pfile, FILE *fp, unsigned int colmax) for (unsigned i = 1; i < d->deps.size (); i++) fprintf (fp, "%s:\n", munge (d->deps[i])); } + + if (!CPP_OPTION (pfile, deps.modules)) + return; + + if (d->modules.size ()) + { + column = make_write_vec (d->targets, fp, 0, colmax, d->quote_lwm); + if (d->cmi_name) + column = make_write_name (d->cmi_name, fp, column, colmax); + fputs (":", fp); + column++; + column = make_write_vec (d->modules, fp, column, colmax, 0, ".c++m"); + fputs ("\n", fp); + } + + if (d->module_name) + { + if (d->cmi_name) + { + /* module-name : cmi-name */ + column = make_write_name (d->module_name, fp, 0, colmax, + true, ".c++m"); + fputs (":", fp); + column++; + column = make_write_name (d->cmi_name, fp, column, colmax); + fputs ("\n", fp); + + column = fprintf (fp, ".PHONY:"); + column = make_write_name (d->module_name, fp, column, colmax, + true, ".c++m"); + fputs ("\n", fp); + } + + if (d->cmi_name && !d->is_header_unit) + { + /* An order-only dependency. + cmi-name :| first-target + We can probably drop this this in favour of Make-4.3's grouped + targets '&:' */ + column = make_write_name (d->cmi_name, fp, 0, colmax); + fputs (":|", fp); + column++; + column = make_write_name (d->targets[0], fp, column, colmax); + fputs ("\n", fp); + } + } + + if (d->modules.size ()) + { + column = fprintf (fp, "CXX_IMPORTS +="); + make_write_vec (d->modules, fp, column, colmax, 0, ".c++m"); + fputs ("\n", fp); + } } /* Write out dependencies according to the selected format (which is From patchwork Tue Nov 3 21:14:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393421 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=afzZcyy8; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjGh6QXvz9sRK for ; Wed, 4 Nov 2020 08:14:52 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A4332398747D; Tue, 3 Nov 2020 21:14:50 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com [IPv6:2607:f8b0:4864:20::f30]) by sourceware.org (Postfix) with ESMTPS id 8C512398747B for ; Tue, 3 Nov 2020 21:14:48 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 8C512398747B Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf30.google.com with SMTP id da2so6465831qvb.0 for ; Tue, 03 Nov 2020 13:14:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=odd+SoWPrKl9EMa8wyjjr3jyz4IK0GM2gbIfhPv8aVk=; b=afzZcyy8H4TQP7gWGEX21McWrMw/UzFVKz9gUZtKzXeR9ee7SikXjVghl0Kq2tYtYo eAFv1NQf9c6uST0dJOHcycOrY2T5BDYZHSylQDPlqYnm7i2nUc3mKQiSN+welAh1dREI xKz9183VMrvdA0R8QZR6uvjo/TPG2YkUIFjCWnOlJ8OKzsrPXFea8Wa5YqrrcgvLdSob X3XbWDGuo7a4j4ADAL1Uc83yU3X7FUu+z6B2c4wZs8P7sohKQJrcEJat0C5LkwaHVB5J 5I3vXDQCW4G303wTP74O6fa7aIV326h8yqQBdV/LO86LoN04wMJ12EqEfFP43AhFYRY/ oW1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=odd+SoWPrKl9EMa8wyjjr3jyz4IK0GM2gbIfhPv8aVk=; b=mVYC4B09ytSc7utaKDS6XhIlUzb8zA6Erc8mvKclJ/u1hAiB8w6RMudG21neE8FL9c cs4P913IaWYu5W0nFd8rFVYF0i5YnzhsZsngbmlPrlTk6r2jEUKM9XKq+nNnBCuX5e+Y apHk77GmJvW0GpVbUKgX+Yt/dU0+ISNkLuJ4SmGQ8b/SNhzDs3AdjVAmCzBa8aZqS8nU SqTtTfwS5zhTfQ2nsJCTd9QpDI/NekWz4AK6bySTFQwZkLqNGVyZDw251VX70thnmuZi jxo5lOxhwN+h4a9f0cI3JZXSMiznrhPou0uMC25z7VH5xYMIowucjq6ADYTex6LqMpvY quMw== X-Gm-Message-State: AOAM533HZqvCpuqcVoyv+AFXCHPSlCTcxnhXwDiUCOKWOR9hpemnVqik ft46WT4n9ru90cFEUJHz4BfWeS2MgQi06w== X-Google-Smtp-Source: ABdhPJx3CZR3ZfvpYBmwAZMXQfdmbZlDkXjvI1WrMkPxM3Y+lg3ILOv9b6JIIP0roSPO1B+AROkN9g== X-Received: by 2002:a05:6214:2a5:: with SMTP id m5mr8730220qvv.58.1604438088073; Tue, 03 Nov 2020 13:14:48 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id n81sm5044451qke.99.2020.11.03.13.14.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:47 -0800 (PST) From: Nathan Sidwell Subject: [09/32] core diagnostics To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> Message-ID: <1fbe900e-a07d-1aa6-32f1-a36ab5b66715@acm.org> Date: Tue, 3 Nov 2020 16:14:45 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <89819c10-e86d-9b01-5673-5223a525a135@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" The 'included from ...' chain that one gets at the start of a diagnostic needs extending to include importing. There are a few combinations to handle, but nothing particularly exciting. diff --git c/gcc/diagnostic.c w/gcc/diagnostic.c index 1b6c9845892..52bf5e0da1f 100644 --- c/gcc/diagnostic.c +++ w/gcc/diagnostic.c @@ -689,12 +705,13 @@ diagnostic_report_current_module (diagnostic_context *context, location_t where) set_last_module (context, map); if (! MAIN_FILE_P (map)) { - bool first = true; + bool first = true, need_inc = true, was_module = MAP_MODULE_P (map); expanded_location s = {}; do { where = linemap_included_from (map); map = linemap_included_from_linemap (line_table, map); + bool is_module = MAP_MODULE_P (map); s.file = LINEMAP_FILE (map); s.line = SOURCE_LINE (map, where); int col = -1; @@ -706,14 +723,24 @@ diagnostic_report_current_module (diagnostic_context *context, location_t where) const char *line_col = maybe_line_and_column (s.line, col); static const char *const msgs[] = { - N_("In file included from"), + NULL, N_(" from"), + N_("In file included from"), /* 2 */ + N_(" included from"), + N_("In module"), /* 4 */ + N_("of module"), + N_("In module imported at"), /* 6 */ + N_("imported at"), }; - unsigned index = !first; + + unsigned index = (was_module ? 6 : is_module ? 4 + : need_inc ? 2 : 0) + !first; + pp_verbatim (context->printer, "%s%s %r%s%s%R", - first ? "" : ",\n", _(msgs[index]), + first ? "" : was_module ? ", " : ",\n", + _(msgs[index]), "locus", s.file, line_col); - first = false; + first = false, need_inc = was_module, was_module = is_module; } while (! MAIN_FILE_P (map)); pp_verbatim (context->printer, ":"); From patchwork Wed Nov 4 19:24:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1394486 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=MYuKvb/h; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CRGnP6hyDz9sSn for ; Thu, 5 Nov 2020 06:24:55 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 7B7A2398795B; Wed, 4 Nov 2020 19:24:52 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) by sourceware.org (Postfix) with ESMTPS id 78FC2385E445 for ; Wed, 4 Nov 2020 19:24:47 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 78FC2385E445 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x732.google.com with SMTP id a65so18177437qkg.13 for ; Wed, 04 Nov 2020 11:24:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=mWz8kCzGW7mvfLmH/ungFslzulzBn7CloBPK0+vUbKs=; b=MYuKvb/hbVFmIL5t1UX0nNIjVf6bFO6KEjowIB7jV2//mk1SsmyRbUL9WfuDz4hh1x B5aKEirExvdk3fW01Vj1rcx1fDDrniuUw8ib2Vx51MjFKrZY4mg9MpinRAR1u5wZKeT3 cyg9n4CaCP1BGdnHwG/9jIDWoSFLo8j3TrLgpU0S2q3OaQf5fqlYWXYOP3cAxgG641b3 8OgJgSrW6fhYoYL/iwZt8ozPBkGfs80mVTQehrOaE+zVeiwzqL+j7affnZ9vVzhmnC9J LgQuiNnAwqqN28mQBoMD/kboAmT/+qQGtS6hmx06t0tbgTisDhr0EPaULYhDIcX20bbo 9kAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=mWz8kCzGW7mvfLmH/ungFslzulzBn7CloBPK0+vUbKs=; b=t+t2gf60TNBruTQ1ulm+qI4K68X068KBVi9lW+CruJHpDbQChZmyzjM1qQQCPxriwp MgQqEXhg3e07GGoCrNidl65kmlyyf7IIQtGGby2uFf+sFIMYN1AD99icp95FUuG3PSmd oZOhYVowI22rcq3k2ibmLU8ErOc0JSQ0jZERT8FpWeMpWm1ha8nWkeD4/8T2F7zjpSPd XHNGZ68qfHXN8qA4g/dUapqUhYhtDzMsdSFS8AKy2StKK3JnolhcR7GsRRVr3EBC72g7 S3mU1rqjwGe7XmKJVsPHqn/iszrbhXuPzPbbszRrIkyvSNLhse2dHTgW6JT6C9eRjLak 8IYA== X-Gm-Message-State: AOAM532dMHlF9wb20vaeZF2kHSq2PbavoMXQNSSNpHMYS9Q5VMb9LNir BzG6hrJEXi1tteIejTct2K/BFWqPpWSbMg== X-Google-Smtp-Source: ABdhPJwvLmXYa/t8wOu8dmd+xqzb2X0RetQ7tORVYSrmPoZU/DeMQ5Iz5T12PeoKz5pexjHijtDsyQ== X-Received: by 2002:a05:620a:69c:: with SMTP id f28mr28643355qkh.437.1604517886852; Wed, 04 Nov 2020 11:24:46 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:65e6:9ed3:a46f:ed07? ([2620:10d:c091:480::1:37fc]) by smtp.googlemail.com with ESMTPSA id w7sm850929qte.83.2020.11.04.11.24.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 04 Nov 2020 11:24:45 -0800 (PST) Subject: [10/32] config From: Nathan Sidwell To: GCC Patches , Jason Merrill , Richard Biener References: <94392c65-0a81-17c3-c5d3-f15a5e91dd79@acm.org> Message-ID: Date: Wed, 4 Nov 2020 14:24:43 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <94392c65-0a81-17c3-c5d3-f15a5e91dd79@acm.org> Content-Language: en-US X-Spam-Status: No, score=-10.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" I managed to flub sending this yesterday. This is the gcc/configure.ac changes (rebuild configure and config.h.in after applying). Generally just checking for network-related functionality. If it's not available, those features of the module mapper will be unavailable. nathan diff --git c/gcc/configure.ac w/gcc/configure.ac index 73034bb902b..168a3bc3625 100644 --- c/gcc/configure.ac +++ w/gcc/configure.ac @@ -1417,8 +1419,8 @@ define(gcc_UNLOCKED_FUNCS, clearerr_unlocked feof_unlocked dnl putchar_unlocked putc_unlocked) AC_CHECK_FUNCS(times clock kill getrlimit setrlimit atoq \ popen sysconf strsignal getrusage nl_langinfo \ - gettimeofday mbstowcs wcswidth mmap setlocale \ - gcc_UNLOCKED_FUNCS madvise mallinfo mallinfo2) + gettimeofday mbstowcs wcswidth mmap memrchr posix_fallocate setlocale \ + gcc_UNLOCKED_FUNCS madvise mallinfo execv mallinfo2 fstatat) if test x$ac_cv_func_mbstowcs = xyes; then AC_CACHE_CHECK(whether mbstowcs works, gcc_cv_func_mbstowcs_works, @@ -1440,6 +1442,10 @@ fi AC_CHECK_TYPE(ssize_t, int) AC_CHECK_TYPE(caddr_t, char *) +AC_CHECK_TYPE(sighander_t, + AC_DEFINE(HAVE_SIGHANDLER_T, 1, + [Define if defines sighandler_t]), + ,signal.h) GCC_AC_FUNC_MMAP_BLACKLIST @@ -1585,6 +1591,146 @@ if test $ac_cv_f_setlkw = yes; then [Define if F_SETLKW supported by fcntl.]) fi +# Check if O_CLOEXEC is defined by fcntl +AC_CACHE_CHECK(for O_CLOEXEC, ac_cv_o_cloexec, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include ]], [[ +return open ("/dev/null", O_RDONLY | O_CLOEXEC);]])], +[ac_cv_o_cloexec=yes],[ac_cv_o_cloexec=no])]) +if test $ac_cv_o_cloexec = yes; then + AC_DEFINE(HOST_HAS_O_CLOEXEC, 1, + [Define if O_CLOEXEC supported by fcntl.]) +fi + +# C++ Modules would like some networking features to provide the mapping +# server. You can still use modules without them though. +# The following network-related checks could probably do with some +# Windows and other non-linux defenses and checking. + +# Local socket connectivity wants AF_UNIX networking +# Check for AF_UNIX networking +AC_CACHE_CHECK(for AF_UNIX, ac_cv_af_unix, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include +#include +#include +#include ]],[[ +sockaddr_un un; +un.sun_family = AF_UNSPEC; +int fd = socket (AF_UNIX, SOCK_STREAM, 0); +connect (fd, (sockaddr *)&un, sizeof (un));]])], +[ac_cv_af_unix=yes], +[ac_cv_af_unix=no])]) +if test $ac_cv_af_unix = yes; then + AC_DEFINE(HAVE_AF_UNIX, 1, + [Define if AF_UNIX supported.]) +fi + +# Remote socket connectivity wants AF_INET6 networking +# Check for AF_INET6 networking +AC_CACHE_CHECK(for AF_INET6, ac_cv_af_inet6, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include +#include +#include +#include ]],[[ +sockaddr_in6 in6; +in6.sin6_family = AF_UNSPEC; +struct addrinfo *addrs = 0; +struct addrinfo hints; +hints.ai_flags = 0; +hints.ai_family = AF_INET6; +hints.ai_socktype = SOCK_STREAM; +hints.ai_protocol = 0; +hints.ai_canonname = 0; +hints.ai_addr = 0; +hints.ai_next = 0; +int e = getaddrinfo ("localhost", 0, &hints, &addrs); +const char *str = gai_strerror (e); +freeaddrinfo (addrs); +int fd = socket (AF_INET6, SOCK_STREAM, 0); +connect (fd, (sockaddr *)&in6, sizeof (in6));]])], +[ac_cv_af_inet6=yes], +[ac_cv_af_inet6=no])]) +if test $ac_cv_af_inet6 = yes; then + AC_DEFINE(HAVE_AF_INET6, 1, + [Define if AF_INET6 supported.]) +fi + +# Efficient server response wants epoll +# Check for epoll_create, epoll_ctl, epoll_pwait +AC_CACHE_CHECK(for epoll, ac_cv_epoll, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include ]],[[ +int fd = epoll_create (1); +epoll_event ev; +ev.events = EPOLLIN; +ev.data.fd = 0; +epoll_ctl (fd, EPOLL_CTL_ADD, 0, &ev); +epoll_pwait (fd, 0, 0, -1, 0);]])], +[ac_cv_epoll=yes], +[ac_cv_epoll=no])]) +if test $ac_cv_epoll = yes; then + AC_DEFINE(HAVE_EPOLL, 1, + [Define if epoll_create, epoll_ctl, epoll_pwait provided.]) +fi + +# If we can't use epoll, try pselect. +# Check for pselect +AC_CACHE_CHECK(for pselect, ac_cv_pselect, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include ]],[[ +pselect (0, 0, 0, 0, 0, 0);]])], +[ac_cv_pselect=yes], +[ac_cv_pselect=no])]) +if test $ac_cv_pselect = yes; then + AC_DEFINE(HAVE_PSELECT, 1, + [Define if pselect provided.]) +fi + +# And failing that, use good old select. +# If we can't even use this, the server is serialized. +# Check for select +AC_CACHE_CHECK(for select, ac_cv_select, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include ]],[[ +select (0, 0, 0, 0, 0);]])], +[ac_cv_select=yes], +[ac_cv_select=no])]) +if test $ac_cv_select = yes; then + AC_DEFINE(HAVE_SELECT, 1, + [Define if select provided.]) +fi + +# Avoid some fnctl calls by using accept4, when available. +# Check for accept4 +AC_CACHE_CHECK(for accept4, ac_cv_accept4, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include ]],[[ +int err = accept4 (1, 0, 0, SOCK_NONBLOCK);]])], +[ac_cv_accept4=yes], +[ac_cv_accept4=no])]) +if test $ac_cv_accept4 = yes; then + AC_DEFINE(HAVE_ACCEPT4, 1, + [Define if accept4 provided.]) +fi + +# For better server messages, look for a way to stringize network addresses +# Check for inet_ntop +AC_CACHE_CHECK(for inet_ntop, ac_cv_inet_ntop, [ +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ +#include +#include ]],[[ +sockaddr_in6 in6; +char buf[INET6_ADDRSTRLEN]; +const char *str = inet_ntop (AF_INET6, &in6, buf, sizeof (buf));]])], +[ac_cv_inet_ntop=yes], +[ac_cv_inet_ntop=no])]) +if test $ac_cv_inet_ntop = yes; then + AC_DEFINE(HAVE_INET_NTOP, 1, + [Define if inet_ntop provided.]) +fi + # Restore CFLAGS, CXXFLAGS from before the gcc_AC_NEED_DECLARATIONS tests. CFLAGS="$saved_CFLAGS" CXXFLAGS="$saved_CXXFLAGS" From patchwork Tue Nov 3 21:14:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393422 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=II/gF2w1; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjGw1qBVz9sWD for ; Wed, 4 Nov 2020 08:15:04 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 20F32398747B; Tue, 3 Nov 2020 21:15:02 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by sourceware.org (Postfix) with ESMTPS id 30FEE3987477 for ; Tue, 3 Nov 2020 21:15:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 30FEE3987477 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x735.google.com with SMTP id a64so13239250qkc.5 for ; Tue, 03 Nov 2020 13:15:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=kj3e+NOumaieSgEuKVjF5Wruv3aA/xVA0p06vfQWxus=; b=II/gF2w1SIDLT+AFBOaZBpwAvXyL0cYmElH+/fOm87yxxunnpbjjU1+3Ns4z0AjlSR BKC4o5oiOezN41qE3g2qCsmAg6b83yGiBHqmSf1CwVw7+YyggzIVdV8GeA0VLQKw5B3s XhVZAGuAoUEKNpf39lZenP2d546H+krHCmHBLLkJBsrv8YlzN6y9vQRqNYCkONeSP2HM iX/4c74R08msLxNxJrSyqNi+GwDIjAQ9NSVXHB1d4//PM0MdJEYmuuTwa6VA75sVMVbj IyaDcR1Aa4lnMLCpQ9snmE5sdMloEFTPKii3sQ4E3FQyLOZUjrsj3pQ83BxrZ9EONK1L yCew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=kj3e+NOumaieSgEuKVjF5Wruv3aA/xVA0p06vfQWxus=; b=J/sXG4Ew0B04fvu+WpFlXlFRSQCRyOojj+dQPuDLgGdESDQoFnv/cU4DKEERikYMR4 e4x2gCAQmj9IMZfxxM0JDES/PpYN5FW49YtmHwZmVANQnEMijO7US4DqAhRqfqjXIKOA GqHqe8gc1mjVK7Gje3jiJseBs9K2Yff77VgHJLFZBAieOnyXB8Y9xMJLcvp/cWilH96Y /XihghkirNooSPtsNA5zzFGT6Pqb83slehxDUOW+juQH5WZaYe3lfDy30ZVGgc76WQ6T DaxZyMIsCSUZzKQnES7zNlZPBlA0Da6ipQ7ubVoHza6oXzyt4L/gz1ivZdj7QhKyBVcu V8eQ== X-Gm-Message-State: AOAM533llUdemfZMJyYgJNaaFfdaw+lkq7+uCybtI6gxL5i1q+mgraaZ xR7xV1LSkK5sasuwCikycCw= X-Google-Smtp-Source: ABdhPJwhOUgyjgTfvQA/B4Li+IiKgPHI0dDKsNZy+NymhokGr67vdN89Rp1y/iodYABw95aKj0yjcg== X-Received: by 2002:a05:620a:8c5:: with SMTP id z5mr21202323qkz.193.1604438099447; Tue, 03 Nov 2020 13:14:59 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 19sm10931146qkj.69.2020.11.03.13.14.58 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:14:58 -0800 (PST) From: Nathan Sidwell Subject: [11/32] instrumentation To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> Message-ID: <9a17881e-65d2-a70a-146e-1f4bb200ff82@acm.org> Date: Tue, 3 Nov 2020 16:14:56 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" I add one new parameter -- the number of concurrently open module files, and 3 instrumentation timers. diff --git c/gcc/params.opt w/gcc/params.opt index 7bac39a9d58..0366584f94e 100644 --- c/gcc/params.opt +++ w/gcc/params.opt @@ -349,6 +349,10 @@ Maximal stack frame growth due to inlining (in percent). Common Joined UInteger Var(param_large_unit_insns) Optimization Init(10000) Param The size of translation unit to be considered large. +-param=lazy-modules= +C++ Joined UInteger Var(param_lazy_modules) Init(32768) Param +Maximum number of concurrently open C++ module files when lazy loading. + -param=lim-expensive= Common Joined UInteger Var(param_lim_expensive) Init(20) Param Optimization The minimum cost of an expensive expression in the loop invariant motion. diff --git c/gcc/timevar.def w/gcc/timevar.def index 08c21c04009..f2bd58f0f58 100644 --- c/gcc/timevar.def +++ w/gcc/timevar.def @@ -145,6 +145,9 @@ DEFTIMEVAR (TV_CONSTEXPR , "constant expression evaluation") DEFTIMEVAR (TV_CONSTRAINT_NORM , "constraint normalization") DEFTIMEVAR (TV_CONSTRAINT_SAT , "constraint satisfaction") DEFTIMEVAR (TV_CONSTRAINT_SUB , "constraint subsumption") +DEFTIMEVAR (TV_MODULE_IMPORT , "module import") +DEFTIMEVAR (TV_MODULE_EXPORT , "module export") +DEFTIMEVAR (TV_MODULE_MAPPER , "module mapper") DEFTIMEVAR (TV_FLATTEN_INLINING , "flatten inlining") DEFTIMEVAR (TV_EARLY_INLINING , "early inlining heuristics") DEFTIMEVAR (TV_INLINE_PARAMETERS , "inline parameters") From patchwork Tue Nov 3 21:15:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393423 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=QSQFLxCg; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjHF3CfBz9sVZ for ; Wed, 4 Nov 2020 08:15:21 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 28BBA3987465; Tue, 3 Nov 2020 21:15:19 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by sourceware.org (Postfix) with ESMTPS id 21BE73987427 for ; Tue, 3 Nov 2020 21:15:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 21BE73987427 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x733.google.com with SMTP id 12so12721277qkl.8 for ; Tue, 03 Nov 2020 13:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=/y21kjGmEzTQP82qCctncXxwgLbzhlwr7DEtZlCCI6I=; b=QSQFLxCgaVIqV83iVLnKgB/xtBPg/jh+X0qjOFRwdo5VmTcV/g8ihf8Dn39O9RBfvV gJRHiUjOICtMQI1Ledm9dtSSz34BJsWWBVvRYmWJUMN7AD2w7hzapHOls8WgnzODeBUU OOD4uzoP3suPfsVJ2FjcOm+97A3QhSV0F1gQwtHCKaSr39u51sL/83YOQH8DJ51zYwkh sCZFYXzWGfS3+/RSqJAvqr7C5aDPX2fExQz2fmxMN2o5KASwoFC4PRRRnRaMI71VdBwJ eUfD2w0WjwloKtlFiQfI8spDbGINO3uVMCC7DvCSjJfV2y6jVzCizaU0Pf6/aWjnweQU bwzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=/y21kjGmEzTQP82qCctncXxwgLbzhlwr7DEtZlCCI6I=; b=s6MkRsE3PNbGhM4WvMsN+yJUC3lEWMANZJlLXDTi2nLU2ZbzFstQ4AHxcexcOH4lMr RqaD1QklBwngilPBBs6on+4LL+6A1b3O/PHJI6J2c8xq2NbSbjBnhn9x2DJK+GncID2r fPLC4l81Dr3T+QKQ/kvDhbG1DmPchMdGOF+gkB6DBtCqXl1Rjv69NbSdhqzLKLobwLH+ jfp0P1IeXbn/f26UTcxtU3CJHLfqRNrcn2e3d/uR6QS19FMZD04/9MG8KZ8Id+fElc1u /P/12+ZyoCNnTx04Zn+FWfLKdmlM4j56DdDXvU81ruYSD21SMH1KcSNnPJrI6kwWg8rH ocpQ== X-Gm-Message-State: AOAM532HnHniS0ZushsgnQOpXpT0QFytno+pnM4X117dyY3JVntR1J79 dqc1Xo8AxVMfBwGjw4JPOsY= X-Google-Smtp-Source: ABdhPJydTFL/98+des246vj8gr/3W9AsOgs/5i6iMag3TIW/RotCSc6l5ZHwCU7ea6RmSj8a3E1jDw== X-Received: by 2002:a05:620a:697:: with SMTP id f23mr20416961qkh.374.1604438114461; Tue, 03 Nov 2020 13:15:14 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id b12sm11025536qkk.71.2020.11.03.13.15.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:13 -0800 (PST) From: Nathan Sidwell Subject: [12/32] user documentation To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> Message-ID: <8e4293f5-24ce-e572-7445-97b82a756b6e@acm.org> Date: Tue, 3 Nov 2020 16:15:11 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" This is the user documentation. diff --git c/gcc/doc/invoke.texi w/gcc/doc/invoke.texi index 492b7dcdf10..3fdee8bbd3b 100644 --- c/gcc/doc/invoke.texi +++ w/gcc/doc/invoke.texi @@ -97,7 +97,7 @@ The usual way to run GCC is to run the executable called @command{gcc}, or When you compile C++ programs, you should invoke GCC as @command{g++} instead. @xref{Invoking G++,,Compiling C++ Programs}, for information about the differences in behavior between @command{gcc} -and @code{g++} when compiling C++ programs. +and @command{g++} when compiling C++ programs. @cindex grouping options @cindex options, grouping @@ -172,6 +172,7 @@ listing and explanation of the binary and decimal byte size prefixes. * Spec Files:: How to pass switches to sub-processes. * Environment Variables:: Env vars that affect GCC. * Precompiled Headers:: Compiling a header once, and using it many times. +* C++ Modules:: Experimental C++20 module system. @end menu @c man begin OPTIONS @@ -595,7 +596,7 @@ Objective-C and Objective-C++ Dialects}. -fpreprocessed -ftabstop=@var{width} -ftrack-macro-expansion @gol -fwide-exec-charset=@var{charset} -fworking-directory @gol -H -imacros @var{file} -include @var{file} @gol --M -MD -MF -MG -MM -MMD -MP -MQ -MT @gol +-M -MD -MF -MG -MM -MMD -MP -Mno-modules -MQ -MT @gol -no-integrated-cpp -P -pthread -remap @gol -traditional -traditional-cpp -trigraphs @gol -U@var{macro} -undef @gol @@ -1567,7 +1568,7 @@ name suffix). This option applies to all following input files until the next @option{-x} option. Possible values for @var{language} are: @smallexample c c-header cpp-output -c++ c++-header c++-cpp-output +c++ c++-header c++-system-header c++-user-header c++-cpp-output objective-c objective-c-header objective-c-cpp-output objective-c++ objective-c++-header objective-c++-cpp-output assembler assembler-with-cpp @@ -3037,6 +3038,63 @@ To save space, do not emit out-of-line copies of inline functions controlled by @code{#pragma implementation}. This causes linker errors if these functions are not inlined everywhere they are called. +@item -fmodules-ts +@itemx -fno-modules-ts +@opindex fmodules-ts +@opindex fno-modules-ts +Enable support for C++ 20 modules. The @option{-fno-modules-ts} is +usually not needed, as that is the default. Even though this is a +C++20 feature, it is not currently implicitly enabled by selecting +that standard version. + +@item -fno-module-lazy +@opindex fno-module-lazy +@opindex fmodule-lazy +Disable lazy module importing and module mapper creation. + +@item -fmodule-header +@itemx -fmodule-header=user +@itemx -fmodule-header=system +@opindex fmodule-header +Compile as a header unit. + +@item -fmodule-implicit-inline +@opindex fmodule-implicit-inline +Memmber functions defined in their class definitions are not +implicitly inline for modular code. This is different to traditional +C++ behaviour, for good reasons. However, it may result in a +difficulty during code porting. This option will make such function +definitions implicitly inline. It does however generate an ABI +incompatibility, so you must use it everywhere or nowhere. (Such +definitions outside of a named module remain implicitly inline, +regardless.) + +@item -fmodule-only +@opindex fmodule-only +Only emit the module CMI, inhibiting any object file. + +@item -flang-info-include-translate +@itemx -flang-info-include-translate=@var{header} +@opindex flang-info-include-translate +Note include translation events. + +@item -Winvalid-imported-macros +@opindex Winvalid-imported-macros +@opindex Wno-invalid-imported-macros +Verify all imported macro definitions are valid at end of +compilation. This is not enabled by default, as it requires +additional processing to determine. + +@item -fmodule-mapper=@r{[}@var{hostname}@r{]}:@var{port}@r{[}?@var{ident}@r{]} +@itemx -fmodule-mapper=|@var{program}@r{[}?@var{ident}@r{]} @var{args...} +@itemx -fmodule-mapper==@var{socket}@r{[}?@var{ident}@r{]} +@itemx -fmodule-mapper=@var{file}@r{[}?@var{ident}@r{]} +@vindex CXX_MODULE_MAPPER @r{environment variable} +@opindex fmodule-mapper +An oracle to query for module name to filename mappings. If +unspecified the @env{CXX_MODULE_MAPPER} environment variable is used, +and if that is unset, a default is provided. + @item -fms-extensions @opindex fms-extensions Disable Wpedantic warnings about constructs used in MFC, such as implicit @@ -14215,7 +14273,7 @@ Note that it is quite common that execution counts of some part of programs depends, for example, on length of temporary file names or memory space randomization (that may affect hash-table collision rate). Such non-reproducible part of programs may be annotated by -@code{no_instrument_function} function attribute. @code{gcov-dump} with +@code{no_instrument_function} function attribute. @command{gcov-dump} with @option{-l} can be used to dump gathered data and verify that they are indeed reproducible. @@ -16585,6 +16643,11 @@ By default, the dump will contain messages about successful optimizations (equivalent to @option{-optimized}) together with low-level details about the analysis. +@item -fdump-lang +@opindex fdump-lang +Dump language-specific information. The file name is made by appending +@file{.lang} to the source file name. + @item -fdump-lang-all @itemx -fdump-lang-@var{switch} @itemx -fdump-lang-@var{switch}-@var{options} @@ -16605,6 +16668,14 @@ Enable all language-specific dumps. Dump class hierarchy information. Virtual table information is emitted unless '@option{slim}' is specified. This option is applicable to C++ only. +@item module +Dump module information. Options @option{lineno} (locations), +@option{graph} (reachability), @option{blocks} (clusters), +@option{uid} (serialization), @option{alias} (mergeable), +@option{asmname} (Elrond), @option{eh} (mapper) & @option{vops} +(macros) may provide additional information. This option is +applicable to C++ only. + @item raw Dump the raw internal tree data. This option is applicable to C++ only. @@ -32246,3 +32317,225 @@ precompiled header, the actual behavior is a mixture of the behavior for the options. For instance, if you use @option{-g} to generate the precompiled header but not when using it, you may or may not get debugging information for routines in the precompiled header. + +@node C++ Modules +@section C++ Modules +@cindex speed of compilation + +Modules are a C++ 20 language feature. As the name suggests, it +provides a modular compilation system, intending to provide both +faster builds and better library isolation. The ``Merging Modules'' +paper @uref{http//wg21.link/p1103}, provides the easiest to read set +of changes to the standard, although it does not capture later +changes. That specification is now in the C++ Working Draft, +@uref{git@@github.com:cplusplus/draft.git}, as of 2020-03-14, it is +considered complete (there may be defect reports to come). +@emph{Modules support is a work in progress. The implementation is +not complete. Command-line options are not stable.} + +Modular compilation is @emph{not} enabled with just the +@option{-std=c++20} option. You must explicitly enable it with the +@option{-fmodules-ts} option. It is independent of the language +version selected. + +No new source file suffixes are required or supported. If you wish to +use a non-standard suffix (@xref{Overall Options}), you will also need +to provide a @option{-x c++} option too.@footnote{Some users like to +distinguish module interface files with a new suffix, such as naming +the source @code{module.cppm}, which involves +teaching all tools about the new suffix. A different scheme, such as +naming @code{module-m.cpp} would be less invasive.} + +Compiling a module interface unit produces an additional output (to +the assembly or object file), called a Compiled Module Interface +(CMI). This encodes the exported declarations of the module. +Importing a module reads in the CMI. The import graph is a Directed +Acyclic Graph (DAG). You must build imports before the importer. + +Header files may themselves be compiled to header units, which are a +transitional ability aiming at faster compilation. The +@option{-fmodule-header} option is used to enable this, and implies +the @option{-fmodules-ts} option. These CMIs are named by the fully +resolved underlying header file, and thus may be a complete pathname +containing subdirectories. If the header file is found at an absolute +pathname, the CMI location is still relative to a CMI root directory. + +As header files often have no suffix, you commonly have to specify a +@option{-x} option to tell the compiler the source is a header file. +You may use @option{-x c++-header}, @option{-x c++-user-header} or +@option{-x c++-system-header}. When used in conjunction with +@option{-fmodules-ts}, these all imply an appropriate +@option{-fmodule-header} option. The latter two variants will use the +user or system include path to search for the file specified. This +allows you to, for instance, compile standard library header files as +header units, without needing to know exactly where they are +installed. Specifying the language as one of these variants also +inhibits output of the object file, as header files have no associated +object file. + +When creating an output CMI any missing directory components are +created in a manner that is safe for concurrent builds creating +multiple, different, CMIs within a common subdirectory tree. + +CMI contents are written to a temporary file, which is then atomically +renamed. Observers will either see old contents (if there is an +existing file), or complete new contents. They will not observe the +CMI during its creation. This is unlike object file writing, which +may be observed by an external process. + +CMIs are read in lazily. Generally blocks are read when name lookup +or template instantiation occurs. To inhibit this, the +@option{-fno-module-lazy} option may be used. + +The @option{-fmodule-only} option disables generation of the +associated object file for compiling a module interface. Only the CMI +is generated. This option is implied when using the +@option{-fmodule-header} option. + +The @option{--param lazy-modules=@var{n}} parameter controls the limit +on the number of concurrently open module files during lazy loading. +Should more modules be imported, an LRU algorithm is used to determine +which files to close -- until that file is needed again. This limit +may be exceeded with deep module dependency hierarchies. With large +code bases there may be more imports than the process limit of file +descriptors. By default, the limit is a few less than the per-process +file descriptor hard limit, if that is determinable.@footnote{Where +applicable the soft limit is incremented as needed towards the hard limit.} + +The @option{-flang-info-include-translate} options note whether +include translation occurs. With no argument, all include translation +is noted. Otherwise, queries about include translation of a specific +header file is noted. The latter form may be repeated. + +The @option{-Winvalid-imported-macros} option causes all imported macros +to be resolved at the end of compilation. Without this, imported +macros are only resolved when expanded or (re)defined. This option +will detect conflicting import definitions for all macros. + +The @option{-fmodule-mapper} family of options are described below. + +@menu +* C++ Module Mapper:: Module Mapper +* C++ Module Preprocessing:: Module Preprocessing +@end menu + +@node C++ Module Mapper +@subsection Module Mapper +@cindex C++ Module Mapper + +A module mapper provides a server or file that the compiler queries to +determine the mapping between module names and CMI files. It is also +used to build CMIs on demand. A mapper may be specified with the +@option{-fmodule-mapper=@var{val}} option or @env{CXX_MODULE_MAPPER} +environment variable. The value may have one of the following forms: + +@table @gcctabopt + +@item @r{[}@var{hostname}@r{]}:@var{port}@r{[}?@var{ident}@r{]} +An optional hostname and a numeric port number to connect to. If the +hostname is omitted, the loopback address is used. If the hostname +corresponds to multiple IPV6 addresses, these are tried in turn, until +one is successful. If your host lacks ipv6, this form is +non-functional. If you must use ipv4 @emph{get with the 21st century}, +or failing that use @option{-fmodule-mapper='|ncat @var{ipv4host} +@var{port}'}. + +@item =@var{socket}@r{[}?@var{ident}@r{]} +A local domain socket. If your host lacks local domain sockets, this +form is non-functional. + +@item |@var{program}@r{[}?@var{ident}@r{]} @r{[}@var{args...}@r{]} +A program to spawn, and communicate with on its stdin/stdout streams. +Your @var{PATH} environment variable is searched for the program. +Arguments are separated by space characters, (it is not possible for +one of the arguments delivered to the program to contain a space). + +@item <>@r{[}?@var{ident}@r{]} +@item <>@var{fdinout}@r{[}?@var{ident}@r{]} +@item <@var{fdin}>@var{fdout}@r{[}?@var{ident}@r{]} +File descriptors to communicate over. The first form, @option{<>}, +communicates over stdin and stdout. The second form specifies a +bidirectional file descriptor and the last form allows specifying +two independent descriptors. Note that other compiler options might +cause the compiler to read stdin or write stdout. + +@item @var{file}@r{[}?@var{ident}@r{]} +A mapping file consisting of space-separated module-name, filename +pairs, one per line. Only the mappings for the direct imports and any +module export name need be provided. If other mappings are provided, +they override those stored in any imported CMI files. A repository +root may be specified in the mapping file by using @samp{$root} as the +module name in the first active line. + +@end table + +As shown, an optional @var{ident} may suffix the first word of the +option, indicated by a @samp{?} prefix. The value is used in the +initial handshake with the module server, or to specify a prefix on +mapping file lines. In the server case, the main source file name is +used if no @var{ident} is specified. In the file case, all non-blank +lines are significant, unless a value is specified, in which case only +lines beginning with @var{ident} are significant. The @var{ident} +must be separated by whitespace from the module name. Be aware that +@samp{<}, @samp{>}, @samp{?} and @samp{|} characters are often +significant to the shell, and therefore may need quoting. + +The mapper is connected to or loaded lazily, when the first module +mapping is required. The networking protocols are only supported on +hosts that provide networking. If no mapper is specified a default is +provided. + +A project-specific mapper is expected to be provided by the build +system that invokes the compiler. It is not expected that a +general-purpose server is provided for all compilations. As such, the +server will know the build configuration, the compiler it invoked, and +the environment (such as working directory) in which that is +operating. As it may parallelize builds, several compilations may +connect to the same socket. + +The default mapper generates CMI files in a @samp{gcm.cache} +directory. CMI files have a @samp{.gcm} suffix. The module unit name +is used directly to provide the basename. Header units construct a +relative path using the underlying header file name. If the path is +already relative, a @samp{,} directory is prepended. Internal +@samp{..} components are translated to @samp{,,}. No attempt is made +to canonicalize these filenames beyond that done by the preprocessor's +include search algorithm, as in general it is ambiguous when symbolic +links are present. + +The mapper protocol was published as ``A Module Mapper'' +@uref{https://wg21.link/p1184}. The implementation is provided by +@command{libcody}, @uref{https://www.github.com/urnathan/libcody}, +which specifies the canonical protocol definition. A proof of concept +server implementation embedded in @command{make} was described in +''Make Me A Module'', @uref{https://wg21.link/p1602}. + +@node C++ Module Preprocessing +@subsection Module Preprocessing +@cindex C++ Module Preprocessing + +Modules affect preprocessing because of header units and include +translation. Some uses of the preprocessor as a separate step will +either not produce a correct output, or require CMIs to be available. + +Header units import macros. These macros can affect later conditional +inclusion, which therefore can cascade to differing import sets. When +preprocessing, it is necessary to load the CMI. If a header unit is +unavailable, the preprocessor will issue a warning and continue (when +no just preprocessing, an error is emitted). Detecting such imports +requires preprocessor tokenization of the input stream to phase 4 +(macro expansion). + +Include translation converts @code{#include}, @code{include_next} and +@code{import} directives to internal @code{import} declarations. +Whether a particular directive is translated is controlled by the +module mapper. Header unit names are canonicalized during +preprocessing. + +Dependency information can be emitted for macro import, extending the +functionality of @option{-MD} and @option{-MMD} options. Detection of +import declarations also requires phase 4 preprocessing, and thus +requires full preprocessing (or compilation). + +The @option{-M}, @option{-MM} and @option{-fdirectives-only} options halt +preprocessing before phase 4. diff --git c/gcc/doc/cppopts.texi w/gcc/doc/cppopts.texi index 7f1849d841f..0df8f4cb446 100644 --- c/gcc/doc/cppopts.texi +++ w/gcc/doc/cppopts.texi @@ -139,6 +139,11 @@ this useless. This feature is used in automatic updating of makefiles. +@item -Mno-modules +@opindex Mno-modules +Disable dependency generation for compiled module interfaces. Some +dependency parsers are not fully make-compatible. + @item -MP @opindex MP This option instructs CPP to add a phony target for each dependency From patchwork Tue Nov 3 21:15:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393424 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=VPO4qd/Y; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjHP4pf9z9sXk for ; Wed, 4 Nov 2020 08:15:28 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 8EE5F3987495; Tue, 3 Nov 2020 21:15:26 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by sourceware.org (Postfix) with ESMTPS id AECB93987427 for ; Tue, 3 Nov 2020 21:15:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org AECB93987427 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72f.google.com with SMTP id b18so16604005qkc.9 for ; Tue, 03 Nov 2020 13:15:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=PutGAIITTAYprHwjVE1AhzN37XdPor/dcHmrEB46pEM=; b=VPO4qd/YTmsDyN6Dig+qJJPKGPi5Id5CIf9/pIiTOr2Cc+r3mPBEo6OLMq+E5MvQSi rWGtmw68Flhrbw7voJ9p0fL8/TJ1Tw8y7n1i+fGmNysqbVS5JH02veGm1XqIWHryID05 U2NvJm4IAy3WE7RmQQA+dq+MMLXEBsb9K5iWpJchF6+lVUtt/86PBBB7gy2kSXDkJ537 0ARBbNJS+dSV/wBk0PpiW+V/2SJ0IC9UXe8b8yVhZOWK2mII8PNhEAjyhBX+gh6cMQ38 FmixjG6OVlZEMQg0ctOVdt8HW7D9rSqwAyFQaRiUMBwAvgRzbtuhze1/7Z12Vhf8LO2w Nhcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=PutGAIITTAYprHwjVE1AhzN37XdPor/dcHmrEB46pEM=; b=Avp/Zo41oD8jNeMGGc2c4auHlO0VfQA3qgYshHQwsDXth+MZEzswvKprT/1veAlXHt T6909hK/RKgdiKQVTHGFb66j9gorNhVAUz2JRVmId2ADudKNnkncpRigsd4eFueGRGhq 8B1m0dJuS+/15zzATz5DYkOdmxRarPuNE9KbranrEURRgUJpb72dYyz/mY5UxZ4YByBO VP4icapsSuxXWU8jAknErMBfZVsrzD4fa3q+JEOVNk1HKy04RC620OMa/kx1n4D/eMXC nGsDa3Qj6rgxMivEqIKN1y14iE9hnvfUx8IpiLhYPhfjGW62w2xXyqCFJYZJvLL7AvPl N4Dg== X-Gm-Message-State: AOAM531kLF7Woo0dBBmHBzdhAM4EFJRLAiqJbP58NsHIMgp++34VaANz 5mp7jojieoUQ99B/9duYP9I= X-Google-Smtp-Source: ABdhPJylIm/KnklNZJfA0oXLF+feq4ivYSCOmIP5JGKjQgQSr/04T6hruCxz/YTQJXPvP/zTFusV/w== X-Received: by 2002:ae9:e807:: with SMTP id a7mr20469073qkg.53.1604438123966; Tue, 03 Nov 2020 13:15:23 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id j16sm11788125qkg.26.2020.11.03.13.15.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:23 -0800 (PST) From: Nathan Sidwell Subject: [13/32] new options To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> Message-ID: <6e4bc3b9-4263-b114-b118-fcb5799976fb@acm.org> Date: Tue, 3 Nov 2020 16:15:21 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Here are the new options, along with the C++ lang-spec changes. Modules is enabled by -fmodules-ts, it is not implicitly enabled by -std=c++20. Usually that's the only option you need to add for a module-aware build. to build a header unit you can either add -fmodule-header to a c++ build, or you can set the language to be c++-header and add -fmodules-ts: g++ -x c++-header -fmodules-ts my-header-file to search the user or system include paths select c++-user-header of c++-system-header as the language. enabling -fmodules-ts will disable PCH, they do not play well together. There is a potential issue down the road when we implicitly enable modules. At that point building a header-unit could become indistinguishable from building a PCH. Perhaps we should consider phasing in an explicit PCH option? diff --git c/gcc/c-family/c-opts.c w/gcc/c-family/c-opts.c index 120f4489f6c..c8f08d9e014 100644 --- c/gcc/c-family/c-opts.c +++ w/gcc/c-family/c-opts.c @@ -234,6 +234,7 @@ c_common_init_options (unsigned int decoded_options_count, cpp_opts = cpp_get_options (parse_in); cpp_opts->dollars_in_ident = DOLLARS_IN_IDENTIFIERS; cpp_opts->objc = c_dialect_objc (); + cpp_opts->deps.modules = true; /* Reset to avoid warnings on internal definitions. We set it just before passing on command-line options to cpplib. */ @@ -367,6 +368,18 @@ c_common_handle_option (size_t scode, const char *arg, HOST_WIDE_INT value, cpp_opts->deps.phony_targets = true; break; + case OPT_Mmodules: + /* Do not set deps_seen, so the user can unconditionally turn + this on or off. */ + cpp_opts->deps.modules = true; + break; + + case OPT_Mno_modules: + /* Do not set deps_seen, so the user can unconditionally turn + this on or off. */ + cpp_opts->deps.modules = false; + break; + case OPT_MQ: case OPT_MT: deps_seen = true; diff --git c/gcc/c-family/c-pch.c w/gcc/c-family/c-pch.c index a2292f46a7d..f55a1fde3ad 100644 --- c/gcc/c-family/c-pch.c +++ w/gcc/c-family/c-pch.c @@ -206,6 +206,10 @@ c_common_valid_pch (cpp_reader *pfile, const char *name, int fd) /* Perform a quick test of whether this is a valid precompiled header for the current language. */ + /* C++ modules and PCH don't play together. */ + if (flag_modules) + return 2; + sizeread = read (fd, ident, IDENT_LENGTH + 16); if (sizeread == -1) fatal_error (input_location, "cannot read %s: %m", name); diff --git c/gcc/c-family/c.opt w/gcc/c-family/c.opt index 10e53ea67c9..1edc7f2b2bd 100644 --- c/gcc/c-family/c.opt +++ w/gcc/c-family/c.opt @@ -236,6 +236,14 @@ MMD C ObjC C++ ObjC++ NoDriverArg Separate MissingArgError(missing filename after %qs) Like -MD but ignore system header files. +Mmodules +C++ +Generate C++ Module dependency information. + +Mno-modules +C++ +; undocumented + MP C ObjC C++ ObjC++ Generate phony targets for all headers. @@ -1666,6 +1674,53 @@ flax-vector-conversions C ObjC C++ ObjC++ Var(flag_lax_vector_conversions) Allow implicit conversions between vectors with differing numbers of subparts and/or differing element types. +fmodules-ts +C++ ObjC++ Var(flag_modules) Integer Init(0) +Enable C++ modules-ts (experimental). + +fno-modules +C++ ObjC++ Undocumented RejectNegative Var(flag_modules,0) Integer +;; undocumented + +fmodule-header +C++ ObjC RejectNegative Var(flag_header_unit,0) Integer +Enable C++ header module (experimental). + +fmodule-header= +C++ ObjC++ Joined RejectNegative Undocumented + +fmodule-implicit-inline +C++ ObjC++ Var(flag_module_implicit_inline,0) Integer +Member functions defined within their class are inline in module purview. + +fmodule-only +C++ ObjC RejectNegative Var(flag_module_only) Integer +Only emit Binary Module Interface. + +fmodule-mapper= +C++ ObjC++ Joined RejectNegative MissingArgError(missing mapper) +Mapper for module to CMI files. + +fmodule-lazy +C++ ObjC++ Var(flag_module_lazy) Init(1) +Enable lazy module importing. + +fmodule-version-ignore +C++ ObjC Var(flag_module_version_ignore) Integer +; undocumented, Very dangerous, but occasionally useful + +Winvalid-imported-macros +C++ ObjC++ Var(warn_imported_macros) +Warn about macros that have conflicting header units definitions. + +flang-info-include-translate +C++ Var(note_include_translate) +Note #include directives translated to import declarations. + +flang-info-include-translate= +C++ Joined RejectNegative MissingArgError(missing header name) +Note a #include translation of a specific header. + fmax-include-depth= C ObjC C++ ObjC++ Joined RejectNegative UInteger fmax-include-depth= Set the maximum depth of the nested #include. diff --git c/gcc/cp/lang-specs.h w/gcc/cp/lang-specs.h index 0ad4a33b93e..1388aaed198 100644 --- c/gcc/cp/lang-specs.h +++ w/gcc/cp/lang-specs.h @@ -40,17 +40,57 @@ along with GCC; see the file COPYING3. If not see {".tcc", "@c++-header", 0, 0, 0}, {".hh", "@c++-header", 0, 0, 0}, {"@c++-header", - "%{E|M|MM:cc1plus -E %(cpp_options) %2 %(cpp_debug_options)}" + "%{E|M|MM:cc1plus -E %{fmodules-ts:-fdirectives-only -fmodule-header}" + " %(cpp_options) %2 %(cpp_debug_options)}" + "%{!E:%{!M:%{!MM:" + " %{save-temps*|no-integrated-cpp:cc1plus -E" + " %{fmodules-ts:-fdirectives-only -fmodule-header}" + " %(cpp_options) %2 -o %{save-temps*:%b.ii} %{!save-temps*:%g.ii} \n}" + " cc1plus %{save-temps*|no-integrated-cpp:-fpreprocessed" + " %{fmodules-ts:-fdirectives-only}" + " %{save-temps*:%b.ii} %{!save-temps*:%g.ii}}" + " %{!save-temps*:%{!no-integrated-cpp:%(cpp_unique_options)}}" + " %{fmodules-ts:-fmodule-header %{fpreprocessed:-fdirectives-only}}" + " %(cc1_options) %2" + " %{!S:-o %g.s%V}" + " %{!fsyntax-only:%{!fmodule-*:%{!fmodules-*:%{!fdump-ada-spec*:" + " %{!o*:--output-pch=%i.gch}%W{o*:--output-pch=%*}}}}}}}}", + CPLUSPLUS_CPP_SPEC, 0, 0}, + {"@c++-system-header", + "%{E|M|MM:cc1plus -E" + " %{fmodules-ts:-fdirectives-only -fmodule-header=system}" + " %(cpp_options) %2 %(cpp_debug_options)}" "%{!E:%{!M:%{!MM:" " %{save-temps*|no-integrated-cpp:cc1plus -E" + " %{fmodules-ts:-fdirectives-only -fmodule-header=system}" " %(cpp_options) %2 -o %{save-temps*:%b.ii} %{!save-temps*:%g.ii} \n}" " cc1plus %{save-temps*|no-integrated-cpp:-fpreprocessed" + " %{fmodules-ts:-fdirectives-only}" " %{save-temps*:%b.ii} %{!save-temps*:%g.ii}}" " %{!save-temps*:%{!no-integrated-cpp:%(cpp_unique_options)}}" + " %{fmodules-ts:-fmodule-header=system %{fpreprocessed:-fdirectives-only}}" " %(cc1_options) %2" - " %{!fsyntax-only:%{!S:-o %g.s}" - " %{!fdump-ada-spec*:%{!o*:--output-pch=%i.gch}" - " %W{o*:--output-pch=%*}}%V}}}}", + " %{!S:-o %g.s%V}" + " %{!fsyntax-only:%{!fmodule-*:%{!fmodules-*:%{!fdump-ada-spec*:" + " %{!o*:--output-pch=%i.gch}%W{o*:--output-pch=%*}}}}}}}}", + CPLUSPLUS_CPP_SPEC, 0, 0}, + {"@c++-user-header", + "%{E|M|MM:cc1plus -E" + " %{fmodules-ts:-fdirectives-only -fmodule-header=user}" + " %(cpp_options) %2 %(cpp_debug_options)}" + "%{!E:%{!M:%{!MM:" + " %{save-temps*|no-integrated-cpp:cc1plus -E" + " %{fmodules-ts:-fdirectives-only -fmodule-header=user}" + " %(cpp_options) %2 -o %{save-temps*:%b.ii} %{!save-temps*:%g.ii} \n}" + " cc1plus %{save-temps*|no-integrated-cpp:-fpreprocessed" + " %{fmodules-ts:-fdirectives-only}" + " %{save-temps*:%b.ii} %{!save-temps*:%g.ii}}" + " %{!save-temps*:%{!no-integrated-cpp:%(cpp_unique_options)}}" + " %{fmodules-ts:-fmodule-header=user %{fpreprocessed:-fdirectives-only}}" + " %(cc1_options) %2" + " %{!S:-o %g.s%V}" + " %{!fsyntax-only:%{!fmodule-*:%{!fmodules-*:%{!fdump-ada-spec*:" + " %{!o*:--output-pch=%i.gch}%W{o*:--output-pch=%*}}}}}}}}", CPLUSPLUS_CPP_SPEC, 0, 0}, {"@c++", "%{E|M|MM:cc1plus -E %(cpp_options) %2 %(cpp_debug_options)}" @@ -60,11 +100,14 @@ along with GCC; see the file COPYING3. If not see " cc1plus %{save-temps*|no-integrated-cpp:-fpreprocessed" " %{save-temps*:%b.ii} %{!save-temps*:%g.ii}}" " %{!save-temps*:%{!no-integrated-cpp:%(cpp_unique_options)}}" - " %(cc1_options) %2" - " %{!fsyntax-only:%(invoke_as)}}}}", + " %(cc1_options) %2" + " %{fmodule-only:%{!S:-o %g.s%V}}" + " %{!fsyntax-only:%{!fmodule-only:%(invoke_as)}}}}}", CPLUSPLUS_CPP_SPEC, 0, 0}, {".ii", "@c++-cpp-output", 0, 0, 0}, {"@c++-cpp-output", "%{!E:%{!M:%{!MM:" " cc1plus -fpreprocessed %i %(cc1_options) %2" - " %{!fsyntax-only:%(invoke_as)}}}}", 0, 0, 0}, + " %{fmodule-only:%{!S:-o %g.s%V}}" + " %{!fsyntax-only:%{!fmodule-only:%{!fmodule-header*:" + " %(invoke_as)}}}}}}", 0, 0, 0}, From patchwork Tue Nov 3 21:15:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393425 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=I3k2QQ0/; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjHc4S2bz9sWd for ; Wed, 4 Nov 2020 08:15:40 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 68A9039874B4; Tue, 3 Nov 2020 21:15:38 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) by sourceware.org (Postfix) with ESMTPS id 58883398748A for ; Tue, 3 Nov 2020 21:15:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 58883398748A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x731.google.com with SMTP id s14so16586613qkg.11 for ; Tue, 03 Nov 2020 13:15:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=mtnJusOIBWfP+k4ZeN3+k/kcMYp2TXfbPn6URgeO8bY=; b=I3k2QQ0/ZMMJjUEy0poUvoS3/SuYDlZhz/WJh7zMJZbieU5qoTONlsBmLAXoRPI1wI ZtO+mW0zu2JziaYE+Jt32ibv5Hx+lEHsxvoUoIt0+G9nG4mTIaw/vNukNROeyVHCKSeZ ByhFBq3f2UGF9qf6yHUY1sUB1oD4uUC4PIhQDq0MOPHbPYu8MN7rtXdIZOsb4S+bmWfg DEVzFS38AvWTbvnlln7RiPJdtGwEvUzADfbVwPYr87/aOFkkj39lo46e2UXq9q7kQLAX nitYYDttW6Ks9qdNS52RTyZlJUOTmbP5yueA97VQMhl3/vSuPA62L8rfwd9SNdMwTfVJ Aw6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=mtnJusOIBWfP+k4ZeN3+k/kcMYp2TXfbPn6URgeO8bY=; b=MluG3CWnKO0gnJLvyF0wGnOBf33n3OPU+LRJ/z2esYLq6t6R72CAd85r5sBSJQv2KR D3eRCRkbLDmfqI/oB9olBzB1YOodIYUSxPt05vDEsJRuQVE4XvggNHT5KxUWXXq8wnNa 84vLeywBkZjp0otyBtwxWeJLxZ04ELMT26szE1Eb+IEawAZJUF4KTZSPnBPYNfhMowCW HSM+g6ik/thopD3/URgKq4hCB/8/2Gn0m9oNgVJXIIwFMe3ZyZ5bom67R3FGzc66Blb+ K9CVE7zVKmAlcMeGN+EttHM1Cdzvv1nwizfzie3AZtc2pyt8aF7tTBVzxFkY3vJF13Dj PmJg== X-Gm-Message-State: AOAM532J1zwAttMD/Lwj3uUtkL42ukSdq2X9MyPI0mE1of+m9a33wsh4 cknO6gsl8OWQ/FzwFZc+az5KnShG9wrHiA== X-Google-Smtp-Source: ABdhPJwgR3Cp09vqZF79Yu4SCBebVBI+5iXO15hmGdYSapKRyx86+HgVHhHH+pBkrAMVvtdFrFttFw== X-Received: by 2002:a37:7c04:: with SMTP id x4mr21029952qkc.441.1604438134714; Tue, 03 Nov 2020 13:15:34 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id a201sm12157810qkc.76.2020.11.03.13.15.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:33 -0800 (PST) From: Nathan Sidwell Subject: [14/32] new keywords To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> Message-ID: <1b723606-40b4-a0f6-485e-320b2533b885@acm.org> Date: Tue, 3 Nov 2020 16:15:32 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We have 3 new keywords. As I mentioned in the preprocessor lexing, the keywords are context-sensitive, and we create internal ones. These internal ones have an 'invisible' space at the end of them. This has the advantage of making them return to normal identifiers in preprocessor output. diff --git c/gcc/c-family/c-common.c w/gcc/c-family/c-common.c index d56238aeb01..33d6ad73125 100644 --- c/gcc/c-family/c-common.c +++ w/gcc/c-family/c-common.c @@ -540,6 +540,12 @@ const struct c_common_resword c_common_reswords[] = { "concept", RID_CONCEPT, D_CXX_CONCEPTS_FLAGS | D_CXXWARN }, { "requires", RID_REQUIRES, D_CXX_CONCEPTS_FLAGS | D_CXXWARN }, + /* Modules-related keywords, these are internal unspellable tokens, + created by the preprocessor. */ + { "module ", RID__MODULE, D_CXX_MODULES_FLAGS | D_CXXWARN }, + { "import ", RID__IMPORT, D_CXX_MODULES_FLAGS | D_CXXWARN }, + { "export ", RID__EXPORT, D_CXX_MODULES_FLAGS | D_CXXWARN }, + /* Coroutines-related keywords */ { "co_await", RID_CO_AWAIT, D_CXX_COROUTINES_FLAGS | D_CXXWARN }, { "co_yield", RID_CO_YIELD, D_CXX_COROUTINES_FLAGS | D_CXXWARN }, diff --git c/gcc/c-family/c-common.h w/gcc/c-family/c-common.h index 18b489d55a3..eb9070b4c6c 100644 --- c/gcc/c-family/c-common.h +++ w/gcc/c-family/c-common.h @@ -190,6 +190,9 @@ enum rid /* C++ concepts */ RID_CONCEPT, RID_REQUIRES, + /* C++ modules. */ + RID__MODULE, RID__IMPORT, RID__EXPORT, /* Internal tokens. */ + /* C++ coroutines */ RID_CO_AWAIT, RID_CO_YIELD, RID_CO_RETURN, @@ -438,9 +445,11 @@ extern machine_mode c_default_pointer_mode; #define D_CXX_CHAR8_T 0X1000 /* In C++, only with -fchar8_t. */ #define D_CXX20 0x2000 /* In C++, C++20 only. */ #define D_CXX_COROUTINES 0x4000 /* In C++, only with coroutines. */ +#define D_CXX_MODULES 0x8000 /* In C++, only with modules. */ #define D_CXX_CONCEPTS_FLAGS D_CXXONLY | D_CXX_CONCEPTS #define D_CXX_CHAR8_T_FLAGS D_CXXONLY | D_CXX_CHAR8_T +#define D_CXX_MODULES_FLAGS (D_CXXONLY | D_CXX_MODULES) #define D_CXX_COROUTINES_FLAGS (D_CXXONLY | D_CXX_COROUTINES) /* The reserved keyword table. */ diff --git c/gcc/c-family/c-cppbuiltin.c w/gcc/c-family/c-cppbuiltin.c index e5ebb79e22a..12043aa4702 100644 --- c/gcc/c-family/c-cppbuiltin.c +++ w/gcc/c-family/c-cppbuiltin.c @@ -1013,6 +1013,10 @@ c_cpp_builtins (cpp_reader *pfile) else cpp_define (pfile, "__cpp_concepts=201507L"); } + if (flag_modules) + /* The std-defined value is 201907L, but I don't think we can + claim victory yet. 201810 is the p1103 date. */ + cpp_define (pfile, "__cpp_modules=201810L"); if (flag_coroutines) cpp_define (pfile, "__cpp_impl_coroutine=201902L"); /* n4861, DIS */ if (flag_tm) diff --git c/gcc/cp/lex.c w/gcc/cp/lex.c index 8a69bc4f170..013cbadf625 100644 --- c/gcc/cp/lex.c +++ w/gcc/cp/lex.c @@ -235,6 +236,8 @@ init_reswords (void) mask |= D_CXX_CONCEPTS; if (!flag_coroutines) mask |= D_CXX_COROUTINES; + if (!flag_modules) + mask |= D_CXX_MODULES; if (!flag_tm) mask |= D_TRANSMEM; if (!flag_char8_t) From patchwork Tue Nov 3 21:15:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=sJubc4fA; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjHn2Xvrz9sXx for ; Wed, 4 Nov 2020 08:15:49 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 50D0439874B0; Tue, 3 Nov 2020 21:15:47 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) by sourceware.org (Postfix) with ESMTPS id 535073987492 for ; Tue, 3 Nov 2020 21:15:44 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 535073987492 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x736.google.com with SMTP id p3so16607590qkk.7 for ; Tue, 03 Nov 2020 13:15:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=TSec9aWcsHxpqILjcPGBc5n4GVkrCBtgsOXFJ5oIajc=; b=sJubc4fALBtYJS285BQYdEUc3y6TYSZR/n5Jdr3ZautyUJRZF/QPd4Q1F/1fUXRpnr osjqA5LxjmXYkxopFSLnpB8WAfF4tImbkEk2rqPDxwJVFeCT11ejKVfoC/nbn9ECANNo XSYLfHuZSbK5Nz8+0N5MzEjD3F85O7wdDa23fjArGVk6TCFotIiFlAwj7+Kyowf1tkjv lQwViQoUPYrrNBOeOLiMIZ+GnRAbRxqui3rGyKAXeWLqJO0KX1VvMyXRv/sbDAdAt4cB pzwVzdYIMyUvsPDhFwIuHZ5AFEvor06Y/maGy7Sjl0+cFiqPwraBYAspdlj/oQKvvZoA e2sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=TSec9aWcsHxpqILjcPGBc5n4GVkrCBtgsOXFJ5oIajc=; b=HOOp5jr9cAsyI9B9hHIidP+sNO2pdYOjWEZxdigB5hZ9pbPqna5MjQfb+clK177pgJ qlKfGs6XTPRuIc8c17/wKNchS2Og9zHoFSbKvP+rQSUE0U849QR71ye/nUbtqDkw7UxF nWYGhj7KFbl0iqDCbMPdqi1vRbIIVw8C2OFc/Nr0BOW5itY3oibzdHxMMKN5Y58yoBT4 grYkokcbU4T48fGiZ4jsWros1HRsEXFUSqRfVOlk4N3jbLxrIL+Zh3ROAINTIvsqXh3j JgsUqPylhYEIogF5UgyTQ5L+iPFpYL1F4swd7Vc4aSwzeEm67bgos35JNwXmCsND7+Mx LhvQ== X-Gm-Message-State: AOAM532dTO+yNszKWVLVUblBh7+tOUnn7wBe0IGbZtERG/LgmiK9PXjO gzlG0B/RVWcvjGxHbRnZWUc= X-Google-Smtp-Source: ABdhPJz8/SmGMnQFLQ9uWAxZKN+SiTKDUJVr/dCGpbEiZK201ZpWwAYgG3P7W7wG5xCBhTPVhK35dw== X-Received: by 2002:a37:886:: with SMTP id 128mr21248161qki.484.1604438143580; Tue, 03 Nov 2020 13:15:43 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 29sm12047970qks.28.2020.11.03.13.15.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:42 -0800 (PST) From: Nathan Sidwell Subject: [15/32] new C++ lexer To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> Message-ID: <7a603e9d-0760-9aa8-11f0-7f8788736909@acm.org> Date: Tue, 3 Nov 2020 16:15:40 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <6666545b-0583-4812-4745-d51994465818@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" Importation of header-units requires the tokenizer to recognize such imports during tokenization so that their macros become available. The C++ parser currently (and continues to) tokenize the entire file before beginning c++ parsing. This implements an explicit coroutine to manage that recognition. It is used both for C++ parsing proper and for just preprocessing. When a module-significant control line is observed, we call into the module machinery to handle it. Usually, we'll also call again later when parsing that declaration. diff --git c/gcc/cp/lex.c w/gcc/cp/lex.c index 8a69bc4f170..013cbadf625 100644 --- c/gcc/cp/lex.c +++ w/gcc/cp/lex.c @@ -32,6 +32,7 @@ along with GCC; see the file COPYING3. If not see #include "c-family/c-objc.h" #include "gcc-rich-location.h" #include "cp-name-hint.h" +#include "langhooks.h" static int interface_strcmp (const char *); static void init_cp_pragma (void); @@ -378,7 +381,206 @@ interface_strcmp (const char* s) return 1; } - +/* We've just read a cpp-token, figure out our next state. Hey, this + is a hand-coded co-routine! */ + +struct token_coro +{ + enum state + { + idle, + module_first, + module_cont, + module_end, + }; + + enum state state : 8; + bool is_import : 1; + bool got_export : 1; + bool got_colon : 1; + bool want_dot : 1; + + location_t token_loc; + cpp_reader *reader; + module_state *module; + module_state *import; + + token_coro (cpp_reader *reader) + : state (idle), is_import (false), + got_export (false), got_colon (false), want_dot (false), + token_loc (UNKNOWN_LOCATION), + reader (reader), module (NULL), import (NULL) + { + }; + + /* Process the next token. Note we cannot see CPP_EOF inside a + pragma -- a CPP_PRAGMA_EOL always happens. */ + uintptr_t resume (int type, int keyword, tree value, location_t loc) + { + unsigned res = 0; + + switch (state) + { + case idle: + if (type == CPP_KEYWORD) + switch (keyword) + { + default: + break; + + case RID__EXPORT: + got_export = true; + res = lang_hooks::PT_begin_pragma; + break; + + case RID__IMPORT: + is_import = true; + /* FALLTHRU */ + case RID__MODULE: + state = module_first; + want_dot = false; + got_colon = false; + token_loc = loc; + import = NULL; + if (!got_export) + res = lang_hooks::PT_begin_pragma; + break; + } + break; + + case module_first: + if (is_import && type == CPP_HEADER_NAME) + { + /* A header name. The preprocessor will have already + done include searching and canonicalization. */ + state = module_end; + goto header_unit; + } + + if (type == CPP_PADDING || type == CPP_COMMENT) + break; + + state = module_cont; + if (type == CPP_COLON && module) + { + got_colon = true; + import = module; + break; + } + /* FALLTHROUGH */ + + case module_cont: + switch (type) + { + case CPP_PADDING: + case CPP_COMMENT: + break; + + default: + /* If we ever need to pay attention to attributes for + header modules, more logic will be needed. */ + state = module_end; + break; + + case CPP_COLON: + if (got_colon) + state = module_end; + got_colon = true; + /* FALLTHROUGH */ + case CPP_DOT: + if (!want_dot) + state = module_end; + want_dot = false; + break; + + case CPP_PRAGMA_EOL: + goto module_end; + + case CPP_NAME: + if (want_dot) + { + /* Got name instead of [.:]. */ + state = module_end; + break; + } + header_unit: + import = get_module (value, import, got_colon); + want_dot = true; + break; + } + break; + + case module_end: + if (type == CPP_PRAGMA_EOL) + { + module_end:; + /* End of the directive, handle the name. */ + if (import) + if (module_state *m + = preprocess_module (import, token_loc, module != NULL, + is_import, got_export, reader)) + if (!module) + module = m; + + is_import = got_export = false; + state = idle; + } + break; + } + + return res; + } +}; + +/* Initialize or teardown. */ + +uintptr_t +module_token_cdtor (cpp_reader *pfile, uintptr_t data_) +{ + if (token_coro *coro = reinterpret_cast (data_)) + { + preprocessed_module (pfile); + delete coro; + data_ = 0; + } + else if (modules_p ()) + data_ = reinterpret_cast (new token_coro (pfile)); + + return data_; +} + +uintptr_t +module_token_lang (int type, int keyword, tree value, location_t loc, + uintptr_t data_) +{ + token_coro *coro = reinterpret_cast (data_); + return coro->resume (type, keyword, value, loc); +} + +uintptr_t +module_token_pre (cpp_reader *pfile, const cpp_token *tok, uintptr_t data_) +{ + if (!tok) + return module_token_cdtor (pfile, data_); + + int type = tok->type; + int keyword = RID_MAX; + tree value = NULL_TREE; + + if (tok->type == CPP_NAME) + { + value = HT_IDENT_TO_GCC_IDENT (HT_NODE (tok->val.node.node)); + if (IDENTIFIER_KEYWORD_P (value)) + { + keyword = C_RID_CODE (value); + type = CPP_KEYWORD; + } + } + else if (tok->type == CPP_HEADER_NAME) + value = build_string (tok->val.str.len, (const char *)tok->val.str.text); + + return module_token_lang (type, keyword, value, tok->src_loc, data_); +} /* Parse a #pragma whose sole argument is a string constant. If OPT is true, the argument is optional. */ diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -6742,6 +6940,10 @@ extern void set_identifier_kind (tree, cp_identifier_kind); extern bool cxx_init (void); extern void cxx_finish (void); extern bool in_main_input_context (void); +extern uintptr_t module_token_pre (cpp_reader *, const cpp_token *, uintptr_t); +extern uintptr_t module_token_cdtor (cpp_reader *, uintptr_t); +extern uintptr_t module_token_lang (int type, int keyword, tree value, + location_t, uintptr_t); /* in method.c */ extern void init_method (void); From patchwork Tue Nov 3 21:15:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393427 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=ag0feDV0; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjHx1mnbz9sXx for ; Wed, 4 Nov 2020 08:15:57 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id C0BCD39874A1; Tue, 3 Nov 2020 21:15:54 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) by sourceware.org (Postfix) with ESMTPS id 09639398748A for ; Tue, 3 Nov 2020 21:15:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 09639398748A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x732.google.com with SMTP id k9so16624944qki.6 for ; Tue, 03 Nov 2020 13:15:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=m9DwVFXhfHKUKuVWEu6S7vgIT97ypx1mlWE7yVr4M44=; b=ag0feDV0J9KwLSD/qnEXd41w440+YEgVvv9tU1xZCcQaYajIvsZi3SPO1zquMm5pXZ uJjphKi8sB++s4uBmCbH4Jbtpbo4whRcW5vzZq8xKl1RjY+xxZKD+erNwyTAzCRMBoci uZNSeqbnGUPTwqbyz0HuhnLfRlg433kWSMUf1h384mnqU/2fJAhDARp16A/0jUsKYZ8V 2pW6xDKnXfFW/EwWLvx3yIKelVDDrYUBjMDl4x3Voo6lQyOCgI9n4cO4SEEijVtUQplF oWI4qxs0z19AqMKuwBQ1EOM5+YFKFl5tQaKTcCSXM+ahjVljALzvucsiPnoiqLwH936H fAXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=m9DwVFXhfHKUKuVWEu6S7vgIT97ypx1mlWE7yVr4M44=; b=ZEABFOv3JV+Hbs8jR6gSKigv1pXr30yZbHmJ98jQNAzKLik4QvPJkhpPYgzsFZ6oS1 Y5vxIjWfA7HlWzmnV/zFRIsZVlGGgYDJGWdLAXxDZjYMSJix5+Hxct0YCpztE9QUaiWH l80Q6dfq/kpS5EOdk8r3KuTbTp84pMpQZWaX/iA0S1oM12C560H8l6er6ydq9JcGXm0y A5OIL6CpXL7FRlDiYWLCaYFSLFc1BufXej4g0MBVnOFR77ZZgO2Rqd357Z91Rz3G2hNy UjeuKA2OuxGjbDcrext42kpJPrwtaEfhk1fy3BHWztx/ZY2yL0nJBLGZwHQxou/U6stO 54Zg== X-Gm-Message-State: AOAM532Pr0D34PgCaeOJMWLWzK9zIBOGl39xdcEMe2tLnkA8km4qIvxc UQjcDNjNKBLqSQ3F3gqpu+o= X-Google-Smtp-Source: ABdhPJyUFAFc2j9U9UlhIRvoHwt1unm7ww60511ed382eKDzqQbSGXdQG5Ja06WegzQ25UfBk4uhPA== X-Received: by 2002:a37:73c6:: with SMTP id o189mr21923277qkc.216.1604438151287; Tue, 03 Nov 2020 13:15:51 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id p77sm11494900qke.39.2020.11.03.13.15.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:50 -0800 (PST) From: Nathan Sidwell Subject: [16/32] new C++ infrastructure To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> Message-ID: <9d1fa530-8f5a-3603-5d60-5ef10c09e9b2@acm.org> Date: Tue, 3 Nov 2020 16:15:49 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We have to expose a few entry points in the c++ handling. when loading a module we need to do template argument comparisons and not resolve typenames. I added a comparing_typenames global flag that;ll end up doing a but more than the 'comparing_specializations' flag we already have. Some of the node creation routines need lower level access. The rtti machinery needs to be driven externally. We need to create template type parameters, and we need to create explicitly dependent array types (we can't call dependent_type_p in the middle of loading something). Oh, this also has some pieces handling parameter packs as structurally-compared types. IIRC when we fixed some of that in the GCC10 release, we tried to handle them as not-types. I never addressed that on the modules branch. nathan diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -5327,6 +5500,10 @@ extern int function_depth; in structrual_comptypes. */ extern int comparing_specializations; +/* Nonzero if we are inside eq_specializations, which affects + resolving of typenames in structural_comptypes. */ +extern int comparing_typenames; + /* In parser.c. */ /* Nonzero if we are parsing an unevaluated operand: an operand to @@ -6457,6 +6651,7 @@ extern bool check_omp_return (void); extern tree make_typename_type (tree, tree, enum tag_types, tsubst_flags_t); extern tree build_typename_type (tree, tree, tree, tag_types); extern tree make_unbound_class_template (tree, tree, tree, tsubst_flags_t); +extern tree make_unbound_class_template_raw (tree, tree, tree); extern tree build_library_fn_ptr (const char *, tree, int); extern tree build_cp_library_fn_ptr (const char *, tree, int); extern tree push_library_fn (tree, tree, tree, int); @@ -6731,8 +6926,11 @@ extern tree unqualified_fn_lookup_error (cp_expr); extern tree make_conv_op_name (tree); extern tree build_lang_decl (enum tree_code, tree, tree); extern tree build_lang_decl_loc (location_t, enum tree_code, tree, tree); +extern bool maybe_add_lang_decl_raw (tree, bool decomp_p); +extern bool maybe_add_lang_type_raw (tree); extern void retrofit_lang_decl (tree); extern void fit_decomposition_lang_decl (tree, tree); +extern void fit_ptrmem_type_decl (tree, tree); extern tree copy_decl (tree CXX_MEM_STAT_INFO); extern tree copy_type (tree CXX_MEM_STAT_INFO); extern tree cxx_make_type (enum tree_code CXX_MEM_STAT_INFO); @@ -6793,6 +7089,7 @@ extern void maybe_show_extern_c_location (void); extern bool literal_integer_zerop (const_tree); /* in pt.c */ +extern tree canonical_type_parameter (tree); extern void push_access_scope (tree); extern void pop_access_scope (tree); extern bool check_template_shadow (tree); @@ -6986,6 +7293,7 @@ extern GTY(()) vec *unemitted_tinfo_decls; extern void init_rtti_processing (void); extern tree build_typeid (tree, tsubst_flags_t); +extern tree get_tinfo_decl_direct (tree, tree, int); extern tree get_tinfo_decl (tree); extern tree get_typeid (tree, tsubst_flags_t); extern tree build_headof (tree); @@ -6993,6 +7301,8 @@ extern tree build_dynamic_cast (location_t, tree, tree, tsubst_flags_t); extern void emit_support_tinfos (void); extern bool emit_tinfo_decl (tree); +extern unsigned get_pseudo_tinfo_index (tree); +extern tree get_pseudo_tinfo_type (unsigned); /* in search.c */ extern bool accessible_base_p (tree, tree, bool); @@ -7340,7 +7650,7 @@ extern bool is_local_temp (tree); extern tree build_aggr_init_expr (tree, tree); extern tree get_target_expr (tree); extern tree get_target_expr_sfinae (tree, tsubst_flags_t); -extern tree build_cplus_array_type (tree, tree); +extern tree build_cplus_array_type (tree, tree, int is_dep = -1); extern tree build_array_of_n_type (tree, int); extern bool array_of_runtime_bound_p (tree); extern bool vla_type_p (tree); diff --git c/gcc/cp/lex.c w/gcc/cp/lex.c index 8a69bc4f170..013cbadf625 100644 --- c/gcc/cp/lex.c +++ w/gcc/cp/lex.c @@ -678,7 +880,7 @@ build_lang_decl_loc (location_t loc, enum tree_code code, tree name, tree type) /* Maybe add a raw lang_decl to T, a decl. Return true if it needed one. */ -static bool +bool maybe_add_lang_decl_raw (tree t, bool decomp_p) { size_t size; @@ -803,6 +1005,9 @@ cxx_dup_lang_specific_decl (tree node) struct lang_decl *ld = (struct lang_decl *) ggc_internal_alloc (size); memcpy (ld, DECL_LANG_SPECIFIC (node), size); DECL_LANG_SPECIFIC (node) = ld; + DECL_MODULE_ENTITY_P (node) = false; + DECL_MODULE_IMPORT_P (node) = false; + DECL_ATTACHED_DECLS_P (node) = false; if (GATHER_STATISTICS) { @@ -831,8 +1036,7 @@ copy_lang_type (tree node) if (! TYPE_LANG_SPECIFIC (node)) return; - struct lang_type *lt - = (struct lang_type *) ggc_internal_alloc (sizeof (struct lang_type)); + auto *lt = (struct lang_type *) ggc_internal_alloc (sizeof (struct lang_type)); memcpy (lt, TYPE_LANG_SPECIFIC (node), (sizeof (struct lang_type))); TYPE_LANG_SPECIFIC (node) = lt; @@ -858,15 +1062,15 @@ copy_type (tree type MEM_STAT_DECL) /* Add a raw lang_type to T, a type, should it need one. */ -static bool +bool maybe_add_lang_type_raw (tree t) { if (!RECORD_OR_UNION_CODE_P (TREE_CODE (t))) return false; - TYPE_LANG_SPECIFIC (t) - = (struct lang_type *) (ggc_internal_cleared_alloc - (sizeof (struct lang_type))); + auto *lt = (struct lang_type *) (ggc_internal_cleared_alloc + (sizeof (struct lang_type))); + TYPE_LANG_SPECIFIC (t) = lt; if (GATHER_STATISTICS) { diff --git c/gcc/cp/pt.c w/gcc/cp/pt.c index aa162d2a4f9..497ac5aafec 100644 --- c/gcc/cp/pt.c +++ w/gcc/cp/pt.c @@ -1709,9 +1708,11 @@ register_specialization (tree spec, tree tmpl, tree args, bool is_friend, return spec; } -/* Returns true iff two spec_entry nodes are equivalent. */ - +/* Restricts tree and type comparisons. */ int comparing_specializations; +int comparing_typenames; + +/* Returns true iff two spec_entry nodes are equivalent. */ bool spec_hasher::equal (spec_entry *e1, spec_entry *e2) @@ -1719,6 +1720,7 @@ spec_hasher::equal (spec_entry *e1, spec_entry *e2) int equal; ++comparing_specializations; + ++comparing_typenames; equal = (e1->tmpl == e2->tmpl && comp_template_args (e1->args, e2->args)); if (equal && flag_concepts @@ -1734,6 +1736,7 @@ spec_hasher::equal (spec_entry *e1, spec_entry *e2) equal = equivalent_constraints (c1, c2); } --comparing_specializations; + --comparing_typenames; return equal; } @@ -4435,7 +4438,7 @@ build_template_parm_index (int index, parameter. Returns the canonical type parameter, which may be TYPE if no such parameter existed. */ -static tree +tree canonical_type_parameter (tree type) { int idx = TEMPLATE_TYPE_IDX (type); @@ -13205,19 +13244,24 @@ tsubst_argument_pack (tree orig_arg, tree args, tsubst_flags_t complain, tree in_decl) { /* Substitute into each of the arguments. */ - tree new_arg = TYPE_P (orig_arg) - ? cxx_make_type (TREE_CODE (orig_arg)) - : make_node (TREE_CODE (orig_arg)); - tree pack_args = tsubst_template_args (ARGUMENT_PACK_ARGS (orig_arg), args, complain, in_decl); - if (pack_args == error_mark_node) - new_arg = error_mark_node; - else - SET_ARGUMENT_PACK_ARGS (new_arg, pack_args); + tree new_arg = error_mark_node; + if (pack_args != error_mark_node) + { + if (TYPE_P (orig_arg)) + { + new_arg = cxx_make_type (TREE_CODE (orig_arg)); + SET_TYPE_STRUCTURAL_EQUALITY (new_arg); + } + else + { + new_arg = make_node (TREE_CODE (orig_arg)); + TREE_CONSTANT (new_arg) = TREE_CONSTANT (orig_arg); + } - if (TREE_CODE (new_arg) == NONTYPE_ARGUMENT_PACK) - TREE_CONSTANT (new_arg) = TREE_CONSTANT (orig_arg); + SET_ARGUMENT_PACK_ARGS (new_arg, pack_args); + } return new_arg; } diff --git c/gcc/cp/typeck.c w/gcc/cp/typeck.c index d3b701610cf..49b895349c8 100644 --- c/gcc/cp/typeck.c +++ w/gcc/cp/typeck.c @@ -1256,16 +1256,15 @@ structural_comptypes (tree t1, tree t2, int strict) gcc_assert (TYPE_P (t1) && TYPE_P (t2)); - if (!comparing_specializations) - { - /* TYPENAME_TYPEs should be resolved if the qualifying scope is the - current instantiation. */ - if (TREE_CODE (t1) == TYPENAME_TYPE) - t1 = resolve_typename_type (t1, /*only_current_p=*/true); - - if (TREE_CODE (t2) == TYPENAME_TYPE) - t2 = resolve_typename_type (t2, /*only_current_p=*/true); - } + /* TYPENAME_TYPEs should be resolved if the qualifying scope is the + current instantiation, and we don't care about typename + structural equality. The comparing_typenames check is after the + code check, in order to early-out the common case. */ + if (TREE_CODE (t1) == TYPENAME_TYPE && !comparing_typenames) + t1 = resolve_typename_type (t1, /*only_current_p=*/true); + + if (TREE_CODE (t2) == TYPENAME_TYPE && !comparing_typenames) + t2 = resolve_typename_type (t2, /*only_current_p=*/true); if (TYPE_PTRMEMFUNC_P (t1)) t1 = TYPE_PTRMEMFUNC_FN_TYPE (t1); diff --git c/gcc/cp/rtti.c w/gcc/cp/rtti.c index 887aae31bf6..0be9eff54ad 100644 --- c/gcc/cp/rtti.c +++ w/gcc/cp/rtti.c @@ -123,7 +123,6 @@ static GTY (()) vec *tinfo_descs; static tree ifnonnull (tree, tree, tsubst_flags_t); static tree tinfo_name (tree, bool); -static tree get_tinfo_decl_direct (tree type, tree name, int pseudo_ix); static tree build_dynamic_cast_1 (location_t, tree, tree, tsubst_flags_t); static tree throw_bad_cast (void); static tree throw_bad_typeid (void); @@ -431,7 +434,7 @@ get_tinfo_decl (tree type) /* Get or create a tinfo VAR_DECL directly from the provided information. The caller must have already checked it is valid to do so. */ -static tree +tree get_tinfo_decl_direct (tree type, tree name, int pseudo_ix) { /* For a class type, the variable is cached in the type node @@ -1491,6 +1495,36 @@ get_tinfo_desc (unsigned ix) return res; } +/* Return an identifying index for the pseudo type_info TYPE. + We wrote the index at the end of the name, so just scan it from + there. This isn't critical, as it's only on the first use of this + type during module stream out. */ + +unsigned +get_pseudo_tinfo_index (tree type) +{ + tree name = DECL_NAME (TYPE_NAME (type)); + unsigned ix = 0, scale = 1; + size_t len = IDENTIFIER_LENGTH (name); + const char *ptr = IDENTIFIER_POINTER (name) + len; + + for (; *--ptr != '_'; scale *= 10) + { + len--; + gcc_checking_assert (len && ISDIGIT (*ptr)); + ix += (*ptr - '0') * scale; + } + + gcc_assert (len != IDENTIFIER_LENGTH (name)); + return ix; +} + +tree +get_pseudo_tinfo_type (unsigned ix) +{ + return get_tinfo_desc (ix)->type; +} + /* We lazily create the type info types. */ static void diff --git c/gcc/cp/tree.c w/gcc/cp/tree.c index 3087c4ab52c..d125fa5e793 100644 --- c/gcc/cp/tree.c +++ w/gcc/cp/tree.c @@ -998,7 +998,7 @@ build_min_array_type (tree elt_type, tree index_type) build_cplus_array_type. */ static void -set_array_type_canon (tree t, tree elt_type, tree index_type) +set_array_type_canon (tree t, tree elt_type, tree index_type, bool dep) { /* Set the canonical type for this new node. */ if (TYPE_STRUCTURAL_EQUALITY_P (elt_type) @@ -1009,30 +1009,33 @@ set_array_type_canon (tree t, tree elt_type, tree index_type) TYPE_CANONICAL (t) = build_cplus_array_type (TYPE_CANONICAL (elt_type), index_type - ? TYPE_CANONICAL (index_type) : index_type); + ? TYPE_CANONICAL (index_type) : index_type, + dep); else TYPE_CANONICAL (t) = t; } /* Like build_array_type, but handle special C++ semantics: an array of a variant element type is a variant of the array of the main variant of - the element type. */ + the element type. IS_DEPENDENT is -ve if we should determine the + dependency. Otherwise its bool value indicates dependency. */ tree -build_cplus_array_type (tree elt_type, tree index_type) +build_cplus_array_type (tree elt_type, tree index_type, int dependent) { tree t; if (elt_type == error_mark_node || index_type == error_mark_node) return error_mark_node; - bool dependent = (uses_template_parms (elt_type) - || (index_type && uses_template_parms (index_type))); + if (dependent < 0) + dependent = (uses_template_parms (elt_type) + || (index_type && uses_template_parms (index_type))); if (elt_type != TYPE_MAIN_VARIANT (elt_type)) /* Start with an array of the TYPE_MAIN_VARIANT. */ t = build_cplus_array_type (TYPE_MAIN_VARIANT (elt_type), - index_type); + index_type, dependent); else if (dependent) { /* Since type_hash_canon calls layout_type, we need to use our own @@ -1062,7 +1065,11 @@ build_cplus_array_type (tree elt_type, tree index_type) *e = t; /* Set the canonical type for this new node. */ - set_array_type_canon (t, elt_type, index_type); + set_array_type_canon (t, elt_type, index_type, dependent); + + /* Mark it as dependent now, this saves time later. */ + TYPE_DEPENDENT_P_VALID (t) = true; + TYPE_DEPENDENT_P (t) = true; } } else @@ -1083,7 +1090,7 @@ build_cplus_array_type (tree elt_type, tree index_type) if (!t) { t = build_min_array_type (elt_type, index_type); - set_array_type_canon (t, elt_type, index_type); + set_array_type_canon (t, elt_type, index_type, dependent); if (!dependent) { layout_type (t); @@ -1319,7 +1326,8 @@ cp_build_qualified_type_real (tree type, if (!t) { - t = build_cplus_array_type (element_type, TYPE_DOMAIN (type)); + t = build_cplus_array_type (element_type, TYPE_DOMAIN (type), + TYPE_DEPENDENT_P (type)); /* Keep the typedef name. */ if (TYPE_NAME (t) != TYPE_NAME (type)) @@ -1555,7 +1563,7 @@ strip_typedefs (tree t, bool *remove_attributes, unsigned int flags) case ARRAY_TYPE: type = strip_typedefs (TREE_TYPE (t), remove_attributes, flags); t0 = strip_typedefs (TYPE_DOMAIN (t), remove_attributes, flags); - result = build_cplus_array_type (type, t0); + result = build_cplus_array_type (type, t0, TYPE_DEPENDENT_P (t)); break; case FUNCTION_TYPE: case METHOD_TYPE: From patchwork Tue Nov 3 21:15:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393428 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=c/94MUWb; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjJ73B8Wz9sWM for ; Wed, 4 Nov 2020 08:16:07 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A59AF39874C9; Tue, 3 Nov 2020 21:16:04 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by sourceware.org (Postfix) with ESMTPS id 7CC72398748A for ; Tue, 3 Nov 2020 21:16:01 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 7CC72398748A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72f.google.com with SMTP id i21so15880651qka.12 for ; Tue, 03 Nov 2020 13:16:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=U4PIjQPgcR1MLSfITQAFJYmhHZxkcP+P2teDsonkm50=; b=c/94MUWbE9GlxIaavj11LkNHMCWXG3XypW6lKi4BSPjgyNqekMudMRrR97pIqvX800 wnYx0zt4b7yLPxTpjxodM6GwdVJ/4/75LfDz0eEEE1KMWYX8LPAD97UXOrj+u1CZklU/ 5Uxfo9Z5zAOwgiOBN0aZAxw8/72VFEPIQCKlG0qUhfsjWGvbMR5UTfBMP1auTMbiz3e6 RPR3qCPjHbBIXNyi+q7KbRhtUzoQn4PAUerF0U3I6JpqHcJN5TmyT7zOU2+OaNzF953X XBkmi7LSZs8nqu9j41vUMNYvpZKPDoNlRRVBlu1UlZj99ZAmxuAjrgtSl2hDobibYg/X IW8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=U4PIjQPgcR1MLSfITQAFJYmhHZxkcP+P2teDsonkm50=; b=p/uXTBnj7bQ87XmqbaamDKd2YylliNSzu6BVznFnsBIZK9tkyOa/DyCiHyJKlhsQC7 ETLtM5YRojAxkop3zhN/PKgxXXWGSHurOtNfJbfgIPmKMUDXP6spTLEGXZXPL0AkZCwg ob0LXElqovNZRSssWDzDtLXOqan+wmwaQTHpfxWom+mPxgRk/lHkRh0HUE0WMPI63Hbn /yOsfkw4sL+m/cO1xNLeqqW+JsbpDk5uWhmahY29d8YuakvW9EjwUjQUYsFTmJ4HqN/6 H+wqekHjg6sGFVGT/ZyLzzeBGdKF3bfzDQssTMRlAzRsjy6q+v8eahmn/R40i5rdm3Jb KskQ== X-Gm-Message-State: AOAM53020Rzybtrsg8tKTgcZ7Uq2RBbDkMrbabqU0GrWK81ud74wBAK/ xqiexFZ8OJ/Q9HbaVLWp8C5mGhg5FFbz8A== X-Google-Smtp-Source: ABdhPJza9LQCTG9M8nhqL01oY6lXqKNgNKUx1432YQjAIQ+w49nS+wLjcd89hoWgLivnKKnqxVdbxw== X-Received: by 2002:a37:9a89:: with SMTP id c131mr21299216qke.80.1604438160673; Tue, 03 Nov 2020 13:16:00 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 19sm12080601qkf.93.2020.11.03.13.15.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:15:59 -0800 (PST) From: Nathan Sidwell Subject: [17/32] new C++ constexpr bits To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:15:58 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We need to load and save constexpr bodies. This exposes that hash table from constexpr diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -7862,9 +8177,20 @@ extern void vtv_recover_class_info (void); extern void vtv_build_vtable_verify_fndecl (void); /* In constexpr.c */ +/* Representation of entries in the constexpr function definition table. */ + +struct GTY((for_user)) constexpr_fundef { + tree decl; + tree body; + tree parms; + tree result; +}; + extern void fini_constexpr (void); extern bool literal_type_p (tree); -extern tree register_constexpr_fundef (tree, tree); +extern tree check_constexpr_fundef (tree, tree); +extern tree register_constexpr_fundef (const constexpr_fundef &); +extern constexpr_fundef *retrieve_constexpr_fundef (tree); extern bool is_valid_constexpr_fn (tree, bool); extern bool check_constexpr_ctor_body (tree, tree, bool); extern tree constexpr_fn_retval (tree); diff --git c/gcc/cp/constexpr.c w/gcc/cp/constexpr.c index b46824f128d..8e94162e0c0 100644 --- c/gcc/cp/constexpr.c +++ w/gcc/cp/constexpr.c @@ -133,19 +133,10 @@ ensure_literal_type_for_constexpr_object (tree decl) return decl; } -/* Representation of entries in the constexpr function definition table. */ - -struct GTY((for_user)) constexpr_fundef { - tree decl; - tree body; - tree parms; - tree result; -}; - struct constexpr_fundef_hasher : ggc_ptr_hash { - static hashval_t hash (constexpr_fundef *); - static bool equal (constexpr_fundef *, constexpr_fundef *); + static hashval_t hash (const constexpr_fundef *); + static bool equal (const constexpr_fundef *, const constexpr_fundef *); }; /* This table holds all constexpr function definitions seen in @@ -158,7 +149,8 @@ static GTY (()) hash_table *constexpr_fundef_table; same constexpr function. */ inline bool -constexpr_fundef_hasher::equal (constexpr_fundef *lhs, constexpr_fundef *rhs) +constexpr_fundef_hasher::equal (const constexpr_fundef *lhs, + const constexpr_fundef *rhs) { return lhs->decl == rhs->decl; } @@ -167,20 +159,20 @@ constexpr_fundef_hasher::equal (constexpr_fundef *lhs, constexpr_fundef *rhs) Return a hash value for the entry pointed to by Q. */ inline hashval_t -constexpr_fundef_hasher::hash (constexpr_fundef *fundef) +constexpr_fundef_hasher::hash (const constexpr_fundef *fundef) { return DECL_UID (fundef->decl); } /* Return a previously saved definition of function FUN. */ -static constexpr_fundef * +constexpr_fundef * retrieve_constexpr_fundef (tree fun) { if (constexpr_fundef_table == NULL) return NULL; - constexpr_fundef fundef = { fun, NULL, NULL, NULL }; + constexpr_fundef fundef = { fun, NULL_TREE, NULL_TREE, NULL_TREE }; return constexpr_fundef_table->find (&fundef); } @@ -669,7 +661,7 @@ get_function_named_in_call (tree t) return fun; } -/* Subroutine of register_constexpr_fundef. BODY is the body of a function +/* Subroutine of check_constexpr_fundef. BODY is the body of a function declared to be constexpr, or a sub-statement thereof. Returns the return value if suitable, error_mark_node for a statement not allowed in a constexpr function, or NULL_TREE if no return value was found. */ @@ -738,7 +730,7 @@ constexpr_fn_retval (tree body) } } -/* Subroutine of register_constexpr_fundef. BODY is the DECL_SAVED_TREE of +/* Subroutine of check_constexpr_fundef. BODY is the DECL_SAVED_TREE of FUN; do the necessary transformations to turn it into a single expression that we can store in the hash table. */ @@ -868,17 +860,14 @@ cx_check_missing_mem_inits (tree ctype, tree body, bool complain) } /* We are processing the definition of the constexpr function FUN. - Check that its BODY fulfills the propriate requirements and + Check that its BODY fulfills the apropriate requirements and enter it in the constexpr function definition table. For constructor BODY is actually the TREE_LIST of the member-initializer list. */ tree -register_constexpr_fundef (tree fun, tree body) +check_constexpr_fundef (tree fun, tree body) { - constexpr_fundef entry; - constexpr_fundef **slot; - if (!is_valid_constexpr_fn (fun, !DECL_GENERATED_P (fun))) return NULL; @@ -903,37 +892,47 @@ register_constexpr_fundef (tree fun, tree body) if (!potential && !DECL_GENERATED_P (fun)) return NULL; - /* Create the constexpr function table if necessary. */ - if (constexpr_fundef_table == NULL) - constexpr_fundef_table - = hash_table::create_ggc (101); - - entry.decl = fun; - tree saved_fn = current_function_decl; + constexpr_fundef entry = {fun, NULL_TREE, NULL_TREE, NULL_TREE}; bool clear_ctx = false; - current_function_decl = fun; if (DECL_RESULT (fun) && DECL_CONTEXT (DECL_RESULT (fun)) == NULL_TREE) { clear_ctx = true; DECL_CONTEXT (DECL_RESULT (fun)) = fun; } - entry.body = copy_fn (fun, entry.parms, entry.result); + tree saved_fn = current_function_decl; + current_function_decl = fun; + entry.body = copy_fn (entry.decl, entry.parms, entry.result); current_function_decl = saved_fn; - slot = constexpr_fundef_table->find_slot (&entry, INSERT); if (clear_ctx) - DECL_CONTEXT (DECL_RESULT (fun)) = NULL_TREE; - + DECL_CONTEXT (DECL_RESULT (entry.decl)) = NULL_TREE; if (!potential) /* For a template instantiation, we want to remember the pre-generic body for explain_invalid_constexpr_fn, but do tell cxx_eval_call_expression that it doesn't need to bother trying to expand the function. */ entry.result = error_mark_node; + return register_constexpr_fundef (entry); +} + +/* BODY is a validated and massaged definition of a constexpr + function. Register it in the hash table. */ + +tree +register_constexpr_fundef (const constexpr_fundef &value) +{ + /* Create the constexpr function table if necessary. */ + if (constexpr_fundef_table == NULL) + constexpr_fundef_table + = hash_table::create_ggc (101); + + constexpr_fundef **slot = constexpr_fundef_table->find_slot + (const_cast (&value), INSERT); + gcc_assert (*slot == NULL); *slot = ggc_alloc (); - **slot = entry; + **slot = value; - return fun; + return value.decl; } /* FUN is a non-constexpr function called in a context that requires a From patchwork Tue Nov 3 21:16:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393429 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=peqaF+jI; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjJH05C2z9sfG for ; Wed, 4 Nov 2020 08:16:15 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 1947539874DB; Tue, 3 Nov 2020 21:16:13 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) by sourceware.org (Postfix) with ESMTPS id A207D3987481 for ; Tue, 3 Nov 2020 21:16:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org A207D3987481 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72a.google.com with SMTP id o205so9841387qke.10 for ; Tue, 03 Nov 2020 13:16:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=MzX2lX96N7e5DU4c8crm+p5iorn5wuzdnR+bXxmNofE=; b=peqaF+jIFt1FZCiXHWiZjuRCIaR0RJd5QjyiI4MojgVNVC66GY8GnKjY+NdseWM7YY mqbisBhuGWqMLU/3flavtWKwdECmbW+gA/zKWk20/baE55aRyh4xa617ujFdfCH6Alan 4MFmEDJdhFhlc4B+AcQwkERtuFHzLnv2Foa4IYfQyeun1E+ZGODFJsB+D9vYhFavN53T oFhox/19bPcIseSczgfQ7nbmKiYWRuJMrpiYZH576+qSktBygJfg4DBPs1PVaDs1Ix2R qEkXBd4+5E2ULBrVWJOF46lriK1PSw1qrvnDRVKUoeIarqIM+EI1Rwsb2i+TI/H2LxBZ yWsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=MzX2lX96N7e5DU4c8crm+p5iorn5wuzdnR+bXxmNofE=; b=mUF/iIfo1IflONIgm6o9hCV1aRFCO90aWOUnZjCdOUhw6jEZD/p99/5PsNnPDpRrCc K/ZBaZLFhe/6Hsv2JnpifCwKaiQlS0XkTFkON7W9rXV3iq0N/ttYp/IO0VTYeN/PpUar zG+Z/VKokrIOMXsyZInvKa+983h8KFBDFHjo4+w/d3B58I4NYDanAt+cxGbcrf3PiR4a MDcm5XdQUGoLhr/SMhvDLxPXhUa0MdfrXSPe2s2683DrluCwyL9tSCYKA+b7APGdxmtr 8lJHy30PKBQBqOHs73IvCwakZcU2i0uIGo62fMNBG+INvnCAX9Sh0tV+ZxSPAdTq9j0J Uv6Q== X-Gm-Message-State: AOAM5331OOxvxu9tq/MeRSfdVWe8zj9pcgkS2H/CijR2Iq/1zX9kQzg8 nXp3VmlALzoQmZUEq3nFnIE= X-Google-Smtp-Source: ABdhPJzqPsFK4d6himigg0k1CikXzgZDE8uBWI3aM5KvUWso5iABxnEYm1QtTrYa7tFwk7MvWP9FYQ== X-Received: by 2002:a05:620a:1375:: with SMTP id d21mr16333966qkl.309.1604438169976; Tue, 03 Nov 2020 13:16:09 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id c27sm71500qkk.57.2020.11.03.13.16.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:16:09 -0800 (PST) From: Nathan Sidwell Subject: [18/32] new C++ template bits To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> Message-ID: <28707e9f-3d66-9b8b-9341-6110243f3392@acm.org> Date: Tue, 3 Nov 2020 16:16:07 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We also need to manipulate the instantiation tables. This exposes that bit of the template machinery. (other patches address instantiations themselves) diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -5386,6 +5563,14 @@ public: hash_map *saved; }; +/* Entry in the specialization hash table. */ +struct GTY((for_user)) spec_entry +{ + tree tmpl; /* The general template this is a specialization of. */ + tree args; /* The args for this (maybe-partial) specialization. */ + tree spec; /* The specialization itself. */ +}; + /* in class.c */ extern int current_class_depth; @@ -6976,6 +7273,16 @@ extern bool copy_guide_p (const_tree); extern bool template_guide_p (const_tree); extern bool builtin_guide_p (const_tree); extern void store_explicit_specifier (tree, tree); +extern void walk_specializations (bool, + void (*)(bool, spec_entry *, + void *), + void *); +extern tree check_mergeable_specialization (bool, spec_entry *); +extern tree match_mergeable_specialization (bool is_decl, tree tmpl, + tree args, tree spec); +extern unsigned get_mergeable_specialization_flags (tree tmpl, tree spec); +extern void add_mergeable_specialization (tree tmpl, tree args, + tree spec, unsigned); extern tree add_outermost_template_args (tree, tree); extern tree add_extra_args (tree, tree); extern tree build_extra_args (tree, tree, tsubst_flags_t); diff --git c/gcc/cp/pt.c w/gcc/cp/pt.c index aa162d2a4f9..497ac5aafec 100644 --- c/gcc/cp/pt.c +++ w/gcc/cp/pt.c @@ -103,13 +106,6 @@ local_specialization_stack::~local_specialization_stack () /* True if we've recursed into fn_type_unification too many times. */ static bool excessive_deduction_depth; -struct GTY((for_user)) spec_entry -{ - tree tmpl; - tree args; - tree spec; -}; - struct spec_hasher : ggc_ptr_hash { static hashval_t hash (spec_entry *); @@ -29613,6 +29695,104 @@ declare_integer_pack (void) CP_BUILT_IN_INTEGER_PACK); } +/* Collect the specializations and explicit instantitions generated + in this module */ + +void +walk_specializations (bool decls_p, + void (*fn) (bool decls_p, spec_entry *entry, void *data), + void *data) +{ + spec_hash_table *table = decls_p ? decl_specializations + : type_specializations; + spec_hash_table::iterator end (table->end ()); + for (spec_hash_table::iterator iter (table->begin ()); iter != end; ++iter) + fn (decls_p, *iter, data); +} + +tree +check_mergeable_specialization (bool decl_p, spec_entry *elt) +{ + hash_table *specializations + = decl_p ? decl_specializations : type_specializations; + hashval_t hash = spec_hasher::hash (elt); + spec_entry **slot = specializations->find_slot_with_hash (elt, + hash, NO_INSERT); + return slot ? (*slot)->spec : NULL_TREE; +} + +/* Lookup the specialization of TMPL,ARGS in the decl or type + specialization table. Return what's there, or add SPEC and return + NULL. */ + +tree +match_mergeable_specialization (bool decl_p, tree tmpl, tree args, tree spec) +{ + gcc_checking_assert (spec); + spec_entry elt = {tmpl, args, spec}; + hash_table *specializations + = decl_p ? decl_specializations : type_specializations; + hashval_t hash = spec_hasher::hash (&elt); + spec_entry **slot = specializations->find_slot_with_hash (&elt, hash, INSERT); + spec_entry *entry = slot ? *slot: NULL; + + if (entry) + return entry->spec; + + entry = ggc_alloc (); + *entry = elt; + *slot = entry; + + return NULL_TREE; +} + +/* Return flags encoding whether SPEC is on the instantiation and/or + specialization lists of TMPL. */ + +unsigned +get_mergeable_specialization_flags (tree tmpl, tree decl) +{ + unsigned flags = 0; + + for (tree inst = DECL_TEMPLATE_INSTANTIATIONS (tmpl); + inst; inst = TREE_CHAIN (inst)) + if (TREE_VALUE (inst) == decl) + { + flags |= 1; + break; + } + + if (CLASS_TYPE_P (TREE_TYPE (decl)) + && CLASSTYPE_TEMPLATE_INFO (TREE_TYPE (decl)) + && CLASSTYPE_USE_TEMPLATE (TREE_TYPE (decl)) == 2) + /* Only need to search if DECL is a partial specialization. */ + for (tree part = DECL_TEMPLATE_SPECIALIZATIONS (tmpl); + part; part = TREE_CHAIN (part)) + if (TREE_VALUE (part) == decl) + { + flags |= 2; + break; + } + + return flags; +} + +void +add_mergeable_specialization (tree tmpl, tree args, tree decl, unsigned flags) +{ + if (flags & 1) + DECL_TEMPLATE_INSTANTIATIONS (tmpl) + = tree_cons (args, decl, DECL_TEMPLATE_INSTANTIATIONS (tmpl)); + + if (flags & 2) + { + DECL_TEMPLATE_SPECIALIZATIONS (tmpl) + = tree_cons (args, decl, DECL_TEMPLATE_SPECIALIZATIONS (tmpl)); + TREE_TYPE (DECL_TEMPLATE_SPECIALIZATIONS (tmpl)) + = TREE_TYPE (DECL_TEMPLATE_RESULT (decl)); + } +} + /* Set up the hash tables for template instantiations. */ void From patchwork Tue Nov 3 21:16:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393430 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=CDR1LaKI; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjJc6NtMz9sWn for ; Wed, 4 Nov 2020 08:16:32 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 875583987481; Tue, 3 Nov 2020 21:16:30 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf34.google.com (mail-qv1-xf34.google.com [IPv6:2607:f8b0:4864:20::f34]) by sourceware.org (Postfix) with ESMTPS id 932AA386F434 for ; Tue, 3 Nov 2020 21:16:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 932AA386F434 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf34.google.com with SMTP id d1so8056765qvl.6 for ; Tue, 03 Nov 2020 13:16:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=GMFB7rbVfFBFDrJ0YH083g9n//DEUyVQXRCFhib8xPs=; b=CDR1LaKI6+55zNYUODHANVmro2m64s1I1khF+YUE+YjMW927HT3zNlQlx78fbn/ACe peQ3zHS9UxhPQp0LkV4bzqfj5y1r2Wg51k8tHh1tNmeTN9jy4O5vHCYkJyhSJhqQeT9z X8JM16C1wGQ+TQgjTcrg4rlT32MxJ7eqXVzi0YX4ruxFiDCB8MyLbWIDYpegzyNR7cb+ oSbzRI8qpyYZo8vcrWthBhkVQ34vLUe1oxeH8DEBoSKxEX0hcbIbXOH3tuLXpBobaEXn RIsLqaqb+WUSkW0Qr3UqMkk8WPpR7AP/s6cUpyyQEh4ZwDUKXTfmQUXq9KVQ1GKcHhek oRWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=GMFB7rbVfFBFDrJ0YH083g9n//DEUyVQXRCFhib8xPs=; b=luD9WWFoDhZFOz1mdYWIwX9Np9mau2XNHD5ffqoFCyLXp12kifJk8cSM+PFWkZA9kl V/3N2iDV0llsyuT8m0Gq23QXGZtBXa+l0XowNqhXYQU63XQAYHjjX9QUVHFuR1oM4Zmz DTcGIrM7UuZ9Y4D539c88hbAALZk/E8xT5o34c4MwZ4ATDGXTiL4VDpj5SSXBFtAuYz/ ppUfE81BostMQeJFLhzdkxtUcUExNevY8wyj462Mq+DM1j8o0vBx0qKTU/PoKC9fff9K VMf+FenfG7VdKPvawpTNlQkgEYza4Dw5mLAg24IH6Jkf8rnEEXfb4N7/AB/prVNGaegc YJbA== X-Gm-Message-State: AOAM533dUTaa87jC8kzmTRUruCMnyurbEIxN05wEBtSDUIdK/nrF26Lz RsX67fBd/MLVaMJnNcH/m2I= X-Google-Smtp-Source: ABdhPJw/HHKVFq0uO+bcmJRSe00I5u8+zCvTZGa0++uy4Mbeds+Au4nRREHZ25TsPHTJdO5p0RWlzQ== X-Received: by 2002:a05:6214:174f:: with SMTP id dc15mr28206378qvb.26.1604438187806; Tue, 03 Nov 2020 13:16:27 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id a21sm11591520qkk.98.2020.11.03.13.16.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:16:26 -0800 (PST) From: Nathan Sidwell Subject: [19/32] global trees To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:16:24 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" We directly reference global trees, and expect them to be immutable. This reorders the global tree arrays, moving the mutable ones after a High Water Mark. Those after the HWM are not directly accessed in the module machinery (we'll reference by name or equivalent). diff --git c/gcc/c-family/c-common.h w/gcc/c-family/c-common.h index 18b489d55a3..eb9070b4c6c 100644 --- c/gcc/c-family/c-common.h +++ w/gcc/c-family/c-common.h @@ -357,13 +360,17 @@ enum c_tree_index CTI_DEFAULT_FUNCTION_TYPE, + CTI_NULL, + /* These are not types, but we have to look them up all the time. */ CTI_FUNCTION_NAME_DECL, CTI_PRETTY_FUNCTION_NAME_DECL, CTI_C99_FUNCTION_NAME_DECL, - CTI_SAVED_FUNCTION_NAME_DECLS, - CTI_NULL, + CTI_MODULE_HWM, + /* Below here entities change during compilation. */ + + CTI_SAVED_FUNCTION_NAME_DECLS, CTI_MAX }; diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -127,12 +127,8 @@ enum cp_tree_index CPTI_INIT_LIST_TYPE, CPTI_VTBL_TYPE, CPTI_VTBL_PTR_TYPE, - CPTI_STD, - CPTI_ABI, CPTI_GLOBAL, CPTI_GLOBAL_TYPE, - CPTI_CONST_TYPE_INFO_TYPE, - CPTI_TYPE_INFO_PTR_TYPE, CPTI_ABORT_FNDECL, CPTI_AGGR_TAG, CPTI_CONV_OP_MARKER, @@ -189,8 +185,28 @@ enum cp_tree_index CPTI_NOEXCEPT_FALSE_SPEC, CPTI_NOEXCEPT_DEFERRED_SPEC, + CPTI_NULLPTR, + CPTI_NULLPTR_TYPE, + + CPTI_ANY_TARG, + + CPTI_MODULE_HWM, + /* Nodes after here change during compilation, or should not be in + the module's global tree table. */ + + /* We must find these via the global namespace. */ + CPTI_STD, + CPTI_ABI, + + /* These are created at init time, but the library/headers provide + definitions. */ + CPTI_ALIGN_TYPE, + CPTI_CONST_TYPE_INFO_TYPE, + CPTI_TYPE_INFO_PTR_TYPE, CPTI_TERMINATE_FN, CPTI_CALL_UNEXPECTED_FN, + + /* These are lazily inited. */ CPTI_GET_EXCEPTION_PTR_FN, CPTI_BEGIN_CATCH_FN, CPTI_END_CATCH_FN, @@ -203,13 +219,6 @@ enum cp_tree_index CPTI_DSO_HANDLE, CPTI_DCAST, - CPTI_NULLPTR, - CPTI_NULLPTR_TYPE, - - CPTI_ALIGN_TYPE, - - CPTI_ANY_TARG, - CPTI_SOURCE_LOCATION_IMPL, CPTI_FALLBACK_DFLOAT32_TYPE, From patchwork Tue Nov 3 21:16:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393431 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=a1mf0sGx; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjJq1HVmz9sWt for ; Wed, 4 Nov 2020 08:16:43 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 07F8C398680B; Tue, 3 Nov 2020 21:16:41 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf2d.google.com (mail-qv1-xf2d.google.com [IPv6:2607:f8b0:4864:20::f2d]) by sourceware.org (Postfix) with ESMTPS id 1D5423986839 for ; Tue, 3 Nov 2020 21:16:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 1D5423986839 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf2d.google.com with SMTP id 63so8670186qva.7 for ; Tue, 03 Nov 2020 13:16:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=aE8oVtj+Ab7Ab8kccS7YorQzBeCrLn3oHn8cA56HvTY=; b=a1mf0sGxRKZKlbI6NWpF8m8n7JHQ02HZ3sONZuhhY1eis/EjPw37LVwylnTqQdkeGU Q94AnUC7SOhsKroEAmkYVHsVauVdElYOpUwbIwGQTi+phqAmXqIQAwPRfjbjWOWKnxZI Xc9AfaxarXCfitjWCiGPMerlj4Mg4LUGDU9bjqdc5XiIoFW97T6AFpbVbZBYshj8Wvo0 sukMojXdMGM51rLR50CDhKqEgi1CtSvRuA87cmJgR1iMtwltLhaGRdCnSPApVp78EnJP lX22ZRtN8L26c51iT9oHRpNuu6jX5KMyEfQ2Rxo1+w35JjtUfvy/IpYThrr4z5Gie1em jtJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=aE8oVtj+Ab7Ab8kccS7YorQzBeCrLn3oHn8cA56HvTY=; b=WH/KQXSZQvgjR+275rlfwHwIgi6NJR9V6gxMThNxJLWrptSKvC5sPYD1V06bCevacT 0+Cd/hKpiOh3yZ+wNVvMm5IcgyojLwLlyYJf1+138c/bW3IWwQ05zVlMQJVwuQkMHx7i 5wgURkcs6wdaeuB1/qoFtimxG6m3t+NlFZwQ8wh6BbaB+4r+WA9HbyCRAuLE0/2P2GK8 XeFoiMg2YAmfUyiqEZbdRMemAmbjPpYOM8Y9t9KN4kveftIGCjCUZFbgiaGdbKojB1oY 8Sqv+9AihLI19Sek2LzDIShg6TBHAMYVbBDZeh11jGDmDA+rOyUfTGHFLovzLzeLgyUK HDVw== X-Gm-Message-State: AOAM531d653d/69ycj+/8m/6LjJwV9H+BRLsTt0zHzS7HU0N/h27fy3b WP52cPXQK0z+EvFDPhyqjt9OEuUqJcB80g== X-Google-Smtp-Source: ABdhPJyAZhyxXKQ5PuofqSFD12RTf3AU2U8zbs/MqVtgVwrrUxhWrpXeKF9TbRtPljGTAZ/FyzIj+Q== X-Received: by 2002:a0c:fbc3:: with SMTP id n3mr28876311qvp.59.1604438197381; Tue, 03 Nov 2020 13:16:37 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 68sm11530676qkg.108.2020.11.03.13.16.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:16:36 -0800 (PST) From: Nathan Sidwell Subject: [20/32] global constructor To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> <01f091a5-cd8b-60b6-9552-2318ecd07025@acm.org> Message-ID: <9f44e989-3519-7111-e848-f426f0d79c08@acm.org> Date: Tue, 3 Nov 2020 16:16:35 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <01f091a5-cd8b-60b6-9552-2318ecd07025@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" the order of global construction is well defined in a module import graph. Each module has a globally callable idempotent constructor function. Every importer calls the ctors of its imports (I optimize that graph walk to avoid calling functions known to be called indirectly.) These are the changes to the global ctor emission. diff --git c/gcc/cp/decl2.c w/gcc/cp/decl2.c index 71107e03010..60bc6444d7c 100644 --- c/gcc/cp/decl2.c +++ w/gcc/cp/decl2.c @@ -3636,35 +3636,45 @@ generate_tls_wrapper (tree fn) static tree start_objects (int method_type, int initp) { - tree body; - tree fndecl; - char type[14]; - /* Make ctor or dtor function. METHOD_TYPE may be 'I' or 'D'. */ + int module_init = 0; - if (initp != DEFAULT_INIT_PRIORITY) + if (initp == DEFAULT_INIT_PRIORITY && method_type == 'I') + module_init = module_initializer_kind (); + + tree name = NULL_TREE; + if (module_init > 0) + name = mangle_module_global_init (0); + else { - char joiner; + char type[14]; + unsigned len = sprintf (type, "sub_%c", method_type); + if (initp != DEFAULT_INIT_PRIORITY) + { + char joiner = '_'; #ifdef JOINER - joiner = JOINER; -#else - joiner = '_'; + joiner = JOINER; #endif + type[len++] = joiner; + sprintf (type + len, "%.5u", initp); + } + name = get_file_function_name (type); + } - sprintf (type, "sub_%c%c%.5u", method_type, joiner, initp); + tree fntype = build_function_type (void_type_node, void_list_node); + tree fndecl = build_lang_decl (FUNCTION_DECL, name, fntype); + DECL_CONTEXT (fndecl) = FROB_CONTEXT (global_namespace); + if (module_init > 0) + { + SET_DECL_ASSEMBLER_NAME (fndecl, name); + TREE_PUBLIC (fndecl) = true; + determine_visibility (fndecl); } else - sprintf (type, "sub_%c", method_type); - - fndecl = build_lang_decl (FUNCTION_DECL, - get_file_function_name (type), - build_function_type_list (void_type_node, - NULL_TREE)); + TREE_PUBLIC (fndecl) = 0; start_preparsed_function (fndecl, /*attrs=*/NULL_TREE, SF_PRE_PARSED); - TREE_PUBLIC (current_function_decl) = 0; - /* Mark as artificial because it's not explicitly in the user's source code. */ DECL_ARTIFICIAL (current_function_decl) = 1; @@ -3678,7 +3688,35 @@ start_objects (int method_type, int initp) else DECL_GLOBAL_DTOR_P (current_function_decl) = 1; - body = begin_compound_stmt (BCS_FN_BODY); + tree body = begin_compound_stmt (BCS_FN_BODY); + + if (module_init > 0) + { + // 'static bool __in_chrg = false; + // if (__inchrg) return; + // __inchrg = true + tree var = build_lang_decl (VAR_DECL, in_charge_identifier, + boolean_type_node); + DECL_CONTEXT (var) = fndecl; + DECL_ARTIFICIAL (var) = true; + TREE_STATIC (var) = true; + pushdecl (var); + cp_finish_decl (var, NULL_TREE, false, NULL_TREE, 0); + + tree if_stmt = begin_if_stmt (); + finish_if_stmt_cond (var, if_stmt); + finish_return_stmt (NULL_TREE); + finish_then_clause (if_stmt); + finish_if_stmt (if_stmt); + + tree assign = build2 (MODIFY_EXPR, boolean_type_node, + var, boolean_true_node); + TREE_SIDE_EFFECTS (assign) = true; + finish_expr_stmt (assign); + } + + if (module_init) + module_add_import_initializers (); return body; } @@ -3689,11 +3727,9 @@ start_objects (int method_type, int initp) static void finish_objects (int method_type, int initp, tree body) { - tree fn; - /* Finish up. */ finish_compound_stmt (body); - fn = finish_function (/*inline_p=*/false); + tree fn = finish_function (/*inline_p=*/false); if (method_type == 'I') { @@ -4228,50 +4264,50 @@ static void generate_ctor_or_dtor_function (bool constructor_p, int priority, location_t *locus) { - char function_key; - tree fndecl; - tree body; - size_t i; - input_location = *locus; - /* ??? */ - /* Was: locus->line++; */ /* We use `I' to indicate initialization and `D' to indicate destruction. */ - function_key = constructor_p ? 'I' : 'D'; + char function_key = constructor_p ? 'I' : 'D'; /* We emit the function lazily, to avoid generating empty global constructors and destructors. */ - body = NULL_TREE; + tree body = NULL_TREE; - /* For Objective-C++, we may need to initialize metadata found in this module. - This must be done _before_ any other static initializations. */ - if (c_dialect_objc () && (priority == DEFAULT_INIT_PRIORITY) - && constructor_p && objc_static_init_needed_p ()) + if (constructor_p && priority == DEFAULT_INIT_PRIORITY) { - body = start_objects (function_key, priority); - objc_generate_static_init_call (NULL_TREE); + bool objc = c_dialect_objc () && objc_static_init_needed_p (); + + /* We may have module initialization to emit and/or insert + before other intializations. */ + if (module_initializer_kind () || objc) + body = start_objects (function_key, priority); + + /* For Objective-C++, we may need to initialize metadata found + in this module. This must be done _before_ any other static + initializations. */ + if (objc) + objc_generate_static_init_call (NULL_TREE); } /* Call the static storage duration function with appropriate arguments. */ + tree fndecl; + size_t i; FOR_EACH_VEC_SAFE_ELT (ssdf_decls, i, fndecl) { /* Calls to pure or const functions will expand to nothing. */ if (! (flags_from_decl_or_type (fndecl) & (ECF_CONST | ECF_PURE))) { - tree call; - if (! body) body = start_objects (function_key, priority); - call = cp_build_function_call_nary (fndecl, tf_warning_or_error, - build_int_cst (NULL_TREE, - constructor_p), - build_int_cst (NULL_TREE, - priority), - NULL_TREE); + tree call = cp_build_function_call_nary (fndecl, tf_warning_or_error, + build_int_cst (NULL_TREE, + constructor_p), + build_int_cst (NULL_TREE, + priority), + NULL_TREE); finish_expr_stmt (call); } } From patchwork Tue Nov 3 21:16:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393432 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=tRh7a+bA; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjJz0RFxz9sVN for ; Wed, 4 Nov 2020 08:16:51 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id D9EF83987509; Tue, 3 Nov 2020 21:16:48 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) by sourceware.org (Postfix) with ESMTPS id 0C16F3986839 for ; Tue, 3 Nov 2020 21:16:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 0C16F3986839 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72f.google.com with SMTP id r7so16653198qkf.3 for ; Tue, 03 Nov 2020 13:16:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=Rs/c3hl/yxBSa/JchBCKlDhuAZprBFPzhuUnRWxyCi0=; b=tRh7a+bAnuJpNyOFW7W0VKn+gpfPw1Vs5zdYCkd3r262SH//C4vwWCcpmaNG7wdz2r VnXcH7wBr8iM8T7LlOn2ZcD6wzqa3xIn2poYd4t+Opykr5rbBt+7Re62X/oz6wsPAiTk hTai0xJ5BGwPfkbfRwjgT3r2QWtMBQpY1A8c4EG7vJybe10XaKjUkt2u+DOwPI1/OKJJ 1I2/trWBxW3WWWOMH8PpbEb0qeMDypf+JGKgaJhuYbEE5jpPLPHtDHYoyGUlRIfAp6pe H/pkn6xpBg4CzbNfRu4KC+pyaKaaFR1R56TXR5EJOAPMEppGgNOo7kP8oXXzFKAz1WgY UYkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=Rs/c3hl/yxBSa/JchBCKlDhuAZprBFPzhuUnRWxyCi0=; b=rjrFe0/G1DzaiQ2sevCFkN07vYm0IlJiq3okFEF63yELFzYe6116ASGIQPrQEPRk8l piDVbdve0c5npGQtTdng3Vd+taT6oyIegl53EnmKRHprvme/k2HxDIwvyaCBmsBm4Qpk MDJ3e5JdhROcAX+7cWDJTnexmwlgJovjUnwXgLLCkMfy1Cqk6npCQ5H7vxf9iDNgMWHT rLgKtrOqMEp8BXS6YKdsSlcM/od3JrgUDcYHCy5vVaozzg5hjsxhb/N2iphN9AJSYAZq 0RzF1Rw/0ZZBnE2t4AB5oJzbXFdIkFKuKfC4coJxahaM+/D1H6WB70HyUAYYUoiGPRAq Psjg== X-Gm-Message-State: AOAM533oPlumooIZD3C8nsHyAO7bfBmArbqKkk69caln1jsNJ5OltYP/ uAy/pS+/xWjF9ZhIwLzruIc= X-Google-Smtp-Source: ABdhPJxmodSwXYSLkhfSvIigXhz6cHWXUAGFreicXEwyDCD0739J7WDzThIs37Q2C04iyOOuoKhJQA== X-Received: by 2002:ae9:efd5:: with SMTP id d204mr21105636qkg.158.1604438205279; Tue, 03 Nov 2020 13:16:45 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id x21sm12207357qkb.78.2020.11.03.13.16.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:16:44 -0800 (PST) From: Nathan Sidwell Subject: [21/32] miscelaneous To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> <01f091a5-cd8b-60b6-9552-2318ecd07025@acm.org> Message-ID: Date: Tue, 3 Nov 2020 16:16:43 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" These are changes to gcc/tree.h adding some raw accessors to nodes, which seemed preferable to direct field access. I also needed access to the integral constant cache diff --git c/gcc/tree.h w/gcc/tree.h index 7f0aa5b8d1d..13062f52919 100644 --- c/gcc/tree.h +++ w/gcc/tree.h @@ -2521,25 +2521,28 @@ extern tree vector_element_bits_tree (const_tree); #define DECL_SIZE(NODE) (DECL_COMMON_CHECK (NODE)->decl_common.size) /* Likewise for the size in bytes. */ #define DECL_SIZE_UNIT(NODE) (DECL_COMMON_CHECK (NODE)->decl_common.size_unit) +#define DECL_ALIGN_RAW(NODE) (DECL_COMMON_CHECK (NODE)->decl_common.align) /* Returns the alignment required for the datum, in bits. It must be a power of two, but an "alignment" of zero is supported (e.g. as "uninitialized" sentinel). */ -#define DECL_ALIGN(NODE) \ - (DECL_COMMON_CHECK (NODE)->decl_common.align \ - ? ((unsigned)1) << ((NODE)->decl_common.align - 1) : 0) +#define DECL_ALIGN(NODE) \ + (DECL_ALIGN_RAW (NODE) \ + ? ((unsigned)1) << (DECL_ALIGN_RAW (NODE) - 1) : 0) /* Specify that DECL_ALIGN(NODE) is X. */ #define SET_DECL_ALIGN(NODE, X) \ - (DECL_COMMON_CHECK (NODE)->decl_common.align = ffs_hwi (X)) + (DECL_ALIGN_RAW (NODE) = ffs_hwi (X)) /* The minimum alignment necessary for the datum, in bits, without warning. */ -#define DECL_WARN_IF_NOT_ALIGN(NODE) \ - (DECL_COMMON_CHECK (NODE)->decl_common.warn_if_not_align \ - ? ((unsigned)1) << ((NODE)->decl_common.warn_if_not_align - 1) : 0) +#define DECL_WARN_IF_NOT_ALIGN_RAW(NODE) \ + (DECL_COMMON_CHECK (NODE)->decl_common.warn_if_not_align) +#define DECL_WARN_IF_NOT_ALIGN(NODE) \ + (DECL_WARN_IF_NOT_ALIGN_RAW (NODE) \ + ? ((unsigned)1) << (DECL_WARN_IF_NOT_ALIGN_RAW (NODE) - 1) : 0) /* Specify that DECL_WARN_IF_NOT_ALIGN(NODE) is X. */ -#define SET_DECL_WARN_IF_NOT_ALIGN(NODE, X) \ - (DECL_COMMON_CHECK (NODE)->decl_common.warn_if_not_align = ffs_hwi (X)) +#define SET_DECL_WARN_IF_NOT_ALIGN(NODE, X) \ + (DECL_WARN_IF_NOT_ALIGN_RAW (NODE) = ffs_hwi (X)) /* The alignment of NODE, in bytes. */ #define DECL_ALIGN_UNIT(NODE) (DECL_ALIGN (NODE) / BITS_PER_UNIT) @@ -5118,7 +5121,7 @@ extern const_tree strip_invariant_refs (const_tree); extern tree lhd_gcc_personality (void); extern void assign_assembler_name_if_needed (tree); extern bool warn_deprecated_use (tree, tree); -extern void cache_integer_cst (tree); +extern tree cache_integer_cst (tree, bool small = false); extern const char *combined_fn_name (combined_fn); /* Compare and hash for any structure which begins with a canonical diff --git c/gcc/tree.c w/gcc/tree.c index 9260772b846..2656a804ea2 100644 --- c/gcc/tree.c +++ w/gcc/tree.c @@ -1727,8 +1727,8 @@ wide_int_to_tree (tree type, const poly_wide_int_ref &value) return build_poly_int_cst (type, value); } -void -cache_integer_cst (tree t) +tree +cache_integer_cst (tree t, bool replace) { tree type = TREE_TYPE (t); int ix = -1; @@ -1822,8 +1822,13 @@ cache_integer_cst (tree t) TYPE_CACHED_VALUES (type) = make_tree_vec (limit); } - gcc_assert (TREE_VEC_ELT (TYPE_CACHED_VALUES (type), ix) == NULL_TREE); - TREE_VEC_ELT (TYPE_CACHED_VALUES (type), ix) = t; + if (tree r = TREE_VEC_ELT (TYPE_CACHED_VALUES (type), ix)) + { + gcc_assert (replace); + t = r; + } + else + TREE_VEC_ELT (TYPE_CACHED_VALUES (type), ix) = t; } else { @@ -1831,12 +1836,18 @@ cache_integer_cst (tree t) tree *slot = int_cst_hash_table->find_slot (t, INSERT); /* If there is already an entry for the number verify it's the same. */ - if (*slot) - gcc_assert (wi::to_wide (tree (*slot)) == wi::to_wide (t)); + if (tree r = *slot) + { + gcc_assert (wi::to_wide (tree (r)) == wi::to_wide (t)); + if (replace) + t = r; + } else /* Otherwise insert this one into the hash table. */ *slot = t; } + + return t; } From patchwork Tue Nov 3 21:16:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393433 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=WtnLTBXZ; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjKJ5J53z9sVk for ; Wed, 4 Nov 2020 08:17:08 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5C9FB3987506; Tue, 3 Nov 2020 21:17:06 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf30.google.com (mail-qv1-xf30.google.com [IPv6:2607:f8b0:4864:20::f30]) by sourceware.org (Postfix) with ESMTPS id 36C8C39874F5 for ; Tue, 3 Nov 2020 21:17:00 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 36C8C39874F5 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf30.google.com with SMTP id ed14so5105981qvb.4 for ; Tue, 03 Nov 2020 13:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=+6ryLV8U3lC3E/Ad53uJB89ZOzwmPVm/GDaxFsVvpOA=; b=WtnLTBXZMLwpF+K5l80ikQwD35zEjDsHrWK2EjjTVvg2v3IGbwL7mBc0I8dzkoJfU8 Cw0WwT/VA0jC+DyaBVnNZk96VFj7zYjM+xIAJZu7i9U007LwyjKnzSJraGVHE7Uscg15 b2LumQwAbA522sk1Cs6BJfPc3RVKbeEHAhPbXgX3l5zP4mMJIU7doqqm+cTTeb9X5/i7 QJxipeuTjmPYpCVnFpPD7VguKcFb3EW5e/dxEvxF/4sUP4WX7zzW5wzQ5D5LA3BXckcN x7u51sHcc6niAbO8rRxNSDCZoxzIuTZQFqX6MzitcNjyW0WNJUCyv8wFNnrFLINL7e/U q/cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=+6ryLV8U3lC3E/Ad53uJB89ZOzwmPVm/GDaxFsVvpOA=; b=ljVz2FbWlTkA3UKiGnifpqQpEFhGUvZn30IsgxVEEr3+32p2gqBjTt5b3hhY8DubkO dQVx16J9Aad3dz+54nl5clSwqpIyuz+bwbp16GeSsYmS8mmZTZPlW+HATtA45/5MD6hn Tw7gbokN9nY20ohowN0/6uOr82Fw7Z5MEv89OE5erJ1Er0i1yIxnVN7oiwrCFXQWnJxI 8yHzJZ8/6IWBgdJt5/f0qpU99vhZV9PMtAWdiqKDq2MjjsoidkN/7SxKlPsPZq3kmRpb U+etKscHpBC1TiqqWb0LhfS+KiDGmns5BLCQLTAcAV3AsgUDhMq2omIOctcjo+/SK9zq GvUQ== X-Gm-Message-State: AOAM531JGcP4Epc29mQt2tVmBF0tqW/Bg1c6FgXonldgDGW6OMFEUxyq s5qBObWDbJtV4GB+2n9zWxV862frbWfyXQ== X-Google-Smtp-Source: ABdhPJw8bujj61JBWkyFC2gSnUZh8CtXh/+VEhZd6Zl9nwhNgl27PjsILdY3/fAxqy0KRx/ZIK3gSA== X-Received: by 2002:ad4:4142:: with SMTP id z2mr29632563qvp.20.1604438219493; Tue, 03 Nov 2020 13:16:59 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id 19sm10938464qkj.69.2020.11.03.13.16.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:16:58 -0800 (PST) From: Nathan Sidwell Subject: [22/32] miscelaneous c++ bits To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <0bdf00a8-d8ad-9e97-134d-6668c0e8c86b@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> <01f091a5-cd8b-60b6-9552-2318ecd07025@acm.org> <5c533ebe-440d-188e-5bdb-38c14898852c@acm.org> Message-ID: <10c2ff88-0c67-8557-6c06-bf822f675c59@acm.org> Date: Tue, 3 Nov 2020 16:16:57 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <5c533ebe-440d-188e-5bdb-38c14898852c@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" This is probably the messiest diff. It's the remaining diff of cp-tree.h. 1) a new tree type MODULE_VECTOR. This is a sparse array used for name lookup. A namespace symbol table entry may contain one of these, which holds the bindings for each loaded module. If a module doesn't bind that name, there'll be no slot for it. The current TU always uses slot 0. The Global Module uses slot 1, and in a named-module partitions share slot 2. Slots 1 & 2 are used for duplicate declaration matching. I steal more flags from lang-decl-base. Firstly, the selector is reduced from 16 to 3 bits. We add a bunch of module-related flags module_purview_p -- the declaration is owned by a module module_import_p -- the declaration is from an import module_entity_p -- the declaration is in the imported entity hash & array module_pending_specializations_p - in a template we know there are specializations we've not loaded module_pending_members_p - in a class we know there are members to be loaded attached_decls_p -- this decl is in the attached-decls hash, and therefore has attached decls (to do with mangling lambdas initializing namespace-scope maybe-template variables) It sprinkles set_originating_module and set_instantiating_module around a few places. The originating module of a decl is the module that declared the decl. The instantiating module is the module that caused code to be emitted for it. Often these two modules are the same, but for templates they can of course be different. diff --git c/gcc/cp/cp-tree.def w/gcc/cp/cp-tree.def index a188576013b..ba02cdc2748 100644 --- c/gcc/cp/cp-tree.def +++ w/gcc/cp/cp-tree.def @@ -233,6 +233,9 @@ DEFTREECODE (TEMPLATE_ID_EXPR, "template_id_expr", tcc_expression, 2) /* One of a set of overloaded functions. */ DEFTREECODE (OVERLOAD, "overload", tcc_exceptional, 0) +/* A vector of module overloads. */ +DEFTREECODE (MODULE_VECTOR, "module_vector", tcc_exceptional, 0) + /* A pseudo-destructor, of the form "OBJECT.~DESTRUCTOR" or "OBJECT.SCOPE::~DESTRUCTOR. The first operand is the OBJECT. The second operand (if non-NULL) is the SCOPE. The third operand is diff --git c/gcc/cp/cp-tree.h w/gcc/cp/cp-tree.h index fdb8ee57f0b..e8e4d0af2d8 100644 --- c/gcc/cp/cp-tree.h +++ w/gcc/cp/cp-tree.h @@ -475,20 +484,22 @@ extern GTY(()) tree cp_global_trees[CPTI_MAX]; CALL_EXPR_ORDERED_ARGS (in CALL_EXPR, AGGR_INIT_EXPR) DECLTYPE_FOR_REF_CAPTURE (in DECLTYPE_TYPE) CONSTRUCTOR_C99_COMPOUND_LITERAL (in CONSTRUCTOR) + DECL_MODULE_EXPORT_P (in _DECL) OVL_NESTED_P (in OVERLOAD) LAMBDA_EXPR_INSTANTIATED (in LAMBDA_EXPR) Reserved for DECL_MODULE_EXPORT (in DECL_) 4: IDENTIFIER_MARKED (IDENTIFIER_NODEs) TREE_HAS_CONSTRUCTOR (in INDIRECT_REF, SAVE_EXPR, CONSTRUCTOR, CALL_EXPR, or FIELD_DECL). - DECL_TINFO_P (in VAR_DECL) + DECL_TINFO_P (in VAR_DECL, TYPE_DECL) FUNCTION_REF_QUALIFIED (in FUNCTION_TYPE, METHOD_TYPE) OVL_LOOKUP_P (in OVERLOAD) - LOOKUP_FOUND_P (in RECORD_TYPE, UNION_TYPE, NAMESPACE_DECL) + LOOKUP_FOUND_P (in RECORD_TYPE, UNION_TYPE, ENUMERAL_TYPE, NAMESPACE_DECL) 5: IDENTIFIER_VIRTUAL_P (in IDENTIFIER_NODE) FUNCTION_RVALUE_QUALIFIED (in FUNCTION_TYPE, METHOD_TYPE) CALL_EXPR_REVERSE_ARGS (in CALL_EXPR, AGGR_INIT_EXPR) CONSTRUCTOR_PLACEHOLDER_BOUNDARY (in CONSTRUCTOR) + OVL_EXPORT_P (in OVL_USING_P OVERLOAD) 6: TYPE_MARKED_P (in _TYPE) DECL_NONTRIVIALLY_INITIALIZED_P (in VAR_DECL) RANGE_FOR_IVDEP (in RANGE_FOR_STMT) @@ -530,6 +541,7 @@ extern GTY(()) tree cp_global_trees[CPTI_MAX]; DECL_ANON_UNION_VAR_P (in a VAR_DECL) DECL_SELF_REFERENCE_P (in a TYPE_DECL) DECL_INVALID_OVERRIDER_P (in a FUNCTION_DECL) + DECL_UNINSTANIATED_TEMPLATE_FRIEND_P (in TEMPLATE_DECL) 5: DECL_INTERFACE_KNOWN. 6: DECL_THIS_STATIC (in VAR_DECL, FUNCTION_DECL or PARM_DECL) DECL_FIELD_IS_BASE (in FIELD_DECL) @@ -744,7 +756,8 @@ typedef struct ptrmem_cst * ptrmem_cst_t; /* Lookup walker marking. */ #define LOOKUP_SEEN_P(NODE) TREE_VISITED(NODE) #define LOOKUP_FOUND_P(NODE) \ - TREE_LANG_FLAG_4 (TREE_CHECK3(NODE,RECORD_TYPE,UNION_TYPE,NAMESPACE_DECL)) + TREE_LANG_FLAG_4 (TREE_CHECK4(NODE,RECORD_TYPE,UNION_TYPE,ENUMERAL_TYPE, \ + NAMESPACE_DECL)) /* These two accessors should only be used by OVL manipulators. Other users should use iterators and convenience functions. */ @@ -763,6 +776,8 @@ typedef struct ptrmem_cst * ptrmem_cst_t; #define OVL_NESTED_P(NODE) TREE_LANG_FLAG_3 (OVERLOAD_CHECK (NODE)) /* If set, this overload was constructed during lookup. */ #define OVL_LOOKUP_P(NODE) TREE_LANG_FLAG_4 (OVERLOAD_CHECK (NODE)) +/* If set, this OVL_USING_P overload is exported. */ +#define OVL_EXPORT_P(NODE) TREE_LANG_FLAG_5 (OVERLOAD_CHECK (NODE)) /* The first decl of an overload. */ #define OVL_FIRST(NODE) ovl_first (NODE) @@ -822,6 +837,11 @@ class ovl_iterator { return fn; } + tree get_using () const + { + gcc_checking_assert (using_p ()); + return ovl; + } public: /* Whether this overload was introduced by a using decl. */ @@ -830,10 +850,21 @@ class ovl_iterator { return (TREE_CODE (ovl) == USING_DECL || (TREE_CODE (ovl) == OVERLOAD && OVL_USING_P (ovl))); } + /* Whether this using is being exported. */ + bool exporting_p () const + { + return OVL_EXPORT_P (get_using ()); + } + bool hidden_p () const { return TREE_CODE (ovl) == OVERLOAD && OVL_HIDDEN_P (ovl); } + void set_dedup () + { + if (TREE_CODE (ovl) == OVERLOAD) + OVL_DEDUP_P (ovl) = true; + } public: tree remove_node (tree head) @@ -924,6 +955,84 @@ struct named_decl_hash : ggc_remove { static void mark_deleted (value_type) { gcc_unreachable (); } }; +/* Bindings for modules are held in a sparse array. There is always a + current TU slot, others are allocated as needed. By construction + of the importing mechanism we only ever need to append to the + array. Rather than have straight index/slot tuples, we bunch them + up for greater packing. + + The cluster representation packs well on a 64-bit system. */ + +#define MODULE_VECTOR_SLOTS_PER_CLUSTER 2 +struct mc_index { + unsigned short base; + unsigned short span; +}; + +struct GTY(()) module_cluster +{ + mc_index GTY((skip)) indices[MODULE_VECTOR_SLOTS_PER_CLUSTER]; + mc_slot slots[MODULE_VECTOR_SLOTS_PER_CLUSTER]; +}; + +/* These two fields overlay lang flags. So don't use those. */ +#define MODULE_VECTOR_ALLOC_CLUSTERS(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.u.dependence_info.clique) +#define MODULE_VECTOR_NUM_CLUSTERS(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.u.dependence_info.base) +#define MODULE_VECTOR_CLUSTER_BASE(NODE) \ + (((tree_module_vec *)MODULE_VECTOR_CHECK (NODE))->vec) +#define MODULE_VECTOR_CLUSTER_LAST(NODE) \ + (&MODULE_VECTOR_CLUSTER (NODE, MODULE_VECTOR_NUM_CLUSTERS (NODE) - 1)) +#define MODULE_VECTOR_CLUSTER(NODE,IX) \ + (((tree_module_vec *)MODULE_VECTOR_CHECK (NODE))->vec[IX]) + +struct GTY(()) tree_module_vec { + struct tree_base base; + tree name; + module_cluster GTY((length ("%h.base.u.dependence_info.base"))) vec[1]; +}; + +/* The name of a module vector. */ +#define MODULE_VECTOR_NAME(NODE) \ + (((tree_module_vec *)MODULE_VECTOR_CHECK (NODE))->name) + +/* tree_module_vec does uses base.u.dependence_info.base field for + length. It does not have lang_flag etc available! */ + +/* These two flags note if a module-vector contains deduplicated + bindings (i.e. multiple declarations in different imports). */ +/* This binding contains duplicate references to a global module + entity. */ +#define MODULE_VECTOR_GLOBAL_DUPS_P(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.static_flag) +/* This binding contains duplicate references to a partioned module + entity. */ +#define MODULE_VECTOR_PARTITION_DUPS_P(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.volatile_flag) + +/* These two flags indicate the provenence of the bindings on this + particular vector slot. We can of course determine this from slot + number, but that's a relatively expensive lookup. This avoids + that when iterating. */ +/* This slot is part of the global module (a header unit). */ +#define MODULE_BINDING_GLOBAL_P(NODE) \ + (OVERLOAD_CHECK (NODE)->base.static_flag) +/* This slot is part of the current module (a partition or primary). */ +#define MODULE_BINDING_PARTITION_P(NODE) \ + (OVERLOAD_CHECK (NODE)->base.volatile_flag) + +/* There are specializations of a template keyed to this binding. */ +#define MODULE_VECTOR_PENDING_SPECIALIZATIONS_P(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.public_flag) +/* The key is in a header unit (not a named module partition or + primary). */ +#define MODULE_VECTOR_PENDING_IS_HEADER_P(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.protected_flag) +/* The key is in a named module (primary or partition). */ +#define MODULE_VECTOR_PENDING_IS_PARTITION_P(NODE) \ + (MODULE_VECTOR_CHECK (NODE)->base.private_flag) + /* Simplified unique_ptr clone to release a tree vec on exit. */ class releasing_vec @@ -1623,6 +1732,48 @@ check_constraint_info (tree t) #define CONSTRAINED_PARM_PROTOTYPE(NODE) \ DECL_INITIAL (TYPE_DECL_CHECK (NODE)) +/* Module defines. */ +// Too many _DECLS: FUNCTION,VAR,TYPE,TEMPLATE,CONCEPT or NAMESPACE +#define DECL_MODULE_CHECK(NODE) (NODE) + +/* In the purview of a module (including header unit). */ +#define DECL_MODULE_PURVIEW_P(N) \ + (DECL_LANG_SPECIFIC (DECL_MODULE_CHECK (N))->u.base.module_purview_p) + +/* True if the live version of the decl was imported. */ +#define DECL_MODULE_IMPORT_P(NODE) \ + (DECL_LANG_SPECIFIC (DECL_MODULE_CHECK (NODE))->u.base.module_import_p) + +/* True if this decl is in the entity hash & array. This means that + some variant was imported, even if DECL_MODULE_IMPORT_P is false. */ +#define DECL_MODULE_ENTITY_P(NODE) \ + (DECL_LANG_SPECIFIC (DECL_MODULE_CHECK (NODE))->u.base.module_entity_p) + +/* True if there are unloaded specializations keyed to this template. */ +#define DECL_MODULE_PENDING_SPECIALIZATIONS_P(NODE) \ + (DECL_LANG_SPECIFIC (TEMPLATE_DECL_CHECK (NODE)) \ + ->u.base.module_pending_specializations_p) + +/* True if this class has unloaded members. These should be loaded + before we do member lookups. While this may be better on the + classtype, I think we can get this behaviour for enums too. But + perhaps those need to be immediately loaded? (Particularly if + unscoped). */ +#define DECL_MODULE_PENDING_MEMBERS_P(NODE) \ + (DECL_LANG_SPECIFIC (TYPE_DECL_CHECK (NODE)) \ + ->u.base.module_pending_members_p) + +/* Whether this is an exported DECL. Held on any decl that can appear + at namespace scope (function, var, type, template, const or + namespace). templates copy from their template_result, consts have + it for unscoped enums. */ +#define DECL_MODULE_EXPORT_P(NODE) TREE_LANG_FLAG_3 (NODE) + +/* DECL that has attached decls for ODR-relatedness. */ +#define DECL_ATTACHED_DECLS_P(NODE) \ + (DECL_LANG_SPECIFIC (NODE)->u.base.attached_decls_p) + + /* The list of local parameters introduced by this requires-expression, in the form of a chain of PARM_DECLs. */ #define REQUIRES_EXPR_PARMS(NODE) \ @@ -1644,6 +1795,7 @@ enum cp_tree_node_structure_enum { TS_CP_TPI, TS_CP_PTRMEM, TS_CP_OVERLOAD, + TS_CP_MODULE_VECTOR, TS_CP_BASELINK, TS_CP_TEMPLATE_DECL, TS_CP_DEFERRED_PARSE, @@ -1665,6 +1817,7 @@ union GTY((desc ("cp_tree_node_structure (&%h)"), struct template_parm_index GTY ((tag ("TS_CP_TPI"))) tpi; struct ptrmem_cst GTY ((tag ("TS_CP_PTRMEM"))) ptrmem; struct tree_overload GTY ((tag ("TS_CP_OVERLOAD"))) overload; + struct tree_module_vec GTY ((tag ("TS_CP_MODULE_VECTOR"))) module_vec; struct tree_baselink GTY ((tag ("TS_CP_BASELINK"))) baselink; struct tree_template_decl GTY ((tag ("TS_CP_TEMPLATE_DECL"))) template_decl; struct tree_deferred_parse GTY ((tag ("TS_CP_DEFERRED_PARSE"))) deferred_parse; @@ -2645,15 +2799,15 @@ enum lang_decl_selector /* Flags shared by all forms of DECL_LANG_SPECIFIC. Some of the flags live here only to make lang_decl_min/fn smaller. Do - not make this struct larger than 32 bits; instead, make sel smaller. */ + not make this struct larger than 32 bits. */ struct GTY(()) lang_decl_base { - /* Larger than necessary for faster access. */ - ENUM_BITFIELD(lang_decl_selector) selector : 16; + ENUM_BITFIELD(lang_decl_selector) selector : 3; ENUM_BITFIELD(languages) language : 1; unsigned use_template : 2; unsigned not_really_extern : 1; /* var or fn */ unsigned initialized_in_class : 1; /* var or fn */ + unsigned threadprivate_or_deleted_p : 1; /* var or fn */ /* anticipated_p is no longer used for anticipated_decls (fn, type or template). It is used as DECL_OMP_PRIVATIZED_MEMBER in @@ -2662,11 +2816,22 @@ struct GTY(()) lang_decl_base { unsigned friend_or_tls : 1; /* var, fn, type or template */ unsigned unknown_bound_p : 1; /* var */ unsigned odr_used : 1; /* var or fn */ - unsigned spare : 1; unsigned concept_p : 1; /* applies to vars and functions */ unsigned var_declared_inline_p : 1; /* var */ unsigned dependent_init_p : 1; /* var */ - /* 2 spare bits */ + + unsigned module_purview_p : 1; /* in module purview (not GMF) */ + unsigned module_import_p : 1; /* from an import */ + unsigned module_entity_p : 1; /* is in the entitity ary & + hash. */ + /* Has specializations or members yet to load. */ + unsigned module_pending_specializations_p : 1; + unsigned module_pending_members_p : 1; + + /* Is in the decl-attached hash table, (with attached decls). */ + unsigned attached_decls_p : 1; + + /* 10 spare bits. */ }; /* True for DECL codes which have template info and access. */ @@ -3099,6 +3264,13 @@ struct GTY(()) lang_decl { (DECL_LANG_SPECIFIC (FUNCTION_DECL_CHECK (NODE)) \ ->u.base.friend_or_tls) +/* True of a TEMPLATE_DECL that is a template class friend. Such + decls are not pushed until instantiated (as they may depend on + parameters of the befriending class). DECL_CHAIN is the + befriending class. */ +#define DECL_UNINSTANTIATED_TEMPLATE_FRIEND_P(NODE) \ + (DECL_LANG_FLAG_4 (TEMPLATE_DECL_CHECK (NODE))) + /* Nonzero if the thread-local variable was declared with __thread as opposed to thread_local. */ #define DECL_GNU_TLS_P(NODE) \ @@ -3340,7 +3512,8 @@ struct GTY(()) lang_decl { /* 1 iff VAR_DECL node NODE is a type-info decl. This flag is set for both the primary typeinfo object and the associated NTBS name. */ -#define DECL_TINFO_P(NODE) TREE_LANG_FLAG_4 (VAR_DECL_CHECK (NODE)) +#define DECL_TINFO_P(NODE) \ + TREE_LANG_FLAG_4 (TREE_CHECK2 (NODE,VAR_DECL,TYPE_DECL)) /* 1 iff VAR_DECL node NODE is virtual table or VTT. We forward to DECL_VIRTUAL_P from the common code, as that has the semantics we @@ -6092,6 +6277,13 @@ struct GTY((chain_next ("%h.next"))) tinst_level { arguments. */ tree tldcl, targs; + /* For modules we need to know (a) the modules on the path of + instantiation and (b) the transitive imports along that path. + Note that these two bitmaps may be inherited from NEXT, if this + decl is in the same module as NEXT (or has no new information). */ + bitmap path; + bitmap visible; + private: /* Return TRUE iff the original node is a split list. */ bool split_list_p () const { return targs; } @@ -6187,6 +6379,7 @@ extern cp_parameter_declarator *no_parameters; /* Various dump ids. */ extern int class_dump_id; +extern int module_dump_id; extern int raw_dump_id; /* in call.c */ @@ -6393,6 +6586,7 @@ extern void check_abi_tags (tree); extern tree missing_abi_tags (tree); extern void fixup_type_variants (tree); extern void fixup_attribute_variants (tree); +extern void build_cdtor_clones (tree, bool, bool, bool); extern void clone_cdtor (tree, bool); extern tree copy_operator_fn (tree, tree_code code); extern void adjust_clone_args (tree); @@ -6775,6 +6977,100 @@ extern bool ctor_omit_inherited_parms (tree); extern tree locate_ctor (tree); extern tree implicitly_declare_fn (special_function_kind, tree, bool, tree, tree); +/* In module.cc */ +class module_state; /* Forward declare. */ +inline bool modules_p () { return flag_modules != 0; } + +#define MK_MODULE (1 << 0) /* This TU is a module. */ +#define MK_GLOBAL (1 << 1) /* Entities are in the global module. */ +#define MK_INTERFACE (1 << 2) /* This TU is an interface. */ +#define MK_PARTITION (1 << 3) /* This TU is a partition. */ +#define MK_EXPORTING (1 << 4) /* We are in an export region. */ +extern unsigned module_kind; + +/* MK_MODULE & MK_GLOBAL have the following combined meanings: + MODULE GLOBAL + 0 0 not a module + 0 1 GMF of named module (we've not yet seen module-decl) + 1 0 purview of named module + 1 1 header unit. */ + +inline bool module_purview_p () +{ return module_kind & MK_MODULE; } +inline bool global_purview_p () +{ return module_kind & MK_GLOBAL; } + +inline bool not_module_p () +{ return (module_kind & (MK_MODULE | MK_GLOBAL)) == 0; } +inline bool named_module_p () +{ /* This is a named module if exactly one of MODULE and GLOBAL is + set. */ + /* The divides are constant shifts! */ + return ((module_kind / MK_MODULE) ^ (module_kind / MK_GLOBAL)) & 1; +} +inline bool header_module_p () +{ return (module_kind & (MK_MODULE | MK_GLOBAL)) == (MK_MODULE | MK_GLOBAL); } +inline bool named_module_purview_p () +{ return (module_kind & (MK_MODULE | MK_GLOBAL)) == MK_MODULE; } +inline bool module_interface_p () +{ return module_kind & MK_INTERFACE; } +inline bool module_partition_p () +{ return module_kind & MK_PARTITION; } +inline bool module_has_cmi_p () +{ return module_kind & (MK_INTERFACE | MK_PARTITION); } + +/* We're currently exporting declarations. */ +inline bool module_exporting_p () +{ return module_kind & MK_EXPORTING; } + +extern module_state *get_module (tree name, module_state *parent = NULL, + bool partition = false); +extern bool module_may_redeclare (tree decl); + +extern int module_initializer_kind (); +extern void module_add_import_initializers (); + +/* Where the namespace-scope decl was originally declared. */ +extern void set_originating_module (tree, bool friend_p = false); +extern tree get_originating_module_decl (tree) ATTRIBUTE_PURE; +extern int get_originating_module (tree, bool for_mangle = false) ATTRIBUTE_PURE; +extern unsigned get_importing_module (tree, bool = false) ATTRIBUTE_PURE; + +/* Where current instance of the decl got declared/defined/instantiated. */ +extern void set_instantiating_module (tree); +extern void set_defining_module (tree); +extern void maybe_attach_decl (tree ctx, tree decl); + +extern void mangle_module (int m, bool include_partition); +extern void mangle_module_fini (); +extern void lazy_load_binding (unsigned mod, tree ns, tree id, mc_slot *mslot); +extern void lazy_load_specializations (tree tmpl); +extern void lazy_load_members (tree decl); +extern bool lazy_specializations_p (unsigned, bool, bool); +extern module_state *preprocess_module (module_state *, location_t, + bool in_purview, + bool is_import, bool export_p, + cpp_reader *reader); +extern void preprocessed_module (cpp_reader *reader); +extern void import_module (module_state *, location_t, bool export_p, + tree attr, cpp_reader *); +extern void declare_module (module_state *, location_t, bool export_p, + tree attr, cpp_reader *); + +extern void module_cpp_undef (cpp_reader *, location_t, cpp_hashnode *); +extern cpp_macro *module_cpp_deferred_macro (cpp_reader *, + location_t, cpp_hashnode *); +extern void init_modules (cpp_reader *); +extern void fini_modules (); +extern void maybe_check_all_macros (cpp_reader *); +extern void finish_module_processing (cpp_reader *); +extern char const *module_name (unsigned, bool header_ok); +extern bitmap get_import_bitmap (); +extern bitmap module_visible_instantiation_path (bitmap *); +extern void module_begin_main_file (cpp_reader *, line_maps *, + const line_map_ordinary *); +extern void module_preprocess_options (cpp_reader *); +extern bool handle_module_option (unsigned opt, const char *arg, int value); /* In optimize.c */ extern bool maybe_clone_body (tree); @@ -7351,6 +7661,7 @@ extern tree hash_tree_cons (tree, tree, tree); extern tree hash_tree_chain (tree, tree); extern tree build_qualified_name (tree, tree, tree, bool); extern tree build_ref_qualified_type (tree, cp_ref_qualifier); +extern tree make_module_vec (tree, unsigned clusters); inline tree ovl_first (tree) ATTRIBUTE_PURE; extern tree ovl_make (tree fn, tree next = NULL_TREE); @@ -7690,6 +8002,9 @@ extern tree mangle_template_parm_object (tree); extern char *get_mangled_vtable_map_var_name (tree); extern bool mangle_return_type_p (tree); extern tree mangle_decomp (tree, vec &); +extern void mangle_module_substitution (int); +extern void mangle_identifier (char, tree); +extern tree mangle_module_global_init (int); /* in dump.c */ extern bool cp_dump_tree (void *, tree); @@ -7959,14 +8285,16 @@ type_unknown_p (const_tree expr) inline hashval_t named_decl_hash::hash (const value_type decl) { - tree name = OVL_NAME (decl); + tree name = (TREE_CODE (decl) == MODULE_VECTOR + ? MODULE_VECTOR_NAME (decl) : OVL_NAME (decl)); return name ? IDENTIFIER_HASH_VALUE (name) : 0; } inline bool named_decl_hash::equal (const value_type existing, compare_type candidate) { - tree name = OVL_NAME (existing); + tree name = (TREE_CODE (existing) == MODULE_VECTOR + ? MODULE_VECTOR_NAME (existing) : OVL_NAME (existing)); return candidate == name; } diff --git c/gcc/cp/decl.c w/gcc/cp/decl.c index 39f56b81275..1ae696a94ad 100644 --- c/gcc/cp/decl.c +++ w/gcc/cp/decl.c @@ -2008,6 +2008,39 @@ duplicate_decls (tree newdecl, tree olddecl, bool hiding, bool was_hidden) if (!validate_constexpr_redeclaration (olddecl, newdecl)) return error_mark_node; + if (modules_p () + && TREE_CODE (CP_DECL_CONTEXT (olddecl)) == NAMESPACE_DECL + && TREE_CODE (olddecl) != NAMESPACE_DECL + && !hiding) + { + if (DECL_ARTIFICIAL (olddecl)) + { + gcc_checking_assert (!(DECL_LANG_SPECIFIC (olddecl) + && DECL_MODULE_IMPORT_P (olddecl))); + if (!(global_purview_p () || not_module_p ())) + error ("declaration %qD conflicts with builtin", newdecl); + else + DECL_MODULE_EXPORT_P (olddecl) = DECL_MODULE_EXPORT_P (newdecl); + } + else + { + if (!module_may_redeclare (olddecl)) + { + error ("declaration %qD conflicts with import", newdecl); + inform (olddecl_loc, "import declared %q#D here", olddecl); + + return error_mark_node; + } + + if (DECL_MODULE_EXPORT_P (newdecl) + && !DECL_MODULE_EXPORT_P (olddecl)) + { + error ("conflicting exporting declaration %qD", newdecl); + inform (olddecl_loc, "previous declaration %q#D here", olddecl); + } + } + } + /* We have committed to returning OLDDECL at this point. */ /* If new decl is `static' and an `extern' was seen previously, @@ -2218,6 +2251,10 @@ duplicate_decls (tree newdecl, tree olddecl, bool hiding, bool was_hidden) } } + DECL_MODULE_IMPORT_P (olddecl) + = DECL_MODULE_IMPORT_P (old_result) + = DECL_MODULE_IMPORT_P (newdecl); + return olddecl; } @@ -2816,6 +2859,20 @@ duplicate_decls (tree newdecl, tree olddecl, bool hiding, bool was_hidden) memcpy ((char *) olddecl + sizeof (struct tree_common), (char *) newdecl + sizeof (struct tree_common), sizeof (struct tree_decl_common) - sizeof (struct tree_common)); + + if (DECL_LANG_SPECIFIC (olddecl) && DECL_TEMPLATE_INFO (olddecl)) + { + /* Repropagate the module information to the template. */ + tree tmpl = DECL_TI_TEMPLATE (olddecl); + + if (DECL_TEMPLATE_RESULT (tmpl) == olddecl) + { + DECL_MODULE_PURVIEW_P (tmpl) = DECL_MODULE_PURVIEW_P (olddecl); + gcc_checking_assert (!DECL_MODULE_IMPORT_P (olddecl)); + DECL_MODULE_IMPORT_P (tmpl) = false; + } + } + switch (TREE_CODE (newdecl)) { case LABEL_DECL: @@ -4102,6 +4159,12 @@ make_unbound_class_template (tree context, tree name, tree parm_list, return tmpl; } + return make_unbound_class_template_raw (context, name, parm_list); +} + +tree +make_unbound_class_template_raw (tree context, tree name, tree parm_list) +{ /* Build the UNBOUND_CLASS_TEMPLATE. */ tree t = cxx_make_type (UNBOUND_CLASS_TEMPLATE); TYPE_CONTEXT (t) = FROB_CONTEXT (context); @@ -4283,7 +4346,8 @@ cxx_init_decl_processing (void) gcc_assert (global_namespace == NULL_TREE); global_namespace = build_lang_decl (NAMESPACE_DECL, global_identifier, void_type_node); - TREE_PUBLIC (global_namespace) = 1; + TREE_PUBLIC (global_namespace) = true; + DECL_MODULE_EXPORT_P (global_namespace) = true; DECL_CONTEXT (global_namespace) = build_translation_unit_decl (get_identifier (main_input_filename)); /* Remember whether we want the empty class passing ABI change warning @@ -4579,6 +4643,9 @@ cxx_init_decl_processing (void) if (! supports_one_only ()) flag_weak = 0; + if (modules_p ()) + init_modules (parse_in); + make_fname_decl = cp_make_fname_decl; start_fname_decls (); @@ -5390,8 +5442,14 @@ start_decl (const cp_declarator *declarator, if ((DECL_EXTERNAL (decl) || TREE_CODE (decl) == FUNCTION_DECL) && current_function_decl) - /* A function-scope decl of some namespace-scope decl. */ - DECL_LOCAL_DECL_P (decl) = true; + { + /* A function-scope decl of some namespace-scope decl. */ + DECL_LOCAL_DECL_P (decl) = true; + if (named_module_purview_p ()) + error_at (declarator->id_loc, + "block-scope extern declaration %q#D not permitted" + " in module purview", decl); + } /* Enter this declaration into the symbol table. Don't push the plain VAR_DECL for a variable template. */ @@ -9810,6 +9868,8 @@ grokfndecl (tree ctype, && !processing_template_decl) deduce_noexcept_on_destructor (decl); + set_originating_module (decl); + decl = check_explicit_specialization (orig_declarator, decl, template_count, 2 * funcdef_flag + @@ -10054,6 +10114,8 @@ grokvardecl (tree type, TREE_PUBLIC (decl) = DECL_EXTERNAL (decl); } + set_originating_module (decl); + if (decl_spec_seq_has_spec_p (declspecs, ds_thread)) { if (DECL_EXTERNAL (decl) || TREE_STATIC (decl)) @@ -12907,6 +12969,8 @@ grokdeclarator (const cp_declarator *declarator, revert this subsequently if it determines that the clones should share a common implementation. */ DECL_ABSTRACT_P (decl) = true; + + set_originating_module (decl); } else if (current_class_type && constructor_name_p (unqualified_id, current_class_type)) @@ -13431,6 +13495,8 @@ grokdeclarator (const cp_declarator *declarator, ; /* We already issued a permerror. */ else if (decl && DECL_NAME (decl)) { + set_originating_module (decl, true); + if (initialized) /* Kludge: We need funcdef_flag to be true in do_friend for in-class defaulted functions, but that breaks grokfndecl. @@ -15070,6 +15135,41 @@ xref_tag_1 (enum tag_types tag_code, tree name, inform (location_of (t), "previous declaration %qD", t); return error_mark_node; } + + if (modules_p () + && how == TAG_how::CURRENT_ONLY) + { + tree decl = TYPE_NAME (t); + if (!module_may_redeclare (decl)) + { + error ("cannot declare %qD in a different module", decl); + inform (DECL_SOURCE_LOCATION (decl), "declared here"); + return error_mark_node; + } + + tree maybe_tmpl = decl; + if (CLASS_TYPE_P (t) && CLASSTYPE_IS_TEMPLATE (t)) + maybe_tmpl = CLASSTYPE_TI_TEMPLATE (t); + + if (DECL_LANG_SPECIFIC (decl) + && DECL_MODULE_IMPORT_P (decl) + && TREE_CODE (CP_DECL_CONTEXT (decl)) == NAMESPACE_DECL) + { + /* Push it into this TU's symbol slot. */ + gcc_checking_assert (current_namespace == CP_DECL_CONTEXT (decl)); + if (maybe_tmpl != decl) + /* We're in the template parm binding level. + Pushtag has logic to slide under that, but we're + not pushing a *new* type. */ + push_nested_namespace (CP_DECL_CONTEXT (decl)); + + pushdecl (maybe_tmpl); + if (maybe_tmpl != decl) + pop_nested_namespace (CP_DECL_CONTEXT (decl)); + } + + set_instantiating_module (maybe_tmpl); + } } return t; @@ -15414,6 +15514,19 @@ start_enum (tree name, tree enumtype, tree underlying_type, "previous definition here"); underlying_type = NULL_TREE; } + + if (flag_modules) + { + if (!module_may_redeclare (TYPE_NAME (enumtype))) + { + error ("cannot define %qD in different module", + TYPE_NAME (enumtype)); + inform (DECL_SOURCE_LOCATION (TYPE_NAME (enumtype)), + "declared here"); + enumtype = error_mark_node; + } + set_instantiating_module (TYPE_NAME (enumtype)); + } } if (!enumtype || TREE_CODE (enumtype) != ENUMERAL_TYPE @@ -15674,6 +15787,11 @@ finish_enum_value_list (tree enumtype) else underlying_type = ENUM_UNDERLYING_TYPE (enumtype); + /* If the enum is exported, mark the consts too. */ + bool export_p = (UNSCOPED_ENUM_P (enumtype) + && DECL_MODULE_EXPORT_P (TYPE_STUB_DECL (enumtype)) + && at_namespace_scope_p ()); + /* Convert each of the enumerators to the type of the underlying type of the enumeration. */ for (values = TYPE_VALUES (enumtype); values; values = TREE_CHAIN (values)) @@ -15696,8 +15814,10 @@ finish_enum_value_list (tree enumtype) TREE_TYPE (value) = enumtype; } DECL_INITIAL (decl) = value; + if (export_p) + DECL_MODULE_EXPORT_P (decl) = true; } /* Fix up all variant types of this enum type. */ for (t = TYPE_MAIN_VARIANT (enumtype); t; t = TYPE_NEXT_VARIANT (t)) TYPE_VALUES (t) = TYPE_VALUES (enumtype); @@ -16866,7 +16988,7 @@ maybe_save_function_definition (tree fun) && DECL_DECLARED_CONSTEXPR_P (fun) && !cp_function_chain->invalid_constexpr && !DECL_CLONED_FUNCTION_P (fun)) - register_constexpr_fundef (fun, DECL_SAVED_TREE (fun)); + check_constexpr_fundef (fun, DECL_SAVED_TREE (fun)); } /* Attempt to add a fix-it hint to RICHLOC suggesting the insertion @@ -17307,10 +17429,25 @@ grokmethod (cp_decl_specifier_seq *declspecs, check_template_shadow (fndecl); - if (TREE_PUBLIC (fndecl)) - DECL_COMDAT (fndecl) = 1; - DECL_DECLARED_INLINE_P (fndecl) = 1; - DECL_NO_INLINE_WARNING_P (fndecl) = 1; + /* p1779 ABI-Isolation makes inline not a default for in-class + definitions in named module purview. If the user explicitly + made it inline, grokdeclarator will already have done the right + things. */ + if ((!named_module_purview_p () + || flag_module_implicit_inline + /* Lambda's operator function remains inline. */ + || LAMBDA_TYPE_P (DECL_CONTEXT (fndecl))) + /* If the user explicitly asked for this to be inline, we don't + need to do more, but more importantly we want to warn if we + can't inline it. */ + && !DECL_DECLARED_INLINE_P (fndecl)) + { + if (TREE_PUBLIC (fndecl)) + DECL_COMDAT (fndecl) = 1; + DECL_DECLARED_INLINE_P (fndecl) = 1; + /* It's ok if we can't inline this. */ + DECL_NO_INLINE_WARNING_P (fndecl) = 1; + } /* We process method specializations in finish_struct_1. */ if (processing_template_decl && !DECL_TEMPLATE_SPECIALIZATION (fndecl)) @@ -17581,6 +17718,7 @@ cp_tree_node_structure (union lang_tree_node * t) case DEFERRED_PARSE: return TS_CP_DEFERRED_PARSE; case IDENTIFIER_NODE: return TS_CP_IDENTIFIER; case LAMBDA_EXPR: return TS_CP_LAMBDA_EXPR; + case MODULE_VECTOR: return TS_CP_MODULE_VECTOR; case OVERLOAD: return TS_CP_OVERLOAD; case PTRMEM_CST: return TS_CP_PTRMEM; case STATIC_ASSERT: return TS_CP_STATIC_ASSERT; diff --git c/gcc/cp/decl2.c w/gcc/cp/decl2.c index 71107e03010..60bc6444d7c 100644 --- c/gcc/cp/decl2.c +++ w/gcc/cp/decl2.c @@ -4470,6 +4506,10 @@ no_linkage_error (tree decl) /* In C++11 it's ok if the decl is defined. */ return; + if (DECL_LANG_SPECIFIC (decl) && DECL_MODULE_IMPORT_P (decl)) + /* An imported decl is ok. */ + return; + tree t = no_linkage_check (TREE_TYPE (decl), /*relaxed_p=*/false); if (t == NULL_TREE) /* The type that got us on no_linkage_decls must have gotten a name for @@ -4904,6 +4944,9 @@ c_parse_final_cleanups (void) instantiate_pending_templates (retries); ggc_collect (); + if (header_module_p ()) + goto skip; + /* Write out virtual tables as required. Writing out the virtual table for a template class may cause the instantiation of members of that class. If we write out @@ -5102,10 +5147,13 @@ c_parse_final_cleanups (void) pending_statics->length ())) reconsider = true; + skip:; retries++; } while (reconsider); + finish_module_processing (parse_in); + lower_var_init (); generate_mangling_aliases (); @@ -5121,6 +5169,10 @@ c_parse_final_cleanups (void) #pragma interface, etc.) we decided not to emit the definition here. */ && !DECL_INITIAL (decl) + /* A defaulted fn in a header module can be synthesized on + demand later. (In non-header modules we should have + synthesized it above.) */ + && !(DECL_DEFAULTED_FN (decl) && header_module_p ()) /* Don't complain if the template was defined. */ && !(DECL_TEMPLATE_INSTANTIATION (decl) && DECL_INITIAL (DECL_TEMPLATE_RESULT @@ -5154,9 +5206,8 @@ c_parse_final_cleanups (void) splay_tree_foreach (priority_info_map, generate_ctor_and_dtor_functions_for_priority, /*data=*/&locus_at_end_of_parsing); - else if (c_dialect_objc () && objc_static_init_needed_p ()) - /* If this is obj-c++ and we need a static init, call - generate_ctor_or_dtor_function. */ + else if ((c_dialect_objc () && objc_static_init_needed_p ()) + || module_initializer_kind ()) generate_ctor_or_dtor_function (/*constructor_p=*/true, DEFAULT_INIT_PRIORITY, &locus_at_end_of_parsing); @@ -5165,6 +5216,8 @@ c_parse_final_cleanups (void) if (priority_info_map) splay_tree_delete (priority_info_map); + fini_modules (); + /* Generate any missing aliases. */ maybe_apply_pending_pragma_weaks (); diff --git c/gcc/cp/error.c w/gcc/cp/error.c index 396558be17f..ad998791bc6 100644 --- c/gcc/cp/error.c +++ w/gcc/cp/error.c @@ -179,6 +179,36 @@ cxx_initialize_diagnostics (diagnostic_context *context) pp->m_format_postprocessor = new cxx_format_postprocessor (); } +static void +dump_module_suffix (cxx_pretty_printer *pp, tree decl) +{ + if (!modules_p ()) + return; + + if (!DECL_CONTEXT (decl)) + return; + + if (TREE_CODE (decl) != CONST_DECL + || !UNSCOPED_ENUM_P (DECL_CONTEXT (decl))) + { + if (!DECL_NAMESPACE_SCOPE_P (decl)) + return; + + if (TREE_CODE (decl) == NAMESPACE_DECL + && !DECL_NAMESPACE_ALIAS (decl) + && (TREE_PUBLIC (decl) || !TREE_PUBLIC (CP_DECL_CONTEXT (decl)))) + return; + } + + if (unsigned m = get_originating_module (decl)) + if (const char *n = module_name (m, false)) + { + pp_character (pp, '@'); + pp->padding = pp_none; + pp_string (pp, n); + } +} + /* Dump a scope, if deemed necessary. */ static void @@ -770,6 +800,8 @@ dump_aggr_type (cxx_pretty_printer *pp, tree t, int flags) else pp_cxx_tree_identifier (pp, DECL_NAME (decl)); + dump_module_suffix (pp, decl); + if (tmplate) dump_template_parms (pp, TYPE_TEMPLATE_INFO (t), !CLASSTYPE_USE_TEMPLATE (t), @@ -1074,6 +1106,9 @@ dump_simple_decl (cxx_pretty_printer *pp, tree t, tree type, int flags) pp_string (pp, M_("")); else pp_string (pp, M_("")); + + dump_module_suffix (pp, t); + if (flags & TFF_DECL_SPECIFIERS) dump_type_suffix (pp, type, flags); } @@ -1891,6 +1926,8 @@ dump_function_name (cxx_pretty_printer *pp, tree t, int flags) else dump_decl (pp, name, flags); + dump_module_suffix (pp, t); + if (DECL_TEMPLATE_INFO (t) && !DECL_FRIEND_PSEUDO_TEMPLATE_INSTANTIATION (t) && (TREE_CODE (DECL_TI_TEMPLATE (t)) != TEMPLATE_DECL diff --git c/gcc/cp/lambda.c w/gcc/cp/lambda.c index 1a1647f465e..f6746d7304b 100644 --- c/gcc/cp/lambda.c +++ w/gcc/cp/lambda.c @@ -1114,6 +1114,8 @@ maybe_add_lambda_conv_op (tree type) while (src) { tree new_node = copy_node (src); + /* We set DECL_CONTEXT of NEW_NODE to the statfn below. + Notice this is creating a recursive type! */ /* Clear TREE_ADDRESSABLE on thunk arguments. */ TREE_ADDRESSABLE (new_node) = 0; @@ -1393,6 +1395,12 @@ record_lambda_scope (tree lambda) { LAMBDA_EXPR_EXTRA_SCOPE (lambda) = lambda_scope; LAMBDA_EXPR_DISCRIMINATOR (lambda) = lambda_count++; + if (lambda_scope) + { + tree closure = LAMBDA_EXPR_CLOSURE (lambda); + gcc_checking_assert (closure); + maybe_attach_decl (lambda_scope, TYPE_NAME (closure)); + } } /* This lambda is an instantiation of a lambda in a template default argument diff --git c/gcc/cp/mangle.c w/gcc/cp/mangle.c index 9fd30011288..6f9c93cb571 100644 --- c/gcc/cp/mangle.c +++ w/gcc/cp/mangle.c @@ -117,6 +117,9 @@ struct GTY(()) globals { /* True if the mangling will be different in C++17 mode. */ bool need_cxx17_warning; + + /* True if we mangled a module name. */ + bool mod; }; static GTY (()) globals G; @@ -832,6 +835,62 @@ write_encoding (const tree decl) } } +/* Interface to substitution and identifer mangling, used by the + module name mangler. */ + +void +mangle_module_substitution (int v) +{ + if (v < 10) + { + write_char ('_'); + write_char ('0' + v); + } + else + { + write_char ('W'); + write_unsigned_number (v - 10); + write_char ('_'); + } +} + +void +mangle_identifier (char c, tree id) +{ + if (c) + write_char (c); + write_source_name (id); +} + +/* If the outermost non-namespace context (including DECL itself) is + a module-linkage decl, mangle the module information. For module + global initializers we need to include the partition part. + + ::= W + E + :: + || _ ;; short backref + || W _ ;; long backref + || P ;; partition introducer +*/ + +static void +write_module (int m, bool include_partition) +{ + G.mod = true; + + write_char ('W'); + mangle_module (m, include_partition); + write_char ('E'); +} + +static void +maybe_write_module (tree decl) +{ + int m = get_originating_module (decl, true); + if (m >= 0) + write_module (m, false); +} + /* Lambdas can have a bit more context for mangling, specifically VAR_DECL or PARM_DECL context, which doesn't belong in DECL_CONTEXT. */ @@ -894,6 +953,9 @@ write_name (tree decl, const int ignore_local_scope) decl = TYPE_NAME (TYPE_MAIN_VARIANT (TREE_TYPE (decl))); } + if (modules_p ()) + maybe_write_module (decl); + context = decl_mangling_context (decl); gcc_assert (context != NULL_TREE); @@ -3806,14 +3868,13 @@ start_mangling (const tree entity) G.entity = entity; G.need_abi_warning = false; G.need_cxx17_warning = false; + G.mod = false; obstack_free (&name_obstack, name_base); mangle_obstack = &name_obstack; name_base = obstack_alloc (&name_obstack, 0); } -/* Done with mangling. If WARN is true, and the name of G.entity will - be mangled differently in a future version of the ABI, issue a - warning. */ +/* Done with mangling. Release the data. */ static void finish_mangling_internal (void) @@ -3821,6 +3882,9 @@ finish_mangling_internal (void) /* Clear all the substitutions. */ vec_safe_truncate (G.substitutions, 0); + if (G.mod) + mangle_module_fini (); + /* Null-terminate the string. */ write_char ('\0'); } @@ -3865,6 +3929,20 @@ init_mangle (void) subst_identifiers[SUBID_BASIC_IOSTREAM] = get_identifier ("basic_iostream"); } +/* Generate a mangling for MODULE's global initializer fn. */ + +tree +mangle_module_global_init (int module) +{ + start_mangling (NULL_TREE); + + write_string ("_ZGI"); + write_module (module, true); + write_char ('v'); + + return finish_mangling_get_identifier (); +} + /* Generate the mangled name of DECL. */ static tree diff --git c/gcc/cp/pt.c w/gcc/cp/pt.c index aa162d2a4f9..497ac5aafec 100644 --- c/gcc/cp/pt.c +++ w/gcc/cp/pt.c @@ -40,6 +42,7 @@ along with GCC; see the file COPYING3. If not see #include "tree-iterator.h" #include "type-utils.h" #include "gimplify.h" +#include "bitmap.h" #include "gcc-rich-location.h" #include "selftest.h" #include "target.h" @@ -965,6 +961,9 @@ maybe_new_partial_specialization (tree type) TREE_PRIVATE (d) = (current_access_specifier == access_private_node); TREE_PROTECTED (d) = (current_access_specifier == access_protected_node); + set_instantiating_module (d); + DECL_MODULE_EXPORT_P (d) = DECL_MODULE_EXPORT_P (tmpl); + return t; } @@ -4926,6 +4929,17 @@ build_template_decl (tree decl, tree parms, bool member_template_p) DECL_SOURCE_LOCATION (tmpl) = DECL_SOURCE_LOCATION (decl); DECL_MEMBER_TEMPLATE_P (tmpl) = member_template_p; + if (modules_p ()) + { + /* Propagate module information from the decl. */ + DECL_MODULE_EXPORT_P (tmpl) = DECL_MODULE_EXPORT_P (decl); + if (DECL_LANG_SPECIFIC (decl)) + { + DECL_MODULE_PURVIEW_P (tmpl) = DECL_MODULE_PURVIEW_P (decl); + gcc_checking_assert (!DECL_MODULE_IMPORT_P (decl)); + } + } + return tmpl; } @@ -6047,6 +6061,14 @@ push_template_decl (tree decl, bool is_friend) tmpl = NULL_TREE; } } + else if (is_friend) + { + /* Record this decl as belonging to the current class. It's + not chained onto anything else. */ + DECL_UNINSTANTIATED_TEMPLATE_FRIEND_P (tmpl) = true; + gcc_checking_assert (!DECL_CHAIN (tmpl)); + DECL_CHAIN (tmpl) = current_scope (); + } } else if (tmpl) /* The type may have been completed, or (erroneously) changed. */ @@ -9770,6 +9792,15 @@ lookup_template_class_1 (tree d1, tree arglist, tree in_decl, tree context, return error_mark_node; gen_tmpl = most_general_template (templ); + if (flag_modules) + { + tree origin = get_originating_module_decl (gen_tmpl); + load_pending_specializations (CP_DECL_CONTEXT (origin), + DECL_NAME (origin)); + if (DECL_MODULE_PENDING_SPECIALIZATIONS_P (gen_tmpl)) + lazy_load_specializations (gen_tmpl); + } + parmlist = DECL_TEMPLATE_PARMS (gen_tmpl); parm_depth = TMPL_PARMS_DEPTH (parmlist); arg_depth = TMPL_ARGS_DEPTH (arglist); @@ -9990,6 +10021,12 @@ lookup_template_class_1 (tree d1, tree arglist, tree in_decl, tree context, = DECL_SOURCE_LOCATION (TYPE_STUB_DECL (template_type)); } + set_instantiating_module (type_decl); + /* Although GEN_TMPL is the TEMPLATE_DECL, it has the same value + of export flag. We want to propagate this because it might + be a friend declaration that pushes a new hidden binding. */ + DECL_MODULE_EXPORT_P (type_decl) = DECL_MODULE_EXPORT_P (gen_tmpl); + if (CLASS_TYPE_P (template_type)) { TREE_PRIVATE (type_decl) @@ -10896,6 +10933,7 @@ push_tinst_level_loc (tree tldcl, tree targs, location_t loc) new_level->errors = errorcount + sorrycount; new_level->next = NULL; new_level->refcount = 0; + new_level->path = new_level->visible = nullptr; set_refcount_ptr (new_level->next, current_tinst_level); set_refcount_ptr (current_tinst_level, new_level); @@ -11046,6 +11084,7 @@ tsubst_friend_function (tree decl, tree args) DECL_USE_TEMPLATE (new_friend) = 0; if (TREE_CODE (new_friend) == TEMPLATE_DECL) { + DECL_UNINSTANTIATED_TEMPLATE_FRIEND_P (new_friend) = false; DECL_USE_TEMPLATE (DECL_TEMPLATE_RESULT (new_friend)) = 0; DECL_SAVED_TREE (DECL_TEMPLATE_RESULT (new_friend)) = DECL_SAVED_TREE (DECL_TEMPLATE_RESULT (decl)); @@ -13891,6 +13935,7 @@ tsubst_function_decl (tree t, tree args, tsubst_flags_t complain, if (!DECL_DELETED_FN (r)) DECL_INITIAL (r) = NULL_TREE; DECL_CONTEXT (r) = ctx; + set_instantiating_module (r); /* Handle explicit(dependent-expr). */ if (DECL_HAS_DEPENDENT_EXPLICIT_SPEC_P (t)) @@ -14214,7 +14259,25 @@ tsubst_template_decl (tree t, tree args, tsubst_flags_t complain, TREE_TYPE (r) = TREE_TYPE (inner); DECL_CONTEXT (r) = DECL_CONTEXT (inner); + if (modules_p ()) + { + /* Propagate module information from the decl. */ + DECL_MODULE_EXPORT_P (r) = DECL_MODULE_EXPORT_P (inner); + if (DECL_LANG_SPECIFIC (inner)) + { + DECL_MODULE_PURVIEW_P (r) = DECL_MODULE_PURVIEW_P (inner); + /* If this is a constrained template, the above tsubst of + inner can find the unconstrained template, which may have + come from an import. This is ok, because we don't + register this instantiation (see below). */ + gcc_checking_assert (!DECL_MODULE_IMPORT_P (inner) + || (TEMPLATE_PARMS_CONSTRAINTS + (DECL_TEMPLATE_PARMS (t)))); + DECL_MODULE_IMPORT_P (r) = false; + } + } + DECL_TEMPLATE_INSTANTIATIONS (r) = NULL_TREE; DECL_TEMPLATE_SPECIALIZATIONS (r) = NULL_TREE; if (PRIMARY_TEMPLATE_P (t)) @@ -14769,6 +14831,8 @@ tsubst_decl (tree t, tree args, tsubst_flags_t complain) SET_DECL_ASSEMBLER_NAME (r, NULL_TREE); if (CODE_CONTAINS_STRUCT (TREE_CODE (t), TS_DECL_WRTL)) SET_DECL_RTL (r, NULL); + set_instantiating_module (r); + /* The initializer must not be expanded until it is required; see [temp.inst]. */ DECL_INITIAL (r) = NULL_TREE; @@ -20843,6 +20914,15 @@ instantiate_template_1 (tree tmpl, tree orig_args, tsubst_flags_t complain) (DECL_TI_ARGS (DECL_TEMPLATE_RESULT (tmpl)), targ_ptr)); + if (flag_modules) + { + tree origin = get_originating_module_decl (gen_tmpl); + load_pending_specializations (CP_DECL_CONTEXT (origin), + DECL_NAME (origin)); + if (DECL_MODULE_PENDING_SPECIALIZATIONS_P (gen_tmpl)) + lazy_load_specializations (gen_tmpl); + } + /* It would be nice to avoid hashing here and then again in tsubst_decl, but it doesn't seem to be on the hot path. */ spec = retrieve_specialization (gen_tmpl, targ_ptr, 0); @@ -20928,6 +21008,8 @@ instantiate_template_1 (tree tmpl, tree orig_args, tsubst_flags_t complain) DECL_TI_TEMPLATE (fndecl) = tmpl; DECL_TI_ARGS (fndecl) = targ_ptr; + set_instantiating_module (fndecl); + /* Now we know the specialization, compute access previously deferred. Do no access control for inheriting constructors, as we already checked access for the inherited constructor. */ @@ -28002,6 +28082,8 @@ finish_concept_definition (cp_expr id, tree init) DECL_CONTEXT (decl) = current_scope (); DECL_INITIAL (decl) = init; + set_originating_module (decl, false); + /* Push the enclosing template. */ return push_template_decl (decl); } diff --git c/gcc/cp/ptree.c w/gcc/cp/ptree.c index a28b722f571..a216ca59e99 100644 --- c/gcc/cp/ptree.c +++ w/gcc/cp/ptree.c @@ -59,6 +59,42 @@ cxx_print_decl (FILE *file, tree node, int indent) bool need_indent = true; + if (TREE_CODE (node) == FUNCTION_DECL + || TREE_CODE (node) == VAR_DECL + || TREE_CODE (node) == TYPE_DECL + || TREE_CODE (node) == TEMPLATE_DECL + || TREE_CODE (node) == CONCEPT_DECL + || TREE_CODE (node) == NAMESPACE_DECL) + { + unsigned m = 0; + if (DECL_LANG_SPECIFIC (node) && DECL_MODULE_IMPORT_P (node)) + m = get_importing_module (node, true); + + if (const char *name = m == ~0u ? "" : module_name (m, true)) + { + if (need_indent) + indent_to (file, indent + 3); + fprintf (file, " module %d:%s", m, name); + need_indent = false; + } + + if (DECL_LANG_SPECIFIC (node) && DECL_MODULE_PURVIEW_P (node)) + { + if (need_indent) + indent_to (file, indent + 3); + fprintf (file, " purview"); + need_indent = false; + } + } + + if (DECL_MODULE_EXPORT_P (node)) + { + if (need_indent) + indent_to (file, indent + 3); + fprintf (file, " exported"); + need_indent = false; + } + if (DECL_EXTERNAL (node) && DECL_NOT_REALLY_EXTERN (node)) { if (need_indent) @@ -250,6 +286,44 @@ cxx_print_xnode (FILE *file, tree node, int indent) print_node (file, "function", OVL_FUNCTION (node), indent + 4); print_node (file, "next", OVL_CHAIN (node), indent + 4); break; + case MODULE_VECTOR: + { + unsigned len = MODULE_VECTOR_NUM_CLUSTERS (node); + print_node (file, "name", MODULE_VECTOR_NAME (node), indent + 4); + fprintf (file, " clusters %u, alloc %u", len, + MODULE_VECTOR_ALLOC_CLUSTERS (node)); + for (unsigned ix = 0; ix != len; ix++) + { + module_cluster *cluster = &MODULE_VECTOR_CLUSTER (node, ix); + char pfx[20]; + for (unsigned jx = 0; jx != MODULE_VECTOR_SLOTS_PER_CLUSTER; jx++) + if (cluster->indices[jx].span) + { + int len = sprintf (pfx, "module:%u", + cluster->indices[jx].base); + if (cluster->indices[jx].span > 1) + len + += sprintf (&pfx[len], "(+%u)", cluster->indices[jx].span); + len += sprintf (&pfx[len], " cluster:%u/%u", ix, jx); + mc_slot &slot = cluster->slots[jx]; + if (slot.is_lazy ()) + { + indent_to (file, indent + 4); + unsigned lazy = slot.get_lazy (); + fprintf (file, "%s snum:%u flags:%d", + pfx, lazy >> 2, lazy & 3); + } + else if (slot) + print_node (file, pfx, slot, indent + 4); + else + { + indent_to (file, indent + 4); + fprintf (file, "%s NULL", pfx); + } + } + } + } + break; case TEMPLATE_PARM_INDEX: print_node (file, "decl", TEMPLATE_PARM_DECL (node), indent+4); indent_to (file, indent + 3); diff --git c/gcc/cp/semantics.c w/gcc/cp/semantics.c index 352ebe03436..a0888bc6c29 100644 --- c/gcc/cp/semantics.c +++ w/gcc/cp/semantics.c @@ -3225,6 +3225,19 @@ begin_class_definition (tree t) t = make_class_type (TREE_CODE (t)); pushtag (TYPE_IDENTIFIER (t), t); } + + if (flag_modules) + { + if (!module_may_redeclare (TYPE_NAME (t))) + { + error ("cannot declare %qD in a different module", TYPE_NAME (t)); + inform (DECL_SOURCE_LOCATION (TYPE_NAME (t)), "declared here"); + return error_mark_node; + } + set_instantiating_module (TYPE_NAME (t)); + set_defining_module (TYPE_NAME (t)); + } + maybe_process_partial_specialization (t); pushclass (t); TYPE_BEING_DEFINED (t) = 1; @@ -4503,7 +4516,8 @@ expand_or_defer_fn_1 (tree fn) it out, even though we haven't. */ TREE_ASM_WRITTEN (fn) = 1; /* If this is a constexpr function, keep DECL_SAVED_TREE. */ - if (!DECL_DECLARED_CONSTEXPR_P (fn)) + if (!DECL_DECLARED_CONSTEXPR_P (fn) + && !(modules_p () && DECL_DECLARED_INLINE_P (fn))) DECL_SAVED_TREE (fn) = NULL_TREE; return false; } diff --git c/gcc/gdbinit.in w/gcc/gdbinit.in index e951c19db63..eb999d2a808 100644 --- c/gcc/gdbinit.in +++ w/gcc/gdbinit.in @@ -333,6 +333,10 @@ b fancy_abort # Put a breakpoint on internal_error to help with debugging ICEs. b internal_error +# Break on module streaming error +b bytes_in::set_overrun if !overrun +b elf::set_error if !err + set complaints 0 # Don't let abort actually run, as it will make # stdio stop working and therefore the `pr' command above as well. diff --git c/gcc/tree-core.h w/gcc/tree-core.h index c9280a8d3b1..73aa0c3f399 100644 --- c/gcc/tree-core.h +++ w/gcc/tree-core.h @@ -769,6 +769,10 @@ enum tree_index { TI_SAT_UDA_TYPE, TI_SAT_UTA_TYPE, + TI_MODULE_HWM, + /* Nodes below here change during compilation, and should therefore + not be in the C++ module's global tree table. */ + TI_OPTIMIZATION_DEFAULT, TI_OPTIMIZATION_CURRENT, TI_TARGET_OPTION_DEFAULT, diff --git c/libcpp/include/cpplib.h w/libcpp/include/cpplib.h index 8e398863cf6..81be6457951 100644 --- c/libcpp/include/cpplib.h +++ w/libcpp/include/cpplib.h @@ -528,6 +538,9 @@ struct cpp_options one. */ bool phony_targets; + /* Generate dependency info for modules. */ + bool modules; + /* If true, no dependency is generated on the main file. */ bool ignore_main_file; diff --git c/gcc/cp/rtti.c w/gcc/cp/rtti.c index 887aae31bf6..0be9eff54ad 100644 --- c/gcc/cp/rtti.c +++ w/gcc/cp/rtti.c @@ -144,16 +143,20 @@ static bool typeinfo_in_lib_p (tree); static int doing_runtime = 0; -static void +static unsigned push_abi_namespace (void) { push_nested_namespace (abi_node); push_visibility ("default", 2); + unsigned flags = module_kind; + module_kind = 0; + return flags; } static void -pop_abi_namespace (void) +pop_abi_namespace (unsigned flags) { + module_kind = flags; pop_visibility (2); pop_nested_namespace (abi_node); } @@ -766,7 +769,7 @@ build_dynamic_cast_1 (location_t loc, tree type, tree expr, dcast_fn = dynamic_cast_node; if (!dcast_fn) { - push_abi_namespace (); + unsigned flags = push_abi_namespace (); tree tinfo_ptr = xref_tag (class_type, get_identifier ("__class_type_info")); tinfo_ptr = cp_build_qualified_type (tinfo_ptr, TYPE_QUAL_CONST); @@ -781,7 +784,7 @@ build_dynamic_cast_1 (location_t loc, tree type, tree expr, NULL_TREE)); dcast_fn = (build_library_fn_ptr (fn_name, fn_type, ECF_LEAF | ECF_PURE | ECF_NOTHROW)); - pop_abi_namespace (); + pop_abi_namespace (flags); dynamic_cast_node = dcast_fn; } result = build_cxx_call (dcast_fn, 4, elems, complain); @@ -955,11 +958,11 @@ tinfo_base_init (tinfo_s *ti, tree target) vtable_ptr = ti->vtable; if (!vtable_ptr) { - push_abi_namespace (); + int flags = push_abi_namespace (); tree real_type = xref_tag (class_type, ti->name); tree real_decl = TYPE_NAME (real_type); DECL_SOURCE_LOCATION (real_decl) = BUILTINS_LOCATION; - pop_abi_namespace (); + pop_abi_namespace (flags); if (!COMPLETE_TYPE_P (real_type)) { @@ -1479,6 +1482,7 @@ get_tinfo_desc (unsigned ix) finish_builtin_struct (pseudo_type, pseudo_name, fields, NULL_TREE); CLASSTYPE_AS_BASE (pseudo_type) = pseudo_type; DECL_CONTEXT (TYPE_NAME (pseudo_type)) = FROB_CONTEXT (global_namespace); + DECL_TINFO_P (TYPE_NAME (pseudo_type)) = true; xref_basetypes (pseudo_type, /*bases=*/NULL_TREE); res->type = cp_build_qualified_type (pseudo_type, TYPE_QUAL_CONST); diff --git c/gcc/cp/tree.c w/gcc/cp/tree.c index 3087c4ab52c..d125fa5e793 100644 --- c/gcc/cp/tree.c +++ w/gcc/cp/tree.c @@ -2218,6 +2226,23 @@ build_ref_qualified_type (tree type, cp_ref_qualifier rqual) return build_cp_fntype_variant (type, rqual, raises, late); } +tree +make_module_vec (tree name, unsigned clusters MEM_STAT_DECL) +{ + /* Stored in an unsigned short, but we're limited to the number of + modules anyway. */ + gcc_checking_assert (clusters <= (unsigned short)(~0)); + size_t length = (clusters * sizeof (module_cluster) + + sizeof (tree_module_vec) - sizeof (module_cluster)); + tree vec = ggc_alloc_cleared_tree_node_stat (length PASS_MEM_STAT); + TREE_SET_CODE (vec, MODULE_VECTOR); + MODULE_VECTOR_NAME (vec) = name; + MODULE_VECTOR_ALLOC_CLUSTERS (vec) = clusters; + MODULE_VECTOR_NUM_CLUSTERS (vec) = 0; + + return vec; +} + /* Make a raw overload node containing FN. */ tree @@ -2237,10 +2262,11 @@ ovl_make (tree fn, tree next) return result; } -/* Add FN to the (potentially NULL) overload set OVL. USING_OR_HIDDEN - is > 0, if FN is via a using declaration. USING_OR_HIDDEN is < 0, - if FN is hidden. (A decl cannot be both using and hidden.) We - keep the hidden decls first, but remaining ones are unordered. */ +/* Add FN to the (potentially NULL) overload set OVL. USING_OR_HIDDEN is > + zero if this is a using-decl. It is > 1 if we're exporting the + using decl. USING_OR_HIDDEN is < 0, if FN is hidden. (A decl + cannot be both using and hidden.) We keep the hidden decls first, + but remaining ones are unordered. */ tree ovl_insert (tree fn, tree maybe_ovl, int using_or_hidden) @@ -2264,7 +2290,11 @@ ovl_insert (tree fn, tree maybe_ovl, int using_or_hidden) if (using_or_hidden < 0) OVL_HIDDEN_P (maybe_ovl) = true; if (using_or_hidden > 0) - OVL_DEDUP_P (maybe_ovl) = OVL_USING_P (maybe_ovl) = true; + { + OVL_DEDUP_P (maybe_ovl) = OVL_USING_P (maybe_ovl) = true; + if (using_or_hidden > 1) + OVL_EXPORT_P (maybe_ovl) = true; + } } else maybe_ovl = fn; diff --git c/gcc/cp/class.c w/gcc/cp/class.c index c03737294eb..f4e44a5feba 100644 --- c/gcc/cp/class.c +++ w/gcc/cp/class.c @@ -4901,7 +4901,7 @@ build_clone (tree fn, tree name, bool need_vtt_parm_p, /* Build the clones of FN, return the number of clones built. These will be inserted onto DECL_CHAIN of FN. */ -static void +void build_cdtor_clones (tree fn, bool needs_vtt_p, bool base_omits_inherited_p, bool update_methods) { @@ -6740,6 +6740,8 @@ layout_class_type (tree t, tree *virtuals_p) TYPE_CONTEXT (base_t) = t; DECL_CONTEXT (base_d) = t; + set_instantiating_module (base_d); + /* If the ABI version is not at least two, and the last field was a bit-field, RLI may not be on a byte boundary. In particular, rli_size_unit_so_far might @@ -8719,6 +8721,7 @@ build_self_reference (void) DECL_ARTIFICIAL (decl) = 1; SET_DECL_SELF_REFERENCE_P (decl); set_underlying_type (decl); + set_instantiating_module (decl); if (processing_template_decl) decl = push_template_decl (decl); From patchwork Tue Nov 3 21:17:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1393434 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=nr55m7W/; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CQjKl298Tz9sVk for ; Wed, 4 Nov 2020 08:17:31 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5C6D9398751D; Tue, 3 Nov 2020 21:17:28 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by sourceware.org (Postfix) with ESMTPS id DEA1E3987419 for ; Tue, 3 Nov 2020 21:17:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org DEA1E3987419 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72e.google.com with SMTP id k9so16631785qki.6 for ; Tue, 03 Nov 2020 13:17:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:subject:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=R4PtiRthpEJshPgB+kdmqZ9Su+v2Wx10I9rSKNUrSLw=; b=nr55m7W/ZZ6VYm7kq9lnup0/mER+kk93tg0j9ILZ0BTHaMa+uGRCl5X7qtThtqmoJA wE3iOur8Senp3Z+B1kIzPTrv5hsDL+POCpXu1oZRqkrIfDxesPuLinplaBpcAtUtxGLv 4Pxb6eovJCYwDbStXtY7bvtJvEgdr+4CFXN+vEs2QiTE+x/mOIAg30c1hTSoD95mr8ci 1RAOFR5nUulMcZvD0DBHW4yxycOwKFXYPReuFakVZNijCzW0oAV6PTE/3D0kfDaeDb2q 2HnRzGCkRvQ+SI18piju+DBRkiokUrPkS15GsQEQzXHtfw2AtdI9HL9jS8AHeLSkRUrB 3xvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:subject:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=R4PtiRthpEJshPgB+kdmqZ9Su+v2Wx10I9rSKNUrSLw=; b=S2MxXLvvi3gKauW1010zGRs/uE8WEnEQWji1MyM3sMmAdTGKRua7o0+2Z4wahUYZ2T wsDEy2VbXUwHKGQjrrW+T4+hd0vGT25ClPbDL33g+O+aAnc6FKA0K+cce3Ugf8AS3biZ +YgXq4txxMyaWuAuIkL08Bpcp10KgCilcAlA3/zV/ocFtwe0XeT6/vXSLtLfgAS4iB/l jaW7BbMY6fVwVmk7yhG61q8c0JcMUWcMBXxABgkPv5CRmZ0kz0bth+uBuj2+haHQFgCI q60NO5mhVI9CnLW3KRw21wq0lZrtxQWQASUMb+Rv1aASpivtfhYeg5rAjWOowXo4vFMs +DnA== X-Gm-Message-State: AOAM530kaCRf4mzogfy6ie+XmI4e4fbK8eJnv0z+ZtUOVbdnlnoA7Gn4 Jwd2CFkCBozESgS+CYOUYU8= X-Google-Smtp-Source: ABdhPJyVUmWTLdNK3Q8eKXFzxrSyszd0R2L/J92lMHwo5LmE5BH0vUQZa1xYCAFzTa7/nRSonDUNfg== X-Received: by 2002:a37:a304:: with SMTP id m4mr21794616qke.287.1604438243090; Tue, 03 Nov 2020 13:17:23 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:e5ef:7eb4:1609:98bb? ([2620:10d:c091:480::1:bc2d]) by smtp.googlemail.com with ESMTPSA id h82sm11571865qke.109.2020.11.03.13.17.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 03 Nov 2020 13:17:22 -0800 (PST) From: Nathan Sidwell Subject: [24/32] module mapper To: GCC Patches , Jason Merrill , Richard Biener References: <7fc9e868-3db9-4972-ed67-6ff249f549c3@acm.org> <85996243-86cf-88b4-5b3b-451eaf3a0df6@acm.org> <4d14b230-3263-9a13-3159-c4853f282761@acm.org> <688bd28f-5998-0def-8c40-03b817832d63@acm.org> <89819c10-e86d-9b01-5673-5223a525a135@acm.org> <35879e15-d74a-c664-4d44-15f4b3783d77@acm.org> <9ae23c4c-67a5-a267-c939-5a96e9488612@acm.org> <6666545b-0583-4812-4745-d51994465818@acm.org> <1ab99df5-3997-0895-c979-f8529f476df7@acm.org> <97e9477b-7173-b7f9-a884-616b972c57ba@acm.org> <01f091a5-cd8b-60b6-9552-2318ecd07025@acm.org> <5c533ebe-440d-188e-5bdb-38c14898852c@acm.org> <9e6ec23c-6b36-de70-7630-55562583696f@acm.org> Message-ID: <6939465f-3896-24e3-e588-e8e246073b52@acm.org> Date: Tue, 3 Nov 2020 16:17:20 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" this is the module mapper client and server pieces. It features a default resolver that can read a text file, or generate default mappings from module name to cmi name. By default the compiler will use an in-process resolved, but with suitable options can be instructed to communicate with a server over a) a pipe to a spawned process's stdin/stdout b) a pair of specified filenos c) a unix-domain socket (if supported) d) an ipv6 host/port (if supported) the server can operat in modes a, c or d. diff --git c/gcc/cp/mapper-client.cc w/gcc/cp/mapper-client.cc new file mode 100644 index 00000000000..de259b0564c --- /dev/null +++ w/gcc/cp/mapper-client.cc @@ -0,0 +1,315 @@ +/* C++ modules. Experimental! + Copyright (C) 2017-2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "system.h" + +#include "line-map.h" +#include "diagnostic-core.h" +#define MAPPER_FOR_GCC 1 +#include "mapper.h" +#include "intl.h" + +module_client::module_client (pex_obj *p, int fd_from, int fd_to) + : Client (fd_from, fd_to), pex (p) +{ +} + +static module_client * +spawn_mapper_program (char const **errmsg, std::string &name, + char const *full_program_name) +{ + /* Split writable at white-space. No space-containing args for + you! */ + // At most every other char could be an argument + char **argv = new char *[name.size () / 2 + 2]; + unsigned arg_no = 0; + char *str = new char[name.size ()]; + memcpy (str, name.c_str () + 1, name.size ()); + + for (auto ptr = str; ; ++ptr) + { + while (*ptr == ' ') + ptr++; + if (!*ptr) + break; + argv[arg_no++] = ptr; + while (*ptr && *ptr != ' ') + ptr++; + if (!*ptr) + break; + *ptr = 0; + } + argv[arg_no] = nullptr; + + auto *pex = pex_init (PEX_USE_PIPES, progname, NULL); + FILE *to = pex_input_pipe (pex, false); + name = argv[0]; + if (!to) + *errmsg = "connecting input"; + else + { + int flags = PEX_SEARCH; + + if (full_program_name) + { + /* Prepend the invoking path. */ + size_t dir_len = progname - full_program_name; + std::string argv0; + argv0.reserve (dir_len + name.size ()); + argv0.append (full_program_name, dir_len).append (name); + name = std::move (argv0); + argv[0] = const_cast (name.c_str ()); + flags = 0; + } + int err; + *errmsg = pex_run (pex, flags, argv[0], argv, NULL, NULL, &err); + } + delete[] str; + delete[] argv; + + int fd_from = -1, fd_to = -1; + if (!*errmsg) + { + FILE *from = pex_read_output (pex, false); + if (from && (fd_to = dup (fileno (to))) >= 0) + fd_from = fileno (from); + else + *errmsg = "connecting output"; + fclose (to); + } + + if (*errmsg) + { + pex_free (pex); + return nullptr; + } + + return new module_client (pex, fd_from, fd_to); +} + +module_client * +module_client::open_module_client (location_t loc, const char *o, + void (*set_repo) (const char *), + char const *full_program_name) +{ + module_client *c = nullptr; + std::string ident; + std::string name; + char const *errmsg = nullptr; + unsigned line = 0; + + if (o && o[0]) + { + /* Maybe a local or ipv6 address. */ + name = o; + auto last = name.find_last_of ('?'); + if (last != name.npos) + { + ident = name.substr (last + 1); + name.erase (last); + } + + if (name.size ()) + { + switch (name[0]) + { + case '<': + // to or <>fromto, or <> + { + int fd_from = -1, fd_to = -1; + char const *ptr = name.c_str (); + char *eptr; + + fd_from = strtoul (++ptr, &eptr, 0); + if (*eptr == '>') + { + ptr = eptr; + fd_to = strtoul (++ptr, &eptr, 0); + if (eptr != ptr && ptr == name.c_str () + 1) + fd_from = fd_to; + } + + if (*eptr) + errmsg = "parsing"; + else + { + if (name.size () == 2) + { + fd_from = fileno (stdin); + fd_to = fileno (stdout); + } + c = new module_client (fd_from, fd_to); + } + } + break; + + case '=': + // =localsocket + { + int fd = -1; +#if CODY_NETWORKING + fd = Cody::OpenLocal (&errmsg, name.c_str () + 1); +#endif + if (fd >= 0) + c = new module_client (fd, fd); + } + break; + + case '|': + // |program and args + c = spawn_mapper_program (&errmsg, name, full_program_name); + break; + + default: + // file or hostname:port + { + auto colon = name.find_last_of (':'); + if (colon != name.npos) + { + char const *cptr = name.c_str () + colon; + char *endp; + unsigned port = strtoul (cptr + 1, &endp, 10); + + if (port && endp != cptr + 1 && !*endp) + { + name[colon] = 0; + int fd = 01; +#if CODY_NETWORKING + fd = Cody::OpenInet6 (&errmsg, name.c_str (), port); +#endif + name[colon] = ':'; + + if (fd >= 0) + c = new module_client (fd, fd); + } + } + + } + break; + } + } + } + + if (!c) + { + // Make a default in-process client + bool file = !errmsg && !name.empty (); + auto r = new module_resolver (!file, true); + + if (file) + { + int fd = open (name.c_str (), O_RDONLY | O_CLOEXEC); + if (fd < 0) + errmsg = "opening"; + else + { + if (int l = r->read_tuple_file (fd, ident, false)) + { + if (l > 0) + line = l; + errmsg = "reading"; + } + + close (fd); + } + } + else + r->set_repo ("gcm.cache"); + + auto *s = new Cody::Server (r); + c = new module_client (s); + } + +#ifdef SIGPIPE + if (!c->IsDirect ()) + /* We need to ignore sig pipe for a while. */ + c->sigpipe = signal (SIGPIPE, SIG_IGN); +#endif + + if (errmsg) + error_at (loc, line ? G_("failed %s mapper %qs line %u") + : G_("failed %s mapper %qs"), errmsg, name.c_str (), line); + + // now wave hello! + c->Cork (); + c->Connect (std::string ("GCC"), ident); + c->ModuleRepo (); + auto packets = c->Uncork (); + + auto &connect = packets[0]; + if (connect.GetCode () == Cody::Client::PC_CONNECT) + ; + else if (connect.GetCode () == Cody::Client::PC_ERROR) + error_at (loc, "failed mapper handshake %s", connect.GetString ().c_str ()); + + auto &repo = packets[1]; + if (repo.GetCode () == Cody::Client::PC_PATHNAME) + set_repo (repo.GetString ().c_str ()); + + return c; +} + +void +module_client::close_module_client (location_t loc, module_client *mapper) +{ + if (mapper->IsDirect ()) + { + auto *s = mapper->GetServer (); + auto *r = s->GetResolver (); + delete s; + delete r; + } + else + { + if (mapper->pex) + { + int fd_write = mapper->GetFDWrite (); + if (fd_write >= 0) + close (fd_write); + + int status; + pex_get_status (mapper->pex, 1, &status); + + pex_free (mapper->pex); + mapper->pex = NULL; + + if (WIFSIGNALED (status)) + error_at (loc, "mapper died by signal %s", + strsignal (WTERMSIG (status))); + else if (WIFEXITED (status) && WEXITSTATUS (status) != 0) + error_at (loc, "mapper exit status %d", + WEXITSTATUS (status)); + } + else + { + int fd_read = mapper->GetFDRead (); + close (fd_read); + } + +#ifdef SIGPIPE + // Restore sigpipe + if (mapper->sigpipe != SIG_IGN) + signal (SIGPIPE, mapper->sigpipe); +#endif + } + + delete mapper; +} diff --git c/gcc/cp/mapper-resolver.cc w/gcc/cp/mapper-resolver.cc new file mode 100644 index 00000000000..7692380812a --- /dev/null +++ w/gcc/cp/mapper-resolver.cc @@ -0,0 +1,266 @@ +/* C++ modules. Experimental! -*- c++ -*- + Copyright (C) 2017-2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "system.h" + +#include "mapper.h" +// C++ +#include +// C +#include +// OS +#include +#include +#include +#include +#include + +module_resolver::module_resolver (bool map, bool xlate) + : default_map (map), default_translate (xlate) +{ +} + +module_resolver::~module_resolver () +{ + if (fd_repo >= 0) + close (fd_repo); +} + +bool +module_resolver::set_repo (std::string &&r, bool force) +{ + if (force || repo.empty ()) + { + repo = std::move (r); + force = true; + } + return force; +} + +bool +module_resolver::add_mapping (std::string &&module, std::string &&file, + bool force) +{ + auto res = map.emplace (std::move (module), std::move (file)); + if (res.second) + force = true; + else if (force) + res.first->second = std::move (file); + + return force; +} + +int +module_resolver::read_tuple_file (int fd, char const *prefix, bool force) +{ + struct stat stat; + if (fstat (fd, &stat) < 0) + return -errno; + + if (!stat.st_size) + return 0; + + // Just map the file, we're gonna read all of it, so no need for + // line buffering + void *buffer = mmap (nullptr, stat.st_size, PROT_READ, MAP_PRIVATE, fd, 0); + if (buffer == MAP_FAILED) + return -errno; + + size_t prefix_len = prefix ? strlen (prefix) : 0; + unsigned lineno = 0; + + for (char const *begin = reinterpret_cast (buffer), + *end = begin + stat.st_size, *eol; + begin != end; begin = eol + 1) + { + lineno++; + eol = std::find (begin, end, '\n'); + if (eol == end) + // last line has no \n, ignore the line, you lose + break; + + auto *pos = begin; + bool pfx_search = prefix_len != 0; + + pfx_search: + while (*pos == ' ' || *pos == '\t') + pos++; + + auto *space = pos; + while (*space != '\n' && *space != ' ' && *space != '\t') + space++; + + if (pos == space) + // at end of line, nothing here + continue; + + if (pfx_search) + { + if (size_t (space - pos) == prefix_len + && std::equal (pos, space, prefix)) + pfx_search = false; + pos = space; + goto pfx_search; + } + + std::string module (pos, space); + while (*space == ' ' || *space == '\t') + space++; + std::string file (space, eol); + + if (module[0] == '$') + { + if (module == "$root") + set_repo (std::move (file)); + else + return lineno; + } + else + { + if (file.empty ()) + file = GetCMIName (module); + add_mapping (std::move (module), std::move (file), force); + } + } + + munmap (buffer, stat.st_size); + + return 0; +} + +char const * +module_resolver::GetCMISuffix () +{ + return "gcm"; +} + +module_resolver * +module_resolver::ConnectRequest (Cody::Server *s, unsigned version, + std::string &a, std::string &i) +{ + if (!version || version > Cody::Version) + s->ErrorResponse ("version mismatch"); + else if (a != "GCC") + // Refuse anything but GCC + ErrorResponse (s, std::string ("only GCC supported")); + else if (!ident.empty () && ident != i) + // Failed ident check + ErrorResponse (s, std::string ("bad ident")); + else + // Success! + s->ConnectResponse ("gcc"); + + return this; +} + +int +module_resolver::ModuleRepoRequest (Cody::Server *s) +{ + s->PathnameResponse (repo); + return 0; +} + +int +module_resolver::cmi_response (Cody::Server *s, std::string &module) +{ + auto iter = map.find (module); + if (iter == map.end ()) + { + std::string file; + if (default_map) + file = std::move (GetCMIName (module)); + auto res = map.emplace (module, file); + iter = res.first; + } + + if (iter->second.empty ()) + s->ErrorResponse ("no such module"); + else + s->PathnameResponse (iter->second); + + return 0; +} + +int +module_resolver::ModuleExportRequest (Cody::Server *s, std::string &module) +{ + return cmi_response (s, module); +} + +int +module_resolver::ModuleImportRequest (Cody::Server *s, std::string &module) +{ + return cmi_response (s, module); +} + +int +module_resolver::IncludeTranslateRequest (Cody::Server *s, std::string &include) +{ + auto iter = map.find (include); + if (iter == map.end () && default_translate) + { + // Not found, look for it + auto file = GetCMIName (include); + struct stat statbuf; + bool ok = true; + +#if HAVE_FSTATAT + int fd_dir = AT_FDCWD; + if (!repo.empty ()) + { + if (fd_repo == -1) + { + fd_repo = open (repo.c_str (), + O_RDONLY | O_CLOEXEC | O_DIRECTORY); + if (fd_repo < 0) + fd_repo = -2; + } + fd_dir = fd_repo; + } + + if (!repo.empty () && fd_repo < 0) + ok = false; + else if (fstatat (fd_dir, file.c_str (), &statbuf, 0) < 0 + || !S_ISREG (statbuf.st_mode)) + ok = false; +#else + auto append = repo; + append.push_back (DIR_SEPARATOR); + append.append (file); + if (stat (append.c_str (), &statbuf) < 0 + || !S_ISREG (statbuf.st_mode)) + ok = false; +#endif + if (!ok) + // Mark as not present + file.clear (); + auto res = map.emplace (include, file); + iter = res.first; + } + + if (iter == map.end () || iter->second.empty ()) + s->BoolResponse (false); + else + s->PathnameResponse (iter->second); + + return 0; +} + diff --git c/gcc/cp/mapper-server.cc w/gcc/cp/mapper-server.cc new file mode 100644 index 00000000000..f0d26810501 --- /dev/null +++ w/gcc/cp/mapper-server.cc @@ -0,0 +1,968 @@ +/* C++ modules. Experimental! + Copyright (C) 2018-2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +#include "config.h" +#include "mapper.h" + +// C++ +#include +#include +// GCC +#define INCLUDE_VECTOR +#define INCLUDE_MAP +#define INCLUDE_SET + +// Network +/* Include network stuff first. Excitingly OSX10.14 uses bcmp here, which + we poison later! */ +#if defined (HAVE_AF_UNIX) || defined (HAVE_AF_INET6) +/* socket, bind, listen, accept{4} */ +# define NETWORKING 1 +# include +# ifdef HAVE_AF_UNIX +/* sockaddr_un */ +# include +# endif +# include +# ifdef HAVE_AF_INET6 +/* sockaddr_in6, getaddrinfo, freeaddrinfo, gai_sterror, ntohs, htons. */ +# include +# endif +#ifdef HAVE_INET_NTOP +/* inet_ntop. */ +#include +#endif +#endif +#ifndef HAVE_AF_INET6 +# define gai_strerror(X) "" +#endif + +// Excitingly Darwin uses bcmp in its network headers, and we poison +// that in our setup. +#include "system.h" +#include "version.h" +#include "intl.h" +#include + +// Select or epoll +#ifdef NETWORKING +#ifdef HAVE_EPOLL +/* epoll_create, epoll_ctl, epoll_pwait */ +#include +#endif +#if defined (HAVE_PSELECT) || defined (HAVE_SELECT) +/* pselect or select */ +#include +#endif +#endif + +#if !HOST_HAS_O_CLOEXEC +#define O_CLOEXEC 0 +#endif + +#ifndef HAVE_SIGHANDLER_T +typedef void (*sighandler_t) (int); +#endif + +#ifdef NETWORKING +struct netmask { + in6_addr addr; + unsigned bits; + + netmask (const in6_addr &a, unsigned b) + { + if (b > sizeof (in6_addr) * 8) + b = sizeof (in6_addr) * 8; + bits = b; + unsigned byte = (b + 7) / 8; + unsigned ix = 0; + for (ix = 0; ix < byte; ix++) + addr.s6_addr[ix] = a.s6_addr[ix]; + for (; ix != sizeof (in6_addr); ix++) + addr.s6_addr[ix] = 0; + if (b & 3) + addr.s6_addr[b/7] &= (255 << 8) >> (b & 3); + } + + bool includes (const in6_addr &a) const + { + unsigned byte = bits / 8; + for (unsigned ix = 0; ix != byte; ix++) + if (addr.s6_addr[ix] != a.s6_addr[ix]) + return false; + if (bits & 3) + if ((addr.s6_addr[byte] ^ a.s6_addr[byte]) >> (8 - (bits & 3))) + return false; + return true; + } +}; + +/* Netmask comparison. */ +struct netmask_cmp { + bool operator() (const netmask &a, const netmask &b) const + { + if (a.bits != b.bits) + return a.bits < b.bits; + for (unsigned ix = 0; ix != sizeof (in6_addr); ix++) + if (a.addr.s6_addr[ix] != b.addr.s6_addr[ix]) + return a.addr.s6_addr[ix] < b.addr.s6_addr[ix]; + return false; + } +}; + +typedef std::set netmask_set_t; +typedef std::vector netmask_vec_t; +#endif + +const char *progname; + +/* Speak thoughts out loud. */ +static bool flag_noisy = false; + +/* One and done. */ +static bool flag_one = false; + +/* Serialize connections. */ +static bool flag_sequential = false; + +/* Fallback to default if map file is unrewarding. */ +static bool flag_map = false; + +/* Fallback to xlate if map file is unrewarding. */ +static bool flag_xlate = false; + +/* Root binary directory. */ +static const char *flag_root = "gcm.cache"; + +#ifdef NETWORKING +static netmask_set_t netmask_set; + +static netmask_vec_t accept_addrs; +#endif + +/* Strip out the source directory from FILE. */ + +static const char * +trim_src_file (const char *file) +{ + static const char me[] = __FILE__; + unsigned pos = 0; + + while (file[pos] == me[pos] && me[pos]) + pos++; + while (pos && !IS_DIR_SEPARATOR (me[pos-1])) + pos--; + + return file + pos; +} + +/* Die screaming. */ + +void ATTRIBUTE_NORETURN ATTRIBUTE_PRINTF_1 ATTRIBUTE_COLD +internal_error (const char *fmt, ...) +{ + fprintf (stderr, "%s:Internal error ", progname); + va_list args; + + va_start (args, fmt); + vfprintf (stderr, fmt, args); + va_end (args); + fprintf (stderr, "\n"); + + exit (FATAL_EXIT_CODE); +} + +/* Hooked to from gcc_assert & gcc_unreachable. */ + +void ATTRIBUTE_NORETURN ATTRIBUTE_COLD +fancy_abort (const char *file, int line, const char *func) +{ + internal_error ("in %s, at %s:%d", func, trim_src_file (file), line); +} + +/* Exploded on a signal. */ + +static void ATTRIBUTE_NORETURN ATTRIBUTE_COLD +crash_signal (int sig) +{ + signal (sig, SIG_DFL); + internal_error ("signal %s", strsignal (sig)); +} + +/* A fatal error of some kind. */ + +static void ATTRIBUTE_NORETURN ATTRIBUTE_COLD ATTRIBUTE_PRINTF_1 +error (const char *msg, ...) +{ + fprintf (stderr, "%s:error: ", progname); + va_list args; + + va_start (args, msg); + vfprintf (stderr, msg, args); + va_end (args); + fprintf (stderr, "\n"); + + exit (1); +} + +#ifdef NETWORKING +/* Progress messages to the user. */ +static bool ATTRIBUTE_PRINTF_1 ATTRIBUTE_COLD +noisy (const char *fmt, ...) +{ + fprintf (stderr, "%s:", progname); + va_list args; + va_start (args, fmt); + vfprintf (stderr, fmt, args); + va_end (args); + fprintf (stderr, "\n"); + + return false; +} +#endif + +/* More messages to the user. */ + +static void ATTRIBUTE_PRINTF_2 +fnotice (FILE *file, const char *fmt, ...) +{ + va_list args; + + va_start (args, fmt); + vfprintf (file, _(fmt), args); + va_end (args); +} + +static void ATTRIBUTE_NORETURN +print_usage (int error_p) +{ + FILE *file = error_p ? stderr : stdout; + int status = error_p ? FATAL_EXIT_CODE : SUCCESS_EXIT_CODE; + + fnotice (file, "Usage: cxx-mapper [OPTION...] [CONNECTION] [MAPPINGS...] \n\n"); + fnotice (file, "C++ Module Mapper.\n\n"); + fnotice (file, " -a, --accept Netmask to accept from\n"); + fnotice (file, " -f, --fallback Use fallback for missing mappings\n"); + fnotice (file, " -h, --help Print this help, then exit\n"); + fnotice (file, " -n, --noisy Print progress messages\n"); + fnotice (file, " -1, --one One connection and then exit\n"); + fnotice (file, " -r, --root DIR Root compiled module directory\n"); + fnotice (file, " -s, --sequential Process connections sequentially\n"); + fnotice (file, " -v, --version Print version number, then exit\n"); + fnotice (file, "Send SIGTERM(%d) to terminate\n", SIGTERM); + fnotice (file, "\nFor bug reporting instructions, please see:\n%s.\n", + bug_report_url); + exit (status); +} + +/* Print version information and exit. */ + +static void ATTRIBUTE_NORETURN +print_version (void) +{ + fnotice (stdout, "cxx-mapper %s%s\n", pkgversion_string, version_string); + fprintf (stdout, "Copyright %s 2018-2020 Free Software Foundation, Inc.\n", + _("(C)")); + fnotice (stdout, + _("This is free software; see the source for copying conditions.\n" + "There is NO warranty; not even for MERCHANTABILITY or \n" + "FITNESS FOR A PARTICULAR PURPOSE.\n\n")); + exit (SUCCESS_EXIT_CODE); +} + +/* ARG is a netmask to accept from. Add it to the table. Return + false if we fail to resolve it. */ + +static bool +accept_from (char *arg ATTRIBUTE_UNUSED) +{ + bool ok = true; +#if HAVE_AF_INET6 + unsigned bits = sizeof (in6_addr) * 8; + char *slash = strrchr (arg, '/'); + if (slash) + { + *slash = 0; + if (slash[1]) + { + char *endp; + bits = strtoul (slash + 1, &endp, 0); + } + } + + addrinfo hints; + + hints.ai_flags = AI_NUMERICSERV; + hints.ai_family = AF_INET6; + hints.ai_socktype = SOCK_STREAM; + hints.ai_protocol = 0; + hints.ai_addrlen = 0; + hints.ai_addr = NULL; + hints.ai_canonname = NULL; + hints.ai_next = NULL; + + struct addrinfo *addrs = NULL; + if (int e = getaddrinfo (slash == arg ? NULL : arg, "0", &hints, &addrs)) + { + noisy ("cannot resolve '%s': %s", arg, gai_strerror (e)); + ok = false; + } + else + for (addrinfo *next = addrs; next; next = next->ai_next) + if (next->ai_family == AF_INET6) + { + netmask mask (((const sockaddr_in6 *)next->ai_addr)->sin6_addr, bits); + netmask_set.insert (mask); + } + freeaddrinfo (addrs); +#endif + return ok; +} + +/* Process args, return index to first non-arg. */ + +static int +process_args (int argc, char **argv) +{ + static const struct option options[] = + { + { "accept", required_argument, NULL, 'a' }, + { "help", no_argument, NULL, 'h' }, + { "map", no_argument, NULL, 'm' }, + { "noisy", no_argument, NULL, 'n' }, + { "one", no_argument, NULL, '1' }, + { "root", required_argument, NULL, 'r' }, + { "sequential", no_argument, NULL, 's' }, + { "translate",no_argument, NULL, 't' }, + { "version", no_argument, NULL, 'v' }, + { 0, 0, 0, 0 } + }; + int opt; + bool bad_accept = false; + const char *opts = "a:fhmn1r:stv"; + while ((opt = getopt_long (argc, argv, opts, options, NULL)) != -1) + { + switch (opt) + { + case 'a': + if (!accept_from (optarg)) + bad_accept = true; + break; + case 'h': + print_usage (false); + /* print_usage will exit. */ + case 'f': // deprecated alias + case 'm': + flag_map = true; + break; + case 'n': + flag_noisy = true; + break; + case '1': + flag_one = true; + break; + case 'r': + flag_root = optarg; + break; + case 's': + flag_sequential = true; + break; + case 't': + flag_xlate = true; + break; + case 'v': + print_version (); + /* print_version will exit. */ + default: + print_usage (true); + /* print_usage will exit. */ + } + } + + if (bad_accept) + error ("failed to resolve all accept addresses"); + + return optind; +} + +#ifdef NETWORKING + +/* Manipulate the EPOLL state, or do nothing, if there is epoll. */ + +#ifdef HAVE_EPOLL +static inline void +do_epoll_ctl (int epoll_fd, int code, int event, int fd, unsigned data) +{ + epoll_event ev; + ev.events = event; + ev.data.u32 = data; + if (epoll_ctl (epoll_fd, code, fd, &ev)) + { + noisy ("epoll_ctl error:%s", xstrerror (errno)); + gcc_unreachable (); + } +} +#define my_epoll_ctl(EFD,C,EV,FD,CL) \ + ((EFD) >= 0 ? do_epoll_ctl (EFD,C,EV,FD,CL) : (void)0) +#else +#define my_epoll_ctl(EFD,C,EV,FD,CL) ((void)(EFD), (void)(FD), (void)(CL)) +#endif + +/* We increment this to tell the server to shut down. */ +static volatile int term = false; +static volatile int kill_sock_fd = -1; +#if !defined (HAVE_PSELECT) && defined (HAVE_SELECT) +static int term_pipe[2] = {-1, -1}; +#else +#define term_pipe ((int *)NULL) +#endif + +/* A terminate signal. Shutdown gracefully. */ + +static void +term_signal (int sig) +{ + signal (sig, term_signal); + term = term + 1; + if (term_pipe && term_pipe[1] >= 0) + write (term_pipe[1], &term_pipe[1], 1); +} + +/* A kill signal. Shutdown immediately. */ + +static void +kill_signal (int sig) +{ + signal (sig, SIG_DFL); + int sock_fd = kill_sock_fd; + if (sock_fd >= 0) + close (sock_fd); + exit (2); +} + +bool process_server (Cody::Server *server, unsigned slot, int epoll_fd) +{ + switch (server->GetDirection ()) + { + case Cody::Server::READING: + if (int err = server->Read ()) + return !(err == EINTR || err == EAGAIN); + server->ProcessRequests (); + server->PrepareToWrite (); + break; + + case Cody::Server::WRITING: + if (int err = server->Write ()) + return !(err == EINTR || err == EAGAIN); + server->PrepareToRead (); + break; + + default: + // We should never get here + return true; + } + + // We've changed direction, so update epoll + gcc_assert (server->GetFDRead () == server->GetFDWrite ()); + my_epoll_ctl (epoll_fd, EPOLL_CTL_MOD, + server->GetDirection () == Cody::Server::READING + ? EPOLLIN : EPOLLOUT, server->GetFDRead (), slot + 1); + + return false; +} + +void close_server (Cody::Server *server, int epoll_fd) +{ + my_epoll_ctl (epoll_fd, EPOLL_CTL_DEL, EPOLLIN, server->GetFDRead (), 0); + + close (server->GetFDRead ()); + + delete server; +} + +int open_server (bool ip6, int sock_fd) +{ + sockaddr_in6 addr; + socklen_t addr_len = sizeof (addr); + +#ifdef HAVE_ACCEPT4 + int client_fd = accept4 (sock_fd, ip6 ? (sockaddr *)&addr : nullptr, + &addr_len, SOCK_NONBLOCK); +#else + int client_fd = accept (sock_fd, ip6 ? (sockaddr *)&addr : nullptr, &addr_len); +#endif + if (client_fd < 0) + { + error ("cannot accept: %s", xstrerror (errno)); + flag_one = true; + } + else if (ip6) + { + const char *str = NULL; +#if HAVE_INET_NTOP + char name[INET6_ADDRSTRLEN]; + str = inet_ntop (addr.sin6_family, &addr.sin6_addr, name, sizeof (name)); +#endif + if (!accept_addrs.empty ()) + { + netmask_vec_t::iterator e = accept_addrs.end (); + for (netmask_vec_t::iterator i = accept_addrs.begin (); + i != e; ++i) + if (i->includes (addr.sin6_addr)) + goto present; + close (client_fd); + client_fd = -1; + noisy ("Rejecting connection from disallowed source '%s'", + str ? str : ""); + present:; + } + if (client_fd >= 0) + flag_noisy && noisy ("Accepting connection from '%s'", str ? str : ""); + } + + return client_fd; +} + +/* A server listening on bound socket SOCK_FD. */ + +static void +server (bool ipv6, int sock_fd, module_resolver *resolver) +{ + int epoll_fd = -1; + + signal (SIGTERM, term_signal); +#ifdef HAVE_EPOLL + epoll_fd = epoll_create (1); +#endif + if (epoll_fd >= 0) + my_epoll_ctl (epoll_fd, EPOLL_CTL_ADD, EPOLLIN, sock_fd, 0); + +#if defined (HAVE_EPOLL) || defined (HAVE_PSELECT) || defined (HAVE_SELECT) + sigset_t mask; + { + sigset_t block; + sigemptyset (&block); + sigaddset (&block, SIGTERM); + sigprocmask (SIG_BLOCK, &block, &mask); + } +#endif + +#ifdef HAVE_EPOLL + const unsigned max_events = 20; + epoll_event events[max_events]; +#endif +#if defined (HAVE_PSELECT) || defined (HAVE_SELECT) + fd_set readers, writers; +#endif + if (term_pipe) + pipe (term_pipe); + + // We need stable references to servers, so this array can contain nulls + std::vector connections; + unsigned live = 0; + while (sock_fd >= 0 || live) + { + /* Wait for one or more events. */ + bool eintr = false; + int event_count; + + if (epoll_fd >= 0) + { +#ifdef HAVE_EPOLL + event_count = epoll_pwait (epoll_fd, events, max_events, -1, &mask); +#endif + } + else + { +#if defined (HAVE_PSELECT) || defined (HAVE_SELECT) + FD_ZERO (&readers); + FD_ZERO (&writers); + + unsigned limit = 0; + if (sock_fd >= 0 + && !(term || (live && (flag_one || flag_sequential)))) + { + FD_SET (sock_fd, &readers); + limit = sock_fd + 1; + } + + if (term_pipe && term_pipe[0] >= 0) + { + FD_SET (term_pipe[0], &readers); + if (unsigned (term_pipe[0]) >= limit) + limit = term_pipe[0] + 1; + } + + for (auto iter = connections.begin (); + iter != connections.end (); ++iter) + if (auto *server = *iter) + { + int fd = -1; + switch (server->GetDirection ()) + { + case Cody::Server::READING: + fd = server->GetFDRead (); + FD_SET (fd, &readers); + break; + case Cody::Server::WRITING: + fd = server->GetFDWrite (); + FD_SET (fd, &writers); + break; + default: + break; + } + + if (fd >= 0 && limit <= unsigned (fd)) + limit = fd + 1; + } + +#ifdef HAVE_PSELECT + event_count = pselect (limit, &readers, &writers, NULL, NULL, &mask); +#else + event_count = select (limit, &readers, &writers, NULL, NULL); +#endif + if (term_pipe && FD_ISSET (term_pipe[0], &readers)) + { + /* Fake up an interrupted system call. */ + event_count = -1; + errno = EINTR; + } +#endif + } + + if (event_count < 0) + { + // Error in waiting + if (errno == EINTR) + { + flag_noisy && noisy ("Interrupted wait"); + eintr = true; + } + else + error ("cannot %s: %s", epoll_fd >= 0 ? "epoll_wait" +#ifdef HAVE_PSELECT + : "pselect", +#else + : "select", +#endif + xstrerror (errno)); + event_count = 0; + } + + auto iter = connections.begin (); + while (event_count--) + { + // Process an event + int active = -2; + + if (epoll_fd >= 0) + { +#ifdef HAVE_EPOLL + /* See PR c++/88664 for why a temporary is used. */ + unsigned data = events[event_count].data.u32; + active = int (data) - 1; +#endif + } + else + { + for (; iter != connections.end (); ++iter) + if (auto *server = *iter) + { + bool found = false; + switch (server->GetDirection ()) + { +#if defined (HAVE_PSELECT) || defined (HAVE_SELECT) + case Cody::Server::READING: + found = FD_ISSET (server->GetFDRead (), &readers); + break; + case Cody::Server::WRITING: + found = FD_ISSET (server->GetFDWrite (), &writers); + break; +#endif + default: + break; + } + + if (found) + { + active = iter - connections.begin (); + ++iter; + break; + } + } + + if (active < 0 && sock_fd >= 0 && FD_ISSET (sock_fd, &readers)) + active = -1; + } + + if (active >= 0) + { + // Do the action + auto *server = connections[active]; + if (process_server (server, active, epoll_fd)) + { + connections[active] = nullptr; + close_server (server, epoll_fd); + live--; + if (flag_sequential) + my_epoll_ctl (epoll_fd, EPOLL_CTL_ADD, EPOLLIN, sock_fd, 0); + } + } + else if (active == -1 && !eintr) + { + // New connection + int fd = open_server (ipv6, sock_fd); + if (fd >= 0) + { +#if !defined (HAVE_ACCEPT4) \ + && (defined (HAVE_EPOLL) || defined (HAVE_PSELECT) || defined (HAVE_SELECT)) + int flags = fcntl (fd, F_GETFL, 0); + fcntl (fd, F_SETFL, flags | O_NONBLOCK); +#endif + auto *server = new Cody::Server (resolver, fd); + + unsigned slot = connections.size (); + if (live == slot) + connections.push_back (server); + else + for (auto iter = connections.begin (); ; ++iter) + if (!*iter) + { + *iter = server; + slot = iter - connections.begin (); + break; + } + live++; + my_epoll_ctl (epoll_fd, EPOLL_CTL_ADD, EPOLLIN, fd, slot + 1); + } + } + + if (sock_fd >= 0 + && (term || (live && (flag_one || flag_sequential)))) + { + /* Stop paying attention to sock_fd. */ + my_epoll_ctl (epoll_fd, EPOLL_CTL_DEL, EPOLLIN, sock_fd, 0); + if (flag_one || term) + { + close (sock_fd); + sock_fd = -1; + } + } + } + } +#if defined (HAVE_EPOLL) || defined (HAVE_PSELECT) || defined (HAVE_SELECT) + /* Restore the signal mask. */ + sigprocmask (SIG_SETMASK, &mask, NULL); +#endif + + gcc_assert (sock_fd < 0); + if (epoll_fd >= 0) + close (epoll_fd); + + if (term_pipe && term_pipe[0] >= 0) + { + close (term_pipe[0]); + close (term_pipe[1]); + } +} + +#endif + +static int maybe_parse_socket (std::string &option, module_resolver *r) +{ + /* Local or ipv6 address. */ + auto last = option.find_last_of ('?'); + if (last != option.npos) + { + r->set_ident (option.c_str () + last + 1); + option.erase (last); + } + int fd = -2; + char const *errmsg = nullptr; + + /* Does it look like a socket? */ + if (option[0] == '=') + { + /* A local socket. */ +#if CODY_NETWORKING + fd = Cody::ListenLocal (&errmsg, option.c_str () + 1); +#endif + } + else + { + auto colon = option.find_last_of (':'); + if (colon != option.npos) + { + /* Try a hostname:port address. */ + char const *cptr = option.c_str () + colon; + char *endp; + unsigned port = strtoul (cptr + 1, &endp, 10); + + if (port && endp != cptr + 1 && !*endp) + { + /* Ends in ':number', treat as ipv6 domain socket. */ + option.erase (colon); +#if CODY_NETWORKING + fd = Cody::ListenInet6 (&errmsg, option.c_str (), port); +#endif + } + } + } + + if (errmsg) + error ("failed to open socket: %s", errmsg); + + return fd; +} + +int +main (int argc, char *argv[]) +{ + const char *p = argv[0] + strlen (argv[0]); + while (p != argv[0] && !IS_DIR_SEPARATOR (p[-1])) + --p; + progname = p; + + xmalloc_set_program_name (progname); + +#ifdef SIGSEGV + signal (SIGSEGV, crash_signal); +#endif +#ifdef SIGILL + signal (SIGILL, crash_signal); +#endif +#ifdef SIGBUS + signal (SIGBUS, crash_signal); +#endif +#ifdef SIGABRT + signal (SIGABRT, crash_signal); +#endif +#ifdef SIGFPE + signal (SIGFPE, crash_signal); +#endif +#ifdef SIGPIPE + /* Ignore sigpipe, so read/write get an error. */ + signal (SIGPIPE, SIG_IGN); +#endif +#ifdef NETWORKING +#ifdef SIGINT + signal (SIGINT, kill_signal); +#endif +#endif + + int argno = process_args (argc, argv); + + std::string name; + int sock_fd = -1; /* Socket fd, otherwise stdin/stdout. */ + module_resolver r (flag_map, flag_xlate); + + if (argno != argc) + { + name = argv[argno]; + sock_fd = maybe_parse_socket (name, &r); + if (!name.empty ()) + argno++; + } + + if (argno != argc) + for (; argno != argc; argno++) + { + std::string option = argv[argno]; + char const *prefix = nullptr; + auto ident = option.find_last_of ('?'); + if (ident != option.npos) + { + prefix = option.c_str () + ident + 1; + option[ident] = 0; + } + int fd = open (option.c_str (), O_RDONLY | O_CLOEXEC); + int err = 0; + if (fd < 0) + err = errno; + else + { + err = r.read_tuple_file (fd, prefix, false); + close (fd); + } + + if (err) + error ("failed reading '%s': %s", option.c_str (), xstrerror (err)); + } + else + r.set_default_map (true); + + if (flag_root) + r.set_repo (flag_root); + +#ifdef HAVE_AF_INET6 + netmask_set_t::iterator end = netmask_set.end (); + for (netmask_set_t::iterator iter = netmask_set.begin (); + iter != end; ++iter) + { + netmask_vec_t::iterator e = accept_addrs.end (); + for (netmask_vec_t::iterator i = accept_addrs.begin (); i != e; ++i) + if (i->includes (iter->addr)) + goto present; + accept_addrs.push_back (*iter); + present:; + } +#endif + +#ifdef NETWORKING + if (sock_fd >= 0) + { + server (name[0] != '=', sock_fd, &r); + if (name[0] == '=') + unlink (name.c_str () + 1); + } + else +#endif + { + gcc_assert (sock_fd < 0); + auto server = Cody::Server (&r, 0, 1); + + int err = 0; + for (;;) + { + server.PrepareToRead (); + while ((err = server.Read ())) + { + if (err == EINTR || err == EAGAIN) + continue; + goto done; + } + + server.ProcessRequests (); + + server.PrepareToWrite (); + while ((err = server.Write ())) + { + if (err == EINTR || err == EAGAIN) + continue; + goto done; + } + } + done:; + if (err > 0) + error ("communication error:%s", xstrerror (err)); + } + + return 0; +} diff --git c/gcc/cp/mapper.h w/gcc/cp/mapper.h new file mode 100644 index 00000000000..86562252b53 --- /dev/null +++ w/gcc/cp/mapper.h @@ -0,0 +1,122 @@ +/* C++ modules. Experimental! -*- c++ -*- + Copyright (C) 2017-2020 Free Software Foundation, Inc. + Written by Nathan Sidwell while at FaceBook + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + +You should have received a copy of the GNU General Public License +along with GCC; see the file COPYING3. If not see +. */ + +// Mapper interface for client and server bits +#include "cody.hh" +// C++ +#include +#include + +// This is a GCC class, so GCC coding conventions on new bits. +class module_resolver : public Cody::Resolver +{ +public: + using parent = Cody::Resolver; + using module_map = std::map; + +private: + std::string repo; + std::string ident; + module_map map; + int fd_repo = -1; + bool default_map = true; + bool default_translate = true; + +public: + module_resolver (bool map = true, bool xlate = false); + virtual ~module_resolver () override; + +public: + void set_default_map (bool d) + { + default_map = d; + } + void set_default_translate (bool d) + { + default_translate = d; + } + void set_ident (char const *i) + { + ident = i; + } + bool set_repo (std::string &&repo, bool force = false); + bool add_mapping (std::string &&module, std::string &&file, + bool force = false); + + // Return +ve line number of error, or -ve errno + int read_tuple_file (int fd, char const *prefix, bool force = false); + int read_tuple_file (int fd, std::string const &prefix, + bool force = false) + { + return read_tuple_file (fd, prefix.empty () ? nullptr : prefix.c_str (), + force); + } + +public: + // Virtual overriders, names are controlle by Cody::Resolver + virtual module_resolver *ConnectRequest (Cody::Server *, unsigned version, + std::string &agent, + std::string &ident) + override; + virtual int ModuleRepoRequest (Cody::Server *) override; + virtual int ModuleExportRequest (Cody::Server *s, std::string &module) + override; + virtual int ModuleImportRequest (Cody::Server *s, std::string &module) + override; + virtual int IncludeTranslateRequest (Cody::Server *s, std::string &include) + override; + +private: + virtual char const *GetCMISuffix () override; + +private: + int cmi_response (Cody::Server *s, std::string &module); +}; + +#ifdef MAPPER_FOR_GCC +#ifndef HAVE_SIGHANDLER_T +typedef void (*sighandler_t) (int); +#endif + +class module_client : public Cody::Client +{ + pex_obj *pex = nullptr; + sighandler_t sigpipe = SIG_IGN; + +public: + module_client (Cody::Server *s) + : Client (s) + { + } + module_client (pex_obj *pex, int fd_from, int fd_to); + + module_client (int fd_from, int fd_to) + : Client (fd_from, fd_to) + { + } + +public: + static module_client *open_module_client (location_t loc, const char *option, + void (*set_repo) (const char *), + char const *); + static void close_module_client (location_t loc, module_client *); +}; + +#endif