From patchwork Wed Nov 25 23:07:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Hubicka X-Patchwork-Id: 548810 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 6437F1402B8 for ; Thu, 26 Nov 2015 10:08:12 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b=swPW2//z; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; q=dns; s=default; b=xOBci882tihBrBMK1 rb1gqbD+rRwmKjpxnU4IumBHilXAvsGmwr4IsiP6kzTOwXxtFGXcXoDHrj5YB+NI Z7HFx27M0iH1KduH/RM9aey3SUJK9WNn6RQovNZ8qPo1iBOJOZAOZlJzJBHLgvAx zVCQUWPAB+bOSNtaQupqfIr0sQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=default; bh=689s4U8Fmklzm14O+UQi1Uj ZKq4=; b=swPW2//zzNs6C8FsL3ERZp+urwNitQV06IxGv7dByuWXJM8ZXAL7UXs RtQYMNdMUF3cvLzpqC3BBBOc/Qd2UtCwQEo+XW6sJbvOaM+LQtw90snYZ11LlInj 2pShMwL4wnp4kzN04n5OQxc+vh3fkD9rh8vKx6s/CNLTQWsezwD4= Received: (qmail 68189 invoked by alias); 25 Nov 2015 23:08:05 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 68168 invoked by uid 89); 25 Nov 2015 23:08:03 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.1 required=5.0 tests=AWL, BAYES_00, KAM_ASCII_DIVIDERS, KAM_LAZY_DOMAIN_SECURITY, RP_MATCHES_RCVD autolearn=no version=3.3.2 X-HELO: nikam.ms.mff.cuni.cz Received: from nikam.ms.mff.cuni.cz (HELO nikam.ms.mff.cuni.cz) (195.113.20.16) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Wed, 25 Nov 2015 23:08:02 +0000 Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id A40BF541E21; Thu, 26 Nov 2015 00:07:58 +0100 (CET) Date: Thu, 26 Nov 2015 00:07:58 +0100 From: Jan Hubicka To: Jan Hubicka Cc: "H.J. Lu" , Richard Biener , GCC Patches , Andi Kleen , Cary Coutant , Ian Lance Taylor Subject: Re: [RFC] Getting LTO incremental linking work Message-ID: <20151125230758.GF20593@kam.mff.cuni.cz> References: <20151125085912.GD58491@kam.mff.cuni.cz> <20151125191740.GB5371@kam.mff.cuni.cz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20151125191740.GB5371@kam.mff.cuni.cz> User-Agent: Mutt/1.5.21 (2010-09-15) Hi, this is the first part of patch that adds -flinker-output flags and gets symbol visibility right. It makes the testcase in the PR to pass, but I do not know how to turn it into a testsuite ready version. I remember there was other PRs related to incremental linking and symbol visibility, I will try to find them and see if any of those can be turned into a testcase. I think this may even be backportable to GCC 5 if it does not cause any new issues. Also the collect2 path can probably indeed be updated, it has info about what type of binary it produces. I will try to look into it incrementally. Bootstrapped/regtested x86_64-linux, comitted. Honza PR lto/67548 * lto-plugin.c (linker_output, linker_output_set): New statics. (all_symbols_read_handler): Add -flinker-output option. (onload): Record linker_output info. * ipa-visibility.c (cgraph_externally_visible_p, varpool_node::externally_visible_p): When doing incremental linking, hidden symbols may be still used later. (update_visibility_by_resolution_info): Do not drop weak during incremental link. (function_and_variable_visibility): Fix formating. * flag-types.h (lto_linker_output): Declare. * common.opt 9flag_incremental_link): New flag. * lto-lang.c (lto_post_options): Process flag_lto_linker_output. * lang.opt (lto_linker_output): New enum. (flinker_output): New flag. Index: lto-plugin/lto-plugin.c =================================================================== --- lto-plugin/lto-plugin.c (revision 230897) +++ lto-plugin/lto-plugin.c (working copy) @@ -167,6 +167,8 @@ static unsigned int num_pass_through_ite static char debug; static char nop; static char *resolution_file = NULL; +static enum ld_plugin_output_file_type linker_output; +static int linker_output_set; /* The version of gold being used, or -1 if not gold. The number is MAJOR * 100 + MINOR. */ @@ -624,8 +626,9 @@ all_symbols_read_handler (void) { unsigned i; unsigned num_lto_args - = num_claimed_files + num_offload_files + lto_wrapper_num_args + 1; + = num_claimed_files + num_offload_files + lto_wrapper_num_args + 2; char **lto_argv; + const char *linker_output_str; const char **lto_arg_ptr; if (num_claimed_files + num_offload_files == 0) return LDPS_OK; @@ -648,6 +651,26 @@ all_symbols_read_handler (void) for (i = 0; i < lto_wrapper_num_args; i++) *lto_arg_ptr++ = lto_wrapper_argv[i]; + assert (linker_output_set); + switch (linker_output) + { + case LDPO_REL: + linker_output_str = "-flinker-output=rel"; + break; + case LDPO_DYN: + linker_output_str = "-flinker-output=dyn"; + break; + case LDPO_PIE: + linker_output_str = "-flinker-output=pie"; + break; + case LDPO_EXEC: + linker_output_str = "-flinker-output=exec"; + break; + default: + message (LDPL_FATAL, "unsupported linker output %i", linker_output); + break; + } + *lto_arg_ptr++ = xstrdup (linker_output_str); for (i = 0; i < num_claimed_files; i++) { struct plugin_file_info *info = &claimed_files[i]; @@ -1100,6 +1123,10 @@ onload (struct ld_plugin_tv *tv) case LDPT_GOLD_VERSION: gold_version = p->tv_u.tv_val; break; + case LDPT_LINKER_OUTPUT: + linker_output = (enum ld_plugin_output_file_type) p->tv_u.tv_val; + linker_output_set = 1; + break; default: break; } Index: gcc/lto/lto-lang.c =================================================================== --- gcc/lto/lto-lang.c (revision 230902) +++ gcc/lto/lto-lang.c (working copy) @@ -819,6 +819,35 @@ lto_post_options (const char **pfilename if (flag_wpa) flag_generate_lto = 1; + /* Initialize the codegen flags according to the output type. */ + switch (flag_lto_linker_output) + { + case LTO_LINKER_OUTPUT_REL: /* .o: incremental link producing LTO IL */ + flag_whole_program = 0; + flag_incremental_link = 1; + break; + + case LTO_LINKER_OUTPUT_DYN: /* .so: PID library */ + /* On some targets, like i386 it makes sense to build PIC library wihout + -fpic for performance reasons. So no need to adjust flags. */ + break; + + case LTO_LINKER_OUTPUT_PIE: /* PIE binary */ + /* If -fPIC or -fPIE was used at compile time, be sure that + flag_pie is 2. */ + flag_pie = MAX (flag_pie, flag_pic); + flag_pic = 0; + break; + + case LTO_LINKER_OUTPUT_EXEC: /* Normal executable */ + flag_pic = 0; + flag_pie = 0; + break; + + case LTO_LINKER_OUTPUT_UNKNOWN: + break; + } + /* Excess precision other than "fast" requires front-end support. */ flag_excess_precision_cmdline = EXCESS_PRECISION_FAST; Index: gcc/lto/lang.opt =================================================================== --- gcc/lto/lang.opt (revision 230902) +++ gcc/lto/lang.opt (working copy) @@ -24,6 +24,29 @@ Language LTO +Enum +Name(lto_linker_output) Type(enum lto_linker_output) UnknownError(unknown linker output %qs) + +EnumValue +Enum(lto_linker_output) String(unknown) Value(LTO_LINKER_OUTPUT_UNKNOWN) + +EnumValue +Enum(lto_linker_output) String(rel) Value(LTO_LINKER_OUTPUT_REL) + +EnumValue +Enum(lto_linker_output) String(dyn) Value(LTO_LINKER_OUTPUT_DYN) + +EnumValue +Enum(lto_linker_output) String(pie) Value(LTO_LINKER_OUTPUT_PIE) + +EnumValue +Enum(lto_linker_output) String(exec) Value(LTO_LINKER_OUTPUT_EXEC) + +flinker-output= +LTO Report Driver Joined RejectNegative Enum(lto_linker_output) Var(flag_lto_linker_output) Init(LTO_LINKER_OUTPUT_UNKNOWN) +Set linker output type (used internally during LTO optimization) + + fltrans LTO Report Var(flag_ltrans) Run the link-time optimizer in local transformation (LTRANS) mode. Index: gcc/flag-types.h =================================================================== --- gcc/flag-types.h (revision 230902) +++ gcc/flag-types.h (working copy) @@ -265,6 +265,14 @@ enum lto_partition_model { LTO_PARTITION_MAX = 4 }; +/* flag_lto_linker_output initialization values. */ +enum lto_linker_output { + LTO_LINKER_OUTPUT_UNKNOWN, + LTO_LINKER_OUTPUT_REL, + LTO_LINKER_OUTPUT_DYN, + LTO_LINKER_OUTPUT_PIE, + LTO_LINKER_OUTPUT_EXEC +}; /* gfortran -finit-real= values. */ Index: gcc/common.opt =================================================================== --- gcc/common.opt (revision 230902) +++ gcc/common.opt (working copy) @@ -46,6 +46,12 @@ int optimize_fast Variable bool in_lto_p = false +; This variable is set to non-0 only by LTO front-end. 1 indicates that +; the output produced will be used for incrmeental linking (thus weak symbols +; can still be bound). +Variable +int flag_incremental_link = 0 + ; 0 means straightforward implementation of complex divide acceptable. ; 1 means wide ranges of inputs must work for complex divide. ; 2 means C99-like requirements for complex multiply and divide. Index: ipa-visibility.c =================================================================== --- ipa-visibility.c (revision 230902) +++ ipa-visibility.c (working copy) @@ -217,13 +217,13 @@ cgraph_externally_visible_p (struct cgra This improves code quality and we know we will duplicate them at most twice (in the case that we are not using plugin and link with object file implementing same COMDAT) */ - if ((in_lto_p || whole_program) + if (((in_lto_p || whole_program) && !flag_incremental_link) && DECL_COMDAT (node->decl) && comdat_can_be_unshared_p (node)) return false; /* When doing link time optimizations, hidden symbols become local. */ - if (in_lto_p + if ((in_lto_p && !flag_incremental_link) && (DECL_VISIBILITY (node->decl) == VISIBILITY_HIDDEN || DECL_VISIBILITY (node->decl) == VISIBILITY_INTERNAL) /* Be sure that node is defined in IR file, not in other object @@ -293,13 +293,13 @@ varpool_node::externally_visible_p (void so this does not enable more optimization, but referring static var is faster for dynamic linking. Also this match logic hidding vtables from LTO symbol tables. */ - if ((in_lto_p || flag_whole_program) + if (((in_lto_p || flag_whole_program) && !flag_incremental_link) && DECL_COMDAT (decl) && comdat_can_be_unshared_p (this)) return false; /* When doing link time optimizations, hidden symbols become local. */ - if (in_lto_p + if (in_lto_p && !flag_incremental_link && (DECL_VISIBILITY (decl) == VISIBILITY_HIDDEN || DECL_VISIBILITY (decl) == VISIBILITY_INTERNAL) /* Be sure that node is defined in IR file, not in other object @@ -405,17 +405,36 @@ update_visibility_by_resolution_info (sy for (symtab_node *next = node->same_comdat_group; next != node; next = next->same_comdat_group) { - next->set_comdat_group (NULL); - DECL_WEAK (next->decl) = false; + /* During incremental linking we need to keep symbol weak for future + linking. We can still drop definition if we know non-LTO world + prevails. */ + if (!flag_incremental_link) + { + DECL_WEAK (next->decl) = false; + next->set_comdat_group (NULL); + } if (next->externally_visible && !define) - DECL_EXTERNAL (next->decl) = true; + { + DECL_EXTERNAL (next->decl) = true; + next->set_comdat_group (NULL); + } } - node->set_comdat_group (NULL); - DECL_WEAK (node->decl) = false; + + /* During incremental linking we need to keep symbol weak for future + linking. We can still drop definition if we know non-LTO world prevails. */ + if (!flag_incremental_link) + { + DECL_WEAK (node->decl) = false; + node->set_comdat_group (NULL); + node->dissolve_same_comdat_group_list (); + } if (!define) - DECL_EXTERNAL (node->decl) = true; - node->dissolve_same_comdat_group_list (); + { + DECL_EXTERNAL (node->decl) = true; + node->set_comdat_group (NULL); + node->dissolve_same_comdat_group_list (); + } } /* Decide on visibility of all symbols. */ @@ -639,8 +658,9 @@ function_and_variable_visibility (bool w { gcc_assert (in_lto_p || whole_program || !TREE_PUBLIC (vnode->decl)); vnode->unique_name = ((vnode->resolution == LDPR_PREVAILING_DEF_IRONLY - || vnode->resolution == LDPR_PREVAILING_DEF_IRONLY_EXP) - && TREE_PUBLIC (vnode->decl)); + || vnode->resolution + == LDPR_PREVAILING_DEF_IRONLY_EXP) + && TREE_PUBLIC (vnode->decl)); if (vnode->same_comdat_group && TREE_PUBLIC (vnode->decl)) { symtab_node *next = vnode;