From patchwork Wed Nov 10 04:24:52 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Hubicka X-Patchwork-Id: 70608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 9BE21B7133 for ; Wed, 10 Nov 2010 15:25:15 +1100 (EST) Received: (qmail 17594 invoked by alias); 10 Nov 2010 04:25:12 -0000 Received: (qmail 17472 invoked by uid 22791); 10 Nov 2010 04:25:06 -0000 X-SWARE-Spam-Status: No, hits=2.0 required=5.0 tests=AWL, BAYES_40, KAM_STOCKTIP, TW_CP, T_RP_MATCHES_RCVD X-Spam-Check-By: sourceware.org Received: from nikam-dmz.ms.mff.cuni.cz (HELO nikam.ms.mff.cuni.cz) (195.113.20.16) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Wed, 10 Nov 2010 04:24:55 +0000 Received: by nikam.ms.mff.cuni.cz (Postfix, from userid 16202) id 03FA99AC6BF; Wed, 10 Nov 2010 05:24:53 +0100 (CET) Date: Wed, 10 Nov 2010 05:24:52 +0100 From: Jan Hubicka To: Richard Henderson Cc: "Joseph S. Myers" , Jan Hubicka , gcc-patches@gcc.gnu.org, rguenther@suse.de, iant@google.com, iains@gcc.gnu.org, tglek@mozilla.com Subject: Re: Group static constructors and destructors in specific subsections, take 2 Message-ID: <20101110042452.GA2722@kam.mff.cuni.cz> References: <20101014203817.GB15487@kam.mff.cuni.cz> <4CB7716E.3000605@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <4CB7716E.3000605@redhat.com> User-Agent: Mutt/1.5.18 (2008-05-17) Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Hi, this is the updated patch that deals with section instead of section names as Richard requested and also uses hook instead of target macros. The hot/cold function partitioning made things somewhat harder since we have special code for cold sections all around. Baic idea is to have named_text_subsection to produce subsections for functions in generic way and function_section hook to decide proper subsection based on profile. I've bootstrapped/regtested the patch on x86_64-linux, I also checked that Hpux and dwarf output seems sane at a glance. Does the patch seem resonable? Honza * targhooks.h (default_function_section): Declare. * target.def (function_section): New hook. * defaults.h (HOT_TEXT_SECTION_NAME, UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Remove. * predict.c (choose_function_section): Remove. (estimate_bb_frequencies): Do not use choose_function_section. * coretypes.h (enum node_frequency): Move here from cgraph.h * cgraph.h (enum node_frequency): Remove. * varasm.c (initialize_cold_section_name, unlikely_text_section, unlikely_text_section_p): Remove. (named_subsection_entry): New structure. (subsection_hash): New static var. (eq_subsections): New function. (hash_subsection): New function. (named_text_subsection): New function. (default_function_section): New function. (function_section_1): Break out from ...; handle profile info. (function_section): ... here. (unlikely_text_section): Remove. (unlikely_text_section_p): Use function_section_1. (assemble_start_function): Do not initialize cold section. (default_section_type_flags): Do not special case cold subsection. (switch_to_section): Likewise. * output.h (named_text_subsection): Define. * config/i386/winnt.c: Do not special case cold section. * config/darwin-protos.h (darwin_function_section): Declare. * config/microblaze/microblaze.h (HOT_TEXT_SECTION_NAME, UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Remove. * config/ia64/hpux.h (HOT_TEXT_SECTION_NAME, UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Remove. (TARGET_ASM_FUNCTION_SECTOIN): Define to NULL. * config/darwin.c (machopic_select_section): Use darwin_function_section. (darwin_function_section): New function. * config/darwin.h (HOT_TEXT_SECTION_NAME, UNLIKELY_EXECUTED_TEXT_SECTION_NAME): Remove. (TARGET_ASM_FUNCTION_SECTION): Define. Index: doc/tm.texi =================================================================== --- doc/tm.texi (revision 166490) +++ doc/tm.texi (working copy) @@ -7335,6 +7335,8 @@ macro is not defined, nothing is output @end defmac @deftypefn {Target Hook} void TARGET_ASM_NAMED_SECTION (const char *@var{name}, unsigned int @var{flags}, tree @var{decl}) + +@deftypefn {Target Hook} {section *} TARGET_ASM_FUNCTION_SECTION (tree @var{decl}, enum node_frequency @var{freq}, bool @var{startup}, bool @var{exit}) Output assembly directives to switch to section @var{name}. The section should have attributes as specified by @var{flags}, which is a bit mask of the @code{SECTION_*} flags defined in @file{output.h}. If @var{decl} Index: targhooks.h =================================================================== --- targhooks.h (revision 166490) +++ targhooks.h (working copy) @@ -169,7 +169,8 @@ extern int default_label_align_after_bar extern int default_loop_align_max_skip (rtx); extern int default_label_align_max_skip (rtx); extern int default_jump_align_max_skip (rtx); - +extern section * default_function_section(tree decl, enum node_frequency freq, + bool startup, bool exit); extern enum machine_mode default_get_reg_raw_mode(int); extern const struct default_options empty_optimization_table[]; Index: target.def =================================================================== --- target.def (revision 166490) +++ target.def (working copy) @@ -281,6 +281,19 @@ DEFHOOK void, (const char *name, unsigned int flags, tree decl), default_no_named_section) +/* Return preferred text (sub)section for function DECL. + Main purpose of this function is to separate cold, normal and hot + functions. STARTUP is true when function is known to be used only + at startup (from static constructors or it is main()). + EXIT is true when function is known to be used only at exit + (from static destructors). + Return NULL if function should go to default text section. */ +DEFHOOK +(function_section, + "", + section *, (tree decl, enum node_frequency freq, bool startup, bool exit), + default_function_section) + /* Return a mask describing how relocations should be treated when selecting sections. Bit 1 should be set if global relocations should be placed in a read-write section; bit 0 should be set if Index: cgraph.h =================================================================== --- cgraph.h (revision 166490) +++ cgraph.h (working copy) @@ -179,20 +179,6 @@ struct GTY(()) cgraph_clone_info bitmap combined_args_to_skip; }; -enum node_frequency { - /* This function most likely won't be executed at all. - (set only when profile feedback is available or via function attribute). */ - NODE_FREQUENCY_UNLIKELY_EXECUTED, - /* For functions that are known to be executed once (i.e. constructors, destructors - and main function. */ - NODE_FREQUENCY_EXECUTED_ONCE, - /* The default value. */ - NODE_FREQUENCY_NORMAL, - /* Optimize this function hard - (set only when profile feedback is available or via function attribute). */ - NODE_FREQUENCY_HOT -}; - /* The cgraph data structure. Each function decl has assigned cgraph_node listing callees and callers. */ Index: defaults.h =================================================================== --- defaults.h (revision 166490) +++ defaults.h (working copy) @@ -897,14 +897,6 @@ see the files COPYING3 and COPYING.RUNTI #define TARGET_DEC_EVAL_METHOD 2 #endif -#ifndef HOT_TEXT_SECTION_NAME -#define HOT_TEXT_SECTION_NAME ".text.hot" -#endif - -#ifndef UNLIKELY_EXECUTED_TEXT_SECTION_NAME -#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME ".text.unlikely" -#endif - #ifndef HAS_LONG_COND_BRANCH #define HAS_LONG_COND_BRANCH 0 #endif Index: predict.c =================================================================== --- predict.c (revision 166490) +++ predict.c (working copy) @@ -78,7 +78,6 @@ static sreal real_zero, real_one, real_a static void combine_predictions_for_insn (rtx, basic_block); static void dump_prediction (FILE *, enum br_predictor, int, basic_block, int); static void predict_paths_leading_to (basic_block, enum br_predictor, enum prediction); -static void choose_function_section (void); static bool can_predict_insn_p (const_rtx); /* Information we hold about each branch predictor. @@ -2185,8 +2184,6 @@ estimate_bb_frequencies (void) free_aux_for_edges (); } compute_function_frequency (); - if (flag_reorder_functions) - choose_function_section (); } /* Decide whether function is hot, cold or unlikely executed. */ @@ -2232,35 +2229,6 @@ compute_function_frequency (void) } } -/* Choose appropriate section for the function. */ -static void -choose_function_section (void) -{ - struct cgraph_node *node = cgraph_node (current_function_decl); - if (DECL_SECTION_NAME (current_function_decl) - || !targetm.have_named_sections - /* Theoretically we can split the gnu.linkonce text section too, - but this requires more work as the frequency needs to match - for all generated objects so we need to merge the frequency - of all instances. For now just never set frequency for these. */ - || DECL_ONE_ONLY (current_function_decl)) - return; - - /* If we are doing the partitioning optimization, let the optimization - choose the correct section into which to put things. */ - - if (flag_reorder_blocks_and_partition) - return; - - if (node->frequency == NODE_FREQUENCY_HOT) - DECL_SECTION_NAME (current_function_decl) = - build_string (strlen (HOT_TEXT_SECTION_NAME), HOT_TEXT_SECTION_NAME); - if (node->frequency == NODE_FREQUENCY_UNLIKELY_EXECUTED) - DECL_SECTION_NAME (current_function_decl) = - build_string (strlen (UNLIKELY_EXECUTED_TEXT_SECTION_NAME), - UNLIKELY_EXECUTED_TEXT_SECTION_NAME); -} - static bool gate_estimate_probability (void) { Index: function.h =================================================================== --- function.h (revision 166490) +++ function.h (working copy) @@ -226,11 +226,6 @@ struct GTY(()) function_subsections { const char *cold_section_label; const char *hot_section_end_label; const char *cold_section_end_label; - - /* String to be used for name of cold text sections, via - targetm.asm_out.named_section. */ - - const char *unlikely_text_section_name; }; /* Describe an empty area of space in the stack frame. These can be chained Index: coretypes.h =================================================================== --- coretypes.h (revision 166490) +++ coretypes.h (working copy) @@ -121,6 +121,22 @@ enum unwind_info_type UI_TARGET }; +/* Callgraph node profile representation. */ +enum node_frequency { + /* This function most likely won't be executed at all. + (set only when profile feedback is available or via function attribute). */ + NODE_FREQUENCY_UNLIKELY_EXECUTED, + /* For functions that are known to be executed once (i.e. constructors, destructors + and main function. */ + NODE_FREQUENCY_EXECUTED_ONCE, + /* The default value. */ + NODE_FREQUENCY_NORMAL, + /* Optimize this function hard + (set only when profile feedback is available or via function attribute). */ + NODE_FREQUENCY_HOT +}; + + struct edge_def; typedef struct edge_def *edge; typedef const struct edge_def *const_edge; Index: varasm.c =================================================================== --- varasm.c (revision 166490) +++ varasm.c (working copy) @@ -368,69 +368,6 @@ create_block_symbol (const char *label, return symbol; } -static void -initialize_cold_section_name (void) -{ - const char *stripped_name; - char *name, *buffer; - tree dsn; - - gcc_assert (cfun && current_function_decl); - if (crtl->subsections.unlikely_text_section_name) - return; - - dsn = DECL_SECTION_NAME (current_function_decl); - if (flag_function_sections && dsn) - { - name = (char *) alloca (TREE_STRING_LENGTH (dsn) + 1); - memcpy (name, TREE_STRING_POINTER (dsn), TREE_STRING_LENGTH (dsn) + 1); - - stripped_name = targetm.strip_name_encoding (name); - - buffer = ACONCAT ((stripped_name, "_unlikely", NULL)); - crtl->subsections.unlikely_text_section_name = ggc_strdup (buffer); - } - else - crtl->subsections.unlikely_text_section_name = UNLIKELY_EXECUTED_TEXT_SECTION_NAME; -} - -/* Tell assembler to switch to unlikely-to-be-executed text section. */ - -section * -unlikely_text_section (void) -{ - if (cfun) - { - if (!crtl->subsections.unlikely_text_section_name) - initialize_cold_section_name (); - - return get_named_section (NULL, crtl->subsections.unlikely_text_section_name, 0); - } - else - return get_named_section (NULL, UNLIKELY_EXECUTED_TEXT_SECTION_NAME, 0); -} - -/* When called within a function context, return true if the function - has been assigned a cold text section and if SECT is that section. - When called outside a function context, return true if SECT is the - default cold section. */ - -bool -unlikely_text_section_p (section *sect) -{ - const char *name; - - if (cfun) - name = crtl->subsections.unlikely_text_section_name; - else - name = UNLIKELY_EXECUTED_TEXT_SECTION_NAME; - - return (name - && sect - && SECTION_STYLE (sect) == SECTION_NAMED - && strcmp (name, sect->named.name) == 0); -} - /* Return a section with a particular name and with whatever SECTION_* flags section_type_flags deems appropriate. The name of the section is taken from NAME if nonnull, otherwise it is taken from DECL's @@ -539,49 +476,202 @@ hot_function_section (tree decl) } #endif +/* Hash table used to cache names of function subsections. */ +struct GTY(()) named_subsection_entry +{ + section *sect; + tree section_name; + const char *suffix; +}; + +static GTY((param_is (struct named_subsection_entry))) htab_t subsection_hash; + +/* Returns nonzero if P1 and P2 are equal. */ + +static int +eq_subsections (const void *p1, const void *p2) +{ + const struct named_subsection_entry *n1 = (const struct named_subsection_entry *) p1; + const struct named_subsection_entry *n2 = (const struct named_subsection_entry *) p2; + return n1->section_name == n2->section_name && n1->suffix == n2->suffix; +} + +/* Compute hash value of subsection. */ +static unsigned int +hash_subsection (const void *p) +{ + const struct named_subsection_entry *n = (const struct named_subsection_entry *)p; + return htab_hash_pointer (n->section_name) ^ htab_hash_pointer (n->suffix); +} + +/* Return section for TEXT_SECITON_NAME if DECL or DECL_SECTION_NAME (DECL) + is NULL. + + If it is non-NULL, return subsection of DECL_SECTION_NAME (DECL) created + by appending NAMED_SECTION_SUFFIX. */ + +section * +named_text_subsection (tree decl, + const char *text_section_name, + const char *named_section_suffix) +{ + if (decl && DECL_SECTION_NAME (decl)) + { + tree dsn = DECL_SECTION_NAME (decl); + struct named_subsection_entry key, *entry, **slot; + + key.suffix = named_section_suffix; + key.section_name = dsn; + if (!subsection_hash) + subsection_hash = + htab_create_ggc (10, hash_subsection, eq_subsections, NULL); + slot = + (struct named_subsection_entry **) htab_find_slot (subsection_hash, + &key, INSERT); + if (!*slot) + { + const char *stripped_name; + char *name, *buffer; + name = (char *) alloca (TREE_STRING_LENGTH (dsn) + 1); + memcpy (name, TREE_STRING_POINTER (dsn), + TREE_STRING_LENGTH (dsn) + 1); + + stripped_name = targetm.strip_name_encoding (name); + + buffer = ACONCAT ((stripped_name, named_section_suffix, NULL)); + entry = ggc_alloc_named_subsection_entry (); + entry->section_name = dsn; + entry->suffix = named_section_suffix; + entry->sect = get_named_section (decl, buffer, 0); + *slot = entry; + } + return (*slot)->sect; + } + return get_named_section (decl, text_section_name, 0); +} + +/* Choose named function section based on its frequency. */ + +section * +default_function_section (tree decl, enum node_frequency freq, + bool startup, bool exit) +{ + /* Startup code should go to startup subsection unless it is + unlikely executed (this happens especially with function splitting + where we can split away unnecesary parts of static constructors. */ + if (startup && freq != NODE_FREQUENCY_UNLIKELY_EXECUTED) + return named_text_subsection (decl, ".text.startup", "_startup"); + + /* Similarly for exit. */ + if (exit && freq != NODE_FREQUENCY_UNLIKELY_EXECUTED) + return named_text_subsection (decl, ".text.exit", "_exit"); + + /* Group cold functions together, similarly for hot code. */ + switch (freq) + { + case NODE_FREQUENCY_UNLIKELY_EXECUTED: + return named_text_subsection (decl, "text.unlikely", "_unlikely"); + case NODE_FREQUENCY_HOT: + return named_text_subsection (decl, "text.hot", "_hot"); + default: + return NULL; + } +} + /* Return the section for function DECL. If DECL is NULL_TREE, return the text section. We can be passed - NULL_TREE under some circumstances by dbxout.c at least. */ + NULL_TREE under some circumstances by dbxout.c at least. -section * -function_section (tree decl) + If FORCE_COLD is true, return cold function section ignoring + the frequency info of cgraph_node. */ + +static section * +function_section_1 (tree decl, bool force_cold) { - int reloc = 0; + section *section = NULL; + enum node_frequency freq = NODE_FREQUENCY_NORMAL; + bool startup = false, exit = false; - if (first_function_block_is_cold) - reloc = 1; + if (decl) + { + struct cgraph_node *node = cgraph_node (decl); + + freq = node->frequency; + startup = node->only_called_at_startup; + exit = node->only_called_at_exit; + } + if (force_cold) + freq = NODE_FREQUENCY_UNLIKELY_EXECUTED; #ifdef USE_SELECT_SECTION_FOR_FUNCTIONS if (decl != NULL_TREE && DECL_SECTION_NAME (decl) != NULL_TREE) - return reloc ? unlikely_text_section () - : get_named_section (decl, NULL, 0); + { + if (targetm.asm_out.function_section) + section = targetm.asm_out.function_section (decl, freq, + startup, exit); + if (section) + return section; + return get_named_section (decl, NULL, 0); + } else - return targetm.asm_out.select_section (decl, reloc, DECL_ALIGN (decl)); + return targetm.asm_out.select_section + (decl, freq == NODE_FREQUENCY_UNLIKELY_EXECUTED, + DECL_ALIGN (decl)); #else - return reloc ? unlikely_text_section () : hot_function_section (decl); + if (targetm.asm_out.function_section) + section = targetm.asm_out.function_section (decl, freq, startup, exit); + if (section) + return section; + return hot_function_section (decl); #endif } +/* Return the section for function DECL. + + If DECL is NULL_TREE, return the text section. We can be passed + NULL_TREE under some circumstances by dbxout.c at least. */ + +section * +function_section (tree decl) +{ + /* Handle cases where function splitting code decides + to put function entry point into unlikely executed section + despite the fact that the function itself is not cold + (i.e. it is called rarely but contains a hot loop that is + better to live in hot subsection for the code locality). */ + return function_section_1 (decl, + first_function_block_is_cold + && flag_function_sections); +} + +/* Return the section for the current function, take IN_COLD_SECTION_P + into account. */ + section * current_function_section (void) { -#ifdef USE_SELECT_SECTION_FOR_FUNCTIONS - if (current_function_decl != NULL_TREE - && DECL_SECTION_NAME (current_function_decl) != NULL_TREE) - return in_cold_section_p ? unlikely_text_section () - : get_named_section (current_function_decl, - NULL, 0); - else - return targetm.asm_out.select_section (current_function_decl, - in_cold_section_p, - DECL_ALIGN (current_function_decl)); -#else - return (in_cold_section_p - ? unlikely_text_section () - : hot_function_section (current_function_decl)); -#endif + return function_section_1 (current_function_decl, in_cold_section_p); +} + +/* Tell assembler to switch to unlikely-to-be-executed text section. */ + +section * +unlikely_text_section (void) +{ + return function_section_1 (current_function_decl, true); +} + +/* When called within a function context, return true if the function + has been assigned a cold text section and if SECT is that section. + When called outside a function context, return true if SECT is the + default cold section. */ + +bool +unlikely_text_section_p (section *sect) +{ + return sect == function_section_1 (current_function_decl, true); } /* Return the read-only data section associated with function DECL. */ @@ -1483,16 +1573,10 @@ assemble_start_function (tree decl, cons else if (DECL_SECTION_NAME (decl)) { /* Calls to function_section rely on first_function_block_is_cold - being accurate. The first block may be cold even if we aren't - doing partitioning, if the entire function was decided by - choose_function_section (predict.c) to be cold. */ - - initialize_cold_section_name (); - - if (crtl->subsections.unlikely_text_section_name - && strcmp (TREE_STRING_POINTER (DECL_SECTION_NAME (decl)), - crtl->subsections.unlikely_text_section_name) == 0) - first_function_block_is_cold = true; + being accurate. */ + first_function_block_is_cold + = (cgraph_node (current_function_decl)->frequency + == NODE_FREQUENCY_UNLIKELY_EXECUTED); } in_cold_section_p = first_function_block_is_cold; @@ -5841,15 +5925,6 @@ default_section_type_flags (tree decl, c flags = SECTION_CODE; else if (decl && decl_readonly_section (decl, reloc)) flags = 0; - else if (current_function_decl - && cfun - && crtl->subsections.unlikely_text_section_name - && strcmp (name, crtl->subsections.unlikely_text_section_name) == 0) - flags = SECTION_CODE; - else if (!decl - && (!current_function_decl || !cfun) - && strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0) - flags = SECTION_CODE; else flags = SECTION_WRITE; @@ -6776,12 +6851,6 @@ switch_to_section (section *new_section) switch (SECTION_STYLE (new_section)) { case SECTION_NAMED: - if (cfun - && !crtl->subsections.unlikely_text_section_name - && strcmp (new_section->named.name, - UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0) - crtl->subsections.unlikely_text_section_name = UNLIKELY_EXECUTED_TEXT_SECTION_NAME; - targetm.asm_out.named_section (new_section->named.name, new_section->named.common.flags, new_section->named.decl); Index: output.h =================================================================== --- output.h (revision 166490) +++ output.h (working copy) @@ -264,6 +264,10 @@ extern bool default_assemble_integer (rt be outputable. */ extern bool assemble_integer (rtx, unsigned, unsigned, int); +/* Return section for TEXT_SECITON_NAME if DECL or DECL_SECTION_NAME (DECL) + is NULL. */ +extern section *named_text_subsection (tree, const char *, const char *); + /* An interface to assemble_integer for the common case in which a value is fully aligned and must be printed. VALUE is the value of the integer object and SIZE is the number of bytes it contains. */ Index: config/i386/winnt.c =================================================================== --- config/i386/winnt.c (revision 166490) +++ config/i386/winnt.c (working copy) @@ -414,15 +414,6 @@ i386_pe_section_type_flags (tree decl, c flags = SECTION_CODE; else if (decl && decl_readonly_section (decl, reloc)) flags = 0; - else if (current_function_decl - && cfun - && crtl->subsections.unlikely_text_section_name - && strcmp (name, crtl->subsections.unlikely_text_section_name) == 0) - flags = SECTION_CODE; - else if (!decl - && (!current_function_decl || !cfun) - && strcmp (name, UNLIKELY_EXECUTED_TEXT_SECTION_NAME) == 0) - flags = SECTION_CODE; else { flags = SECTION_WRITE; Index: config/darwin-protos.h =================================================================== --- config/darwin-protos.h (revision 166490) +++ config/darwin-protos.h (working copy) @@ -57,6 +57,7 @@ extern section *machopic_select_section extern section *machopic_select_rtx_section (enum machine_mode, rtx, unsigned HOST_WIDE_INT); +extern section *darwin_function_section (tree, enum node_frequency, bool, bool); extern void darwin_unique_section (tree decl, int reloc); extern void darwin_asm_named_section (const char *, unsigned int, tree); extern void darwin_non_lazy_pcrel (FILE *, rtx); Index: config/microblaze/microblaze.h =================================================================== --- config/microblaze/microblaze.h (revision 166490) +++ config/microblaze/microblaze.h (working copy) @@ -880,10 +880,6 @@ do { \ #define SBSS_SECTION_ASM_OP "\t.sbss" /* Small RW uninitialized data */ #define SBSS2_SECTION_ASM_OP "\t.sbss2" /* Small RO uninitialized data */ -#define HOT_TEXT_SECTION_NAME ".text.hot" -#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME \ - ".text.unlikely" - /* We do this to save a few 10s of code space that would be taken up by the call_FUNC () wrappers, used by the generic CRT_CALL_STATIC_FUNCTION definition in crtstuff.c. */ Index: config/ia64/hpux.h =================================================================== --- config/ia64/hpux.h (revision 166490) +++ config/ia64/hpux.h (working copy) @@ -221,8 +221,4 @@ do { \ it is fixed, prevent code from being put into .text.unlikely or .text.hot. */ -#undef UNLIKELY_EXECUTED_TEXT_SECTION_NAME -#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME ".text" - -#undef HOT_TEXT_SECTION_NAME -#define HOT_TEXT_SECTION_NAME ".text" +#define TARGET_ASM_FUNCTION_SECTOIN NULL Index: config/darwin.c =================================================================== --- config/darwin.c (revision 166490) +++ config/darwin.c (working copy) @@ -1233,12 +1233,22 @@ machopic_select_section (tree decl, && DECL_WEAK (decl) && !lookup_attribute ("weak_import", DECL_ATTRIBUTES (decl))); - section *base_section; + section *base_section = NULL; switch (categorize_decl_for_section (decl, reloc)) { case SECCAT_TEXT: - base_section = darwin_text_section (reloc, weak); + { + struct cgraph_node *node; + if (decl && TREE_CODE (decl) == FUNCTION_DECL + && (node = cgraph_get_node (decl)) != NULL) + base_section = darwin_function_section (decl, + node->frequency, + node->only_called_at_startup, + node->only_called_at_exit); + if (!base_section) + base_section = darwin_text_section (reloc, weak); + } break; case SECCAT_RODATA: @@ -2362,4 +2372,38 @@ darwin_enter_string_into_cfstring_table } } +/* Choose named function section based on its frequency. */ + +section * +darwin_function_section (tree decl, enum node_frequency freq, + bool startup, bool exit) +{ + /* Startup code should go to startup subsection unless it is + unlikely executed (this happens especially with function splitting + where we can split away unnecesary parts of static constructors. */ + if (startup && freq != NODE_FREQUENCY_UNLIKELY_EXECUTED) + return named_text_subsection + (decl, "__TEXT,__startup,regular,pure_instructions", "_startup"); + + /* Similarly for exit. */ + if (exit && freq != NODE_FREQUENCY_UNLIKELY_EXECUTED) + return named_text_subsection (decl, + "__TEXT,__exit,regular,pure_instructions", + "_exit"); + + /* Group cold functions together, similarly for hot code. */ + switch (freq) + { + case NODE_FREQUENCY_UNLIKELY_EXECUTED: + return named_text_subsection + (decl, + "__TEXT,__unlikely,regular,pure_instructions", "_ulinkely"); + case NODE_FREQUENCY_HOT: + return named_text_subsection + (decl, "__TEXT,__hot,regular,pure_instructions", "_hot"); + default: + return NULL; + } +} + #include "gt-darwin.h" Index: config/darwin.h =================================================================== --- config/darwin.h (revision 166490) +++ config/darwin.h (working copy) @@ -710,11 +710,6 @@ int darwin_label_is_anonymous_local_objc /* The generic version, archs should over-ride where required. */ #define MACHOPIC_NL_SYMBOL_PTR_SECTION ".non_lazy_symbol_pointer" -/* These are used by -fbranch-probabilities */ -#define HOT_TEXT_SECTION_NAME "__TEXT,__text,regular,pure_instructions" -#define UNLIKELY_EXECUTED_TEXT_SECTION_NAME \ - "__TEXT,__unlikely,regular,pure_instructions" - /* Declare the section variables. */ #ifndef USED_FOR_TARGET enum darwin_section_enum { @@ -729,6 +724,8 @@ extern GTY(()) section * darwin_sections #undef TARGET_ASM_SELECT_SECTION #define TARGET_ASM_SELECT_SECTION machopic_select_section #define USE_SELECT_SECTION_FOR_FUNCTIONS +#undef TARGET_ASM_FUNCTION_SECTION +#define TARGET_ASM_FUNCTION_SECTION darwin_function_section #undef TARGET_ASM_SELECT_RTX_SECTION #define TARGET_ASM_SELECT_RTX_SECTION machopic_select_rtx_section