From patchwork Wed Nov 8 03:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 1861385 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SQB0R20Jlz1yQg for ; Wed, 8 Nov 2023 14:49:11 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2A4493856948 for ; Wed, 8 Nov 2023 03:49:09 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg1.qq.com (smtpbgsg1.qq.com [54.254.200.92]) by sourceware.org (Postfix) with ESMTPS id 496353858C66 for ; Wed, 8 Nov 2023 03:47:56 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 496353858C66 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 496353858C66 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.254.200.92 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699415284; cv=none; b=upjDves8MGFENd1bY1+d4AC4oOm/Lu94SNaUblOlmAy3653bOMYMhfY/NHUqb8W6+ubgKBsXOwas10yJyddRVWrg6cWOQDod2Idj7Gykzcedlvbwv/N7J5JM/TxFgyE6j5nYP31oOP+McsHbsdPowsr3YbTRdn6TMKaB4M4KKHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699415284; c=relaxed/simple; bh=r0WkdUJRuiPiC2qnxSeuxfpgUuc0g0HQU0EsTYMbv9U=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=jihhNT1+P2Gn6w8KcQno5HnJypq62VV/ed5e2MJBDeMTSOhDLlGQn/3tu9S3PH15ccGQFIFL+aot1sG+SNrgdlpTcYwq6olLajMQ313PAqaWWMAInlYVecfaPYJAV4kZ0UpTigejzzN35nhpr8K+Xm/Dtp1sie3twXN6pvmFF4g= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp81t1699415271tohivwaj Received: from rios-cad121.hadoop.rioslab.org ( [58.60.1.9]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 08 Nov 2023 11:47:50 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: rZJGTgY0+YMFfx21uTpoY2RVPTQYeOzWdiEVhViwKXtCzLoTcDoldcqevCG+9 8Z9xSAkMPJWg91jgY3qmQSvWDTOX8neTn3NjJUCXh5nQtl9oD7zj0ATIVOIp3IJ03HuoxIH +pLMibQRJrDs/bYLdtXS+gCOIO9y9tKi/sP/ZsPIKrMFAneMPneaxQSpKW6xMSS8jBkmB7a Re8IYixHuW80V58MXc4NSHTnG1qZXtz9qoltLrQN1nBSc/ZOdxUDlfJxlDmO71jKI9tn5U9 3VeCRUjkh2xP6XhU9aUOcNkcEYVVeL56bprbbOUV/N7d8lpqgFIx5asZOAJ23azLOTiehAS 2Pe7AkXJKlBrrxp6HsbPn/ccdVJZoIHxqSXUu1mTR84cLO3lDkH+ruLojsW/R4DFpY+lFis X-QQ-GoodBg: 2 X-BIZMAIL-ID: 6175444661060236510 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 3/7] ira: Support subreg live range track Date: Wed, 8 Nov 2023 11:47:36 +0800 Message-Id: <20231108034740.834590-4-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20231108034740.834590-1-lehua.ding@rivai.ai> References: <20231108034740.834590-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz6a-0 X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org This patch extends the reg live range in ira to track the lifecycle of subreg, thus enabling more granular tracking of the live range and conflict of a pseudo subreg part. This patch will divide allocno into two categories: one has single object, and the other is the case where it contains subreg objects. gcc/ChangeLog: * ira-build.cc (init_object_start_and_nregs): Removed. (ira_create_object): Adjust. (find_object): New. (find_object_anyway): New. (ira_create_allocno): Removed regs_with_subreg. (ira_set_allocno_class): Adjust. (get_range): New. (ira_copy_allocno_objects): New. (merge_hard_reg_conflicts): Adjust. (create_cap_allocno): Adjust. (find_subreg_p): New. (add_subregs): New. (create_insn_allocnos): Adjust. (create_bb_allocnos): Adjust. (move_allocno_live_ranges): Adjust. (copy_allocno_live_ranges): Adjust. (setup_min_max_allocno_live_range_point): Adjust. (init_regs_with_subreg): Removed. (ira_build): Removed. (ira_destroy): Removed. * ira-color.cc (INCLUDE_MAP): use std::map (setup_left_conflict_sizes_p): Adjust. (push_allocno_to_stack): Adjust. * ira-conflicts.cc (record_object_conflict): Adjust. (build_object_conflicts): Adjust. (build_conflicts): Adjust. (print_allocno_conflicts): Adjust. * ira-emit.cc (modify_move_list): Adjust. * ira-int.h (struct ira_object): Adjust. (struct ira_allocno): Adjust. (OBJECT_SIZE): New. (OBJECT_OFFSET): New. (OBJECT_SUBWORD): New. (find_object): New. (find_object_anyway): New. (ira_copy_allocno_objects): New. * ira-lives.cc (INCLUDE_VECTOR): use std::vector. (set_subreg_conflict_hard_regs): New. (make_hard_regno_dead): Adjust. (make_object_live): Adjust. (update_allocno_pressure_excess_length): Adjust. (make_object_dead): Adjust. (mark_pseudo_regno_live): New. (add_subreg_point): New. (mark_pseudo_object_live): New. (mark_pseudo_regno_subword_live): Removed. (mark_pseudo_regno_subreg_live): New. (mark_pseudo_regno_subregs_live): New. (mark_pseudo_reg_live): New. (mark_pseudo_regno_dead): Removed. (mark_pseudo_object_dead): New. (mark_pseudo_regno_subword_dead): Removed. (mark_pseudo_regno_subreg_dead): New. (mark_pseudo_reg_dead): Adjust. (process_single_reg_class_operands): Adjust. (process_out_of_region_eh_regs): Adjust. (process_bb_node_lives): Adjust. (class subreg_live_item): New. (create_subregs_live_ranges): New. (ira_create_allocno_live_ranges): Adjust. * subreg-live-range.h: New fields. --- gcc/ira-build.cc | 275 +++++++++++++-------- gcc/ira-color.cc | 68 ++++-- gcc/ira-conflicts.cc | 48 ++-- gcc/ira-emit.cc | 2 +- gcc/ira-int.h | 21 +- gcc/ira-lives.cc | 522 +++++++++++++++++++++++++++++----------- gcc/subreg-live-range.h | 16 ++ 7 files changed, 653 insertions(+), 299 deletions(-) diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc index 7df98164503..5fb7a9f800f 100644 --- a/gcc/ira-build.cc +++ b/gcc/ira-build.cc @@ -29,10 +29,12 @@ along with GCC; see the file COPYING3. If not see #include "insn-config.h" #include "regs.h" #include "memmodel.h" +#include "tm_p.h" #include "ira.h" #include "ira-int.h" #include "sparseset.h" #include "cfgloop.h" +#include "subreg-live-range.h" static ira_copy_t find_allocno_copy (ira_allocno_t, ira_allocno_t, rtx_insn *, ira_loop_tree_node_t); @@ -440,49 +442,14 @@ initiate_allocnos (void) memset (ira_regno_allocno_map, 0, max_reg_num () * sizeof (ira_allocno_t)); } -/* Update OBJ's start and nregs field according A and OBJ info. */ -static void -init_object_start_and_nregs (ira_allocno_t a, ira_object_t obj) -{ - enum reg_class aclass = ALLOCNO_CLASS (a); - gcc_assert (aclass != NO_REGS); - - machine_mode mode = ALLOCNO_MODE (a); - int nregs = ira_reg_class_max_nregs[aclass][mode]; - if (ALLOCNO_TRACK_SUBREG_P (a)) - { - poly_int64 end = OBJECT_OFFSET (obj) + OBJECT_SIZE (obj); - for (int i = 0; i < nregs; i += 1) - { - poly_int64 right = ALLOCNO_UNIT_SIZE (a) * (i + 1); - if (OBJECT_START (obj) < 0 && maybe_lt (OBJECT_OFFSET (obj), right)) - { - OBJECT_START (obj) = i; - } - if (OBJECT_NREGS (obj) < 0 && maybe_le (end, right)) - { - OBJECT_NREGS (obj) = i + 1 - OBJECT_START (obj); - break; - } - } - gcc_assert (OBJECT_START (obj) >= 0 && OBJECT_NREGS (obj) > 0); - } - else - { - OBJECT_START (obj) = 0; - OBJECT_NREGS (obj) = nregs; - } -} - /* Create and return an object corresponding to a new allocno A. */ static ira_object_t -ira_create_object (ira_allocno_t a, int subword) +ira_create_object (ira_allocno_t a, int start, int nregs) { enum reg_class aclass = ALLOCNO_CLASS (a); ira_object_t obj = object_pool.allocate (); OBJECT_ALLOCNO (obj) = a; - OBJECT_SUBWORD (obj) = subword; OBJECT_CONFLICT_ID (obj) = ira_objects_num; OBJECT_CONFLICT_VEC_P (obj) = false; OBJECT_CONFLICT_ARRAY (obj) = NULL; @@ -494,19 +461,14 @@ ira_create_object (ira_allocno_t a, int subword) OBJECT_MIN (obj) = INT_MAX; OBJECT_MAX (obj) = -1; OBJECT_LIVE_RANGES (obj) = NULL; - OBJECT_SIZE (obj) = UNITS_PER_WORD; - OBJECT_OFFSET (obj) = subword * UNITS_PER_WORD; - OBJECT_START (obj) = -1; - OBJECT_NREGS (obj) = -1; + OBJECT_START (obj) = start; + OBJECT_NREGS (obj) = nregs; ira_object_id_map_vec.safe_push (obj); ira_object_id_map = ira_object_id_map_vec.address (); ira_objects_num = ira_object_id_map_vec.length (); - if (aclass != NO_REGS) - init_object_start_and_nregs (a, obj); - a->objects.push_back (obj); return obj; @@ -524,6 +486,52 @@ find_object (ira_allocno_t a, int start, int nregs) return NULL; } +ira_object_t +find_object (ira_allocno_t a, poly_int64 offset, poly_int64 size) +{ + enum reg_class aclass = ALLOCNO_CLASS (a); + machine_mode mode = ALLOCNO_MODE (a); + int nregs = ira_reg_class_max_nregs[aclass][mode]; + + if (!has_subreg_object_p (a) + || maybe_eq (GET_MODE_SIZE (ALLOCNO_MODE (a)), size)) + return find_object (a, 0, nregs); + + gcc_assert (maybe_lt (size, GET_MODE_SIZE (ALLOCNO_MODE (a))) + && maybe_le (offset + size, GET_MODE_SIZE (ALLOCNO_MODE (a)))); + + int subreg_start = -1; + int subreg_nregs = -1; + for (int i = 0; i < nregs; i += 1) + { + poly_int64 right = ALLOCNO_UNIT_SIZE (a) * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + { + subreg_start = i; + } + if (subreg_nregs < 0 && maybe_le (offset + size, right)) + { + subreg_nregs = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nregs > 0); + return find_object (a, subreg_start, subreg_nregs); +} + +/* Return the object in allocno A which match START & NREGS. Create when not + found. */ +ira_object_t +find_object_anyway (ira_allocno_t a, int start, int nregs) +{ + ira_object_t obj = find_object (a, start, nregs); + if (obj == NULL && ALLOCNO_TRACK_SUBREG_P (a)) + obj = ira_create_object (a, start, nregs); + + gcc_assert (obj != NULL); + return obj; +} + /* Create and return the allocno corresponding to REGNO in LOOP_TREE_NODE. Add the allocno to the list of allocnos with the same regno if CAP_P is FALSE. */ @@ -591,9 +599,6 @@ ira_create_allocno (int regno, bool cap_p, return a; } -/* Record the regs referenced by subreg. */ -static bitmap_head regs_with_subreg; - /* Set up register class for A and update its conflict hard registers. */ void @@ -614,8 +619,7 @@ ira_set_allocno_class (ira_allocno_t a, enum reg_class aclass) /* SET the unit_size of one register. */ machine_mode mode = ALLOCNO_MODE (a); int nregs = ira_reg_class_max_nregs[aclass][mode]; - if (nregs == 2 && maybe_eq (GET_MODE_SIZE (mode), nregs * UNITS_PER_WORD) - && bitmap_bit_p (®s_with_subreg, ALLOCNO_REGNO (a))) + if (nregs == 2 && maybe_eq (GET_MODE_SIZE (mode), nregs * UNITS_PER_WORD)) { ALLOCNO_UNIT_SIZE (a) = UNITS_PER_WORD; ALLOCNO_TRACK_SUBREG_P (a) = true; @@ -623,6 +627,39 @@ ira_set_allocno_class (ira_allocno_t a, enum reg_class aclass) } } +/* Return the subreg range of rtx SUBREG. */ +static subreg_range +get_range (rtx subreg) +{ + gcc_assert (read_modify_subreg_p (subreg)); + rtx reg = SUBREG_REG (subreg); + machine_mode reg_mode = GET_MODE (reg); + + machine_mode subreg_mode = GET_MODE (subreg); + int nblocks = get_nblocks (reg_mode); + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode); + + poly_int64 offset = SUBREG_BYTE (subreg); + poly_int64 left = offset + GET_MODE_SIZE (subreg_mode); + + int subreg_start = -1; + int subreg_nblocks = -1; + for (int i = 0; i < nblocks; i += 1) + { + poly_int64 right = unit_size * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + subreg_start = i; + if (subreg_nblocks < 0 && maybe_le (left, right)) + { + subreg_nblocks = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0); + + return subreg_range (subreg_start, subreg_start + subreg_nblocks); +} + /* Determine the number of objects we should associate with allocno A and allocate them. */ void @@ -630,15 +667,37 @@ ira_create_allocno_objects (ira_allocno_t a) { machine_mode mode = ALLOCNO_MODE (a); enum reg_class aclass = ALLOCNO_CLASS (a); - int n = ira_reg_class_max_nregs[aclass][mode]; - int i; + int nregs = ira_reg_class_max_nregs[aclass][mode]; - if (n != 2 || maybe_ne (GET_MODE_SIZE (mode), n * UNITS_PER_WORD) - || !bitmap_bit_p (®s_with_subreg, ALLOCNO_REGNO (a))) - n = 1; + ira_create_object (a, 0, nregs); - for (i = 0; i < n; i++) - ira_create_object (a, i); + if (aclass == NO_REGS || !ALLOCNO_TRACK_SUBREG_P (a) || a->subregs.empty ()) + return; + + int nblocks = get_nblocks (ALLOCNO_MODE (a)); + int times = nblocks / ALLOCNO_NREGS (a); + gcc_assert (times >= 1 && nblocks % ALLOCNO_NREGS (a) == 0); + + for (const auto &range : a->subregs) + { + int start = range.start / times; + int end = CEIL (range.end, times); + if (find_object (a, start, end - start) != NULL) + continue; + ira_create_object (a, start, end - start); + } + + a->subregs.clear (); +} + +/* Copy the objects from FROM to TO. */ +void +ira_copy_allocno_objects (ira_allocno_t to, ira_allocno_t from) +{ + ira_allocno_object_iterator oi; + ira_object_t obj; + FOR_EACH_ALLOCNO_OBJECT (from, obj, oi) + ira_create_object (to, OBJECT_START (obj), OBJECT_NREGS (obj)); } /* For each allocno, set ALLOCNO_NUM_OBJECTS and create the @@ -662,11 +721,11 @@ merge_hard_reg_conflicts (ira_allocno_t from, ira_allocno_t to, bool total_only) { int i; - gcc_assert (ALLOCNO_NUM_OBJECTS (to) == ALLOCNO_NUM_OBJECTS (from)); - for (i = 0; i < ALLOCNO_NUM_OBJECTS (to); i++) + for (i = 0; i < ALLOCNO_NUM_OBJECTS (from); i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); if (!total_only) OBJECT_CONFLICT_HARD_REGS (to_obj) @@ -960,7 +1019,7 @@ create_cap_allocno (ira_allocno_t a) ALLOCNO_WMODE (cap) = ALLOCNO_WMODE (a); aclass = ALLOCNO_CLASS (a); ira_set_allocno_class (cap, aclass); - ira_create_allocno_objects (cap); + ira_copy_allocno_objects (cap, a); ALLOCNO_CAP_MEMBER (cap) = a; ALLOCNO_CAP (a) = cap; ALLOCNO_CLASS_COST (cap) = ALLOCNO_CLASS_COST (a); @@ -1902,6 +1961,26 @@ ira_traverse_loop_tree (bool bb_p, ira_loop_tree_node_t loop_node, /* The basic block currently being processed. */ static basic_block curr_bb; +/* Return true if A's subregs has a subreg with same SIZE and OFFSET. */ +static bool +find_subreg_p (ira_allocno_t a, const subreg_range &r) +{ + for (const auto &item : a->subregs) + if (item.start == r.start && item.end == r.end) + return true; + return false; +} + +/* Return start and nregs subregs from DF_LIVE_SUBREG. */ +static void +add_subregs (ira_allocno_t a, const subreg_ranges &sr) +{ + gcc_assert (get_nblocks (ALLOCNO_MODE (a)) == sr.max); + for (const subreg_range &r : sr.ranges) + if (!find_subreg_p (a, r)) + a->subregs.push_back (r); +} + /* This recursive function creates allocnos corresponding to pseudo-registers containing in X. True OUTPUT_P means that X is an lvalue. OUTER corresponds to the parent expression of X. */ @@ -1931,6 +2010,14 @@ create_insn_allocnos (rtx x, rtx outer, bool output_p) } } + /* Collect subreg reference. */ + if (outer != NULL && read_modify_subreg_p (outer)) + { + const subreg_range r = get_range (outer); + if (!find_subreg_p (a, r)) + a->subregs.push_back (r); + } + ALLOCNO_NREFS (a)++; ALLOCNO_FREQ (a) += REG_FREQ_FROM_BB (curr_bb); if (output_p) @@ -1998,8 +2085,21 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node) EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_PARTIAL_IN (bb), FIRST_PSEUDO_REGISTER, i, bi) - if (ira_curr_regno_allocno_map[i] == NULL) - ira_create_allocno (i, false, ira_curr_loop_tree_node); + { + if (ira_curr_regno_allocno_map[i] == NULL) + ira_create_allocno (i, false, ira_curr_loop_tree_node); + add_subregs (ira_curr_regno_allocno_map[i], + DF_LIVE_SUBREG_RANGE_IN (bb)->lives.at (i)); + } + + EXECUTE_IF_SET_IN_REG_SET (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, i, bi) + { + if (ira_curr_regno_allocno_map[i] == NULL) + ira_create_allocno (i, false, ira_curr_loop_tree_node); + add_subregs (ira_curr_regno_allocno_map[i], + DF_LIVE_SUBREG_RANGE_OUT (bb)->lives.at (i)); + } } /* Create allocnos corresponding to pseudo-registers living on edge E @@ -2214,20 +2314,20 @@ move_allocno_live_ranges (ira_allocno_t from, ira_allocno_t to) int i; int n = ALLOCNO_NUM_OBJECTS (from); - gcc_assert (n == ALLOCNO_NUM_OBJECTS (to)); - for (i = 0; i < n; i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); live_range_t lr = OBJECT_LIVE_RANGES (from_obj); if (internal_flag_ira_verbose > 4 && ira_dump_file != NULL) { fprintf (ira_dump_file, - " Moving ranges of a%dr%d to a%dr%d: ", + " Moving ranges of a%dr%d_obj%d to a%dr%d_obj%d: ", ALLOCNO_NUM (from), ALLOCNO_REGNO (from), - ALLOCNO_NUM (to), ALLOCNO_REGNO (to)); + OBJECT_INDEX (from_obj), ALLOCNO_NUM (to), + ALLOCNO_REGNO (to), OBJECT_INDEX (to_obj)); ira_print_live_range_list (ira_dump_file, lr); } change_object_in_range_list (lr, to_obj); @@ -2243,12 +2343,11 @@ copy_allocno_live_ranges (ira_allocno_t from, ira_allocno_t to) int i; int n = ALLOCNO_NUM_OBJECTS (from); - gcc_assert (n == ALLOCNO_NUM_OBJECTS (to)); - for (i = 0; i < n; i++) { ira_object_t from_obj = ALLOCNO_OBJECT (from, i); - ira_object_t to_obj = ALLOCNO_OBJECT (to, i); + ira_object_t to_obj = find_object_anyway (to, OBJECT_START (from_obj), + OBJECT_NREGS (from_obj)); live_range_t lr = OBJECT_LIVE_RANGES (from_obj); if (internal_flag_ira_verbose > 4 && ira_dump_file != NULL) @@ -2860,15 +2959,17 @@ setup_min_max_allocno_live_range_point (void) ira_assert (OBJECT_LIVE_RANGES (obj) == NULL); OBJECT_MAX (obj) = 0; OBJECT_MIN (obj) = 1; - continue; } ira_assert (ALLOCNO_CAP_MEMBER (a) == NULL); /* Accumulation of range info. */ if (ALLOCNO_CAP (a) != NULL) { - for (cap = ALLOCNO_CAP (a); cap != NULL; cap = ALLOCNO_CAP (cap)) + for (cap = ALLOCNO_CAP (a); cap != NULL; + cap = ALLOCNO_CAP (cap)) { - ira_object_t cap_obj = ALLOCNO_OBJECT (cap, j); + ira_object_t cap_obj = find_object (cap, OBJECT_START (obj), + OBJECT_NREGS (obj)); + gcc_assert (cap_obj != NULL); if (OBJECT_MAX (cap_obj) < OBJECT_MAX (obj)) OBJECT_MAX (cap_obj) = OBJECT_MAX (obj); if (OBJECT_MIN (cap_obj) > OBJECT_MIN (obj)) @@ -2879,7 +2980,9 @@ setup_min_max_allocno_live_range_point (void) if ((parent = ALLOCNO_LOOP_TREE_NODE (a)->parent) == NULL) continue; parent_a = parent->regno_allocno_map[i]; - parent_obj = ALLOCNO_OBJECT (parent_a, j); + parent_obj + = find_object (parent_a, OBJECT_START (obj), OBJECT_NREGS (obj)); + gcc_assert (parent_obj != NULL); if (OBJECT_MAX (parent_obj) < OBJECT_MAX (obj)) OBJECT_MAX (parent_obj) = OBJECT_MAX (obj); if (OBJECT_MIN (parent_obj) > OBJECT_MIN (obj)) @@ -3538,30 +3641,6 @@ update_conflict_hard_reg_costs (void) } } -/* Traverse all instructions to determine which ones have access through subreg. - */ -static void -init_regs_with_subreg () -{ - bitmap_initialize (®s_with_subreg, ®_obstack); - basic_block bb; - rtx_insn *insn; - df_ref def, use; - FOR_ALL_BB_FN (bb, cfun) - FOR_BB_INSNS (bb, insn) - { - if (!NONDEBUG_INSN_P (insn)) - continue; - df_insn_info *insn_info = DF_INSN_INFO_GET (insn); - FOR_EACH_INSN_INFO_DEF (def, insn_info) - if (DF_REF_FLAGS (def) & (DF_REF_PARTIAL | DF_REF_SUBREG)) - bitmap_set_bit (®s_with_subreg, DF_REF_REGNO (def)); - FOR_EACH_INSN_INFO_USE (use, insn_info) - if (DF_REF_FLAGS (use) & (DF_REF_PARTIAL | DF_REF_SUBREG)) - bitmap_set_bit (®s_with_subreg, DF_REF_REGNO (use)); - } -} - /* Create a internal representation (IR) for IRA (allocnos, copies, loop tree nodes). The function returns TRUE if we generate loop structure (besides nodes representing all function and the basic @@ -3577,7 +3656,6 @@ ira_build (void) initiate_allocnos (); initiate_prefs (); initiate_copies (); - init_regs_with_subreg (); create_loop_tree_nodes (); form_loop_tree (); create_allocnos (); @@ -3668,5 +3746,4 @@ ira_destroy (void) finish_allocnos (); finish_cost_vectors (); ira_finish_allocno_live_ranges (); - bitmap_clear (®s_with_subreg); } diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc index f1b96d1aee6..8aed25144b9 100644 --- a/gcc/ira-color.cc +++ b/gcc/ira-color.cc @@ -19,6 +19,7 @@ along with GCC; see the file COPYING3. If not see . */ #include "config.h" +#define INCLUDE_MAP #include "system.h" #include "coretypes.h" #include "backend.h" @@ -852,18 +853,17 @@ setup_left_conflict_sizes_p (ira_allocno_t a) node_preorder_num = node->preorder_num; node_set = node->hard_regs->set; node_check_tick++; + /* Collect conflict objects. */ + std::map allocno_conflict_regs; for (k = 0; k < nobj; k++) { ira_object_t obj = ALLOCNO_OBJECT (a, k); ira_object_t conflict_obj; ira_object_conflict_iterator oci; - + FOR_EACH_OBJECT_CONFLICT (obj, conflict_obj, oci) { - int size; - ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); - allocno_hard_regs_node_t conflict_node, temp_node; - HARD_REG_SET conflict_node_set; + ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); allocno_color_data_t conflict_data; conflict_data = ALLOCNO_COLOR_DATA (conflict_a); @@ -872,6 +872,24 @@ setup_left_conflict_sizes_p (ira_allocno_t a) conflict_data ->profitable_hard_regs)) continue; + int num = ALLOCNO_NUM (conflict_a); + if (allocno_conflict_regs.count (num) == 0) + allocno_conflict_regs.insert ({num, ira_allocate_bitmap ()}); + bitmap_head temp; + bitmap_initialize (&temp, ®_obstack); + bitmap_set_range (&temp, OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj)); + bitmap_and_compl_into (&temp, allocno_conflict_regs.at (num)); + int size = bitmap_count_bits (&temp); + bitmap_clear (&temp); + if (size == 0) + continue; + + bitmap_set_range (allocno_conflict_regs.at (num), + OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj)); + allocno_hard_regs_node_t conflict_node, temp_node; + HARD_REG_SET conflict_node_set; conflict_node = conflict_data->hard_regs_node; conflict_node_set = conflict_node->hard_regs->set; if (hard_reg_set_subset_p (node_set, conflict_node_set)) @@ -886,14 +904,13 @@ setup_left_conflict_sizes_p (ira_allocno_t a) temp_node->check = node_check_tick; temp_node->conflict_size = 0; } - size = (ira_reg_class_max_nregs - [ALLOCNO_CLASS (conflict_a)][ALLOCNO_MODE (conflict_a)]); - if (ALLOCNO_NUM_OBJECTS (conflict_a) > 1) - /* We will deal with the subwords individually. */ - size = 1; temp_node->conflict_size += size; } } + /* Setup conflict nregs of ALLOCNO. */ + for (auto &kv : allocno_conflict_regs) + ira_free_bitmap (kv.second); + for (i = 0; i < data->hard_regs_subnodes_num; i++) { allocno_hard_regs_node_t temp_node; @@ -2746,21 +2763,16 @@ push_allocno_to_stack (ira_allocno_t a) { enum reg_class aclass; allocno_color_data_t data, conflict_data; - int size, i, n = ALLOCNO_NUM_OBJECTS (a); - + int i, n = ALLOCNO_NUM_OBJECTS (a); + data = ALLOCNO_COLOR_DATA (a); data->in_graph_p = false; allocno_stack_vec.safe_push (a); aclass = ALLOCNO_CLASS (a); if (aclass == NO_REGS) return; - size = ira_reg_class_max_nregs[aclass][ALLOCNO_MODE (a)]; - if (n > 1) - { - /* We will deal with the subwords individually. */ - gcc_assert (size == ALLOCNO_NUM_OBJECTS (a)); - size = 1; - } + /* Already collect conflict objects. */ + std::map allocno_conflict_regs; for (i = 0; i < n; i++) { ira_object_t obj = ALLOCNO_OBJECT (a, i); @@ -2785,6 +2797,21 @@ push_allocno_to_stack (ira_allocno_t a) continue; ira_assert (bitmap_bit_p (coloring_allocno_bitmap, ALLOCNO_NUM (conflict_a))); + + int num = ALLOCNO_NUM (conflict_a); + if (allocno_conflict_regs.count (num) == 0) + allocno_conflict_regs.insert ({num, ira_allocate_bitmap ()}); + bitmap_head temp; + bitmap_initialize (&temp, ®_obstack); + bitmap_set_range (&temp, OBJECT_START (obj), OBJECT_NREGS (obj)); + bitmap_and_compl_into (&temp, allocno_conflict_regs.at (num)); + int size = bitmap_count_bits (&temp); + bitmap_clear (&temp); + if (size == 0) + continue; + + bitmap_set_range (allocno_conflict_regs.at (num), OBJECT_START (obj), + OBJECT_NREGS (obj)); if (update_left_conflict_sizes_p (conflict_a, a, size)) { delete_allocno_from_bucket @@ -2800,6 +2827,9 @@ push_allocno_to_stack (ira_allocno_t a) } } + + for (auto &kv : allocno_conflict_regs) + ira_free_bitmap (kv.second); } /* Put ALLOCNO onto the coloring stack and remove it from its bucket. diff --git a/gcc/ira-conflicts.cc b/gcc/ira-conflicts.cc index a4d93c8d734..0585ad10043 100644 --- a/gcc/ira-conflicts.cc +++ b/gcc/ira-conflicts.cc @@ -60,23 +60,8 @@ static IRA_INT_TYPE **conflicts; static void record_object_conflict (ira_object_t obj1, ira_object_t obj2) { - ira_allocno_t a1 = OBJECT_ALLOCNO (obj1); - ira_allocno_t a2 = OBJECT_ALLOCNO (obj2); - int w1 = OBJECT_SUBWORD (obj1); - int w2 = OBJECT_SUBWORD (obj2); - int id1, id2; - - /* Canonicalize the conflict. If two identically-numbered words - conflict, always record this as a conflict between words 0. That - is the only information we need, and it is easier to test for if - it is collected in each allocno's lowest-order object. */ - if (w1 == w2 && w1 > 0) - { - obj1 = ALLOCNO_OBJECT (a1, 0); - obj2 = ALLOCNO_OBJECT (a2, 0); - } - id1 = OBJECT_CONFLICT_ID (obj1); - id2 = OBJECT_CONFLICT_ID (obj2); + int id1 = OBJECT_CONFLICT_ID (obj1); + int id2 = OBJECT_CONFLICT_ID (obj2); SET_MINMAX_SET_BIT (conflicts[id1], id2, OBJECT_MIN (obj1), OBJECT_MAX (obj1)); @@ -606,8 +591,8 @@ build_object_conflicts (ira_object_t obj) if (parent_a == NULL) return; ira_assert (ALLOCNO_CLASS (a) == ALLOCNO_CLASS (parent_a)); - ira_assert (ALLOCNO_NUM_OBJECTS (a) == ALLOCNO_NUM_OBJECTS (parent_a)); - parent_obj = ALLOCNO_OBJECT (parent_a, OBJECT_SUBWORD (obj)); + parent_obj + = find_object_anyway (parent_a, OBJECT_START (obj), OBJECT_NREGS (obj)); parent_num = OBJECT_CONFLICT_ID (parent_obj); parent_min = OBJECT_MIN (parent_obj); parent_max = OBJECT_MAX (parent_obj); @@ -616,7 +601,6 @@ build_object_conflicts (ira_object_t obj) { ira_object_t another_obj = ira_object_id_map[i]; ira_allocno_t another_a = OBJECT_ALLOCNO (another_obj); - int another_word = OBJECT_SUBWORD (another_obj); ira_assert (ira_reg_classes_intersect_p [ALLOCNO_CLASS (a)][ALLOCNO_CLASS (another_a)]); @@ -627,11 +611,11 @@ build_object_conflicts (ira_object_t obj) ira_assert (ALLOCNO_NUM (another_parent_a) >= 0); ira_assert (ALLOCNO_CLASS (another_a) == ALLOCNO_CLASS (another_parent_a)); - ira_assert (ALLOCNO_NUM_OBJECTS (another_a) - == ALLOCNO_NUM_OBJECTS (another_parent_a)); SET_MINMAX_SET_BIT (conflicts[parent_num], - OBJECT_CONFLICT_ID (ALLOCNO_OBJECT (another_parent_a, - another_word)), + OBJECT_CONFLICT_ID ( + find_object_anyway (another_parent_a, + OBJECT_START (another_obj), + OBJECT_NREGS (another_obj))), parent_min, parent_max); } } @@ -659,9 +643,10 @@ build_conflicts (void) build_object_conflicts (obj); for (cap = ALLOCNO_CAP (a); cap != NULL; cap = ALLOCNO_CAP (cap)) { - ira_object_t cap_obj = ALLOCNO_OBJECT (cap, j); - gcc_assert (ALLOCNO_NUM_OBJECTS (cap) == ALLOCNO_NUM_OBJECTS (a)); - build_object_conflicts (cap_obj); + ira_object_t cap_obj + = find_object_anyway (cap, OBJECT_START (obj), + OBJECT_NREGS (obj)); + build_object_conflicts (cap_obj); } } } @@ -736,7 +721,8 @@ print_allocno_conflicts (FILE * file, bool reg_p, ira_allocno_t a) } if (n > 1) - fprintf (file, "\n;; subobject %d:", i); + fprintf (file, "\n;; subobject s%d,n%d,f%d:", OBJECT_START (obj), + OBJECT_NREGS (obj), ALLOCNO_NREGS (a)); FOR_EACH_OBJECT_CONFLICT (obj, conflict_obj, oci) { ira_allocno_t conflict_a = OBJECT_ALLOCNO (conflict_obj); @@ -746,8 +732,10 @@ print_allocno_conflicts (FILE * file, bool reg_p, ira_allocno_t a) { fprintf (file, " a%d(r%d", ALLOCNO_NUM (conflict_a), ALLOCNO_REGNO (conflict_a)); - if (ALLOCNO_NUM_OBJECTS (conflict_a) > 1) - fprintf (file, ",w%d", OBJECT_SUBWORD (conflict_obj)); + if (has_subreg_object_p (conflict_a)) + fprintf (file, ",s%d,n%d,f%d", OBJECT_START (conflict_obj), + OBJECT_NREGS (conflict_obj), + ALLOCNO_NREGS (conflict_a)); if ((bb = ALLOCNO_LOOP_TREE_NODE (conflict_a)->bb) != NULL) fprintf (file, ",b%d", bb->index); else diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc index 84ed482e568..9dc7f3c655e 100644 --- a/gcc/ira-emit.cc +++ b/gcc/ira-emit.cc @@ -854,7 +854,7 @@ modify_move_list (move_t list) ALLOCNO_MODE (new_allocno) = ALLOCNO_MODE (set_move->to); ira_set_allocno_class (new_allocno, ALLOCNO_CLASS (set_move->to)); - ira_create_allocno_objects (new_allocno); + ira_copy_allocno_objects (new_allocno, set_move->to); ALLOCNO_ASSIGNED_P (new_allocno) = true; ALLOCNO_HARD_REGNO (new_allocno) = -1; ALLOCNO_EMIT_DATA (new_allocno)->reg diff --git a/gcc/ira-int.h b/gcc/ira-int.h index b6281d3df6d..b9e24328867 100644 --- a/gcc/ira-int.h +++ b/gcc/ira-int.h @@ -24,6 +24,7 @@ along with GCC; see the file COPYING3. If not see #include "recog.h" #include "function-abi.h" #include +#include "subreg-live-range.h" /* To provide consistency in naming, all IRA external variables, functions, common typedefs start with prefix ira_. */ @@ -223,7 +224,7 @@ extern int ira_max_point; extern live_range_t *ira_start_point_ranges, *ira_finish_point_ranges; /* A structure representing conflict information for an allocno - (or one of its subwords). */ + (or one of its subregs). */ struct ira_object { /* The allocno associated with this record. */ @@ -237,17 +238,9 @@ struct ira_object ranges in the list are not intersected and ordered by decreasing their program points*. */ live_range_t live_ranges; - /* The subword within ALLOCNO which is represented by this object. - Zero means the lowest-order subword (or the entire allocno in case - it is not being tracked in subwords). */ - int subword; /* Reprensent OBJECT occupied [start, start + nregs) registers of it's ALLOCNO. */ int start, nregs; - /* Reprensent the size and offset of current object, use to track subreg - range, For full reg, the size is GET_MODE_SIZE (ALLOCNO_MODE (allocno)), - offset is 0. */ - poly_int64 size, offset; /* Allocated size of the conflicts array. */ unsigned int conflicts_array_size; /* A unique number for every instance of this structure, which is used @@ -400,6 +393,9 @@ struct ira_allocno more than one such object in cases where the allocno represents a multi-hardreg pesudo. */ std::vector objects; + /* An array of structures decribing the subreg mode start and subreg end for + this allocno. */ + std::vector subregs; /* Registers clobbered by intersected calls. */ HARD_REG_SET crossed_calls_clobbered_regs; /* Array of usage costs (accumulated and the one updated during @@ -526,9 +522,6 @@ allocno_emit_reg (ira_allocno_t a) } #define OBJECT_ALLOCNO(O) ((O)->allocno) -#define OBJECT_SIZE(O) ((O)->size) -#define OBJECT_OFFSET(O) ((O)->offset) -#define OBJECT_SUBWORD(O) ((O)->subword) #define OBJECT_CONFLICT_ARRAY(O) ((O)->conflicts_array) #define OBJECT_CONFLICT_VEC(O) ((ira_object_t *)(O)->conflicts_array) #define OBJECT_CONFLICT_BITVEC(O) ((IRA_INT_TYPE *)(O)->conflicts_array) @@ -1062,6 +1055,10 @@ extern bool ira_build (void); extern void ira_destroy (void); extern ira_object_t find_object (ira_allocno_t, int, int); +extern ira_object_t find_object (ira_allocno_t, poly_int64, poly_int64); +ira_object_t +find_object_anyway (ira_allocno_t a, int start, int nregs); +extern void ira_copy_allocno_objects (ira_allocno_t, ira_allocno_t); /* ira-costs.cc */ extern void ira_init_costs_once (void); diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc index 60e6be0b0ae..e00898c0ccd 100644 --- a/gcc/ira-lives.cc +++ b/gcc/ira-lives.cc @@ -19,6 +19,7 @@ along with GCC; see the file COPYING3. If not see . */ #include "config.h" +#define INCLUDE_VECTOR #include "system.h" #include "coretypes.h" #include "backend.h" @@ -35,6 +36,7 @@ along with GCC; see the file COPYING3. If not see #include "sparseset.h" #include "function-abi.h" #include "except.h" +#include "subreg-live-range.h" /* The code in this file is similar to one in global but the code works on the allocno basis and creates live ranges instead of @@ -91,6 +93,9 @@ static alternative_mask preferred_alternatives; we should not add a conflict with the copy's destination operand. */ static rtx ignore_reg_for_conflicts; +/* Store def/use point of has_subreg_object_p register. */ +static class subregs_live_points *subreg_live_points; + /* Record hard register REGNO as now being live. */ static void make_hard_regno_live (int regno) @@ -98,6 +103,33 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); } +/* Update conflict hard regs of ALLOCNO a for current live part. */ +static void +set_subreg_conflict_hard_regs (ira_allocno_t a, HARD_REG_SET regs) +{ + gcc_assert (has_subreg_object_p (a)); + + if (subreg_live_points->subreg_live_ranges.count (ALLOCNO_NUM (a)) == 0) + return; + + for (const subreg_range &r : + subreg_live_points->subreg_live_ranges.at (ALLOCNO_NUM (a)).ranges) + { + ira_object_t obj = find_object_anyway (a, r.start, r.end - r.start); + OBJECT_CONFLICT_HARD_REGS (obj) |= regs; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= regs; + } +} + +static void +set_subreg_conflict_hard_regs (ira_allocno_t a, unsigned int regno) +{ + HARD_REG_SET set; + CLEAR_HARD_REG_SET (set); + SET_HARD_REG_BIT (set, regno); + set_subreg_conflict_hard_regs (a, set); +} + /* Process the definition of hard register REGNO. This updates hard_regs_live and hard reg conflict information for living allocnos. */ static void @@ -113,8 +145,13 @@ make_hard_regno_dead (int regno) == (unsigned int) ALLOCNO_REGNO (OBJECT_ALLOCNO (obj))) continue; - SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); - SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); + if (has_subreg_object_p (OBJECT_ALLOCNO (obj))) + set_subreg_conflict_hard_regs (OBJECT_ALLOCNO (obj), regno); + else + { + SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); + SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); + } } CLEAR_HARD_REG_BIT (hard_regs_live, regno); } @@ -127,9 +164,29 @@ make_object_live (ira_object_t obj) sparseset_set_bit (objects_live, OBJECT_CONFLICT_ID (obj)); live_range_t lr = OBJECT_LIVE_RANGES (obj); - if (lr == NULL - || (lr->finish != curr_point && lr->finish + 1 != curr_point)) - ira_add_live_range_to_object (obj, curr_point, -1); + if (lr == NULL || (lr->finish != curr_point && lr->finish + 1 != curr_point)) + { + ira_add_live_range_to_object (obj, curr_point, -1); + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf (ira_dump_file, + " add new live_range for a%d(r%d): [%d...-1]\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), curr_point); + } + } + else + { + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf ( + ira_dump_file, + " use old live_range for a%d(r%d): [%d...%d], curr: %d\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), lr->start, lr->finish, + curr_point); + } + } } /* Update ALLOCNO_EXCESS_PRESSURE_POINTS_NUM for the allocno @@ -140,7 +197,6 @@ update_allocno_pressure_excess_length (ira_object_t obj) ira_allocno_t a = OBJECT_ALLOCNO (obj); int start, i; enum reg_class aclass, pclass, cl; - live_range_t p; aclass = ALLOCNO_CLASS (a); pclass = ira_pressure_class_translate[aclass]; @@ -152,10 +208,18 @@ update_allocno_pressure_excess_length (ira_object_t obj) continue; if (high_pressure_start_point[cl] < 0) continue; - p = OBJECT_LIVE_RANGES (obj); - ira_assert (p != NULL); - start = (high_pressure_start_point[cl] > p->start - ? high_pressure_start_point[cl] : p->start); + int start_point; + if (has_subreg_object_p (a)) + start_point = subreg_live_points->get_start_point (ALLOCNO_NUM (a)); + else + { + live_range_t p = OBJECT_LIVE_RANGES (obj); + ira_assert (p != NULL); + start_point = p->start; + } + start = (high_pressure_start_point[cl] > start_point + ? high_pressure_start_point[cl] + : start_point); ALLOCNO_EXCESS_PRESSURE_POINTS_NUM (a) += curr_point - start + 1; } } @@ -201,6 +265,14 @@ make_object_dead (ira_object_t obj) CLEAR_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); lr = OBJECT_LIVE_RANGES (obj); + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) + { + fprintf (ira_dump_file, + " finish a live_range a%d(r%d): [%d...%d] => [%d...%d]\n", + ALLOCNO_NUM (OBJECT_ALLOCNO (obj)), + ALLOCNO_REGNO (OBJECT_ALLOCNO (obj)), lr->start, lr->finish, + lr->start, curr_point); + } ira_assert (lr != NULL); lr->finish = curr_point; update_allocno_pressure_excess_length (obj); @@ -295,77 +367,144 @@ pseudo_regno_single_word_and_live_p (int regno) return sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj)); } -/* Mark the pseudo register REGNO as live. Update all information about - live ranges and register pressure. */ +/* Collect the point which the OBJ be def/use. */ static void -mark_pseudo_regno_live (int regno) +add_subreg_point (ira_object_t obj, bool is_def, bool is_dec = true) { - ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - enum reg_class pclass; - int i, n, nregs; - - if (a == NULL) - return; + ira_allocno_t a = OBJECT_ALLOCNO (obj); + if (is_def) + { + OBJECT_CONFLICT_HARD_REGS (obj) |= hard_regs_live; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= hard_regs_live; + if (is_dec) + { + enum reg_class pclass + = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + dec_register_pressure (pclass, ALLOCNO_NREGS (a)); + } + update_allocno_pressure_excess_length (obj); + } + else + { + enum reg_class pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + inc_register_pressure (pclass, ALLOCNO_NREGS (a)); + } - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + subreg_range r = subreg_range ( + {OBJECT_START (obj), OBJECT_START (obj) + OBJECT_NREGS (obj)}); + subreg_live_points->add_point (ALLOCNO_NUM (a), ALLOCNO_NREGS (a), r, is_def, + curr_point); - n = ALLOCNO_NUM_OBJECTS (a); - pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; - if (n > 1) + if (internal_flag_ira_verbose > 8 && ira_dump_file != NULL) { - /* We track every subobject separately. */ - gcc_assert (nregs == n); - nregs = 1; + fprintf (ira_dump_file, " %s a%d(r%d", is_def ? "def" : "use", + ALLOCNO_NUM (a), ALLOCNO_REGNO (a)); + if (ALLOCNO_CLASS (a) != NO_REGS + && ALLOCNO_NREGS (a) != OBJECT_NREGS (obj)) + fprintf (ira_dump_file, " [subreg: start %d, nregs %d]", + OBJECT_START (obj), OBJECT_NREGS (obj)); + else + fprintf (ira_dump_file, " [full: nregs %d]", OBJECT_NREGS (obj)); + fprintf (ira_dump_file, ") at point %d\n", curr_point); } - for (i = 0; i < n; i++) - { - ira_object_t obj = ALLOCNO_OBJECT (a, i); + gcc_assert (has_subreg_object_p (a)); + gcc_assert (subreg_live_points->subreg_live_ranges.count (ALLOCNO_NUM (a)) + != 0); + + const subreg_ranges &sr + = subreg_live_points->subreg_live_ranges.at (ALLOCNO_NUM (a)); + ira_object_t main_obj = find_object (a, 0, ALLOCNO_NREGS (a)); + gcc_assert (main_obj != NULL); + if (sr.empty_p () + && sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (main_obj))) + sparseset_clear_bit (objects_live, OBJECT_CONFLICT_ID (main_obj)); + else if (!sr.empty_p () + && !sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (main_obj))) + sparseset_set_bit (objects_live, OBJECT_CONFLICT_ID (main_obj)); +} +/* Mark the object OBJ as live. */ +static void +mark_pseudo_object_live (ira_allocno_t a, ira_object_t obj) +{ + /* Invalidate because it is referenced. */ + allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + + if (has_subreg_object_p (a)) + add_subreg_point (obj, false); + else + { if (sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - continue; + return; - inc_register_pressure (pclass, nregs); + enum reg_class pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + inc_register_pressure (pclass, ALLOCNO_NREGS (a)); make_object_live (obj); } } +/* Mark the pseudo register REGNO as live. Update all information about + live ranges and register pressure. */ +static void +mark_pseudo_regno_live (int regno) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + + if (a == NULL) + return; + + int nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; + ira_object_t obj = find_object (a, 0, nregs); + gcc_assert (obj != NULL); + + mark_pseudo_object_live (a, obj); +} + /* Like mark_pseudo_regno_live, but try to only mark one subword of the pseudo as live. SUBWORD indicates which; a value of 0 indicates the low part. */ static void -mark_pseudo_regno_subword_live (int regno, int subword) +mark_pseudo_regno_subreg_live (int regno, rtx subreg) { ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n; - enum reg_class pclass; - ira_object_t obj; if (a == NULL) return; - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + ira_object_t obj + = find_object (a, SUBREG_BYTE (subreg), GET_MODE_SIZE (GET_MODE (subreg))); + gcc_assert (obj != NULL); + + mark_pseudo_object_live (a, obj); +} - n = ALLOCNO_NUM_OBJECTS (a); - if (n == 1) +/* Mark objects in subreg ranges SR as live. Update all information about + live ranges and register pressure. */ +static void +mark_pseudo_regno_subregs_live (int regno, const subreg_ranges &sr) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + if (a == NULL) + return; + + if (!ALLOCNO_TRACK_SUBREG_P (a)) { mark_pseudo_regno_live (regno); return; } - pclass = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - gcc_assert - (n == ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]); - obj = ALLOCNO_OBJECT (a, subword); - - if (sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - return; - - inc_register_pressure (pclass, 1); - make_object_live (obj); + int times = sr.max / ALLOCNO_NREGS (a); + gcc_assert (sr.max >= ALLOCNO_NREGS (a) + && times * ALLOCNO_NREGS (a) == sr.max); + for (const subreg_range &range : sr.ranges) + { + int start = range.start / times; + int end = CEIL (range.end, times); + ira_object_t obj = find_object (a, start, end - start); + gcc_assert (obj != NULL); + mark_pseudo_object_live (a, obj); + } } /* Mark the register REG as live. Store a 1 in hard_regs_live for @@ -403,10 +542,7 @@ static void mark_pseudo_reg_live (rtx orig_reg, unsigned regno) { if (read_modify_subreg_p (orig_reg)) - { - mark_pseudo_regno_subword_live (regno, - subreg_lowpart_p (orig_reg) ? 0 : 1); - } + mark_pseudo_regno_subreg_live (regno, orig_reg); else mark_pseudo_regno_live (regno); } @@ -427,72 +563,59 @@ mark_ref_live (df_ref ref) mark_hard_reg_live (reg); } -/* Mark the pseudo register REGNO as dead. Update all information about - live ranges and register pressure. */ +/* Mark object as dead. */ static void -mark_pseudo_regno_dead (int regno) +mark_pseudo_object_dead (ira_allocno_t a, ira_object_t obj) { - ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n, i, nregs; - enum reg_class cl; - - if (a == NULL) - return; - /* Invalidate because it is referenced. */ allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; - n = ALLOCNO_NUM_OBJECTS (a); - cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; - if (n > 1) - { - /* We track every subobject separately. */ - gcc_assert (nregs == n); - nregs = 1; - } - for (i = 0; i < n; i++) + if (has_subreg_object_p (a)) + add_subreg_point (obj, true); + else { - ira_object_t obj = ALLOCNO_OBJECT (a, i); if (!sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) - continue; + return; - dec_register_pressure (cl, nregs); + enum reg_class cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; + dec_register_pressure (cl, ALLOCNO_NREGS (a)); make_object_dead (obj); } } -/* Like mark_pseudo_regno_dead, but called when we know that only part of the - register dies. SUBWORD indicates which; a value of 0 indicates the low part. */ +/* Mark the pseudo register REGNO as dead. Update all information about + live ranges and register pressure. */ static void -mark_pseudo_regno_subword_dead (int regno, int subword) +mark_pseudo_regno_dead (int regno) { ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - int n; - enum reg_class cl; - ira_object_t obj; if (a == NULL) return; - /* Invalidate because it is referenced. */ - allocno_saved_at_call[ALLOCNO_NUM (a)] = 0; + int nregs = ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]; + ira_object_t obj = find_object (a, 0, nregs); + gcc_assert (obj != NULL); - n = ALLOCNO_NUM_OBJECTS (a); - if (n == 1) - /* The allocno as a whole doesn't die in this case. */ - return; + mark_pseudo_object_dead (a, obj); +} - cl = ira_pressure_class_translate[ALLOCNO_CLASS (a)]; - gcc_assert - (n == ira_reg_class_max_nregs[ALLOCNO_CLASS (a)][ALLOCNO_MODE (a)]); +/* Like mark_pseudo_regno_dead, but called when we know that only part of the + register dies. SUBWORD indicates which; a value of 0 indicates the low part. + */ +static void +mark_pseudo_regno_subreg_dead (int regno, rtx subreg) +{ + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; - obj = ALLOCNO_OBJECT (a, subword); - if (!sparseset_bit_p (objects_live, OBJECT_CONFLICT_ID (obj))) + if (a == NULL) return; - dec_register_pressure (cl, 1); - make_object_dead (obj); + ira_object_t obj + = find_object (a, SUBREG_BYTE (subreg), GET_MODE_SIZE (GET_MODE (subreg))); + gcc_assert (obj != NULL); + + mark_pseudo_object_dead (a, obj); } /* Process the definition of hard register REG. This updates hard_regs_live @@ -528,10 +651,7 @@ static void mark_pseudo_reg_dead (rtx orig_reg, unsigned regno) { if (read_modify_subreg_p (orig_reg)) - { - mark_pseudo_regno_subword_dead (regno, - subreg_lowpart_p (orig_reg) ? 0 : 1); - } + mark_pseudo_regno_subreg_dead (regno, orig_reg); else mark_pseudo_regno_dead (regno); } @@ -1059,8 +1179,14 @@ process_single_reg_class_operands (bool in_p, int freq) /* We could increase costs of A instead of making it conflicting with the hard register. But it works worse because it will be spilled in reload in anyway. */ - OBJECT_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; - OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; + if (has_subreg_object_p (a)) + set_subreg_conflict_hard_regs (OBJECT_ALLOCNO (obj), + reg_class_contents[cl]); + else + { + OBJECT_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= reg_class_contents[cl]; + } } } } @@ -1198,17 +1324,15 @@ process_out_of_region_eh_regs (basic_block bb) bi) { ira_allocno_t a = ira_curr_regno_allocno_map[i]; - for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--) + ira_object_t obj = find_object (a, 0, ALLOCNO_NREGS (a)); + for (int k = 0;; k++) { - ira_object_t obj = ALLOCNO_OBJECT (a, n); - for (int k = 0;; k++) - { - unsigned int regno = EH_RETURN_DATA_REGNO (k); - if (regno == INVALID_REGNUM) - break; - SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); - SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); - } + unsigned int regno = EH_RETURN_DATA_REGNO (k); + if (regno == INVALID_REGNUM) + break; + + SET_HARD_REG_BIT (OBJECT_CONFLICT_HARD_REGS (obj), regno); + SET_HARD_REG_BIT (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj), regno); } } } @@ -1234,6 +1358,10 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) bb = loop_tree_node->bb; if (bb != NULL) { + if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) + fprintf (ira_dump_file, "\n BB exit(l%d): point = %d\n", + loop_tree_node->parent->loop_num, curr_point); + for (i = 0; i < ira_pressure_classes_num; i++) { curr_reg_pressure[ira_pressure_classes[i]] = 0; @@ -1242,6 +1370,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) curr_bb_node = loop_tree_node; reg_live_out = DF_LIVE_SUBREG_OUT (bb); sparseset_clear (objects_live); + subreg_live_points->clear_live_ranges (); REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out); hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs); for (i = 0; i < FIRST_PSEUDO_REGISTER; i++) @@ -1265,9 +1394,17 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) <= ira_class_hard_regs_num[cl]); } } - EXECUTE_IF_SET_IN_BITMAP (reg_live_out, FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_FULL_OUT (bb), + FIRST_PSEUDO_REGISTER, j, bi) mark_pseudo_regno_live (j); + EXECUTE_IF_SET_IN_BITMAP (DF_LIVE_SUBREG_PARTIAL_OUT (bb), + FIRST_PSEUDO_REGISTER, j, bi) + { + mark_pseudo_regno_subregs_live ( + j, DF_LIVE_SUBREG_RANGE_OUT (bb)->lives.at (j)); + } + #ifdef EH_RETURN_DATA_REGNO process_out_of_region_eh_regs (bb); #endif @@ -1381,27 +1518,33 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) || (!targetm.setjmp_preserves_nonvolatile_regs_p () && (find_reg_note (insn, REG_SETJMP, NULL_RTX) != NULL_RTX))) + { + if (has_subreg_object_p (a)) + { + HARD_REG_SET regs; + SET_HARD_REG_SET (regs); + set_subreg_conflict_hard_regs (a, regs); + } + else + { + SET_HARD_REG_SET (OBJECT_CONFLICT_HARD_REGS (obj)); + SET_HARD_REG_SET ( + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj)); + } + } + if (can_throw_internal (insn)) { - SET_HARD_REG_SET (OBJECT_CONFLICT_HARD_REGS (obj)); - SET_HARD_REG_SET (OBJECT_TOTAL_CONFLICT_HARD_REGS (obj)); - } - eh_region r; - eh_landing_pad lp; - rtx_code_label *landing_label; - basic_block landing_bb; - if (can_throw_internal (insn) - && (r = get_eh_region_from_rtx (insn)) != NULL - && (lp = gen_eh_landing_pad (r)) != NULL - && (landing_label = lp->landing_pad) != NULL - && (landing_bb = BLOCK_FOR_INSN (landing_label)) != NULL - && (r->type != ERT_CLEANUP - || bitmap_bit_p (df_get_live_in (landing_bb), - ALLOCNO_REGNO (a)))) - { - HARD_REG_SET new_conflict_regs - = callee_abi.mode_clobbers (ALLOCNO_MODE (a)); - OBJECT_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; - OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) |= new_conflict_regs; + if (has_subreg_object_p (a)) + set_subreg_conflict_hard_regs (a, + callee_abi.mode_clobbers ( + ALLOCNO_MODE (a))); + else + { + OBJECT_CONFLICT_HARD_REGS (obj) + |= callee_abi.mode_clobbers (ALLOCNO_MODE (a)); + OBJECT_TOTAL_CONFLICT_HARD_REGS (obj) + |= callee_abi.mode_clobbers (ALLOCNO_MODE (a)); + } } if (sparseset_bit_p (allocnos_processed, num)) continue; @@ -1443,7 +1586,14 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) /* Mark each used value as live. */ FOR_EACH_INSN_USE (use, insn) - mark_ref_live (use); + { + unsigned regno = DF_REF_REGNO (use); + ira_allocno_t a = ira_curr_regno_allocno_map[regno]; + if (a && has_subreg_object_p (a) + && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG)) + continue; + mark_ref_live (use); + } process_single_reg_class_operands (true, freq); @@ -1473,6 +1623,10 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) } ignore_reg_for_conflicts = NULL_RTX; + if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) + fprintf (ira_dump_file, "\n BB head(l%d): point = %d\n", + loop_tree_node->parent->loop_num, curr_point); + if (bb_has_eh_pred (bb)) for (j = 0; ; ++j) { @@ -1526,10 +1680,15 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) } EXECUTE_IF_SET_IN_SPARSESET (objects_live, i) - make_object_dead (ira_object_id_map[i]); + { + ira_object_t obj = ira_object_id_map[i]; + if (has_subreg_object_p (OBJECT_ALLOCNO (obj))) + add_subreg_point (obj, true, false); + else + make_object_dead (obj); + } curr_point++; - } /* Propagate register pressure to upper loop tree nodes. */ if (loop_tree_node != ira_loop_tree_root) @@ -1730,6 +1889,86 @@ ira_debug_live_ranges (void) print_live_ranges (stderr); } +class subreg_live_item +{ +public: + subreg_ranges subreg; + int start, finish; +}; + +/* Create subreg live ranges from objects def/use point info. */ +static void +create_subregs_live_ranges () +{ + for (const auto &subreg_point_it : subreg_live_points->subreg_points) + { + unsigned int allocno_num = subreg_point_it.first; + const class live_points &points = subreg_point_it.second; + ira_allocno_t a = ira_allocnos[allocno_num]; + std::vector temps; + gcc_assert (has_subreg_object_p (a)); + for (const auto &point_it : points.points) + { + int point = point_it.first; + const live_point ®s = point_it.second; + gcc_assert (temps.empty () || temps.back ().finish <= point); + if (!regs.use_reg.empty_p ()) + { + if (temps.empty ()) + temps.push_back ({regs.use_reg, point, -1}); + else if (temps.back ().finish == -1) + { + if (!temps.back ().subreg.same_p (regs.use_reg)) + { + if (temps.back ().start == point) + temps.back ().subreg.add_ranges (regs.use_reg); + else + { + temps.back ().finish = point - 1; + + subreg_ranges temp = regs.use_reg; + temp.add_ranges (temps.back ().subreg); + temps.push_back ({temp, point, -1}); + } + } + } + else if (temps.back ().subreg.same_p (regs.use_reg) + && (temps.back ().finish == point + || temps.back ().finish + 1 == point)) + temps.back ().finish = -1; + else + temps.push_back ({regs.use_reg, point, -1}); + } + if (!regs.def_reg.empty_p ()) + { + gcc_assert (!temps.empty ()); + if (regs.def_reg.include_ranges_p (temps.back ().subreg)) + temps.back ().finish = point; + else if (temps.back ().subreg.include_ranges_p (regs.def_reg)) + { + temps.back ().finish = point; + + subreg_ranges diff = temps.back ().subreg; + diff.remove_ranges (regs.def_reg); + temps.push_back ({diff, point + 1, -1}); + } + else + gcc_unreachable (); + } + } + for (const subreg_live_item &item : temps) + for (const subreg_range &r : item.subreg.ranges) + { + ira_object_t obj = find_object_anyway (a, r.start, r.end - r.start); + live_range_t lr = OBJECT_LIVE_RANGES (obj); + if (lr != NULL && lr->finish + 1 == item.start) + lr->finish = item.finish; + else + ira_add_live_range_to_object (obj, item.start, item.finish); + } + } +} + /* The main entry function creates live ranges, set up CONFLICT_HARD_REGS and TOTAL_CONFLICT_HARD_REGS for objects, and calculate register pressure info. */ @@ -1743,13 +1982,20 @@ ira_create_allocno_live_ranges (void) allocno_saved_at_call = (int *) ira_allocate (ira_allocnos_num * sizeof (int)); memset (allocno_saved_at_call, 0, ira_allocnos_num * sizeof (int)); + subreg_live_points = new subregs_live_points (); ira_traverse_loop_tree (true, ira_loop_tree_root, NULL, process_bb_node_lives); ira_max_point = curr_point; + create_subregs_live_ranges (); create_start_finish_chains (); if (internal_flag_ira_verbose > 2 && ira_dump_file != NULL) - print_live_ranges (ira_dump_file); + { + fprintf (ira_dump_file, ";; subreg live points:\n"); + subreg_live_points->dump (ira_dump_file); + print_live_ranges (ira_dump_file); + } /* Clean up. */ + delete subreg_live_points; ira_free (allocno_saved_at_call); sparseset_free (objects_live); sparseset_free (allocnos_processed); diff --git a/gcc/subreg-live-range.h b/gcc/subreg-live-range.h index 56931b53550..bee97708a52 100644 --- a/gcc/subreg-live-range.h +++ b/gcc/subreg-live-range.h @@ -275,11 +275,20 @@ class subregs_live_points { public: std::map subreg_points; + std::map last_start_points; std::map subreg_live_ranges; void add_point (int id, int max, const subreg_range &range, bool is_def, int point) { + if (!is_def && empty_live_p (id)) + { + if (last_start_points.count (id) == 0) + last_start_points.insert ({id, point}); + else + last_start_points.at (id) = point; + } + if (subreg_points.count (id) == 0) subreg_points.insert ({id, live_points (id, max)}); @@ -317,6 +326,13 @@ public: || subreg_live_ranges.at (id).empty_p (); } + int get_start_point (int id) + { + int start_point = last_start_points.at (id); + gcc_assert (start_point != -1); + return start_point; + } + void clear_live_ranges () { subreg_live_ranges.clear (); } /* Debug methods. */