From patchwork Sat Feb 3 10:50:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 1894900 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TRqFq4LFZz23gM for ; Sat, 3 Feb 2024 21:51:43 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 94FAE385782B for ; Sat, 3 Feb 2024 10:51:41 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgbr2.qq.com (smtpbgbr2.qq.com [54.207.22.56]) by sourceware.org (Postfix) with ESMTPS id 7BE113858289 for ; Sat, 3 Feb 2024 10:50:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7BE113858289 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 7BE113858289 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.207.22.56 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957441; cv=none; b=H/RHz/G59vjh4HZjpD1wv7l26etO6fXDgZ14fV2yR2xwTSKzE/VeALywZBqswF9DR+AsZpn/T68SAfDeS4Sey53Uctiok0M54NBIL3wrPat+d/Vbub0Jsssc5MzcA0sxq6iBg9V1stML1UijFBlHL8UD+SxpOCQGu6ghce9SjKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957441; c=relaxed/simple; bh=XJ+5GLBfB5kXviuDAXYPAzoqhs3er06qleaSS3IA7EE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=wTUBkuqqw/kmARwtT7TSurvNIUndYizEMYtyOiFectBekxCRdIqB/i8nnYCIkSfMg7vzo5wyYYWGKl5qt0epD0lnmqlYVWu3hgf9cDO12jaxtZbyVYgFSdaVpii6GlapvrLKubnnVWtSpVm7ykBproAA5QnxVz+n1fFb1rqn3Tk= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp72t1706957427t4fhb6xg X-QQ-Originating-IP: 5x5BkfjicudGiOEOqxY4XBEjgjUTxE7gwZHEc1hFEtE= Received: from rios-cad5.localdomain ( [58.60.1.25]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 03 Feb 2024 18:50:26 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: +ynUkgUhZJlq9vRSwPu3aF3w/Bbjf/BRg0WdacLMrrJAe3KPRmrrBWaX26fF7 BiHH57Kmdaw+UN6CmT3UOiOxClgglGQrkhDOGqVHZdkoh6WKAF3vKQOpNmh2Xskbvh+YRcj KozHybFWQZtoNLaC6BGDbNRjiXtmDWrcAu7BQ90cVC67jbzVx21Wc9ZmPvXvHcx3g5nz3g5 56hhRhNejLlqYmDCuIQ0BIWr+AOCldxmpHB4fkkUpP4wY8ND8X40g3UCBAq4Yf/Z5X5GLmg TOKm/Hk88nSFjutdO3cNPuQGdGLUBj7Sx4DTiUYSIBJNEGJgBHv3r+CizVM+wu0K/rJFYea rWGz0n5c80BsrIh2qxU7pXQyQ1Abkgr2xBzIe0DrMz/j+CQ0Xv+rh09yKGjhdgmEfMWUsKW X-QQ-GoodBg: 2 X-BIZMAIL-ID: 7222153971514329300 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 4/4] lra: Apply DF_LIVE_SUBREG data Date: Sat, 3 Feb 2024 18:50:12 +0800 Message-Id: <20240203105012.208998-5-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240203105012.208998-1-lehua.ding@rivai.ai> References: <20240203105012.208998-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz5a-3 X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org This patch apply the DF_LIVE_SUBREG to LRA pass. More changes were made to the LRA than the IRA since the LRA will modify the DF data directly. The main big changes are centered on the lra-lives.cc file. gcc/ChangeLog: * lra-coalesce.cc (update_live_info): Extend to DF_LIVE_SUBREG. (lra_coalesce): Ditto. * lra-constraints.cc (update_ebb_live_info): Ditto. (get_live_on_other_edges): Ditto. (inherit_in_ebb): Ditto. (lra_inheritance): Ditto. (fix_bb_live_info): Ditto. (remove_inheritance_pseudos): Ditto. * lra-int.h (GCC_LRA_INT_H): include subreg-live-range.h (struct lra_insn_reg): Add op filed to record the corresponding rtx. * lra-lives.cc (class bb_data_pseudos): Extend the bb_data_pseudos to include new partial_def/use and range_def/use fileds for DF_LIVE_SUBREG problem. (need_track_subreg_p): checking is the regno need to be tracked. (make_hard_regno_live): switch to live_subreg filed. (make_hard_regno_dead): Ditto. (mark_regno_live): Support record subreg liveness. (mark_regno_dead): Ditto. (live_trans_fun): Adjust transfer function to support subreg liveness. (live_con_fun_0): Adjust Confluence function to support subreg liveness. (live_con_fun_n): Ditto. (initiate_live_solver): Ditto. (finish_live_solver): Ditto. (process_bb_lives): Ditto. (lra_create_live_ranges_1): Dump subreg liveness. * lra-remat.cc (dump_candidates_and_remat_bb_data): Switch to DF_LIVE_SUBREG df data. (calculate_livein_cands): Ditto. (do_remat): Ditto. * lra-spills.cc (spill_pseudos): Ditto. * lra.cc (new_insn_reg): New argument op. (add_regs_to_insn_regno_info): Add new argument op. --- gcc/lra-coalesce.cc | 27 +++- gcc/lra-constraints.cc | 109 ++++++++++--- gcc/lra-int.h | 4 + gcc/lra-lives.cc | 357 ++++++++++++++++++++++++++++++++--------- gcc/lra-remat.cc | 8 +- gcc/lra-spills.cc | 27 +++- gcc/lra.cc | 10 +- 7 files changed, 430 insertions(+), 112 deletions(-) diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc index a9b5b51cb3f..9416775a009 100644 --- a/gcc/lra-coalesce.cc +++ b/gcc/lra-coalesce.cc @@ -186,19 +186,28 @@ static bitmap_head used_pseudos_bitmap; /* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info bitmap). */ static void -update_live_info (bitmap lr_bitmap) +update_live_info (bitmap all, bitmap full, bitmap partial) { unsigned int j; bitmap_iterator bi; bitmap_clear (&used_pseudos_bitmap); - EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap, + EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all, FIRST_PSEUDO_REGISTER, j, bi) bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]); - if (! bitmap_empty_p (&used_pseudos_bitmap)) + if (!bitmap_empty_p (&used_pseudos_bitmap)) { - bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap); - bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap); + bitmap_and_compl_into (all, &coalesced_pseudos_bitmap); + bitmap_ior_into (all, &used_pseudos_bitmap); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (full, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial); + + bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full); + } } } @@ -301,8 +310,12 @@ lra_coalesce (void) bitmap_initialize (&used_pseudos_bitmap, ®_obstack); FOR_EACH_BB_FN (bb, cfun) { - update_live_info (df_get_live_in (bb)); - update_live_info (df_get_live_out (bb)); + update_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb)); + update_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb)); FOR_BB_INSNS_SAFE (bb, insn, next) if (INSN_P (insn) && bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn))) diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc index 0ae81c1ff9c..d1316620f51 100644 --- a/gcc/lra-constraints.cc +++ b/gcc/lra-constraints.cc @@ -6505,34 +6505,86 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail) { if (prev_bb != NULL) { - /* Update df_get_live_in (prev_bb): */ + /* Update subreg live (prev_bb): */ + bitmap subreg_all_in = df_get_subreg_live_in (prev_bb); + bitmap subreg_full_in = df_get_subreg_live_full_in (prev_bb); + bitmap subreg_partial_in = df_get_subreg_live_partial_in (prev_bb); + subregs_live *range_in = df_get_subreg_live_range_in (prev_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) if (bitmap_bit_p (&live_regs, j)) - bitmap_set_bit (df_get_live_in (prev_bb), j); - else - bitmap_clear_bit (df_get_live_in (prev_bb), j); + { + bitmap_set_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_in, j)) + { + bitmap_clear_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } } + bitmap subreg_all_out = df_get_subreg_live_out (curr_bb); if (curr_bb != last_bb) { - /* Update df_get_live_out (curr_bb): */ + /* Update subreg live (curr_bb): */ + bitmap subreg_full_out = df_get_subreg_live_full_out (curr_bb); + bitmap subreg_partial_out = df_get_subreg_live_partial_out (curr_bb); + subregs_live *range_out = df_get_subreg_live_range_out (curr_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) { live_p = bitmap_bit_p (&live_regs, j); if (! live_p) FOR_EACH_EDGE (e, ei, curr_bb->succs) - if (bitmap_bit_p (df_get_live_in (e->dest), j)) + if (bitmap_bit_p (df_get_subreg_live_in (e->dest), j)) { live_p = true; break; } if (live_p) - bitmap_set_bit (df_get_live_out (curr_bb), j); - else - bitmap_clear_bit (df_get_live_out (curr_bb), j); + { + bitmap_set_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_out, j)) + { + bitmap_clear_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } } } prev_bb = curr_bb; - bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb)); + bitmap_and (&live_regs, &check_only_regs, subreg_all_out); } if (! NONDEBUG_INSN_P (curr_insn)) continue; @@ -6649,7 +6701,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res) bitmap_clear (res); FOR_EACH_EDGE (e, ei, from->succs) if (e->dest != to) - bitmap_ior_into (res, df_get_live_in (e->dest)); + bitmap_ior_into (res, df_get_subreg_live_in (e->dest)); last = get_last_insertion_point (from); if (! JUMP_P (last)) return; @@ -6721,7 +6773,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We are at the end of BB. Add qualified living pseudos for potential splitting. */ - to_process = df_get_live_out (curr_bb); + to_process = df_get_subreg_live_out (curr_bb); if (last_processed_bb != NULL) { /* We are somewhere in the middle of EBB. */ @@ -7093,7 +7145,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We reached the beginning of the current block -- do rest of spliting in the current BB. */ - to_process = df_get_live_in (curr_bb); + to_process = df_get_subreg_live_in (curr_bb); if (BLOCK_FOR_INSN (head) != curr_bb) { /* We are somewhere in the middle of EBB. */ @@ -7170,7 +7222,7 @@ lra_inheritance (void) fprintf (lra_dump_file, "EBB"); /* Form a EBB starting with BB. */ bitmap_clear (&ebb_global_regs); - bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_in (bb)); for (;;) { if (lra_dump_file != NULL) @@ -7186,7 +7238,7 @@ lra_inheritance (void) break; bb = bb->next_bb; } - bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_out (bb)); if (lra_dump_file != NULL) fprintf (lra_dump_file, "\n"); if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb))) @@ -7215,15 +7267,26 @@ int lra_undo_inheritance_iter; /* Fix BB live info LIVE after removing pseudos created on pass doing inheritance/split which are REMOVED_PSEUDOS. */ static void -fix_bb_live_info (bitmap live, bitmap removed_pseudos) +fix_bb_live_info (bitmap all, bitmap full, bitmap partial, + bitmap removed_pseudos) { unsigned int regno; bitmap_iterator bi; EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi) - if (bitmap_clear_bit (live, regno) - && REG_P (lra_reg_info[regno].restore_rtx)) - bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx)); + { + if (bitmap_clear_bit (all, regno) + && REG_P (lra_reg_info[regno].restore_rtx)) + { + bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx)); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (full, regno); + bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx)); + gcc_assert (!bitmap_bit_p (partial, regno)); + } + } + } } /* Return regno of the (subreg of) REG. Otherwise, return a negative @@ -7289,8 +7352,12 @@ remove_inheritance_pseudos (bitmap remove_pseudos) constraint pass. */ FOR_EACH_BB_FN (bb, cfun) { - fix_bb_live_info (df_get_live_in (bb), remove_pseudos); - fix_bb_live_info (df_get_live_out (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb), remove_pseudos); FOR_BB_INSNS_REVERSE (bb, curr_insn) { if (! INSN_P (curr_insn)) diff --git a/gcc/lra-int.h b/gcc/lra-int.h index 5f605c3ae41..d73ff4e31b5 100644 --- a/gcc/lra-int.h +++ b/gcc/lra-int.h @@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_LRA_INT_H #define GCC_LRA_INT_H +#include "subreg-live-range.h" + #define lra_assert(c) gcc_checking_assert (c) /* The parameter used to prevent infinite reloading for an insn. Each @@ -161,6 +163,8 @@ struct lra_insn_reg int regno; /* Next reg info of the same insn. */ struct lra_insn_reg *next; + /* The corresponding reg or subreg RTX. */ + rtx op; }; /* Static part (common info for insns with the same ICODE) of LRA diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc index 66c6577e5d6..291f9420e06 100644 --- a/gcc/lra-lives.cc +++ b/gcc/lra-lives.cc @@ -272,8 +272,34 @@ update_pseudo_point (int regno, int point, enum point_type type) } } -/* The corresponding bitmaps of BB currently being processed. */ -static bitmap bb_killed_pseudos, bb_gen_pseudos; +/* Structure describing local BB data used for pseudo + live-analysis. */ +class bb_data_pseudos : public df_live_subreg_local_bb_info +{ +public: + /* Basic block about which the below data are. */ + basic_block bb; +}; + +/* Array for all BB data. Indexed by the corresponding BB index. */ +typedef class bb_data_pseudos *bb_data_t; + +/* All basic block data are referred through the following array. */ +static bb_data_t bb_data; + +/* The corresponding df local data of BB currently being processed. */ +static bb_data_t curr_bb_info; + +/* The regs which need to be tracked it's subreg liveness. */ +static bitmap_head tracked_regs; + +/* Return true if the REGNO need be track with subreg liveness. */ + +static bool +need_track_subreg_p (unsigned regno) +{ + return bitmap_bit_p (&tracked_regs, regno); +} /* Record hard register REGNO as now being live. It updates living hard regs and START_LIVING. */ @@ -287,7 +313,7 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); sparseset_set_bit (start_living, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); } /* Process the definition of hard register REGNO. This updates @@ -310,8 +336,8 @@ make_hard_regno_dead (int regno) sparseset_set_bit (start_dying, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) { - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); } } @@ -343,7 +369,7 @@ mark_pseudo_dead (int regno) /* Mark register REGNO (pseudo or hard register) in MODE as being live and update BB_GEN_PSEUDOS. */ static void -mark_regno_live (int regno, machine_mode mode) +mark_regno_live (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -355,15 +381,23 @@ mark_regno_live (int regno, machine_mode mode) else { mark_pseudo_live (regno); - bitmap_set_bit (bb_gen_pseudos, regno); + if (!need_track_subreg_p (regno)) + bitmap_set_bit (&curr_bb_info->full_use, regno); + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + add_subreg_range (curr_bb_info, regno, range, false); + } } } - /* Mark register REGNO (pseudo or hard register) in MODE as being dead and update BB_GEN_PSEUDOS and BB_KILLED_PSEUDOS. */ static void -mark_regno_dead (int regno, machine_mode mode) +mark_regno_dead (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -375,8 +409,20 @@ mark_regno_dead (int regno, machine_mode mode) else { mark_pseudo_dead (regno); - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + if (!need_track_subreg_p (regno)) + { + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); + } + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + remove_subreg_range (curr_bb_info, regno, range); + add_subreg_range (curr_bb_info, regno, range, true); + } } } @@ -387,23 +433,6 @@ mark_regno_dead (int regno, machine_mode mode) border. That might be a consequence of some global transformations in LRA, e.g. PIC pseudo reuse or rematerialization. */ -/* Structure describing local BB data used for pseudo - live-analysis. */ -class bb_data_pseudos -{ -public: - /* Basic block about which the below data are. */ - basic_block bb; - bitmap_head killed_pseudos; /* pseudos killed in the BB. */ - bitmap_head gen_pseudos; /* pseudos generated in the BB. */ -}; - -/* Array for all BB data. Indexed by the corresponding BB index. */ -typedef class bb_data_pseudos *bb_data_t; - -/* All basic block data are referred through the following array. */ -static bb_data_t bb_data; - /* Two small functions for access to the bb data. */ static inline bb_data_t get_bb_data (basic_block bb) @@ -429,14 +458,73 @@ static bitmap_head all_hard_regs_bitmap; static bool live_trans_fun (int bb_index) { - basic_block bb = get_bb_data_by_index (bb_index)->bb; - bitmap bb_liveout = df_get_live_out (bb); - bitmap bb_livein = df_get_live_in (bb); - bb_data_t bb_info = get_bb_data (bb); - - bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap); - return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos, - &temp_bitmap, &bb_info->killed_pseudos); + bb_data_t local_bb_info = get_bb_data_by_index (bb_index); + bitmap full_in = df_get_subreg_live_full_in (local_bb_info->bb); + bitmap full_out = df_get_subreg_live_full_out (local_bb_info->bb); + + bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap); + bool changed = bitmap_ior_and_compl (full_in, &local_bb_info->full_use, + &temp_bitmap, &local_bb_info->full_def); + + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap partial_in = df_get_subreg_live_partial_in (local_bb_info->bb); + bitmap partial_out = df_get_subreg_live_partial_out (local_bb_info->bb); + subregs_live *range_in = df_get_subreg_live_range_in (local_bb_info->bb); + subregs_live *range_out + = df_get_subreg_live_range_out (local_bb_info->bb); + + if (!bitmap_empty_p (partial_out) + || !bitmap_empty_p (&local_bb_info->partial_use)) + { + unsigned int regno; + bitmap_iterator bi; + bitmap_head temp_partial_out; + subregs_live temp_range_out; + + /* TEMP = (OUT & ~DEF) */ + bitmap_initialize (&temp_partial_out, &bitmap_default_obstack); + EXECUTE_IF_SET_IN_BITMAP (partial_out, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap out_range = range_out->get_range (regno); + temp_range_out.add_range (regno, out_range); + if (bitmap_bit_p (&local_bb_info->partial_def, regno)) + { + sbitmap def_range + = local_bb_info->range_def->get_range (regno); + temp_range_out.remove_range (regno, def_range); + if (!temp_range_out.empty_p (regno)) + bitmap_set_bit (&temp_partial_out, regno); + } + else + bitmap_set_bit (&temp_partial_out, regno); + } + + /* TEMP = USE | TEMP */ + EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap use_range = local_bb_info->range_use->get_range (regno); + temp_range_out.add_range (regno, use_range); + } + bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use); + + /* IN = TEMP */ + changed |= range_in->copy_lives (temp_range_out); + bitmap_copy (partial_in, &temp_partial_out); + df_live_subreg_check_result (full_in, partial_in, range_in); + } + else if (!bitmap_empty_p (partial_in)) + { + changed = true; + bitmap_clear (partial_in); + range_in->clear (); + } + } + + return changed; } /* The confluence function used by the DF equation solver to set up @@ -444,7 +532,9 @@ live_trans_fun (int bb_index) static void live_con_fun_0 (basic_block bb) { - bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap); + bitmap_and_into (df_get_subreg_live_out (bb), &all_hard_regs_bitmap); + if (flag_track_subreg_liveness) + bitmap_and_into (df_get_subreg_live_full_out (bb), &all_hard_regs_bitmap); } /* The confluence function used by the DF equation solver to propagate @@ -456,13 +546,37 @@ live_con_fun_0 (basic_block bb) static bool live_con_fun_n (edge e) { - basic_block bb = e->src; - basic_block dest = e->dest; - bitmap bb_liveout = df_get_live_out (bb); - bitmap dest_livein = df_get_live_in (dest); + bitmap src_full_out = df_get_subreg_live_full_out (e->src); + bitmap dest_full_in = df_get_subreg_live_full_in (e->dest); + + bool changed = bitmap_ior_and_compl_into (src_full_out, dest_full_in, + &all_hard_regs_bitmap); + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap src_partial_out = df_get_subreg_live_partial_out (e->src); + subregs_live *src_range_out = df_get_subreg_live_range_out (e->src); + bitmap dest_partial_in = df_get_subreg_live_partial_in (e->dest); + subregs_live *dest_range_in = df_get_subreg_live_range_in (e->dest); + + if (bitmap_empty_p (dest_partial_in)) + return changed; + + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (dest_partial_in, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap dest_range = dest_range_in->get_range (regno); + changed |= src_range_out->add_range (regno, dest_range); + } + changed |= bitmap_ior_into (src_partial_out, dest_partial_in); - return bitmap_ior_and_compl_into (bb_liveout, - dest_livein, &all_hard_regs_bitmap); + df_live_subreg_check_result (src_full_out, src_partial_out, + src_range_out); + } + + return changed; } /* Indexes of all function blocks. */ @@ -479,12 +593,47 @@ initiate_live_solver (void) bitmap_initialize (&all_blocks, ®_obstack); basic_block bb; + if (flag_track_subreg_liveness) + { + bitmap_initialize (&tracked_regs, ®_obstack); + FOR_ALL_BB_FN (bb, cfun) + { + rtx_insn *insn; + df_ref use; + FOR_BB_INSNS (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* A multireg which is used via subreg pattern. */ + if (multireg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_SUBREG)) + bitmap_set_bit (&tracked_regs, regno); + } + } + } + } + + FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); bb_info->bb = bb; - bitmap_initialize (&bb_info->killed_pseudos, ®_obstack); - bitmap_initialize (&bb_info->gen_pseudos, ®_obstack); + bitmap_initialize (&bb_info->full_def, ®_obstack); + bitmap_initialize (&bb_info->full_use, ®_obstack); + if (flag_track_subreg_liveness) + { + bitmap_initialize (&bb_info->partial_def, ®_obstack); + bitmap_initialize (&bb_info->partial_use, ®_obstack); + size_t num_regs = bitmap_count_bits (&tracked_regs); + bb_info->range_def = new subregs_live (num_regs); + bb_info->range_use = new subregs_live (num_regs); + } bitmap_set_bit (&all_blocks, bb->index); } } @@ -496,11 +645,19 @@ finish_live_solver (void) basic_block bb; bitmap_clear (&all_blocks); + bitmap_clear (&tracked_regs); FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap_clear (&bb_info->killed_pseudos); - bitmap_clear (&bb_info->gen_pseudos); + bitmap_clear (&bb_info->full_def); + bitmap_clear (&bb_info->full_use); + if (flag_track_subreg_liveness) + { + bitmap_clear (&bb_info->partial_def); + bitmap_clear (&bb_info->partial_use); + delete bb_info->range_def; + delete bb_info->range_use; + } } free (bb_data); bitmap_clear (&all_hard_regs_bitmap); @@ -663,7 +820,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) /* Only has a meaningful value once we've seen a call. */ function_abi last_call_abi = default_function_abi; - reg_live_out = df_get_live_out (bb); + reg_live_out = df_get_subreg_live_out (bb); sparseset_clear (pseudos_live); sparseset_clear (pseudos_live_through_calls); sparseset_clear (pseudos_live_through_setjumps); @@ -675,10 +832,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_live (j); } - bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos; - bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos; - bitmap_clear (bb_gen_pseudos); - bitmap_clear (bb_killed_pseudos); + curr_bb_info = get_bb_data (bb); + bitmap_clear (&curr_bb_info->full_use); + bitmap_clear (&curr_bb_info->full_def); + if (flag_track_subreg_liveness) + { + bitmap_clear (&curr_bb_info->partial_use); + bitmap_clear (&curr_bb_info->partial_def); + curr_bb_info->range_use->clear (); + curr_bb_info->range_def->clear (); + } freq = REG_FREQ_FROM_BB (bb); if (lra_dump_file != NULL) @@ -860,7 +1023,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (reg->type != OP_IN) { update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); /* ??? Should be a no-op for unused registers. */ check_pseudos_live_through_calls (reg->regno, last_call_abi); } @@ -886,7 +1049,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -945,8 +1108,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_IN) update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); check_pseudos_live_through_calls (reg->regno, last_call_abi); + /* Ignore the use of subreg which is used as dest operand. */ + if (need_track_subreg_p (reg->regno) && reg->subreg_p + && reg->type == OP_INOUT) + continue; + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -970,12 +1137,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); /* We're done processing inputs, so make sure early clobber operands that are both inputs and outputs are still live. */ if (reg->type == OP_INOUT) - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -1099,8 +1266,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) bool live_change_p = false; /* Check if bb border live info was changed. */ unsigned int live_pseudos_num = 0; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), - FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { live_pseudos_num++; if (! sparseset_bit_p (pseudos_live, j)) @@ -1118,7 +1285,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) live_change_p = true; if (lra_dump_file != NULL) EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j) - if (! bitmap_bit_p (df_get_live_in (bb), j)) + if (! bitmap_bit_p (df_get_subreg_live_in (bb), j)) fprintf (lra_dump_file, " r%d is added to live at bb%d start\n", j, bb->index); } @@ -1133,7 +1300,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_dead (i); } - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { if (sparseset_cardinality (pseudos_live_through_calls) == 0) break; @@ -1149,7 +1317,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i)) continue; - if (bitmap_bit_p (df_get_live_in (bb), i)) + if (bitmap_bit_p (df_get_subreg_live_in (bb), i)) continue; live_change_p = true; @@ -1157,7 +1325,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) fprintf (lra_dump_file, " hard reg r%d is added to live at bb%d start\n", i, bb->index); - bitmap_set_bit (df_get_live_in (bb), i); + bitmap_set_bit (df_get_subreg_live_in (bb), i); + if (flag_track_subreg_liveness) + bitmap_set_bit (df_get_subreg_live_full_in (bb), i); } if (need_curr_point_incr) @@ -1421,12 +1591,30 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) { /* We need to clear pseudo live info as some pseudos can disappear, e.g. pseudos with used equivalences. */ - FOR_EACH_BB_FN (bb, cfun) + FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), + EXIT_BLOCK_PTR_FOR_FN (cfun), next_bb) { - bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); - bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); + if (flag_track_subreg_liveness) + { + bitmap_clear_range (df_get_subreg_live_full_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_full_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + df_get_subreg_live_range_in (bb)->clear (); + df_get_subreg_live_range_out (bb)->clear (); + } } /* As we did not change CFG since LRA start we can use DF-infrastructure solver to solve live data flow problem. */ @@ -1439,6 +1627,10 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) (DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n, live_trans_fun, &all_blocks, df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD)); + + if (flag_track_subreg_liveness) + df_live_subreg_finalize (&all_blocks); + if (lra_dump_file != NULL) { fprintf (lra_dump_file, @@ -1447,16 +1639,33 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) FOR_EACH_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap bb_livein = df_get_live_in (bb); - bitmap bb_liveout = df_get_live_out (bb); fprintf (lra_dump_file, "\nBB %d:\n", bb->index); - lra_dump_bitmap_with_title (" gen:", - &bb_info->gen_pseudos, bb->index); - lra_dump_bitmap_with_title (" killed:", - &bb_info->killed_pseudos, bb->index); - lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index); - lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index); + lra_dump_bitmap_with_title (" full use", &bb_info->full_use, + bb->index); + lra_dump_bitmap_with_title (" full def", &bb_info->full_def, + bb->index); + lra_dump_bitmap_with_title (" live in full", + df_get_subreg_live_full_in (bb), + bb->index); + lra_dump_bitmap_with_title (" live out full", + df_get_subreg_live_full_out (bb), + bb->index); + if (flag_track_subreg_liveness) + { + lra_dump_bitmap_with_title (" partial use", + &bb_info->partial_use, bb->index); + lra_dump_bitmap_with_title (" partial def", + &bb_info->partial_def, bb->index); + lra_dump_bitmap_with_title (" live in partial", + df_get_subreg_live_partial_in ( + bb), + bb->index); + lra_dump_bitmap_with_title (" live out partial", + df_get_subreg_live_partial_out ( + bb), + bb->index); + } } } } diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc index c84bf3c9938..ef0157513b0 100644 --- a/gcc/lra-remat.cc +++ b/gcc/lra-remat.cc @@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void) fprintf (lra_dump_file, "\nBB %d:\n", bb->index); /* Livein */ fprintf (lra_dump_file, " register live in:"); - dump_regset (df_get_live_in (bb), lra_dump_file); + dump_regset (df_get_subreg_live_in (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Liveout */ fprintf (lra_dump_file, " register live out:"); - dump_regset (df_get_live_out (bb), lra_dump_file); + dump_regset (df_get_subreg_live_out (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Changed/dead regs: */ fprintf (lra_dump_file, " changed regs:"); @@ -727,7 +727,7 @@ calculate_livein_cands (void) FOR_EACH_BB_FN (bb, cfun) { - bitmap livein_regs = df_get_live_in (bb); + bitmap livein_regs = df_get_subreg_live_in (bb); bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands; for (unsigned int i = 0; i < cands_num; i++) { @@ -1064,7 +1064,7 @@ do_remat (void) FOR_EACH_BB_FN (bb, cfun) { CLEAR_HARD_REG_SET (live_hard_regs); - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), 0, regno, bi) { int hard_regno = regno < FIRST_PSEUDO_REGISTER ? regno diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc index 8b1a945d632..08ec247bc00 100644 --- a/gcc/lra-spills.cc +++ b/gcc/lra-spills.cc @@ -566,8 +566,31 @@ spill_pseudos (void) "Debug insn #%u is reset because it referenced " "removed pseudo\n", INSN_UID (insn)); } - bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos); - bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_in (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_out (bb), spilled_pseudos); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (df_get_subreg_live_full_in (bb), + spilled_pseudos); + bitmap partial_in = df_get_subreg_live_partial_in (bb); + subregs_live *range_in = df_get_subreg_live_range_in (bb); + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_in->remove_range (regno); + bitmap_and_compl_into (partial_in, spilled_pseudos); + + bitmap_and_compl_into (df_get_subreg_live_full_out (bb), + spilled_pseudos); + bitmap partial_out = df_get_subreg_live_partial_out (bb); + subregs_live *range_out = df_get_subreg_live_range_out (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_out->remove_range (regno); + bitmap_and_compl_into (partial_out, spilled_pseudos); + } } } } diff --git a/gcc/lra.cc b/gcc/lra.cc index fb32e134004..a6fabe7d118 100644 --- a/gcc/lra.cc +++ b/gcc/lra.cc @@ -574,7 +574,7 @@ object_allocator lra_insn_reg_pool ("insn regs"); EARLY_CLOBBER_ALTS. */ static struct lra_insn_reg * new_insn_reg (rtx_insn *insn, int regno, enum op_type type, - machine_mode mode, bool subreg_p, + machine_mode mode, bool subreg_p, rtx op, alternative_mask early_clobber_alts, struct lra_insn_reg *next) { @@ -586,6 +586,7 @@ new_insn_reg (rtx_insn *insn, int regno, enum op_type type, ir->subreg_p = subreg_p; ir->early_clobber_alts = early_clobber_alts; ir->regno = regno; + ir->op = op; ir->next = next; return ir; } @@ -926,7 +927,7 @@ collect_non_operand_hard_regs (rtx_insn *insn, rtx *x, && ! (FIRST_STACK_REG <= regno && regno <= LAST_STACK_REG)); #endif - list = new_insn_reg (data->insn, regno, type, mode, subreg_p, + list = new_insn_reg (data->insn, regno, type, mode, subreg_p, *x, early_clobber ? ALL_ALTERNATIVES : 0, list); } } @@ -1484,6 +1485,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, code = GET_CODE (x); mode = GET_MODE (x); subreg_p = false; + rtx op = x; if (GET_CODE (x) == SUBREG) { mode = wider_subreg_mode (x); @@ -1501,7 +1503,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, if (bitmap_set_bit (&lra_reg_info[regno].insn_bitmap, INSN_UID (insn))) { data->regs = new_insn_reg (data->insn, regno, type, mode, subreg_p, - early_clobber_alts, data->regs); + op, early_clobber_alts, data->regs); return; } else @@ -1513,7 +1515,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, /* The info cannot be integrated into the found structure. */ data->regs = new_insn_reg (data->insn, regno, type, mode, - subreg_p, early_clobber_alts, + subreg_p, op, early_clobber_alts, data->regs); else {