From patchwork Tue Feb 4 14:54:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1233378 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-518892-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha1 header.s=default header.b=CIT3pNlS; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=lsPY1GBy; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48Bnlw4xN9zB3xW for ; Wed, 5 Feb 2020 01:54:34 +1100 (AEDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:references:mime-version:content-type :in-reply-to; q=dns; s=default; b=kn/WbihvEq71Mhb9F36cqupVCgQ8fn cqxJClzt+0e8lJ7V9z/2Pz7yoRuKDUGghFflCHQgyXGbfnPFFzdkJqz7rOcDTpBt KJURlnMm9r9TfRxs46ydr1mnG0kRo/cxuN4wfp3tz33OkrCo00kmrU0G5XVVWP8a hm1UqF6Ah6VDE= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:date :from:to:subject:message-id:references:mime-version:content-type :in-reply-to; s=default; bh=uObd8vXoSwAtiy0/+lVx+xp+H8c=; b=CIT3 pNlScA5mlwMfA1S1ZtXak8E6h+7qNeTUR/Mn25I/a5vLeToafJnLTsSHeNg4h6Nh ZBFSijE9NFuc8U3FKEbTgbY48k5ag6AO4Rxi/Ew4WTMAVKiMIz2WGvcL3lqaKFco aRaTB1wAJDUAc5N4HnNqr2FSjlaYqJK4Zs6CL5s= Received: (qmail 97951 invoked by alias); 4 Feb 2020 14:54:26 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 97943 invoked by uid 89); 4 Feb 2020 14:54:25 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.7 required=5.0 tests=AWL, BAYES_00, FREEMAIL_FROM, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=gate X-HELO: mail-pj1-f68.google.com Received: from mail-pj1-f68.google.com (HELO mail-pj1-f68.google.com) (209.85.216.68) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 04 Feb 2020 14:54:22 +0000 Received: by mail-pj1-f68.google.com with SMTP id e9so1486696pjr.4 for ; Tue, 04 Feb 2020 06:54:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=/PUbG3/q2hPWQ6bjhkDjfXdvp+Gryb90e8ceRXyH2RU=; b=lsPY1GBy708iMJ3MPrJCWw6p/e9W8UXE/Bzh0CzCeo/QJKqc6O4iRtYNp/YiYptcQ8 bv3KIOGsAejhiXVlTME3RqFywWxFhw89Ar5I3CG4sR2UiJCiCoFHfcpE5ynlW/B0mq66 k2HkF+IRPlD33Hvh5mstlVItHmdEhvKMWIpuKyNYPQmkd0f+fmzTU8VIHslZyGEEpbc8 st2bbn+yDNK1WXmstkSR2x+c7kqOQ+i2zXtoQyjbi0Sx2VKVwdwjSbmZlnaLSeW+E8uN H7FD8NbywpZU9HsksvjVqUXh7gorzgKy4K95O0Lu64oZ6OIrB5pZ+vnBKqab+a8MNSGk riyQ== Received: from gnu-cfl-2.localdomain (c-73-93-86-59.hsd1.ca.comcast.net. [73.93.86.59]) by smtp.gmail.com with ESMTPSA id e2sm3755913pjs.25.2020.02.04.06.54.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2020 06:54:19 -0800 (PST) Received: by gnu-cfl-2.localdomain (Postfix, from userid 1000) id 16883C037B; Tue, 4 Feb 2020 06:54:13 -0800 (PST) Date: Tue, 4 Feb 2020 06:54:13 -0800 From: "H.J. Lu" To: GCC Patches , Uros Bizjak Subject: [PATCH] x86: Add UNSPECV_PATCHABLE_AREA Message-ID: <20200204145413.GA90890@gmail.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-IsSubscribed: yes On Mon, Feb 03, 2020 at 06:10:49PM -0800, H.J. Lu wrote: > On Mon, Feb 3, 2020 at 4:02 PM H.J. Lu wrote: > > > > On Mon, Feb 3, 2020 at 10:35 AM H.J. Lu wrote: > > > > > > Define TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY to make sure that the > > > ENDBR are emitted before the patch area. When -mfentry -pg is also used > > > together, there should be no ENDBR before "call __fentry__". > > > > > > OK for master if there is no regression? > > > > > > Thanks. > > > > > > H.J. > > > -- > > > gcc/ > > > > > > PR target/93492 > > > * config/i386/i386.c (ix86_asm_output_function_label): Set > > > function_label_emitted to true. > > > (ix86_print_patchable_function_entry): New function. > > > > > > gcc/testsuite/ > > > > > > PR target/93492 > > > * gcc.target/i386/pr93492-1.c: New test. > > > * gcc.target/i386/pr93492-2.c: Likewise. > > > * gcc.target/i386/pr93492-3.c: Likewise. > > > > > > > This version works with both .cfi_startproc and DWARF debug info. > > > > -g -fpatchable-function-entry=1 doesn't work together: > Here is a differnt approach with UNSPECV_PATCHABLE_AREA. H.J. --- Currently patchable area is at the wrong place. It is placed immediately after function label, before both .cfi_startproc and ENDBR. This patch adds UNSPECV_PATCHABLE_AREA for pseudo patchable area instruction and changes ENDBR insertion pass to also insert a dummy patchable area. TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY is defined to provide the actual size for patchable area. It places patchable area immediately after .cfi_startproc and ENDBR. gcc/ PR target/93492 * config/i386/i386-features.c (rest_of_insert_endbranch): Renamed to ... (rest_of_insert_endbr_and_patchable_area): Change return type to void. Don't call timevar_push nor timevar_pop. Replace endbr_queued_at_entrance with insn_queued_at_entrance. Insert UNSPECV_PATCHABLE_AREA for patchable area. (pass_data_insert_endbranch): Renamed to ... (pass_data_insert_endbr_and_patchable_area): This. Change pass name to endbr_and_patchable_area. (pass_insert_endbranch): Renamed to ... (pass_insert_endbr_and_patchable_area): This. Add need_endbr and need_patchable_area. (pass_insert_endbr_and_patchable_area::gate): Set and check need_endbr/need_patchable_area. (pass_insert_endbr_and_patchable_area::execute): Call timevar_push and timevar_pop. Pass need_endbr amd need_patchable_area to rest_of_insert_endbr_and_patchable_area. (make_pass_insert_endbranch): Renamed to ... (make_pass_insert_endbr_and_patchable_area): This. * config/i386/i386-passes.def: Replace pass_insert_endbranch with pass_insert_endbr_and_patchable_area. * config/i386/i386-protos.h (ix86_output_patchable_area): New. (make_pass_insert_endbranch): Renamed to ... (make_pass_insert_endbr_and_patchable_area): This. * config/i386/i386.c (ix86_asm_output_function_label): Set function_label_emitted to true. (ix86_print_patchable_function_entry): New function. (ix86_output_patchable_area): Likewise. (x86_function_profiler): Replace endbr_queued_at_entrance with insn_queued_at_entrance. Generate ENDBR only for TYPE_ENDBR. Call ix86_output_patchable_area to generate patchable area. (TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY): New. * i386.h (queued_insn_type): New. (machine_function): Add patch_area_size, record_patch_area and function_label_emitted. Replace endbr_queued_at_entrance with insn_queued_at_entrance. * config/i386/i386.md (UNSPECV_PATCHABLE_AREA): New. (patchable_area): New. gcc/testsuite/ PR target/93492 * gcc.target/i386/pr93492-1.c: New test. * gcc.target/i386/pr93492-2.c: Likewise. * gcc.target/i386/pr93492-3.c: Likewise. * gcc.target/i386/pr93492-4.c: Likewise. * gcc.target/i386/pr93492-5.c: Likewise. --- gcc/config/i386/i386-features.c | 139 ++++++++++++++-------- gcc/config/i386/i386-passes.def | 2 +- gcc/config/i386/i386-protos.h | 5 +- gcc/config/i386/i386.c | 90 +++++++++++++- gcc/config/i386/i386.h | 20 +++- gcc/config/i386/i386.md | 14 +++ gcc/testsuite/gcc.target/i386/pr93492-1.c | 73 ++++++++++++ gcc/testsuite/gcc.target/i386/pr93492-2.c | 12 ++ gcc/testsuite/gcc.target/i386/pr93492-3.c | 13 ++ gcc/testsuite/gcc.target/i386/pr93492-4.c | 11 ++ gcc/testsuite/gcc.target/i386/pr93492-5.c | 12 ++ 11 files changed, 337 insertions(+), 54 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/pr93492-1.c create mode 100644 gcc/testsuite/gcc.target/i386/pr93492-2.c create mode 100644 gcc/testsuite/gcc.target/i386/pr93492-3.c create mode 100644 gcc/testsuite/gcc.target/i386/pr93492-4.c create mode 100644 gcc/testsuite/gcc.target/i386/pr93492-5.c diff --git a/gcc/config/i386/i386-features.c b/gcc/config/i386/i386-features.c index b49e6f8d408..479b0d8fdd4 100644 --- a/gcc/config/i386/i386-features.c +++ b/gcc/config/i386/i386-features.c @@ -1937,43 +1937,78 @@ make_pass_stv (gcc::context *ctxt) return new pass_stv (ctxt); } -/* Inserting ENDBRANCH instructions. */ +/* Inserting ENDBR and pseudo patchable-area instructions. */ -static unsigned int -rest_of_insert_endbranch (void) +static void +rest_of_insert_endbr_and_patchable_area (bool need_endbr, + bool need_patchable_area) { - timevar_push (TV_MACH_DEP); - - rtx cet_eb; - rtx_insn *insn; + rtx endbr; + rtx_insn *endbr_insn = NULL, *insn; basic_block bb; - /* Currently emit EB if it's a tracking function, i.e. 'nocf_check' is - absent among function attributes. Later an optimization will be - introduced to make analysis if an address of a static function is - taken. A static function whose address is not taken will get a - nocf_check attribute. This will allow to reduce the number of EB. */ + if (need_endbr) + { + /* Currently emit EB if it's a tracking function, i.e. 'nocf_check' + is absent among function attributes. Later an optimization will + be introduced to make analysis if an address of a static function + is taken. A static function whose address is not taken will get + a nocf_check attribute. This will allow to reduce the number of + EB. */ + if (!lookup_attribute ("nocf_check", + TYPE_ATTRIBUTES (TREE_TYPE (cfun->decl))) + && (!flag_manual_endbr + || lookup_attribute ("cf_check", + DECL_ATTRIBUTES (cfun->decl))) + && !cgraph_node::get (cfun->decl)->only_called_directly_p ()) + { + if (crtl->profile && flag_fentry) + { + /* Queue ENDBR insertion to x86_function_profiler. + NB: Any patchable-area insn will be inserted after + ENDBR. */ + cfun->machine->insn_queued_at_entrance = TYPE_ENDBR; + } + else + { + endbr = gen_nop_endbr (); + bb = ENTRY_BLOCK_PTR_FOR_FN (cfun)->next_bb; + rtx_insn *insn = BB_HEAD (bb); + endbr_insn = emit_insn_before (endbr, insn); + } + } + } - if (!lookup_attribute ("nocf_check", - TYPE_ATTRIBUTES (TREE_TYPE (cfun->decl))) - && (!flag_manual_endbr - || lookup_attribute ("cf_check", - DECL_ATTRIBUTES (cfun->decl))) - && !cgraph_node::get (cfun->decl)->only_called_directly_p ()) + if (need_patchable_area) { - /* Queue ENDBR insertion to x86_function_profiler. */ if (crtl->profile && flag_fentry) - cfun->machine->endbr_queued_at_entrance = true; + { + /* Queue patchable-area insertion to x86_function_profiler. + NB: If there is a queued ENDBR, x86_function_profiler + will also handle patchable-area. */ + if (!cfun->machine->insn_queued_at_entrance) + cfun->machine->insn_queued_at_entrance = TYPE_PATCHABLE_AREA; + } else { - cet_eb = gen_nop_endbr (); - - bb = ENTRY_BLOCK_PTR_FOR_FN (cfun)->next_bb; - insn = BB_HEAD (bb); - emit_insn_before (cet_eb, insn); + /* ix86_print_patchable_function_entry will provide actual + size. */ + rtx patchable_area = gen_patchable_area (GEN_INT (0), + GEN_INT (0)); + if (endbr_insn) + emit_insn_after (patchable_area, endbr_insn); + else + { + bb = ENTRY_BLOCK_PTR_FOR_FN (cfun)->next_bb; + insn = BB_HEAD (bb); + emit_insn_before (patchable_area, insn); + } } } + if (!need_endbr) + return; + bb = 0; FOR_EACH_BB_FN (bb, cfun) { @@ -1982,7 +2017,6 @@ rest_of_insert_endbranch (void) { if (CALL_P (insn)) { - bool need_endbr; need_endbr = find_reg_note (insn, REG_SETJMP, NULL) != NULL; if (!need_endbr && !SIBLING_CALL_P (insn)) { @@ -2013,8 +2047,8 @@ rest_of_insert_endbranch (void) /* Generate ENDBRANCH after CALL, which can return more than twice, setjmp-like functions. */ - cet_eb = gen_nop_endbr (); - emit_insn_after_setloc (cet_eb, insn, INSN_LOCATION (insn)); + endbr = gen_nop_endbr (); + emit_insn_after_setloc (endbr, insn, INSN_LOCATION (insn)); continue; } @@ -2039,36 +2073,33 @@ rest_of_insert_endbranch (void) FOR_EACH_EDGE (e, ei, bb->succs) { - rtx_insn *insn; - dest_blk = e->dest; insn = BB_HEAD (dest_blk); gcc_assert (LABEL_P (insn)); - cet_eb = gen_nop_endbr (); - emit_insn_after (cet_eb, insn); + endbr = gen_nop_endbr (); + emit_insn_after (endbr, insn); } continue; } if (LABEL_P (insn) && LABEL_PRESERVE_P (insn)) { - cet_eb = gen_nop_endbr (); - emit_insn_after (cet_eb, insn); + endbr = gen_nop_endbr (); + emit_insn_after (endbr, insn); continue; } } } - timevar_pop (TV_MACH_DEP); - return 0; + return; } namespace { -const pass_data pass_data_insert_endbranch = +const pass_data pass_data_insert_endbr_and_patchable_area = { RTL_PASS, /* type. */ - "cet", /* name. */ + "endbr_and_patchable_area", /* name. */ OPTGROUP_NONE, /* optinfo_flags. */ TV_MACH_DEP, /* tv_id. */ 0, /* properties_required. */ @@ -2078,32 +2109,44 @@ const pass_data pass_data_insert_endbranch = 0, /* todo_flags_finish. */ }; -class pass_insert_endbranch : public rtl_opt_pass +class pass_insert_endbr_and_patchable_area : public rtl_opt_pass { public: - pass_insert_endbranch (gcc::context *ctxt) - : rtl_opt_pass (pass_data_insert_endbranch, ctxt) + pass_insert_endbr_and_patchable_area (gcc::context *ctxt) + : rtl_opt_pass (pass_data_insert_endbr_and_patchable_area, ctxt) {} /* opt_pass methods: */ - virtual bool gate (function *) - { - return ((flag_cf_protection & CF_BRANCH)); + virtual bool gate (function *fun) + { + need_endbr = (flag_cf_protection & CF_BRANCH) != 0; + need_patchable_area + = (function_entry_patch_area_size + || lookup_attribute ("patchable_function_entry", + DECL_ATTRIBUTES (fun->decl))); + return need_endbr || need_patchable_area; } virtual unsigned int execute (function *) { - return rest_of_insert_endbranch (); + timevar_push (TV_MACH_DEP); + rest_of_insert_endbr_and_patchable_area (need_endbr, + need_patchable_area); + timevar_pop (TV_MACH_DEP); + return 0; } -}; // class pass_insert_endbranch +private: + bool need_endbr; + bool need_patchable_area; +}; // class pass_insert_endbr_and_patchable_area } // anon namespace rtl_opt_pass * -make_pass_insert_endbranch (gcc::context *ctxt) +make_pass_insert_endbr_and_patchable_area (gcc::context *ctxt) { - return new pass_insert_endbranch (ctxt); + return new pass_insert_endbr_and_patchable_area (ctxt); } /* At entry of the nearest common dominator for basic blocks with diff --git a/gcc/config/i386/i386-passes.def b/gcc/config/i386/i386-passes.def index 41386a13d88..d83c7b956b1 100644 --- a/gcc/config/i386/i386-passes.def +++ b/gcc/config/i386/i386-passes.def @@ -30,6 +30,6 @@ along with GCC; see the file COPYING3. If not see CONSTM1_RTX generated by the STV pass can be CSEed. */ INSERT_PASS_BEFORE (pass_cse2, 1, pass_stv, true /* timode_p */); - INSERT_PASS_BEFORE (pass_shorten_branches, 1, pass_insert_endbranch); + INSERT_PASS_BEFORE (pass_shorten_branches, 1, pass_insert_endbr_and_patchable_area); INSERT_PASS_AFTER (pass_combine, 1, pass_remove_partial_avx_dependency); diff --git a/gcc/config/i386/i386-protos.h b/gcc/config/i386/i386-protos.h index 266381ca5a6..cbcc08ebfe8 100644 --- a/gcc/config/i386/i386-protos.h +++ b/gcc/config/i386/i386-protos.h @@ -87,6 +87,8 @@ extern const char *output_fp_compare (rtx_insn *, rtx*, bool, bool); extern const char *output_adjust_stack_and_probe (rtx); extern const char *output_probe_stack_range (rtx, rtx); +extern void ix86_output_patchable_area (unsigned int, bool); + extern void ix86_expand_clear (rtx); extern void ix86_expand_move (machine_mode, rtx[]); extern void ix86_expand_vector_move (machine_mode, rtx[]); @@ -376,6 +378,7 @@ class rtl_opt_pass; extern rtl_opt_pass *make_pass_insert_vzeroupper (gcc::context *); extern rtl_opt_pass *make_pass_stv (gcc::context *); -extern rtl_opt_pass *make_pass_insert_endbranch (gcc::context *); +extern rtl_opt_pass *make_pass_insert_endbr_and_patchable_area + (gcc::context *); extern rtl_opt_pass *make_pass_remove_partial_avx_dependency (gcc::context *); diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c index ffda3e8fd21..071bce8733f 100644 --- a/gcc/config/i386/i386.c +++ b/gcc/config/i386/i386.c @@ -1563,6 +1563,9 @@ ix86_asm_output_function_label (FILE *asm_out_file, const char *fname, { bool is_ms_hook = ix86_function_ms_hook_prologue (decl); + if (cfun) + cfun->machine->function_label_emitted = true; + if (is_ms_hook) { int i, filler_count = (TARGET_64BIT ? 32 : 16); @@ -9118,6 +9121,80 @@ ix86_output_function_epilogue (FILE *file ATTRIBUTE_UNUSED) } } +/* Implement TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY. */ + +void +ix86_print_patchable_function_entry (FILE *file, + unsigned HOST_WIDE_INT patch_area_size, + bool record_p) +{ + if (cfun->machine->function_label_emitted) + { + /* The insert_endbr_and_patchable_area pass inserted a dummy + UNSPECV_PATCHABLE_AREA with 0 patchable area size. If the + patchable area is placed after the function label, we replace + 0 patchable area size with the real one. Otherwise, the + dummy UNSPECV_PATCHABLE_AREA will be ignored. */ + if (cfun->machine->insn_queued_at_entrance) + { + /* Record the patchable area. Both ENDBR and patchable area + will be inserted by x86_function_profiler later. */ + cfun->machine->patch_area_size = patch_area_size; + cfun->machine->record_patch_area = record_p; + return; + } + + /* We can have + + UNSPECV_NOP_ENDBR + UNSPECV_PATCHABLE_AREA + + or just + + UNSPECV_PATCHABLE_AREA + */ + rtx_insn *patchable_insn; + rtx_insn *insn = next_real_nondebug_insn (get_insns ()); + if (insn + && INSN_P (insn) + && GET_CODE (PATTERN (insn)) == UNSPEC_VOLATILE + && XINT (PATTERN (insn), 1) == UNSPECV_NOP_ENDBR) + patchable_insn = next_real_nondebug_insn (insn); + else + patchable_insn = insn; + + if (patchable_insn && INSN_P (patchable_insn)) + { + /* Replace the dummy patchable area size with the real one. */ + rtx pattern = PATTERN (patchable_insn); + if (GET_CODE (pattern) == UNSPEC_VOLATILE + && XINT (pattern, 1) == UNSPECV_PATCHABLE_AREA) + { + XVECEXP (pattern, 0, 0) = GEN_INT (patch_area_size); + XVECEXP (pattern, 0, 1) = GEN_INT (record_p); + } + return; + } + + gcc_unreachable (); + } + + default_print_patchable_function_entry (file, patch_area_size, + record_p); +} + +/* Output patchable area. */ + +void +ix86_output_patchable_area (unsigned int patch_area_size, + bool record_p) +{ + if (patch_area_size) + default_print_patchable_function_entry (asm_out_file, + patch_area_size, + record_p); +} + /* Return a scratch register to use in the split stack prologue. The split stack prologue is used for -fsplit-stack. It is the first instructions in the function, even before the regular prologue. @@ -20151,8 +20228,13 @@ current_fentry_section (const char **name) void x86_function_profiler (FILE *file, int labelno ATTRIBUTE_UNUSED) { - if (cfun->machine->endbr_queued_at_entrance) - fprintf (file, "\t%s\n", TARGET_64BIT ? "endbr64" : "endbr32"); + if (cfun->machine->insn_queued_at_entrance) + { + if (cfun->machine->insn_queued_at_entrance == TYPE_ENDBR) + fprintf (file, "\t%s\n", TARGET_64BIT ? "endbr64" : "endbr32"); + ix86_output_patchable_area (cfun->machine->patch_area_size, + cfun->machine->record_patch_area); + } const char *mcount_name = MCOUNT_NAME; @@ -22744,6 +22826,10 @@ ix86_run_selftests (void) #undef TARGET_ASM_FUNCTION_EPILOGUE #define TARGET_ASM_FUNCTION_EPILOGUE ix86_output_function_epilogue +#undef TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY +#define TARGET_ASM_PRINT_PATCHABLE_FUNCTION_ENTRY \ + ix86_print_patchable_function_entry + #undef TARGET_ENCODE_SECTION_INFO #ifndef SUBTARGET_ENCODE_SECTION_INFO #define TARGET_ENCODE_SECTION_INFO ix86_encode_section_info diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h index 943e9a5c783..7c8f0cd4c70 100644 --- a/gcc/config/i386/i386.h +++ b/gcc/config/i386/i386.h @@ -2751,6 +2751,13 @@ enum function_type TYPE_EXCEPTION }; +enum queued_insn_type +{ + TYPE_NONE = 0, + TYPE_ENDBR, + TYPE_PATCHABLE_AREA +}; + struct GTY(()) machine_function { struct stack_local_entry *stack_locals; int varargs_gpr_size; @@ -2767,6 +2774,12 @@ struct GTY(()) machine_function { structure. */ rtx split_stack_varargs_pointer; + /* The size of the patchable area at function entry. */ + unsigned int patch_area_size; + + /* If true, record patchable area at function entry. */ + BOOL_BITFIELD record_patch_area : 1; + /* This value is used for amd64 targets and specifies the current abi to be used. MS_ABI means ms abi. Otherwise SYSV_ABI means sysv abi. */ ENUM_BITFIELD(calling_abi) call_abi : 8; @@ -2841,8 +2854,11 @@ struct GTY(()) machine_function { /* Nonzero if the function places outgoing arguments on stack. */ BOOL_BITFIELD outgoing_args_on_stack : 1; - /* If true, ENDBR is queued at function entrance. */ - BOOL_BITFIELD endbr_queued_at_entrance : 1; + /* If true, ENDBR or patchable area is queued at function entrance. */ + ENUM_BITFIELD(queued_insn_type) insn_queued_at_entrance : 2; + + /* If true, the function label has been emitted. */ + BOOL_BITFIELD function_label_emitted : 1; /* True if the function needs a stack frame. */ BOOL_BITFIELD stack_frame_required : 1; diff --git a/gcc/config/i386/i386.md b/gcc/config/i386/i386.md index 46b442dae51..75bf8fd2348 100644 --- a/gcc/config/i386/i386.md +++ b/gcc/config/i386/i386.md @@ -300,6 +300,9 @@ (define_c_enum "unspecv" [ ;; For ENQCMD and ENQCMDS support UNSPECV_ENQCMD UNSPECV_ENQCMDS + + ;; For patchable area support + UNSPECV_PATCHABLE_AREA ]) ;; Constants to represent rounding modes in the ROUND instruction @@ -21280,6 +21283,17 @@ (define_insn "speculation_barrier" [(set_attr "type" "other") (set_attr "length" "3")]) +(define_insn "patchable_area" + [(unspec_volatile [(match_operand 0 "const_int_operand") + (match_operand 1 "const_int_operand")] + UNSPECV_PATCHABLE_AREA)] + "" +{ + ix86_output_patchable_area (INTVAL (operands[0]), + INTVAL (operands[1]) != 0); + return ""; +}) + (include "mmx.md") (include "sse.md") (include "sync.md") diff --git a/gcc/testsuite/gcc.target/i386/pr93492-1.c b/gcc/testsuite/gcc.target/i386/pr93492-1.c new file mode 100644 index 00000000000..f978d2e5220 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr93492-1.c @@ -0,0 +1,73 @@ +/* { dg-do "compile" } */ +/* { dg-options "-O1 -fcf-protection -mmanual-endbr" } */ +/* { dg-final { check-function-bodies "**" "" } } */ + +/* Note: this test only checks the instructions in the function bodies, + not the placement of the patch label or nops before the function. */ + +/* +**f10_none: +** nop +** ret +*/ +void +__attribute__ ((nocf_check,patchable_function_entry (1, 0))) +f10_none (void) +{ +} + +/* +**f10_endbr: +** endbr(32|64) +** nop +** ret +*/ +void +__attribute__ ((cf_check,patchable_function_entry (1, 0))) +f10_endbr (void) +{ +} + +/* +**f11_none: +** ret +*/ +void +__attribute__ ((nocf_check,patchable_function_entry (1, 1))) +f11_none (void) +{ +} + +/* +**f11_endbr: +** endbr(32|64) +** ret +*/ +void +__attribute__ ((cf_check,patchable_function_entry (1, 1))) +f11_endbr (void) +{ +} + +/* +**f21_none: +** nop +** ret +*/ +void +__attribute__ ((nocf_check,patchable_function_entry (2, 1))) +f21_none (void) +{ +} + +/* +**f21_endbr: +** endbr(32|64) +** nop +** ret +*/ +void +__attribute__ ((cf_check,patchable_function_entry (2, 1))) +f21_endbr (void) +{ +} diff --git a/gcc/testsuite/gcc.target/i386/pr93492-2.c b/gcc/testsuite/gcc.target/i386/pr93492-2.c new file mode 100644 index 00000000000..ec26d4cc367 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr93492-2.c @@ -0,0 +1,12 @@ +/* { dg-do "compile" } */ +/* { dg-options "-O1 -fcf-protection -mmanual-endbr -fasynchronous-unwind-tables" } */ + +/* Test the placement of the .LPFE1 label. */ + +void +__attribute__ ((cf_check,patchable_function_entry (1, 0))) +f10_endbr (void) +{ +} + +/* { dg-final { scan-assembler "\t\.cfi_startproc\n\tendbr(32|64)\n.*\.LPFE1:\n\tnop\n\tret\n" } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr93492-3.c b/gcc/testsuite/gcc.target/i386/pr93492-3.c new file mode 100644 index 00000000000..1f03c627120 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr93492-3.c @@ -0,0 +1,13 @@ +/* { dg-do "compile" } */ +/* { dg-require-effective-target mfentry } */ +/* { dg-options "-O1 -fcf-protection -mmanual-endbr -mfentry -pg -fasynchronous-unwind-tables" } */ + +/* Test the placement of the .LPFE1 label. */ + +void +__attribute__ ((cf_check,patchable_function_entry (1, 0))) +f10_endbr (void) +{ +} + +/* { dg-final { scan-assembler "\t\.cfi_startproc\n\tendbr(32|64)\n.*\.LPFE1:\n\tnop\n1:\tcall\t__fentry__\n\tret\n" } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr93492-4.c b/gcc/testsuite/gcc.target/i386/pr93492-4.c new file mode 100644 index 00000000000..d193df8e66d --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr93492-4.c @@ -0,0 +1,11 @@ +/* { dg-do "compile" } */ +/* { dg-options "-O1 -fpatchable-function-entry=1 -fasynchronous-unwind-tables" } */ + +/* Test the placement of the .LPFE1 label. */ + +void +foo (void) +{ +} + +/* { dg-final { scan-assembler "\t\.cfi_startproc\n.*\.LPFE1:\n\tnop\n\tret\n" } } */ diff --git a/gcc/testsuite/gcc.target/i386/pr93492-5.c b/gcc/testsuite/gcc.target/i386/pr93492-5.c new file mode 100644 index 00000000000..d04077c6007 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr93492-5.c @@ -0,0 +1,12 @@ +/* { dg-do "compile" } */ +/* { dg-require-effective-target mfentry } */ +/* { dg-options "-O1 -fpatchable-function-entry=1 -mfentry -pg -fasynchronous-unwind-tables" } */ + +/* Test the placement of the .LPFE1 label. */ + +void +foo (void) +{ +} + +/* { dg-final { scan-assembler "\t\.cfi_startproc\n.*\.LPFE1:\n\tnop\n1:\tcall\t__fentry__\n\tret\n" } } */