From patchwork Wed Nov 17 20:01:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "H.J. Lu" X-Patchwork-Id: 1556368 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=bIGg8EMz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HvYkh56Gpz9sPf for ; Thu, 18 Nov 2021 07:02:52 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9B8503858424 for ; Wed, 17 Nov 2021 20:02:50 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9B8503858424 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1637179370; bh=yIjEOgwaOn7gQxiFK4ZliZk6d7T4wbQASxIIG324EzY=; h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=bIGg8EMzX0Il7XLqvmrXjU9Xm7yToBbkQ0dZ6iEDy21wX5dBmOGhoyjlp4V6nhRQt pzrlkVdBtBmH+s7PgePDIo9A2HEVm+vMu5LRiZj2jJdXgUgTF7T0edKwmmLv+Qym9W FYJyom7lncv12Nx0A3t9o8dCTvbRXl6mzoEgECP8= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by sourceware.org (Postfix) with ESMTPS id CAA0B3858409 for ; Wed, 17 Nov 2021 20:02:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org CAA0B3858409 Received: by mail-pg1-x52a.google.com with SMTP id p17so3238417pgj.2 for ; Wed, 17 Nov 2021 12:02:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yIjEOgwaOn7gQxiFK4ZliZk6d7T4wbQASxIIG324EzY=; b=uQXzTOIxmTIqFUvSUqLJzUV3Wp0C8a3F08P7Tf9/b0o2eiqSmKcck/EBXcxcKq4zBf inC2EmsbxnB7qV5p5YfGVUKArQeYZscUIzyr8ONfEMB0QKf0AV0jNpd1AKtzdPRmRfNt G0GiZtzadxHTpWlYz2bA00E7lQ7/VkdXKBOkYcZcbNmlZ7lOAlVDK67Em/VOQcwJGvSZ VdmAmrfYBHWOuglXMiHph7gQhmaxSL1Q6i07uZfvcsRPUObPEysqpVCyljF37gA/hnVj W9l4JBIcnYCoT0uGYO2Sbx0/sbgYhPu5sL1/H3YTsmptbC9CRGo/7MtaQodDkXis8Thq 1SHA== X-Gm-Message-State: AOAM532TgD5AogRm5MqnShy7doVGT32xsJ8XVB8kGmSnI4gMu08YD14K AmSme+HeT3qWb+MmcfOgvww9t8WTacmZDNvDMfs= X-Google-Smtp-Source: ABdhPJyQxPEtJa/2h+mmf7jZ/CwWKaDxxUgFvwmjVOq/4+T4RmnT6OxuOWl4ASeRitxe6ucsM1RdZ7NuQxoZfuFlKgk= X-Received: by 2002:a05:6a00:8cd:b0:4a2:82d7:1703 with SMTP id s13-20020a056a0008cd00b004a282d71703mr9787957pfu.43.1637179347375; Wed, 17 Nov 2021 12:02:27 -0800 (PST) MIME-Version: 1.0 References: <20211117153522.230700-1-hjl.tools@gmail.com> In-Reply-To: Date: Wed, 17 Nov 2021 12:01:51 -0800 Message-ID: Subject: [PATCH v3] x86: Add -mharden-sls=[none|all|return|indirect-branch] To: Uros Bizjak X-Spam-Status: No, score=-3029.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: "H.J. Lu via Gcc-patches" From: "H.J. Lu" Reply-To: "H.J. Lu" Cc: "gcc-patches@gcc.gnu.org" Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" On Wed, Nov 17, 2021 at 7:53 AM Uros Bizjak wrote: > > On Wed, Nov 17, 2021 at 4:35 PM H.J. Lu wrote: > > > > Add -mharden-sls= to mitigate against straight line speculation (SLS) > > for function return and indirect branch by adding an INT3 instruction > > after function return and indirect branch. > > > > gcc/ > > > > PR target/102952 > > * config/i386/i386-opts.h (harden_sls): New enum. > > * config/i386/i386.c (output_indirect_thunk): Mitigate against > > SLS for function return. > > (ix86_output_function_return): Likewise. > > (ix86_output_jmp_thunk_or_indirect): Mitigate against indirect > > branch. > > (ix86_output_indirect_jmp): Likewise. > > (ix86_output_call_insn): Likewise. > > * config/i386/i386.opt: Add -mharden-sls=. > > * doc/invoke.texi: Document -mharden-sls=. > > > > gcc/testsuite/ > > > > PR target/102952 > > * gcc.target/i386/harden-sls-1.c: New test. > > * gcc.target/i386/harden-sls-2.c: Likewise. > > * gcc.target/i386/harden-sls-3.c: Likewise. > > * gcc.target/i386/harden-sls-4.c: Likewise. > > * gcc.target/i386/harden-sls-5.c: Likewise. > > --- > > gcc/config/i386/i386-opts.h | 7 ++++++ > > gcc/config/i386/i386.c | 23 ++++++++++++++------ > > gcc/config/i386/i386.opt | 20 +++++++++++++++++ > > gcc/doc/invoke.texi | 10 ++++++++- > > gcc/testsuite/gcc.target/i386/harden-sls-1.c | 14 ++++++++++++ > > gcc/testsuite/gcc.target/i386/harden-sls-2.c | 14 ++++++++++++ > > gcc/testsuite/gcc.target/i386/harden-sls-3.c | 14 ++++++++++++ > > gcc/testsuite/gcc.target/i386/harden-sls-4.c | 16 ++++++++++++++ > > gcc/testsuite/gcc.target/i386/harden-sls-5.c | 17 +++++++++++++++ > > 9 files changed, 127 insertions(+), 8 deletions(-) > > create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-1.c > > create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-2.c > > create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-3.c > > create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-4.c > > create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-5.c > > > > diff --git a/gcc/config/i386/i386-opts.h b/gcc/config/i386/i386-opts.h > > index 04e4ad608fb..171d3106d0a 100644 > > --- a/gcc/config/i386/i386-opts.h > > +++ b/gcc/config/i386/i386-opts.h > > @@ -121,4 +121,11 @@ enum instrument_return { > > instrument_return_nop5 > > }; > > > > +enum harden_sls { > > + harden_sls_none = 0, > > + harden_sls_return = 1 << 0, > > + harden_sls_indirect_branch = 1 << 1, > > + harden_sls_all = harden_sls_return | harden_sls_indirect_branch > > +}; > > + > > #endif > > diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c > > index 73c4d5115bb..8bbf6ae9875 100644 > > --- a/gcc/config/i386/i386.c > > +++ b/gcc/config/i386/i386.c > > @@ -5914,6 +5914,8 @@ output_indirect_thunk (unsigned int regno) > > } > > > > fputs ("\tret\n", asm_out_file); > > + if ((ix86_harden_sls & harden_sls_return)) > > + fputs ("\tint3\n", asm_out_file); > > } > > > > /* Output a funtion with a call and return thunk for indirect branch. > > @@ -15984,6 +15986,8 @@ ix86_output_jmp_thunk_or_indirect (const char *thunk_name, const int regno) > > fprintf (asm_out_file, "\tjmp\t"); > > assemble_name (asm_out_file, thunk_name); > > putc ('\n', asm_out_file); > > + if ((ix86_harden_sls & harden_sls_indirect_branch)) > > + fputs ("\tint3\n", asm_out_file); > > } > > else > > output_indirect_thunk (regno); > > @@ -16206,10 +16210,10 @@ ix86_output_indirect_jmp (rtx call_op) > > gcc_unreachable (); > > > > ix86_output_indirect_branch (call_op, "%0", true); > > - return ""; > > } > > else > > - return "%!jmp\t%A0"; > > + output_asm_insn ("%!jmp\t%A0", &call_op); > > + return (ix86_harden_sls & harden_sls_indirect_branch) ? "int3" : ""; > > } > > > > /* Output return instrumentation for current function if needed. */ > > @@ -16277,10 +16281,10 @@ ix86_output_function_return (bool long_p) > > return ""; > > } > > > > - if (!long_p) > > - return "%!ret"; > > - > > - return "rep%; ret"; > > + if ((ix86_harden_sls & harden_sls_return)) > > + long_p = false; > > Is the above really needed? This will change "rep ret" to a "[notrack] > ret" when SLS hardening is in effect, with a conditional [notrack] > prefix, even when long ret was requested. Fixed in the v3 patch. > On a related note, "notrack ret" does not assemble for me, the > assembler reports: > > notrack.s:1: Error: expecting indirect branch instruction after `notrack' > > Can you please clarify the above change? I opened: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=103307 Here is the v3 patch. From ed5e4a06b0488bff1fcdf218d93b54e0abf7ff3b Mon Sep 17 00:00:00 2001 From: "H.J. Lu" Date: Wed, 27 Oct 2021 07:48:54 -0700 Subject: [PATCH v3] x86: Add -mharden-sls=[none|all|return|indirect-branch] Add -mharden-sls= to mitigate against straight line speculation (SLS) for function return and indirect branch by adding an INT3 instruction after function return and indirect branch. gcc/ PR target/102952 * config/i386/i386-opts.h (harden_sls): New enum. * config/i386/i386.c (output_indirect_thunk): Mitigate against SLS for function return. (ix86_output_function_return): Likewise. (ix86_output_jmp_thunk_or_indirect): Mitigate against indirect branch. (ix86_output_indirect_jmp): Likewise. (ix86_output_call_insn): Likewise. * config/i386/i386.opt: Add -mharden-sls=. * doc/invoke.texi: Document -mharden-sls=. gcc/testsuite/ PR target/102952 * gcc.target/i386/harden-sls-1.c: New test. * gcc.target/i386/harden-sls-2.c: Likewise. * gcc.target/i386/harden-sls-3.c: Likewise. * gcc.target/i386/harden-sls-4.c: Likewise. * gcc.target/i386/harden-sls-5.c: Likewise. --- gcc/config/i386/i386-opts.h | 7 +++++++ gcc/config/i386/i386.c | 21 +++++++++++++------- gcc/config/i386/i386.opt | 20 +++++++++++++++++++ gcc/doc/invoke.texi | 10 +++++++++- gcc/testsuite/gcc.target/i386/harden-sls-1.c | 14 +++++++++++++ gcc/testsuite/gcc.target/i386/harden-sls-2.c | 14 +++++++++++++ gcc/testsuite/gcc.target/i386/harden-sls-3.c | 14 +++++++++++++ gcc/testsuite/gcc.target/i386/harden-sls-4.c | 16 +++++++++++++++ gcc/testsuite/gcc.target/i386/harden-sls-5.c | 17 ++++++++++++++++ 9 files changed, 125 insertions(+), 8 deletions(-) create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-1.c create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-2.c create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-3.c create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-4.c create mode 100644 gcc/testsuite/gcc.target/i386/harden-sls-5.c diff --git a/gcc/config/i386/i386-opts.h b/gcc/config/i386/i386-opts.h index 04e4ad608fb..171d3106d0a 100644 --- a/gcc/config/i386/i386-opts.h +++ b/gcc/config/i386/i386-opts.h @@ -121,4 +121,11 @@ enum instrument_return { instrument_return_nop5 }; +enum harden_sls { + harden_sls_none = 0, + harden_sls_return = 1 << 0, + harden_sls_indirect_branch = 1 << 1, + harden_sls_all = harden_sls_return | harden_sls_indirect_branch +}; + #endif diff --git a/gcc/config/i386/i386.c b/gcc/config/i386/i386.c index 73c4d5115bb..633cac2451d 100644 --- a/gcc/config/i386/i386.c +++ b/gcc/config/i386/i386.c @@ -5914,6 +5914,8 @@ output_indirect_thunk (unsigned int regno) } fputs ("\tret\n", asm_out_file); + if ((ix86_harden_sls & harden_sls_return)) + fputs ("\tint3\n", asm_out_file); } /* Output a funtion with a call and return thunk for indirect branch. @@ -15984,6 +15986,8 @@ ix86_output_jmp_thunk_or_indirect (const char *thunk_name, const int regno) fprintf (asm_out_file, "\tjmp\t"); assemble_name (asm_out_file, thunk_name); putc ('\n', asm_out_file); + if ((ix86_harden_sls & harden_sls_indirect_branch)) + fputs ("\tint3\n", asm_out_file); } else output_indirect_thunk (regno); @@ -16206,10 +16210,10 @@ ix86_output_indirect_jmp (rtx call_op) gcc_unreachable (); ix86_output_indirect_branch (call_op, "%0", true); - return ""; } else - return "%!jmp\t%A0"; + output_asm_insn ("%!jmp\t%A0", &call_op); + return (ix86_harden_sls & harden_sls_indirect_branch) ? "int3" : ""; } /* Output return instrumentation for current function if needed. */ @@ -16277,10 +16281,8 @@ ix86_output_function_return (bool long_p) return ""; } - if (!long_p) - return "%!ret"; - - return "rep%; ret"; + output_asm_insn (long_p ? "rep%; ret" : "ret", nullptr); + return (ix86_harden_sls & harden_sls_return) ? "int3" : ""; } /* Output indirect function return. RET_OP is the function return @@ -16375,7 +16377,12 @@ ix86_output_call_insn (rtx_insn *insn, rtx call_op) if (output_indirect_p && !direct_p) ix86_output_indirect_branch (call_op, xasm, true); else - output_asm_insn (xasm, &call_op); + { + output_asm_insn (xasm, &call_op); + if (!direct_p + && (ix86_harden_sls & harden_sls_indirect_branch)) + return "int3"; + } return ""; } diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index 46fad3cc038..8d499a5a4df 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -1117,6 +1117,26 @@ mrecord-return Target Var(ix86_flag_record_return) Init(0) Generate a __return_loc section pointing to all return instrumentation code. +mharden-sls= +Target RejectNegative Joined Enum(harden_sls) Var(ix86_harden_sls) Init(harden_sls_none) +Generate code to mitigate against straight line speculation. + +Enum +Name(harden_sls) Type(enum harden_sls) +Known choices for mitigation against straight line speculation with -mharden-sls=: + +EnumValue +Enum(harden_sls) String(none) Value(harden_sls_none) + +EnumValue +Enum(harden_sls) String(all) Value(harden_sls_all) + +EnumValue +Enum(harden_sls) String(return) Value(harden_sls_return) + +EnumValue +Enum(harden_sls) String(indirect-branch) Value(harden_sls_indirect_branch) + mavx512bf16 Target Mask(ISA2_AVX512BF16) Var(ix86_isa_flags2) Save Support MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX512F and diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index a22758d18ee..0265c160e02 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -1427,7 +1427,7 @@ See RS/6000 and PowerPC Options. -mstack-protector-guard-symbol=@var{symbol} @gol -mgeneral-regs-only -mcall-ms2sysv-xlogues -mrelax-cmpxchg-loop @gol -mindirect-branch=@var{choice} -mfunction-return=@var{choice} @gol --mindirect-branch-register -mneeded} +-mindirect-branch-register -mharden-sls=@var{choice} -mneeded} @emph{x86 Windows Options} @gccoptlist{-mconsole -mcygwin -mno-cygwin -mdll @gol @@ -32401,6 +32401,14 @@ not be reachable in the large code model. @opindex mindirect-branch-register Force indirect call and jump via register. +@item -mharden-sls=@var{choice} +@opindex mharden-sls +Generate code to mitigate against straight line speculation (SLS) with +@var{choice}. The default is @samp{none} which disables all SLS +hardening. @samp{return} enables SLS hardening for function return. +@samp{indirect-branch} enables SLS hardening for indirect branch. +@samp{all} enables all SLS hardening. + @end table These @samp{-m} switches are supported in addition to the above diff --git a/gcc/testsuite/gcc.target/i386/harden-sls-1.c b/gcc/testsuite/gcc.target/i386/harden-sls-1.c new file mode 100644 index 00000000000..6f70dc94a23 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/harden-sls-1.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mindirect-branch=thunk-extern -mharden-sls=all" } */ +/* { dg-additional-options "-fno-pic" { target { ! *-*-darwin* } } } */ + +extern void foo (void); + +void +bar (void) +{ + foo (); +} + +/* { dg-final { scan-assembler "jmp\[ \t\]+_?foo" } } */ +/* { dg-final { scan-assembler-not {int3} } } */ diff --git a/gcc/testsuite/gcc.target/i386/harden-sls-2.c b/gcc/testsuite/gcc.target/i386/harden-sls-2.c new file mode 100644 index 00000000000..a7c59078d03 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/harden-sls-2.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mindirect-branch=thunk-extern -mharden-sls=all" } */ +/* { dg-additional-options "-fno-pic" { target { ! *-*-darwin* } } } */ + +extern void (*fptr) (void); + +void +foo (void) +{ + fptr (); +} + +/* { dg-final { scan-assembler "jmp\[ \t\]+_?__x86_indirect_thunk_(r|e)ax" } } */ +/* { dg-final { scan-assembler-times "int3" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/harden-sls-3.c b/gcc/testsuite/gcc.target/i386/harden-sls-3.c new file mode 100644 index 00000000000..1a6056b6d7b --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/harden-sls-3.c @@ -0,0 +1,14 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mindirect-branch=thunk -mharden-sls=all" } */ +/* { dg-additional-options "-fno-pic" { target { ! *-*-darwin* } } } */ + +extern void (*fptr) (void); + +void +foo (void) +{ + fptr (); +} + +/* { dg-final { scan-assembler "jmp\[ \t\]+_?__x86_indirect_thunk_(r|e)ax" } } */ +/* { dg-final { scan-assembler-times "int3" 2 } } */ diff --git a/gcc/testsuite/gcc.target/i386/harden-sls-4.c b/gcc/testsuite/gcc.target/i386/harden-sls-4.c new file mode 100644 index 00000000000..f70dd1379d3 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/harden-sls-4.c @@ -0,0 +1,16 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mindirect-branch=keep -mharden-sls=all" } */ +/* { dg-additional-options "-fno-pic" { target { ! *-*-darwin* } } } */ + +extern void (*fptr) (void); + +void +foo (void) +{ + fptr (); +} + +/* { dg-final { scan-assembler "jmp\[ \t\]+\\*_?fptr" { target { ! x32 } } } } */ +/* { dg-final { scan-assembler "movl\[ \t\]+fptr\\(%rip\\), %eax" { target x32 } } } */ +/* { dg-final { scan-assembler "jmp\[ \t\]+\\*%rax" { target x32 } } } */ +/* { dg-final { scan-assembler-times "int3" 1 } } */ diff --git a/gcc/testsuite/gcc.target/i386/harden-sls-5.c b/gcc/testsuite/gcc.target/i386/harden-sls-5.c new file mode 100644 index 00000000000..613c44c6f82 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/harden-sls-5.c @@ -0,0 +1,17 @@ +/* { dg-do compile } */ +/* { dg-options "-O2 -mno-indirect-branch-register -mfunction-return=keep -mindirect-branch=thunk-extern -mharden-sls=return" } */ +/* { dg-additional-options "-fno-pic" { target { ! *-*-darwin* } } } */ + +typedef void (*dispatch_t)(long offset); + +dispatch_t dispatch; + +int +male_indirect_jump (long offset) +{ + dispatch(offset); + return 0; +} + +/* { dg-final { scan-assembler-times "ret" 1 } } */ +/* { dg-final { scan-assembler-times "int3" 1 } } */ -- 2.33.1