[v2,1/2] meson: mitigate against ROP exploits with -fzero-call-used-regs

To quote wikipedia:

  "Return-oriented programming (ROP) is a computer security exploit
   technique that allows an attacker to execute code in the presence
   of security defenses such as executable space protection and code
   signing.

   In this technique, an attacker gains control of the call stack to
   hijack program control flow and then executes carefully chosen
   machine instruction sequences that are already present in the
   machine's memory, called "gadgets". Each gadget typically ends in
   a return instruction and is located in a subroutine within the
   existing program and/or shared library code. Chained together,
   these gadgets allow an attacker to perform arbitrary operations
   on a machine employing defenses that thwart simpler attacks."

QEMU is by no means perfect with an ever growing set of CVEs from
flawed hardware device emulation, which could potentially be
exploited using ROP techniques.

Since GCC 11 there has been a compiler option that can mitigate
against this exploit technique:

    -fzero-call-user-regs

To understand it refer to these two resources:

   https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/
   https://gcc.gnu.org/pipermail/gcc-patches/2020-August/552262.html

I used two programs to scan qemu-system-x86_64 for ROP gadgets:

  https://github.com/0vercl0k/rp
  https://github.com/JonathanSalwan/ROPgadget

When asked to find 8 byte gadgets, the 'rp' tool reports:

  A total of 440278 gadgets found.
  You decided to keep only the unique ones, 156143 unique gadgets found.

While the ROPgadget tool reports:

  Unique gadgets found: 353122

With the --ropchain argument, the latter attempts to use the found
gadgets to product a chain that can execute arbitrary syscalls. With
current QEMU it succeeds in this task, which is an undesirable
situation.

With QEMU modified to use -fzero-call-user-regs=used-gpr the 'rp' tool
reports

  A total of 528991 gadgets found.
  You decided to keep only the unique ones, 121128 unique gadgets found.

This is 22% fewer unique gadgets

While the ROPgadget tool reports:

  Unique gadgets found: 328605

This is 7% fewer unique gadgets. Crucially though, despite this more
modest reduction, the ROPgadget tool is no longer able to identify a
chain of gadgets for executing arbitrary syscalls. It fails at the
very first step, unable to find gadgets for populating registers for
a future syscall. Having said that, more advanced tools do still
manage to put together a viable ROP chain.

Also this only takes into account QEMU code. QEMU links to many 3rd
party shared libraries and ideally all of them would be compiled with
this same hardening. That becomes a distro policy question though.

In terms of performance impact, TCG was used as an evaluation test
case. We're not interested in protecting TCG since it isn't designed
to provide a security barrier, but it is performance sensitive code,
so useful as a guide to how other areas of QEMU might be impacted.
With the -fzero-call-user-regs=used-gpr argument present, using the
real world test of booting a linux kernel and having init immediately
poweroff, there is a ~1% slow down in performance under TCG. The QEMU
binary size also grows by approximately 1%.

By comparison, using the more aggressive -fzero-call-user-regs=all,
results in a slowdown of over 25% in TCG, which is clearly not an
acceptable impact, and a binary size increase of 5%.

Considering that 'used-gpr' succesfully stopped ROPgadget assembling
a chain, this more targetted protection is a justifiable hardening
/ performance tradeoff.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 meson.build | 11 +++++++++++
 1 file changed, 11 insertions(+)

Daniel P. Berrangé <berrange@redhat.com> writes:

> To quote wikipedia:
>
>   "Return-oriented programming (ROP) is a computer security exploit
>    technique that allows an attacker to execute code in the presence
>    of security defenses such as executable space protection and code
>    signing.
>
>    In this technique, an attacker gains control of the call stack to
>    hijack program control flow and then executes carefully chosen
>    machine instruction sequences that are already present in the
>    machine's memory, called "gadgets". Each gadget typically ends in
>    a return instruction and is located in a subroutine within the
>    existing program and/or shared library code. Chained together,
>    these gadgets allow an attacker to perform arbitrary operations
>    on a machine employing defenses that thwart simpler attacks."
>
> QEMU is by no means perfect with an ever growing set of CVEs from
> flawed hardware device emulation, which could potentially be
> exploited using ROP techniques.
>
> Since GCC 11 there has been a compiler option that can mitigate
> against this exploit technique:
>
>     -fzero-call-user-regs
>
> To understand it refer to these two resources:
>
>    https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/
>    https://gcc.gnu.org/pipermail/gcc-patches/2020-August/552262.html
>
> I used two programs to scan qemu-system-x86_64 for ROP gadgets:
>
>   https://github.com/0vercl0k/rp
>   https://github.com/JonathanSalwan/ROPgadget
>
> When asked to find 8 byte gadgets, the 'rp' tool reports:
>
>   A total of 440278 gadgets found.
>   You decided to keep only the unique ones, 156143 unique gadgets found.
>
> While the ROPgadget tool reports:
>
>   Unique gadgets found: 353122
>
> With the --ropchain argument, the latter attempts to use the found
> gadgets to product a chain that can execute arbitrary syscalls. With
> current QEMU it succeeds in this task, which is an undesirable
> situation.
>
> With QEMU modified to use -fzero-call-user-regs=used-gpr the 'rp' tool
> reports
>
>   A total of 528991 gadgets found.
>   You decided to keep only the unique ones, 121128 unique gadgets found.
>
> This is 22% fewer unique gadgets
>
> While the ROPgadget tool reports:
>
>   Unique gadgets found: 328605
>
> This is 7% fewer unique gadgets. Crucially though, despite this more
> modest reduction, the ROPgadget tool is no longer able to identify a
> chain of gadgets for executing arbitrary syscalls. It fails at the
> very first step, unable to find gadgets for populating registers for
> a future syscall. Having said that, more advanced tools do still
> manage to put together a viable ROP chain.
>
> Also this only takes into account QEMU code. QEMU links to many 3rd
> party shared libraries and ideally all of them would be compiled with
> this same hardening. That becomes a distro policy question though.
>
> In terms of performance impact, TCG was used as an evaluation test
> case. We're not interested in protecting TCG since it isn't designed
> to provide a security barrier, but it is performance sensitive code,
> so useful as a guide to how other areas of QEMU might be impacted.
> With the -fzero-call-user-regs=used-gpr argument present, using the
> real world test of booting a linux kernel and having init immediately
> poweroff, there is a ~1% slow down in performance under TCG. The QEMU
> binary size also grows by approximately 1%.
>
> By comparison, using the more aggressive -fzero-call-user-regs=all,
> results in a slowdown of over 25% in TCG, which is clearly not an
> acceptable impact, and a binary size increase of 5%.
>
> Considering that 'used-gpr' succesfully stopped ROPgadget assembling
> a chain, this more targetted protection is a justifiable hardening
> / performance tradeoff.

Have you also considered 'used-arg'?

> Reviewed-by: Thomas Huth <thuth@redhat.com>
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> ---
>  meson.build | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/meson.build b/meson.build
> index 6c77d9687d..eaa20d241d 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -433,6 +433,17 @@ if get_option('fuzzing')
>    endif
>  endif
>  
> +# Check further flags that make QEMU more robust against malicious parties
> +
> +hardening_flags = [
> +    # Zero out registers used during a function call
> +    # upon its return. This makes it harder to assemble
> +    # ROP gadgets into something usable
> +    '-fzero-call-used-regs=used-gpr',
> +]
> +
> +qemu_common_flags += cc.get_supported_arguments(hardening_flags)
> +
>  add_global_arguments(qemu_common_flags, native: false, language: all_languages)
>  add_global_link_arguments(qemu_ldflags, native: false, language: all_languages)

On Tue, Jan 09, 2024 at 03:54:07PM +0100, Markus Armbruster wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > To quote wikipedia:
> >
> >   "Return-oriented programming (ROP) is a computer security exploit
> >    technique that allows an attacker to execute code in the presence
> >    of security defenses such as executable space protection and code
> >    signing.
> >
> >    In this technique, an attacker gains control of the call stack to
> >    hijack program control flow and then executes carefully chosen
> >    machine instruction sequences that are already present in the
> >    machine's memory, called "gadgets". Each gadget typically ends in
> >    a return instruction and is located in a subroutine within the
> >    existing program and/or shared library code. Chained together,
> >    these gadgets allow an attacker to perform arbitrary operations
> >    on a machine employing defenses that thwart simpler attacks."
> >
> > QEMU is by no means perfect with an ever growing set of CVEs from
> > flawed hardware device emulation, which could potentially be
> > exploited using ROP techniques.
> >
> > Since GCC 11 there has been a compiler option that can mitigate
> > against this exploit technique:
> >
> >     -fzero-call-user-regs
> >
> > To understand it refer to these two resources:
> >
> >    https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/
> >    https://gcc.gnu.org/pipermail/gcc-patches/2020-August/552262.html
> >
> > I used two programs to scan qemu-system-x86_64 for ROP gadgets:
> >
> >   https://github.com/0vercl0k/rp
> >   https://github.com/JonathanSalwan/ROPgadget
> >
> > When asked to find 8 byte gadgets, the 'rp' tool reports:
> >
> >   A total of 440278 gadgets found.
> >   You decided to keep only the unique ones, 156143 unique gadgets found.
> >
> > While the ROPgadget tool reports:
> >
> >   Unique gadgets found: 353122
> >
> > With the --ropchain argument, the latter attempts to use the found
> > gadgets to product a chain that can execute arbitrary syscalls. With
> > current QEMU it succeeds in this task, which is an undesirable
> > situation.
> >
> > With QEMU modified to use -fzero-call-user-regs=used-gpr the 'rp' tool
> > reports
> >
> >   A total of 528991 gadgets found.
> >   You decided to keep only the unique ones, 121128 unique gadgets found.
> >
> > This is 22% fewer unique gadgets
> >
> > While the ROPgadget tool reports:
> >
> >   Unique gadgets found: 328605
> >
> > This is 7% fewer unique gadgets. Crucially though, despite this more
> > modest reduction, the ROPgadget tool is no longer able to identify a
> > chain of gadgets for executing arbitrary syscalls. It fails at the
> > very first step, unable to find gadgets for populating registers for
> > a future syscall. Having said that, more advanced tools do still
> > manage to put together a viable ROP chain.
> >
> > Also this only takes into account QEMU code. QEMU links to many 3rd
> > party shared libraries and ideally all of them would be compiled with
> > this same hardening. That becomes a distro policy question though.
> >
> > In terms of performance impact, TCG was used as an evaluation test
> > case. We're not interested in protecting TCG since it isn't designed
> > to provide a security barrier, but it is performance sensitive code,
> > so useful as a guide to how other areas of QEMU might be impacted.
> > With the -fzero-call-user-regs=used-gpr argument present, using the
> > real world test of booting a linux kernel and having init immediately
> > poweroff, there is a ~1% slow down in performance under TCG. The QEMU
> > binary size also grows by approximately 1%.
> >
> > By comparison, using the more aggressive -fzero-call-user-regs=all,
> > results in a slowdown of over 25% in TCG, which is clearly not an
> > acceptable impact, and a binary size increase of 5%.
> >
> > Considering that 'used-gpr' succesfully stopped ROPgadget assembling
> > a chain, this more targetted protection is a justifiable hardening
> > / performance tradeoff.
> 
> Have you also considered 'used-arg'?

No, not in any detail.  I was mostly guided by the writeup here:

  https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/

which indicates Linux chose 'used-gpr'. I figured if Kees Cook
decide that was a good tradeoff for Linux, we might as well follow
it.

'used-gpr' will target any general purpose registers
that are used in a method.  'used-arg' will taget any registers
used for parameters. IIUC, this makes 'used-gpr' be a slightly
stronger protection as it covers register usage even for things
which aren't args.

> 
> > Reviewed-by: Thomas Huth <thuth@redhat.com>
> > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
> > ---
> >  meson.build | 11 +++++++++++
> >  1 file changed, 11 insertions(+)
> >
> > diff --git a/meson.build b/meson.build
> > index 6c77d9687d..eaa20d241d 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -433,6 +433,17 @@ if get_option('fuzzing')
> >    endif
> >  endif
> >  
> > +# Check further flags that make QEMU more robust against malicious parties
> > +
> > +hardening_flags = [
> > +    # Zero out registers used during a function call
> > +    # upon its return. This makes it harder to assemble
> > +    # ROP gadgets into something usable
> > +    '-fzero-call-used-regs=used-gpr',
> > +]
> > +
> > +qemu_common_flags += cc.get_supported_arguments(hardening_flags)
> > +
> >  add_global_arguments(qemu_common_flags, native: false, language: all_languages)
> >  add_global_link_arguments(qemu_ldflags, native: false, language: all_languages)
> 

With regards,
Daniel

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Tue, Jan 09, 2024 at 03:54:07PM +0100, Markus Armbruster wrote:
>> Daniel P. Berrangé <berrange@redhat.com> writes:
>> 
>> > To quote wikipedia:
>> >
>> >   "Return-oriented programming (ROP) is a computer security exploit
>> >    technique that allows an attacker to execute code in the presence
>> >    of security defenses such as executable space protection and code
>> >    signing.
>> >
>> >    In this technique, an attacker gains control of the call stack to
>> >    hijack program control flow and then executes carefully chosen
>> >    machine instruction sequences that are already present in the
>> >    machine's memory, called "gadgets". Each gadget typically ends in
>> >    a return instruction and is located in a subroutine within the
>> >    existing program and/or shared library code. Chained together,
>> >    these gadgets allow an attacker to perform arbitrary operations
>> >    on a machine employing defenses that thwart simpler attacks."
>> >
>> > QEMU is by no means perfect with an ever growing set of CVEs from
>> > flawed hardware device emulation, which could potentially be
>> > exploited using ROP techniques.
>> >
>> > Since GCC 11 there has been a compiler option that can mitigate
>> > against this exploit technique:
>> >
>> >     -fzero-call-user-regs
>> >
>> > To understand it refer to these two resources:
>> >
>> >    https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/
>> >    https://gcc.gnu.org/pipermail/gcc-patches/2020-August/552262.html
>> >
>> > I used two programs to scan qemu-system-x86_64 for ROP gadgets:
>> >
>> >   https://github.com/0vercl0k/rp
>> >   https://github.com/JonathanSalwan/ROPgadget
>> >
>> > When asked to find 8 byte gadgets, the 'rp' tool reports:
>> >
>> >   A total of 440278 gadgets found.
>> >   You decided to keep only the unique ones, 156143 unique gadgets found.
>> >
>> > While the ROPgadget tool reports:
>> >
>> >   Unique gadgets found: 353122
>> >
>> > With the --ropchain argument, the latter attempts to use the found
>> > gadgets to product a chain that can execute arbitrary syscalls. With
>> > current QEMU it succeeds in this task, which is an undesirable
>> > situation.
>> >
>> > With QEMU modified to use -fzero-call-user-regs=used-gpr the 'rp' tool
>> > reports
>> >
>> >   A total of 528991 gadgets found.
>> >   You decided to keep only the unique ones, 121128 unique gadgets found.
>> >
>> > This is 22% fewer unique gadgets
>> >
>> > While the ROPgadget tool reports:
>> >
>> >   Unique gadgets found: 328605
>> >
>> > This is 7% fewer unique gadgets. Crucially though, despite this more
>> > modest reduction, the ROPgadget tool is no longer able to identify a
>> > chain of gadgets for executing arbitrary syscalls. It fails at the
>> > very first step, unable to find gadgets for populating registers for
>> > a future syscall. Having said that, more advanced tools do still
>> > manage to put together a viable ROP chain.
>> >
>> > Also this only takes into account QEMU code. QEMU links to many 3rd
>> > party shared libraries and ideally all of them would be compiled with
>> > this same hardening. That becomes a distro policy question though.
>> >
>> > In terms of performance impact, TCG was used as an evaluation test
>> > case. We're not interested in protecting TCG since it isn't designed
>> > to provide a security barrier, but it is performance sensitive code,
>> > so useful as a guide to how other areas of QEMU might be impacted.
>> > With the -fzero-call-user-regs=used-gpr argument present, using the
>> > real world test of booting a linux kernel and having init immediately
>> > poweroff, there is a ~1% slow down in performance under TCG. The QEMU
>> > binary size also grows by approximately 1%.
>> >
>> > By comparison, using the more aggressive -fzero-call-user-regs=all,
>> > results in a slowdown of over 25% in TCG, which is clearly not an
>> > acceptable impact, and a binary size increase of 5%.
>> >
>> > Considering that 'used-gpr' succesfully stopped ROPgadget assembling
>> > a chain, this more targetted protection is a justifiable hardening
>> > / performance tradeoff.
>> 
>> Have you also considered 'used-arg'?
>
> No, not in any detail.  I was mostly guided by the writeup here:
>
>   https://www.jerkeby.se/newsletter/posts/rop-reduction-zero-call-user-regs/
>
> which indicates Linux chose 'used-gpr'. I figured if Kees Cook
> decide that was a good tradeoff for Linux, we might as well follow
> it.

Makes sense.

> 'used-gpr' will target any general purpose registers
> that are used in a method.  'used-arg' will taget any registers
> used for parameters. IIUC, this makes 'used-gpr' be a slightly
> stronger protection as it covers register usage even for things
> which aren't args.

The docs lead me to suspect it will *not* cover non-gpr registers that
are used for passing arguments.  Requires a calling convention that can
pass arguments in non-gpr registers, such as floating-point and vector
registers.  I figure these are less useful for exploits than gprs.

Thanks!

[...]