[v2,0/3] Mips support for PT_GNU_STACK
mbox series

Message ID 1563214941-16203-1-git-send-email-dmladjenovic@wavecomp.com
Headers show
Series
  • Mips support for PT_GNU_STACK
Related show

Message

Dragan Mladjenovic July 15, 2019, 6:23 p.m. UTC
Hello everyone,

Patches in this series are a slight variation of work done previously by
Faraz Shahbazker [1] in 2016.

A brief summary of the issue this is trying to address:

Up until the Linux kernel version 4.8 [2] MIPS FPU emulator used a small trampoline,
created on user stack, to handle delay slots when emulating FPU branches.
Because of this non-executable stack could not be enabled by default on MIPS.
The compatibility issue is that these old kernels respect PT_GNU_STACK,
making the stack non-executable if requested, and could crash the user process if
there would be need to emulate an instruction in the delay slot of an FPU branch.

In order to allow for the tool-chain to safely use PT_GNU_STACK by default and to
provide the compatibility with pre-4.8 kernels, the original patch would revert
stack protection back to executable stack if it could not detect that kernel
supports non-executable stack.

The form of detection the patch proposes is not yet provided by the kernel.
Instead, this version of the patch does kernel version check at runtime and
provides compatible behavior if it cannot detect the 4.8 kernel or newer.
 
The last patch increments the ABI Version number in order to disallow new
binaries to run with older glibc. The number is not set in stone.
I'm assuming it will probably land after GNU_HASH [3] support which consumes
ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
part gets finalized.

Even if this part doesn't get in the next release due to issues [4] with ABI
version handling, it would be still nice if the back-compat support gets in.
I would like to hear your thoughts on this.

Changes from v1 [5]: Moved stack override logic behind inline _dl_exec_stack_override.

Best regards,

Dragan

[1] https://sourceware.org/ml/libc-alpha/2016-02/msg00076.html
[2] https://github.com/torvalds/linux/commit/432c6bacbd0c16ec210c43da411ccc3855c4c010
[3] https://sourceware.org/ml/libc-alpha/2019-06/msg00456.html
[4] https://sourceware.org/ml/libc-alpha/2019-06/msg00730.html
[5] https://sourceware.org/ml/libc-alpha/2019-06/msg00889.html

Dragan Mladjenovic (3):
  [ELF] Allow the machine support to enforce executable stack
  [MIPS] Define DL_EXEC_STACK_OVERRIDE
  [RFC][MIPS] Define GNU_STACK ABI

 elf/dl-exec-stack-override.h                       | 36 ++++++++++++++++++++++
 elf/dl-support.c                                   |  3 ++
 elf/rtld.c                                         |  3 ++
 sysdeps/generic/ldsodefs.h                         |  4 +++
 sysdeps/unix/sysv/linux/mips/Makefile              | 26 +++++++++++++---
 sysdeps/unix/sysv/linux/mips/configure.ac          |  3 ++
 sysdeps/unix/sysv/linux/mips/ldsodefs.h            | 13 +++++++-
 sysdeps/unix/sysv/linux/mips/libc-abis             |  2 ++
 .../sysv/linux/mips/tst-execstack-ovrd-static.c    |  1 +
 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd.c  |  2 ++
 .../sysv/linux/mips/tst-execstack-ovrd1-static.c   |  1 +
 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1.c | 10 ++++++
 12 files changed, 99 insertions(+), 5 deletions(-)
 create mode 100644 elf/dl-exec-stack-override.h
 create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd-static.c
 create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd.c
 create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1-static.c
 create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1.c

Comments

Adhemerval Zanella July 17, 2019, 7:43 p.m. UTC | #1
On 15/07/2019 15:23, Dragan Mladjenovic wrote:
> Hello everyone,
> 
> Patches in this series are a slight variation of work done previously by
> Faraz Shahbazker [1] in 2016.
> 
> A brief summary of the issue this is trying to address:
> 
> Up until the Linux kernel version 4.8 [2] MIPS FPU emulator used a small trampoline,
> created on user stack, to handle delay slots when emulating FPU branches.
> Because of this non-executable stack could not be enabled by default on MIPS.
> The compatibility issue is that these old kernels respect PT_GNU_STACK,
> making the stack non-executable if requested, and could crash the user process if
> there would be need to emulate an instruction in the delay slot of an FPU branch.
> 
> In order to allow for the tool-chain to safely use PT_GNU_STACK by default and to
> provide the compatibility with pre-4.8 kernels, the original patch would revert
> stack protection back to executable stack if it could not detect that kernel
> supports non-executable stack.
> 
> The form of detection the patch proposes is not yet provided by the kernel.
> Instead, this version of the patch does kernel version check at runtime and
> provides compatible behavior if it cannot detect the 4.8 kernel or newer.

I think checking the kernel version is the wrong approach, it prevents a distribution
to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
other architectures do for similar kernel features and not tie it to any specific
version.

>  
> The last patch increments the ABI Version number in order to disallow new
> binaries to run with older glibc. The number is not set in stone.
> I'm assuming it will probably land after GNU_HASH [3] support which consumes
> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
> part gets finalized.

If the idea is to fallback to executable stack for the case of underlying missing
kernel support, which is the net gain in adding this requirement? My understanding
it ABI bump should be used to fail early for the cases where the new binaries
requires loader support that can not be provided (iFUNC or new relocations), not
for hardening.

> 
> Even if this part doesn't get in the next release due to issues [4] with ABI
> version handling, it would be still nice if the back-compat support gets in.
> I would like to hear your thoughts on this.

If you don't tie it support to kernel version checks or require bump the ABI version,
it should be fair straight to backport it.

> 
> Changes from v1 [5]: Moved stack override logic behind inline _dl_exec_stack_override.
> 
> Best regards,
> 
> Dragan
> 
> [1] https://sourceware.org/ml/libc-alpha/2016-02/msg00076.html
> [2] https://github.com/torvalds/linux/commit/432c6bacbd0c16ec210c43da411ccc3855c4c010
> [3] https://sourceware.org/ml/libc-alpha/2019-06/msg00456.html
> [4] https://sourceware.org/ml/libc-alpha/2019-06/msg00730.html
> [5] https://sourceware.org/ml/libc-alpha/2019-06/msg00889.html
> 
> Dragan Mladjenovic (3):
>   [ELF] Allow the machine support to enforce executable stack
>   [MIPS] Define DL_EXEC_STACK_OVERRIDE
>   [RFC][MIPS] Define GNU_STACK ABI
> 
>  elf/dl-exec-stack-override.h                       | 36 ++++++++++++++++++++++
>  elf/dl-support.c                                   |  3 ++
>  elf/rtld.c                                         |  3 ++
>  sysdeps/generic/ldsodefs.h                         |  4 +++
>  sysdeps/unix/sysv/linux/mips/Makefile              | 26 +++++++++++++---
>  sysdeps/unix/sysv/linux/mips/configure.ac          |  3 ++
>  sysdeps/unix/sysv/linux/mips/ldsodefs.h            | 13 +++++++-
>  sysdeps/unix/sysv/linux/mips/libc-abis             |  2 ++
>  .../sysv/linux/mips/tst-execstack-ovrd-static.c    |  1 +
>  sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd.c  |  2 ++
>  .../sysv/linux/mips/tst-execstack-ovrd1-static.c   |  1 +
>  sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1.c | 10 ++++++
>  12 files changed, 99 insertions(+), 5 deletions(-)
>  create mode 100644 elf/dl-exec-stack-override.h
>  create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd-static.c
>  create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd.c
>  create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1-static.c
>  create mode 100644 sysdeps/unix/sysv/linux/mips/tst-execstack-ovrd1.c
>
Dragan Mladjenovic July 17, 2019, 9:47 p.m. UTC | #2
On 17.07.2019. 21:43, Adhemerval Zanella wrote:
>
> On 15/07/2019 15:23, Dragan Mladjenovic wrote:
>> Hello everyone,
>>
>> Patches in this series are a slight variation of work done previously by
>> Faraz Shahbazker [1] in 2016.
>>
>> A brief summary of the issue this is trying to address:
>>
>> Up until the Linux kernel version 4.8 [2] MIPS FPU emulator used a small trampoline,
>> created on user stack, to handle delay slots when emulating FPU branches.
>> Because of this non-executable stack could not be enabled by default on MIPS.
>> The compatibility issue is that these old kernels respect PT_GNU_STACK,
>> making the stack non-executable if requested, and could crash the user process if
>> there would be need to emulate an instruction in the delay slot of an FPU branch.
>>
>> In order to allow for the tool-chain to safely use PT_GNU_STACK by default and to
>> provide the compatibility with pre-4.8 kernels, the original patch would revert
>> stack protection back to executable stack if it could not detect that kernel
>> supports non-executable stack.
>>
>> The form of detection the patch proposes is not yet provided by the kernel.
>> Instead, this version of the patch does kernel version check at runtime and
>> provides compatible behavior if it cannot detect the 4.8 kernel or newer.
> I think checking the kernel version is the wrong approach, it prevents a distribution
> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
> other architectures do for similar kernel features and not tie it to any specific
> version.

First, thanks for the comments.
 
I agree with everything and the original patch did propose the usage of AT_FLAGS
auxval for this purpose. But, sadly, the kernel still doesn't advertise this information
to the user mode and it would take some time for it to be back-ported even if it gets
implemented, let's say in next kernel release. Meanwhile, this approach, while lacking,
would allow for distributions to enable PT_GNU_STACK way earlier and to actually
enjoy the benefit of it on existing 4.8+ kernels while remaining compatible with pre-4.8
ones.

>>  
>> The last patch increments the ABI Version number in order to disallow new
>> binaries to run with older glibc. The number is not set in stone.
>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>> part gets finalized.
> If the idea is to fallback to executable stack for the case of underlying missing
> kernel support, which is the net gain in adding this requirement? My understanding
> it ABI bump should be used to fail early for the cases where the new binaries
> requires loader support that can not be provided (iFUNC or new relocations), not
> for hardening.

True, and that is the reason I'm not hard pressed for this part of the patch. Though one
nice benefit form it is that it prevents you from accidentally using the GNU.stack enabled
mips toolchain to build glibc w/o this patch. I guess that can be prevented on gcc side if
glibc is built as part of the toolchain build, but otherwise, this would require some discipline
form the end user.

Without this, the dynamically liked binaries/shared objects would still work with older glibcs.
Given that existing distributions are build w/o PT_GNU_STACK they would get the executable
stack anyway. Statically linked binaries would be handled by linked libc. So we are safe on
pre-4.8 kernels.

This should work as long as there is no RW PT_GNU_STACK built glibc w/o this patch in "the wild".

From my point of view this is more to enable, at least for glibc based environment,
RW PT_GNU_STACK by default w/o having the end-user worrying on which kernel will it run
than it is to enforce the hardening. 

>> Even if this part doesn't get in the next release due to issues [4] with ABI
>> version handling, it would be still nice if the back-compat support gets in.
>> I would like to hear your thoughts on this.
> If you don't tie it support to kernel version checks or require bump the ABI version,
> it should be fair straight to backport it.

Another option that came up during the v1 discussion is to actually require 4.8 as minimal
kernel version if glibc gets build with RW PT_GNU_STACK.

If it gets back-ported far enough, that approach would satisfy the requirement that there is no
RW PT_GNU_STACK glibc running on the pre-4.8 kernels which would still allow us to enable 
GNU.stack on GCC by default, at least for mips*-*-gnu* triplets.


Best regards,

Dragan
Faraz Shahbazker July 17, 2019, 10:59 p.m. UTC | #3
On 07/17/2019 12:43 PM, Adhemerval Zanella wrote:
> I think checking the kernel version is the wrong approach, it prevents a distribution
> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
> other architectures do for similar kernel features and not tie it to any specific
> version.

The original proposal was to advertise through AT_FLAGS. I've heard this suggestion of using
hwcap from multiple sources, so I am curious - what other *purely* software kernel features
are advertised using hwcap?

>> The last patch increments the ABI Version number in order to disallow new
>> binaries to run with older glibc. The number is not set in stone.
>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>> part gets finalized.
> 
> If the idea is to fallback to executable stack for the case of underlying missing
> kernel support, which is the net gain in adding this requirement? My understanding
> it ABI bump should be used to fail early for the cases where the new binaries
> requires loader support that can not be provided (iFUNC or new relocations), not
> for hardening.

It is not really hardening, the way glibc handles PT_GNU_STACK is along the lines of
'may have non-executable stack' rather than 'must have non-executable stack'. However
the MIPS backend overrides *any* incoming PT_GNU_STACK permissions using the default (RWX)
permissions for MIPS, in effect enforcing 'will not have non-executable stack'. What is
being indicated here is a change in this default behaviour. The ABI version bump would
indicate whether PT_GNU_STACK permissions will be honoured (at least to the extent it is
by other architectures) or simply ignored (as it has historically been).

IMO, the current behaviour of PT_GNU_STACK for MIPS is an anomaly in itself. What should
have been, is a rejection of non-executable PT_GNU_STACK at some level, instead of silently
overriding it in glibc. So are you of the opinion that this change in glibc behaviour is not
worth being published at all, or that it should be advertised using a different mechanism
instead of an ABI version bump?

Regards,
Faraz
Adhemerval Zanella July 18, 2019, 1:33 p.m. UTC | #4
On 17/07/2019 18:47, Dragan Mladjenovic wrote:
> 
> On 17.07.2019. 21:43, Adhemerval Zanella wrote:
>>
>> On 15/07/2019 15:23, Dragan Mladjenovic wrote:
>>> Hello everyone,
>>>
>>> Patches in this series are a slight variation of work done previously by
>>> Faraz Shahbazker [1] in 2016.
>>>
>>> A brief summary of the issue this is trying to address:
>>>
>>> Up until the Linux kernel version 4.8 [2] MIPS FPU emulator used a small trampoline,
>>> created on user stack, to handle delay slots when emulating FPU branches.
>>> Because of this non-executable stack could not be enabled by default on MIPS.
>>> The compatibility issue is that these old kernels respect PT_GNU_STACK,
>>> making the stack non-executable if requested, and could crash the user process if
>>> there would be need to emulate an instruction in the delay slot of an FPU branch.
>>>
>>> In order to allow for the tool-chain to safely use PT_GNU_STACK by default and to
>>> provide the compatibility with pre-4.8 kernels, the original patch would revert
>>> stack protection back to executable stack if it could not detect that kernel
>>> supports non-executable stack.
>>>
>>> The form of detection the patch proposes is not yet provided by the kernel.
>>> Instead, this version of the patch does kernel version check at runtime and
>>> provides compatible behavior if it cannot detect the 4.8 kernel or newer.
>> I think checking the kernel version is the wrong approach, it prevents a distribution
>> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
>> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
>> other architectures do for similar kernel features and not tie it to any specific
>> version.
> 
> First, thanks for the comments.
>  
> I agree with everything and the original patch did propose the usage of AT_FLAGS
> auxval for this purpose. But, sadly, the kernel still doesn't advertise this information
> to the user mode and it would take some time for it to be back-ported even if it gets
> implemented, let's say in next kernel release. Meanwhile, this approach, while lacking,
> would allow for distributions to enable PT_GNU_STACK way earlier and to actually
> enjoy the benefit of it on existing 4.8+ kernels while remaining compatible with pre-4.8
> ones.

It seems the kernel commit that actually added this support on mips was 
432c6bacbd0c16ec210c43da411ccc3855c4c010.  Isn't there any other mechanism to check 
for kernel support without tying a kernel version check? 

The code seems to reserve an extra page for the delay slot emulation, could we
infer based on vdso placement, page size, and maximum stack size? 

Otherwise I think we should do as usual, which is just assumes non-executable stack if
glibc is configure for kernel higher than 4.8 (with --enable-kernel). Checking on
patch submission history, Florian also stated that runtime version check is not
an option here.

> 
>>>  
>>> The last patch increments the ABI Version number in order to disallow new
>>> binaries to run with older glibc. The number is not set in stone.
>>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>>> part gets finalized.
>> If the idea is to fallback to executable stack for the case of underlying missing
>> kernel support, which is the net gain in adding this requirement? My understanding
>> it ABI bump should be used to fail early for the cases where the new binaries
>> requires loader support that can not be provided (iFUNC or new relocations), not
>> for hardening.
> 
> True, and that is the reason I'm not hard pressed for this part of the patch. Though one
> nice benefit form it is that it prevents you from accidentally using the GNU.stack enabled
> mips toolchain to build glibc w/o this patch. I guess that can be prevented on gcc side if
> glibc is built as part of the toolchain build, but otherwise, this would require some discipline
> form the end user.
> 
> Without this, the dynamically liked binaries/shared objects would still work with older glibcs.
> Given that existing distributions are build w/o PT_GNU_STACK they would get the executable
> stack anyway. Statically linked binaries would be handled by linked libc. So we are safe on
> pre-4.8 kernels.
> 
> This should work as long as there is no RW PT_GNU_STACK built glibc w/o this patch in "the wild".
> 
> From my point of view this is more to enable, at least for glibc based environment,
> RW PT_GNU_STACK by default w/o having the end-user worrying on which kernel will it run
> than it is to enforce the hardening. 

I don't have a strong opinion here, although the main issue here is runtime support
provided by underlying kernel rather than an ABI issue in fact.  

One option is just to check if compilers sets GNU.stack as default to require a
minimum kernel version of 3.8 (as we already did for 2008 NaN on d5f2798a0ac9).

> 
>>> Even if this part doesn't get in the next release due to issues [4] with ABI
>>> version handling, it would be still nice if the back-compat support gets in.
>>> I would like to hear your thoughts on this.
>> If you don't tie it support to kernel version checks or require bump the ABI version,
>> it should be fair straight to backport it.
> 
> Another option that came up during the v1 discussion is to actually require 4.8 as minimal
> kernel version if glibc gets build with RW PT_GNU_STACK.
> 
> If it gets back-ported far enough, that approach would satisfy the requirement that there is no
> RW PT_GNU_STACK glibc running on the pre-4.8 kernels which would still allow us to enable 
> GNU.stack on GCC by default, at least for mips*-*-gnu* triplets.

If a distribution aims to backport this features, it will need to coordinate on 
multiple parts (kernel, compiler, runtime), so I prefer to avoid trying to 
provision this beforehand.  I would go to with a simpler solution, instead of
adding runtime/configure checks.

> 
> 
> Best regards,
> 
> Dragan
>
Adhemerval Zanella July 18, 2019, 1:38 p.m. UTC | #5
On 17/07/2019 19:59, Faraz Shahbazker wrote:
> On 07/17/2019 12:43 PM, Adhemerval Zanella wrote:
>> I think checking the kernel version is the wrong approach, it prevents a distribution
>> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
>> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
>> other architectures do for similar kernel features and not tie it to any specific
>> version.
> 
> The original proposal was to advertise through AT_FLAGS. I've heard this suggestion of using
> hwcap from multiple sources, so I am curious - what other *purely* software kernel features
> are advertised using hwcap?

It is not common, but on powerpc has both PPC_FEATURE2_HTM_NOSC and 
PPC_FEATURE2_HTM_NO_SUSPEND which is kernel behaviour regarding transaction
memory state and syscall execution.

> 
>>> The last patch increments the ABI Version number in order to disallow new
>>> binaries to run with older glibc. The number is not set in stone.
>>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>>> part gets finalized.
>>
>> If the idea is to fallback to executable stack for the case of underlying missing
>> kernel support, which is the net gain in adding this requirement? My understanding
>> it ABI bump should be used to fail early for the cases where the new binaries
>> requires loader support that can not be provided (iFUNC or new relocations), not
>> for hardening.
> 
> It is not really hardening, the way glibc handles PT_GNU_STACK is along the lines of
> 'may have non-executable stack' rather than 'must have non-executable stack'. However
> the MIPS backend overrides *any* incoming PT_GNU_STACK permissions using the default (RWX)
> permissions for MIPS, in effect enforcing 'will not have non-executable stack'. What is
> being indicated here is a change in this default behaviour. The ABI version bump would
> indicate whether PT_GNU_STACK permissions will be honoured (at least to the extent it is
> by other architectures) or simply ignored (as it has historically been).
> 
> IMO, the current behaviour of PT_GNU_STACK for MIPS is an anomaly in itself. What should
> have been, is a rejection of non-executable PT_GNU_STACK at some level, instead of silently
> overriding it in glibc. So are you of the opinion that this change in glibc behaviour is not
> worth being published at all, or that it should be advertised using a different mechanism
> instead of an ABI version bump?

Since non-executable stack is tied with underlying kernel support rather than 
ABI, my suggestion is just to assume non-executable stack as default, without
permission override, if glibc is configure for kernel higher than 3.8. We will
need to live with old behaviour for default builds.
Maciej W. Rozycki July 18, 2019, 2:50 p.m. UTC | #6
On Wed, 17 Jul 2019, Adhemerval Zanella wrote:

> > The form of detection the patch proposes is not yet provided by the kernel.
> > Instead, this version of the patch does kernel version check at runtime and
> > provides compatible behavior if it cannot detect the 4.8 kernel or newer.
> 
> I think checking the kernel version is the wrong approach, it prevents a 
> distribution to backport the kernel fix without also applying a 
> out-of-tree patch to fix it on glibc as well.

 Good point!

>  IMHO the proper way would be to make kernel advertise it through hwcap, 
> as other architectures do for similar kernel features and not tie it to 
> any specific version.

 As other people have pointed out to use AT_HWCAP* for a plain software 
feature might not be the most fortunate choice and we could use AT_FLAGS 
or even define AT_LINUX_FEATURES (or AT_MIPS_FEATURES), but that is 
secondary; it's not like we're going to run out of auxiliary vector's 
space.

 FWIW,

  Maciej
Faraz Shahbazker July 18, 2019, 3:34 p.m. UTC | #7
On 7/18/19 6:38 AM, Adhemerval Zanella wrote:> On 17/07/2019 19:59, Faraz Shahbazker wrote:
>> On 07/17/2019 12:43 PM, Adhemerval Zanella wrote:
>> IMO, the current behaviour of PT_GNU_STACK for MIPS is an anomaly in itself. What should
>> have been, is a rejection of non-executable PT_GNU_STACK at some level, instead of silently
>> overriding it in glibc. So are you of the opinion that this change in glibc behaviour is not
>> worth being published at all, or that it should be advertised using a different mechanism
>> instead of an ABI version bump?
> 
> Since non-executable stack is tied with underlying kernel support rather than 
> ABI, my suggestion is just to assume non-executable stack as default, without
> permission override, if glibc is configure for kernel higher than *4.8*. We will
> need to live with old behaviour for default builds.

Frankly, I can live with that. If someone really wants to back-port they have to do the
leg-work across multiple components to get it to work. This theoretically still leaves
the door open for the kernel to start advertising non-exec stack support to allow for
easier back-ports.
Regards,
Faraz
Dragan Mladjenovic July 18, 2019, 7:49 p.m. UTC | #8
On 18.07.2019. 15:38, Adhemerval Zanella wrote:
>
>
> On 17/07/2019 19:59, Faraz Shahbazker wrote:
>> On 07/17/2019 12:43 PM, Adhemerval Zanella wrote:
>>> I think checking the kernel version is the wrong approach, it prevents a distribution
>>> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
>>> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
>>> other architectures do for similar kernel features and not tie it to any specific
>>> version.
>>
>> The original proposal was to advertise through AT_FLAGS. I've heard this suggestion of using
>> hwcap from multiple sources, so I am curious - what other *purely* software kernel features
>> are advertised using hwcap?
>
> It is not common, but on powerpc has both PPC_FEATURE2_HTM_NOSC and
> PPC_FEATURE2_HTM_NO_SUSPEND which is kernel behaviour regarding transaction
> memory state and syscall execution.
>
>>
>>>> The last patch increments the ABI Version number in order to disallow new
>>>> binaries to run with older glibc. The number is not set in stone.
>>>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>>>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>>>> part gets finalized.
>>>
>>> If the idea is to fallback to executable stack for the case of underlying missing
>>> kernel support, which is the net gain in adding this requirement? My understanding
>>> it ABI bump should be used to fail early for the cases where the new binaries
>>> requires loader support that can not be provided (iFUNC or new relocations), not
>>> for hardening.
>>
>> It is not really hardening, the way glibc handles PT_GNU_STACK is along the lines of
>> 'may have non-executable stack' rather than 'must have non-executable stack'. However
>> the MIPS backend overrides *any* incoming PT_GNU_STACK permissions using the default (RWX)
>> permissions for MIPS, in effect enforcing 'will not have non-executable stack'. What is
>> being indicated here is a change in this default behaviour. The ABI version bump would
>> indicate whether PT_GNU_STACK permissions will be honoured (at least to the extent it is
>> by other architectures) or simply ignored (as it has historically been).
>>
>> IMO, the current behaviour of PT_GNU_STACK for MIPS is an anomaly in itself. What should
>> have been, is a rejection of non-executable PT_GNU_STACK at some level, instead of silently
>> overriding it in glibc. So are you of the opinion that this change in glibc behaviour is not
>> worth being published at all, or that it should be advertised using a different mechanism
>> instead of an ABI version bump?
>
> Since non-executable stack is tied with underlying kernel support rather than
> ABI, my suggestion is just to assume non-executable stack as default, without
> permission override, if glibc is configure for kernel higher than 3.8. We will
> need to live with old behaviour for default builds.
>

If I follow correctly this can be simplified to either require 4.8 
kernel when built with non-executable stack or force executable stack at 
build time for default (< 4.8) builds?

Just to be on the same page. This "old" behavior of overriding 
PT_GNU_STACK permission to RWX on mips is introduced in the second patch 
of the series.

What I understand is that previously you could build a glibc with RW 
PT_GNU_STACK and it may or may not work at runtime depending on if the 
kernel actually needed to use you stack for emulation and if you 
hardware is capable of enforcing NX page protection.

If linker, glibc or kernel did historically disallow RW PT_GNU_STACK we 
would not have this issue.

I interpret this abi version bump more as assurance that we are not 
running in the environment that might crash just because we requested
RW PT_GNU_STACK. (I'm ignoring existence of other libc implementations 
here.)

Best regards,

Dragan
Dragan Mladjenovic July 18, 2019, 8 p.m. UTC | #9
On 18.07.2019. 15:33, Adhemerval Zanella wrote:
>
>
> On 17/07/2019 18:47, Dragan Mladjenovic wrote:
>>
>> On 17.07.2019. 21:43, Adhemerval Zanella wrote:
>>>
>>> On 15/07/2019 15:23, Dragan Mladjenovic wrote:
>>>> Hello everyone,
>>>>
>>>> Patches in this series are a slight variation of work done previously by
>>>> Faraz Shahbazker [1] in 2016.
>>>>
>>>> A brief summary of the issue this is trying to address:
>>>>
>>>> Up until the Linux kernel version 4.8 [2] MIPS FPU emulator used a small trampoline,
>>>> created on user stack, to handle delay slots when emulating FPU branches.
>>>> Because of this non-executable stack could not be enabled by default on MIPS.
>>>> The compatibility issue is that these old kernels respect PT_GNU_STACK,
>>>> making the stack non-executable if requested, and could crash the user process if
>>>> there would be need to emulate an instruction in the delay slot of an FPU branch.
>>>>
>>>> In order to allow for the tool-chain to safely use PT_GNU_STACK by default and to
>>>> provide the compatibility with pre-4.8 kernels, the original patch would revert
>>>> stack protection back to executable stack if it could not detect that kernel
>>>> supports non-executable stack.
>>>>
>>>> The form of detection the patch proposes is not yet provided by the kernel.
>>>> Instead, this version of the patch does kernel version check at runtime and
>>>> provides compatible behavior if it cannot detect the 4.8 kernel or newer.
>>> I think checking the kernel version is the wrong approach, it prevents a distribution
>>> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
>>> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
>>> other architectures do for similar kernel features and not tie it to any specific
>>> version.
>>
>> First, thanks for the comments.
>>
>> I agree with everything and the original patch did propose the usage of AT_FLAGS
>> auxval for this purpose. But, sadly, the kernel still doesn't advertise this information
>> to the user mode and it would take some time for it to be back-ported even if it gets
>> implemented, let's say in next kernel release. Meanwhile, this approach, while lacking,
>> would allow for distributions to enable PT_GNU_STACK way earlier and to actually
>> enjoy the benefit of it on existing 4.8+ kernels while remaining compatible with pre-4.8
>> ones.
>
> It seems the kernel commit that actually added this support on mips was
> 432c6bacbd0c16ec210c43da411ccc3855c4c010.  Isn't there any other mechanism to check
> for kernel support without tying a kernel version check?
>
> The code seems to reserve an extra page for the delay slot emulation, could we
> infer based on vdso placement, page size, and maximum stack size?
>

Will have to check on this. So far it doesn't seem we can. I think that 
on pre-vdso kernels the same page was used for sigreturn trampoline. I 
either way even if we could do this, we could end up painting ourselves 
in a corner similar as x86_64 with vsyscalls page. It would become the 
part of userspace ABI. For now the emutrap page is just an 
implementation detail.


Best regards,

Dragan
Adhemerval Zanella July 18, 2019, 8:54 p.m. UTC | #10
On 18/07/2019 16:49, Dragan Mladjenovic wrote:
> 
> 
> On 18.07.2019. 15:38, Adhemerval Zanella wrote:
>>
>>
>> On 17/07/2019 19:59, Faraz Shahbazker wrote:
>>> On 07/17/2019 12:43 PM, Adhemerval Zanella wrote:
>>>> I think checking the kernel version is the wrong approach, it prevents a distribution
>>>> to backport the kernel fix without also applying a out-of-tree patch to fix it on glibc
>>>> as well.  IMHO the proper way would be to make kernel advertise it through hwcap, as
>>>> other architectures do for similar kernel features and not tie it to any specific
>>>> version.
>>>
>>> The original proposal was to advertise through AT_FLAGS. I've heard this suggestion of using
>>> hwcap from multiple sources, so I am curious - what other *purely* software kernel features
>>> are advertised using hwcap?
>>
>> It is not common, but on powerpc has both PPC_FEATURE2_HTM_NOSC and
>> PPC_FEATURE2_HTM_NO_SUSPEND which is kernel behaviour regarding transaction
>> memory state and syscall execution.
>>
>>>
>>>>> The last patch increments the ABI Version number in order to disallow new
>>>>> binaries to run with older glibc. The number is not set in stone.
>>>>> I'm assuming it will probably land after GNU_HASH [3] support which consumes
>>>>> ABI version 5 for MIPS. I will send a proposal for Binutils and GCC after this
>>>>> part gets finalized.
>>>>
>>>> If the idea is to fallback to executable stack for the case of underlying missing
>>>> kernel support, which is the net gain in adding this requirement? My understanding
>>>> it ABI bump should be used to fail early for the cases where the new binaries
>>>> requires loader support that can not be provided (iFUNC or new relocations), not
>>>> for hardening.
>>>
>>> It is not really hardening, the way glibc handles PT_GNU_STACK is along the lines of
>>> 'may have non-executable stack' rather than 'must have non-executable stack'. However
>>> the MIPS backend overrides *any* incoming PT_GNU_STACK permissions using the default (RWX)
>>> permissions for MIPS, in effect enforcing 'will not have non-executable stack'. What is
>>> being indicated here is a change in this default behaviour. The ABI version bump would
>>> indicate whether PT_GNU_STACK permissions will be honoured (at least to the extent it is
>>> by other architectures) or simply ignored (as it has historically been).
>>>
>>> IMO, the current behaviour of PT_GNU_STACK for MIPS is an anomaly in itself. What should
>>> have been, is a rejection of non-executable PT_GNU_STACK at some level, instead of silently
>>> overriding it in glibc. So are you of the opinion that this change in glibc behaviour is not
>>> worth being published at all, or that it should be advertised using a different mechanism
>>> instead of an ABI version bump?
>>
>> Since non-executable stack is tied with underlying kernel support rather than
>> ABI, my suggestion is just to assume non-executable stack as default, without
>> permission override, if glibc is configure for kernel higher than 3.8. We will
>> need to live with old behaviour for default builds.
>>
> 
> If I follow correctly this can be simplified to either require 4.8 
> kernel when built with non-executable stack or force executable stack at 
> build time for default (< 4.8) builds?
> 
> Just to be on the same page. This "old" behavior of overriding 
> PT_GNU_STACK permission to RWX on mips is introduced in the second patch 
> of the series.
> 
> What I understand is that previously you could build a glibc with RW 
> PT_GNU_STACK and it may or may not work at runtime depending on if the 
> kernel actually needed to use you stack for emulation and if you 
> hardware is capable of enforcing NX page protection.
> 
> If linker, glibc or kernel did historically disallow RW PT_GNU_STACK we 
> would not have this issue.
> 
> I interpret this abi version bump more as assurance that we are not 
> running in the environment that might crash just because we requested
> RW PT_GNU_STACK. (I'm ignoring existence of other libc implementations 
> here.)

My understanding is ELFOSABI_SYSV check is only required if we build glibc
with support for kernel older than 3.8, to make newer binaries fail early 
on kernels that might not honour the PT_GNU_STACK.

So indeed it might a good idea to bump ELFOSABI_SYSV for to handle this case.