diff mbox series

[3/6] linux-user: Prohibit brk() to to shrink below initial heap address

Message ID 20230717213545.142598-4-deller@gmx.de
State New
Headers show
Series linux-user: brk() syscall fixes and armhf static binary fix | expand

Commit Message

Helge Deller July 17, 2023, 9:35 p.m. UTC 
Since commit 86f04735ac ("linux-user: Fix brk() to release pages") it's
possible for userspace applications to reduce memory footprint by
calling brk() with a lower address and free up memory. Before that guest
heap memory never was unmapped.

But the Linux kernel prohibits to reduce brk() below the initial memory
address which is set at startup by the program, while this check was
missed in commit 86f04735ac.

This patch adds the missing check by storing the initial brk value in
initial_target_brk and verify new brk addresses against that value.

Tested with the i386 upx binary from
https://github.com/upx/upx/releases/download/v4.0.2/upx-4.0.2-i386_linux.tar.xz

Signed-off-by: Helge Deller <deller@gmx.de>
Tested-by: Markus F.X.J. Oberhumer <notifications@github.com>
Fixes: 86f04735ac ("linux-user: Fix brk() to release pages")
Buglink: https://github.com/upx/upx/issues/683
---
 linux-user/syscall.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

--
2.41.0
diff mbox series

Patch

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index f877156ed3..92d146f8fb 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -801,12 +801,13 @@  static inline int host_to_target_sock_type(int host_type)
     return target_type;
 }

-static abi_ulong target_brk;
+static abi_ulong target_brk, initial_target_brk;
 static abi_ulong brk_page;

 void target_set_brk(abi_ulong new_brk)
 {
     target_brk = new_brk;
+    initial_target_brk = new_brk;
     brk_page = HOST_PAGE_ALIGN(target_brk);
 }

@@ -824,6 +825,11 @@  abi_long do_brk(abi_ulong brk_val)
         return target_brk;
     }

+    /* do not allow to shrink below initial brk value */
+    if (brk_val < initial_target_brk) {
+        brk_val = initial_target_brk;
+    }
+
     new_brk = TARGET_PAGE_ALIGN(brk_val);
     new_host_brk_page = HOST_PAGE_ALIGN(brk_val);