diff mbox series

[v3] linux-user: do setrlimit selectively

Message ID 20180917181314.22551-1-jcmvbkbc@gmail.com
State New
Headers show
Series [v3] linux-user: do setrlimit selectively | expand

Commit Message

Max Filippov Sept. 17, 2018, 6:13 p.m. UTC
setrlimit guest calls that affect memory resources
(RLIMIT_{AS,DATA,STACK}) may interfere with QEMU internal memory
management. They may result in QEMU lockup because mprotect call in
page_unprotect would fail with ENOMEM error code, causing infinite loop
of SIGSEGV. E.g. it happens when running libstdc++ testsuite for xtensa
target on x86_64 host.

Don't call host setrlimit for memory-related resources.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
---
Changes v2->v3:
- add comment sugessted by Peter

Changes v1->v2:
- don't limit change to 32-bit guest on 64-bit host case

 linux-user/syscall.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

Comments

Laurent Vivier Sept. 18, 2018, 4 p.m. UTC | #1
Le 17/09/2018 à 20:13, Max Filippov a écrit :
> setrlimit guest calls that affect memory resources
> (RLIMIT_{AS,DATA,STACK}) may interfere with QEMU internal memory
> management. They may result in QEMU lockup because mprotect call in
> page_unprotect would fail with ENOMEM error code, causing infinite loop
> of SIGSEGV. E.g. it happens when running libstdc++ testsuite for xtensa
> target on x86_64 host.
> 
> Don't call host setrlimit for memory-related resources.
> 
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
> ---
> Changes v2->v3:
> - add comment sugessted by Peter
> 
> Changes v1->v2:
> - don't limit change to 32-bit guest on 64-bit host case
> 
>  linux-user/syscall.c | 16 +++++++++++++++-
>  1 file changed, 15 insertions(+), 1 deletion(-)

rebased on master and applied on my branch linux-user-for-3.1

Thanks,
Laurent
diff mbox series

Patch

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index bb42a225eb58..f06307ebd945 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -9275,7 +9275,21 @@  abi_long do_syscall(void *cpu_env, int num, abi_long arg1,
             rlim.rlim_cur = target_to_host_rlim(target_rlim->rlim_cur);
             rlim.rlim_max = target_to_host_rlim(target_rlim->rlim_max);
             unlock_user_struct(target_rlim, arg2, 0);
-            ret = get_errno(setrlimit(resource, &rlim));
+            /*
+             * If we just passed through resource limit settings for memory then
+             * they would also apply to QEMU's own allocations, and QEMU will
+             * crash or hang or die if its allocations fail. Ideally we would
+             * track the guest allocations in QEMU and apply the limits ourselves.
+             * For now, just tell the guest the call succeeded but don't actually
+             * limit anything.
+             */
+            if (resource != RLIMIT_AS &&
+                resource != RLIMIT_DATA &&
+                resource != RLIMIT_STACK) {
+                return get_errno(setrlimit(resource, &rlim));
+            } else {
+                return 0;
+            }
         }
         break;
 #endif