From patchwork Tue Nov 14 09:42:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 837764 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="SGmmMGyk"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3ybjK25Z0fz9s7m for ; Tue, 14 Nov 2017 20:44:38 +1100 (AEDT) Received: from localhost ([::1]:58515 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEXlk-0006TB-GA for incoming@patchwork.ozlabs.org; Tue, 14 Nov 2017 04:44:36 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58403) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eEXlM-0006R3-Un for qemu-devel@nongnu.org; Tue, 14 Nov 2017 04:44:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eEXlH-0005ez-SM for qemu-devel@nongnu.org; Tue, 14 Nov 2017 04:44:13 -0500 Received: from mail-wr0-x243.google.com ([2a00:1450:400c:c0c::243]:53170) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eEXlH-0005eS-Hm for qemu-devel@nongnu.org; Tue, 14 Nov 2017 04:44:07 -0500 Received: by mail-wr0-x243.google.com with SMTP id j23so16914333wra.9 for ; Tue, 14 Nov 2017 01:44:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=WaMrwHciPAETRSvhXhXwAGx5KtbYSUVKuFVkDY9duRs=; b=SGmmMGyk+fAfbdaFD0WDQ1zx5jJVqW8ib87vj5o1nXt9fXa+SUROwX1QBY80efxZtc IwtHWc1JmIA0O/3NRhPbnzSGZlD8xLanESiHn1+MwzgAI/YJ9mcXyl1HAlEfFpg6bOaR 66KlkaijWuUXfeBayHl5WivfvCMCX6ov7BeyY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=WaMrwHciPAETRSvhXhXwAGx5KtbYSUVKuFVkDY9duRs=; b=L6l9L8V1sYRUgZ1MpykwYa/8RfSTjOgMyQ8w5UlEEiZGoBTA9dRHRFZmB6GuNZpvva yeeUGzgukyYmGnIAr8gjTBMqHlt/DTfuC+Uc2u+McLA8Zw8TgGrxIEaKFyjgPvH1RMLH BI68q3K/j5zD7qAl3k9k8VXQfJ08qjqkCbtRqbLHYB66iba9KcO+qBJyzEuT+iOJ9vqU WPKdimdMw+6Ys1K5h6BMqPcIeZGTuYDXfR8mjpP+hjosMvY1Xzpsm5xJPXOASIy3GDmI kd+MheK8TEDmGa6kSlxvIwvB+EHH/xWGBOZWI51rdu6o+fHgFPpAL+iEBHgNY/naadht ApEw== X-Gm-Message-State: AJaThX677fHJbnnv8edb6OTTiYsE+axEDwmxbgNK/Vx6O09LRAxNKO/q voO/gihDZeeYzzQF8wnWb+4fvR8Oyfs= X-Google-Smtp-Source: AGs4zMZ7N1eLIS+nF1Ls0yGM5pDf2WiCThu8LI0xuuJDBP+64UzIzxmh2z5tOhJy9nr47jfnE0GpAg== X-Received: by 10.223.129.41 with SMTP id 38mr10296748wrm.57.1510652646106; Tue, 14 Nov 2017 01:44:06 -0800 (PST) Received: from cloudburst.twiddle.net (176.red-213-99-135.dynamicip.rima-tde.net. [213.99.135.176]) by smtp.gmail.com with ESMTPSA id h8sm5693217wme.30.2017.11.14.01.44.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Nov 2017 01:44:05 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Tue, 14 Nov 2017 10:42:03 +0100 Message-Id: <20171114094203.28030-1-richard.henderson@linaro.org> X-Mailer: git-send-email 2.13.6 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::243 Subject: [Qemu-devel] [PATCH] tcg: Record code_gen_buffer address for user-only memory helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When we handle a signal from a fault within a user-only memory helper, we cannot cpu_restore_state with the PC found within the signal frame. Use a TLS variable, helper_retaddr, to record the unwind start point to find the faulting guest insn. Reported-by: Peter Maydell Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée Tested-by: Alex Bennée --- Tested only with a silly test case -- int main() { int new = 1, old = 0; __atomic_compare_exchange((int *)0, &old, &new, 0, 0, 0); return old; } which even before the patch does not fail in the way Peter describes. As I post this, I remember in theory we should use __atomic_signal_fence after setting helper_retaddr, but as far as I know this is a no-op on all supported hosts. It might still generate a compiler barrier though, so it's worth considering. r~ --- accel/tcg/atomic_template.h | 32 +++++++++++++---- include/exec/cpu_ldst.h | 2 ++ include/exec/cpu_ldst_useronly_template.h | 14 ++++++-- accel/tcg/cputlb.c | 1 + accel/tcg/user-exec.c | 58 +++++++++++++++++++++++++------ 5 files changed, 87 insertions(+), 20 deletions(-) diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index b400b2a3d3..1c7c17526c 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -62,7 +62,9 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv EXTRA_ARGS) { DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; - return atomic_cmpxchg__nocheck(haddr, cmpv, newv); + DATA_TYPE ret = atomic_cmpxchg__nocheck(haddr, cmpv, newv); + ATOMIC_MMU_CLEANUP; + return ret; } #if DATA_SIZE >= 16 @@ -70,6 +72,7 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS) { DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP; __atomic_load(haddr, &val, __ATOMIC_RELAXED); + ATOMIC_MMU_CLEANUP; return val; } @@ -78,13 +81,16 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, { DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; __atomic_store(haddr, &val, __ATOMIC_RELAXED); + ATOMIC_MMU_CLEANUP; } #else ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val EXTRA_ARGS) { DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; - return atomic_xchg__nocheck(haddr, val); + DATA_TYPE ret = atomic_xchg__nocheck(haddr, val); + ATOMIC_MMU_CLEANUP; + return ret; } #define GEN_ATOMIC_HELPER(X) \ @@ -92,8 +98,10 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val EXTRA_ARGS) \ { \ DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; \ - return atomic_##X(haddr, val); \ -} \ + DATA_TYPE ret = atomic_##X(haddr, val); \ + ATOMIC_MMU_CLEANUP; \ + return ret; \ +} GEN_ATOMIC_HELPER(fetch_add) GEN_ATOMIC_HELPER(fetch_and) @@ -123,7 +131,9 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv EXTRA_ARGS) { DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; - return BSWAP(atomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv))); + DATA_TYPE ret = atomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv)); + ATOMIC_MMU_CLEANUP; + return BSWAP(ret); } #if DATA_SIZE >= 16 @@ -131,6 +141,7 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS) { DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP; __atomic_load(haddr, &val, __ATOMIC_RELAXED); + ATOMIC_MMU_CLEANUP; return BSWAP(val); } @@ -140,13 +151,16 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; val = BSWAP(val); __atomic_store(haddr, &val, __ATOMIC_RELAXED); + ATOMIC_MMU_CLEANUP; } #else ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val EXTRA_ARGS) { DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; - return BSWAP(atomic_xchg__nocheck(haddr, BSWAP(val))); + ABI_TYPE ret = atomic_xchg__nocheck(haddr, BSWAP(val)); + ATOMIC_MMU_CLEANUP; + return BSWAP(ret); } #define GEN_ATOMIC_HELPER(X) \ @@ -154,7 +168,9 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val EXTRA_ARGS) \ { \ DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP; \ - return BSWAP(atomic_##X(haddr, BSWAP(val))); \ + DATA_TYPE ret = atomic_##X(haddr, BSWAP(val)); \ + ATOMIC_MMU_CLEANUP; \ + return BSWAP(ret); \ } GEN_ATOMIC_HELPER(fetch_and) @@ -180,6 +196,7 @@ ABI_TYPE ATOMIC_NAME(fetch_add)(CPUArchState *env, target_ulong addr, sto = BSWAP(ret + val); ldn = atomic_cmpxchg__nocheck(haddr, ldo, sto); if (ldn == ldo) { + ATOMIC_MMU_CLEANUP; return ret; } ldo = ldn; @@ -198,6 +215,7 @@ ABI_TYPE ATOMIC_NAME(add_fetch)(CPUArchState *env, target_ulong addr, sto = BSWAP(ret); ldn = atomic_cmpxchg__nocheck(haddr, ldo, sto); if (ldn == ldo) { + ATOMIC_MMU_CLEANUP; return ret; } ldo = ldn; diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 6eb5fe80dc..191f2e962a 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -76,6 +76,8 @@ #if defined(CONFIG_USER_ONLY) +extern __thread uintptr_t helper_retaddr; + /* In user-only mode we provide only the _code and _data accessors. */ #define MEMSUFFIX _data diff --git a/include/exec/cpu_ldst_useronly_template.h b/include/exec/cpu_ldst_useronly_template.h index 7b8c7c506e..c168f31bba 100644 --- a/include/exec/cpu_ldst_useronly_template.h +++ b/include/exec/cpu_ldst_useronly_template.h @@ -73,7 +73,11 @@ glue(glue(glue(cpu_ld, USUFFIX), MEMSUFFIX), _ra)(CPUArchState *env, target_ulong ptr, uintptr_t retaddr) { - return glue(glue(cpu_ld, USUFFIX), MEMSUFFIX)(env, ptr); + RES_TYPE ret; + helper_retaddr = retaddr; + ret = glue(glue(cpu_ld, USUFFIX), MEMSUFFIX)(env, ptr); + helper_retaddr = 0; + return ret; } #if DATA_SIZE <= 2 @@ -93,7 +97,11 @@ glue(glue(glue(cpu_lds, SUFFIX), MEMSUFFIX), _ra)(CPUArchState *env, target_ulong ptr, uintptr_t retaddr) { - return glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(env, ptr); + int ret; + helper_retaddr = retaddr; + ret = glue(glue(cpu_lds, SUFFIX), MEMSUFFIX)(env, ptr); + helper_retaddr = 0; + return ret; } #endif @@ -116,7 +124,9 @@ glue(glue(glue(cpu_st, SUFFIX), MEMSUFFIX), _ra)(CPUArchState *env, RES_TYPE v, uintptr_t retaddr) { + helper_retaddr = retaddr; glue(glue(cpu_st, SUFFIX), MEMSUFFIX)(env, ptr, v); + helper_retaddr = 0; } #endif diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index a23919c3a8..d071ca4d14 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1041,6 +1041,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, #define ATOMIC_NAME(X) \ HELPER(glue(glue(glue(atomic_ ## X, SUFFIX), END), _mmu)) #define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, oi, retaddr) +#define ATOMIC_MMU_CLEANUP do { } while (0) #define DATA_SIZE 1 #include "atomic_template.h" diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 492ea0826c..0324ba8ad1 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -39,6 +39,8 @@ #include #endif +__thread uintptr_t helper_retaddr; + //#define DEBUG_SIGNAL /* exit the current TB from a signal handler. The host registers are @@ -62,6 +64,27 @@ static inline int handle_cpu_signal(uintptr_t pc, unsigned long address, CPUClass *cc; int ret; + /* We must handle PC addresses from two different sources: + * a call return address and a signal frame address. + * + * Within cpu_restore_state_from_tb we assume the former and adjust + * the address by -GETPC_ADJ so that the address is within the call + * insn so that addr does not accidentally match the beginning of the + * next guest insn. + * + * However, when the PC comes from the signal frame, it points to + * the actual faulting host insn and not a call insn. Subtracting + * GETPC_ADJ in that case may accidentally match the previous guest insn. + * + * So for the later case, adjust forward to compensate for what + * will be done later by cpu_restore_state_from_tb. + */ + if (helper_retaddr) { + pc = helper_retaddr; + } else { + pc += GETPC_ADJ; + } + /* For synchronous signals we expect to be coming from the vCPU * thread (so current_cpu should be valid) and either from running * code or during translation which can fault as we cross pages. @@ -84,21 +107,24 @@ static inline int handle_cpu_signal(uintptr_t pc, unsigned long address, switch (page_unprotect(h2g(address), pc)) { case 0: /* Fault not caused by a page marked unwritable to protect - * cached translations, must be the guest binary's problem + * cached translations, must be the guest binary's problem. */ break; case 1: /* Fault caused by protection of cached translation; TBs - * invalidated, so resume execution + * invalidated, so resume execution. Retain helper_retaddr + * for a possible second fault. */ return 1; case 2: /* Fault caused by protection of cached translation, and the * currently executing TB was modified and must be exited - * immediately. + * immediately. Clear helper_retaddr for next execution. */ + helper_retaddr = 0; cpu_exit_tb_from_sighandler(cpu, old_set); - g_assert_not_reached(); + /* NORETURN */ + default: g_assert_not_reached(); } @@ -112,17 +138,25 @@ static inline int handle_cpu_signal(uintptr_t pc, unsigned long address, /* see if it is an MMU fault */ g_assert(cc->handle_mmu_fault); ret = cc->handle_mmu_fault(cpu, address, is_write, MMU_USER_IDX); + + if (ret == 0) { + /* The MMU fault was handled without causing real CPU fault. + * Retain helper_retaddr for a possible second fault. + */ + return 1; + } + + /* All other paths lead to cpu_exit; clear helper_retaddr + * for next execution. + */ + helper_retaddr = 0; + if (ret < 0) { return 0; /* not an MMU fault */ } - if (ret == 0) { - return 1; /* the MMU fault was handled without causing real CPU fault */ - } - /* Now we have a real cpu fault. Since this is the exact location of - * the exception, we must undo the adjustment done by cpu_restore_state - * for handling call return addresses. */ - cpu_restore_state(cpu, pc + GETPC_ADJ); + /* Now we have a real cpu fault. */ + cpu_restore_state(cpu, pc); sigprocmask(SIG_SETMASK, old_set, NULL); cpu_loop_exit(cpu); @@ -585,11 +619,13 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, if (unlikely(addr & (size - 1))) { cpu_loop_exit_atomic(ENV_GET_CPU(env), retaddr); } + helper_retaddr = retaddr; return g2h(addr); } /* Macro to call the above, with local variables from the use context. */ #define ATOMIC_MMU_LOOKUP atomic_mmu_lookup(env, addr, DATA_SIZE, GETPC()) +#define ATOMIC_MMU_CLEANUP do { helper_retaddr = 0; } while (0) #define ATOMIC_NAME(X) HELPER(glue(glue(atomic_ ## X, SUFFIX), END)) #define EXTRA_ARGS