From patchwork Mon Jul 2 16:05:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938023 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="C2Md0bNO"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KBvn4BsXz9s4Z for ; Tue, 3 Jul 2018 02:06:45 +1000 (AEST) Received: from localhost ([::1]:33969 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Lf-0004fc-6I for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:06:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38397) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Ks-0004dz-Q1 for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Kp-0005OF-A3 for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:54 -0400 Received: from mail-pf0-x229.google.com ([2607:f8b0:400e:c00::229]:43876) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Kp-0005O1-2C for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:51 -0400 Received: by mail-pf0-x229.google.com with SMTP id y8-v6so7719193pfm.10 for ; Mon, 02 Jul 2018 09:05:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IGwmuLsh0l2gOpw4+GfJn6vetrwThkU6G7pxtiS16GI=; b=C2Md0bNOg8HAVyFm49dK5PPHCcxs7mTOgcPZmIGUuagFiirOaGPQZ4sHEsTVkWqjFC Z9zD5Hh6eFppqogCGYbF05nK9RbMeCon0/4ahhI/6yOzoTP+rSLAIoEvalrBI/IRJiFY tZcnQWMoCdC5swHleyYK0AWAK7V5y9RxSItgo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IGwmuLsh0l2gOpw4+GfJn6vetrwThkU6G7pxtiS16GI=; b=AyL2f1VNCbCw1dDYSaWoNl8Zw+itiLtxjbgnAVzsGHJoXe7aGQ8GPixg78rhUVgAQZ kTWgJDiwGHXxbj9go4HNPuIs+q9DYFvMsNk3S9AQ8DeOreKUppwJiXFiib6LpPfIbAqu BjlRmbw4C/v0+vWwvHWvtLqI5LUrf52dNy+XZsDYRu6vxDRDPZ6PlNXFJWSesi769bO2 qOCsUec5HMCGIRGFGJYgv4wlybNUU8JxFPb9YQdkapBwzrrcQnDeTTqXbp3pdcxF3hZp +kdb8tVSQnKFUemb/n3UNBisp+b8WOkqWqiZ4e9uS4rqFmddHK1Efb1R1vbTMAwT2VgC n4GA== X-Gm-Message-State: APt69E0gz+jlQO1BhB1/lIn0WcLtT4a7K1WF5K7DmL8/rhm1TJ86Aklp swui0427VXnQN9Y/TBeHqEx4eTAxFbc= X-Google-Smtp-Source: AAOMgpdUqQN8iVewX6QFIekakt4/l+mx1rk1xgHfHIl1uu/jxoLwVGjGUiTW1x26nnQF4zJrE0O6pA== X-Received: by 2002:a62:b20c:: with SMTP id x12-v6mr17059845pfe.64.1530547549826; Mon, 02 Jul 2018 09:05:49 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:41 -0700 Message-Id: <20180702160546.31969-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::229 Subject: [Qemu-devel] [PULL 1/6] translate-all: fix locking of TBs whose two pages share the same physical page X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" Commit 0b5c91f ("translate-all: use per-page locking in !user-mode", 2018-06-15) introduced per-page locking. It assumed that the physical pages corresponding to a TB (at most two pages) are always distinct, which is wrong. For instance, an xtensa test provided by Max Filippov is broken by the commit, since the test maps two virtual pages to the same physical page: virt1: 7fff, virt2: 8000 phys1 6000fff, phys2 6000000 Fix it by removing the assumption from page_lock_pair. If the two physical page addresses are equal, we only lock the PageDesc once. Note that the two callers of page_lock_pair, namely page_unlock_tb and tb_link_page, are also updated so that we do not try to unlock the same PageDesc twice. Fixes: 0b5c91f74f3c83a36f37740969df8c775c997e69 Reported-by: Max Filippov Tested-by: Max Filippov Tested-by: Philippe Mathieu-Daudé Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Message-Id: <1529944302-14186-1-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson --- accel/tcg/translate-all.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index e8228bf3e6..170b95793f 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -669,9 +669,15 @@ static inline void page_lock_tb(const TranslationBlock *tb) static inline void page_unlock_tb(const TranslationBlock *tb) { - page_unlock(page_find(tb->page_addr[0] >> TARGET_PAGE_BITS)); + PageDesc *p1 = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS); + + page_unlock(p1); if (unlikely(tb->page_addr[1] != -1)) { - page_unlock(page_find(tb->page_addr[1] >> TARGET_PAGE_BITS)); + PageDesc *p2 = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS); + + if (p2 != p1) { + page_unlock(p2); + } } } @@ -850,22 +856,34 @@ static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1, PageDesc **ret_p2, tb_page_addr_t phys2, int alloc) { PageDesc *p1, *p2; + tb_page_addr_t page1; + tb_page_addr_t page2; assert_memory_lock(); - g_assert(phys1 != -1 && phys1 != phys2); - p1 = page_find_alloc(phys1 >> TARGET_PAGE_BITS, alloc); + g_assert(phys1 != -1); + + page1 = phys1 >> TARGET_PAGE_BITS; + page2 = phys2 >> TARGET_PAGE_BITS; + + p1 = page_find_alloc(page1, alloc); if (ret_p1) { *ret_p1 = p1; } if (likely(phys2 == -1)) { page_lock(p1); return; + } else if (page1 == page2) { + page_lock(p1); + if (ret_p2) { + *ret_p2 = p1; + } + return; } - p2 = page_find_alloc(phys2 >> TARGET_PAGE_BITS, alloc); + p2 = page_find_alloc(page2, alloc); if (ret_p2) { *ret_p2 = p2; } - if (phys1 < phys2) { + if (page1 < page2) { page_lock(p1); page_lock(p2); } else { @@ -1623,7 +1641,7 @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc, tb = existing_tb; } - if (p2) { + if (p2 && p2 != p) { page_unlock(p2); } page_unlock(p); From patchwork Mon Jul 2 16:05:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="M7al8khq"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KBwk5b2Bz9s4Z for ; Tue, 3 Jul 2018 02:07:34 +1000 (AEST) Received: from localhost ([::1]:33980 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1MS-0005Kr-Ap for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:07:32 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38396) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Ks-0004dy-PZ for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Kr-0005P1-5E for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:54 -0400 Received: from mail-pf0-x230.google.com ([2607:f8b0:400e:c00::230]:33241) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Kq-0005Ok-Sn for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:53 -0400 Received: by mail-pf0-x230.google.com with SMTP id b17-v6so7726945pfi.0 for ; Mon, 02 Jul 2018 09:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VgOqH893kHg6WQZS2vwvDG+LDV6FGymw5UhSUPXJ9D8=; b=M7al8khqD+s3eG1CifM+uA0fIYWWAj+N72hc9Q2yaxtQrDqtOU1TMaychQc4vUaEb5 pjSz+PIkh1p5RgxWKdlJvCwIP/zdrzLht7TeM53jr4xTOE9rnIDBzbnqowkXJbDjVbXH 8YYHx0d6MZxAEowYcgl9PYiD/9Q74ZECxGa7U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VgOqH893kHg6WQZS2vwvDG+LDV6FGymw5UhSUPXJ9D8=; b=fkQYJ2ZYZt0Mxpo2adC0jYEZstEKofhCNS5M+jKGaTD84f7fmglLrH4YI2B1RirptP CcGBrk05MfVdAg0/qszC+a2dNAu3jZSAtjRVfihhA8sXLI+o8LEoPqwtyZBn4kGrORmp exA2E/LBGusVD5dK1N8tu5yPzkTWN9ycj1LsSwL9g6fcx64iwhcAk8vQILaUiJOAuNA5 mBOpcM4SAGDZ0HmBfK5SNHKqcXNfYhxVmDM262XN9J1PljH/Oj6+kZQi4aKhvOsRW2Fa xFEMQFUbBSWg/BD6t196ZCgVY1yqFeeGLzJdsgyrWlPEcb7KzA4vtbVXTJ7M2dEwEylI ZYWg== X-Gm-Message-State: APt69E2bn9CcyKbJneqrqK+klu+KjC7KFj0b47Z/3JoBbTEZNuZmP+ea rln4EQr9fxfEAukl3k9PkKv+WxX1BFU= X-Google-Smtp-Source: AAOMgpeQkmJ1I4ha9ylxsIhAuewg4YglhFnHI49JtQw450sqibQs2pk0oAka7Q/UjvVZ9kH1WvxhbA== X-Received: by 2002:a62:700a:: with SMTP id l10-v6mr7481319pfc.71.1530547551624; Mon, 02 Jul 2018 09:05:51 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.49 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:42 -0700 Message-Id: <20180702160546.31969-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::230 Subject: [Qemu-devel] [PULL 2/6] tcg: Define and use new tlb_hit() and tlb_hit_page() functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peter Maydell The condition to check whether an address has hit against a particular TLB entry is not completely trivial. We do this in various places, and in fact in one place (get_page_addr_code()) we have got the condition wrong. Abstract it out into new tlb_hit() and tlb_hit_page() inline functions (one for a known-page-aligned address and one for an arbitrary address), and use them in all the places where we had the condition correct. This is a no-behaviour-change patch; we leave fixing the buggy code in get_page_addr_code() to a subsequent patch. Reviewed-by: Richard Henderson Signed-off-by: Peter Maydell Message-Id: <20180629162122.19376-2-peter.maydell@linaro.org> Signed-off-by: Richard Henderson --- accel/tcg/softmmu_template.h | 16 ++++++---------- include/exec/cpu-all.h | 23 +++++++++++++++++++++++ include/exec/cpu_ldst.h | 3 +-- accel/tcg/cputlb.c | 15 +++++---------- 4 files changed, 35 insertions(+), 22 deletions(-) diff --git a/accel/tcg/softmmu_template.h b/accel/tcg/softmmu_template.h index c47591c970..badbf14880 100644 --- a/accel/tcg/softmmu_template.h +++ b/accel/tcg/softmmu_template.h @@ -123,8 +123,7 @@ WORD_TYPE helper_le_ld_name(CPUArchState *env, target_ulong addr, } /* If the TLB entry is for a different page, reload and try again. */ - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { if (!VICTIM_TLB_HIT(ADDR_READ, addr)) { tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, READ_ACCESS_TYPE, mmu_idx, retaddr); @@ -191,8 +190,7 @@ WORD_TYPE helper_be_ld_name(CPUArchState *env, target_ulong addr, } /* If the TLB entry is for a different page, reload and try again. */ - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { if (!VICTIM_TLB_HIT(ADDR_READ, addr)) { tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, READ_ACCESS_TYPE, mmu_idx, retaddr); @@ -286,8 +284,7 @@ void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, } /* If the TLB entry is for a different page, reload and try again. */ - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { if (!VICTIM_TLB_HIT(addr_write, addr)) { tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); @@ -322,7 +319,7 @@ void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, page2 = (addr + DATA_SIZE) & TARGET_PAGE_MASK; index2 = (page2 >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); tlb_addr2 = env->tlb_table[mmu_idx][index2].addr_write; - if (page2 != (tlb_addr2 & (TARGET_PAGE_MASK | TLB_INVALID_MASK)) + if (!tlb_hit_page(tlb_addr2, page2) && !VICTIM_TLB_HIT(addr_write, page2)) { tlb_fill(ENV_GET_CPU(env), page2, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); @@ -364,8 +361,7 @@ void helper_be_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, } /* If the TLB entry is for a different page, reload and try again. */ - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { if (!VICTIM_TLB_HIT(addr_write, addr)) { tlb_fill(ENV_GET_CPU(env), addr, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); @@ -400,7 +396,7 @@ void helper_be_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, page2 = (addr + DATA_SIZE) & TARGET_PAGE_MASK; index2 = (page2 >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); tlb_addr2 = env->tlb_table[mmu_idx][index2].addr_write; - if (page2 != (tlb_addr2 & (TARGET_PAGE_MASK | TLB_INVALID_MASK)) + if (!tlb_hit_page(tlb_addr2, page2) && !VICTIM_TLB_HIT(addr_write, page2)) { tlb_fill(ENV_GET_CPU(env), page2, DATA_SIZE, MMU_DATA_STORE, mmu_idx, retaddr); diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index 7338f57062..117d2fbbca 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -339,6 +339,29 @@ CPUArchState *cpu_copy(CPUArchState *env); #define TLB_FLAGS_MASK (TLB_INVALID_MASK | TLB_NOTDIRTY | TLB_MMIO \ | TLB_RECHECK) +/** + * tlb_hit_page: return true if page aligned @addr is a hit against the + * TLB entry @tlb_addr + * + * @addr: virtual address to test (must be page aligned) + * @tlb_addr: TLB entry address (a CPUTLBEntry addr_read/write/code value) + */ +static inline bool tlb_hit_page(target_ulong tlb_addr, target_ulong addr) +{ + return addr == (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK)); +} + +/** + * tlb_hit: return true if @addr is a hit against the TLB entry @tlb_addr + * + * @addr: virtual address to test (need not be page aligned) + * @tlb_addr: TLB entry address (a CPUTLBEntry addr_read/write/code value) + */ +static inline bool tlb_hit(target_ulong tlb_addr, target_ulong addr) +{ + return tlb_hit_page(tlb_addr, addr & TARGET_PAGE_MASK); +} + void dump_exec_info(FILE *f, fprintf_function cpu_fprintf); void dump_opcount_info(FILE *f, fprintf_function cpu_fprintf); #endif /* !CONFIG_USER_ONLY */ diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 5de8c8a5af..0f2cb717b1 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -422,8 +422,7 @@ static inline void *tlb_vaddr_to_host(CPUArchState *env, target_ulong addr, g_assert_not_reached(); } - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { /* TLB entry is for a different page */ return NULL; } diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index eebe97dabb..adb711963b 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -239,12 +239,9 @@ void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) { - if (addr == (tlb_entry->addr_read & - (TARGET_PAGE_MASK | TLB_INVALID_MASK)) || - addr == (tlb_entry->addr_write & - (TARGET_PAGE_MASK | TLB_INVALID_MASK)) || - addr == (tlb_entry->addr_code & - (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (tlb_hit_page(tlb_entry->addr_read, addr) || + tlb_hit_page(tlb_entry->addr_write, addr) || + tlb_hit_page(tlb_entry->addr_code, addr)) { memset(tlb_entry, -1, sizeof(*tlb_entry)); } } @@ -1046,8 +1043,7 @@ void probe_write(CPUArchState *env, target_ulong addr, int size, int mmu_idx, int index = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); target_ulong tlb_addr = env->tlb_table[mmu_idx][index].addr_write; - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { /* TLB entry is for a different page */ if (!VICTIM_TLB_HIT(addr_write, addr)) { tlb_fill(ENV_GET_CPU(env), addr, size, MMU_DATA_STORE, @@ -1091,8 +1087,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, } /* Check TLB entry and enforce page permissions. */ - if ((addr & TARGET_PAGE_MASK) - != (tlb_addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK))) { + if (!tlb_hit(tlb_addr, addr)) { if (!VICTIM_TLB_HIT(addr_write, addr)) { tlb_fill(ENV_GET_CPU(env), addr, 1 << s_bits, MMU_DATA_STORE, mmu_idx, retaddr); From patchwork Mon Jul 2 16:05:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="gX0te6vo"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KBzW2qK4z9s2R for ; Tue, 3 Jul 2018 02:09:59 +1000 (AEST) Received: from localhost ([::1]:33994 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1On-0007MK-2q for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:09:57 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38418) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Ku-0004eu-8P for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Kt-0005QP-Dj for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:56 -0400 Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:37146) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Kt-0005Pj-8J for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:55 -0400 Received: by mail-pg0-x242.google.com with SMTP id n15-v6so4107346pgv.4 for ; Mon, 02 Jul 2018 09:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Za3snJKKi+Qz2FZHbHN1FIGuHvykZW/vRpgWAxCoJDI=; b=gX0te6voMZKYtOtnTunt1wAT2EwR6oqps3FnNr8qqS3ZOxl6fqBM/+DpbZ0E12bsk3 Bk8RM2T+ii1dQN/by82TFzp3c1KA6NgVsf2kJ+yHIDEgFphTuJaJAa/l3z16TYnUaM/D AU5gpfwomtWuBoOvZ2qNe8hk+0ZbAvCpZpsLs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Za3snJKKi+Qz2FZHbHN1FIGuHvykZW/vRpgWAxCoJDI=; b=aQnPQzwprR9d5cNA51om+03agwnnQCtP8K1ygxJuCACppUU4j3A+HpisjUYBTrjbNk Is1jYJxarb2R/WyeKP/I2XAe8x8U2rYwIeDFgCULTcqhEQgnW8gxaPT9TmJfIfC7Q8Rd IaiY+CPPJvRT7qzmuBkhMNlU6goVZTnwK/UqenUSWPJO9XPSWsXLoeZ+RzpB/X8rNyIO 9o4Ec50VUpRBbn0NVfsGhL0AKnrBxCsDORTdEowC0u5QkxKGo9Bl+4a9VwFjodB+iQAs kMIlEjcyPaTjr8nojcaG5HVQ+FrD6TDUuFfCnsFCEZP9TEgoZVziDje4njHX4nKo3qsn 9O7A== X-Gm-Message-State: APt69E1NccDDj/MVNBVISYBdGRxSQcv5Lcb1/6a2JOmWij29zpbNv7Wz uanZRQwy57jOJ1YuHocLRmRjJVarrjo= X-Google-Smtp-Source: AAOMgpdLv9+rzbmR2owX4qsbITbfPOv7q2VblfKYV6aGStOfqzM4LzoTsT8sCE2/gYtd1KvsD1gy5Q== X-Received: by 2002:a62:6882:: with SMTP id d124-v6mr25951637pfc.122.1530547553723; Mon, 02 Jul 2018 09:05:53 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.51 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:43 -0700 Message-Id: <20180702160546.31969-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::242 Subject: [Qemu-devel] [PULL 3/6] accel/tcg: Correct "is this a TLB miss" check in get_page_addr_code() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peter Maydell In commit 71b9a45330fe220d1 we changed the condition we use to determine whether we need to refill the TLB in get_page_addr_code() to if (unlikely(env->tlb_table[mmu_idx][index].addr_code != (addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK)))) { This isn't the right check (it will falsely fail if the input addr happens to have the low bit corresponding to TLB_INVALID_MASK set, for instance). Replace it with a use of the new tlb_hit() function, which is the correct test. Reviewed-by: Richard Henderson Signed-off-by: Peter Maydell Message-Id: <20180629162122.19376-3-peter.maydell@linaro.org> Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index adb711963b..3ae1198c24 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -957,8 +957,7 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, target_ulong addr) index = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); mmu_idx = cpu_mmu_index(env, true); - if (unlikely(env->tlb_table[mmu_idx][index].addr_code != - (addr & (TARGET_PAGE_MASK | TLB_INVALID_MASK)))) { + if (unlikely(!tlb_hit(env->tlb_table[mmu_idx][index].addr_code, addr))) { if (!VICTIM_TLB_HIT(addr_read, addr)) { tlb_fill(ENV_GET_CPU(env), addr, 0, MMU_INST_FETCH, mmu_idx, 0); } From patchwork Mon Jul 2 16:05:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938026 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="SGVPSy1w"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KByt4CnCz9s2R for ; Tue, 3 Jul 2018 02:09:26 +1000 (AEST) Received: from localhost ([::1]:33989 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1OG-0006w8-7P for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:09:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38431) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Kv-0004gH-Pw for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:06:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Ku-0005RG-RM for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:57 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:44420) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Ku-0005Qk-LF for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:56 -0400 Received: by mail-pf0-x244.google.com with SMTP id j3-v6so7715719pfh.11 for ; Mon, 02 Jul 2018 09:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CFbsIeZbs3j7ab2Zy+DvzMawS9IkD3pOZfIsc+sHoIc=; b=SGVPSy1wikf9SgKwbCepFRIKax2PVh63yNZvipA8xiGYkjv3HuY1gUuAq3JKsC0fEZ XRfL7ZV+F5m7uVIknOV1UcKORk7IaU5PrFFXJyAysD7x7hSx8SNI/E8PpKGBjqLMUkGM 75oog7OXQ6Xo0wOaT7Cok+QHf04GMwb4ObFTU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CFbsIeZbs3j7ab2Zy+DvzMawS9IkD3pOZfIsc+sHoIc=; b=sUhDJpuMTHL73os53vriywRwEWv4LIRdbxkhyTieORrCY+OY2aIexlGkK852Z/hA6o beYp6nBa1K5F9BuSxOGlQQbgfxMXzNazLTKJH/WoF6zYM4NnKA9fkvNZ0xawcmZPHrwY FV3y8Y3EcGfEE/fJysZ8nEvaQGt0afH3uYsOnFUe8+M4q/2MQuSw5fwXhiHTsF1Lc1Mq wC/ccwXdAtfeuO1I3YVqqRucPw/9+Y5nEkbr+QmyDzfR9UNzTJCo3AnWeY0YJVWkc4xy Mq83eC58b88mAQOCWDCwSOfp8LtJzgu4t1UgfaHuacyQI9UK5oLBAUc0WWxRy6DqBwDB FvxQ== X-Gm-Message-State: APt69E3QDAhyZC2W/3EaKoeouH1sIOakIwU5TmNx4pWcneXR99sEM9l4 ZnjPv6N9sufWAK0DgjwG9tcg70EyB2A= X-Google-Smtp-Source: AAOMgpfC/5yif9A3kyy5XBr7eJQ1r2QhOu8E/uD1UjCSlZ+00TxJfu0JP+o1cXeduqOaMeHp+WkNRg== X-Received: by 2002:a63:ad46:: with SMTP id y6-v6mr14905202pgo.144.1530547555498; Mon, 02 Jul 2018 09:05:55 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.53 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:44 -0700 Message-Id: <20180702160546.31969-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PULL 4/6] accel/tcg: Don't treat invalid TLB entries as needing recheck X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peter Maydell In get_page_addr_code() when we check whether the TLB entry is marked as TLB_RECHECK, we should not go down that code path if the TLB entry is not valid at all (ie the TLB_INVALID bit is set). Tested-by: Laurent Vivier Reported-by: Laurent Vivier Reviewed-by: Richard Henderson Signed-off-by: Peter Maydell Message-Id: <20180629161731.16239-1-peter.maydell@linaro.org> Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 3ae1198c24..cc90a5fe92 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -963,7 +963,8 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env, target_ulong addr) } } - if (unlikely(env->tlb_table[mmu_idx][index].addr_code & TLB_RECHECK)) { + if (unlikely((env->tlb_table[mmu_idx][index].addr_code & + (TLB_RECHECK | TLB_INVALID_MASK)) == TLB_RECHECK)) { /* * This is a TLB_RECHECK access, where the MMU protection * covers a smaller range than a target page, and we must From patchwork Mon Jul 2 16:05:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938028 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="Mm9FA4lS"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KC135DTNz9s2R for ; Tue, 3 Jul 2018 02:11:19 +1000 (AEST) Received: from localhost ([::1]:34003 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Q5-0008H5-Bt for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:11:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38450) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Ky-0004gU-0J for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:06:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Kw-0005Sg-QN for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:59 -0400 Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:34857) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Kw-0005SF-IT for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:05:58 -0400 Received: by mail-pg0-x241.google.com with SMTP id i7-v6so7345960pgp.2 for ; Mon, 02 Jul 2018 09:05:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LjSXEo7wznIsBtYxIKACJ2dssod61S6F92h6i+lGTPo=; b=Mm9FA4lSL7xljPo6oTntTchB3OgJrW0vIQWHi3hz3xmartekLZeq/vfEGdxwAauTqv A9o1+h2hieG26cBQwxF1tboIkaNvheYS7VqqjqDDkHTGKQKx0irXhakAsWIm/yid3hac GpxR8gdQOUfHTIgdQskv5SQRln0I7JKQ2LCHc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LjSXEo7wznIsBtYxIKACJ2dssod61S6F92h6i+lGTPo=; b=OzqaLAejrTPuAvFWGEjoXTJWmfGiSk1fLamcrdEedw1SlRs2+BD3T66Eh1OYaX2Wko nUPPhizSuOGScUhZg2kbjG6hJatGaVCgeryef6ajLKTGIqo/oSN7N/zASlQKJweQGbDr LZA3c9NWOi/1hFawvKwgE1SIN17lqhdUGKAmTVAtp/E0BMtuq6lcJF9IhehZXf577GJQ 8wSBQBgzd0dExWqB3UAYfBcyt+BooKdb2JZXKslN3wmi3qd8AtdqmM9QcuYZHDu+XITi 2vD67QajvI0DMUjYxhAFC3hkVFhwZxszHOZypYc0J/N4wX2hMvrC53CKtowXYrJYRVuz ejvQ== X-Gm-Message-State: APt69E1WrOC696nibFs60d/qZdwJLQXPLKjGinwNwNRc7y3nVBpJ+i5n kMKlGg+q2v8gKu5e/W3Ah0vnO7HjnUg= X-Google-Smtp-Source: AAOMgpfbLyJzSTvxNWmVzAU6bgF8G2/BI9ylVX/Xiu8pO+8CgUm5KWI1HYO/u/8DjwrCD2vGwJeSrg== X-Received: by 2002:a63:6e0a:: with SMTP id j10-v6mr5689753pgc.321.1530547557347; Mon, 02 Jul 2018 09:05:57 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:45 -0700 Message-Id: <20180702160546.31969-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::241 Subject: [Qemu-devel] [PULL 5/6] accel/tcg: Avoid caching overwritten tlb entries X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When installing a TLB entry, remove any cached version of the same page in the VTLB. If the existing TLB entry matches, do not copy into the VTLB, but overwrite it. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 63 ++++++++++++++++++++++++++-------------------- 1 file changed, 36 insertions(+), 27 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index cc90a5fe92..20c147d655 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -235,17 +235,30 @@ void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, async_safe_run_on_cpu(src_cpu, fn, RUN_ON_CPU_HOST_INT(idxmap)); } - - -static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) +static inline bool tlb_hit_page_anyprot(CPUTLBEntry *tlb_entry, + target_ulong page) { - if (tlb_hit_page(tlb_entry->addr_read, addr) || - tlb_hit_page(tlb_entry->addr_write, addr) || - tlb_hit_page(tlb_entry->addr_code, addr)) { + return tlb_hit_page(tlb_entry->addr_read, page) || + tlb_hit_page(tlb_entry->addr_write, page) || + tlb_hit_page(tlb_entry->addr_code, page); +} + +static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong page) +{ + if (tlb_hit_page_anyprot(tlb_entry, page)) { memset(tlb_entry, -1, sizeof(*tlb_entry)); } } +static inline void tlb_flush_vtlb_page(CPUArchState *env, int mmu_idx, + target_ulong page) +{ + int k; + for (k = 0; k < CPU_VTLB_SIZE; k++) { + tlb_flush_entry(&env->tlb_v_table[mmu_idx][k], page); + } +} + static void tlb_flush_page_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; @@ -271,14 +284,7 @@ static void tlb_flush_page_async_work(CPUState *cpu, run_on_cpu_data data) i = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { tlb_flush_entry(&env->tlb_table[mmu_idx][i], addr); - } - - /* check whether there are entries that need to be flushed in the vtlb */ - for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - int k; - for (k = 0; k < CPU_VTLB_SIZE; k++) { - tlb_flush_entry(&env->tlb_v_table[mmu_idx][k], addr); - } + tlb_flush_vtlb_page(env, mmu_idx, addr); } tb_flush_jmp_cache(cpu, addr); @@ -310,7 +316,6 @@ static void tlb_flush_page_by_mmuidx_async_work(CPUState *cpu, unsigned long mmu_idx_bitmap = addr_and_mmuidx & ALL_MMUIDX_BITS; int page = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); int mmu_idx; - int i; assert_cpu_is_self(cpu); @@ -320,11 +325,7 @@ static void tlb_flush_page_by_mmuidx_async_work(CPUState *cpu, for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { if (test_bit(mmu_idx, &mmu_idx_bitmap)) { tlb_flush_entry(&env->tlb_table[mmu_idx][page], addr); - - /* check whether there are vltb entries that need to be flushed */ - for (i = 0; i < CPU_VTLB_SIZE; i++) { - tlb_flush_entry(&env->tlb_v_table[mmu_idx][i], addr); - } + tlb_flush_vtlb_page(env, mmu_idx, addr); } } @@ -609,10 +610,9 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, target_ulong address; target_ulong code_address; uintptr_t addend; - CPUTLBEntry *te, *tv, tn; + CPUTLBEntry *te, tn; hwaddr iotlb, xlat, sz, paddr_page; target_ulong vaddr_page; - unsigned vidx = env->vtlb_index++ % CPU_VTLB_SIZE; int asidx = cpu_asidx_from_attrs(cpu, attrs); assert_cpu_is_self(cpu); @@ -654,19 +654,28 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, addend = (uintptr_t)memory_region_get_ram_ptr(section->mr) + xlat; } + /* Make sure there's no cached translation for the new page. */ + tlb_flush_vtlb_page(env, mmu_idx, vaddr_page); + code_address = address; iotlb = memory_region_section_get_iotlb(cpu, section, vaddr_page, paddr_page, xlat, prot, &address); index = (vaddr_page >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); te = &env->tlb_table[mmu_idx][index]; - /* do not discard the translation in te, evict it into a victim tlb */ - tv = &env->tlb_v_table[mmu_idx][vidx]; - /* addr_write can race with tlb_reset_dirty_range */ - copy_tlb_helper(tv, te, true); + /* + * Only evict the old entry to the victim tlb if it's for a + * different page; otherwise just overwrite the stale data. + */ + if (!tlb_hit_page_anyprot(te, vaddr_page)) { + unsigned vidx = env->vtlb_index++ % CPU_VTLB_SIZE; + CPUTLBEntry *tv = &env->tlb_v_table[mmu_idx][vidx]; - env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index]; + /* Evict the old entry into the victim tlb. */ + copy_tlb_helper(tv, te, true); + env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index]; + } /* refill the tlb */ /* From patchwork Mon Jul 2 16:05:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 938024 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.b="bklPN9Mo"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41KBvx1Bs2z9s2R for ; Tue, 3 Jul 2018 02:06:53 +1000 (AEST) Received: from localhost ([::1]:33976 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1Lm-0004nh-Pd for incoming@patchwork.ozlabs.org; Mon, 02 Jul 2018 12:06:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38477) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa1L3-0004kc-Og for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:06:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa1Ky-0005Tu-CG for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:06:05 -0400 Received: from mail-pg0-x230.google.com ([2607:f8b0:400e:c05::230]:37186) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fa1Ky-0005TO-6J for qemu-devel@nongnu.org; Mon, 02 Jul 2018 12:06:00 -0400 Received: by mail-pg0-x230.google.com with SMTP id n15-v6so4107458pgv.4 for ; Mon, 02 Jul 2018 09:06:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=EDg7+xQrWVFN2B7aGOJKgtLDfQ0mJ1smJERcqBGPq1Y=; b=bklPN9Mo1esbQfs5g1uySsYWn5iG7sSE/cyH6o3z1rQrTzIpqZMMMUq31BmrKYSFWH RjAyJpX5s0/t+UvELYAZEbwfkpMOyP/+gGqRLVbUi4HB+IzpJdG/fOrCmz6SOG+zhJSK T6U9i8AGBP8rKir9wl1PGdNfjC3PAkvsM0kd0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=EDg7+xQrWVFN2B7aGOJKgtLDfQ0mJ1smJERcqBGPq1Y=; b=DIZlahop0yJTvUwdW3u4m8Npd8oJMGYWIwS+4YOvQpkqghPpfeSyYC4ZTHUVPvjw+u 3VnGqRAotuDLThSaHJJFV3GzBNTgYZSi9igjDxVTfOyxRpU8jQ3z7BFETcKGrUpcsbvR Ax+h8HtVG3uKVI3gR9aIuaYhfYvTrAA4PncqpdSAOhQRuGtr61E3ITSRWTWEBJRjgU8M hOWmq8oUeno/ztmVFcwHIuNkiAp7YWKdgk/zH+EFeN39gKx2x9pmMeQMGJ18iRGMY8or PwGo4lEId0GwZnZ9+IQyfNoXt9yS3QXWEDbDo2m+FkXNIF8mHD+K7RiuphyQxF4sgPeK fnSg== X-Gm-Message-State: APt69E3ebXTWikB8HBT7c4URruJRGNAoohNhRqlD6uqNVyDyTktXBeZR NwmVxAESGTO0vBwIBttO1edhR/qDkeE= X-Google-Smtp-Source: ADUXVKL2mEoPbzmjb+tUZH/quuleuuM1KYT3VoeGoFPXlYi7DQKtmKTyvZ+V4dXsITOSOBgRj3rqTQ== X-Received: by 2002:a63:6004:: with SMTP id u4-v6mr22162315pgb.441.1530547559032; Mon, 02 Jul 2018 09:05:59 -0700 (PDT) Received: from cloudburst.twiddle.net (97-126-112-211.tukw.qwest.net. [97.126.112.211]) by smtp.gmail.com with ESMTPSA id e17-v6sm38107002pfd.15.2018.07.02.09.05.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 02 Jul 2018 09:05:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 2 Jul 2018 09:05:46 -0700 Message-Id: <20180702160546.31969-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180702160546.31969-1-richard.henderson@linaro.org> References: <20180702160546.31969-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::230 Subject: [Qemu-devel] [PULL 6/6] cpu: Assert asidx_from_attrs return value in range X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- include/qom/cpu.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/include/qom/cpu.h b/include/qom/cpu.h index cce2fd6acc..bd796579ee 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -620,11 +620,13 @@ static inline hwaddr cpu_get_phys_page_debug(CPUState *cpu, vaddr addr) static inline int cpu_asidx_from_attrs(CPUState *cpu, MemTxAttrs attrs) { CPUClass *cc = CPU_GET_CLASS(cpu); + int ret = 0; if (cc->asidx_from_attrs) { - return cc->asidx_from_attrs(cpu, attrs); + ret = cc->asidx_from_attrs(cpu, attrs); + assert(ret < cpu->num_ases && ret >= 0); } - return 0; + return ret; } #endif