From patchwork Mon Aug 7 23:52:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emilio Cota X-Patchwork-Id: 798968 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=braap.org header.i=@braap.org header.b="WaYoL+bn"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="mHPxH8RK"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xRDvd4MkDz9sPr for ; Tue, 8 Aug 2017 09:56:29 +1000 (AEST) Received: from localhost ([::1]:40051 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dersp-0000Hy-CE for incoming@patchwork.ozlabs.org; Mon, 07 Aug 2017 19:56:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47812) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1derpN-0005yb-HW for qemu-devel@nongnu.org; Mon, 07 Aug 2017 19:53:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1derpH-0005iw-9k for qemu-devel@nongnu.org; Mon, 07 Aug 2017 19:52:53 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:54627) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1derpH-0005iJ-3f for qemu-devel@nongnu.org; Mon, 07 Aug 2017 19:52:47 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id BC5292196B; Mon, 7 Aug 2017 19:52:46 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute4.internal (MEProxy); Mon, 07 Aug 2017 19:52:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=braap.org; h=cc :date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=5FI wsjnnxlQpL+6SEi8k+c4b91JHEc/SKgAXUJVLpeE=; b=WaYoL+bnPIX3y5qU1ul oqe/FhXzdQFfVI7roJnbAePLCAblL742aNflqT8gFB24xeiEZeMvtdwNCWRfwTxD kwNFOKFgPj61fWpTEj+J/2LTfblDxx/lXtvG4YgPNxzhKffwjPYgZ31CYUCAfn9W k0LcTqYCBJa2bHzliXY3A7/A= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc :x-sasl-enc; s=fm1; bh=5FIwsjnnxlQpL+6SEi8k+c4b91JHEc/SKgAXUJVLp eE=; b=mHPxH8RKnlZwQt3a8nWnOozdC+0WLS/4M4Ym/vrvRSWjzug0CemGKw3oQ DGMB81tgNPP/rw+0s2Pjx8OdIir8a6yiywMg2W21P6vHJMTDnX8PkjM/cKfaEnUd imypwCvXY1dqVG5N7ABCaNt2bvsWfNW0FtmAcHBzT864J3dwXR9zPVbpuScjRrVM 4TYZFiQ2CKXJ72GkvW2MiwuFlhsILtfDL01wuS1loN1x7n95a5R4ZxWnvpr/wTR9 EzcUTNbGEVE+a7b3Lz94OrIF9UeuahABGmXNrTOCT9DW5UgwsMRuj4m1JzBxsBDa mUnk5mK1oe34rVtaPO9+JlbE0qgxA== X-ME-Sender: X-Sasl-enc: Elo6Gl5iYGswtQZhR2vkxpEIXaqUo+A0xN0edYKZ+KJa 1502149966 Received: from localhost (flamenco.cs.columbia.edu [128.59.20.216]) by mail.messagingengine.com (Postfix) with ESMTPA id 7D3717E6AD; Mon, 7 Aug 2017 19:52:46 -0400 (EDT) From: "Emilio G. Cota" To: qemu-devel@nongnu.org Date: Mon, 7 Aug 2017 19:52:26 -0400 Message-Id: <1502149958-23381-11-git-send-email-cota@braap.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1502149958-23381-1-git-send-email-cota@braap.org> References: <1502149958-23381-1-git-send-email-cota@braap.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.111.4.25 Subject: [Qemu-devel] [PATCH 10/22] translate-all: iterate over TBs in a page with page_for_each_tb X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Richard Henderson Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This commit does several things, but to avoid churn I merged them all into the same commit. To wit: - Use uintptr_t instead of TranslationBlock * for the list of TBs in a page. Just like we did in (c37e6d7e "tcg: Use uintptr_t type for jmp_list_{next|first} fields of TB"), the rationale is the same: these are tagged pointers, not pointers. So use a more appropriate type. - Only check the least significant bit of the tagged pointers. Masking with 3/~3 is unnecessary and confusing. - Use the new tb_for_each_tagged macros to define page_for_each_tb(_safe), which improves readability. - Update tb_page_remove to use page_for_each_tb_safe(). In case there is a bug and we attempt to remove a TB that is not in the list, instead of segfaulting (since the list is NULL-terminated) we will reach g_assert_not_reached(). Signed-off-by: Emilio G. Cota --- include/exec/exec-all.h | 2 +- accel/tcg/translate-all.c | 57 ++++++++++++++++++++--------------------------- 2 files changed, 25 insertions(+), 34 deletions(-) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 32e97e1..26084c3 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -381,7 +381,7 @@ struct TranslationBlock { struct TranslationBlock *orig_tb; /* first and second physical page containing code. The lower bit of the pointer tells the index in page_next[] */ - struct TranslationBlock *page_next[2]; + uintptr_t page_next[2]; tb_page_addr_t page_addr[2]; /* The following data are used to directly call another TB from diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index 396c10c..8e5e5b2 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -104,7 +104,7 @@ typedef struct PageDesc { /* list of TBs intersecting this ram page */ - TranslationBlock *first_tb; + uintptr_t first_tb; #ifdef CONFIG_SOFTMMU /* in order to optimize self modifying code, we count the number of lookups we do to a given page to use a bitmap */ @@ -134,6 +134,12 @@ typedef struct PageDesc { n = (uintptr_t)*prev & 1, \ tb = (TranslationBlock *)(*prev & ~1)) +#define page_for_each_tb(pagedesc, tb, n) \ + tb_for_each_tagged((pagedesc)->first_tb, tb, n, page_next) + +#define page_for_each_tb_safe(pagedesc, tb, n, prev) \ + tb_for_each_tagged_safe((pagedesc)->first_tb, tb, n, page_next, prev) + /* In system mode we want L1_MAP to be based on ram offsets, while in user mode we want it to be based on virtual addresses. */ #if !defined(CONFIG_USER_ONLY) @@ -834,7 +840,7 @@ static void page_flush_tb_1(int level, void **lp) PageDesc *pd = *lp; for (i = 0; i < V_L2_SIZE; ++i) { - pd[i].first_tb = NULL; + pd[i].first_tb = (uintptr_t)NULL; invalidate_page_bitmap(pd + i); } } else { @@ -962,21 +968,19 @@ static void tb_page_check(void) #endif /* CONFIG_USER_ONLY */ -static inline void tb_page_remove(TranslationBlock **ptb, TranslationBlock *tb) +static inline void tb_page_remove(PageDesc *pd, TranslationBlock *tb) { TranslationBlock *tb1; + uintptr_t *prev; unsigned int n1; - for (;;) { - tb1 = *ptb; - n1 = (uintptr_t)tb1 & 3; - tb1 = (TranslationBlock *)((uintptr_t)tb1 & ~3); + page_for_each_tb_safe(pd, tb1, n1, prev) { if (tb1 == tb) { - *ptb = tb1->page_next[n1]; - break; + *prev = tb1->page_next[n1]; + return; } - ptb = &tb1->page_next[n1]; } + g_assert_not_reached(); } /* remove the TB from a list of TBs jumping to the n-th jump target of the TB */ @@ -1062,12 +1066,12 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr) /* remove the TB from the page list */ if (tb->page_addr[0] != page_addr) { p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } if (tb->page_addr[1] != -1 && tb->page_addr[1] != page_addr) { p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS); - tb_page_remove(&p->first_tb, tb); + tb_page_remove(p, tb); invalidate_page_bitmap(p); } @@ -1098,10 +1102,7 @@ static void build_page_bitmap(PageDesc *p) p->code_bitmap = bitmap_new(TARGET_PAGE_SIZE); - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + page_for_each_tb(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1116,7 +1117,6 @@ static void build_page_bitmap(PageDesc *p) tb_end = ((tb->pc + tb->size) & ~TARGET_PAGE_MASK); } bitmap_set(p->code_bitmap, tb_start, tb_end - tb_start); - tb = tb->page_next[n]; } } #endif @@ -1139,9 +1139,9 @@ static inline void tb_alloc_page(TranslationBlock *tb, p = page_find_alloc(page_addr >> TARGET_PAGE_BITS, 1); tb->page_next[n] = p->first_tb; #ifndef CONFIG_USER_ONLY - page_already_protected = p->first_tb != NULL; + page_already_protected = p->first_tb != (uintptr_t)NULL; #endif - p->first_tb = (TranslationBlock *)((uintptr_t)tb | n); + p->first_tb = (uintptr_t)tb | n; invalidate_page_bitmap(p); #if defined(CONFIG_USER_ONLY) @@ -1401,7 +1401,7 @@ void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end) void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, int is_cpu_write_access) { - TranslationBlock *tb, *tb_next; + TranslationBlock *tb; #if defined(TARGET_HAS_PRECISE_SMC) CPUState *cpu = current_cpu; CPUArchState *env = NULL; @@ -1434,11 +1434,7 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, /* we remove all the TBs in the range [start, end[ */ /* XXX: see if in some cases it could be faster to invalidate all the code */ - tb = p->first_tb; - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); - tb_next = tb->page_next[n]; + page_for_each_tb(p, tb, n) { /* NOTE: this is subtle as a TB may span two physical pages */ if (n == 0) { /* NOTE: tb_end may be after the end of the page, but @@ -1475,7 +1471,6 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, -1); } - tb = tb_next; } #if !defined(CONFIG_USER_ONLY) /* if no code remaining, no need to continue to use slow writes */ @@ -1572,18 +1567,15 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } tb_lock(); - tb = p->first_tb; #ifdef TARGET_HAS_PRECISE_SMC - if (tb && pc != 0) { + if (p->first_tb && pc != 0) { current_tb = tcg_tb_lookup(pc); } if (cpu != NULL) { env = cpu->env_ptr; } #endif - while (tb != NULL) { - n = (uintptr_t)tb & 3; - tb = (TranslationBlock *)((uintptr_t)tb & ~3); + page_for_each_tb(p, tb, n) { #ifdef TARGET_HAS_PRECISE_SMC if (current_tb == tb && (current_tb->cflags & CF_COUNT_MASK) != 1) { @@ -1600,9 +1592,8 @@ static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc) } #endif /* TARGET_HAS_PRECISE_SMC */ tb_phys_invalidate(tb, addr); - tb = tb->page_next[n]; } - p->first_tb = NULL; + p->first_tb = (uintptr_t)NULL; #ifdef TARGET_HAS_PRECISE_SMC if (current_tb_modified) { /* we generate a block containing just the instruction