From patchwork Sat Oct 22 21:05:00 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 685458 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3t1b7K0FCLz9t0J for ; Sun, 23 Oct 2016 08:20:44 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=a2hEqxxa; dkim-atps=neutral Received: from localhost ([::1]:38741 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1by3ic-0000cm-Lr for incoming@patchwork.ozlabs.org; Sat, 22 Oct 2016 17:20:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40401) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1by3U0-00054M-5x for qemu-devel@nongnu.org; Sat, 22 Oct 2016 17:05:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1by3Ty-0001rz-Tn for qemu-devel@nongnu.org; Sat, 22 Oct 2016 17:05:36 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:34907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1by3Ty-0001rH-Hk for qemu-devel@nongnu.org; Sat, 22 Oct 2016 17:05:34 -0400 Received: by mail-pf0-x243.google.com with SMTP id s8so12071603pfj.2 for ; Sat, 22 Oct 2016 14:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=akFy4sdEJKPKjjt7v5lGfcSmcpykEhEGMbskKFJfsmE=; b=a2hEqxxa250wiQejx8lUpROv0Dzog3IdhASMr9raL3uAObusGmiHOdIt9+PC4ri8YY junjio4aM9ONV3R5X9T0OBQfcw3gISejwFtG/r+lw2J+1LjSAR8w2q3RdlZhfcQmdf2b 0n4UfWjq32thgzMSvOoZB8fsmhudnjbMf6XQtzEMUIwoUcddsb+qbuwUOvXDEfoKQgK/ q2gS8Oyv6EgsD6b8t77dyBwA7S9QWO6Ecfqnb9cnApIawA/93TDnEA7SdC0Mk/5I9R1D UrImMEznQhx6EgYyMvnv8YYgn0ZKlVYLWto+cRGzrzlnsqJjkedSnZFqiQnSthGyqNUs CcCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=akFy4sdEJKPKjjt7v5lGfcSmcpykEhEGMbskKFJfsmE=; b=mJ2tgSu67QHrhP0sxobVYM2ejgysmpOSKdEC4bGOkw9yFcDxWvUQAbkOuXzvUIrKQ9 RKik47CnY0k51HCDmGAC9fteWT5BBnh2/wCKN9CZb0WQBUTBjckMcNPEaL9qT7EoCH6Q OqhMcZgT1sQx4Cj0MSbLrepifMe91GNzAbMdETwNHhUqaVTDF+Wa7SG8UWz44QX3r0ce vJcU77uhXxhkzMTcD80nTqoTsCJX9dw0Ga/NKvUrc4iRorOJYuO8t+VeUQQEUn74MZZV vDKDJiRahFD77HAT4BjqV7qWmF4srzdjZBGxiP/BgZ1q0ZpMuuYcR9xsd10faPbjC5Op gkZQ== X-Gm-Message-State: ABUngvfa3BY91M36W6TdJuuWojOElnla1src2HODToPzPCZupn8S+vcwcBoum/LTzORPtg== X-Received: by 10.99.172.26 with SMTP id v26mr3758339pge.164.1477170333432; Sat, 22 Oct 2016 14:05:33 -0700 (PDT) Received: from bigtime.twiddle.net (174-24-157-40.tukw.qwest.net. [174.24.157.40]) by smtp.gmail.com with ESMTPSA id f129sm14300679pfc.3.2016.10.22.14.05.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 22 Oct 2016 14:05:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Sat, 22 Oct 2016 14:05:00 -0700 Message-Id: <1477170324-17783-12-git-send-email-rth@twiddle.net> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1477170324-17783-1-git-send-email-rth@twiddle.net> References: <1477170324-17783-1-git-send-email-rth@twiddle.net> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PULL 11/35] cputlb: Move most of iotlb code out of line X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Saves 2k code size off of a cold path. Reviewed-by: Emilio G. Cota Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- cputlb.c | 37 +++++++++++++++++++++++++++++++++++++ softmmu_template.h | 52 ++++++++++------------------------------------------ 2 files changed, 47 insertions(+), 42 deletions(-) diff --git a/cputlb.c b/cputlb.c index 0c9b77b..1bee47d 100644 --- a/cputlb.c +++ b/cputlb.c @@ -498,6 +498,43 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr) return qemu_ram_addr_from_host_nofail(p); } +static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, + target_ulong addr, uintptr_t retaddr, int size) +{ + CPUState *cpu = ENV_GET_CPU(env); + hwaddr physaddr = iotlbentry->addr; + MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); + uint64_t val; + + physaddr = (physaddr & TARGET_PAGE_MASK) + addr; + cpu->mem_io_pc = retaddr; + if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { + cpu_io_recompile(cpu, retaddr); + } + + cpu->mem_io_vaddr = addr; + memory_region_dispatch_read(mr, physaddr, &val, size, iotlbentry->attrs); + return val; +} + +static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry, + uint64_t val, target_ulong addr, + uintptr_t retaddr, int size) +{ + CPUState *cpu = ENV_GET_CPU(env); + hwaddr physaddr = iotlbentry->addr; + MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); + + physaddr = (physaddr & TARGET_PAGE_MASK) + addr; + if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { + cpu_io_recompile(cpu, retaddr); + } + + cpu->mem_io_vaddr = addr; + cpu->mem_io_pc = retaddr; + memory_region_dispatch_write(mr, physaddr, val, size, iotlbentry->attrs); +} + /* Return true if ADDR is present in the victim tlb, and has been copied back to the main tlb. */ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index, diff --git a/softmmu_template.h b/softmmu_template.h index b9532a4..035ffc8 100644 --- a/softmmu_template.h +++ b/softmmu_template.h @@ -112,25 +112,12 @@ #ifndef SOFTMMU_CODE_ACCESS static inline DATA_TYPE glue(io_read, SUFFIX)(CPUArchState *env, - CPUIOTLBEntry *iotlbentry, + size_t mmu_idx, size_t index, target_ulong addr, uintptr_t retaddr) { - uint64_t val; - CPUState *cpu = ENV_GET_CPU(env); - hwaddr physaddr = iotlbentry->addr; - MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); - - physaddr = (physaddr & TARGET_PAGE_MASK) + addr; - cpu->mem_io_pc = retaddr; - if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { - cpu_io_recompile(cpu, retaddr); - } - - cpu->mem_io_vaddr = addr; - memory_region_dispatch_read(mr, physaddr, &val, DATA_SIZE, - iotlbentry->attrs); - return val; + CPUIOTLBEntry *iotlbentry = &env->iotlb[mmu_idx][index]; + return io_readx(env, iotlbentry, addr, retaddr, DATA_SIZE); } #endif @@ -161,15 +148,13 @@ WORD_TYPE helper_le_ld_name(CPUArchState *env, target_ulong addr, /* Handle an IO access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; if ((addr & (DATA_SIZE - 1)) != 0) { goto do_unaligned_access; } - iotlbentry = &env->iotlb[mmu_idx][index]; /* ??? Note that the io helpers always read data in the target byte ordering. We should push the LE/BE request down into io. */ - res = glue(io_read, SUFFIX)(env, iotlbentry, addr, retaddr); + res = glue(io_read, SUFFIX)(env, mmu_idx, index, addr, retaddr); res = TGT_LE(res); return res; } @@ -230,15 +215,13 @@ WORD_TYPE helper_be_ld_name(CPUArchState *env, target_ulong addr, /* Handle an IO access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; if ((addr & (DATA_SIZE - 1)) != 0) { goto do_unaligned_access; } - iotlbentry = &env->iotlb[mmu_idx][index]; /* ??? Note that the io helpers always read data in the target byte ordering. We should push the LE/BE request down into io. */ - res = glue(io_read, SUFFIX)(env, iotlbentry, addr, retaddr); + res = glue(io_read, SUFFIX)(env, mmu_idx, index, addr, retaddr); res = TGT_BE(res); return res; } @@ -289,24 +272,13 @@ WORD_TYPE helper_be_lds_name(CPUArchState *env, target_ulong addr, #endif static inline void glue(io_write, SUFFIX)(CPUArchState *env, - CPUIOTLBEntry *iotlbentry, + size_t mmu_idx, size_t index, DATA_TYPE val, target_ulong addr, uintptr_t retaddr) { - CPUState *cpu = ENV_GET_CPU(env); - hwaddr physaddr = iotlbentry->addr; - MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); - - physaddr = (physaddr & TARGET_PAGE_MASK) + addr; - if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { - cpu_io_recompile(cpu, retaddr); - } - - cpu->mem_io_vaddr = addr; - cpu->mem_io_pc = retaddr; - memory_region_dispatch_write(mr, physaddr, val, DATA_SIZE, - iotlbentry->attrs); + CPUIOTLBEntry *iotlbentry = &env->iotlb[mmu_idx][index]; + return io_writex(env, iotlbentry, val, addr, retaddr, DATA_SIZE); } void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, @@ -334,16 +306,14 @@ void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, /* Handle an IO access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; if ((addr & (DATA_SIZE - 1)) != 0) { goto do_unaligned_access; } - iotlbentry = &env->iotlb[mmu_idx][index]; /* ??? Note that the io helpers always read data in the target byte ordering. We should push the LE/BE request down into io. */ val = TGT_LE(val); - glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); + glue(io_write, SUFFIX)(env, mmu_idx, index, val, addr, retaddr); return; } @@ -412,16 +382,14 @@ void helper_be_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, /* Handle an IO access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; if ((addr & (DATA_SIZE - 1)) != 0) { goto do_unaligned_access; } - iotlbentry = &env->iotlb[mmu_idx][index]; /* ??? Note that the io helpers always read data in the target byte ordering. We should push the LE/BE request down into io. */ val = TGT_BE(val); - glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); + glue(io_write, SUFFIX)(env, mmu_idx, index, val, addr, retaddr); return; }