From patchwork Tue May 26 16:54:50 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 476666 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id BA49214029C for ; Wed, 27 May 2015 03:08:04 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=Z3pUFYeQ; dkim-atps=neutral Received: from localhost ([::1]:49360 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YxIKg-00029K-Rb for incoming@patchwork.ozlabs.org; Tue, 26 May 2015 13:08:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46586) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YxI9F-00056j-Q8 for qemu-devel@nongnu.org; Tue, 26 May 2015 12:56:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YxI99-0004mz-Dp for qemu-devel@nongnu.org; Tue, 26 May 2015 12:56:13 -0400 Received: from mail-wi0-x236.google.com ([2a00:1450:400c:c05::236]:37635) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YxI99-0004mr-2B for qemu-devel@nongnu.org; Tue, 26 May 2015 12:56:07 -0400 Received: by wifw1 with SMTP id w1so38384725wif.0 for ; Tue, 26 May 2015 09:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:subject:date:message-id:in-reply-to:references; bh=gh4MRcJPa8YtK5zOjLM2uzv6DxVxPgYekgt0nH8LxqE=; b=Z3pUFYeQbnauFEiJPIZEpFZBzBfycTAFLWe4KMXJLPLOtJ+1iZ63KdPgi6oUtNOa1p uMzJEbYAJQUhGg4YOqDbZdeMae+DadcDFnBu3addxsHpZGYCPsmHMtxeK4E4SoxnS9y8 p82uO9FYlEpowSgPB4s0SOh8+3coPVSWOfXu659LPohSrV20HKFy77etqeULLh5oZEKK 3GBikRKuh2PqTw4HIozwZkrSaGX8DYwRK90hrMPd/P+6wFOcChjI+3nY5cJA0gpg10Rh cwcElQY3EsoegUBNPGqgN9HJsQoobg3kHofni4AN07caMI7OPtNiLseXYYA695AYPkox DjNw== X-Received: by 10.180.83.40 with SMTP id n8mr42788171wiy.57.1432659351291; Tue, 26 May 2015 09:55:51 -0700 (PDT) Received: from 640k.localdomain (dynamic-adsl-94-39-199-114.clienti.tiscali.it. [94.39.199.114]) by mx.google.com with ESMTPSA id x3sm17749309wiy.20.2015.05.26.09.55.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 May 2015 09:55:50 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Tue, 26 May 2015 18:54:50 +0200 Message-Id: <1432659305-54578-15-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1432659305-54578-1-git-send-email-pbonzini@redhat.com> References: <1432659305-54578-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-Received-From: 2a00:1450:400c:c05::236 Subject: [Qemu-devel] [PATCH v3 14/29] exec: use memory_region_get_dirty_log_mask to optimize dirty tracking X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The memory API can now return the exact set of bitmaps that have to be tracked. Use it instead of the in_migration variable. In the next patches, we will also use it to set only DIRTY_MEMORY_VGA or DIRTY_MEMORY_MIGRATION if necessary. This can make a difference for dataplane, especially after the dirty bitmap is changed to use more expensive atomic operations. Of some interest is the change to stl_phys_notdirty. When migration was introduced, stl_phys_notdirty was changed to effectively behave as stl_phys during migration. In fact, if one looks at the function as it was in the beginning (commit 8df1cd0, physical memory access functions, 2005-01-28), at the time the dirty bitmap was the equivalent of DIRTY_MEMORY_CODE nowadays; hence, the function simply should not touch the dirty code bits. This patch changes it to do the intended thing. Signed-off-by: Paolo Bonzini Reviewed-by: Fam Zheng --- exec.c | 59 +++++++++++++++++++---------------------------------------- 1 file changed, 19 insertions(+), 40 deletions(-) diff --git a/exec.c b/exec.c index c90dc72..9486347 100644 --- a/exec.c +++ b/exec.c @@ -59,8 +59,6 @@ //#define DEBUG_SUBPAGE #if !defined(CONFIG_USER_ONLY) -static bool in_migration; - /* ram_list is read under rcu_read_lock()/rcu_read_unlock(). Writes * are protected by the ramlist lock. */ @@ -870,11 +868,6 @@ void cpu_physical_memory_reset_dirty(ram_addr_t start, ram_addr_t length, } } -static void cpu_physical_memory_set_dirty_tracking(bool enable) -{ - in_migration = enable; -} - /* Called from RCU critical section */ hwaddr memory_region_section_get_iotlb(CPUState *cpu, MemoryRegionSection *section, @@ -2165,22 +2158,6 @@ static void tcg_commit(MemoryListener *listener) } } -static void core_log_global_start(MemoryListener *listener) -{ - cpu_physical_memory_set_dirty_tracking(true); -} - -static void core_log_global_stop(MemoryListener *listener) -{ - cpu_physical_memory_set_dirty_tracking(false); -} - -static MemoryListener core_memory_listener = { - .log_global_start = core_log_global_start, - .log_global_stop = core_log_global_stop, - .priority = 1, -}; - void address_space_init_dispatch(AddressSpace *as) { as->dispatch = NULL; @@ -2220,8 +2197,6 @@ static void memory_map_init(void) memory_region_init_io(system_io, NULL, &unassigned_io_ops, NULL, "io", 65536); address_space_init(&address_space_io, system_io, "I/O"); - - memory_listener_register(&core_memory_listener, &address_space_memory); } MemoryRegion *get_system_memory(void) @@ -2279,12 +2254,18 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr, #else -static void invalidate_and_set_dirty(hwaddr addr, +static void invalidate_and_set_dirty(MemoryRegion *mr, hwaddr addr, hwaddr length) { if (cpu_physical_memory_range_includes_clean(addr, length)) { - tb_invalidate_phys_range(addr, addr + length, 0); - cpu_physical_memory_set_dirty_range_nocode(addr, length); + uint8_t dirty_log_mask = memory_region_get_dirty_log_mask(mr); + if (dirty_log_mask & (1 << DIRTY_MEMORY_CODE)) { + tb_invalidate_phys_range(addr, addr + length, 0); + dirty_log_mask &= ~(1 << DIRTY_MEMORY_CODE); + } + if (dirty_log_mask) { + cpu_physical_memory_set_dirty_range_nocode(addr, length); + } } else { xen_modified_memory(addr, length); } @@ -2372,7 +2353,7 @@ MemTxResult address_space_rw(AddressSpace *as, hwaddr addr, MemTxAttrs attrs, /* RAM case */ ptr = qemu_get_ram_ptr(addr1); memcpy(ptr, buf, l); - invalidate_and_set_dirty(addr1, l); + invalidate_and_set_dirty(mr, addr1, l); } } else { if (!memory_access_is_direct(mr, is_write)) { @@ -2469,7 +2450,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as, switch (type) { case WRITE_DATA: memcpy(ptr, buf, l); - invalidate_and_set_dirty(addr1, l); + invalidate_and_set_dirty(mr, addr1, l); break; case FLUSH_CACHE: flush_icache_range((uintptr_t)ptr, (uintptr_t)ptr + l); @@ -2694,7 +2675,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len, mr = qemu_ram_addr_from_host(buffer, &addr1); assert(mr != NULL); if (is_write) { - invalidate_and_set_dirty(addr1, access_len); + invalidate_and_set_dirty(mr, addr1, access_len); } if (xen_enabled()) { xen_invalidate_map_cache_entry(buffer); @@ -3023,6 +3004,7 @@ void address_space_stl_notdirty(AddressSpace *as, hwaddr addr, uint32_t val, hwaddr l = 4; hwaddr addr1; MemTxResult r; + uint8_t dirty_log_mask; rcu_read_lock(); mr = address_space_translate(as, addr, &addr1, &l, @@ -3034,13 +3016,10 @@ void address_space_stl_notdirty(AddressSpace *as, hwaddr addr, uint32_t val, ptr = qemu_get_ram_ptr(addr1); stl_p(ptr, val); - if (unlikely(in_migration)) { - if (cpu_physical_memory_is_clean(addr1)) { - /* invalidate code */ - tb_invalidate_phys_page_range(addr1, addr1 + 4, 0); - /* set dirty bit */ - cpu_physical_memory_set_dirty_range_nocode(addr1, 4); - } + dirty_log_mask = memory_region_get_dirty_log_mask(mr); + dirty_log_mask &= ~(1 << DIRTY_MEMORY_CODE); + if (dirty_log_mask) { + cpu_physical_memory_set_dirty_range_nocode(addr1, 4); } r = MEMTX_OK; } @@ -3097,7 +3076,7 @@ static inline void address_space_stl_internal(AddressSpace *as, stl_p(ptr, val); break; } - invalidate_and_set_dirty(addr1, 4); + invalidate_and_set_dirty(mr, addr1, 4); r = MEMTX_OK; } if (result) { @@ -3201,7 +3180,7 @@ static inline void address_space_stw_internal(AddressSpace *as, stw_p(ptr, val); break; } - invalidate_and_set_dirty(addr1, 2); + invalidate_and_set_dirty(mr, addr1, 2); r = MEMTX_OK; } if (result) {