From patchwork Tue Nov 20 12:31:47 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 200331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id E0A912C0087 for ; Wed, 21 Nov 2012 00:18:19 +1100 (EST) Received: from localhost ([::1]:49474 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Tan0a-0002fJ-F2 for incoming@patchwork.ozlabs.org; Tue, 20 Nov 2012 07:32:56 -0500 Received: from eggs.gnu.org ([208.118.235.92]:45400) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Tan04-0001jM-HD for qemu-devel@nongnu.org; Tue, 20 Nov 2012 07:32:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Tamzw-0006yZ-6G for qemu-devel@nongnu.org; Tue, 20 Nov 2012 07:32:24 -0500 Received: from mx1.redhat.com ([209.132.183.28]:41335) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Tamzv-0006yN-V1 for qemu-devel@nongnu.org; Tue, 20 Nov 2012 07:32:16 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qAKCWFc8002010 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 20 Nov 2012 07:32:15 -0500 Received: from localhost (ovpn-112-23.ams2.redhat.com [10.36.112.23]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qAKCWDh4002801; Tue, 20 Nov 2012 07:32:14 -0500 From: Stefan Hajnoczi To: Date: Tue, 20 Nov 2012 13:31:47 +0100 Message-Id: <1353414712-27072-4-git-send-email-stefanha@redhat.com> In-Reply-To: <1353414712-27072-1-git-send-email-stefanha@redhat.com> References: <1353414712-27072-1-git-send-email-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: Kevin Wolf , Anthony Liguori , "Michael S. Tsirkin" , khoa@us.ibm.com, Stefan Hajnoczi , Paolo Bonzini , asias@redhat.com Subject: [Qemu-devel] [PATCH v2 3/8] dataplane: add host memory mapping code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The data plane thread needs to map guest physical addresses to host pointers. Normally this is done with cpu_physical_memory_map() but the function assumes the global mutex is held. The data plane thread does not touch the global mutex and therefore needs a thread-safe memory mapping mechanism. Hostmem registers a MemoryListener similar to how vhost collects and pushes memory region information into the kernel. There is a fine-grained lock on the regions list which is held during lookup and when installing a new regions list. When the physical memory map changes the MemoryListener callbacks are invoked. They build up a new list of memory regions which is finally installed when the list has been completed. Note that this approach is not safe across memory hotplug because mapped pointers may still be in used across memory unplug. However, this is currently a problem for QEMU in general and needs to be addressed in the future. Signed-off-by: Stefan Hajnoczi --- hw/dataplane/Makefile.objs | 3 + hw/dataplane/hostmem.c | 165 +++++++++++++++++++++++++++++++++++++++++++++ hw/dataplane/hostmem.h | 52 ++++++++++++++ 3 files changed, 220 insertions(+) create mode 100644 hw/dataplane/Makefile.objs create mode 100644 hw/dataplane/hostmem.c create mode 100644 hw/dataplane/hostmem.h diff --git a/hw/dataplane/Makefile.objs b/hw/dataplane/Makefile.objs new file mode 100644 index 0000000..8c8dea1 --- /dev/null +++ b/hw/dataplane/Makefile.objs @@ -0,0 +1,3 @@ +ifeq ($(CONFIG_VIRTIO), y) +common-obj-$(CONFIG_VIRTIO_BLK_DATA_PLANE) += hostmem.o +endif diff --git a/hw/dataplane/hostmem.c b/hw/dataplane/hostmem.c new file mode 100644 index 0000000..ac9d613 --- /dev/null +++ b/hw/dataplane/hostmem.c @@ -0,0 +1,165 @@ +/* + * Thread-safe guest to host memory mapping + * + * Copyright 2012 Red Hat, Inc. and/or its affiliates + * + * Authors: + * Stefan Hajnoczi + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "exec-memory.h" +#include "hostmem.h" + +static int hostmem_lookup_cmp(const void *phys_, const void *region_) +{ + hwaddr phys = *(const hwaddr *)phys_; + const HostmemRegion *region = region_; + + if (phys < region->guest_addr) { + return -1; + } else if (phys >= region->guest_addr + region->size) { + return 1; + } else { + return 0; + } +} + +/** + * Map guest physical address to host pointer + */ +void *hostmem_lookup(Hostmem *hostmem, hwaddr phys, hwaddr len, bool is_write) +{ + HostmemRegion *region; + void *host_addr = NULL; + hwaddr offset_within_region; + + qemu_mutex_lock(&hostmem->current_regions_lock); + region = bsearch(&phys, hostmem->current_regions, + hostmem->num_current_regions, + sizeof(hostmem->current_regions[0]), + hostmem_lookup_cmp); + if (!region) { + goto out; + } + if (is_write && region->readonly) { + goto out; + } + offset_within_region = phys - region->guest_addr; + if (offset_within_region + len <= region->size) { + host_addr = region->host_addr + offset_within_region; + } +out: + qemu_mutex_unlock(&hostmem->current_regions_lock); + + return host_addr; +} + +/** + * Install new regions list + */ +static void hostmem_listener_commit(MemoryListener *listener) +{ + Hostmem *hostmem = container_of(listener, Hostmem, listener); + + qemu_mutex_lock(&hostmem->current_regions_lock); + g_free(hostmem->current_regions); + hostmem->current_regions = hostmem->new_regions; + hostmem->num_current_regions = hostmem->num_new_regions; + qemu_mutex_unlock(&hostmem->current_regions_lock); + + /* Reset new regions list */ + hostmem->new_regions = NULL; + hostmem->num_new_regions = 0; +} + +/** + * Add a MemoryRegionSection to the new regions list + */ +static void hostmem_append_new_region(Hostmem *hostmem, + MemoryRegionSection *section) +{ + void *ram_ptr = memory_region_get_ram_ptr(section->mr); + size_t num = hostmem->num_new_regions; + size_t new_size = (num + 1) * sizeof(hostmem->new_regions[0]); + + hostmem->new_regions = g_realloc(hostmem->new_regions, new_size); + hostmem->new_regions[num] = (HostmemRegion){ + .host_addr = ram_ptr + section->offset_within_region, + .guest_addr = section->offset_within_address_space, + .size = section->size, + .readonly = section->readonly, + }; + hostmem->num_new_regions++; +} + +static void hostmem_listener_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + Hostmem *hostmem = container_of(listener, Hostmem, listener); + + if (memory_region_is_ram(section->mr)) { + hostmem_append_new_region(hostmem, section); + } +} + +/* We don't implement most MemoryListener callbacks, use these nop stubs */ +static void hostmem_listener_nop(MemoryListener *listener) +{ +} + +static void hostmem_listener_section_nop(MemoryListener *listener, + MemoryRegionSection *section) +{ +} + +static void hostmem_listener_eventfd_nop(MemoryListener *listener, + MemoryRegionSection *section, + bool match_data, uint64_t data, + EventNotifier *e) +{ +} + +static void hostmem_listener_coalesced_mmio_nop(MemoryListener *listener, + MemoryRegionSection *section, + hwaddr addr, hwaddr len) +{ +} + +void hostmem_init(Hostmem *hostmem) +{ + memset(hostmem, 0, sizeof(*hostmem)); + + hostmem->listener = (MemoryListener){ + .begin = hostmem_listener_nop, + .commit = hostmem_listener_commit, + .region_add = hostmem_listener_region_add, + .region_del = hostmem_listener_section_nop, + .region_nop = hostmem_listener_region_add, + .log_start = hostmem_listener_section_nop, + .log_stop = hostmem_listener_section_nop, + .log_sync = hostmem_listener_section_nop, + .log_global_start = hostmem_listener_nop, + .log_global_stop = hostmem_listener_nop, + .eventfd_add = hostmem_listener_eventfd_nop, + .eventfd_del = hostmem_listener_eventfd_nop, + .coalesced_mmio_add = hostmem_listener_coalesced_mmio_nop, + .coalesced_mmio_del = hostmem_listener_coalesced_mmio_nop, + .priority = 10, + }; + + memory_listener_register(&hostmem->listener, &address_space_memory); + if (hostmem->num_new_regions > 0) { + hostmem_listener_commit(&hostmem->listener); + } +} + +void hostmem_finalize(Hostmem *hostmem) +{ + memory_listener_unregister(&hostmem->listener); + g_free(hostmem->new_regions); + g_free(hostmem->current_regions); +} diff --git a/hw/dataplane/hostmem.h b/hw/dataplane/hostmem.h new file mode 100644 index 0000000..a833b74 --- /dev/null +++ b/hw/dataplane/hostmem.h @@ -0,0 +1,52 @@ +/* + * Thread-safe guest to host memory mapping + * + * Copyright 2012 Red Hat, Inc. and/or its affiliates + * + * Authors: + * Stefan Hajnoczi + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#ifndef HOSTMEM_H +#define HOSTMEM_H + +#include "memory.h" +#include "qemu-thread.h" + +typedef struct { + void *host_addr; + hwaddr guest_addr; + uint64_t size; + bool readonly; +} HostmemRegion; + +typedef struct { + /* The listener is invoked when regions change and a new list of regions is + * built up completely before they are installed. + */ + MemoryListener listener; + HostmemRegion *new_regions; + size_t num_new_regions; + + /* Current regions are accessed from multiple threads either to lookup + * addresses or to install a new list of regions. The lock protects the + * pointer and the regions. + */ + QemuMutex current_regions_lock; + HostmemRegion *current_regions; + size_t num_current_regions; +} Hostmem; + +void hostmem_init(Hostmem *hostmem); +void hostmem_finalize(Hostmem *hostmem); + +/** + * Map a guest physical address to a pointer + */ +void *hostmem_lookup(Hostmem *hostmem, hwaddr phys, hwaddr len, bool is_write); + +#endif /* HOSTMEM_H */