{"id":814157,"url":"http://patchwork.ozlabs.org/api/patches/814157/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170915084030.40988-12-aik@ozlabs.ru/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20170915084030.40988-12-aik@ozlabs.ru>","list_archive_url":null,"date":"2017-09-15T08:40:28","name":"[qemu,v2,11/13] memory: Share FlatView's and dispatch trees between address spaces","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"d99d55baf956f8e9c27b09504084d1aadbe2e9b8","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/?format=json","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170915084030.40988-12-aik@ozlabs.ru/mbox/","series":[{"id":3250,"url":"http://patchwork.ozlabs.org/api/series/3250/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=3250","date":"2017-09-15T08:40:21","name":"memory: Reduce memory use","version":2,"mbox":"http://patchwork.ozlabs.org/series/3250/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/814157/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/814157/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtptp6lpCz9t2f\n\tfor ;\n\tFri, 15 Sep 2017 18:47:30 +1000 (AEST)","from localhost ([::1]:51979 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsmHZ-0005Fn-1c\n\tfor incoming@patchwork.ozlabs.org; Fri, 15 Sep 2017 04:47:29 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:38269)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsmB9-00071e-Oo\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 04:40:53 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsmB6-0002YM-Dw\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 04:40:51 -0400","from ozlabs.ru ([107.173.13.209]:44668)\n\tby eggs.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsmB5-0002Ql-US\n\tfor qemu-devel@nongnu.org; Fri, 15 Sep 2017 04:40:48 -0400","from vpl1.ozlabs.ibm.com (localhost [IPv6:::1])\n\tby ozlabs.ru (Postfix) with ESMTP id 6F8EE3A60063;\n\tFri, 15 Sep 2017 04:42:01 -0400 (EDT)"],"From":"Alexey Kardashevskiy ","To":"qemu-devel@nongnu.org","Date":"Fri, 15 Sep 2017 18:40:28 +1000","Message-Id":"<20170915084030.40988-12-aik@ozlabs.ru>","X-Mailer":"git-send-email 2.11.0","In-Reply-To":"<20170915084030.40988-1-aik@ozlabs.ru>","References":"<20170915084030.40988-1-aik@ozlabs.ru>","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 3.x [fuzzy]","X-Received-From":"107.173.13.209","Subject":"[Qemu-devel] [PATCH qemu v2 11/13] memory: Share FlatView's and\n\tdispatch trees between address spaces","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , Paolo Bonzini ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"},"content":"This allows sharing flat views between address spaces when the same root\nmemory region is used when creating a new address space.\n\nThis adds a global list of flat views and a list of attached address\nspaces per a flat view. Each address space references a flat view.\n\nSigned-off-by: Alexey Kardashevskiy \n---\n include/exec/memory.h | 3 +-\n memory.c | 148 ++++++++++++++++++++++++++++++++++++--------------\n 2 files changed, 108 insertions(+), 43 deletions(-)","diff":"diff --git a/include/exec/memory.h b/include/exec/memory.h\nindex be8cc1ccd3..04993f8eca 100644\n--- a/include/exec/memory.h\n+++ b/include/exec/memory.h\n@@ -308,8 +308,6 @@ struct AddressSpace {\n /* All fields are private. */\n struct rcu_head rcu;\n char *name;\n- int ref_count;\n- bool malloced;\n \n /* Accessed via RCU. */\n struct FlatView *current_map;\n@@ -318,6 +316,7 @@ struct AddressSpace {\n struct MemoryRegionIoeventfd *ioeventfds;\n QTAILQ_HEAD(memory_listeners_as, MemoryListener) listeners;\n QTAILQ_ENTRY(AddressSpace) address_spaces_link;\n+ QTAILQ_ENTRY(AddressSpace) flat_view_link;\n };\n \n FlatView *address_space_to_flatview(AddressSpace *as);\ndiff --git a/memory.c b/memory.c\nindex 29f8588945..f3da9379df 100644\n--- a/memory.c\n+++ b/memory.c\n@@ -47,6 +47,9 @@ static QTAILQ_HEAD(memory_listeners, MemoryListener) memory_listeners\n static QTAILQ_HEAD(, AddressSpace) address_spaces\n = QTAILQ_HEAD_INITIALIZER(address_spaces);\n \n+static QTAILQ_HEAD(FlatViewList, FlatView) flat_views\n+ = QTAILQ_HEAD_INITIALIZER(flat_views);\n+\n typedef struct AddrRange AddrRange;\n \n /*\n@@ -232,6 +235,8 @@ struct FlatView {\n unsigned nr_allocated;\n struct AddressSpaceDispatch *dispatch;\n MemoryRegion *root;\n+ QTAILQ_ENTRY(FlatView) flat_views_link;\n+ QTAILQ_HEAD(address_spaces, AddressSpace) address_spaces;\n };\n \n typedef struct AddressSpaceOps AddressSpaceOps;\n@@ -269,6 +274,7 @@ static FlatView *flatview_alloc(MemoryRegion *mr_root)\n view->ref = 1;\n view->root = mr_root;\n memory_region_ref(mr_root);\n+ QTAILQ_INIT(&view->address_spaces);\n \n return view;\n }\n@@ -953,28 +959,99 @@ static void flatview_render_new(FlatView *old_view, FlatView *new_view)\n address_space_dispatch_compact(new_view->dispatch);\n }\n \n-static void address_space_update_topology(AddressSpace *as)\n+static MemoryRegion *memory_region_unalias_entire(MemoryRegion *mr)\n {\n- FlatView *old_view = address_space_get_flatview(as);\n- FlatView *new_view = generate_memory_topology(old_view->root);\n-\n- flatview_render_new(old_view, new_view);\n- address_space_update_topology_pass(as, old_view, new_view, false);\n- address_space_update_topology_pass(as, old_view, new_view, true);\n-\n- /* Writes are protected by the BQL. */\n- atomic_rcu_set(&as->current_map, new_view);\n- call_rcu(old_view, flatview_unref, rcu);\n-\n- /* Note that all the old MemoryRegions are still alive up to this\n- * point. This relieves most MemoryListeners from the need to\n- * ref/unref the MemoryRegions they get---unless they use them\n- * outside the iothread mutex, in which case precise reference\n- * counting is necessary.\n- */\n- flatview_unref(old_view);\n-\n- address_space_update_ioeventfds(as);\n+ while (mr->alias && !mr->alias_offset &&\n+ int128_ge(mr->size, mr->alias->size)) {\n+ /* The alias is included in its entirety. Use it as\n+ * the \"real\" root, so that we can share more FlatViews.\n+ */\n+ mr = mr->alias;\n+ }\n+\n+ return mr;\n+}\n+\n+static bool flatview_can_share(FlatView *old_view, FlatView *new_view)\n+{\n+ MemoryRegion *old_root = memory_region_unalias_entire(old_view->root);\n+ MemoryRegion *new_root = memory_region_unalias_entire(new_view->root);\n+\n+ if (old_root == new_root) {\n+ return true;\n+ }\n+\n+ if (!old_root->enabled && !new_root->enabled) {\n+ return true;\n+ }\n+\n+ return false;\n+}\n+\n+static void flatview_update_topology(void)\n+{\n+ AddressSpace *as, *asnext;\n+ FlatView *old_view, *new_view, *vnext;\n+ struct FlatViewList fvs_tmp = QTAILQ_HEAD_INITIALIZER(fvs_tmp);\n+ bool found;\n+\n+ /* Build list of unique FlatViews, FV::root is the key */\n+ QTAILQ_FOREACH(old_view, &flat_views, flat_views_link) {\n+ found = false;\n+ QTAILQ_FOREACH(new_view, &fvs_tmp, flat_views_link) {\n+ if (flatview_can_share(old_view, new_view)) {\n+ found = true;\n+ break;\n+ }\n+ }\n+ if (found) {\n+ continue;\n+ }\n+\n+ new_view = generate_memory_topology(old_view->root);\n+ flatview_render_new(old_view, new_view);\n+ QTAILQ_INSERT_TAIL(&fvs_tmp, new_view, flat_views_link);\n+ }\n+\n+ /* Replace old FVs with new ones */\n+ QTAILQ_FOREACH_SAFE(old_view, &flat_views, flat_views_link, vnext) {\n+ flatview_ref(old_view);\n+\n+ found = false;\n+ QTAILQ_FOREACH(new_view, &fvs_tmp, flat_views_link) {\n+ if (flatview_can_share(old_view, new_view)) {\n+ found = true;\n+ break;\n+ }\n+ }\n+ assert(found);\n+\n+ QTAILQ_FOREACH_SAFE(as, &old_view->address_spaces, flat_view_link,\n+ asnext) {\n+ address_space_update_topology_pass(as, old_view, new_view, false);\n+ address_space_update_topology_pass(as, old_view, new_view, true);\n+\n+ QTAILQ_REMOVE(&old_view->address_spaces, as, flat_view_link);\n+ flatview_unref(old_view);\n+\n+ atomic_rcu_set(&as->current_map, new_view);\n+\n+ flatview_ref(new_view);\n+ QTAILQ_INSERT_TAIL(&new_view->address_spaces, as, flat_view_link);\n+ }\n+\n+ QTAILQ_REMOVE(&flat_views, old_view, flat_views_link);\n+\n+ flatview_unref(old_view); /* unref from beginning of the scope */\n+ }\n+\n+ /* Copy new FVs to the global list */\n+ QTAILQ_FOREACH_SAFE(new_view, &fvs_tmp, flat_views_link, vnext) {\n+ QTAILQ_REMOVE(&fvs_tmp, new_view, flat_views_link);\n+ flatview_unref(new_view);\n+\n+ QTAILQ_INSERT_HEAD(&flat_views, new_view, flat_views_link);\n+ }\n }\n \n void memory_region_transaction_begin(void)\n@@ -994,9 +1071,10 @@ void memory_region_transaction_commit(void)\n if (!memory_region_transaction_depth) {\n if (memory_region_update_pending) {\n MEMORY_LISTENER_CALL_GLOBAL(begin, Forward);\n+ flatview_update_topology();\n \n QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {\n- address_space_update_topology(as);\n+ address_space_update_ioeventfds(as);\n }\n memory_region_update_pending = false;\n MEMORY_LISTENER_CALL_GLOBAL(commit, Forward);\n@@ -2685,8 +2763,6 @@ void memory_region_invalidate_mmio_ptr(MemoryRegion *mr, hwaddr offset,\n void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)\n {\n memory_region_transaction_begin();\n- as->ref_count = 1;\n- as->malloced = false;\n as->current_map = flatview_alloc(root);\n as->ioeventfd_nb = 0;\n as->ioeventfds = NULL;\n@@ -2694,6 +2770,8 @@ void address_space_init(AddressSpace *as, MemoryRegion *root, const char *name)\n QTAILQ_INSERT_TAIL(&address_spaces, as, address_spaces_link);\n as->name = g_strdup(name ? name : \"anonymous\");\n memory_region_update_pending |= root->enabled;\n+ QTAILQ_INSERT_TAIL(&flat_views, as->current_map, flat_views_link);\n+ QTAILQ_INSERT_TAIL(&as->current_map->address_spaces, as, flat_view_link);\n memory_region_transaction_commit();\n }\n \n@@ -2704,41 +2782,29 @@ MemoryRegion *address_space_root(AddressSpace *as)\n \n static void do_address_space_destroy(AddressSpace *as)\n {\n- bool do_free = as->malloced;\n-\n assert(QTAILQ_EMPTY(&as->listeners));\n \n+ QTAILQ_REMOVE(&as->current_map->address_spaces, as, flat_view_link);\n+\n flatview_unref(as->current_map);\n+\n g_free(as->name);\n g_free(as->ioeventfds);\n- if (do_free) {\n- g_free(as);\n- }\n+ g_free(as);\n }\n \n AddressSpace *address_space_init_shareable(MemoryRegion *root, const char *name)\n {\n AddressSpace *as;\n \n- QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) {\n- if (root == address_space_root(as) && as->malloced) {\n- as->ref_count++;\n- return as;\n- }\n- }\n-\n as = g_malloc0(sizeof *as);\n address_space_init(as, root, name);\n- as->malloced = true;\n+\n return as;\n }\n \n void address_space_destroy(AddressSpace *as)\n {\n- as->ref_count--;\n- if (as->ref_count) {\n- return;\n- }\n /* Flush out anything from MemoryListeners listening in on this */\n memory_region_transaction_begin();\n memory_region_transaction_commit();\n","prefixes":["qemu","v2","11/13"]}