[{"id":1770223,"web_url":"http://patchwork.ozlabs.org/comment/1770223/","msgid":"","list_archive_url":null,"date":"2017-09-18T14:54:31","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 18/09/2017 12:16, Alexey Kardashevskiy wrote:\n> Most devices use at least one address space and every time a new address\n> space is added, flat views and dispatch trees are rebuild for all address\n> spaces. This is not a problem for a relatively small amount of devices but\n> even 50 virtio-pci devices use more than 8GB of RAM.\n> \n> What happens that on every flatview/dispatch rebuild, new arrays are\n> allocated and old ones release but the release is done via RCU so until\n> an entire machine is build, they are not released.\n> \n> This wraps devices creation into memory_region_transaction_begin/commit\n> to massively reduce amount of flat view/dispatch tree (re)allocations.\n> \n> Signed-off-by: Alexey Kardashevskiy \n> ---\n> Changes:\n> v2:\n> * wrapped qemu_run_machine_init_done_notifiers() as well\n> ---\n> vl.c | 9 +++++++++\n> 1 file changed, 9 insertions(+)\n> \n> diff --git a/vl.c b/vl.c\n> index 9e62e92aea..e4f2ece590 100644\n> --- a/vl.c\n> +++ b/vl.c\n> @@ -4741,12 +4741,16 @@ int main(int argc, char **argv, char **envp)\n> igd_gfx_passthru();\n> \n> /* init generic devices */\n> + memory_region_transaction_begin();\n> +\n> rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);\n> if (qemu_opts_foreach(qemu_find_opts(\"device\"),\n> device_init_func, NULL, NULL)) {\n> exit(1);\n> }\n> \n> + memory_region_transaction_commit();\n> +\n> cpu_synchronize_all_post_init();\n> \n> rom_reset_order_override();\n> @@ -4829,8 +4833,13 @@ int main(int argc, char **argv, char **envp)\n> /* TODO: once all bus devices are qdevified, this should be done\n> * when bus is created by qdev.c */\n> qemu_register_reset(qbus_reset_all_fn, sysbus_get_default());\n> +\n> + memory_region_transaction_begin();\n> +\n> qemu_run_machine_init_done_notifiers();\n> \n> + memory_region_transaction_commit();\n> +\n> if (rom_check_and_register_reset() != 0) {\n> error_report(\"rom check and register reset failed\");\n> exit(1);\n> \n\nThis should not be necessary given the other patches; the PCI devices\nhave an empty address space at the beginning, so there are other less\nintrusive optimizations to do instead with the same effect:\n\n1) as a start, the \"|= root->enabled\" can resolve aliases. This should\nbe enough for the PCI device case.\n\n2) also, after patch 2 you know that the address space has no listeners\nhere, so the begin/commit isn't really needed. Instead you can use the\nopen-coded loop to directly generate the FlatView. This avoids touching\n_all_ address spaces, which is already an improvement from O(n^2) to\nO(n) rebuilds on device startup.\n\n3) you can consult the list (or hash table :)) of live FlatViews (which\nmeans you keep it live after memory_region_transaction_commit ends, and\nonly clear it on the next call), and reuse an existing FlatView. Note\nthat the number of distinct FlatViews should be very few, so feel free\nto revert from hash table to list in v4 if you prefer.\n\n4) you can skip address_space_update_topology_pass if\nQTAILQ_EMPTY(&as->listeners). This can provide some startup speed\nimprovements.\n\nOptimizations 2/3/4 should be moved to the end of the series, or even in\na separate post. The first can be done in the beginning too, as you prefer.\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwpvX1yqbz9s3w\n\tfor ;\n\tTue, 19 Sep 2017 00:55:03 +1000 (AEST)","from localhost ([::1]:37090 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtxRs-0006mX-Oh\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 10:55:00 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34127)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtxRW-0006mO-J2\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:54:39 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtxRT-0006Sm-B1\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:54:38 -0400","from mx1.redhat.com ([209.132.183.28]:46542)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dtxRT-0006Qv-34\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 10:54:35 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id CABB280481;\n\tMon, 18 Sep 2017 14:54:33 +0000 (UTC)","from [10.32.181.85] (unknown [10.32.181.85])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 493AB61342;\n\tMon, 18 Sep 2017 14:54:32 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com CABB280481","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>","From":"Paolo Bonzini ","Message-ID":"","Date":"Mon, 18 Sep 2017 16:54:31 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<20170918101709.30421-2-aik@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.28]);\n\tMon, 18 Sep 2017 14:54:33 +0000 (UTC)","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770611,"web_url":"http://patchwork.ozlabs.org/comment/1770611/","msgid":"<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>","list_archive_url":null,"date":"2017-09-19T02:09:32","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 19/09/17 00:54, Paolo Bonzini wrote:\n> On 18/09/2017 12:16, Alexey Kardashevskiy wrote:\n>> Most devices use at least one address space and every time a new address\n>> space is added, flat views and dispatch trees are rebuild for all address\n>> spaces. This is not a problem for a relatively small amount of devices but\n>> even 50 virtio-pci devices use more than 8GB of RAM.\n>>\n>> What happens that on every flatview/dispatch rebuild, new arrays are\n>> allocated and old ones release but the release is done via RCU so until\n>> an entire machine is build, they are not released.\n>>\n>> This wraps devices creation into memory_region_transaction_begin/commit\n>> to massively reduce amount of flat view/dispatch tree (re)allocations.\n>>\n>> Signed-off-by: Alexey Kardashevskiy \n>> ---\n>> Changes:\n>> v2:\n>> * wrapped qemu_run_machine_init_done_notifiers() as well\n>> ---\n>> vl.c | 9 +++++++++\n>> 1 file changed, 9 insertions(+)\n>>\n>> diff --git a/vl.c b/vl.c\n>> index 9e62e92aea..e4f2ece590 100644\n>> --- a/vl.c\n>> +++ b/vl.c\n>> @@ -4741,12 +4741,16 @@ int main(int argc, char **argv, char **envp)\n>> igd_gfx_passthru();\n>> \n>> /* init generic devices */\n>> + memory_region_transaction_begin();\n>> +\n>> rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);\n>> if (qemu_opts_foreach(qemu_find_opts(\"device\"),\n>> device_init_func, NULL, NULL)) {\n>> exit(1);\n>> }\n>> \n>> + memory_region_transaction_commit();\n>> +\n>> cpu_synchronize_all_post_init();\n>> \n>> rom_reset_order_override();\n>> @@ -4829,8 +4833,13 @@ int main(int argc, char **argv, char **envp)\n>> /* TODO: once all bus devices are qdevified, this should be done\n>> * when bus is created by qdev.c */\n>> qemu_register_reset(qbus_reset_all_fn, sysbus_get_default());\n>> +\n>> + memory_region_transaction_begin();\n>> +\n>> qemu_run_machine_init_done_notifiers();\n>> \n>> + memory_region_transaction_commit();\n>> +\n>> if (rom_check_and_register_reset() != 0) {\n>> error_report(\"rom check and register reset failed\");\n>> exit(1);\n>>\n> \n> This should not be necessary given the other patches; the PCI devices\n> have an empty address space at the beginning, so there are other less\n> intrusive optimizations to do instead with the same effect:\n> \n> 1) as a start, the \"|= root->enabled\" can resolve aliases. This should\n> be enough for the PCI device case.\n> \n> 2) also, after patch 2 you know that the address space has no listeners\n> here, so the begin/commit isn't really needed. Instead you can use the\n> open-coded loop to directly generate the FlatView. This avoids touching\n> _all_ address spaces, which is already an improvement from O(n^2) to\n> O(n) rebuilds on device startup.\n> \n> 3) you can consult the list (or hash table :)) of live FlatViews (which\n> means you keep it live after memory_region_transaction_commit ends, and\n> only clear it on the next call), and reuse an existing FlatView. Note\n> that the number of distinct FlatViews should be very few, \n\n\nI keep missing this bit - why few? Each virtio-pci device creates 2 AS,\nwith proxy->modern_bar and pci_dev->bus_master_container_region which are\nunique and not aliases. Remember, 500 virtio devices is my test case ;)\n\n\n> so feel free\n> to revert from hash table to list in v4 if you prefer.\n>> 4) you can skip address_space_update_topology_pass if\n> QTAILQ_EMPTY(&as->listeners). This can provide some startup speed\n> improvements.\n> \n> Optimizations 2/3/4 should be moved to the end of the series, or even in\n> a separate post. The first can be done in the beginning too, as you prefer.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"FEG1ash2\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xx5tc0YlJz9s3w\n\tfor ;\n\tTue, 19 Sep 2017 12:10:16 +1000 (AEST)","from localhost ([::1]:39698 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1du7zK-0003X5-62\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 22:10:14 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:60953)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1du7yp-0003Va-1L\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 22:09:44 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1du7yk-0004Db-Q2\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 22:09:42 -0400","from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:36943)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1du7yk-0004Cy-Gr\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 22:09:38 -0400","by mail-pg0-x242.google.com with SMTP id v5so1206468pgn.4\n\tfor ; Mon, 18 Sep 2017 19:09:38 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tt17sm777274pfl.61.2017.09.18.19.09.35\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tMon, 18 Sep 2017 19:09:36 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:references:from:message-id:date:user-agent:mime-version\n\t:in-reply-to:content-language:content-transfer-encoding;\n\tbh=yxyDsoyrT3a4/sPXVH0WOhVL5OIQAL/H8BAjLz2IrVo=;\n\tb=FEG1ash2X1y6NUqLtdy/gPNJTe8EDXzvUT3J3dBvCv3jZjoZjeZjFR0/ac/mg7rhtu\n\t45cDzNXv/8jE7uWbiX1D9LoYD4z2Sob/RC8+ui1WKm2JV7utZAXP7nK0gWSj0LpgRvsa\n\t65PoeOXNcXgZobNqOnBqcd3g5glKtlJ35h+BmWUTP4iKalssXGpMGPS3YVZZB49wiFdq\n\tf9NBh3U2JAMrtzOIyWonDynIJV90IN2VZb8GjB4Fo6tPP3ryGLLrrGWrjZHsVSG73xPW\n\tgjdUKfLeZE4XXdJ/G/eAcSt0g3Ow/mBSWl0ZxJnM2SwboQ4+fcA/szl+2FpaMIXXwrjO\n\tJwvg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=yxyDsoyrT3a4/sPXVH0WOhVL5OIQAL/H8BAjLz2IrVo=;\n\tb=ogPM7NSfRlw95snbpDTHoQ7Pohzl72Zx7P2le7IqfMhZOc3At/ofQqtGW2qhF4mrmu\n\t5mm+afid+c0hv2szz6Ds0pcWFmXDcbC11qURIGgNMhPO8tVcgc8T0Q1ZtuiuplY1Fjjf\n\tWShxZ7siywZGK8f1NPaQ9I7mjIuD4K0eRjfd3rNPWRSRwbxFQG8L+/P08fmdPKK1oZu4\n\tLYnJp7FL6Z7O3gxjAweNzTtWuWrpM554nqmrHpARgRnxw962jVCmLc/FQV9zQZ/Vlqnl\n\tC/yZ7xPp2TZHl0XjY8l9KMNPtpo2w9hGq1IDrEJ9wEOGsKeJ6jP4GRR9tpxjAZMm23aV\n\tNjgw==","X-Gm-Message-State":"AHPjjUjPhusZMt47BDituSdKY8R+PZ8k49mWBG5VZxqQ7uAFal0XUu6v\n\th6ZI/UnBsIfXyobvlJw=","X-Google-Smtp-Source":"AOwi7QC5/UVMLBNnPCLLcvXYdYR0CmciK9XPbeL5V/eyoTcV7rIaXSNY6wsaoWscgwIoDIClf/YHkA==","X-Received":"by 10.84.135.34 with SMTP id 31mr539154pli.306.1505786977185;\n\tMon, 18 Sep 2017 19:09:37 -0700 (PDT)","To":"Paolo Bonzini , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t","From":"Alexey Kardashevskiy ","Message-ID":"<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>","Date":"Tue, 19 Sep 2017 12:09:32 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::242","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770683,"web_url":"http://patchwork.ozlabs.org/comment/1770683/","msgid":"","list_archive_url":null,"date":"2017-09-19T06:57:17","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 19/09/17 12:09, Alexey Kardashevskiy wrote:\n> On 19/09/17 00:54, Paolo Bonzini wrote:\n>> On 18/09/2017 12:16, Alexey Kardashevskiy wrote:\n>>> Most devices use at least one address space and every time a new address\n>>> space is added, flat views and dispatch trees are rebuild for all address\n>>> spaces. This is not a problem for a relatively small amount of devices but\n>>> even 50 virtio-pci devices use more than 8GB of RAM.\n>>>\n>>> What happens that on every flatview/dispatch rebuild, new arrays are\n>>> allocated and old ones release but the release is done via RCU so until\n>>> an entire machine is build, they are not released.\n>>>\n>>> This wraps devices creation into memory_region_transaction_begin/commit\n>>> to massively reduce amount of flat view/dispatch tree (re)allocations.\n>>>\n>>> Signed-off-by: Alexey Kardashevskiy \n>>> ---\n>>> Changes:\n>>> v2:\n>>> * wrapped qemu_run_machine_init_done_notifiers() as well\n>>> ---\n>>> vl.c | 9 +++++++++\n>>> 1 file changed, 9 insertions(+)\n>>>\n>>> diff --git a/vl.c b/vl.c\n>>> index 9e62e92aea..e4f2ece590 100644\n>>> --- a/vl.c\n>>> +++ b/vl.c\n>>> @@ -4741,12 +4741,16 @@ int main(int argc, char **argv, char **envp)\n>>> igd_gfx_passthru();\n>>> \n>>> /* init generic devices */\n>>> + memory_region_transaction_begin();\n>>> +\n>>> rom_set_order_override(FW_CFG_ORDER_OVERRIDE_DEVICE);\n>>> if (qemu_opts_foreach(qemu_find_opts(\"device\"),\n>>> device_init_func, NULL, NULL)) {\n>>> exit(1);\n>>> }\n>>> \n>>> + memory_region_transaction_commit();\n>>> +\n>>> cpu_synchronize_all_post_init();\n>>> \n>>> rom_reset_order_override();\n>>> @@ -4829,8 +4833,13 @@ int main(int argc, char **argv, char **envp)\n>>> /* TODO: once all bus devices are qdevified, this should be done\n>>> * when bus is created by qdev.c */\n>>> qemu_register_reset(qbus_reset_all_fn, sysbus_get_default());\n>>> +\n>>> + memory_region_transaction_begin();\n>>> +\n>>> qemu_run_machine_init_done_notifiers();\n>>> \n>>> + memory_region_transaction_commit();\n>>> +\n>>> if (rom_check_and_register_reset() != 0) {\n>>> error_report(\"rom check and register reset failed\");\n>>> exit(1);\n>>>\n>>\n>> This should not be necessary given the other patches; the PCI devices\n>> have an empty address space at the beginning, so there are other less\n>> intrusive optimizations to do instead with the same effect:\n>>\n>> 1) as a start, the \"|= root->enabled\" can resolve aliases. This should\n>> be enough for the PCI device case.\n>>\n>> 2) also, after patch 2 you know that the address space has no listeners\n>> here, so the begin/commit isn't really needed. Instead you can use the\n>> open-coded loop to directly generate the FlatView. This avoids touching\n>> _all_ address spaces, which is already an improvement from O(n^2) to\n>> O(n) rebuilds on device startup.\n\n03/13 does this already, no?\n\n>>\n>> 3) you can consult the list (or hash table :)) of live FlatViews (which\n>> means you keep it live after memory_region_transaction_commit ends, and\n>> only clear it on the next call), and reuse an existing FlatView. Note\n>> that the number of distinct FlatViews should be very few, \n> \n> \n> I keep missing this bit - why few? Each virtio-pci device creates 2 AS,\n> with proxy->modern_bar and pci_dev->bus_master_container_region which are\n> unique and not aliases. Remember, 500 virtio devices is my test case ;)\n\n\nMore details: pci_dev->bus_master_container_region is a root and it is\nenabled but its only child pci_dev->bus_master_enable_region is not. Ok, in\nflatview_topology_update() I can render a FV, see that it is empty\n(view->nr==0) and share an empty FV in this case too, this halves the\nnumber of FVs (from ~1000 to ~500 for 500 virtio devices).\n\nBut proxy->modern_bar (which has an modern_cfg alias which is a root of an\nAS) is enabled since it is created and I could disable it and enable\nafterwards but since a PCI device enablement is done by writing to the\nconfig space, I kind of stuck here.\n\n\nI can do something like this and it helps a lot (now with -S I end up\nhaving 4 FVs and much better start time) but it is kinda hacky and \"memory:\nPostpone flatview and dispatch tree building till all devices are added\"\nsolves this better imho, no?\n\n\ndiff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c\nindex 3268c16966..fa2cd7cf2c 100644\n--- a/hw/virtio/virtio-pci.c\n+++ b/hw/virtio/virtio-pci.c\n@@ -629,6 +629,8 @@ static void virtio_write_config(PCIDevice *pci_dev,\nuint32_t address,\n VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);\n struct virtio_pci_cfg_cap *cfg;\n\n+ memory_region_set_enabled(&proxy->modern_bar, true);\n+\n pci_default_write_config(pci_dev, address, val, len);\n\n if (range_covers_byte(address, len, PCI_COMMAND) &&\n@@ -662,6 +664,8 @@ static uint32_t virtio_read_config(PCIDevice *pci_dev,\n VirtIOPCIProxy *proxy = DO_UPCAST(VirtIOPCIProxy, pci_dev, pci_dev);\n struct virtio_pci_cfg_cap *cfg;\n\n+ memory_region_set_enabled(&proxy->modern_bar, true);\n+\n if (proxy->config_cap &&\n ranges_overlap(address, len, proxy->config_cap + offsetof(struct\nvirtio_pci_cfg_cap,\n\npci_cfg_data),\n@@ -1790,6 +1794,8 @@ static void virtio_pci_realize(PCIDevice *pci_dev,\nError **errp)\n 0,\n memory_region_size(&proxy->modern_bar));\n\n+ memory_region_set_enabled(&proxy->modern_bar, false);\n+\n address_space_init(&proxy->modern_as, &proxy->modern_cfg,\n\"virtio-pci-cfg-as\");\n\n\n\n\n\n> \n> \n>> so feel free\n>> to revert from hash table to list in v4 if you prefer.\n>>> 4) you can skip address_space_update_topology_pass if\n>> QTAILQ_EMPTY(&as->listeners). This can provide some startup speed\n>> improvements.\n>>\n>> Optimizations 2/3/4 should be moved to the end of the series, or even in\n>> a separate post. The first can be done in the beginning too, as you prefer.\n> \n> \n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"qn70yZ7M\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxDGm1Dlzz9rvt\n\tfor ;\n\tTue, 19 Sep 2017 16:58:05 +1000 (AEST)","from localhost ([::1]:40344 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duCTo-0003hs-AE\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 02:58:00 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:58735)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duCTI-0003gR-CD\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 02:57:29 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duCTE-0003vx-82\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 02:57:28 -0400","from mail-it0-x244.google.com ([2607:f8b0:4001:c0b::244]:33729)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1duCTD-0003uq-W1\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 02:57:24 -0400","by mail-it0-x244.google.com with SMTP id g18so1878477itg.0\n\tfor ; Mon, 18 Sep 2017 23:57:23 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tn63sm580091itb.16.2017.09.18.23.57.20\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tMon, 18 Sep 2017 23:57:21 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:from:to:references:message-id:date:user-agent:mime-version\n\t:in-reply-to:content-language:content-transfer-encoding;\n\tbh=2LCa/zjytbLvPxk4nsFzdjrx4itg7KJdwtDFgJCNIp8=;\n\tb=qn70yZ7M0SwBVVZi75b6PpzsseC3aZFVPuIemVJo9vYNfegbBktvrnS7nKz+UFPiRm\n\tJxVtup8rZOam/zL213jWR5IqSq9Mg7w6IhDkSIXMls2VasM4cpOLcM9Qap0kZ8a2NWGY\n\tkk6ZXW1HPuSpgs/PCuu9lM8Oe68X6xjqeoofKgPReJof9wPAm68EfCbvklIBPWsoi7DG\n\tRS9ekc77ZSHgEpnVUFK4xBs9qLdCdL+0EALlhFypKNo+bqdG3z2CsKjE57OrteZl1kY3\n\tC/TXr3FO8hWEtrmBPSulh1+BBT42OFh33eAnO4Fr+jDVHI8RifsG5FY1gUjGIXVQfbmX\n\tvy8Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:from:to:references:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=2LCa/zjytbLvPxk4nsFzdjrx4itg7KJdwtDFgJCNIp8=;\n\tb=r7AuyCu4C3zBeRmDlu0XmEVKNyP1PJv2uKjMcJNY/sUXNN0frLv4XGkOLeiGiYp1KV\n\tRfpz/HOD20AZT4Vo8aQ8wJuW1xF7pT/aXwwu2j9qn8hgcsXfHwIpflz1QoyHQVrHXwGq\n\tza9DRIrxozTqUWPXE2aIXjLGJcFgwzvw/QHL8q7v5MSp/xYlYeTICa/1emgzwhIS2fJ6\n\tjUkrJgxBhr/SRho/CHZ/KYtX3BjP7c79nku94ST5pWy4f7OaanW6ZwwxPtvU6TGwB+6h\n\tOkhj1y6sYp0YSs7h3ay8eFDjottUQVbW/aASk9KPA1dAx+W32dSj+5/LoJYoHwZs9D/3\n\teVNQ==","X-Gm-Message-State":"AHPjjUgkGr0Daj8L82WTYHUTtGI9JilBXy7S0dmaIsF9jes4lFSy2QYC\n\tla2XBZiLMSqJ5roNEizOnVsG0F+0","X-Google-Smtp-Source":"AOwi7QDTatAdNeNTDIsckZBh7VTmSkolzd2uKhIYGJDQtuxDHXE1lH/puGw508Q/8nUGZZ3a6Xuy1Q==","X-Received":"by 10.36.46.140 with SMTP id i134mr127424ita.137.1505804242464; \n\tMon, 18 Sep 2017 23:57:22 -0700 (PDT)","From":"Alexey Kardashevskiy ","To":"Paolo Bonzini , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>","Message-ID":"","Date":"Tue, 19 Sep 2017 16:57:17 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:4001:c0b::244","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770704,"web_url":"http://patchwork.ozlabs.org/comment/1770704/","msgid":"<11f3ee38-7c70-41d8-0df1-774fbe95cfcb@redhat.com>","list_archive_url":null,"date":"2017-09-19T07:40:57","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 19/09/2017 08:57, Alexey Kardashevskiy wrote:\n>>> 3) you can consult the list (or hash table :)) of live FlatViews (which\n>>> means you keep it live after memory_region_transaction_commit ends, and\n>>> only clear it on the next call), and reuse an existing FlatView. Note\n>>> that the number of distinct FlatViews should be very few, \n>>\n>> I keep missing this bit - why few? Each virtio-pci device creates 2 AS,\n>> with proxy->modern_bar and pci_dev->bus_master_container_region which are\n>> unique and not aliases. Remember, 500 virtio devices is my test case ;)\n\nArgh, no, it's me who keeps missing the bit.\n\n> More details: pci_dev->bus_master_container_region is a root and it is\n> enabled but its only child pci_dev->bus_master_enable_region is not. Ok, in\n> flatview_topology_update() I can render a FV, see that it is empty\n> (view->nr==0) and share an empty FV in this case too, this halves the\n> number of FVs (from ~1000 to ~500 for 500 virtio devices).\n\nWe can also resolve \"fake\" aliases, obtained through a region with only\none subregion.\n\n> But proxy->modern_bar (which has an modern_cfg alias which is a root of an\n> AS) is enabled since it is created and I could disable it and enable\n> afterwards but since a PCI device enablement is done by writing to the\n> config space, I kind of stuck here.\n> \n> I can do something like this and it helps a lot (now with -S I end up\n> having 4 FVs and much better start time) but it is kinda hacky and \"memory:\n> Postpone flatview and dispatch tree building till all devices are added\"\n> solves this better imho, no?\n\nI think modern_bar (actually modern_cfg) shouldn't have its own address\nspace, it's a pretty wasteful way to compute the dispatch tree where a\nsimple linked list or array would be enough:\n\nmemory-region: virtio-pci\n 00000000fe000000-00000000fe003fff (prio 1, i/o): virtio-pci\n 00000000fe000000-00000000fe000fff (prio 0, i/o): virtio-pci-common\n 00000000fe001000-00000000fe001fff (prio 0, i/o): virtio-pci-isr\n 00000000fe002000-00000000fe002fff (prio 0, i/o): virtio-pci-device\n 00000000fe003000-00000000fe003fff (prio 0, i/o): virtio-pci-notify\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFDm15yYz9s7g\n\tfor ;\n\tTue, 19 Sep 2017 17:41:28 +1000 (AEST)","from localhost ([::1]:40549 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duD9q-0002sU-9l\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 03:41:26 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:44490)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duD9X-0002sC-3N\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:41:08 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duD9R-0005JC-Fq\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:41:07 -0400","from mx1.redhat.com ([209.132.183.28]:55070)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duD9R-0005Ic-9S\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:41:01 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 3468580F7A;\n\tTue, 19 Sep 2017 07:41:00 +0000 (UTC)","from [10.36.117.61] (ovpn-117-61.ams2.redhat.com [10.36.117.61])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 8C3E7600C2;\n\tTue, 19 Sep 2017 07:40:59 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 3468580F7A","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t","From":"Paolo Bonzini ","Message-ID":"<11f3ee38-7c70-41d8-0df1-774fbe95cfcb@redhat.com>","Date":"Tue, 19 Sep 2017 09:40:57 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]);\n\tTue, 19 Sep 2017 07:41:00 +0000 (UTC)","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770706,"web_url":"http://patchwork.ozlabs.org/comment/1770706/","msgid":"","list_archive_url":null,"date":"2017-09-19T07:42:42","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 19/09/2017 08:57, Alexey Kardashevskiy wrote:\n>>> 2) also, after patch 2 you know that the address space has no listeners\n>>> here, so the begin/commit isn't really needed. Instead you can use the\n>>> open-coded loop to directly generate the FlatView. This avoids touching\n>>> _all_ address spaces, which is already an improvement from O(n^2) to\n>>> O(n) rebuilds on device startup.\n> 03/13 does this already, no?\n> \n\nNo, there's still\n\n as->dispatch = NULL;\n memory_region_update_pending |= root->enabled;\n memory_region_transaction_commit();\n\nwhen creating the address space, which touches all address spaces.\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFH146g7z9s7g\n\tfor ;\n\tTue, 19 Sep 2017 17:43:25 +1000 (AEST)","from localhost ([::1]:40556 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duDBj-0003xW-LZ\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 03:43:23 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45072)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duDBE-0003wI-34\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:42:53 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duDB9-0006iC-Uu\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:42:52 -0400","from mx1.redhat.com ([209.132.183.28]:35750)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duDB9-0006hx-P5\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:42:47 -0400","from smtp.corp.redhat.com\n\t(int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id B50EBFED4;\n\tTue, 19 Sep 2017 07:42:46 +0000 (UTC)","from [10.36.117.61] (ovpn-117-61.ams2.redhat.com [10.36.117.61])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 201EF5D97A;\n\tTue, 19 Sep 2017 07:42:45 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com B50EBFED4","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t","From":"Paolo Bonzini ","Message-ID":"","Date":"Tue, 19 Sep 2017 09:42:42 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.14","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.29]);\n\tTue, 19 Sep 2017 07:42:46 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770721,"web_url":"http://patchwork.ozlabs.org/comment/1770721/","msgid":"","list_archive_url":null,"date":"2017-09-19T07:59:10","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 19/09/2017 09:57, Alexey Kardashevskiy wrote:\n>> We can also resolve \"fake\" aliases, obtained through a region with only\n>> one subregion.\n> I just do this now (will post soon):\n> \n> \n> empty_view = generate_memory_topology(NULL);\n> [...]\n> \n> new_view = generate_memory_topology(physmr);\n> \n> if (!new_view->nr) {\n> flatview_ref(empty_view);\n> flatview_unref(new_view);\n> new_view = empty_view;\n> \n> [...]\n> \n> g_hash_table_insert(views, physmr, new_view);\n> \n> and that's it. For non empty FVs I need to render FVs anyway and for\n> all-disabled/empty FVs it is really cheap so no big loss here.\n\nIs that enough to get 1 shared FV for all bus master address spaces? In\nany case, your patches provide a very good infrastructure and further\noptimization can be done easily on top.\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx05.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFgg6SfKz9ryT\n\tfor ;\n\tTue, 19 Sep 2017 18:01:19 +1000 (AEST)","from localhost ([::1]:40666 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duDT3-0001yX-V7\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 04:01:18 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:53966)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duDR5-0000cz-P1\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:59:16 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duDR4-0007n4-UX\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:59:15 -0400","from mx1.redhat.com ([209.132.183.28]:46972)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duDR4-0007mO-PO\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:59:14 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 9104A13A49;\n\tTue, 19 Sep 2017 07:59:13 +0000 (UTC)","from [10.36.117.61] (ovpn-117-61.ams2.redhat.com [10.36.117.61])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id EEB2F61342;\n\tTue, 19 Sep 2017 07:59:12 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 9104A13A49","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t<11f3ee38-7c70-41d8-0df1-774fbe95cfcb@redhat.com>\n\t","From":"Paolo Bonzini ","Message-ID":"","Date":"Tue, 19 Sep 2017 09:59:10 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.29]);\n\tTue, 19 Sep 2017 07:59:13 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770725,"web_url":"http://patchwork.ozlabs.org/comment/1770725/","msgid":"","list_archive_url":null,"date":"2017-09-19T07:57:36","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 19/09/17 17:40, Paolo Bonzini wrote:\n> On 19/09/2017 08:57, Alexey Kardashevskiy wrote:\n>>>> 3) you can consult the list (or hash table :)) of live FlatViews (which\n>>>> means you keep it live after memory_region_transaction_commit ends, and\n>>>> only clear it on the next call), and reuse an existing FlatView. Note\n>>>> that the number of distinct FlatViews should be very few, \n>>>\n>>> I keep missing this bit - why few? Each virtio-pci device creates 2 AS,\n>>> with proxy->modern_bar and pci_dev->bus_master_container_region which are\n>>> unique and not aliases. Remember, 500 virtio devices is my test case ;)\n> \n> Argh, no, it's me who keeps missing the bit.\n> \n>> More details: pci_dev->bus_master_container_region is a root and it is\n>> enabled but its only child pci_dev->bus_master_enable_region is not. Ok, in\n>> flatview_topology_update() I can render a FV, see that it is empty\n>> (view->nr==0) and share an empty FV in this case too, this halves the\n>> number of FVs (from ~1000 to ~500 for 500 virtio devices).\n> \n> We can also resolve \"fake\" aliases, obtained through a region with only\n> one subregion.\n\nI just do this now (will post soon):\n\n\nempty_view = generate_memory_topology(NULL);\n[...]\n\nnew_view = generate_memory_topology(physmr);\n\nif (!new_view->nr) {\n flatview_ref(empty_view);\n flatview_unref(new_view);\n new_view = empty_view;\n\n[...]\n\ng_hash_table_insert(views, physmr, new_view);\n\nand that's it. For non empty FVs I need to render FVs anyway and for\nall-disabled/empty FVs it is really cheap so no big loss here.\n\n\n> \n>> But proxy->modern_bar (which has an modern_cfg alias which is a root of an\n>> AS) is enabled since it is created and I could disable it and enable\n>> afterwards but since a PCI device enablement is done by writing to the\n>> config space, I kind of stuck here.\n>>\n>> I can do something like this and it helps a lot (now with -S I end up\n>> having 4 FVs and much better start time) but it is kinda hacky and \"memory:\n>> Postpone flatview and dispatch tree building till all devices are added\"\n>> solves this better imho, no?\n> \n> I think modern_bar (actually modern_cfg) shouldn't have its own address\n> space, it's a pretty wasteful way to compute the dispatch tree where a\n> simple linked list or array would be enough:\n\nWhy would we need an array there anyway? This AS is only used for\nvirtio_address_space_write() and virtio_address_space_read() so we only\nneed here s/virtio_address_space_write/memory_region_dispatch_write/ (and\nthe same for reading), no?\n\n>\n> memory-region: virtio-pci\n> 00000000fe000000-00000000fe003fff (prio 1, i/o): virtio-pci\n> 00000000fe000000-00000000fe000fff (prio 0, i/o): virtio-pci-common\n> 00000000fe001000-00000000fe001fff (prio 0, i/o): virtio-pci-isr\n> 00000000fe002000-00000000fe002fff (prio 0, i/o): virtio-pci-device\n> 00000000fe003000-00000000fe003fff (prio 0, i/o): virtio-pci-notify","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"xIxtej6H\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxFkT3gW1z9ryT\n\tfor ;\n\tTue, 19 Sep 2017 18:03:45 +1000 (AEST)","from localhost ([::1]:40677 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duDVP-0003gw-Jo\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 04:03:43 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:53228)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duDPe-0007sR-05\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:57:47 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duDPa-00075Z-Km\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:57:46 -0400","from mail-it0-x234.google.com ([2607:f8b0:4001:c0b::234]:45807)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1duDPa-000751-B3\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 03:57:42 -0400","by mail-it0-x234.google.com with SMTP id v19so1086295ite.0\n\tfor ; Tue, 19 Sep 2017 00:57:42 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tb14sm5069445ioa.6.2017.09.19.00.57.38\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 19 Sep 2017 00:57:40 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:references:from:message-id:date:user-agent:mime-version\n\t:in-reply-to:content-language:content-transfer-encoding;\n\tbh=T3S1emgGrsc7S9xiqNTYTKbwTUjBpWV1z1Etc5cSf4w=;\n\tb=xIxtej6HaxzPFoRoDEr8Mac5O1xa4dB0PfqEn/L+Dyt04iCz5e9wAIW6ie25N6WxZf\n\tPojeVEDUxEfryquiUiJX1dflT8Q14577YX1C/EI6HH2qBXldo8AW7T8iAnbQMH+L4G/Z\n\tVAQZHV+n1Mkh1Zzz0bxvLHlRbdBlt7wrT/oRlw2Z7ufgZ/Kugzh+dz7J9iapIXsMcN7l\n\trFnfrib06j3RTYAfWoPUoxHma18Mfo5BSp1iFwPJS9lcvAOApqkFlYRPtGCNrTBAQLbp\n\tNvQhGVTf03PJRqGenuq3vDGYywZ/gOO2vh+Mz22cbDRApQ+ANTahah0URJowXJ8dKEXc\n\teSrQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=T3S1emgGrsc7S9xiqNTYTKbwTUjBpWV1z1Etc5cSf4w=;\n\tb=ZxaMhhXO8bpW8UHGpt5xqyFELhNEoxux4DP8gTkOcHOYNyJAIPyID+hIkC7AArPS+Y\n\tMWK/uSSSrJ4ujS3tmgJMprETsIqpZlq/eKEbAVkSMyohbfnrENmjtQr0zpnQBNTmm8Qt\n\tWz8a4y5ZDKvYET9dYu5sk/UW2byHvJSMB342ClqMYkrO6jyYvvfFY2zYYLQ+jQtrPuXi\n\tGhCMO+KBX5S3aH5ZhDQG8aXOnQhvdmWt/uCNxohP/03bXW4AS+l/BdUp3DeqWKV9Lrex\n\thHmV3ADNOL7/2sS82Yv6SDHAWUV0aQ7BLXg3tDVnkyRZdgfcyTRsKmOJ4O4z8UZSYMvH\n\taMfQ==","X-Gm-Message-State":"AHPjjUhPjESsSiUlSW63SO0vagWd3LVVFj+KljnLO9AiY1H/WoEat4HU\n\tE4SW3TV3jLnsNH6ilYuq6nh9J7Mx","X-Google-Smtp-Source":"AOwi7QCBvbbzttAcysrs8oULcBIoMQZTWQNV5ck18QsZhggwhrGTudoyAtels2g12V46Kn+LoD9rkg==","X-Received":"by 10.36.172.88 with SMTP id m24mr331871iti.134.1505807861336;\n\tTue, 19 Sep 2017 00:57:41 -0700 (PDT)","To":"Paolo Bonzini , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t<11f3ee38-7c70-41d8-0df1-774fbe95cfcb@redhat.com>","From":"Alexey Kardashevskiy ","Message-ID":"","Date":"Tue, 19 Sep 2017 17:57:36 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<11f3ee38-7c70-41d8-0df1-774fbe95cfcb@redhat.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:4001:c0b::234","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770857,"web_url":"http://patchwork.ozlabs.org/comment/1770857/","msgid":"<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>","list_archive_url":null,"date":"2017-09-19T10:39:13","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 19/09/17 17:42, Paolo Bonzini wrote:\n> On 19/09/2017 08:57, Alexey Kardashevskiy wrote:\n>>>> 2) also, after patch 2 you know that the address space has no listeners\n>>>> here, so the begin/commit isn't really needed. Instead you can use the\n>>>> open-coded loop to directly generate the FlatView. This avoids touching\n>>>> _all_ address spaces, which is already an improvement from O(n^2) to\n>>>> O(n) rebuilds on device startup.\n>> 03/13 does this already, no?\n>>\n> \n> No, there's still\n> \n> as->dispatch = NULL;\n> memory_region_update_pending |= root->enabled;\n> memory_region_transaction_commit();\n> \n> when creating the address space, which touches all address spaces.\n\n\nThis does not seem helping a lot - my test without such an optimization\nallocates 2966044 FVs and 2388768 with it, which is just 25%, i.e. worth\nmaking a patch but yet getting rid of those nasty virtio-modern address\nspaces will do a lot better job.\n\nWhat is the exact concern about the \"[PATCH qemu v3 01/13] memory: Postpone\nflatview and dispatch tree building till all devices are added\"? Is it that\nyou missed that modern AS thing in virtio and thought we can do well even\nwithout it? Or there is something wrong with the approach?","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"l+sBSQmi\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxKSl47nRz9sBZ\n\tfor ;\n\tTue, 19 Sep 2017 20:52:07 +1000 (AEST)","from localhost ([::1]:41437 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duG8L-00056s-LU\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 06:52:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:52712)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duFw3-0003X8-C6\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:39:24 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duFw0-0004RO-8p\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:39:23 -0400","from mail-pg0-x22f.google.com ([2607:f8b0:400e:c05::22f]:47145)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1duFw0-0004QO-1t\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:39:20 -0400","by mail-pg0-x22f.google.com with SMTP id d8so1789426pgt.4\n\tfor ; Tue, 19 Sep 2017 03:39:18 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tg68sm2804136pfb.120.2017.09.19.03.39.15\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 19 Sep 2017 03:39:17 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:references:from:message-id:date:user-agent:mime-version\n\t:in-reply-to:content-language:content-transfer-encoding;\n\tbh=gDiTiSR9X0Nr2w4wdfyo4zGLXQjTcCB7DorjcqmAL7g=;\n\tb=l+sBSQmi9KKHIJlg1kwdUvHdk9Rg6OIU/+J2Yw0hZU5HqRS69HtzE19XCjvHTxyBq3\n\tcALi5wOMlcsTvL60uRCu86KoA+j8azn+RvWhzprLL6nm6rx4FYgWQvx2kkW4hcx0rdmN\n\tSg65LMliJrlCXaI/Hg1IK+L+b5fN2GqWYdhUyUPdfmCewTotdMZYWd5Yq2xyiv7gpCkq\n\tV65oBIOM2A9z0aTmrJSfN9Jqd2ddnlROpNjj0ys3cSDN+3hjxI/bruneYNX+hC5UYYGP\n\tJhs6L7fPRIKME/fP0G+Fa0nsuUoMOq7eIrezd4v/324QIpaF2sn87PSIrKcNo2XOV1Bu\n\t425Q==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=gDiTiSR9X0Nr2w4wdfyo4zGLXQjTcCB7DorjcqmAL7g=;\n\tb=sOOS5cXuQ+q9vzfTxm1fmLuqgvc7AJ+Gx1Qi764yDd509J4AZU5qwr5zDW09MK9yEk\n\tL2qtl0eDBNB5OoBEtNOV17Mkx8qY+CpvnpYYmYlgwzHvUNRbI1MfGs8uftLrE+ReGgFM\n\tOca0hL8V8dOiSBEekYAszPxGccsMg9ebmwLyvCFhba/pgYQ7esjOLt6gZfUxPJWt6KSG\n\tCuZ+MXo7vOZ+yGgmDf8zhCvZ/pmFCi/z5EEnmhWaXX6NJzvNb3ZkgpA0epQw3TaZa/65\n\tnZEex/ZUMASjBoFd9PSru6lpEz1pEZ4olQpSZEYbFAOMSRdal8yKHQN4lKVRiHh+yYgU\n\t3cug==","X-Gm-Message-State":"AHPjjUglrSCKgCLaexhqQwl4g+Ajcri8nEo7/fTOydIXPw5F4WLrg7M9\n\t9iD8Xh2StSjDHToIMRFBbE891aOP","X-Google-Smtp-Source":"AOwi7QDeOQUK16afkN52kmd/i5Pd/gzRuCk5/jbAEqPyjt2xyvvIKa4MjDCpndATi9yOcfKbINGwAA==","X-Received":"by 10.98.34.85 with SMTP id i82mr861369pfi.111.1505817557638;\n\tTue, 19 Sep 2017 03:39:17 -0700 (PDT)","To":"Paolo Bonzini , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t","From":"Alexey Kardashevskiy ","Message-ID":"<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>","Date":"Tue, 19 Sep 2017 20:39:13 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::22f","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770878,"web_url":"http://patchwork.ozlabs.org/comment/1770878/","msgid":"","list_archive_url":null,"date":"2017-09-19T10:47:22","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 19/09/2017 12:39, Alexey Kardashevskiy wrote:\n>> No, there's still\n>>\n>> as->dispatch = NULL;\n>> memory_region_update_pending |= root->enabled;\n>> memory_region_transaction_commit();\n>>\n>> when creating the address space, which touches all address spaces.\n> \n> This does not seem helping a lot - my test without such an optimization\n> allocates 2966044 FVs and 2388768 with it, which is just 25%, i.e. worth\n> making a patch but yet getting rid of those nasty virtio-modern address\n> spaces will do a lot better job.\n\nHave you tried resolving aliases that are implemented with containers\nand one subregion?\n\n> What is the exact concern about the \"[PATCH qemu v3 01/13] memory: Postpone\n> flatview and dispatch tree building till all devices are added\"? Is it that\n> you missed that modern AS thing in virtio and thought we can do well even\n> without it?\n\nI am afraid that some of the special sysbus devices (not PCI, those go\nthrough the bus master DMA region and it's empty on startup) might write\nto memory during initialization.\n\nThanks,\n\nPaolo\n\n> Or there is something wrong with the approach?","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=pbonzini@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxKsl6nC2z9rxj\n\tfor ;\n\tTue, 19 Sep 2017 21:10:19 +1000 (AEST)","from localhost ([::1]:41578 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duGPy-00039a-2f\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 07:10:18 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56538)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duG3s-00027W-R9\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:47:29 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duG3p-0008St-Ok\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:47:28 -0400","from mx1.redhat.com ([209.132.183.28]:50888)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duG3p-0008SS-He\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:47:25 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 7E039267C8;\n\tTue, 19 Sep 2017 10:47:24 +0000 (UTC)","from [10.36.117.61] (ovpn-117-61.ams2.redhat.com [10.36.117.61])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id DA4AE5D6A4;\n\tTue, 19 Sep 2017 10:47:23 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 7E039267C8","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t\n\t<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>","From":"Paolo Bonzini ","Message-ID":"","Date":"Tue, 19 Sep 2017 12:47:22 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.30]);\n\tTue, 19 Sep 2017 10:47:24 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771065,"web_url":"http://patchwork.ozlabs.org/comment/1771065/","msgid":"<2f185edf-3281-e01f-5859-cc43a5df413e@ozlabs.ru>","list_archive_url":null,"date":"2017-09-19T14:12:33","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 19/09/17 20:47, Paolo Bonzini wrote:\n> On 19/09/2017 12:39, Alexey Kardashevskiy wrote:\n>>> No, there's still\n>>>\n>>> as->dispatch = NULL;\n>>> memory_region_update_pending |= root->enabled;\n>>> memory_region_transaction_commit();\n>>>\n>>> when creating the address space, which touches all address spaces.\n>>\n>> This does not seem helping a lot - my test without such an optimization\n>> allocates 2966044 FVs and 2388768 with it, which is just 25%, i.e. worth\n>> making a patch but yet getting rid of those nasty virtio-modern address\n>> spaces will do a lot better job.\n> \n> Have you tried resolving aliases that are implemented with containers\n> and one subregion?\n\nYes, for 10 virtio-net and 8 cpus it looks like below, #5 is that shared\nthing. And if I use Intel e1000 instead, it is just 4 FVs.\n\n\n(qemu) info mtree -f -d\nFlatView #0\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #1\n AS \"I/O\", root: io\n Root memory region: io\n\nFlatView #2\n AS \"pci@800000020000000\", root: pci@800000020000000.iommu-root\n Root memory region: pci@800000020000000.iommu-root\n\nFlatView #3\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #4\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #5\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n AS \"virtio-net-pci\", root: bus master container\n Root memory region: (none)\n No rendered FlatView\n\nFlatView #6\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #7\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #8\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #9\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #10\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #11\n AS \"memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n AS \"cpu-memory\", root: system\n Root memory region: system\n\nFlatView #12\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\nFlatView #13\n AS \"virtio-pci-cfg-as\", root: virtio-pci-cfg, alias virtio-pci\n Root memory region: virtio-pci\n\n\n>> What is the exact concern about the \"[PATCH qemu v3 01/13] memory: Postpone\n>> flatview and dispatch tree building till all devices are added\"? Is it that\n>> you missed that modern AS thing in virtio and thought we can do well even\n>> without it?\n> \n> I am afraid that some of the special sysbus devices (not PCI, those go\n> through the bus master DMA region and it's empty on startup) might write\n> to memory during initialization.\n\n\nAnd somehow QEMU guarantees that all RAM MRs are rendered to FlatViews even\nthough some might be added via -device (can they btw?)?\n\n\n\n> \n> Thanks,\n> \n> Paolo\n> \n>> Or there is something wrong with the approach?\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"mr/S77GJ\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxPwb1RGdz9s7c\n\tfor ;\n\tWed, 20 Sep 2017 00:13:03 +1000 (AEST)","from localhost ([::1]:43157 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duJGn-0005FG-BV\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 10:13:01 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:44128)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duJGV-0005Eo-2P\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:12:44 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duJGR-0003ow-Nf\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:12:43 -0400","from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:33735)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1duJGR-0003oK-F1\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:12:39 -0400","by mail-pg0-x241.google.com with SMTP id i130so2123981pgc.0\n\tfor ; Tue, 19 Sep 2017 07:12:39 -0700 (PDT)","from [192.168.10.22] (124-171-134-202.dyn.iinet.net.au.\n\t[124.171.134.202]) by smtp.googlemail.com with ESMTPSA id\n\tv71sm3995402pfa.45.2017.09.19.07.12.36\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 19 Sep 2017 07:12:37 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:references:from:message-id:date:user-agent:mime-version\n\t:in-reply-to:content-language:content-transfer-encoding;\n\tbh=F14zQdK3W+jHyeh2lWeIDtiFBdnNqfUrvUA301bsVvc=;\n\tb=mr/S77GJcJhieaAH9Zhhe5Y9FC45jjXNNC3RiUfdz+/HxvbPrAP/j62wcpSaAcD2Wx\n\tt9GeTJa2Ns2+kzWOPiWon6rRPcwsr3SWhrQnQ2KxPpxoRyctK6RibLOTP34el3dVpZ5j\n\t4S0yyJcLa+IpVJfVUKEdZ3sfPmB16JESVVPSDhjOKtePPiXAbkTArCB6bcqk6T4izmBs\n\temKc6K1Cp7AXv3JrYW3s+Krwm0FgZh9GaUkFyKrCW0IJNNVNKoarfy5+w1lVqyv75cL3\n\tfLNrYlWpKNNsj2o+x4pAXRQ8mx7JbvKdrF9kmNQ+C2vuXFhxurOyjx+y2oafHIKF7mT3\n\t3Gnw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=F14zQdK3W+jHyeh2lWeIDtiFBdnNqfUrvUA301bsVvc=;\n\tb=XJzbX3+Oozww68wplIw6Xc5HNtpjvG4q3sHiI3V5LLCHyoGvJ2ePuXj4cAuhSdfy0w\n\tJ6u6CdycEs2aprPJXsv+BAjMNB6a/nmvJCnojFSWuxO/M++F3EloO+v4rm1wyh51aohG\n\toUupmILhKbI/F651bcbNZmSVl7RFLrXuGHqmn/0Lq/yTg5ymWZ9jTn8T8rq0O8FnrBz8\n\tWSPNJoOxMC5BQNcGRP6yU26VHD/dXT+hJirf/hbKlNUVv437cJx9OZN+yzdOIa2rupUj\n\t0Y3slJZgWADGyeLMJA9L7LUXw+ThGVteJnN9H4HbKMx75pBf+LV4Y/GCCChS3FvZEVz5\n\tjS9w==","X-Gm-Message-State":"AHPjjUjLqw/BqxlxaE6Etrn8wQKLLHp4/wuFl3orB5OhVl/163QSW44A\n\tY7zSmz9vV/uEWzqq02W/CZWBJl9U","X-Google-Smtp-Source":"AOwi7QD+1z0PEk7SeQXDmvk8BudQ/st/v3JwbaaRFBQALpkc2ACbX/QgP8+RMopI46/1xTtuSZxxUw==","X-Received":"by 10.98.218.18 with SMTP id c18mr1415090pfh.256.1505830358204; \n\tTue, 19 Sep 2017 07:12:38 -0700 (PDT)","To":"Paolo Bonzini , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t\n\t<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>\n\t","From":"Alexey Kardashevskiy ","Message-ID":"<2f185edf-3281-e01f-5859-cc43a5df413e@ozlabs.ru>","Date":"Wed, 20 Sep 2017 00:12:33 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::241","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771076,"web_url":"http://patchwork.ozlabs.org/comment/1771076/","msgid":"<89a582f4-e4b8-a8ae-8e8e-2fa83e1fd31d@redhat.com>","list_archive_url":null,"date":"2017-09-19T14:23:13","subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","submitter":{"id":2701,"url":"http://patchwork.ozlabs.org/api/people/2701/","name":"Paolo Bonzini","email":"pbonzini@redhat.com"},"content":"On 19/09/2017 16:12, Alexey Kardashevskiy wrote:\n>>> What is the exact concern about the \"[PATCH qemu v3 01/13] memory: Postpone\n>>> flatview and dispatch tree building till all devices are added\"? Is it that\n>>> you missed that modern AS thing in virtio and thought we can do well even\n>>> without it?\n>> I am afraid that some of the special sysbus devices (not PCI, those go\n>> through the bus master DMA region and it's empty on startup) might write\n>> to memory during initialization.\n> \n> And somehow QEMU guarantees that all RAM MRs are rendered to FlatViews even\n> though some might be added via -device (can they btw?)?\n\nI don't know, but as discussed on IRC, I think this is a pretty\ndifferent change from the rest of the series---which should have zero\nchange except for going faster---so I'd prefer to keep it out for now\nand see where optimization can get us.\n\nPaolo","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxQ9V3RJfz9s7m\n\tfor ;\n\tWed, 20 Sep 2017 00:24:14 +1000 (AEST)","from localhost ([::1]:43208 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duJRc-0002gy-4j\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 10:24:12 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:50649)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duJQn-0002aO-7O\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:23:27 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duJQj-000227-4u\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:23:21 -0400","from mail-wm0-f49.google.com ([74.125.82.49]:47157)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1duJQi-00021b-Vq\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 10:23:17 -0400","by mail-wm0-f49.google.com with SMTP id 13so406611wmq.2\n\tfor ; Tue, 19 Sep 2017 07:23:16 -0700 (PDT)","from [192.168.10.165]\n\t(dynamic-adsl-78-12-246-117.clienti.tiscali.it.\n\t[78.12.246.117]) by smtp.gmail.com with ESMTPSA id\n\tm128sm2764426wmf.0.2017.09.19.07.23.14\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tTue, 19 Sep 2017 07:23:14 -0700 (PDT)"],"X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=F8HLu2YET/ti4IZNVfsVaKwRGi19+V6VqkBAWdiVyZU=;\n\tb=D78o6EkBenctsBtUpEcGyd6LZdkGUu3dMDROAzR24nX4xbK32ZgpOUCZu/GrMIpY/3\n\tctzNgZ8deWMtvAshWKRDFIQiyadQedPQwIS1MtrFVky/HSjuSoqwEVm8qDXDmn9BAeH9\n\t6dN4g8CI7KQQj13D/cj9WydHmJzBNM5eHYhm3z5i2p4kfzvaQOK7bVmB/DpA+vFyFLHe\n\tiwzs2oUC1HX+LrbONvfnS+mlFERtElkCN1MWq3A2kFk0YTCW0jcoTLwSXT75iBGuzaIH\n\trkcCXlyGQrvB8jRior55ATt/qebaIxcfxgzkGFoTh4hQyrAf+EPVjSaxnLWwTWdYDQKO\n\tk3vg==","X-Gm-Message-State":"AHPjjUjKB89G+ME/XJoExgndQAelAq8mkGb9VoyucMRXR3Jxk+W5yZEr\n\tf/U0n8xJE0YB6qs7B6nFZvSoqKco2xs=","X-Google-Smtp-Source":"AOwi7QDTe0U9UuZ2sfz9fTgxkg0tjPQQeyAsCs4CCdITS7BwGFfnCU2tPbe7e1m38kkoZuthSpCAfA==","X-Received":"by 10.28.150.199 with SMTP id y190mr1322060wmd.51.1505830995505; \n\tTue, 19 Sep 2017 07:23:15 -0700 (PDT)","To":"Alexey Kardashevskiy , qemu-devel@nongnu.org","References":"<20170918101709.30421-1-aik@ozlabs.ru>\n\t<20170918101709.30421-2-aik@ozlabs.ru>\n\t\n\t<2debed31-4c28-6dc3-b159-1fdc3fbf87f5@ozlabs.ru>\n\t\n\t\n\t<9c707630-570a-d7a1-76a4-3ddd5ae0f9b6@ozlabs.ru>\n\t\n\t<2f185edf-3281-e01f-5859-cc43a5df413e@ozlabs.ru>","From":"Paolo Bonzini ","Message-ID":"<89a582f4-e4b8-a8ae-8e8e-2fa83e1fd31d@redhat.com>","Date":"Tue, 19 Sep 2017 16:23:13 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<2f185edf-3281-e01f-5859-cc43a5df413e@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-US","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"74.125.82.49","Subject":"Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview\n\tand dispatch tree building till all devices are added","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]