[{"id":1764621,"web_url":"http://patchwork.ozlabs.org/comment/1764621/","msgid":"<20170907095142.GA2194@work-vm>","list_archive_url":null,"date":"2017-09-07T09:51:42","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":48102,"url":"http://patchwork.ozlabs.org/api/people/48102/","name":"Dr. David Alan Gilbert","email":"dgilbert@redhat.com"},"content":"* Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n> \n> What happens ithere is that every virtio block device creates 2 address\n> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n> for busmaster (common pci thing, called after the device name,\n> in my case \"virtio-blk-pci\").\n> \n> Each address_space_init() updates topology for every address space.\n> Every topology update (address_space_update_topology()) creates a new\n> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n> sections (48KB) and destroys the old one.\n> \n> However the dispatch destructor is postponed via RCU which does not\n> get a chance to execute until the machine is initialized but before\n> we get there, memory is not returned to the pool, and this is a lot\n> of memory which grows n^2.\n> \n> These patches are trying to address the memory use and boot time\n> issues but tbh only the first one provides visible outcome.\n\nDo you have a feel for how much memory is saved?\n\nDave\n\n> There are still things to polish and double check the use of RCU,\n> I'd like to get any feedback before proceeding - is this going\n> the right way or way too ugly?\n> \n> \n> This is based on sha1\n> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n> \n> Please comment. Thanks.\n> \n> \n> \n> Alexey Kardashevskiy (4):\n> memory: Postpone flatview and dispatch tree building till all devices\n> are added\n> memory: Prepare for shared flat views\n> memory: Share flat views and dispatch trees between address spaces\n> memory: Add flat views to HMP \"info mtree\"\n> \n> include/exec/memory-internal.h | 6 +-\n> include/exec/memory.h | 93 +++++++++----\n> exec.c | 242 +++++++++++++++++++--------------\n> hw/alpha/typhoon.c | 2 +-\n> hw/dma/rc4030.c | 4 +-\n> hw/i386/amd_iommu.c | 2 +-\n> hw/i386/intel_iommu.c | 9 +-\n> hw/intc/openpic_kvm.c | 2 +-\n> hw/pci-host/apb.c | 2 +-\n> hw/pci/pci.c | 3 +-\n> hw/ppc/spapr_iommu.c | 4 +-\n> hw/s390x/s390-pci-bus.c | 2 +-\n> hw/vfio/common.c | 6 +-\n> hw/virtio/vhost.c | 6 +-\n> memory.c | 299 +++++++++++++++++++++++++++--------------\n> monitor.c | 3 +-\n> vl.c | 4 +\n> hmp-commands-info.hx | 7 +-\n> 18 files changed, 448 insertions(+), 248 deletions(-)\n> \n> -- \n> 2.11.0\n> \n> \n--\nDr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx09.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx09.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=dgilbert@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xnwjf5brWz9t2R\n\tfor ;\n\tThu, 7 Sep 2017 19:52:38 +1000 (AEST)","from localhost ([::1]:39602 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dptUC-00006G-SC\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 05:52:36 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:40900)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dptTU-0008TH-HA\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 05:51:57 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dptTQ-0004TQ-5u\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 05:51:52 -0400","from mx1.redhat.com ([209.132.183.28]:36516)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dptTP-0004Ss-Li\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 05:51:47 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 966A04ACBB;\n\tThu, 7 Sep 2017 09:51:46 +0000 (UTC)","from work-vm (ovpn-117-81.ams2.redhat.com [10.36.117.81])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id E66536058F;\n\tThu, 7 Sep 2017 09:51:44 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 966A04ACBB","Date":"Thu, 7 Sep 2017 10:51:42 +0100","From":"\"Dr. David Alan Gilbert\" ","To":"Alexey Kardashevskiy ","Message-ID":"<20170907095142.GA2194@work-vm>","References":"<20170907092010.3605-1-aik@ozlabs.ru>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170907092010.3605-1-aik@ozlabs.ru>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.38]);\n\tThu, 07 Sep 2017 09:51:46 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1764632,"web_url":"http://patchwork.ozlabs.org/comment/1764632/","msgid":"<20170907100848.GL2735@umbus.fritz.box>","list_archive_url":null,"date":"2017-09-07T10:08:48","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Thu, Sep 07, 2017 at 10:51:42AM +0100, Dr. David Alan Gilbert wrote:\n> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> > This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n> > \n> > What happens ithere is that every virtio block device creates 2 address\n> > spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n> > for busmaster (common pci thing, called after the device name,\n> > in my case \"virtio-blk-pci\").\n> > \n> > Each address_space_init() updates topology for every address space.\n> > Every topology update (address_space_update_topology()) creates a new\n> > dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n> > sections (48KB) and destroys the old one.\n> > \n> > However the dispatch destructor is postponed via RCU which does not\n> > get a chance to execute until the machine is initialized but before\n> > we get there, memory is not returned to the pool, and this is a lot\n> > of memory which grows n^2.\n> > \n> > These patches are trying to address the memory use and boot time\n> > issues but tbh only the first one provides visible outcome.\n> \n> Do you have a feel for how much memory is saved?\n\nI think that's a bit hard to answer.\n\nAs noted above there's O(n^2) (or more) space complexity here - one\nwhich shouldn't be required by the data we actually have to track.\nThat means the amount of \"excess\" memory depends on how many devices\nthere are.\n\nI haven't yet looked at these patches in detail, to know if it truly\nfixes that O(n^2) or just pares the constant. If it does fix the\nO(n^2) then the amount is going to vary from \"probably not enough to\nworry about\" in normal use cases to hundreds of gigabytes in cases\nwith many devices.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"hRpdBZTY\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xnx5R6n4bz9t2f\n\tfor ;\n\tThu, 7 Sep 2017 20:09:47 +1000 (AEST)","from localhost ([::1]:39656 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dptko-0006ia-14\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 06:09:46 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:47068)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dptkH-0006hS-Bq\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 06:09:14 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dptkG-00050C-A0\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 06:09:13 -0400","from ozlabs.org ([2401:3900:2:1::2]:51971)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dptkF-0004z2-Ty\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 06:09:12 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3xnx4V5NKvz9t2f; Thu, 7 Sep 2017 20:08:58 +1000 (AEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1504778938;\n\tbh=3cGMwz28G6KQ059kXnlvQZdWtbt+Gpz1bAALqxdGMTY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=hRpdBZTYNykEibmvYpn5bhJbjfYFNh3T2xo35WQ+mxeCgI6GlBA909Fp7/By9X7CC\n\tH5x57kcjmvjPpyClIopDEQ3nlMJkDgVjtgnEHBeLz1avuCEQky9cnrWN38TZ6/lQHs\n\tkMx2Q+3/kIzPdjbWEd+Z71kHlN1WUalijctFo4TI=","Date":"Thu, 7 Sep 2017 20:08:48 +1000","From":"David Gibson ","To":"\"Dr. David Alan Gilbert\" ","Message-ID":"<20170907100848.GL2735@umbus.fritz.box>","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"qo7zVO9a9OQ5oQtr\"","Content-Disposition":"inline","In-Reply-To":"<20170907095142.GA2194@work-vm>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , Paolo Bonzini ,\n\tPeter Maydell , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1764767,"web_url":"http://patchwork.ozlabs.org/comment/1764767/","msgid":"","list_archive_url":null,"date":"2017-09-07T14:44:45","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 07/09/17 19:51, Dr. David Alan Gilbert wrote:\n> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n>> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n>>\n>> What happens ithere is that every virtio block device creates 2 address\n>> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n>> for busmaster (common pci thing, called after the device name,\n>> in my case \"virtio-blk-pci\").\n>>\n>> Each address_space_init() updates topology for every address space.\n>> Every topology update (address_space_update_topology()) creates a new\n>> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n>> sections (48KB) and destroys the old one.\n>>\n>> However the dispatch destructor is postponed via RCU which does not\n>> get a chance to execute until the machine is initialized but before\n>> we get there, memory is not returned to the pool, and this is a lot\n>> of memory which grows n^2.\n>>\n>> These patches are trying to address the memory use and boot time\n>> issues but tbh only the first one provides visible outcome.\n> \n> Do you have a feel for how much memory is saved?\n\n\nThe 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci\ndevices. These GB figures are the peak values (but it does not matter for\nOOM killer), memory gets released in one go when RCU kicks in, it just\nhappens too late.\n\nThe 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so\nmuch). Strangely, I do not see the difference in valgrind output when I run\na guest with 1024 or just 8 CPUs, probably \"massif\" is not the right tool\nto catch this.\n\n> \n> Dave\n> \n>> There are still things to polish and double check the use of RCU,\n>> I'd like to get any feedback before proceeding - is this going\n>> the right way or way too ugly?\n>>\n>>\n>> This is based on sha1\n>> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n>>\n>> Please comment. Thanks.\n>>\n>>\n>>\n>> Alexey Kardashevskiy (4):\n>> memory: Postpone flatview and dispatch tree building till all devices\n>> are added\n>> memory: Prepare for shared flat views\n>> memory: Share flat views and dispatch trees between address spaces\n>> memory: Add flat views to HMP \"info mtree\"\n>>\n>> include/exec/memory-internal.h | 6 +-\n>> include/exec/memory.h | 93 +++++++++----\n>> exec.c | 242 +++++++++++++++++++--------------\n>> hw/alpha/typhoon.c | 2 +-\n>> hw/dma/rc4030.c | 4 +-\n>> hw/i386/amd_iommu.c | 2 +-\n>> hw/i386/intel_iommu.c | 9 +-\n>> hw/intc/openpic_kvm.c | 2 +-\n>> hw/pci-host/apb.c | 2 +-\n>> hw/pci/pci.c | 3 +-\n>> hw/ppc/spapr_iommu.c | 4 +-\n>> hw/s390x/s390-pci-bus.c | 2 +-\n>> hw/vfio/common.c | 6 +-\n>> hw/virtio/vhost.c | 6 +-\n>> memory.c | 299 +++++++++++++++++++++++++++--------------\n>> monitor.c | 3 +-\n>> vl.c | 4 +\n>> hmp-commands-info.hx | 7 +-\n>> 18 files changed, 448 insertions(+), 248 deletions(-)\n>>\n>> -- \n>> 2.11.0\n>>\n>>\n> --\n> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"bekx9nM6\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xp3Cc6N2Rz9sRY\n\tfor ;\n\tFri, 8 Sep 2017 00:45:30 +1000 (AEST)","from localhost ([::1]:40779 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpy3b-0003qQ-H8\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 10:45:27 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33630)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpy3A-0003mE-PH\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:45:05 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpy32-0004fe-RA\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:45:00 -0400","from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:37009)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1dpy32-0004ef-Ji\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:44:52 -0400","by mail-pf0-x242.google.com with SMTP id a2so4601218pfj.4\n\tfor ; Thu, 07 Sep 2017 07:44:51 -0700 (PDT)","from [192.168.10.22] (124-171-136-142.dyn.iinet.net.au.\n\t[124.171.136.142]) by smtp.googlemail.com with ESMTPSA id\n\te126sm4743847pfh.183.2017.09.07.07.44.47\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 07 Sep 2017 07:44:49 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=yzkaamRsFowInRjsdWwK9iByyEJHzcVGPBJ7TDegkJ0=;\n\tb=bekx9nM6+pLl4/B1aLIzH6y6bS+3rgkWk/2ToPz4q/5GL19wGLWvqyJF0UTUpBE1oc\n\tYUrN4J9wkLYb1ZR3xX8RSBIH8fN+U3oTlyvF5uqn6k9oTxCbLaTqbQ8akbiXv0cq199l\n\tYaRodrcfFmfoo/WXRk6OYDTY1sIko7NS9/Ed1y8EVBfRj6PM5rOqC/s7wHqK7ise41sZ\n\tGBvR8qv6WX9jZpsjkCXTU85dZ5GfxUN2pImAcWFCjWGqD4HDZkDph5Ulyw86D8VHHS7r\n\tTOo0Xsh6/qtRUUQXbKyYbg6WlDlk5x6pjW88qJA7EeK91A4OGmf0SHvxdzxU3VeJWk9y\n\t5erA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=yzkaamRsFowInRjsdWwK9iByyEJHzcVGPBJ7TDegkJ0=;\n\tb=dIjmbPwPsSc3pnXdtEIfFr3yZZ5+ItJlh52+AHQjOb0NxtKDAcEEsqDrfYFsdTNztU\n\txD4m36lclBsPPWyXoStRjkooXr0pm9yBJNsEx3VsCpCt1UjmZ9I7X8C8DKv40Bw7ylMN\n\td2a30xlqE9X/RxOsmRbRGwBd7N0JC+9tiz1TiCEmQdlQ/J6y8JcXjD5l1dbwJPpkIgXb\n\t7KK1vhkN6/pg2RZyMODnjt2Fg0pdm/gnjoC4jpAzpGjStRQD58WtT4Rin7HIRWrpcrl+\n\twvys9DV2rcTetOlKAwc4WI4wi807c4Zz4t15h3cViX+FSi/SVvRnH0byIj6dSSqkAjP1\n\t9pHw==","X-Gm-Message-State":"AHPjjUjMta2JJy/bmvDGJie2gC+Q9/1c3Cn2w4JtY730Ut6gP262jhjS\n\t6EKxm3RWQKJoar33","X-Google-Smtp-Source":"ADKCNb4XEiaY1bnuSTmSmPKcDnDYNwSkzXYfN4oHDrgIHPxhgj9Ivi3jbbrY/BYzvA4pd56bXs/RQQ==","X-Received":"by 10.99.171.9 with SMTP id p9mr2996851pgf.30.1504795491043;\n\tThu, 07 Sep 2017 07:44:51 -0700 (PDT)","To":"\"Dr. David Alan Gilbert\" ","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>","From":"Alexey Kardashevskiy ","Message-ID":"","Date":"Fri, 8 Sep 2017 00:44:45 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<20170907095142.GA2194@work-vm>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c00::242","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1764771,"web_url":"http://patchwork.ozlabs.org/comment/1764771/","msgid":"<20170907145403.GR2098@work-vm>","list_archive_url":null,"date":"2017-09-07T14:54:03","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":48102,"url":"http://patchwork.ozlabs.org/api/people/48102/","name":"Dr. David Alan Gilbert","email":"dgilbert@redhat.com"},"content":"* Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> On 07/09/17 19:51, Dr. David Alan Gilbert wrote:\n> > * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> >> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n> >>\n> >> What happens ithere is that every virtio block device creates 2 address\n> >> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n> >> for busmaster (common pci thing, called after the device name,\n> >> in my case \"virtio-blk-pci\").\n> >>\n> >> Each address_space_init() updates topology for every address space.\n> >> Every topology update (address_space_update_topology()) creates a new\n> >> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n> >> sections (48KB) and destroys the old one.\n> >>\n> >> However the dispatch destructor is postponed via RCU which does not\n> >> get a chance to execute until the machine is initialized but before\n> >> we get there, memory is not returned to the pool, and this is a lot\n> >> of memory which grows n^2.\n> >>\n> >> These patches are trying to address the memory use and boot time\n> >> issues but tbh only the first one provides visible outcome.\n> > \n> > Do you have a feel for how much memory is saved?\n> \n> \n> The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci\n> devices. These GB figures are the peak values (but it does not matter for\n> OOM killer), memory gets released in one go when RCU kicks in, it just\n> happens too late.\n\nNice saving! Still, why is it using 11GB?\nWhat's it like for more sane configurations, say 2-3 virtio devices - is\nthere anything noticable or is it just the huge setups?\n\nDave\n\n\n> The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so\n> much). Strangely, I do not see the difference in valgrind output when I run\n> a guest with 1024 or just 8 CPUs, probably \"massif\" is not the right tool\n> to catch this.\n> \n> > \n> > Dave\n> > \n> >> There are still things to polish and double check the use of RCU,\n> >> I'd like to get any feedback before proceeding - is this going\n> >> the right way or way too ugly?\n> >>\n> >>\n> >> This is based on sha1\n> >> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n> >>\n> >> Please comment. Thanks.\n> >>\n> >>\n> >>\n> >> Alexey Kardashevskiy (4):\n> >> memory: Postpone flatview and dispatch tree building till all devices\n> >> are added\n> >> memory: Prepare for shared flat views\n> >> memory: Share flat views and dispatch trees between address spaces\n> >> memory: Add flat views to HMP \"info mtree\"\n> >>\n> >> include/exec/memory-internal.h | 6 +-\n> >> include/exec/memory.h | 93 +++++++++----\n> >> exec.c | 242 +++++++++++++++++++--------------\n> >> hw/alpha/typhoon.c | 2 +-\n> >> hw/dma/rc4030.c | 4 +-\n> >> hw/i386/amd_iommu.c | 2 +-\n> >> hw/i386/intel_iommu.c | 9 +-\n> >> hw/intc/openpic_kvm.c | 2 +-\n> >> hw/pci-host/apb.c | 2 +-\n> >> hw/pci/pci.c | 3 +-\n> >> hw/ppc/spapr_iommu.c | 4 +-\n> >> hw/s390x/s390-pci-bus.c | 2 +-\n> >> hw/vfio/common.c | 6 +-\n> >> hw/virtio/vhost.c | 6 +-\n> >> memory.c | 299 +++++++++++++++++++++++++++--------------\n> >> monitor.c | 3 +-\n> >> vl.c | 4 +\n> >> hmp-commands-info.hx | 7 +-\n> >> 18 files changed, 448 insertions(+), 248 deletions(-)\n> >>\n> >> -- \n> >> 2.11.0\n> >>\n> >>\n> > --\n> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n> > \n> \n> \n> -- \n> Alexey\n--\nDr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx06.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=dgilbert@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xp3QQ5cX3z9t2r\n\tfor ;\n\tFri, 8 Sep 2017 00:54:53 +1000 (AEST)","from localhost ([::1]:40807 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpyCh-0006j9-4e\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 10:54:51 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:37396)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpyCI-0006hJ-5K\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:54:31 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpyCD-0001vS-AY\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:54:26 -0400","from mx1.redhat.com ([209.132.183.28]:55222)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dpyCD-0001v9-1W\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 10:54:21 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 8591C356F0;\n\tThu, 7 Sep 2017 14:54:19 +0000 (UTC)","from work-vm (ovpn-117-81.ams2.redhat.com [10.36.117.81])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id B69726AD04;\n\tThu, 7 Sep 2017 14:54:12 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 8591C356F0","Date":"Thu, 7 Sep 2017 15:54:03 +0100","From":"\"Dr. David Alan Gilbert\" ","To":"Alexey Kardashevskiy ","Message-ID":"<20170907145403.GR2098@work-vm>","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.30]);\n\tThu, 07 Sep 2017 14:54:20 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765041,"web_url":"http://patchwork.ozlabs.org/comment/1765041/","msgid":"<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","list_archive_url":null,"date":"2017-09-08T02:08:08","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 08/09/17 00:54, Dr. David Alan Gilbert wrote:\n> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n>> On 07/09/17 19:51, Dr. David Alan Gilbert wrote:\n>>> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n>>>> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n>>>>\n>>>> What happens ithere is that every virtio block device creates 2 address\n>>>> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n>>>> for busmaster (common pci thing, called after the device name,\n>>>> in my case \"virtio-blk-pci\").\n>>>>\n>>>> Each address_space_init() updates topology for every address space.\n>>>> Every topology update (address_space_update_topology()) creates a new\n>>>> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n>>>> sections (48KB) and destroys the old one.\n>>>>\n>>>> However the dispatch destructor is postponed via RCU which does not\n>>>> get a chance to execute until the machine is initialized but before\n>>>> we get there, memory is not returned to the pool, and this is a lot\n>>>> of memory which grows n^2.\n>>>>\n>>>> These patches are trying to address the memory use and boot time\n>>>> issues but tbh only the first one provides visible outcome.\n>>>\n>>> Do you have a feel for how much memory is saved?\n>>\n>>\n>> The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci\n>> devices. These GB figures are the peak values (but it does not matter for\n>> OOM killer), memory gets released in one go when RCU kicks in, it just\n>> happens too late.\n> \n> Nice saving! Still, why is it using 11GB?\n\nYet to be discovered :) Not clear at the moment.\n\n\n> What's it like for more sane configurations, say 2-3 virtio devices - is\n> there anything noticable or is it just the huge setups?\n> \n> Dave\n> \n> \n>> The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so\n>> much). Strangely, I do not see the difference in valgrind output when I run\n>> a guest with 1024 or just 8 CPUs, probably \"massif\" is not the right tool\n>> to catch this.\n\nI did some more tests.\n\nv2.10:\n1024 CPUs, no virtio: 0:47 490.8MB 38/34\n1 CPU, 500 virtio-block: 5:03 59.69GB 2354438/3\n\n1/4 applied:\n1024 CPUs, no virtio:\t 0:49 490.8MB 38/34\n1 CPU, 500 virtio-block: 1:57 17.74GB 2186/3\n\n3/4 applied:\n1024 CPUs, no virtio: 0:53 491.1MB 20/17\n1 CPU, 500 virtio-block: 2:01 17.7GB 2167/0\n\n\nTime is what it takes to start QEMU with -S and then Q-Ax.\n\nMemory amount is peak use from valgrind massif.\n\nLast 2 numbers - \"38/34\" for example - 38 is the number of g_new(FlatView,\n1), 34 is the number of g_free(view); the numbers are printed at\nhttps://git.qemu.org/?p=qemu.git;a=blob;f=vl.c;h=8e247cc2a239ae8fb3d3cdf6d4ee78fd723d1053;hb=1ab5eb4efb91a3d4569b0df6e824cc08ab4bd8ec#l4666\nbefore RCU kicks in.\n\n500 virtio-block + bridges use around 1100 address spaces.\n\n\n\n\n\n>>\n>>>\n>>> Dave\n>>>\n>>>> There are still things to polish and double check the use of RCU,\n>>>> I'd like to get any feedback before proceeding - is this going\n>>>> the right way or way too ugly?\n>>>>\n>>>>\n>>>> This is based on sha1\n>>>> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n>>>>\n>>>> Please comment. Thanks.\n>>>>\n>>>>\n>>>>\n>>>> Alexey Kardashevskiy (4):\n>>>> memory: Postpone flatview and dispatch tree building till all devices\n>>>> are added\n>>>> memory: Prepare for shared flat views\n>>>> memory: Share flat views and dispatch trees between address spaces\n>>>> memory: Add flat views to HMP \"info mtree\"\n>>>>\n>>>> include/exec/memory-internal.h | 6 +-\n>>>> include/exec/memory.h | 93 +++++++++----\n>>>> exec.c | 242 +++++++++++++++++++--------------\n>>>> hw/alpha/typhoon.c | 2 +-\n>>>> hw/dma/rc4030.c | 4 +-\n>>>> hw/i386/amd_iommu.c | 2 +-\n>>>> hw/i386/intel_iommu.c | 9 +-\n>>>> hw/intc/openpic_kvm.c | 2 +-\n>>>> hw/pci-host/apb.c | 2 +-\n>>>> hw/pci/pci.c | 3 +-\n>>>> hw/ppc/spapr_iommu.c | 4 +-\n>>>> hw/s390x/s390-pci-bus.c | 2 +-\n>>>> hw/vfio/common.c | 6 +-\n>>>> hw/virtio/vhost.c | 6 +-\n>>>> memory.c | 299 +++++++++++++++++++++++++++--------------\n>>>> monitor.c | 3 +-\n>>>> vl.c | 4 +\n>>>> hmp-commands-info.hx | 7 +-\n>>>> 18 files changed, 448 insertions(+), 248 deletions(-)\n>>>>\n>>>> -- \n>>>> 2.11.0\n>>>>\n>>>>\n>>> --\n>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n>>>\n>>\n>>\n>> -- \n>> Alexey\n> --\n> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"lfwek4sG\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpLVn0zhDz9ryT\n\tfor ;\n\tFri, 8 Sep 2017 12:14:41 +1000 (AEST)","from localhost ([::1]:42977 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dq8oZ-00028o-5E\n\tfor incoming@patchwork.ozlabs.org; Thu, 07 Sep 2017 22:14:39 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33733)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dq8iR-0005D4-Dq\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 22:08:24 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dq8iM-0003lM-1Q\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 22:08:19 -0400","from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:36325)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1dq8iL-0003kf-Od\n\tfor qemu-devel@nongnu.org; Thu, 07 Sep 2017 22:08:13 -0400","by mail-pg0-x242.google.com with SMTP id d8so674164pgt.3\n\tfor ; Thu, 07 Sep 2017 19:08:13 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\th1sm1079917pfg.153.2017.09.07.19.08.09\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 07 Sep 2017 19:08:12 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=noVwxEdd+0Zq6hg/Ym6czyvKyxWMKDoW3G1Li7heAMo=;\n\tb=lfwek4sGzDinnN2u8aSEXD6OwdsSEobFf072YL1vINIZuzZvQRynfiu0NukwllVU7w\n\tTAo0UoAmIZWvBOACPUUTSBCc2E1Bh+rfUqgiGsZ0CymuJJfzFlqPmTuQGooXgPI/P/95\n\tgIM1gJTkw1wmaUJdhSduJM2IcwSsHuhRac1aO/LKeNzFN9CuEZCoI6CSIG+AEY5Qf66R\n\t5ZfV/ZYjkCdW8zKDRqvewJfzfxXlPFAZNo6AVHq6lyzKebEGyXR2AKbNE+2HiYVEQYHa\n\tekDyyMM3bBffWWroon7uZO9v9buIsDlpTG59Sx30f6vnUsPjCjl1+faFl981PdQ8kXxC\n\t1GSw==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=noVwxEdd+0Zq6hg/Ym6czyvKyxWMKDoW3G1Li7heAMo=;\n\tb=Xsn9dD0JXsO2Zc5XhlueyWJKzBIPVpleTlt2yDpJolcn5sXy7xPee92QJTVOYazVse\n\tcyCz1rS+DQlUUtLbndiAMTfBcxK5Y+v/fj3b76ay1TAlh9KPjwQp8Ti7k+duUfm3HYng\n\tDB2t9E4MknYfDP3A1eJr9IeD1hXJ+84nyDi2i54Xa2jzBOxPCLKIfLikDvTMkaoFge4d\n\tfhHomINHGgV8el8sgDydsl6dYneLc8g+SFvS76QekkYLYgAmqtrXKh8AAl+tG9LcXO2j\n\tBjZ6Egk4FNWxuS91eOApQBMmxG7c7XX+cl8akZflwCdvSC9V/lfVGcMhDv9TmPa1eyqD\n\tvPqw==","X-Gm-Message-State":"AHPjjUhP66hpUpK6I2Q0+Qtmn4U6uOLTNjDx19k74nFtf0O9kDVHOWzd\n\trh95irAG7wDUazzY","X-Google-Smtp-Source":"ADKCNb5jGIQbMgjQNtdfCXLfR54Oo9oQJ50GRDgMLn67Ss3mjMcdPApyj7Z4i6seSrK7dfGJIJkCtQ==","X-Received":"by 10.99.125.25 with SMTP id y25mr1450788pgc.45.1504836492648;\n\tThu, 07 Sep 2017 19:08:12 -0700 (PDT)","To":"\"Dr. David Alan Gilbert\" ","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>\n\t\n\t<20170907145403.GR2098@work-vm>","From":"Alexey Kardashevskiy ","Message-ID":"<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","Date":"Fri, 8 Sep 2017 12:08:08 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<20170907145403.GR2098@work-vm>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c05::242","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765070,"web_url":"http://patchwork.ozlabs.org/comment/1765070/","msgid":"<31a8c1fc-c363-55c0-0811-6c5997911f4c@ozlabs.ru>","list_archive_url":null,"date":"2017-09-08T04:04:15","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":7621,"url":"http://patchwork.ozlabs.org/api/people/7621/","name":"Alexey Kardashevskiy","email":"aik@ozlabs.ru"},"content":"On 08/09/17 12:08, Alexey Kardashevskiy wrote:\n> On 08/09/17 00:54, Dr. David Alan Gilbert wrote:\n>> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n>>> On 07/09/17 19:51, Dr. David Alan Gilbert wrote:\n>>>> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n>>>>> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n>>>>>\n>>>>> What happens ithere is that every virtio block device creates 2 address\n>>>>> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n>>>>> for busmaster (common pci thing, called after the device name,\n>>>>> in my case \"virtio-blk-pci\").\n>>>>>\n>>>>> Each address_space_init() updates topology for every address space.\n>>>>> Every topology update (address_space_update_topology()) creates a new\n>>>>> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n>>>>> sections (48KB) and destroys the old one.\n>>>>>\n>>>>> However the dispatch destructor is postponed via RCU which does not\n>>>>> get a chance to execute until the machine is initialized but before\n>>>>> we get there, memory is not returned to the pool, and this is a lot\n>>>>> of memory which grows n^2.\n>>>>>\n>>>>> These patches are trying to address the memory use and boot time\n>>>>> issues but tbh only the first one provides visible outcome.\n>>>>\n>>>> Do you have a feel for how much memory is saved?\n>>>\n>>>\n>>> The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci\n>>> devices. These GB figures are the peak values (but it does not matter for\n>>> OOM killer), memory gets released in one go when RCU kicks in, it just\n>>> happens too late.\n>>\n>> Nice saving! Still, why is it using 11GB?\n> \n> Yet to be discovered :) Not clear at the moment.\n> \n> \n>> What's it like for more sane configurations, say 2-3 virtio devices - is\n>> there anything noticable or is it just the huge setups?\n>>\n>> Dave\n>>\n>>\n>>> The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so\n>>> much). Strangely, I do not see the difference in valgrind output when I run\n>>> a guest with 1024 or just 8 CPUs, probably \"massif\" is not the right tool\n>>> to catch this.\n> \n> I did some more tests.\n> \n> v2.10:\n> 1024 CPUs, no virtio: 0:47 490.8MB 38/34\n> 1 CPU, 500 virtio-block: 5:03 59.69GB 2354438/3\n> \n> 1/4 applied:\n> 1024 CPUs, no virtio:\t 0:49 490.8MB 38/34\n> 1 CPU, 500 virtio-block: 1:57 17.74GB 2186/3\n> \n> 3/4 applied:\n> 1024 CPUs, no virtio: 0:53 491.1MB 20/17\n> 1 CPU, 500 virtio-block: 2:01 17.7GB 2167/0\n> \n> \n> Time is what it takes to start QEMU with -S and then Q-Ax.\n> \n> Memory amount is peak use from valgrind massif.\n> \n> Last 2 numbers - \"38/34\" for example - 38 is the number of g_new(FlatView,\n> 1), 34 is the number of g_free(view); the numbers are printed at\n> https://git.qemu.org/?p=qemu.git;a=blob;f=vl.c;h=8e247cc2a239ae8fb3d3cdf6d4ee78fd723d1053;hb=1ab5eb4efb91a3d4569b0df6e824cc08ab4bd8ec#l4666\n> before RCU kicks in.\n> \n> 500 virtio-block + bridges use around 1100 address spaces.\n\n\nNumbers are quite wrong for the [3/4] patch as\nqemu_run_machine_init_done_notifiers() does the same dance all over again.\n\nI can bring memory peak use down to 3.75GB (yay!) for \"1 CPU, 500\nvirtio-block\" with the below fix. It belongs to [1/4] and we might want\njust extend that begin+commit to cover both devices creation and notifiers\ninitialization.\n\nMy guess that the general problem is that memory_region_transaction_begin()\nand memory_region_transaction_commit() update all address spaces (or flat\nviews after [3/4] applied) while they do not really need to - they could\nreceive a hint (an AS? a flatview) and update only respective bits, I just\ndo not see an easy way to provide with a hint other than storing an AS\npointer in an MR.\n\nIn my test, I have 500 devices and 2 layers of PCI bridges. For every PCI\ndevice probed by SLOF (our boot firmware), _all_ flatviews are rebuild 8\ntimes (at least, sometime more, like 20, when a next bridge gets activated,\ndunno):\n\n- config space writes at 0x20 0x24 0x28 0x2c 0x1c - we end up in\npci_bridge_update_mappings();\n- config space writes at 0x4 for a device and a bridge - pci_update_mappings();\n- memory_region_set_enabled() on d->bus_master_enable_region\nenable bridge.\n\nEach of these operations re-renders all 1000-ish flatviews and dispatch\ntrees, making device probing really, really slow. But at least memory is\nnot sort of leaking or hold by RCU :) And I cannot wrap these config writes\ninto memory_region_transaction_begin/commit as every config space access\ncomes independently from SLOF.\n\n\n\ndiff --git a/vl.c b/vl.c\nindex 89fb58c1de..4317ef01b4 100644\n--- a/vl.c\n+++ b/vl.c\n@@ -4754,8 +4754,13 @@ int main(int argc, char **argv, char **envp)\n /* TODO: once all bus devices are qdevified, this should be done\n * when bus is created by qdev.c */\n qemu_register_reset(qbus_reset_all_fn, sysbus_get_default());\n+\n+ memory_region_transaction_begin();\n+\n qemu_run_machine_init_done_notifiers();\n\n+ memory_region_transaction_commit();\n+\n if (rom_check_and_register_reset() != 0) {\n error_report(\"rom check and register reset failed\");\n exit(1);\n\n\n\n\n> \n> \n> \n> \n> \n>>>\n>>>>\n>>>> Dave\n>>>>\n>>>>> There are still things to polish and double check the use of RCU,\n>>>>> I'd like to get any feedback before proceeding - is this going\n>>>>> the right way or way too ugly?\n>>>>>\n>>>>>\n>>>>> This is based on sha1\n>>>>> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n>>>>>\n>>>>> Please comment. Thanks.\n>>>>>\n>>>>>\n>>>>>\n>>>>> Alexey Kardashevskiy (4):\n>>>>> memory: Postpone flatview and dispatch tree building till all devices\n>>>>> are added\n>>>>> memory: Prepare for shared flat views\n>>>>> memory: Share flat views and dispatch trees between address spaces\n>>>>> memory: Add flat views to HMP \"info mtree\"\n>>>>>\n>>>>> include/exec/memory-internal.h | 6 +-\n>>>>> include/exec/memory.h | 93 +++++++++----\n>>>>> exec.c | 242 +++++++++++++++++++--------------\n>>>>> hw/alpha/typhoon.c | 2 +-\n>>>>> hw/dma/rc4030.c | 4 +-\n>>>>> hw/i386/amd_iommu.c | 2 +-\n>>>>> hw/i386/intel_iommu.c | 9 +-\n>>>>> hw/intc/openpic_kvm.c | 2 +-\n>>>>> hw/pci-host/apb.c | 2 +-\n>>>>> hw/pci/pci.c | 3 +-\n>>>>> hw/ppc/spapr_iommu.c | 4 +-\n>>>>> hw/s390x/s390-pci-bus.c | 2 +-\n>>>>> hw/vfio/common.c | 6 +-\n>>>>> hw/virtio/vhost.c | 6 +-\n>>>>> memory.c | 299 +++++++++++++++++++++++++++--------------\n>>>>> monitor.c | 3 +-\n>>>>> vl.c | 4 +\n>>>>> hmp-commands-info.hx | 7 +-\n>>>>> 18 files changed, 448 insertions(+), 248 deletions(-)\n>>>>>\n>>>>> -- \n>>>>> 2.11.0\n>>>>>\n>>>>>\n>>>> --\n>>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n>>>>\n>>>\n>>>\n>>> -- \n>>> Alexey\n>> --\n>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n>>\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=ozlabs-ru.20150623.gappssmtp.com\n\theader.i=@ozlabs-ru.20150623.gappssmtp.com\n\theader.b=\"A1jC/mcV\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpNy60SD5z9sBZ\n\tfor ;\n\tFri, 8 Sep 2017 14:05:00 +1000 (AEST)","from localhost ([::1]:43206 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqAXI-0003Jv-OB\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 00:04:56 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48068)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqAWq-0003JJ-PK\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 00:04:34 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqAWl-00017c-DM\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 00:04:28 -0400","from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:38508)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from ) id 1dqAWl-00016D-4v\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 00:04:23 -0400","by mail-pf0-x244.google.com with SMTP id q76so825089pfq.5\n\tfor ; Thu, 07 Sep 2017 21:04:21 -0700 (PDT)","from [10.61.2.175] ([122.99.82.10])\n\tby smtp.googlemail.com with ESMTPSA id\n\tm9sm1095151pgt.49.2017.09.07.21.04.17\n\t(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);\n\tThu, 07 Sep 2017 21:04:19 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=ozlabs-ru.20150623.gappssmtp.com; s=20150623;\n\th=subject:from:to:cc:references:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=HXOXwnZ5Im2fIm/zlLwmYTbAOoh7jLSRHEIqKp7d120=;\n\tb=A1jC/mcVfbg5IbkLacAKpCOcfbFdrS6wO0IIuy1IDjLlzrABkH0iODHApuUkmfcdKi\n\tezneUqW9gRk9gGDsiI2m7ELQDliICo242LBTEhgU6Q8USUZOeVClCd6g3zo78ydoC7QF\n\tdUvBivpmYgCTrEfLXbTgFm1oschRCK3TMO2fuyMZ6ay0+QkfjDdfRI39/IeWQqm62v1c\n\tMmg7OdbxM3qZuEICdPUCRahPBuj1GeD0uTj7O0KcBlKFCTL0MjUHvc1r73A5upI+UrXc\n\tSdmKq6ifYPBiENR62Os8pceuRjqMd+r14rOwmv3oYn9r1LP83RrKb4HVGTDUVVSbzNmF\n\tXZkQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:from:to:cc:references:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=HXOXwnZ5Im2fIm/zlLwmYTbAOoh7jLSRHEIqKp7d120=;\n\tb=H0WLxhi8/KuTqJmSbYhI+sT7t3LXxKx5B1z6ShC/Ip/FXawTt0zbL82CGl0bo6+0VK\n\thIXvxq3BeMON3bR8IZy6f4KaHcmLJfs1LDmRr6ErFO+8rjjgHMaUo4NSaihhvQhEs1JY\n\tUnoffNgPCNXlhISPpPR29qfj4/RbY29XbrQKgZntxXBuDEzPdJ9CQbz4YWVKnueB8McJ\n\tQnza4LtOkj2IhhNc9v7GqD4SMBo/YghHIriZpzRrkoZETzUeBQ6Qfi9yrUSUiz9l2sJH\n\t5f/afcMzMvkoLfsWr4WxtIP3MP7W6tiNDh9Q6ARlABYAQbbuRhJzJ4Hf0gIjFtXyEZCd\n\tYZsg==","X-Gm-Message-State":"AHPjjUgy5vIsr/zXmt2M1Q8RWg98sF7BD8SpLJUoPifsC3MHWWzggM/u\n\trTf6iZQZoQws2egR","X-Google-Smtp-Source":"ADKCNb6AFNo2+Wol2dxuScVK4iT+t+WR2mt2vXV5dkUTXBKviOJittCyZmV6Wuqu0gUYVhWhobiupg==","X-Received":"by 10.84.213.23 with SMTP id f23mr1882181pli.214.1504843460725; \n\tThu, 07 Sep 2017 21:04:20 -0700 (PDT)","From":"Alexey Kardashevskiy ","To":"\"Dr. David Alan Gilbert\" ","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>\n\t\n\t<20170907145403.GR2098@work-vm>\n\t<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","Message-ID":"<31a8c1fc-c363-55c0-0811-6c5997911f4c@ozlabs.ru>","Date":"Fri, 8 Sep 2017 14:04:15 +1000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.2.1","MIME-Version":"1.0","In-Reply-To":"<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-AU","Content-Transfer-Encoding":"7bit","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2607:f8b0:400e:c00::244","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765263,"web_url":"http://patchwork.ozlabs.org/comment/1765263/","msgid":"<20170908111214.GB2095@work-vm>","list_archive_url":null,"date":"2017-09-08T11:12:14","subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","submitter":{"id":48102,"url":"http://patchwork.ozlabs.org/api/people/48102/","name":"Dr. David Alan Gilbert","email":"dgilbert@redhat.com"},"content":"* Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> On 08/09/17 00:54, Dr. David Alan Gilbert wrote:\n> > * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> >> On 07/09/17 19:51, Dr. David Alan Gilbert wrote:\n> >>> * Alexey Kardashevskiy (aik@ozlabs.ru) wrote:\n> >>>> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593\n> >>>>\n> >>>> What happens ithere is that every virtio block device creates 2 address\n> >>>> spaces - for modern config space (called \"virtio-pci-cfg-as\") and\n> >>>> for busmaster (common pci thing, called after the device name,\n> >>>> in my case \"virtio-blk-pci\").\n> >>>>\n> >>>> Each address_space_init() updates topology for every address space.\n> >>>> Every topology update (address_space_update_topology()) creates a new\n> >>>> dispatch tree - AddressSpaceDispatch with nodes (1KB) and\n> >>>> sections (48KB) and destroys the old one.\n> >>>>\n> >>>> However the dispatch destructor is postponed via RCU which does not\n> >>>> get a chance to execute until the machine is initialized but before\n> >>>> we get there, memory is not returned to the pool, and this is a lot\n> >>>> of memory which grows n^2.\n> >>>>\n> >>>> These patches are trying to address the memory use and boot time\n> >>>> issues but tbh only the first one provides visible outcome.\n> >>>\n> >>> Do you have a feel for how much memory is saved?\n> >>\n> >>\n> >> The 1/4 saves ~33GB (~44GB -> 11GB) for a 2GB guest and 400 virtio-pci\n> >> devices. These GB figures are the peak values (but it does not matter for\n> >> OOM killer), memory gets released in one go when RCU kicks in, it just\n> >> happens too late.\n> > \n> > Nice saving! Still, why is it using 11GB?\n> \n> Yet to be discovered :) Not clear at the moment.\n> \n> \n> > What's it like for more sane configurations, say 2-3 virtio devices - is\n> > there anything noticable or is it just the huge setups?\n> > \n> > Dave\n> > \n> > \n> >> The 3/4 saves less, I'd say 50KB per VCPU (more if you count peaks but so\n> >> much). Strangely, I do not see the difference in valgrind output when I run\n> >> a guest with 1024 or just 8 CPUs, probably \"massif\" is not the right tool\n> >> to catch this.\n> \n> I did some more tests.\n> \n> v2.10:\n> 1024 CPUs, no virtio: 0:47 490.8MB 38/34\n> 1 CPU, 500 virtio-block: 5:03 59.69GB 2354438/3\n> \n> 1/4 applied:\n> 1024 CPUs, no virtio:\t 0:49 490.8MB 38/34\n> 1 CPU, 500 virtio-block: 1:57 17.74GB 2186/3\n> \n> 3/4 applied:\n> 1024 CPUs, no virtio: 0:53 491.1MB 20/17\n> 1 CPU, 500 virtio-block: 2:01 17.7GB 2167/0\n> \n> \n> Time is what it takes to start QEMU with -S and then Q-Ax.\n> \n> Memory amount is peak use from valgrind massif.\n> \n> Last 2 numbers - \"38/34\" for example - 38 is the number of g_new(FlatView,\n> 1), 34 is the number of g_free(view); the numbers are printed at\n> https://git.qemu.org/?p=qemu.git;a=blob;f=vl.c;h=8e247cc2a239ae8fb3d3cdf6d4ee78fd723d1053;hb=1ab5eb4efb91a3d4569b0df6e824cc08ab4bd8ec#l4666\n> before RCU kicks in.\n> \n> 500 virtio-block + bridges use around 1100 address spaces.\n\nWhat I find interesting is the effect even on small VMs, I'm using\nvalgrind --tool=exp-dhat as per your bz comment, on a qemu close to\nhead:\n\nvalgrind --tool=exp-dhat ~/try/x86_64-softmmu/qemu-system-x86_64 -nographic -device sga -m 1G -M pc,accel=kvm -drive file=/home/vmimages/littlefed20.img,id=d1,if=none -device virtio-blk,drive=d1 -drive file=/home/vmimages/dummy1,id=d2,if=none -device virtio-blk,drive=d2 -drive file=/home/vmimages/dummy2,id=d3,if=none -device virtio-blk,drive=d3 -device virtio-serial -device virtio-serial -device virtio-serial -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0\n\n==5945== guest_insns: 2,845,498,404\n==5945== max_live: 73,745,261 in 45,395 blocks\n==5945== tot_alloc: 615,696,752 in 515,110 blocks\n\nwith your 1-4 patches:\n==14661== guest_insns: 2,626,826,254\n==14661== max_live: 27,825,659 in 28,950 blocks\n==14661== tot_alloc: 529,978,686 in 444,043 blocks\n\nso that's a 45MB saving on a simple VM - those type of numbers add up\nfor people running lots of small VMs; they notice when their total\nqemu RAM overhead for their box goes up by a few GB.\n\nDave\n\n> \n> \n> \n> \n> >>\n> >>>\n> >>> Dave\n> >>>\n> >>>> There are still things to polish and double check the use of RCU,\n> >>>> I'd like to get any feedback before proceeding - is this going\n> >>>> the right way or way too ugly?\n> >>>>\n> >>>>\n> >>>> This is based on sha1\n> >>>> 1ab5eb4efb Peter Maydell \"Update version for v2.10.0 release\".\n> >>>>\n> >>>> Please comment. Thanks.\n> >>>>\n> >>>>\n> >>>>\n> >>>> Alexey Kardashevskiy (4):\n> >>>> memory: Postpone flatview and dispatch tree building till all devices\n> >>>> are added\n> >>>> memory: Prepare for shared flat views\n> >>>> memory: Share flat views and dispatch trees between address spaces\n> >>>> memory: Add flat views to HMP \"info mtree\"\n> >>>>\n> >>>> include/exec/memory-internal.h | 6 +-\n> >>>> include/exec/memory.h | 93 +++++++++----\n> >>>> exec.c | 242 +++++++++++++++++++--------------\n> >>>> hw/alpha/typhoon.c | 2 +-\n> >>>> hw/dma/rc4030.c | 4 +-\n> >>>> hw/i386/amd_iommu.c | 2 +-\n> >>>> hw/i386/intel_iommu.c | 9 +-\n> >>>> hw/intc/openpic_kvm.c | 2 +-\n> >>>> hw/pci-host/apb.c | 2 +-\n> >>>> hw/pci/pci.c | 3 +-\n> >>>> hw/ppc/spapr_iommu.c | 4 +-\n> >>>> hw/s390x/s390-pci-bus.c | 2 +-\n> >>>> hw/vfio/common.c | 6 +-\n> >>>> hw/virtio/vhost.c | 6 +-\n> >>>> memory.c | 299 +++++++++++++++++++++++++++--------------\n> >>>> monitor.c | 3 +-\n> >>>> vl.c | 4 +\n> >>>> hmp-commands-info.hx | 7 +-\n> >>>> 18 files changed, 448 insertions(+), 248 deletions(-)\n> >>>>\n> >>>> -- \n> >>>> 2.11.0\n> >>>>\n> >>>>\n> >>> --\n> >>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n> >>>\n> >>\n> >>\n> >> -- \n> >> Alexey\n> > --\n> > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK\n> > \n> \n> \n> -- \n> Alexey\n--\nDr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=dgilbert@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xpZmR1ntrz9s83\n\tfor ;\n\tFri, 8 Sep 2017 21:27:19 +1000 (AEST)","from localhost ([::1]:44757 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqHRN-0007NL-AV\n\tfor incoming@patchwork.ozlabs.org; Fri, 08 Sep 2017 07:27:17 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45391)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqHCz-0000rR-M5\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 07:12:31 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqHCu-0007uq-Hf\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 07:12:25 -0400","from mx1.redhat.com ([209.132.183.28]:51918)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dqHCu-0007uA-8e\n\tfor qemu-devel@nongnu.org; Fri, 08 Sep 2017 07:12:20 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 209397E424;\n\tFri, 8 Sep 2017 11:12:19 +0000 (UTC)","from work-vm (ovpn-116-125.ams2.redhat.com [10.36.116.125])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 73A41600CA;\n\tFri, 8 Sep 2017 11:12:17 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 209397E424","Date":"Fri, 8 Sep 2017 12:12:14 +0100","From":"\"Dr. David Alan Gilbert\" ","To":"Alexey Kardashevskiy ","Message-ID":"<20170908111214.GB2095@work-vm>","References":"<20170907092010.3605-1-aik@ozlabs.ru>\n\t<20170907095142.GA2194@work-vm>\n\t\n\t<20170907145403.GR2098@work-vm>\n\t<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<3b4eb5d7-a73a-b0b4-97d4-8033cc3aed8c@ozlabs.ru>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]);\n\tFri, 08 Sep 2017 11:12:19 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Peter Maydell ,\n\tPaolo Bonzini ,\n\tDavid Gibson ,\n\tqemu-devel@nongnu.org, Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]