[{"id":1763363,"web_url":"http://patchwork.ozlabs.org/comment/1763363/","msgid":"<20170905131648.GW7570@localhost.localdomain>","list_archive_url":null,"date":"2017-09-05T13:16:48","subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","submitter":{"id":195,"url":"http://patchwork.ozlabs.org/api/people/195/","name":"Eduardo Habkost","email":"ehabkost@redhat.com"},"content":"On Tue, Sep 05, 2017 at 05:30:05PM +0800, Gonglei wrote:\n> Starting with Windows Server 2012 and Windows 8, if\n> CPUID.40000005.EAX contains a value of -1, Windows assumes specific\n> limit to the number of VPs. In this case, Windows Server 2012\n> guest VMs may use more than 64 VPs, up to the maximum supported\n> number of processors applicable to the specific Windows\n> version being used.\n> \n> https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs\n> \n> For compatibility, Let's introduce a new property for X86CPU,\n> named \"hv-cpuid-limits-eax\" as Paolo's suggestion, and set it\n> to \"on\" before machine 2.10.\n> \n> Signed-off-by: Gonglei \n> ---\n> include/hw/i386/pc.h | 5 +++++\n> target/i386/cpu.c | 1 +\n> target/i386/cpu.h | 2 ++\n> target/i386/kvm.c | 18 +++++++++++++++++-\n> 4 files changed, 25 insertions(+), 1 deletion(-)\n> \n> diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h\n> index 8226904..db32e58 100644\n> --- a/include/hw/i386/pc.h\n> +++ b/include/hw/i386/pc.h\n> @@ -371,6 +371,11 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);\n> \n> #define PC_COMPAT_2_10 \\\n> HW_COMPAT_2_10 \\\n> + {\\\n> + .driver = TYPE_X86_CPU,\\\n> + .property = \"hv_cpuid_limits_eax\",\\\n\nThe property name is hv-cpuid-limits-eax.\n\n> + .value = \"on\",\\\n> + },\\\n> \n> #define PC_COMPAT_2_9 \\\n> HW_COMPAT_2_9 \\\n> diff --git a/target/i386/cpu.c b/target/i386/cpu.c\n> index 69676e1..0d47bdd 100644\n> --- a/target/i386/cpu.c\n> +++ b/target/i386/cpu.c\n> @@ -4145,6 +4145,7 @@ static Property x86_cpu_properties[] = {\n> false),\n> DEFINE_PROP_BOOL(\"vmware-cpuid-freq\", X86CPU, vmware_cpuid_freq, true),\n> DEFINE_PROP_BOOL(\"tcg-cpuid\", X86CPU, expose_tcg, true),\n> + DEFINE_PROP_BOOL(\"hv-cpuid-limits-eax\", X86CPU, hv_cpuid_limits_eax, false),\n\nThe property name \"hv-cpuid-limits-eax\" doesn't say anything\nabout what it does exactly when set to true.\n\nWhat about just making it int32? e.g.:\n\n DEFINE_PROP_INT32(\"x-hv-max-vps\", X86CPU, hv_max_vps, -1)\n[...]\n {\\\n .driver = TYPE_X86_CPU,\\\n .property = \"x-hv-max-vps\",\\\n .value = \"0x40\",\\\n },\\\n[...]\n c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n c->eax = cpu->hv_max_vps;\n\n\n(The \"x-\" prefix indicates that the property is not supposed to\nbe a stable user interface.)\n\n\n> DEFINE_PROP_END_OF_LIST()\n> };\n> \n> diff --git a/target/i386/cpu.h b/target/i386/cpu.h\n> index 525d35d..f8b455a 100644\n> --- a/target/i386/cpu.h\n> +++ b/target/i386/cpu.h\n> @@ -1282,6 +1282,8 @@ struct X86CPU {\n> int32_t socket_id;\n> int32_t core_id;\n> int32_t thread_id;\n> +\n> + bool hv_cpuid_limits_eax;\n> };\n> \n> static inline X86CPU *x86_env_get_cpu(CPUX86State *env)\n> diff --git a/target/i386/kvm.c b/target/i386/kvm.c\n> index 6db7783..cf6ef96 100644\n> --- a/target/i386/kvm.c\n> +++ b/target/i386/kvm.c\n> @@ -751,7 +751,23 @@ int kvm_arch_init_vcpu(CPUState *cs)\n> \n> c = &cpuid_data.entries[cpuid_i++];\n> c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> - c->eax = 0x40;\n> +\n> + if (!cpu->hv_cpuid_limits_eax) {\n> + /*\n> + * Starting with Windows Server 2012 and Windows 8, if\n> + * CPUID.40000005.EAX contains a value of -1, Windows\n> + * assumes specific limit to the number of VPs. In this case,\n> + * Windows Server 2012 guest VMs may use more than 64 VPs,\n> + * up to the maximum supported number of processors\n> + * applicable to the specific Windows version being used.\n\nThat was a direct quote from a document, so I recommend citing\nthe specific document you quoted. e.g.:\n\n /*\n * From \"Requirements for Implementing the Microsoft\n * Hypervisor Interface\":\n * https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs\n *\n * \"Starting with Windows Server 2012 and Windows 8, if\n * CPUID.40000005.EAX contains a value of -1, Windows assumes\n * specific limit to the number of VPs. In this case, Windows\n * Server 2012 guest VMs may use more than 64 VPs, up to the\n * maximum supported number of processors applicable to the\n * specific Windows version being used.\"\n */\n\n\n> + *\n> + * https://docs.microsoft.com/en-us/virtualization/\n> + * hyper-v-on-windows/reference/tlfs\n\nIMO a long line is preferable to a broken URL.\n\n> + */\n> + c->eax = -1;\n> + } else {\n> + c->eax = 0x40;\n> + }\n> c->ebx = 0x40;\n> \n> kvm_base = KVM_CPUID_SIGNATURE_NEXT;\n> -- \n> 1.8.3.1\n> \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=ehabkost@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xmnsx3R8bz9t2W\n\tfor ;\n\tTue, 5 Sep 2017 23:40:52 +1000 (AEST)","from localhost ([::1]:59007 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpE5u-0008BX-WE\n\tfor incoming@patchwork.ozlabs.org; Tue, 05 Sep 2017 09:40:47 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:55651)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpE5G-00084i-BD\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 09:40:15 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpE56-0004q1-QI\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 09:40:06 -0400","from mx1.redhat.com ([209.132.183.28]:51406)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dpE56-0004oV-EI\n\tfor qemu-devel@nongnu.org; Tue, 05 Sep 2017 09:39:56 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 88B3CC01FA91;\n\tTue, 5 Sep 2017 13:39:54 +0000 (UTC)","from localhost (ovpn-116-45.gru2.redhat.com [10.97.116.45])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 5C89418200;\n\tTue, 5 Sep 2017 13:39:51 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 88B3CC01FA91","Date":"Tue, 5 Sep 2017 10:16:48 -0300","From":"Eduardo Habkost ","To":"Gonglei ","Message-ID":"<20170905131648.GW7570@localhost.localdomain>","References":"<1504603805-180240-1-git-send-email-arei.gonglei@huawei.com>\n\t<1504603805-180240-3-git-send-email-arei.gonglei@huawei.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<1504603805-180240-3-git-send-email-arei.gonglei@huawei.com>","X-Fnord":"you can see the fnord","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tTue, 05 Sep 2017 13:39:54 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"weidong.huang@huawei.com, mst@redhat.com, mtosatti@redhat.com,\n\tqemu-devel@nongnu.org, vrozenfe@redhat.com, pbonzini@redhat.com,\n\trth@twiddle.net","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1764448,"web_url":"http://patchwork.ozlabs.org/comment/1764448/","msgid":"<33183CC9F5247A488A2544077AF19020DA3A2534@DGGEMA505-MBX.china.huawei.com>","list_archive_url":null,"date":"2017-09-07T01:05:33","subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","submitter":{"id":35948,"url":"http://patchwork.ozlabs.org/api/people/35948/","name":"Gonglei","email":"arei.gonglei@huawei.com"},"content":"> -----Original Message-----\n> From: Eduardo Habkost [mailto:ehabkost@redhat.com]\n> Sent: Tuesday, September 05, 2017 9:17 PM\n> To: Gonglei (Arei)\n> Cc: qemu-devel@nongnu.org; mst@redhat.com; pbonzini@redhat.com;\n> rth@twiddle.net; mtosatti@redhat.com; vrozenfe@redhat.com;\n> Huangweidong (C)\n> Subject: Re: [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for windows\n> guests\n> \n> On Tue, Sep 05, 2017 at 05:30:05PM +0800, Gonglei wrote:\n> > Starting with Windows Server 2012 and Windows 8, if\n> > CPUID.40000005.EAX contains a value of -1, Windows assumes specific\n> > limit to the number of VPs. In this case, Windows Server 2012\n> > guest VMs may use more than 64 VPs, up to the maximum supported\n> > number of processors applicable to the specific Windows\n> > version being used.\n> >\n> >\n> https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/referenc\n> e/tlfs\n> >\n> > For compatibility, Let's introduce a new property for X86CPU,\n> > named \"hv-cpuid-limits-eax\" as Paolo's suggestion, and set it\n> > to \"on\" before machine 2.10.\n> >\n> > Signed-off-by: Gonglei \n> > ---\n> > include/hw/i386/pc.h | 5 +++++\n> > target/i386/cpu.c | 1 +\n> > target/i386/cpu.h | 2 ++\n> > target/i386/kvm.c | 18 +++++++++++++++++-\n> > 4 files changed, 25 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h\n> > index 8226904..db32e58 100644\n> > --- a/include/hw/i386/pc.h\n> > +++ b/include/hw/i386/pc.h\n> > @@ -371,6 +371,11 @@ bool e820_get_entry(int, uint32_t, uint64_t *,\n> uint64_t *);\n> >\n> > #define PC_COMPAT_2_10 \\\n> > HW_COMPAT_2_10 \\\n> > + {\\\n> > + .driver = TYPE_X86_CPU,\\\n> > + .property = \"hv_cpuid_limits_eax\",\\\n> \n> The property name is hv-cpuid-limits-eax.\n> \nMake sense to me.\n\n> > + .value = \"on\",\\\n> > + },\\\n> >\n> > #define PC_COMPAT_2_9 \\\n> > HW_COMPAT_2_9 \\\n> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c\n> > index 69676e1..0d47bdd 100644\n> > --- a/target/i386/cpu.c\n> > +++ b/target/i386/cpu.c\n> > @@ -4145,6 +4145,7 @@ static Property x86_cpu_properties[] = {\n> > false),\n> > DEFINE_PROP_BOOL(\"vmware-cpuid-freq\", X86CPU,\n> vmware_cpuid_freq, true),\n> > DEFINE_PROP_BOOL(\"tcg-cpuid\", X86CPU, expose_tcg, true),\n> > + DEFINE_PROP_BOOL(\"hv-cpuid-limits-eax\", X86CPU,\n> hv_cpuid_limits_eax, false),\n> \n> The property name \"hv-cpuid-limits-eax\" doesn't say anything\n> about what it does exactly when set to true.\n> \n> What about just making it int32? e.g.:\n> \n> DEFINE_PROP_INT32(\"x-hv-max-vps\", X86CPU, hv_max_vps, -1)\n> [...]\n> {\\\n> .driver = TYPE_X86_CPU,\\\n> .property = \"x-hv-max-vps\",\\\n> .value = \"0x40\",\\\n> },\\\n> [...]\n> c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> c->eax = cpu->hv_max_vps;\n> \n> \n> (The \"x-\" prefix indicates that the property is not supposed to\n> be a stable user interface.)\n> \nI thought about this as well.\nbut actually we can't assure that users set the x-hv-max-vps an invalid value if\nwe use this method. Do we really need to expose eax directly?\n\n> \n> > DEFINE_PROP_END_OF_LIST()\n> > };\n> >\n> > diff --git a/target/i386/cpu.h b/target/i386/cpu.h\n> > index 525d35d..f8b455a 100644\n> > --- a/target/i386/cpu.h\n> > +++ b/target/i386/cpu.h\n> > @@ -1282,6 +1282,8 @@ struct X86CPU {\n> > int32_t socket_id;\n> > int32_t core_id;\n> > int32_t thread_id;\n> > +\n> > + bool hv_cpuid_limits_eax;\n> > };\n> >\n> > static inline X86CPU *x86_env_get_cpu(CPUX86State *env)\n> > diff --git a/target/i386/kvm.c b/target/i386/kvm.c\n> > index 6db7783..cf6ef96 100644\n> > --- a/target/i386/kvm.c\n> > +++ b/target/i386/kvm.c\n> > @@ -751,7 +751,23 @@ int kvm_arch_init_vcpu(CPUState *cs)\n> >\n> > c = &cpuid_data.entries[cpuid_i++];\n> > c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> > - c->eax = 0x40;\n> > +\n> > + if (!cpu->hv_cpuid_limits_eax) {\n> > + /*\n> > + * Starting with Windows Server 2012 and Windows 8, if\n> > + * CPUID.40000005.EAX contains a value of -1, Windows\n> > + * assumes specific limit to the number of VPs. In this case,\n> > + * Windows Server 2012 guest VMs may use more than 64\n> VPs,\n> > + * up to the maximum supported number of processors\n> > + * applicable to the specific Windows version being used.\n> \n> That was a direct quote from a document, so I recommend citing\n> the specific document you quoted. e.g.:\n> \n> /*\n> * From \"Requirements for Implementing the Microsoft\n> * Hypervisor Interface\":\n> *\n> https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/referenc\n> e/tlfs\n> *\n> * \"Starting with Windows Server 2012 and Windows 8, if\n> * CPUID.40000005.EAX contains a value of -1, Windows assumes\n> * specific limit to the number of VPs. In this case, Windows\n> * Server 2012 guest VMs may use more than 64 VPs, up to the\n> * maximum supported number of processors applicable to the\n> * specific Windows version being used.\"\n> */\n> \n> \n> > + *\n> > + * https://docs.microsoft.com/en-us/virtualization/\n> > + * hyper-v-on-windows/reference/tlfs\n> \n> IMO a long line is preferable to a broken URL.\n> \nMake sense to me.\n\nThanks,\n-Gonglei","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xnj2R0HZNz9sCZ\n\tfor ;\n\tThu, 7 Sep 2017 11:06:20 +1000 (AEST)","from localhost ([::1]:38510 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dplGr-0003kZ-QM\n\tfor incoming@patchwork.ozlabs.org; Wed, 06 Sep 2017 21:06:17 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:58318)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dplGV-0003kL-SN\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 21:05:57 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dplGS-0006Yy-LG\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 21:05:55 -0400","from szxga03-in.huawei.com ([45.249.212.189]:4354)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)\n\t(envelope-from ) id 1dplGR-0006Xc-LH\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 21:05:52 -0400","from 172.30.72.54 (EHLO DGGEMA402-HUB.china.huawei.com)\n\t([172.30.72.54])\n\tby dggrg03-dlp.huawei.com (MOS 4.4.6-GA FastPath queued)\n\twith ESMTP id AUW77765; Thu, 07 Sep 2017 09:05:42 +0800 (CST)","from DGGEMA505-MBX.china.huawei.com ([169.254.1.46]) by\n\tDGGEMA402-HUB.china.huawei.com ([10.3.20.43]) with mapi id\n\t14.03.0301.000; Thu, 7 Sep 2017 09:05:33 +0800"],"From":"\"Gonglei (Arei)\" ","To":"Eduardo Habkost ","Thread-Topic":"[PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for windows\n\tguests","Thread-Index":"AQHTJimZLXU3wSlkuE64qwXCq+YQuKKlwAUAgALcSRA=","Date":"Thu, 7 Sep 2017 01:05:33 +0000","Message-ID":"<33183CC9F5247A488A2544077AF19020DA3A2534@DGGEMA505-MBX.china.huawei.com>","References":"<1504603805-180240-1-git-send-email-arei.gonglei@huawei.com>\n\t<1504603805-180240-3-git-send-email-arei.gonglei@huawei.com>\n\t<20170905131648.GW7570@localhost.localdomain>","In-Reply-To":"<20170905131648.GW7570@localhost.localdomain>","Accept-Language":"zh-CN, en-US","Content-Language":"zh-CN","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[10.177.18.62]","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"quoted-printable","MIME-Version":"1.0","X-CFilter-Loop":"Reflected","X-Mirapoint-Virus-RAPID-Raw":"score=unknown(0),\n\trefid=str=0001.0A020203.59B09B66.0107, ss=1, re=0.000, recu=0.000,\n\treip=0.000, cl=1, cld=1, fgs=0, ip=169.254.1.46,\n\tso=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32","X-Mirapoint-Loop-Id":"ad8551ff4d16591e3a47d2928158dec2","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic]\n\t[fuzzy]","X-Received-From":"45.249.212.189","Subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Huangweidong \\(C\\)\" ,\n\t\"mst@redhat.com\" ,\n\t\"mtosatti@redhat.com\" ,\n\t\"qemu-devel@nongnu.org\" ,\n\t\"vrozenfe@redhat.com\" ,\n\t\"pbonzini@redhat.com\" ,\n\t\"rth@twiddle.net\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1765863,"web_url":"http://patchwork.ozlabs.org/comment/1765863/","msgid":"<20170909204629.GI7570@localhost.localdomain>","list_archive_url":null,"date":"2017-09-09T20:46:29","subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","submitter":{"id":195,"url":"http://patchwork.ozlabs.org/api/people/195/","name":"Eduardo Habkost","email":"ehabkost@redhat.com"},"content":"On Thu, Sep 07, 2017 at 01:05:33AM +0000, Gonglei (Arei) wrote:\n> \n> \n> > -----Original Message-----\n> > From: Eduardo Habkost [mailto:ehabkost@redhat.com]\n> > Sent: Tuesday, September 05, 2017 9:17 PM\n> > To: Gonglei (Arei)\n> > Cc: qemu-devel@nongnu.org; mst@redhat.com; pbonzini@redhat.com;\n> > rth@twiddle.net; mtosatti@redhat.com; vrozenfe@redhat.com;\n> > Huangweidong (C)\n> > Subject: Re: [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for windows\n> > guests\n> > \n> > On Tue, Sep 05, 2017 at 05:30:05PM +0800, Gonglei wrote:\n> > > Starting with Windows Server 2012 and Windows 8, if\n> > > CPUID.40000005.EAX contains a value of -1, Windows assumes specific\n> > > limit to the number of VPs. In this case, Windows Server 2012\n> > > guest VMs may use more than 64 VPs, up to the maximum supported\n> > > number of processors applicable to the specific Windows\n> > > version being used.\n> > >\n> > >\n> > https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/referenc\n> > e/tlfs\n> > >\n> > > For compatibility, Let's introduce a new property for X86CPU,\n> > > named \"hv-cpuid-limits-eax\" as Paolo's suggestion, and set it\n> > > to \"on\" before machine 2.10.\n> > >\n> > > Signed-off-by: Gonglei \n> > > ---\n> > > include/hw/i386/pc.h | 5 +++++\n> > > target/i386/cpu.c | 1 +\n> > > target/i386/cpu.h | 2 ++\n> > > target/i386/kvm.c | 18 +++++++++++++++++-\n> > > 4 files changed, 25 insertions(+), 1 deletion(-)\n> > >\n> > > diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h\n> > > index 8226904..db32e58 100644\n> > > --- a/include/hw/i386/pc.h\n> > > +++ b/include/hw/i386/pc.h\n> > > @@ -371,6 +371,11 @@ bool e820_get_entry(int, uint32_t, uint64_t *,\n> > uint64_t *);\n> > >\n> > > #define PC_COMPAT_2_10 \\\n> > > HW_COMPAT_2_10 \\\n> > > + {\\\n> > > + .driver = TYPE_X86_CPU,\\\n> > > + .property = \"hv_cpuid_limits_eax\",\\\n> > \n> > The property name is hv-cpuid-limits-eax.\n> > \n> Make sense to me.\n> \n> > > + .value = \"on\",\\\n> > > + },\\\n> > >\n> > > #define PC_COMPAT_2_9 \\\n> > > HW_COMPAT_2_9 \\\n> > > diff --git a/target/i386/cpu.c b/target/i386/cpu.c\n> > > index 69676e1..0d47bdd 100644\n> > > --- a/target/i386/cpu.c\n> > > +++ b/target/i386/cpu.c\n> > > @@ -4145,6 +4145,7 @@ static Property x86_cpu_properties[] = {\n> > > false),\n> > > DEFINE_PROP_BOOL(\"vmware-cpuid-freq\", X86CPU,\n> > vmware_cpuid_freq, true),\n> > > DEFINE_PROP_BOOL(\"tcg-cpuid\", X86CPU, expose_tcg, true),\n> > > + DEFINE_PROP_BOOL(\"hv-cpuid-limits-eax\", X86CPU,\n> > hv_cpuid_limits_eax, false),\n> > \n> > The property name \"hv-cpuid-limits-eax\" doesn't say anything\n> > about what it does exactly when set to true.\n> > \n> > What about just making it int32? e.g.:\n> > \n> > DEFINE_PROP_INT32(\"x-hv-max-vps\", X86CPU, hv_max_vps, -1)\n> > [...]\n> > {\\\n> > .driver = TYPE_X86_CPU,\\\n> > .property = \"x-hv-max-vps\",\\\n> > .value = \"0x40\",\\\n> > },\\\n> > [...]\n> > c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> > c->eax = cpu->hv_max_vps;\n> > \n> > \n> > (The \"x-\" prefix indicates that the property is not supposed to\n> > be a stable user interface.)\n> > \n> I thought about this as well.\n> but actually we can't assure that users set the x-hv-max-vps an invalid value if\n> we use this method. Do we really need to expose eax directly?\n\nWe don't really need to expose eax directly and I'm not against a\nboolean property, but I think an integer property with the actual\nCPUID value makes the compat code simpler and the purpose of the\nentry at PC_COMPAT_* more obvious.\n\nProperties prefixed with \"x-\" are for internal QEMU usage or\ndebugging, if users want to fiddle with it, they do it at their\nown risk. I don't see a problem with that.\n\n> \n> > \n> > > DEFINE_PROP_END_OF_LIST()\n> > > };\n> > >\n> > > diff --git a/target/i386/cpu.h b/target/i386/cpu.h\n> > > index 525d35d..f8b455a 100644\n> > > --- a/target/i386/cpu.h\n> > > +++ b/target/i386/cpu.h\n> > > @@ -1282,6 +1282,8 @@ struct X86CPU {\n> > > int32_t socket_id;\n> > > int32_t core_id;\n> > > int32_t thread_id;\n> > > +\n> > > + bool hv_cpuid_limits_eax;\n> > > };\n> > >\n> > > static inline X86CPU *x86_env_get_cpu(CPUX86State *env)\n> > > diff --git a/target/i386/kvm.c b/target/i386/kvm.c\n> > > index 6db7783..cf6ef96 100644\n> > > --- a/target/i386/kvm.c\n> > > +++ b/target/i386/kvm.c\n> > > @@ -751,7 +751,23 @@ int kvm_arch_init_vcpu(CPUState *cs)\n> > >\n> > > c = &cpuid_data.entries[cpuid_i++];\n> > > c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> > > - c->eax = 0x40;\n> > > +\n> > > + if (!cpu->hv_cpuid_limits_eax) {\n> > > + /*\n> > > + * Starting with Windows Server 2012 and Windows 8, if\n> > > + * CPUID.40000005.EAX contains a value of -1, Windows\n> > > + * assumes specific limit to the number of VPs. In this case,\n> > > + * Windows Server 2012 guest VMs may use more than 64\n> > VPs,\n> > > + * up to the maximum supported number of processors\n> > > + * applicable to the specific Windows version being used.\n> > \n> > That was a direct quote from a document, so I recommend citing\n> > the specific document you quoted. e.g.:\n> > \n> > /*\n> > * From \"Requirements for Implementing the Microsoft\n> > * Hypervisor Interface\":\n> > *\n> > https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/referenc\n> > e/tlfs\n> > *\n> > * \"Starting with Windows Server 2012 and Windows 8, if\n> > * CPUID.40000005.EAX contains a value of -1, Windows assumes\n> > * specific limit to the number of VPs. In this case, Windows\n> > * Server 2012 guest VMs may use more than 64 VPs, up to the\n> > * maximum supported number of processors applicable to the\n> > * specific Windows version being used.\"\n> > */\n> > \n> > \n> > > + *\n> > > + * https://docs.microsoft.com/en-us/virtualization/\n> > > + * hyper-v-on-windows/reference/tlfs\n> > \n> > IMO a long line is preferable to a broken URL.\n> > \n> Make sense to me.\n> \n> Thanks,\n> -Gonglei\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=ehabkost@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xqR7q6QNdz9sBZ\n\tfor ;\n\tSun, 10 Sep 2017 06:47:03 +1000 (AEST)","from localhost ([::1]:50841 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dqmec-0005Kx-2B\n\tfor incoming@patchwork.ozlabs.org; Sat, 09 Sep 2017 16:47:02 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48249)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dqmeD-0005KN-DK\n\tfor qemu-devel@nongnu.org; Sat, 09 Sep 2017 16:46:39 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dqmeA-0002bE-96\n\tfor qemu-devel@nongnu.org; Sat, 09 Sep 2017 16:46:37 -0400","from mx1.redhat.com ([209.132.183.28]:58744)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dqmeA-0002aZ-05\n\tfor qemu-devel@nongnu.org; Sat, 09 Sep 2017 16:46:34 -0400","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 032A0C04B928;\n\tSat, 9 Sep 2017 20:46:33 +0000 (UTC)","from localhost (ovpn-116-45.gru2.redhat.com [10.97.116.45])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id AEF3060621;\n\tSat, 9 Sep 2017 20:46:30 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 032A0C04B928","Date":"Sat, 9 Sep 2017 17:46:29 -0300","From":"Eduardo Habkost ","To":"\"Gonglei (Arei)\" ","Message-ID":"<20170909204629.GI7570@localhost.localdomain>","References":"<1504603805-180240-1-git-send-email-arei.gonglei@huawei.com>\n\t<1504603805-180240-3-git-send-email-arei.gonglei@huawei.com>\n\t<20170905131648.GW7570@localhost.localdomain>\n\t<33183CC9F5247A488A2544077AF19020DA3A2534@DGGEMA505-MBX.china.huawei.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<33183CC9F5247A488A2544077AF19020DA3A2534@DGGEMA505-MBX.china.huawei.com>","X-Fnord":"you can see the fnord","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.13","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tSat, 09 Sep 2017 20:46:33 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Huangweidong \\(C\\)\" ,\n\t\"mst@redhat.com\" ,\n\t\"mtosatti@redhat.com\" ,\n\t\"qemu-devel@nongnu.org\" ,\n\t\"vrozenfe@redhat.com\" ,\n\t\"pbonzini@redhat.com\" ,\n\t\"rth@twiddle.net\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1766025,"web_url":"http://patchwork.ozlabs.org/comment/1766025/","msgid":"<33183CC9F5247A488A2544077AF19020DA3A7FFA@DGGEMA505-MBX.china.huawei.com>","list_archive_url":null,"date":"2017-09-11T00:55:26","subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","submitter":{"id":35948,"url":"http://patchwork.ozlabs.org/api/people/35948/","name":"Gonglei","email":"arei.gonglei@huawei.com"},"content":"> -----Original Message-----\n> From: Eduardo Habkost [mailto:ehabkost@redhat.com]\n> Sent: Sunday, September 10, 2017 4:46 AM\n> To: Gonglei (Arei)\n> Cc: qemu-devel@nongnu.org; mst@redhat.com; pbonzini@redhat.com;\n> rth@twiddle.net; mtosatti@redhat.com; vrozenfe@redhat.com;\n> Huangweidong (C)\n> Subject: Re: [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for windows\n> guests\n> \n> On Thu, Sep 07, 2017 at 01:05:33AM +0000, Gonglei (Arei) wrote:\n> >\n> >\n> > > -----Original Message-----\n> > > From: Eduardo Habkost [mailto:ehabkost@redhat.com]\n> > > Sent: Tuesday, September 05, 2017 9:17 PM\n> > > To: Gonglei (Arei)\n> > > Cc: qemu-devel@nongnu.org; mst@redhat.com; pbonzini@redhat.com;\n> > > rth@twiddle.net; mtosatti@redhat.com; vrozenfe@redhat.com;\n> > > Huangweidong (C)\n> > > Subject: Re: [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for\n> windows\n> > > guests\n> > >\n> > > On Tue, Sep 05, 2017 at 05:30:05PM +0800, Gonglei wrote:\n> > > > Starting with Windows Server 2012 and Windows 8, if\n> > > > CPUID.40000005.EAX contains a value of -1, Windows assumes specific\n> > > > limit to the number of VPs. In this case, Windows Server 2012\n> > > > guest VMs may use more than 64 VPs, up to the maximum supported\n> > > > number of processors applicable to the specific Windows\n> > > > version being used.\n> > > >\n> > > >\n> > >\n> https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/referenc\n> > > e/tlfs\n> > > >\n> > > > For compatibility, Let's introduce a new property for X86CPU,\n> > > > named \"hv-cpuid-limits-eax\" as Paolo's suggestion, and set it\n> > > > to \"on\" before machine 2.10.\n> > > >\n> > > > Signed-off-by: Gonglei \n> > > > ---\n> > > > include/hw/i386/pc.h | 5 +++++\n> > > > target/i386/cpu.c | 1 +\n> > > > target/i386/cpu.h | 2 ++\n> > > > target/i386/kvm.c | 18 +++++++++++++++++-\n> > > > 4 files changed, 25 insertions(+), 1 deletion(-)\n> > > >\n> > > > diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h\n> > > > index 8226904..db32e58 100644\n> > > > --- a/include/hw/i386/pc.h\n> > > > +++ b/include/hw/i386/pc.h\n> > > > @@ -371,6 +371,11 @@ bool e820_get_entry(int, uint32_t, uint64_t *,\n> > > uint64_t *);\n> > > >\n> > > > #define PC_COMPAT_2_10 \\\n> > > > HW_COMPAT_2_10 \\\n> > > > + {\\\n> > > > + .driver = TYPE_X86_CPU,\\\n> > > > + .property = \"hv_cpuid_limits_eax\",\\\n> > >\n> > > The property name is hv-cpuid-limits-eax.\n> > >\n> > Make sense to me.\n> >\n> > > > + .value = \"on\",\\\n> > > > + },\\\n> > > >\n> > > > #define PC_COMPAT_2_9 \\\n> > > > HW_COMPAT_2_9 \\\n> > > > diff --git a/target/i386/cpu.c b/target/i386/cpu.c\n> > > > index 69676e1..0d47bdd 100644\n> > > > --- a/target/i386/cpu.c\n> > > > +++ b/target/i386/cpu.c\n> > > > @@ -4145,6 +4145,7 @@ static Property x86_cpu_properties[] = {\n> > > > false),\n> > > > DEFINE_PROP_BOOL(\"vmware-cpuid-freq\", X86CPU,\n> > > vmware_cpuid_freq, true),\n> > > > DEFINE_PROP_BOOL(\"tcg-cpuid\", X86CPU, expose_tcg, true),\n> > > > + DEFINE_PROP_BOOL(\"hv-cpuid-limits-eax\", X86CPU,\n> > > hv_cpuid_limits_eax, false),\n> > >\n> > > The property name \"hv-cpuid-limits-eax\" doesn't say anything\n> > > about what it does exactly when set to true.\n> > >\n> > > What about just making it int32? e.g.:\n> > >\n> > > DEFINE_PROP_INT32(\"x-hv-max-vps\", X86CPU, hv_max_vps, -1)\n> > > [...]\n> > > {\\\n> > > .driver = TYPE_X86_CPU,\\\n> > > .property = \"x-hv-max-vps\",\\\n> > > .value = \"0x40\",\\\n> > > },\\\n> > > [...]\n> > > c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;\n> > > c->eax = cpu->hv_max_vps;\n> > >\n> > >\n> > > (The \"x-\" prefix indicates that the property is not supposed to\n> > > be a stable user interface.)\n> > >\n> > I thought about this as well.\n> > but actually we can't assure that users set the x-hv-max-vps an invalid value if\n> > we use this method. Do we really need to expose eax directly?\n> \n> We don't really need to expose eax directly and I'm not against a\n> boolean property, but I think an integer property with the actual\n> CPUID value makes the compat code simpler and the purpose of the\n> entry at PC_COMPAT_* more obvious.\n> \n> Properties prefixed with \"x-\" are for internal QEMU usage or\n> debugging, if users want to fiddle with it, they do it at their\n> own risk. I don't see a problem with that.\n> \nOk, it's fair. Will do it as your suggestion.\n\nThanks,\n-Gonglei","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xr8dJ0bPjz9s76\n\tfor ;\n\tMon, 11 Sep 2017 10:56:32 +1000 (AEST)","from localhost ([::1]:54787 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1drD1Z-0008TB-1S\n\tfor incoming@patchwork.ozlabs.org; Sun, 10 Sep 2017 20:56:29 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:33016)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1drD11-0008Sq-S1\n\tfor qemu-devel@nongnu.org; Sun, 10 Sep 2017 20:55:57 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1drD0y-0006FI-PB\n\tfor qemu-devel@nongnu.org; Sun, 10 Sep 2017 20:55:55 -0400","from szxga02-in.huawei.com ([45.249.212.188]:4421)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.71)\n\t(envelope-from ) id 1drD0x-0006DK-8K\n\tfor qemu-devel@nongnu.org; Sun, 10 Sep 2017 20:55:52 -0400","from 172.30.72.53 (EHLO DGGEMA403-HUB.china.huawei.com)\n\t([172.30.72.53])\n\tby dggrg02-dlp.huawei.com (MOS 4.4.6-GA FastPath queued)\n\twith ESMTP id AVC13575; Mon, 11 Sep 2017 08:55:34 +0800 (CST)","from DGGEMA505-MBX.china.huawei.com ([169.254.1.46]) by\n\tDGGEMA403-HUB.china.huawei.com ([10.3.20.44]) with mapi id\n\t14.03.0301.000; Mon, 11 Sep 2017 08:55:26 +0800"],"From":"\"Gonglei (Arei)\" ","To":"Eduardo Habkost ","Thread-Topic":"[PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus for windows\n\tguests","Thread-Index":"AQHTJimZLXU3wSlkuE64qwXCq+YQuKKlwAUAgALcSRCAA+qugIACXXZg","Date":"Mon, 11 Sep 2017 00:55:26 +0000","Message-ID":"<33183CC9F5247A488A2544077AF19020DA3A7FFA@DGGEMA505-MBX.china.huawei.com>","References":"<1504603805-180240-1-git-send-email-arei.gonglei@huawei.com>\n\t<1504603805-180240-3-git-send-email-arei.gonglei@huawei.com>\n\t<20170905131648.GW7570@localhost.localdomain>\n\t<33183CC9F5247A488A2544077AF19020DA3A2534@DGGEMA505-MBX.china.huawei.com>\n\t<20170909204629.GI7570@localhost.localdomain>","In-Reply-To":"<20170909204629.GI7570@localhost.localdomain>","Accept-Language":"zh-CN, en-US","Content-Language":"zh-CN","X-MS-Has-Attach":"","X-MS-TNEF-Correlator":"","x-originating-ip":"[10.177.18.62]","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"quoted-printable","MIME-Version":"1.0","X-CFilter-Loop":"Reflected","X-Mirapoint-Virus-RAPID-Raw":"score=unknown(0),\n\trefid=str=0001.0A090203.59B5DF07.002C, ss=1, re=0.000, recu=0.000,\n\treip=0.000, cl=1, cld=1, fgs=0, ip=169.254.1.46,\n\tso=2014-11-16 11:51:01, dmn=2013-03-21 17:37:32","X-Mirapoint-Loop-Id":"a6f1ea7434cbe6863cf253e424f14437","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic]\n\t[fuzzy]","X-Received-From":"45.249.212.188","Subject":"Re: [Qemu-devel] [PATCH 2/2] i386/cpu/hyperv: support over 64 vcpus\n\tfor windows guests","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"\"Huangweidong \\(C\\)\" ,\n\t\"mst@redhat.com\" ,\n\t\"mtosatti@redhat.com\" ,\n\t\"qemu-devel@nongnu.org\" ,\n\t\"vrozenfe@redhat.com\" ,\n\t\"pbonzini@redhat.com\" ,\n\t\"rth@twiddle.net\" ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]