[{"id":1770862,"web_url":"http://patchwork.ozlabs.org/comment/1770862/","msgid":"<20170919082020.GT27153@umbus>","list_archive_url":null,"date":"2017-09-19T08:20:20","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n> negotiation process determines whether the guest operates with an\n> interrupt controller using the XICS legacy model, as found on POWER8,\n> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n> \n> Follows a model for the XIVE interrupt controller and support for the\n> Hypervisor's calls which are used to configure the interrupt sources\n> and the event/notification queues of the guest. The last patch\n> integrates XIVE in the sPAPR machine.\n> \n> Code is here:\n\n\nAn overall comment:\n\nI note in several replies here that I think the way XICS objects are\nre-used for XIVE is really ugly, and I think it will make future\nmaintenance pretty painful.\n\nI'm thinking maybe trying to support the CAS negotiation of interrupt\ncontroller from day 1 is warping the design. A better approach might\nbe first to implement XIVE only when given a specific machine option -\nguest gets one or the other and can't negotiate.\n\nThat should allow a more natural XIVE design to emerge, *then* we can\nlook at what's necessary to make boot-time negotiation possible.\n> \n> https://github.com/legoater/qemu/commits/xive\n> \n> Caveats :\n> \n> - IRQ allocator : making progress\n> \n> The sPAPR machine make uses of the interrupt controller very early\n> in the initialization sequence to allocate IRQ numbers and populate\n> the device tree. CAS requires XIVE to be able to switch interrupt\n> model and consequently have the models share a common IRQ allocator. \n> \n> I have chosen to link the sPAPR XICS interrupt source into XIVE to\n> share the ICSIRQState array which acts as an IRQ allocator. This\n> can be improved.\n> \n> - Interrupt presenter :\n> \n> The register data is directly stored under the ICPState structure\n> which is shared with all other sPAPR interrupt controller models.\n> \n> - KVM support : not addressed yet\n> \n> The guest needs to be run with kernel_irqchip=off on a POWER9 system.\n> \n> - LSI : lightly tested.\n> \n> Thanks,\n> \n> C.\n> \n> Changes since RFC v1:\n> \n> - removed initial complexity due to a tentative try to support\n> PowerNV. This will come later.\n> - removed specific XIVE interrupt source and presenter models\n> - renamed files and typedefs\n> - removed print_info() handler\n> - introduced a CAS reset to rebuild the device tree\n> - linked the XIVE model with the sPAPR XICS interrupt source to share\n> the IRQ allocator \n> - improved hcall support (still some missing but they are not used\n> under Linux)\n> - improved device tree\n> - should have addressed comments in first RFC\n> - and much more ... Next version should have a better changelog.\n> \n> \n> Cédric Le Goater (21):\n> ppc/xive: introduce a skeleton for the sPAPR XIVE interrupt controller\n> migration: add VMSTATE_STRUCT_VARRAY_UINT32_ALLOC\n> ppc/xive: define the XIVE internal tables\n> ppc/xive: provide a link to the sPAPR ICS object under XIVE\n> ppc/xive: allocate IRQ numbers for the IPIs\n> ppc/xive: introduce handlers for interrupt sources\n> ppc/xive: add MMIO handlers for the XIVE interrupt sources\n> ppc/xive: describe the XIVE interrupt source flags\n> ppc/xive: extend the interrupt presenter model for XIVE\n> ppc/xive: add MMIO handlers for the XIVE TIMA\n> ppc/xive: push the EQ data in OS event queue\n> ppc/xive: notify the CPU when interrupt priority is more privileged\n> ppc/xive: handle interrupt acknowledgment by the O/S\n> ppc/xive: add support for the SET_OS_PENDING command\n> spapr: modify spapr_populate_pci_dt() to use a 'nr_irqs' argument\n> spapr: add a XIVE object to the sPAPR machine\n> ppc/xive: add hcalls support\n> ppc/xive: add device tree support\n> ppc/xive: introduce a helper to map the XIVE memory regions\n> ppc/xics: introduce a qirq_get() helper in the XICSFabric\n> spapr: activate XIVE exploitation mode\n> \n> default-configs/ppc64-softmmu.mak | 1 +\n> hw/intc/Makefile.objs | 1 +\n> hw/intc/spapr_xive.c | 821 +++++++++++++++++++++++++++++++++\n> hw/intc/spapr_xive_hcall.c | 930 ++++++++++++++++++++++++++++++++++++++\n> hw/intc/xics.c | 11 +-\n> hw/intc/xive-internal.h | 189 ++++++++\n> hw/ppc/spapr.c | 110 ++++-\n> hw/ppc/spapr_hcall.c | 6 +\n> hw/ppc/spapr_pci.c | 4 +-\n> include/hw/pci-host/spapr.h | 2 +-\n> include/hw/ppc/spapr.h | 17 +-\n> include/hw/ppc/spapr_xive.h | 75 +++\n> include/hw/ppc/xics.h | 7 +\n> include/migration/vmstate.h | 10 +\n> 14 files changed, 2169 insertions(+), 15 deletions(-)\n> create mode 100644 hw/intc/spapr_xive.c\n> create mode 100644 hw/intc/spapr_xive_hcall.c\n> create mode 100644 hw/intc/xive-internal.h\n> create mode 100644 include/hw/ppc/spapr_xive.h\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"nT1xSYJq\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxKZw64Flz9sNw\n\tfor ;\n\tTue, 19 Sep 2017 20:57:28 +1000 (AEST)","from localhost ([::1]:41462 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duGDW-000180-S2\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 06:57:26 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:51330)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duFta-0001J0-Pj\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:36:53 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duFtW-0002xh-NC\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:36:50 -0400","from ozlabs.org ([103.22.144.67]:45431)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1duFtW-0002s9-7z; Tue, 19 Sep 2017 06:36:46 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3xxK6q5Q1gz9sP1; Tue, 19 Sep 2017 20:36:34 +1000 (AEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1505817395;\n\tbh=KN50rO0M4o535bchjJFb/zzJdcCr7jPs0eXwSxyKXxo=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=nT1xSYJq0p4JMAMh4GTXCtVMy6IMYCSIdNehgKabQPN6PfdeDP4v9VCvKgJUVFooM\n\tZl7quOX/MevDIiFg7TalIYWxRo43AVxh0egu8PQyaEIkX1BXeDQNf1tA9CQ6cTUD9C\n\tY2AcrpMcBrFjti6ZVy5g09muT6CXu1XMufQZoe18=","Date":"Tue, 19 Sep 2017 18:20:20 +1000","From":"David Gibson ","To":"=?iso-8859-1?q?C=E9dric?= Le Goater ","Message-ID":"<20170919082020.GT27153@umbus>","References":"<20170911171235.29331-1-clg@kaod.org>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"bxF9Dep5HzwGj9mC\"","Content-Disposition":"inline","In-Reply-To":"<20170911171235.29331-1-clg@kaod.org>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"103.22.144.67","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770868,"web_url":"http://patchwork.ozlabs.org/comment/1770868/","msgid":"<20170919084618.GY27153@umbus>","list_archive_url":null,"date":"2017-09-19T08:46:18","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n> > On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n> > negotiation process determines whether the guest operates with an\n> > interrupt controller using the XICS legacy model, as found on POWER8,\n> > or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n> > patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n> > \n> > Follows a model for the XIVE interrupt controller and support for the\n> > Hypervisor's calls which are used to configure the interrupt sources\n> > and the event/notification queues of the guest. The last patch\n> > integrates XIVE in the sPAPR machine.\n> > \n> > Code is here:\n> \n> \n> An overall comment:\n> \n> I note in several replies here that I think the way XICS objects are\n> re-used for XIVE is really ugly, and I think it will make future\n> maintenance pretty painful.\n> \n> I'm thinking maybe trying to support the CAS negotiation of interrupt\n> controller from day 1 is warping the design. A better approach might\n> be first to implement XIVE only when given a specific machine option -\n> guest gets one or the other and can't negotiate.\n> \n> That should allow a more natural XIVE design to emerge, *then* we can\n> look at what's necessary to make boot-time negotiation possible.\n\nActually, it just occurred to me that we might be making life hard for\nourselves by trying to actually switch between full XICS and XIVE\nmodels. Coudln't we have new machine types always construct the XIVE\ninfrastructure, but then implement the XICS RTAS and hcalls in terms\nof the XIVE virtual hardware. Since something more or less equivalent\nhas already been done in both OPAL and the host kernel, I'm guessing\nthis shouldn't be too hard at this point.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"eYcohKyC\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxKhw165qz9rxj\n\tfor ;\n\tTue, 19 Sep 2017 21:02:40 +1000 (AEST)","from localhost ([::1]:41501 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duGIY-0005XU-4m\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 07:02:38 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:51347)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duFtb-0001JL-0f\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:36:53 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duFtX-0002zw-Tj\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 06:36:50 -0400","from ozlabs.org ([2401:3900:2:1::2]:42199)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1duFtX-0002tx-GV; Tue, 19 Sep 2017 06:36:47 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3xxK6r1L3vz9t4g; Tue, 19 Sep 2017 20:36:35 +1000 (AEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1505817396;\n\tbh=Ut/4GCK83ytQfTwYVqtR0SW9PWr7QaZaVhDC7/qmv50=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=eYcohKyCACsmGzqR2/NY5/VcvAeqBjQ005r2Se59MFWSkB7JvRxDOLt2Uro2qRfRZ\n\tZU4A20GqeQ68A3mvdXnpSEk4tVdDkwhIIy+pwQyuy0XLDwQLhhYwIshjanlfH5PrbV\n\to7SF66NfWzzQqu0GIqt6lPfMhLLixC68WMrAtIXk=","Date":"Tue, 19 Sep 2017 18:46:18 +1000","From":"David Gibson ","To":"=?iso-8859-1?q?C=E9dric?= Le Goater ","Message-ID":"<20170919084618.GY27153@umbus>","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"t3Rhqlz/9+3FgEet\"","Content-Disposition":"inline","In-Reply-To":"<20170919082020.GT27153@umbus>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771834,"web_url":"http://patchwork.ozlabs.org/comment/1771834/","msgid":"<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","list_archive_url":null,"date":"2017-09-20T12:33:37","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":68548,"url":"http://patchwork.ozlabs.org/api/people/68548/","name":"Cédric Le Goater","email":"clg@kaod.org"},"content":"On 09/19/2017 10:46 AM, David Gibson wrote:\n> On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n>> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n>>> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n>>> negotiation process determines whether the guest operates with an\n>>> interrupt controller using the XICS legacy model, as found on POWER8,\n>>> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n>>> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n>>>\n>>> Follows a model for the XIVE interrupt controller and support for the\n>>> Hypervisor's calls which are used to configure the interrupt sources\n>>> and the event/notification queues of the guest. The last patch\n>>> integrates XIVE in the sPAPR machine.\n>>>\n>>> Code is here:\n>>\n>>\n>> An overall comment:\n>>\n>> I note in several replies here that I think the way XICS objects are\n>> re-used for XIVE is really ugly, and I think it will make future\n>> maintenance pretty painful.\n\nI agree. That was one way to identify what we need for migration \ncompatibility and CAS reset. \n\n>> I'm thinking maybe trying to support the CAS negotiation of interrupt\n>> controller from day 1 is warping the design. A better approach might\n>> be first to implement XIVE only when given a specific machine option -\n>> guest gets one or the other and can't negotiate.\n\nok. \n\nCAS is not the most complex problem, we mostly need to share \nthe ICSIRQState array and the source offset. migration from older\nmachine is a problem. We are doomed to keep the existing XICS\nframework available.\n\n>> That should allow a more natural XIVE design to emerge, *then* we can\n>> look at what's necessary to make boot-time negotiation possible.\n> \n> Actually, it just occurred to me that we might be making life hard for\n> ourselves by trying to actually switch between full XICS and XIVE\n> models. Coudln't we have new machine types always construct the XIVE\n> infrastructure, \n\nyes.\n\n> but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n> hardware.\n\nok but migration will not be supported.\n\n> Since something more or less equivalent\n> has already been done in both OPAL and the host kernel, I'm guessing\n> this shouldn't be too hard at this point.\n\nIndeed that is how it is working currently on P9 kvm guests. hcalls are\nimplemented on top of XIVE native.\n\nThanks,\n\n\nC.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xy0c64zHcz9s06\n\tfor ;\n\tWed, 20 Sep 2017 23:15:49 +1000 (AEST)","from localhost ([::1]:48007 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dueqw-0001j1-7t\n\tfor incoming@patchwork.ozlabs.org; Wed, 20 Sep 2017 09:15:46 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:41237)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duemz-0007bS-MC\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 09:12:23 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duemR-0006hl-1t\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 09:11:41 -0400","from 3.mo2.mail-out.ovh.net ([46.105.58.226]:35160)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1duemQ-0006dE-Rh\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 09:11:06 -0400","from player157.ha.ovh.net (b9.ovh.net [213.186.33.59])\n\tby mo2.mail-out.ovh.net (Postfix) with ESMTP id 5DFFBAC984\n\tfor ; Wed, 20 Sep 2017 14:33:44 +0200 (CEST)","from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr\n\t[90.76.52.173]) (Authenticated sender: postmaster@kaod.org)\n\tby player157.ha.ovh.net (Postfix) with ESMTPSA id C29A95000A8;\n\tWed, 20 Sep 2017 14:33:37 +0200 (CEST)"],"To":"David Gibson ","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>","From":"=?utf-8?q?C=C3=A9dric_Le_Goater?= ","Message-ID":"<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","Date":"Wed, 20 Sep 2017 14:33:37 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170919084618.GY27153@umbus>","Content-Type":"text/plain; charset=windows-1252","Content-Language":"en-US","X-Ovh-Tracer-Id":"6928788027427556216","X-VR-SPAMSTATE":"OK","X-VR-SPAMSCORE":"-100","X-VR-SPAMCAUSE":"gggruggvucftvghtrhhoucdtuddrfeelledriedtgddvgecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"46.105.58.226","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772355,"web_url":"http://patchwork.ozlabs.org/comment/1772355/","msgid":"<20170921012508.GA4998@umbus.fritz.box>","list_archive_url":null,"date":"2017-09-21T01:25:08","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Wed, Sep 20, 2017 at 02:33:37PM +0200, Cédric Le Goater wrote:\n> On 09/19/2017 10:46 AM, David Gibson wrote:\n> > On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n> >> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n> >>> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n> >>> negotiation process determines whether the guest operates with an\n> >>> interrupt controller using the XICS legacy model, as found on POWER8,\n> >>> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n> >>> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n> >>>\n> >>> Follows a model for the XIVE interrupt controller and support for the\n> >>> Hypervisor's calls which are used to configure the interrupt sources\n> >>> and the event/notification queues of the guest. The last patch\n> >>> integrates XIVE in the sPAPR machine.\n> >>>\n> >>> Code is here:\n> >>\n> >>\n> >> An overall comment:\n> >>\n> >> I note in several replies here that I think the way XICS objects are\n> >> re-used for XIVE is really ugly, and I think it will make future\n> >> maintenance pretty painful.\n> \n> I agree. That was one way to identify what we need for migration \n> compatibility and CAS reset. \n> \n> >> I'm thinking maybe trying to support the CAS negotiation of interrupt\n> >> controller from day 1 is warping the design. A better approach might\n> >> be first to implement XIVE only when given a specific machine option -\n> >> guest gets one or the other and can't negotiate.\n> \n> ok. \n> \n> CAS is not the most complex problem, we mostly need to share \n> the ICSIRQState array and the source offset. migration from older\n> machine is a problem.\n\nUh.. what? Migration from an older machine isn't a thing. We can\nmigrate from an older qemu, but the machine type (and version) has to\nbe identical at each end. That's *why* we keep around the older\nmachine types on newer qemus.\n\n> We are doomed to keep the existing XICS\n> framework available.\n> \n> >> That should allow a more natural XIVE design to emerge, *then* we can\n> >> look at what's necessary to make boot-time negotiation possible.\n> > \n> > Actually, it just occurred to me that we might be making life hard for\n> > ourselves by trying to actually switch between full XICS and XIVE\n> > models. Coudln't we have new machine types always construct the XIVE\n> > infrastructure, \n> \n> yes.\n> \n> > but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n> > hardware.\n> \n> ok but migration will not be supported.\n\nRight, this would only be for newer machine types, and you can never\nmigrate between different machine types.\n\n> > Since something more or less equivalent\n> > has already been done in both OPAL and the host kernel, I'm guessing\n> > this shouldn't be too hard at this point.\n> \n> Indeed that is how it is working currently on P9 kvm guests. hcalls are\n> implemented on top of XIVE native.\n> \n> Thanks,\n> \n> \n> C.\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"fcieJhCP\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xyK576TGZz9s81\n\tfor ;\n\tThu, 21 Sep 2017 11:38:35 +1000 (AEST)","from localhost ([::1]:51367 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duqRl-0003H3-UH\n\tfor incoming@patchwork.ozlabs.org; Wed, 20 Sep 2017 21:38:33 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:47607)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duqR9-0003Gk-2r\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 21:37:56 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duqR7-0006YJ-O3\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 21:37:55 -0400","from ozlabs.org ([2401:3900:2:1::2]:51335)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1duqR7-0006UB-7d; Wed, 20 Sep 2017 21:37:53 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3xyK4F0bZmz9sBZ; Thu, 21 Sep 2017 11:37:48 +1000 (AEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1505957869;\n\tbh=cBggIdhXyH4EDpv7OiSGLLMtSdxwTfGYfeU1y/mFaj4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=fcieJhCPODakKhIYqE85ZoB/8RoNysqjO2QGSWHGmmY1FVC0lKPLos3bRgKq3Bfwl\n\tXElnUuqoK8UK778j40WH1q6vBPAGQ/aduPFJCL6EmtaoW12+GC0C5BkwcQ1lRyaoYi\n\tVTuMgUL17rSbvx5gchjLDxAPwrl5Tv3kZGMoAHxM=","Date":"Thu, 21 Sep 2017 11:25:08 +1000","From":"David Gibson ","To":"=?iso-8859-1?q?C=E9dric?= Le Goater ","Message-ID":"<20170921012508.GA4998@umbus.fritz.box>","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>\n\t<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"wRRV7LY7NUeQGEoC\"","Content-Disposition":"inline","In-Reply-To":"<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","User-Agent":"Mutt/1.9.0 (2017-09-02)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1772831,"web_url":"http://patchwork.ozlabs.org/comment/1772831/","msgid":"<53b4c971-920f-816b-59fb-f77a03663d81@kaod.org>","list_archive_url":null,"date":"2017-09-21T14:18:33","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":68548,"url":"http://patchwork.ozlabs.org/api/people/68548/","name":"Cédric Le Goater","email":"clg@kaod.org"},"content":"On 09/21/2017 03:25 AM, David Gibson wrote:\n> On Wed, Sep 20, 2017 at 02:33:37PM +0200, Cédric Le Goater wrote:\n>> On 09/19/2017 10:46 AM, David Gibson wrote:\n>>> On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n>>>> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n>>>>> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n>>>>> negotiation process determines whether the guest operates with an\n>>>>> interrupt controller using the XICS legacy model, as found on POWER8,\n>>>>> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n>>>>> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n>>>>>\n>>>>> Follows a model for the XIVE interrupt controller and support for the\n>>>>> Hypervisor's calls which are used to configure the interrupt sources\n>>>>> and the event/notification queues of the guest. The last patch\n>>>>> integrates XIVE in the sPAPR machine.\n>>>>>\n>>>>> Code is here:\n>>>>\n>>>>\n>>>> An overall comment:\n>>>>\n>>>> I note in several replies here that I think the way XICS objects are\n>>>> re-used for XIVE is really ugly, and I think it will make future\n>>>> maintenance pretty painful.\n>>\n>> I agree. That was one way to identify what we need for migration \n>> compatibility and CAS reset. \n>>\n>>>> I'm thinking maybe trying to support the CAS negotiation of interrupt\n>>>> controller from day 1 is warping the design. A better approach might\n>>>> be first to implement XIVE only when given a specific machine option -\n>>>> guest gets one or the other and can't negotiate.\n>>\n>> ok. \n>>\n>> CAS is not the most complex problem, we mostly need to share \n>> the ICSIRQState array and the source offset. migration from older\n>> machine is a problem.\n> \n> Uh.. what? Migration from an older machine isn't a thing. We can\n> migrate from an older qemu, but the machine type (and version) has to\n> be identical at each end. That's *why* we keep around the older\n> machine types on newer qemus.\n\nyes. I am just wondering how I am going to handle a xics-only \nmachine migrating to a xics/xive machine. \n\nThe xive machine option we are talking about will activate \nthe xive interrupt mode and instantiate the objects behind it. \nSo when we migrate from an older machine we will need to start \nthe target machine with xive=off. I guess that is OK. \n\nThanks for the insights and the time to review the code,\n\nC. \n\n>> We are doomed to keep the existing XICS\n>> framework available.\n>>\n>>>> That should allow a more natural XIVE design to emerge, *then* we can\n>>>> look at what's necessary to make boot-time negotiation possible.\n>>>\n>>> Actually, it just occurred to me that we might be making life hard for\n>>> ourselves by trying to actually switch between full XICS and XIVE\n>>> models. Coudln't we have new machine types always construct the XIVE\n>>> infrastructure, \n>>\n>> yes.\n>>\n>>> but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n>>> hardware.\n>>\n>> ok but migration will not be supported.\n> \n> Right, this would only be for newer machine types, and you can never\n> migrate between different machine types.\n> \n>>> Since something more or less equivalent\n>>> has already been done in both OPAL and the host kernel, I'm guessing\n>>> this shouldn't be too hard at this point.\n>>\n>> Indeed that is how it is working currently on P9 kvm guests. hcalls are\n>> implemented on top of XIVE native.\n>>\n>> Thanks,\n>>\n>>\n>> C.\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xydyt1W9Vz9t4B\n\tfor ;\n\tFri, 22 Sep 2017 00:19:18 +1000 (AEST)","from localhost ([::1]:53943 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dv2Jw-0000iV-7F\n\tfor incoming@patchwork.ozlabs.org; Thu, 21 Sep 2017 10:19:16 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:44725)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dv2JQ-0000gX-NY\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 10:18:50 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dv2JN-0008Qe-HP\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 10:18:44 -0400","from 5.mo2.mail-out.ovh.net ([87.98.181.248]:39065)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dv2JN-0008PX-BI\n\tfor qemu-devel@nongnu.org; Thu, 21 Sep 2017 10:18:41 -0400","from player157.ha.ovh.net (b9.ovh.net [213.186.33.59])\n\tby mo2.mail-out.ovh.net (Postfix) with ESMTP id D6453ACAC8\n\tfor ; Thu, 21 Sep 2017 16:18:39 +0200 (CEST)","from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr\n\t[90.76.52.173]) (Authenticated sender: postmaster@kaod.org)\n\tby player157.ha.ovh.net (Postfix) with ESMTPSA id 7F45250009F;\n\tThu, 21 Sep 2017 16:18:33 +0200 (CEST)"],"To":"David Gibson ","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>\n\t<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>\n\t<20170921012508.GA4998@umbus.fritz.box>","From":"=?utf-8?q?C=C3=A9dric_Le_Goater?= ","Message-ID":"<53b4c971-920f-816b-59fb-f77a03663d81@kaod.org>","Date":"Thu, 21 Sep 2017 16:18:33 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170921012508.GA4998@umbus.fritz.box>","Content-Type":"text/plain; charset=windows-1252","Content-Language":"en-US","X-Ovh-Tracer-Id":"14573366922429762424","X-VR-SPAMSTATE":"OK","X-VR-SPAMSCORE":"-100","X-VR-SPAMCAUSE":"gggruggvucftvghtrhhoucdtuddrfeelledriedvgdejjecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"87.98.181.248","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1773466,"web_url":"http://patchwork.ozlabs.org/comment/1773466/","msgid":"<20170922103350.GL4998@umbus.fritz.box>","list_archive_url":null,"date":"2017-09-22T10:33:50","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":47,"url":"http://patchwork.ozlabs.org/api/people/47/","name":"David Gibson","email":"david@gibson.dropbear.id.au"},"content":"On Thu, Sep 21, 2017 at 04:18:33PM +0200, Cédric Le Goater wrote:\n> On 09/21/2017 03:25 AM, David Gibson wrote:\n> > On Wed, Sep 20, 2017 at 02:33:37PM +0200, Cédric Le Goater wrote:\n> >> On 09/19/2017 10:46 AM, David Gibson wrote:\n> >>> On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n> >>>> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n> >>>>> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n> >>>>> negotiation process determines whether the guest operates with an\n> >>>>> interrupt controller using the XICS legacy model, as found on POWER8,\n> >>>>> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n> >>>>> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n> >>>>>\n> >>>>> Follows a model for the XIVE interrupt controller and support for the\n> >>>>> Hypervisor's calls which are used to configure the interrupt sources\n> >>>>> and the event/notification queues of the guest. The last patch\n> >>>>> integrates XIVE in the sPAPR machine.\n> >>>>>\n> >>>>> Code is here:\n> >>>>\n> >>>>\n> >>>> An overall comment:\n> >>>>\n> >>>> I note in several replies here that I think the way XICS objects are\n> >>>> re-used for XIVE is really ugly, and I think it will make future\n> >>>> maintenance pretty painful.\n> >>\n> >> I agree. That was one way to identify what we need for migration \n> >> compatibility and CAS reset. \n> >>\n> >>>> I'm thinking maybe trying to support the CAS negotiation of interrupt\n> >>>> controller from day 1 is warping the design. A better approach might\n> >>>> be first to implement XIVE only when given a specific machine option -\n> >>>> guest gets one or the other and can't negotiate.\n> >>\n> >> ok. \n> >>\n> >> CAS is not the most complex problem, we mostly need to share \n> >> the ICSIRQState array and the source offset. migration from older\n> >> machine is a problem.\n> > \n> > Uh.. what? Migration from an older machine isn't a thing. We can\n> > migrate from an older qemu, but the machine type (and version) has to\n> > be identical at each end. That's *why* we keep around the older\n> > machine types on newer qemus.\n> \n> yes. I am just wondering how I am going to handle a xics-only \n> machine migrating to a xics/xive machine. \n\nWon't ever happen. Older machine types will always be xics, newer\nmachine type will always be xive (at least with POWER9).\n\n> The xive machine option we are talking about will activate \n> the xive interrupt mode and instantiate the objects behind it. \n> So when we migrate from an older machine we will need to start \n> the target machine with xive=off. I guess that is OK.\n\nAgain, we *don't* migrate from an older machine. Ever. We only ever\nmigrate from an older qemu version to a newer qemu using the older\nmachine type.\n> \n> Thanks for the insights and the time to review the code,\n> \n> C. \n> \n> >> We are doomed to keep the existing XICS\n> >> framework available.\n> >>\n> >>>> That should allow a more natural XIVE design to emerge, *then* we can\n> >>>> look at what's necessary to make boot-time negotiation possible.\n> >>>\n> >>> Actually, it just occurred to me that we might be making life hard for\n> >>> ourselves by trying to actually switch between full XICS and XIVE\n> >>> models. Coudln't we have new machine types always construct the XIVE\n> >>> infrastructure, \n> >>\n> >> yes.\n> >>\n> >>> but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n> >>> hardware.\n> >>\n> >> ok but migration will not be supported.\n> > \n> > Right, this would only be for newer machine types, and you can never\n> > migrate between different machine types.\n> > \n> >>> Since something more or less equivalent\n> >>> has already been done in both OPAL and the host kernel, I'm guessing\n> >>> this shouldn't be too hard at this point.\n> >>\n> >> Indeed that is how it is working currently on P9 kvm guests. hcalls are\n> >> implemented on top of XIVE native.\n> >>\n> >> Thanks,\n> >>\n> >>\n> >> C.\n> >>\n> > \n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=gibson.dropbear.id.au\n\theader.i=@gibson.dropbear.id.au header.b=\"ox/dTti6\"; \n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xz92K1bhvz9s82\n\tfor ;\n\tFri, 22 Sep 2017 20:39:04 +1000 (AEST)","from localhost ([::1]:57882 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dvLMK-0006Lj-Oe\n\tfor incoming@patchwork.ozlabs.org; Fri, 22 Sep 2017 06:39:00 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:58274)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dvLLu-0006LQ-JF\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 06:38:36 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dvLLt-0000xT-AT\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 06:38:34 -0400","from ozlabs.org ([2401:3900:2:1::2]:58749)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dvLLs-0000vx-VZ; Fri, 22 Sep 2017 06:38:33 -0400","by ozlabs.org (Postfix, from userid 1007)\n\tid 3xz91c6qvtz9sNw; Fri, 22 Sep 2017 20:38:28 +1000 (AEST)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple;\n\td=gibson.dropbear.id.au; s=201602; t=1506076708;\n\tbh=27i8NbME/MZI+9cyPkE2iIoCETIGNYNdy9rr4+uR3dQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=ox/dTti6vp8qwzbt/LNk+ZTv7XCR8ATf8g4Dg2/qIZa7TH7zpqa4JPXZVHMlyhW/u\n\t5cy62ln/Nr9CzcJfaLsZ41gBBC11pIWUt4KwJPdlSBajBIIDDIb2Fjqis6Yx6uxQrg\n\tasZZ8LCPybYtkKB+ZNlgjAN/WGbSQ9Fg9SQqIOyM=","Date":"Fri, 22 Sep 2017 20:33:50 +1000","From":"David Gibson ","To":"=?iso-8859-1?q?C=E9dric?= Le Goater ","Message-ID":"<20170922103350.GL4998@umbus.fritz.box>","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>\n\t<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>\n\t<20170921012508.GA4998@umbus.fritz.box>\n\t<53b4c971-920f-816b-59fb-f77a03663d81@kaod.org>","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\"; boundary=\"zH41lVBEV8cLJnCl\"","Content-Disposition":"inline","In-Reply-To":"<53b4c971-920f-816b-59fb-f77a03663d81@kaod.org>","User-Agent":"Mutt/1.9.0 (2017-09-02)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2401:3900:2:1::2","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1773557,"web_url":"http://patchwork.ozlabs.org/comment/1773557/","msgid":"<77052464-e455-757e-c2a7-e55bb7d26d14@kaod.org>","list_archive_url":null,"date":"2017-09-22T12:32:33","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":68548,"url":"http://patchwork.ozlabs.org/api/people/68548/","name":"Cédric Le Goater","email":"clg@kaod.org"},"content":"On 09/22/2017 12:33 PM, David Gibson wrote:\n> On Thu, Sep 21, 2017 at 04:18:33PM +0200, Cédric Le Goater wrote:\n>> On 09/21/2017 03:25 AM, David Gibson wrote:\n>>> On Wed, Sep 20, 2017 at 02:33:37PM +0200, Cédric Le Goater wrote:\n>>>> On 09/19/2017 10:46 AM, David Gibson wrote:\n>>>>> On Tue, Sep 19, 2017 at 06:20:20PM +1000, David Gibson wrote:\n>>>>>> On Mon, Sep 11, 2017 at 07:12:14PM +0200, Cédric Le Goater wrote:\n>>>>>>> On a POWER9 sPAPR machine, the Client Architecture Support (CAS)\n>>>>>>> negotiation process determines whether the guest operates with an\n>>>>>>> interrupt controller using the XICS legacy model, as found on POWER8,\n>>>>>>> or in XIVE exploitation mode, the newer POWER9 interrupt model. This\n>>>>>>> patchset is a proposal to add XIVE support in POWER9 sPAPR machine.\n>>>>>>>\n>>>>>>> Follows a model for the XIVE interrupt controller and support for the\n>>>>>>> Hypervisor's calls which are used to configure the interrupt sources\n>>>>>>> and the event/notification queues of the guest. The last patch\n>>>>>>> integrates XIVE in the sPAPR machine.\n>>>>>>>\n>>>>>>> Code is here:\n>>>>>>\n>>>>>>\n>>>>>> An overall comment:\n>>>>>>\n>>>>>> I note in several replies here that I think the way XICS objects are\n>>>>>> re-used for XIVE is really ugly, and I think it will make future\n>>>>>> maintenance pretty painful.\n>>>>\n>>>> I agree. That was one way to identify what we need for migration \n>>>> compatibility and CAS reset. \n>>>>\n>>>>>> I'm thinking maybe trying to support the CAS negotiation of interrupt\n>>>>>> controller from day 1 is warping the design. A better approach might\n>>>>>> be first to implement XIVE only when given a specific machine option -\n>>>>>> guest gets one or the other and can't negotiate.\n>>>>\n>>>> ok. \n>>>>\n>>>> CAS is not the most complex problem, we mostly need to share \n>>>> the ICSIRQState array and the source offset. migration from older\n>>>> machine is a problem.\n>>>\n>>> Uh.. what? Migration from an older machine isn't a thing. We can\n>>> migrate from an older qemu, but the machine type (and version) has to\n>>> be identical at each end. That's *why* we keep around the older\n>>> machine types on newer qemus.\n>>\n>> yes. I am just wondering how I am going to handle a xics-only \n>> machine migrating to a xics/xive machine. \n> \n> Won't ever happen. Older machine types will always be xics, newer\n> machine type will always be xive (at least with POWER9).\n> \n>> The xive machine option we are talking about will activate \n>> the xive interrupt mode and instantiate the objects behind it. \n>> So when we migrate from an older machine we will need to start \n>> the target machine with xive=off. I guess that is OK.\n> \n> Again, we *don't* migrate from an older machine. Ever. We only ever\n> migrate from an older qemu version to a newer qemu using the older\n> machine type.\n\nSorry I was talking about QEMU version, and not machine version.\nI still have to look at how both machines will cohabitate in the \nnewer QEMU. \n\nThanks,\n\nC. \n\n\n>>\n>> Thanks for the insights and the time to review the code,\n>>\n>> C. \n>>\n>>>> We are doomed to keep the existing XICS\n>>>> framework available.\n>>>>\n>>>>>> That should allow a more natural XIVE design to emerge, *then* we can\n>>>>>> look at what's necessary to make boot-time negotiation possible.\n>>>>>\n>>>>> Actually, it just occurred to me that we might be making life hard for\n>>>>> ourselves by trying to actually switch between full XICS and XIVE\n>>>>> models. Coudln't we have new machine types always construct the XIVE\n>>>>> infrastructure, \n>>>>\n>>>> yes.\n>>>>\n>>>>> but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n>>>>> hardware.\n>>>>\n>>>> ok but migration will not be supported.\n>>>\n>>> Right, this would only be for newer machine types, and you can never\n>>> migrate between different machine types.\n>>>\n>>>>> Since something more or less equivalent\n>>>>> has already been done in both OPAL and the host kernel, I'm guessing\n>>>>> this shouldn't be too hard at this point.\n>>>>\n>>>> Indeed that is how it is working currently on P9 kvm guests. hcalls are\n>>>> implemented on top of XIVE native.\n>>>>\n>>>> Thanks,\n>>>>\n>>>>\n>>>> C.\n>>>>\n>>>\n>>\n>","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xzCnV5XPjz9sPk\n\tfor ;\n\tFri, 22 Sep 2017 22:43:10 +1000 (AEST)","from localhost ([::1]:58674 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dvNIS-0000WK-Pr\n\tfor incoming@patchwork.ozlabs.org; Fri, 22 Sep 2017 08:43:08 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56368)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dvN8V-0000Z6-5j\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 08:32:52 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dvN8R-0005og-1j\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 08:32:51 -0400","from 5.mo179.mail-out.ovh.net ([46.105.43.140]:54850)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dvN8Q-0005nr-PR\n\tfor qemu-devel@nongnu.org; Fri, 22 Sep 2017 08:32:46 -0400","from player690.ha.ovh.net (b6.ovh.net [213.186.33.56])\n\tby mo179.mail-out.ovh.net (Postfix) with ESMTP id 6321964E06\n\tfor ; Fri, 22 Sep 2017 14:32:45 +0200 (CEST)","from zorba.kaod.org (LFbn-1-2231-173.w90-76.abo.wanadoo.fr\n\t[90.76.52.173]) (Authenticated sender: postmaster@kaod.org)\n\tby player690.ha.ovh.net (Postfix) with ESMTPSA id 53194540071;\n\tFri, 22 Sep 2017 14:32:39 +0200 (CEST)"],"To":"David Gibson ","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>\n\t<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>\n\t<20170921012508.GA4998@umbus.fritz.box>\n\t<53b4c971-920f-816b-59fb-f77a03663d81@kaod.org>\n\t<20170922103350.GL4998@umbus.fritz.box>","From":"=?utf-8?q?C=C3=A9dric_Le_Goater?= ","Message-ID":"<77052464-e455-757e-c2a7-e55bb7d26d14@kaod.org>","Date":"Fri, 22 Sep 2017 14:32:33 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170922103350.GL4998@umbus.fritz.box>","Content-Type":"text/plain; charset=windows-1252","Content-Language":"en-US","X-Ovh-Tracer-Id":"210824758166063992","X-VR-SPAMSTATE":"OK","X-VR-SPAMSCORE":"-100","X-VR-SPAMCAUSE":"gggruggvucftvghtrhhoucdtuddrfeelledrieeggdehhecutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfqggfjpdevjffgvefmvefgnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd","Content-Transfer-Encoding":"quoted-printable","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"46.105.43.140","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1776855,"web_url":"http://patchwork.ozlabs.org/comment/1776855/","msgid":"<1506587002.25626.20.camel@kernel.crashing.org>","list_archive_url":null,"date":"2017-09-28T08:23:22","subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","submitter":{"id":38,"url":"http://patchwork.ozlabs.org/api/people/38/","name":"Benjamin Herrenschmidt","email":"benh@kernel.crashing.org"},"content":"On Wed, 2017-09-20 at 14:33 +0200, Cédric Le Goater wrote:\n> > > I'm thinking maybe trying to support the CAS negotiation of interrupt\n> > > controller from day 1 is warping the design. A better approach might\n> > > be first to implement XIVE only when given a specific machine option -\n> > > guest gets one or the other and can't negotiate.\n> \n> ok. \n> \n> CAS is not the most complex problem, we mostly need to share \n> the ICSIRQState array and the source offset. migration from older\n> machine is a problem. We are doomed to keep the existing XICS\n> framework available.\n\nI don't like sharing anything. I'd rather we had separate objects\nalltogether. If needed we can implement CAS by doing a partition reboot\nlike pHyp does, at least initially, until we add ways to tear down and\nrebuild objects.\n\nThe main issue is whether we can keep a consistent number space so the\nDT doesn't have to be completely rebuilt. If it does, then reboot will\nbe the only practical option I'm afraid.\n\n> > > That should allow a more natural XIVE design to emerge, *then* we can\n> > > look at what's necessary to make boot-time negotiation possible.\n> > \n> > Actually, it just occurred to me that we might be making life hard for\n> > ourselves by trying to actually switch between full XICS and XIVE\n> > models. Coudln't we have new machine types always construct the XIVE\n> > infrastructure, \n> \n> yes.\n> \n> > but then implement the XICS RTAS and hcalls in terms of the XIVE virtual \n> > hardware.\n\nThat's gross :-)\n\nThis is also exactly what KVM does with real XIVE HW and there's also\nsuch an emulation in OPAL. I'd be weary of creating a 3rd one...\n\nI'd much prefer if we managed to:\n\n - Split the source numbering from the various state tracking objects\nso we can have that common\n\n - Either delay the creation to after CAS or tear down & re-create the\nstate tracking objects at CAS time.\n\n> ok but migration will not be supported.\n> \n> > Since something more or less equivalent\n> > has already been done in both OPAL and the host kernel, I'm guessing\n> > this shouldn't be too hard at this point.\n\nIt would very much suck to have yet another one of these.\n\nAlso we need to understand how that would work in a KVM context, the\nkernel will provide a \"XICS\" state even on top of XIVE unless we switch\nthe kernel object to native, but then the kernel will expect full\nexploitation.\n\n> Indeed that is how it is working currently on P9 kvm guests. hcalls are\n> implemented on top of XIVE native.\n> \n> Thanks,\n> \n> \n> C.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":"ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3y2nm14LG3z9tXd\n\tfor ;\n\tThu, 28 Sep 2017 18:24:15 +1000 (AEST)","from localhost ([::1]:57924 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dxU7B-0007BR-Ao\n\tfor incoming@patchwork.ozlabs.org; Thu, 28 Sep 2017 04:24:13 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:45691)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dxU6k-0007B8-GU\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 04:23:48 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dxU6h-00026B-Ak\n\tfor qemu-devel@nongnu.org; Thu, 28 Sep 2017 04:23:46 -0400","from gate.crashing.org ([63.228.1.57]:56614)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dxU6g-00025b-V4; Thu, 28 Sep 2017 04:23:43 -0400","from localhost (localhost.localdomain [127.0.0.1])\n\tby gate.crashing.org (8.14.1/8.13.8) with ESMTP id v8S8NMge028985;\n\tThu, 28 Sep 2017 03:23:24 -0500"],"Message-ID":"<1506587002.25626.20.camel@kernel.crashing.org>","From":"Benjamin Herrenschmidt ","To":"=?iso-8859-1?q?C=E9dric?= Le Goater , David Gibson\n\t","Date":"Thu, 28 Sep 2017 10:23:22 +0200","In-Reply-To":"<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","References":"<20170911171235.29331-1-clg@kaod.org>\n\t<20170919082020.GT27153@umbus> <20170919084618.GY27153@umbus>\n\t<98b6f737-a554-1c83-79f9-2c8021e1cf69@kaod.org>","Content-Type":"text/plain; charset=\"UTF-8\"","X-Mailer":"Evolution 3.24.5 (3.24.5-1.fc26) ","Mime-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","X-MIME-Autoconverted":"from 8bit to quoted-printable by gate.crashing.org id\n\tv8S8NMge028985","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy]","X-Received-From":"63.228.1.57","Subject":"Re: [Qemu-devel] [RFC PATCH v2 00/21] Guest exploitation of the\n\tXIVE interrupt controller (POWER9)","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Alexey Kardashevskiy , qemu-ppc@nongnu.org,\n\tqemu-devel@nongnu.org, Alexander Graf ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]