From patchwork Tue Feb 19 17:18:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kurz X-Patchwork-Id: 1044816 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=kaod.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 443nl15SW0z9s7h for ; Wed, 20 Feb 2019 04:28:29 +1100 (AEDT) Received: from localhost ([127.0.0.1]:52483 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gw9Bz-0001eP-Ie for incoming@patchwork.ozlabs.org; Tue, 19 Feb 2019 12:28:27 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33697) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gw92M-0002Vd-R9 for qemu-devel@nongnu.org; Tue, 19 Feb 2019 12:18:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gw92K-0007cb-En for qemu-devel@nongnu.org; Tue, 19 Feb 2019 12:18:30 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:33062 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gw92K-0007aa-46 for qemu-devel@nongnu.org; Tue, 19 Feb 2019 12:18:28 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1JHElB0108034 for ; Tue, 19 Feb 2019 12:18:26 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 2qrm5rpgxm-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 19 Feb 2019 12:18:26 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 19 Feb 2019 17:18:23 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 19 Feb 2019 17:18:20 -0000 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x1JHIJEB60489754 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 19 Feb 2019 17:18:19 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 21E5B42042; Tue, 19 Feb 2019 17:18:19 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B8044204B; Tue, 19 Feb 2019 17:18:19 +0000 (GMT) Received: from smtp.lab.toulouse-stg.fr.ibm.com (unknown [9.101.4.1]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 19 Feb 2019 17:18:19 +0000 (GMT) Received: from bahia.lab.toulouse-stg.fr.ibm.com (bahia.lab.toulouse-stg.fr.ibm.com [9.101.4.41]) by smtp.lab.toulouse-stg.fr.ibm.com (Postfix) with ESMTP id CF69022019A; Tue, 19 Feb 2019 18:18:18 +0100 (CET) From: Greg Kurz To: David Gibson Date: Tue, 19 Feb 2019 18:18:18 +0100 In-Reply-To: <155059665292.1466090.8750653555749574947.stgit@bahia.lab.toulouse-stg.fr.ibm.com> References: <155059665292.1466090.8750653555749574947.stgit@bahia.lab.toulouse-stg.fr.ibm.com> User-Agent: StGit/unknown-version MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19021917-0012-0000-0000-000002F7A37F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19021917-0013-0000-0000-0000212F314B Message-Id: <155059669881.1466090.13515030705986041517.stgit@bahia.lab.toulouse-stg.fr.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-02-19_11:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1034 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1902190127 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH v5 09/17] spapr_pci: add PHB unrealize X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , Cornelia Huck , Gerd Hoffmann , Michael Roth , "Michael S. Tsirkin" , Alexey Kardashevskiy , David Hildenbrand , qemu-devel@nongnu.org, Greg Kurz , qemu-s390x@nongnu.org, Dmitry Fleytman , qemu-ppc@nongnu.org, =?utf-8?q?C=C3=A9dric?= Le Goater , Marcel Apfelbaum , Paolo Bonzini , Eduardo Habkost Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To support PHB hotplug we need to clean up lingering references, memory, child properties, etc. prior to the PHB object being finalized. Generally this will be called as a result of calling object_unparent() on the PHB object, which in turn would normally be called as the result of an unplug() operation. When the PHB is finalized, child objects will be unparented in turn, and finalized if the PHB was the only reference holder. so we don't bother to explicitly unparent child objects of the PHB, with the notable exception of DRCs. This is needed to avoid a QEMU crash when unplugging a PHB and resetting the machine before the guest could handle the event. The DRCs are removed from the QOM tree by pci_unregister_root_bus() and we must make sure we're not leaving stale aliases under the global /dr-connector path. The formula that gives the number of DMA windows is moved to an inline function in the hw/pci-host/spapr.h header because it will have other users. The unrealize function is able to cope with partially realized PHBs. It is hence used to implement proper rollback on the realize error path. Signed-off-by: Michael Roth Signed-off-by: Greg Kurz Reviewed-by: David Gibson --- v5: - unparent child DRCs at unrealize v4: - reverted to v2 v3: - don't free LSIs at unrealize v2: - implement rollback with unrealize function --- --- hw/ppc/spapr_pci.c | 86 +++++++++++++++++++++++++++++++++++++++++-- include/hw/pci-host/spapr.h | 5 +++ 2 files changed, 87 insertions(+), 4 deletions(-) diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index e2bc9fec824b..ede928b0bff3 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1570,6 +1570,75 @@ static void spapr_pci_unplug_request(HotplugHandler *plug_handler, } } +static void spapr_phb_finalizefn(Object *obj) +{ + sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(obj); + + g_free(sphb->dtbusname); + sphb->dtbusname = NULL; +} + +static void spapr_phb_unrealize(DeviceState *dev, Error **errp) +{ + sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine()); + SysBusDevice *s = SYS_BUS_DEVICE(dev); + PCIHostState *phb = PCI_HOST_BRIDGE(s); + sPAPRPHBState *sphb = SPAPR_PCI_HOST_BRIDGE(phb); + sPAPRTCETable *tcet; + int i; + const unsigned windows_supported = spapr_phb_windows_supported(sphb); + + if (sphb->msi) { + g_hash_table_unref(sphb->msi); + sphb->msi = NULL; + } + + /* + * Remove IO/MMIO subregions and aliases, rest should get cleaned + * via PHB's unrealize->object_finalize + */ + for (i = windows_supported - 1; i >= 0; i--) { + tcet = spapr_tce_find_by_liobn(sphb->dma_liobn[i]); + if (tcet) { + memory_region_del_subregion(&sphb->iommu_root, + spapr_tce_get_iommu(tcet)); + } + } + + if (sphb->dr_enabled) { + for (i = PCI_SLOT_MAX * 8 - 1; i >= 0; i--) { + sPAPRDRConnector *drc = spapr_drc_by_id(TYPE_SPAPR_DRC_PCI, + (sphb->index << 16) | i); + + if (drc) { + object_unparent(OBJECT(drc)); + } + } + } + + for (i = PCI_NUM_PINS - 1; i >= 0; i--) { + if (sphb->lsi_table[i].irq) { + spapr_irq_free(spapr, sphb->lsi_table[i].irq, 1); + sphb->lsi_table[i].irq = 0; + } + } + + QLIST_REMOVE(sphb, list); + + memory_region_del_subregion(&sphb->iommu_root, &sphb->msiwindow); + + address_space_destroy(&sphb->iommu_as); + + qbus_set_hotplug_handler(BUS(phb->bus), NULL, &error_abort); + pci_unregister_root_bus(phb->bus); + + memory_region_del_subregion(get_system_memory(), &sphb->iowindow); + if (sphb->mem64_win_pciaddr != (hwaddr)-1) { + memory_region_del_subregion(get_system_memory(), &sphb->mem64window); + } + memory_region_del_subregion(get_system_memory(), &sphb->mem32window); +} + static void spapr_phb_realize(DeviceState *dev, Error **errp) { /* We don't use SPAPR_MACHINE() in order to exit gracefully if the user @@ -1587,8 +1656,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) PCIBus *bus; uint64_t msi_window_size = 4096; sPAPRTCETable *tcet; - const unsigned windows_supported = - sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1; + const unsigned windows_supported = spapr_phb_windows_supported(sphb); if (!spapr) { error_setg(errp, TYPE_SPAPR_PCI_HOST_BRIDGE " needs a pseries machine"); @@ -1745,6 +1813,10 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) if (local_err) { error_propagate_prepend(errp, local_err, "can't allocate LSIs: "); + /* + * Older machines will never support PHB hotplug, ie, this is an + * init only path and QEMU will terminate. No need to rollback. + */ return; } } @@ -1752,7 +1824,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) spapr_irq_claim(spapr, irq, true, &local_err); if (local_err) { error_propagate_prepend(errp, local_err, "can't allocate LSIs: "); - return; + goto unrealize; } sphb->lsi_table[i].irq = irq; @@ -1772,13 +1844,17 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) if (!tcet) { error_setg(errp, "Creating window#%d failed for %s", i, sphb->dtbusname); - return; + goto unrealize; } memory_region_add_subregion(&sphb->iommu_root, 0, spapr_tce_get_iommu(tcet)); } sphb->msi = g_hash_table_new_full(g_int_hash, g_int_equal, g_free, g_free); + return; + +unrealize: + spapr_phb_unrealize(dev, NULL); } static int spapr_phb_children_reset(Object *child, void *opaque) @@ -1977,6 +2053,7 @@ static void spapr_phb_class_init(ObjectClass *klass, void *data) hc->root_bus_path = spapr_phb_root_bus_path; dc->realize = spapr_phb_realize; + dc->unrealize = spapr_phb_unrealize; dc->props = spapr_phb_properties; dc->reset = spapr_phb_reset; dc->vmsd = &vmstate_spapr_pci; @@ -1992,6 +2069,7 @@ static const TypeInfo spapr_phb_info = { .name = TYPE_SPAPR_PCI_HOST_BRIDGE, .parent = TYPE_PCI_HOST_BRIDGE, .instance_size = sizeof(sPAPRPHBState), + .instance_finalize = spapr_phb_finalizefn, .class_init = spapr_phb_class_init, .interfaces = (InterfaceInfo[]) { { TYPE_HOTPLUG_HANDLER }, diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h index f6e43f48fefa..4b0443f4cfe4 100644 --- a/include/hw/pci-host/spapr.h +++ b/include/hw/pci-host/spapr.h @@ -165,4 +165,9 @@ static inline void spapr_phb_vfio_reset(DeviceState *qdev) void spapr_phb_dma_reset(sPAPRPHBState *sphb); +static inline unsigned spapr_phb_windows_supported(sPAPRPHBState *sphb) +{ + return sphb->ddw_enabled ? SPAPR_PCI_DMA_MAX_WINDOWS : 1; +} + #endif /* PCI_HOST_SPAPR_H */