From patchwork Fri Nov 27 10:07:06 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 549368 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 6441714031C for ; Fri, 27 Nov 2015 21:08:15 +1100 (AEDT) Received: from localhost ([::1]:55624 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a2Fwq-0001BU-UM for incoming@patchwork.ozlabs.org; Fri, 27 Nov 2015 05:08:12 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58904) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a2FwB-00089c-Bi for qemu-devel@nongnu.org; Fri, 27 Nov 2015 05:07:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a2Fw6-0003rg-UO for qemu-devel@nongnu.org; Fri, 27 Nov 2015 05:07:31 -0500 Received: from e06smtp16.uk.ibm.com ([195.75.94.112]:41576) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a2Fw6-0003qj-LE for qemu-devel@nongnu.org; Fri, 27 Nov 2015 05:07:26 -0500 Received: from localhost by e06smtp16.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 27 Nov 2015 10:07:25 -0000 Received: from d06dlp02.portsmouth.uk.ibm.com (9.149.20.14) by e06smtp16.uk.ibm.com (192.168.101.146) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 27 Nov 2015 10:07:22 -0000 X-IBM-Helo: d06dlp02.portsmouth.uk.ibm.com X-IBM-MailFrom: cornelia.huck@de.ibm.com X-IBM-RcptTo: qemu-devel@nongnu.org Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by d06dlp02.portsmouth.uk.ibm.com (Postfix) with ESMTP id 3C70A219004D for ; Fri, 27 Nov 2015 10:07:16 +0000 (GMT) Received: from d06av04.portsmouth.uk.ibm.com (d06av04.portsmouth.uk.ibm.com [9.149.37.216]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id tARA7MOC8847756 for ; Fri, 27 Nov 2015 10:07:22 GMT Received: from d06av04.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av04.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id tARA7KWD008757 for ; Fri, 27 Nov 2015 03:07:22 -0700 Received: from gondolin.boeblingen.de.ibm.com (dyn-9-152-224-100.boeblingen.de.ibm.com [9.152.224.100]) by d06av04.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id tARA78Tp008157 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Nov 2015 03:07:17 -0700 From: Cornelia Huck To: qemu-devel@nongnu.org Date: Fri, 27 Nov 2015 11:07:06 +0100 Message-Id: <1448618826-29546-5-git-send-email-cornelia.huck@de.ibm.com> X-Mailer: git-send-email 2.6.3 In-Reply-To: <1448618826-29546-1-git-send-email-cornelia.huck@de.ibm.com> References: <1448618826-29546-1-git-send-email-cornelia.huck@de.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15112710-0025-0000-0000-000007F18696 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 195.75.94.112 Cc: Cornelia Huck , borntraeger@de.ibm.com, jfrei@linux.vnet.ibm.com, Yi Min Zhao , agraf@suse.de Subject: [Qemu-devel] [PATCH for-2.5 4/4] s390x/pci: fix up IOMMU size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Yi Min Zhao Present code uses @size==UINT64_MAX to initialize IOMMU. It infers that it can map any 64-bit IOVA whatsoever. But in fact, the largest DMA range for each PCI Device on s390x is from ZPCI_SDMA_ADDR to ZPCI_EDMA_ADDR. The largest value is returned from hardware, which is to indicate the largest range hardware can support. But the real IOMMU size for specific PCI Device is obtained once qemu intercepts mpcifc instruction that guest is requesting a DMA range for that PCI Device. Therefore, before intercepting mpcifc instruction, qemu cannot be aware of the size of IOMMU region that guest will use. Moreover, iommu replay during device initialization for the whole region in 4k steps takes a very long time. In conclusion, this patch intializes IOMMU region for each PCI Device when intercept mpcifc instruction which is to register DMA range for the PCI Device. And then, destroy IOMMU region when guest wants to deregister IOAT. Signed-off-by: Yi Min Zhao Reviewed-by: Cornelia Huck Signed-off-by: Cornelia Huck --- hw/s390x/s390-pci-bus.c | 26 ++++++++++++++++++++++---- hw/s390x/s390-pci-bus.h | 2 ++ hw/s390x/s390-pci-inst.c | 7 ++++++- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index d5712ae..98c726c 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -308,7 +308,7 @@ static IOMMUTLBEntry s390_translate_iommu(MemoryRegion *iommu, hwaddr addr, { uint64_t pte; uint32_t flags; - S390PCIBusDevice *pbdev = container_of(iommu, S390PCIBusDevice, mr); + S390PCIBusDevice *pbdev = container_of(iommu, S390PCIBusDevice, iommu_mr); S390pciState *s; IOMMUTLBEntry ret = { .target_as = &address_space_memory, @@ -454,14 +454,32 @@ static const MemoryRegionOps s390_msi_ctrl_ops = { .endianness = DEVICE_LITTLE_ENDIAN, }; +void s390_pcihost_iommu_configure(S390PCIBusDevice *pbdev, bool enable) +{ + pbdev->configured = false; + + if (enable) { + uint64_t size = pbdev->pal - pbdev->pba + 1; + memory_region_init_iommu(&pbdev->iommu_mr, OBJECT(&pbdev->mr), + &s390_iommu_ops, "iommu-s390", size); + memory_region_add_subregion(&pbdev->mr, pbdev->pba, &pbdev->iommu_mr); + } else { + memory_region_del_subregion(&pbdev->mr, &pbdev->iommu_mr); + } + + pbdev->configured = true; +} + static void s390_pcihost_init_as(S390pciState *s) { int i; + S390PCIBusDevice *pbdev; for (i = 0; i < PCI_SLOT_MAX; i++) { - memory_region_init_iommu(&s->pbdev[i].mr, OBJECT(s), - &s390_iommu_ops, "iommu-s390", UINT64_MAX); - address_space_init(&s->pbdev[i].as, &s->pbdev[i].mr, "iommu-pci"); + pbdev = &s->pbdev[i]; + memory_region_init(&pbdev->mr, OBJECT(s), + "iommu-root-s390", UINT64_MAX); + address_space_init(&pbdev->as, &pbdev->mr, "iommu-pci"); } memory_region_init_io(&s->msix_notify_mr, OBJECT(s), diff --git a/hw/s390x/s390-pci-bus.h b/hw/s390x/s390-pci-bus.h index 464a92e..80345da 100644 --- a/hw/s390x/s390-pci-bus.h +++ b/hw/s390x/s390-pci-bus.h @@ -231,6 +231,7 @@ typedef struct S390PCIBusDevice { AdapterRoutes routes; AddressSpace as; MemoryRegion mr; + MemoryRegion iommu_mr; } S390PCIBusDevice; typedef struct S390pciState { @@ -244,6 +245,7 @@ typedef struct S390pciState { int chsc_sei_nt2_get_event(void *res); int chsc_sei_nt2_have_event(void); void s390_pci_sclp_configure(int configure, SCCB *sccb); +void s390_pcihost_iommu_configure(S390PCIBusDevice *pbdev, bool enable); S390PCIBusDevice *s390_pci_find_dev_by_idx(uint32_t idx); S390PCIBusDevice *s390_pci_find_dev_by_fh(uint32_t fh); S390PCIBusDevice *s390_pci_find_dev_by_fid(uint32_t fid); diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index f9151a9..8c1dc82 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -528,7 +528,7 @@ int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2) goto out; } - mr = pci_device_iommu_address_space(pbdev->pdev)->root; + mr = &pbdev->iommu_mr; while (start < end) { entry = mr->iommu_ops->translate(mr, start, 0); @@ -689,6 +689,9 @@ static int reg_ioat(CPUS390XState *env, S390PCIBusDevice *pbdev, ZpciFib fib) pbdev->pba = pba; pbdev->pal = pal; pbdev->g_iota = g_iota; + + s390_pcihost_iommu_configure(pbdev, true); + return 0; } @@ -697,6 +700,8 @@ static void dereg_ioat(S390PCIBusDevice *pbdev) pbdev->pba = 0; pbdev->pal = 0; pbdev->g_iota = 0; + + s390_pcihost_iommu_configure(pbdev, false); } int mpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t ar)