From patchwork Tue May 30 17:59:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787804 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=V65n4XPU; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Zz3z5rz20Pc for ; Wed, 31 May 2023 04:01:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43dn-0008Cl-NP; Tue, 30 May 2023 14:00:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dl-0008Bp-Cp for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:13 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dd-0002Bt-FU for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:13 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4wGu017005; Tue, 30 May 2023 18:00:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=gL0CLVOm8SGnDZaw05UgGdYm3M9XJQzrTCLBW7+qBi0=; b=V65n4XPUr80R3rxZ/4Cmcx2fxEZ1NMyqXxSASzYQnGnavULEwhO9uH2id2XuZyAEtOaz nIorNuXV6NRjtnrzDuLjaerHD+p5M6/HZjoNhzcH/z2qvrKPFgNEwF5pLYvJYVGm3j3e R60308FHS6Gpr5gRPO07WIEqf39qyeWq+wFpi6e2NDJN8gFFIkDrG2XwYTWO+ZE913hX 1IN5gLfYqos5Jxoyum6E4gXTwsiI33J6hgkfPaBc4UxRLQEMdoldMqtNSuOQ7PIPMDBn LkPDuz9GX37SOZsbNjhEwOKYZZo5F2+AOAZAIe1yX0rn2d4iBU3dsJh2OMeA1e/zAYOf eQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjkketn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:00 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHaGvd014712; Tue, 30 May 2023 17:59:59 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48t12-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 17:59:59 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspD011459; Tue, 30 May 2023 17:59:58 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-2; Tue, 30 May 2023 17:59:58 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 01/15] hw/pci: Refactor pci_device_iommu_address_space() Date: Tue, 30 May 2023 18:59:23 +0100 Message-Id: <20230530175937.24202-2-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=833 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: qIc0ztAymb9v3Bgs5IgKvop2mUn3P9_g X-Proofpoint-GUID: qIc0ztAymb9v3Bgs5IgKvop2mUn3P9_g Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Rename pci_device_iommu_address_space() into pci_device_iommu_info(). In the new function return a new type PCIAddressSpace that encapsulates the AddressSpace pointer that originally was returned. The new type is added in preparation to expanding it to include the IOMMU memory region as a new field, such that we are able to fetch attributes of the vIOMMU e.g. at vfio migration setup. Signed-off-by: Joao Martins --- hw/pci/pci.c | 9 ++++++--- include/hw/pci/pci.h | 21 ++++++++++++++++++++- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 1cc7c89036b5..ecf8a543aa77 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2633,11 +2633,12 @@ static void pci_device_class_base_init(ObjectClass *klass, void *data) } } -AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +PCIAddressSpace pci_device_iommu_info(PCIDevice *dev) { PCIBus *bus = pci_get_bus(dev); PCIBus *iommu_bus = bus; uint8_t devfn = dev->devfn; + AddressSpace *as = NULL; while (iommu_bus && !iommu_bus->iommu_fn && iommu_bus->parent_dev) { PCIBus *parent_bus = pci_get_bus(iommu_bus->parent_dev); @@ -2678,10 +2679,12 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) iommu_bus = parent_bus; } + + as = &address_space_memory; if (!pci_bus_bypass_iommu(bus) && iommu_bus && iommu_bus->iommu_fn) { - return iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); + as = iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); } - return &address_space_memory; + return as_to_pci_as(as); } void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index e6d0574a2999..9ffaf47fe2ab 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -363,9 +363,28 @@ void pci_bus_get_w64_range(PCIBus *bus, Range *range); void pci_device_deassert_intx(PCIDevice *dev); +typedef struct PCIAddressSpace { + AddressSpace *as; +} PCIAddressSpace; + typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); +static inline PCIAddressSpace as_to_pci_as(AddressSpace *as) +{ + PCIAddressSpace ret = { .as = as }; + + return ret; +} +static inline AddressSpace *pci_as_to_as(PCIAddressSpace pci_as) +{ + return pci_as.as; +} + +PCIAddressSpace pci_device_iommu_info(PCIDevice *dev); +static inline AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +{ + return pci_as_to_as(pci_device_iommu_info(dev)); +} -AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); pcibus_t pci_bar_address(PCIDevice *d, From patchwork Tue May 30 17:59:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787797 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=HXWcnb8T; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Yn6wYQz20Pc for ; Wed, 31 May 2023 04:00:45 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43dl-0008Bm-Dx; Tue, 30 May 2023 14:00:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dj-00089Z-IL for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:11 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dh-0002Mv-OX for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:11 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4jsG021696; Tue, 30 May 2023 18:00:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=CXnIDx3G2VitmhWITlFxhYYmGCk6JD4I7r2Gk/ajJzo=; b=HXWcnb8TUJ5bjXkw66qrtHskjEvPtPCGvRh8VHJ52xvFibvwVsgWgo0hRZwEpVstyJCO 14wBXWekaetL42Cg2Bp3fZ7iYAaZtyrfDJEr1SZTpO1hhOHixI0kfBGB/rfiqoe52+/S x8xBRIIgWuJwiyDEaLXX8zPmQMWu343wCLuHKrRhVbf13+lwF0XJ/jZsxkIitA64I4yq 0lmiGKGH/HKXaJu8NXP6C5RctXVDG9eZQCSArFydRiHfpYB3aPLvFpmZZXCpBJE9WLCW Ngzleqa4uc6HMARTrtj0nFstWJgpRcVKLVSrdQ7vSZiUTA5bhZbwAohwN/PRBe2M5j+2 4w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjh3ehj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:04 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHcPx0014630; Tue, 30 May 2023 18:00:03 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48t5c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:03 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspF011459; Tue, 30 May 2023 18:00:03 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-3; Tue, 30 May 2023 18:00:02 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 02/15] hw/pci: Add a pci_setup_iommu_info() helper Date: Tue, 30 May 2023 18:59:24 +0100 Message-Id: <20230530175937.24202-3-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=940 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: 7Wsz83WunnVOsdentylhcdvxMJ4fdrSU X-Proofpoint-ORIG-GUID: 7Wsz83WunnVOsdentylhcdvxMJ4fdrSU Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Provide a second PCI iommu bus initialization function which returns PCIAddressSpace rather than an AddressSpace. The function is meant to superseed pci_setup_iommu(). Under the hood in pci_device_iommu_info() if the new function pointer is set in the device bus, use that instead and return the new object. This is preparation for vIOMMU MR to be stored in PCIAddressSpace, thus made available regardless of guest behaviour. Signed-off-by: Joao Martins --- hw/pci/pci.c | 14 ++++++++++++-- include/hw/pci/pci.h | 2 ++ include/hw/pci/pci_bus.h | 1 + 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index ecf8a543aa77..ac10333b5097 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2681,8 +2681,12 @@ PCIAddressSpace pci_device_iommu_info(PCIDevice *dev) } as = &address_space_memory; - if (!pci_bus_bypass_iommu(bus) && iommu_bus && iommu_bus->iommu_fn) { - as = iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); + if (!pci_bus_bypass_iommu(bus) && iommu_bus) { + if (iommu_bus->iommu_fn) { + as = iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); + } else if (iommu_bus->iommu_as_fn) { + return iommu_bus->iommu_as_fn(bus, iommu_bus->iommu_opaque, devfn); + } } return as_to_pci_as(as); } @@ -2693,6 +2697,12 @@ void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) bus->iommu_opaque = opaque; } +void pci_setup_iommu_info(PCIBus *bus, PCIIOMMUASFunc fn, void *opaque) +{ + bus->iommu_as_fn = fn; + bus->iommu_opaque = opaque; +} + static void pci_dev_get_w64(PCIBus *b, PCIDevice *dev, void *opaque) { Range *range = opaque; diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index 9ffaf47fe2ab..d2c87d87a24e 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -368,6 +368,7 @@ typedef struct PCIAddressSpace { } PCIAddressSpace; typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); +typedef PCIAddressSpace (*PCIIOMMUASFunc)(PCIBus *, void *, int); static inline PCIAddressSpace as_to_pci_as(AddressSpace *as) { PCIAddressSpace ret = { .as = as }; @@ -386,6 +387,7 @@ static inline AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) } void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); +void pci_setup_iommu_info(PCIBus *bus, PCIIOMMUASFunc fn, void *opaque); pcibus_t pci_bar_address(PCIDevice *d, int reg, uint8_t type, pcibus_t size); diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h index 56531759578f..a2795b23a3b0 100644 --- a/include/hw/pci/pci_bus.h +++ b/include/hw/pci/pci_bus.h @@ -34,6 +34,7 @@ struct PCIBus { BusState qbus; enum PCIBusFlags flags; PCIIOMMUFunc iommu_fn; + PCIIOMMUASFunc iommu_as_fn; void *iommu_opaque; uint8_t devfn_min; uint32_t slot_reserved_mask; From patchwork Tue May 30 17:59:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787810 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=vU7IFxS6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0fq1BpVz20Q4 for ; Wed, 31 May 2023 04:05:07 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43dp-0008DQ-Cg; Tue, 30 May 2023 14:00:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dm-0008Cb-MH for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:15 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dk-0002Np-Ti for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:14 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4uZU016936; Tue, 30 May 2023 18:00:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=o7vdUV+u0swzyTnaTw7LEKypTHbSrpeAeS+oiO09Tbk=; b=vU7IFxS6X921/0jjfZlnMRx0+kAf+2tzSXwEQUsige8UFPBCk5kd7WaF+ur20uFRIwK+ 8YmY8nAEAoEVnDrVchGKR1jnBmpqnz0SoOmnW7yXXSzx+DK5fllu+T4vleRvpUhB17K8 qT83kpoEgEFdN6aZekkvaLpBAtrS7BVD10iiN3Mitt5fB2t6nQUgWCMNl74AJnT5hLJf jIrrfA4xi2l9/jngR216cS6NmDs2zFN9NJbggDZiIvvNTWdYRnK0/Tmyt9ui2Ff1e3jY 1AvXXuui7pXrFLcbEaEMt72av14ohAUJLSeVG6hRCqGXoVZ8QEkKN6UIM9ZxBSDtycMz 3w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjkkety-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:08 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHhRLC014722; Tue, 30 May 2023 18:00:08 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48ta2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:08 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspH011459; Tue, 30 May 2023 18:00:07 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-4; Tue, 30 May 2023 18:00:07 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 03/15] hw/pci: Add a pci_device_iommu_memory_region() helper Date: Tue, 30 May 2023 18:59:25 +0100 Message-Id: <20230530175937.24202-4-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=805 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: WG0YZoTsWjfPjPbE2EPEWZSizjDXu93s X-Proofpoint-GUID: WG0YZoTsWjfPjPbE2EPEWZSizjDXu93s Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Today's VFIO model of understanding if an IOMMU is behind the PCIDevice is to check whether the address space backing the device is memory or not, which is done via pci_device_iommu_address_space(pdev). On the other hand, the IOMMU MR is used for example to fetch attributes, or when doing an vIOMMU map and figuring out if it's under the vIOMMU. However, such object is only available today by the time the IOMMU map notifier is called which depends on the guest doing a DMA map or not. Thus there's no way to get access to the IOMMU memory region early i.e. at vfio device initialization where we attest migration support and impose LM blockers or not. Much like pci_device_iommu_address_space() fetches the IOMMU AS, add a pci_device_iommu_memory_region() which lets it return an the IOMMU MR associated with it. The IOMMU MR is returned correctly for vIOMMUs using pci_setup_iommu_info(). Note that today most vIOMMUs create the address space and IOMMU MR at the same time, it's just mainly that there's API to make the latter available. This is in preparation for VFIO to track both the AS and IOMMU MR into VFIOSpace without being tied to a map taking place by the guest. The IOMMU MR will then provide access to upper layers about various IOMMU attributes. Signed-off-by: Joao Martins --- include/hw/pci/pci.h | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index d2c87d87a24e..0177f50e96a3 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -365,13 +365,14 @@ void pci_device_deassert_intx(PCIDevice *dev); typedef struct PCIAddressSpace { AddressSpace *as; + IOMMUMemoryRegion *iommu_mr; } PCIAddressSpace; typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); typedef PCIAddressSpace (*PCIIOMMUASFunc)(PCIBus *, void *, int); static inline PCIAddressSpace as_to_pci_as(AddressSpace *as) { - PCIAddressSpace ret = { .as = as }; + PCIAddressSpace ret = { .as = as, .iommu_mr = NULL }; return ret; } @@ -379,6 +380,13 @@ static inline AddressSpace *pci_as_to_as(PCIAddressSpace pci_as) { return pci_as.as; } +static inline PCIAddressSpace to_pci_as(AddressSpace *as, + IOMMUMemoryRegion *iommu_mr) +{ + PCIAddressSpace ret = { .as = as, .iommu_mr = iommu_mr }; + + return ret; +} PCIAddressSpace pci_device_iommu_info(PCIDevice *dev); static inline AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) @@ -386,6 +394,13 @@ static inline AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) return pci_as_to_as(pci_device_iommu_info(dev)); } +static inline IOMMUMemoryRegion *pci_device_iommu_memory_region(PCIDevice *dev) +{ + PCIAddressSpace ret = pci_device_iommu_info(dev); + + return ret.iommu_mr; +} + void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); void pci_setup_iommu_info(PCIBus *bus, PCIIOMMUASFunc fn, void *opaque); From patchwork Tue May 30 17:59:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787795 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=1JyV13rI; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Yh36Q7z20Q5 for ; Wed, 31 May 2023 04:00:40 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43dv-0008Hj-4R; Tue, 30 May 2023 14:00:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dr-0008Fc-P6 for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:20 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43do-0002PB-Vl for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:19 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4umR016924; Tue, 30 May 2023 18:00:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=bNWd+8qg+HQMGADbnG4d7mUSP5zBAyGayHx2RMjErd0=; b=1JyV13rIqfN9b5y6yGfVSQP9UlWFJsylzEOXNOBF5KDeCNM+QvUvAdGL6jad2bggdohR vCE55p0sTSmnLAW7nxWU0JOcjLCArBmhn/MusJYFJ61Pc4yuYczsQR9nHpWjdFDyglUJ FpK0BZFlg8C4h3pKISs2h3iTmnUGHWH2GZyEJ8i88Pvlzi/b1QrSfP0m41W01k8hrJpu 6ROJyMMXYtMpQSU1K6XywAK3jVx5BMku/jwe0Yc8CLy2q0IgoptToxCWYSlzE0/x81X8 jIzAijYOa6Tm9978I1nlP7sEhm2io9qt/NxJ13AJQFEi/MvSBEKOL+p5F3OTOSz486HC Kg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjkkeu1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:13 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHfg4d014618; Tue, 30 May 2023 18:00:12 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48teu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:12 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspJ011459; Tue, 30 May 2023 18:00:11 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-5; Tue, 30 May 2023 18:00:11 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 04/15] intel-iommu: Switch to pci_setup_iommu_info() Date: Tue, 30 May 2023 18:59:26 +0100 Message-Id: <20230530175937.24202-5-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: Us0O5CmZc99Blx5peVHeh2uvs1EuJW8g X-Proofpoint-GUID: Us0O5CmZc99Blx5peVHeh2uvs1EuJW8g Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Use the PCI IOMMU setup function that allows returning the IOMMUMemoryRegion in addition to the AddressSpace. To faciliate conversion use the new to_pci_as()/pci_as_to_as() helpers. Signed-off-by: Joao Martins --- hw/i386/intel_iommu.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 94d52f4205d2..c344d49fe290 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -4021,7 +4021,7 @@ static void vtd_reset(DeviceState *dev) vtd_address_space_refresh_all(s); } -static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) +static PCIAddressSpace vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) { IntelIOMMUState *s = opaque; VTDAddressSpace *vtd_as; @@ -4029,7 +4029,7 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) assert(0 <= devfn && devfn < PCI_DEVFN_MAX); vtd_as = vtd_find_add_as(s, bus, devfn, PCI_NO_PASID); - return &vtd_as->as; + return to_pci_as(&vtd_as->as, &vtd_as->iommu); } static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) @@ -4155,9 +4155,10 @@ static void vtd_realize(DeviceState *dev, Error **errp) g_free, g_free); vtd_init(s); sysbus_mmio_map(SYS_BUS_DEVICE(s), 0, Q35_HOST_BRIDGE_IOMMU_ADDR); - pci_setup_iommu(bus, vtd_host_dma_iommu, dev); + pci_setup_iommu_info(bus, vtd_host_dma_iommu, dev); /* Pseudo address space under root PCI bus. */ - x86ms->ioapic_as = vtd_host_dma_iommu(bus, s, Q35_PSEUDO_DEVFN_IOAPIC); + x86ms->ioapic_as = pci_as_to_as(vtd_host_dma_iommu(bus, + s, Q35_PSEUDO_DEVFN_IOAPIC)); qemu_add_machine_init_done_notifier(&vtd_machine_done_notify); } From patchwork Tue May 30 17:59:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787802 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=B2fTCUwH; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0ZN23NJz20Q5 for ; Wed, 31 May 2023 04:01:16 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43dy-0008Rv-EW; Tue, 30 May 2023 14:00:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dv-0008JH-Ce for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:23 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dt-0002R4-Bn for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:22 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4i2V021684; Tue, 30 May 2023 18:00:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=ahvtNwQMdZfZKC1RYnrq2f+vHhBjZIejmeLgkq1wK6A=; b=B2fTCUwHiGWd32xMwKEDd5Mh0bG3fHkK8bzwhXWDv0VB86mGwsG6+cB2cP4boaooVScd R3nDF7P6+caScFzIEnM6VmF0ULR1xtICI33B5Mf5HjssOwqqraDs0xnYNj6suB6axiE5 PznSSrpf0QkeVg2zadCaMp+SqJxIz4U3Ry0nx3NTq9TcBefRBqE8dhv57N/NWSo7X2TU MkmxycABtJrtujnWFPM9A86p9ygMQZ8r+D3Lz3QZrjeyEr8OnlUOArac/bSwzVCDMUkL W3xmSZbibhaJB0+4YlU4jKpz3sDElT8Ez8fcAkl7t/r2qFtGCpIx9C0IGOJfiSpkEk7T Gw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjh3eht-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:17 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UH0NUb014665; Tue, 30 May 2023 18:00:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48tm0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:16 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspL011459; Tue, 30 May 2023 18:00:16 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-6; Tue, 30 May 2023 18:00:15 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 05/15] vfio/common: Track the IOMMU MR behind the device in addition to the AS Date: Tue, 30 May 2023 18:59:27 +0100 Message-Id: <20230530175937.24202-6-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=711 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: jeH73bkJz02jIbUYlnTfzdRP7TlNk9V7 X-Proofpoint-ORIG-GUID: jeH73bkJz02jIbUYlnTfzdRP7TlNk9V7 Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org vfio_get_group() allocates and fills the group/container/space on success which will store the AddressSpace inside the VFIOSpace struct. Use the newly added pci_device_iommu_memory_region() and store the IOMMU MR too. Today, this means only intel-iommu sets it. Signed-off-by: Joao Martins --- hw/vfio/pci.c | 4 ++++ include/hw/vfio/vfio-common.h | 1 + 2 files changed, 5 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 73874a94de12..9dc9a7d834ec 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2900,6 +2900,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) VFIOPCIDevice *vdev = VFIO_PCI(pdev); VFIODevice *vbasedev = &vdev->vbasedev; VFIODevice *vbasedev_iter; + VFIOAddressSpace *space; VFIOGroup *group; char *tmp, *subsys, group_path[PATH_MAX], *group_name; Error *err = NULL; @@ -2961,6 +2962,9 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) goto error; } + space = group->container->space; + space->iommu_mr = pci_device_iommu_memory_region(pdev); + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { if (strcmp(vbasedev_iter->name, vbasedev->name) == 0) { error_setg(errp, "device is already attached"); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index eed244f25f34..92e6514fd757 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -70,6 +70,7 @@ typedef struct VFIOMigration { typedef struct VFIOAddressSpace { AddressSpace *as; + IOMMUMemoryRegion *iommu_mr; QLIST_HEAD(, VFIOContainer) containers; QLIST_ENTRY(VFIOAddressSpace) list; } VFIOAddressSpace; From patchwork Tue May 30 17:59:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787799 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=B1GFiDDM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Z45zl3z20Q5 for ; Wed, 31 May 2023 04:01:00 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43e4-0000XC-2w; Tue, 30 May 2023 14:00:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43e1-0000NI-2g for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:29 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43dy-0002UN-0q for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:28 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4wGx017005; Tue, 30 May 2023 18:00:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=KQFYtR0JFMAeqZa5xmyrG53ZgATBRYJx00uHF/BU6wY=; b=B1GFiDDMIO1Grdhc+70nRAMvwj6KMXidfOsJcRqjuLWM/dXATowLseTOwjBbvu4EDy6b ZmtittFCJ8tT8r61Ojo2aAq/3dwTnT6IghwMmpZQ2U1CFVrnaNqmRmJTKz/v9dxnoAti Qjz66BDiDXWrnb60ZPQ2CrYgwrIKVk3fxNrBe1VFk3rOl0vmGzmAa493lmfZQic1VaTQ nrVLJyOttycB6tNh+jzj8sFyD8o81VwlNJ7rKJudTMFj/HRdVIiOvFHqS50K8E5sFI3a zE8UrzghoEZ13euunKYNB/lXIPnM7bqlG8SGLrlQ1ISMdXbTDrARhk3hujQ3oK9BpfGa TA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjkkeu7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:21 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UGGEZT014607; Tue, 30 May 2023 18:00:21 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48tsr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:21 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspN011459; Tue, 30 May 2023 18:00:20 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-7; Tue, 30 May 2023 18:00:20 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 06/15] memory/iommu: Add IOMMU_ATTR_DMA_TRANSLATION attribute Date: Tue, 30 May 2023 18:59:28 +0100 Message-Id: <20230530175937.24202-7-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=953 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: dyleHYivGTg1dG0dQoR3MLr2mXZX801X X-Proofpoint-GUID: dyleHYivGTg1dG0dQoR3MLr2mXZX801X Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a new IOMMU attribute IOMMU_ATTR_DMA_TRANSLATION which indicates whether the IOMMU supports DMA Translation. This attribute will be used by VFIO device dirty page tracking such that it can restrict the IOVA under tracking to the memory map when vIOMMU is configured in IR-only mode without dma translation supported/advertised. Signed-off-by: Joao Martins --- include/exec/memory.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index c3661b2276c7..a02496f34180 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -316,7 +316,8 @@ typedef struct MemoryRegionClass { enum IOMMUMemoryRegionAttr { - IOMMU_ATTR_SPAPR_TCE_FD + IOMMU_ATTR_SPAPR_TCE_FD, + IOMMU_ATTR_DMA_TRANSLATION, }; /* From patchwork Tue May 30 17:59:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787801 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=GJ055BHp; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0ZM5bGnz20Pc for ; Wed, 31 May 2023 04:01:15 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43e8-0000mw-4I; Tue, 30 May 2023 14:00:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43e6-0000iD-As for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:34 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43e3-0002Xd-LE for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:34 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4soo008463; Tue, 30 May 2023 18:00:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=aIMBMLsFEbFOA4jOhB634CmGHntU/r9jnKkv1QWnnFk=; b=GJ055BHpRKdeXP6UFB4AAxvmWq7dyHmX20yos9eRmTl1+5xpAyJWk/Fgmg60H5SAZD78 tGg3UAYDywbJprF55oDqieZiQcSqSZcjF4bdpnkOu667/7M6UF1Q2a7U66hSYFpislp/ x5frwN/BhXm4Hr9gJ/mwpjyafeY0QT1EMluxdpGqSxUrdxJhSsyjoZcZm5X8XGq3dbas pX+mxiwABDyLag9bi89E/clu6IGBwHK9arBOquK/NHGpDC9PPNf8ye+eEXrTErTNFOov ZEE2A6YN6/A02iw+LGnJ9WZoYcSHsC+DNoTyalEHGxrx9xvJyl4i6LFn4zvgwmXvkD9f kQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhmjker0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:25 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHHZna014645; Tue, 30 May 2023 18:00:25 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48ty3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:25 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspP011459; Tue, 30 May 2023 18:00:24 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-8; Tue, 30 May 2023 18:00:24 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 07/15] intel-iommu: Implement get_attr() method Date: Tue, 30 May 2023 18:59:29 +0100 Message-Id: <20230530175937.24202-8-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: F5CB5OTTby07sR8w8sJi6cMh-3NXYNSv X-Proofpoint-GUID: F5CB5OTTby07sR8w8sJi6cMh-3NXYNSv Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Implement get_attr() method and use the IntelIOMMUState::dma_translation field to report the IOMMU_ATTR_DMA_TRANSLATION attribute. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/i386/intel_iommu.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index c344d49fe290..1906f3a67960 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3861,6 +3861,29 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) return; } +static int vtd_iommu_get_attr(IOMMUMemoryRegion *iommu_mr, + enum IOMMUMemoryRegionAttr attr, void *data) +{ + VTDAddressSpace *vtd_as = container_of(iommu_mr, VTDAddressSpace, iommu); + IntelIOMMUState *s = vtd_as->iommu_state; + int ret = 0; + + switch (attr) { + case IOMMU_ATTR_DMA_TRANSLATION: + { + bool *enabled = data; + + *enabled = s->dma_translation; + break; + } + default: + ret = -EINVAL; + break; + } + + return ret; +} + /* Do the initialization. It will also be called when reset, so pay * attention when adding new initialization stuff. */ @@ -4194,6 +4217,7 @@ static void vtd_iommu_memory_region_class_init(ObjectClass *klass, imrc->translate = vtd_iommu_translate; imrc->notify_flag_changed = vtd_iommu_notify_flag_changed; imrc->replay = vtd_iommu_replay; + imrc->get_attr = vtd_iommu_get_attr; } static const TypeInfo vtd_iommu_memory_region_info = { From patchwork Tue May 30 17:59:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787809 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=xC+r6DQM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0d96g8Cz20Pc for ; Wed, 31 May 2023 04:03:41 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43e9-0000rz-4l; Tue, 30 May 2023 14:00:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43e7-0000lX-Hf for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:35 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43e5-0002YJ-Q3 for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:35 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE54a0020898; Tue, 30 May 2023 18:00:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=oeF/xgJgz8YKtGSx6YH+wUQ+893UdJNG0ePybKsC06Q=; b=xC+r6DQM7jgRj6klgdp84tREkEODwU7h7Dc7ACvo9snSafAmaSlJA13hhGKKZHsblw5y kULnEX10/zDYzolr4LwI4z8vxJltSkrJ7G0FDGRimD+9QzVLIWGY+ywR4drKlmS4JRHa iAe5lKNjQLKuK79mysOIyGMep4H+QyWGxHD819PgGpEwxcb+xCprxnzG50isokydg8kG 3H9AjuC1w/03yuAk/79EWyxL0eNzmijek6rbG9PYL7zYCVstS1m2vMFB4hAH+zGBff6D Buci2ETG+qgYDK7NHyKBVBhrbE7h6tGfUaqlayMXkBwKhSfTs/nNtIcV9MKFI9T7EtaA fA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhb93ge4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:29 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHTTA1014592; Tue, 30 May 2023 18:00:29 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48u4d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:29 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspR011459; Tue, 30 May 2023 18:00:28 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-9; Tue, 30 May 2023 18:00:28 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 08/15] vfio/common: Relax vIOMMU detection when DMA translation is off Date: Tue, 30 May 2023 18:59:30 +0100 Message-Id: <20230530175937.24202-9-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: 0LmWS7nQmjh2Qd6Zn1vfwm_gsDbnNjb1 X-Proofpoint-ORIG-GUID: 0LmWS7nQmjh2Qd6Zn1vfwm_gsDbnNjb1 Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Relax the vIOMMU migration blocker when the underlying IOMMU reports that DMA translation disabled. When it is disabled there will be no DMA mappings via the vIOMMU and the guest only uses it for Interrupt Remapping. The latter is done via the vfio_viommu_preset() return value whereby in addition to validating that the address space is memory, we also check whether the IOMMU MR has DMA translation on. Assume it is enabled if there's no IOMMU MR or if no dma-translation property is supported. Signed-off-by: Joao Martins --- hw/vfio/common.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index fa8fd949b1cf..060acccb3443 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -414,12 +414,32 @@ void vfio_unblock_multiple_devices_migration(void) multiple_devices_migration_blocker = NULL; } +static bool vfio_viommu_dma_translation_enabled(VFIOAddressSpace *space) +{ + bool enabled = false; + int ret; + + if (!space->iommu_mr) { + return true; + } + + ret = memory_region_iommu_get_attr(space->iommu_mr, + IOMMU_ATTR_DMA_TRANSLATION, + &enabled); + if (ret || enabled) { + return true; + } + + return false; +} + static bool vfio_viommu_preset(void) { VFIOAddressSpace *space; QLIST_FOREACH(space, &vfio_address_spaces, list) { - if (space->as != &address_space_memory) { + if ((space->as != &address_space_memory) && + vfio_viommu_dma_translation_enabled(space)) { return true; } } From patchwork Tue May 30 17:59:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787798 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=bztkFj63; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Z45bw1z20Pc for ; Wed, 31 May 2023 04:01:00 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43eG-0001S9-Ss; Tue, 30 May 2023 14:00:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eE-0001IW-LY for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:42 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eD-0002a5-1M for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:42 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4iHO021679; Tue, 30 May 2023 18:00:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Lr/C3gX8ZIXheuATdDxod9Punf7oG6IsG/pJunOuypY=; b=bztkFj63J2BqDBFzIqABVDVDlRKxrlHd8Z4P/ByfqCrH832byO6455esLj/oaHmxUihv vsZzhQaVTIeBHfruCZ8vIkxMOg1tQLN++xWQZihwKiky/2kpWx4+RoQrLS31hMkrs3NI RqLfOgszndMqwH4KaianNtkDCt8dVlmaII1sJqz5j8oaiAhreBe8COK2k9jdabnHla+t wBzNabmbKthwslAHORncE1STBEdAHc/x+ocNQ+s6miR6/b8r/IfMxv//VTk0kYeJSa6t Le2fR8o34yv/tBzj4IiF7fNQIWkLPnDG9gKOjNYOUWNmDkdXy9p20IUq8IagOU89gB79 9Q== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjh3ejq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:36 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHZLZe014673; Tue, 30 May 2023 18:00:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48udw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:35 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspT011459; Tue, 30 May 2023 18:00:35 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-10; Tue, 30 May 2023 18:00:32 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 09/15] memory/iommu: Add IOMMU_ATTR_MAX_IOVA attribute Date: Tue, 30 May 2023 18:59:31 +0100 Message-Id: <20230530175937.24202-10-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=995 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: KnvnpNNqU2stl4GCI5QwXBYEPRrufkX2 X-Proofpoint-ORIG-GUID: KnvnpNNqU2stl4GCI5QwXBYEPRrufkX2 Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Add a new IOMMU attribute IOMMU_ATTR_MAX_IOVA which indicates the maximal IOVA that an IOMMU can use. This attribute will be used by VFIO device dirty page tracking so it can track the entire IOVA space when needed (i.e. when vIOMMU is enabled). Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Acked-by: Peter Xu --- include/exec/memory.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index a02496f34180..b09b1c9846fe 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -318,6 +318,7 @@ typedef struct MemoryRegionClass { enum IOMMUMemoryRegionAttr { IOMMU_ATTR_SPAPR_TCE_FD, IOMMU_ATTR_DMA_TRANSLATION, + IOMMU_ATTR_MAX_IOVA, }; /* From patchwork Tue May 30 17:59:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787803 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=u0TsVeNm; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Zt2znLz20Pc for ; Wed, 31 May 2023 04:01:42 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43eO-0001ZK-UV; Tue, 30 May 2023 14:00:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eN-0001Yv-EF for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:51 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eL-0002fA-SW for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:51 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4sBA008456; Tue, 30 May 2023 18:00:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=38V6sqKnz0J4b6q5r6+HMr9dMFtBKs/EXConcqapz8k=; b=u0TsVeNmzK+71JgDgB0m1vgDK1i5aS+2vFNdDmyaadYQxnbyKsSDk87uh8don4yLD7GG TPgC8pT+5v3+Mztqq7OXlUVTyACTGsK2Ok+wozhH1BYQ/LXrZZddIn9a5LRk030fBNpq T16NV/oSikblgvTr/pFZpEhZ2Pe05wjCDaVy9m8xc47ABnY30PC7l/IMiZgPBrP4ROkh uTDUGNezgahQyzyn+WxMMaKQC64jTnBuSE+KUyRuthRS0Q8HyPwQB36msIBnbflncMJ/ k/26FPyCOFhBfWQrbgGbo7O49qaE4dkjARRN1WNtafDCooKiWLJOxfCgwl2KsUyJ99NB RQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhmjkesg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:43 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHhTVm014716; Tue, 30 May 2023 18:00:42 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48uqn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:42 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspV011459; Tue, 30 May 2023 18:00:42 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-11; Tue, 30 May 2023 18:00:39 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 10/15] intel-iommu: Implement IOMMU_ATTR_MAX_IOVA get_attr() attribute Date: Tue, 30 May 2023 18:59:32 +0100 Message-Id: <20230530175937.24202-11-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: jfbONwdVxv8vgdA2euEPS68h9O6uefiW X-Proofpoint-GUID: jfbONwdVxv8vgdA2euEPS68h9O6uefiW Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Implement get_attr() method and use the address width property to report the IOMMU_ATTR_MAX_IOVA attribute. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/i386/intel_iommu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 1906f3a67960..829dd6eadc6c 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3876,6 +3876,13 @@ static int vtd_iommu_get_attr(IOMMUMemoryRegion *iommu_mr, *enabled = s->dma_translation; break; } + case IOMMU_ATTR_MAX_IOVA: + { + hwaddr *max_iova = data; + + *max_iova = (1ULL << s->aw_bits) - 1; + break; + } default: ret = -EINVAL; break; From patchwork Tue May 30 17:59:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787800 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=i5VpUtr9; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0Z85QQnz20Pc for ; Wed, 31 May 2023 04:01:04 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43eR-0001dA-FA; Tue, 30 May 2023 14:00:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eP-0001ZX-7I for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:53 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eN-0002hm-LW for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:52 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4tkr008512; Tue, 30 May 2023 18:00:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Ec2yjzcuWoT8Qe2E87T1LVSwwhoAWx9rdoJA8pHb+Ok=; b=i5VpUtr9riioSi3Iuwen9Es82WopMNT15RWZJmR9jBsZLp7qjxb/5ET/pwOT121HX0pR XxNBzVfbg9fiqklmEeKWspqXyoFOmH5cElmEQyloNgTSLc0Tj+CCx72nUglt19wRF2sE GUwl1m0/gL4wGCxcBq394oeyB/ulW0FpaXcCCWJ7Agvac6gIfx5c00Kn1l8LPjMeL4u1 b8DP0b0muSsoH2ERgqNw0Y9QpUePEprIjBqfCAUk0nm5V4djl6PBCM+la6fhrTyVWWsK S+drdlqkRlDaU8zpK7ng8iAVCteQ8q6rx02MYQbrG8+dYgjWTiUfy4NAEF1WOAX8orKv 4w== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhmjkesw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:47 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHmbFm014623; Tue, 30 May 2023 18:00:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48uwv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:46 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspX011459; Tue, 30 May 2023 18:00:46 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-12; Tue, 30 May 2023 18:00:45 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 11/15] vfio/common: Move dirty tracking ranges update to helper Date: Tue, 30 May 2023 18:59:33 +0100 Message-Id: <20230530175937.24202-12-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: akBRhUbZ4ZK7OFE2gJj6uasiIVzHqHsH X-Proofpoint-GUID: akBRhUbZ4ZK7OFE2gJj6uasiIVzHqHsH Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Separate the code that update the ranges from the listener, in preparation to expand its use to vIOMMU support. Signed-off-by: Joao Martins --- hw/vfio/common.c | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 060acccb3443..b8d97577f856 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1432,20 +1432,10 @@ typedef struct VFIODirtyRangesListener { MemoryListener listener; } VFIODirtyRangesListener; -static void vfio_dirty_tracking_update(MemoryListener *listener, - MemoryRegionSection *section) +static void vfio_dirty_tracking_update(hwaddr iova, hwaddr end, + VFIODirtyRanges *range) { - VFIODirtyRangesListener *dirty = container_of(listener, - VFIODirtyRangesListener, - listener); - VFIODirtyRanges *range = &dirty->ranges; - hwaddr iova, end, *min, *max; - - if (!vfio_listener_valid_section(section, "tracking_update") || - !vfio_get_section_iova_range(dirty->container, section, - &iova, &end, NULL)) { - return; - } + hwaddr *min, *max; /* * The address space passed to the dirty tracker is reduced to two ranges: @@ -1469,12 +1459,28 @@ static void vfio_dirty_tracking_update(MemoryListener *listener, } trace_vfio_device_dirty_tracking_update(iova, end, *min, *max); - return; +} + +static void vfio_listener_dirty_tracking_update(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIODirtyRangesListener *dirty = container_of(listener, + VFIODirtyRangesListener, + listener); + hwaddr iova, end; + + if (!vfio_listener_valid_section(section, "tracking_update") || + !vfio_get_section_iova_range(dirty->container, section, + &iova, &end, NULL)) { + return; + } + + vfio_dirty_tracking_update(iova, end, &dirty->ranges); } static const MemoryListener vfio_dirty_tracking_listener = { .name = "vfio-tracking", - .region_add = vfio_dirty_tracking_update, + .region_add = vfio_listener_dirty_tracking_update, }; static void vfio_dirty_tracking_init(VFIOContainer *container, From patchwork Tue May 30 17:59:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787805 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=YzOkA8/I; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0bS5mJJz20Pc for ; Wed, 31 May 2023 04:02:12 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43eW-00026e-8y; Tue, 30 May 2023 14:01:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eU-0001vs-JG for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:58 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eS-0002jh-4K for qemu-devel@nongnu.org; Tue, 30 May 2023 14:00:58 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE54jf020902; Tue, 30 May 2023 18:00:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=hHvviXGKXz7/JvAkgQz3l7QwnqzLJhzckRGwXC/3Pmg=; b=YzOkA8/I0hn/bNMv1kb/UMi7g1Jf/+Q/FsQEgqXgGI2qupz9AKeg3WyApkg3dSrv6O/x 9NrWPhnRAEQb65LovxLXYm+Ie6xGnm4Yb/44urH13+x6mSJ1rNj0fCp65av5laWTcIX+ 5W83vpepuPB4J3/HnP9LygoGJVCq0yVUqoIalCDHWsTWym84ckAmClFCbpUt1KjetPjX 8NzPYzoM2dQ2DmlZr7LirvjJ2QSwS3jdqP3tz8PYPs4pit/JklXuxJmyBdup4ZAWVwYX VcvsmcIZIYrC+WEA6eXFGc4aqT1E4MXc85ln3UPEd1sDd4u6B0MZsNBS9wqu4IBiG5rL dw== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhb93gfv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:51 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHfg5K014618; Tue, 30 May 2023 18:00:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48v34-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:50 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspZ011459; Tue, 30 May 2023 18:00:50 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-13; Tue, 30 May 2023 18:00:49 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 12/15] vfio/common: Support device dirty page tracking with vIOMMU Date: Tue, 30 May 2023 18:59:34 +0100 Message-Id: <20230530175937.24202-13-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: gvG2qt7Ghnse6XfhqvrkWZ4taSyibWv2 X-Proofpoint-ORIG-GUID: gvG2qt7Ghnse6XfhqvrkWZ4taSyibWv2 Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently, device dirty page tracking with vIOMMU is not supported, and a blocker is added and the migration is prevented. When vIOMMU is used, IOVA ranges are DMA mapped/unmapped on the fly as requesting by the vIOMMU. These IOVA ranges can potentially be mapped anywhere in the vIOMMU IOVA space as advertised by the VMM. To support device dirty tracking when vIOMMU enabled instead create the dirty ranges based on the vIOMMU provided limits, which leads to the tracking of the whole IOVA space regardless of what devices use. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 64 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 54 insertions(+), 10 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index b8d97577f856..d2897aceedae 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -447,6 +447,31 @@ static bool vfio_viommu_preset(void) return false; } +static int vfio_viommu_get_max_iova(hwaddr *max_iova) +{ + VFIOAddressSpace *space; + int ret = -EINVAL; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + if (space->as == &address_space_memory) { + continue; + } + + if (!space->iommu_mr) { + break; + } + + ret = memory_region_iommu_get_attr(space->iommu_mr, + IOMMU_ATTR_MAX_IOVA, + max_iova); + if (ret) { + break; + } + } + + return ret; +} + int vfio_block_giommu_migration(Error **errp) { int ret; @@ -1483,10 +1508,11 @@ static const MemoryListener vfio_dirty_tracking_listener = { .region_add = vfio_listener_dirty_tracking_update, }; -static void vfio_dirty_tracking_init(VFIOContainer *container, +static int vfio_dirty_tracking_init(VFIOContainer *container, VFIODirtyRanges *ranges) { VFIODirtyRangesListener dirty; + int ret; memset(&dirty, 0, sizeof(dirty)); dirty.ranges.min32 = UINT32_MAX; @@ -1494,17 +1520,29 @@ static void vfio_dirty_tracking_init(VFIOContainer *container, dirty.listener = vfio_dirty_tracking_listener; dirty.container = container; - memory_listener_register(&dirty.listener, - container->space->as); + if (vfio_viommu_preset()) { + hwaddr iommu_max_iova; + + ret = vfio_viommu_get_max_iova(&iommu_max_iova); + if (ret) { + return ret; + } + + vfio_dirty_tracking_update(0, iommu_max_iova, &dirty.ranges); + } else { + memory_listener_register(&dirty.listener, + container->space->as); + /* + * The memory listener is synchronous, and used to calculate the range + * to dirty tracking. Unregister it after we are done as we are not + * interested in any follow-up updates. + */ + memory_listener_unregister(&dirty.listener); + } *ranges = dirty.ranges; - /* - * The memory listener is synchronous, and used to calculate the range - * to dirty tracking. Unregister it after we are done as we are not - * interested in any follow-up updates. - */ - memory_listener_unregister(&dirty.listener); + return 0; } static void vfio_devices_dma_logging_stop(VFIOContainer *container) @@ -1609,7 +1647,13 @@ static int vfio_devices_dma_logging_start(VFIOContainer *container) VFIOGroup *group; int ret = 0; - vfio_dirty_tracking_init(container, &ranges); + ret = vfio_dirty_tracking_init(container, &ranges); + if (ret) { + error_report("Failed to init DMA logging ranges, err %d", + ret); + return -EOPNOTSUPP; + } + feature = vfio_device_feature_dma_logging_start_create(container, &ranges); if (!feature) { From patchwork Tue May 30 17:59:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787806 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=RBdm4PAX; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0bX3xPtz20Pc for ; Wed, 31 May 2023 04:02:16 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43es-0002mz-Dc; Tue, 30 May 2023 14:01:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eb-0002Un-0m for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:08 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43eZ-0002lF-5R for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:04 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4tJD008515; Tue, 30 May 2023 18:00:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=QCKWQjmfZCLCZPMNKQB8uFMHMlrcY9ep0tlM1sAePfw=; b=RBdm4PAXIbv27YoKJ2u0901tjqAsaLPer/flq+IAmWX1/ZhFiGzml1LFMjmoi6Ck5eWS hCyLicC9bJR57hhZ/ozNNk+bxr0XzlYt5L8T+dw2COqir7CEXzvXEmet0caXf+i6u4xA HBSsikNuiDD08xmngnsv5SBBqiYNf768HVLRlDyGAsfNWlXRWj9OGF323vtNUfMLTbSw N8nZyBXWcc/nNAZ4bOo67ENs/n/7mAgwMqmj77mSQABh+Y2ebg8mNQqNhiCkIwE5weR0 slviAToNmbWIKcdKCFrWiKd6cU6QiPW2dseeF3wp+TXWWciaud4WHFNR66xwOXEeBhz8 QQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhmjkett-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:58 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UH0NVL014665; Tue, 30 May 2023 18:00:57 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48vbk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:00:57 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspb011459; Tue, 30 May 2023 18:00:57 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-14; Tue, 30 May 2023 18:00:54 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 13/15] vfio/common: Extract vIOMMU code from vfio_sync_dirty_bitmap() Date: Tue, 30 May 2023 18:59:35 +0100 Message-Id: <20230530175937.24202-14-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=884 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: dn4NIzkDEirpuTEmWEBTk4rmPWwK0tb2 X-Proofpoint-GUID: dn4NIzkDEirpuTEmWEBTk4rmPWwK0tb2 Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Extract vIOMMU code from vfio_sync_dirty_bitmap() to a new function and restructure the code. This is done in preparation for optimizing vIOMMU deviice dirty page tracking. No functional changes intended. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 63 +++++++++++++++++++++++++++++------------------- 1 file changed, 38 insertions(+), 25 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index d2897aceedae..733f0bd7825f 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1939,37 +1939,50 @@ static int vfio_sync_ram_discard_listener_dirty_bitmap(VFIOContainer *container, &vrdl); } +static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, + MemoryRegionSection *section) +{ + VFIOGuestIOMMU *giommu; + bool found = false; + Int128 llend; + vfio_giommu_dirty_notifier gdn; + int idx; + + QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { + if (MEMORY_REGION(giommu->iommu_mr) == section->mr && + giommu->n.start == section->offset_within_region) { + found = true; + break; + } + } + + if (!found) { + return 0; + } + + gdn.giommu = giommu; + idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, + MEMTXATTRS_UNSPECIFIED); + + llend = int128_add(int128_make64(section->offset_within_region), + section->size); + llend = int128_sub(llend, int128_one()); + + iommu_notifier_init(&gdn.n, vfio_iommu_map_dirty_notify, IOMMU_NOTIFIER_MAP, + section->offset_within_region, int128_get64(llend), + idx); + memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); + + return 0; +} + static int vfio_sync_dirty_bitmap(VFIOContainer *container, MemoryRegionSection *section) { ram_addr_t ram_addr; if (memory_region_is_iommu(section->mr)) { - VFIOGuestIOMMU *giommu; - - QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { - if (MEMORY_REGION(giommu->iommu_mr) == section->mr && - giommu->n.start == section->offset_within_region) { - Int128 llend; - vfio_giommu_dirty_notifier gdn = { .giommu = giommu }; - int idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, - MEMTXATTRS_UNSPECIFIED); - - llend = int128_add(int128_make64(section->offset_within_region), - section->size); - llend = int128_sub(llend, int128_one()); - - iommu_notifier_init(&gdn.n, - vfio_iommu_map_dirty_notify, - IOMMU_NOTIFIER_MAP, - section->offset_within_region, - int128_get64(llend), - idx); - memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); - break; - } - } - return 0; + return vfio_sync_iommu_dirty_bitmap(container, section); } else if (memory_region_has_ram_discard_manager(section->mr)) { return vfio_sync_ram_discard_listener_dirty_bitmap(container, section); } From patchwork Tue May 30 17:59:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787807 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=YLc34s/H; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0bm1y24z20Q5 for ; Wed, 31 May 2023 04:02:28 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43f8-0003Na-GP; Tue, 30 May 2023 14:01:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43ei-0002Wp-8T for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:12 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43ed-0002mK-Tp for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:09 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4juV021715; Tue, 30 May 2023 18:01:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=MCVvu0ZT0Efg7fIZpsFsdi88Rn/1HAZ8J63Xek+QgJQ=; b=YLc34s/Hj/5FoIkBXtvRFBiDlaHX+k0rUtClfJ7UExCn5RJZHr+/7NcJNESo+z8pmnN1 7I9APSbwztGWA8/lHEykoNYbfFizMd90nXRPhHc3zsQPhaK1eQ4yHrZ5P1WcrDkdcJrH UgjwHQGPTg+0RL2wLId9u1V+ObjObGuSWbAtU6WZ2u4XsKEe2jDzONTz41urG2II3qIL DWWFSPFFVQcDpcmEG1iXdXMcAe0W6gM+PuowdcZHT4sVQQVw7Ea8EeR8scvAT77+fFeB GGCBujhFjK7HYeJFGH3AuCnWWckb7JOxDJ1TQ8DXv+KMI9Vz7dgGHWJ/nrwDoW2fugfA tA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjh3emy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:01:04 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UGjDqD014692; Tue, 30 May 2023 18:01:03 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48vjw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:01:03 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspd011459; Tue, 30 May 2023 18:01:02 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-15; Tue, 30 May 2023 18:01:02 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 14/15] vfio/common: Optimize device dirty page tracking with vIOMMU Date: Tue, 30 May 2023 18:59:36 +0100 Message-Id: <20230530175937.24202-15-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=780 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-GUID: Gz7CqUk9Nrwm2XtUVwL2CgX4RiPxpcBb X-Proofpoint-ORIG-GUID: Gz7CqUk9Nrwm2XtUVwL2CgX4RiPxpcBb Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon When vIOMMU is enabled, syncing dirty page bitmaps is done by replaying the vIOMMU mappings and querying the dirty bitmap for each mapping. With device dirty tracking this causes a lot of overhead, since the HW is queried many times (even with small idle guest this can end up with thousands of calls to HW). Optimize this by de-coupling dirty bitmap query from vIOMMU replay. Now a single dirty bitmap is queried per vIOMMU MR section, which is then used for all corresponding vIOMMU mappings within that MR section. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 72 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 733f0bd7825f..5b211380306a 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1857,8 +1857,36 @@ out: typedef struct { IOMMUNotifier n; VFIOGuestIOMMU *giommu; + VFIOBitmap vbmap; } vfio_giommu_dirty_notifier; +static int vfio_iommu_set_dirty_bitmap(VFIOContainer *container, + vfio_giommu_dirty_notifier *gdn, + hwaddr iova, hwaddr size, + ram_addr_t ram_addr) +{ + VFIOBitmap *vbmap = &gdn->vbmap; + VFIOBitmap dst_vbmap; + hwaddr start_iova = REAL_HOST_PAGE_ALIGN(gdn->n.start); + hwaddr copy_offset; + int ret; + + ret = vfio_bitmap_alloc(&dst_vbmap, size); + if (ret) { + return -ENOMEM; + } + + copy_offset = (iova - start_iova) / qemu_real_host_page_size(); + bitmap_copy_with_src_offset(dst_vbmap.bitmap, vbmap->bitmap, copy_offset, + dst_vbmap.pages); + + cpu_physical_memory_set_dirty_lebitmap(dst_vbmap.bitmap, ram_addr, + dst_vbmap.pages); + g_free(dst_vbmap.bitmap); + + return 0; +} + static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) { vfio_giommu_dirty_notifier *gdn = container_of(n, @@ -1879,8 +1907,15 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) rcu_read_lock(); if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { - ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1, - translated_addr); + if (gdn->vbmap.bitmap) { + ret = vfio_iommu_set_dirty_bitmap(container, gdn, iova, + iotlb->addr_mask + 1, + translated_addr); + } else { + ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1, + translated_addr); + } + if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%s)", @@ -1961,6 +1996,7 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, } gdn.giommu = giommu; + gdn.vbmap.bitmap = NULL; idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, MEMTXATTRS_UNSPECIFIED); @@ -1968,10 +2004,44 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, section->size); llend = int128_sub(llend, int128_one()); + /* + * Optimize device dirty tracking if the MR section is at least partially + * tracked. Optimization is done by querying a single dirty bitmap for the + * entire range instead of querying dirty bitmap for each vIOMMU mapping. + */ + if (vfio_devices_all_device_dirty_tracking(container)) { + hwaddr start = REAL_HOST_PAGE_ALIGN(section->offset_within_region); + hwaddr end = int128_get64(llend); + hwaddr iommu_max_iova; + hwaddr size; + int ret; + + ret = vfio_viommu_get_max_iova(&iommu_max_iova); + if (ret) { + return -EINVAL; + } + + size = REAL_HOST_PAGE_ALIGN(MIN(iommu_max_iova, end) - start); + + ret = vfio_bitmap_alloc(&gdn.vbmap, size); + if (ret) { + return -ENOMEM; + } + + ret = vfio_devices_query_dirty_bitmap(container, &gdn.vbmap, + start, size); + if (ret) { + g_free(gdn.vbmap.bitmap); + + return ret; + } + } + iommu_notifier_init(&gdn.n, vfio_iommu_map_dirty_notify, IOMMU_NOTIFIER_MAP, section->offset_within_region, int128_get64(llend), idx); memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); + g_free(gdn.vbmap.bitmap); return 0; } From patchwork Tue May 30 17:59:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1787808 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=3xvU3pC6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QW0bq1njyz20Pc for ; Wed, 31 May 2023 04:02:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q43f9-0003Z0-EJ; Tue, 30 May 2023 14:01:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43el-0002mX-9s for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:18 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q43ej-0002nV-C5 for qemu-devel@nongnu.org; Tue, 30 May 2023 14:01:14 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34UE4udJ016942; Tue, 30 May 2023 18:01:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=NHP3YX3xN1KFlY3tE2FocA1gNKdffoZzE3n9vQTADMI=; b=3xvU3pC6H1/IGfVPY9snPBk2XqbdaP62OrujCOVUx6WPTuZeXAa/+0Hn8YBT+a6Vs6Xr pz/K8vb3n6LChez5QgvZKtld2bBTmygzCyDKU27unwmtK9QbeELLILmPXMKItmFD/bej 15RsqSFHPr+EXAOzabdV+hZLUmDPDPO44PA/m2NB31VaiDsXExLFMZhjreoam4f8zTHf Qz1b/wseSRhSzbL4Mni2zsunAPP4LB0cHv+94M7dJrbKU2cJr8KzWLsIQ2zo0NrSpulm W0pqAmO97P3A3qj+4gq2qkIgPID3KIf4oNcnllByD+XRVXL4hQcgk8l0aZ6+9n1N5SXK KA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qvhjkkexd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:01:09 +0000 Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34UHhRMH014722; Tue, 30 May 2023 18:01:08 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3qu8a48vs1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 30 May 2023 18:01:08 +0000 Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34UHxspf011459; Tue, 30 May 2023 18:01:08 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-191-61.vpn.oracle.com [10.175.191.61]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3qu8a48srr-16; Tue, 30 May 2023 18:01:07 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v3 15/15] vfio/common: Block migration with vIOMMUs without address width limits Date: Tue, 30 May 2023 18:59:37 +0100 Message-Id: <20230530175937.24202-16-joao.m.martins@oracle.com> In-Reply-To: <20230530175937.24202-1-joao.m.martins@oracle.com> References: <20230530175937.24202-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-30_13,2023-05-30_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 spamscore=0 phishscore=0 malwarescore=0 mlxlogscore=717 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305300144 X-Proofpoint-ORIG-GUID: kU_iBsfymTH9mQECYztpOBfWqHjmGQWy X-Proofpoint-GUID: kU_iBsfymTH9mQECYztpOBfWqHjmGQWy Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Only block the case when the underlying vIOMMU model does not report any address space limits, in addition to DMA translation being off, or no vIOMMU being present. The limits are needed such that can define the IOVA limits to arm the device dirty tracker. Additionally, reword the migration blocker error message to clarify that we the configured vIOMMU does not support migration, as opposed to implying that just being there blocks migration. Signed-off-by: Joao Martins --- hw/vfio/common.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 5b211380306a..b0cf86559032 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -474,15 +474,17 @@ static int vfio_viommu_get_max_iova(hwaddr *max_iova) int vfio_block_giommu_migration(Error **errp) { + hwaddr max; int ret; if (giommu_migration_blocker || - !vfio_viommu_preset()) { + !vfio_viommu_preset() || + (vfio_viommu_preset() && !vfio_viommu_get_max_iova(&max))) { return 0; } error_setg(&giommu_migration_blocker, - "Migration is currently not supported with vIOMMU enabled"); + "Migration is currently not supported with the configured vIOMMU"); ret = migrate_add_blocker(giommu_migration_blocker, errp); if (ret < 0) { error_free(giommu_migration_blocker);