From patchwork Fri Sep 21 08:17:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972916 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmrS4Pchz9s9G for ; Fri, 21 Sep 2018 18:25:44 +1000 (AEST) Received: from localhost ([::1]:54381 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gkw-00014b-4Q for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:25:42 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47311) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GeU-0003de-HG for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GeR-0001NS-8W for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53482) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GeL-00010T-GI; Fri, 21 Sep 2018 04:18:53 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B71C55C9; Fri, 21 Sep 2018 08:18:52 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6D41B7D904; Fri, 21 Sep 2018 08:18:50 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:52 +0200 Message-Id: <20180921081819.9203-2-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 21 Sep 2018 08:18:52 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 01/28] hw/arm/smmu-common: Fix the name of the iommu memory regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" At the point smmu_find_add_as() gets called, the bus number might not be computed. Let's change the name of IOMMU memory region and just use the devfn and an incrementing index. The name only is used for debug. Signed-off-by: Eric Auger --- hw/arm/smmu-common.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 55c75d65d2..3f55cfd193 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -311,6 +311,7 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, void *opaque, int devfn) SMMUState *s = opaque; SMMUPciBus *sbus = g_hash_table_lookup(s->smmu_pcibus_by_busptr, bus); SMMUDevice *sdev; + static uint index; if (!sbus) { sbus = g_malloc0(sizeof(SMMUPciBus) + @@ -321,9 +322,8 @@ static AddressSpace *smmu_find_add_as(PCIBus *bus, void *opaque, int devfn) sdev = sbus->pbdev[devfn]; if (!sdev) { - char *name = g_strdup_printf("%s-%d-%d", - s->mrtypename, - pci_bus_num(bus), devfn); + char *name = g_strdup_printf("%s-%d-%d", s->mrtypename, devfn, index++); + sdev = sbus->pbdev[devfn] = g_new0(SMMUDevice, 1); sdev->smmu = s; From patchwork Fri Sep 21 08:17:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972922 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmxW2XlTz9sBJ for ; Fri, 21 Sep 2018 18:30:05 +1000 (AEST) Received: from localhost ([::1]:54413 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gp8-0005WY-Hc for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:30:02 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47326) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GeY-0003hX-7G for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GeR-0001NR-8U for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52652) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GeO-00019O-9R; Fri, 21 Sep 2018 04:18:56 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5568C308FBA9; Fri, 21 Sep 2018 08:18:55 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0D1527D665; Fri, 21 Sep 2018 08:18:52 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:53 +0200 Message-Id: <20180921081819.9203-3-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Fri, 21 Sep 2018 08:18:55 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 02/28] hw/arm/smmuv3: fix eventq recording and IRQ triggerring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The event queue management is broken today. Event records are not properly written as EVT_SET_* macro was not updating the actual event record. Also the event queue interrupt is not correctly triggered. Fixes: bb981004eaf4 ("hw/arm/smmuv3: Event queue recording helper") Signed-off-by: Eric Auger --- hw/arm/smmuv3-internal.h | 26 +++++++++++++------------- hw/arm/smmuv3.c | 2 +- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/hw/arm/smmuv3-internal.h b/hw/arm/smmuv3-internal.h index bab25d640e..19540f8f41 100644 --- a/hw/arm/smmuv3-internal.h +++ b/hw/arm/smmuv3-internal.h @@ -442,17 +442,17 @@ typedef struct SMMUEventInfo { #define EVT_Q_OVERFLOW (1 << 31) -#define EVT_SET_TYPE(x, v) deposit32((x)->word[0], 0 , 8 , v) -#define EVT_SET_SSV(x, v) deposit32((x)->word[0], 11, 1 , v) -#define EVT_SET_SSID(x, v) deposit32((x)->word[0], 12, 20, v) -#define EVT_SET_SID(x, v) ((x)->word[1] = v) -#define EVT_SET_STAG(x, v) deposit32((x)->word[2], 0 , 16, v) -#define EVT_SET_STALL(x, v) deposit32((x)->word[2], 31, 1 , v) -#define EVT_SET_PNU(x, v) deposit32((x)->word[3], 1 , 1 , v) -#define EVT_SET_IND(x, v) deposit32((x)->word[3], 2 , 1 , v) -#define EVT_SET_RNW(x, v) deposit32((x)->word[3], 3 , 1 , v) -#define EVT_SET_S2(x, v) deposit32((x)->word[3], 7 , 1 , v) -#define EVT_SET_CLASS(x, v) deposit32((x)->word[3], 8 , 2 , v) +#define EVT_SET_TYPE(x, v) ((x)->word[0] = deposit32((x)->word[0], 0 , 8 , v)) +#define EVT_SET_SSV(x, v) ((x)->word[0] = deposit32((x)->word[0], 11, 1 , v)) +#define EVT_SET_SSID(x, v) ((x)->word[0] = deposit32((x)->word[0], 12, 20, v)) +#define EVT_SET_SID(x, v) ((x)->word[1] = v) +#define EVT_SET_STAG(x, v) ((x)->word[2] = deposit32((x)->word[2], 0 , 16, v)) +#define EVT_SET_STALL(x, v) ((x)->word[2] = deposit32((x)->word[2], 31, 1 , v)) +#define EVT_SET_PNU(x, v) ((x)->word[3] = deposit32((x)->word[3], 1 , 1 , v)) +#define EVT_SET_IND(x, v) ((x)->word[3] = deposit32((x)->word[3], 2 , 1 , v)) +#define EVT_SET_RNW(x, v) ((x)->word[3] = deposit32((x)->word[3], 3 , 1 , v)) +#define EVT_SET_S2(x, v) ((x)->word[3] = deposit32((x)->word[3], 7 , 1 , v)) +#define EVT_SET_CLASS(x, v) ((x)->word[3] = deposit32((x)->word[3], 8 , 2 , v)) #define EVT_SET_ADDR(x, addr) \ do { \ (x)->word[5] = (uint32_t)(addr >> 32); \ @@ -460,8 +460,8 @@ typedef struct SMMUEventInfo { } while (0) #define EVT_SET_ADDR2(x, addr) \ do { \ - deposit32((x)->word[7], 3, 29, addr >> 16); \ - deposit32((x)->word[7], 0, 16, addr & 0xffff);\ + (x)->word[7] = deposit32((x)->word[7], 3, 29, addr >> 16); \ + (x)->word[7] = deposit32((x)->word[7], 0, 16, addr & 0xffff);\ } while (0) void smmuv3_record_event(SMMUv3State *s, SMMUEventInfo *event); diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index bb6a24e9b8..8c4e99fecc 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -136,7 +136,7 @@ static MemTxResult smmuv3_write_eventq(SMMUv3State *s, Evt *evt) return r; } - if (smmuv3_q_empty(q)) { + if (!smmuv3_q_empty(q)) { smmuv3_trigger_irq(s, SMMU_IRQ_EVTQ, 0); } return MEMTX_OK; From patchwork Fri Sep 21 08:17:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972921 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmxW2bQzz9sCS for ; Fri, 21 Sep 2018 18:30:06 +1000 (AEST) Received: from localhost ([::1]:54415 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gp9-0005Yk-QP for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:30:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47459) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gel-0003sr-2D for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gek-0002fb-CI for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53568) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gee-00029P-7c; Fri, 21 Sep 2018 04:19:13 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1B83519CF61; Fri, 21 Sep 2018 08:19:09 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F4127DFC0; Fri, 21 Sep 2018 08:18:55 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:54 +0200 Message-Id: <20180921081819.9203-4-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 21 Sep 2018 08:19:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 03/28] update-linux-headers: Import iommu.h X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Update the script to import the new iommu.h uapi header. Signed-off-by: Eric Auger --- scripts/update-linux-headers.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index 0a964fe240..3f11e49758 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -125,7 +125,7 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" -for header in kvm.h vfio.h vfio_ccw.h vhost.h \ +for header in kvm.h vfio.h iommu.h vfio_ccw.h vhost.h \ psci.h psp-sev.h userfaultfd.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done From patchwork Fri Sep 21 08:17:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972931 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn140SQdz9sCV for ; Fri, 21 Sep 2018 18:33:12 +1000 (AEST) Received: from localhost ([::1]:54435 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gs9-00087F-J6 for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:33:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47480) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gem-0003uP-Ff for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gel-0002i1-2I for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52698) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Geg-0002HF-6B; Fri, 21 Sep 2018 04:19:14 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B27E6308FBA9; Fri, 21 Sep 2018 08:19:11 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 656497D904; Fri, 21 Sep 2018 08:19:09 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:55 +0200 Message-Id: <20180921081819.9203-5-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Fri, 21 Sep 2018 08:19:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 04/28] linux-headers: Partial header update X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This imports both the iommu.h and vfio.h headers found on branch https://github.com/eauger/linux/tree/v4.19-rc4-2stage-rfc-v2 ([RFC v2 00/20] SMMUv3 Nested Stage Setup) Signed-off-by: Eric Auger --- v1 -> v2: - addition of the default reporting API - renamings - CD removal from iommu_pasid_smmuv3 --- linux-headers/linux/iommu.h | 237 ++++++++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 48 ++++++++ 2 files changed, 285 insertions(+) create mode 100644 linux-headers/linux/iommu.h diff --git a/linux-headers/linux/iommu.h b/linux-headers/linux/iommu.h new file mode 100644 index 0000000000..9d55e7d053 --- /dev/null +++ b/linux-headers/linux/iommu.h @@ -0,0 +1,237 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * IOMMU user API definitions + * + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#ifndef _IOMMU_H +#define _IOMMU_H + +#include + +/** + * SMMUv3 Stream Table Entry stage 1 related information + * @s1contextptr: Context Descriptor Table GPA + * @abort: shall the STE lead to abort + * @s1fmt: STE s1fmt field as set by the guest + * @s1cdmax: STE s1cdmax as set by the guest + * @s1dss: STE s1dss as set by the guest + * All field names match the smmu 3.0/3.1 spec (ARM IHI 0070A) + */ +struct iommu_pasid_smmuv3 { + __u64 s1contextptr; + __u8 bypass; + __u8 abort; + __u8 s1fmt; + __u8 s1cdmax; + __u8 s1dss; +}; + +/** + * PASID table data used to bind guest PASID table to the host IOMMU + * Note PASID table corresponds to the Context Table on ARM SMMUv3. + * + * @version: API version to prepare for future extensions + * @format: format of the PASID table + * + */ +struct iommu_pasid_table_config { +#define PASID_TABLE_CFG_VERSION_1 1 + __u32 version; +#define IOMMU_PASID_FORMAT_SMMUV3 (1 << 0) + __u32 format; + union { + struct iommu_pasid_smmuv3 smmuv3; + }; +}; + +/** + * enum iommu_inv_granularity - Generic invalidation granularity + * @IOMMU_INV_GRANU_DOMAIN_ALL_PASID: TLB entries or PASID caches of all + * PASIDs associated with a domain ID + * @IOMMU_INV_GRANU_PASID_SEL: TLB entries or PASID cache associated + * with a PASID and a domain + * @IOMMU_INV_GRANU_PAGE_PASID: TLB entries of selected page range + * within a PASID + * + * When an invalidation request is passed down to IOMMU to flush translation + * caches, it may carry different granularity levels, which can be specific + * to certain types of translation caches. + * This enum is a collection of granularities for all types of translation + * caches. The idea is to make it easy for IOMMU model specific driver to + * convert from generic to model specific value. Each IOMMU driver + * can enforce check based on its own conversion table. The conversion is + * based on 2D look-up with inputs as follows: + * - translation cache types + * - granularity + * + * type | DTLB | TLB | PASID | + * granule | | | cache | + * -----------------+-----------+-----------+-----------+ + * DN_ALL_PASID | Y | Y | Y | + * PASID_SEL | Y | Y | Y | + * PAGE_PASID | Y | Y | N/A | + * + */ +enum iommu_inv_granularity { + IOMMU_INV_GRANU_DOMAIN_ALL_PASID, + IOMMU_INV_GRANU_PASID_SEL, + IOMMU_INV_GRANU_PAGE_PASID, + IOMMU_INV_NR_GRANU, +}; + +/** + * enum iommu_inv_type - Generic translation cache types for invalidation + * + * @IOMMU_INV_TYPE_DTLB: device IOTLB + * @IOMMU_INV_TYPE_TLB: IOMMU paging structure cache + * @IOMMU_INV_TYPE_PASID: PASID cache + * Invalidation requests sent to IOMMU for a given device need to indicate + * which type of translation cache to be operated on. Combined with enum + * iommu_inv_granularity, model specific driver can do a simple lookup to + * convert from generic to model specific value. + */ +enum iommu_inv_type { + IOMMU_INV_TYPE_DTLB, + IOMMU_INV_TYPE_TLB, + IOMMU_INV_TYPE_PASID, + IOMMU_INV_NR_TYPE +}; + +/** + * Translation cache invalidation header that contains mandatory meta data. + * @version: info format version, expecting future extesions + * @type: type of translation cache to be invalidated + */ +struct iommu_cache_invalidate_hdr { + __u32 version; +#define TLB_INV_HDR_VERSION_1 1 + enum iommu_inv_type type; +}; + +/** + * Translation cache invalidation information, contains generic IOMMU + * data which can be parsed based on model ID by model specific drivers. + * Since the invalidation of second level page tables are included in the + * unmap operation, this info is only applicable to the first level + * translation caches, i.e. DMA request with PASID. + * + * @granularity: requested invalidation granularity, type dependent + * @size: 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc. + * @nr_pages: number of pages to invalidate + * @pasid: processor address space ID value per PCI spec. + * @arch_id: architecture dependent id characterizing a context + * and tagging the caches, ie. domain Identfier on VTD, + * asid on ARM SMMU + * @addr: page address to be invalidated + * @flags IOMMU_INVALIDATE_ADDR_LEAF: leaf paging entries + * IOMMU_INVALIDATE_GLOBAL_PAGE: global pages + * + */ +struct iommu_cache_invalidate_info { + struct iommu_cache_invalidate_hdr hdr; + enum iommu_inv_granularity granularity; + __u32 flags; +#define IOMMU_INVALIDATE_ADDR_LEAF (1 << 0) +#define IOMMU_INVALIDATE_GLOBAL_PAGE (1 << 1) + __u8 size; + __u64 nr_pages; + __u32 pasid; + __u64 arch_id; + __u64 addr; +}; + +struct iommu_guest_msi_binding { + __u64 iova; + __u64 gpa; + __u32 granule; +}; + +/* Generic fault types, can be expanded IRQ remapping fault */ +enum iommu_fault_type { + IOMMU_FAULT_DMA_UNRECOV = 1, /* unrecoverable fault */ + IOMMU_FAULT_PAGE_REQ, /* page request fault */ +}; + +enum iommu_fault_reason { + IOMMU_FAULT_REASON_UNKNOWN = 0, + + /* IOMMU internal error, no specific reason to report out */ + IOMMU_FAULT_REASON_INTERNAL, + + /* Could not access the PASID table (fetch caused external abort) */ + IOMMU_FAULT_REASON_PASID_FETCH, + + /* could not access the device context (fetch caused external abort) */ + IOMMU_FAULT_REASON_DEVICE_CONTEXT_FETCH, + + /* pasid entry is invalid or has configuration errors */ + IOMMU_FAULT_REASON_BAD_PASID_ENTRY, + + /* device context entry is invalid or has configuration errors */ + IOMMU_FAULT_REASON_BAD_DEVICE_CONTEXT_ENTRY, + /* + * PASID is out of range (e.g. exceeds the maximum PASID + * supported by the IOMMU) or disabled. + */ + IOMMU_FAULT_REASON_PASID_INVALID, + + /* source id is out of range */ + IOMMU_FAULT_REASON_SOURCEID_INVALID, + + /* + * An external abort occurred fetching (or updating) a translation + * table descriptor + */ + IOMMU_FAULT_REASON_WALK_EABT, + + /* + * Could not access the page table entry (Bad address), + * actual translation fault + */ + IOMMU_FAULT_REASON_PTE_FETCH, + + /* Protection flag check failed */ + IOMMU_FAULT_REASON_PERMISSION, + + /* access flag check failed */ + IOMMU_FAULT_REASON_ACCESS, + + /* Output address of a translation stage caused Address Size fault */ + IOMMU_FAULT_REASON_OOR_ADDRESS +}; + +/** + * struct iommu_fault - Generic fault data + * + * @type contains fault type + * @reason fault reasons if relevant outside IOMMU driver. + * IOMMU driver internal faults are not reported. + * @addr: tells the offending page address + * @fetch_addr: tells the address that caused an abort, if any + * @pasid: contains process address space ID, used in shared virtual memory + * @page_req_group_id: page request group index + * @last_req: last request in a page request group + * @pasid_valid: indicates if the PRQ has a valid PASID + * @prot: page access protection flag: + * IOMMU_FAULT_READ, IOMMU_FAULT_WRITE + */ + +struct iommu_fault { + __u32 type; /* enum iommu_fault_type */ + __u32 reason; /* enum iommu_fault_reason */ + __u64 addr; + __u64 fetch_addr; + __u32 pasid; + __u32 page_req_group_id; + __u32 last_req; + __u32 pasid_valid; + __u32 prot; + __u32 access; +}; +#endif /* _IOMMU_H */ + diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 3615a269d3..250c09f3cf 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -14,6 +14,7 @@ #include #include +#include #define VFIO_API_VERSION 0 @@ -665,6 +666,53 @@ struct vfio_iommu_type1_dma_unmap { #define VFIO_IOMMU_ENABLE _IO(VFIO_TYPE, VFIO_BASE + 15) #define VFIO_IOMMU_DISABLE _IO(VFIO_TYPE, VFIO_BASE + 16) +struct vfio_iommu_type1_bind_pasid_table { + __u32 argsz; + __u32 flags; + struct iommu_pasid_table_config config; +}; +#define VFIO_IOMMU_BIND_PASID_TABLE _IO(VFIO_TYPE, VFIO_BASE + 22) + +struct vfio_iommu_type1_cache_invalidate { + __u32 argsz; + __u32 flags; + struct iommu_cache_invalidate_info info; +}; +#define VFIO_IOMMU_CACHE_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) + +struct vfio_iommu_type1_bind_guest_msi { + __u32 argsz; + __u32 flags; + struct iommu_guest_msi_binding binding; +}; +#define VFIO_IOMMU_BIND_MSI _IO(VFIO_TYPE, VFIO_BASE + 24) + +struct vfio_iommu_type1_guest_fault_config { +#define VFIO_IOMMU_FAULT_UNRECOVERABLE (1 << 0) + __u32 flags; + union { + __u8 qs; /* queue size, log2(entries) */ + }; +}; + +struct vfio_iommu_type1_set_fault_eventfd { + __u32 argsz; + __u32 flags; + __u32 eventfd; + struct vfio_iommu_type1_guest_fault_config config; +}; +#define VFIO_IOMMU_SET_FAULT_EVENTFD _IO(VFIO_TYPE, VFIO_BASE + 25) + +struct vfio_iommu_type1_get_fault_events { + __u32 argsz; + __u32 flags; + __u32 count; /* number of faults returned */ + __u32 reserved; + struct iommu_fault events[]; +}; + +#define VFIO_IOMMU_GET_FAULT_EVENTS _IO(VFIO_TYPE, VFIO_BASE + 26) + /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /* From patchwork Fri Sep 21 08:17:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972919 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmtG6MxCz9sBJ for ; Fri, 21 Sep 2018 18:27:18 +1000 (AEST) Received: from localhost ([::1]:54396 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GmS-0002PJ-9a for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:27:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47460) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gel-0003ss-2i for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gek-0002fT-BD for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38030) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Geh-0002Sm-3H; Fri, 21 Sep 2018 04:19:15 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4BEEBC04AC42; Fri, 21 Sep 2018 08:19:14 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0662D7D904; Fri, 21 Sep 2018 08:19:11 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:56 +0200 Message-Id: <20180921081819.9203-6-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 21 Sep 2018 08:19:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 05/28] memory: add IOMMU_ATTR_VFIO_NESTED IOMMU memory region attribute X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We introduce a new IOMMU Memory Region attribute, IOMMU_ATTR_VFIO_NESTED which tells whether the virtual IOMMU requires physical nested stages for VFIO integration. Intel virtual IOMMU supports Caching Mode and does not require 2 stages at physical level. However virtual ARM SMMU does not implement such caching mode and requires to use physical stage 1 for VFIO integration. Signed-off-by: Eric Auger --- include/exec/memory.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index eb4f2fb249..b6e59c139c 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -209,7 +209,8 @@ struct MemoryRegionOps { }; enum IOMMUMemoryRegionAttr { - IOMMU_ATTR_SPAPR_TCE_FD + IOMMU_ATTR_SPAPR_TCE_FD, + IOMMU_ATTR_VFIO_NESTED, }; /** From patchwork Fri Sep 21 08:17:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972918 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmsK3Zzmz9sBq for ; Fri, 21 Sep 2018 18:26:29 +1000 (AEST) Received: from localhost ([::1]:54391 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gle-0001hp-VK for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:26:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47635) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gf8-0004BT-Jo for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gf3-0003X8-Hf for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Geu-00036c-G3; Fri, 21 Sep 2018 04:19:28 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AE67D86676; Fri, 21 Sep 2018 08:19:27 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9736F7D665; Fri, 21 Sep 2018 08:19:14 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:57 +0200 Message-Id: <20180921081819.9203-7-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 21 Sep 2018 08:19:27 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 06/28] hw/arm/smmuv3: Implement get_attr API to report IOMMU_ATTR_VFIO_NESTED X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Virtual SMMUv3 requires physical nested stages for VFIO integration. Let's advertise this. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 8c4e99fecc..d6895071a5 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -1504,6 +1504,17 @@ static void smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu, } } +static int smmuv3_get_attr(IOMMUMemoryRegion *iommu, + enum IOMMUMemoryRegionAttr attr, + void *data) +{ + if (attr == IOMMU_ATTR_VFIO_NESTED) { + *(bool *) data = true; + return 0; + } + return -EINVAL; +} + static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, void *data) { @@ -1511,6 +1522,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, imrc->translate = smmuv3_translate; imrc->notify_flag_changed = smmuv3_notify_flag_changed; + imrc->get_attr = smmuv3_get_attr; } static const TypeInfo smmuv3_type_info = { From patchwork Fri Sep 21 08:17:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972940 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn5K6w0mz9sBq for ; Fri, 21 Sep 2018 18:36:53 +1000 (AEST) Received: from localhost ([::1]:54464 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gvj-0003G1-EL for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:36:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47636) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gf8-0004BU-Ju for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gf3-0003X9-Hh for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52752) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gex-0003DK-3A; Fri, 21 Sep 2018 04:19:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5105330022E8; Fri, 21 Sep 2018 08:19:30 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 041CB7D665; Fri, 21 Sep 2018 08:19:27 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:58 +0200 Message-Id: <20180921081819.9203-8-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Fri, 21 Sep 2018 08:19:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 07/28] hw/vfio/common: Refactor container initialization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" To prepare for testing yet another extension, let's refactor the code. We introduce vfio_iommu_get_type() helper which selects the richest API (v2 first). Then vfio_init_container() does the SET_CONTAINER and SET_IOMMU ioctl calls. So we end up with a switch/case on the iommu_type which should be a little bit more readable when introducing the NESTING extension check. Also ioctl's get called once per iommu_type. Signed-off-by: Eric Auger Reviewed-by: Greg Kurz --- hw/vfio/common.c | 102 ++++++++++++++++++++++++++++++----------------- 1 file changed, 65 insertions(+), 37 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7c185e5a2e..53b8f773cc 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1036,12 +1036,58 @@ static void vfio_put_address_space(VFIOAddressSpace *space) } } +/* + * vfio_iommu_get_type - selects the richest iommu_type (v2 first) + * nested only is selected if requested by @force_nested + */ +static int vfio_iommu_get_type(VFIOContainer *container, + Error **errp) +{ + int fd = container->fd; + + if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { + return VFIO_TYPE1v2_IOMMU; + } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU)) { + return VFIO_TYPE1_IOMMU; + } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_v2_IOMMU)) { + return VFIO_SPAPR_TCE_v2_IOMMU; + } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_IOMMU)) { + return VFIO_SPAPR_TCE_IOMMU; + } else { + error_setg(errp, "No available IOMMU models"); + return -EINVAL; + } +} + +static int vfio_init_container(VFIOContainer *container, int group_fd, + int iommu_type, Error **errp) +{ + int ret; + + ret = ioctl(group_fd, VFIO_GROUP_SET_CONTAINER, &container->fd); + if (ret) { + error_setg_errno(errp, errno, "failed to set group container"); + return -errno; + } + + ret = ioctl(container->fd, VFIO_SET_IOMMU, iommu_type); + if (ret) { + error_setg_errno(errp, errno, "failed to set iommu for container"); + return -errno; + } + container->iommu_type = iommu_type; + return 0; +} + static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, Error **errp) { VFIOContainer *container; int ret, fd; VFIOAddressSpace *space; + int iommu_type; + bool v2 = false; + space = vfio_get_address_space(as); @@ -1101,23 +1147,20 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, container->fd = fd; QLIST_INIT(&container->giommu_list); QLIST_INIT(&container->hostwin_list); - if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) || - ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { - bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU); + + iommu_type = vfio_iommu_get_type(container, errp); + if (iommu_type < 0) { + goto free_container_exit; + } + + switch (iommu_type) { + case VFIO_TYPE1v2_IOMMU: + case VFIO_TYPE1_IOMMU: + { struct vfio_iommu_type1_info info; - ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd); + ret = vfio_init_container(container, group->fd, iommu_type, errp); if (ret) { - error_setg_errno(errp, errno, "failed to set group container"); - ret = -errno; - goto free_container_exit; - } - - container->iommu_type = v2 ? VFIO_TYPE1v2_IOMMU : VFIO_TYPE1_IOMMU; - ret = ioctl(fd, VFIO_SET_IOMMU, container->iommu_type); - if (ret) { - error_setg_errno(errp, errno, "failed to set iommu for container"); - ret = -errno; goto free_container_exit; } @@ -1137,28 +1180,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, } vfio_host_win_add(container, 0, (hwaddr)-1, info.iova_pgsizes); container->pgsizes = info.iova_pgsizes; - } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_IOMMU) || - ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_v2_IOMMU)) { + break; + } + case VFIO_SPAPR_TCE_v2_IOMMU: + v2 = true; + case VFIO_SPAPR_TCE_IOMMU: + { struct vfio_iommu_spapr_tce_info info; - bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_v2_IOMMU); - ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd); + ret = vfio_init_container(container, group->fd, iommu_type, errp); if (ret) { - error_setg_errno(errp, errno, "failed to set group container"); - ret = -errno; - goto free_container_exit; - } - container->iommu_type = - v2 ? VFIO_SPAPR_TCE_v2_IOMMU : VFIO_SPAPR_TCE_IOMMU; - ret = ioctl(fd, VFIO_SET_IOMMU, container->iommu_type); - if (ret) { - container->iommu_type = VFIO_SPAPR_TCE_IOMMU; - v2 = false; - ret = ioctl(fd, VFIO_SET_IOMMU, container->iommu_type); - } - if (ret) { - error_setg_errno(errp, errno, "failed to set iommu for container"); - ret = -errno; goto free_container_exit; } @@ -1222,10 +1253,7 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, info.dma32_window_size - 1, 0x1000); } - } else { - error_setg(errp, "No available IOMMU models"); - ret = -EINVAL; - goto free_container_exit; + } } vfio_kvm_device_add_group(group); From patchwork Fri Sep 21 08:17:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972942 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn8f2zjNz9sBJ for ; Fri, 21 Sep 2018 18:39:46 +1000 (AEST) Received: from localhost ([::1]:54478 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GyW-0005qC-2n for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:39:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47686) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GfF-0004IM-Bd for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GfC-0003q4-7f for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:49 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40342) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gf3-0003Rl-IC; Fri, 21 Sep 2018 04:19:38 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 49F473082B03; Fri, 21 Sep 2018 08:19:35 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A2B07D665; Fri, 21 Sep 2018 08:19:30 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:17:59 +0200 Message-Id: <20180921081819.9203-9-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Fri, 21 Sep 2018 08:19:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 08/28] hw/vfio/common: Force nested if iommu requires it X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" In case we detect the address space is translated by a virtual IOMMU which requires nested stages, let's set up the container with the VFIO_TYPE1_NESTING_IOMMU iommu_type. Signed-off-by: Eric Auger --- hw/vfio/common.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 53b8f773cc..1416892f1c 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1041,11 +1041,15 @@ static void vfio_put_address_space(VFIOAddressSpace *space) * nested only is selected if requested by @force_nested */ static int vfio_iommu_get_type(VFIOContainer *container, - Error **errp) + bool force_nested, Error **errp) { int fd = container->fd; - if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { + if (force_nested && + ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_NESTING_IOMMU)) { + /* NESTED implies v2 */ + return VFIO_TYPE1_NESTING_IOMMU; + } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { return VFIO_TYPE1v2_IOMMU; } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU)) { return VFIO_TYPE1_IOMMU; @@ -1085,9 +1089,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, VFIOContainer *container; int ret, fd; VFIOAddressSpace *space; + IOMMUMemoryRegion *iommu_mr; int iommu_type; + bool force_nested = false; bool v2 = false; + if (as != &address_space_memory && memory_region_is_iommu(as->root)) { + iommu_mr = IOMMU_MEMORY_REGION(as->root); + memory_region_iommu_get_attr(iommu_mr, IOMMU_ATTR_VFIO_NESTED, + (void *)&force_nested); + } space = vfio_get_address_space(as); @@ -1148,12 +1159,18 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, QLIST_INIT(&container->giommu_list); QLIST_INIT(&container->hostwin_list); - iommu_type = vfio_iommu_get_type(container, errp); + iommu_type = vfio_iommu_get_type(container, force_nested, errp); if (iommu_type < 0) { goto free_container_exit; } + if (force_nested && iommu_type != VFIO_TYPE1_NESTING_IOMMU) { + error_setg(errp, "nested mode requested by the virtual IOMMU " + "but not supported by the vfio iommu"); + } + switch (iommu_type) { + case VFIO_TYPE1_NESTING_IOMMU: case VFIO_TYPE1v2_IOMMU: case VFIO_TYPE1_IOMMU: { From patchwork Fri Sep 21 08:18:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972920 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gmw13yqvz9sBJ for ; Fri, 21 Sep 2018 18:28:49 +1000 (AEST) Received: from localhost ([::1]:54402 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gnv-0004V2-3m for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:28:47 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47753) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GfK-0004NH-An for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GfJ-00043Q-0r for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50094) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GfG-0003wO-2b; Fri, 21 Sep 2018 04:19:50 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2958A356F6; Fri, 21 Sep 2018 08:19:49 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 974AC7D665; Fri, 21 Sep 2018 08:19:35 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:00 +0200 Message-Id: <20180921081819.9203-10-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 21 Sep 2018 08:19:49 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 09/28] memory: Introduce IOMMUIOLTBNotifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Current IOMMUNotifiers dedicate to IOTLB related notifications. We want to introduce notifiers for virtual IOMMU config changes. Let's create a new IOMMUIOLTBNotifier datatype. This paves the way to the introduction of an IOMMUConfigNotifier. IOMMUNotifier now has an iotlb_notifier field. We change all calling sites. We also rename IOMMU_NOTIFIER_ALL into IOMMU_NOTIFIER_IOTLB_ALL Signed-off-by: Eric Auger --- exec.c | 12 ++++++------ hw/arm/smmu-common.c | 4 ++-- hw/i386/intel_iommu.c | 6 +++--- hw/vfio/common.c | 12 ++++++------ hw/virtio/vhost.c | 12 ++++++------ include/exec/memory.h | 25 +++++++++++++++---------- memory.c | 10 +++++----- 7 files changed, 43 insertions(+), 38 deletions(-) diff --git a/exec.c b/exec.c index 6826c8337d..1411660289 100644 --- a/exec.c +++ b/exec.c @@ -683,12 +683,12 @@ static void tcg_register_iommu_notifier(CPUState *cpu, * just register interest in the whole thing, on the assumption * that iommu reconfiguration will be rare. */ - iommu_notifier_init(¬ifier->n, - tcg_iommu_unmap_notify, - IOMMU_NOTIFIER_UNMAP, - 0, - HWADDR_MAX, - iommu_idx); + iommu_iotlb_notifier_init(¬ifier->n, + tcg_iommu_unmap_notify, + IOMMU_NOTIFIER_UNMAP, + 0, + HWADDR_MAX, + iommu_idx); memory_region_register_iommu_notifier(notifier->mr, ¬ifier->n); } diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index 3f55cfd193..ad6ef2135b 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -391,9 +391,9 @@ static void smmu_unmap_notifier_range(IOMMUNotifier *n) IOMMUTLBEntry entry; entry.target_as = &address_space_memory; - entry.iova = n->start; + entry.iova = n->iotlb_notifier.start; entry.perm = IOMMU_NONE; - entry.addr_mask = n->end - n->start; + entry.addr_mask = n->iotlb_notifier.end - n->iotlb_notifier.start; memory_region_notify_one(n, &entry); } diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 3dfada19a6..0dbd9512f2 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -2984,8 +2984,8 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) { IOMMUTLBEntry entry; hwaddr size; - hwaddr start = n->start; - hwaddr end = n->end; + hwaddr start = n->iotlb_notifier.start; + hwaddr end = n->iotlb_notifier.end; IntelIOMMUState *s = as->iommu_state; DMAMap map; @@ -3021,7 +3021,7 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) entry.target_as = &address_space_memory; /* Adjust iova for the size */ - entry.iova = n->start & ~(size - 1); + entry.iova = n->iotlb_notifier.start & ~(size - 1); /* This field is meaningless for unmap */ entry.translated_addr = 0; entry.perm = IOMMU_NONE; diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 1416892f1c..19dfc279d6 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -527,11 +527,11 @@ static void vfio_listener_region_add(MemoryListener *listener, llend = int128_sub(llend, int128_one()); iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, MEMTXATTRS_UNSPECIFIED); - iommu_notifier_init(&giommu->n, vfio_iommu_map_notify, - IOMMU_NOTIFIER_ALL, - section->offset_within_region, - int128_get64(llend), - iommu_idx); + iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_map_notify, + IOMMU_NOTIFIER_IOTLB_ALL, + section->offset_within_region, + int128_get64(llend), + iommu_idx); QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); memory_region_register_iommu_notifier(section->mr, &giommu->n); @@ -625,7 +625,7 @@ static void vfio_listener_region_del(MemoryListener *listener, QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { if (MEMORY_REGION(giommu->iommu) == section->mr && - giommu->n.start == section->offset_within_region) { + giommu->n.iotlb_notifier.start == section->offset_within_region) { memory_region_unregister_iommu_notifier(section->mr, &giommu->n); QLIST_REMOVE(giommu, giommu_next); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index d4cb5894a8..c21b9b8be9 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -677,11 +677,11 @@ static void vhost_iommu_region_add(MemoryListener *listener, end = int128_sub(end, int128_one()); iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, MEMTXATTRS_UNSPECIFIED); - iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, - IOMMU_NOTIFIER_UNMAP, - section->offset_within_region, - int128_get64(end), - iommu_idx); + iommu_iotlb_notifier_init(&iommu->n, vhost_iommu_unmap_notify, + IOMMU_NOTIFIER_UNMAP, + section->offset_within_region, + int128_get64(end), + iommu_idx); iommu->mr = section->mr; iommu->iommu_offset = section->offset_within_address_space - section->offset_within_region; @@ -704,7 +704,7 @@ static void vhost_iommu_region_del(MemoryListener *listener, QLIST_FOREACH(iommu, &dev->iommu_list, iommu_next) { if (iommu->mr == section->mr && - iommu->n.start == section->offset_within_region) { + iommu->n.iotlb_notifier.start == section->offset_within_region) { memory_region_unregister_iommu_notifier(iommu->mr, &iommu->n); QLIST_REMOVE(iommu, iommu_next); diff --git a/include/exec/memory.h b/include/exec/memory.h index b6e59c139c..31fc859c6b 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -86,18 +86,22 @@ typedef enum { IOMMU_NOTIFIER_MAP = 0x2, } IOMMUNotifierFlag; -#define IOMMU_NOTIFIER_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) +#define IOMMU_NOTIFIER_IOTLB_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) struct IOMMUNotifier; typedef void (*IOMMUNotify)(struct IOMMUNotifier *notifier, IOMMUTLBEntry *data); -struct IOMMUNotifier { +typedef struct IOMMUIOLTBNotifier { IOMMUNotify notify; - IOMMUNotifierFlag notifier_flags; /* Notify for address space range start <= addr <= end */ hwaddr start; hwaddr end; +} IOMMUIOLTBNotifier; + +struct IOMMUNotifier { + IOMMUNotifierFlag notifier_flags; + IOMMUIOLTBNotifier iotlb_notifier; int iommu_idx; QLIST_ENTRY(IOMMUNotifier) node; }; @@ -126,15 +130,16 @@ typedef struct IOMMUNotifier IOMMUNotifier; /* RAM is a persistent kind memory */ #define RAM_PMEM (1 << 5) -static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, - IOMMUNotifierFlag flags, - hwaddr start, hwaddr end, - int iommu_idx) +static inline void iommu_iotlb_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, + IOMMUNotifierFlag flags, + hwaddr start, hwaddr end, + int iommu_idx) { - n->notify = fn; + assert(flags & IOMMU_NOTIFIER_MAP || flags & IOMMU_NOTIFIER_UNMAP); n->notifier_flags = flags; - n->start = start; - n->end = end; + n->iotlb_notifier.notify = fn; + n->iotlb_notifier.start = start; + n->iotlb_notifier.end = end; n->iommu_idx = iommu_idx; } diff --git a/memory.c b/memory.c index 9b73892768..b7e2e43b68 100644 --- a/memory.c +++ b/memory.c @@ -1800,7 +1800,7 @@ void memory_region_register_iommu_notifier(MemoryRegion *mr, /* We need to register for at least one bitfield */ iommu_mr = IOMMU_MEMORY_REGION(mr); assert(n->notifier_flags != IOMMU_NOTIFIER_NONE); - assert(n->start <= n->end); + assert(n->iotlb_notifier.start <= n->iotlb_notifier.end); assert(n->iommu_idx >= 0 && n->iommu_idx < memory_region_iommu_num_indexes(iommu_mr)); @@ -1836,7 +1836,7 @@ void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) for (addr = 0; addr < memory_region_size(mr); addr += granularity) { iotlb = imrc->translate(iommu_mr, addr, IOMMU_NONE, n->iommu_idx); if (iotlb.perm != IOMMU_NONE) { - n->notify(n, &iotlb); + n->iotlb_notifier.notify(n, &iotlb); } /* if (2^64 - MR size) < granularity, it's possible to get an @@ -1879,8 +1879,8 @@ void memory_region_notify_one(IOMMUNotifier *notifier, * Skip the notification if the notification does not overlap * with registered range. */ - if (notifier->start > entry->iova + entry->addr_mask || - notifier->end < entry->iova) { + if (notifier->iotlb_notifier.start > entry->iova + entry->addr_mask || + notifier->iotlb_notifier.end < entry->iova) { return; } @@ -1891,7 +1891,7 @@ void memory_region_notify_one(IOMMUNotifier *notifier, } if (notifier->notifier_flags & request_flags) { - notifier->notify(notifier, entry); + notifier->iotlb_notifier.notify(notifier, entry); } } From patchwork Fri Sep 21 08:18:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972939 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn5F0VNKz9sBq for ; Fri, 21 Sep 2018 18:36:49 +1000 (AEST) Received: from localhost ([::1]:54463 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gve-0003Am-Jk for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:36:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47791) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GfO-0004RN-IK for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GfL-00047J-JQ for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:19:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50118) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GfI-00041s-Ep; Fri, 21 Sep 2018 04:19:52 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B8E4C20A9B; Fri, 21 Sep 2018 08:19:51 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 734A07D665; Fri, 21 Sep 2018 08:19:49 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:01 +0200 Message-Id: <20180921081819.9203-11-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Fri, 21 Sep 2018 08:19:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 10/28] memory: rename memory_region notify_iommu, notify_one X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Le's rename those notification functions to clearly discriminate iotlb notifications from looming config notifications. Signed-off-by: Eric Auger --- hw/arm/smmu-common.c | 2 +- hw/arm/smmuv3.c | 2 +- hw/i386/intel_iommu.c | 10 +++++----- hw/misc/tz-mpc.c | 8 ++++---- hw/ppc/spapr_iommu.c | 2 +- hw/s390x/s390-pci-inst.c | 4 ++-- include/exec/memory.h | 20 ++++++++++---------- memory.c | 12 ++++++------ 8 files changed, 30 insertions(+), 30 deletions(-) diff --git a/hw/arm/smmu-common.c b/hw/arm/smmu-common.c index ad6ef2135b..70b014e618 100644 --- a/hw/arm/smmu-common.c +++ b/hw/arm/smmu-common.c @@ -395,7 +395,7 @@ static void smmu_unmap_notifier_range(IOMMUNotifier *n) entry.perm = IOMMU_NONE; entry.addr_mask = n->iotlb_notifier.end - n->iotlb_notifier.start; - memory_region_notify_one(n, &entry); + memory_region_iotlb_notify_one(n, &entry); } /* Unmap all notifiers attached to @mr */ diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index d6895071a5..b8c41916aa 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -822,7 +822,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, entry.addr_mask = (1 << tt->granule_sz) - 1; entry.perm = IOMMU_NONE; - memory_region_notify_one(n, &entry); + memory_region_iotlb_notify_one(n, &entry); } /* invalidate an asid/iova tuple in all mr's */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 0dbd9512f2..dfd3cae054 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -1031,7 +1031,7 @@ static int vtd_dev_to_context_entry(IntelIOMMUState *s, uint8_t bus_num, static int vtd_sync_shadow_page_hook(IOMMUTLBEntry *entry, void *private) { - memory_region_notify_iommu((IOMMUMemoryRegion *)private, 0, *entry); + memory_region_iotlb_notify_iommu((IOMMUMemoryRegion *)private, 0, *entry); return 0; } @@ -1592,7 +1592,7 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, .addr_mask = size - 1, .perm = IOMMU_NONE, }; - memory_region_notify_iommu(&vtd_as->iommu, 0, entry); + memory_region_iotlb_notify_iommu(&vtd_as->iommu, 0, entry); } } } @@ -2031,7 +2031,7 @@ static bool vtd_process_device_iotlb_desc(IntelIOMMUState *s, entry.iova = addr; entry.perm = IOMMU_NONE; entry.translated_addr = 0; - memory_region_notify_iommu(&vtd_dev_as->iommu, 0, entry); + memory_region_iotlb_notify_iommu(&vtd_dev_as->iommu, 0, entry); done: return true; @@ -3036,7 +3036,7 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) map.size = entry.addr_mask; iova_tree_remove(as->iova_tree, &map); - memory_region_notify_one(n, &entry); + memory_region_iotlb_notify_one(n, &entry); } static void vtd_address_space_unmap_all(IntelIOMMUState *s) @@ -3053,7 +3053,7 @@ static void vtd_address_space_unmap_all(IntelIOMMUState *s) static int vtd_replay_hook(IOMMUTLBEntry *entry, void *private) { - memory_region_notify_one((IOMMUNotifier *)private, entry); + memory_region_iotlb_notify_one((IOMMUNotifier *)private, entry); return 0; } diff --git a/hw/misc/tz-mpc.c b/hw/misc/tz-mpc.c index e0c58ba37e..a585e7b475 100644 --- a/hw/misc/tz-mpc.c +++ b/hw/misc/tz-mpc.c @@ -100,8 +100,8 @@ static void tz_mpc_iommu_notify(TZMPC *s, uint32_t lutidx, entry.translated_addr = addr; entry.perm = IOMMU_NONE; - memory_region_notify_iommu(&s->upstream, IOMMU_IDX_S, entry); - memory_region_notify_iommu(&s->upstream, IOMMU_IDX_NS, entry); + memory_region_iotlb_notify_iommu(&s->upstream, IOMMU_IDX_S, entry); + memory_region_iotlb_notify_iommu(&s->upstream, IOMMU_IDX_NS, entry); entry.perm = IOMMU_RW; if (block_is_ns) { @@ -109,13 +109,13 @@ static void tz_mpc_iommu_notify(TZMPC *s, uint32_t lutidx, } else { entry.target_as = &s->downstream_as; } - memory_region_notify_iommu(&s->upstream, IOMMU_IDX_S, entry); + memory_region_iotlb_notify_iommu(&s->upstream, IOMMU_IDX_S, entry); if (block_is_ns) { entry.target_as = &s->downstream_as; } else { entry.target_as = &s->blocked_io_as; } - memory_region_notify_iommu(&s->upstream, IOMMU_IDX_NS, entry); + memory_region_iotlb_notify_iommu(&s->upstream, IOMMU_IDX_NS, entry); } } diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c index 1b0880ac9e..680fb6ad24 100644 --- a/hw/ppc/spapr_iommu.c +++ b/hw/ppc/spapr_iommu.c @@ -429,7 +429,7 @@ static target_ulong put_tce_emu(sPAPRTCETable *tcet, target_ulong ioba, entry.translated_addr = tce & page_mask; entry.addr_mask = ~page_mask; entry.perm = spapr_tce_iommu_access_flags(tce); - memory_region_notify_iommu(&tcet->iommu, 0, entry); + memory_region_iotlb_notify_iommu(&tcet->iommu, 0, entry); return H_SUCCESS; } diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 7b61367ee3..a5ad72d19c 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -589,7 +589,7 @@ static void s390_pci_update_iotlb(S390PCIIOMMU *iommu, S390IOTLBEntry *entry) } notify.perm = IOMMU_NONE; - memory_region_notify_iommu(&iommu->iommu_mr, 0, notify); + memory_region_iotlb_notify_iommu(&iommu->iommu_mr, 0, notify); notify.perm = entry->perm; } @@ -601,7 +601,7 @@ static void s390_pci_update_iotlb(S390PCIIOMMU *iommu, S390IOTLBEntry *entry) g_hash_table_replace(iommu->iotlb, &cache->iova, cache); } - memory_region_notify_iommu(&iommu->iommu_mr, 0, notify); + memory_region_iotlb_notify_iommu(&iommu->iommu_mr, 0, notify); } int rpcit_service_call(S390CPU *cpu, uint8_t r1, uint8_t r2, uintptr_t ra) diff --git a/include/exec/memory.h b/include/exec/memory.h index 31fc859c6b..5ef9bf6d21 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1024,10 +1024,10 @@ static inline IOMMUMemoryRegionClass *memory_region_get_iommu_class_nocheck( uint64_t memory_region_iommu_get_min_page_size(IOMMUMemoryRegion *iommu_mr); /** - * memory_region_notify_iommu: notify a change in an IOMMU translation entry. + * memory_region_iotlb_notify_iommu: notify a change in an IOMMU translation + * entry. * * The notification type will be decided by entry.perm bits: - * * - For UNMAP (cache invalidation) notifies: set entry.perm to IOMMU_NONE. * - For MAP (newly added entry) notifies: set entry.perm to the * permission of the page (which is definitely !IOMMU_NONE). @@ -1041,15 +1041,15 @@ uint64_t memory_region_iommu_get_min_page_size(IOMMUMemoryRegion *iommu_mr); * replaces all old entries for the same virtual I/O address range. * Deleted entries have .@perm == 0. */ -void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, - int iommu_idx, - IOMMUTLBEntry entry); +void memory_region_iotlb_notify_iommu(IOMMUMemoryRegion *iommu_mr, + int iommu_idx, + IOMMUTLBEntry entry); /** - * memory_region_notify_one: notify a change in an IOMMU translation - * entry to a single notifier + * memory_region_iotlb_notify_one: notify a change in an IOMMU translation + * entry to a single notifier * - * This works just like memory_region_notify_iommu(), but it only + * This works just like memory_region_iotlb_notify_iommu(), but it only * notifies a specific notifier, not all of them. * * @notifier: the notifier to be notified @@ -1057,8 +1057,8 @@ void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, * replaces all old entries for the same virtual I/O address range. * Deleted entries have .@perm == 0. */ -void memory_region_notify_one(IOMMUNotifier *notifier, - IOMMUTLBEntry *entry); +void memory_region_iotlb_notify_one(IOMMUNotifier *notifier, + IOMMUTLBEntry *entry); /** * memory_region_register_iommu_notifier: register a notifier for changes to diff --git a/memory.c b/memory.c index b7e2e43b68..8ee5cbdbad 100644 --- a/memory.c +++ b/memory.c @@ -1870,8 +1870,8 @@ void memory_region_unregister_iommu_notifier(MemoryRegion *mr, memory_region_update_iommu_notify_flags(iommu_mr); } -void memory_region_notify_one(IOMMUNotifier *notifier, - IOMMUTLBEntry *entry) +void memory_region_iotlb_notify_one(IOMMUNotifier *notifier, + IOMMUTLBEntry *entry) { IOMMUNotifierFlag request_flags; @@ -1895,9 +1895,9 @@ void memory_region_notify_one(IOMMUNotifier *notifier, } } -void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, - int iommu_idx, - IOMMUTLBEntry entry) +void memory_region_iotlb_notify_iommu(IOMMUMemoryRegion *iommu_mr, + int iommu_idx, + IOMMUTLBEntry entry) { IOMMUNotifier *iommu_notifier; @@ -1905,7 +1905,7 @@ void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, IOMMU_NOTIFIER_FOREACH(iommu_notifier, iommu_mr) { if (iommu_notifier->iommu_idx == iommu_idx) { - memory_region_notify_one(iommu_notifier, &entry); + memory_region_iotlb_notify_one(iommu_notifier, &entry); } } } From patchwork Fri Sep 21 08:18:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972946 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnCw4tJsz9sCD for ; Fri, 21 Sep 2018 18:42:36 +1000 (AEST) Received: from localhost ([::1]:54498 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3H1G-0008Km-5f for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:42:34 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47811) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GfQ-0004T1-Uy for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GfP-0004ES-P7 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38174) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GfL-00045n-2o; Fri, 21 Sep 2018 04:19:55 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 57C7E30832EF; Fri, 21 Sep 2018 08:19:54 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 109017D665; Fri, 21 Sep 2018 08:19:51 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:02 +0200 Message-Id: <20180921081819.9203-12-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 21 Sep 2018 08:19:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 11/28] memory: Add IOMMUConfigNotifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" With this patch, an IOMMUNotifier can now be either an IOTLB notifier or a config notifier. A config notifier is supposed to be called on guest translation config change. This gives host a chance to update the physical IOMMU configuration so that is consistent with the guest view. The notifier is passed an IOMMUConfig. The first type of configuration introduced here consists in the PASID configuration. We introduce the associated helpers, iommu_config_notifier_init, memory_region_config_notify_iommu Signed-off-by: Eric Auger --- v1 -> v2: - use pasid_table config - pass IOMMUNotifierFlag flags to iommu_config_notifier_init to prepare for other config flags - Introduce IOMMUConfig - s/IOMMU_NOTIFIER_S1_CFG/IOMMU_NOTIFIER_PASID_CFG - remove unused IOMMUStage1ConfigType --- hw/vfio/common.c | 14 +++++++--- include/exec/memory.h | 62 +++++++++++++++++++++++++++++++++++++++++-- memory.c | 34 ++++++++++++++++++++++-- 3 files changed, 102 insertions(+), 8 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 19dfc279d6..5dcb502f42 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -624,10 +624,16 @@ static void vfio_listener_region_del(MemoryListener *listener, VFIOGuestIOMMU *giommu; QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { - if (MEMORY_REGION(giommu->iommu) == section->mr && - giommu->n.iotlb_notifier.start == section->offset_within_region) { - memory_region_unregister_iommu_notifier(section->mr, - &giommu->n); + if (MEMORY_REGION(giommu->iommu) == section->mr) { + if (is_iommu_iotlb_notifier(&giommu->n) && + giommu->n.iotlb_notifier.start == + section->offset_within_region) { + memory_region_unregister_iommu_notifier(section->mr, + &giommu->n); + } else if (is_iommu_config_notifier(&giommu->n)) { + memory_region_unregister_iommu_notifier(section->mr, + &giommu->n); + } QLIST_REMOVE(giommu, giommu_next); g_free(giommu); break; diff --git a/include/exec/memory.h b/include/exec/memory.h index 5ef9bf6d21..d4486d4e6b 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -26,6 +26,9 @@ #include "qom/object.h" #include "qemu/rcu.h" #include "hw/qdev-core.h" +#ifdef CONFIG_LINUX +#include +#endif #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -74,6 +77,14 @@ struct IOMMUTLBEntry { IOMMUAccessFlags perm; }; +typedef struct IOMMUConfig { + union { +#ifdef __linux__ + struct iommu_pasid_table_config pasid_cfg; +#endif + }; +} IOMMUConfig; + /* * Bitmap for different IOMMUNotifier capabilities. Each notifier can * register with one or multiple IOMMU Notifier capability bit(s). @@ -84,13 +95,19 @@ typedef enum { IOMMU_NOTIFIER_UNMAP = 0x1, /* Notify entry changes (newly created entries) */ IOMMU_NOTIFIER_MAP = 0x2, + /* Notify stage 1 config changes */ + IOMMU_NOTIFIER_PASID_CFG = 0x4, } IOMMUNotifierFlag; #define IOMMU_NOTIFIER_IOTLB_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) +#define IOMMU_NOTIFIER_CONFIG_ALL (IOMMU_NOTIFIER_PASID_CFG) struct IOMMUNotifier; +struct IOMMUConfig; typedef void (*IOMMUNotify)(struct IOMMUNotifier *notifier, IOMMUTLBEntry *data); +typedef void (*IOMMUConfigNotify)(struct IOMMUNotifier *notifier, + IOMMUConfig *cfg); typedef struct IOMMUIOLTBNotifier { IOMMUNotify notify; @@ -99,9 +116,16 @@ typedef struct IOMMUIOLTBNotifier { hwaddr end; } IOMMUIOLTBNotifier; +typedef struct IOMMUConfigNotifier { + IOMMUConfigNotify notify; +} IOMMUConfigNotifier; + struct IOMMUNotifier { IOMMUNotifierFlag notifier_flags; - IOMMUIOLTBNotifier iotlb_notifier; + union { + IOMMUIOLTBNotifier iotlb_notifier; + IOMMUConfigNotifier config_notifier; + }; int iommu_idx; QLIST_ENTRY(IOMMUNotifier) node; }; @@ -143,6 +167,16 @@ static inline void iommu_iotlb_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, n->iommu_idx = iommu_idx; } +static inline void iommu_config_notifier_init(IOMMUNotifier *n, + IOMMUConfigNotify fn, + IOMMUNotifierFlag flags, + int iommu_idx) +{ + n->notifier_flags = flags; + n->iommu_idx = iommu_idx; + n->config_notifier.notify = fn; +} + /* * Memory region callbacks */ @@ -639,6 +673,17 @@ void memory_region_init_resizeable_ram(MemoryRegion *mr, uint64_t length, void *host), Error **errp); + +static inline bool is_iommu_iotlb_notifier(IOMMUNotifier *n) +{ + return n->notifier_flags & IOMMU_NOTIFIER_IOTLB_ALL; +} + +static inline bool is_iommu_config_notifier(IOMMUNotifier *n) +{ + return n->notifier_flags & IOMMU_NOTIFIER_CONFIG_ALL; +} + #ifdef __linux__ /** @@ -1045,6 +1090,19 @@ void memory_region_iotlb_notify_iommu(IOMMUMemoryRegion *iommu_mr, int iommu_idx, IOMMUTLBEntry entry); +/** + * memory_region_config_notify_iommu: notify a change in a translation + * configuration structure. + * @iommu_mr: the memory region that was changed + * @iommu_idx: the IOMMU index for the translation table which has changed + * @flag: config change type + * @config: new guest config + */ +void memory_region_config_notify_iommu(IOMMUMemoryRegion *iommu_mr, + int iommu_idx, + IOMMUNotifierFlag flag, + IOMMUConfig *config); + /** * memory_region_iotlb_notify_one: notify a change in an IOMMU translation * entry to a single notifier @@ -1062,7 +1120,7 @@ void memory_region_iotlb_notify_one(IOMMUNotifier *notifier, /** * memory_region_register_iommu_notifier: register a notifier for changes to - * IOMMU translation entries. + * IOMMU translation entries or translation config settings. * * @mr: the memory region to observe * @n: the IOMMUNotifier to be added; the notify callback receives a diff --git a/memory.c b/memory.c index 8ee5cbdbad..2c320bc14b 100644 --- a/memory.c +++ b/memory.c @@ -49,6 +49,8 @@ static GHashTable *flat_views; typedef struct AddrRange AddrRange; +struct iommu_pasid_table_config; + /* * Note that signed integers are needed for negative offsetting in aliases * (large MemoryRegion::alias_offset). @@ -1800,7 +1802,9 @@ void memory_region_register_iommu_notifier(MemoryRegion *mr, /* We need to register for at least one bitfield */ iommu_mr = IOMMU_MEMORY_REGION(mr); assert(n->notifier_flags != IOMMU_NOTIFIER_NONE); - assert(n->iotlb_notifier.start <= n->iotlb_notifier.end); + if (is_iommu_iotlb_notifier(n)) { + assert(n->iotlb_notifier.start <= n->iotlb_notifier.end); + } assert(n->iommu_idx >= 0 && n->iommu_idx < memory_region_iommu_num_indexes(iommu_mr)); @@ -1870,6 +1874,13 @@ void memory_region_unregister_iommu_notifier(MemoryRegion *mr, memory_region_update_iommu_notify_flags(iommu_mr); } +static void +memory_region_config_notify_one(IOMMUNotifier *notifier, + IOMMUConfig *cfg) +{ + notifier->config_notifier.notify(notifier, cfg); +} + void memory_region_iotlb_notify_one(IOMMUNotifier *notifier, IOMMUTLBEntry *entry) { @@ -1904,12 +1915,31 @@ void memory_region_iotlb_notify_iommu(IOMMUMemoryRegion *iommu_mr, assert(memory_region_is_iommu(MEMORY_REGION(iommu_mr))); IOMMU_NOTIFIER_FOREACH(iommu_notifier, iommu_mr) { - if (iommu_notifier->iommu_idx == iommu_idx) { + if (iommu_notifier->iommu_idx == iommu_idx && + is_iommu_iotlb_notifier(iommu_notifier)) { memory_region_iotlb_notify_one(iommu_notifier, &entry); } } } +void memory_region_config_notify_iommu(IOMMUMemoryRegion *iommu_mr, + int iommu_idx, + IOMMUNotifierFlag flag, + IOMMUConfig *config) +{ + IOMMUNotifier *iommu_notifier; + + assert(memory_region_is_iommu(MEMORY_REGION(iommu_mr))); + + IOMMU_NOTIFIER_FOREACH(iommu_notifier, iommu_mr) { + if (iommu_notifier->iommu_idx == iommu_idx && + is_iommu_config_notifier(iommu_notifier) && + iommu_notifier->notifier_flags == flag) { + memory_region_config_notify_one(iommu_notifier, config); + } + } +} + int memory_region_iommu_get_attr(IOMMUMemoryRegion *iommu_mr, enum IOMMUMemoryRegionAttr attr, void *data) From patchwork Fri Sep 21 08:18:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972932 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn166Hv5z9sBq for ; Fri, 21 Sep 2018 18:33:14 +1000 (AEST) Received: from localhost ([::1]:54436 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GsC-0008Gx-D1 for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:33:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47981) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gfz-00051N-U0 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gft-00057H-Bp for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38272) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gfd-0004SX-BV; Fri, 21 Sep 2018 04:20:15 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 461DD30832C5; Fri, 21 Sep 2018 08:20:09 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id A25EF7D665; Fri, 21 Sep 2018 08:19:54 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:03 +0200 Message-Id: <20180921081819.9203-13-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 21 Sep 2018 08:20:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 12/28] memory: Add arch_id in IOTLBEntry X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" TLB entries are usually tagged with some ids such as the asid or pasid. When propagating an invalidation command from the guest to the host, we need to pass this id. Signed-off-by: Eric Auger --- include/exec/memory.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index d4486d4e6b..93150e1450 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -75,6 +75,7 @@ struct IOMMUTLBEntry { hwaddr translated_addr; hwaddr addr_mask; /* 0xfff = 4k translation */ IOMMUAccessFlags perm; + uint32_t arch_id; /* architecture specific ID tagging the TLB */ }; typedef struct IOMMUConfig { From patchwork Fri Sep 21 08:18:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972943 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn8f6SGxz9sC7 for ; Fri, 21 Sep 2018 18:39:46 +1000 (AEST) Received: from localhost ([::1]:54477 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GyW-0005pp-EN for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:39:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47949) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gfx-0004z8-6k for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:34 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gft-00057I-C5 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:31 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40502) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gfl-0004Vb-Nt; Fri, 21 Sep 2018 04:20:22 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E9B753082B03; Fri, 21 Sep 2018 08:20:11 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 902E97D665; Fri, 21 Sep 2018 08:20:09 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:04 +0200 Message-Id: <20180921081819.9203-14-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Fri, 21 Sep 2018 08:20:12 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 13/28] hw/arm/smmuv3: Store s1ctrptr in translation config data X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For VFIO integration we will need to pass the context descriptor table GPA to the host. So let's decode and store it. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 1 + include/hw/arm/smmu-common.h | 1 + 2 files changed, 2 insertions(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index b8c41916aa..afe1758830 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -351,6 +351,7 @@ static int decode_ste(SMMUv3State *s, SMMUTransCfg *cfg, "SMMUv3 S1 stalling fault model not allowed yet\n"); goto bad_ste; } + cfg->s1ctxptr = STE_CTXPTR(ste); return 0; bad_ste: diff --git a/include/hw/arm/smmu-common.h b/include/hw/arm/smmu-common.h index b07cadd0ef..52073a23b4 100644 --- a/include/hw/arm/smmu-common.h +++ b/include/hw/arm/smmu-common.h @@ -68,6 +68,7 @@ typedef struct SMMUTransCfg { uint8_t tbi; /* Top Byte Ignore */ uint16_t asid; SMMUTransTableInfo tt[2]; + dma_addr_t s1ctxptr; uint32_t iotlb_hits; /* counts IOTLB hits for this asid */ uint32_t iotlb_misses; /* counts IOTLB misses for this asid */ } SMMUTransCfg; From patchwork Fri Sep 21 08:18:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972923 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GmxW2XnQz9sBq for ; Fri, 21 Sep 2018 18:30:05 +1000 (AEST) Received: from localhost ([::1]:54414 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gp9-0005Y6-Bs for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:30:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48033) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gg4-00055L-1j for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:43 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gg1-0005On-7p for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:38 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38346) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gft-00055N-E7; Fri, 21 Sep 2018 04:20:29 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EFAB3C04AC51; Fri, 21 Sep 2018 08:20:27 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 397CF7D665; Fri, 21 Sep 2018 08:20:12 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:05 +0200 Message-Id: <20180921081819.9203-15-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 21 Sep 2018 08:20:28 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 14/28] hw/arm/smmuv3: Implement dummy replay X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The default implementation of memory_region_iommu_replay() shall not be used as it forces the translation of the whole RAM range. The purpose of this function is to update the shadow page tables. However in case of nested stage, there is no shadow page table so we can simply return. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index afe1758830..d609038724 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -1516,6 +1516,11 @@ static int smmuv3_get_attr(IOMMUMemoryRegion *iommu, return -EINVAL; } +static inline void +smmuv3_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) +{ +} + static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, void *data) { @@ -1524,6 +1529,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, imrc->translate = smmuv3_translate; imrc->notify_flag_changed = smmuv3_notify_flag_changed; imrc->get_attr = smmuv3_get_attr; + imrc->replay = smmuv3_replay; } static const TypeInfo smmuv3_type_info = { From patchwork Fri Sep 21 08:18:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972938 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn4d4p9Sz9sBq for ; Fri, 21 Sep 2018 18:36:16 +1000 (AEST) Received: from localhost ([::1]:54454 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gv5-0002oF-Vh for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:36:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48087) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GgA-00057b-2y for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:47 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gg8-0005Y7-4f for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:45 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38384) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gfz-0005Ca-Aa; Fri, 21 Sep 2018 04:20:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4E6CDC04F4C4; Fri, 21 Sep 2018 08:20:32 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 47C227D665; Fri, 21 Sep 2018 08:20:28 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:06 +0200 Message-Id: <20180921081819.9203-16-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Fri, 21 Sep 2018 08:20:32 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 15/28] hw/arm/smmuv3: Notify on config changes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" In case IOMMU config notifiers are attached to the IOMMU memory region, we execute them, passing as argument the iommu_pasid_table_config struct updated with the new viommu translation config. Config notifiers are called on STE changes. At physical level, they translate into CMD_CFGI_STE_* commands. Signed-off-by: Eric Auger --- v1 -> v2: - do not notify anymore on CD change. Anyway the smmuv3 linux driver is not sending any CD invalidation commands. If we were to propagate CD invalidation commands, we would use the CACHE_INVALIDATE VFIO ioctl. - notify a precise config flags to prepare for addition of new flags --- hw/arm/smmuv3.c | 61 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 44 insertions(+), 17 deletions(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index d609038724..196835739f 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -16,6 +16,8 @@ * with this program; if not, see . */ +#include "linux/iommu.h" + #include "qemu/osdep.h" #include "hw/boards.h" #include "sysemu/sysemu.h" @@ -843,6 +845,46 @@ static void smmuv3_inv_notifiers_iova(SMMUState *s, int asid, dma_addr_t iova) } } +static void smmuv3_notify_config_change(SMMUState *bs, uint32_t sid) +{ + IOMMUMemoryRegion *mr = smmu_iommu_mr(bs, sid); + SMMUEventInfo event = {.type = SMMU_EVT_NONE, .sid = sid}; + SMMUTransCfg *cfg; + SMMUDevice *sdev; + + if (!mr) { + return; + } + + sdev = container_of(mr, SMMUDevice, iommu); + + /* flush QEMU config cache */ + smmuv3_flush_config(sdev); + + if (mr->iommu_notify_flags & IOMMU_NOTIFIER_PASID_CFG) { + /* force a guest RAM config structure decoding */ + cfg = smmuv3_get_config(sdev, &event); + + if (cfg) { + IOMMUConfig *iommu_config = g_new0(IOMMUConfig, 1); + + iommu_config->pasid_cfg.format = IOMMU_PASID_FORMAT_SMMUV3; + iommu_config->pasid_cfg.smmuv3.bypass = + cfg->disabled || cfg->bypassed; + iommu_config->pasid_cfg.smmuv3.abort = cfg->aborted; + iommu_config->pasid_cfg.smmuv3.s1contextptr = cfg->s1ctxptr; + + memory_region_config_notify_iommu(mr, 0, IOMMU_NOTIFIER_PASID_CFG, + iommu_config); + g_free(iommu_config); + } else { + qemu_log_mask(LOG_GUEST_ERROR, + "%s error decoding the configuration for iommu mr=%s\n", + __func__, mr->parent_obj.name); + } + } +} + static int smmuv3_cmdq_consume(SMMUv3State *s) { SMMUState *bs = ARM_SMMU(s); @@ -893,22 +935,14 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) case SMMU_CMD_CFGI_STE: { uint32_t sid = CMD_SID(&cmd); - IOMMUMemoryRegion *mr = smmu_iommu_mr(bs, sid); - SMMUDevice *sdev; if (CMD_SSEC(&cmd)) { cmd_error = SMMU_CERROR_ILL; break; } - if (!mr) { - break; - } - trace_smmuv3_cmdq_cfgi_ste(sid); - sdev = container_of(mr, SMMUDevice, iommu); - smmuv3_flush_config(sdev); - + smmuv3_notify_config_change(bs, sid); break; } case SMMU_CMD_CFGI_STE_RANGE: /* same as SMMU_CMD_CFGI_ALL */ @@ -925,14 +959,7 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) trace_smmuv3_cmdq_cfgi_ste_range(start, end); for (i = start; i <= end; i++) { - IOMMUMemoryRegion *mr = smmu_iommu_mr(bs, i); - SMMUDevice *sdev; - - if (!mr) { - continue; - } - sdev = container_of(mr, SMMUDevice, iommu); - smmuv3_flush_config(sdev); + smmuv3_notify_config_change(bs, i); } break; } From patchwork Fri Sep 21 08:18:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972949 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnGw3tR0z9sC7 for ; Fri, 21 Sep 2018 18:45:12 +1000 (AEST) Received: from localhost ([::1]:54513 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3H3l-000391-TL for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:45:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48076) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gg8-00057a-Sl for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gg8-0005YH-76 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51324) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Gg0-0005JR-79; Fri, 21 Sep 2018 04:20:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E6F5931524C5; Fri, 21 Sep 2018 08:20:34 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99FA87D665; Fri, 21 Sep 2018 08:20:32 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:07 +0200 Message-Id: <20180921081819.9203-17-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Fri, 21 Sep 2018 08:20:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 16/28] hw/arm/smmuv3: Fill the IOTLBEntry arch_id on NH_VA invalidation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When the guest invalidates one S1 entry, it passes the asid. When propagating this invalidation downto the host, the asid information also must be passed. So let's fill the arch_id field introduced for that purpose. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 196835739f..1d8deae37f 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -824,6 +824,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, entry.iova = iova; entry.addr_mask = (1 << tt->granule_sz) - 1; entry.perm = IOMMU_NONE; + entry.arch_id = asid; memory_region_iotlb_notify_one(n, &entry); } From patchwork Fri Sep 21 08:18:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972941 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn7z4YP4z9sBJ for ; Fri, 21 Sep 2018 18:39:11 +1000 (AEST) Received: from localhost ([::1]:54472 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gxx-0005IL-6o for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:39:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48184) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GgL-0005Gw-K6 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GgK-0006G1-Kk for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47316) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GgD-0005k8-Pu; Fri, 21 Sep 2018 04:20:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8234488312; Fri, 21 Sep 2018 08:20:48 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3C1597D904; Fri, 21 Sep 2018 08:20:35 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:08 +0200 Message-Id: <20180921081819.9203-18-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 21 Sep 2018 08:20:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 17/28] hw/vfio/common: Introduce vfio_alloc_guest_iommu helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Soon this code will be called several times. So let's introduce an helper. Signed-off-by: Eric Auger --- hw/vfio/common.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 5dcb502f42..ce13c102a5 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -24,6 +24,7 @@ #include #endif #include +#include #include "hw/vfio/vfio-common.h" #include "hw/vfio/vfio.h" @@ -401,6 +402,19 @@ out: rcu_read_unlock(); } +static VFIOGuestIOMMU *vfio_alloc_guest_iommu(VFIOContainer *container, + IOMMUMemoryRegion *iommu, + hwaddr offset) +{ + VFIOGuestIOMMU *giommu = g_new0(VFIOGuestIOMMU, 1); + + giommu->container = container; + giommu->iommu = iommu; + giommu->iommu_offset = offset; + /* notifier will be registered separately */ + return giommu; +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -508,6 +522,7 @@ static void vfio_listener_region_add(MemoryListener *listener, if (memory_region_is_iommu(section->mr)) { VFIOGuestIOMMU *giommu; IOMMUMemoryRegion *iommu_mr = IOMMU_MEMORY_REGION(section->mr); + hwaddr offset; int iommu_idx; trace_vfio_listener_region_add_iommu(iova, end); @@ -517,11 +532,11 @@ static void vfio_listener_region_add(MemoryListener *listener, * would be the right place to wire that up (tell the KVM * device emulation the VFIO iommu handles to use). */ - giommu = g_malloc0(sizeof(*giommu)); - giommu->iommu = iommu_mr; - giommu->iommu_offset = section->offset_within_address_space - - section->offset_within_region; - giommu->container = container; + + offset = section->offset_within_address_space - + section->offset_within_region; + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + llend = int128_add(int128_make64(section->offset_within_region), section->size); llend = int128_sub(llend, int128_one()); From patchwork Fri Sep 21 08:18:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972933 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42Gn1H4L1fz9sBq for ; Fri, 21 Sep 2018 18:33:23 +1000 (AEST) Received: from localhost ([::1]:54438 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GsL-0008Od-3C for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:33:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48247) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GgN-0005J5-V1 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GgM-0006LQ-Gf for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:20:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53812) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GgJ-0005sh-Ci; Fri, 21 Sep 2018 04:20:55 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 206BBC04C27D; Fri, 21 Sep 2018 08:20:51 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id CCC727D665; Fri, 21 Sep 2018 08:20:48 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:09 +0200 Message-Id: <20180921081819.9203-19-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Fri, 21 Sep 2018 08:20:51 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 18/28] hw/vfio/common: Introduce vfio_dma_(un)map_ram_section helpers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This code is going to be duplicated soon, so let's introduce an helper which dma (unp)maps a ram memory section. No functional change. Signed-off-by: Eric Auger --- hw/vfio/common.c | 198 ++++++++++++++++++++++++++----------------- hw/vfio/trace-events | 4 +- 2 files changed, 123 insertions(+), 79 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index ce13c102a5..83f5f2263d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -415,13 +415,130 @@ static VFIOGuestIOMMU *vfio_alloc_guest_iommu(VFIOContainer *container, return giommu; } +static int vfio_dma_map_ram_section(VFIOContainer *container, + MemoryRegionSection *section) +{ + VFIOHostDMAWindow *hostwin; + Int128 llend, llsize; + bool hostwin_found; + hwaddr iova, end; + void *vaddr; + int ret; + + assert(memory_region_is_ram(section->mr)); + + iova = TARGET_PAGE_ALIGN(section->offset_within_address_space); + llend = int128_make64(section->offset_within_address_space); + llend = int128_add(llend, section->size); + llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK)); + end = int128_get64(int128_sub(llend, int128_one())); + + vaddr = memory_region_get_ram_ptr(section->mr) + + section->offset_within_region + + (iova - section->offset_within_address_space); + + hostwin_found = false; + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { + if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { + hostwin_found = true; + break; + } + } + + if (!hostwin_found) { + error_report("vfio: IOMMU container %p can't map guest IOVA region" + " 0x%"HWADDR_PRIx"..0x%"HWADDR_PRIx, + container, iova, end); + return -EFAULT; + } + + trace_vfio_dma_map_ram(iova, end, vaddr); + + llsize = int128_sub(llend, int128_make64(iova)); + + if (memory_region_is_ram_device(section->mr)) { + hwaddr pgmask = (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; + + if ((iova & pgmask) || (int128_get64(llsize) & pgmask)) { + trace_vfio_listener_region_add_no_dma_map( + memory_region_name(section->mr), + section->offset_within_address_space, + int128_getlo(section->size), + pgmask + 1); + return 0; + } + } + + ret = vfio_dma_map(container, iova, int128_get64(llsize), + vaddr, section->readonly); + if (ret) { + error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " + "0x%"HWADDR_PRIx", %p) = %d (%m)", + container, iova, int128_get64(llsize), vaddr, ret); + if (memory_region_is_ram_device(section->mr)) { + /* Allow unexpected mappings not to be fatal for RAM devices */ + return 0; + } + return -EINVAL; + } + return 0; +} + +static void vfio_dma_unmap_ram_section(VFIOContainer *container, + MemoryRegionSection *section) +{ + Int128 llend, llsize; + hwaddr iova, end; + bool try_unmap = true; + int ret; + + iova = TARGET_PAGE_ALIGN(section->offset_within_address_space); + llend = int128_make64(section->offset_within_address_space); + llend = int128_add(llend, section->size); + llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK)); + + if (int128_ge(int128_make64(iova), llend)) { + return; + } + end = int128_get64(int128_sub(llend, int128_one())); + + llsize = int128_sub(llend, int128_make64(iova)); + + trace_vfio_dma_unmap_ram(iova, end); + + if (memory_region_is_ram_device(section->mr)) { + hwaddr pgmask; + VFIOHostDMAWindow *hostwin; + bool hostwin_found = false; + + QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { + if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { + hostwin_found = true; + break; + } + } + assert(hostwin_found); /* or region_add() would have failed */ + + pgmask = (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; + try_unmap = !((iova & pgmask) || (int128_get64(llsize) & pgmask)); + } + + if (try_unmap) { + ret = vfio_dma_unmap(container, iova, int128_get64(llsize)); + if (ret) { + error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " + "0x%"HWADDR_PRIx") = %d (%m)", + container, iova, int128_get64(llsize), ret); + } + } +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); hwaddr iova, end; - Int128 llend, llsize; - void *vaddr; + Int128 llend; int ret; VFIOHostDMAWindow *hostwin; bool hostwin_found; @@ -556,41 +673,10 @@ static void vfio_listener_region_add(MemoryListener *listener, } /* Here we assume that memory_region_is_ram(section->mr)==true */ - - vaddr = memory_region_get_ram_ptr(section->mr) + - section->offset_within_region + - (iova - section->offset_within_address_space); - - trace_vfio_listener_region_add_ram(iova, end, vaddr); - - llsize = int128_sub(llend, int128_make64(iova)); - - if (memory_region_is_ram_device(section->mr)) { - hwaddr pgmask = (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; - - if ((iova & pgmask) || (int128_get64(llsize) & pgmask)) { - trace_vfio_listener_region_add_no_dma_map( - memory_region_name(section->mr), - section->offset_within_address_space, - int128_getlo(section->size), - pgmask + 1); - return; - } - } - - ret = vfio_dma_map(container, iova, int128_get64(llsize), - vaddr, section->readonly); + ret = vfio_dma_map_ram_section(container, section); if (ret) { - error_report("vfio_dma_map(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx", %p) = %d (%m)", - container, iova, int128_get64(llsize), vaddr, ret); - if (memory_region_is_ram_device(section->mr)) { - /* Allow unexpected mappings not to be fatal for RAM devices */ - return; - } goto fail; } - return; fail: @@ -616,10 +702,6 @@ static void vfio_listener_region_del(MemoryListener *listener, MemoryRegionSection *section) { VFIOContainer *container = container_of(listener, VFIOContainer, listener); - hwaddr iova, end; - Int128 llend, llsize; - int ret; - bool try_unmap = true; if (vfio_listener_skipped_section(section)) { trace_vfio_listener_region_del_skip( @@ -664,45 +746,7 @@ static void vfio_listener_region_del(MemoryListener *listener, */ } - iova = TARGET_PAGE_ALIGN(section->offset_within_address_space); - llend = int128_make64(section->offset_within_address_space); - llend = int128_add(llend, section->size); - llend = int128_and(llend, int128_exts64(TARGET_PAGE_MASK)); - - if (int128_ge(int128_make64(iova), llend)) { - return; - } - end = int128_get64(int128_sub(llend, int128_one())); - - llsize = int128_sub(llend, int128_make64(iova)); - - trace_vfio_listener_region_del(iova, end); - - if (memory_region_is_ram_device(section->mr)) { - hwaddr pgmask; - VFIOHostDMAWindow *hostwin; - bool hostwin_found = false; - - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { - hostwin_found = true; - break; - } - } - assert(hostwin_found); /* or region_add() would have failed */ - - pgmask = (1ULL << ctz64(hostwin->iova_pgsizes)) - 1; - try_unmap = !((iova & pgmask) || (int128_get64(llsize) & pgmask)); - } - - if (try_unmap) { - ret = vfio_dma_unmap(container, iova, int128_get64(llsize)); - if (ret) { - error_report("vfio_dma_unmap(%p, 0x%"HWADDR_PRIx", " - "0x%"HWADDR_PRIx") = %d (%m)", - container, iova, int128_get64(llsize), ret); - } - } + vfio_dma_unmap_ram_section(container, section); memory_region_unref(section->mr); diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events index a85e8662ea..cee49ef124 100644 --- a/hw/vfio/trace-events +++ b/hw/vfio/trace-events @@ -93,10 +93,10 @@ vfio_region_read(char *name, int index, uint64_t addr, unsigned size, uint64_t d vfio_iommu_map_notify(const char *op, uint64_t iova_start, uint64_t iova_end) "iommu %s @ 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_add_skip(uint64_t start, uint64_t end) "SKIPPING region_add 0x%"PRIx64" - 0x%"PRIx64 vfio_listener_region_add_iommu(uint64_t start, uint64_t end) "region_add [iommu] 0x%"PRIx64" - 0x%"PRIx64 -vfio_listener_region_add_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]" +vfio_dma_map_ram(uint64_t iova_start, uint64_t iova_end, void *vaddr) "region_add [ram] 0x%"PRIx64" - 0x%"PRIx64" [%p]" vfio_listener_region_add_no_dma_map(const char *name, uint64_t iova, uint64_t size, uint64_t page_size) "Region \"%s\" 0x%"PRIx64" size=0x%"PRIx64" is not aligned to 0x%"PRIx64" and cannot be mapped for DMA" vfio_listener_region_del_skip(uint64_t start, uint64_t end) "SKIPPING region_del 0x%"PRIx64" - 0x%"PRIx64 -vfio_listener_region_del(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64 +vfio_dma_unmap_ram(uint64_t start, uint64_t end) "region_del 0x%"PRIx64" - 0x%"PRIx64 vfio_disconnect_container(int fd) "close container->fd=%d" vfio_put_group(int fd) "close group->fd=%d" vfio_get_device(const char * name, unsigned int flags, unsigned int num_regions, unsigned int num_irqs) "Device %s flags: %u, regions: %u, irqs: %u" From patchwork Fri Sep 21 08:18:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972953 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnNR0lJWz9sCD for ; Fri, 21 Sep 2018 18:49:59 +1000 (AEST) Received: from localhost ([::1]:54543 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3H8O-0008Dq-M9 for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:49:56 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48307) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GgR-0005Nf-K8 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GgQ-0006TL-C2 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52996) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GgL-0006GL-8o; Fri, 21 Sep 2018 04:20:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8370F308FBA9; Fri, 21 Sep 2018 08:20:56 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 687F07D665; Fri, 21 Sep 2018 08:20:51 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:10 +0200 Message-Id: <20180921081819.9203-20-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Fri, 21 Sep 2018 08:20:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 19/28] hw/vfio/common: Register specific nested mode notifiers and memory_listener X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" In nested mode, legacy vfio_iommu_map_notify MAP/UNMAP notifier cannot be used anymore. Indeed there is no caching mode in place that allows to trap MAP events. Only configuration change and UNMAP events can be trapped. As such we register - one configuration notifier, whose role is to propagate the configuration update downto the host - one UNMAP notifier, whose role is to propagate the TLB invalidation at physical IOMMU level. Those notifiers propagate the guest stage 1 mappings at physical level. Also as there is no MAP event, the stage 2 mapping is not handled anymore by the vfio_iommu_map_notify notifier. We register a prereg_listener whose role is to dma_(un)map the RAM memory regions. This programs the stage 2. Signed-off-by: Eric Auger --- v1 -> v2: - adapt to uapi changes - pass the asid - pass IOMMU_NOTIFIER_S1_CFG when initializing the config notifier --- hw/vfio/common.c | 167 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 137 insertions(+), 30 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 83f5f2263d..77bdb76ade 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -349,6 +349,65 @@ static bool vfio_get_vaddr(IOMMUTLBEntry *iotlb, void **vaddr, return true; } +/* Program the guest @cfg on physical IOMMU stage 1 (nested mode) */ +static void vfio_iommu_nested_notify(IOMMUNotifier *n, + IOMMUConfig *cfg) +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOContainer *container = giommu->container; + struct vfio_iommu_type1_bind_pasid_table info; + int ret; + + info.argsz = sizeof(info); + info.flags = 0; + memcpy(&info.config, &cfg->pasid_cfg, sizeof(cfg->pasid_cfg)); + + ret = ioctl(container->fd, VFIO_IOMMU_BIND_PASID_TABLE, &info); + if (ret) { + error_report("%s: failed to pass S1 config to the host (%d)", + __func__, ret); + } +} + +/* Propagate a guest invalidation downto the physical IOMMU (nested mode) */ +static void vfio_iommu_unmap_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + hwaddr start = iotlb->iova + giommu->iommu_offset; + + VFIOContainer *container = giommu->container; + struct vfio_iommu_type1_cache_invalidate ustruct; + int ret; + + assert(iotlb->perm == IOMMU_NONE); + + ustruct.argsz = sizeof(ustruct); + ustruct.flags = 0; + ustruct.info.hdr.version = TLB_INV_HDR_VERSION_1; + ustruct.info.hdr.type = IOMMU_INV_TYPE_TLB; + ustruct.info.granularity = IOMMU_INV_NR_GRANU; + ustruct.info.flags = IOMMU_INVALIDATE_GLOBAL_PAGE; + /* 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc. */ + ustruct.info.size = ctz64(~iotlb->addr_mask) - 12; + /* + * TODO: at the moment we invalidate the whole ASID instead + * of invalidating the given nb_pages (nb_pages = 0): + * mask covering the whole GPA range is observed: in this case we shall + * invalidate the whole ASID (NH_ASID) and not induce storm of + * NH_VA commands. + */ + ustruct.info.nr_pages = 0; + ustruct.info.addr = start; + ustruct.info.arch_id = iotlb->arch_id; + + ret = ioctl(container->fd, VFIO_IOMMU_CACHE_INVALIDATE, &ustruct); + if (ret) { + error_report("%s: failed to invalidate CACHE for 0x%"PRIx64 + " mask=0x%"PRIx64" (%d)", + __func__, start, iotlb->addr_mask, ret); + } +} + static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) { VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); @@ -533,6 +592,32 @@ static void vfio_dma_unmap_ram_section(VFIOContainer *container, } } +static void vfio_prereg_listener_region_add(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container = + container_of(listener, VFIOContainer, prereg_listener); + + if (!memory_region_is_ram(section->mr)) { + return; + } + + vfio_dma_map_ram_section(container, section); + +} +static void vfio_prereg_listener_region_del(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIOContainer *container = + container_of(listener, VFIOContainer, prereg_listener); + + if (!memory_region_is_ram(section->mr)) { + return; + } + + vfio_dma_unmap_ram_section(container, section); +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -541,7 +626,6 @@ static void vfio_listener_region_add(MemoryListener *listener, Int128 llend; int ret; VFIOHostDMAWindow *hostwin; - bool hostwin_found; if (vfio_listener_skipped_section(section)) { trace_vfio_listener_region_add_skip( @@ -618,26 +702,10 @@ static void vfio_listener_region_add(MemoryListener *listener, #endif } - hostwin_found = false; - QLIST_FOREACH(hostwin, &container->hostwin_list, hostwin_next) { - if (hostwin->min_iova <= iova && end <= hostwin->max_iova) { - hostwin_found = true; - break; - } - } - - if (!hostwin_found) { - error_report("vfio: IOMMU container %p can't map guest IOVA region" - " 0x%"HWADDR_PRIx"..0x%"HWADDR_PRIx, - container, iova, end); - ret = -EFAULT; - goto fail; - } - memory_region_ref(section->mr); if (memory_region_is_iommu(section->mr)) { - VFIOGuestIOMMU *giommu; + VFIOGuestIOMMU *giommu = NULL; IOMMUMemoryRegion *iommu_mr = IOMMU_MEMORY_REGION(section->mr); hwaddr offset; int iommu_idx; @@ -652,21 +720,40 @@ static void vfio_listener_region_add(MemoryListener *listener, offset = section->offset_within_address_space - section->offset_within_region; - giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); - llend = int128_add(int128_make64(section->offset_within_region), section->size); llend = int128_sub(llend, int128_one()); iommu_idx = memory_region_iommu_attrs_to_index(iommu_mr, MEMTXATTRS_UNSPECIFIED); - iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_map_notify, - IOMMU_NOTIFIER_IOTLB_ALL, - section->offset_within_region, - int128_get64(llend), - iommu_idx); - QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); - memory_region_register_iommu_notifier(section->mr, &giommu->n); + if (container->iommu_type == VFIO_TYPE1_NESTING_IOMMU) { + /* Config notifier to propagate guest stage 1 config changes */ + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + iommu_config_notifier_init(&giommu->n, vfio_iommu_nested_notify, + IOMMU_NOTIFIER_PASID_CFG, iommu_idx); + QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + memory_region_register_iommu_notifier(section->mr, &giommu->n); + + /* IOTLB unmap notifier to propagate guest IOTLB invalidations */ + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_unmap_notify, + IOMMU_NOTIFIER_UNMAP, + section->offset_within_region, + int128_get64(llend), + iommu_idx); + QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + memory_region_register_iommu_notifier(section->mr, &giommu->n); + } else { + /* MAP/UNMAP IOTLB notifier */ + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_map_notify, + IOMMU_NOTIFIER_IOTLB_ALL, + section->offset_within_region, + int128_get64(llend), + iommu_idx); + QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + memory_region_register_iommu_notifier(section->mr, &giommu->n); + } memory_region_iommu_replay(giommu->iommu, &giommu->n); return; @@ -679,7 +766,7 @@ static void vfio_listener_region_add(MemoryListener *listener, } return; -fail: + fail: if (memory_region_is_ram_device(section->mr)) { error_report("failed to vfio_dma_map. pci p2p may not work"); return; @@ -763,15 +850,21 @@ static void vfio_listener_region_del(MemoryListener *listener, } } -static const MemoryListener vfio_memory_listener = { +static MemoryListener vfio_memory_listener = { .region_add = vfio_listener_region_add, .region_del = vfio_listener_region_del, }; +static MemoryListener vfio_memory_prereg_listener = { + .region_add = vfio_prereg_listener_region_add, + .region_del = vfio_prereg_listener_region_del, +}; + static void vfio_listener_release(VFIOContainer *container) { memory_listener_unregister(&container->listener); - if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU) { + if (container->iommu_type == VFIO_SPAPR_TCE_v2_IOMMU || + container->iommu_type == VFIO_TYPE1_NESTING_IOMMU) { memory_listener_unregister(&container->prereg_listener); } } @@ -1262,6 +1355,20 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, } vfio_host_win_add(container, 0, (hwaddr)-1, info.iova_pgsizes); container->pgsizes = info.iova_pgsizes; + + if (container->iommu_type == VFIO_TYPE1_NESTING_IOMMU) { + container->prereg_listener = vfio_memory_prereg_listener; + + memory_listener_register(&container->prereg_listener, + &address_space_memory); + if (container->error) { + memory_listener_unregister(&container->prereg_listener); + ret = container->error; + error_setg(errp, + "RAM memory listener initialization failed for container"); + goto free_container_exit; + } + } break; } case VFIO_SPAPR_TCE_v2_IOMMU: From patchwork Fri Sep 21 08:18:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972962 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnWm1BMgz9sC7 for ; Fri, 21 Sep 2018 18:56:19 +1000 (AEST) Received: from localhost ([::1]:54589 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HEV-0005LA-Pm for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:56:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48453) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Ggm-0005e3-73 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Ggl-00078M-DW for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38576) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggf-0006m1-0L; Fri, 21 Sep 2018 04:21:18 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B24C23083391; Fri, 21 Sep 2018 08:21:11 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id CE25A7D90A; Fri, 21 Sep 2018 08:20:56 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:11 +0200 Message-Id: <20180921081819.9203-21-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Fri, 21 Sep 2018 08:21:11 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 20/28] hw/vfio/common: Register MAP notifier for MSI binding X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Register a MAP notifier to propage MSI stage 1 bindings to the host. When the notifier gets called, we pass the guest stage 1 MSI binding to the host. The host can then build a S2 binding whose entry is the guest MSI doorbell GPA. Signed-off-by: Eric Auger --- hw/vfio/common.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 77bdb76ade..172ddd396f 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -408,6 +408,26 @@ static void vfio_iommu_unmap_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) } } +static void vfio_iommu_msi_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOContainer *container = giommu->container; + struct vfio_iommu_type1_bind_guest_msi ustruct; + int ret; + + ustruct.argsz = sizeof(struct vfio_iommu_type1_bind_guest_msi); + ustruct.flags = 0; + ustruct.binding.iova = iotlb->iova; + ustruct.binding.gpa = iotlb->translated_addr; + ustruct.binding.granule = ctz64(~iotlb->addr_mask); + + ret = ioctl(container->fd, VFIO_IOMMU_BIND_MSI , &ustruct); + if (ret) { + error_report("%s: failed to pass MSI binding (%d)", + __func__, ret); + } +} + static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) { VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); @@ -743,6 +763,15 @@ static void vfio_listener_region_add(MemoryListener *listener, iommu_idx); QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); memory_region_register_iommu_notifier(section->mr, &giommu->n); + + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_msi_map_notify, + IOMMU_NOTIFIER_MAP, + section->offset_within_region, + int128_get64(llend), + iommu_idx); + QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + memory_region_register_iommu_notifier(section->mr, &giommu->n); } else { /* MAP/UNMAP IOTLB notifier */ giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); From patchwork Fri Sep 21 08:18:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972952 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnMb0ptPz9sC7 for ; Fri, 21 Sep 2018 18:49:15 +1000 (AEST) Received: from localhost ([::1]:54540 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3H7g-0007fX-Lm for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:49:12 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48463) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Ggm-0005eK-Eo for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Ggl-00078o-Jb for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33960) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggh-0006qF-2y; Fri, 21 Sep 2018 04:21:20 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8791E86672; Fri, 21 Sep 2018 08:21:14 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 06B287D904; Fri, 21 Sep 2018 08:21:11 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:12 +0200 Message-Id: <20180921081819.9203-22-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 21 Sep 2018 08:21:14 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 21/28] target/arm/kvm: Notifies IOMMU on MSI stage 1 binding X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When the MSI route is setup, we know the MSI IOVA and the doorbell GPA . At that point we can communicate this guest stage 1 binding to the host. Then the host will be able to construct a stage 2 binding taking as input address the doorbell GPA. We also directly use the iommu memory region translate() callback as the addr_mask is returned in IOTLB entry. address_space_translate does not return this information. Signed-off-by: Eric Auger --- TODO: access to as->root field may be cleaned later on --- target/arm/kvm.c | 46 ++++++++++++++++++++-------------------------- 1 file changed, 20 insertions(+), 26 deletions(-) diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 65f867d569..6f905215b8 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -661,41 +661,35 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route, uint64_t address, uint32_t data, PCIDevice *dev) { AddressSpace *as = pci_device_iommu_address_space(dev); - hwaddr xlat, len, doorbell_gpa; - MemoryRegionSection mrs; - MemoryRegion *mr; - int ret = 1; + IOMMUMemoryRegionClass *imrc; + IOMMUMemoryRegion *iommu_mr; + IOMMUTLBEntry entry; if (as == &address_space_memory) { return 0; } + iommu_mr = IOMMU_MEMORY_REGION(as->root); + imrc = memory_region_get_iommu_class_nocheck(iommu_mr); + /* MSI doorbell address is translated by an IOMMU */ rcu_read_lock(); - mr = address_space_translate(as, address, &xlat, &len, true, - MEMTXATTRS_UNSPECIFIED); - if (!mr) { - goto unlock; - } - mrs = memory_region_find(mr, xlat, 1); - if (!mrs.mr) { - goto unlock; - } - - doorbell_gpa = mrs.offset_within_address_space; - memory_region_unref(mrs.mr); - - route->u.msi.address_lo = doorbell_gpa; - route->u.msi.address_hi = doorbell_gpa >> 32; - - trace_kvm_arm_fixup_msi_route(address, doorbell_gpa); - - ret = 0; - -unlock: + entry = imrc->translate(iommu_mr, address, IOMMU_WO, 0); rcu_read_unlock(); - return ret; + + if (entry.perm == IOMMU_NONE) { + return -ENOENT; + } + + route->u.msi.address_lo = entry.translated_addr; + route->u.msi.address_hi = entry.translated_addr >> 32; + + memory_region_iotlb_notify_iommu(iommu_mr, 0, entry); + + trace_kvm_arm_fixup_msi_route(address, entry.translated_addr); + + return 0; } int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route, From patchwork Fri Sep 21 08:18:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972958 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnT53Ms5z9sC7 for ; Fri, 21 Sep 2018 18:54:01 +1000 (AEST) Received: from localhost ([::1]:54571 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HCJ-0003FT-0k for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:53:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48447) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Ggm-0005du-2P for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Ggl-000789-At for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:24 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38722) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggi-0006vF-Tg; Fri, 21 Sep 2018 04:21:21 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 27CDD3084049; Fri, 21 Sep 2018 08:21:17 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF69B7D665; Fri, 21 Sep 2018 08:21:14 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:13 +0200 Message-Id: <20180921081819.9203-23-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Fri, 21 Sep 2018 08:21:17 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 22/28] vfio/pci: Always set up MSI route before enabling vectors X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" As we enable the vectors, we shall have an MSI route setup. The notification of the stage 1 binding is done on MSI route setup. Signed-off-by: Eric Auger --- hw/vfio/pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 866f0deeb7..842b0921d4 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -522,6 +522,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, if (vdev->nr_vectors < nr + 1) { vfio_disable_irqindex(&vdev->vbasedev, VFIO_PCI_MSIX_IRQ_INDEX); vdev->nr_vectors = nr + 1; + vfio_add_kvm_msi_virq(vdev, vector, nr, true); ret = vfio_enable_vectors(vdev, true); if (ret) { error_report("vfio: failed to enable vectors, %d", ret); From patchwork Fri Sep 21 08:18:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972965 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnZJ59LFz9sC7 for ; Fri, 21 Sep 2018 18:58:32 +1000 (AEST) Received: from localhost ([::1]:54600 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HGf-0007mu-WC for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:58:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48599) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Ggy-0005qF-9p for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Ggx-0007aH-Ik for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40346) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggt-0007PZ-8Z; Fri, 21 Sep 2018 04:21:31 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 64CFF88E61; Fri, 21 Sep 2018 08:21:30 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 73D967D665; Fri, 21 Sep 2018 08:21:17 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:14 +0200 Message-Id: <20180921081819.9203-24-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Fri, 21 Sep 2018 08:21:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 23/28] hw/arm/smmuv3: Remove warning about unsupported MAP notifiers X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" SMMUv3 now is integrated with VFIO by enable the 2 stages of the physical SMMUv3. This relies on a MAP notifier for MSI stage 1 binding notification. So let's remove this warning. Signed-off-by: Eric Auger --- hw/arm/smmuv3.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index 1d8deae37f..c6dec63f07 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -1504,14 +1504,6 @@ static void smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu, SMMUNotifierNode *node = NULL; SMMUNotifierNode *next_node = NULL; - if (new & IOMMU_NOTIFIER_MAP) { - int bus_num = pci_bus_num(sdev->bus); - PCIDevice *pcidev = pci_find_device(sdev->bus, bus_num, sdev->devfn); - - warn_report("SMMUv3 does not support notification on MAP: " - "device %s will not function properly", pcidev->name); - } - if (old == IOMMU_NOTIFIER_NONE) { trace_smmuv3_notify_flag_add(iommu->parent_obj.name); node = g_malloc0(sizeof(*node)); From patchwork Fri Sep 21 08:18:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972956 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnRL1P02z9sC7 for ; Fri, 21 Sep 2018 18:52:30 +1000 (AEST) Received: from localhost ([::1]:54561 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HAp-0001nl-Oo for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:52:27 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Ggy-0005qN-Dt for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Ggx-0007aS-Je for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:36 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50838) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggv-0007VP-N1; Fri, 21 Sep 2018 04:21:33 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0411F3002A92; Fri, 21 Sep 2018 08:21:33 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id AF1947D665; Fri, 21 Sep 2018 08:21:30 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:15 +0200 Message-Id: <20180921081819.9203-25-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Fri, 21 Sep 2018 08:21:33 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 24/28] memory: Introduce IOMMU_NOTIFIER_INIT_CFG IOMMU Config Notifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds a new IOMMU config notifier. It aims at configuring the fault reporting from host to guest. Signed-off-by: Eric Auger --- I acknowledge I am not a big fan of having a vfio uapi struct referenced in IOMMUConfig. However this config really sets something at VFIO layer. Also I may rename IOMMU_NOTIFIER_INIT_CFG into IOMMU_NOTIFIER_FAULT_CONFIG. Globally it is also questionnable at this stage if such a notifier really is requested. I devised that to allow the SMMU to pass a fifo size to the kernel. This size can be aligned to the size as seen/set by the guest. However for unrecoverable errors we may consider that having an arbitrary size determined by vfio/common.c can be sufficient? --- include/exec/memory.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 93150e1450..f54c39b248 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -28,6 +28,7 @@ #include "hw/qdev-core.h" #ifdef CONFIG_LINUX #include +#include #endif #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -82,6 +83,7 @@ typedef struct IOMMUConfig { union { #ifdef __linux__ struct iommu_pasid_table_config pasid_cfg; + struct vfio_iommu_type1_guest_fault_config fault_cfg; #endif }; } IOMMUConfig; @@ -98,10 +100,13 @@ typedef enum { IOMMU_NOTIFIER_MAP = 0x2, /* Notify stage 1 config changes */ IOMMU_NOTIFIER_PASID_CFG = 0x4, + /* Notify IOMMU initial setup */ + IOMMU_NOTIFIER_INIT_CFG = 0x8, } IOMMUNotifierFlag; #define IOMMU_NOTIFIER_IOTLB_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) -#define IOMMU_NOTIFIER_CONFIG_ALL (IOMMU_NOTIFIER_PASID_CFG) +#define IOMMU_NOTIFIER_CONFIG_ALL (IOMMU_NOTIFIER_PASID_CFG | \ + IOMMU_NOTIFIER_INIT_CFG) struct IOMMUNotifier; struct IOMMUConfig; From patchwork Fri Sep 21 08:18:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972960 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnVK65d2z9sCD for ; Fri, 21 Sep 2018 18:55:05 +1000 (AEST) Received: from localhost ([::1]:54577 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HDL-0004Bc-BK for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:55:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48643) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3Gh1-0005tQ-A6 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:40 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3Gh0-0007kF-G4 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47506) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3Ggy-0007b9-FM; Fri, 21 Sep 2018 04:21:36 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A1E408830C; Fri, 21 Sep 2018 08:21:35 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5090C7D665; Fri, 21 Sep 2018 08:21:33 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:16 +0200 Message-Id: <20180921081819.9203-26-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 21 Sep 2018 08:21:35 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 25/28] memory: Introduce IOMMU Memory Region inject_faults API X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This new API allows to inject @count iommu_faults into the IOMMU memory region. Signed-off-by: Eric Auger --- include/exec/memory.h | 26 ++++++++++++++++++++++++++ memory.c | 12 ++++++++++++ 2 files changed, 38 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index f54c39b248..45bcca2081 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -391,6 +391,20 @@ typedef struct IOMMUMemoryRegionClass { * @iommu: the IOMMUMemoryRegion */ int (*num_indexes)(IOMMUMemoryRegion *iommu); + +#ifdef CONFIG_LINUX + /* Inject @count faults into the IOMMU memory region + * + * Optional method: if this method is not provided, then + * memory_region_injection_faults() will return -ENOENT + * + * @iommu: the IOMMU memory region to inject the faults in + * @count: number of faults to inject + * @buf: fault buffer + */ + int (*inject_faults)(IOMMUMemoryRegion *iommu, int count, + struct iommu_fault *buf); +#endif } IOMMUMemoryRegionClass; typedef struct CoalescedMemoryRange CoalescedMemoryRange; @@ -1203,6 +1217,18 @@ int memory_region_iommu_attrs_to_index(IOMMUMemoryRegion *iommu_mr, */ int memory_region_iommu_num_indexes(IOMMUMemoryRegion *iommu_mr); +#ifdef CONFIG_LINUX +/** + * memory_region_inject_faults : inject @count faults stored in @buf + * + * @iommu_mr: the IOMMU memory region + * @count: number of faults to be injected + * @buf: buffer containing the faults + */ +int memory_region_inject_faults(IOMMUMemoryRegion *iommu_mr, int count, + struct iommu_fault *buf); +#endif + /** * memory_region_name: get a memory region's name * diff --git a/memory.c b/memory.c index 2c320bc14b..6ba8d31773 100644 --- a/memory.c +++ b/memory.c @@ -1976,6 +1976,18 @@ int memory_region_iommu_num_indexes(IOMMUMemoryRegion *iommu_mr) return imrc->num_indexes(iommu_mr); } +#ifdef CONFIG_LINUX +int memory_region_inject_faults(IOMMUMemoryRegion *iommu_mr, int count, + struct iommu_fault *buf) +{ + IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_GET_CLASS(iommu_mr); + if (!imrc->inject_faults) { + return -ENOENT; + } + return imrc->inject_faults(iommu_mr, count, buf); +} +#endif + void memory_region_set_log(MemoryRegion *mr, bool log, unsigned client) { uint8_t mask = 1 << client; From patchwork Fri Sep 21 08:18:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972957 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnSS5LfXz9sC7 for ; Fri, 21 Sep 2018 18:53:28 +1000 (AEST) Received: from localhost ([::1]:54568 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HBm-0002gB-8s for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:53:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48778) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GhH-00067u-Ar for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GhG-00007a-Au for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52322) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GhB-0008D3-HR; Fri, 21 Sep 2018 04:21:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id BE5994E919; Fri, 21 Sep 2018 08:21:48 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id E6F657D90A; Fri, 21 Sep 2018 08:21:35 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:17 +0200 Message-Id: <20180921081819.9203-27-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Fri, 21 Sep 2018 08:21:48 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 26/28] hw/vfio/common: Handle fault_handler X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We introduce a new IOMMU notifier, whose role is to register the fault notifier. The patch also implements the userspace fault handler. When the eventfd is signalled, the fault handler injects the set of collected faults into the IOMMU memory region. Signed-off-by: Eric Auger --- hw/vfio/common.c | 118 ++++++++++++++++++++++++++++++++++ include/hw/vfio/vfio-common.h | 1 + 2 files changed, 119 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 172ddd396f..9548e962e5 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -37,12 +37,16 @@ #include "sysemu/kvm.h" #include "trace.h" #include "qapi/error.h" +#include "qemu/event_notifier.h" +#include "qemu/main-loop.h" struct vfio_group_head vfio_group_list = QLIST_HEAD_INITIALIZER(vfio_group_list); struct vfio_as_head vfio_address_spaces = QLIST_HEAD_INITIALIZER(vfio_address_spaces); +#define MAX_QUERIED_FAULT_EVENTS 10 + #ifdef CONFIG_KVM /* * We have a single VFIO pseudo device per KVM VM. Once created it lives @@ -349,6 +353,111 @@ static bool vfio_get_vaddr(IOMMUTLBEntry *iotlb, void **vaddr, return true; } +/* + * At the moment we retrieve MAX_QUERIED_FAULT_EVENTS events + * TODO: loop until the ioctl returns count = 0 + */ +static void vfio_iommu_fault_handler(void *data) +{ + VFIOGuestIOMMU *giommu = (VFIOGuestIOMMU *)data; + IOMMUMemoryRegion *iommu = giommu->iommu; + struct vfio_iommu_type1_get_fault_events *ustruct; + size_t fault_buffer_size = + MAX_QUERIED_FAULT_EVENTS * sizeof(struct iommu_fault); + size_t argsz = sizeof(*ustruct) + fault_buffer_size; + int ret; + + ustruct = g_malloc0(argsz); + ustruct->argsz = argsz; + ustruct->count = 0; + + ret = ioctl(giommu->container->fd, VFIO_IOMMU_GET_FAULT_EVENTS, ustruct); + if (ret) { + error_report("%s: failed to get pending faults (%d)", + __func__, ret); + } + + assert(ustruct->count > 0); + memory_region_inject_faults(iommu, ustruct->count, ustruct->events); + + g_free(ustruct); + + ret = event_notifier_test_and_clear(giommu->fault_notifier); + if (!ret) { + error_report("Error when clearing fd=%d (ret = %d)", + event_notifier_get_fd(giommu->fault_notifier), ret); + } +} + +static void +vfio_iommu_fault_handler_register(IOMMUNotifier *n, IOMMUConfig *cfg) + +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOContainer *container = giommu->container; + struct vfio_iommu_type1_set_fault_eventfd info; + int ret; + + info.argsz = sizeof(info); + info.flags = 0; + + giommu->fault_notifier = g_malloc0(sizeof(EventNotifier)); + ret = event_notifier_init(giommu->fault_notifier, 0); + if (ret) { + error_report("%s: failed to init fault eventfd (%d)", + __func__, ret); + goto out; + } + + info.eventfd = event_notifier_get_fd(giommu->fault_notifier); + qemu_set_fd_handler(info.eventfd, vfio_iommu_fault_handler, NULL, giommu); + + memcpy(&info.config, &cfg->fault_cfg, sizeof(cfg->fault_cfg)); + + ret = ioctl(container->fd, VFIO_IOMMU_SET_FAULT_EVENTFD, &info); + if (ret) { + error_report("%s: failed to setup iommu fault propagation (%d)", + __func__, ret); + goto cleanup; + } + return; +cleanup: + qemu_set_fd_handler(info.eventfd, NULL, NULL, NULL); + event_notifier_cleanup(giommu->fault_notifier); +out: + g_free(giommu->fault_notifier); +} + +static void vfio_iommu_fault_handler_unregister(IOMMUNotifier *n) +{ + VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOContainer *container = giommu->container; + struct vfio_iommu_type1_set_fault_eventfd info; + int eventfd, ret; + + if (!(n->notifier_flags & IOMMU_NOTIFIER_INIT_CFG) || + !giommu->fault_notifier) { + return; + } + + eventfd = event_notifier_get_fd(giommu->fault_notifier); + if (eventfd < 0) { + return; + } + + info.argsz = sizeof(info); + info.flags = 0; + info.eventfd = -1; + ret = ioctl(container->fd, VFIO_IOMMU_SET_FAULT_EVENTFD, &info); + if (ret) { + error_report("%s: failed to stop iommu fault propagation (%d)", + __func__, ret); + } + qemu_set_fd_handler(eventfd, NULL, NULL, NULL); + event_notifier_cleanup(giommu->fault_notifier); + g_free(giommu->fault_notifier); +} + /* Program the guest @cfg on physical IOMMU stage 1 (nested mode) */ static void vfio_iommu_nested_notify(IOMMUNotifier *n, IOMMUConfig *cfg) @@ -754,6 +863,14 @@ static void vfio_listener_region_add(MemoryListener *listener, QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); memory_region_register_iommu_notifier(section->mr, &giommu->n); + /* Config notifier to register fault handler */ + giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); + iommu_config_notifier_init(&giommu->n, + vfio_iommu_fault_handler_register, + IOMMU_NOTIFIER_INIT_CFG, iommu_idx); + QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + memory_region_register_iommu_notifier(section->mr, &giommu->n); + /* IOTLB unmap notifier to propagate guest IOTLB invalidations */ giommu = vfio_alloc_guest_iommu(container, iommu_mr, offset); iommu_iotlb_notifier_init(&giommu->n, vfio_iommu_unmap_notify, @@ -846,6 +963,7 @@ static void vfio_listener_region_del(MemoryListener *listener, } else if (is_iommu_config_notifier(&giommu->n)) { memory_region_unregister_iommu_notifier(section->mr, &giommu->n); + vfio_iommu_fault_handler_unregister(&giommu->n); } QLIST_REMOVE(giommu, giommu_next); g_free(giommu); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 821def0565..db9cd93d77 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -91,6 +91,7 @@ typedef struct VFIOGuestIOMMU { hwaddr iommu_offset; IOMMUNotifier n; QLIST_ENTRY(VFIOGuestIOMMU) giommu_next; + EventNotifier *fault_notifier; } VFIOGuestIOMMU; typedef struct VFIOHostDMAWindow { From patchwork Fri Sep 21 08:18:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972963 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnXs2LByz9sC7 for ; Fri, 21 Sep 2018 18:57:17 +1000 (AEST) Received: from localhost ([::1]:54592 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HFT-0006HH-05 for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:57:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48803) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GhI-00069F-SQ for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GhI-0000HK-52 for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:21:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38456) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GhG-000061-6T; Fri, 21 Sep 2018 04:21:54 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7A38880F88; Fri, 21 Sep 2018 08:21:53 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 14A567D904; Fri, 21 Sep 2018 08:21:48 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:18 +0200 Message-Id: <20180921081819.9203-28-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 21 Sep 2018 08:21:53 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 27/28] hw/arm/smmuv3: Init fault handling X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We configure the kernel fault event fifo when the guest sizes the event queue. Signed-off-by: Eric Auger --- At the moment we arbitrarily cap the fifo size to 2^5 events --- hw/arm/smmuv3.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index c6dec63f07..ae27cfdfa0 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -1066,6 +1066,25 @@ static int smmuv3_cmdq_consume(SMMUv3State *s) return 0; } +static int smmuv3_init_fault_queues(SMMUv3State *s) +{ + SMMUState *base = &s->smmu_state; + SMMUNotifierNode *node; + IOMMUConfig config; + /* TODO: today we cap the shift to 5 to avoid the kfifo to become too big*/ + int qs = MIN(5, s->eventq.log2size); + + config.fault_cfg.qs = qs; + + QLIST_FOREACH(node, &base->notifiers_list, next) { + IOMMUMemoryRegion *iommu_mr = &node->sdev->iommu; + + memory_region_config_notify_iommu(iommu_mr, 0, IOMMU_NOTIFIER_INIT_CFG, + &config); + } + return 0; +} + static MemTxResult smmu_writell(SMMUv3State *s, hwaddr offset, uint64_t data, MemTxAttrs attrs) { @@ -1089,6 +1108,7 @@ static MemTxResult smmu_writell(SMMUv3State *s, hwaddr offset, if (s->eventq.log2size > SMMU_EVENTQS) { s->eventq.log2size = SMMU_EVENTQS; } + smmuv3_init_fault_queues(s); return MEMTX_OK; case A_EVENTQ_IRQ_CFG0: s->eventq_irq_cfg0 = data; From patchwork Fri Sep 21 08:18:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 972961 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42GnWC5bhSz9sC7 for ; Fri, 21 Sep 2018 18:55:51 +1000 (AEST) Received: from localhost ([::1]:54585 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3HE5-0004wg-D8 for incoming@patchwork.ozlabs.org; Fri, 21 Sep 2018 04:55:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48887) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g3GhP-0006D0-Pg for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:22:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g3GhO-0000bA-OR for qemu-devel@nongnu.org; Fri, 21 Sep 2018 04:22:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:54494) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g3GhI-0000Hn-UC; Fri, 21 Sep 2018 04:21:57 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 199A019CF61; Fri, 21 Sep 2018 08:21:56 +0000 (UTC) Received: from laptop.redhat.com (ovpn-117-87.ams2.redhat.com [10.36.117.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id C3D907D665; Fri, 21 Sep 2018 08:21:53 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, qemu-devel@nongnu.org, qemu-arm@nongnu.org, peter.maydell@linaro.org Date: Fri, 21 Sep 2018 10:18:19 +0200 Message-Id: <20180921081819.9203-29-eric.auger@redhat.com> In-Reply-To: <20180921081819.9203-1-eric.auger@redhat.com> References: <20180921081819.9203-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 21 Sep 2018 08:21:56 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [RFC v2 28/28] hw/arm/smmuv3: Implement fault injection X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: yi.l.liu@intel.com, cdall@kernel.org, mst@redhat.com, jean-philippe.brucker@arm.com, peterx@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We convert iommu_fault structs received from the kernel into the data struct used by the emulation code and record the evnts into the virtual event queue. Signed-off-by: Eric Auger --- Exhaustive mapping remains to be done --- hw/arm/smmuv3.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c index ae27cfdfa0..7ecc6c9669 100644 --- a/hw/arm/smmuv3.c +++ b/hw/arm/smmuv3.c @@ -1561,6 +1561,81 @@ smmuv3_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) { } +static inline int +smmuv3_inject_faults(IOMMUMemoryRegion *iommu_mr, int count, + struct iommu_fault *buf) +{ + SMMUDevice *sdev = container_of(iommu_mr, SMMUDevice, iommu); + SMMUv3State *s3 = sdev->smmu; + uint32_t sid = smmu_get_sid(sdev); + int i; + + for (i = 0; i < count; i++) { + SMMUEventInfo info = {}; + + if (buf[i].type != IOMMU_FAULT_DMA_UNRECOV) { + continue; + } + + info.sid = sid; + + switch (buf[i].reason) { + case IOMMU_FAULT_REASON_UNKNOWN: + case IOMMU_FAULT_REASON_INTERNAL: + break; + case IOMMU_FAULT_REASON_PASID_FETCH: + info.type = SMMU_EVT_F_CD_FETCH; + break; + case IOMMU_FAULT_REASON_DEVICE_CONTEXT_FETCH: + info.type = SMMU_EVT_F_STE_FETCH; + info.u.f_cd_fetch.addr = buf[i].fetch_addr; + break; + case IOMMU_FAULT_REASON_BAD_PASID_ENTRY: + info.type = SMMU_EVT_C_BAD_CD; + /* TODO further fill info.u.c_bad_cd */ + break; + case IOMMU_FAULT_REASON_BAD_DEVICE_CONTEXT_ENTRY: + info.type = SMMU_EVT_C_BAD_STE; + /* TODO further fill info.u.f_ste_fetch */ + break; + case IOMMU_FAULT_REASON_PASID_INVALID: + info.type = SMMU_EVT_C_BAD_SUBSTREAMID; + /* TODO further fill info.u.c_bad_substream */ + break; + case IOMMU_FAULT_REASON_SOURCEID_INVALID: + info.type = SMMU_EVT_C_BAD_STREAMID; + /* TODO further fill info.u.c_bad_streamid */ + break; + case IOMMU_FAULT_REASON_WALK_EABT: + info.type = SMMU_EVT_F_WALK_EABT; + info.u.f_walk_eabt.addr = buf[i].addr; + info.u.f_walk_eabt.addr2 = buf[i].fetch_addr; + break; + case IOMMU_FAULT_REASON_PTE_FETCH: + info.type = SMMU_EVT_F_TRANSLATION; + info.u.f_translation.addr = buf[i].addr; + break; + case IOMMU_FAULT_REASON_PERMISSION: + info.type = SMMU_EVT_F_PERMISSION; + info.u.f_permission.addr = buf[i].addr; + break; + case IOMMU_FAULT_REASON_ACCESS: + info.type = SMMU_EVT_F_ACCESS; + info.u.f_access.addr = buf[i].addr; + break; + case IOMMU_FAULT_REASON_OOR_ADDRESS: + info.type = SMMU_EVT_F_ADDR_SIZE; + info.u.f_addr_size.addr = buf[i].addr; + break; + default: + break; + } + + smmuv3_record_event(s3, &info); + } + return 0; +} + static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, void *data) { @@ -1570,6 +1645,7 @@ static void smmuv3_iommu_memory_region_class_init(ObjectClass *klass, imrc->notify_flag_changed = smmuv3_notify_flag_changed; imrc->get_attr = smmuv3_get_attr; imrc->replay = smmuv3_replay; + imrc->inject_faults = smmuv3_inject_faults; } static const TypeInfo smmuv3_type_info = {