diff mbox series

[1/1] PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU

Message ID 20220216050140.426720-2-jeffrey.lane@canonical.com
State New
Headers show
Series PCI: vmd: Do not disable MSI-X remapping if interrupt remapping is enabled by IOMMU | expand

Commit Message

Jeff Lane Feb. 16, 2022, 5:01 a.m. UTC 
From: Adrian Huang <ahuang12@lenovo.com>

BugLink: https://bugs.launchpad.net/bugs/1937295

When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
the host OS cannot boot successfully with the following error message:

  nvme nvme0: I/O 12 QID 0 timeout, completion polled
  nvme nvme0: Shutdown timeout set to 6 seconds
  DMAR: DRHD: handling fault status reg 2
  DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request

The request device is the VMD controller:
  # lspci -s 0000:00.5 -nn
  0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume
  Management Device NVMe RAID Controller [8086:28c0] (rev 04)

`git bisect` points to this offending commit ee81ee84f873 ("PCI:
vmd: Disable MSI-X remapping when possible"), which disables VMD MSI
remapping. The IOMMU hardware blocks the compatibility format
interrupt request because Interrupt Remapping Enable Status (IRES) and
Extended Interrupt Mode Enable (EIME) are enabled. Please refer to
section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d
spec.

To fix the issue, VMD driver still enables the interrupt remapping
irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem
enables the interrupt remapping.

Test configuration is shown as follows:
  * Two VMD controllers
    1. 8086:28c0 (Whitley's VMD)
    2. 8086:201d (Purley's VMD: The issue does not appear in this
       controller. Just make sure if any side effect occurs.)
  * w/wo intremap=off

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219
Link: https://lore.kernel.org/r/20210901124047.1615-1-adrianhuang0701@gmail.com
Signed-off-by: Adrian Huang <ahuang12@lenovo.com>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
Cc: Jon Derrick <jonathan.derrick@intel.com>
Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
Cc: Joerg Roedel <jroedel@suse.de>
(cherry picked from commit 2565e5b69c44b4e42469afea3cc5a97e74d1ed45)
Signed-off-by: Jeff Lane <jeffrey.lane@canonical.com>
---
 drivers/pci/controller/vmd.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Comments

Krzysztof Kozlowski Feb. 16, 2022, 12:45 p.m. UTC | #1 
On 16/02/2022 06:01, Jeff Lane wrote:
> From: Adrian Huang <ahuang12@lenovo.com>
> 
> BugLink: https://bugs.launchpad.net/bugs/1937295
> 
> When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
> the host OS cannot boot successfully with the following error message:
> 
>   nvme nvme0: I/O 12 QID 0 timeout, completion polled
>   nvme nvme0: Shutdown timeout set to 6 seconds
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request
> 
> The request device is the VMD controller:
>   # lspci -s 0000:00.5 -nn
>   0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume
>   Management Device NVMe RAID Controller [8086:28c0] (rev 04)
> 
> `git bisect` points to this offending commit ee81ee84f873 ("PCI:
> vmd: Disable MSI-X remapping when possible"), which disables VMD MSI
> remapping. The IOMMU hardware blocks the compatibility format
> interrupt request because Interrupt Remapping Enable Status (IRES) and
> Extended Interrupt Mode Enable (EIME) are enabled. Please refer to
> section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d
> spec.
> 
> To fix the issue, VMD driver still enables the interrupt remapping
> irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem
> enables the interrupt remapping.
> 
> Test configuration is shown as follows:
>   * Two VMD controllers
>     1. 8086:28c0 (Whitley's VMD)
>     2. 8086:201d (Purley's VMD: The issue does not appear in this
>        controller. Just make sure if any side effect occurs.)
>   * w/wo intremap=off
> 
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219
> Link: https://lore.kernel.org/r/20210901124047.1615-1-adrianhuang0701@gmail.com
> Signed-off-by: Adrian Huang <ahuang12@lenovo.com>
> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
> Cc: Jon Derrick <jonathan.derrick@intel.com>
> Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
> Cc: Joerg Roedel <jroedel@suse.de>
> (cherry picked from commit 2565e5b69c44b4e42469afea3cc5a97e74d1ed45)
> Signed-off-by: Jeff Lane <jeffrey.lane@canonical.com>
> ---
>  drivers/pci/controller/vmd.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 

Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>


Best regards,
Krzysztof
diff mbox series

Patch

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 274e42e967f6..1441ed746eaf 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -6,6 +6,7 @@ 
 
 #include <linux/device.h>
 #include <linux/interrupt.h>
+#include <linux/iommu.h>
 #include <linux/irq.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
@@ -776,7 +777,8 @@  static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 	 * acceptable because the guest is usually CPU-limited and MSI
 	 * remapping doesn't become a performance bottleneck.
 	 */
-	if (!(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
+	if (iommu_capable(vmd->dev->dev.bus, IOMMU_CAP_INTR_REMAP) ||
+	    !(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
 	    offset[0] || offset[1]) {
 		ret = vmd_alloc_irqs(vmd);
 		if (ret)