From patchwork Fri Dec 6 20:48:36 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 298261 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 08D6E2C00A8 for ; Sat, 7 Dec 2013 07:50:57 +1100 (EST) Received: from localhost ([::1]:33412 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vp2MQ-0008UA-3Z for incoming@patchwork.ozlabs.org; Fri, 06 Dec 2013 15:50:54 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:38191) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vp2KL-0005gO-QA for qemu-devel@nongnu.org; Fri, 06 Dec 2013 15:48:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vp2KF-00088A-Pz for qemu-devel@nongnu.org; Fri, 06 Dec 2013 15:48:45 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57954) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vp2KF-00087t-Ch for qemu-devel@nongnu.org; Fri, 06 Dec 2013 15:48:39 -0500 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id rB6KmbSS015703 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 6 Dec 2013 15:48:37 -0500 Received: from bling.home (ovpn-113-87.phx2.redhat.com [10.3.113.87]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id rB6KmbUw005185; Fri, 6 Dec 2013 15:48:37 -0500 To: aliguori@amazon.com From: Alex Williamson Date: Fri, 06 Dec 2013 13:48:36 -0700 Message-ID: <20131206204836.16731.95235.stgit@bling.home> In-Reply-To: <20131206204715.16731.12627.stgit@bling.home> References: <20131206204715.16731.12627.stgit@bling.home> User-Agent: StGit/0.16 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [Qemu-devel] [PULL 3/5] vfio-pci: Fix Nvidia MSI ACK through 0x88000 quirk X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When MSI is enabled on Nvidia GeForce cards the driver seems to acknowledge the interrupt by writing a 0xff byte to the MSI capability ID register using the PCI config space mirror at offset 0x88000 from BAR0. Without this, the device will only fire a single interrupt. VFIO handles the PCI capability ID/next registers as virtual w/o write support, so any write through config space is currently dropped. Add a check for this and allow the write through the BAR window. The registers are read-only anyway. Signed-off-by: Alex Williamson --- hw/misc/vfio.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index f1aa4a7..09bdddf 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -1811,6 +1811,34 @@ static void vfio_probe_nvidia_bar5_window_quirk(VFIODevice *vdev, int nr) vdev->host.function); } +static void vfio_nvidia_88000_quirk_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + PCIDevice *pdev = &vdev->pdev; + hwaddr base = quirk->data.address_match & TARGET_PAGE_MASK; + + vfio_generic_quirk_write(opaque, addr, data, size); + + /* + * Nvidia seems to acknowledge MSI interrupts by writing 0xff to the + * MSI capability ID register. Both the ID and next register are + * read-only, so we allow writes covering either of those to real hw. + * NB - only fixed for the 0x88000 MMIO window. + */ + if ((pdev->cap_present & QEMU_PCI_CAP_MSI) && + vfio_range_contained(addr, size, pdev->msi_cap, PCI_MSI_FLAGS)) { + vfio_bar_write(&vdev->bars[quirk->data.bar], addr + base, data, size); + } +} + +static const MemoryRegionOps vfio_nvidia_88000_quirk = { + .read = vfio_generic_quirk_read, + .write = vfio_nvidia_88000_quirk_write, + .endianness = DEVICE_LITTLE_ENDIAN, +}; + /* * Finally, BAR0 itself. We want to redirect any accesses to either * 0x1800 or 0x88000 through the PCI config space access functions. @@ -1837,7 +1865,7 @@ static void vfio_probe_nvidia_bar0_88000_quirk(VFIODevice *vdev, int nr) quirk->data.address_mask = PCIE_CONFIG_SPACE_SIZE - 1; quirk->data.bar = nr; - memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_generic_quirk, + memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_nvidia_88000_quirk, quirk, "vfio-nvidia-bar0-88000-quirk", TARGET_PAGE_ALIGN(quirk->data.address_mask + 1)); memory_region_add_subregion_overlap(&vdev->bars[nr].mem,