From patchwork Mon Mar 16 16:31:25 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 450665 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 1CF8C140077 for ; Tue, 17 Mar 2015 03:33:46 +1100 (AEDT) Received: from localhost ([::1]:50521 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YXXxY-0007oe-Ay for incoming@patchwork.ozlabs.org; Mon, 16 Mar 2015 12:33:44 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46530) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YXXwT-0005qO-E7 for qemu-devel@nongnu.org; Mon, 16 Mar 2015 12:32:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YXXwK-0001ar-4v for qemu-devel@nongnu.org; Mon, 16 Mar 2015 12:32:37 -0400 Received: from e23smtp09.au.ibm.com ([202.81.31.142]:42973) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YXXwJ-0001aN-HN for qemu-devel@nongnu.org; Mon, 16 Mar 2015 12:32:28 -0400 Received: from /spool/local by e23smtp09.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 17 Mar 2015 02:32:24 +1000 Received: from d23dlp01.au.ibm.com (202.81.31.203) by e23smtp09.au.ibm.com (202.81.31.206) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 17 Mar 2015 02:32:22 +1000 Received: from d23relay09.au.ibm.com (d23relay09.au.ibm.com [9.185.63.181]) by d23dlp01.au.ibm.com (Postfix) with ESMTP id E426E2CE804E; Tue, 17 Mar 2015 03:32:19 +1100 (EST) Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay09.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t2GGWBXG16711912; Tue, 17 Mar 2015 03:32:19 +1100 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t2GGVkPa015700; Tue, 17 Mar 2015 03:31:46 +1100 Received: from shangw ([9.192.168.31]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with SMTP id t2GGViZl015399; Tue, 17 Mar 2015 03:31:45 +1100 Received: by shangw (Postfix, from userid 1000) id E60E63E08BE; Tue, 17 Mar 2015 03:31:27 +1100 (EST) From: Gavin Shan To: qemu-devel@nongnu.org Date: Tue, 17 Mar 2015 03:31:25 +1100 Message-Id: <1426523486-9794-2-git-send-email-gwshan@linux.vnet.ibm.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1426523486-9794-1-git-send-email-gwshan@linux.vnet.ibm.com> References: <1426523486-9794-1-git-send-email-gwshan@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15031616-0033-0000-0000-0000012D0D5C X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 202.81.31.142 Cc: aik@ozlabs.ru, agraf@suse.de, Gavin Shan , alex.williamson@redhat.com, qemu-ppc@nongnu.org, david@gibson.dropbear.id.au Subject: [Qemu-devel] [PATCH v2 2/3] VFIO: Disable INTx interrupt on EEH reset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When Linux guest recovers from EEH error on the following Emulex adapter, the MSIx interrupts are disabled and the INTx emulation is enabled. One INTx interrupt is injected to the guest by host because of detected pending INTx interrupts on the adapter. QEMU disables mmap'ed BAR regions and starts a timer to enable those regions at later point the INTx interrupt handler. Unfortunately, "VFIOPCIDevice->intx.pending" isn't cleared, meaning those disabled mapp'ed BAR regions won't be reenabled properly. It leads to EEH recovery failure at guest side because of hanged MMIO access. # lspci | grep Emulex 0000:01:00.0 Ethernet controller: Emulex Corporation \ OneConnect 10Gb NIC (be3) (rev 02) 0000:01:00.1 Ethernet controller: Emulex Corporation \ OneConnect 10Gb NIC (be3) (rev 02) The patch disables INTx interrupt before doing EEH reset to avoid the issue. Signed-off-by: Gavin Shan --- hw/vfio/pci.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index fca1edc..bfa3d0c 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3340,6 +3340,14 @@ int vfio_container_eeh_event(AddressSpace *as, int32_t groupid, * disable it so that it can be reenabled properly. Also, * the cached MSIx table should be cleared as it's not * reflecting the contents in hardware. + * + * We might have INTx interrupt whose handler disables the + * memory mapped BARs. The INTx pending state can't be + * cleared with memory BAR access in slow path. The timer + * kicked by the INTx interrupt handler won't enable those + * disabled memory mapped BARs, which leads to hanged MMIO + * register access and EEH recovery failure. We simply disable + * INTx if it has been enabled. */ QLIST_FOREACH(vbasedev, &group->device_list, next) { vdev = container_of(vbasedev, VFIOPCIDevice, vbasedev); @@ -3348,6 +3356,11 @@ int vfio_container_eeh_event(AddressSpace *as, int32_t groupid, } msix_reset(&vdev->pdev); + + /* Disable INTx */ + if (vdev->interrupt == VFIO_INT_INTx) { + vfio_disable_intx(vdev); + } } break;