From patchwork Wed Oct 4 15:52:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 821369 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3y6gXd1zPxz9t2c for ; Thu, 5 Oct 2017 02:57:53 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 895E688632; Wed, 4 Oct 2017 15:57:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p8zx9RHdLKKP; Wed, 4 Oct 2017 15:57:50 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 838C587C4E; Wed, 4 Oct 2017 15:57:50 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 2A3521CE750 for ; Wed, 4 Oct 2017 15:57:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 1E4D787638 for ; Wed, 4 Oct 2017 15:57:49 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dLgpfp-MvpBl for ; Wed, 4 Oct 2017 15:57:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 3CD4D8762A for ; Wed, 4 Oct 2017 15:57:48 +0000 (UTC) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP; 04 Oct 2017 08:57:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,477,1500966000"; d="scan'208"; a="1021580123" Received: from unknown (HELO anguy11-linux.jf.intel.com) ([10.165.248.57]) by orsmga003.jf.intel.com with ESMTP; 04 Oct 2017 08:57:47 -0700 From: Tony Nguyen To: linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org Date: Wed, 4 Oct 2017 08:52:58 -0700 Message-Id: <20171004155258.35634-1-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.9.5 Cc: bhelgaas@google.com Subject: [Intel-wired-lan] [PATCH] PCI: Check/Set ARI capability before setting numVFs X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This fixes a bug that can occur if an AER error is encountered while SRIOV devices are present. This issue was seen by doing the following. Inject an AER error to a device that has SRIOV devices. After the device has recovered, remove the driver. Reload the driver and enable SRIOV which causes the following crash to occur: kernel BUG at drivers/pci/iov.c:157! invalid opcode: 0000 [#1] SMP CPU: 36 PID: 2295 Comm: bash Not tainted 4.14.0-rc1+ #74 Hardware name: Supermicro X9DAi/X9DAi, BIOS 3.0a 04/29/2014 task: ffff9fa41cd45a00 task.stack: ffffb4b2036e8000 RIP: 0010:pci_iov_add_virtfn+0x2eb/0x350 RSP: 0018:ffffb4b2036ebcb8 EFLAGS: 00010286 RAX: 00000000fffffff0 RBX: ffff9fa42c1c8800 RCX: ffff9fa421ce2388 RDX: 00000000df900000 RSI: ffff9fa8214fb388 RDI: 00000000df903fff RBP: ffffb4b2036ebd18 R08: ffff9fa421ce23b8 R09: ffffb4b2036ebc2c R10: ffff9fa42c1a5548 R11: 000000000000058e R12: ffff9fa8214fb000 R13: ffff9fa42c1a5000 R14: ffff9fa8214fb388 R15: 0000000000000000 FS: 00007f60724b6700(0000) GS:ffff9fa82f300000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000559eca8b0f40 CR3: 0000000864146000 CR4: 00000000001606e0 Call Trace: pci_enable_sriov+0x353/0x440 ixgbe_pci_sriov_configure+0xd5/0x1f0 [ixgbe] sriov_numvfs_store+0xf7/0x170 dev_attr_store+0x18/0x30 sysfs_kf_write+0x37/0x40 kernfs_fop_write+0x120/0x1b0 __vfs_write+0x37/0x170 ? __alloc_fd+0x3f/0x170 ? set_close_on_exec+0x30/0x70 vfs_write+0xb5/0x1a0 SyS_write+0x55/0xc0 entry_SYSCALL_64_fastpath+0x1a/0xa5 RIP: 0033:0x7f6071bafc20 RSP: 002b:00007ffe7d42ba48 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 RAX: ffffffffffffffda RBX: 0000559eca8b0f30 RCX: 00007f6071bafc20 RDX: 0000000000000002 RSI: 0000559eca961f60 RDI: 0000000000000001 RBP: 00007f6071e78ae0 R08: 00007f6071e7a740 R09: 00007f60724b6700 R10: 0000000000000073 R11: 0000000000000246 R12: 0000000000000000 R13: 0000000000000000 R14: 0000000000000000 R15: 0000559eca892170 RIP: pci_iov_add_virtfn+0x2eb/0x350 RSP: ffffb4b2036ebcb8 The occurs since during AER recovery the ARI Capable Hierarchy bit, which can affect the values for First VF Offset and VF Stride, is not set until after pci_iov_set_numvfs() is called. This can cause the iov structure to be populated with values that are incorrect if the bit is later set. Check and set this bit, if needed, before calling pci_iov_set_numvfs() so that the values being populated properly take the ARI bit into account. CC: Alexander Duyck CC: Emil Tantilov Signed-off-by: Tony Nguyen Acked-by: Alexander Duyck --- drivers/pci/iov.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c index 7492a65..a8896c7 100644 --- a/drivers/pci/iov.c +++ b/drivers/pci/iov.c @@ -497,6 +497,10 @@ static void sriov_restore_state(struct pci_dev *dev) if (ctrl & PCI_SRIOV_CTRL_VFE) return; + if ((iov->ctrl & PCI_SRIOV_CTRL_ARI) && !(ctrl & PCI_SRIOV_CTRL_ARI)) + pci_write_config_word(dev, iov->pos + PCI_SRIOV_CTRL, + ctrl | PCI_SRIOV_CTRL_ARI); + for (i = PCI_IOV_RESOURCES; i <= PCI_IOV_RESOURCE_END; i++) pci_update_resource(dev, i);