From patchwork Mon Aug 13 18:19:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Myron Stowe X-Patchwork-Id: 957138 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41q3sq1Kqsz9s4V for ; Tue, 14 Aug 2018 04:19:43 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729984AbeHMVDB (ORCPT ); Mon, 13 Aug 2018 17:03:01 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:45646 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728821AbeHMVDB (ORCPT ); Mon, 13 Aug 2018 17:03:01 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E92FB7D84D; Mon, 13 Aug 2018 18:19:40 +0000 (UTC) Received: from tak.stowe (ovpn-120-77.rdu2.redhat.com [10.10.120.77]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B90A1C70A; Mon, 13 Aug 2018 18:19:40 +0000 (UTC) Subject: [PATCH 1/2] PCI: Skip MPS logic for Virtual Functions (VFs) From: Myron Stowe To: bhelgaas@google.com, linux-pci@vger.kernel.org Cc: keith.busch@intel.com, jdmason@kudzu.us, okaya@kernel.org Date: Mon, 13 Aug 2018 12:19:39 -0600 Message-ID: <20180813181939.105655.50410.stgit@tak.stowe> In-Reply-To: <20180813181931.105655.22182.stgit@tak.stowe> References: <20180813181931.105655.22182.stgit@tak.stowe> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Mon, 13 Aug 2018 18:19:40 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Mon, 13 Aug 2018 18:19:40 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'myron.stowe@redhat.com' RCPT:'' Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Section 9.3.5.4 "Device Control Register" specifically shows both Max_Payload_Size (MPS) and Max_Read_request_Size (MRRS) to be 'RsvdP' for VFs in Table 9-16 [1]. Just prior to the table it states: "PF and VF functionality is defined in Section 7.5.3.4 except where noted in Table 9-16. For VF fields marked 'RsvdP', the PF setting applies to the VF." All of which implies that with respect to Max_Payload_Size Supported (MPSS), MPS, and MRRS values, we should not be paying any attention to the VF's fields, but rather only to the PF's. Only looking at the PF's fields also logically makes sense as it's the sole physical interface to the PCIe bus. [1] PCI Express Base Specification, Revision 4.0 Version 1.0 (2017). Link: https://bugzilla.kernel.org/show_bug.cgi?id=200527 Cc: Keith Busch Cc: Sinan Kaya Cc: Dongdong Liu Cc: Jon Mason Fixes: 27d868b5e6cf ("PCI: Set MPS to match upstream bridge") Cc: stable@vger.kernel.org # 4.3+ Signed-off-by: Myron Stowe --- drivers/pci/probe.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 611adcd..b285786 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1730,6 +1730,10 @@ static void pci_configure_mps(struct pci_dev *dev) if (!pci_is_pcie(dev) || !bridge || !pci_is_pcie(bridge)) return; + /* MPS and MRRS fields are of type 'RsvdP' for VFs, short-circuit out */ + if (dev->is_virtfn) + return; + mps = pcie_get_mps(dev); p_mps = pcie_get_mps(bridge); From patchwork Mon Aug 13 18:19:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Myron Stowe X-Patchwork-Id: 957139 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41q3sx2vPRz9s4V for ; Tue, 14 Aug 2018 04:19:49 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729988AbeHMVDI (ORCPT ); Mon, 13 Aug 2018 17:03:08 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:59530 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729858AbeHMVDI (ORCPT ); Mon, 13 Aug 2018 17:03:08 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 04F264023461; Mon, 13 Aug 2018 18:19:46 +0000 (UTC) Received: from tak.stowe (ovpn-120-77.rdu2.redhat.com [10.10.120.77]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6C3E02142F20; Mon, 13 Aug 2018 18:19:46 +0000 (UTC) Subject: [PATCH 2/2] PCI: Match Root Port's MPS to endpoint's MPSS as necessary From: Myron Stowe To: bhelgaas@google.com, linux-pci@vger.kernel.org Cc: keith.busch@intel.com, jdmason@kudzu.us, okaya@kernel.org Date: Mon, 13 Aug 2018 12:19:46 -0600 Message-ID: <20180813181946.105655.94535.stgit@tak.stowe> In-Reply-To: <20180813181931.105655.22182.stgit@tak.stowe> References: <20180813181931.105655.22182.stgit@tak.stowe> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Mon, 13 Aug 2018 18:19:47 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Mon, 13 Aug 2018 18:19:47 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'myron.stowe@redhat.com' RCPT:'' Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org In commit 27d868b5e6cf ("PCI: Set MPS to match upstream bridge"), we made sure every device's MPS setting matches its upstream bridge, making it more likely that a hot-added device will work in a system with an optimized MPS configuration. Recently I've started encountering systems where the endpoint device's MPSS capability is less than its Root Port's current MPS value, thus the endpoint is not capable of matching its upstream bridge's MPS setting (see: bugzilla via "Link:" below). This leaves the system vulnerable - the upstream Root Port could respond with larger sized TLPs than the device can handle, and the device will consider them to be 'Malformed'. One could use the "pci=pcie_bus_safe" kernel parameter to resolve the issue, but, it both forces a user to have to supply a kernel parameter to get the system to function reliable, and may end up limiting MPS settings of other, non-related, sub-topologies which could benefit from maintaining their larger values. This patch augments Keith's approach to include tuning down a Root Port's MPS setting when its hot-added endpoint device is not capable of matching it. Link: https://bugzilla.kernel.org/show_bug.cgi?id=200527 Cc: Keith Busch Cc: Sinan Kaya Cc: Dongdong Liu Acked-by: Jon Mason Signed-off-by: Myron Stowe --- drivers/pci/probe.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index b285786..a1a243e 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -1725,7 +1725,7 @@ int pci_setup_device(struct pci_dev *dev) static void pci_configure_mps(struct pci_dev *dev) { struct pci_dev *bridge = pci_upstream_bridge(dev); - int mps, p_mps, rc; + int mps, mpss, p_mps, rc; if (!pci_is_pcie(dev) || !bridge || !pci_is_pcie(bridge)) return; @@ -1753,6 +1753,14 @@ static void pci_configure_mps(struct pci_dev *dev) if (pcie_bus_config != PCIE_BUS_DEFAULT) return; + mpss = 128 << dev->pcie_mpss; + if (mpss < p_mps && pci_pcie_type(bridge) == PCI_EXP_TYPE_ROOT_PORT) { + pcie_set_mps(bridge, mpss); + pci_info(dev, "Upstream bridge's Max Payload Size set to %d (was %d, max %d)\n", + mpss, p_mps, 128 << bridge->pcie_mpss); + p_mps = pcie_get_mps(bridge); + } + rc = pcie_set_mps(dev, p_mps); if (rc) { pci_warn(dev, "can't set Max Payload Size to %d; if necessary, use \"pci=pcie_bus_safe\" and report a bug\n", @@ -1761,7 +1769,7 @@ static void pci_configure_mps(struct pci_dev *dev) } pci_info(dev, "Max Payload Size set to %d (was %d, max %d)\n", - p_mps, mps, 128 << dev->pcie_mpss); + p_mps, mps, mpss); } static struct hpp_type0 pci_default_type0 = {