From patchwork Wed Mar 29 23:05:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 744992 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vtk0n6J5zz9s0Z for ; Thu, 30 Mar 2017 10:06:49 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3vtk0n2tr1zDqKy for ; Thu, 30 Mar 2017 10:06:49 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3vtk0d4pxczDqJx for ; Thu, 30 Mar 2017 10:06:41 +1100 (AEDT) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v2TMxIFL176236 for ; Wed, 29 Mar 2017 19:06:39 -0400 Received: from e23smtp08.au.ibm.com (e23smtp08.au.ibm.com [202.81.31.141]) by mx0b-001b2d01.pphosted.com with ESMTP id 29gfk20u04-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 29 Mar 2017 19:06:38 -0400 Received: from localhost by e23smtp08.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 30 Mar 2017 09:06:36 +1000 Received: from d23relay07.au.ibm.com (202.81.31.226) by e23smtp08.au.ibm.com (202.81.31.205) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 30 Mar 2017 09:06:33 +1000 Received: from d23av01.au.ibm.com (d23av01.au.ibm.com [9.190.234.96]) by d23relay07.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v2TN6PhD32899196 for ; Thu, 30 Mar 2017 10:06:33 +1100 Received: from d23av01.au.ibm.com (localhost [127.0.0.1]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v2TN60FV012794 for ; Thu, 30 Mar 2017 10:06:01 +1100 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av01.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v2TN60fZ012599; Thu, 30 Mar 2017 10:06:00 +1100 Received: from bran.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id 8FA6FA034A; Thu, 30 Mar 2017 10:05:36 +1100 (AEDT) Received: from gwshan.ozlabs.ibm.com (shangw.ozlabs.ibm.com [10.61.2.199]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id 84E15E3AA7; Thu, 30 Mar 2017 10:05:36 +1100 (AEDT) Received: by gwshan.ozlabs.ibm.com (Postfix, from userid 1000) id 7A1D4AC3233; Thu, 30 Mar 2017 10:05:36 +1100 (AEDT) From: Gavin Shan To: skiboot@lists.ozlabs.org Date: Thu, 30 Mar 2017 10:05:28 +1100 X-Mailer: git-send-email 2.7.4 In-Reply-To: <1490828731-14497-1-git-send-email-gwshan@linux.vnet.ibm.com> References: <1490828731-14497-1-git-send-email-gwshan@linux.vnet.ibm.com> X-TM-AS-MML: disable x-cbid: 17032923-0048-0000-0000-0000021830B2 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17032923-0049-0000-0000-000047C410AD Message-Id: <1490828731-14497-4-git-send-email-gwshan@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-29_18:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703290192 Subject: [Skiboot] [PATCH 3/6] core/pci: Fix lost NVMe adapter behind PMC 8546 switch X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: markes@us.ibm.com, cjlarson@us.ibm.com MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The NVMe adapter in below PCI topology is lost. The root cause is the presence bit on its PCI slot is missed, but the PCIe link has been up. The PCI core doesn't probe the adapter behind the slot, leading to lost NVMe adapter in the particular case. PHB3 root port PLX switch 8748 (10b5:8748) PLX swich 9733 (10b5:9733) PMC 8546 swtich (11f8:8546) NVMe adapter (1c58:0023) This fixes the issue by overriding the PCI slot presence bit with PCIe link state bit. Reported-by: Mark E Schreiter Signed-off-by: Gavin Shan Tested-by: Mark E Schreiter --- core/pci.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/core/pci.c b/core/pci.c index ecb94c2..6864e6f 100644 --- a/core/pci.c +++ b/core/pci.c @@ -360,12 +360,19 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) uint16_t bctl; bool was_reset = false; int64_t ecap = 0; + uint32_t lcap = 0; + uint16_t lstat; /* Disable master aborts, clear errors */ pci_cfg_read16(phb, pd->bdfn, PCI_CFG_BRCTL, &bctl); bctl &= ~PCI_CFG_BRCTL_MABORT_REPORT; pci_cfg_write16(phb, pd->bdfn, PCI_CFG_BRCTL, bctl); + if (pci_has_cap(pd, PCI_CFG_CAP_ID_EXP, false)) { + ecap = pci_cap(pd, PCI_CFG_CAP_ID_EXP, false); + pci_cfg_read32(phb, pd->bdfn, ecap+PCICAP_EXP_LCAP, &lcap); + } + /* PCI-E bridge, check the slot state. We don't do that on the * root complex as this is handled separately and not all our * RCs implement the standard register set. @@ -374,7 +381,21 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) pd->dev_type == PCIE_TYPE_SWITCH_DNPORT) { uint16_t slctl, slcap, slsta, lctl; - ecap = pci_cap(pd, PCI_CFG_CAP_ID_EXP, false); + /* + * No need to touch the power supply if the PCIe link has + * been up. Further more, the slot presence bit is lost while + * the PCIe link is up on the specific PCI topology. In that + * case, we need ignore the slot presence bit and go ahead for + * probing. Otherwise, the NVMe adapter won't be probed. + * + * PHB3 root port, PLX switch 8748 (10b5:8748), PLX swich 9733 + * (10b5:9733), PMC 8546 swtich (11f8:8546), NVMe adapter + * (1c58:0023). + */ + pci_cfg_read16(phb, pd->bdfn, ecap+PCICAP_EXP_LSTAT, &lstat); + if ((lcap & PCICAP_EXP_LCAP_DL_ACT_REP) && + (lstat & PCICAP_EXP_LSTAT_DLLL_ACT)) + return true; /* Read the slot status & check for presence detect */ pci_cfg_read16(phb, pd->bdfn, ecap+PCICAP_EXP_SLOTSTAT, &slsta); @@ -430,11 +451,6 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) /* PCI-E bridge, wait for link */ if (pd->dev_type == PCIE_TYPE_ROOT_PORT || pd->dev_type == PCIE_TYPE_SWITCH_DNPORT) { - uint32_t lcap; - - /* Read link caps */ - pci_cfg_read32(phb, pd->bdfn, ecap+PCICAP_EXP_LCAP, &lcap); - /* Did link capability say we got reporting ? * * If yes, wait up to 10s, if not, wait 1s if we didn't already