From patchwork Thu Sep 11 19:26:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 388353 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id C4957140119 for ; Fri, 12 Sep 2014 05:26:58 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753480AbaIKT04 (ORCPT ); Thu, 11 Sep 2014 15:26:56 -0400 Received: from mail-ig0-f172.google.com ([209.85.213.172]:34597 "EHLO mail-ig0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752285AbaIKT0z (ORCPT ); Thu, 11 Sep 2014 15:26:55 -0400 Received: by mail-ig0-f172.google.com with SMTP id h3so935724igd.5 for ; Thu, 11 Sep 2014 12:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=t/nOjsFuLJfc6Cf9rtdrB5Wh5wep4hqR+4Y0aR20wg4=; b=VLOMsGpsgO6nrkViq0uMrCwDXVmD6Hk2mHX7/e68RKZ5O8tvH04l/89wH9uxqUDBqJ EUJ8Oja8Oa1M+bnHdahvJnJDy2HZO9/hUwQ69THVvaz3S/Tnt7GCzy/o4IUW6jD/C5tX iL8p00WrtwfYmIfFj/w3Dk9STB+hzZglBG//UJMW6yxdRiS125X9Gcv4cl8sFOYi5sjz zTqrD7DwAmU5juS0AUGNKgCM7BTppre/hLtv/CKyMVi2ylInODB4JQP2hKX0oe8KLrfl rcgoPDckv3RUIilER0bfGjf/JPgjC3fjx9X8aB1PoXKr6BRccQTssm86DzLpF9ZBjqal S0aw== MIME-Version: 1.0 X-Received: by 10.50.153.83 with SMTP id ve19mr11108929igb.4.1410463614214; Thu, 11 Sep 2014 12:26:54 -0700 (PDT) Received: by 10.64.171.11 with HTTP; Thu, 11 Sep 2014 12:26:54 -0700 (PDT) In-Reply-To: References: Date: Thu, 11 Sep 2014 12:26:54 -0700 X-Google-Sender-Auth: 4VrduG-h_MbOg_jIZIH6XUcHdSE Message-ID: Subject: Re: [BUG] Bisected Problem with LSI PCI FC Adapter From: Yinghai Lu To: Bjorn Helgaas , Linus Torvalds Cc: Dirk Gouders , Andreas Noever , Linux Kernel , "linux-pci@vger.kernel.org" Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, Sep 11, 2014 at 10:30 AM, Bjorn Helgaas wrote: > [+cc linux-pci] > > > On Thu, Sep 11, 2014 at 7:43 AM, Dirk Gouders wrote: >> Andreas Noever writes: >> >>> On Wed, Sep 3, 2014 at 2:47 PM, Dirk Gouders wrote: >>>> Andreas Noever writes: >>>> >>>>> On Wed, Sep 3, 2014 at 12:57 PM, Dirk Gouders wrote: >>>>>> On a Tyan VX50 (B4985) I ran into problems when updating the kernel: the >>>>>> PCI FC Adapter is no longer recognized. >>>>> >>>>> Can you provide the output of lspci -vvv and the output of dmesg from >>>>> a working boot? Which card is the one that is not recognized? >>>> >>>> Sure, the card that disappeared is: >>>> >>>> 0a:00.0 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02) >>> >>> As far as I can tell the following is happening: >>> The root bus resource window (advertised by the bios?) is to small: >>> pci_bus 0000:00: root bus resource [bus 00-07] >>> Previously we didn't really care. There is a resource conflict but we >>> ignored it: >>> pci_bus 0000:0a: busn_res: can not insert [bus 0a] under [bus 00-07] >>> (conflicts with (null) [bus 00-07]) >>> With the patch we mark the bridge as broken and reassign the bus to 06: >>> pci 0000:00:0e.0: bridge configuration invalid ([bus 0a-0a]), reconfiguring >>> pci 0000:00:0e.0: PCI bridge to [bus 06-07] >>> pci 0000:00:0e.0: bridge window [io 0x3000-0x3fff] >>> pci 0000:00:0e.0: bridge window [mem 0xd4200000-0xd42fffff] >>> pci_bus 0000:06: busn_res: [bus 06-07] end is updated to 06 > Thanks for following up on this. It had fallen off my radar, so I > opened https://bugzilla.kernel.org/show_bug.cgi?id=84281 to make sure > I don't forget again. Please continue the debug discussion here in > email. Two problems here: 1. This is amd two node systems. amd_bus.c tell us bus [00, 7f] is from first socket, but _OSC says only [0,7] is from first socket. So solution (1): According to Linus's principle, we should always trust HW than firmware, so should we just adjust bus range from _OSC before we use it? 2. After moving, LSI FC card from bus 0a to bus 07, the LSI refuse to respond. During my testing with pci busn allocation patchset, I found that if changing LSI Erie card to different bus, it will refuse to responding. Only thing that will make the LSI card again, is resetting the pcie link. This should be LSI firmware bug. Dirk, please check if you can apply attached patches to use echo 1 > /sys/bus/pci/devices/0000\:00\0e.0/link_disable echo 0 > /sys/bus/pci/devices/0000\:00\0e.0/link_disable to reset the link. Solution (2) To workaround the problem, we could reset the pcie link after change bus num in the pcie bridges ? Soultion (3) Or we just revert the offending 1820ffdccb9b4398 (PCI: Make sure bus number resources stay within their parents bounds) ? Thanks Yinghai Subject: [PATCH] PCI: Add link_disable in /sysfs for pcie device Found PCIe cards from one vendor, will not respond to scan from bridge, if we change bus number setting in bridge device. Have to do link disable/enable on the pcie root port. So try to expose link disable bit of pcie link control register. We can use echo 1 > /sys/..../link_disable echo 0 > /sys/..../link_disable to bring the pcie device back to respond to scan. Signed-off-by: Yinghai Lu --- drivers/pci/pcie-sysfs.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) Index: linux-2.6/drivers/pci/pcie-sysfs.c =================================================================== --- linux-2.6.orig/drivers/pci/pcie-sysfs.c +++ linux-2.6/drivers/pci/pcie-sysfs.c @@ -1,7 +1,35 @@ #include #include +static ssize_t +pcie_link_disable_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct pci_dev *pdev = to_pci_dev(dev); + + return sprintf(buf, "%u\n", pcie_link_disable_get(pdev)); +} +static ssize_t +pcie_link_disable_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct pci_dev *pdev = to_pci_dev(dev); + unsigned long val; + + if (kstrtoul(buf, 0, &val) < 0) + return -EINVAL; + + pcie_link_disable_set(pdev, val); + + return count; +} + +static struct device_attribute pcie_link_disable_attr = + __ATTR(pcie_link_disable, 0644, + pcie_link_disable_show, pcie_link_disable_store); + static struct attribute *pci_dev_pcie_dev_attrs[] = { + &pcie_link_disable_attr.attr, NULL, }; @@ -14,6 +42,11 @@ static umode_t pci_dev_pcie_attrs_are_vi if (!pci_is_pcie(pdev)) return 0; + if (a == &pcie_link_disable_attr.attr) + if ((pci_pcie_type(pdev) != PCI_EXP_TYPE_ROOT_PORT) && + (pci_pcie_type(pdev) != PCI_EXP_TYPE_DOWNSTREAM)) + return 0; + return a->mode; }