From patchwork Tue Mar 31 20:28:31 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 457010 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 4BC9914007D for ; Wed, 1 Apr 2015 07:29:05 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=gmail.com header.i=@gmail.com header.b=leKc4Wwb; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755792AbbCaU2j (ORCPT ); Tue, 31 Mar 2015 16:28:39 -0400 Received: from mail-ie0-f177.google.com ([209.85.223.177]:33070 "EHLO mail-ie0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755786AbbCaU2c (ORCPT ); Tue, 31 Mar 2015 16:28:32 -0400 Received: by iebmp1 with SMTP id mp1so19125464ieb.0; Tue, 31 Mar 2015 13:28:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=WZ9MVRvDiaSJ7p1Ymnv6wLBYMkFIL/e7M9+HpFs5x8I=; b=leKc4Wwbp8wPGHDgO+ROa4LcGDSBGwEqM9ldGCZ7fAXtKhYII/66Ym6b7NJ+wuYe8r s3q671P/+vqblzpMVfwMPiVKU1sgBZVLE/HQE1UiRgvepLx67ntvvFixkT3tqCyx5BKr 9rxMRaui3RnZRH//d7/3/7bmaXEgoFonBH7fH0ere6EX0q2Q1yMs9zEukL4s7stdfOCh gPNDnREOdoSd35cLXpjI9sxfLX4nGpj4uus42U+B83XPnCk07ID200fDGlVyjxirXnSI cjkxHaToF4Pe5zuEcxG0E4oRpzNjQxQu9V/dsMxEdAVtN/Jn1EEae4Q9spX08CYf+d0E R/Gg== MIME-Version: 1.0 X-Received: by 10.43.64.203 with SMTP id xj11mr71652993icb.54.1427833711586; Tue, 31 Mar 2015 13:28:31 -0700 (PDT) Received: by 10.64.208.43 with HTTP; Tue, 31 Mar 2015 13:28:31 -0700 (PDT) In-Reply-To: <551AD384.50000@oracle.com> References: <5514391F.2030300@oracle.com> <551495EE.20201@oracle.com> <5515F6B8.8020606@oracle.com> <5515F782.7060301@oracle.com> <551623C1.2040300@oracle.com> <5516BF4B.9030901@oracle.com> <55181065.8090807@oracle.com> <5519D40D.20903@oracle.com> <551A1E32.1010309@oracle.com> <551AD384.50000@oracle.com> Date: Tue, 31 Mar 2015 13:28:31 -0700 X-Google-Sender-Auth: PVSwKb1x8WDZah_SRkYE6A6nFUg Message-ID: Subject: Re: d63e2e1f3df breaks sparc/T5-8 From: Yinghai Lu To: David Ahern , David Miller Cc: Bjorn Helgaas , "linux-pci@vger.kernel.org" , "sparclinux@vger.kernel.org" , LKML Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Tue, Mar 31, 2015 at 10:04 AM, David Ahern wrote: >> Clear out the old and apply these new ones. > > > I take DaveM's response to mean the patches (3rd one?) needs another > version. > > I will be on PTO Wed-Fri with limited access through Sunday. If you have > something to try out later today I can do that; else it needs to wait until > next week. Given the likelihood that Linus will release 4.0 this weekend > that means both 3.19 and 4.0 will be broken for these systems. Please check attached three patches on top of current linus tree. Thanks Yinghai Subject: [RFC PATCH v2] PCI: Set pref for mem64 resource of pcie device We still get "no compatible bridge window" warning on sparc T5-8 after we add support for 64bit resource for root bus. [ 286.647560] PCI: scan_bus[/pci@300/pci@1/pci@0/pci@6] bus no 8 [ 286.921232] PCI: Claiming 0000:00:01.0: Resource 15: 0000800100000000..00008004afffffff [220c] [ 287.229190] PCI: Claiming 0000:01:00.0: Resource 15: 0000800100000000..00008004afffffff [220c] [ 287.533428] PCI: Claiming 0000:02:04.0: Resource 15: 0000800100000000..000080012fffffff [220c] [ 288.149831] PCI: Claiming 0000:03:00.0: Resource 15: 0000800100000000..000080012fffffff [220c] [ 288.252466] PCI: Claiming 0000:04:06.0: Resource 14: 0000800100000000..000080010fffffff [220c] [ 288.867196] PCI: Claiming 0000:05:00.0: Resource 0: 0000800100000000..0000800100001fff [204] [ 288.968221] pci 0000:05:00.0: can't claim BAR 0 [mem 0x800100000000-0x800100001fff]: no compatible bridge window All the bridges have pref mem 64-bit resource, but the device resource does not have pref set, then we can not find parent for the device resource, as we can not put non-pref mem under pref mem. According to pcie spec errta https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf page 13, in some case it is ok to mark some as pref. only set pref for 64bit mmio when the entire path from the host to the adapter is over PCI Express. Signed-off-by: Yinghai Lu --- -v2: set pref for mmio 64 when whole path is PCI Express. --- drivers/pci/probe.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) Index: linux-2.6/drivers/pci/probe.c =================================================================== --- linux-2.6.orig/drivers/pci/probe.c +++ linux-2.6/drivers/pci/probe.c @@ -1508,6 +1508,53 @@ static void pci_init_capabilities(struct pci_enable_acs(dev); } +static bool pci_up_path_over_pcie(struct pci_bus *bus) +{ + if (!bus) + return true; + + if (bus->self && !pci_is_pcie(bus->self)) + return false; + + return pci_up_path_over_pcie(bus->parent); +} + +/* + * According to + * https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf + * page 13, system firmware could put some 64bit non-pref under 64bit pref, + * on some cases. + * Let's set pref bit for 64bit mmio when entire path from the host to + * the adapter is over PCI Express. + */ +static void set_pcie_64bit_pref(struct pci_dev *dev) +{ + int i; + + if (!pci_is_pcie(dev)) + return; + + if (!pci_up_path_over_pcie(dev->bus)) + return; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; i++) { + struct resource *res = &dev->resource[i]; + enum pci_bar_type type; + int reg; + + if (!(res->flags & IORESOURCE_MEM_64)) + continue; + + if (res->flags & IORESOURCE_PREFETCH) + continue; + + reg = pci_resource_bar(dev, i, &type); + dev_printk(KERN_DEBUG, &dev->dev, "reg 0x%x %pR + pref\n", + reg, res); + res->flags |= IORESOURCE_PREFETCH; + } +} + void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) { int ret; @@ -1538,6 +1585,9 @@ void pci_device_add(struct pci_dev *dev, /* Initialize various capabilities */ pci_init_capabilities(dev); + /* After pcie_cap is assigned and sriov bar is probed */ + set_pcie_64bit_pref(dev); + /* * Add the device to our list of discovered devices * and the bus list for fixup functions, etc.