From patchwork Tue Mar 31 01:06:02 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yinghai Lu X-Patchwork-Id: 456436 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 45BB614018C for ; Tue, 31 Mar 2015 12:06:07 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="verification failed; unprotected key" header.d=gmail.com header.i=@gmail.com header.b=w6Hs9Idt; dkim-adsp=none (unprotected policy); dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753760AbbCaBGF (ORCPT ); Mon, 30 Mar 2015 21:06:05 -0400 Received: from mail-ig0-f172.google.com ([209.85.213.172]:34073 "EHLO mail-ig0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753824AbbCaBGD (ORCPT ); Mon, 30 Mar 2015 21:06:03 -0400 Received: by igcau2 with SMTP id au2so3431122igc.1; Mon, 30 Mar 2015 18:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pKJD41hHvlRE5M+evWQ/KH3Rgv8PNUmy9KdIC7PTzYA=; b=w6Hs9IdtcIAE/nHO8oiYvVdue6+48JvSCchrP92qB3+eKN1WGRAQM4XmGZlW4cIZwb 5WlSMxia6wAqQn274UhR1jq7Zm1U/jCNuXbNwAe2j6WOtcF4BZElZJU/wx1ljdGaeZMo DUc5hHw2rDA1/qdbrjxYKLfSfKux8q97omy6CouPPfRD9A1Z1/X1pWEFlj1eXoM/A+AD MxTi87j+tZXXcUOi+pqmOHDf6JFv+a21j4EnnNYmD9rz5/+pWRZPOP109MuQ6kd0hY8Q 8m+3vQM/qAviF9fiCJM6AxPYtFvre8K2cg4gti/ZvUhzlvtmup1LfTJ2Xe92ku+c/dUV flAg== MIME-Version: 1.0 X-Received: by 10.107.135.75 with SMTP id j72mr53484709iod.0.1427763962342; Mon, 30 Mar 2015 18:06:02 -0700 (PDT) Received: by 10.64.208.43 with HTTP; Mon, 30 Mar 2015 18:06:02 -0700 (PDT) In-Reply-To: <5519D40D.20903@oracle.com> References: <5514391F.2030300@oracle.com> <551495EE.20201@oracle.com> <5515F6B8.8020606@oracle.com> <5515F782.7060301@oracle.com> <551623C1.2040300@oracle.com> <5516BF4B.9030901@oracle.com> <55181065.8090807@oracle.com> <5519D40D.20903@oracle.com> Date: Mon, 30 Mar 2015 18:06:02 -0700 X-Google-Sender-Auth: fJY_FSngnH-PdgZqPGDWINDQTNA Message-ID: Subject: Re: d63e2e1f3df breaks sparc/T5-8 From: Yinghai Lu To: David Ahern Cc: David Miller , Bjorn Helgaas , "linux-pci@vger.kernel.org" , "sparclinux@vger.kernel.org" , LKML Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Mon, Mar 30, 2015 at 3:54 PM, David Ahern wrote: > On 3/29/15 2:07 PM, Yinghai Lu wrote: >> >> [ 286.647560] PCI: scan_bus[/pci@300/pci@1/pci@0/pci@6] bus no 8 >> [ 286.921232] PCI: Claiming 0000:00:01.0: Resource 15: >> 0000800100000000..00008004afffffff [220c] >> [ 287.229190] PCI: Claiming 0000:01:00.0: Resource 15: >> 0000800100000000..00008004afffffff [220c] >> [ 287.533428] PCI: Claiming 0000:02:04.0: Resource 15: >> 0000800100000000..000080012fffffff [220c] >> [ 288.149831] PCI: Claiming 0000:03:00.0: Resource 15: >> 0000800100000000..000080012fffffff [220c] >> [ 288.252466] PCI: Claiming 0000:04:06.0: Resource 14: >> 0000800100000000..000080010fffffff [220c] >> [ 288.867196] PCI: Claiming 0000:05:00.0: Resource 0: >> 0000800100000000..0000800100001fff [204] >> [ 288.968221] pci 0000:05:00.0: can't claim BAR 0 [mem >> 0x800100000000-0x800100001fff]: no compatible bridge window >> >> the bridge resource has IORESOURCE_PREFETCH, but the device doesn't have >> that. > > # lspci -vvxxx -s 0000:05:00.0 > 0000:05:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host > Controller (rev 03) (prog-if 30 [XHCI]) > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > SERR- Latency: 0, Cache Line Size: 64 bytes > Interrupt: pin A routed to IRQ 00000004 > Region 0: Memory at 100000000 (64-bit, non-prefetchable) [size=8K] ok, that is really non-pref mmio 64bit. We can workaround the problem by honoring firmware setting, according to https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf page 13 Please check attached updated patches that should fix the regression and kill those "no compatible window" warnings. Thanks Yinghai Subject: [RFC PATCH] PCI: Set pref for mem64 resource of pcie device We still get "no compatible bridge window" warning on sparc T5-8 after we add support for 64bit resource for root bus. [ 286.647560] PCI: scan_bus[/pci@300/pci@1/pci@0/pci@6] bus no 8 [ 286.921232] PCI: Claiming 0000:00:01.0: Resource 15: 0000800100000000..00008004afffffff [220c] [ 287.229190] PCI: Claiming 0000:01:00.0: Resource 15: 0000800100000000..00008004afffffff [220c] [ 287.533428] PCI: Claiming 0000:02:04.0: Resource 15: 0000800100000000..000080012fffffff [220c] [ 288.149831] PCI: Claiming 0000:03:00.0: Resource 15: 0000800100000000..000080012fffffff [220c] [ 288.252466] PCI: Claiming 0000:04:06.0: Resource 14: 0000800100000000..000080010fffffff [220c] [ 288.867196] PCI: Claiming 0000:05:00.0: Resource 0: 0000800100000000..0000800100001fff [204] [ 288.968221] pci 0000:05:00.0: can't claim BAR 0 [mem 0x800100000000-0x800100001fff]: no compatible bridge window All the bridges have pref mem 64-bit resource, but the device resource does not have pref set, then we can not find parent for the device resource, as we can not put non-pref mem under pref mem. According to pcie spec errta https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf page 13, in some case it is ok to mark some as pref. only set the bit when the mmio is above 4G by BIOS. Signed-off-by: Yinghai Lu --- drivers/pci/probe.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) Index: linux-2.6/drivers/pci/probe.c =================================================================== --- linux-2.6.orig/drivers/pci/probe.c +++ linux-2.6/drivers/pci/probe.c @@ -1508,6 +1508,43 @@ static void pci_init_capabilities(struct pci_enable_acs(dev); } +/* + * According to + * https://www.pcisig.com/specifications/pciexpress/base2/PCIe_Base_r2.1_Errata_08Jun10.pdf + * page 13, system firmware could put some 64bit non-pref under 64bit pref, + * on some cases. + * Let's set pref bit when pci bus address is above 4G. + */ +static void set_pcie_64bit_pref(struct pci_dev *dev) +{ + int i; + + if (!pci_is_pcie(dev)) + return; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; i++) { + struct resource *res = &dev->resource[i]; + struct pci_bus_region r; + enum pci_bar_type type; + int reg; + + if (!(res->flags & IORESOURCE_MEM_64)) + continue; + + if (res->flags & IORESOURCE_PREFETCH) + continue; + + pcibios_resource_to_bus(dev->bus, &r, res); + if (r.start < 0xffffffff) + continue; + + reg = pci_resource_bar(dev, i, &type); + dev_printk(KERN_DEBUG, &dev->dev, "reg %d %pR + pref\n", + reg, res); + res->flags |= IORESOURCE_PREFETCH; + } +} + void pci_device_add(struct pci_dev *dev, struct pci_bus *bus) { int ret; @@ -1538,6 +1575,9 @@ void pci_device_add(struct pci_dev *dev, /* Initialize various capabilities */ pci_init_capabilities(dev); + /* After pcie_cap is assigned and sriov bar is probed */ + set_pcie_64bit_pref(dev); + /* * Add the device to our list of discovered devices * and the bus list for fixup functions, etc.