From patchwork Tue Feb 24 08:34:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 442847 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id E460214016B for ; Tue, 24 Feb 2015 19:34:54 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752028AbbBXIer (ORCPT ); Tue, 24 Feb 2015 03:34:47 -0500 Received: from mail-pd0-f175.google.com ([209.85.192.175]:43053 "EHLO mail-pd0-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752256AbbBXIep (ORCPT ); Tue, 24 Feb 2015 03:34:45 -0500 Received: by pdev10 with SMTP id v10so31777778pde.10 for ; Tue, 24 Feb 2015 00:34:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=subject:to:from:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=+EgmFlrHpCPtz+dIWASdEPlEe/qKdufRkllWNLa4aXI=; b=YLukMFbq3uth9e2Re/fwAuVzMgWIV38Bz2LNf75T2C3ioc9JPsiP/kgMzgqiJUMuls m/dIBeyi10umOODAn/zB32CvxKvE/kGDLEQWj7OPgSOXsQaG4UnWcLneXyjyTbjPXT8S TkTj3RmeiplKx/JmCDi3RLQPFTj/4NmxbxFVDdcgM7X59J4jEuT1qsz5FihzrHCRFIbr Yh42gdmcVfGBFJnjEjRZK8ty2h64OoB8oodkKcasdhCjcQzYCQ/mpPqvWdi+q7p4q7GD Q5bKxOClI+XdEHLezeGQPP3pWLFlKFCTc2RC8l4Cbtvq8AOHRNSQ6DcypDjorEQM9QYk JhYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding; bh=+EgmFlrHpCPtz+dIWASdEPlEe/qKdufRkllWNLa4aXI=; b=IwuQBcFQ3oDPck8ouE3P6hk8YEVezRvFeyl8I5JgIO4ojCTtzLoys8mLykRqK7U2Vi aYlo10zWJOE7pKh3HGTwwQ1yV+UL3nV19+hBX8DWa+fJJSr0X7ouN4EPV0yC7GbbSdFM M2mepM6xlanugovYnWT/0/1vW/D1OxtZrtWmOP/w3mthnRPYCIqPFc4mu0/Cam/mjt/X 4sU+DN4XZnWBGUW3qzHZdAjAwsj3v94IEphhZD87bEB+zMsOSafGdW57RYcn+XTY9VNG cnmET4mQUcRjVM5HCyW0LO5KMfvn+cWEXpKk5dEZ7ichtjYs1S0MCYT+I4AWHltUpH7K OftA== X-Gm-Message-State: ALoCoQkn2TSqKC//iEn5J7FENELVP+LfT5HqiBoRNDjFf1iJojXEttgnPbqI4bxGfkYyouPWPqsb X-Received: by 10.66.62.201 with SMTP id a9mr26823932pas.101.1424766884671; Tue, 24 Feb 2015 00:34:44 -0800 (PST) Received: from localhost ([12.23.74.29]) by mx.google.com with ESMTPSA id in1sm37387407pbc.19.2015.02.24.00.34.43 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 24 Feb 2015 00:34:43 -0800 (PST) Subject: [PATCH v12 15/21] powerpc/powernv: Reserve additional space for IOV BAR according to the number of total_pe To: Wei Yang , benh@au1.ibm.com, gwshan@linux.vnet.ibm.com From: Bjorn Helgaas Cc: linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org Date: Tue, 24 Feb 2015 02:34:42 -0600 Message-ID: <20150224083442.32124.96716.stgit@bhelgaas-glaptop2.roam.corp.google.com> In-Reply-To: <20150224082939.32124.45744.stgit@bhelgaas-glaptop2.roam.corp.google.com> References: <20150224082939.32124.45744.stgit@bhelgaas-glaptop2.roam.corp.google.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Wei Yang On PHB3, PF IOV BAR will be covered by M64 window to have better PE isolation. The total_pe number is usually different from total_VFs, which can lead to a conflict between MMIO space and the PE number. For example, if total_VFs is 128 and total_pe is 256, the second half of M64 window will be part of other PCI device, which may already belong to other PEs. Prevent the conflict by reserving additional space for the PF IOV BAR, which is total_pe number of VF's BAR size. [bhelgaas: make dev_printk() output more consistent, index resource[] conventionally] Signed-off-by: Wei Yang Signed-off-by: Bjorn Helgaas --- arch/powerpc/include/asm/machdep.h | 4 ++ arch/powerpc/include/asm/pci-bridge.h | 3 ++ arch/powerpc/kernel/pci-common.c | 5 +++ arch/powerpc/platforms/powernv/pci-ioda.c | 58 +++++++++++++++++++++++++++++ 4 files changed, 70 insertions(+) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/powerpc/include/asm/machdep.h b/arch/powerpc/include/asm/machdep.h index c8175a3fe560..965547c58497 100644 --- a/arch/powerpc/include/asm/machdep.h +++ b/arch/powerpc/include/asm/machdep.h @@ -250,6 +250,10 @@ struct machdep_calls { /* Reset the secondary bus of bridge */ void (*pcibios_reset_secondary_bus)(struct pci_dev *dev); +#ifdef CONFIG_PCI_IOV + void (*pcibios_fixup_sriov)(struct pci_bus *bus); +#endif /* CONFIG_PCI_IOV */ + /* Called to shutdown machine specific hardware not already controlled * by other drivers. */ diff --git a/arch/powerpc/include/asm/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h index 513f8f27060d..de11de7d4547 100644 --- a/arch/powerpc/include/asm/pci-bridge.h +++ b/arch/powerpc/include/asm/pci-bridge.h @@ -175,6 +175,9 @@ struct pci_dn { #define IODA_INVALID_PE (-1) #ifdef CONFIG_PPC_POWERNV int pe_number; +#ifdef CONFIG_PCI_IOV + u16 max_vfs; /* number of VFs IOV BAR expended */ +#endif /* CONFIG_PCI_IOV */ #endif struct list_head child_list; struct list_head list; diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c index 82031011522f..022e9feeb1f2 100644 --- a/arch/powerpc/kernel/pci-common.c +++ b/arch/powerpc/kernel/pci-common.c @@ -1646,6 +1646,11 @@ void pcibios_scan_phb(struct pci_controller *hose) if (ppc_md.pcibios_fixup_phb) ppc_md.pcibios_fixup_phb(hose); +#ifdef CONFIG_PCI_IOV + if (ppc_md.pcibios_fixup_sriov) + ppc_md.pcibios_fixup_sriov(bus); +#endif /* CONFIG_PCI_IOV */ + /* Configure PCI Express settings */ if (bus && !pci_has_flag(PCI_PROBE_ONLY)) { struct pci_bus *child; diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index cd1a56160ded..36c533da5ccb 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -1749,6 +1749,61 @@ static void pnv_pci_init_ioda_msis(struct pnv_phb *phb) static void pnv_pci_init_ioda_msis(struct pnv_phb *phb) { } #endif /* CONFIG_PCI_MSI */ +#ifdef CONFIG_PCI_IOV +static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) +{ + struct pci_controller *hose; + struct pnv_phb *phb; + struct resource *res; + int i; + resource_size_t size; + struct pci_dn *pdn; + + if (!pdev->is_physfn || pdev->is_added) + return; + + hose = pci_bus_to_host(pdev->bus); + phb = hose->private_data; + + pdn = pci_get_pdn(pdev); + pdn->max_vfs = 0; + + for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) { + res = &pdev->resource[i + PCI_IOV_RESOURCES]; + if (!res->flags || res->parent) + continue; + if (!pnv_pci_is_mem_pref_64(res->flags)) { + dev_warn(&pdev->dev, "Skipping expanding VF BAR%d: %pR\n", + i, res); + continue; + } + + dev_dbg(&pdev->dev, " Fixing VF BAR%d: %pR to\n", i, res); + size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); + res->end = res->start + size * phb->ioda.total_pe - 1; + dev_dbg(&pdev->dev, " %pR\n", res); + dev_info(&pdev->dev, "VF BAR%d: %pR (expanded to %d VFs for PE alignment)", + i, res, phb->ioda.total_pe); + } + pdn->max_vfs = phb->ioda.total_pe; +} + +static void pnv_pci_ioda_fixup_sriov(struct pci_bus *bus) +{ + struct pci_dev *pdev; + struct pci_bus *b; + + list_for_each_entry(pdev, &bus->devices, bus_list) { + b = pdev->subordinate; + + if (b) + pnv_pci_ioda_fixup_sriov(b); + + pnv_pci_ioda_fixup_iov_resources(pdev); + } +} +#endif /* CONFIG_PCI_IOV */ + /* * This function is supposed to be called on basis of PE from top * to bottom style. So the the I/O or MMIO segment assigned to @@ -2125,6 +2180,9 @@ static void __init pnv_pci_init_ioda_phb(struct device_node *np, ppc_md.pcibios_enable_device_hook = pnv_pci_enable_device_hook; ppc_md.pcibios_window_alignment = pnv_pci_window_alignment; ppc_md.pcibios_reset_secondary_bus = pnv_pci_reset_secondary_bus; +#ifdef CONFIG_PCI_IOV + ppc_md.pcibios_fixup_sriov = pnv_pci_ioda_fixup_sriov; +#endif /* CONFIG_PCI_IOV */ pci_add_flags(PCI_REASSIGN_ALL_RSRC); /* Reset IODA tables to a clean state */