From patchwork Tue Feb 24 08:35:04 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bjorn Helgaas X-Patchwork-Id: 442873 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 73D6F140182 for ; Tue, 24 Feb 2015 19:47:46 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 5C9F31A2EB1 for ; Tue, 24 Feb 2015 19:47:46 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from mail-pa0-x230.google.com (mail-pa0-x230.google.com [IPv6:2607:f8b0:400e:c03::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 1BF741A10C8 for ; Tue, 24 Feb 2015 19:35:09 +1100 (AEDT) Received: by pabrd3 with SMTP id rd3so34441998pab.1 for ; Tue, 24 Feb 2015 00:35:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=subject:to:from:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=Pa5tgkVu3CssC6PMVNJj5y80S0udrin/wqXQFc6j4fU=; b=WCzigCnOVFtBDqgf0aIBYlQ6/HHeyvdrnjYIkM7CXUzEySV2egZH/6M/Ujc23rMmi/ h4wOuDFe7u3eLB7UuSSjI5fm+IyK+B77QdnogYXDSeLgYqo5lN9+l48Tapqyxhh6w0Vd xDl0zcdx+5rmuK3xxCWpVf6dFh6TbgLH/2dGjEYw8Qq8msf1IPIaJSDifoUS8ZJ9XY8j 5srjh4r4N4EloDpXxcI09LzLKc2jFh6sg/cAfhNtH79hzmVAZ847syj1Afif4ssEPQWP vtBhZbtiGbH70EsDCHjO9LYM/d2jM9kge//HXU/O4P8gl1SZjUKChVmj/xb+ZXim9tIW O9aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:from:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-type :content-transfer-encoding; bh=Pa5tgkVu3CssC6PMVNJj5y80S0udrin/wqXQFc6j4fU=; b=litgIIOsiC3bpKzqeN/TRsbqMwkJw6o/5NEk2xvOLHMkzuPSe5ymyCEFqMjiSi4To+ o93bbDsoLBsSSrGDiuadq+sn+amFiaZLAMvP+sJiCUmMqFjsn2QxLsnICBDaPqA7461x nYwWDjL7uknvJHY7MfU1saPf/KTUld/MuOq8/Tz4I+lZ2SD5KwOkyw2a9wQMjbnSni25 hNiSgF8Tilq6MH8P+y1W/MVZiLJt0qPu8RUcTO4qlEnOhsyb8eZ09bqHLn2DLT7e6NfJ ZmjfrA2GdfTM9/SSHWntQJ3rixzFbjyB+v4Q+WKmQJTlEYu+jYe/oLhckAynAFzG+hd5 lFDA== X-Gm-Message-State: ALoCoQl2xj7n5nyj3eGAxnQUqx6nIkcXDYAoCD1HOFdAmyhUsf8P2BRFOC+bKnI7273wrJbo23/Z X-Received: by 10.70.133.130 with SMTP id pc2mr26298532pdb.99.1424766907386; Tue, 24 Feb 2015 00:35:07 -0800 (PST) Received: from localhost ([12.23.74.29]) by mx.google.com with ESMTPSA id ez1sm38246524pdb.22.2015.02.24.00.35.06 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 24 Feb 2015 00:35:06 -0800 (PST) Subject: [PATCH v12 18/21] powerpc/powernv: Reserve additional space for IOV BAR, with m64_per_iov supported To: Wei Yang , benh@au1.ibm.com, gwshan@linux.vnet.ibm.com From: Bjorn Helgaas Date: Tue, 24 Feb 2015 02:35:04 -0600 Message-ID: <20150224083504.32124.23374.stgit@bhelgaas-glaptop2.roam.corp.google.com> In-Reply-To: <20150224082939.32124.45744.stgit@bhelgaas-glaptop2.roam.corp.google.com> References: <20150224082939.32124.45744.stgit@bhelgaas-glaptop2.roam.corp.google.com> User-Agent: StGit/0.16 MIME-Version: 1.0 Cc: linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From: Wei Yang M64 aperture size is limited on PHB3. When the IOV BAR is too big, this will exceed the limitation and failed to be assigned. Introduce a different mechanism based on the IOV BAR size: - if IOV BAR size is smaller than 64MB, expand to total_pe - if IOV BAR size is bigger than 64MB, roundup power2 [bhelgaas: make dev_printk() output more consistent, use PCI_SRIOV_NUM_BARS] Signed-off-by: Wei Yang Signed-off-by: Bjorn Helgaas --- arch/powerpc/include/asm/pci-bridge.h | 2 ++ arch/powerpc/platforms/powernv/pci-ioda.c | 33 ++++++++++++++++++++++++++--- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/include/asm/pci-bridge.h b/arch/powerpc/include/asm/pci-bridge.h index 011340df8583..d824bb184ab8 100644 --- a/arch/powerpc/include/asm/pci-bridge.h +++ b/arch/powerpc/include/asm/pci-bridge.h @@ -179,6 +179,8 @@ struct pci_dn { u16 max_vfs; /* number of VFs IOV BAR expended */ u16 vf_pes; /* VF PE# under this PF */ int offset; /* PE# for the first VF PE */ +#define M64_PER_IOV 4 + int m64_per_iov; #define IODA_INVALID_M64 (-1) int m64_wins[PCI_SRIOV_NUM_BARS]; #endif /* CONFIG_PCI_IOV */ diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index a3c2fbe35fc8..30b7c3909746 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -2242,6 +2242,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) int i; resource_size_t size; struct pci_dn *pdn; + int mul, total_vfs; if (!pdev->is_physfn || pdev->is_added) return; @@ -2252,6 +2253,32 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) pdn = pci_get_pdn(pdev); pdn->max_vfs = 0; + total_vfs = pci_sriov_get_totalvfs(pdev); + pdn->m64_per_iov = 1; + mul = phb->ioda.total_pe; + + for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) { + res = &pdev->resource[i + PCI_IOV_RESOURCES]; + if (!res->flags || res->parent) + continue; + if (!pnv_pci_is_mem_pref_64(res->flags)) { + dev_warn(&pdev->dev, " non M64 VF BAR%d: %pR\n", + i, res); + continue; + } + + size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); + + /* bigger than 64M */ + if (size > (1 << 26)) { + dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than 64M, roundup power2\n", + i, res); + pdn->m64_per_iov = M64_PER_IOV; + mul = __roundup_pow_of_two(total_vfs); + break; + } + } + for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) { res = &pdev->resource[i + PCI_IOV_RESOURCES]; if (!res->flags || res->parent) @@ -2264,12 +2291,12 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) dev_dbg(&pdev->dev, " Fixing VF BAR%d: %pR to\n", i, res); size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); - res->end = res->start + size * phb->ioda.total_pe - 1; + res->end = res->start + size * mul - 1; dev_dbg(&pdev->dev, " %pR\n", res); dev_info(&pdev->dev, "VF BAR%d: %pR (expanded to %d VFs for PE alignment)", - i, res, phb->ioda.total_pe); + i, res, mul); } - pdn->max_vfs = phb->ioda.total_pe; + pdn->max_vfs = mul; } static void pnv_pci_ioda_fixup_sriov(struct pci_bus *bus)