From patchwork Fri Jul 17 00:14:43 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 496957 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id BE6481402B5 for ; Fri, 17 Jul 2015 10:17:53 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 9F3B11A1D1F for ; Fri, 17 Jul 2015 10:17:53 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from e23smtp05.au.ibm.com (e23smtp05.au.ibm.com [202.81.31.147]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id F33281A075D for ; Fri, 17 Jul 2015 10:15:45 +1000 (AEST) Received: from /spool/local by e23smtp05.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 17 Jul 2015 10:15:45 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp05.au.ibm.com (202.81.31.211) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 17 Jul 2015 10:15:43 +1000 X-Helo: d23dlp02.au.ibm.com X-MailFrom: gwshan@linux.vnet.ibm.com X-RcptTo: linuxppc-dev@lists.ozlabs.org Received: from d23relay07.au.ibm.com (d23relay07.au.ibm.com [9.190.26.37]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id 834332BB004D for ; Fri, 17 Jul 2015 10:15:42 +1000 (EST) Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay07.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t6H0FZmV41615580 for ; Fri, 17 Jul 2015 10:15:43 +1000 Received: from d23av03.au.ibm.com (localhost [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t6H0F94J008419 for ; Fri, 17 Jul 2015 10:15:09 +1000 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id t6H0F9Oa007649; Fri, 17 Jul 2015 10:15:09 +1000 Received: from bran.ozlabs.ibm.com (unknown [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id 00AE8A03CB; Fri, 17 Jul 2015 10:14:45 +1000 (AEST) Received: from gwshan (shangw.ozlabs.ibm.com [10.61.2.199]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id ECAB4E38E6; Fri, 17 Jul 2015 10:14:44 +1000 (AEST) Received: by gwshan (Postfix, from userid 1000) id E2CB1941E93; Fri, 17 Jul 2015 10:14:44 +1000 (AEST) From: Gavin Shan To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 2/2] powerpc/powernv: Double VF BAR size for compound PE Date: Fri, 17 Jul 2015 10:14:43 +1000 Message-Id: <1437092083-20672-3-git-send-email-gwshan@linux.vnet.ibm.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1437092083-20672-1-git-send-email-gwshan@linux.vnet.ibm.com> References: <1437092083-20672-1-git-send-email-gwshan@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15071700-0017-0000-0000-0000018D0F86 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, paulus@samba.org, Gavin Shan MIME-Version: 1.0 Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" When VF BAR size is equal to 128MB or bigger than that, we extend the corresponding PF's IOV BAR to cover number of total VFs supported by the PF. Otherwise, we extend the PF's IOV BAR to cover 256 VFs. For the former case, we have to create compound PE, which includes 4 VFs. Those 4 VFs included in the compound PE can't be passed through to different guests, which isn't good. The gate (128MB) was choosen based on the assumption that each PHB supports 64GB M64 space and one PF's IOV BAR can be extended to be as huge as 1/4 of that, which is 16GB. However, the IOV BAR can be extended to half of PHB's M64 window when the PF seats behind the root port. In that case, the gate can be enlarged to be 256MB to avoid compound PE as we can. Signed-off-by: Gavin Shan --- arch/powerpc/platforms/powernv/pci-ioda.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index 6ec62b9..5b2e88f 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -2721,6 +2721,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) struct resource *res; int i; resource_size_t size; + resource_size_t limit; struct pci_dn *pdn; int mul, total_vfs; @@ -2730,6 +2731,18 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) hose = pci_bus_to_host(pdev->bus); phb = hose->private_data; + /* + * When the PF seats behind root port, the IOV BAR can + * consume half of the PHB's M64 window. Otherwise, + * 1/4 of the PHB's M64 window can be consumed to the + * maximal degree. + */ + if (!pci_is_root_bus(pdev->bus) && + pci_is_root_bus(pdev->bus->self->bus)) + limit = 128; + else + limit = 256; + pdn = pci_get_pdn(pdev); pdn->vfs_expanded = 0; @@ -2748,11 +2761,9 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) } size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); - - /* bigger than 64M */ - if (size > (1 << 26)) { - dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than 64M, roundup power2\n", - i, res); + if (size >= (limit * 0x100000)) { + dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than %lldMB, roundup power2\n", + i, res, limit); pdn->m64_per_iov = M64_PER_IOV; mul = roundup_pow_of_two(total_vfs); break;