| Message ID | 1437092083-20672-3-git-send-email-gwshan@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | Rejected |
| Headers | show |
On Fri, Jul 17, 2015 at 10:14:43AM +1000, Gavin Shan wrote: >When VF BAR size is equal to 128MB or bigger than that, we extend >the corresponding PF's IOV BAR to cover number of total VFs supported >by the PF. Otherwise, we extend the PF's IOV BAR to cover 256 VFs. >For the former case, we have to create compound PE, which includes >4 VFs. Those 4 VFs included in the compound PE can't be passed through >to different guests, which isn't good. > >The gate (128MB) was choosen based on the assumption that each PHB >supports 64GB M64 space and one PF's IOV BAR can be extended to be >as huge as 1/4 of that, which is 16GB. However, the IOV BAR can be >extended to half of PHB's M64 window when the PF seats behind the >root port. In that case, the gate can be enlarged to be 256MB to >avoid compound PE as we can. > >Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com> >--- > arch/powerpc/platforms/powernv/pci-ioda.c | 21 ++++++++++++++++----- > 1 file changed, 16 insertions(+), 5 deletions(-) > >diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c >index 6ec62b9..5b2e88f 100644 >--- a/arch/powerpc/platforms/powernv/pci-ioda.c >+++ b/arch/powerpc/platforms/powernv/pci-ioda.c >@@ -2721,6 +2721,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > struct resource *res; > int i; > resource_size_t size; >+ resource_size_t limit; > struct pci_dn *pdn; > int mul, total_vfs; > >@@ -2730,6 +2731,18 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > hose = pci_bus_to_host(pdev->bus); > phb = hose->private_data; > >+ /* >+ * When the PF seats behind root port, the IOV BAR can >+ * consume half of the PHB's M64 window. Otherwise, >+ * 1/4 of the PHB's M64 window can be consumed to the >+ * maximal degree. >+ */ >+ if (!pci_is_root_bus(pdev->bus) && >+ pci_is_root_bus(pdev->bus->self->bus)) >+ limit = 128; >+ else >+ limit = 256; >+ I sent it too fast. The limit should be reversed: 256 when PF seats behind the root port. Otherwise, it should be 128. I will send follow-up v2 after waiting for couple of days in case there are some comments for this revision. > pdn = pci_get_pdn(pdev); > pdn->vfs_expanded = 0; > >@@ -2748,11 +2761,9 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) > } > > size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); >- >- /* bigger than 64M */ >- if (size > (1 << 26)) { >- dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than 64M, roundup power2\n", >- i, res); >+ if (size >= (limit * 0x100000)) { >+ dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than %lldMB, roundup power2\n", >+ i, res, limit); > pdn->m64_per_iov = M64_PER_IOV; > mul = roundup_pow_of_two(total_vfs); > break; Thanks, Gavin
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index 6ec62b9..5b2e88f 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -2721,6 +2721,7 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) struct resource *res; int i; resource_size_t size; + resource_size_t limit; struct pci_dn *pdn; int mul, total_vfs; @@ -2730,6 +2731,18 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) hose = pci_bus_to_host(pdev->bus); phb = hose->private_data; + /* + * When the PF seats behind root port, the IOV BAR can + * consume half of the PHB's M64 window. Otherwise, + * 1/4 of the PHB's M64 window can be consumed to the + * maximal degree. + */ + if (!pci_is_root_bus(pdev->bus) && + pci_is_root_bus(pdev->bus->self->bus)) + limit = 128; + else + limit = 256; + pdn = pci_get_pdn(pdev); pdn->vfs_expanded = 0; @@ -2748,11 +2761,9 @@ static void pnv_pci_ioda_fixup_iov_resources(struct pci_dev *pdev) } size = pci_iov_resource_size(pdev, i + PCI_IOV_RESOURCES); - - /* bigger than 64M */ - if (size > (1 << 26)) { - dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than 64M, roundup power2\n", - i, res); + if (size >= (limit * 0x100000)) { + dev_info(&pdev->dev, "PowerNV: VF BAR%d: %pR IOV size is bigger than %lldMB, roundup power2\n", + i, res, limit); pdn->m64_per_iov = M64_PER_IOV; mul = roundup_pow_of_two(total_vfs); break;
When VF BAR size is equal to 128MB or bigger than that, we extend the corresponding PF's IOV BAR to cover number of total VFs supported by the PF. Otherwise, we extend the PF's IOV BAR to cover 256 VFs. For the former case, we have to create compound PE, which includes 4 VFs. Those 4 VFs included in the compound PE can't be passed through to different guests, which isn't good. The gate (128MB) was choosen based on the assumption that each PHB supports 64GB M64 space and one PF's IOV BAR can be extended to be as huge as 1/4 of that, which is 16GB. However, the IOV BAR can be extended to half of PHB's M64 window when the PF seats behind the root port. In that case, the gate can be enlarged to be 256MB to avoid compound PE as we can. Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com> --- arch/powerpc/platforms/powernv/pci-ioda.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)