| Message ID | 20200331012338.23773-1-aik@ozlabs.ru (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 54fc3c681ded9437e4548e2501dc1136b23cfa9a |
| Headers | show |
| Series | [kernel] powerpc/pseries/ddw: Extend upper limit for huge DMA window for persistent memory | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | Successfully applied on branch powerpc/merge (c6624071c338732402e8c726df6a4074473eaa0e) |
| snowpatch_ozlabs/build-ppc64le | success | Build succeeded |
| snowpatch_ozlabs/build-ppc64be | success | Build succeeded |
| snowpatch_ozlabs/build-ppc64e | success | Build succeeded |
| snowpatch_ozlabs/build-pmac32 | success | Build succeeded |
| snowpatch_ozlabs/checkpatch | warning | total: 0 errors, 0 warnings, 1 checks, 15 lines checked |
| snowpatch_ozlabs/needsstable | success | Patch has no Fixes tags |
Hi Wen, Can you please try this? It contains 3 patches from Christoph Hellwig plus my patch on top, this should improve performance when DMA-ing not to/from persistent memory: https://github.com/aik/linux/commits/dma-bypass.3 I am looking for any benchmarks not related to persistent memory. Thanks, On 03/04/2020 09:09, Wen Xiong wrote: > I applied the patch on top of the latest upstream kernel. I ran HTX over > pmem nodes for several hours and it works. > > Tested-by: Wen Xiong<wenxiong@linux.vnet.ibm.com> > > Thanks, > Wendy > > ----- Original message ----- > From: Alexey Kardashevskiy <aik@ozlabs.ru> > To: linuxppc-dev@lists.ozlabs.org > Cc: Alexey Kardashevskiy <aik@ozlabs.ru>, David Gibson > <david@gibson.dropbear.id.au>, Michael Ellerman > <mpe@ellerman.id.au>, Oliver O'Halloran <oohall@gmail.com>, "Aneesh > Kumar K . V" <aneesh.kumar@linux.ibm.com>, Wen Xiong > <wenxiong@us.ibm.com>, Brian J King <bjking1@us.ibm.com> > Subject: [EXTERNAL] [PATCH kernel] powerpc/pseries/ddw: Extend upper > limit for huge DMA window for persistent memory > Date: Mon, Mar 30, 2020 8:23 PM > > Unlike normal memory ("memory" compatible type in the FDT), > the persistent memory ("ibm,pmemory" in the FDT) can be mapped anywhere > in the guest physical space and it can be used for DMA. > > In order to maintain 1:1 mapping via the huge DMA window, we need to > know the maximum physical address at the time of the window setup. > So far we've been looking at "memory" nodes but "ibm,pmemory" does not > have fixed addresses and the persistent memory may be mapped afterwards. > > Since the persistent memory is still backed with page structs, > use MAX_PHYSMEM_BITS as the upper limit. > > This effectively disables huge DMA window in LPAR under pHyp if > persistent memory is present but this is the best we can do. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > arch/powerpc/platforms/pseries/iommu.c | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/arch/powerpc/platforms/pseries/iommu.c > b/arch/powerpc/platforms/pseries/iommu.c > index 2e0a8eab5588..6d47b4a3ce39 100644 > --- a/arch/powerpc/platforms/pseries/iommu.c > +++ b/arch/powerpc/platforms/pseries/iommu.c > @@ -945,6 +945,15 @@ static phys_addr_t ddw_memory_hotplug_max(void) > phys_addr_t max_addr = memory_hotplug_max(); > struct device_node *memory; > > + /* > + * The "ibm,pmemory" can appear anywhere in the address space. > + * Assuming it is still backed by page structs, set the upper limit > + * for the huge DMA window as MAX_PHYSMEM_BITS. > + */ > + if (of_find_node_by_type(NULL, "ibm,pmemory")) > + return (sizeof(phys_addr_t) * 8 <= MAX_PHYSMEM_BITS) ? > + (phys_addr_t) -1 : (1ULL << MAX_PHYSMEM_BITS); > + > for_each_node_by_type(memory, "memory") { > unsigned long start, size; > int n_mem_addr_cells, n_mem_size_cells, len; > -- > 2.17.1 > > > >
Alexey Kardashevskiy <aik@ozlabs.ru> writes: > Unlike normal memory ("memory" compatible type in the FDT), > the persistent memory ("ibm,pmemory" in the FDT) can be mapped anywhere > in the guest physical space and it can be used for DMA. > > In order to maintain 1:1 mapping via the huge DMA window, we need to > know the maximum physical address at the time of the window setup. > So far we've been looking at "memory" nodes but "ibm,pmemory" does not > have fixed addresses and the persistent memory may be mapped afterwards. > > Since the persistent memory is still backed with page structs, > use MAX_PHYSMEM_BITS as the upper limit. > > This effectively disables huge DMA window in LPAR under pHyp if > persistent memory is present but this is the best we can do. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> Thanks for the patch Alexy, LGTM. Reviewed-by: Vaibhav Jain <vaibhav@linux.ibm.com> > --- > arch/powerpc/platforms/pseries/iommu.c | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c > index 2e0a8eab5588..6d47b4a3ce39 100644 > --- a/arch/powerpc/platforms/pseries/iommu.c > +++ b/arch/powerpc/platforms/pseries/iommu.c > @@ -945,6 +945,15 @@ static phys_addr_t ddw_memory_hotplug_max(void) > phys_addr_t max_addr = memory_hotplug_max(); > struct device_node *memory; > > + /* > + * The "ibm,pmemory" can appear anywhere in the address space. > + * Assuming it is still backed by page structs, set the upper limit > + * for the huge DMA window as MAX_PHYSMEM_BITS. > + */ > + if (of_find_node_by_type(NULL, "ibm,pmemory")) > + return (sizeof(phys_addr_t) * 8 <= MAX_PHYSMEM_BITS) ? > + (phys_addr_t) -1 : (1ULL << MAX_PHYSMEM_BITS); > + > for_each_node_by_type(memory, "memory") { > unsigned long start, size; > int n_mem_addr_cells, n_mem_size_cells, len; > -- > 2.17.1 >
On Tue, 2020-03-31 at 01:23:38 UTC, Alexey Kardashevskiy wrote: > Unlike normal memory ("memory" compatible type in the FDT), > the persistent memory ("ibm,pmemory" in the FDT) can be mapped anywhere > in the guest physical space and it can be used for DMA. > > In order to maintain 1:1 mapping via the huge DMA window, we need to > know the maximum physical address at the time of the window setup. > So far we've been looking at "memory" nodes but "ibm,pmemory" does not > have fixed addresses and the persistent memory may be mapped afterwards. > > Since the persistent memory is still backed with page structs, > use MAX_PHYSMEM_BITS as the upper limit. > > This effectively disables huge DMA window in LPAR under pHyp if > persistent memory is present but this is the best we can do. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> Applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/54fc3c681ded9437e4548e2501dc1136b23cfa9a cheers
diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 2e0a8eab5588..6d47b4a3ce39 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -945,6 +945,15 @@ static phys_addr_t ddw_memory_hotplug_max(void) phys_addr_t max_addr = memory_hotplug_max(); struct device_node *memory; + /* + * The "ibm,pmemory" can appear anywhere in the address space. + * Assuming it is still backed by page structs, set the upper limit + * for the huge DMA window as MAX_PHYSMEM_BITS. + */ + if (of_find_node_by_type(NULL, "ibm,pmemory")) + return (sizeof(phys_addr_t) * 8 <= MAX_PHYSMEM_BITS) ? + (phys_addr_t) -1 : (1ULL << MAX_PHYSMEM_BITS); + for_each_node_by_type(memory, "memory") { unsigned long start, size; int n_mem_addr_cells, n_mem_size_cells, len;
Unlike normal memory ("memory" compatible type in the FDT), the persistent memory ("ibm,pmemory" in the FDT) can be mapped anywhere in the guest physical space and it can be used for DMA. In order to maintain 1:1 mapping via the huge DMA window, we need to know the maximum physical address at the time of the window setup. So far we've been looking at "memory" nodes but "ibm,pmemory" does not have fixed addresses and the persistent memory may be mapped afterwards. Since the persistent memory is still backed with page structs, use MAX_PHYSMEM_BITS as the upper limit. This effectively disables huge DMA window in LPAR under pHyp if persistent memory is present but this is the best we can do. Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> --- arch/powerpc/platforms/pseries/iommu.c | 9 +++++++++ 1 file changed, 9 insertions(+)