| Message ID | cover.1545066805.git.gustavo.pimentel@synopsys.com |
|---|---|
| Headers | show |
| Series | dmaengine: Add Synopsys eDMA IP driver (version 0) | expand |
On Mon, Dec 17, 2018 at 06:19:32PM +0100, Gustavo Pimentel wrote: > Add Synopsys eDMA IP driver (version 0) to Linux kernel. This IP is generally > distributed with Synopsys PCIe EndPoint IP (depends of the use and licensing > agreement), which supports: > - legacy and unroll modes > - 16 independent and concurrent channels (8 write + 8 read) > - supports linked list (scatter-gather) transfer > - each linked list descriptor can transfer from 1 byte to 4 Gbytes > - PCIe EndPoint glue-logic > > Gustavo Pimentel (6): > dmaengine: Add Synopsys eDMA IP core driver > dmaengine: Add Synopsys eDMA IP version 0 support > dmaengine: Add Synopsys eDMA IP version 0 debugfs support > PCI: Add Synopsys endpoint EDDA Device id > dmaengine: Add Synopsys eDMA IP PCIe glue-logic > MAINTAINERS: Add Synopsys eDMA IP driver maintainer Hello Gustavo, Nice to see support for the embedded DMA controller used in the DWC PCIe controller. Some kind of documentation in Documentation/ would be nice. The eDMA can be used both while the PCIe controller is in EP and RC mode, right? I guess that this support is only to use the eDMA for RC mode? Do you have any benchmarks to share? (Perhaps with different buffer sizes). Not sure what we will see, lower CPU utilization but more IRQs? Have you thought about doing IRQ coalescing? Usually we need dmas and dma-names in device tree, however, maybe that isn't needed here, since it is an embedded DMA controller. Don't we even need/want a DT property in the PCIe node, that says if we want to enable the eDMA or not? Usually a PCIe EP driver will call pci_ioremap_bar() and then use readl()/writel() directly. But since we now need to setup a DMA transfer, and wait for an IRQ, how does this work? I have a board that has a CC_DMA_ENABLE set, so if you give me some detailed instructions how to test, I might be able to provide you with a Tested-by. Kind regards, Niklas > > MAINTAINERS | 7 + > drivers/dma/Kconfig | 2 + > drivers/dma/Makefile | 1 + > drivers/dma/dw-edma/Kconfig | 18 + > drivers/dma/dw-edma/Makefile | 7 + > drivers/dma/dw-edma/dw-edma-core.c | 845 +++++++++++++++++++++++++++++++ > drivers/dma/dw-edma/dw-edma-core.h | 147 ++++++ > drivers/dma/dw-edma/dw-edma-pcie.c | 258 ++++++++++ > drivers/dma/dw-edma/dw-edma-v0-core.c | 361 +++++++++++++ > drivers/dma/dw-edma/dw-edma-v0-core.h | 26 + > drivers/dma/dw-edma/dw-edma-v0-debugfs.c | 357 +++++++++++++ > drivers/dma/dw-edma/dw-edma-v0-debugfs.h | 24 + > drivers/dma/dw-edma/dw-edma-v0-regs.h | 145 ++++++ > drivers/misc/pci_endpoint_test.c | 2 +- > include/linux/dma/edma.h | 43 ++ > include/linux/pci_ids.h | 1 + > 16 files changed, 2243 insertions(+), 1 deletion(-) > create mode 100644 drivers/dma/dw-edma/Kconfig > create mode 100644 drivers/dma/dw-edma/Makefile > create mode 100644 drivers/dma/dw-edma/dw-edma-core.c > create mode 100644 drivers/dma/dw-edma/dw-edma-core.h > create mode 100644 drivers/dma/dw-edma/dw-edma-pcie.c > create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.c > create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.h > create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.c > create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.h > create mode 100644 drivers/dma/dw-edma/dw-edma-v0-regs.h > create mode 100644 include/linux/dma/edma.h > > Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com> > Cc: Vinod Koul <vkoul@kernel.org> > Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> > Cc: Russell King <rmk+kernel@armlinux.org.uk> > Cc: Eugeniy Paltsev <paltsev@synopsys.com> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > Cc: Bjorn Helgaas <bhelgaas@google.com> > Cc: Kishon Vijay Abraham I <kishon@ti.com> > Cc: Niklas Cassel <niklas.cassel@linaro.org> > Cc: Joao Pinto <jpinto@synopsys.com> > Cc: Jose Abreu <jose.abreu@synopsys.com> > Cc: Luis Oliveira <lolivei@synopsys.com> > Cc: Vitor Soares <vitor.soares@synopsys.com> > Cc: Nelson Costa <nelson.costa@synopsys.com> > Cc: Pedro Sousa <pedrom.sousa@synopsys.com> > > -- > 2.7.4 >
Hi, On 17/12/2018 22:21, Niklas Cassel wrote: > On Mon, Dec 17, 2018 at 06:19:32PM +0100, Gustavo Pimentel wrote: >> Add Synopsys eDMA IP driver (version 0) to Linux kernel. This IP is generally >> distributed with Synopsys PCIe EndPoint IP (depends of the use and licensing >> agreement), which supports: >> - legacy and unroll modes >> - 16 independent and concurrent channels (8 write + 8 read) >> - supports linked list (scatter-gather) transfer >> - each linked list descriptor can transfer from 1 byte to 4 Gbytes >> - PCIe EndPoint glue-logic >> >> Gustavo Pimentel (6): >> dmaengine: Add Synopsys eDMA IP core driver >> dmaengine: Add Synopsys eDMA IP version 0 support >> dmaengine: Add Synopsys eDMA IP version 0 debugfs support >> PCI: Add Synopsys endpoint EDDA Device id >> dmaengine: Add Synopsys eDMA IP PCIe glue-logic >> MAINTAINERS: Add Synopsys eDMA IP driver maintainer > > Hello Gustavo, > > Nice to see support for the embedded DMA controller used in the DWC PCIe > controller. Yes, I included you because you were from the beginning one of those who requested this feature. :) > > Some kind of documentation in Documentation/ would be nice. Yes, I thought so too, but under of DOCUMENTATION folder doesn't have anything related with DMA drivers implementations either on topic of DMA or PCI. I thought maybe putting on the dw-edma folder but that doesn't follow the standard documentation flow. > > > The eDMA can be used both while the PCIe controller is in EP and RC mode, > right? For now I can not give this certainty. My setup only have the eDMA on the EP side and has the eDMA mapped on PCI BAR, which allows me to configure it through PCI. > I guess that this support is only to use the eDMA for RC mode? For now this driver runs on the RC side (host/CPU) and configures the eDMA EP. I've some plans to support eDMA configuration but on the EP side. I'm guessing that also typical use case. > > Do you have any benchmarks to share? (Perhaps with different buffer sizes). I'm currently working on the dw-edma-test driver (similar to dmatest driver) that supports DEV_TO_MEM and MEM_TO_DEV directions instead of MEM_TO_MEM (dmatest), and adapted to WRITE and READ channels with multiple threads. After finishing this driver I'll able to do some benchmarks and comparisons. :) Given your enthusiasm, when I'm finished we could do some brainstorm and define some test cases / benchmarks, what do you think? > Not sure what we will see, lower CPU utilization but more IRQs? Probably yes. Let me explain you With this implementation, you can transfer several chunks (a chunk is linked list of burst elements). Each burst basically defines the source, destination addresses and size to transfer (from 1 Byte to 4 GB). This linked list will be written on EP's RAM (mapped on a PCI BAR) which the HW block will consume it. The linked list size is dependent of this EP's RAM size, i.e. 8 KB will allow 340 elements (bursts). On my currently setup I'm testing the driver implementation like this: - 4 chunks - each chunk is composed by 340 elements (bursts) - each burst have a transfer size of 100 bytes (can be up to 4GB and the size can be also different for each burst) Chunk #0 |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) Chunk #1 |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) Chunk #2 |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) Chunk #3 |--> Burst #0 --> ... --> Burst #34 -> IRQ (End transfer) In total I'm writing ((4 * 340) + 35) * 100 = 139500 Bytes <=> 136 KB > Have you thought about doing IRQ coalescing? No. I'm only focus to implement the driver for now. After the implementation and the benchmarks, maybe we could think about it. :) > > Usually we need dmas and dma-names in device tree, however, maybe that > isn't needed here, since it is an embedded DMA controller. > Don't we even need/want a DT property in the PCIe node, that says if we > want to enable the eDMA or not? That will be necessary in the future when we have eDMA in RC or when we implement the driver to control the eDMA on the EP side. > > Usually a PCIe EP driver will call pci_ioremap_bar() and then use > readl()/writel() directly. > But since we now need to setup a DMA transfer, and wait for an IRQ, > how does this work? I gave a brief explanation in a previous comment in this email (see info about the chunks and bursts) I'm dividing this patch series like this: - eDMA core + eDMA core v0 driver (implementing the interface with DMAengine controller APIs and the interface with the eDMA HW block) - eDMA PCIe glue-logic driver (attaches to EP and provides memory access to eDMA core driver), for now is attach to Synopsys Vendor and Device ID and has a platform data specific to our EP. - eDMA Test driver (is a sample driver which aims to test the eDMA, but also provides a example how an customer driver would use the eDMA driver, through the DMAengine client APIs) (under developing, I hope to make it available before xmas :)) Sounds logic? > > I have a board that has a CC_DMA_ENABLE set, so if you give me some > detailed instructions how to test, I might be able to provide you with a > Tested-by. Ok, let me just finish the eDMA test and append it to the patch series. I'm happy to help you with this. :) Thanks! Gustavo > > > Kind regards, > Niklas > > >> >> MAINTAINERS | 7 + >> drivers/dma/Kconfig | 2 + >> drivers/dma/Makefile | 1 + >> drivers/dma/dw-edma/Kconfig | 18 + >> drivers/dma/dw-edma/Makefile | 7 + >> drivers/dma/dw-edma/dw-edma-core.c | 845 +++++++++++++++++++++++++++++++ >> drivers/dma/dw-edma/dw-edma-core.h | 147 ++++++ >> drivers/dma/dw-edma/dw-edma-pcie.c | 258 ++++++++++ >> drivers/dma/dw-edma/dw-edma-v0-core.c | 361 +++++++++++++ >> drivers/dma/dw-edma/dw-edma-v0-core.h | 26 + >> drivers/dma/dw-edma/dw-edma-v0-debugfs.c | 357 +++++++++++++ >> drivers/dma/dw-edma/dw-edma-v0-debugfs.h | 24 + >> drivers/dma/dw-edma/dw-edma-v0-regs.h | 145 ++++++ >> drivers/misc/pci_endpoint_test.c | 2 +- >> include/linux/dma/edma.h | 43 ++ >> include/linux/pci_ids.h | 1 + >> 16 files changed, 2243 insertions(+), 1 deletion(-) >> create mode 100644 drivers/dma/dw-edma/Kconfig >> create mode 100644 drivers/dma/dw-edma/Makefile >> create mode 100644 drivers/dma/dw-edma/dw-edma-core.c >> create mode 100644 drivers/dma/dw-edma/dw-edma-core.h >> create mode 100644 drivers/dma/dw-edma/dw-edma-pcie.c >> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.c >> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.h >> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.c >> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.h >> create mode 100644 drivers/dma/dw-edma/dw-edma-v0-regs.h >> create mode 100644 include/linux/dma/edma.h >> >> Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com> >> Cc: Vinod Koul <vkoul@kernel.org> >> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> >> Cc: Russell King <rmk+kernel@armlinux.org.uk> >> Cc: Eugeniy Paltsev <paltsev@synopsys.com> >> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >> Cc: Bjorn Helgaas <bhelgaas@google.com> >> Cc: Kishon Vijay Abraham I <kishon@ti.com> >> Cc: Niklas Cassel <niklas.cassel@linaro.org> >> Cc: Joao Pinto <jpinto@synopsys.com> >> Cc: Jose Abreu <jose.abreu@synopsys.com> >> Cc: Luis Oliveira <lolivei@synopsys.com> >> Cc: Vitor Soares <vitor.soares@synopsys.com> >> Cc: Nelson Costa <nelson.costa@synopsys.com> >> Cc: Pedro Sousa <pedrom.sousa@synopsys.com> >> >> -- >> 2.7.4 >>
On Tue, Dec 18, 2018 at 10:36:58AM +0000, Gustavo Pimentel wrote: > Hi, > > On 17/12/2018 22:21, Niklas Cassel wrote: > > On Mon, Dec 17, 2018 at 06:19:32PM +0100, Gustavo Pimentel wrote: > >> Add Synopsys eDMA IP driver (version 0) to Linux kernel. This IP is generally > >> distributed with Synopsys PCIe EndPoint IP (depends of the use and licensing > >> agreement), which supports: > >> - legacy and unroll modes > >> - 16 independent and concurrent channels (8 write + 8 read) > >> - supports linked list (scatter-gather) transfer > >> - each linked list descriptor can transfer from 1 byte to 4 Gbytes > >> - PCIe EndPoint glue-logic > >> > >> Gustavo Pimentel (6): > >> dmaengine: Add Synopsys eDMA IP core driver > >> dmaengine: Add Synopsys eDMA IP version 0 support > >> dmaengine: Add Synopsys eDMA IP version 0 debugfs support > >> PCI: Add Synopsys endpoint EDDA Device id > >> dmaengine: Add Synopsys eDMA IP PCIe glue-logic > >> MAINTAINERS: Add Synopsys eDMA IP driver maintainer > > > > Hello Gustavo, > > > > Nice to see support for the embedded DMA controller used in the DWC PCIe > > controller. > > Yes, I included you because you were from the beginning one of those who > requested this feature. :) Hello Gustavo, sorry for the delayed response, I took an extra long christmas vacation :) Unfortunately, it was my previous employer (Axis) who was interested in this. Right now I am simply looking at this patch series out of my own curiosity :) (snip) > > The eDMA can be used both while the PCIe controller is in EP and RC mode, > > right? > > For now I can not give this certainty. My setup only have the eDMA on the EP > side and has the eDMA mapped on PCI BAR, which allows me to configure it through > PCI. Ok, I looked at the databook, and it does mention support for both modes. Perhaps the cover letter could mention more clearly that this is just for eDMA on the EP side. (Since the IP seems to support eDMA for both RC and EP side.) (snip) > > Do you have any benchmarks to share? (Perhaps with different buffer sizes). > > I'm currently working on the dw-edma-test driver (similar to dmatest driver) > that supports DEV_TO_MEM and MEM_TO_DEV directions instead of MEM_TO_MEM > (dmatest), and adapted to WRITE and READ channels with multiple threads. After > finishing this driver I'll able to do some benchmarks and comparisons. :) > > Given your enthusiasm, when I'm finished we could do some brainstorm and define > some test cases / benchmarks, what do you think? I might have the enthusiasm, but I don't have the expertise to come up with some relevant use-cases :p > > > Not sure what we will see, lower CPU utilization but more IRQs? > > Probably yes. Let me explain you > > With this implementation, you can transfer several chunks (a chunk is linked > list of burst elements). > > Each burst basically defines the source, destination addresses and size to > transfer (from 1 Byte to 4 GB). > > This linked list will be written on EP's RAM (mapped on a PCI BAR) which the HW > block will consume it. The linked list size is dependent of this EP's RAM size, > i.e. 8 KB will allow 340 elements (bursts). > > On my currently setup I'm testing the driver implementation like this: > - 4 chunks > - each chunk is composed by 340 elements (bursts) > - each burst have a transfer size of 100 bytes (can be up to 4GB and the size > can be also different for each burst) > > Chunk #0 > |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) > Chunk #1 > |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) > Chunk #2 > |--> Burst #0 --> ... --> Burst #339 -> IRQ (Reload new linked list) > Chunk #3 > |--> Burst #0 --> ... --> Burst #34 -> IRQ (End transfer) > > In total I'm writing ((4 * 340) + 35) * 100 = 139500 Bytes <=> 136 KB This is a nice explaination, and it probably belongs in the cover letter, and/or commit message, and/or somewhere in Documentation/, and/or as a comment in the source code. (snip) > I'm dividing this patch series like this: > > - eDMA core + eDMA core v0 driver (implementing the interface with DMAengine > controller APIs and the interface with the eDMA HW block) > > - eDMA PCIe glue-logic driver (attaches to EP and provides memory access to > eDMA core driver), for now is attach to Synopsys Vendor and Device ID and has a > platform data specific to our EP. > > - eDMA Test driver (is a sample driver which aims to test the eDMA, but also > provides a example how an customer driver would use the eDMA driver, through the > DMAengine client APIs) (under developing, I hope to make it available before > xmas :)) > > Sounds logic? This definitely belongs in the cover letter :) > > > > > I have a board that has a CC_DMA_ENABLE set, so if you give me some > > detailed instructions how to test, I might be able to provide you with a > > Tested-by. > > Ok, let me just finish the eDMA test and append it to the patch series. > I'm happy to help you with this. :) The hardware I currently have access to (Qualcomm hardware), hasn't implemented the software support for running the PCIe controller in endpoint mode yet. This means that I won't be able to help you with testing this series. Sorry for that. (I wanted to help out here, promise :p). Hopefully there is someone else who has this IP configured with CC_DMA_ENABLE + has EP support implemented, who can help you out :) Kind regards, Niklas
Add Synopsys eDMA IP driver (version 0) to Linux kernel. This IP is generally distributed with Synopsys PCIe EndPoint IP (depends of the use and licensing agreement), which supports: - legacy and unroll modes - 16 independent and concurrent channels (8 write + 8 read) - supports linked list (scatter-gather) transfer - each linked list descriptor can transfer from 1 byte to 4 Gbytes - PCIe EndPoint glue-logic Gustavo Pimentel (6): dmaengine: Add Synopsys eDMA IP core driver dmaengine: Add Synopsys eDMA IP version 0 support dmaengine: Add Synopsys eDMA IP version 0 debugfs support PCI: Add Synopsys endpoint EDDA Device id dmaengine: Add Synopsys eDMA IP PCIe glue-logic MAINTAINERS: Add Synopsys eDMA IP driver maintainer MAINTAINERS | 7 + drivers/dma/Kconfig | 2 + drivers/dma/Makefile | 1 + drivers/dma/dw-edma/Kconfig | 18 + drivers/dma/dw-edma/Makefile | 7 + drivers/dma/dw-edma/dw-edma-core.c | 845 +++++++++++++++++++++++++++++++ drivers/dma/dw-edma/dw-edma-core.h | 147 ++++++ drivers/dma/dw-edma/dw-edma-pcie.c | 258 ++++++++++ drivers/dma/dw-edma/dw-edma-v0-core.c | 361 +++++++++++++ drivers/dma/dw-edma/dw-edma-v0-core.h | 26 + drivers/dma/dw-edma/dw-edma-v0-debugfs.c | 357 +++++++++++++ drivers/dma/dw-edma/dw-edma-v0-debugfs.h | 24 + drivers/dma/dw-edma/dw-edma-v0-regs.h | 145 ++++++ drivers/misc/pci_endpoint_test.c | 2 +- include/linux/dma/edma.h | 43 ++ include/linux/pci_ids.h | 1 + 16 files changed, 2243 insertions(+), 1 deletion(-) create mode 100644 drivers/dma/dw-edma/Kconfig create mode 100644 drivers/dma/dw-edma/Makefile create mode 100644 drivers/dma/dw-edma/dw-edma-core.c create mode 100644 drivers/dma/dw-edma/dw-edma-core.h create mode 100644 drivers/dma/dw-edma/dw-edma-pcie.c create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.c create mode 100644 drivers/dma/dw-edma/dw-edma-v0-core.h create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.c create mode 100644 drivers/dma/dw-edma/dw-edma-v0-debugfs.h create mode 100644 drivers/dma/dw-edma/dw-edma-v0-regs.h create mode 100644 include/linux/dma/edma.h Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com> Cc: Vinod Koul <vkoul@kernel.org> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Cc: Russell King <rmk+kernel@armlinux.org.uk> Cc: Eugeniy Paltsev <paltsev@synopsys.com> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Kishon Vijay Abraham I <kishon@ti.com> Cc: Niklas Cassel <niklas.cassel@linaro.org> Cc: Joao Pinto <jpinto@synopsys.com> Cc: Jose Abreu <jose.abreu@synopsys.com> Cc: Luis Oliveira <lolivei@synopsys.com> Cc: Vitor Soares <vitor.soares@synopsys.com> Cc: Nelson Costa <nelson.costa@synopsys.com> Cc: Pedro Sousa <pedrom.sousa@synopsys.com>