| Message ID | 20220715192219.1489403-1-Frank.Li@nxp.com |
|---|---|
| State | New |
| Headers | show |
On Fri, Jul 15, 2022 at 02:22:15PM -0500, Frank Li wrote: > > ┌───────┐ ┌──────────┐ > │ │ │ │ > ┌─────────────┐ │ │ │ PCI Host │ > │ MSI │◄┐ │ │ │ │ > │ Controller │ │ │ │ │ │ > └─────────────┘ └─┼───────┼──────────┼─Bar0 │ > │ PCI │ │ Bar1 │ > │ Func │ │ Bar2 │ > │ │ │ Bar3 │ > │ │ │ Bar4 │ > │ ├─────────►│ │ > └───────┘ └──────────┘ Nice diagram and explanation. I suggest rewrapping to fit in 75 columns and including in one of the patches, probably the pci-epf-vntb.c one. Then it will be more likely to make it to the git history where it will be useful. > Many PCI controllers provided Endpoint functions. > Generally PCI endpoint is hardware, which is not running a rich OS, like linux. > > But Linux also supports endpoint functions. PCI Host write bar<n> space like > write to memory. The EP side can't know memory changed by the Host driver. > > PCI Spec has not defined a standard method to do that. Only define MSI(x) to let > EP notified RC status change. > > The basic idea is to trigger an irq when PCI RC writes to a memory address. That's > what MSI controller provided. EP drivers just need to request a platform MSI interrupt, > struct msi_msg *msg will pass down a memory address and data. EP driver will > map such memory address to one of PCI bar<n>. Host just writes such an address to > trigger EP side irq. > > If system have gic-its, only need update PCI EP side driver. But i.MX have not chip > support gic-ites yet. So we have to use MU to simulate a MSI controller. Although > only 4 MSI irqs are simulated, it matched vntd network requirmenent. > > After enable MSI, ping delay reduce < 1ms from ~8ms > > irqchip: imx mu worked as msi controller: > let imx mu worked as MSI controllers. Although IP is not design as MSI controller, > we still can use it if limiated irq number to 4. > > pcie: endpoint: pci-epf-vntb: add endpoint msi support > Based on ntb-next branch. https://github.com/jonmason/ntb/commits/ntb-next > Using MSI as door bell registers > > i.MX EP function driver is upstreaming by Richard Zhu. > Some dts change missed at this patches. below is reference dts change s/bar/BAR/ (several) s/irq/IRQ/ (several) s/irqs/IRQs/ s/msi/MSI/ s/gic-ites/? (capitalize if it's an acronym) s/requirmenent/requirement/ s/limiated/limited/ You use both "gic-its" and "gic-ites". I assume they should be the same. Not sure what "vntd" refers to. Capitalize if it's an acronym. > --- a/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi > +++ b/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi > @@ -160,5 +160,6 @@ pcieb_ep: pcie_ep@5f010000 { > num-ib-windows = <6>; > num-ob-windows = <6>; > status = "disabled"; > + msi-parent = <&lsio_mu12>; > }; > > --- a/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi > +++ b/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi > @@ -172,6 +172,19 @@ lsio_mu6: mailbox@5d210000 { > status = "disabled"; > }; > > + lsio_mu12: mailbox@5d270000 { > + compatible = "fsl,imx6sx-mu-msi"; > + msi-controller; > + interrupt-controller; > + reg = <0x5d270000 0x10000>, /* A side */ > + <0x5d300000 0x10000>; /* B side */ > + reg-names = "a", "b"; > + interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>; > + power-domains = <&pd IMX_SC_R_MU_12A>, > + <&pd IMX_SC_R_MU_12B>; > + power-domain-names = "a", "b"; > + }; > + > > Change Log > - from V1 to V2 > Fixed fsl,mu-msi.yaml's problem > Fixed irq-imx-mu-msi.c problem according Marc Zyngier's feeback > Added a new patch to allow pass down .pm by IRQCHIP_PLATFORM_DRIVER_END > > -- > 2.35.1 >
--- a/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8-ss-hsio.dtsi @@ -160,5 +160,6 @@ pcieb_ep: pcie_ep@5f010000 { num-ib-windows = <6>; num-ob-windows = <6>; status = "disabled"; + msi-parent = <&lsio_mu12>; }; --- a/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi +++ b/arch/arm64/boot/dts/freescale/imx8-ss-lsio.dtsi @@ -172,6 +172,19 @@ lsio_mu6: mailbox@5d210000 { status = "disabled"; }; + lsio_mu12: mailbox@5d270000 { + compatible = "fsl,imx6sx-mu-msi"; + msi-controller; + interrupt-controller; + reg = <0x5d270000 0x10000>, /* A side */ + <0x5d300000 0x10000>; /* B side */ + reg-names = "a", "b"; + interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_HIGH>; + power-domains = <&pd IMX_SC_R_MU_12A>, + <&pd IMX_SC_R_MU_12B>; + power-domain-names = "a", "b"; + }; + Change Log - from V1 to V2 Fixed fsl,mu-msi.yaml's problem
┌───────┐ ┌──────────┐ │ │ │ │ ┌─────────────┐ │ │ │ PCI Host │ │ MSI │◄┐ │ │ │ │ │ Controller │ │ │ │ │ │ └─────────────┘ └─┼───────┼──────────┼─Bar0 │ │ PCI │ │ Bar1 │ │ Func │ │ Bar2 │ │ │ │ Bar3 │ │ │ │ Bar4 │ │ ├─────────►│ │ └───────┘ └──────────┘ Many PCI controllers provided Endpoint functions. Generally PCI endpoint is hardware, which is not running a rich OS, like linux. But Linux also supports endpoint functions. PCI Host write bar<n> space like write to memory. The EP side can't know memory changed by the Host driver. PCI Spec has not defined a standard method to do that. Only define MSI(x) to let EP notified RC status change. The basic idea is to trigger an irq when PCI RC writes to a memory address. That's what MSI controller provided. EP drivers just need to request a platform MSI interrupt, struct msi_msg *msg will pass down a memory address and data. EP driver will map such memory address to one of PCI bar<n>. Host just writes such an address to trigger EP side irq. If system have gic-its, only need update PCI EP side driver. But i.MX have not chip support gic-ites yet. So we have to use MU to simulate a MSI controller. Although only 4 MSI irqs are simulated, it matched vntd network requirmenent. After enable MSI, ping delay reduce < 1ms from ~8ms irqchip: imx mu worked as msi controller: let imx mu worked as MSI controllers. Although IP is not design as MSI controller, we still can use it if limiated irq number to 4. pcie: endpoint: pci-epf-vntb: add endpoint msi support Based on ntb-next branch. https://github.com/jonmason/ntb/commits/ntb-next Using MSI as door bell registers i.MX EP function driver is upstreaming by Richard Zhu. Some dts change missed at this patches. below is reference dts change Fixed irq-imx-mu-msi.c problem according Marc Zyngier's feeback Added a new patch to allow pass down .pm by IRQCHIP_PLATFORM_DRIVER_END