| Message ID | 1437670365-20704-3-git-send-email-mark.rutland@arm.com |
|---|---|
| State | Superseded, archived |
| Headers | show |
looks good. possible to describe the chip we have. > On Jul 23, 2015, at 9:52 AM, Mark Rutland <mark.rutland@arm.com> wrote: > > Currently msi-parent is used by a few bindings to describe the > relationship between a PCI root complex and a single MSI controller, but > this property does not have a generic binding document. > > Additionally, msi-parent is insufficient to describe more complex > relationships between MSI controllers and devices under a root complex, > where devices may be able to target multiple MSI controllers, or where > MSI controllers use (non-probeable) sideband information to distinguish > devices. > > This patch adds a generic binding for mapping PCI devices to MSI > controllers. This document covers msi-parent, and a new msi-map property > (specific to PCI*) which may be used to map devices (identified by their > Requester ID) to sideband data for each MSI controller that they may > target. > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > --- > Documentation/devicetree/bindings/pci/pci-msi.txt | 220 ++++++++++++++++++++++ > 1 file changed, 220 insertions(+) > create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt b/Documentation/devicetree/bindings/pci/pci-msi.txt > new file mode 100644 > index 0000000..9b3cc81 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > @@ -0,0 +1,220 @@ > +This document describes the generic device tree binding for describing the > +relationship between PCI devices and MSI controllers. > + > +Each PCI device under a root complex is uniquely identified by its Requester ID > +(AKA RID). A Requester ID is a triplet of a Bus number, Device number, and > +Function number. > + > +For the purpose of this document, when treated as a numeric value, a RID is > +formatted such that: > + > +* Bits [15:8] are the Bus number. > +* Bits [7:3] are the Device number. > +* Bits [2:0] are the Function number. > +* Any other bits required for padding must be zero. > + > +MSIs may be distinguished in part through the use of sideband data accompanying > +writes. In the case of PCI devices, this sideband data may be derived from the > +Requester ID. A mechanism is required to associate a device with both the MSI > +controllers it can address, and the sideband data that will be associated with > +its writes to those controllers. > + > +For generic MSI bindings, see > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > + > + > +PCI root complex > +================ > + > +Optional properties > +------------------- > + > +- msi-map: Maps a Requester ID to an MSI controller and associated > + msi-specifier data. The property is an arbitrary number of tuples of > + (rid-base,msi-controller,msi-base,length), where: > + > + * rid-base is a single cell describing the first RID matched by the entry. > + > + * msi-controller is a single phandle to an MSI controller > + > + * msi-base is an msi-specifier describing the msi-specifier produced for the > + first RID matched by the entry. > + > + * length is a single cell describing how many consecutive RIDs are matched > + following the rid-base. > + > + Any RID r in the interval [rid-base, rid-base + length) is associated with > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > + > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > + to an msi-specifier per the msi-map property. > + > +- msi-parent: Describes the MSI parent of the root complex itself. Where > + the root complex and MSI controller do not pass sideband data with MSI > + writes, this property may be used to describe the MSI controller(s) > + used by PCI devices under the root complex, if defined as such in the > + binding for the root complex. > + > + > +Example (1) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, identity-mapped. > + */ > + msi-map = <0x0 &msi_a 0x0 0x10000>, > + }; > +}; > + > + > +Example (2) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, masked to only the device and function bits. > + */ > + msi-map = <0x0 &msi_a 0x0 0x100>, > + msi-map-mask = <0xff> > + }; > +}; > + > + > +Example (3) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, but the high bit of the bus number is > + * ignored. > + */ > + msi-map = <0x0000 &msi 0x0000 0x8000>, > + <0x8000 &msi 0x0000 0x8000>; > + }; > +}; > + > + > +Example (4) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, but the high bit of the bus number is > + * negated. > + */ > + msi-map = <0x0000 &msi 0x8000 0x8000>, > + <0x8000 &msi 0x0000 0x8000>; > + }; > +}; > + > + > +Example (5) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi_a: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + msi_b: msi-controller@b { > + reg = <0xb 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + msi_c: msi-controller@c { > + reg = <0xc 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@c { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to MSI controller a is the > + * RID, but the high bit of the bus number is negated. > + * The sideband data provided to MSI controller b is the > + * RID, identity-mapped. > + * MSI controller c is not addressable. > + */ > + msi-map = <0x0000 &msi_a 0x8000 0x08000>, > + <0x8000 &msi_a 0x0000 0x08000>, > + <0x0000 &msi_b 0x0000 0x10000>; > + }; they can be identical right? like <0x8000 &msi_a 0x0000 0x08000>, <0x8000 &msi_b 0x0000 0x08000>; Thanks, Tirumalesh. > +}; > -- > 1.9.1 > -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 23/07/15 17:52, Mark Rutland wrote: > Currently msi-parent is used by a few bindings to describe the > relationship between a PCI root complex and a single MSI controller, but > this property does not have a generic binding document. > > Additionally, msi-parent is insufficient to describe more complex > relationships between MSI controllers and devices under a root complex, > where devices may be able to target multiple MSI controllers, or where > MSI controllers use (non-probeable) sideband information to distinguish > devices. > > This patch adds a generic binding for mapping PCI devices to MSI > controllers. This document covers msi-parent, and a new msi-map property > (specific to PCI*) which may be used to map devices (identified by their > Requester ID) to sideband data for each MSI controller that they may > target. > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > --- > Documentation/devicetree/bindings/pci/pci-msi.txt | 220 ++++++++++++++++++++++ > 1 file changed, 220 insertions(+) > create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt b/Documentation/devicetree/bindings/pci/pci-msi.txt > new file mode 100644 > index 0000000..9b3cc81 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > @@ -0,0 +1,220 @@ > +This document describes the generic device tree binding for describing the > +relationship between PCI devices and MSI controllers. > + > +Each PCI device under a root complex is uniquely identified by its Requester ID > +(AKA RID). A Requester ID is a triplet of a Bus number, Device number, and > +Function number. > + > +For the purpose of this document, when treated as a numeric value, a RID is > +formatted such that: > + > +* Bits [15:8] are the Bus number. > +* Bits [7:3] are the Device number. > +* Bits [2:0] are the Function number. > +* Any other bits required for padding must be zero. > + > +MSIs may be distinguished in part through the use of sideband data accompanying > +writes. In the case of PCI devices, this sideband data may be derived from the > +Requester ID. A mechanism is required to associate a device with both the MSI > +controllers it can address, and the sideband data that will be associated with > +its writes to those controllers. > + > +For generic MSI bindings, see > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > + > + > +PCI root complex > +================ > + > +Optional properties > +------------------- > + > +- msi-map: Maps a Requester ID to an MSI controller and associated > + msi-specifier data. The property is an arbitrary number of tuples of > + (rid-base,msi-controller,msi-base,length), where: > + > + * rid-base is a single cell describing the first RID matched by the entry. > + > + * msi-controller is a single phandle to an MSI controller > + > + * msi-base is an msi-specifier describing the msi-specifier produced for the > + first RID matched by the entry. > + > + * length is a single cell describing how many consecutive RIDs are matched > + following the rid-base. > + > + Any RID r in the interval [rid-base, rid-base + length) is associated with > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > + > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > + to an msi-specifier per the msi-map property. > + > +- msi-parent: Describes the MSI parent of the root complex itself. Where > + the root complex and MSI controller do not pass sideband data with MSI > + writes, this property may be used to describe the MSI controller(s) > + used by PCI devices under the root complex, if defined as such in the > + binding for the root complex. Right, this is where I'd expect some details about #msi-cells. Is it meant to be ignored? The lack of symmetry between the PCI and non-PCI use cases feels a bit inelegant (not to mention that it precludes having an unified parser for both cases). This otherwise looks good to me. Thanks, M.
> > +Example (5) > > +=========== > > + > > +/ { > > + #address-cells = <1>; > > + #size-cells = <1>; > > + > > + msi_a: msi-controller@a { > > + reg = <0xa 0x1>; > > + compatible = "vendor,some-controller"; > > + msi-controller; > > + #msi-cells = <1>; > > + }; > > + > > + msi_b: msi-controller@b { > > + reg = <0xb 0x1>; > > + compatible = "vendor,some-controller"; > > + msi-controller; > > + #msi-cells = <1>; > > + }; > > + > > + msi_c: msi-controller@c { > > + reg = <0xc 0x1>; > > + compatible = "vendor,some-controller"; > > + msi-controller; > > + #msi-cells = <1>; > > + }; > > + > > + pci: pci@c { > > + reg = <0xf 0x1>; > > + compatible = "vendor,pcie-root-complex"; > > + device_type = "pci"; > > + > > + /* > > + * The sideband data provided to MSI controller a is the > > + * RID, but the high bit of the bus number is negated. > > + * The sideband data provided to MSI controller b is the > > + * RID, identity-mapped. > > + * MSI controller c is not addressable. > > + */ > > + msi-map = <0x0000 &msi_a 0x8000 0x08000>, > > + <0x8000 &msi_a 0x0000 0x08000>, > > + <0x0000 &msi_b 0x0000 0x10000>; > > + }; > > they can be identical right? like > <0x8000 &msi_a 0x0000 0x08000>, > <0x8000 &msi_b 0x0000 0x08000>; In general that would be valid, yes. In this case two entries are required for MSI controller a because the high bit passed to it is negated. This does not occur for MSI controller b, so it only requires a single entry to describe the transformation. Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Mark Thanks for the patch. Please find my comment inline. Regards Varun > -----Original Message----- > From: iommu-bounces@lists.linux-foundation.org [mailto:iommu- > bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland > Sent: Thursday, July 23, 2015 10:23 PM > To: devicetree@vger.kernel.org > Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de; > marc.zyngier@arm.com; will.deacon@arm.com; linux- > kernel@vger.kernel.org; ddaney@caviumnetworks.com; iommu@lists.linux- > foundation.org; tirumalesh.chalamarla@caviumnetworks.com; > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com; > treding@nvidia.com; linux-arm-kernel@lists.infradead.org; > majun258@huawei.com > Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > Currently msi-parent is used by a few bindings to describe the relationship > between a PCI root complex and a single MSI controller, but this property > does not have a generic binding document. > > Additionally, msi-parent is insufficient to describe more complex > relationships between MSI controllers and devices under a root complex, > where devices may be able to target multiple MSI controllers, or where MSI > controllers use (non-probeable) sideband information to distinguish devices. > > This patch adds a generic binding for mapping PCI devices to MSI controllers. > This document covers msi-parent, and a new msi-map property (specific to > PCI*) which may be used to map devices (identified by their Requester ID) to > sideband data for each MSI controller that they may target. > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > --- > Documentation/devicetree/bindings/pci/pci-msi.txt | 220 > ++++++++++++++++++++++ > 1 file changed, 220 insertions(+) > create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt > b/Documentation/devicetree/bindings/pci/pci-msi.txt > new file mode 100644 > index 0000000..9b3cc81 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > @@ -0,0 +1,220 @@ > +This document describes the generic device tree binding for describing > +the relationship between PCI devices and MSI controllers. > + > +Each PCI device under a root complex is uniquely identified by its > +Requester ID (AKA RID). A Requester ID is a triplet of a Bus number, > +Device number, and Function number. > + > +For the purpose of this document, when treated as a numeric value, a > +RID is formatted such that: > + > +* Bits [15:8] are the Bus number. > +* Bits [7:3] are the Device number. > +* Bits [2:0] are the Function number. > +* Any other bits required for padding must be zero. > + > +MSIs may be distinguished in part through the use of sideband data > +accompanying writes. In the case of PCI devices, this sideband data may > +be derived from the Requester ID. A mechanism is required to associate > +a device with both the MSI controllers it can address, and the sideband > +data that will be associated with its writes to those controllers. > + > +For generic MSI bindings, see > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > + > + > +PCI root complex > +================ > + > +Optional properties > +------------------- > + > +- msi-map: Maps a Requester ID to an MSI controller and associated > + msi-specifier data. The property is an arbitrary number of tuples of > + (rid-base,msi-controller,msi-base,length), where: [varun] How would we account for hot plug PCI devices and SR-IOV use cases, with the rid base and length? How do we take in to account for a PCIe bridge, while setting up the requestor ID base and length? > + > + * rid-base is a single cell describing the first RID matched by the entry. > + > + * msi-controller is a single phandle to an MSI controller > + > + * msi-base is an msi-specifier describing the msi-specifier produced for the > + first RID matched by the entry. > + > + * length is a single cell describing how many consecutive RIDs are matched > + following the rid-base. > + > + Any RID r in the interval [rid-base, rid-base + length) is associated > + with the listed msi-controller, with the msi-specifier (r - rid-base + msi- > base). > + > +- msi-map-mask: A mask to be applied to each Requester ID prior to > +being mapped > + to an msi-specifier per the msi-map property. > + [varun] Can you please elaborate on a use case, where this would help. > +- msi-parent: Describes the MSI parent of the root complex itself. > +Where > + the root complex and MSI controller do not pass sideband data with > +MSI > + writes, this property may be used to describe the MSI controller(s) > + used by PCI devices under the root complex, if defined as such in the > + binding for the root complex. > + > + > +Example (1) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, identity-mapped. > + */ > + msi-map = <0x0 &msi_a 0x0 0x10000>, > + }; > +}; > + > + > +Example (2) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, masked to only the device and function bits. > + */ > + msi-map = <0x0 &msi_a 0x0 0x100>, > + msi-map-mask = <0xff> > + }; > +}; > + > + > +Example (3) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, but the high bit of the bus number is > + * ignored. > + */ > + msi-map = <0x0000 &msi 0x0000 0x8000>, > + <0x8000 &msi 0x0000 0x8000>; > + }; > +}; > + > + > +Example (4) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@f { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to the MSI controller is > + * the RID, but the high bit of the bus number is > + * negated. > + */ > + msi-map = <0x0000 &msi 0x8000 0x8000>, > + <0x8000 &msi 0x0000 0x8000>; > + }; > +}; > + > + > +Example (5) > +=========== > + > +/ { > + #address-cells = <1>; > + #size-cells = <1>; > + > + msi_a: msi-controller@a { > + reg = <0xa 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + msi_b: msi-controller@b { > + reg = <0xb 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + msi_c: msi-controller@c { > + reg = <0xc 0x1>; > + compatible = "vendor,some-controller"; > + msi-controller; > + #msi-cells = <1>; > + }; > + > + pci: pci@c { > + reg = <0xf 0x1>; > + compatible = "vendor,pcie-root-complex"; > + device_type = "pci"; > + > + /* > + * The sideband data provided to MSI controller a is the > + * RID, but the high bit of the bus number is negated. > + * The sideband data provided to MSI controller b is the > + * RID, identity-mapped. > + * MSI controller c is not addressable. > + */ > + msi-map = <0x0000 &msi_a 0x8000 0x08000>, > + <0x8000 &msi_a 0x0000 0x08000>, > + <0x0000 &msi_b 0x0000 0x10000>; > + }; > +}; > -- > 1.9.1 > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> From: Mark Rutland <mark.rutland@arm.com> > Date: Thu, Jul 23, 2015 at 11:52 AM [cut] > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt > b/Documentation/devicetree/bindings/pci/pci-msi.txt > new file mode 100644 > index 0000000..9b3cc81 > --- /dev/null > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > @@ -0,0 +1,220 @@ > +This document describes the generic device tree binding for describing the > +relationship between PCI devices and MSI controllers. > + > +Each PCI device under a root complex is uniquely identified by its Requester ID > +(AKA RID). A Requester ID is a triplet of a Bus number, Device number, and > +Function number. > + > +For the purpose of this document, when treated as a numeric value, a RID is > +formatted such that: > + > +* Bits [15:8] are the Bus number. > +* Bits [7:3] are the Device number. > +* Bits [2:0] are the Function number. > +* Any other bits required for padding must be zero. > + > +MSIs may be distinguished in part through the use of sideband data accompanying > +writes. In the case of PCI devices, this sideband data may be derived from the > +Requester ID. A mechanism is required to associate a device with both the MSI > +controllers it can address, and the sideband data that will be associated with > +its writes to those controllers. > + > +For generic MSI bindings, see > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > + > + > +PCI root complex > +================ > + > +Optional properties > +------------------- > + > +- msi-map: Maps a Requester ID to an MSI controller and associated > + msi-specifier data. The property is an arbitrary number of tuples of > + (rid-base,msi-controller,msi-base,length), where: > + > + * rid-base is a single cell describing the first RID matched by the entry. > + > + * msi-controller is a single phandle to an MSI controller > + > + * msi-base is an msi-specifier describing the msi-specifier produced for the > + first RID matched by the entry. > + > + * length is a single cell describing how many consecutive RIDs are matched > + following the rid-base. > + > + Any RID r in the interval [rid-base, rid-base + length) is associated with > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > + > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > + to an msi-specifier per the msi-map property. Can we extend the msi-map-mask definition to say: "A mask value of 0x0 is valid and indicates that no RIDs are _currently_ mapped to any msi-specifier." We have an SoC with a programmable hardware table in the PCI controller that maps requester ID to stream ID, so the overall msi-map (and iommu-map) definition fit into that scheme. But, we would like to be able make the RID->stream-ID mapping decision _lazily_, in Linux, based on actual usage of PCI devices. pcie@3600000 { compatible = "fsl,ls2085a-pcie", "snps,dw-pcie"; device_type = "pci"; ... msi-map = <0x0 &msi_a 0x7 4>, msi-map-mask = <0x0> }; That specifies the there are 4 stream IDs starting at stream ID 0x7, but the requester ID's are not mapped (because the mask is 0x0). This tells the PCI controller driver that there are 4 msi-specifiers (e.g. stream IDs) available and what they are. (same definition would apply to the iommu-map-mask) Thanks, Stuart
On Wed, Aug 05, 2015 at 05:39:33PM +0100, Varun Sethi wrote: > Hi Mark > Thanks for the patch. Please find my comment inline. > > Regards > Varun > > > -----Original Message----- > > From: iommu-bounces@lists.linux-foundation.org [mailto:iommu- > > bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland > > Sent: Thursday, July 23, 2015 10:23 PM > > To: devicetree@vger.kernel.org > > Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de; > > marc.zyngier@arm.com; will.deacon@arm.com; linux- > > kernel@vger.kernel.org; ddaney@caviumnetworks.com; iommu@lists.linux- > > foundation.org; tirumalesh.chalamarla@caviumnetworks.com; > > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com; > > treding@nvidia.com; linux-arm-kernel@lists.infradead.org; > > majun258@huawei.com > > Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > > > Currently msi-parent is used by a few bindings to describe the relationship > > between a PCI root complex and a single MSI controller, but this property > > does not have a generic binding document. > > > > Additionally, msi-parent is insufficient to describe more complex > > relationships between MSI controllers and devices under a root complex, > > where devices may be able to target multiple MSI controllers, or where MSI > > controllers use (non-probeable) sideband information to distinguish devices. > > > > This patch adds a generic binding for mapping PCI devices to MSI controllers. > > This document covers msi-parent, and a new msi-map property (specific to > > PCI*) which may be used to map devices (identified by their Requester ID) to > > sideband data for each MSI controller that they may target. > > > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > > --- > > Documentation/devicetree/bindings/pci/pci-msi.txt | 220 > > ++++++++++++++++++++++ > > 1 file changed, 220 insertions(+) > > create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt > > > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt > > b/Documentation/devicetree/bindings/pci/pci-msi.txt > > new file mode 100644 > > index 0000000..9b3cc81 > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > > @@ -0,0 +1,220 @@ > > +This document describes the generic device tree binding for describing > > +the relationship between PCI devices and MSI controllers. > > + > > +Each PCI device under a root complex is uniquely identified by its > > +Requester ID (AKA RID). A Requester ID is a triplet of a Bus number, > > +Device number, and Function number. > > + > > +For the purpose of this document, when treated as a numeric value, a > > +RID is formatted such that: > > + > > +* Bits [15:8] are the Bus number. > > +* Bits [7:3] are the Device number. > > +* Bits [2:0] are the Function number. > > +* Any other bits required for padding must be zero. > > + > > +MSIs may be distinguished in part through the use of sideband data > > +accompanying writes. In the case of PCI devices, this sideband data may > > +be derived from the Requester ID. A mechanism is required to associate > > +a device with both the MSI controllers it can address, and the sideband > > +data that will be associated with its writes to those controllers. > > + > > +For generic MSI bindings, see > > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > > + > > + > > +PCI root complex > > +================ > > + > > +Optional properties > > +------------------- > > + > > +- msi-map: Maps a Requester ID to an MSI controller and associated > > + msi-specifier data. The property is an arbitrary number of tuples of > > + (rid-base,msi-controller,msi-base,length), where: > [varun] How would we account for hot plug PCI devices and SR-IOV use cases, with the rid base and length? For hotplug, you simply need the mapping from RID to msi-specifier to be defined in advance in the DT, for the set of RIDs that could possibly occur. For SR-IOV, are you asking about ARI? I should update the description of the RID to describe that for ARI it has the format: * Bits [15:8] are the Bus number * Bits [7:0] are the Identifier Other than that, the handling would be identical to the non-ARI case. What else am I missing? > How do we take in to account for a PCIe bridge, while setting up the requestor ID base and length? I'm not sure I follow the question. I don't see why this is any different to any other requester ID. What do you see as being the problem for this case? > > + > > + * rid-base is a single cell describing the first RID matched by the entry. > > + > > + * msi-controller is a single phandle to an MSI controller > > + > > + * msi-base is an msi-specifier describing the msi-specifier produced for the > > + first RID matched by the entry. > > + > > + * length is a single cell describing how many consecutive RIDs are matched > > + following the rid-base. > > + > > + Any RID r in the interval [rid-base, rid-base + length) is associated > > + with the listed msi-controller, with the msi-specifier (r - rid-base + msi- > > base). > > + > > +- msi-map-mask: A mask to be applied to each Requester ID prior to > > +being mapped > > + to an msi-specifier per the msi-map property. > > + > [varun] Can you please elaborate on a use case, where this would help. It may be the case that at the MSI controller's ID space is smaller than the RID space, and so only a subset of RID bits matter. For example, it might be the case that only the Bus ID matters. Using the msi-map-mask allows for a much smaller msi-map for this case, e.g. msi-map-mask = <0xff00>; msi-map = <0x0000 &msi 0x00 1>, <0x0100 &msi 0x01 1>, <0x0200 &msi 0x01 1>, <0x0300 &msi 0x01 1>, ... <0xff00 &msi 0xff 1>; Rather than: msi-map = <0x0000 &msi 0x00 1>, <0x0001 &msi 0x00 1>, <0x0002 &msi 0x00 1>, <0x0003 &msi 0x00 1>, ... <0x00ff &msi 0x00 1>, <0x0100 &msi 0x00 1>, <0x0101 &msi 0x00 1>, <0x0102 &msi 0x00 1>, <0x0103 &msi 0x00 1>, ... <0x01ff &msi 0x00 1>, ... <0xff00 &msi 0xff 1>, <0xff01 &msi 0xff 1>, <0xff02 &msi 0xff 1>, <0xff03 &msi 0xff 1>, .... <0xffff &msi 0xff 1>; Or for the case that everything maps to a single ID: msi-map-mask = <0x0000>; msi-map = <0x0000 &msi 0x0000 1>; Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[...] > > +PCI root complex > > +================ > > + > > +Optional properties > > +------------------- > > + > > +- msi-map: Maps a Requester ID to an MSI controller and associated > > + msi-specifier data. The property is an arbitrary number of tuples of > > + (rid-base,msi-controller,msi-base,length), where: > > + > > + * rid-base is a single cell describing the first RID matched by the entry. > > + > > + * msi-controller is a single phandle to an MSI controller > > + > > + * msi-base is an msi-specifier describing the msi-specifier produced for the > > + first RID matched by the entry. > > + > > + * length is a single cell describing how many consecutive RIDs are matched > > + following the rid-base. > > + > > + Any RID r in the interval [rid-base, rid-base + length) is associated with > > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > > + > > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > > + to an msi-specifier per the msi-map property. > > Can we extend the msi-map-mask definition to say: "A mask value of 0x0 is valid > and indicates that no RIDs are _currently_ mapped to any msi-specifier." That would break a valid case of the mask being all zeroes. Consider the case that all RIDs get mapped to a single msi-specifier; the obvious way to write that is: msi-map-mask = <0x0000>; msi-map = <0x0000 &msi (msi-specifier) 1>; In this case all RIDS are always mapped to the single msi-specifier. > We have an SoC with a programmable hardware table in the PCI controller that maps > requester ID to stream ID, so the overall msi-map (and iommu-map) definition fit > into that scheme. But, we would like to be able make the RID->stream-ID mapping > decision _lazily_, in Linux, based on actual usage of PCI devices. Dynamically programming the mapping is at odds to this binding. I don't see how that can fit. Why can the RID->SID mapping not be statically configured prior to entering the OS? Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> -----Original Message----- > From: Mark Rutland [mailto:mark.rutland@arm.com] > Sent: Thursday, August 06, 2015 1:15 PM > To: Yoder Stuart-B08248; Marc Zyngier; Will Deacon > Cc: devicetree@vger.kernel.org; Lorenzo Pieralisi; arnd@arndb.de; linux-kernel@vger.kernel.org; > ddaney@caviumnetworks.com; iommu@lists.linux-foundation.org; tirumalesh.chalamarla@caviumnetworks.com; > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com; treding@nvidia.com; linux-arm- > kernel@lists.infradead.org; majun258@huawei.com > Subject: Re: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > [...] > > > > +PCI root complex > > > +================ > > > + > > > +Optional properties > > > +------------------- > > > + > > > +- msi-map: Maps a Requester ID to an MSI controller and associated > > > + msi-specifier data. The property is an arbitrary number of tuples of > > > + (rid-base,msi-controller,msi-base,length), where: > > > + > > > + * rid-base is a single cell describing the first RID matched by the entry. > > > + > > > + * msi-controller is a single phandle to an MSI controller > > > + > > > + * msi-base is an msi-specifier describing the msi-specifier produced for the > > > + first RID matched by the entry. > > > + > > > + * length is a single cell describing how many consecutive RIDs are matched > > > + following the rid-base. > > > + > > > + Any RID r in the interval [rid-base, rid-base + length) is associated with > > > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > > > + > > > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > > > + to an msi-specifier per the msi-map property. > > > > Can we extend the msi-map-mask definition to say: "A mask value of 0x0 is valid > > and indicates that no RIDs are _currently_ mapped to any msi-specifier." > > That would break a valid case of the mask being all zeroes. > > Consider the case that all RIDs get mapped to a single msi-specifier; > the obvious way to write that is: > > msi-map-mask = <0x0000>; > msi-map = <0x0000 &msi (msi-specifier) 1>; > > In this case all RIDS are always mapped to the single msi-specifier. Does it really break that case? We could have this (your example): msi-map-mask = <0x0000>; msi-map = <0x0000 &msi 7 1>; // map all RIDs to msi-spec 7 Or, my example: msi-map-mask = <0x0000>; msi-map = <0x0000 &msi 7 4>; // all RIDs map to any of msi-spec 7,8,9,10 > > We have an SoC with a programmable hardware table in the PCI controller that maps > > requester ID to stream ID, so the overall msi-map (and iommu-map) definition fit > > into that scheme. But, we would like to be able make the RID->stream-ID mapping > > decision _lazily_, in Linux, based on actual usage of PCI devices. > > Dynamically programming the mapping is at odds to this binding. I don't > see how that can fit. My example above obviously doesn't make sense for a static binding, but can't we allow both a mask value of 0x0 and a length > 1 at the same time. > Why can the RID->SID mapping not be statically configured prior to > entering the OS? The problem for us is the limited number of SIDs available on our SoC. We have an SMMU-500 with 128 SIDs / 64-contexts total. An SR-IOV card could enable VFs dynamically and suddenly there are 64 new RIDs on a PCI bus. There are 4 PCI controllers and firmware doesn't know what might be enabled on which bus. We run out of available SIDs. In that example we want all RIDs mapped to one SID by default, but want the option of setting our dynamic RID->SID table for a situation where say someone assigns a VF to a KVM VM and thus needs an SID to use for SMMU mappings. I think the binding can work for the dynamic case as long as we allow the example I showed above. So, I would propose changing my proposed text to: "A mask value of 0x0 is valid and indicates that all RIDS map to the specified msi-specifier(s). If the mask is 0x0 and length > 1 it indicates that all RIDs map to any of the msi-specifiers with the actual mapping left unspecified by the msi-map property." Thanks, Stuart -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Mark, Thanks for the response. Please find my comments inline. Regards Varun > -----Original Message----- > From: Mark Rutland [mailto:mark.rutland@arm.com] > Sent: Thursday, August 06, 2015 11:09 PM > To: Sethi Varun-B16395 > Cc: devicetree@vger.kernel.org; Lorenzo Pieralisi; arnd@arndb.de; Marc > Zyngier; Will Deacon; linux-kernel@vger.kernel.org; > ddaney@caviumnetworks.com; iommu@lists.linux-foundation.org; > tirumalesh.chalamarla@caviumnetworks.com; > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com; > treding@nvidia.com; linux-arm-kernel@lists.infradead.org; > majun258@huawei.com; Yoder Stuart-B08248 > Subject: Re: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > On Wed, Aug 05, 2015 at 05:39:33PM +0100, Varun Sethi wrote: > > Hi Mark > > Thanks for the patch. Please find my comment inline. > > > > Regards > > Varun > > > > > -----Original Message----- > > > From: iommu-bounces@lists.linux-foundation.org [mailto:iommu- > > > bounces@lists.linux-foundation.org] On Behalf Of Mark Rutland > > > Sent: Thursday, July 23, 2015 10:23 PM > > > To: devicetree@vger.kernel.org > > > Cc: Mark Rutland; lorenzo.pieralisi@arm.com; arnd@arndb.de; > > > marc.zyngier@arm.com; will.deacon@arm.com; linux- > > > kernel@vger.kernel.org; ddaney@caviumnetworks.com; > > > iommu@lists.linux- foundation.org; > > > tirumalesh.chalamarla@caviumnetworks.com; > > > laurent.pinchart@ideasonboard.com; thunder.leizhen@huawei.com; > > > treding@nvidia.com; linux-arm-kernel@lists.infradead.org; > > > majun258@huawei.com > > > Subject: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > > > > > Currently msi-parent is used by a few bindings to describe the > > > relationship between a PCI root complex and a single MSI controller, > > > but this property does not have a generic binding document. > > > > > > Additionally, msi-parent is insufficient to describe more complex > > > relationships between MSI controllers and devices under a root > > > complex, where devices may be able to target multiple MSI > > > controllers, or where MSI controllers use (non-probeable) sideband > information to distinguish devices. > > > > > > This patch adds a generic binding for mapping PCI devices to MSI > controllers. > > > This document covers msi-parent, and a new msi-map property > > > (specific to > > > PCI*) which may be used to map devices (identified by their > > > Requester ID) to sideband data for each MSI controller that they may > target. > > > > > > Signed-off-by: Mark Rutland <mark.rutland@arm.com> > > > --- > > > Documentation/devicetree/bindings/pci/pci-msi.txt | 220 > > > ++++++++++++++++++++++ > > > 1 file changed, 220 insertions(+) > > > create mode 100644 > > > Documentation/devicetree/bindings/pci/pci-msi.txt > > > > > > diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt > > > b/Documentation/devicetree/bindings/pci/pci-msi.txt > > > new file mode 100644 > > > index 0000000..9b3cc81 > > > --- /dev/null > > > +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt > > > @@ -0,0 +1,220 @@ > > > +This document describes the generic device tree binding for > > > +describing the relationship between PCI devices and MSI controllers. > > > + > > > +Each PCI device under a root complex is uniquely identified by its > > > +Requester ID (AKA RID). A Requester ID is a triplet of a Bus > > > +number, Device number, and Function number. > > > + > > > +For the purpose of this document, when treated as a numeric value, > > > +a RID is formatted such that: > > > + > > > +* Bits [15:8] are the Bus number. > > > +* Bits [7:3] are the Device number. > > > +* Bits [2:0] are the Function number. > > > +* Any other bits required for padding must be zero. > > > + > > > +MSIs may be distinguished in part through the use of sideband data > > > +accompanying writes. In the case of PCI devices, this sideband data > > > +may be derived from the Requester ID. A mechanism is required to > > > +associate a device with both the MSI controllers it can address, > > > +and the sideband data that will be associated with its writes to those > controllers. > > > + > > > +For generic MSI bindings, see > > > +Documentation/devicetree/bindings/interrupt-controller/msi.txt. > > > + > > > + > > > +PCI root complex > > > +================ > > > + > > > +Optional properties > > > +------------------- > > > + > > > +- msi-map: Maps a Requester ID to an MSI controller and associated > > > + msi-specifier data. The property is an arbitrary number of tuples > > > +of > > > + (rid-base,msi-controller,msi-base,length), where: > > [varun] How would we account for hot plug PCI devices and SR-IOV use > cases, with the rid base and length? > > For hotplug, you simply need the mapping from RID to msi-specifier to be > defined in advance in the DT, for the set of RIDs that could possibly occur. > > For SR-IOV, are you asking about ARI? I should update the description of the > RID to describe that for ARI it has the format: > > * Bits [15:8] are the Bus number > * Bits [7:0] are the Identifier > > Other than that, the handling would be identical to the non-ARI case. > > What else am I missing? > > > How do we take in to account for a PCIe bridge, while setting up the > requestor ID base and length? > > I'm not sure I follow the question. I don't see why this is any different to any > other requester ID. > > What do you see as being the problem for this case? [varun] Would the boot loader be responsible for the PCI bus probe and setting up of the requestor id information in the device tree. Also, during bus probe the boot loader would understand the topology, and handle cases like non transparent host bridge and non standard devices (e.g. require special handling based on quirks). Also, the msi identifier would typically be the stream ID provided by the device. So, this would be limited by the number of SMRs, right? Considering that the each device can have its specific IOMMU domain, which would also require mapping for MSI access, we may also be restricted by the number of context banks. So, we may be able to support a restricted number of msi identifiers. We already have the PCI bus probe in Linux. If my assumption about the boot loader doing the probe is correct, why do we want to duplicate the work. Also, can we just provide a list of possible MSI identifiers to Linux for each PCIe controller. The IOMMU driver in the Linux kernel can interface with PCIe controller for setting up "requestor id"->"msi identifier" translation. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi Mark, First of all: Thanks for working on this. I now have a prototype implementation for irq-gic-v3-its.c that is using this binding on Cavium's ThunderX platform. Q: Have you guys had any more thoughts on this that might require changing the binding? If not, I will be sending out my patches for your consideration. Thanks, David Daney On 07/27/2015 01:16 AM, Marc Zyngier wrote: > On 23/07/15 17:52, Mark Rutland wrote: >> Currently msi-parent is used by a few bindings to describe the >> relationship between a PCI root complex and a single MSI controller, but >> this property does not have a generic binding document. >> >> Additionally, msi-parent is insufficient to describe more complex >> relationships between MSI controllers and devices under a root complex, >> where devices may be able to target multiple MSI controllers, or where >> MSI controllers use (non-probeable) sideband information to distinguish >> devices. >> >> This patch adds a generic binding for mapping PCI devices to MSI >> controllers. This document covers msi-parent, and a new msi-map property >> (specific to PCI*) which may be used to map devices (identified by their >> Requester ID) to sideband data for each MSI controller that they may >> target. >> >> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: David Daney <david.daney@cavium.com> >> --- >> Documentation/devicetree/bindings/pci/pci-msi.txt | 220 ++++++++++++++++++++++ >> 1 file changed, 220 insertions(+) >> create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt >> >> diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt b/Documentation/devicetree/bindings/pci/pci-msi.txt >> new file mode 100644 >> index 0000000..9b3cc81 >> --- /dev/null >> +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt >> @@ -0,0 +1,220 @@ >> +This document describes the generic device tree binding for describing the >> +relationship between PCI devices and MSI controllers. >> + >> +Each PCI device under a root complex is uniquely identified by its Requester ID >> +(AKA RID). A Requester ID is a triplet of a Bus number, Device number, and >> +Function number. >> + >> +For the purpose of this document, when treated as a numeric value, a RID is >> +formatted such that: >> + >> +* Bits [15:8] are the Bus number. >> +* Bits [7:3] are the Device number. >> +* Bits [2:0] are the Function number. >> +* Any other bits required for padding must be zero. >> + >> +MSIs may be distinguished in part through the use of sideband data accompanying >> +writes. In the case of PCI devices, this sideband data may be derived from the >> +Requester ID. A mechanism is required to associate a device with both the MSI >> +controllers it can address, and the sideband data that will be associated with >> +its writes to those controllers. >> + >> +For generic MSI bindings, see >> +Documentation/devicetree/bindings/interrupt-controller/msi.txt. >> + >> + >> +PCI root complex >> +================ >> + >> +Optional properties >> +------------------- >> + >> +- msi-map: Maps a Requester ID to an MSI controller and associated >> + msi-specifier data. The property is an arbitrary number of tuples of >> + (rid-base,msi-controller,msi-base,length), where: >> + >> + * rid-base is a single cell describing the first RID matched by the entry. >> + >> + * msi-controller is a single phandle to an MSI controller >> + >> + * msi-base is an msi-specifier describing the msi-specifier produced for the >> + first RID matched by the entry. >> + >> + * length is a single cell describing how many consecutive RIDs are matched >> + following the rid-base. >> + >> + Any RID r in the interval [rid-base, rid-base + length) is associated with >> + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). >> + >> +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped >> + to an msi-specifier per the msi-map property. >> + >> +- msi-parent: Describes the MSI parent of the root complex itself. Where >> + the root complex and MSI controller do not pass sideband data with MSI >> + writes, this property may be used to describe the MSI controller(s) >> + used by PCI devices under the root complex, if defined as such in the >> + binding for the root complex. > > Right, this is where I'd expect some details about #msi-cells. Is it > meant to be ignored? The lack of symmetry between the PCI and non-PCI > use cases feels a bit inelegant (not to mention that it precludes having > an unified parser for both cases). > > This otherwise looks good to me. > > Thanks, > > M. > -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> > +PCI root complex > > +================ > > + > > +Optional properties > > +------------------- > > + > > +- msi-map: Maps a Requester ID to an MSI controller and associated > > + msi-specifier data. The property is an arbitrary number of tuples of > > + (rid-base,msi-controller,msi-base,length), where: > > + > > + * rid-base is a single cell describing the first RID matched by the entry. > > + > > + * msi-controller is a single phandle to an MSI controller > > + > > + * msi-base is an msi-specifier describing the msi-specifier produced for the > > + first RID matched by the entry. > > + > > + * length is a single cell describing how many consecutive RIDs are matched > > + following the rid-base. > > + > > + Any RID r in the interval [rid-base, rid-base + length) is associated with > > + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). > > + > > +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped > > + to an msi-specifier per the msi-map property. > > + > > +- msi-parent: Describes the MSI parent of the root complex itself. Where > > + the root complex and MSI controller do not pass sideband data with MSI > > + writes, this property may be used to describe the MSI controller(s) > > + used by PCI devices under the root complex, if defined as such in the > > + binding for the root complex. > > Right, this is where I'd expect some details about #msi-cells. Is it > meant to be ignored? The lack of symmetry between the PCI and non-PCI > use cases feels a bit inelegant (not to mention that it precludes having > an unified parser for both cases). I would expect that the msi-parent's #msi-cells could be non-zero. As I see it, there are three possible interpretations (and the choice between these would be specific to a PCI root complex): (1) Devices under the root complex generate MSIs via the MSI parent, using common sideband data described in the msi cells (and therefore cannot be distingushed from each other). (2) The root complex itself generates MSIs via the MSI parent, with any sideband data described in the msi cells. (3) Both the root complex and devices underneath it generate MSIs via the MSI parent, using common sideband data described in the msi cells (and therefore cannot be distinguished from each other). Of these, (1) is the common case today, though (2) would be more in keeping with how this style of property usually works. I'm not sure how to capture that. Thanks, Mark. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 04, 2015 at 11:33:35PM +0100, David Daney wrote: > Hi Mark, Hi David, > I now have a prototype implementation for irq-gic-v3-its.c that is using > this binding on Cavium's ThunderX platform. > > Q: Have you guys had any more thoughts on this that might require > changing the binding? Having discussed this with Stuart and others at Linux Plumbers, I think that the binding is sufficient, and unlikely to change greatly unless there is a strong objection (Stuart, please correct me if I am wrong). Per Marc's comments there are probably some edge cases and/or wording details to sort out, but I think the common/simple case is sorted. I'll send a v2 once those have been settled. > If not, I will be sending out my patches for your consideration. Please do! > Thanks, > David Daney > > On 07/27/2015 01:16 AM, Marc Zyngier wrote: > > On 23/07/15 17:52, Mark Rutland wrote: > >> Currently msi-parent is used by a few bindings to describe the > >> relationship between a PCI root complex and a single MSI controller, but > >> this property does not have a generic binding document. > >> > >> Additionally, msi-parent is insufficient to describe more complex > >> relationships between MSI controllers and devices under a root complex, > >> where devices may be able to target multiple MSI controllers, or where > >> MSI controllers use (non-probeable) sideband information to distinguish > >> devices. > >> > >> This patch adds a generic binding for mapping PCI devices to MSI > >> controllers. This document covers msi-parent, and a new msi-map property > >> (specific to PCI*) which may be used to map devices (identified by their > >> Requester ID) to sideband data for each MSI controller that they may > >> target. > >> > >> Signed-off-by: Mark Rutland <mark.rutland@arm.com> > > Acked-by: David Daney <david.daney@cavium.com> Thanks! Mark. -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> -----Original Message----- > From: Mark Rutland [mailto:mark.rutland@arm.com] > Sent: Monday, September 07, 2015 1:05 PM > To: David Daney > Cc: Marc Zyngier; tirumalesh.chalamarla@caviumnetworks.com; Richter, Robert; Chintakuntla, Radha; > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org; iommu@lists.linux-foundation.org; linux-arm- > kernel@lists.infradead.org; Will Deacon; Robin Murphy; Lorenzo Pieralisi; arnd@arndb.de; treding@nvidia.com; > majun258@huawei.com; thunder.leizhen@huawei.com; laurent.pinchart@ideasonboard.com; Yoder Stuart-B08248 > Subject: Re: [PATCH 2/3] Docs: dt: Add PCI MSI map bindings > > On Fri, Sep 04, 2015 at 11:33:35PM +0100, David Daney wrote: > > Hi Mark, > > Hi David, > > > I now have a prototype implementation for irq-gic-v3-its.c that is using > > this binding on Cavium's ThunderX platform. > > > > Q: Have you guys had any more thoughts on this that might require > > changing the binding? > > Having discussed this with Stuart and others at Linux Plumbers, I think > that the binding is sufficient, and unlikely to change greatly unless > there is a strong objection (Stuart, please correct me if I am wrong). > > Per Marc's comments there are probably some edge cases and/or wording > details to sort out, but I think the common/simple case is sorted. I'll > send a v2 once those have been settled. I think the binding as-is, is sufficient for the static description of RID to SID. I think the binding can be extended in a backwards compatible way to support dynamic RID->SID mappings if we need that in the future. The case that we (Freescale) are concerned with are where someone plugs an SRIOV card into an SoC and enables, say 64 VFs. It's like hot-plugging 64 new PCI devices at once. If all those devices that show up belong to the host they belong to one 'isolation context' and could share the same streamID, same SMMU mappings, and the same ITT in the GIC ITS. So a static mapping could work. But, as soon as I want to assign VF#5 to a virtual machine I need a new RID->SID mapping for VF#5. To require firmware to do that mapping ahead of time is a _real_ pain. I think longer term we need some mechanism to allow lazy, dynamic RID->SID mappings by Linux. We can start with the static approach, but we need to keep in the back of our minds that there may be cases in the near future where a static mapping is too inflexible. Thanks, Stuart -- To unsubscribe from this list: send the line "unsubscribe devicetree" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/Documentation/devicetree/bindings/pci/pci-msi.txt b/Documentation/devicetree/bindings/pci/pci-msi.txt new file mode 100644 index 0000000..9b3cc81 --- /dev/null +++ b/Documentation/devicetree/bindings/pci/pci-msi.txt @@ -0,0 +1,220 @@ +This document describes the generic device tree binding for describing the +relationship between PCI devices and MSI controllers. + +Each PCI device under a root complex is uniquely identified by its Requester ID +(AKA RID). A Requester ID is a triplet of a Bus number, Device number, and +Function number. + +For the purpose of this document, when treated as a numeric value, a RID is +formatted such that: + +* Bits [15:8] are the Bus number. +* Bits [7:3] are the Device number. +* Bits [2:0] are the Function number. +* Any other bits required for padding must be zero. + +MSIs may be distinguished in part through the use of sideband data accompanying +writes. In the case of PCI devices, this sideband data may be derived from the +Requester ID. A mechanism is required to associate a device with both the MSI +controllers it can address, and the sideband data that will be associated with +its writes to those controllers. + +For generic MSI bindings, see +Documentation/devicetree/bindings/interrupt-controller/msi.txt. + + +PCI root complex +================ + +Optional properties +------------------- + +- msi-map: Maps a Requester ID to an MSI controller and associated + msi-specifier data. The property is an arbitrary number of tuples of + (rid-base,msi-controller,msi-base,length), where: + + * rid-base is a single cell describing the first RID matched by the entry. + + * msi-controller is a single phandle to an MSI controller + + * msi-base is an msi-specifier describing the msi-specifier produced for the + first RID matched by the entry. + + * length is a single cell describing how many consecutive RIDs are matched + following the rid-base. + + Any RID r in the interval [rid-base, rid-base + length) is associated with + the listed msi-controller, with the msi-specifier (r - rid-base + msi-base). + +- msi-map-mask: A mask to be applied to each Requester ID prior to being mapped + to an msi-specifier per the msi-map property. + +- msi-parent: Describes the MSI parent of the root complex itself. Where + the root complex and MSI controller do not pass sideband data with MSI + writes, this property may be used to describe the MSI controller(s) + used by PCI devices under the root complex, if defined as such in the + binding for the root complex. + + +Example (1) +=========== + +/ { + #address-cells = <1>; + #size-cells = <1>; + + msi: msi-controller@a { + reg = <0xa 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + pci: pci@f { + reg = <0xf 0x1>; + compatible = "vendor,pcie-root-complex"; + device_type = "pci"; + + /* + * The sideband data provided to the MSI controller is + * the RID, identity-mapped. + */ + msi-map = <0x0 &msi_a 0x0 0x10000>, + }; +}; + + +Example (2) +=========== + +/ { + #address-cells = <1>; + #size-cells = <1>; + + msi: msi-controller@a { + reg = <0xa 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + pci: pci@f { + reg = <0xf 0x1>; + compatible = "vendor,pcie-root-complex"; + device_type = "pci"; + + /* + * The sideband data provided to the MSI controller is + * the RID, masked to only the device and function bits. + */ + msi-map = <0x0 &msi_a 0x0 0x100>, + msi-map-mask = <0xff> + }; +}; + + +Example (3) +=========== + +/ { + #address-cells = <1>; + #size-cells = <1>; + + msi: msi-controller@a { + reg = <0xa 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + pci: pci@f { + reg = <0xf 0x1>; + compatible = "vendor,pcie-root-complex"; + device_type = "pci"; + + /* + * The sideband data provided to the MSI controller is + * the RID, but the high bit of the bus number is + * ignored. + */ + msi-map = <0x0000 &msi 0x0000 0x8000>, + <0x8000 &msi 0x0000 0x8000>; + }; +}; + + +Example (4) +=========== + +/ { + #address-cells = <1>; + #size-cells = <1>; + + msi: msi-controller@a { + reg = <0xa 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + pci: pci@f { + reg = <0xf 0x1>; + compatible = "vendor,pcie-root-complex"; + device_type = "pci"; + + /* + * The sideband data provided to the MSI controller is + * the RID, but the high bit of the bus number is + * negated. + */ + msi-map = <0x0000 &msi 0x8000 0x8000>, + <0x8000 &msi 0x0000 0x8000>; + }; +}; + + +Example (5) +=========== + +/ { + #address-cells = <1>; + #size-cells = <1>; + + msi_a: msi-controller@a { + reg = <0xa 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + msi_b: msi-controller@b { + reg = <0xb 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + msi_c: msi-controller@c { + reg = <0xc 0x1>; + compatible = "vendor,some-controller"; + msi-controller; + #msi-cells = <1>; + }; + + pci: pci@c { + reg = <0xf 0x1>; + compatible = "vendor,pcie-root-complex"; + device_type = "pci"; + + /* + * The sideband data provided to MSI controller a is the + * RID, but the high bit of the bus number is negated. + * The sideband data provided to MSI controller b is the + * RID, identity-mapped. + * MSI controller c is not addressable. + */ + msi-map = <0x0000 &msi_a 0x8000 0x08000>, + <0x8000 &msi_a 0x0000 0x08000>, + <0x0000 &msi_b 0x0000 0x10000>; + }; +};
Currently msi-parent is used by a few bindings to describe the relationship between a PCI root complex and a single MSI controller, but this property does not have a generic binding document. Additionally, msi-parent is insufficient to describe more complex relationships between MSI controllers and devices under a root complex, where devices may be able to target multiple MSI controllers, or where MSI controllers use (non-probeable) sideband information to distinguish devices. This patch adds a generic binding for mapping PCI devices to MSI controllers. This document covers msi-parent, and a new msi-map property (specific to PCI*) which may be used to map devices (identified by their Requester ID) to sideband data for each MSI controller that they may target. Signed-off-by: Mark Rutland <mark.rutland@arm.com> --- Documentation/devicetree/bindings/pci/pci-msi.txt | 220 ++++++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 Documentation/devicetree/bindings/pci/pci-msi.txt