Message ID | 1686847842-33780-1-git-send-email-lizhi.hou@amd.com |
---|---|
Headers | show |
Series | Generate device tree node for pci devices | expand |
On Thu, Jun 15, 2023 at 09:50:40AM -0700, Lizhi Hou wrote: > For PCI endpoint defined quirks to generate device tree node, it requires > 'ranges' property to translate iomem addresses for its downstream devices. I'm not following why this patch is separate from patch 2 nor how patch 3 would function without it. > > Signed-off-by: Lizhi Hou <lizhi.hou@amd.com> > --- > drivers/pci/of_property.c | 33 ++++++++++++++++++++++----------- > drivers/pci/quirks.c | 1 + > 2 files changed, 23 insertions(+), 11 deletions(-) > > diff --git a/drivers/pci/of_property.c b/drivers/pci/of_property.c > index bdd756c8d7de..08654740f314 100644 > --- a/drivers/pci/of_property.c > +++ b/drivers/pci/of_property.c > @@ -84,15 +84,22 @@ static int of_pci_prop_ranges(struct pci_dev *pdev, struct of_changeset *ocs, > struct of_pci_range *rp; > struct resource *res; > int i = 0, j, ret; > + u32 flags, num; > u64 val64; > - u32 flags; > > - rp = kcalloc(PCI_BRIDGE_RESOURCE_NUM, sizeof(*rp), GFP_KERNEL); > + if (pci_is_bridge(pdev)) { > + num = PCI_BRIDGE_RESOURCE_NUM; > + res = &pdev->resource[PCI_BRIDGE_RESOURCES]; > + } else { > + num = PCI_STD_NUM_BARS; > + res = &pdev->resource[PCI_STD_RESOURCES]; > + } > + > + rp = kcalloc(num, sizeof(*rp), GFP_KERNEL); > if (!rp) > return -ENOMEM; > > - res = &pdev->resource[PCI_BRIDGE_RESOURCES]; > - for (j = 0; j < PCI_BRIDGE_RESOURCE_NUM; j++) { > + for (j = 0; j < num; j++) { > if (!resource_size(&res[j])) > continue; > > @@ -102,8 +109,12 @@ static int of_pci_prop_ranges(struct pci_dev *pdev, struct of_changeset *ocs, > val64 = res[j].start; > of_pci_set_address(pdev, rp[i].parent_addr, val64, 0, flags, > false); > - memcpy(rp[i].child_addr, rp[i].parent_addr, > - sizeof(rp[i].child_addr)); > + if (pci_is_bridge(pdev)) { > + memcpy(rp[i].child_addr, rp[i].parent_addr, > + sizeof(rp[i].child_addr)); > + } else { > + rp[i].child_addr[0] = j; A comment that child address lower 64-bits is always 0x0 would be helpful here. > + } > > val64 = resource_size(&res[j]); > rp[i].size[0] = upper_32_bits(val64); > @@ -161,13 +172,13 @@ int of_pci_add_properties(struct pci_dev *pdev, struct of_changeset *ocs, > if (pci_is_bridge(pdev)) { > ret |= of_changeset_add_prop_string(ocs, np, "device_type", > "pci"); > - ret |= of_changeset_add_prop_u32(ocs, np, "#address-cells", > - OF_PCI_ADDRESS_CELLS); > - ret |= of_changeset_add_prop_u32(ocs, np, "#size-cells", > - OF_PCI_SIZE_CELLS); > - ret |= of_pci_prop_ranges(pdev, ocs, np); > } > > + ret |= of_pci_prop_ranges(pdev, ocs, np); > + ret |= of_changeset_add_prop_u32(ocs, np, "#address-cells", > + OF_PCI_ADDRESS_CELLS); > + ret |= of_changeset_add_prop_u32(ocs, np, "#size-cells", > + OF_PCI_SIZE_CELLS); > ret |= of_pci_prop_reg(pdev, ocs, np); > ret |= of_pci_prop_compatible(pdev, ocs, np); > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index c8f3acea752d..51945b631628 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -6052,3 +6052,4 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9a31, dpc_log_size); > */ > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_XILINX, 0x5020, of_pci_make_dev_node); > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_XILINX, 0x5021, of_pci_make_dev_node); > +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_REDHAT, 0x0005, of_pci_make_dev_node); > -- > 2.34.1 >
On Thu, Jun 15, 2023 at 09:50:36AM -0700, Lizhi Hou wrote: > This patch series introduces OF overlay support for PCI devices which > primarily addresses two use cases. First, it provides a data driven method > to describe hardware peripherals that are present in a PCI endpoint and > hence can be accessed by the PCI host. Second, it allows reuse of a OF > compatible driver -- often used in SoC platforms -- in a PCI host based > system. > > There are 2 series devices rely on this patch: > > 1) Xilinx Alveo Accelerator cards (FPGA based device) > 2) Microchip LAN9662 Ethernet Controller > > Please see: https://lore.kernel.org/lkml/20220427094502.456111-1-clement.leger@bootlin.com/ > > Normally, the PCI core discovers PCI devices and their BARs using the > PCI enumeration process. However, the process does not provide a way to > discover the hardware peripherals that are present in a PCI device, and > which can be accessed through the PCI BARs. Also, the enumeration process > does not provide a way to associate MSI-X vectors of a PCI device with the > hardware peripherals that are present in the device. PCI device drivers > often use header files to describe the hardware peripherals and their > resources as there is no standard data driven way to do so. This patch > series proposes to use flattened device tree blob to describe the > peripherals in a data driven way. Based on previous discussion, using > device tree overlay is the best way to unflatten the blob and populate > platform devices. To use device tree overlay, there are three obvious > problems that need to be resolved. > > First, we need to create a base tree for non-DT system such as x86_64. A > patch series has been submitted for this: > https://lore.kernel.org/lkml/20220624034327.2542112-1-frowand.list@gmail.com/ > https://lore.kernel.org/lkml/20220216050056.311496-1-lizhi.hou@xilinx.com/ > > Second, a device tree node corresponding to the PCI endpoint is required > for overlaying the flattened device tree blob for that PCI endpoint. > Because PCI is a self-discoverable bus, a device tree node is usually not > created for PCI devices. This series adds support to generate a device > tree node for a PCI device which advertises itself using PCI quirks > infrastructure. > > Third, we need to generate device tree nodes for PCI bridges since a child > PCI endpoint may choose to have a device tree node created. > > This patch series is made up of three patches. > > The first patch is adding OF interface to create or destroy OF node > dynamically. > > The second patch introduces a kernel option, CONFIG_DYNAMIC_PCI_OF_NODEX. > When the option is turned on, the kernel will generate device tree nodes > for all PCI bridges unconditionally. The patch also shows how to use the > PCI quirks infrastructure, DECLARE_PCI_FIXUP_FINAL to generate a device > tree node for a device. Specifically, the patch generates a device tree > node for Xilinx Alveo U50 PCIe accelerator device. The generated device > tree nodes do not have any property. > > The third patch adds basic properties ('reg', 'compatible' and > 'device_type') to the dynamically generated device tree nodes. More > properties can be added in the future. > > Here is the example of device tree nodes generated within the ARM64 QEMU. > # lspci -t > -[0000:00]-+-00.0 > +-01.0 > +-03.0-[01-03]----00.0-[02-03]----00.0-[03]----00.0 > +-03.1-[04]-- > \-04.0-[05-06]----00.0-[06]-- > # tree /sys/firmware/devicetree/base/pcie\@10000000 > /sys/firmware/devicetree/base/pcie@10000000 > |-- #address-cells > |-- #interrupt-cells > |-- #size-cells > |-- bus-range > |-- compatible > |-- device_type > |-- dma-coherent > |-- interrupt-map > |-- interrupt-map-mask > |-- linux,pci-domain > |-- msi-map > |-- name > |-- pci@3,0 > | |-- #address-cells > | |-- #size-cells > | |-- compatible > | |-- device_type > | |-- pci@0,0 > | | |-- #address-cells > | | |-- #size-cells > | | |-- compatible > | | |-- device_type > | | |-- pci@0,0 > | | | |-- #address-cells > | | | |-- #size-cells > | | | |-- compatible > | | | |-- dev@0,0 > | | | | |-- #address-cells > | | | | |-- #size-cells > | | | | |-- compatible > | | | | |-- ranges > | | | | `-- reg > | | | |-- device_type > | | | |-- ranges > | | | `-- reg > | | |-- ranges > | | `-- reg > | |-- ranges > | `-- reg > |-- pci@3,1 > | |-- #address-cells > | |-- #size-cells > | |-- compatible > | |-- device_type > | |-- ranges > | `-- reg > |-- pci@4,0 > | |-- #address-cells > | |-- #size-cells > | |-- compatible > | |-- device_type > | |-- pci@0,0 > | | |-- #address-cells > | | |-- #size-cells > | | |-- compatible > | | |-- device_type > | | |-- ranges > | | `-- reg > | |-- ranges > | `-- reg > |-- ranges > `-- reg > > Changes since v8: > - Added patches to create unit test to verifying address translation > The test relies on QEMU PCI Test Device, please see > https://github.com/houlz0507/xoclv2/blob/pci-dt-0329/pci-dt-patch-0329/README > for test setup > - Minor code review fixes > > Changes since v7: > - Modified dynamic node creation interfaces > - Added unittest for new added interfaces > > Changes since v6: > - Removed single line wrapper functions > - Added Signed-off-by Clément Léger <clement.leger@bootlin.com> > > Changes since v5: > - Fixed code review comments > - Fixed incorrect 'ranges' and 'reg' properties > > Changes since RFC v4: > - Fixed code review comments > > Changes since RFC v3: > - Split the Xilinx Alveo U50 PCI quirk to a separate patch > - Minor changes in commit description and code comment > > Changes since RFC v2: > - Merged patch 3 with patch 2 > - Added OF interfaces of_changeset_add_prop_* and use them to create > properties. > - Added '#address-cells', '#size-cells' and 'ranges' properties. > > Changes since RFC v1: > - Added one patch to create basic properties. > - To move DT related code out of PCI subsystem, replaced of_node_alloc() > with of_create_node()/of_destroy_node() > > Lizhi Hou (6): > of: dynamic: Add interfaces for creating device node dynamically > PCI: Create device tree node for selected devices > PCI: Add PCI quirks to generate device tree node for Xilinx Alveo U50 > PCI: Add ranges property for pci endpoint > of: overlay: Extend of_overlay_fdt_apply() to specify the target node > of: unittest: Add pci_dt_testdrv pci driver > > drivers/of/dynamic.c | 164 ++++++++++++++ > drivers/of/overlay.c | 42 +++- > drivers/of/unittest-data/Makefile | 3 +- > .../of/unittest-data/overlay_pci_node.dtso | 22 ++ > drivers/of/unittest.c | 210 +++++++++++++++++- > drivers/pci/Kconfig | 12 + > drivers/pci/Makefile | 1 + > drivers/pci/bus.c | 2 + > drivers/pci/of.c | 81 ++++++- > drivers/pci/of_property.c | 190 ++++++++++++++++ > drivers/pci/pci.h | 19 ++ > drivers/pci/quirks.c | 12 + > drivers/pci/remove.c | 1 + > include/linux/of.h | 25 ++- > 14 files changed, 768 insertions(+), 16 deletions(-) > create mode 100644 drivers/of/unittest-data/overlay_pci_node.dtso > create mode 100644 drivers/pci/of_property.c Bjorn, I think this is pretty close to being in shape for merging. Do you have any comments on the PCI bits? Would you prefer that I ack the DT bit and you take it or vice-versa? Rob
On 6/20/23 15:00, Rob Herring wrote: > On Thu, Jun 15, 2023 at 09:50:40AM -0700, Lizhi Hou wrote: >> For PCI endpoint defined quirks to generate device tree node, it requires >> 'ranges' property to translate iomem addresses for its downstream devices. > I'm not following why this patch is separate from patch 2 nor how patch > 3 would function without it. Ok. I will merge this with patch 2. > >> Signed-off-by: Lizhi Hou <lizhi.hou@amd.com> >> --- >> drivers/pci/of_property.c | 33 ++++++++++++++++++++++----------- >> drivers/pci/quirks.c | 1 + >> 2 files changed, 23 insertions(+), 11 deletions(-) >> >> diff --git a/drivers/pci/of_property.c b/drivers/pci/of_property.c >> index bdd756c8d7de..08654740f314 100644 >> --- a/drivers/pci/of_property.c >> +++ b/drivers/pci/of_property.c >> @@ -84,15 +84,22 @@ static int of_pci_prop_ranges(struct pci_dev *pdev, struct of_changeset *ocs, >> struct of_pci_range *rp; >> struct resource *res; >> int i = 0, j, ret; >> + u32 flags, num; >> u64 val64; >> - u32 flags; >> >> - rp = kcalloc(PCI_BRIDGE_RESOURCE_NUM, sizeof(*rp), GFP_KERNEL); >> + if (pci_is_bridge(pdev)) { >> + num = PCI_BRIDGE_RESOURCE_NUM; >> + res = &pdev->resource[PCI_BRIDGE_RESOURCES]; >> + } else { >> + num = PCI_STD_NUM_BARS; >> + res = &pdev->resource[PCI_STD_RESOURCES]; >> + } >> + >> + rp = kcalloc(num, sizeof(*rp), GFP_KERNEL); >> if (!rp) >> return -ENOMEM; >> >> - res = &pdev->resource[PCI_BRIDGE_RESOURCES]; >> - for (j = 0; j < PCI_BRIDGE_RESOURCE_NUM; j++) { >> + for (j = 0; j < num; j++) { >> if (!resource_size(&res[j])) >> continue; >> >> @@ -102,8 +109,12 @@ static int of_pci_prop_ranges(struct pci_dev *pdev, struct of_changeset *ocs, >> val64 = res[j].start; >> of_pci_set_address(pdev, rp[i].parent_addr, val64, 0, flags, >> false); >> - memcpy(rp[i].child_addr, rp[i].parent_addr, >> - sizeof(rp[i].child_addr)); >> + if (pci_is_bridge(pdev)) { >> + memcpy(rp[i].child_addr, rp[i].parent_addr, >> + sizeof(rp[i].child_addr)); >> + } else { >> + rp[i].child_addr[0] = j; > A comment that child address lower 64-bits is always 0x0 would be > helpful here. Sure, I will add a comment. Thanks, Lizhi > >> + } >> >> val64 = resource_size(&res[j]); >> rp[i].size[0] = upper_32_bits(val64); >> @@ -161,13 +172,13 @@ int of_pci_add_properties(struct pci_dev *pdev, struct of_changeset *ocs, >> if (pci_is_bridge(pdev)) { >> ret |= of_changeset_add_prop_string(ocs, np, "device_type", >> "pci"); >> - ret |= of_changeset_add_prop_u32(ocs, np, "#address-cells", >> - OF_PCI_ADDRESS_CELLS); >> - ret |= of_changeset_add_prop_u32(ocs, np, "#size-cells", >> - OF_PCI_SIZE_CELLS); >> - ret |= of_pci_prop_ranges(pdev, ocs, np); >> } >> >> + ret |= of_pci_prop_ranges(pdev, ocs, np); >> + ret |= of_changeset_add_prop_u32(ocs, np, "#address-cells", >> + OF_PCI_ADDRESS_CELLS); >> + ret |= of_changeset_add_prop_u32(ocs, np, "#size-cells", >> + OF_PCI_SIZE_CELLS); >> ret |= of_pci_prop_reg(pdev, ocs, np); >> ret |= of_pci_prop_compatible(pdev, ocs, np); >> >> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c >> index c8f3acea752d..51945b631628 100644 >> --- a/drivers/pci/quirks.c >> +++ b/drivers/pci/quirks.c >> @@ -6052,3 +6052,4 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x9a31, dpc_log_size); >> */ >> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_XILINX, 0x5020, of_pci_make_dev_node); >> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_XILINX, 0x5021, of_pci_make_dev_node); >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_REDHAT, 0x0005, of_pci_make_dev_node); >> -- >> 2.34.1 >>
On Tue, Jun 20, 2023 at 04:08:18PM -0600, Rob Herring wrote: > On Thu, Jun 15, 2023 at 09:50:36AM -0700, Lizhi Hou wrote: > > This patch series introduces OF overlay support for PCI devices which > > primarily addresses two use cases. First, it provides a data driven method > > to describe hardware peripherals that are present in a PCI endpoint and > > hence can be accessed by the PCI host. Second, it allows reuse of a OF > > compatible driver -- often used in SoC platforms -- in a PCI host based > > system. > > drivers/of/dynamic.c | 164 ++++++++++++++ > > drivers/of/overlay.c | 42 +++- > > drivers/of/unittest-data/Makefile | 3 +- > > .../of/unittest-data/overlay_pci_node.dtso | 22 ++ > > drivers/of/unittest.c | 210 +++++++++++++++++- > > drivers/pci/Kconfig | 12 + > > drivers/pci/Makefile | 1 + > > drivers/pci/bus.c | 2 + > > drivers/pci/of.c | 81 ++++++- > > drivers/pci/of_property.c | 190 ++++++++++++++++ > > drivers/pci/pci.h | 19 ++ > > drivers/pci/quirks.c | 12 + > > drivers/pci/remove.c | 1 + > > include/linux/of.h | 25 ++- > > 14 files changed, 768 insertions(+), 16 deletions(-) > > create mode 100644 drivers/of/unittest-data/overlay_pci_node.dtso > > create mode 100644 drivers/pci/of_property.c > > Bjorn, I think this is pretty close to being in shape for merging. Do > you have any comments on the PCI bits? Would you prefer that I ack the > DT bit and you take it or vice-versa? I acked the PCI bits (modulo some trivial comments), and this seems more DTish than PCIish, so happy if you would take them. Bjorn