| Message ID | 20220430084846.3127041-5-chenhuacai@loongson.cn |
|---|---|
| State | New |
| Headers | show |
| Series | PCI: Loongson pci improvements and quirks | expand |
On Sat, Apr 30, 2022 at 04:48:44PM +0800, Huacai Chen wrote: > In new revision of LS7A, some PCIe ports support larger value than 256, > but their maximum supported MRRS values are not detectable. Moreover, > the current loongson_mrrs_quirk() cannot avoid devices increasing its > MRRS after pci_enable_device(), and some devices (e.g. Realtek 8169) > will actually set a big value in its driver. So the only possible way > is configure MRRS of all devices in BIOS, and add a pci host bridge bit > flag (i.e., no_inc_mrrs) to stop the increasing MRRS operations. > > However, according to PCIe Spec, it is legal for an OS to program any > value for MRRS, and it is also legal for an endpoint to generate a Read > Request with any size up to its MRRS. As the hardware engineers say, the > root cause here is LS7A doesn't break up large read requests. In detail, > LS7A PCIe port reports CA (Completer Abort) if it receives a Memory Read > request with a size that's "too big" ("too big" means larger than the > PCIe ports can handle, which means 256 for some ports and 4096 for the > others, and of course this is a problem in the LS7A's hardware design). This seems essentially similar to ks_pcie_quirk() [1]. Why are they different, and why do you need no_inc_mrrs, when keystone doesn't? Or *does* keystone need it and we just haven't figured that out yet? Are all callers of pcie_set_readrq() vulnerable to issues there? Whatever we do should be as uniform as possible across host controllers. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pci-keystone.c?id=v5.18#n528 > Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> > --- > drivers/pci/controller/pci-loongson.c | 44 +++++++++------------------ > drivers/pci/pci.c | 6 ++++ > include/linux/pci.h | 1 + > 3 files changed, 22 insertions(+), 29 deletions(-) > > diff --git a/drivers/pci/controller/pci-loongson.c b/drivers/pci/controller/pci-loongson.c > index 48316daa1f23..83447264048a 100644 > --- a/drivers/pci/controller/pci-loongson.c > +++ b/drivers/pci/controller/pci-loongson.c > @@ -67,37 +67,23 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, > DEV_LS7A_LPC, system_bus_quirk); > > -static void loongson_mrrs_quirk(struct pci_dev *dev) > +static void loongson_mrrs_quirk(struct pci_dev *pdev) > { > - struct pci_bus *bus = dev->bus; > - struct pci_dev *bridge; > - static const struct pci_device_id bridge_devids[] = { > - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_0) }, > - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_1) }, > - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_2) }, > - { 0, }, > - }; > - > - /* look for the matching bridge */ > - while (!pci_is_root_bus(bus)) { > - bridge = bus->self; > - bus = bus->parent; > - /* > - * Some Loongson PCIe ports have a h/w limitation of > - * 256 bytes maximum read request size. They can't handle > - * anything larger than this. So force this limit on > - * any devices attached under these ports. > - */ > - if (pci_match_id(bridge_devids, bridge)) { > - if (pcie_get_readrq(dev) > 256) { > - pci_info(dev, "limiting MRRS to 256\n"); > - pcie_set_readrq(dev, 256); > - } > - break; > - } > - } > + /* > + * Some Loongson PCIe ports have h/w limitations of maximum read > + * request size. They can't handle anything larger than this. So > + * force this limit on any devices attached under these ports. > + */ > + struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus); > + > + bridge->no_inc_mrrs = 1; > } > -DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, loongson_mrrs_quirk); > +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, > + DEV_PCIE_PORT_0, loongson_mrrs_quirk); > +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, > + DEV_PCIE_PORT_1, loongson_mrrs_quirk); > +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, > + DEV_PCIE_PORT_2, loongson_mrrs_quirk); > > static struct loongson_pci *pci_bus_to_loongson_pci(struct pci_bus *bus) > { > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index 9ecce435fb3f..72a15bf9eee8 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -5993,6 +5993,7 @@ int pcie_set_readrq(struct pci_dev *dev, int rq) > { > u16 v; > int ret; > + struct pci_host_bridge *bridge = pci_find_host_bridge(dev->bus); > > if (rq < 128 || rq > 4096 || !is_power_of_2(rq)) > return -EINVAL; > @@ -6011,6 +6012,11 @@ int pcie_set_readrq(struct pci_dev *dev, int rq) > > v = (ffs(rq) - 8) << 12; > > + if (bridge->no_inc_mrrs) { > + if (rq > pcie_get_readrq(dev)) > + return -EINVAL; > + } > + > ret = pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL, > PCI_EXP_DEVCTL_READRQ, v); > > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 60adf42460ab..d146eb28e6da 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -569,6 +569,7 @@ struct pci_host_bridge { > void *release_data; > unsigned int ignore_reset_delay:1; /* For entire hierarchy */ > unsigned int no_ext_tags:1; /* No Extended Tags */ > + unsigned int no_inc_mrrs:1; /* No Increase MRRS */ > unsigned int native_aer:1; /* OS may use PCIe AER */ > unsigned int native_pcie_hotplug:1; /* OS may use PCIe hotplug */ > unsigned int native_shpc_hotplug:1; /* OS may use SHPC hotplug */ > -- > 2.27.0 >
在2022年6月1日六月 上午3:22，Bjorn Helgaas写道： > On Sat, Apr 30, 2022 at 04:48:44PM +0800, Huacai Chen wrote: >> In new revision of LS7A, some PCIe ports support larger value than 256, >> but their maximum supported MRRS values are not detectable. Moreover, >> the current loongson_mrrs_quirk() cannot avoid devices increasing its >> MRRS after pci_enable_device(), and some devices (e.g. Realtek 8169) >> will actually set a big value in its driver. So the only possible way >> is configure MRRS of all devices in BIOS, and add a pci host bridge bit >> flag (i.e., no_inc_mrrs) to stop the increasing MRRS operations. >> >> However, according to PCIe Spec, it is legal for an OS to program any >> value for MRRS, and it is also legal for an endpoint to generate a Read >> Request with any size up to its MRRS. As the hardware engineers say, the >> root cause here is LS7A doesn't break up large read requests. In detail, >> LS7A PCIe port reports CA (Completer Abort) if it receives a Memory Read >> request with a size that's "too big" ("too big" means larger than the >> PCIe ports can handle, which means 256 for some ports and 4096 for the >> others, and of course this is a problem in the LS7A's hardware design). > > This seems essentially similar to ks_pcie_quirk() [1]. Why are they > different, and why do you need no_inc_mrrs, when keystone doesn't? > > Or *does* keystone need it and we just haven't figured that out yet? > Are all callers of pcie_set_readrq() vulnerable to issues there? Yes actually keystone may need to set this flag as well. I think Huacai missed a point in his commit message about why he removed the process of walking through the bus and set proper MRRS. That’s because Loongson’s firmware will set proper MRRS and the only thing that Kernel needs to do is leave it as is. no_inc_mrrs is introduced for this purpose. In keystone’s case it’s likely that their firmware won’t do such thing, so their workaround shouldn’t be removed. And no_inc_mrrs should be set for them to prevent device drivers modifying MRRS afterwards. Thanks - Jiaxun > > Whatever we do should be as uniform as possible across host > controllers. > > [1] > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pci-keystone.c?id=v5.18#n528 >
Hi, Bjorn, On Wed, Jun 1, 2022 at 8:00 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote: > > > > 在2022年6月1日六月 上午3:22，Bjorn Helgaas写道： > > On Sat, Apr 30, 2022 at 04:48:44PM +0800, Huacai Chen wrote: > >> In new revision of LS7A, some PCIe ports support larger value than 256, > >> but their maximum supported MRRS values are not detectable. Moreover, > >> the current loongson_mrrs_quirk() cannot avoid devices increasing its > >> MRRS after pci_enable_device(), and some devices (e.g. Realtek 8169) > >> will actually set a big value in its driver. So the only possible way > >> is configure MRRS of all devices in BIOS, and add a pci host bridge bit > >> flag (i.e., no_inc_mrrs) to stop the increasing MRRS operations. > >> > >> However, according to PCIe Spec, it is legal for an OS to program any > >> value for MRRS, and it is also legal for an endpoint to generate a Read > >> Request with any size up to its MRRS. As the hardware engineers say, the > >> root cause here is LS7A doesn't break up large read requests. In detail, > >> LS7A PCIe port reports CA (Completer Abort) if it receives a Memory Read > >> request with a size that's "too big" ("too big" means larger than the > >> PCIe ports can handle, which means 256 for some ports and 4096 for the > >> others, and of course this is a problem in the LS7A's hardware design). > > > > This seems essentially similar to ks_pcie_quirk() [1]. Why are they > > different, and why do you need no_inc_mrrs, when keystone doesn't? > > > > Or *does* keystone need it and we just haven't figured that out yet? > > Are all callers of pcie_set_readrq() vulnerable to issues there? > > Yes actually keystone may need to set this flag as well. > > I think Huacai missed a point in his commit message about why he removed > the process of walking through the bus and set proper MRRS. That’s > because Loongson’s firmware will set proper MRRS and the only thing > that Kernel needs to do is leave it as is. no_inc_mrrs is introduced for > this purpose. > > In keystone’s case it’s likely that their firmware won’t do such thing, so > their workaround shouldn’t be removed. > And no_inc_mrrs should be set for them to prevent device drivers modifying > MRRS afterwards. Yes, the fact is the same as Jiaxun says. Huacai > > Thanks > - Jiaxun > > > > > Whatever we do should be as uniform as possible across host > > controllers. > > > > [1] > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pci-keystone.c?id=v5.18#n528 > > > -- > - Jiaxun
[+cc Jingoo, Gustavo, Kishon, Krzysztof] On Wed, Jun 01, 2022 at 12:59:50PM +0100, Jiaxun Yang wrote: > 在2022年6月1日六月 上午3:22，Bjorn Helgaas写道： > > On Sat, Apr 30, 2022 at 04:48:44PM +0800, Huacai Chen wrote: > >> In new revision of LS7A, some PCIe ports support larger value than 256, > >> but their maximum supported MRRS values are not detectable. Moreover, > >> the current loongson_mrrs_quirk() cannot avoid devices increasing its > >> MRRS after pci_enable_device(), and some devices (e.g. Realtek 8169) > >> will actually set a big value in its driver. So the only possible way > >> is configure MRRS of all devices in BIOS, and add a pci host bridge bit > >> flag (i.e., no_inc_mrrs) to stop the increasing MRRS operations. > >> > >> However, according to PCIe Spec, it is legal for an OS to program any > >> value for MRRS, and it is also legal for an endpoint to generate a Read > >> Request with any size up to its MRRS. As the hardware engineers say, the > >> root cause here is LS7A doesn't break up large read requests. In detail, > >> LS7A PCIe port reports CA (Completer Abort) if it receives a Memory Read > >> request with a size that's "too big" ("too big" means larger than the > >> PCIe ports can handle, which means 256 for some ports and 4096 for the > >> others, and of course this is a problem in the LS7A's hardware design). > > > > This seems essentially similar to ks_pcie_quirk() [1]. Why are they > > different, and why do you need no_inc_mrrs, when keystone doesn't? > > > > Or *does* keystone need it and we just haven't figured that out yet? > > Are all callers of pcie_set_readrq() vulnerable to issues there? > > Yes actually keystone may need to set this flag as well. > > I think Huacai missed a point in his commit message about why he removed > the process of walking through the bus and set proper MRRS. That’s > because Loongson’s firmware will set proper MRRS and the only thing > that Kernel needs to do is leave it as is. no_inc_mrrs is introduced for > this purpose. I'd really like to have a single implementation of whatever quirk works around this. I don't think we should have multiple copies just because we assume some firmware takes care of part of this for us. > In keystone’s case it’s likely that their firmware won’t do such thing, so > their workaround shouldn’t be removed. > And no_inc_mrrs should be set for them to prevent device drivers modifying > MRRS afterwards. I have the vague impression that this issue is related to an arm64 AXI bus property [2] or maybe a DesignWare controller property [3], so this might affect several PCIe controller drivers. > > Whatever we do should be as uniform as possible across host > > controllers. > > > > [1] > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pci-keystone.c?id=v5.18#n528 [2] https://lore.kernel.org/all/20211126083119.16570-4-kishon@ti.com/ [3] https://lore.kernel.org/all/m3r1f08p83.fsf@t19.piap.pl/
Hi Bjorn et al., Bjorn Helgaas <helgaas@kernel.org> writes: > I'd really like to have a single implementation of whatever quirk > works around this. I don't think we should have multiple copies just > because we assume some firmware takes care of part of this for us. I second this. I think it should work this way: MPS affects the whole buses, i.e., packets are not fragmented by PCIe bridges. MPS works for both RX and TX. This means the CPU MPS (if any) must be enforced (set in the registers) over the whole bus (system). The system may use different (smaller) MPSes for different devices, though. Perhaps the user should be able to ask for smaller value (currently it's done using enum pcie_bus_config_types). MRRS can be larger than MPS (a single read causes multiple packets of response), and can be different for different devices. Still, all devices must be programmed the system's limit at most (or less if the user wishes to). IMHO this means we should use max_mps and max_mrrs for the whole system, and then e.g. platform PCIe controller driver or a device driver could lower them, triggering writes to the PCI config registers down the buses. Individual devices/drivers could use smaller values without changing the global variables. > I have the vague impression that this issue is related to an arm64 AXI > bus property [2] or maybe a DesignWare controller property [3], so > this might affect several PCIe controller drivers. [2] seems like a bug in TI specific SoC and revision only. [3] it seems all DWC PCIe hosts (and maybe devices) need a limit (two limits). E.g.: - i.MX6 needs MRRS = 512 (or lower at user's discretion) and MPS = 128. - CNS3xxx needs MRRS = MPS = 128 IIRC.
在2022年6月2日六月 下午5:20，Bjorn Helgaas写道：. > > I'd really like to have a single implementation of whatever quirk > works around this. I don't think we should have multiple copies just > because we assume some firmware takes care of part of this for us. > Yeah that was my idea when I was writing the present version of workaround. However in later LS7A revisions Loongson somehow raised MRRS for several PCIe controllers on chip to 1024 and other ports remains to be 256. Kernel have no way to aware of this change and we can only rely on firmware to set proper value. I have no idea how Loongson achieved this in hardware. All those PCIe controllers are attached under the same AXI bus should share the same AXI to HyperTransport bridge as AXI slave behind a bus matrix. Perhaps instead of fixing error handling of their AXI protocol implementation they just increased the buffer size in AXI bridge so it can accomplish larger requests at one time. >> In keystone’s case it’s likely that their firmware won’t do such thing, so >> their workaround shouldn’t be removed. >> And no_inc_mrrs should be set for them to prevent device drivers modifying >> MRRS afterwards. > > I have the vague impression that this issue is related to an arm64 AXI > bus property [2] or maybe a DesignWare controller property [3], so > this might affect several PCIe controller drivers. In my understanding it’s likely to be a AXI implementation issue. Thanks > >> > Whatever we do should be as uniform as possible across host >> > controllers. >> > >> > [1] >> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/dwc/pci-keystone.c?id=v5.18#n528 > > [2] https://lore.kernel.org/all/20211126083119.16570-4-kishon@ti.com/ > [3] https://lore.kernel.org/all/m3r1f08p83.fsf@t19.piap.pl/
On Fri, Jun 03, 2022 at 11:57:47PM +0100, Jiaxun Yang wrote: > 在2022年6月2日六月 下午5:20，Bjorn Helgaas写道：. > > > > I'd really like to have a single implementation of whatever quirk > > works around this. I don't think we should have multiple copies > > just because we assume some firmware takes care of part of this > > for us. > > > Yeah that was my idea when I was writing the present version of > workaround. However in later LS7A revisions Loongson somehow raised > MRRS for several PCIe controllers on chip to 1024 and other ports > remains to be 256. Kernel have no way to aware of this change and we > can only rely on firmware to set proper value. That's fine; we need a controller-specific way to find the limit (whether it's fixed for all versions or discovered from firmware settings or whatever). My hope is that given that controller-specific value, we can have a single quirk that works on keystone, loongson, etc. to enforce the limit on all relevant devices. Some platform firmware might do that configuration already, but it's OK if a generic quirk re-does it. I don't think it's worth having two quirks, one that does the configuration, and another that relies on firmware having done it. > I have no idea how Loongson achieved this in hardware. All those > PCIe controllers are attached under the same AXI bus should share > the same AXI to HyperTransport bridge as AXI slave behind a bus > matrix. Perhaps instead of fixing error handling of their AXI > protocol implementation they just increased the buffer size in AXI > bridge so it can accomplish larger requests at one time. > >> In keystone’s case it’s likely that their firmware won’t do such thing, so > >> their workaround shouldn’t be removed. > >> And no_inc_mrrs should be set for them to prevent device drivers modifying > >> MRRS afterwards. > > > > I have the vague impression that this issue is related to an arm64 AXI > > bus property [2] or maybe a DesignWare controller property [3], so > > this might affect several PCIe controller drivers. > > In my understanding it’s likely to be a AXI implementation issue. I know almost nothing about AXI, but this concerns me because it sounds like other drivers could be affected. Bjorn
Hi, Bjorn, Jiaxun, On Sat, Jun 4, 2022 at 8:07 AM Bjorn Helgaas <helgaas@kernel.org> wrote: > > On Fri, Jun 03, 2022 at 11:57:47PM +0100, Jiaxun Yang wrote: > > 在2022年6月2日六月 下午5:20，Bjorn Helgaas写道：. > > > > > > I'd really like to have a single implementation of whatever quirk > > > works around this. I don't think we should have multiple copies > > > just because we assume some firmware takes care of part of this > > > for us. > > > > > Yeah that was my idea when I was writing the present version of > > workaround. However in later LS7A revisions Loongson somehow raised > > MRRS for several PCIe controllers on chip to 1024 and other ports > > remains to be 256. Kernel have no way to aware of this change and we > > can only rely on firmware to set proper value. > > That's fine; we need a controller-specific way to find the limit > (whether it's fixed for all versions or discovered from firmware > settings or whatever). > > My hope is that given that controller-specific value, we can have a > single quirk that works on keystone, loongson, etc. to enforce the > limit on all relevant devices. Some platform firmware might do that > configuration already, but it's OK if a generic quirk re-does it. > > I don't think it's worth having two quirks, one that does the > configuration, and another that relies on firmware having done it. I think it is better to let keystone and loongson to both use the no_inc_mrrs quirk. Huacai > > > I have no idea how Loongson achieved this in hardware. All those > > PCIe controllers are attached under the same AXI bus should share > > the same AXI to HyperTransport bridge as AXI slave behind a bus > > matrix. Perhaps instead of fixing error handling of their AXI > > protocol implementation they just increased the buffer size in AXI > > bridge so it can accomplish larger requests at one time. > > > >> In keystone’s case it’s likely that their firmware won’t do such thing, so > > >> their workaround shouldn’t be removed. > > >> And no_inc_mrrs should be set for them to prevent device drivers modifying > > >> MRRS afterwards. > > > > > > I have the vague impression that this issue is related to an arm64 AXI > > > bus property [2] or maybe a DesignWare controller property [3], so > > > this might affect several PCIe controller drivers. > > > > In my understanding it’s likely to be a AXI implementation issue. > > I know almost nothing about AXI, but this concerns me because it > sounds like other drivers could be affected. > > Bjorn
diff --git a/drivers/pci/controller/pci-loongson.c b/drivers/pci/controller/pci-loongson.c index 48316daa1f23..83447264048a 100644 --- a/drivers/pci/controller/pci-loongson.c +++ b/drivers/pci/controller/pci-loongson.c @@ -67,37 +67,23 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, DEV_LS7A_LPC, system_bus_quirk); -static void loongson_mrrs_quirk(struct pci_dev *dev) +static void loongson_mrrs_quirk(struct pci_dev *pdev) { - struct pci_bus *bus = dev->bus; - struct pci_dev *bridge; - static const struct pci_device_id bridge_devids[] = { - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_0) }, - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_1) }, - { PCI_VDEVICE(LOONGSON, DEV_PCIE_PORT_2) }, - { 0, }, - }; - - /* look for the matching bridge */ - while (!pci_is_root_bus(bus)) { - bridge = bus->self; - bus = bus->parent; - /* - * Some Loongson PCIe ports have a h/w limitation of - * 256 bytes maximum read request size. They can't handle - * anything larger than this. So force this limit on - * any devices attached under these ports. - */ - if (pci_match_id(bridge_devids, bridge)) { - if (pcie_get_readrq(dev) > 256) { - pci_info(dev, "limiting MRRS to 256\n"); - pcie_set_readrq(dev, 256); - } - break; - } - } + /* + * Some Loongson PCIe ports have h/w limitations of maximum read + * request size. They can't handle anything larger than this. So + * force this limit on any devices attached under these ports. + */ + struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus); + + bridge->no_inc_mrrs = 1; } -DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, loongson_mrrs_quirk); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, + DEV_PCIE_PORT_0, loongson_mrrs_quirk); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, + DEV_PCIE_PORT_1, loongson_mrrs_quirk); +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_LOONGSON, + DEV_PCIE_PORT_2, loongson_mrrs_quirk); static struct loongson_pci *pci_bus_to_loongson_pci(struct pci_bus *bus) { diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 9ecce435fb3f..72a15bf9eee8 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5993,6 +5993,7 @@ int pcie_set_readrq(struct pci_dev *dev, int rq) { u16 v; int ret; + struct pci_host_bridge *bridge = pci_find_host_bridge(dev->bus); if (rq < 128 || rq > 4096 || !is_power_of_2(rq)) return -EINVAL; @@ -6011,6 +6012,11 @@ int pcie_set_readrq(struct pci_dev *dev, int rq) v = (ffs(rq) - 8) << 12; + if (bridge->no_inc_mrrs) { + if (rq > pcie_get_readrq(dev)) + return -EINVAL; + } + ret = pcie_capability_clear_and_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_DEVCTL_READRQ, v); diff --git a/include/linux/pci.h b/include/linux/pci.h index 60adf42460ab..d146eb28e6da 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -569,6 +569,7 @@ struct pci_host_bridge { void *release_data; unsigned int ignore_reset_delay:1; /* For entire hierarchy */ unsigned int no_ext_tags:1; /* No Extended Tags */ + unsigned int no_inc_mrrs:1; /* No Increase MRRS */ unsigned int native_aer:1; /* OS may use PCIe AER */ unsigned int native_pcie_hotplug:1; /* OS may use PCIe hotplug */ unsigned int native_shpc_hotplug:1; /* OS may use SHPC hotplug */
In new revision of LS7A, some PCIe ports support larger value than 256, but their maximum supported MRRS values are not detectable. Moreover, the current loongson_mrrs_quirk() cannot avoid devices increasing its MRRS after pci_enable_device(), and some devices (e.g. Realtek 8169) will actually set a big value in its driver. So the only possible way is configure MRRS of all devices in BIOS, and add a pci host bridge bit flag (i.e., no_inc_mrrs) to stop the increasing MRRS operations. However, according to PCIe Spec, it is legal for an OS to program any value for MRRS, and it is also legal for an endpoint to generate a Read Request with any size up to its MRRS. As the hardware engineers say, the root cause here is LS7A doesn't break up large read requests. In detail, LS7A PCIe port reports CA (Completer Abort) if it receives a Memory Read request with a size that's "too big" ("too big" means larger than the PCIe ports can handle, which means 256 for some ports and 4096 for the others, and of course this is a problem in the LS7A's hardware design). Signed-off-by: Huacai Chen <chenhuacai@loongson.cn> --- drivers/pci/controller/pci-loongson.c | 44 +++++++++------------------ drivers/pci/pci.c | 6 ++++ include/linux/pci.h | 1 + 3 files changed, 22 insertions(+), 29 deletions(-)