| Message ID | 20210611214823.4898-1-mikel@mikelr.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] PCI: Add quirk for 64-bit DMA on RS690 chipset | expand |
On Fri, Jun 11, 2021 at 05:48:23PM -0400, Mikel Rychliski wrote: > Although the AMD RS690 chipset has 64-bit DMA support, BIOS implementations > sometimes fail to configure the memory limit registers correctly. > > The Acer F690GVM mainboard uses this chipset and a Marvell 88E8056 NIC. The > sky2 driver programs the NIC to use 64-bit DMA, which will not work: > > sky2 0000:02:00.0: error interrupt status=0x8 > sky2 0000:02:00.0 eth0: tx timeout > sky2 0000:02:00.0 eth0: transmit ring 0 .. 22 report=0 done=0 > > Other drivers required by this mainboard either don't support 64-bit DMA, > or have it disabled using driver specific quirks. For example, the ahci > driver has quirks to enable or disable 64-bit DMA depending on the BIOS > version (see ahci_sb600_enable_64bit() in ahci.c). This ahci quirk matches > against the SB600 SATA controller, but the real issue is almost certainly > with the RS690 PCI host that it was commonly attached to. > > To avoid this issue in all drivers with 64-bit DMA support, fix the > configuration of the PCI host. If the kernel is aware of physical memory > above 4GB, but the BIOS never configured the PCI host with this > information, update the registers with our values. > > Signed-off-by: Mikel Rychliski <mikel@mikelr.com> Applied to for-linus for v5.13, thanks! I replaced PCI_DEVICE_ID_ATI_RS690 with 0x7910 because we don't usually add those #defines when they're only used in one place. > --- > arch/x86/pci/fixup.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ > include/linux/pci_ids.h | 1 + > 2 files changed, 46 insertions(+) > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > index 02dc64625e64..4cc479f332ba 100644 > --- a/arch/x86/pci/fixup.c > +++ b/arch/x86/pci/fixup.c > @@ -779,4 +779,49 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, pci_amd_enable_64bit_bar); > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, pci_amd_enable_64bit_bar); > DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, pci_amd_enable_64bit_bar); > > +#define RS690_LOWER_TOP_OF_DRAM2 0x30 > +#define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 > +#define RS690_UPPER_TOP_OF_DRAM2 0x31 > +#define RS690_HTIU_NB_INDEX 0xA8 > +#define RS690_HTIU_NB_INDEX_WR_ENABLE 0x100 > +#define RS690_HTIU_NB_DATA 0xAC > + > +/* > + * Some BIOS implementations support RAM above 4GB, but do not configure the > + * PCI host to respond to bus master accesses for these addresses. These > + * implementations set the TOP_OF_DRAM_SLOT1 register correctly, so PCI DMA > + * works as expected for addresses below 4GB. > + * > + * Reference: "AMD RS690 ASIC Family Register Reference Guide" (pg. 2-57) > + * https://www.amd.com/system/files/TechDocs/43372_rs690_rrg_3.00o.pdf > + */ > +static void rs690_fix_64bit_dma(struct pci_dev *pdev) > +{ > + u32 val = 0; > + phys_addr_t top_of_dram = __pa(high_memory - 1) + 1; > + > + if (top_of_dram <= (1ULL << 32)) > + return; > + > + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, > + RS690_LOWER_TOP_OF_DRAM2); > + pci_read_config_dword(pdev, RS690_HTIU_NB_DATA, &val); > + > + if (val) > + return; > + > + pci_info(pdev, "Adjusting top of DRAM to %pa for 64-bit DMA support\n", &top_of_dram); > + > + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, > + RS690_UPPER_TOP_OF_DRAM2 | RS690_HTIU_NB_INDEX_WR_ENABLE); > + pci_write_config_dword(pdev, RS690_HTIU_NB_DATA, top_of_dram >> 32); > + > + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, > + RS690_LOWER_TOP_OF_DRAM2 | RS690_HTIU_NB_INDEX_WR_ENABLE); > + pci_write_config_dword(pdev, RS690_HTIU_NB_DATA, > + top_of_dram | RS690_LOWER_TOP_OF_DRAM2_VALID); > +} > +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS690, > + rs690_fix_64bit_dma); > + > #endif > diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h > index 4c3fa5293d76..0a7fe2ed520b 100644 > --- a/include/linux/pci_ids.h > +++ b/include/linux/pci_ids.h > @@ -381,6 +381,7 @@ > #define PCI_DEVICE_ID_ATI_RS400_166 0x5a32 > #define PCI_DEVICE_ID_ATI_RS400_200 0x5a33 > #define PCI_DEVICE_ID_ATI_RS480 0x5950 > +#define PCI_DEVICE_ID_ATI_RS690 0x7910 > /* ATI IXP Chipset */ > #define PCI_DEVICE_ID_ATI_IXP200_IDE 0x4349 > #define PCI_DEVICE_ID_ATI_IXP200_SMBUS 0x4353 > -- > 2.13.7 >
diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index 02dc64625e64..4cc479f332ba 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -779,4 +779,49 @@ DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1571, pci_amd_enable_64bit_bar); DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x15b1, pci_amd_enable_64bit_bar); DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_AMD, 0x1601, pci_amd_enable_64bit_bar); +#define RS690_LOWER_TOP_OF_DRAM2 0x30 +#define RS690_LOWER_TOP_OF_DRAM2_VALID 0x1 +#define RS690_UPPER_TOP_OF_DRAM2 0x31 +#define RS690_HTIU_NB_INDEX 0xA8 +#define RS690_HTIU_NB_INDEX_WR_ENABLE 0x100 +#define RS690_HTIU_NB_DATA 0xAC + +/* + * Some BIOS implementations support RAM above 4GB, but do not configure the + * PCI host to respond to bus master accesses for these addresses. These + * implementations set the TOP_OF_DRAM_SLOT1 register correctly, so PCI DMA + * works as expected for addresses below 4GB. + * + * Reference: "AMD RS690 ASIC Family Register Reference Guide" (pg. 2-57) + * https://www.amd.com/system/files/TechDocs/43372_rs690_rrg_3.00o.pdf + */ +static void rs690_fix_64bit_dma(struct pci_dev *pdev) +{ + u32 val = 0; + phys_addr_t top_of_dram = __pa(high_memory - 1) + 1; + + if (top_of_dram <= (1ULL << 32)) + return; + + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, + RS690_LOWER_TOP_OF_DRAM2); + pci_read_config_dword(pdev, RS690_HTIU_NB_DATA, &val); + + if (val) + return; + + pci_info(pdev, "Adjusting top of DRAM to %pa for 64-bit DMA support\n", &top_of_dram); + + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, + RS690_UPPER_TOP_OF_DRAM2 | RS690_HTIU_NB_INDEX_WR_ENABLE); + pci_write_config_dword(pdev, RS690_HTIU_NB_DATA, top_of_dram >> 32); + + pci_write_config_dword(pdev, RS690_HTIU_NB_INDEX, + RS690_LOWER_TOP_OF_DRAM2 | RS690_HTIU_NB_INDEX_WR_ENABLE); + pci_write_config_dword(pdev, RS690_HTIU_NB_DATA, + top_of_dram | RS690_LOWER_TOP_OF_DRAM2_VALID); +} +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RS690, + rs690_fix_64bit_dma); + #endif diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index 4c3fa5293d76..0a7fe2ed520b 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -381,6 +381,7 @@ #define PCI_DEVICE_ID_ATI_RS400_166 0x5a32 #define PCI_DEVICE_ID_ATI_RS400_200 0x5a33 #define PCI_DEVICE_ID_ATI_RS480 0x5950 +#define PCI_DEVICE_ID_ATI_RS690 0x7910 /* ATI IXP Chipset */ #define PCI_DEVICE_ID_ATI_IXP200_IDE 0x4349 #define PCI_DEVICE_ID_ATI_IXP200_SMBUS 0x4353
Although the AMD RS690 chipset has 64-bit DMA support, BIOS implementations sometimes fail to configure the memory limit registers correctly. The Acer F690GVM mainboard uses this chipset and a Marvell 88E8056 NIC. The sky2 driver programs the NIC to use 64-bit DMA, which will not work: sky2 0000:02:00.0: error interrupt status=0x8 sky2 0000:02:00.0 eth0: tx timeout sky2 0000:02:00.0 eth0: transmit ring 0 .. 22 report=0 done=0 Other drivers required by this mainboard either don't support 64-bit DMA, or have it disabled using driver specific quirks. For example, the ahci driver has quirks to enable or disable 64-bit DMA depending on the BIOS version (see ahci_sb600_enable_64bit() in ahci.c). This ahci quirk matches against the SB600 SATA controller, but the real issue is almost certainly with the RS690 PCI host that it was commonly attached to. To avoid this issue in all drivers with 64-bit DMA support, fix the configuration of the PCI host. If the kernel is aware of physical memory above 4GB, but the BIOS never configured the PCI host with this information, update the registers with our values. Signed-off-by: Mikel Rychliski <mikel@mikelr.com> --- arch/x86/pci/fixup.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci_ids.h | 1 + 2 files changed, 46 insertions(+)