| Message ID | 20140521231817.26447.55150.stgit@bhelgaas-glaptop.roam.corp.google.com |
|---|---|
| State | Rejected |
| Headers | show |
On Wed, May 21, 2014 at 05:18:17PM -0600, Bjorn Helgaas wrote: > ECS is an AMD mechanism that allows access to extended PCI config space > (offsets 256-4096) via I/O ports CF8/CFCh. We normally use ECAM, i.e., > MMCONFIG, to access that space, but apparently old machines have issues > that meant we couldn't use ECAM. > > The solution was to enable ECS so we could use configuration mechanism #1 > (I/O ports CF8/CFCh) to access extended config space. See 831d991821da > ("x86: add PCI extended config space access for AMD Barcelona"). > > New machines should be able to use ECAM, which means they don't need the > CPU-specific code to enable ECS. This patch leaves ECS the same on all > existing platforms, but stops enabling it on platforms after Fam16h. > > Those future platforms should be able to use the standard ACPI MCFG/_CBA > descriptions of the PCIe ECAM mechanism. > > Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> > CC: Robert Richter <rric@kernel.org> > --- > arch/x86/pci/amd_bus.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/arch/x86/pci/amd_bus.c b/arch/x86/pci/amd_bus.c > index aa936e3a2019..67dadf179348 100644 > --- a/arch/x86/pci/amd_bus.c > +++ b/arch/x86/pci/amd_bus.c > @@ -405,7 +405,9 @@ static int __init amd_postcore_init(void) > return 0; > > early_fill_mp_bus_info(); > - pci_io_ecs_init(); > + > + if (boot_cpu_data.x86 <= 0x16) Fam 0x16, i.e. hex? I think we talked about fam 0x10, i.e. 16 decimal. Oh well, that's an AMD call now.
On Wed, May 21, 2014 at 5:38 PM, Borislav Petkov <bp@alien8.de> wrote: > On Wed, May 21, 2014 at 05:18:17PM -0600, Bjorn Helgaas wrote: >> ECS is an AMD mechanism that allows access to extended PCI config space >> (offsets 256-4096) via I/O ports CF8/CFCh. We normally use ECAM, i.e., >> MMCONFIG, to access that space, but apparently old machines have issues >> that meant we couldn't use ECAM. >> >> The solution was to enable ECS so we could use configuration mechanism #1 >> (I/O ports CF8/CFCh) to access extended config space. See 831d991821da >> ("x86: add PCI extended config space access for AMD Barcelona"). >> >> New machines should be able to use ECAM, which means they don't need the >> CPU-specific code to enable ECS. This patch leaves ECS the same on all >> existing platforms, but stops enabling it on platforms after Fam16h. >> >> Those future platforms should be able to use the standard ACPI MCFG/_CBA >> descriptions of the PCIe ECAM mechanism. >> >> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> >> CC: Robert Richter <rric@kernel.org> >> --- >> arch/x86/pci/amd_bus.c | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/arch/x86/pci/amd_bus.c b/arch/x86/pci/amd_bus.c >> index aa936e3a2019..67dadf179348 100644 >> --- a/arch/x86/pci/amd_bus.c >> +++ b/arch/x86/pci/amd_bus.c >> @@ -405,7 +405,9 @@ static int __init amd_postcore_init(void) >> return 0; >> >> early_fill_mp_bus_info(); >> - pci_io_ecs_init(); >> + >> + if (boot_cpu_data.x86 <= 0x16) > > Fam 0x16, i.e. hex? I think we talked about fam 0x10, i.e. 16 decimal. > > Oh well, that's an AMD call now. I chose Fam16h (0x16) because it looks like that's the newest stuff that's in the field. I suspect things would probably work if we changed this patch to leave ECS disabled on some Fam16h, Fam15h, etc., but that would change behavior on existing systems, which obviously adds some risk. I didn't think there was much benefit that makes the risk worthwhile. My goal is to stop needing CPU-specific changes in the future, not necessarily to remove the CPU-specific code we already have. Does that make sense? I'm not sure whether I understood your real question. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, May 22, 2014 at 11:56:03AM -0600, Bjorn Helgaas wrote: > I chose Fam16h (0x16) because it looks like that's the newest stuff > that's in the field. I suspect things would probably work if we > changed this patch to leave ECS disabled on some Fam16h, Fam15h, etc., > but that would change behavior on existing systems, which obviously > adds some risk. I didn't think there was much benefit that makes the > risk worthwhile. > > My goal is to stop needing CPU-specific changes in the future, not > necessarily to remove the CPU-specific code we already have. > > Does that make sense? I'm not sure whether I understood your real > question. No, you got it right. I'm just wondering why only the newest stuff. MMCONFIG is supposed to work just fine on everything from Fam10h onwards, I'm not sure all Fam10h supported it though. Maybe Suravee can verify that...
On Thu, May 22, 2014 at 1:17 PM, Borislav Petkov <bp@alien8.de> wrote: > On Thu, May 22, 2014 at 11:56:03AM -0600, Bjorn Helgaas wrote: >> I chose Fam16h (0x16) because it looks like that's the newest stuff >> that's in the field. I suspect things would probably work if we >> changed this patch to leave ECS disabled on some Fam16h, Fam15h, etc., >> but that would change behavior on existing systems, which obviously >> adds some risk. I didn't think there was much benefit that makes the >> risk worthwhile. >> >> My goal is to stop needing CPU-specific changes in the future, not >> necessarily to remove the CPU-specific code we already have. >> >> Does that make sense? I'm not sure whether I understood your real >> question. > > No, you got it right. I'm just wondering why only the newest stuff. > MMCONFIG is supposed to work just fine on everything from Fam10h > onwards, I'm not sure all Fam10h supported it though. Maybe Suravee can > verify that... Even if MMCONFIG does work fine on everything from Fam10h onwards, we still depend on the BIOS to provide a correct MCFG table. I don't think we can guarantee that changing from ECS to MMCONFIG on a Fam16h box in the field is safe, because we'd then be using a feature we've never used before. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, May 22, 2014 at 02:20:04PM -0600, Bjorn Helgaas wrote: > Even if MMCONFIG does work fine on everything from Fam10h onwards, we > still depend on the BIOS to provide a correct MCFG table. It is very hazy memory of mine that starting with F10h, we had an MCFG table always present but I don't remember anymore. Maybe I'm even remembering it wrong. In any case, as Andreas and I pointed out, this is AMD's decision so if they're fine with F16h, then it is definitely not my place to disagree. :-)
On 5/22/2014 3:20 PM, Bjorn Helgaas wrote: > On Thu, May 22, 2014 at 1:17 PM, Borislav Petkov <bp@alien8.de> wrote: >> On Thu, May 22, 2014 at 11:56:03AM -0600, Bjorn Helgaas wrote: >>> I chose Fam16h (0x16) because it looks like that's the newest stuff >>> that's in the field. I suspect things would probably work if we >>> changed this patch to leave ECS disabled on some Fam16h, Fam15h, etc., >>> but that would change behavior on existing systems, which obviously >>> adds some risk. I didn't think there was much benefit that makes the >>> risk worthwhile. >>> >>> My goal is to stop needing CPU-specific changes in the future, not >>> necessarily to remove the CPU-specific code we already have. >>> >>> Does that make sense? I'm not sure whether I understood your real >>> question. >> >> No, you got it right. I'm just wondering why only the newest stuff. >> MMCONFIG is supposed to work just fine on everything from Fam10h >> onwards, I'm not sure all Fam10h supported it though. Maybe Suravee can >> verify that... > > Even if MMCONFIG does work fine on everything from Fam10h onwards, we > still depend on the BIOS to provide a correct MCFG table. I don't > think we can guarantee that changing from ECS to MMCONFIG on a Fam16h > box in the field is safe, because we'd then be using a feature we've > never used before. > > Bjorn > At this point, family11h and later (upto 16h which is our most current processor) should already have supports for the MCFG. However, we can't guarantee that all the systems currently out there would not use the ECS. So, I think it is ok to say we won't support it post 16h as Bjorn suggests. Suravee -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, May 22, 2014 at 5:39 PM, Suravee Suthikulanit <suravee.suthikulpanit@amd.com> wrote: > On 5/22/2014 3:20 PM, Bjorn Helgaas wrote: >> On Thu, May 22, 2014 at 1:17 PM, Borislav Petkov <bp@alien8.de> wrote: >>> On Thu, May 22, 2014 at 11:56:03AM -0600, Bjorn Helgaas wrote: >>>> >>>> I chose Fam16h (0x16) because it looks like that's the newest stuff >>>> that's in the field. I suspect things would probably work if we >>>> changed this patch to leave ECS disabled on some Fam16h, Fam15h, etc., >>>> but that would change behavior on existing systems, which obviously >>>> adds some risk. I didn't think there was much benefit that makes the >>>> risk worthwhile. >>>> >>>> My goal is to stop needing CPU-specific changes in the future, not >>>> necessarily to remove the CPU-specific code we already have. >>>> >>>> Does that make sense? I'm not sure whether I understood your real >>>> question. >>> >>> No, you got it right. I'm just wondering why only the newest stuff. >>> MMCONFIG is supposed to work just fine on everything from Fam10h >>> onwards, I'm not sure all Fam10h supported it though. Maybe Suravee can >>> verify that... >> >> Even if MMCONFIG does work fine on everything from Fam10h onwards, we >> still depend on the BIOS to provide a correct MCFG table. I don't >> think we can guarantee that changing from ECS to MMCONFIG on a Fam16h >> box in the field is safe, because we'd then be using a feature we've >> never used before. > > At this point, family11h and later (upto 16h which is our most current > processor) should already have supports for the MCFG. However, we can't > guarantee that all the systems currently out there would not use the ECS. > So, I think it is ok to say we won't support it post 16h as Bjorn suggests. I think this is more a BIOS question than a hardware question. I'm sure all current AMD hardware supports ECAM just fine. But it's still up to the BIOS to produce a valid MCFG table, so OEMs could still have issues, and *that's* what I'm worried about. I've been poking around for recent dmesg logs that contain "PCI: Using configuration type 1 for extended access", and there are quite a few. In most cases there *is* an MCFG table, but apparently we decide not to use it for some reason (unfortunately we don't print the specific reason). One example is at https://bugzilla.kernel.org/show_bug.cgi?id=68591 . I'm going to go out on a limb and guess that Windows does not enable ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing of MCFG is broken in some way, and we probably *could* use ECAM in all these cases I'm seeing. It would probably be prudent to figure out why Linux is rejecting these MCFG tables. We'll probably see similar tables on Fam17h systems, and if we continue rejecting them, and we don't turn on ECS, we won't be able to access extended config space. I opened a bugzilla for this issue: https://bugzilla.kernel.org/show_bug.cgi?id=76771 I'm wavering on whether it's a good idea to put this patch in before understanding the issue. As much as I'd like to stop fiddling with ECS, we'd likely end up with a v3.15 where extended config space doesn't work on some Fam17h systems. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 22.05.14 20:54:54, Bjorn Helgaas wrote: > I'm going to go out on a limb and guess that Windows does not enable > ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing > of MCFG is broken in some way, and we probably *could* use ECAM in all > these cases I'm seeing. Even if ECS is not enabled the system should be fine anyway, as ECS is only used to enable certain features. For family 10h this was originally the IBS EILVT (extended interrupt local vector table, needed for hw profiling) setup which need to be set by the OS which the BIOS didn't right. This should be fixed now and properly set by the BIOS on 15h+ systems. I don't remember what was added to 16h where ECS was needed, I think there was one (Suravee?). Not sure if this is essential. So using MCFG should be fine, since if in the rare case when it is broken, the system should work properly anyway without it and only some special features, if any, do not work anymore (e.g. IBS should work fine). -Robert -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, May 23, 2014 at 5:56 AM, Robert Richter <rric@kernel.org> wrote: > On 22.05.14 20:54:54, Bjorn Helgaas wrote: >> I'm going to go out on a limb and guess that Windows does not enable >> ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing >> of MCFG is broken in some way, and we probably *could* use ECAM in all >> these cases I'm seeing. > > Even if ECS is not enabled the system should be fine anyway, as ECS is > only used to enable certain features. For family 10h this was > originally the IBS EILVT (extended interrupt local vector table, > needed for hw profiling) setup which need to be set by the OS which > the BIOS didn't right. This should be fixed now and properly set by > the BIOS on 15h+ systems. > > I don't remember what was added to 16h where ECS was needed, I think > there was one (Suravee?). Not sure if this is essential. > > So using MCFG should be fine, since if in the rare case when it is > broken, the system should work properly anyway without it and only > some special features, if any, do not work anymore (e.g. IBS should > work fine). [I guess I've been using the wrong term here. I think "ECS" just refers to the extended config space itself, and I should have been saying "IO ECS" or "EnableCf8ExtCfg".] My understanding was that if we don't enable IO ECS and we don't have MCFG, we will not be able to access extended config space. The system can certainly boot without extended config space, but some drivers may not work correctly, so it would be a regression from the user point of view. The case where MCFG is broken may be rare, but I still want to avoid the regression. And my guess is that the MCFG is not really broken in these cases, because I suspect Windows is using it. Our MCFG parsing code is a mess, and I suspect that we are claiming MCFG is broken when it really isn't. If I understand correctly, IO ECS is also limited to domain 0, so if we have multiple PCI host bridges, i.e., multiple PNP0A03/PNP0A08 devices, and we want them in different domains, we'd have to use ECAM. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 23.05.14 07:01:41, Bjorn Helgaas wrote: > [I guess I've been using the wrong term here. I think "ECS" just > refers to the extended config space itself, and I should have been > saying "IO ECS" or "EnableCf8ExtCfg".] > > My understanding was that if we don't enable IO ECS and we don't have > MCFG, we will not be able to access extended config space. The system > can certainly boot without extended config space, but some drivers may > not work correctly, so it would be a regression from the user point of > view. No, I got you right. If we disable IO ECS there is no fallback if MCFG fails... and this may cause a regression then. I might be completely wrong here, but as I remember IO ECS only affects access to cpu devices (bus 0, slot 0x18-0x1f). Thus, esp. PCIe extended config space access works since a different host controller handles this. So only cpu devices would see a regression and thus cpu bringup code. I don't think ECS (either MCFG or IO ECS) is needed anymore for cpu bringup (this is true for IBS, but I don't know of other cpu features requiring ECS, though family 16h might introduced new ones). IMHO device drivers are not affected and wont get broken. -Robert -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 5/23/2014 6:56 AM, Robert Richter wrote: > On 22.05.14 20:54:54, Bjorn Helgaas wrote: >> I'm going to go out on a limb and guess that Windows does not enable >> ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing >> of MCFG is broken in some way, and we probably *could* use ECAM in all >> these cases I'm seeing. > > Even if ECS is not enabled the system should be fine anyway, as ECS is > only used to enable certain features. For family 10h this was > originally the IBS EILVT (extended interrupt local vector table, > needed for hw profiling) setup which need to be set by the OS which > the BIOS didn't right. This should be fixed now and properly set by > the BIOS on 15h+ systems. > > I don't remember what was added to 16h where ECS was needed, I think > there was one (Suravee?). Not sure if this is essential. I am not aware of anything specific in the family16h which require IO_ECS to be enabled. Suravee -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 5/22/2014 9:54 PM, Bjorn Helgaas wrote: > I've been poking around for recent dmesg logs that contain "PCI: Using > configuration type 1 for extended access", and there are quite a few. > In most cases there*is* an MCFG table, but apparently we decide not > to use it for some reason (unfortunately we don't print the specific > reason). One example is at > https://bugzilla.kernel.org/show_bug.cgi?id=68591 . > > I'm going to go out on a limb and guess that Windows does not enable > ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing > of MCFG is broken in some way, and we probably*could* use ECAM in all > these cases I'm seeing. > > It would probably be prudent to figure out why Linux is rejecting > these MCFG tables. We'll probably see similar tables on Fam17h > systems, and if we continue rejecting them, and we don't turn on ECS, > we won't be able to access extended config space. > > I opened a bugzilla for this issue: > https://bugzilla.kernel.org/show_bug.cgi?id=76771 > > I'm wavering on whether it's a good idea to put this patch in before > understanding the issue. As much as I'd like to stop fiddling with > ECS, we'd likely end up with a v3.15 where extended config space > doesn't work on some Fam17h systems. So, I have located a system which presents issue with MMCONFIG. Here is my investigation: DEBUG: pci_io_ecs_init: pci_probe = 4000f ACPI: bus type PCI registered DEBUG: -----> pci_mmcfg_early_init DEBUG: pci_parse_mcfg PCI: MMCONFIG for domain 0000 [bus 00-01] at [mem 0xe0000000-0xe01fffff] (base 0xe0000000) DEBUG: pci_mmcfg_check_reserved DEBUG: is_mmconf_reserved: method = E820 PCI: not using MMCONFIG DEBUG: pci_direct_init PCI: Using configuration type 1 for base access PCI: Using configuration type 1 for extended access ACPI: Added _OSI(Module Device) ACPI: Added _OSI(Processor Device) ACPI: Added _OSI(3.0 _SCP Extensions) ACPI: Added _OSI(Processor Aggregator Device) [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored \_SB_:_OSC invalid UUID _OSC request data:1 1f ACPI: Interpreter enabled ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20140214/hwxface-580) ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20140214/hwxface-580) ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S4_] (20140214/hwxface-580) ACPI: (supports S0 S3 S5) ACPI: Using IOAPIC for interrupt routing DEBUG: ----> pci_mmcfg_late_init DEBUG: pci_parse_mcfg PCI: MMCONFIG for domain 0000 [bus 00-01] at [mem 0xe0000000-0xe01fffff] (base 0xe0000000) DEBUG: pci_mmcfg_check_reserved DEBUG: is_mmconf_reserved: method = ACPI motherboard resources PCI: MMCONFIG at [mem 0xe0000000-0xe01fffff] reserved in ACPI motherboard resources During pci_mmcfg_early_init(), the MMCONFIG failed because the range 0xe0000000 is not showing as reserved in the E820 mapping. Here is the snippet of E820 mapping from the system: ........ BIOS-e820: [mem 0x00000000c7eb0000-0x00000000c7ec0fff] ACPI data BIOS-e820: [mem 0x00000000c7ec1000-0x00000000c7ec2fff] ACPI NVS BIOS-e820: [mem 0x00000000c7ec3000-0x00000000c7efefff] reserved BIOS-e820: [mem 0x00000000c7f00000-0x00000000c7ffffff] reserved BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved However, during pci_mmcfg_late_init(), the area is reserved in the "ACPI motherboard resources", and the pci_mmcfg_check_reserved() does not fail here. But this is too late since we already setup the "raw_pci_ext_ops" in the "arch/x86/pci/direct.c: pci_direct_init()" (during to use the IO_ECS. Suravee -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, May 23, 2014 at 6:31 PM, Suravee Suthikulanit <suravee.suthikulpanit@amd.com> wrote: > On 5/22/2014 9:54 PM, Bjorn Helgaas wrote: >> >> I've been poking around for recent dmesg logs that contain "PCI: Using >> configuration type 1 for extended access", and there are quite a few. >> In most cases there*is* an MCFG table, but apparently we decide not >> >> to use it for some reason (unfortunately we don't print the specific >> reason). One example is at >> https://bugzilla.kernel.org/show_bug.cgi?id=68591 . >> >> I'm going to go out on a limb and guess that Windows does not enable >> ECS, so it probably uses ECAM. Therefore, I suspect Linux's parsing >> of MCFG is broken in some way, and we probably*could* use ECAM in all >> >> these cases I'm seeing. >> >> It would probably be prudent to figure out why Linux is rejecting >> these MCFG tables. We'll probably see similar tables on Fam17h >> systems, and if we continue rejecting them, and we don't turn on ECS, >> we won't be able to access extended config space. >> >> I opened a bugzilla for this issue: >> https://bugzilla.kernel.org/show_bug.cgi?id=76771 >> >> I'm wavering on whether it's a good idea to put this patch in before >> understanding the issue. As much as I'd like to stop fiddling with >> ECS, we'd likely end up with a v3.15 where extended config space >> doesn't work on some Fam17h systems. > > > So, I have located a system which presents issue with MMCONFIG. Here is my > investigation: > > DEBUG: pci_io_ecs_init: pci_probe = 4000f > ACPI: bus type PCI registered > DEBUG: -----> pci_mmcfg_early_init > DEBUG: pci_parse_mcfg > PCI: MMCONFIG for domain 0000 [bus 00-01] at [mem 0xe0000000-0xe01fffff] > (base 0xe0000000) > DEBUG: pci_mmcfg_check_reserved > DEBUG: is_mmconf_reserved: method = E820 > PCI: not using MMCONFIG > DEBUG: pci_direct_init > PCI: Using configuration type 1 for base access > > PCI: Using configuration type 1 for extended access > ACPI: Added _OSI(Module Device) > ACPI: Added _OSI(Processor Device) > ACPI: Added _OSI(3.0 _SCP Extensions) > ACPI: Added _OSI(Processor Aggregator Device) > [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored > \_SB_:_OSC invalid UUID > _OSC request data:1 1f > ACPI: Interpreter enabled > ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] > (20140214/hwxface-580) > ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] > (20140214/hwxface-580) > ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S4_] > (20140214/hwxface-580) > ACPI: (supports S0 S3 S5) > ACPI: Using IOAPIC for interrupt routing > DEBUG: ----> pci_mmcfg_late_init > DEBUG: pci_parse_mcfg > PCI: MMCONFIG for domain 0000 [bus 00-01] at [mem 0xe0000000-0xe01fffff] > (base 0xe0000000) > DEBUG: pci_mmcfg_check_reserved > DEBUG: is_mmconf_reserved: method = ACPI motherboard resources > PCI: MMCONFIG at [mem 0xe0000000-0xe01fffff] reserved in ACPI motherboard > resources > > During pci_mmcfg_early_init(), the MMCONFIG failed because the range > 0xe0000000 is not showing as reserved in the E820 mapping. Here is the > snippet of E820 mapping from the system: > ........ > BIOS-e820: [mem 0x00000000c7eb0000-0x00000000c7ec0fff] ACPI data > BIOS-e820: [mem 0x00000000c7ec1000-0x00000000c7ec2fff] ACPI NVS > BIOS-e820: [mem 0x00000000c7ec3000-0x00000000c7efefff] reserved > BIOS-e820: [mem 0x00000000c7f00000-0x00000000c7ffffff] reserved > BIOS-e820: [mem 0x00000000fec00000-0x00000000fec0ffff] reserved > > However, during pci_mmcfg_late_init(), the area is reserved in the "ACPI > motherboard resources", and the pci_mmcfg_check_reserved() does not fail > here. But this is too late since we already setup the "raw_pci_ext_ops" in > the "arch/x86/pci/direct.c: pci_direct_init()" (during to use the IO_ECS. Thanks for checking this out. I'm going to going to drop the IO ECS-related patch for now, and merge the others for v3.16 (the branch is here: http://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git/log/?h=pci/amd-numa). I don't understand why MCFG init is split into two phases, and I don't have time to sort all that out before v3.16. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/arch/x86/pci/amd_bus.c b/arch/x86/pci/amd_bus.c index aa936e3a2019..67dadf179348 100644 --- a/arch/x86/pci/amd_bus.c +++ b/arch/x86/pci/amd_bus.c @@ -405,7 +405,9 @@ static int __init amd_postcore_init(void) return 0; early_fill_mp_bus_info(); - pci_io_ecs_init(); + + if (boot_cpu_data.x86 <= 0x16) + pci_io_ecs_init(); return 0; }
ECS is an AMD mechanism that allows access to extended PCI config space (offsets 256-4096) via I/O ports CF8/CFCh. We normally use ECAM, i.e., MMCONFIG, to access that space, but apparently old machines have issues that meant we couldn't use ECAM. The solution was to enable ECS so we could use configuration mechanism #1 (I/O ports CF8/CFCh) to access extended config space. See 831d991821da ("x86: add PCI extended config space access for AMD Barcelona"). New machines should be able to use ECAM, which means they don't need the CPU-specific code to enable ECS. This patch leaves ECS the same on all existing platforms, but stops enabling it on platforms after Fam16h. Those future platforms should be able to use the standard ACPI MCFG/_CBA descriptions of the PCIe ECAM mechanism. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> CC: Robert Richter <rric@kernel.org> --- arch/x86/pci/amd_bus.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html