| Message ID | 1342002726-18258-6-git-send-email-vasilis.liaskovitis@profitbricks.com |
|---|---|
| State | New |
| Headers | show |
On 07/11/12 12:31, Vasilis Liaskovitis wrote: > In order to hotplug memory between RamSize and BUILD_PCIMEM_START, the pci > window needs to start at BUILD_PCIMEM_START (0xe0000000). > Otherwise, the guest cannot online new dimms at those ranges due to pci_root > window conflicts. (workaround for linux guest is booting with pci=nocrs) > static void pci_bios_map_devices(struct pci_bus *busses) > { > - pcimem_start = RamSize; > + pcimem_start = BUILD_PCIMEM_START; It isn't that simple. For the 32bit pci window it will work, but will leaves address space unused instead of assigning it to the 32bit pci window. For the 64bit pci window it will not work. You have to walk the dimms and figure what the highest used address is, for both below-4g and above-4g. Then fill two variable with it and make the pci init code use that instead of RamSize and RamSizeOver4G. cheers, Gerd
Hi, On Wed, Jul 11, 2012 at 01:56:19PM +0200, Gerd Hoffmann wrote: > On 07/11/12 12:31, Vasilis Liaskovitis wrote: > > In order to hotplug memory between RamSize and BUILD_PCIMEM_START, the pci > > window needs to start at BUILD_PCIMEM_START (0xe0000000). > > Otherwise, the guest cannot online new dimms at those ranges due to pci_root > > window conflicts. (workaround for linux guest is booting with pci=nocrs) > > > static void pci_bios_map_devices(struct pci_bus *busses) > > { > > - pcimem_start = RamSize; > > + pcimem_start = BUILD_PCIMEM_START; > > It isn't that simple. For the 32bit pci window it will work, but will > leaves address space unused instead of assigning it to the 32bit pci > window. For the 64bit pci window it will not work. > > You have to walk the dimms and figure what the highest used address is, > for both below-4g and above-4g. Then fill two variable with it and make > the pci init code use that instead of RamSize and RamSizeOver4G. I see. I already have these values values computed in qemu-kvm, so I can pass them in a paravirt struct, or infer them from the dimm/srat paravirt info that I already pass to seabios. If i understand correctly, we would like the pcimem windows to use the maximum possible address space (constrained by the exact dimms/ranges which are defined) instead of leaving unused space. thanks, - Vasilis
On 07/11/12 18:45, Vasilis Liaskovitis wrote: > Hi, > > On Wed, Jul 11, 2012 at 01:56:19PM +0200, Gerd Hoffmann wrote: >> On 07/11/12 12:31, Vasilis Liaskovitis wrote: >>> In order to hotplug memory between RamSize and BUILD_PCIMEM_START, the pci >>> window needs to start at BUILD_PCIMEM_START (0xe0000000). >>> Otherwise, the guest cannot online new dimms at those ranges due to pci_root >>> window conflicts. (workaround for linux guest is booting with pci=nocrs) >> >>> static void pci_bios_map_devices(struct pci_bus *busses) >>> { >>> - pcimem_start = RamSize; >>> + pcimem_start = BUILD_PCIMEM_START; >> >> It isn't that simple. For the 32bit pci window it will work, but will >> leaves address space unused instead of assigning it to the 32bit pci >> window. For the 64bit pci window it will not work. >> >> You have to walk the dimms and figure what the highest used address is, >> for both below-4g and above-4g. Then fill two variable with it and make >> the pci init code use that instead of RamSize and RamSizeOver4G. > > I see. I already have these values values computed in qemu-kvm, so I can pass > them in a paravirt struct, or infer them from the dimm/srat paravirt info that I > already pass to seabios. I'd suggest to infer from the dimm info, to limit the amout of information which needs to be passed from qemu to seabios. > If i understand correctly, we would like the pcimem windows to use the maximum > possible address space (constrained by the exact dimms/ranges which are defined) > instead of leaving unused space. Yes, for the 32bit pci window. The 64bit pci window is mapped above all memory, and it must likewise consider defined+unfilled dimms so the start address doesn't collide with memory hot-plugged above 4G later on. cheers, Gerd
On Thu, Jul 12, 2012 at 09:22:14AM +0200, Gerd Hoffmann wrote: > On 07/11/12 18:45, Vasilis Liaskovitis wrote: > > Hi, > > > > On Wed, Jul 11, 2012 at 01:56:19PM +0200, Gerd Hoffmann wrote: > >> On 07/11/12 12:31, Vasilis Liaskovitis wrote: > >>> In order to hotplug memory between RamSize and BUILD_PCIMEM_START, the pci > >>> window needs to start at BUILD_PCIMEM_START (0xe0000000). > >>> Otherwise, the guest cannot online new dimms at those ranges due to pci_root > >>> window conflicts. (workaround for linux guest is booting with pci=nocrs) > >> > >>> static void pci_bios_map_devices(struct pci_bus *busses) > >>> { > >>> - pcimem_start = RamSize; > >>> + pcimem_start = BUILD_PCIMEM_START; > >> > >> It isn't that simple. For the 32bit pci window it will work, but will > >> leaves address space unused instead of assigning it to the 32bit pci > >> window. For the 64bit pci window it will not work. > >> > >> You have to walk the dimms and figure what the highest used address is, > >> for both below-4g and above-4g. Then fill two variable with it and make > >> the pci init code use that instead of RamSize and RamSizeOver4G. > > > > I see. I already have these values values computed in qemu-kvm, so I can pass > > them in a paravirt struct, or infer them from the dimm/srat paravirt info that I > > already pass to seabios. > > I'd suggest to infer from the dimm info, to limit the amout of > information which needs to be passed from qemu to seabios. ok.Currently dimm info is processed in bios_init_tables(), which is called after pci_setup(). I 'll see if i can do the processing earlier. > > > If i understand correctly, we would like the pcimem windows to use the maximum > > possible address space (constrained by the exact dimms/ranges which are defined) > > instead of leaving unused space. > > Yes, for the 32bit pci window. > > The 64bit pci window is mapped above all memory, and it must likewise > consider defined+unfilled dimms so the start address doesn't collide > with memory hot-plugged above 4G later on. yes, understood. thanks, - Vasilis
diff --git a/src/pciinit.c b/src/pciinit.c index 68f302a..0b11cbe 100644 --- a/src/pciinit.c +++ b/src/pciinit.c @@ -592,7 +592,7 @@ static void pci_region_map_entries(struct pci_bus *busses, struct pci_region *r) static void pci_bios_map_devices(struct pci_bus *busses) { - pcimem_start = RamSize; + pcimem_start = BUILD_PCIMEM_START; if (pci_bios_init_root_regions(busses)) { struct pci_region r64_mem, r64_pref;
In order to hotplug memory between RamSize and BUILD_PCIMEM_START, the pci window needs to start at BUILD_PCIMEM_START (0xe0000000). Otherwise, the guest cannot online new dimms at those ranges due to pci_root window conflicts. (workaround for linux guest is booting with pci=nocrs) Signed-off-by: Vasilis Liaskovitis <vasilis.liaskovitis@profitbricks.com> --- src/pciinit.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)