| Message ID | 20190122125133.1191-2-david@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | s390x/pci: PCI bridge plugging fixes | expand |
On 2019-01-22 13:51, David Hildenbrand wrote: > The primary bus number corresponds always to the bus number of the > bus the bridge is attached to. > > Right now, if we have two bridges attached to the same bus (e.g. root > bus) this is however not the case. Fix assignment. > > While at it > - Add a comment why we have to reassign durign every reset (which I s/durign/during/ > found to be supprising) s/supprising/surprising/ > - Drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. As we are > setting it via a DFS and not via a BFS (as discussed e.g. in [1]), this > is not necessary. The last number when we return is the highest > number. I think that explanation is slightly wrong / misleading. It's not about DFS vs. BFS, it's about guest code vs. QEMU code. If you do a bridge setup from the guest side, you've got to set the subordinate bus number to 0xff to make sure that the bridge forwards all config space accesses to the attached devices while you're scanning the devices that are attached to the bridge. But this code is not running in the guest, it is running in QEMU, and so it can access the config space of the attached devices directly via pci_default_write_config() - the write do not need to pass the parent bridge in this case, and thus the subordinate bus number in the bridge is ignored for the config space write access. > Please note that hotplugging of bridges is in general still broken, will > be fixed next. > > [1] http://www.science.unitn.it/~fiorella/guidelinux/tlk/node76.html > > Signed-off-by: David Hildenbrand <david@redhat.com> > --- > hw/s390x/s390-pci-bus.c | 13 ++++++++----- > 1 file changed, 8 insertions(+), 5 deletions(-) With the commit message fixed: Reviewed-by: Thomas Huth <thuth@redhat.com>
On 23.01.19 11:26, Thomas Huth wrote: > On 2019-01-22 13:51, David Hildenbrand wrote: >> The primary bus number corresponds always to the bus number of the >> bus the bridge is attached to. >> >> Right now, if we have two bridges attached to the same bus (e.g. root >> bus) this is however not the case. Fix assignment. >> >> While at it >> - Add a comment why we have to reassign durign every reset (which I > > s/durign/during/ > >> found to be supprising) > > s/supprising/surprising/ > >> - Drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. As we are >> setting it via a DFS and not via a BFS (as discussed e.g. in [1]), this >> is not necessary. The last number when we return is the highest >> number. > > I think that explanation is slightly wrong / misleading. It's not about > DFS vs. BFS, it's about guest code vs. QEMU code. If you do a bridge > setup from the guest side, you've got to set the subordinate bus number > to 0xff to make sure that the bridge forwards all config space accesses > to the attached devices while you're scanning the devices that are > attached to the bridge. > But this code is not running in the guest, it is running in QEMU, and so > it can access the config space of the attached devices directly via > pci_default_write_config() - the write do not need to pass the parent > bridge in this case, and thus the subordinate bus number in the bridge > is ignored for the config space write access. Indeed, I phrased that better in the spapr/pci patch I sent, What about this: " The primary bus number corresponds always to the bus number of the bus the bridge is attached to. Right now, if we have two bridges attached to the same bus (e.g. root bus) this is however not the case. The first bridge will have primary bus 0, the second bridge primary bus 1, which is wrong. Fix the assignment. While at it, drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. Setting it temporarily to that value (as discussed e.g. in [1]), is only relevant for a running system that probes the buses. The value is effectively unused for us just doing a DFS. Also add a comment why we have to reassign during every reset (which I found to be surprising. Please note that hotplugging of bridges is in general still broken, will be fixed next. " > >> Please note that hotplugging of bridges is in general still broken, will >> be fixed next. >> >> [1] http://www.science.unitn.it/~fiorella/guidelinux/tlk/node76.html >> >> Signed-off-by: David Hildenbrand <david@redhat.com> >> --- >> hw/s390x/s390-pci-bus.c | 13 ++++++++----- >> 1 file changed, 8 insertions(+), 5 deletions(-) > > With the commit message fixed: @Conny can you fix up when applying if there are no other comments? > > Reviewed-by: Thomas Huth <thuth@redhat.com> > Thanks Thomas!
On Wed, 23 Jan 2019 11:32:48 +0100 David Hildenbrand <david@redhat.com> wrote: > On 23.01.19 11:26, Thomas Huth wrote: > > On 2019-01-22 13:51, David Hildenbrand wrote: > >> The primary bus number corresponds always to the bus number of the > >> bus the bridge is attached to. > >> > >> Right now, if we have two bridges attached to the same bus (e.g. root > >> bus) this is however not the case. Fix assignment. > >> > >> While at it > >> - Add a comment why we have to reassign durign every reset (which I > > > > s/durign/during/ > > > >> found to be supprising) > > > > s/supprising/surprising/ > > > >> - Drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. As we are > >> setting it via a DFS and not via a BFS (as discussed e.g. in [1]), this > >> is not necessary. The last number when we return is the highest > >> number. > > > > I think that explanation is slightly wrong / misleading. It's not about > > DFS vs. BFS, it's about guest code vs. QEMU code. If you do a bridge > > setup from the guest side, you've got to set the subordinate bus number > > to 0xff to make sure that the bridge forwards all config space accesses > > to the attached devices while you're scanning the devices that are > > attached to the bridge. > > But this code is not running in the guest, it is running in QEMU, and so > > it can access the config space of the attached devices directly via > > pci_default_write_config() - the write do not need to pass the parent > > bridge in this case, and thus the subordinate bus number in the bridge > > is ignored for the config space write access. > > Indeed, I phrased that better in the spapr/pci patch I sent, What about > this: > > " > The primary bus number corresponds always to the bus number of the > bus the bridge is attached to. > > Right now, if we have two bridges attached to the same bus (e.g. root > bus) this is however not the case. The first bridge will have primary > bus 0, the second bridge primary bus 1, which is wrong. Fix the assignment. > > While at it, drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. > Setting it temporarily to that value (as discussed e.g. in [1]), is > only relevant for a running system that probes the buses. The value is > effectively unused for us just doing a DFS. > > Also add a comment why we have to reassign during every reset (which I > found to be surprising. > > Please note that hotplugging of bridges is in general still broken, will > be fixed next. > " > > > > >> Please note that hotplugging of bridges is in general still broken, will > >> be fixed next. > >> > >> [1] http://www.science.unitn.it/~fiorella/guidelinux/tlk/node76.html > >> > >> Signed-off-by: David Hildenbrand <david@redhat.com> > >> --- > >> hw/s390x/s390-pci-bus.c | 13 ++++++++----- > >> 1 file changed, 8 insertions(+), 5 deletions(-) > > > > With the commit message fixed: > > @Conny can you fix up when applying if there are no other comments? Sure, can do. Waiting for zpci maintainer ack :) > > > > > Reviewed-by: Thomas Huth <thuth@redhat.com> > > > > Thanks Thomas! > >
On 2019-01-23 11:32, David Hildenbrand wrote: > On 23.01.19 11:26, Thomas Huth wrote: >> On 2019-01-22 13:51, David Hildenbrand wrote: >>> The primary bus number corresponds always to the bus number of the >>> bus the bridge is attached to. >>> >>> Right now, if we have two bridges attached to the same bus (e.g. root >>> bus) this is however not the case. Fix assignment. >>> >>> While at it >>> - Add a comment why we have to reassign durign every reset (which I >> >> s/durign/during/ >> >>> found to be supprising) >> >> s/supprising/surprising/ >> >>> - Drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. As we are >>> setting it via a DFS and not via a BFS (as discussed e.g. in [1]), this >>> is not necessary. The last number when we return is the highest >>> number. >> >> I think that explanation is slightly wrong / misleading. It's not about >> DFS vs. BFS, it's about guest code vs. QEMU code. If you do a bridge >> setup from the guest side, you've got to set the subordinate bus number >> to 0xff to make sure that the bridge forwards all config space accesses >> to the attached devices while you're scanning the devices that are >> attached to the bridge. >> But this code is not running in the guest, it is running in QEMU, and so >> it can access the config space of the attached devices directly via >> pci_default_write_config() - the write do not need to pass the parent >> bridge in this case, and thus the subordinate bus number in the bridge >> is ignored for the config space write access. > > Indeed, I phrased that better in the spapr/pci patch I sent, What about > this: > > " > The primary bus number corresponds always to the bus number of the > bus the bridge is attached to. > > Right now, if we have two bridges attached to the same bus (e.g. root > bus) this is however not the case. The first bridge will have primary > bus 0, the second bridge primary bus 1, which is wrong. Fix the assignment. > > While at it, drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. > Setting it temporarily to that value (as discussed e.g. in [1]), is > only relevant for a running system that probes the buses. The value is > effectively unused for us just doing a DFS. I'd rather replace the last sentence with: The value is not necessary in QEMU code since we can access the config space of the devices directly here, without the need to pass the config space write requests through the superordinate bridges. Apart from that, text looks fine to me now. Thomas
diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index f017c1ded0..b7c4613fde 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -862,7 +862,8 @@ static void s390_pcihost_plug(HotplugHandler *hotplug_dev, DeviceState *dev, qbus_set_hotplug_handler(bus, DEVICE(s), errp); if (dev->hotplugged) { - pci_default_write_config(pdev, PCI_PRIMARY_BUS, s->bus_no, 1); + pci_default_write_config(pdev, PCI_PRIMARY_BUS, + pci_dev_bus_num(pdev), 1); s->bus_no += 1; pci_default_write_config(pdev, PCI_SECONDARY_BUS, s->bus_no, 1); do { @@ -1016,8 +1017,6 @@ static void s390_pci_enumerate_bridge(PCIBus *bus, PCIDevice *pdev, void *opaque) { S390pciState *s = opaque; - unsigned int primary = s->bus_no; - unsigned int subordinate = 0xff; PCIBus *sec_bus = NULL; if ((pci_default_read_config(pdev, PCI_HEADER_TYPE, 1) != @@ -1026,7 +1025,7 @@ static void s390_pci_enumerate_bridge(PCIBus *bus, PCIDevice *pdev, } (s->bus_no)++; - pci_default_write_config(pdev, PCI_PRIMARY_BUS, primary, 1); + pci_default_write_config(pdev, PCI_PRIMARY_BUS, pci_dev_bus_num(pdev), 1); pci_default_write_config(pdev, PCI_SECONDARY_BUS, s->bus_no, 1); pci_default_write_config(pdev, PCI_SUBORDINATE_BUS, s->bus_no, 1); @@ -1035,7 +1034,7 @@ static void s390_pci_enumerate_bridge(PCIBus *bus, PCIDevice *pdev, return; } - pci_default_write_config(pdev, PCI_SUBORDINATE_BUS, subordinate, 1); + /* Assign numbers to all child bridges. The last is the highest number. */ pci_for_each_device(sec_bus, pci_bus_num(sec_bus), s390_pci_enumerate_bridge, s); pci_default_write_config(pdev, PCI_SUBORDINATE_BUS, s->bus_no, 1); @@ -1046,6 +1045,10 @@ static void s390_pcihost_reset(DeviceState *dev) S390pciState *s = S390_PCI_HOST_BRIDGE(dev); PCIBus *bus = s->parent_obj.bus; + /* + * When resetting a PCI bridge, the assigned numbers are set to 0. So + * on every system reset, we also have to reassign numbers. + */ s->bus_no = 0; pci_for_each_device(bus, pci_bus_num(bus), s390_pci_enumerate_bridge, s); }
The primary bus number corresponds always to the bus number of the bus the bridge is attached to. Right now, if we have two bridges attached to the same bus (e.g. root bus) this is however not the case. Fix assignment. While at it - Add a comment why we have to reassign durign every reset (which I found to be supprising) - Drop setting the PCI_SUBORDINATE_BUS temporarily to 0xff. As we are setting it via a DFS and not via a BFS (as discussed e.g. in [1]), this is not necessary. The last number when we return is the highest number. Please note that hotplugging of bridges is in general still broken, will be fixed next. [1] http://www.science.unitn.it/~fiorella/guidelinux/tlk/node76.html Signed-off-by: David Hildenbrand <david@redhat.com> --- hw/s390x/s390-pci-bus.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)