| Message ID | PSXP216MB04386BA48874B56BC5CB0292803C0@PSXP216MB0438.KORP216.PROD.OUTLOOK.COM |
|---|---|
| State | New |
| Headers | show |
| Series | PCI: Allow Thunderbolt to work with resources from pci=hpmemsize | expand |
Thanks a lot for splitting these up. It makes these dramatically easier to read. On Mon, Jan 06, 2020 at 03:47:46PM +0000, Nicholas Johnson wrote: > Change extend_bridge_window() to set resource size directly instead of > using additional resource lists. > > Because additional resource lists are optional resources, any algorithm > that requires guaranteed allocation that uses them cannot be guaranteed > to work. There is never a guarantee that PCI resource assignment will work. It's always possible that we don't have enough resources to allow us to enable a device. So I'm not sure what this is telling me, and it doesn't seem like a justification for setting the resource size directly here. Prior to this patch, I think all the assignment and changes to dev->resource[] were in __assign_resources_sorted(). Maybe it's safe to do some here and some in __assign_resources_sorted(), but I don't understand it well enough to be confident. > Remove the resource from add_list, as a zero-sized additional resource > is redundant. > > Update comment in pci_bus_distribute_available_resources() to reflect > the above changes. > > Signed-off-by: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au> > --- > drivers/pci/setup-bus.c | 25 ++++++++----------------- > 1 file changed, 8 insertions(+), 17 deletions(-) > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c > index de43815be..0c51f4937 100644 > --- a/drivers/pci/setup-bus.c > +++ b/drivers/pci/setup-bus.c > @@ -1836,7 +1836,7 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > struct list_head *add_list, > resource_size_t new_size) > { > - struct pci_dev_resource *dev_res; > + resource_size_t add_size; > > if (res->parent) > return; > @@ -1844,17 +1844,10 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > if (resource_size(res) >= new_size) > return; > > - dev_res = res_to_dev_res(add_list, res); > - if (!dev_res) > - return; > - > - /* Is there room to extend the window? */ > - if (new_size - resource_size(res) <= dev_res->add_size) > - return; > - > - dev_res->add_size = new_size - resource_size(res); > - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, > - &dev_res->add_size); > + add_size = new_size - resource_size(res); > + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); > + res->end = res->start + new_size - 1; How do we know it's safe to extend this, i.e., how do we know there's nothing immediately after res? > + remove_from_list(add_list, res); > } > > static void pci_bus_distribute_available_resources(struct pci_bus *bus, > @@ -1889,11 +1882,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, > mmio_pref.start = min(ALIGN(mmio_pref.start, align), > mmio_pref.end + 1); > > - /* > - * Update additional resource list (add_list) to fill all the > - * extra resource space available for this port except the space > - * calculated in __pci_bus_size_bridges() which covers all the > - * devices currently connected to the port and below. > + /* > + * Now that we have adjusted for alignment, update the bridge window > + * resources to fill as much remaining resource space as possible. > */ > adjust_bridge_window(bridge, io_res, add_list, resource_size(&io)); > adjust_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio)); > -- > 2.24.1 >
On Mon, Jan 06, 2020 at 02:23:01PM -0600, Bjorn Helgaas wrote: > Thanks a lot for splitting these up. It makes these dramatically > easier to read. > > On Mon, Jan 06, 2020 at 03:47:46PM +0000, Nicholas Johnson wrote: > > Change extend_bridge_window() to set resource size directly instead of > > using additional resource lists. > > > > Because additional resource lists are optional resources, any algorithm > > that requires guaranteed allocation that uses them cannot be guaranteed > > to work. > > There is never a guarantee that PCI resource assignment will work. > It's always possible that we don't have enough resources to allow us > to enable a device. So I'm not sure what this is telling me, and it > doesn't seem like a justification for setting the resource size > directly here. More reasons why: I have had failure to assign when adding multiple daisy-chained Thunderbolt devices in one enumeration (single native hotplug interrupt) when using add_size, but way too obscure to describe and understand. All I know is it went away when setting size directly. I do not use this as the reason because I do not understand why the failure happens, and this makes it difficult to defend. Also, does avoiding unnecessary complexity count as a reason? The most compelling might be to say that it is preparation for the next commit which allows it to shrink. Suppose a resource has X actual size and Y add_size. What if we want to shrink below X size? The code will have to be much more complicated to handle the logic. How do we word a commit that is in preparation for the next patch? "In preparation for next patch in series, ...."? > > Prior to this patch, I think all the assignment and changes to > dev->resource[] were in __assign_resources_sorted(). Maybe it's safe > to do some here and some in __assign_resources_sorted(), but I don't > understand it well enough to be confident. The resource assignment right now is a black box to me. I do not understand how the sorting happens and what it is trying to do. I would prefer reduce our reliance on add_size if possible. I believe you once agreed you dislike how we have to do it in multiple passes, that we do not have the confidence it will work the first time. Is this how you feel? If we do not have multiple passes, a lot of the complexity goes away. > > > Remove the resource from add_list, as a zero-sized additional resource > > is redundant. > > > > Update comment in pci_bus_distribute_available_resources() to reflect > > the above changes. > > > > Signed-off-by: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au> > > --- > > drivers/pci/setup-bus.c | 25 ++++++++----------------- > > 1 file changed, 8 insertions(+), 17 deletions(-) > > > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c > > index de43815be..0c51f4937 100644 > > --- a/drivers/pci/setup-bus.c > > +++ b/drivers/pci/setup-bus.c > > @@ -1836,7 +1836,7 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > > struct list_head *add_list, > > resource_size_t new_size) > > { > > - struct pci_dev_resource *dev_res; > > + resource_size_t add_size; > > > > if (res->parent) > > return; > > @@ -1844,17 +1844,10 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > > if (resource_size(res) >= new_size) > > return; > > > > - dev_res = res_to_dev_res(add_list, res); > > - if (!dev_res) > > - return; > > - > > - /* Is there room to extend the window? */ > > - if (new_size - resource_size(res) <= dev_res->add_size) > > - return; > > - > > - dev_res->add_size = new_size - resource_size(res); > > - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, > > - &dev_res->add_size); > > + add_size = new_size - resource_size(res); > > + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); > > + res->end = res->start + new_size - 1; > > How do we know it's safe to extend this, i.e., how do we know there's > nothing immediately after res? > > > + remove_from_list(add_list, res); > > } > > > > static void pci_bus_distribute_available_resources(struct pci_bus *bus, > > @@ -1889,11 +1882,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, > > mmio_pref.start = min(ALIGN(mmio_pref.start, align), > > mmio_pref.end + 1); > > > > - /* > > - * Update additional resource list (add_list) to fill all the > > - * extra resource space available for this port except the space > > - * calculated in __pci_bus_size_bridges() which covers all the > > - * devices currently connected to the port and below. > > + /* > > + * Now that we have adjusted for alignment, update the bridge window > > + * resources to fill as much remaining resource space as possible. > > */ > > adjust_bridge_window(bridge, io_res, add_list, resource_size(&io)); > > adjust_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio)); > > -- > > 2.24.1 > > Cheers. Regards, Nicholas
On Mon, Jan 06, 2020 at 03:47:46PM +0000, Nicholas Johnson wrote: > Change extend_bridge_window() to set resource size directly instead of > using additional resource lists. > > Because additional resource lists are optional resources, any algorithm > that requires guaranteed allocation that uses them cannot be guaranteed > to work. > > Remove the resource from add_list, as a zero-sized additional resource > is redundant. > > Update comment in pci_bus_distribute_available_resources() to reflect > the above changes. > > Signed-off-by: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au> > --- > drivers/pci/setup-bus.c | 25 ++++++++----------------- > 1 file changed, 8 insertions(+), 17 deletions(-) > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c > index de43815be..0c51f4937 100644 > --- a/drivers/pci/setup-bus.c > +++ b/drivers/pci/setup-bus.c > @@ -1836,7 +1836,7 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > struct list_head *add_list, > resource_size_t new_size) > { > - struct pci_dev_resource *dev_res; > + resource_size_t add_size; > > if (res->parent) > return; > @@ -1844,17 +1844,10 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > if (resource_size(res) >= new_size) > return; > > - dev_res = res_to_dev_res(add_list, res); > - if (!dev_res) > - return; > - > - /* Is there room to extend the window? */ > - if (new_size - resource_size(res) <= dev_res->add_size) > - return; > - > - dev_res->add_size = new_size - resource_size(res); > - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, > - &dev_res->add_size); > + add_size = new_size - resource_size(res); > + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); > + res->end = res->start + new_size - 1; > + remove_from_list(add_list, res); > } > > static void pci_bus_distribute_available_resources(struct pci_bus *bus, > @@ -1889,11 +1882,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, > mmio_pref.start = min(ALIGN(mmio_pref.start, align), > mmio_pref.end + 1); > > - /* > - * Update additional resource list (add_list) to fill all the > - * extra resource space available for this port except the space > - * calculated in __pci_bus_size_bridges() which covers all the > - * devices currently connected to the port and below. > + /* Indentation is wrong here. > + * Now that we have adjusted for alignment, update the bridge window > + * resources to fill as much remaining resource space as possible. > */ > adjust_bridge_window(bridge, io_res, add_list, resource_size(&io)); > adjust_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio)); > -- > 2.24.1
On Mon, Jan 06, 2020 at 02:23:01PM -0600, Bjorn Helgaas wrote: > Thanks a lot for splitting these up. It makes these dramatically > easier to read. > > On Mon, Jan 06, 2020 at 03:47:46PM +0000, Nicholas Johnson wrote: > > Change extend_bridge_window() to set resource size directly instead of > > using additional resource lists. > > > > Because additional resource lists are optional resources, any algorithm > > that requires guaranteed allocation that uses them cannot be guaranteed > > to work. > > There is never a guarantee that PCI resource assignment will work. > It's always possible that we don't have enough resources to allow us > to enable a device. So I'm not sure what this is telling me, and it > doesn't seem like a justification for setting the resource size > directly here. > > Prior to this patch, I think all the assignment and changes to > dev->resource[] were in __assign_resources_sorted(). Maybe it's safe > to do some here and some in __assign_resources_sorted(), but I don't > understand it well enough to be confident. > > > Remove the resource from add_list, as a zero-sized additional resource > > is redundant. > > > > Update comment in pci_bus_distribute_available_resources() to reflect > > the above changes. > > > > Signed-off-by: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au> > > --- > > drivers/pci/setup-bus.c | 25 ++++++++----------------- > > 1 file changed, 8 insertions(+), 17 deletions(-) > > > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c > > index de43815be..0c51f4937 100644 > > --- a/drivers/pci/setup-bus.c > > +++ b/drivers/pci/setup-bus.c > > @@ -1836,7 +1836,7 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > > struct list_head *add_list, > > resource_size_t new_size) > > { > > - struct pci_dev_resource *dev_res; > > + resource_size_t add_size; > > > > if (res->parent) > > return; > > @@ -1844,17 +1844,10 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, > > if (resource_size(res) >= new_size) > > return; > > > > - dev_res = res_to_dev_res(add_list, res); > > - if (!dev_res) > > - return; > > - > > - /* Is there room to extend the window? */ > > - if (new_size - resource_size(res) <= dev_res->add_size) > > - return; > > - > > - dev_res->add_size = new_size - resource_size(res); > > - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, > > - &dev_res->add_size); > > + add_size = new_size - resource_size(res); > > + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); > > + res->end = res->start + new_size - 1; > > How do we know it's safe to extend this, i.e., how do we know there's > nothing immediately after res? Sorry, I do not remember answering this one before, must have missed it. Because we return if assigned (res->parent not NULL), this resource has not been assigned. Hence, the desired alignment is .start and the desired size is set by setting the .end = .start + size - 1. The address is not yet absolute. If we make it bigger then the kernel will take that into account when handing out addresses. Also, since this resource is not yet assigned, neither are the other ones around it. pci_bus_distribute_available_resources() is called on the native hotplug interrupt when a device (such as Thunderbolt 3) is added. > > > + remove_from_list(add_list, res); > > } > > > > static void pci_bus_distribute_available_resources(struct pci_bus *bus, > > @@ -1889,11 +1882,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, > > mmio_pref.start = min(ALIGN(mmio_pref.start, align), > > mmio_pref.end + 1); > > > > - /* > > - * Update additional resource list (add_list) to fill all the > > - * extra resource space available for this port except the space > > - * calculated in __pci_bus_size_bridges() which covers all the > > - * devices currently connected to the port and below. > > + /* > > + * Now that we have adjusted for alignment, update the bridge window > > + * resources to fill as much remaining resource space as possible. > > */ > > adjust_bridge_window(bridge, io_res, add_list, resource_size(&io)); > > adjust_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio)); > > -- > > 2.24.1 > >
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index de43815be..0c51f4937 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1836,7 +1836,7 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, struct list_head *add_list, resource_size_t new_size) { - struct pci_dev_resource *dev_res; + resource_size_t add_size; if (res->parent) return; @@ -1844,17 +1844,10 @@ static void adjust_bridge_window(struct pci_dev *bridge, struct resource *res, if (resource_size(res) >= new_size) return; - dev_res = res_to_dev_res(add_list, res); - if (!dev_res) - return; - - /* Is there room to extend the window? */ - if (new_size - resource_size(res) <= dev_res->add_size) - return; - - dev_res->add_size = new_size - resource_size(res); - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, - &dev_res->add_size); + add_size = new_size - resource_size(res); + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); + res->end = res->start + new_size - 1; + remove_from_list(add_list, res); } static void pci_bus_distribute_available_resources(struct pci_bus *bus, @@ -1889,11 +1882,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, mmio_pref.start = min(ALIGN(mmio_pref.start, align), mmio_pref.end + 1); - /* - * Update additional resource list (add_list) to fill all the - * extra resource space available for this port except the space - * calculated in __pci_bus_size_bridges() which covers all the - * devices currently connected to the port and below. + /* + * Now that we have adjusted for alignment, update the bridge window + * resources to fill as much remaining resource space as possible. */ adjust_bridge_window(bridge, io_res, add_list, resource_size(&io)); adjust_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio));
Change extend_bridge_window() to set resource size directly instead of using additional resource lists. Because additional resource lists are optional resources, any algorithm that requires guaranteed allocation that uses them cannot be guaranteed to work. Remove the resource from add_list, as a zero-sized additional resource is redundant. Update comment in pci_bus_distribute_available_resources() to reflect the above changes. Signed-off-by: Nicholas Johnson <nicholas.johnson-opensource@outlook.com.au> --- drivers/pci/setup-bus.c | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-)