| Message ID | 1526639490-12167-1-git-send-email-ulf.hansson@linaro.org |
|---|---|
| Headers | show |
| Series | PM / Domains: Add support for multi PM domains per device | expand |
Hi Ulf, On Fri, May 18, 2018 at 12:31 PM, Ulf Hansson <ulf.hansson@linaro.org> wrote: > To be able to describe topologies where devices are partitioned across > multiple power domains, let's extend the power-domain property to allow > being a list of phandles. > > Cc: Rob Herring <robh+dt@kernel.org> > Cc: devicetree@vger.kernel.org > Suggested-by: Jon Hunter <jonathanh@nvidia.com> > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> Thanks for your patch! > --- a/Documentation/devicetree/bindings/power/power_domain.txt > +++ b/Documentation/devicetree/bindings/power/power_domain.txt > @@ -111,8 +111,9 @@ Example 3: > ==PM domain consumers== > > Required properties: > - - power-domains : A phandle and PM domain specifier as defined by bindings of > - the power controller specified by phandle. > + - power-domains : A phandle and PM domain specifier, or a list of phandles, as A list of PM domain specifiers? (A PM domain specifier consists of a phandle, and zero or more indices) > + defined by bindings of the power controller specified by > + phandle. > > Example: > > @@ -122,9 +123,16 @@ Example: > power-domains = <&power 0>; > }; > > -The node above defines a typical PM domain consumer device, which is located > -inside a PM domain with index 0 of a power controller represented by a node > -with the label "power". > + leaky-device@12350000 { I know it's just an example, but this uses the same unit-address and reg property as the device node above. There's a similar issue with the two other examples below, but they use different node names (leaky-device0 and leaky-device1). > + compatible = "foo,i-leak-current"; > + reg = <0x12350000 0x1000>; > + power-domains = <&power0 0>, <&power1 0> ; > + }; > + > +The first example above defines a typical PM domain consumer device, which is > +located inside a PM domain with index 0 of a power controller represented by a > +node with the label "power". In the second example the consumer device are > +partitioned across two PM domains. > > Optional properties: > - required-opps: This contains phandle to an OPP node in another device's OPP Gr{oetje,eeting}s, Geert
[...] >> >> +/** >> + * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >> + * @dev: Device to attach. >> + * @index: The index of the PM domain. >> + * >> + * Parse device's OF node to find a PM domain specifier at the provided @index. >> + * If such is found, allocates a new device and attaches it to retrieved >> + * pm_domain ops. >> + * >> + * Returns the allocated device if successfully attached PM domain, NULL when >> + * the device don't need a PM domain or have a single PM domain, else PTR_ERR() >> + * in case of failures. Note that if a power-domain exists for the device, but >> + * cannot be found or turned on, then return PTR_ERR(-EPROBE_DEFER) to ensure >> + * that the device is not probed and to re-try again later. >> + */ >> +struct device *genpd_dev_pm_attach_by_id(struct device *dev, >> + unsigned int index) >> +{ >> + struct device *genpd_dev; >> + int num_domains; >> + int ret; >> + >> + if (!dev->of_node) >> + return NULL; >> + >> + /* Deal only with devices using multiple PM domains. */ >> + num_domains = of_count_phandle_with_args(dev->of_node, "power-domains", >> + "#power-domain-cells"); >> + if (num_domains < 2 || index >= num_domains) >> + return NULL; >> + >> + /* Allocate and register device on the genpd bus. */ >> + genpd_dev = kzalloc(sizeof(*genpd_dev), GFP_KERNEL); >> + if (!genpd_dev) >> + return ERR_PTR(-ENOMEM); >> + >> + dev_set_name(genpd_dev, "genpd:%u:%s", index, dev_name(dev)); >> + genpd_dev->bus = &genpd_bus_type; >> + genpd_dev->release = genpd_release_dev; >> + >> + ret = device_register(genpd_dev); >> + if (ret) { >> + kfree(genpd_dev); >> + return ERR_PTR(ret); >> + } >> + >> + /* Try to attach the device to the PM domain at the specified index. */ >> + ret = __genpd_dev_pm_attach(genpd_dev, dev->of_node, index); >> + if (ret < 1) { >> + device_unregister(genpd_dev); >> + return ret ? ERR_PTR(ret) : NULL; >> + } >> + >> + pm_runtime_set_active(genpd_dev); >> + pm_runtime_enable(genpd_dev); >> + >> + return genpd_dev; >> +} >> +EXPORT_SYMBOL_GPL(genpd_dev_pm_attach_by_id); > > Thanks for sending this. Believe it or not this has still been on my to-do list > and so we definitely need a solution for Tegra. > > Looking at the above it appears that additional power-domains exposed as devices > to the client device. So I assume that this means that the drivers for devices > with multiple power-domains will need to call RPM APIs for each of these > additional power-domains. Is that correct? They can, but should not! Instead, the driver shall use device_link_add() and device_link_del(), dynamically, depending on what PM domain that their original device needs for the current running use case. In that way, they keep existing runtime PM deployment, operating on its original device. > > If so, I can see that that would work, but it does not seem to fit the RPM model > where currently for something like device clocks, the RPM callbacks can handle > all clocks at once. > > I was wondering why we could not add a list of genpd domains to the > dev_pm_domain struct for the device? For example ... See above answer, hopefully that explains it. FYI, that's why I also discovered the bug around using device links with runtime PM during probe. https://patchwork.kernel.org/patch/10408825/ > > diff --git a/include/linux/pm.h b/include/linux/pm.h > index e723b78d8357..a11d6db3c077 100644 > --- a/include/linux/pm.h > +++ b/include/linux/pm.h > @@ -659,6 +659,7 @@ extern void dev_pm_put_subsys_data(struct device *dev); > * subsystem-level and driver-level callbacks. > */ > struct dev_pm_domain { > + struct list_head genpd_list; > struct dev_pm_ops ops; > void (*detach)(struct device *dev, bool power_off); > int (*activate)(struct device *dev); > @@ -666,6 +667,11 @@ struct dev_pm_domain { > void (*dismiss)(struct device *dev); > }; > > +struct dev_pm_domain_link { > + struct generic_pm_domain *genpd; > + struct list_head node; > +}; > + > /* > * The PM_EVENT_ messages are also used by drivers implementing the legacy > * suspend framework, based on the ->suspend() and ->resume() callbacks common > diff --git a/include/linux/pm_domain.h b/include/linux/pm_domain.h > index e61b5cd79fe2..019593804670 100644 > --- a/include/linux/pm_domain.h > +++ b/include/linux/pm_domain.h > @@ -51,7 +51,6 @@ struct dev_pm_opp; > > struct generic_pm_domain { > struct device dev; > - struct dev_pm_domain domain; /* PM domain operations */ > struct list_head gpd_list_node; /* Node in the global PM domains list */ > struct list_head master_links; /* Links with PM domain as a master */ > struct list_head slave_links; /* Links with PM domain as a slave */ > @@ -99,11 +98,6 @@ struct generic_pm_domain { > > }; > > -static inline struct generic_pm_domain *pd_to_genpd(struct dev_pm_domain *pd) > -{ > - return container_of(pd, struct generic_pm_domain, domain); > -} > - > > Obviously the above will not compile but the intent would be to allocate a > dev_pm_domain_link struct per power-domain that the device needs and add > to the genpd_list for the device. It would be a much bigger change in > having to iterate through all the power-domains when turning devices on > and off, however, it would simplify the client driver. > > Cheers > Jon > > -- > nvpublic Kind regards Uffe
On 22/05/18 15:47, Ulf Hansson wrote: > [...] > >>> >>> +/** >>> + * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >>> + * @dev: Device to attach. >>> + * @index: The index of the PM domain. >>> + * >>> + * Parse device's OF node to find a PM domain specifier at the provided @index. >>> + * If such is found, allocates a new device and attaches it to retrieved >>> + * pm_domain ops. >>> + * >>> + * Returns the allocated device if successfully attached PM domain, NULL when >>> + * the device don't need a PM domain or have a single PM domain, else PTR_ERR() >>> + * in case of failures. Note that if a power-domain exists for the device, but >>> + * cannot be found or turned on, then return PTR_ERR(-EPROBE_DEFER) to ensure >>> + * that the device is not probed and to re-try again later. >>> + */ >>> +struct device *genpd_dev_pm_attach_by_id(struct device *dev, >>> + unsigned int index) >>> +{ >>> + struct device *genpd_dev; >>> + int num_domains; >>> + int ret; >>> + >>> + if (!dev->of_node) >>> + return NULL; >>> + >>> + /* Deal only with devices using multiple PM domains. */ >>> + num_domains = of_count_phandle_with_args(dev->of_node, "power-domains", >>> + "#power-domain-cells"); >>> + if (num_domains < 2 || index >= num_domains) >>> + return NULL; >>> + >>> + /* Allocate and register device on the genpd bus. */ >>> + genpd_dev = kzalloc(sizeof(*genpd_dev), GFP_KERNEL); >>> + if (!genpd_dev) >>> + return ERR_PTR(-ENOMEM); >>> + >>> + dev_set_name(genpd_dev, "genpd:%u:%s", index, dev_name(dev)); >>> + genpd_dev->bus = &genpd_bus_type; >>> + genpd_dev->release = genpd_release_dev; >>> + >>> + ret = device_register(genpd_dev); >>> + if (ret) { >>> + kfree(genpd_dev); >>> + return ERR_PTR(ret); >>> + } >>> + >>> + /* Try to attach the device to the PM domain at the specified index. */ >>> + ret = __genpd_dev_pm_attach(genpd_dev, dev->of_node, index); >>> + if (ret < 1) { >>> + device_unregister(genpd_dev); >>> + return ret ? ERR_PTR(ret) : NULL; >>> + } >>> + >>> + pm_runtime_set_active(genpd_dev); >>> + pm_runtime_enable(genpd_dev); >>> + >>> + return genpd_dev; >>> +} >>> +EXPORT_SYMBOL_GPL(genpd_dev_pm_attach_by_id); >> >> Thanks for sending this. Believe it or not this has still been on my to-do list >> and so we definitely need a solution for Tegra. >> >> Looking at the above it appears that additional power-domains exposed as devices >> to the client device. So I assume that this means that the drivers for devices >> with multiple power-domains will need to call RPM APIs for each of these >> additional power-domains. Is that correct? > > They can, but should not! > > Instead, the driver shall use device_link_add() and device_link_del(), > dynamically, depending on what PM domain that their original device > needs for the current running use case. > > In that way, they keep existing runtime PM deployment, operating on > its original device. OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? Thanks Jon
On 05/23/2018 02:25 AM, Jon Hunter wrote: > > On 22/05/18 15:47, Ulf Hansson wrote: >> [...] >> >>>> >>>> +/** >>>> + * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >>>> + * @dev: Device to attach. >>>> + * @index: The index of the PM domain. >>>> + * >>>> + * Parse device's OF node to find a PM domain specifier at the provided @index. >>>> + * If such is found, allocates a new device and attaches it to retrieved >>>> + * pm_domain ops. >>>> + * >>>> + * Returns the allocated device if successfully attached PM domain, NULL when >>>> + * the device don't need a PM domain or have a single PM domain, else PTR_ERR() >>>> + * in case of failures. Note that if a power-domain exists for the device, but >>>> + * cannot be found or turned on, then return PTR_ERR(-EPROBE_DEFER) to ensure >>>> + * that the device is not probed and to re-try again later. >>>> + */ >>>> +struct device *genpd_dev_pm_attach_by_id(struct device *dev, >>>> + unsigned int index) >>>> +{ >>>> + struct device *genpd_dev; >>>> + int num_domains; >>>> + int ret; >>>> + >>>> + if (!dev->of_node) >>>> + return NULL; >>>> + >>>> + /* Deal only with devices using multiple PM domains. */ >>>> + num_domains = of_count_phandle_with_args(dev->of_node, "power-domains", >>>> + "#power-domain-cells"); >>>> + if (num_domains < 2 || index >= num_domains) >>>> + return NULL; >>>> + >>>> + /* Allocate and register device on the genpd bus. */ >>>> + genpd_dev = kzalloc(sizeof(*genpd_dev), GFP_KERNEL); >>>> + if (!genpd_dev) >>>> + return ERR_PTR(-ENOMEM); >>>> + >>>> + dev_set_name(genpd_dev, "genpd:%u:%s", index, dev_name(dev)); >>>> + genpd_dev->bus = &genpd_bus_type; >>>> + genpd_dev->release = genpd_release_dev; >>>> + >>>> + ret = device_register(genpd_dev); >>>> + if (ret) { >>>> + kfree(genpd_dev); >>>> + return ERR_PTR(ret); >>>> + } >>>> + >>>> + /* Try to attach the device to the PM domain at the specified index. */ >>>> + ret = __genpd_dev_pm_attach(genpd_dev, dev->of_node, index); >>>> + if (ret < 1) { >>>> + device_unregister(genpd_dev); >>>> + return ret ? ERR_PTR(ret) : NULL; >>>> + } >>>> + >>>> + pm_runtime_set_active(genpd_dev); >>>> + pm_runtime_enable(genpd_dev); >>>> + >>>> + return genpd_dev; >>>> +} >>>> +EXPORT_SYMBOL_GPL(genpd_dev_pm_attach_by_id); >>> >>> Thanks for sending this. Believe it or not this has still been on my to-do list >>> and so we definitely need a solution for Tegra. >>> >>> Looking at the above it appears that additional power-domains exposed as devices >>> to the client device. So I assume that this means that the drivers for devices >>> with multiple power-domains will need to call RPM APIs for each of these >>> additional power-domains. Is that correct? >> >> They can, but should not! >> >> Instead, the driver shall use device_link_add() and device_link_del(), >> dynamically, depending on what PM domain that their original device >> needs for the current running use case. >> >> In that way, they keep existing runtime PM deployment, operating on >> its original device. > > OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ then a default linking of all would help.
Rajendra, Jon, On 23 May 2018 at 06:51, Rajendra Nayak <rnayak@codeaurora.org> wrote: > > > On 05/23/2018 02:25 AM, Jon Hunter wrote: >> >> On 22/05/18 15:47, Ulf Hansson wrote: >>> [...] >>> >>>>> >>>>> +/** >>>>> + * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >>>>> + * @dev: Device to attach. >>>>> + * @index: The index of the PM domain. >>>>> + * >>>>> + * Parse device's OF node to find a PM domain specifier at the provided @index. >>>>> + * If such is found, allocates a new device and attaches it to retrieved >>>>> + * pm_domain ops. >>>>> + * >>>>> + * Returns the allocated device if successfully attached PM domain, NULL when >>>>> + * the device don't need a PM domain or have a single PM domain, else PTR_ERR() >>>>> + * in case of failures. Note that if a power-domain exists for the device, but >>>>> + * cannot be found or turned on, then return PTR_ERR(-EPROBE_DEFER) to ensure >>>>> + * that the device is not probed and to re-try again later. >>>>> + */ >>>>> +struct device *genpd_dev_pm_attach_by_id(struct device *dev, >>>>> + unsigned int index) >>>>> +{ >>>>> + struct device *genpd_dev; >>>>> + int num_domains; >>>>> + int ret; >>>>> + >>>>> + if (!dev->of_node) >>>>> + return NULL; >>>>> + >>>>> + /* Deal only with devices using multiple PM domains. */ >>>>> + num_domains = of_count_phandle_with_args(dev->of_node, "power-domains", >>>>> + "#power-domain-cells"); >>>>> + if (num_domains < 2 || index >= num_domains) >>>>> + return NULL; >>>>> + >>>>> + /* Allocate and register device on the genpd bus. */ >>>>> + genpd_dev = kzalloc(sizeof(*genpd_dev), GFP_KERNEL); >>>>> + if (!genpd_dev) >>>>> + return ERR_PTR(-ENOMEM); >>>>> + >>>>> + dev_set_name(genpd_dev, "genpd:%u:%s", index, dev_name(dev)); >>>>> + genpd_dev->bus = &genpd_bus_type; >>>>> + genpd_dev->release = genpd_release_dev; >>>>> + >>>>> + ret = device_register(genpd_dev); >>>>> + if (ret) { >>>>> + kfree(genpd_dev); >>>>> + return ERR_PTR(ret); >>>>> + } >>>>> + >>>>> + /* Try to attach the device to the PM domain at the specified index. */ >>>>> + ret = __genpd_dev_pm_attach(genpd_dev, dev->of_node, index); >>>>> + if (ret < 1) { >>>>> + device_unregister(genpd_dev); >>>>> + return ret ? ERR_PTR(ret) : NULL; >>>>> + } >>>>> + >>>>> + pm_runtime_set_active(genpd_dev); >>>>> + pm_runtime_enable(genpd_dev); >>>>> + >>>>> + return genpd_dev; >>>>> +} >>>>> +EXPORT_SYMBOL_GPL(genpd_dev_pm_attach_by_id); >>>> >>>> Thanks for sending this. Believe it or not this has still been on my to-do list >>>> and so we definitely need a solution for Tegra. >>>> >>>> Looking at the above it appears that additional power-domains exposed as devices >>>> to the client device. So I assume that this means that the drivers for devices >>>> with multiple power-domains will need to call RPM APIs for each of these >>>> additional power-domains. Is that correct? >>> >>> They can, but should not! >>> >>> Instead, the driver shall use device_link_add() and device_link_del(), >>> dynamically, depending on what PM domain that their original device >>> needs for the current running use case. >>> >>> In that way, they keep existing runtime PM deployment, operating on >>> its original device. >> >> OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? > > I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control > at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ > then a default linking of all would help. First, I think we need to decide on *where* the linking should be done, not at both places, as that would just mess up synchronization of who is responsible for calling the device_link_del() at detach. Second, It would in principle be fine to call device_link_add() and device_link_del() as a part of the attach/detach APIs. However, there is a downside to such solution, which would be that the driver then needs call the detach API, just to do device_link_del(). Of course then it would also needs to call the attach API later if/when needed. Doing this adds unnecessary overhead - comparing to just let the driver call device_link_add|del() when needed. On the upside, yes, it would put less burden on the drivers as it then only needs to care about using one set of functions. Which solution do you prefer? Kind regards Uffe
On 23/05/18 07:12, Ulf Hansson wrote: ... >>>>> Thanks for sending this. Believe it or not this has still been on my to-do list >>>>> and so we definitely need a solution for Tegra. >>>>> >>>>> Looking at the above it appears that additional power-domains exposed as devices >>>>> to the client device. So I assume that this means that the drivers for devices >>>>> with multiple power-domains will need to call RPM APIs for each of these >>>>> additional power-domains. Is that correct? >>>> >>>> They can, but should not! >>>> >>>> Instead, the driver shall use device_link_add() and device_link_del(), >>>> dynamically, depending on what PM domain that their original device >>>> needs for the current running use case. >>>> >>>> In that way, they keep existing runtime PM deployment, operating on >>>> its original device. >>> >>> OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? >> >> I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control >> at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ >> then a default linking of all would help. > > First, I think we need to decide on *where* the linking should be > done, not at both places, as that would just mess up synchronization > of who is responsible for calling the device_link_del() at detach. > > Second, It would in principle be fine to call device_link_add() and > device_link_del() as a part of the attach/detach APIs. However, there > is a downside to such solution, which would be that the driver then > needs call the detach API, just to do device_link_del(). Of course > then it would also needs to call the attach API later if/when needed. > Doing this adds unnecessary overhead - comparing to just let the > driver call device_link_add|del() when needed. On the upside, yes, it > would put less burden on the drivers as it then only needs to care > about using one set of functions. > > Which solution do you prefer? Any reason why we could not add a 'boolean' argument to the API to indicate whether the new device should be linked? I think that I prefer the API handles it, but I can see there could be instances where drivers may wish to handle it themselves. Rajendra, do you have a use-case right now where the driver would want to handle the linking? Cheers Jon
On 05/23/2018 02:37 PM, Jon Hunter wrote: > > On 23/05/18 07:12, Ulf Hansson wrote: > > ... > >>>>>> Thanks for sending this. Believe it or not this has still been on my to-do list >>>>>> and so we definitely need a solution for Tegra. >>>>>> >>>>>> Looking at the above it appears that additional power-domains exposed as devices >>>>>> to the client device. So I assume that this means that the drivers for devices >>>>>> with multiple power-domains will need to call RPM APIs for each of these >>>>>> additional power-domains. Is that correct? >>>>> >>>>> They can, but should not! >>>>> >>>>> Instead, the driver shall use device_link_add() and device_link_del(), >>>>> dynamically, depending on what PM domain that their original device >>>>> needs for the current running use case. >>>>> >>>>> In that way, they keep existing runtime PM deployment, operating on >>>>> its original device. >>>> >>>> OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? >>> >>> I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control >>> at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ >>> then a default linking of all would help. >> >> First, I think we need to decide on *where* the linking should be >> done, not at both places, as that would just mess up synchronization >> of who is responsible for calling the device_link_del() at detach. >> >> Second, It would in principle be fine to call device_link_add() and >> device_link_del() as a part of the attach/detach APIs. However, there >> is a downside to such solution, which would be that the driver then >> needs call the detach API, just to do device_link_del(). Of course >> then it would also needs to call the attach API later if/when needed. >> Doing this adds unnecessary overhead - comparing to just let the >> driver call device_link_add|del() when needed. On the upside, yes, it >> would put less burden on the drivers as it then only needs to care >> about using one set of functions. >> >> Which solution do you prefer? > > Any reason why we could not add a 'boolean' argument to the API to indicate whether the new device should be linked? I think that I prefer the API handles it, but I can see there could be instances where drivers may wish to handle it themselves. > > Rajendra, do you have a use-case right now where the driver would want to handle the linking? So if I understand this right, any driver which does want to control individual powerdomain state would need to do the linking itself right? What I am saying is, if I have device A, with powerdomains X and Y, and if I want to turn on only X, then I would want only X to be linked with A, and at a later point if I want both X and Y to be turned on, I would then go ahead and link both X and Y to A? Is that correct or did I get it all wrong? I know atleast Camera on msm8996 would need to do this since it has 2 vfe powerdoamins, which can be turned on one at a time (depending on what resolution needs to be supported) or both together if we really need very high resolution using both vfe modules.
On 23 May 2018 at 11:27, Rajendra Nayak <rnayak@codeaurora.org> wrote: > > > On 05/23/2018 02:37 PM, Jon Hunter wrote: >> >> On 23/05/18 07:12, Ulf Hansson wrote: >> >> ... >> >>>>>>> Thanks for sending this. Believe it or not this has still been on my to-do list >>>>>>> and so we definitely need a solution for Tegra. >>>>>>> >>>>>>> Looking at the above it appears that additional power-domains exposed as devices >>>>>>> to the client device. So I assume that this means that the drivers for devices >>>>>>> with multiple power-domains will need to call RPM APIs for each of these >>>>>>> additional power-domains. Is that correct? >>>>>> >>>>>> They can, but should not! >>>>>> >>>>>> Instead, the driver shall use device_link_add() and device_link_del(), >>>>>> dynamically, depending on what PM domain that their original device >>>>>> needs for the current running use case. >>>>>> >>>>>> In that way, they keep existing runtime PM deployment, operating on >>>>>> its original device. >>>>> >>>>> OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? >>>> >>>> I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control >>>> at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ >>>> then a default linking of all would help. >>> >>> First, I think we need to decide on *where* the linking should be >>> done, not at both places, as that would just mess up synchronization >>> of who is responsible for calling the device_link_del() at detach. >>> >>> Second, It would in principle be fine to call device_link_add() and >>> device_link_del() as a part of the attach/detach APIs. However, there >>> is a downside to such solution, which would be that the driver then >>> needs call the detach API, just to do device_link_del(). Of course >>> then it would also needs to call the attach API later if/when needed. >>> Doing this adds unnecessary overhead - comparing to just let the >>> driver call device_link_add|del() when needed. On the upside, yes, it >>> would put less burden on the drivers as it then only needs to care >>> about using one set of functions. >>> >>> Which solution do you prefer? >> >> Any reason why we could not add a 'boolean' argument to the API to indicate whether the new device should be linked? I think that I prefer the API handles it, but I can see there could be instances where drivers may wish to handle it themselves. >> >> Rajendra, do you have a use-case right now where the driver would want to handle the linking? > > So if I understand this right, any driver which does want to control individual powerdomain state would > need to do the linking itself right? > > What I am saying is, if I have device A, with powerdomains X and Y, and if I want to turn on only X, > then I would want only X to be linked with A, and at a later point if I want both X and Y to be turned on, > I would then go ahead and link both X and Y to A? Is that correct or did I get it all wrong? Correct! > > I know atleast Camera on msm8996 would need to do this since it has 2 vfe powerdoamins, which can be > turned on one at a time (depending on what resolution needs to be supported) or both together if we > really need very high resolution using both vfe modules. I think this is also the case for the Tegra XUSB subsystem. The usb device is always attached to one PM domain, but depending on if super-speed mode is used, another PM domain for that logic needs to be powered on as well. Jon, please correct me if I am wrong! Kind regards Uffe
On 23/05/18 10:33, Ulf Hansson wrote: > On 23 May 2018 at 11:27, Rajendra Nayak <rnayak@codeaurora.org> wrote: >> >> >> On 05/23/2018 02:37 PM, Jon Hunter wrote: >>> >>> On 23/05/18 07:12, Ulf Hansson wrote: >>> >>> ... >>> >>>>>>>> Thanks for sending this. Believe it or not this has still been on my to-do list >>>>>>>> and so we definitely need a solution for Tegra. >>>>>>>> >>>>>>>> Looking at the above it appears that additional power-domains exposed as devices >>>>>>>> to the client device. So I assume that this means that the drivers for devices >>>>>>>> with multiple power-domains will need to call RPM APIs for each of these >>>>>>>> additional power-domains. Is that correct? >>>>>>> >>>>>>> They can, but should not! >>>>>>> >>>>>>> Instead, the driver shall use device_link_add() and device_link_del(), >>>>>>> dynamically, depending on what PM domain that their original device >>>>>>> needs for the current running use case. >>>>>>> >>>>>>> In that way, they keep existing runtime PM deployment, operating on >>>>>>> its original device. >>>>>> >>>>>> OK, sounds good. Any reason why the linking cannot be handled by the above API? Is there a use-case where you would not want it linked? >>>>> >>>>> I am guessing the linking is what would give the driver the ability to decide which subset of powerdomains it actually wants to control >>>>> at any point using runtime PM. If we have cases wherein the driver would want to turn on/off _all_ its associated powerdomains _always_ >>>>> then a default linking of all would help. >>>> >>>> First, I think we need to decide on *where* the linking should be >>>> done, not at both places, as that would just mess up synchronization >>>> of who is responsible for calling the device_link_del() at detach. >>>> >>>> Second, It would in principle be fine to call device_link_add() and >>>> device_link_del() as a part of the attach/detach APIs. However, there >>>> is a downside to such solution, which would be that the driver then >>>> needs call the detach API, just to do device_link_del(). Of course >>>> then it would also needs to call the attach API later if/when needed. >>>> Doing this adds unnecessary overhead - comparing to just let the >>>> driver call device_link_add|del() when needed. On the upside, yes, it >>>> would put less burden on the drivers as it then only needs to care >>>> about using one set of functions. >>>> >>>> Which solution do you prefer? >>> >>> Any reason why we could not add a 'boolean' argument to the API to indicate whether the new device should be linked? I think that I prefer the API handles it, but I can see there could be instances where drivers may wish to handle it themselves. >>> >>> Rajendra, do you have a use-case right now where the driver would want to handle the linking? >> >> So if I understand this right, any driver which does want to control individual powerdomain state would >> need to do the linking itself right? >> >> What I am saying is, if I have device A, with powerdomains X and Y, and if I want to turn on only X, >> then I would want only X to be linked with A, and at a later point if I want both X and Y to be turned on, >> I would then go ahead and link both X and Y to A? Is that correct or did I get it all wrong? > > Correct! > >> >> I know atleast Camera on msm8996 would need to do this since it has 2 vfe powerdoamins, which can be >> turned on one at a time (depending on what resolution needs to be supported) or both together if we >> really need very high resolution using both vfe modules. > > I think this is also the case for the Tegra XUSB subsystem. > > The usb device is always attached to one PM domain, but depending on > if super-speed mode is used, another PM domain for that logic needs to > be powered on as well. > > Jon, please correct me if I am wrong! Yes this is technically correct, however, in reality I think we are always going to enable the superspeed domain if either the host or device domain is enabled. So we would probably always link the superspeed with the host and device devices. Cheers Jon
On 23 May 2018 at 11:45, Jon Hunter <jonathanh@nvidia.com> wrote: > > On 23/05/18 10:33, Ulf Hansson wrote: >> >> On 23 May 2018 at 11:27, Rajendra Nayak <rnayak@codeaurora.org> wrote: >>> >>> >>> >>> On 05/23/2018 02:37 PM, Jon Hunter wrote: >>>> >>>> >>>> On 23/05/18 07:12, Ulf Hansson wrote: >>>> >>>> ... >>>> >>>>>>>>> Thanks for sending this. Believe it or not this has still been on >>>>>>>>> my to-do list >>>>>>>>> and so we definitely need a solution for Tegra. >>>>>>>>> >>>>>>>>> Looking at the above it appears that additional power-domains >>>>>>>>> exposed as devices >>>>>>>>> to the client device. So I assume that this means that the drivers >>>>>>>>> for devices >>>>>>>>> with multiple power-domains will need to call RPM APIs for each of >>>>>>>>> these >>>>>>>>> additional power-domains. Is that correct? >>>>>>>> >>>>>>>> >>>>>>>> They can, but should not! >>>>>>>> >>>>>>>> Instead, the driver shall use device_link_add() and >>>>>>>> device_link_del(), >>>>>>>> dynamically, depending on what PM domain that their original device >>>>>>>> needs for the current running use case. >>>>>>>> >>>>>>>> In that way, they keep existing runtime PM deployment, operating on >>>>>>>> its original device. >>>>>>> >>>>>>> >>>>>>> OK, sounds good. Any reason why the linking cannot be handled by the >>>>>>> above API? Is there a use-case where you would not want it linked? >>>>>> >>>>>> >>>>>> I am guessing the linking is what would give the driver the ability to >>>>>> decide which subset of powerdomains it actually wants to control >>>>>> at any point using runtime PM. If we have cases wherein the driver >>>>>> would want to turn on/off _all_ its associated powerdomains _always_ >>>>>> then a default linking of all would help. >>>>> >>>>> >>>>> First, I think we need to decide on *where* the linking should be >>>>> done, not at both places, as that would just mess up synchronization >>>>> of who is responsible for calling the device_link_del() at detach. >>>>> >>>>> Second, It would in principle be fine to call device_link_add() and >>>>> device_link_del() as a part of the attach/detach APIs. However, there >>>>> is a downside to such solution, which would be that the driver then >>>>> needs call the detach API, just to do device_link_del(). Of course >>>>> then it would also needs to call the attach API later if/when needed. >>>>> Doing this adds unnecessary overhead - comparing to just let the >>>>> driver call device_link_add|del() when needed. On the upside, yes, it >>>>> would put less burden on the drivers as it then only needs to care >>>>> about using one set of functions. >>>>> >>>>> Which solution do you prefer? >>>> >>>> >>>> Any reason why we could not add a 'boolean' argument to the API to >>>> indicate whether the new device should be linked? I think that I prefer the >>>> API handles it, but I can see there could be instances where drivers may >>>> wish to handle it themselves. >>>> >>>> Rajendra, do you have a use-case right now where the driver would want >>>> to handle the linking? >>> >>> >>> So if I understand this right, any driver which does want to control >>> individual powerdomain state would >>> need to do the linking itself right? >>> >>> What I am saying is, if I have device A, with powerdomains X and Y, and >>> if I want to turn on only X, >>> then I would want only X to be linked with A, and at a later point if I >>> want both X and Y to be turned on, >>> I would then go ahead and link both X and Y to A? Is that correct or did >>> I get it all wrong? >> >> >> Correct! >> >>> >>> I know atleast Camera on msm8996 would need to do this since it has 2 vfe >>> powerdoamins, which can be >>> turned on one at a time (depending on what resolution needs to be >>> supported) or both together if we >>> really need very high resolution using both vfe modules. >> >> >> I think this is also the case for the Tegra XUSB subsystem. >> >> The usb device is always attached to one PM domain, but depending on >> if super-speed mode is used, another PM domain for that logic needs to >> be powered on as well. >> >> Jon, please correct me if I am wrong! > > > Yes this is technically correct, however, in reality I think we are always > going to enable the superspeed domain if either the host or device domain is > enabled. So we would probably always link the superspeed with the host and > device devices. Why? Wouldn't that waste power if the superspeed mode isn't used? Kind regards Uffe
On 23/05/18 10:47, Ulf Hansson wrote: > On 23 May 2018 at 11:45, Jon Hunter <jonathanh@nvidia.com> wrote: >> >> On 23/05/18 10:33, Ulf Hansson wrote: >>> >>> On 23 May 2018 at 11:27, Rajendra Nayak <rnayak@codeaurora.org> wrote: >>>> >>>> >>>> >>>> On 05/23/2018 02:37 PM, Jon Hunter wrote: >>>>> >>>>> >>>>> On 23/05/18 07:12, Ulf Hansson wrote: >>>>> >>>>> ... >>>>> >>>>>>>>>> Thanks for sending this. Believe it or not this has still been on >>>>>>>>>> my to-do list >>>>>>>>>> and so we definitely need a solution for Tegra. >>>>>>>>>> >>>>>>>>>> Looking at the above it appears that additional power-domains >>>>>>>>>> exposed as devices >>>>>>>>>> to the client device. So I assume that this means that the drivers >>>>>>>>>> for devices >>>>>>>>>> with multiple power-domains will need to call RPM APIs for each of >>>>>>>>>> these >>>>>>>>>> additional power-domains. Is that correct? >>>>>>>>> >>>>>>>>> >>>>>>>>> They can, but should not! >>>>>>>>> >>>>>>>>> Instead, the driver shall use device_link_add() and >>>>>>>>> device_link_del(), >>>>>>>>> dynamically, depending on what PM domain that their original device >>>>>>>>> needs for the current running use case. >>>>>>>>> >>>>>>>>> In that way, they keep existing runtime PM deployment, operating on >>>>>>>>> its original device. >>>>>>>> >>>>>>>> >>>>>>>> OK, sounds good. Any reason why the linking cannot be handled by the >>>>>>>> above API? Is there a use-case where you would not want it linked? >>>>>>> >>>>>>> >>>>>>> I am guessing the linking is what would give the driver the ability to >>>>>>> decide which subset of powerdomains it actually wants to control >>>>>>> at any point using runtime PM. If we have cases wherein the driver >>>>>>> would want to turn on/off _all_ its associated powerdomains _always_ >>>>>>> then a default linking of all would help. >>>>>> >>>>>> >>>>>> First, I think we need to decide on *where* the linking should be >>>>>> done, not at both places, as that would just mess up synchronization >>>>>> of who is responsible for calling the device_link_del() at detach. >>>>>> >>>>>> Second, It would in principle be fine to call device_link_add() and >>>>>> device_link_del() as a part of the attach/detach APIs. However, there >>>>>> is a downside to such solution, which would be that the driver then >>>>>> needs call the detach API, just to do device_link_del(). Of course >>>>>> then it would also needs to call the attach API later if/when needed. >>>>>> Doing this adds unnecessary overhead - comparing to just let the >>>>>> driver call device_link_add|del() when needed. On the upside, yes, it >>>>>> would put less burden on the drivers as it then only needs to care >>>>>> about using one set of functions. >>>>>> >>>>>> Which solution do you prefer? >>>>> >>>>> >>>>> Any reason why we could not add a 'boolean' argument to the API to >>>>> indicate whether the new device should be linked? I think that I prefer the >>>>> API handles it, but I can see there could be instances where drivers may >>>>> wish to handle it themselves. >>>>> >>>>> Rajendra, do you have a use-case right now where the driver would want >>>>> to handle the linking? >>>> >>>> >>>> So if I understand this right, any driver which does want to control >>>> individual powerdomain state would >>>> need to do the linking itself right? >>>> >>>> What I am saying is, if I have device A, with powerdomains X and Y, and >>>> if I want to turn on only X, >>>> then I would want only X to be linked with A, and at a later point if I >>>> want both X and Y to be turned on, >>>> I would then go ahead and link both X and Y to A? Is that correct or did >>>> I get it all wrong? >>> >>> >>> Correct! >>> >>>> >>>> I know atleast Camera on msm8996 would need to do this since it has 2 vfe >>>> powerdoamins, which can be >>>> turned on one at a time (depending on what resolution needs to be >>>> supported) or both together if we >>>> really need very high resolution using both vfe modules. >>> >>> >>> I think this is also the case for the Tegra XUSB subsystem. >>> >>> The usb device is always attached to one PM domain, but depending on >>> if super-speed mode is used, another PM domain for that logic needs to >>> be powered on as well. >>> >>> Jon, please correct me if I am wrong! >> >> >> Yes this is technically correct, however, in reality I think we are always >> going to enable the superspeed domain if either the host or device domain is >> enabled. So we would probably always link the superspeed with the host and >> device devices. > > Why? Wouldn't that waste power if the superspeed mode isn't used? Simply to reduce complexity. Jon
On 23 May 2018 at 11:07, Jon Hunter <jonathanh@nvidia.com> wrote: > > On 23/05/18 07:12, Ulf Hansson wrote: > > ... > > >>>>>> Thanks for sending this. Believe it or not this has still been on my >>>>>> to-do list >>>>>> and so we definitely need a solution for Tegra. >>>>>> >>>>>> Looking at the above it appears that additional power-domains exposed >>>>>> as devices >>>>>> to the client device. So I assume that this means that the drivers for >>>>>> devices >>>>>> with multiple power-domains will need to call RPM APIs for each of >>>>>> these >>>>>> additional power-domains. Is that correct? >>>>> >>>>> >>>>> They can, but should not! >>>>> >>>>> Instead, the driver shall use device_link_add() and device_link_del(), >>>>> dynamically, depending on what PM domain that their original device >>>>> needs for the current running use case. >>>>> >>>>> In that way, they keep existing runtime PM deployment, operating on >>>>> its original device. >>>> >>>> >>>> OK, sounds good. Any reason why the linking cannot be handled by the >>>> above API? Is there a use-case where you would not want it linked? >>> >>> >>> I am guessing the linking is what would give the driver the ability to >>> decide which subset of powerdomains it actually wants to control >>> at any point using runtime PM. If we have cases wherein the driver would >>> want to turn on/off _all_ its associated powerdomains _always_ >>> then a default linking of all would help. >> >> >> First, I think we need to decide on *where* the linking should be >> done, not at both places, as that would just mess up synchronization >> of who is responsible for calling the device_link_del() at detach. >> >> Second, It would in principle be fine to call device_link_add() and >> device_link_del() as a part of the attach/detach APIs. However, there >> is a downside to such solution, which would be that the driver then >> needs call the detach API, just to do device_link_del(). Of course >> then it would also needs to call the attach API later if/when needed. >> Doing this adds unnecessary overhead - comparing to just let the >> driver call device_link_add|del() when needed. On the upside, yes, it >> would put less burden on the drivers as it then only needs to care >> about using one set of functions. >> >> Which solution do you prefer? > > > Any reason why we could not add a 'boolean' argument to the API to indicate > whether the new device should be linked? I think that I prefer the API > handles it, but I can see there could be instances where drivers may wish to > handle it themselves. Coming back to this question. Both Tegra XUSB and Qcom Camera use case, would benefit from doing the linking themselves, as it needs different PM domains to be powered on depending on the current use case - as to avoid wasting power. However, I can understand that you prefer some simplicity over optimizations, as you told us. Then, does it mean that you are insisting on extending the APIs with a boolean for linking, or are you fine with the driver to call device_link_add()? [...] Kind regards Uffe
On 24/05/18 08:04, Ulf Hansson wrote: ... >> Any reason why we could not add a 'boolean' argument to the API to indicate >> whether the new device should be linked? I think that I prefer the API >> handles it, but I can see there could be instances where drivers may wish to >> handle it themselves. > > Coming back to this question. Both Tegra XUSB and Qcom Camera use > case, would benefit from doing the linking themselves, as it needs > different PM domains to be powered on depending on the current use > case - as to avoid wasting power. > > However, I can understand that you prefer some simplicity over > optimizations, as you told us. Then, does it mean that you are > insisting on extending the APIs with a boolean for linking, or are you > fine with the driver to call device_link_add()? I am fine with the driver calling device_link_add(), but I just wonder if we will find a several drivers doing this and then we will end up doing this later anyway. The current API is called ... * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. * @dev: Device to attach. * @index: The index of the PM domain. This naming and description is a bit misleading, because really it is not attaching the device that is passed, but creating a new device to attach a PM domain to. So we should consider renaming and changing the description and indicate that users need to link the device. Finally, how is a PM domain attached via calling genpd_dev_pm_attach_by_id() detached? Cheers Jon
On 24 May 2018 at 11:36, Jon Hunter <jonathanh@nvidia.com> wrote: > > On 24/05/18 08:04, Ulf Hansson wrote: > > ... > >>> Any reason why we could not add a 'boolean' argument to the API to >>> indicate >>> whether the new device should be linked? I think that I prefer the API >>> handles it, but I can see there could be instances where drivers may wish >>> to >>> handle it themselves. >> >> >> Coming back to this question. Both Tegra XUSB and Qcom Camera use >> case, would benefit from doing the linking themselves, as it needs >> different PM domains to be powered on depending on the current use >> case - as to avoid wasting power. >> >> However, I can understand that you prefer some simplicity over >> optimizations, as you told us. Then, does it mean that you are >> insisting on extending the APIs with a boolean for linking, or are you >> fine with the driver to call device_link_add()? > > > I am fine with the driver calling device_link_add(), but I just wonder if we > will find a several drivers doing this and then we will end up doing this > later anyway. Okay. > > The current API is called ... > > * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. > * @dev: Device to attach. > * @index: The index of the PM domain. > > This naming and description is a bit misleading, because really it is not > attaching the device that is passed, but creating a new device to attach a > PM domain to. So we should consider renaming and changing the description > and indicate that users need to link the device. I picked the name to be consistent with the existing genpd_dev_pm_attach(). Do you have a better suggestion? I agree, some details is missing to the description, let me try to improve it. Actually, I was trying to follow existing descriptions from genpd_dev_pm_attach(), so perhaps that also needs a little update. However, do note that, neither genpd_dev_pm_attach() or genpd_dev_pm_attach_by_id() is supposed to be called by drivers, but rather only by the driver core. So description may not be so important. In regards to good descriptions, for sure the API added in patch9, dev_pm_domain_attach_by_id(), needs a good one, as this is what drivers should be using. > > Finally, how is a PM domain attached via calling genpd_dev_pm_attach_by_id() > detached? Via the existing genpd_dev_pm_detach(), according to what I have described in the change log. I clarify the description in regards to this as well. Kind regards Uffe
On 24/05/18 13:17, Ulf Hansson wrote: > On 24 May 2018 at 11:36, Jon Hunter <jonathanh@nvidia.com> wrote: >> >> On 24/05/18 08:04, Ulf Hansson wrote: >> >> ... >> >>>> Any reason why we could not add a 'boolean' argument to the API to >>>> indicate >>>> whether the new device should be linked? I think that I prefer the API >>>> handles it, but I can see there could be instances where drivers may wish >>>> to >>>> handle it themselves. >>> >>> >>> Coming back to this question. Both Tegra XUSB and Qcom Camera use >>> case, would benefit from doing the linking themselves, as it needs >>> different PM domains to be powered on depending on the current use >>> case - as to avoid wasting power. >>> >>> However, I can understand that you prefer some simplicity over >>> optimizations, as you told us. Then, does it mean that you are >>> insisting on extending the APIs with a boolean for linking, or are you >>> fine with the driver to call device_link_add()? >> >> >> I am fine with the driver calling device_link_add(), but I just wonder if we >> will find a several drivers doing this and then we will end up doing this >> later anyway. > > Okay. > >> >> The current API is called ... >> >> * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >> * @dev: Device to attach. >> * @index: The index of the PM domain. >> >> This naming and description is a bit misleading, because really it is not >> attaching the device that is passed, but creating a new device to attach a >> PM domain to. So we should consider renaming and changing the description >> and indicate that users need to link the device. > > I picked the name to be consistent with the existing > genpd_dev_pm_attach(). Do you have a better suggestion? Well, it appears to get more of a 'get' function and so I don't see why we could not have 'genpd_dev_get_by_id()' and then we could have a genpd_dev_put() as well (which would call genpd_dev_pm_detach). > I agree, some details is missing to the description, let me try to > improve it. Actually, I was trying to follow existing descriptions > from genpd_dev_pm_attach(), so perhaps that also needs a little > update. > > However, do note that, neither genpd_dev_pm_attach() or > genpd_dev_pm_attach_by_id() is supposed to be called by drivers, but > rather only by the driver core. So description may not be so > important. > > In regards to good descriptions, for sure the API added in patch9, > dev_pm_domain_attach_by_id(), needs a good one, as this is what > drivers should be using. OK. Same appears to apply here to the description as I mentioned above. Still seems to be more of a 'get' than an attach. So I wonder if it should be dev_pm_domain_get_by_id() instead? >> Finally, how is a PM domain attached via calling genpd_dev_pm_attach_by_id() >> detached? > > Via the existing genpd_dev_pm_detach(), according to what I have > described in the change log. I clarify the description in regards to > this as well. OK, so this bit is a to-do as that is not yet exposed AFAICT. I see that you said 'although we need to extend it to cover cleanup of the earlier registered device, via calling device_unregister().' So if we do this then that would be fine. Cheers! Jon
On 18/05/18 11:31, Ulf Hansson wrote: > The existing dev_pm_domain_attach() function, allows a single PM domain to > be attached per device. To be able to support devices that are partitioned > across multiple PM domains, let's introduce a new interface, > dev_pm_domain_attach_by_id(). > > The dev_pm_domain_attach_by_id() returns a new allocated struct device with > the corresponding attached PM domain. This enables for example a driver to > operate on the new device from a power management point of view. The driver > may then also benefit from using the received device, to set up so called > device-links towards its original device. Depending on the situation, these > links may then be dynamically changed. > > The new interface is typically called by drivers during their probe phase, > in case they manages devices which uses multiple PM domains. If that is the > case, the driver also becomes responsible of managing the detaching of the > PM domains, which typically should be done at the remove phase. Detaching > is done by calling the existing dev_pm_domain_detach() function and for > each of the received devices from dev_pm_domain_attach_by_id(). > > Note, currently its only genpd that supports multiple PM domains per > device, but dev_pm_domain_attach_by_id() can easily by extended to cover > other PM domain types, if/when needed. > > Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> > --- > drivers/base/power/common.c | 33 ++++++++++++++++++++++++++++++++- > include/linux/pm_domain.h | 7 +++++++ > 2 files changed, 39 insertions(+), 1 deletion(-) > > diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c > index 7ae62b6..d3db974 100644 > --- a/drivers/base/power/common.c > +++ b/drivers/base/power/common.c > @@ -117,13 +117,44 @@ int dev_pm_domain_attach(struct device *dev, bool power_on) > EXPORT_SYMBOL_GPL(dev_pm_domain_attach); > > /** > + * dev_pm_domain_attach_by_id - Attach a device to one of its PM domains. Isn't this more of a 'get'? > + * @index: The index of the PM domain. > + * @dev: Device to attach. Isn't this just the device associated with the PM domain we are getting? > + * > + * As @dev may only be attached to a single PM domain, the backend PM domain > + * provider should create a virtual device to attach instead. As attachment > + * succeeds, the ->detach() callback in the struct dev_pm_domain should be > + * assigned by the corresponding backend attach function. > + * > + * This function should typically be invoked from drivers during probe phase. > + * Especially for those that manages devices which requires power management > + * through more than one PM domain. > + * > + * Callers must ensure proper synchronization of this function with power > + * management callbacks. > + * > + * Returns the virtual attached device in case successfully attached PM domain, > + * NULL in case @dev don't need a PM domain, else a PTR_ERR(). Should this be 'NULL in the case where the @dev already has a power-domain'? > + */ > +struct device *dev_pm_domain_attach_by_id(struct device *dev, > + unsigned int index) > +{ > + if (dev->pm_domain) I wonder if this is worthy of a ... if (WARN_ON(dev->pm_domain)) > + return NULL; Don't we consider this an error case? I wonder why not return PTR_ERR here as well? This would be consistent with dev_pm_domain_attach(). Cheers Jon
On 24 May 2018 at 17:48, Jon Hunter <jonathanh@nvidia.com> wrote: > > On 18/05/18 11:31, Ulf Hansson wrote: >> >> The existing dev_pm_domain_attach() function, allows a single PM domain to >> be attached per device. To be able to support devices that are partitioned >> across multiple PM domains, let's introduce a new interface, >> dev_pm_domain_attach_by_id(). >> >> The dev_pm_domain_attach_by_id() returns a new allocated struct device >> with >> the corresponding attached PM domain. This enables for example a driver to >> operate on the new device from a power management point of view. The >> driver >> may then also benefit from using the received device, to set up so called >> device-links towards its original device. Depending on the situation, >> these >> links may then be dynamically changed. >> >> The new interface is typically called by drivers during their probe phase, >> in case they manages devices which uses multiple PM domains. If that is >> the >> case, the driver also becomes responsible of managing the detaching of the >> PM domains, which typically should be done at the remove phase. Detaching >> is done by calling the existing dev_pm_domain_detach() function and for >> each of the received devices from dev_pm_domain_attach_by_id(). >> >> Note, currently its only genpd that supports multiple PM domains per >> device, but dev_pm_domain_attach_by_id() can easily by extended to cover >> other PM domain types, if/when needed. >> >> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >> --- >> drivers/base/power/common.c | 33 ++++++++++++++++++++++++++++++++- >> include/linux/pm_domain.h | 7 +++++++ >> 2 files changed, 39 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c >> index 7ae62b6..d3db974 100644 >> --- a/drivers/base/power/common.c >> +++ b/drivers/base/power/common.c >> @@ -117,13 +117,44 @@ int dev_pm_domain_attach(struct device *dev, bool >> power_on) >> EXPORT_SYMBOL_GPL(dev_pm_domain_attach); >> /** >> + * dev_pm_domain_attach_by_id - Attach a device to one of its PM domains. > > > Isn't this more of a 'get'? I don't think so, at least according to the common understanding of how we use get and put APIs. For example, clk_get() returns a cookie to a clk, which you then can do a hole bunch of different clk specific operations on. This is different, there are no specific PM domain operations the caller can or should do. Instead the idea is to keep drivers more or less transparent, still using runtime PM as before. The only care the caller need to take is to use device links, which BTW isn't a PM domain specific thing. > >> + * @index: The index of the PM domain. >> + * @dev: Device to attach. > > > Isn't this just the device associated with the PM domain we are getting? Correct, but please note, as stated above, I don't think it's a good idea to return a special PM domain cookie, because we don't want the caller to run special PM domain operations. > >> + * >> + * As @dev may only be attached to a single PM domain, the backend PM >> domain >> + * provider should create a virtual device to attach instead. As >> attachment >> + * succeeds, the ->detach() callback in the struct dev_pm_domain should >> be >> + * assigned by the corresponding backend attach function. >> + * >> + * This function should typically be invoked from drivers during probe >> phase. >> + * Especially for those that manages devices which requires power >> management >> + * through more than one PM domain. >> + * >> + * Callers must ensure proper synchronization of this function with power >> + * management callbacks. >> + * >> + * Returns the virtual attached device in case successfully attached PM >> domain, >> + * NULL in case @dev don't need a PM domain, else a PTR_ERR(). > > > Should this be 'NULL in the case where the @dev already has a power-domain'? Yes, I think so. The intent is to be consistent with how dev_pm_domain_attach() works and according to the latest changes I did for it. It's queued for 4.18, please have a look in Rafael's tree and you will see. > >> + */ >> +struct device *dev_pm_domain_attach_by_id(struct device *dev, >> + unsigned int index) >> +{ >> + if (dev->pm_domain) > > > I wonder if this is worthy of a ... > > if (WARN_ON(dev->pm_domain)) > >> + return NULL; > > > Don't we consider this an error case? I wonder why not return PTR_ERR here > as well? This would be consistent with dev_pm_domain_attach(). Please see above comment. Kind regards Uffe
[...] >>> >>> * genpd_dev_pm_attach_by_id() - Attach a device to one of its PM domain. >>> * @dev: Device to attach. >>> * @index: The index of the PM domain. >>> >>> This naming and description is a bit misleading, because really it is not >>> attaching the device that is passed, but creating a new device to attach >>> a >>> PM domain to. So we should consider renaming and changing the description >>> and indicate that users need to link the device. >> >> >> I picked the name to be consistent with the existing >> genpd_dev_pm_attach(). Do you have a better suggestion? > > > Well, it appears to get more of a 'get' function and so I don't see why we > could not have 'genpd_dev_get_by_id()' and then we could have a > genpd_dev_put() as well (which would call genpd_dev_pm_detach). > >> I agree, some details is missing to the description, let me try to >> improve it. Actually, I was trying to follow existing descriptions >> from genpd_dev_pm_attach(), so perhaps that also needs a little >> update. >> >> However, do note that, neither genpd_dev_pm_attach() or >> genpd_dev_pm_attach_by_id() is supposed to be called by drivers, but >> rather only by the driver core. So description may not be so >> important. >> >> In regards to good descriptions, for sure the API added in patch9, >> dev_pm_domain_attach_by_id(), needs a good one, as this is what >> drivers should be using. > > > OK. Same appears to apply here to the description as I mentioned above. > Still seems to be more of a 'get' than an attach. So I wonder if it should > be dev_pm_domain_get_by_id() instead? Regarding "get" vs "attach", I suggest we continue to discuss that in patch 9. Whatever is decided, $subject patch needs to follow. > >>> Finally, how is a PM domain attached via calling >>> genpd_dev_pm_attach_by_id() >>> detached? >> >> >> Via the existing genpd_dev_pm_detach(), according to what I have >> described in the change log. I clarify the description in regards to >> this as well. > > > OK, so this bit is a to-do as that is not yet exposed AFAICT. I see that you > said 'although we need to extend it to cover cleanup of the earlier > registered device, via calling device_unregister().' So if we do this then > that would be fine. Let me clarify the changelog. It's not a to-do, as it's already done as part of $subject patch. So I guess we are in agreement that we don't need another API to deal with detach? Kind regards Uffe
On 24/05/18 22:21, Ulf Hansson wrote: ... >> OK, so this bit is a to-do as that is not yet exposed AFAICT. I see that you >> said 'although we need to extend it to cover cleanup of the earlier >> registered device, via calling device_unregister().' So if we do this then >> that would be fine. > > Let me clarify the changelog. It's not a to-do, as it's already done > as part of $subject patch. Yes I see it now. OK, then that's fine. Jon
On 24/05/18 22:11, Ulf Hansson wrote: > On 24 May 2018 at 17:48, Jon Hunter <jonathanh@nvidia.com> wrote: >> >> On 18/05/18 11:31, Ulf Hansson wrote: >>> >>> The existing dev_pm_domain_attach() function, allows a single PM domain to >>> be attached per device. To be able to support devices that are partitioned >>> across multiple PM domains, let's introduce a new interface, >>> dev_pm_domain_attach_by_id(). >>> >>> The dev_pm_domain_attach_by_id() returns a new allocated struct device >>> with >>> the corresponding attached PM domain. This enables for example a driver to >>> operate on the new device from a power management point of view. The >>> driver >>> may then also benefit from using the received device, to set up so called >>> device-links towards its original device. Depending on the situation, >>> these >>> links may then be dynamically changed. >>> >>> The new interface is typically called by drivers during their probe phase, >>> in case they manages devices which uses multiple PM domains. If that is >>> the >>> case, the driver also becomes responsible of managing the detaching of the >>> PM domains, which typically should be done at the remove phase. Detaching >>> is done by calling the existing dev_pm_domain_detach() function and for >>> each of the received devices from dev_pm_domain_attach_by_id(). >>> >>> Note, currently its only genpd that supports multiple PM domains per >>> device, but dev_pm_domain_attach_by_id() can easily by extended to cover >>> other PM domain types, if/when needed. >>> >>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >>> --- >>> drivers/base/power/common.c | 33 ++++++++++++++++++++++++++++++++- >>> include/linux/pm_domain.h | 7 +++++++ >>> 2 files changed, 39 insertions(+), 1 deletion(-) >>> >>> diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c >>> index 7ae62b6..d3db974 100644 >>> --- a/drivers/base/power/common.c >>> +++ b/drivers/base/power/common.c >>> @@ -117,13 +117,44 @@ int dev_pm_domain_attach(struct device *dev, bool >>> power_on) >>> EXPORT_SYMBOL_GPL(dev_pm_domain_attach); >>> /** >>> + * dev_pm_domain_attach_by_id - Attach a device to one of its PM domains. >> >> >> Isn't this more of a 'get'? > > I don't think so, at least according to the common understanding of > how we use get and put APIs. > > For example, clk_get() returns a cookie to a clk, which you then can > do a hole bunch of different clk specific operations on. > > This is different, there are no specific PM domain operations the > caller can or should do. Instead the idea is to keep drivers more or > less transparent, still using runtime PM as before. The only care the > caller need to take is to use device links, which BTW isn't a PM > domain specific thing. Yes, but a user can still call pm_runtime_get/put with the device returned if they wish to, right? >> >>> + * @index: The index of the PM domain. >>> + * @dev: Device to attach. >> >> >> Isn't this just the device associated with the PM domain we are getting? > > Correct, but please note, as stated above, I don't think it's a good > idea to return a special PM domain cookie, because we don't want the > caller to run special PM domain operations. > >> >>> + * >>> + * As @dev may only be attached to a single PM domain, the backend PM >>> domain >>> + * provider should create a virtual device to attach instead. As >>> attachment >>> + * succeeds, the ->detach() callback in the struct dev_pm_domain should >>> be >>> + * assigned by the corresponding backend attach function. >>> + * >>> + * This function should typically be invoked from drivers during probe >>> phase. >>> + * Especially for those that manages devices which requires power >>> management >>> + * through more than one PM domain. >>> + * >>> + * Callers must ensure proper synchronization of this function with power >>> + * management callbacks. >>> + * >>> + * Returns the virtual attached device in case successfully attached PM >>> domain, >>> + * NULL in case @dev don't need a PM domain, else a PTR_ERR(). >> >> >> Should this be 'NULL in the case where the @dev already has a power-domain'? > > Yes, I think so. The intent is to be consistent with how > dev_pm_domain_attach() works and according to the latest changes I did > for it. It's queued for 4.18, please have a look in Rafael's tree and > you will see. Ah, I see your change now. >> >>> + */ >>> +struct device *dev_pm_domain_attach_by_id(struct device *dev, >>> + unsigned int index) >>> +{ >>> + if (dev->pm_domain) >> >> >> I wonder if this is worthy of a ... >> >> if (WARN_ON(dev->pm_domain)) >> >>> + return NULL; >> >> >> Don't we consider this an error case? I wonder why not return PTR_ERR here >> as well? This would be consistent with dev_pm_domain_attach(). > > Please see above comment. Right, but this case still seems like an error. My understanding is that only drivers will use this API directly and it will not be used by the device driver core (unlike dev_pm_domain_attach), so if anyone calls this attempting to attach another PM domain when one is already attached, they are doing something wrong. Cheers Jon
On 25 May 2018 at 10:31, Jon Hunter <jonathanh@nvidia.com> wrote: > > On 24/05/18 22:11, Ulf Hansson wrote: >> >> On 24 May 2018 at 17:48, Jon Hunter <jonathanh@nvidia.com> wrote: >>> >>> >>> On 18/05/18 11:31, Ulf Hansson wrote: >>>> >>>> >>>> The existing dev_pm_domain_attach() function, allows a single PM domain >>>> to >>>> be attached per device. To be able to support devices that are >>>> partitioned >>>> across multiple PM domains, let's introduce a new interface, >>>> dev_pm_domain_attach_by_id(). >>>> >>>> The dev_pm_domain_attach_by_id() returns a new allocated struct device >>>> with >>>> the corresponding attached PM domain. This enables for example a driver >>>> to >>>> operate on the new device from a power management point of view. The >>>> driver >>>> may then also benefit from using the received device, to set up so >>>> called >>>> device-links towards its original device. Depending on the situation, >>>> these >>>> links may then be dynamically changed. >>>> >>>> The new interface is typically called by drivers during their probe >>>> phase, >>>> in case they manages devices which uses multiple PM domains. If that is >>>> the >>>> case, the driver also becomes responsible of managing the detaching of >>>> the >>>> PM domains, which typically should be done at the remove phase. >>>> Detaching >>>> is done by calling the existing dev_pm_domain_detach() function and for >>>> each of the received devices from dev_pm_domain_attach_by_id(). >>>> >>>> Note, currently its only genpd that supports multiple PM domains per >>>> device, but dev_pm_domain_attach_by_id() can easily by extended to cover >>>> other PM domain types, if/when needed. >>>> >>>> Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org> >>>> --- >>>> drivers/base/power/common.c | 33 ++++++++++++++++++++++++++++++++- >>>> include/linux/pm_domain.h | 7 +++++++ >>>> 2 files changed, 39 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/base/power/common.c b/drivers/base/power/common.c >>>> index 7ae62b6..d3db974 100644 >>>> --- a/drivers/base/power/common.c >>>> +++ b/drivers/base/power/common.c >>>> @@ -117,13 +117,44 @@ int dev_pm_domain_attach(struct device *dev, bool >>>> power_on) >>>> EXPORT_SYMBOL_GPL(dev_pm_domain_attach); >>>> /** >>>> + * dev_pm_domain_attach_by_id - Attach a device to one of its PM >>>> domains. >>> >>> >>> >>> Isn't this more of a 'get'? >> >> >> I don't think so, at least according to the common understanding of >> how we use get and put APIs. >> >> For example, clk_get() returns a cookie to a clk, which you then can >> do a hole bunch of different clk specific operations on. >> >> This is different, there are no specific PM domain operations the >> caller can or should do. Instead the idea is to keep drivers more or >> less transparent, still using runtime PM as before. The only care the >> caller need to take is to use device links, which BTW isn't a PM >> domain specific thing. > > > Yes, but a user can still call pm_runtime_get/put with the device returned > if they wish to, right? Correct. [...] >>>> + */ >>>> +struct device *dev_pm_domain_attach_by_id(struct device *dev, >>>> + unsigned int index) >>>> +{ >>>> + if (dev->pm_domain) >>> >>> >>> >>> I wonder if this is worthy of a ... >>> >>> if (WARN_ON(dev->pm_domain)) >>> >>>> + return NULL; >>> >>> >>> >>> Don't we consider this an error case? I wonder why not return PTR_ERR >>> here >>> as well? This would be consistent with dev_pm_domain_attach(). >> >> >> Please see above comment. > > > Right, but this case still seems like an error. My understanding is that > only drivers will use this API directly and it will not be used by the > device driver core (unlike dev_pm_domain_attach), so if anyone calls this > attempting to attach another PM domain when one is already attached, they > are doing something wrong. [...] You may be right! What I was thinking of is whether multiple PM domains may be optional in some cases, but instead a PM domain have already been attached by dev_pm_domain_attach(), prior the driver starts to probe. Then, assuming we return an error for this case, that means the caller then need to check the dev->pm_domain pointer, prior calling dev_pm_domain_attach_by_id(). Wouldn't it? Perhaps that is more clear though? Kind regards Uffe
On 25/05/18 11:45, Ulf Hansson wrote: ... >> Right, but this case still seems like an error. My understanding is that >> only drivers will use this API directly and it will not be used by the >> device driver core (unlike dev_pm_domain_attach), so if anyone calls this >> attempting to attach another PM domain when one is already attached, they >> are doing something wrong. > > [...] > > You may be right! > > What I was thinking of is whether multiple PM domains may be optional > in some cases, but instead a PM domain have already been attached by > dev_pm_domain_attach(), prior the driver starts to probe. > > Then, assuming we return an error for this case, that means the caller > then need to check the dev->pm_domain pointer, prior calling > dev_pm_domain_attach_by_id(). Wouldn't it? Perhaps that is more clear > though? IMO the driver should know whether is needs multiple power-domains or not and if it needs multiple then it should just call dev_pm_domain_attach_by_id() N times without needing to checking dev->pm_domain first. If it fails then either the PM domain core did something wrong or power-domains are missing from DT, but either way there is an error, so let it fail. Cheers Jon
On 25 May 2018 at 13:07, Jon Hunter <jonathanh@nvidia.com> wrote: > > > On 25/05/18 11:45, Ulf Hansson wrote: > > ... > >>> Right, but this case still seems like an error. My understanding is that >>> only drivers will use this API directly and it will not be used by the >>> device driver core (unlike dev_pm_domain_attach), so if anyone calls this >>> attempting to attach another PM domain when one is already attached, they >>> are doing something wrong. >> >> >> [...] >> >> You may be right! >> >> What I was thinking of is whether multiple PM domains may be optional >> in some cases, but instead a PM domain have already been attached by >> dev_pm_domain_attach(), prior the driver starts to probe. >> >> Then, assuming we return an error for this case, that means the caller >> then need to check the dev->pm_domain pointer, prior calling >> dev_pm_domain_attach_by_id(). Wouldn't it? Perhaps that is more clear >> though? > > > IMO the driver should know whether is needs multiple power-domains or not > and if it needs multiple then it should just call > dev_pm_domain_attach_by_id() N times without needing to checking > dev->pm_domain first. If it fails then either the PM domain core did > something wrong or power-domains are missing from DT, but either way there > is an error, so let it fail. Right, sounds reasonable! Kind regards Uffe
There are devices that are partitioned across multiple PM domains. Currently these can't be supported well by the available PM infrastructures we have in the kernel. This series is an attempt to address this. The interesting parts happens from patch 5 an onwards, including a minor DT update to the existing power-domain bindings, the 4 earlier are just trivial clean-ups of some related code in genpd, which I happened to stumble over. Some additional background: One existing case where devices are partitioned across multiple PM domains, is the Nvida Tegra 124/210 X-USB subsystem. A while ago Jon Hunter (Nvidia) sent a series, trying to address these issues, however this is a new approach, while it re-uses the same concepts from DT point of view. The Tegra 124/210 X-USB subsystem contains of a host controller and a device controller. Each controller have its own independent PM domain, but are being partitioned across another shared PM domain for the USB super-speed logic. Currently to make the drivers work, either the related PM domains needs to stay powered on always or the PM domain topology needs to be in-correctly modelled through sub-domains. In both cases PM domains may be powered on while they don't need to be, so in the end this means - wasting power -. As stated above, this series intends to address these problem from a PM infrastructure point of view. More details are available in each changelog. It should be noted that this series has been tested on HW, however only by using a home-cooked test PM domain driver for genpd and together with a test driver. This allowed me to play with PM domain (genpd), runtime PM and device links. Any further deployment for real use cases are greatly appreciated. I am happy to to help, if needed! Kind regards Ulf Hansson Ulf Hansson (9): PM / Domains: Drop extern declarations of functions in pm_domain.h PM / Domains: Drop __pm_genpd_add_device() PM / Domains: Drop genpd as in-param for pm_genpd_remove_device() PM / Domains: Drop unused parameter in genpd_allocate_dev_data() PM / Domains: dt: Allow power-domain property to be a list of phandles PM / Domains: Don't attach devices in genpd with multi PM domains PM / Domains: Split genpd_dev_pm_attach() PM / Domains: Add support for multi PM domains per device to genpd PM / Domains: Add dev_pm_domain_attach_by_id() to manage multi PM domains .../devicetree/bindings/power/power_domain.txt | 18 ++- drivers/base/power/common.c | 33 ++++- drivers/base/power/domain.c | 154 ++++++++++++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c | 2 +- include/linux/pm_domain.h | 79 +++++------ 5 files changed, 208 insertions(+), 78 deletions(-)