[RFC,2/3] PM / Domains: Add support for devices with multiple domains

Submitted by Jon Hunter on Sept. 20, 2016, 10:28 a.m.

Details

Message ID 1474367287-10402-3-git-send-email-jonathanh@nvidia.com
State New
Headers show

Commit Message

Jon Hunter Sept. 20, 2016, 10:28 a.m.
Some devices may require more than one PM domain to operate and this is
not currently by the PM domain framework. Furthermore, the current Linux
'device' structure only allows devices to be associated with a single PM
domain and so cannot easily be associated with more than one. To allow
devices to be associated with more than one PM domain, if multiple
domains are defined for a given device (eg. via device-tree), then:
1. Create a new PM domain for this device. The name of the new PM domain
   created matches the device name for which it was created for.
2. Register the new PM domain as a sub-domain for all PM domains
   required by the device.
3. Attach the device to the new PM domain.

By default the newly created PM domain is assumed to be in the 'off'
state to ensure that any parent PM domains will be turned on if not
already on when the new PM domain is turned on.

When a device associated with more than one PM domain is detached,
wait for any power-off work to complete, then remove the PM domain that
was created for the device by calling pm_genpd_remove() (this also
removes it as a child to any other PM domains) and free the memory
for the PM domain.

For devices using device-tree, devices that require multiple PM domains
are detected by seeing if the 'power-domains' property has more than one
entry defined.

Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
---

Here is an example output from pm_genpd_summary following this change
for the Tegra210 XHCI device:

domain                          status          slaves
    /device                                             runtime status
----------------------------------------------------------------------
70090000.usb                    on
    /devices/platform/70090000.usb                      unsupported
xusbc                           on              70090000.usb
xusbb                           off-0
xusba                           on              70090000.usb

I am not sure if this is confusing to have a device and domain with the
same name. So let me know if you have any thoughts!


 drivers/base/power/domain.c | 102 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 88 insertions(+), 14 deletions(-)

Comments

Jon Hunter Sept. 20, 2016, 5:54 p.m.
On 20/09/16 11:28, Jon Hunter wrote:
> Some devices may require more than one PM domain to operate and this is
> not currently by the PM domain framework. Furthermore, the current Linux
> 'device' structure only allows devices to be associated with a single PM
> domain and so cannot easily be associated with more than one. To allow
> devices to be associated with more than one PM domain, if multiple
> domains are defined for a given device (eg. via device-tree), then:
> 1. Create a new PM domain for this device. The name of the new PM domain
>    created matches the device name for which it was created for.
> 2. Register the new PM domain as a sub-domain for all PM domains
>    required by the device.
> 3. Attach the device to the new PM domain.
> 
> By default the newly created PM domain is assumed to be in the 'off'
> state to ensure that any parent PM domains will be turned on if not
> already on when the new PM domain is turned on.
> 
> When a device associated with more than one PM domain is detached,
> wait for any power-off work to complete, then remove the PM domain that
> was created for the device by calling pm_genpd_remove() (this also
> removes it as a child to any other PM domains) and free the memory
> for the PM domain.
> 
> For devices using device-tree, devices that require multiple PM domains
> are detected by seeing if the 'power-domains' property has more than one
> entry defined.
> 
> Signed-off-by: Jon Hunter <jonathanh@nvidia.com>
> ---
> 
> Here is an example output from pm_genpd_summary following this change
> for the Tegra210 XHCI device:
> 
> domain                          status          slaves
>     /device                                             runtime status
> ----------------------------------------------------------------------
> 70090000.usb                    on
>     /devices/platform/70090000.usb                      unsupported
> xusbc                           on              70090000.usb
> xusbb                           off-0
> xusba                           on              70090000.usb
> 
> I am not sure if this is confusing to have a device and domain with the
> same name. So let me know if you have any thoughts!
> 
> 
>  drivers/base/power/domain.c | 102 ++++++++++++++++++++++++++++++++++++++------
>  1 file changed, 88 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
> index 382735949591..ee39824c03b3 100644
> --- a/drivers/base/power/domain.c
> +++ b/drivers/base/power/domain.c
> @@ -1826,7 +1826,7 @@ static void genpd_dev_pm_detach(struct device *dev, bool power_off)
>  {
>  	struct generic_pm_domain *pd;
>  	unsigned int i;
> -	int ret = 0;
> +	int count, ret = 0;
>  
>  	pd = genpd_lookup_dev(dev);
>  	if (!pd)
> @@ -1851,6 +1851,19 @@ static void genpd_dev_pm_detach(struct device *dev, bool power_off)
>  
>  	/* Check if PM domain can be powered off after removing this device. */
>  	genpd_queue_power_off_work(pd);
> +
> +	count = of_count_phandle_with_args(dev->of_node, "power-domains",
> +					   "#power-domain-cells");
> +	if (count > 1) {
> +		cancel_work_sync(&pd->power_off_work);
> +
> +		ret = pm_genpd_remove(pd);
> +		if (ret < 0)
> +			dev_err(dev, "failed to remove PM domain %s: %d\n",
> +				pd->name, ret);
> +
> +		kfree(pd);

I realise I am missing a return if pm_genpd_remove() returns a failure!
Will correct this.

Jon
Geert Uytterhoeven Sept. 21, 2016, 8:53 a.m.
Hi Jon,

On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
> Some devices may require more than one PM domain to operate and this is
> not currently by the PM domain framework. Furthermore, the current Linux
> 'device' structure only allows devices to be associated with a single PM
> domain and so cannot easily be associated with more than one. To allow
> devices to be associated with more than one PM domain, if multiple
> domains are defined for a given device (eg. via device-tree), then:
> 1. Create a new PM domain for this device. The name of the new PM domain
>    created matches the device name for which it was created for.
> 2. Register the new PM domain as a sub-domain for all PM domains
>    required by the device.
> 3. Attach the device to the new PM domain.

This looks a suboptimal to me: if you have n devices sharing the same PM
domains, you would add n new subdomains?

Having a clean way to specify multiple PM domains is very useful, though.

E.g. on Renesas ARM SoCs, devices are usually part of two PM domains:
  1. A power area (can be "always-on",
  2. The clock domain.

As power areas and clock domains are fairly orthogonal (the former use the
.power_{off,on}() callbacks, the latter set GENPD_FLAG_PM_CLK and use the
{at,de}tach_dev() callbacks), we currently setup both in the same driver
(SYSC, for controlling power areas), which forwards the clock domain operations
to the clock driver (CPG/MSTP or CPG/MSSR).
Hence we have only single references in the power-domains properties, but
having two would allow to drop the hardcoded links between the two drivers.

(Oh no, more DT backwards compatibility issues if this is accepted ;-)

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jon Hunter Sept. 21, 2016, 10:01 a.m.
Hi Geert,

On 21/09/16 09:53, Geert Uytterhoeven wrote:
> Hi Jon,
> 
> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>> Some devices may require more than one PM domain to operate and this is
>> not currently by the PM domain framework. Furthermore, the current Linux
>> 'device' structure only allows devices to be associated with a single PM
>> domain and so cannot easily be associated with more than one. To allow
>> devices to be associated with more than one PM domain, if multiple
>> domains are defined for a given device (eg. via device-tree), then:
>> 1. Create a new PM domain for this device. The name of the new PM domain
>>    created matches the device name for which it was created for.
>> 2. Register the new PM domain as a sub-domain for all PM domains
>>    required by the device.
>> 3. Attach the device to the new PM domain.
> 
> This looks a suboptimal to me: if you have n devices sharing the same PM
> domains, you would add n new subdomains?

Yes you would and so in that case it would appear to be suboptimal, I
agree. One option would be to name the new PM domain after its parents
and then see if any PM domains exist that matches the name and verify it
has the required parents. We could even add a prefix to the name to
indicate that this is a PM domain added by the core. Only problem is we
could get some long names!

> Having a clean way to specify multiple PM domains is very useful, though.
> 
> E.g. on Renesas ARM SoCs, devices are usually part of two PM domains:
>   1. A power area (can be "always-on",
>   2. The clock domain.
> 
> As power areas and clock domains are fairly orthogonal (the former use the
> .power_{off,on}() callbacks, the latter set GENPD_FLAG_PM_CLK and use the
> {at,de}tach_dev() callbacks), we currently setup both in the same driver
> (SYSC, for controlling power areas), which forwards the clock domain operations
> to the clock driver (CPG/MSTP or CPG/MSSR).
> Hence we have only single references in the power-domains properties, but
> having two would allow to drop the hardcoded links between the two drivers.

Glad to see others could benefit from this ...

> (Oh no, more DT backwards compatibility issues if this is accepted ;-)

... despite DT compat issues :-(

Jon
Jon Hunter Sept. 21, 2016, 2:37 p.m.
Hi Geert,

On 21/09/16 09:53, Geert Uytterhoeven wrote:
> Hi Jon,
> 
> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>> Some devices may require more than one PM domain to operate and this is
>> not currently by the PM domain framework. Furthermore, the current Linux
>> 'device' structure only allows devices to be associated with a single PM
>> domain and so cannot easily be associated with more than one. To allow
>> devices to be associated with more than one PM domain, if multiple
>> domains are defined for a given device (eg. via device-tree), then:
>> 1. Create a new PM domain for this device. The name of the new PM domain
>>    created matches the device name for which it was created for.
>> 2. Register the new PM domain as a sub-domain for all PM domains
>>    required by the device.
>> 3. Attach the device to the new PM domain.
> 
> This looks a suboptimal to me: if you have n devices sharing the same PM
> domains, you would add n new subdomains?

BTW, would this be the case today for some renesas devices or are you
just pointing this out as something that could be optimised/improved?

Cheers
Jon
Geert Uytterhoeven Sept. 21, 2016, 2:57 p.m.
Hi Jon,

On Wed, Sep 21, 2016 at 4:37 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
> On 21/09/16 09:53, Geert Uytterhoeven wrote:
>> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>> Some devices may require more than one PM domain to operate and this is
>>> not currently by the PM domain framework. Furthermore, the current Linux
>>> 'device' structure only allows devices to be associated with a single PM
>>> domain and so cannot easily be associated with more than one. To allow
>>> devices to be associated with more than one PM domain, if multiple
>>> domains are defined for a given device (eg. via device-tree), then:
>>> 1. Create a new PM domain for this device. The name of the new PM domain
>>>    created matches the device name for which it was created for.
>>> 2. Register the new PM domain as a sub-domain for all PM domains
>>>    required by the device.
>>> 3. Attach the device to the new PM domain.
>>
>> This looks a suboptimal to me: if you have n devices sharing the same PM
>> domains, you would add n new subdomains?
>
> BTW, would this be the case today for some renesas devices or are you
> just pointing this out as something that could be optimised/improved?

This is the case for all Renesas SoCs that have power areas: devices belong
to both the PM domain for the power area, and to the PM domain for the clock
domain.

See setting of .attach_dev in
    arch/arm/mach-shmobile/pm-rmobile.c (power areas + clock domain)
    drivers/clk/renesas/clk-mstp.c (pure clock domain)
    drivers/clk/renesas/renesas-cpg-mssr.c (pure clock domain)
    drivers/soc/renesas/rcar-sysc.c (power areas + clock domain)

AFAIK there are no devices that belong to multiple power areas (power areas may
be nested though, which is handled fine by the current framework, as created by
Rafael when he worked on Renesas SoCs).

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jon Hunter Sept. 23, 2016, 12:57 p.m.
Hi Geert,

On 21/09/16 15:57, Geert Uytterhoeven wrote:
> Hi Jon,
> 
> On Wed, Sep 21, 2016 at 4:37 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>> On 21/09/16 09:53, Geert Uytterhoeven wrote:
>>> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>>> Some devices may require more than one PM domain to operate and this is
>>>> not currently by the PM domain framework. Furthermore, the current Linux
>>>> 'device' structure only allows devices to be associated with a single PM
>>>> domain and so cannot easily be associated with more than one. To allow
>>>> devices to be associated with more than one PM domain, if multiple
>>>> domains are defined for a given device (eg. via device-tree), then:
>>>> 1. Create a new PM domain for this device. The name of the new PM domain
>>>>    created matches the device name for which it was created for.
>>>> 2. Register the new PM domain as a sub-domain for all PM domains
>>>>    required by the device.
>>>> 3. Attach the device to the new PM domain.
>>>
>>> This looks a suboptimal to me: if you have n devices sharing the same PM
>>> domains, you would add n new subdomains?
>>
>> BTW, would this be the case today for some renesas devices or are you
>> just pointing this out as something that could be optimised/improved?
> 
> This is the case for all Renesas SoCs that have power areas: devices belong
> to both the PM domain for the power area, and to the PM domain for the clock
> domain.

To quantify this a bit, for the Renesas case, how many of these
duplicated domains would there be if you were to use this approach as-is?

I would like to see some agreement about whether we would allow the
'power-domains' property to have more than one power-domain. We could
always improve the implementation in the future. I am quite happy to
re-work this RFC to avoid duplicated domains for devices like Renesas if
people are on board with the overall proposal.

Cheers
Jon
Geert Uytterhoeven Sept. 23, 2016, 2:27 p.m.
Hi Jon,

On Fri, Sep 23, 2016 at 2:57 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
> On 21/09/16 15:57, Geert Uytterhoeven wrote:
>> On Wed, Sep 21, 2016 at 4:37 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>> On 21/09/16 09:53, Geert Uytterhoeven wrote:
>>>> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>>>> Some devices may require more than one PM domain to operate and this is
>>>>> not currently by the PM domain framework. Furthermore, the current Linux
>>>>> 'device' structure only allows devices to be associated with a single PM
>>>>> domain and so cannot easily be associated with more than one. To allow
>>>>> devices to be associated with more than one PM domain, if multiple
>>>>> domains are defined for a given device (eg. via device-tree), then:
>>>>> 1. Create a new PM domain for this device. The name of the new PM domain
>>>>>    created matches the device name for which it was created for.
>>>>> 2. Register the new PM domain as a sub-domain for all PM domains
>>>>>    required by the device.
>>>>> 3. Attach the device to the new PM domain.
>>>>
>>>> This looks a suboptimal to me: if you have n devices sharing the same PM
>>>> domains, you would add n new subdomains?
>>>
>>> BTW, would this be the case today for some renesas devices or are you
>>> just pointing this out as something that could be optimised/improved?
>>
>> This is the case for all Renesas SoCs that have power areas: devices belong
>> to both the PM domain for the power area, and to the PM domain for the clock
>> domain.
>
> To quantify this a bit, for the Renesas case, how many of these
> duplicated domains would there be if you were to use this approach as-is?

for i in $(git grep -l renesas, -- "*dts*") ; do echo --- $i ---; git
grep -w power-domains $i | sort | uniq -c | sort -n;done

tells you how many (supported) devices are (currently) present in each
PM domain.
Most of these (all but devices in CPU/SCU power areas) are also part of a
clock domain.
The synthetic R8A779*_PD_ALWAYS_ON domains could be dropped again,
as we could just refer to the CPG/MSSR node for the clock domain instead.

For older SH/R-Mobile SoCs with lots of hierarchical domains, that gives us,
after removing the above:

      1 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4mp>;
      1 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_d4>;
      2 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_c5>;
      3 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4r>;
      6 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4s>;
     15 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a3sp>;

R-Car Gen1/Gen2 have all devices in the "always on" PM domain, so they're
not affected.

R-Car Gen3 again has devices in power areas, mostly for graphics related
purposes:

     16 arch/arm64/boot/dts/renesas/r8a7795.dtsi:
 power-domains = <&sysc R8A7795_PD_A3VP>;

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jon Hunter Sept. 30, 2016, 8:05 a.m.
Hi PM posse!

On 23/09/16 15:27, Geert Uytterhoeven wrote:
> Hi Jon,
> 
> On Fri, Sep 23, 2016 at 2:57 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>> On 21/09/16 15:57, Geert Uytterhoeven wrote:
>>> On Wed, Sep 21, 2016 at 4:37 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>>> On 21/09/16 09:53, Geert Uytterhoeven wrote:
>>>>> On Tue, Sep 20, 2016 at 12:28 PM, Jon Hunter <jonathanh@nvidia.com> wrote:
>>>>>> Some devices may require more than one PM domain to operate and this is
>>>>>> not currently by the PM domain framework. Furthermore, the current Linux
>>>>>> 'device' structure only allows devices to be associated with a single PM
>>>>>> domain and so cannot easily be associated with more than one. To allow
>>>>>> devices to be associated with more than one PM domain, if multiple
>>>>>> domains are defined for a given device (eg. via device-tree), then:
>>>>>> 1. Create a new PM domain for this device. The name of the new PM domain
>>>>>>    created matches the device name for which it was created for.
>>>>>> 2. Register the new PM domain as a sub-domain for all PM domains
>>>>>>    required by the device.
>>>>>> 3. Attach the device to the new PM domain.
>>>>>
>>>>> This looks a suboptimal to me: if you have n devices sharing the same PM
>>>>> domains, you would add n new subdomains?
>>>>
>>>> BTW, would this be the case today for some renesas devices or are you
>>>> just pointing this out as something that could be optimised/improved?
>>>
>>> This is the case for all Renesas SoCs that have power areas: devices belong
>>> to both the PM domain for the power area, and to the PM domain for the clock
>>> domain.
>>
>> To quantify this a bit, for the Renesas case, how many of these
>> duplicated domains would there be if you were to use this approach as-is?
> 
> for i in $(git grep -l renesas, -- "*dts*") ; do echo --- $i ---; git
> grep -w power-domains $i | sort | uniq -c | sort -n;done
> 
> tells you how many (supported) devices are (currently) present in each
> PM domain.
> Most of these (all but devices in CPU/SCU power areas) are also part of a
> clock domain.
> The synthetic R8A779*_PD_ALWAYS_ON domains could be dropped again,
> as we could just refer to the CPG/MSSR node for the clock domain instead.
> 
> For older SH/R-Mobile SoCs with lots of hierarchical domains, that gives us,
> after removing the above:
> 
>       1 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4mp>;
>       1 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_d4>;
>       2 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_c5>;
>       3 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4r>;
>       6 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a4s>;
>      15 arch/arm/boot/dts/r8a7740.dtsi:         power-domains = <&pd_a3sp>;
> 
> R-Car Gen1/Gen2 have all devices in the "always on" PM domain, so they're
> not affected.
> 
> R-Car Gen3 again has devices in power areas, mostly for graphics related
> purposes:
> 
>      16 arch/arm64/boot/dts/renesas/r8a7795.dtsi:
>  power-domains = <&sysc R8A7795_PD_A3VP>;

Does anyone have any more inputs comments on this? Does it look complete
bonkers or should I forge ahead with this?

Cheers
Jon
Kevin Hilman Oct. 7, 2016, 9:14 a.m.
Jon Hunter <jonathanh@nvidia.com> writes:

> Some devices may require more than one PM domain to operate and this is
> not currently by the PM domain framework. Furthermore, the current Linux
> 'device' structure only allows devices to be associated with a single PM
> domain and so cannot easily be associated with more than one. To allow
> devices to be associated with more than one PM domain, if multiple
> domains are defined for a given device (eg. via device-tree), then:
> 1. Create a new PM domain for this device. The name of the new PM domain
>    created matches the device name for which it was created for.
> 2. Register the new PM domain as a sub-domain for all PM domains
>    required by the device.
> 3. Attach the device to the new PM domain.

Did you look at what might be involved to extend struct device to hace a
list of pm_domains?  Like Ulf, I'm a bit unsettled by this
implementation that has to work around the basic limitation in the
driver model.

Having devices in multitple domains is needed for SoCs I'm familiar with
also, so is a needed feature.

I think removing the struct device limitation and corresponding
assumptions in the driver and PM core is a prerequisite for this
feature.

Doing that will lead to several questions about how to handle runtime PM
operations (e.g. which of the multiple PM domains should the one to call
the drivers runtime PM hooks when a device changes runtime PM state?)

Anyways, even with the potential complexities, I think attempting this
is the right way forward.

Kevin
--
To unsubscribe from this list: send the line "unsubscribe linux-tegra" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jon Hunter Oct. 10, 2016, 11:24 a.m.
On 07/10/16 10:14, Kevin Hilman wrote:
> Jon Hunter <jonathanh@nvidia.com> writes:
> 
>> Some devices may require more than one PM domain to operate and this is
>> not currently by the PM domain framework. Furthermore, the current Linux
>> 'device' structure only allows devices to be associated with a single PM
>> domain and so cannot easily be associated with more than one. To allow
>> devices to be associated with more than one PM domain, if multiple
>> domains are defined for a given device (eg. via device-tree), then:
>> 1. Create a new PM domain for this device. The name of the new PM domain
>>    created matches the device name for which it was created for.
>> 2. Register the new PM domain as a sub-domain for all PM domains
>>    required by the device.
>> 3. Attach the device to the new PM domain.
> 
> Did you look at what might be involved to extend struct device to hace a
> list of pm_domains?  Like Ulf, I'm a bit unsettled by this
> implementation that has to work around the basic limitation in the
> driver model.

I had but it was going to be a much bigger and intrusive change. So I
went with this as a initial idea to see if others also had a need for
it. I am happy to start looking at extended the device struct if this is
the preferred path and I would agree that would make most sense.

> Having devices in multitple domains is needed for SoCs I'm familiar with
> also, so is a needed feature.

Ok great.

> I think removing the struct device limitation and corresponding
> assumptions in the driver and PM core is a prerequisite for this
> feature.
> 
> Doing that will lead to several questions about how to handle runtime PM
> operations (e.g. which of the multiple PM domains should the one to call
> the drivers runtime PM hooks when a device changes runtime PM state?)

Right. My initial thought would be that at least for device-tree based
configuration, that the order in which the pm-domains are defined in DT
would determine the order in which the pm-domains are power-on/off.

> Anyways, even with the potential complexities, I think attempting this
> is the right way forward.

Ok. I will start having a think about this but will probably not get
back to this for a few weeks.

Cheers
Jon

Patch hide | download patch | download mbox

diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
index 382735949591..ee39824c03b3 100644
--- a/drivers/base/power/domain.c
+++ b/drivers/base/power/domain.c
@@ -1826,7 +1826,7 @@  static void genpd_dev_pm_detach(struct device *dev, bool power_off)
 {
 	struct generic_pm_domain *pd;
 	unsigned int i;
-	int ret = 0;
+	int count, ret = 0;
 
 	pd = genpd_lookup_dev(dev);
 	if (!pd)
@@ -1851,6 +1851,19 @@  static void genpd_dev_pm_detach(struct device *dev, bool power_off)
 
 	/* Check if PM domain can be powered off after removing this device. */
 	genpd_queue_power_off_work(pd);
+
+	count = of_count_phandle_with_args(dev->of_node, "power-domains",
+					   "#power-domain-cells");
+	if (count > 1) {
+		cancel_work_sync(&pd->power_off_work);
+
+		ret = pm_genpd_remove(pd);
+		if (ret < 0)
+			dev_err(dev, "failed to remove PM domain %s: %d\n",
+				pd->name, ret);
+
+		kfree(pd);
+	}
 }
 
 static void genpd_dev_pm_sync(struct device *dev)
@@ -1898,6 +1911,73 @@  static int genpd_dev_pm_attach_device(struct device *dev,
 
 }
 
+static int genpd_dev_pm_attach_one(struct device *dev)
+{
+	struct generic_pm_domain *pd;
+	int ret;
+
+	mutex_lock(&gpd_list_lock);
+	pd = genpd_dev_pm_lookup(dev, 0);
+	if (IS_ERR(pd)) {
+		mutex_unlock(&gpd_list_lock);
+		return PTR_ERR(pd);
+	}
+
+	ret = genpd_dev_pm_attach_device(dev, pd);
+	mutex_unlock(&gpd_list_lock);
+
+	return ret;
+}
+
+static int genpd_dev_pm_attach_many(struct device *dev, unsigned int count)
+{
+	struct generic_pm_domain *pd, *parent;
+	unsigned int i;
+	int ret;
+
+	pd = kzalloc(sizeof(*pd), GFP_KERNEL);
+	if (!pd)
+		return -ENOMEM;
+
+	pd->name = dev_name(dev);
+
+	ret = pm_genpd_init(pd, NULL, true);
+	if (ret < 0)
+		goto err_free;
+
+	mutex_lock(&gpd_list_lock);
+	for (i = 0; i < count; i++) {
+		parent = genpd_dev_pm_lookup(dev, i);
+		if (IS_ERR(parent)) {
+			ret = PTR_ERR(parent);
+			goto err_remove;
+		}
+
+		ret = genpd_add_subdomain(parent, pd);
+		if (ret < 0)
+			goto err_remove;
+
+	}
+
+	ret = genpd_dev_pm_attach_device(dev, pd);
+	if (ret < 0)
+		goto err_remove;
+
+	mutex_unlock(&gpd_list_lock);
+
+	return ret;
+
+err_remove:
+	WARN_ON(genpd_remove(pd));
+
+	mutex_unlock(&gpd_list_lock);
+
+err_free:
+	kfree(pd);
+
+	return ret;
+}
+
 /**
  * genpd_dev_pm_attach - Attach a device to its PM domain using DT.
  * @dev: Device to attach.
@@ -1915,8 +1995,7 @@  static int genpd_dev_pm_attach_device(struct device *dev,
  */
 int genpd_dev_pm_attach(struct device *dev)
 {
-	struct generic_pm_domain *pd;
-	int ret;
+	int count;
 
 	if (!dev->of_node)
 		return -ENODEV;
@@ -1924,17 +2003,12 @@  int genpd_dev_pm_attach(struct device *dev)
 	if (dev->pm_domain)
 		return -EEXIST;
 
-	mutex_lock(&gpd_list_lock);
-	pd = genpd_dev_pm_lookup(dev, 0);
-	if (IS_ERR(pd)) {
-		mutex_unlock(&gpd_list_lock);
-		return -EPROBE_DEFER;
-	}
-
-	ret = genpd_dev_pm_attach_device(dev, pd);
-	mutex_lock(&gpd_list_lock);
-
-	return ret;
+	count = of_count_phandle_with_args(dev->of_node, "power-domains",
+					   "#power-domain-cells");
+	if (count > 1)
+		return genpd_dev_pm_attach_many(dev, count);
+	else
+		return genpd_dev_pm_attach_one(dev);
 }
 EXPORT_SYMBOL_GPL(genpd_dev_pm_attach);
 #endif /* CONFIG_PM_GENERIC_DOMAINS_OF */