Patchwork Xen ACPI - Precise SRU

login
register
mail settings
Submitter Stefan Bader
Date April 24, 2012, 6:47 p.m.
Message ID <1335293228-21551-1-git-send-email-stefan.bader@canonical.com>
Download mbox
Permalink /patch/154742/
State New
Headers show

Pull-request

git://kernel.ubuntu.com/smb/ubuntu-precise.git xen-acpi

Comments

Stefan Bader - April 24, 2012, 6:47 p.m.
While looking into this a bit more I realized that the patch which
modifies privcmd to provide a misc device interface is not needed
at all to backport the cpufreq functionality. The only issues were
Kconfig and Makefile in drivers/xen which could be resolved easily.

So the remaining patches only make some smaller changes and add
the new xen-acpi-processor driver (which only activates when in
dom0). The greatly improves the unlikelyhood of regressions outside
xen. It has not changed compared to before but it is simpler to
see.

The other modification I did was to make the xen-acpi-processor
driver a built-in. It won't load outside dom0 but as a module it
would require manual intervention and so its probably simpler to
make it work right out of the box.

So for testing:
 - On my usual Xen test box this does not work (only to the effect
   of the driver failing the acpi_processor_register_performance()
   call in init).
   Though this is not really expected (since the AMD Opteron has
   frequency scaling normally through powernow-k8) it does at least
   not make things worse. Before and after I am running at full
   speed.
 - On an Intel i7 box the driver works and I can see the frequencies
   changing through xenpm. Checking a Lucid and Precise HVM guest,
   those seem to be unaffected. Also without any visible problems
   was a Precise PVM guest. The only theoretical worry would
   be whether there is a chance the guest notices the cpu frequency
   changing and whether it cares if it would.
 - Also on the i7: I am not sure what to make of this output. 
   For example for cpu#0 I get the following:
   Max C-state: C7

   cpu id               : 0
   total C-states       : 2
   idle time(ms)        : 3315614
   C0                   : transition [00000000000000000001]
                          residency  [00000000000000069798 ms]
   C1                   : transition [00000000000000000001]
                          residency  [00000000000003315614 ms]
   pc3                  : [00000000000000000000 ms]
   pc6                  : [00000000000000000000 ms]
   pc7                  : [00000000000000000000 ms]
   cc3                  : [00000000000000000000 ms]
   cc6                  : [00000000000000000000 ms]

   And gathering statistics there is also only the cpu cstates being
   used and the data for the socket looks like:
   Socket 0
	PC3	0 ms	0.00%
	PC6	0 ms	0.00%
	PC7	0 ms	0.00%
	 Core 0 CPU 0
		CC3	0 ms	0.00%
		CC6	0 ms	0.00%

	 Core 1 CPU 2
		CC3	0 ms	0.00%
		CC6	0 ms	0.00%

	 Core 2 CPU 4
		CC3	0 ms	0.00%
		CC6	0 ms	0.00%

	 Core 3 CPU 6
		CC3	0 ms	0.00%
		CC6	0 ms	0.00%
  Again it might be not as expected but at least the same as without
  the change.

In summary, the frequency scaling improved at least on the i5, there was
no change on the AMD Opteron (which might be a bug). The C-state handling
seems unaffected (which may or may not be a bug) on both. At least the
results are in all cases the same or better.

We could SRU the new driver now and add fixes when we get them, or wait.
Probably I should try to quantify the gain by running some more tests
with a (not-cking-grade) power meter attached.

-Stefan

The following changes since commit 7885fb139c30858f3a7d8700b44b788a5c931198:

  Linux 3.2.16 (2012-04-23 09:13:23 -0600)

are available in the git repository at:

  git://kernel.ubuntu.com/smb/ubuntu-precise.git xen-acpi

for you to fetch changes up to 641bfd4d37224761d5dffa5f1bec17dc6da07556:

  xen/cpufreq: Disable the cpu frequency scaling drivers from loading. (2012-04-24 18:54:26 +0200)

----------------------------------------------------------------
Konrad Rzeszutek Wilk (6):
      provide disable_cpufreq() function to disable the API.
      xen/acpi-processor: C and P-state driver that uploads said data to hypervisor.
      xen/acpi-processor: Do not depend on CPU frequency scaling drivers.
      xen/acpi: Fix Kconfig dependency on CPU_FREQ
      xen/acpi: Remove the WARN's as they just create noise.
      xen/cpufreq: Disable the cpu frequency scaling drivers from loading.

Tim Gardner (1):
      UBUNTU: [Config] CONFIG_XEN_ACPI_PROCESSOR=y

 arch/x86/xen/setup.c                      |    2 +
 debian.master/config/config.common.ubuntu |    1 +
 drivers/cpufreq/cpufreq.c                 |   24 ++
 drivers/xen/Kconfig                       |   17 +
 drivers/xen/Makefile                      |    2 +-
 drivers/xen/xen-acpi-processor.c          |  562 +++++++++++++++++++++++++++++
 include/linux/cpufreq.h                   |    2 +
 include/xen/interface/platform.h          |   17 +
 8 files changed, 626 insertions(+), 1 deletion(-)
 create mode 100644 drivers/xen/xen-acpi-processor.c
Stefan Bader - April 25, 2012, 11:47 a.m.
On 24.04.2012 20:47, Stefan Bader wrote:
> While looking into this a bit more I realized that the patch which
> modifies privcmd to provide a misc device interface is not needed
> at all to backport the cpufreq functionality. The only issues were
> Kconfig and Makefile in drivers/xen which could be resolved easily.
> 
> So the remaining patches only make some smaller changes and add
> the new xen-acpi-processor driver (which only activates when in
> dom0). The greatly improves the unlikelyhood of regressions outside
> xen. It has not changed compared to before but it is simpler to
> see.
> 
> The other modification I did was to make the xen-acpi-processor
> driver a built-in. It won't load outside dom0 but as a module it
> would require manual intervention and so its probably simpler to
> make it work right out of the box.
> 
> So for testing:
>  - On my usual Xen test box this does not work (only to the effect
>    of the driver failing the acpi_processor_register_performance()
>    call in init).
>    Though this is not really expected (since the AMD Opteron has
>    frequency scaling normally through powernow-k8) it does at least
>    not make things worse. Before and after I am running at full
>    speed.
>  - On an Intel i7 box the driver works and I can see the frequencies
>    changing through xenpm. Checking a Lucid and Precise HVM guest,
>    those seem to be unaffected. Also without any visible problems
>    was a Precise PVM guest. The only theoretical worry would
>    be whether there is a chance the guest notices the cpu frequency
>    changing and whether it cares if it would.
>  - Also on the i7: I am not sure what to make of this output. 
>    For example for cpu#0 I get the following:
>    Max C-state: C7
> 
>    cpu id               : 0
>    total C-states       : 2
>    idle time(ms)        : 3315614
>    C0                   : transition [00000000000000000001]
>                           residency  [00000000000000069798 ms]
>    C1                   : transition [00000000000000000001]
>                           residency  [00000000000003315614 ms]
>    pc3                  : [00000000000000000000 ms]
>    pc6                  : [00000000000000000000 ms]
>    pc7                  : [00000000000000000000 ms]
>    cc3                  : [00000000000000000000 ms]
>    cc6                  : [00000000000000000000 ms]
> 
>    And gathering statistics there is also only the cpu cstates being
>    used and the data for the socket looks like:
>    Socket 0
> 	PC3	0 ms	0.00%
> 	PC6	0 ms	0.00%
> 	PC7	0 ms	0.00%
> 	 Core 0 CPU 0
> 		CC3	0 ms	0.00%
> 		CC6	0 ms	0.00%
> 
> 	 Core 1 CPU 2
> 		CC3	0 ms	0.00%
> 		CC6	0 ms	0.00%
> 
> 	 Core 2 CPU 4
> 		CC3	0 ms	0.00%
> 		CC6	0 ms	0.00%
> 
> 	 Core 3 CPU 6
> 		CC3	0 ms	0.00%
> 		CC6	0 ms	0.00%
>   Again it might be not as expected but at least the same as without
>   the change.
> 
> In summary, the frequency scaling improved at least on the i5, there was
> no change on the AMD Opteron (which might be a bug). The C-state handling
> seems unaffected (which may or may not be a bug) on both. At least the
> results are in all cases the same or better.
> 
> We could SRU the new driver now and add fixes when we get them, or wait.
> Probably I should try to quantify the gain by running some more tests
> with a (not-cking-grade) power meter attached.

At least from what I can measure, I am not that much impressed by the change:

Native kernel:		175W
Xen without patchset:	183W (with a bit of good will maths 183.60W)
Xen with patchset:	183W (calculated 183.08W)

So any difference may just be my poor meter which only resolves 1W and 1/100h
and those on two screens I have to switch between taking the values. Or not
really much more than 0.9W on a 4 core i7.
I would suspect that the output of xenpm really does mean that C3 and C6 are not
used and that doing so would give a higher power saving.

With that I would say in this form there does not seem to be such a huge benefit
from backporting the changes. I would contact Xen upstream and try to find out
what is a) wrong with the AMD Opteron and b) whether the way the c-states are
used is as it is expected.

-Stefan

> 
> -Stefan
> 
> The following changes since commit 7885fb139c30858f3a7d8700b44b788a5c931198:
> 
>   Linux 3.2.16 (2012-04-23 09:13:23 -0600)
> 
> are available in the git repository at:
> 
>   git://kernel.ubuntu.com/smb/ubuntu-precise.git xen-acpi
> 
> for you to fetch changes up to 641bfd4d37224761d5dffa5f1bec17dc6da07556:
> 
>   xen/cpufreq: Disable the cpu frequency scaling drivers from loading. (2012-04-24 18:54:26 +0200)
> 
> ----------------------------------------------------------------
> Konrad Rzeszutek Wilk (6):
>       provide disable_cpufreq() function to disable the API.
>       xen/acpi-processor: C and P-state driver that uploads said data to hypervisor.
>       xen/acpi-processor: Do not depend on CPU frequency scaling drivers.
>       xen/acpi: Fix Kconfig dependency on CPU_FREQ
>       xen/acpi: Remove the WARN's as they just create noise.
>       xen/cpufreq: Disable the cpu frequency scaling drivers from loading.
> 
> Tim Gardner (1):
>       UBUNTU: [Config] CONFIG_XEN_ACPI_PROCESSOR=y
> 
>  arch/x86/xen/setup.c                      |    2 +
>  debian.master/config/config.common.ubuntu |    1 +
>  drivers/cpufreq/cpufreq.c                 |   24 ++
>  drivers/xen/Kconfig                       |   17 +
>  drivers/xen/Makefile                      |    2 +-
>  drivers/xen/xen-acpi-processor.c          |  562 +++++++++++++++++++++++++++++
>  include/linux/cpufreq.h                   |    2 +
>  include/xen/interface/platform.h          |   17 +
>  8 files changed, 626 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/xen/xen-acpi-processor.c
>
Stefan Bader - April 25, 2012, 12:18 p.m.
On 25.04.2012 13:47, Stefan Bader wrote:
> On 24.04.2012 20:47, Stefan Bader wrote:
>> While looking into this a bit more I realized that the patch which
>> modifies privcmd to provide a misc device interface is not needed
>> at all to backport the cpufreq functionality. The only issues were
>> Kconfig and Makefile in drivers/xen which could be resolved easily.
>>
>> So the remaining patches only make some smaller changes and add
>> the new xen-acpi-processor driver (which only activates when in
>> dom0). The greatly improves the unlikelyhood of regressions outside
>> xen. It has not changed compared to before but it is simpler to
>> see.
>>
>> The other modification I did was to make the xen-acpi-processor
>> driver a built-in. It won't load outside dom0 but as a module it
>> would require manual intervention and so its probably simpler to
>> make it work right out of the box.
>>
>> So for testing:
>>  - On my usual Xen test box this does not work (only to the effect
>>    of the driver failing the acpi_processor_register_performance()
>>    call in init).
>>    Though this is not really expected (since the AMD Opteron has
>>    frequency scaling normally through powernow-k8) it does at least
>>    not make things worse. Before and after I am running at full
>>    speed.
>>  - On an Intel i7 box the driver works and I can see the frequencies
>>    changing through xenpm. Checking a Lucid and Precise HVM guest,
>>    those seem to be unaffected. Also without any visible problems
>>    was a Precise PVM guest. The only theoretical worry would
>>    be whether there is a chance the guest notices the cpu frequency
>>    changing and whether it cares if it would.
>>  - Also on the i7: I am not sure what to make of this output. 
>>    For example for cpu#0 I get the following:
>>    Max C-state: C7
>>
>>    cpu id               : 0
>>    total C-states       : 2
>>    idle time(ms)        : 3315614
>>    C0                   : transition [00000000000000000001]
>>                           residency  [00000000000000069798 ms]
>>    C1                   : transition [00000000000000000001]
>>                           residency  [00000000000003315614 ms]
>>    pc3                  : [00000000000000000000 ms]
>>    pc6                  : [00000000000000000000 ms]
>>    pc7                  : [00000000000000000000 ms]
>>    cc3                  : [00000000000000000000 ms]
>>    cc6                  : [00000000000000000000 ms]
>>
>>    And gathering statistics there is also only the cpu cstates being
>>    used and the data for the socket looks like:
>>    Socket 0
>> 	PC3	0 ms	0.00%
>> 	PC6	0 ms	0.00%
>> 	PC7	0 ms	0.00%
>> 	 Core 0 CPU 0
>> 		CC3	0 ms	0.00%
>> 		CC6	0 ms	0.00%
>>
>> 	 Core 1 CPU 2
>> 		CC3	0 ms	0.00%
>> 		CC6	0 ms	0.00%
>>
>> 	 Core 2 CPU 4
>> 		CC3	0 ms	0.00%
>> 		CC6	0 ms	0.00%
>>
>> 	 Core 3 CPU 6
>> 		CC3	0 ms	0.00%
>> 		CC6	0 ms	0.00%
>>   Again it might be not as expected but at least the same as without
>>   the change.
>>
>> In summary, the frequency scaling improved at least on the i5, there was
>> no change on the AMD Opteron (which might be a bug). The C-state handling
>> seems unaffected (which may or may not be a bug) on both. At least the
>> results are in all cases the same or better.
>>
>> We could SRU the new driver now and add fixes when we get them, or wait.
>> Probably I should try to quantify the gain by running some more tests
>> with a (not-cking-grade) power meter attached.
> 
> At least from what I can measure, I am not that much impressed by the change:
> 
> Native kernel:		175W
> Xen without patchset:	183W (with a bit of good will maths 183.60W)
> Xen with patchset:	183W (calculated 183.08W)
> 
> So any difference may just be my poor meter which only resolves 1W and 1/100h
> and those on two screens I have to switch between taking the values. Or not
> really much more than 0.9W on a 4 core i7.
> I would suspect that the output of xenpm really does mean that C3 and C6 are not
> used and that doing so would give a higher power saving.
> 
> With that I would say in this form there does not seem to be such a huge benefit
> from backporting the changes. I would contact Xen upstream and try to find out
> what is a) wrong with the AMD Opteron and b) whether the way the c-states are
> used is as it is expected.
> 

Hm, ok, the main reasoning in the bug report is that with cpufreq management,
the cpu can use turbo-mode. How useful this is iirc also does not only depend on
the frequency but also on how many cores are in c3 (or deeper).
But also this would require a different approach in testing. I'd need to pin a
guest to use a single cpu and find something to measure the cpu performance. And
then compare the two kernels.


> -Stefan
> 
>>
>> -Stefan
>>
>> The following changes since commit 7885fb139c30858f3a7d8700b44b788a5c931198:
>>
>>   Linux 3.2.16 (2012-04-23 09:13:23 -0600)
>>
>> are available in the git repository at:
>>
>>   git://kernel.ubuntu.com/smb/ubuntu-precise.git xen-acpi
>>
>> for you to fetch changes up to 641bfd4d37224761d5dffa5f1bec17dc6da07556:
>>
>>   xen/cpufreq: Disable the cpu frequency scaling drivers from loading. (2012-04-24 18:54:26 +0200)
>>
>> ----------------------------------------------------------------
>> Konrad Rzeszutek Wilk (6):
>>       provide disable_cpufreq() function to disable the API.
>>       xen/acpi-processor: C and P-state driver that uploads said data to hypervisor.
>>       xen/acpi-processor: Do not depend on CPU frequency scaling drivers.
>>       xen/acpi: Fix Kconfig dependency on CPU_FREQ
>>       xen/acpi: Remove the WARN's as they just create noise.
>>       xen/cpufreq: Disable the cpu frequency scaling drivers from loading.
>>
>> Tim Gardner (1):
>>       UBUNTU: [Config] CONFIG_XEN_ACPI_PROCESSOR=y
>>
>>  arch/x86/xen/setup.c                      |    2 +
>>  debian.master/config/config.common.ubuntu |    1 +
>>  drivers/cpufreq/cpufreq.c                 |   24 ++
>>  drivers/xen/Kconfig                       |   17 +
>>  drivers/xen/Makefile                      |    2 +-
>>  drivers/xen/xen-acpi-processor.c          |  562 +++++++++++++++++++++++++++++
>>  include/linux/cpufreq.h                   |    2 +
>>  include/xen/interface/platform.h          |   17 +
>>  8 files changed, 626 insertions(+), 1 deletion(-)
>>  create mode 100644 drivers/xen/xen-acpi-processor.c
>>
> 
> 
> 
>