mbox series

[B+F:linux-azure,SRU,v2,0/4] LP#:1875467: Enable invariant TSC

Message ID 20200812184246.14738-1-kelsey.skunberg@canonical.com
Headers show
Series LP#:1875467: Enable invariant TSC | expand

Message

Kelsey Skunberg Aug. 12, 2020, 6:42 p.m. UTC
BugLink: https://bugs.launchpad.net/bugs/1875467

[Impact]

Microsoft would like to request two commits in all releases supported on
Azure. These commits benefit SAP workloads.

These requested commits will enable support for Invariant TSC in Linux
guest VMs. The invariant TSC means that the TSC continues at a fixed
rate regardless of the C-state or frequency of the processor (as long as
the processor remains in the ACPI S0 state).

These patches will allow guests to access the
HvSyntheticInvariantTscControl MSR and set bit 0 of this synthetic MSR,
which then enables the InvariantTSC feature. After setting the synthetic
MSR, CPUID will enumerate support for InvariantTSC.

[Test Case]

If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
configuration version. Bit 15 corresponds to the
AccessTscInvariantControls privilege. If this privilege bit is set,
guests can access the HvSyntheticInvariantTscControl MSR: guests can set
bit 0 of this synthetic MSR to enable the InvariantTSC feature.  After
setting the synthetic MSR, CPUID will enumerate support for
InvariantTSC.

[Regression Potential]

If the patches fail to enable support for Invariant TSC or doesn't set
the TSC clocksource as the default when Invariant TSC is available,
while this would be an error it wouldn't be a regression. Possibility of
finding problems using the TSC clocksource once support has been
enabled.

a 4.15 test kernel was sent to Microsoft for testing. Result came back
verified to allow enabling support for Invariant TSC and there was no
report of known regression.

The 4.15, 5.3, & 5.4 kernels with patches applied were boot tested and
verified no changes to dmesg logs relating to clocksource or tsc. tsc
autotest ran successfully on the 4.15 before and after applying the
requested patches.

We are no longer interested in applying to Eoan/linux-azure (5.3) and
only targeted to apply the patches to bionic/azure-4.15 and focal/azure.


Changes since v1: More testing was done by microsoft and myself. Results
are included in the Bug and cover letter. Also brought the Focal and
Bionic patches together into one patch series. 


Bionic/azure-4.15:

Andrea Parri (2):
  x86/hyperv: Allow guests to enable InvariantTSC
  x86/hyperv: Set TSC clocksource as default w/ InvariantTSC

 arch/x86/hyperv/hv_init.c          | 13 +++++++++++--
 arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
 arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
 3 files changed, 22 insertions(+), 3 deletions(-)


Focal/azure-5.3:

Andrea Parri (2):
  x86/hyperv: Allow guests to enable InvariantTSC
  clocksource/drivers/hyper-v: Set TSC clocksource as default w/
    InvariantTSC

 arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
 arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
 drivers/clocksource/hyperv_timer.c | 12 ++++++++++--
 3 files changed, 21 insertions(+), 3 deletions(-)

Comments

Thadeu Lima de Souza Cascardo Aug. 12, 2020, 7:05 p.m. UTC | #1
On Wed, Aug 12, 2020 at 12:42:42PM -0600, Kelsey Skunberg wrote:
> BugLink: https://bugs.launchpad.net/bugs/1875467
> 
> [Impact]
> 
> Microsoft would like to request two commits in all releases supported on
> Azure. These commits benefit SAP workloads.
> 
> These requested commits will enable support for Invariant TSC in Linux
> guest VMs. The invariant TSC means that the TSC continues at a fixed
> rate regardless of the C-state or frequency of the processor (as long as
> the processor remains in the ACPI S0 state).
> 
> These patches will allow guests to access the
> HvSyntheticInvariantTscControl MSR and set bit 0 of this synthetic MSR,
> which then enables the InvariantTSC feature. After setting the synthetic
> MSR, CPUID will enumerate support for InvariantTSC.
> 
> [Test Case]
> 
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can set
> bit 0 of this synthetic MSR to enable the InvariantTSC feature.  After
> setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
> 

Hi, Kelsey.

This test case section doesn't tell how one could check that the instance has
picked up TSC as its clocksource, or still used Hyper-V clocksource after the
change in instances where TSC is still marked as unstable.

> [Regression Potential]
> 
> If the patches fail to enable support for Invariant TSC or doesn't set
> the TSC clocksource as the default when Invariant TSC is available,
> while this would be an error it wouldn't be a regression. Possibility of
> finding problems using the TSC clocksource once support has been
> enabled.
> 
> a 4.15 test kernel was sent to Microsoft for testing. Result came back
> verified to allow enabling support for Invariant TSC and there was no
> report of known regression.
> 
> The 4.15, 5.3, & 5.4 kernels with patches applied were boot tested and
> verified no changes to dmesg logs relating to clocksource or tsc. tsc
> autotest ran successfully on the 4.15 before and after applying the
> requested patches.

Potential regressions that I see is that TSC is not reliable as a clocksource;
we have MSR writes spillups; or even a totally different clocksource is used
because ratings were changed.

Those are all stil okay, as we got good test results from Microsoft.

We these considerations in mind:

Acked-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>

> 
> We are no longer interested in applying to Eoan/linux-azure (5.3) and
> only targeted to apply the patches to bionic/azure-4.15 and focal/azure.
> 
> 
> Changes since v1: More testing was done by microsoft and myself. Results
> are included in the Bug and cover letter. Also brought the Focal and
> Bionic patches together into one patch series. 
> 
> 
> Bionic/azure-4.15:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   x86/hyperv: Set TSC clocksource as default w/ InvariantTSC
> 
>  arch/x86/hyperv/hv_init.c          | 13 +++++++++++--
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
> 
> Focal/azure-5.3:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   clocksource/drivers/hyper-v: Set TSC clocksource as default w/
>     InvariantTSC
> 
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  drivers/clocksource/hyperv_timer.c | 12 ++++++++++--
>  3 files changed, 21 insertions(+), 3 deletions(-)
> -- 
> 2.25.1
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
Kelsey Skunberg Aug. 12, 2020, 7:13 p.m. UTC | #2
On 2020-08-12 16:05:41 , Thadeu Lima de Souza Cascardo wrote:
> On Wed, Aug 12, 2020 at 12:42:42PM -0600, Kelsey Skunberg wrote:
> > BugLink: https://bugs.launchpad.net/bugs/1875467
> > 
> > [Impact]
> > 
> > Microsoft would like to request two commits in all releases supported on
> > Azure. These commits benefit SAP workloads.
> > 
> > These requested commits will enable support for Invariant TSC in Linux
> > guest VMs. The invariant TSC means that the TSC continues at a fixed
> > rate regardless of the C-state or frequency of the processor (as long as
> > the processor remains in the ACPI S0 state).
> > 
> > These patches will allow guests to access the
> > HvSyntheticInvariantTscControl MSR and set bit 0 of this synthetic MSR,
> > which then enables the InvariantTSC feature. After setting the synthetic
> > MSR, CPUID will enumerate support for InvariantTSC.
> > 
> > [Test Case]
> > 
> > If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> > HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> > configuration version. Bit 15 corresponds to the
> > AccessTscInvariantControls privilege. If this privilege bit is set,
> > guests can access the HvSyntheticInvariantTscControl MSR: guests can set
> > bit 0 of this synthetic MSR to enable the InvariantTSC feature.  After
> > setting the synthetic MSR, CPUID will enumerate support for
> > InvariantTSC.
> > 
> 
> Hi, Kelsey.
> 
> This test case section doesn't tell how one could check that the instance has
> picked up TSC as its clocksource, or still used Hyper-V clocksource after the
> change in instances where TSC is still marked as unstable.
>

Oh! I can add the commands you shared with me into the Bug. Good
thought. 

> > [Regression Potential]
> > 
> > If the patches fail to enable support for Invariant TSC or doesn't set
> > the TSC clocksource as the default when Invariant TSC is available,
> > while this would be an error it wouldn't be a regression. Possibility of
> > finding problems using the TSC clocksource once support has been
> > enabled.
> > 
> > a 4.15 test kernel was sent to Microsoft for testing. Result came back
> > verified to allow enabling support for Invariant TSC and there was no
> > report of known regression.
> > 
> > The 4.15, 5.3, & 5.4 kernels with patches applied were boot tested and
> > verified no changes to dmesg logs relating to clocksource or tsc. tsc
> > autotest ran successfully on the 4.15 before and after applying the
> > requested patches.
> 
> Potential regressions that I see is that TSC is not reliable as a clocksource;
> we have MSR writes spillups; or even a totally different clocksource is used
> because ratings were changed.
> 
> Those are all stil okay, as we got good test results from Microsoft.
>

I struggled a bit with this part in seeing what other regressions could
be potential since it's a relatively new research point for me. I'll
check into the MSR writes spillups, I'm curious about that. And valid
for the rating changes. 

appreciate your input on this! Definitely on the list to get better with
writing the regression potentials which I'm sure will get easier with
more experience. 

> We these considerations in mind:
> 
> Acked-by: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
> 

Thank you!

-Kelsey

> > 
> > We are no longer interested in applying to Eoan/linux-azure (5.3) and
> > only targeted to apply the patches to bionic/azure-4.15 and focal/azure.
> > 
> > 
> > Changes since v1: More testing was done by microsoft and myself. Results
> > are included in the Bug and cover letter. Also brought the Focal and
> > Bionic patches together into one patch series. 
> > 
> > 
> > Bionic/azure-4.15:
> > 
> > Andrea Parri (2):
> >   x86/hyperv: Allow guests to enable InvariantTSC
> >   x86/hyperv: Set TSC clocksource as default w/ InvariantTSC
> > 
> >  arch/x86/hyperv/hv_init.c          | 13 +++++++++++--
> >  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
> >  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
> >  3 files changed, 22 insertions(+), 3 deletions(-)
> > 
> > 
> > Focal/azure-5.3:
> > 
> > Andrea Parri (2):
> >   x86/hyperv: Allow guests to enable InvariantTSC
> >   clocksource/drivers/hyper-v: Set TSC clocksource as default w/
> >     InvariantTSC
> > 
> >  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
> >  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
> >  drivers/clocksource/hyperv_timer.c | 12 ++++++++++--
> >  3 files changed, 21 insertions(+), 3 deletions(-)
> > -- 
> > 2.25.1
> > 
> > 
> > -- 
> > kernel-team mailing list
> > kernel-team@lists.ubuntu.com
> > https://lists.ubuntu.com/mailman/listinfo/kernel-team
Stefan Bader Aug. 14, 2020, 7:26 a.m. UTC | #3
On 12.08.20 20:42, Kelsey Skunberg wrote:
> BugLink: https://bugs.launchpad.net/bugs/1875467
> 
> [Impact]
> 
> Microsoft would like to request two commits in all releases supported on
> Azure. These commits benefit SAP workloads.
> 
> These requested commits will enable support for Invariant TSC in Linux
> guest VMs. The invariant TSC means that the TSC continues at a fixed
> rate regardless of the C-state or frequency of the processor (as long as
> the processor remains in the ACPI S0 state).
> 
> These patches will allow guests to access the
> HvSyntheticInvariantTscControl MSR and set bit 0 of this synthetic MSR,
> which then enables the InvariantTSC feature. After setting the synthetic
> MSR, CPUID will enumerate support for InvariantTSC.
> 
> [Test Case]
> 
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can set
> bit 0 of this synthetic MSR to enable the InvariantTSC feature.  After
> setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
> 
> [Regression Potential]
> 
> If the patches fail to enable support for Invariant TSC or doesn't set
> the TSC clocksource as the default when Invariant TSC is available,
> while this would be an error it wouldn't be a regression. Possibility of
> finding problems using the TSC clocksource once support has been
> enabled.
> 
> a 4.15 test kernel was sent to Microsoft for testing. Result came back
> verified to allow enabling support for Invariant TSC and there was no
> report of known regression.
> 
> The 4.15, 5.3, & 5.4 kernels with patches applied were boot tested and
> verified no changes to dmesg logs relating to clocksource or tsc. tsc
> autotest ran successfully on the 4.15 before and after applying the
> requested patches.
> 
> We are no longer interested in applying to Eoan/linux-azure (5.3) and
> only targeted to apply the patches to bionic/azure-4.15 and focal/azure.
> 
> 
> Changes since v1: More testing was done by microsoft and myself. Results
> are included in the Bug and cover letter. Also brought the Focal and
> Bionic patches together into one patch series. 
> 
> 
> Bionic/azure-4.15:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   x86/hyperv: Set TSC clocksource as default w/ InvariantTSC
> 
>  arch/x86/hyperv/hv_init.c          | 13 +++++++++++--
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
> 
> Focal/azure-5.3:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   clocksource/drivers/hyper-v: Set TSC clocksource as default w/
>     InvariantTSC
> 
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  drivers/clocksource/hyperv_timer.c | 12 ++++++++++--
>  3 files changed, 21 insertions(+), 3 deletions(-)
> 
Acked-by: Stefan Bader <stefan.bader@canonical.com>
Khaled Elmously Aug. 28, 2020, 4:46 a.m. UTC | #4
On 2020-08-12 12:42:42 , Kelsey Skunberg wrote:
> BugLink: https://bugs.launchpad.net/bugs/1875467
> 
> [Impact]
> 
> Microsoft would like to request two commits in all releases supported on
> Azure. These commits benefit SAP workloads.
> 
> These requested commits will enable support for Invariant TSC in Linux
> guest VMs. The invariant TSC means that the TSC continues at a fixed
> rate regardless of the C-state or frequency of the processor (as long as
> the processor remains in the ACPI S0 state).
> 
> These patches will allow guests to access the
> HvSyntheticInvariantTscControl MSR and set bit 0 of this synthetic MSR,
> which then enables the InvariantTSC feature. After setting the synthetic
> MSR, CPUID will enumerate support for InvariantTSC.
> 
> [Test Case]
> 
> If the hardware supports TSC scaling, Hyper-V will set bit 15 of the
> HV_PARTITION_PRIVILEGE_MASK in guest VMs with a compatible Hyper-V
> configuration version. Bit 15 corresponds to the
> AccessTscInvariantControls privilege. If this privilege bit is set,
> guests can access the HvSyntheticInvariantTscControl MSR: guests can set
> bit 0 of this synthetic MSR to enable the InvariantTSC feature.  After
> setting the synthetic MSR, CPUID will enumerate support for
> InvariantTSC.
> 
> [Regression Potential]
> 
> If the patches fail to enable support for Invariant TSC or doesn't set
> the TSC clocksource as the default when Invariant TSC is available,
> while this would be an error it wouldn't be a regression. Possibility of
> finding problems using the TSC clocksource once support has been
> enabled.
> 
> a 4.15 test kernel was sent to Microsoft for testing. Result came back
> verified to allow enabling support for Invariant TSC and there was no
> report of known regression.
> 
> The 4.15, 5.3, & 5.4 kernels with patches applied were boot tested and
> verified no changes to dmesg logs relating to clocksource or tsc. tsc
> autotest ran successfully on the 4.15 before and after applying the
> requested patches.
> 
> We are no longer interested in applying to Eoan/linux-azure (5.3) and
> only targeted to apply the patches to bionic/azure-4.15 and focal/azure.
> 
> 
> Changes since v1: More testing was done by microsoft and myself. Results
> are included in the Bug and cover letter. Also brought the Focal and
> Bionic patches together into one patch series. 
> 
> 
> Bionic/azure-4.15:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   x86/hyperv: Set TSC clocksource as default w/ InvariantTSC
> 
>  arch/x86/hyperv/hv_init.c          | 13 +++++++++++--
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  3 files changed, 22 insertions(+), 3 deletions(-)
> 
> 
> Focal/azure-5.3:
> 
> Andrea Parri (2):
>   x86/hyperv: Allow guests to enable InvariantTSC
>   clocksource/drivers/hyper-v: Set TSC clocksource as default w/
>     InvariantTSC
> 
>  arch/x86/include/asm/hyperv-tlfs.h |  5 +++++
>  arch/x86/kernel/cpu/mshyperv.c     |  7 ++++++-
>  drivers/clocksource/hyperv_timer.c | 12 ++++++++++--
>  3 files changed, 21 insertions(+), 3 deletions(-)
> -- 
> 2.25.1
> 
> 
> -- 
> kernel-team mailing list
> kernel-team@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team