| Message ID | 20190703012022.15644-1-sjitindarsingh@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [1/3] KVM: PPC: Book3S HV: Always save guest pmu for guest capable of nesting | expand |
Suraj Jitindar Singh <sjitindarsingh@gmail.com> writes: > The performance monitoring unit (PMU) registers are saved on guest exit > when the guest has set the pmcregs_in_use flag in its lppaca, if it > exists, or unconditionally if it doesn't. If a nested guest is being > run then the hypervisor doesn't, and in most cases can't, know if the > pmu registers are in use since it doesn't know the location of the lppaca > for the nested guest, although it may have one for its immediate guest. > This results in the values of these registers being lost across nested > guest entry and exit in the case where the nested guest was making use > of the performance monitoring facility while it's nested guest hypervisor > wasn't. > > Further more the hypervisor could interrupt a guest hypervisor between > when it has loaded up the pmu registers and it calling H_ENTER_NESTED or > between returning from the nested guest to the guest hypervisor and the > guest hypervisor reading the pmu registers, in kvmhv_p9_guest_entry(). > This means that it isn't sufficient to just save the pmu registers when > entering or exiting a nested guest, but that it is necessary to always > save the pmu registers whenever a guest is capable of running nested guests > to ensure the register values aren't lost in the context switch. > > Ensure the pmu register values are preserved by always saving their > value into the vcpu struct when a guest is capable of running nested > guests. > > This should have minimal performance impact however any impact can be > avoided by booting a guest with "-machine pseries,cap-nested-hv=false" > on the qemu commandline. > > Fixes: 95a6432ce903 "KVM: PPC: Book3S HV: Streamlined guest entry/exit path on P9 for radix guests" I'm not clear why this and the next commit are marked as fixing the above commit. Wasn't it broken prior to that commit as well? cheers > Signed-off-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com> > --- > arch/powerpc/kvm/book3s_hv.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c > index ec1804f822af..b682a429f3ef 100644 > --- a/arch/powerpc/kvm/book3s_hv.c > +++ b/arch/powerpc/kvm/book3s_hv.c > @@ -3654,6 +3654,8 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, > vcpu->arch.vpa.dirty = 1; > save_pmu = lp->pmcregs_in_use; > } > + /* Must save pmu if this guest is capable of running nested guests */ > + save_pmu |= nesting_enabled(vcpu->kvm); > > kvmhv_save_guest_pmu(vcpu, save_pmu); > > -- > 2.13.6
On Sat, 2019-07-13 at 13:47 +1000, Michael Ellerman wrote: > Suraj Jitindar Singh <sjitindarsingh@gmail.com> writes: > > The performance monitoring unit (PMU) registers are saved on guest > > exit > > when the guest has set the pmcregs_in_use flag in its lppaca, if it > > exists, or unconditionally if it doesn't. If a nested guest is > > being > > run then the hypervisor doesn't, and in most cases can't, know if > > the > > pmu registers are in use since it doesn't know the location of the > > lppaca > > for the nested guest, although it may have one for its immediate > > guest. > > This results in the values of these registers being lost across > > nested > > guest entry and exit in the case where the nested guest was making > > use > > of the performance monitoring facility while it's nested guest > > hypervisor > > wasn't. > > > > Further more the hypervisor could interrupt a guest hypervisor > > between > > when it has loaded up the pmu registers and it calling > > H_ENTER_NESTED or > > between returning from the nested guest to the guest hypervisor and > > the > > guest hypervisor reading the pmu registers, in > > kvmhv_p9_guest_entry(). > > This means that it isn't sufficient to just save the pmu registers > > when > > entering or exiting a nested guest, but that it is necessary to > > always > > save the pmu registers whenever a guest is capable of running > > nested guests > > to ensure the register values aren't lost in the context switch. > > > > Ensure the pmu register values are preserved by always saving their > > value into the vcpu struct when a guest is capable of running > > nested > > guests. > > > > This should have minimal performance impact however any impact can > > be > > avoided by booting a guest with "-machine pseries,cap-nested- > > hv=false" > > on the qemu commandline. > > > > Fixes: 95a6432ce903 "KVM: PPC: Book3S HV: Streamlined guest > > entry/exit path on P9 for radix guests" > > I'm not clear why this and the next commit are marked as fixing the > above commit. Wasn't it broken prior to that commit as well? That was the commit which introduced the entry path which we use for a nested guest, the path on which we need to be saving and restoring the pmu registers and so where the new code was introduced. It wasn't technically broken prior to that commit since you couldn't run nested prior to that commit, and in fact it's a few commits after that one where we actually enabled the ability to run nested guests. However since that's the code which introduced the nested entry path it seemed like the best fit for the fixes tag for people who will be looking for fixes in that area. Also all the other nested entry path fixes used that fixes tag so it ties them together nicely. Thanks, Suraj > > cheers > > > Signed-off-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com> > > --- > > arch/powerpc/kvm/book3s_hv.c | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/arch/powerpc/kvm/book3s_hv.c > > b/arch/powerpc/kvm/book3s_hv.c > > index ec1804f822af..b682a429f3ef 100644 > > --- a/arch/powerpc/kvm/book3s_hv.c > > +++ b/arch/powerpc/kvm/book3s_hv.c > > @@ -3654,6 +3654,8 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu > > *vcpu, u64 time_limit, > > vcpu->arch.vpa.dirty = 1; > > save_pmu = lp->pmcregs_in_use; > > } > > + /* Must save pmu if this guest is capable of running > > nested guests */ > > + save_pmu |= nesting_enabled(vcpu->kvm); > > > > kvmhv_save_guest_pmu(vcpu, save_pmu); > > > > -- > > 2.13.6
Suraj Jitindar Singh <sjitindarsingh@gmail.com> writes: > On Sat, 2019-07-13 at 13:47 +1000, Michael Ellerman wrote: >> Suraj Jitindar Singh <sjitindarsingh@gmail.com> writes: ... >> > >> > Fixes: 95a6432ce903 "KVM: PPC: Book3S HV: Streamlined guest >> > entry/exit path on P9 for radix guests" >> >> I'm not clear why this and the next commit are marked as fixing the >> above commit. Wasn't it broken prior to that commit as well? > > That was the commit which introduced the entry path which we use for a > nested guest, the path on which we need to be saving and restoring the > pmu registers and so where the new code was introduced. OK, I thought that commit was an unrelated optimisation. Agree that is a good target if it is the commit that introduced the nested path. cheers
On Wed, 2019-07-03 at 01:20:20 UTC, Suraj Jitindar Singh wrote: > The performance monitoring unit (PMU) registers are saved on guest exit > when the guest has set the pmcregs_in_use flag in its lppaca, if it > exists, or unconditionally if it doesn't. If a nested guest is being > run then the hypervisor doesn't, and in most cases can't, know if the > pmu registers are in use since it doesn't know the location of the lppaca > for the nested guest, although it may have one for its immediate guest. > This results in the values of these registers being lost across nested > guest entry and exit in the case where the nested guest was making use > of the performance monitoring facility while it's nested guest hypervisor > wasn't. > > Further more the hypervisor could interrupt a guest hypervisor between > when it has loaded up the pmu registers and it calling H_ENTER_NESTED or > between returning from the nested guest to the guest hypervisor and the > guest hypervisor reading the pmu registers, in kvmhv_p9_guest_entry(). > This means that it isn't sufficient to just save the pmu registers when > entering or exiting a nested guest, but that it is necessary to always > save the pmu registers whenever a guest is capable of running nested guests > to ensure the register values aren't lost in the context switch. > > Ensure the pmu register values are preserved by always saving their > value into the vcpu struct when a guest is capable of running nested > guests. > > This should have minimal performance impact however any impact can be > avoided by booting a guest with "-machine pseries,cap-nested-hv=false" > on the qemu commandline. > > Fixes: 95a6432ce903 "KVM: PPC: Book3S HV: Streamlined guest entry/exit path on P9 for radix guests" > > Signed-off-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com> Series applied to powerpc fixes, thanks. https://git.kernel.org/powerpc/c/63279eeb7f93abb1692573c26f1e038e1a87358b cheers
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index ec1804f822af..b682a429f3ef 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -3654,6 +3654,8 @@ int kvmhv_p9_guest_entry(struct kvm_vcpu *vcpu, u64 time_limit, vcpu->arch.vpa.dirty = 1; save_pmu = lp->pmcregs_in_use; } + /* Must save pmu if this guest is capable of running nested guests */ + save_pmu |= nesting_enabled(vcpu->kvm); kvmhv_save_guest_pmu(vcpu, save_pmu);
The performance monitoring unit (PMU) registers are saved on guest exit when the guest has set the pmcregs_in_use flag in its lppaca, if it exists, or unconditionally if it doesn't. If a nested guest is being run then the hypervisor doesn't, and in most cases can't, know if the pmu registers are in use since it doesn't know the location of the lppaca for the nested guest, although it may have one for its immediate guest. This results in the values of these registers being lost across nested guest entry and exit in the case where the nested guest was making use of the performance monitoring facility while it's nested guest hypervisor wasn't. Further more the hypervisor could interrupt a guest hypervisor between when it has loaded up the pmu registers and it calling H_ENTER_NESTED or between returning from the nested guest to the guest hypervisor and the guest hypervisor reading the pmu registers, in kvmhv_p9_guest_entry(). This means that it isn't sufficient to just save the pmu registers when entering or exiting a nested guest, but that it is necessary to always save the pmu registers whenever a guest is capable of running nested guests to ensure the register values aren't lost in the context switch. Ensure the pmu register values are preserved by always saving their value into the vcpu struct when a guest is capable of running nested guests. This should have minimal performance impact however any impact can be avoided by booting a guest with "-machine pseries,cap-nested-hv=false" on the qemu commandline. Fixes: 95a6432ce903 "KVM: PPC: Book3S HV: Streamlined guest entry/exit path on P9 for radix guests" Signed-off-by: Suraj Jitindar Singh <sjitindarsingh@gmail.com> --- arch/powerpc/kvm/book3s_hv.c | 2 ++ 1 file changed, 2 insertions(+)