| Message ID | 1494234377-1773-3-git-send-email-bharata@linux.vnet.ibm.com |
|---|---|
| State | New |
| Headers | show |
On Mon, May 08, 2017 at 02:36:17PM +0530, Bharata B Rao wrote: > Currently HTAB savevm handlers get registered by default and migration > of radix guest will fail. > > - Ensure that HTAB savevm handlers are not registered for radix case. > - Ensure that we issue KVM_PPC_CONFIGURE_V3_MMU for radix case post > migration. > > TODO: Right now I have delayed the HTAB savevm handler registration > to CAS call where we know if the guest is radix or hash. Another approach > is to let the HTAB handlers to be registered by default (as it is being > done currently, but unregister them from CAS when we discover radix > capability). Option 2 there sounds messy. I also suspect it could break if you try to migrate an (eventually) radix guest before it's done CAS. Strictly speaking only registering at CAS time will break old hash guests that don't do CAS at all. However such guests are really, really ancient, and I suspect we don't work with them already. You do, however, need to deregister (and allow the choice to be made again) on guest reset. On KVM we can only (for now) support either hash or radix guests. Under TCG, however, we could run a radix guest then reboot to a hash guest or vice versa. > > Reported-by: Nageswara R Sastry <rnsastry@linux.vnet.ibm.com> > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com> > --- > hw/ppc/spapr.c | 18 +++++++++++++++--- > hw/ppc/spapr_hcall.c | 5 +++++ > include/hw/ppc/spapr.h | 3 +++ > 3 files changed, 23 insertions(+), 3 deletions(-) > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index e2dc77c..e14f55c 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -1436,6 +1436,14 @@ static int spapr_post_load(void *opaque, int version_id) > err = spapr_rtc_import_offset(&spapr->rtc, spapr->rtc_offset); > } > > + if (spapr->patb_entry && (spapr->patb_flags & SPAPR_PROC_TABLE_RADIX) && patb_entry already tells you whether the guest is radix or not (PATBE1_GR), you shouldn't need extra flags. > + kvmppc_has_cap_mmu_radix() && kvm_enabled()) { You should also fail the migration if you have an incoming radix guest, but the your new KVM host can't do radix. Or the reverse, for that matter. > + err = kvmppc_configure_v3_mmu(POWERPC_CPU(first_cpu), > + spapr->patb_flags & > + SPAPR_PROC_TABLE_RADIX, > + spapr->patb_flags & SPAPR_PROC_TABLE_GTSE, > + spapr->patb_entry); > + } > return err; > } > > @@ -1520,6 +1528,7 @@ static const VMStateDescription vmstate_spapr_patb_entry = { > .needed = spapr_patb_entry_needed, > .fields = (VMStateField[]) { > VMSTATE_UINT64(patb_entry, sPAPRMachineState), > + VMSTATE_UINT64(patb_flags, sPAPRMachineState), > VMSTATE_END_OF_LIST() > }, > }; > @@ -1869,6 +1878,12 @@ static SaveVMHandlers savevm_htab_handlers = { > .load_state = htab_load, > }; > > +void spapr_htab_savevm_register(sPAPRMachineState *spapr) > +{ > + register_savevm_live(NULL, "spapr/htab", -1, 1, > + &savevm_htab_handlers, spapr); > +} > + > static void spapr_boot_set(void *opaque, const char *boot_device, > Error **errp) > { > @@ -2331,9 +2346,6 @@ static void ppc_spapr_init(MachineState *machine) > * interface, this is a legacy from the sPAPREnvironment structure > * which predated MachineState but had a similar function */ > vmstate_register(NULL, 0, &vmstate_spapr, spapr); > - register_savevm_live(NULL, "spapr/htab", -1, 1, > - &savevm_htab_handlers, spapr); > - > /* used by RTAS */ > QTAILQ_INIT(&spapr->ccs_list); > qemu_register_reset(spapr_ccs_reset_hook, spapr); > diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c > index 3600b0e..fa03ca0 100644 > --- a/hw/ppc/spapr_hcall.c > +++ b/hw/ppc/spapr_hcall.c > @@ -988,6 +988,7 @@ static target_ulong h_register_process_table(PowerPCCPU *cpu, > spapr_check_setup_free_hpt(spapr, spapr->patb_entry, cproc); > > spapr->patb_entry = cproc; /* Save new process table */ > + spapr->patb_flags = flags; /* Save the flags */ > > /* Update the UPRT and GTSE bits in the LPCR for all cpus */ > CPU_FOREACH(cs) { > @@ -1169,6 +1170,10 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu, > } > } > > + /* Register savevm handlers for HTAB case */ > + if (!guest_radix || !kvmppc_has_cap_mmu_radix()) { > + spapr_htab_savevm_register(spapr); > + } > return H_SUCCESS; > } > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h > index a692e63..0d0e9e4 100644 > --- a/include/hw/ppc/spapr.h > +++ b/include/hw/ppc/spapr.h > @@ -75,6 +75,7 @@ struct sPAPRMachineState { > void *htab; > uint32_t htab_shift; > uint64_t patb_entry; /* Process tbl registed in H_REGISTER_PROCESS_TABLE */ > + uint64_t patb_flags; > hwaddr rma_size; > int vrma_adjust; > ssize_t rtas_size; > @@ -691,4 +692,6 @@ void spapr_do_system_reset_on_cpu(CPUState *cs, run_on_cpu_data arg); > #define SPAPR_PROC_TABLE_HPT_PT 0x02 > #define SPAPR_PROC_TABLE_GTSE 0x01 > > +void spapr_htab_savevm_register(sPAPRMachineState *spapr); > + > #endif /* HW_SPAPR_H */
On Thu, May 11, 2017 at 11:02:20AM +1000, David Gibson wrote: > On Mon, May 08, 2017 at 02:36:17PM +0530, Bharata B Rao wrote: > > Currently HTAB savevm handlers get registered by default and migration > > of radix guest will fail. > > > > - Ensure that HTAB savevm handlers are not registered for radix case. > > - Ensure that we issue KVM_PPC_CONFIGURE_V3_MMU for radix case post > > migration. > > > > TODO: Right now I have delayed the HTAB savevm handler registration > > to CAS call where we know if the guest is radix or hash. Another approach > > is to let the HTAB handlers to be registered by default (as it is being > > done currently, but unregister them from CAS when we discover radix > > capability). > > Option 2 there sounds messy. I also suspect it could break if you try > to migrate an (eventually) radix guest before it's done CAS. > > Strictly speaking only registering at CAS time will break old hash > guests that don't do CAS at all. However such guests are really, > really ancient, and I suspect we don't work with them already. > > You do, however, need to deregister (and allow the choice to be made > again) on guest reset. On KVM we can only (for now) support either > hash or radix guests. Under TCG, however, we could run a radix guest > then reboot to a hash guest or vice versa. Took care of this in v1. > > > > > > Reported-by: Nageswara R Sastry <rnsastry@linux.vnet.ibm.com> > > Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com> > > --- > > hw/ppc/spapr.c | 18 +++++++++++++++--- > > hw/ppc/spapr_hcall.c | 5 +++++ > > include/hw/ppc/spapr.h | 3 +++ > > 3 files changed, 23 insertions(+), 3 deletions(-) > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > > index e2dc77c..e14f55c 100644 > > --- a/hw/ppc/spapr.c > > +++ b/hw/ppc/spapr.c > > @@ -1436,6 +1436,14 @@ static int spapr_post_load(void *opaque, int version_id) > > err = spapr_rtc_import_offset(&spapr->rtc, spapr->rtc_offset); > > } > > > > + if (spapr->patb_entry && (spapr->patb_flags & SPAPR_PROC_TABLE_RADIX) && > > patb_entry already tells you whether the guest is radix or not > (PATBE1_GR), you shouldn't need extra flags. > > > + kvmppc_has_cap_mmu_radix() && kvm_enabled()) { > > You should also fail the migration if you have an incoming radix > guest, but the your new KVM host can't do radix. Or the reverse, for > that matter. I have checks in v1 to fail migration of radix guest to a host that doesn't support radix. But I don't see how we can detect and fail the migration of hash guests into hosts that don't support hash from here (i,e., from spapr_post_load). The hash guest's migration stream would have htab savevm entries and the target will fail as it knows not about htab savevm entries. Regards, Bharata.
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index e2dc77c..e14f55c 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1436,6 +1436,14 @@ static int spapr_post_load(void *opaque, int version_id) err = spapr_rtc_import_offset(&spapr->rtc, spapr->rtc_offset); } + if (spapr->patb_entry && (spapr->patb_flags & SPAPR_PROC_TABLE_RADIX) && + kvmppc_has_cap_mmu_radix() && kvm_enabled()) { + err = kvmppc_configure_v3_mmu(POWERPC_CPU(first_cpu), + spapr->patb_flags & + SPAPR_PROC_TABLE_RADIX, + spapr->patb_flags & SPAPR_PROC_TABLE_GTSE, + spapr->patb_entry); + } return err; } @@ -1520,6 +1528,7 @@ static const VMStateDescription vmstate_spapr_patb_entry = { .needed = spapr_patb_entry_needed, .fields = (VMStateField[]) { VMSTATE_UINT64(patb_entry, sPAPRMachineState), + VMSTATE_UINT64(patb_flags, sPAPRMachineState), VMSTATE_END_OF_LIST() }, }; @@ -1869,6 +1878,12 @@ static SaveVMHandlers savevm_htab_handlers = { .load_state = htab_load, }; +void spapr_htab_savevm_register(sPAPRMachineState *spapr) +{ + register_savevm_live(NULL, "spapr/htab", -1, 1, + &savevm_htab_handlers, spapr); +} + static void spapr_boot_set(void *opaque, const char *boot_device, Error **errp) { @@ -2331,9 +2346,6 @@ static void ppc_spapr_init(MachineState *machine) * interface, this is a legacy from the sPAPREnvironment structure * which predated MachineState but had a similar function */ vmstate_register(NULL, 0, &vmstate_spapr, spapr); - register_savevm_live(NULL, "spapr/htab", -1, 1, - &savevm_htab_handlers, spapr); - /* used by RTAS */ QTAILQ_INIT(&spapr->ccs_list); qemu_register_reset(spapr_ccs_reset_hook, spapr); diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c index 3600b0e..fa03ca0 100644 --- a/hw/ppc/spapr_hcall.c +++ b/hw/ppc/spapr_hcall.c @@ -988,6 +988,7 @@ static target_ulong h_register_process_table(PowerPCCPU *cpu, spapr_check_setup_free_hpt(spapr, spapr->patb_entry, cproc); spapr->patb_entry = cproc; /* Save new process table */ + spapr->patb_flags = flags; /* Save the flags */ /* Update the UPRT and GTSE bits in the LPCR for all cpus */ CPU_FOREACH(cs) { @@ -1169,6 +1170,10 @@ static target_ulong h_client_architecture_support(PowerPCCPU *cpu, } } + /* Register savevm handlers for HTAB case */ + if (!guest_radix || !kvmppc_has_cap_mmu_radix()) { + spapr_htab_savevm_register(spapr); + } return H_SUCCESS; } diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index a692e63..0d0e9e4 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -75,6 +75,7 @@ struct sPAPRMachineState { void *htab; uint32_t htab_shift; uint64_t patb_entry; /* Process tbl registed in H_REGISTER_PROCESS_TABLE */ + uint64_t patb_flags; hwaddr rma_size; int vrma_adjust; ssize_t rtas_size; @@ -691,4 +692,6 @@ void spapr_do_system_reset_on_cpu(CPUState *cs, run_on_cpu_data arg); #define SPAPR_PROC_TABLE_HPT_PT 0x02 #define SPAPR_PROC_TABLE_GTSE 0x01 +void spapr_htab_savevm_register(sPAPRMachineState *spapr); + #endif /* HW_SPAPR_H */
Currently HTAB savevm handlers get registered by default and migration of radix guest will fail. - Ensure that HTAB savevm handlers are not registered for radix case. - Ensure that we issue KVM_PPC_CONFIGURE_V3_MMU for radix case post migration. TODO: Right now I have delayed the HTAB savevm handler registration to CAS call where we know if the guest is radix or hash. Another approach is to let the HTAB handlers to be registered by default (as it is being done currently, but unregister them from CAS when we discover radix capability). Reported-by: Nageswara R Sastry <rnsastry@linux.vnet.ibm.com> Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com> --- hw/ppc/spapr.c | 18 +++++++++++++++--- hw/ppc/spapr_hcall.c | 5 +++++ include/hw/ppc/spapr.h | 3 +++ 3 files changed, 23 insertions(+), 3 deletions(-)