| Message ID | 20190529065017.15149-42-david@gibson.dropbear.id.au |
|---|---|
| State | New |
| Headers | show |
| Series | [PULL,01/44] tests: Fix up docker cross builds for ppc64 (BE) targets | expand |
On 29/05/2019 08:50, David Gibson wrote: > From: Cédric Le Goater <clg@kaod.org> > > Now that XIVE support is complete (QEMU emulated and KVM devices), > change the pseries machine to advertise both interrupt modes: XICS > (P7/P8) and XIVE (P9). > > The machine default interrupt modes depends on the version. Current > settings are: > > pseries default interrupt mode > > 4.1 dual > 4.0 xics > 3.1 xics > 3.0 legacy xics (different IRQ number space layout) > > Signed-off-by: Cédric Le Goater <clg@kaod.org> > Message-Id: <20190522074016.10521-3-clg@kaod.org> > Reviewed-by: Greg Kurz <groug@kaod.org> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au> > --- > hw/ppc/spapr.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index 39e698e9b0..4fd16b43f0 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; > smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF; > spapr_caps_add_properties(smc, &error_abort); > - smc->irq = &spapr_irq_xics; > + smc->irq = &spapr_irq_dual; > smc->dr_phb_enabled = true; > } > > @@ -4430,6 +4430,7 @@ static void spapr_machine_4_0_class_options(MachineClass *mc) > spapr_machine_4_1_class_options(mc); > compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len); > smc->phb_placement = phb_placement_4_0; > + smc->irq = &spapr_irq_xics; > } > > DEFINE_SPAPR_MACHINE(4_0, "4.0", false); > This patch breaks the '-no-reboot' parameter (I think the "dual" mode breaks the -no-reboot parameter) After grub loads the kernel and starts it, the kernel aborts: OF stdout device is: /vdevice/vty@71000000 Preparing to boot Linux version 4.18.0-112.el8.ppc64le (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507 (Red Hat 8.3.1-4) (GCC)) #1 SMP Fri Jul 5 11:21:28 UTC 2019 Detected machine type: 0000000000000101 command line: BOOT_IMAGE=/vmlinuz-4.18.0-112.el8.ppc64le root=/dev/mapper/rhel_ibm--p8--kvm--03--guest--02-root ro crashkernel=auto rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/root rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/swap Max number of cores passed to firmware: 256 (NR_CPUS = 2048) Calling ibm,client-architecture-support...[lvivier@localhost ~]$ I bisected to this patch, and then after I understood the problem is with the -no-reboot parameter as the machine is reset by the CAS negotiation... and the -no-reboot prevents this reset. I don't know if it's a real problem or not. Thanks, Laurent
On Wed, Jul 10, 2019 at 06:26:09PM +0200, Laurent Vivier wrote: > On 29/05/2019 08:50, David Gibson wrote: > > From: Cédric Le Goater <clg@kaod.org> > > > > Now that XIVE support is complete (QEMU emulated and KVM devices), > > change the pseries machine to advertise both interrupt modes: XICS > > (P7/P8) and XIVE (P9). > > > > The machine default interrupt modes depends on the version. Current > > settings are: > > > > pseries default interrupt mode > > > > 4.1 dual > > 4.0 xics > > 3.1 xics > > 3.0 legacy xics (different IRQ number space layout) > > > > Signed-off-by: Cédric Le Goater <clg@kaod.org> > > Message-Id: <20190522074016.10521-3-clg@kaod.org> > > Reviewed-by: Greg Kurz <groug@kaod.org> > > Signed-off-by: David Gibson <david@gibson.dropbear.id.au> > > --- > > hw/ppc/spapr.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > > index 39e698e9b0..4fd16b43f0 100644 > > --- a/hw/ppc/spapr.c > > +++ b/hw/ppc/spapr.c > > @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > > smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; > > smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF; > > spapr_caps_add_properties(smc, &error_abort); > > - smc->irq = &spapr_irq_xics; > > + smc->irq = &spapr_irq_dual; > > smc->dr_phb_enabled = true; > > } > > > > @@ -4430,6 +4430,7 @@ static void spapr_machine_4_0_class_options(MachineClass *mc) > > spapr_machine_4_1_class_options(mc); > > compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len); > > smc->phb_placement = phb_placement_4_0; > > + smc->irq = &spapr_irq_xics; > > } > > > > DEFINE_SPAPR_MACHINE(4_0, "4.0", false); > > > > This patch breaks the '-no-reboot' parameter (I think the "dual" mode > breaks the -no-reboot parameter) > > After grub loads the kernel and starts it, the kernel aborts: > > OF stdout device is: /vdevice/vty@71000000 > Preparing to boot Linux version 4.18.0-112.el8.ppc64le > (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507 > (Red Hat 8.3.1-4) (GCC)) #1 SMP Fri Jul 5 11:21:28 UTC 2019 > Detected machine type: 0000000000000101 > command line: BOOT_IMAGE=/vmlinuz-4.18.0-112.el8.ppc64le > root=/dev/mapper/rhel_ibm--p8--kvm--03--guest--02-root ro > crashkernel=auto rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/root > rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/swap > Max number of cores passed to firmware: 256 (NR_CPUS = 2048) > Calling ibm,client-architecture-support...[lvivier@localhost ~]$ > > I bisected to this patch, and then after I understood the problem is > with the -no-reboot parameter as the machine is reset by the CAS > negotiation... and the -no-reboot prevents this reset. > > I don't know if it's a real problem or not. Ah, bother. I didn't think of the interaction between the CAS reboot and -no-reboot. I guess that's more reason to work out a way to do the xics/xive switch without a full reset. People were already not thrilled with the extra reboots here.
On 11/07/2019 03:26, David Gibson wrote: > On Wed, Jul 10, 2019 at 06:26:09PM +0200, Laurent Vivier wrote: >> On 29/05/2019 08:50, David Gibson wrote: >>> From: Cédric Le Goater <clg@kaod.org> >>> >>> Now that XIVE support is complete (QEMU emulated and KVM devices), >>> change the pseries machine to advertise both interrupt modes: XICS >>> (P7/P8) and XIVE (P9). >>> >>> The machine default interrupt modes depends on the version. Current >>> settings are: >>> >>> pseries default interrupt mode >>> >>> 4.1 dual >>> 4.0 xics >>> 3.1 xics >>> 3.0 legacy xics (different IRQ number space layout) >>> >>> Signed-off-by: Cédric Le Goater <clg@kaod.org> >>> Message-Id: <20190522074016.10521-3-clg@kaod.org> >>> Reviewed-by: Greg Kurz <groug@kaod.org> >>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au> >>> --- >>> hw/ppc/spapr.c | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c >>> index 39e698e9b0..4fd16b43f0 100644 >>> --- a/hw/ppc/spapr.c >>> +++ b/hw/ppc/spapr.c >>> @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) >>> smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; >>> smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF; >>> spapr_caps_add_properties(smc, &error_abort); >>> - smc->irq = &spapr_irq_xics; >>> + smc->irq = &spapr_irq_dual; >>> smc->dr_phb_enabled = true; >>> } >>> >>> @@ -4430,6 +4430,7 @@ static void spapr_machine_4_0_class_options(MachineClass *mc) >>> spapr_machine_4_1_class_options(mc); >>> compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len); >>> smc->phb_placement = phb_placement_4_0; >>> + smc->irq = &spapr_irq_xics; >>> } >>> >>> DEFINE_SPAPR_MACHINE(4_0, "4.0", false); >>> >> >> This patch breaks the '-no-reboot' parameter (I think the "dual" mode >> breaks the -no-reboot parameter) >> >> After grub loads the kernel and starts it, the kernel aborts: >> >> OF stdout device is: /vdevice/vty@71000000 >> Preparing to boot Linux version 4.18.0-112.el8.ppc64le >> (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507 >> (Red Hat 8.3.1-4) (GCC)) #1 SMP Fri Jul 5 11:21:28 UTC 2019 >> Detected machine type: 0000000000000101 >> command line: BOOT_IMAGE=/vmlinuz-4.18.0-112.el8.ppc64le >> root=/dev/mapper/rhel_ibm--p8--kvm--03--guest--02-root ro >> crashkernel=auto rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/root >> rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/swap >> Max number of cores passed to firmware: 256 (NR_CPUS = 2048) >> Calling ibm,client-architecture-support...[lvivier@localhost ~]$ >> >> I bisected to this patch, and then after I understood the problem is >> with the -no-reboot parameter as the machine is reset by the CAS >> negotiation... and the -no-reboot prevents this reset. >> >> I don't know if it's a real problem or not. > > Ah, bother. I didn't think of the interaction between the CAS reboot > and -no-reboot. I guess that's more reason to work out a way to do > the xics/xive switch without a full reset. People were already not > thrilled with the extra reboots here. QEMU builds a device tree depending on the interrupt mode negotiated at CAS time. Can we dynamically add/remove nodes ? C.
On Mon, Jul 15, 2019 at 12:19:03PM +0200, Cédric Le Goater wrote: > On 11/07/2019 03:26, David Gibson wrote: > > On Wed, Jul 10, 2019 at 06:26:09PM +0200, Laurent Vivier wrote: > >> On 29/05/2019 08:50, David Gibson wrote: > >>> From: Cédric Le Goater <clg@kaod.org> > >>> > >>> Now that XIVE support is complete (QEMU emulated and KVM devices), > >>> change the pseries machine to advertise both interrupt modes: XICS > >>> (P7/P8) and XIVE (P9). > >>> > >>> The machine default interrupt modes depends on the version. Current > >>> settings are: > >>> > >>> pseries default interrupt mode > >>> > >>> 4.1 dual > >>> 4.0 xics > >>> 3.1 xics > >>> 3.0 legacy xics (different IRQ number space layout) > >>> > >>> Signed-off-by: Cédric Le Goater <clg@kaod.org> > >>> Message-Id: <20190522074016.10521-3-clg@kaod.org> > >>> Reviewed-by: Greg Kurz <groug@kaod.org> > >>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au> > >>> --- > >>> hw/ppc/spapr.c | 3 ++- > >>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>> > >>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > >>> index 39e698e9b0..4fd16b43f0 100644 > >>> --- a/hw/ppc/spapr.c > >>> +++ b/hw/ppc/spapr.c > >>> @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > >>> smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; > >>> smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF; > >>> spapr_caps_add_properties(smc, &error_abort); > >>> - smc->irq = &spapr_irq_xics; > >>> + smc->irq = &spapr_irq_dual; > >>> smc->dr_phb_enabled = true; > >>> } > >>> > >>> @@ -4430,6 +4430,7 @@ static void spapr_machine_4_0_class_options(MachineClass *mc) > >>> spapr_machine_4_1_class_options(mc); > >>> compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len); > >>> smc->phb_placement = phb_placement_4_0; > >>> + smc->irq = &spapr_irq_xics; > >>> } > >>> > >>> DEFINE_SPAPR_MACHINE(4_0, "4.0", false); > >>> > >> > >> This patch breaks the '-no-reboot' parameter (I think the "dual" mode > >> breaks the -no-reboot parameter) > >> > >> After grub loads the kernel and starts it, the kernel aborts: > >> > >> OF stdout device is: /vdevice/vty@71000000 > >> Preparing to boot Linux version 4.18.0-112.el8.ppc64le > >> (mockbuild@ppc-061.build.eng.bos.redhat.com) (gcc version 8.3.1 20190507 > >> (Red Hat 8.3.1-4) (GCC)) #1 SMP Fri Jul 5 11:21:28 UTC 2019 > >> Detected machine type: 0000000000000101 > >> command line: BOOT_IMAGE=/vmlinuz-4.18.0-112.el8.ppc64le > >> root=/dev/mapper/rhel_ibm--p8--kvm--03--guest--02-root ro > >> crashkernel=auto rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/root > >> rd.lvm.lv=rhel_ibm-p8-kvm-03-guest-02/swap > >> Max number of cores passed to firmware: 256 (NR_CPUS = 2048) > >> Calling ibm,client-architecture-support...[lvivier@localhost ~]$ > >> > >> I bisected to this patch, and then after I understood the problem is > >> with the -no-reboot parameter as the machine is reset by the CAS > >> negotiation... and the -no-reboot prevents this reset. > >> > >> I don't know if it's a real problem or not. > > > > Ah, bother. I didn't think of the interaction between the CAS reboot > > and -no-reboot. I guess that's more reason to work out a way to do > > the xics/xive switch without a full reset. People were already not > > thrilled with the extra reboots here. > > QEMU builds a device tree depending on the interrupt mode negotiated > at CAS time. Can we dynamically add/remove nodes ? I believe we can make essentially arbitrary change to the device tree as part of the CAS process without a reboot. Re-doing this is kind of awkward with the current structure, but it's probably what we're going to want.
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 39e698e9b0..4fd16b43f0 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -4352,7 +4352,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) smc->default_caps.caps[SPAPR_CAP_LARGE_DECREMENTER] = SPAPR_CAP_ON; smc->default_caps.caps[SPAPR_CAP_CCF_ASSIST] = SPAPR_CAP_OFF; spapr_caps_add_properties(smc, &error_abort); - smc->irq = &spapr_irq_xics; + smc->irq = &spapr_irq_dual; smc->dr_phb_enabled = true; } @@ -4430,6 +4430,7 @@ static void spapr_machine_4_0_class_options(MachineClass *mc) spapr_machine_4_1_class_options(mc); compat_props_add(mc->compat_props, hw_compat_4_0, hw_compat_4_0_len); smc->phb_placement = phb_placement_4_0; + smc->irq = &spapr_irq_xics; } DEFINE_SPAPR_MACHINE(4_0, "4.0", false);