| Message ID | 20191019234715.25750-12-f4bug@amsat.org |
|---|---|
| State | New |
| Headers | show |
| Series | hw/arm/raspi: Add thermal/timer, improve address space, run U-boot | expand |
On Sun, 20 Oct 2019 at 00:47, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: > > The abstract TYPE_BCM283X device provides a 'enabled-cpus' property > to restrict the number of cores powered on reset. This because on > real hardware the GPU is responsible of starting the cores and keep > them spinning until the Linux kernel is ready to use them. > When using the -kernel paramenter, QEMU does this by installing the > 'raspi_smpboot' code when arm_boot_info::write_board_setup() is > called. This is a special feature to help the Linux kernel, and can > only be used with a Linux kernel. > > Even if loaded with the -kernel option, U-boot is not Linux, thus > is not recognized as it and the raspi_smpboot code is not installed. > > Upon introduction of this machine in commit 1df7d1f9303, the -smp <N> > option allowd to limit the number of cores powered on reset. > Unfortunately later commit 72649619341 added a check which made this > feature unusable: > > $ qemu-system-aarch64 -M raspi3 -smp 1 > qemu-system-aarch64: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi3' is 4 > > Fortunately, the -smp option allow various kind of CPU topology: > > -smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets] > set the number of CPUs to 'n' [default=1] > maxcpus= maximum number of total cpus, including > offline CPUs for hotplug, etc > cores= number of CPU cores on one socket (for PC, it's on one die) > threads= number of threads on one CPU core > dies= number of CPU dies on one socket (for PC only) > sockets= number of discrete sockets in the system > > Let's use the 'cores' argument to specify the number of cores powered > at reset to restore this feature, and allow to boot U-boot. From the help message quoted, 'cores' is about NUMA topology, it's nothing to do with "how many cores do we have powerd on on reset". We should definitely not be borrowing it for an unrelated purpose. > We can now run U-boot using: > > $ qemu-system-aarch64 -M raspi3 -smp 4,cores=1 ... Either: (a) we need to support "act like the firmware/GPU" for booting guest binaries on this board, in the same way that our PSCI support is effectively emulating firmware-level facilities (b) the u-boot binary should be provided to QEMU in the right image format to use the kernel boot ABI (c) u-boot could be wrapped in a small guest binary that deals with handling all the secondary cores I'm not really a huge fan of (a) because we already have a huge pile of random weird ways of loading guest images that are undocumented, behave differently from board to board or architecture to architecture, etc. I'm not too keen on making that swamp worse, even though there's an argument that the right UI for a raspi board model is to provide the same "load an ELF file" handling that the real hardware does for ELF files via the GPU/firmware combo. thanks -- PMM
On 10/24/19 3:23 PM, Peter Maydell wrote: > On Sun, 20 Oct 2019 at 00:47, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote: >> >> The abstract TYPE_BCM283X device provides a 'enabled-cpus' property >> to restrict the number of cores powered on reset. This because on >> real hardware the GPU is responsible of starting the cores and keep >> them spinning until the Linux kernel is ready to use them. >> When using the -kernel paramenter, QEMU does this by installing the >> 'raspi_smpboot' code when arm_boot_info::write_board_setup() is >> called. This is a special feature to help the Linux kernel, and can >> only be used with a Linux kernel. >> >> Even if loaded with the -kernel option, U-boot is not Linux, thus >> is not recognized as it and the raspi_smpboot code is not installed. >> >> Upon introduction of this machine in commit 1df7d1f9303, the -smp <N> >> option allowd to limit the number of cores powered on reset. >> Unfortunately later commit 72649619341 added a check which made this >> feature unusable: >> >> $ qemu-system-aarch64 -M raspi3 -smp 1 >> qemu-system-aarch64: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi3' is 4 >> >> Fortunately, the -smp option allow various kind of CPU topology: >> >> -smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets] >> set the number of CPUs to 'n' [default=1] >> maxcpus= maximum number of total cpus, including >> offline CPUs for hotplug, etc >> cores= number of CPU cores on one socket (for PC, it's on one die) >> threads= number of threads on one CPU core >> dies= number of CPU dies on one socket (for PC only) >> sockets= number of discrete sockets in the system >> >> Let's use the 'cores' argument to specify the number of cores powered >> at reset to restore this feature, and allow to boot U-boot. > > From the help message quoted, 'cores' is about NUMA topology, > it's nothing to do with "how many cores do we have powerd on > on reset". We should definitely not be borrowing it for > an unrelated purpose. Yes, I was not sure. Is -smp only about NUMA topology? Should I try adding another argument, such 'online_cpus_mask'? >> We can now run U-boot using: >> >> $ qemu-system-aarch64 -M raspi3 -smp 4,cores=1 ... > > Either: > (a) we need to support "act like the firmware/GPU" for > booting guest binaries on this board, in the same way that our PSCI > support is effectively emulating firmware-level facilities > (b) the u-boot binary should be provided to QEMU in the > right image format to use the kernel boot ABI Well, u-boot is in the same format than the /boot/bootcode.bin closed-source firmware provided by the Raspberry Pi foundation. This is the format the GPU firmware understand when loading it on the 1st ARM core. I'll see what I can do (ab)using the -bios parameter. > (c) u-boot could be wrapped in a small guest binary that > deals with handling all the secondary cores > > I'm not really a huge fan of (a) because we already have a > huge pile of random weird ways of loading guest images > that are undocumented, behave differently from board to > board or architecture to architecture, etc. I'm not too > keen on making that swamp worse, even though there's an > argument that the right UI for a raspi board model is to > provide the same "load an ELF file" handling that the real > hardware does for ELF files via the GPU/firmware combo. This patch doesn't modify hw/arm/boot.c and reuse -smp features, it didn't seemed too invasive. Anyway I'll look for another path. Thanks, Phil.
On 10/20/19 1:47 AM, Philippe Mathieu-Daudé wrote: > The abstract TYPE_BCM283X device provides a 'enabled-cpus' property > to restrict the number of cores powered on reset. This because on > real hardware the GPU is responsible of starting the cores and keep > them spinning until the Linux kernel is ready to use them. > When using the -kernel paramenter, QEMU does this by installing the > 'raspi_smpboot' code when arm_boot_info::write_board_setup() is > called. This is a special feature to help the Linux kernel, and can > only be used with a Linux kernel. > > Even if loaded with the -kernel option, U-boot is not Linux, thus > is not recognized as it and the raspi_smpboot code is not installed. > > Upon introduction of this machine in commit 1df7d1f9303, the -smp <N> > option allowd to limit the number of cores powered on reset. > Unfortunately later commit 72649619341 added a check which made this > feature unusable: > > $ qemu-system-aarch64 -M raspi3 -smp 1 > qemu-system-aarch64: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi3' is 4 > > Fortunately, the -smp option allow various kind of CPU topology: > > -smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets] > set the number of CPUs to 'n' [default=1] > maxcpus= maximum number of total cpus, including > offline CPUs for hotplug, etc > cores= number of CPU cores on one socket (for PC, it's on one die) > threads= number of threads on one CPU core > dies= number of CPU dies on one socket (for PC only) > sockets= number of discrete sockets in the system > > Let's use the 'cores' argument to specify the number of cores powered > at reset to restore this feature, and allow to boot U-boot. > > We can now run U-boot using: > > $ qemu-system-aarch64 -M raspi3 -smp 4,cores=1 ... > > Reported-by: Laurent Bonnans <laurent.bonnans@here.com> > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > hw/arm/raspi.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c > index 569d85c11a..45d3f91f95 100644 > --- a/hw/arm/raspi.c > +++ b/hw/arm/raspi.c > @@ -190,8 +190,8 @@ static void raspi_init(MachineState *machine, int version) > /* Setup the SOC */ > object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram), > &error_abort); > - object_property_set_int(OBJECT(&s->soc), machine->smp.cpus, "enabled-cpus", > - &error_abort); > + object_property_set_int(OBJECT(&s->soc), machine->smp.cores, > + "enabled-cpus", &error_abort); > int board_rev = version == 3 ? 0xa02082 : 0xa21041; > object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev", > &error_abort); Hi Phil, Thanks for the patch, I finally got the chance to make some tests (sorry about the delay). Using the proposed -smp options indeed helps to run u-boot elfs directly. However, the cores fail to start when switching to linux (tested on raspi2): [ 0.071030] smp: Bringing up secondary CPUs ... [ 1.157876] CPU1: failed to come online [ 2.219899] CPU2: failed to come online [ 3.285412] CPU3: failed to come online [ 3.286137] smp: Brought up 1 node, 1 CPU [ 3.286766] SMP: Total of 1 processors activated (125.00 BogoMIPS). [ 3.287442] CPU: All CPU(s) started in SVC mode. The behavior persist even without using the option on the command line. The normal behavior is restored if I use "-smp 4, cores=4" Greetings, Laurent
diff --git a/hw/arm/raspi.c b/hw/arm/raspi.c index 569d85c11a..45d3f91f95 100644 --- a/hw/arm/raspi.c +++ b/hw/arm/raspi.c @@ -190,8 +190,8 @@ static void raspi_init(MachineState *machine, int version) /* Setup the SOC */ object_property_add_const_link(OBJECT(&s->soc), "ram", OBJECT(&s->ram), &error_abort); - object_property_set_int(OBJECT(&s->soc), machine->smp.cpus, "enabled-cpus", - &error_abort); + object_property_set_int(OBJECT(&s->soc), machine->smp.cores, + "enabled-cpus", &error_abort); int board_rev = version == 3 ? 0xa02082 : 0xa21041; object_property_set_int(OBJECT(&s->soc), board_rev, "board-rev", &error_abort);
The abstract TYPE_BCM283X device provides a 'enabled-cpus' property to restrict the number of cores powered on reset. This because on real hardware the GPU is responsible of starting the cores and keep them spinning until the Linux kernel is ready to use them. When using the -kernel paramenter, QEMU does this by installing the 'raspi_smpboot' code when arm_boot_info::write_board_setup() is called. This is a special feature to help the Linux kernel, and can only be used with a Linux kernel. Even if loaded with the -kernel option, U-boot is not Linux, thus is not recognized as it and the raspi_smpboot code is not installed. Upon introduction of this machine in commit 1df7d1f9303, the -smp <N> option allowd to limit the number of cores powered on reset. Unfortunately later commit 72649619341 added a check which made this feature unusable: $ qemu-system-aarch64 -M raspi3 -smp 1 qemu-system-aarch64: Invalid SMP CPUs 1. The min CPUs supported by machine 'raspi3' is 4 Fortunately, the -smp option allow various kind of CPU topology: -smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets] set the number of CPUs to 'n' [default=1] maxcpus= maximum number of total cpus, including offline CPUs for hotplug, etc cores= number of CPU cores on one socket (for PC, it's on one die) threads= number of threads on one CPU core dies= number of CPU dies on one socket (for PC only) sockets= number of discrete sockets in the system Let's use the 'cores' argument to specify the number of cores powered at reset to restore this feature, and allow to boot U-boot. We can now run U-boot using: $ qemu-system-aarch64 -M raspi3 -smp 4,cores=1 ... Reported-by: Laurent Bonnans <laurent.bonnans@here.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> --- hw/arm/raspi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)