| Message ID | 20200409063137.803522-1-joel@jms.id.au |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] aspeed: Add boot stub for smp booting | expand |
On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote: > > This is a boot stub that is similar to the code u-boot runs, allowing > the kernel to boot the secondary CPU. > +static void aspeed_write_smpboot(ARMCPU *cpu, > + const struct arm_boot_info *info) > +{ > + static const uint32_t poll_mailbox_ready[] = { > + /* > + * r2 = per-cpu go sign value > + * r1 = AST_SMP_MBOX_FIELD_ENTRY > + * r0 = AST_SMP_MBOX_FIELD_GOSIGN > + */ > + 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */ > + 0xe21000ff, /* ands r0, r0, #255 */ > + 0xe59f201c, /* ldr r2, [pc, #28] */ > + 0xe1822000, /* orr r2, r2, r0 */ > + > + 0xe59f1018, /* ldr r1, [pc, #24] */ > + 0xe59f0018, /* ldr r0, [pc, #24] */ > + > + 0xe320f002, /* wfe */ > + 0xe5904000, /* ldr r4, [r0] */ > + 0xe1520004, /* cmp r2, r4 */ > + 0x1afffffb, /* bne <wfe> */ Note that unlike "wfi", QEMU's "wfe" implementation is merely a 'yield', so a secondary-CPU boot loop that has wfe in it will basically be a busy-loop of those vcpu threads. (This is why the smpboot code in hw/arm/boot.c uses wfi.) I don't suppose the secondary boot protocol on these boards is such that a wfi loop will work ? (Depends on what the primary code in the kernel does to prod the secondary after writing the magic value.) > + 0xe591f000, /* ldr pc, [r1] */ > + AST_SMP_MBOX_GOSIGN, > + AST_SMP_MBOX_FIELD_ENTRY, > + AST_SMP_MBOX_FIELD_GOSIGN, > + }; thanks -- PMM
On Thu, 16 Apr 2020 at 16:48, Peter Maydell <peter.maydell@linaro.org> wrote: > > On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote: > > > > This is a boot stub that is similar to the code u-boot runs, allowing > > the kernel to boot the secondary CPU. > > > +static void aspeed_write_smpboot(ARMCPU *cpu, > > + const struct arm_boot_info *info) > > +{ > > + static const uint32_t poll_mailbox_ready[] = { > > + /* > > + * r2 = per-cpu go sign value > > + * r1 = AST_SMP_MBOX_FIELD_ENTRY > > + * r0 = AST_SMP_MBOX_FIELD_GOSIGN > > + */ > > + 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */ > > + 0xe21000ff, /* ands r0, r0, #255 */ > > + 0xe59f201c, /* ldr r2, [pc, #28] */ > > + 0xe1822000, /* orr r2, r2, r0 */ > > + > > + 0xe59f1018, /* ldr r1, [pc, #24] */ > > + 0xe59f0018, /* ldr r0, [pc, #24] */ > > + > > + 0xe320f002, /* wfe */ > > + 0xe5904000, /* ldr r4, [r0] */ > > + 0xe1520004, /* cmp r2, r4 */ > > + 0x1afffffb, /* bne <wfe> */ > > Note that unlike "wfi", QEMU's "wfe" implementation is merely > a 'yield', so a secondary-CPU boot loop that has wfe in it > will basically be a busy-loop of those vcpu threads. > (This is why the smpboot code in hw/arm/boot.c uses wfi.) > > I don't suppose the secondary boot protocol on these boards > is such that a wfi loop will work ? (Depends on what the > primary code in the kernel does to prod the secondary after > writing the magic value.) Thanks for the review. I chose wfe as it's what aspeed's u-boot has. I don't have a strong understand of this area of ARM processors, nor do I have any insight into why that choice was made. I did some testing and wfi in the qemu stub works fine with Linux so I will respin with that instead. Cheers, Joel
diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c index 88bcd6ff3fbd..e363e495ef48 100644 --- a/hw/arm/aspeed.c +++ b/hw/arm/aspeed.c @@ -116,6 +116,58 @@ static const MemoryRegionOps max_ram_ops = { .endianness = DEVICE_NATIVE_ENDIAN, }; +#define AST_SMP_MAILBOX_BASE 0x1e6e2180 +#define AST_SMP_MBOX_FIELD_ENTRY (AST_SMP_MAILBOX_BASE + 0x0) +#define AST_SMP_MBOX_FIELD_GOSIGN (AST_SMP_MAILBOX_BASE + 0x4) +#define AST_SMP_MBOX_FIELD_READY (AST_SMP_MAILBOX_BASE + 0x8) +#define AST_SMP_MBOX_FIELD_POLLINSN (AST_SMP_MAILBOX_BASE + 0xc) +#define AST_SMP_MBOX_CODE (AST_SMP_MAILBOX_BASE + 0x10) +#define AST_SMP_MBOX_GOSIGN 0xabbaab00 + +static void aspeed_write_smpboot(ARMCPU *cpu, + const struct arm_boot_info *info) +{ + static const uint32_t poll_mailbox_ready[] = { + /* + * r2 = per-cpu go sign value + * r1 = AST_SMP_MBOX_FIELD_ENTRY + * r0 = AST_SMP_MBOX_FIELD_GOSIGN + */ + 0xee100fb0, /* mrc p15, 0, r0, c0, c0, 5 */ + 0xe21000ff, /* ands r0, r0, #255 */ + 0xe59f201c, /* ldr r2, [pc, #28] */ + 0xe1822000, /* orr r2, r2, r0 */ + + 0xe59f1018, /* ldr r1, [pc, #24] */ + 0xe59f0018, /* ldr r0, [pc, #24] */ + + 0xe320f002, /* wfe */ + 0xe5904000, /* ldr r4, [r0] */ + 0xe1520004, /* cmp r2, r4 */ + 0x1afffffb, /* bne <wfe> */ + 0xe591f000, /* ldr pc, [r1] */ + AST_SMP_MBOX_GOSIGN, + AST_SMP_MBOX_FIELD_ENTRY, + AST_SMP_MBOX_FIELD_GOSIGN, + }; + + rom_add_blob_fixed("aspeed.smpboot", poll_mailbox_ready, + sizeof(poll_mailbox_ready), + info->smp_loader_start); +} + +static void aspeed_reset_secondary(ARMCPU *cpu, + const struct arm_boot_info *info) +{ + AddressSpace *as = arm_boot_address_space(cpu, info); + CPUState *cs = CPU(cpu); + + /* info->smp_bootreg_addr */ + address_space_stl_notdirty(as, AST_SMP_MBOX_FIELD_GOSIGN, 0, + MEMTXATTRS_UNSPECIFIED, NULL); + cpu_set_pc(cs, info->smp_loader_start); +} + #define FIRMWARE_ADDR 0x0 static void write_boot_rom(DriveInfo *dinfo, hwaddr addr, size_t rom_size, @@ -270,6 +322,19 @@ static void aspeed_machine_init(MachineState *machine) } } + if (machine->kernel_filename && bmc->soc.num_cpus > 1) { + /* With no u-boot we must set up a boot stub for the secondary CPU */ + MemoryRegion *smpboot = g_new(MemoryRegion, 1); + memory_region_init_ram(smpboot, OBJECT(bmc), "aspeed.smpboot", + 0x80, &error_abort); + memory_region_add_subregion(get_system_memory(), + AST_SMP_MAILBOX_BASE, smpboot); + + aspeed_board_binfo.write_secondary_boot = aspeed_write_smpboot; + aspeed_board_binfo.secondary_cpu_reset_hook = aspeed_reset_secondary; + aspeed_board_binfo.smp_loader_start = AST_SMP_MBOX_CODE; + } + aspeed_board_binfo.ram_size = ram_size; aspeed_board_binfo.loader_start = sc->memmap[ASPEED_SDRAM]; aspeed_board_binfo.nb_cpus = bmc->soc.num_cpus;
This is a boot stub that is similar to the code u-boot runs, allowing the kernel to boot the secondary CPU. u-boot works as follows: 1. Initialises the SMP mailbox area in the SCU at 0x1e6e2180 with default values 2. Copies a stub named 'mailbox_insn' from flash to the SCU, just above the mailbox area 3. Sets AST_SMP_MBOX_FIELD_READY to a magic value to indicate the secondary can begin execution from the stub 4. The stub waits until the AST_SMP_MBOX_FIELD_GOSIGN register is set to a magic value 5. Jumps to the address in AST_SMP_MBOX_FIELD_ENTRY, starting Linux Linux indicates it is ready by writing the address of its entrypoint function to AST_SMP_MBOX_FIELD_ENTRY and the 'go' magic number to AST_SMP_MBOX_FIELD_GOSIGN. The secondary CPU sees this at step 4 and breaks out of it's loop. To be compatible, a fixed qemu stub is loaded into the mailbox area. As qemu can ensure the stub is loaded before execution starts, we do not need to emulate the AST_SMP_MBOX_FIELD_READY behaviour of u-boot. The secondary CPU's program counter points to the beginning of the stub, allowing qemu to start secondaries at step four. Reboot behaviour is preserved by resetting AST_SMP_MBOX_FIELD_GOSIGN when the secondaries are reset. This is only configured when the system is booted with -kernel and qemu does not execute u-boot first. Reviewed-by: Cédric Le Goater <clg@kaod.org> Tested-by: Cédric Le Goater <clg@kaod.org> Signed-off-by: Joel Stanley <joel@jms.id.au> --- v2: test for number of CPUs --- hw/arm/aspeed.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+)