| Message ID | 1456793788-31178-1-git-send-email-oohall@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
Oliver O'Halloran <oohall@gmail.com> writes: > In very early boot, skiboot places secondary threads in holding loop > until the primary thread has finished relocating skiboot. The secondary > threads wait until a flag at INITAL_OFFSET + offsetof(boot_flag) is set > to one before continuing. After relocating the skiboot image and > applying ELF relocations the sets this flag and continues booting. > > To avoid having to copy skiboot at runtime within the simulation e605691e > introduced a change to have the skiboot.tcl mambo script load skiboot at > it's preferred location of 0x30000000 rather than 0x00000000. > Unfortunately, the relocation code path is also responsible for saving the > inital offset pointer (into r15). When not taken this results in the main > thread writing to r15 + offsetof(boot_flag) instead, which leaves the > secondary threads stuck waiting for boot_flag and the main thread stuck > when trying to call-in the secondary threads during CPU initialisation. > > This patch fixes the bug by saving the inital offset before checking if > relocation is necessary and adds a few comments to better document the > relocation process. You know what'd be awesome? A multi-threaded mambo boot test :)
Oliver O'Halloran <oohall@gmail.com> writes: > In very early boot, skiboot places secondary threads in holding loop > until the primary thread has finished relocating skiboot. The secondary > threads wait until a flag at INITAL_OFFSET + offsetof(boot_flag) is set > to one before continuing. After relocating the skiboot image and > applying ELF relocations the sets this flag and continues booting. > > To avoid having to copy skiboot at runtime within the simulation e605691e > introduced a change to have the skiboot.tcl mambo script load skiboot at > it's preferred location of 0x30000000 rather than 0x00000000. > Unfortunately, the relocation code path is also responsible for saving the > inital offset pointer (into r15). When not taken this results in the main > thread writing to r15 + offsetof(boot_flag) instead, which leaves the > secondary threads stuck waiting for boot_flag and the main thread stuck > when trying to call-in the secondary threads during CPU initialisation. > > This patch fixes the bug by saving the inital offset before checking if > relocation is necessary and adds a few comments to better document the > relocation process. > > Signed-off-by: Oliver O'Halloran <oohall@gmail.com> Appears to make some sense, and verified by trying it with/without this patch. MErged to master as of 1141ca0f74d9cf8dc8456f5ea9248e82b468d36b
diff --git a/asm/head.S b/asm/head.S index b514b22..66d405f 100644 --- a/asm/head.S +++ b/asm/head.S @@ -310,6 +310,12 @@ boot_entry: /* Initialize thread SPRs */ bl init_replicated_sprs + /* Save the initial offset. The secondary threads will spin on boot_flag + * before relocation so we need to keep track of its location to wake + * them up. + */ + mr %r15,%r30 + /* Check if we need to copy ourselves up and update %r30 to * be our new offset */ @@ -319,17 +325,19 @@ boot_entry: srdi %r3,%r3,3 mtctr %r3 mr %r4,%r30 - mr %r15,%r30 mr %r30,%r29 + /* copy the skiboot image to the new offset */ 1: ld %r0,0(%r4) std %r0,0(%r29) addi %r29,%r29,8 addi %r4,%r4,8 bdnz 1b + /* flush caches, etc */ sync icbi 0,%r29 sync isync + /* branch to the new image location and continue */ LOAD_IMM32(%r3, 2f - __head) add %r3,%r3,%r30 mtctr %r3
In very early boot, skiboot places secondary threads in holding loop until the primary thread has finished relocating skiboot. The secondary threads wait until a flag at INITAL_OFFSET + offsetof(boot_flag) is set to one before continuing. After relocating the skiboot image and applying ELF relocations the sets this flag and continues booting. To avoid having to copy skiboot at runtime within the simulation e605691e introduced a change to have the skiboot.tcl mambo script load skiboot at it's preferred location of 0x30000000 rather than 0x00000000. Unfortunately, the relocation code path is also responsible for saving the inital offset pointer (into r15). When not taken this results in the main thread writing to r15 + offsetof(boot_flag) instead, which leaves the secondary threads stuck waiting for boot_flag and the main thread stuck when trying to call-in the secondary threads during CPU initialisation. This patch fixes the bug by saving the inital offset before checking if relocation is necessary and adds a few comments to better document the relocation process. Signed-off-by: Oliver O'Halloran <oohall@gmail.com> --- asm/head.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)