| Message ID | 20180920224856.GJ487685@eidolon.nox.tf (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [ppc-next] powerpc/fsl-booke: don't load early TLB at once | expand |
| Context | Check | Description |
|---|---|---|
| snowpatch_ozlabs/apply_patch | success | next/apply_patch Successfully applied |
| snowpatch_ozlabs/checkpatch | success | Test checkpatch on branch next |
| snowpatch_ozlabs/build-ppc64le | success | Test build-ppc64le on branch next |
| snowpatch_ozlabs/build-ppc64be | success | Test build-ppc64be on branch next |
| snowpatch_ozlabs/build-ppc64e | success | Test build-ppc64e on branch next |
| snowpatch_ozlabs/build-ppc32 | success | Test build-ppc32 on branch next |
On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: > This is a *partial* revert of "powerpc/85xx: Load all early TLB entries > at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) > > My dusty old P4080DS just completely fails to boot (no output at all) > without this revert. I have no clue what's going on here, I just > bisected it down and since it looks like an optimization to me I just > reverted it - and voilá, the P4080 boots again. It's not an optimization; it was required to get kdump working, at least for certain choices of crash kernel location. I just tried booting a 32- bit kernel and did not see this problem -- but I don't have access to a p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit isn't really supported on e6500, but you do get output even without that). Do you have a JTAG that can be used to find out where it's hanging? If not, I can try to get early debug working (needs an early IOMMU mapping). York, can you try booting the latest kernel on p4080ds? -Scott
On Thu, Sep 20, 2018 at 07:31:33PM -0500, Scott Wood wrote: > On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: > > This is a *partial* revert of "powerpc/85xx: Load all early TLB entries > > at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) > > > > My dusty old P4080DS just completely fails to boot (no output at all) > > without this revert. I have no clue what's going on here, I just > > bisected it down and since it looks like an optimization to me I just > > reverted it - and voilá, the P4080 boots again. > > It's not an optimization; it was required to get kdump working, at least Oh, my apologies for misunderstanding the code. > for certain choices of crash kernel location. I just tried booting a 32- > bit kernel and did not see this problem -- but I don't have access to a > p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit > kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit > isn't really supported on e6500, but you do get output even without that). Hrm, maybe I should just make it use the old mechanism under an #ifdef CONFIG_PPC32? Better to boot and have kdump not work than not boot. (But obviously finding the actual problem would be best.) > Do you have a JTAG that can be used to find out where it's hanging? If > not, I can try to get early debug working (needs an early IOMMU mapping). I only have JTAG tools for ARM chips available; hardware wise I could probably solder up an adapter but software wise I have absoutely no clue how to fire up a session on anything PPC... I'm a novice openocd user, that's it. > York, can you try booting the latest kernel on p4080ds? d9e1831a42 has been around for quite some time, 4.4 already has it. I was a bit surprised noone has run into this, but then again P4080 is not exactly the most recent/interesting hardware. Cheers, -David
On 09/20/2018 05:31 PM, Scott Wood wrote: > On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: >> This is a *partial* revert of "powerpc/85xx: Load all early TLB entries >> at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) >> >> My dusty old P4080DS just completely fails to boot (no output at all) >> without this revert. I have no clue what's going on here, I just >> bisected it down and since it looks like an optimization to me I just >> reverted it - and voilá, the P4080 boots again. > > It's not an optimization; it was required to get kdump working, at least > for certain choices of crash kernel location. I just tried booting a 32- > bit kernel and did not see this problem -- but I don't have access to a > p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit > kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit > isn't really supported on e6500, but you do get output even without that). > > Do you have a JTAG that can be used to find out where it's hanging? If > not, I can try to get early debug working (needs an early IOMMU mapping). > > York, can you try booting the latest kernel on p4080ds? > Scott, I haven't tried P4080DS for a long time. What defconfig do you use for this board? I tried latest master branch (commit a27fb6d983c7b5) with corenet_basic_defconfig, it didn't boot up. Kernel has an exception very early (pc ffffae80). However, before I claimed the board, someone booted Linux 4.14.71 on this board. I need to track down where the image came from. York
On Fri, 2018-09-21 at 17:40 +0000, York Sun wrote: > On 09/20/2018 05:31 PM, Scott Wood wrote: > > On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: > > > This is a *partial* revert of "powerpc/85xx: Load all early TLB entries > > > at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) > > > > > > My dusty old P4080DS just completely fails to boot (no output at all) > > > without this revert. I have no clue what's going on here, I just > > > bisected it down and since it looks like an optimization to me I just > > > reverted it - and voilá, the P4080 boots again. > > > > It's not an optimization; it was required to get kdump working, at least > > for certain choices of crash kernel location. I just tried booting a 32- > > bit kernel and did not see this problem -- but I don't have access to a > > p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit > > kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit > > isn't really supported on e6500, but you do get output even without that). > > > > Do you have a JTAG that can be used to find out where it's hanging? If > > not, I can try to get early debug working (needs an early IOMMU mapping). > > > > York, can you try booting the latest kernel on p4080ds? > > > > Scott, > > I haven't tried P4080DS for a long time. What defconfig do you use for > this board? I tried latest master branch (commit a27fb6d983c7b5) with > corenet_basic_defconfig, it didn't boot up. Kernel has an exception very > early (pc ffffae80). > > However, before I claimed the board, someone booted Linux 4.14.71 on > this board. I need to track down where the image came from. Use corenet32_smp_defconfig corenet_basic_defconfig is just a fragment used by the makefiles in assembling the final config. -Scott
On 09/21/2018 10:47 AM, Scott Wood wrote: > On Fri, 2018-09-21 at 17:40 +0000, York Sun wrote: >> On 09/20/2018 05:31 PM, Scott Wood wrote: >>> On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: >>>> This is a *partial* revert of "powerpc/85xx: Load all early TLB entries >>>> at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) >>>> >>>> My dusty old P4080DS just completely fails to boot (no output at all) >>>> without this revert. I have no clue what's going on here, I just >>>> bisected it down and since it looks like an optimization to me I just >>>> reverted it - and voilá, the P4080 boots again. >>> >>> It's not an optimization; it was required to get kdump working, at least >>> for certain choices of crash kernel location. I just tried booting a 32- >>> bit kernel and did not see this problem -- but I don't have access to a >>> p4080ds anymore. I tried with qemu e500mc, and also running a 32-bit >>> kernel on e6500 (needs a tiny change to get past SMP init, since 32-bit >>> isn't really supported on e6500, but you do get output even without that). >>> >>> Do you have a JTAG that can be used to find out where it's hanging? If >>> not, I can try to get early debug working (needs an early IOMMU mapping). >>> >>> York, can you try booting the latest kernel on p4080ds? >>> >> >> Scott, >> >> I haven't tried P4080DS for a long time. What defconfig do you use for >> this board? I tried latest master branch (commit a27fb6d983c7b5) with >> corenet_basic_defconfig, it didn't boot up. Kernel has an exception very >> early (pc ffffae80). >> >> However, before I claimed the board, someone booted Linux 4.14.71 on >> this board. I need to track down where the image came from. > > Use corenet32_smp_defconfig > > corenet_basic_defconfig is just a fragment used by the makefiles in assembling > the final config. > Thanks for the instruction. Linux comes up OK with corenet32_smp_defconfig. root@p4080ds:~# uname -a Linux p4080ds 4.19.0-rc4-00206-ga27fb6d #1 SMP Fri Sep 21 10:56:36 PDT 2018 ppc GNU/Linux York
On Fri, Sep 21, 2018 at 03:00:19AM +0200, David Lamparter wrote: > On Thu, Sep 20, 2018 at 07:31:33PM -0500, Scott Wood wrote: > > Do you have a JTAG that can be used to find out where it's hanging? If > > not, I can try to get early debug working (needs an early IOMMU mapping). s/IOMMU mapping/MMU mapping for MMIO/ :-P > I only have JTAG tools for ARM chips available; hardware wise I could > probably solder up an adapter but software wise I have absoutely no clue > how to fire up a session on anything PPC... I'm a novice openocd user, > that's it. > > > York, can you try booting the latest kernel on p4080ds? > > d9e1831a42 has been around for quite some time, 4.4 already has it. I > was a bit surprised noone has run into this, but then again P4080 is not > exactly the most recent/interesting hardware. I don't suppose you're running a relocatable kernel at a non-zero address, and/or are running in an environment that sets HID0[EN_L2MMU_MHD] (neither standard U-boot nor Linux sets this bit, though they probably should)? On 32-bit, we're already running in an AS1 trampoline when loadcam_multi() is called, but loadcam_multi() sets up its own. This happens to not be catastrophic in standard scenarios, but it does add a duplicate TLB entry, and we return to AS0 sooner than expected. I think your patch, plus ifdefs to make the change 32-bit only, is the appropriate fix. I also got an earlier udbg for e500 working (and happened to decide to test it with a relocatable kernel); I'll send that out once I've cleaned it up (or sooner with the extra TLB dumping included if the above doesn't explain why you're hitting this bug). -Scott
David Lamparter <equinox@diac24.net> writes: > This is a *partial* revert of "powerpc/85xx: Load all early TLB entries > at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) > > My dusty old P4080DS just completely fails to boot (no output at all) > without this revert. I have no clue what's going on here, I just > bisected it down and since it looks like an optimization to me I just > reverted it - and voilá, the P4080 boots again. > > Cc: Scott Wood <swood@redhat.com> > Signed-off-by: David Lamparter <equinox@diac24.net> > --- > I'm not subscribed to linuxppc-dev, please keep me in Cc:. Also I > contribute to the kernel only very rarely and have no clue how to find > out when the appropriate time in the merge window is to submit things to > linuxppc-dev... my apologies. Don't worry about the timing of sending patches. Your patch will be tracked in patchwork, so if it's not a good time we'll just ignore it until it is a good time :) cheers
(Sorry about the delay on my end, deadlines ...) On Fri, Sep 21, 2018 at 06:07:48PM +0000, York Sun wrote: > On 09/21/2018 10:47 AM, Scott Wood wrote: > > On Fri, 2018-09-21 at 17:40 +0000, York Sun wrote: > >> On 09/20/2018 05:31 PM, Scott Wood wrote: > >>> On Fri, 2018-09-21 at 00:48 +0200, David Lamparter wrote: > >>>> My dusty old P4080DS just completely fails to boot (no output at all) > >>>> without this revert. I have no clue what's going on here, I just > >>>> bisected it down and since it looks like an optimization to me I just > >>>> reverted it - and voilá, the P4080 boots again. > >>> > >>> It's not an optimization; it was required to get kdump working, at least > >>> for certain choices of crash kernel location. [...] > >>> York, can you try booting the latest kernel on p4080ds? [...] > > Thanks for the instruction. Linux comes up OK with corenet32_smp_defconfig. > > root@p4080ds:~# uname -a > Linux p4080ds 4.19.0-rc4-00206-ga27fb6d #1 SMP Fri Sep 21 10:56:36 PDT > 2018 ppc GNU/Linux Well, shoot. I guess the next likely thing is that I have something weird enabled in my .config, so I'll try booting corenet32_smp_defconfig on my P4080DS. (I vaguely think I did that at some point but I can't swear to it so I'll just retry.) If that doesn't boot, could either of you provide me with your built uImage so I can exclude toolchain stupidity? => I'll send another mail when I got to try corenet32_smp_defconfig. Aaaand, during the past week I noticed this is a rev1.0 P4080 chip, which apparently only exists on some early P4080DS boards - and Freescale even did a replacement program to get rev2.0 boards out (I guess this one was sitting in some broom closet.) I don't have access to erratas for this, I only know the entire QMan/FMan stuff is royally f*cked - no idea whether the PPC core has issues too. FWIW, my board is running perfectly stable with that patch I posted and I'll just carry it locally if it's an issue for this one specific board I have here. I just don't have sufficient information to tell if that is indeed the case. Thanks a lot for your input and help, -David
On Sat, Sep 22, 2018 at 12:45:16AM -0500, Scott Wood wrote: > I don't suppose you're running a relocatable kernel at a non-zero # CONFIG_RELOCATABLE is not set > address, and/or are running in an environment that sets > HID0[EN_L2MMU_MHD] (neither standard U-boot nor Linux sets this bit, It's u-boot v2018.09-rc3; bit "33" / 30 isn't even defined as a constant, much less touched anywhere. HID0 in u-boot seems to be 0x80000080 aka EMCP | EN_MAS7_UPDATE. > though they probably should)? On 32-bit, we're already running in an AS1 > trampoline when loadcam_multi() is called, but loadcam_multi() sets up > its own. This happens to not be catastrophic in standard scenarios, but > it does add a duplicate TLB entry, and we return to AS0 sooner than > expected. I think your patch, plus ifdefs to make the change 32-bit > only, is the appropriate fix. I'll resend it with ifdefs inserted. Meanwhile, still trying the default config as mentioned in my previous mail. > I also got an earlier udbg for e500 working (and happened to decide to > test it with a relocatable kernel); I'll send that out once I've cleaned > it up (or sooner with the extra TLB dumping included if the above doesn't > explain why you're hitting this bug). Thanks & Cheers, -David
diff --git a/arch/powerpc/mm/fsl_booke_mmu.c b/arch/powerpc/mm/fsl_booke_mmu.c index 080d49b26c3a..f301a4c32c66 100644 --- a/arch/powerpc/mm/fsl_booke_mmu.c +++ b/arch/powerpc/mm/fsl_booke_mmu.c @@ -143,6 +143,8 @@ static void settlbcam(int index, unsigned long virt, phys_addr_t phys, tlbcam_addrs[index].start = virt; tlbcam_addrs[index].limit = virt + size - 1; tlbcam_addrs[index].phys = phys; + + loadcam_entry(index); } unsigned long calc_cam_sz(unsigned long ram, unsigned long virt, @@ -195,7 +197,6 @@ static unsigned long map_mem_in_cams_addr(phys_addr_t phys, unsigned long virt, if (dryrun) return amount_mapped; - loadcam_multi(0, i, max_cam_idx); tlbcam_index = i; #ifdef CONFIG_PPC64 diff --git a/arch/powerpc/mm/mmu_decl.h b/arch/powerpc/mm/mmu_decl.h index e5d779eed181..9a14de6a3ca9 100644 --- a/arch/powerpc/mm/mmu_decl.h +++ b/arch/powerpc/mm/mmu_decl.h @@ -140,7 +140,6 @@ extern int switch_to_as1(void); extern void restore_to_as0(int esel, int offset, void *dt_ptr, int bootcpu); #endif extern void loadcam_entry(unsigned int index); -extern void loadcam_multi(int first_idx, int num, int tmp_idx); struct tlbcam { u32 MAS0; diff --git a/arch/powerpc/mm/tlb_nohash_low.S b/arch/powerpc/mm/tlb_nohash_low.S index e066a658acac..b5fa6c6a940f 100644 --- a/arch/powerpc/mm/tlb_nohash_low.S +++ b/arch/powerpc/mm/tlb_nohash_low.S @@ -402,7 +402,6 @@ _GLOBAL(set_context) * extern void loadcam_entry(unsigned int index) * * Load TLBCAM[index] entry in to the L2 CAM MMU - * Must preserve r7, r8, r9, and r10 */ _GLOBAL(loadcam_entry) mflr r5 @@ -426,66 +425,4 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_BIG_PHYS) tlbwe isync blr - -/* - * Load multiple TLB entries at once, using an alternate-space - * trampoline so that we don't have to care about whether the same - * TLB entry maps us before and after. - * - * r3 = first entry to write - * r4 = number of entries to write - * r5 = temporary tlb entry - */ -_GLOBAL(loadcam_multi) - mflr r8 - - /* - * Set up temporary TLB entry that is the same as what we're - * running from, but in AS=1. - */ - bl 1f -1: mflr r6 - tlbsx 0,r8 - mfspr r6,SPRN_MAS1 - ori r6,r6,MAS1_TS - mtspr SPRN_MAS1,r6 - mfspr r6,SPRN_MAS0 - rlwimi r6,r5,MAS0_ESEL_SHIFT,MAS0_ESEL_MASK - mr r7,r5 - mtspr SPRN_MAS0,r6 - isync - tlbwe - isync - - /* Switch to AS=1 */ - mfmsr r6 - ori r6,r6,MSR_IS|MSR_DS - mtmsr r6 - isync - - mr r9,r3 - add r10,r3,r4 -2: bl loadcam_entry - addi r9,r9,1 - cmpw r9,r10 - mr r3,r9 - blt 2b - - /* Return to AS=0 and clear the temporary entry */ - mfmsr r6 - rlwinm. r6,r6,0,~(MSR_IS|MSR_DS) - mtmsr r6 - isync - - li r6,0 - mtspr SPRN_MAS1,r6 - rlwinm r6,r7,MAS0_ESEL_SHIFT,MAS0_ESEL_MASK - oris r6,r6,MAS0_TLBSEL(1)@h - mtspr SPRN_MAS0,r6 - isync - tlbwe - isync - - mtlr r8 - blr #endif
This is a *partial* revert of "powerpc/85xx: Load all early TLB entries at once" (d9e1831a420267a7ced708bb259d65b0a3c0344d.) My dusty old P4080DS just completely fails to boot (no output at all) without this revert. I have no clue what's going on here, I just bisected it down and since it looks like an optimization to me I just reverted it - and voilá, the P4080 boots again. Cc: Scott Wood <swood@redhat.com> Signed-off-by: David Lamparter <equinox@diac24.net> --- I'm not subscribed to linuxppc-dev, please keep me in Cc:. Also I contribute to the kernel only very rarely and have no clue how to find out when the appropriate time in the merge window is to submit things to linuxppc-dev... my apologies. --- arch/powerpc/mm/fsl_booke_mmu.c | 3 +- arch/powerpc/mm/mmu_decl.h | 1 - arch/powerpc/mm/tlb_nohash_low.S | 63 -------------------------------- 3 files changed, 2 insertions(+), 65 deletions(-)