diff mbox

[U-Boot,issue,report] mainline u-boot was stuck before booting kernel

Message ID 57970375.2000300@rock-chips.com
State RFC
Delegated to: Tom Rini
Headers show

Commit Message

Xu Ziyuan July 26, 2016, 6:30 a.m. UTC
Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux 
kernel entry.:-)

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() 
which to disable dcache and MMU. But it just tell processor that  memory 
had change, not real ISB for armv7.

dcache_disable
         ==>flush_dcache_all()
         ==>set_cr()   disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory")
static inline void set_cr(unsigned int val)
{
     if (is_hyp())
         asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                   : "r" (val)
                                   : "cc");
     else
         asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                   : "r" (val)
                                   : "cc");
     isb();
}

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from 
the each processor followed by an ISB to ensure the MMU disable 
operation is complete, then followed by a DSB to drain previous memory 
transactions."

In my humble opinion, maybe instructions tlb had alter, and cause 
running away??? I'm not sure about it.
@Sandy, could you have a try with my update?

@Simon,  did you hit this glitch? I hope you can help ...:-)

On 2016年07月25日 23:00, Sandy Patterson wrote:
> Ah, thanks. Your debugging looks the same as what I've seen from 
> printf debugging. I'll try to verify though.
>
> On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu@rock-chips.com 
> <mailto:xzy.xu@rock-chips.com>> wrote:
>
>     hi Stany,
>
>     The difference is that you print out assertion log.
>
>     Reset not supported on this platform
>     ### ERROR ### Please RESET the board ###
>
>     You can add show_boot_progress() function in your bsp file to find
>     out something.
>     #define CONFIG_SHOW_BOOT_PROGRESS
>     void show_boot_progress(int progress)
>     {
>         printf("Boot reached stage %d\n", progress);
>
>     }
>
>
>     On 2016年07月25日 22:12, Ziyuan Xu wrote:
>
>         Hi All,
>
>         I'm sorry to tell you that I failed to boot linux kernel with
>         the mainline u-boot on rk3288 board(both evb-rk3288 &
>         fennec-rk3288). It was stuck in cleanup_before_linux() before
>         jumping to linux, and the boot_stage_flag is
>         BOOTSTAGE_ID_BOOTM_HANDOFF.
>
>         ## Current stack ends at 0x7df638b0 *  kernel: cmdline image
>         address = 0x02000000
>         ## No init Ramdisk
>            ramdisk start = 0x00000000, ramdisk end = 0x00000000
>         ## No Flattened Device Tree
>         Continuing to boot without FDT
>         Initial value for argc=3
>         Final value for argc=3
>         using: ATAGS
>         ## Transferring control to Linux (at address 02000000)...
>
>         Starting kernel ...
>
>         With the further investigation, it never returnned back from
>         invalidate_dcache_all(). I mean that I can't reach stage 4 as
>         below.
>
>         cleanup_before_linux
>             ==>cleanup_before_linux_select
>                 ==>1.disable_interrupts
>                 ==>2.dcache_disable
>                 ==>3.invalidate_dcache_all
>                 ==>4.icache_disable
>
>         Debug further, invalidate_dcache_all  invalidate all cache
>         one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or
>         UNIFIED. And invalidate way from one set to another set in
>         order. The problem is that the PC ran away in invalidate way
>         loop  [cache level L1!!!].
>
>         I add some serial output code in __v7_flush_dcache_all to
>         figure out the bog.
>         I expect:
>         Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d.....
>         0x01, 0x00 (hex)
>         In fact, print the value of r9 in sequence at first, but print
>         unexpected value afterwards.   e.g 0x7f, 0x7e, 0x7d, ..,0x73,
>         0x40, 0x85,....
>
>         ENTRY(__v7_flush_dcache_all)
>         [snip]
>         loop1:
>             mov    r9, r7                @ create working copy of max
>         index
>         loop2:
>         +      stmfd    sp!, {r0}
>         +      ldr     r0, =0xff690000
>         +      str     r9, [r0]
>         +      ldmfd    sp!, {r0}
>          ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and
>         cache number into r11
>          THUMB(    lsl    r6, r4, r5        )
>          THUMB(    orr    r11, r10, r6        )    @ factor way and
>         cache number into r11
>          ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index
>         number into r11
>          THUMB(    lsl    r6, r9, r2        )
>          THUMB(    orr    r11, r11, r6        )    @ factor index
>         number into r11
>             mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate
>         by set/way
>             subs    r9, r9, #1            @ decrement the index
>             bge    loop2
>             subs    r4, r4, #1            @ decrement the way
>             bge    loop1
>         skip:
>         ENDPROC(__v7_flush_dcache_all)
>
>         I don't have the jtag, hence I can't address the current pc. I
>         have no doubt that if any glitches in __v7_flush_dcache_all, I
>         reviewed several times, also copy kernel's implement to here.
>         :-( No effect.
>         A more interesting thing is that Sandy had report it. He and I
>         have similar problem. Everything work fine after I applied his
>         patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
>         @Stany, I'm sorry that I disable dcache during hack.. That was
>         a mistake:-(
>
>         @Simon & hieko,
>         Can you boot linux with the mainline u-boot? have a try?
>
>
>         _______________________________________________
>         U-Boot mailing list
>         U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
>         http://lists.denx.de/mailman/listinfo/u-boot
>
>
>
>
>
>
>

Comments

Xu Ziyuan July 26, 2016, 6:32 a.m. UTC | #1
+Simon and heiko

On 2016年07月26日 14:30, Ziyuan Xu wrote:
> Dear All,
>
> I add the ISB operation after dcache_disable(), and I can jump to 
> linux kernel entry.:-)
>
> diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c
> index 6eac5ef..5cc09ba 100644
> --- a/arch/arm/cpu/armv7/cpu.c
> +++ b/arch/arm/cpu/armv7/cpu.c
> @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
>                 */
>                 dcache_disable();
>                 v7_outer_cache_disable();
> +               ISB;
>
>                 /*
>                 * After D-cache is flushed and before it is disabled 
> there may
>
> Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() 
> which to disable dcache and MMU. But it just tell processor that  
> memory had change, not real ISB for armv7.
>
> dcache_disable
>         ==>flush_dcache_all()
>         ==>set_cr()   disable dcache and MMU.
>
> #define isb() __asm__ __volatile__ ("" : : : "memory")
> static inline void set_cr(unsigned int val)
> {
>     if (is_hyp())
>         asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
>                                   : "r" (val)
>                                   : "cc");
>     else
>         asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
>                                   : "r" (val)
>                                   : "cc");
>     isb();
> }
>
> I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from 
> the each processor followed by an ISB to ensure the MMU disable 
> operation is complete, then followed by a DSB to drain previous memory 
> transactions."
>
> In my humble opinion, maybe instructions tlb had alter, and cause 
> running away??? I'm not sure about it.
> @Sandy, could you have a try with my update?
>
> @Simon,  did you hit this glitch? I hope you can help ...:-)
>
> On 2016年07月25日 23:00, Sandy Patterson wrote:
>> Ah, thanks. Your debugging looks the same as what I've seen from 
>> printf debugging. I'll try to verify though.
>>
>> On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu@rock-chips.com 
>> <mailto:xzy.xu@rock-chips.com>> wrote:
>>
>>     hi Stany,
>>
>>     The difference is that you print out assertion log.
>>
>>     Reset not supported on this platform
>>     ### ERROR ### Please RESET the board ###
>>
>>     You can add show_boot_progress() function in your bsp file to find
>>     out something.
>>     #define CONFIG_SHOW_BOOT_PROGRESS
>>     void show_boot_progress(int progress)
>>     {
>>         printf("Boot reached stage %d\n", progress);
>>
>>     }
>>
>>
>>     On 2016年07月25日 22:12, Ziyuan Xu wrote:
>>
>>         Hi All,
>>
>>         I'm sorry to tell you that I failed to boot linux kernel with
>>         the mainline u-boot on rk3288 board(both evb-rk3288 &
>>         fennec-rk3288). It was stuck in cleanup_before_linux() before
>>         jumping to linux, and the boot_stage_flag is
>>         BOOTSTAGE_ID_BOOTM_HANDOFF.
>>
>>         ## Current stack ends at 0x7df638b0 *  kernel: cmdline image
>>         address = 0x02000000
>>         ## No init Ramdisk
>>            ramdisk start = 0x00000000, ramdisk end = 0x00000000
>>         ## No Flattened Device Tree
>>         Continuing to boot without FDT
>>         Initial value for argc=3
>>         Final value for argc=3
>>         using: ATAGS
>>         ## Transferring control to Linux (at address 02000000)...
>>
>>         Starting kernel ...
>>
>>         With the further investigation, it never returnned back from
>>         invalidate_dcache_all(). I mean that I can't reach stage 4 as
>>         below.
>>
>>         cleanup_before_linux
>>             ==>cleanup_before_linux_select
>>                 ==>1.disable_interrupts
>>                 ==>2.dcache_disable
>>                 ==>3.invalidate_dcache_all
>>                 ==>4.icache_disable
>>
>>         Debug further, invalidate_dcache_all  invalidate all cache
>>         one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or
>>         UNIFIED. And invalidate way from one set to another set in
>>         order. The problem is that the PC ran away in invalidate way
>>         loop  [cache level L1!!!].
>>
>>         I add some serial output code in __v7_flush_dcache_all to
>>         figure out the bog.
>>         I expect:
>>         Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d.....
>>         0x01, 0x00 (hex)
>>         In fact, print the value of r9 in sequence at first, but print
>>         unexpected value afterwards.   e.g 0x7f, 0x7e, 0x7d, ..,0x73,
>>         0x40, 0x85,....
>>
>>         ENTRY(__v7_flush_dcache_all)
>>         [snip]
>>         loop1:
>>             mov    r9, r7                @ create working copy of max
>>         index
>>         loop2:
>>         +      stmfd    sp!, {r0}
>>         +      ldr     r0, =0xff690000
>>         +      str     r9, [r0]
>>         +      ldmfd    sp!, {r0}
>>          ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and
>>         cache number into r11
>>          THUMB(    lsl    r6, r4, r5        )
>>          THUMB(    orr    r11, r10, r6        )    @ factor way and
>>         cache number into r11
>>          ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index
>>         number into r11
>>          THUMB(    lsl    r6, r9, r2        )
>>          THUMB(    orr    r11, r11, r6        )    @ factor index
>>         number into r11
>>             mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate
>>         by set/way
>>             subs    r9, r9, #1            @ decrement the index
>>             bge    loop2
>>             subs    r4, r4, #1            @ decrement the way
>>             bge    loop1
>>         skip:
>>         ENDPROC(__v7_flush_dcache_all)
>>
>>         I don't have the jtag, hence I can't address the current pc. I
>>         have no doubt that if any glitches in __v7_flush_dcache_all, I
>>         reviewed several times, also copy kernel's implement to here.
>>         :-( No effect.
>>         A more interesting thing is that Sandy had report it. He and I
>>         have similar problem. Everything work fine after I applied his
>>         patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
>>         @Stany, I'm sorry that I disable dcache during hack.. That was
>>         a mistake:-(
>>
>>         @Simon & hieko,
>>         Can you boot linux with the mainline u-boot? have a try?
>>
>>
>>         _______________________________________________
>>         U-Boot mailing list
>>         U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
>>         http://lists.denx.de/mailman/listinfo/u-boot
>>
>>
>>
>>
>>
>>
>>
>
>
> _______________________________________________
> U-Boot mailing list
> U-Boot@lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
Xu Ziyuan July 26, 2016, 12:15 p.m. UTC | #2
+ Simon and heiko

On 2016年07月26日 14:30, Ziyuan Xu wrote:
> Dear All,
>
> I add the ISB operation after dcache_disable(), and I can jump to 
> linux kernel entry.:-)
>
> diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c
> index 6eac5ef..5cc09ba 100644
> --- a/arch/arm/cpu/armv7/cpu.c
> +++ b/arch/arm/cpu/armv7/cpu.c
> @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
>                 */
>                 dcache_disable();
>                 v7_outer_cache_disable();
> +               ISB;
>
>                 /*
>                 * After D-cache is flushed and before it is disabled 
> there may
>
> Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() 
> which to disable dcache and MMU. But it just tell processor that  
> memory had change, not real ISB for armv7.
>
> dcache_disable
>         ==>flush_dcache_all()
>         ==>set_cr()   disable dcache and MMU.
>
> #define isb() __asm__ __volatile__ ("" : : : "memory")
> static inline void set_cr(unsigned int val)
> {
>     if (is_hyp())
>         asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
>                                   : "r" (val)
>                                   : "cc");
>     else
>         asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
>                                   : "r" (val)
>                                   : "cc");
>     isb();
> }
>
> I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from 
> the each processor followed by an ISB to ensure the MMU disable 
> operation is complete, then followed by a DSB to drain previous memory 
> transactions."
>
> In my humble opinion, maybe instructions tlb had alter, and cause 
> running away??? I'm not sure about it.
> @Sandy, could you have a try with my update?
>
> @Simon,  did you hit this glitch? I hope you can help ...:-)
>
> On 2016年07月25日 23:00, Sandy Patterson wrote:
>> Ah, thanks. Your debugging looks the same as what I've seen from 
>> printf debugging. I'll try to verify though.
>>
>> On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu@rock-chips.com 
>> <mailto:xzy.xu@rock-chips.com>> wrote:
>>
>>     hi Stany,
>>
>>     The difference is that you print out assertion log.
>>
>>     Reset not supported on this platform
>>     ### ERROR ### Please RESET the board ###
>>
>>     You can add show_boot_progress() function in your bsp file to find
>>     out something.
>>     #define CONFIG_SHOW_BOOT_PROGRESS
>>     void show_boot_progress(int progress)
>>     {
>>         printf("Boot reached stage %d\n", progress);
>>
>>     }
>>
>>
>>     On 2016年07月25日 22:12, Ziyuan Xu wrote:
>>
>>         Hi All,
>>
>>         I'm sorry to tell you that I failed to boot linux kernel with
>>         the mainline u-boot on rk3288 board(both evb-rk3288 &
>>         fennec-rk3288). It was stuck in cleanup_before_linux() before
>>         jumping to linux, and the boot_stage_flag is
>>         BOOTSTAGE_ID_BOOTM_HANDOFF.
>>
>>         ## Current stack ends at 0x7df638b0 *  kernel: cmdline image
>>         address = 0x02000000
>>         ## No init Ramdisk
>>            ramdisk start = 0x00000000, ramdisk end = 0x00000000
>>         ## No Flattened Device Tree
>>         Continuing to boot without FDT
>>         Initial value for argc=3
>>         Final value for argc=3
>>         using: ATAGS
>>         ## Transferring control to Linux (at address 02000000)...
>>
>>         Starting kernel ...
>>
>>         With the further investigation, it never returnned back from
>>         invalidate_dcache_all(). I mean that I can't reach stage 4 as
>>         below.
>>
>>         cleanup_before_linux
>>             ==>cleanup_before_linux_select
>>                 ==>1.disable_interrupts
>>                 ==>2.dcache_disable
>>                 ==>3.invalidate_dcache_all
>>                 ==>4.icache_disable
>>
>>         Debug further, invalidate_dcache_all  invalidate all cache
>>         one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or
>>         UNIFIED. And invalidate way from one set to another set in
>>         order. The problem is that the PC ran away in invalidate way
>>         loop  [cache level L1!!!].
>>
>>         I add some serial output code in __v7_flush_dcache_all to
>>         figure out the bog.
>>         I expect:
>>         Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d.....
>>         0x01, 0x00 (hex)
>>         In fact, print the value of r9 in sequence at first, but print
>>         unexpected value afterwards.   e.g 0x7f, 0x7e, 0x7d, ..,0x73,
>>         0x40, 0x85,....
>>
>>         ENTRY(__v7_flush_dcache_all)
>>         [snip]
>>         loop1:
>>             mov    r9, r7                @ create working copy of max
>>         index
>>         loop2:
>>         +      stmfd    sp!, {r0}
>>         +      ldr     r0, =0xff690000
>>         +      str     r9, [r0]
>>         +      ldmfd    sp!, {r0}
>>          ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and
>>         cache number into r11
>>          THUMB(    lsl    r6, r4, r5        )
>>          THUMB(    orr    r11, r10, r6        )    @ factor way and
>>         cache number into r11
>>          ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index
>>         number into r11
>>          THUMB(    lsl    r6, r9, r2        )
>>          THUMB(    orr    r11, r11, r6        )    @ factor index
>>         number into r11
>>             mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate
>>         by set/way
>>             subs    r9, r9, #1            @ decrement the index
>>             bge    loop2
>>             subs    r4, r4, #1            @ decrement the way
>>             bge    loop1
>>         skip:
>>         ENDPROC(__v7_flush_dcache_all)
>>
>>         I don't have the jtag, hence I can't address the current pc. I
>>         have no doubt that if any glitches in __v7_flush_dcache_all, I
>>         reviewed several times, also copy kernel's implement to here.
>>         :-( No effect.
>>         A more interesting thing is that Sandy had report it. He and I
>>         have similar problem. Everything work fine after I applied his
>>         patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
>>         @Stany, I'm sorry that I disable dcache during hack.. That was
>>         a mistake:-(
>>
>>         @Simon & hieko,
>>         Can you boot linux with the mainline u-boot? have a try?
>>
>>
>>         _______________________________________________
>>         U-Boot mailing list
>>         U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
>>         http://lists.denx.de/mailman/listinfo/u-boot
>>
>>
>>
>>
>>
>>
>>
>
>
> _______________________________________________
> U-Boot mailing list
> U-Boot@lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
diff mbox

Patch

diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c
index 6eac5ef..5cc09ba 100644
--- a/arch/arm/cpu/armv7/cpu.c
+++ b/arch/arm/cpu/armv7/cpu.c
@@ -43,6 +43,7 @@  int cleanup_before_linux_select(int flags)
                 */
                 dcache_disable();
                 v7_outer_cache_disable();
+               ISB;

                 /*
                 * After D-cache is flushed and before it is disabled