| Message ID | 20200914142303.21307-4-seanga2@gmail.com |
|---|---|
| State | Superseded |
| Delegated to: | Andes |
| Headers | show |
| Series | riscv: Correctly handle IPIs already pending upon boot | expand |
On Mon, Sep 14, 2020 at 10:23 PM Sean Anderson <seanga2@gmail.com> wrote: > > Some IPIs may already be pending when U-Boot is started. This could be a > problem if a secondary hart tries to handle an IPI before the boot hart has > initialized the IPI device. > > This commit introduces a valid bit so secondary harts know when and IPI > originates from U-boot, and it is safe to use the IPI API. The valid bit is nits: U-Boot > initialized to 0 by board_init_f_init_reserve. Before this, secondary harts > wait in wait_for_gd_init. > > Signed-off-by: Sean Anderson <seanga2@gmail.com> > --- > > Changes in v2: > - Use a valid bit instead of addr to validate IPIs > > arch/riscv/include/asm/smp.h | 7 +++++++ > arch/riscv/lib/smp.c | 16 ++++++++++++++-- > 2 files changed, 21 insertions(+), 2 deletions(-) > Reviewed-by: Bin Meng <bin.meng@windriver.com>
> Some IPIs may already be pending when U-Boot is started. This could be a > problem if a secondary hart tries to handle an IPI before the boot hart has > initialized the IPI device. > > This commit introduces a valid bit so secondary harts know when and IPI > originates from U-boot, and it is safe to use the IPI API. The valid bit is > initialized to 0 by board_init_f_init_reserve. Before this, secondary harts > wait in wait_for_gd_init. > > Signed-off-by: Sean Anderson <seanga2@gmail.com> > --- > > Changes in v2: > - Use a valid bit instead of addr to validate IPIs > > arch/riscv/include/asm/smp.h | 7 +++++++ > arch/riscv/lib/smp.c | 16 ++++++++++++++-- > 2 files changed, 21 insertions(+), 2 deletions(-) Reviewed-by: Rick Chen <rick@andestech.com>
On Mon, Sep 14, 2020 at 10:22:59AM -0400, Sean Anderson wrote: > Some IPIs may already be pending when U-Boot is started. This could be a > problem if a secondary hart tries to handle an IPI before the boot hart has > initialized the IPI device. > > This commit introduces a valid bit so secondary harts know when and IPI > originates from U-boot, and it is safe to use the IPI API. The valid bit is > initialized to 0 by board_init_f_init_reserve. Before this, secondary harts > wait in wait_for_gd_init. > > Signed-off-by: Sean Anderson <seanga2@gmail.com> > Reviewed-by: Bin Meng <bin.meng@windriver.com> > Reviewed-by: Rick Chen <rick@andestech.com> > --- > > Changes in v2: > - Use a valid bit instead of addr to validate IPIs > > arch/riscv/include/asm/smp.h | 7 +++++++ > arch/riscv/lib/smp.c | 16 ++++++++++++++-- > 2 files changed, 21 insertions(+), 2 deletions(-) > > diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h > index 1b428856b2..2dae0800ce 100644 > --- a/arch/riscv/include/asm/smp.h > +++ b/arch/riscv/include/asm/smp.h > @@ -18,14 +18,21 @@ > * IPI data structure. The hart ID is inserted by the hart handling the IPI and > * calling the function. > * > + * @valid is used to determine whether a sent IPI originated from U-Boot. It is > + * initialized to zero by board_init_f_alloc_reserve. When U-Boot sends its > + * first IPI, it is set to 1. This prevents already-pending IPIs not sent by > + * U-Boot from being taken. > + * > * @addr: Address of function > * @arg0: First argument of function > * @arg1: Second argument of function > + * @valid: Whether this IPI is valid > */ > struct ipi_data { > ulong addr; > ulong arg0; > ulong arg1; > + unsigned int valid; > }; > > /** > diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c > index ab6d8bd7fa..8f33ce1fe3 100644 > --- a/arch/riscv/lib/smp.c > +++ b/arch/riscv/lib/smp.c > @@ -54,7 +54,13 @@ static int send_ipi_many(struct ipi_data *ipi, int wait) > gd->arch.ipi[reg].arg0 = ipi->arg0; > gd->arch.ipi[reg].arg1 = ipi->arg1; > > - __smp_mb(); > + /* > + * Ensure valid only becomes set when the IPI parameters are > + * set. An IPI may already be pending on other harts, so we > + * need a way to signal that the IPI device has been > + * initialized, and that it is ok to call the function. > + */ > + __smp_store_release(&gd->arch.ipi[reg].valid, 1); > > ret = riscv_send_ipi(reg); > if (ret) { > @@ -83,7 +89,13 @@ void handle_ipi(ulong hart) > if (hart >= CONFIG_NR_CPUS) > return; > > - __smp_mb(); > + /* > + * If valid is not set, then U-Boot has not requested the IPI. The > + * IPI device may not be initialized, so all we can do is wait for > + * U-Boot to initialize it and send an IPI > + */ > + if (!__smp_load_acquire(&gd->arch.ipi[hart].valid)) > + return; > > smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; > invalidate_icache_all(); Reviewed-by: Leo Liang <ycliang@andestech.com>
diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index 1b428856b2..2dae0800ce 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -18,14 +18,21 @@ * IPI data structure. The hart ID is inserted by the hart handling the IPI and * calling the function. * + * @valid is used to determine whether a sent IPI originated from U-Boot. It is + * initialized to zero by board_init_f_alloc_reserve. When U-Boot sends its + * first IPI, it is set to 1. This prevents already-pending IPIs not sent by + * U-Boot from being taken. + * * @addr: Address of function * @arg0: First argument of function * @arg1: Second argument of function + * @valid: Whether this IPI is valid */ struct ipi_data { ulong addr; ulong arg0; ulong arg1; + unsigned int valid; }; /** diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c index ab6d8bd7fa..8f33ce1fe3 100644 --- a/arch/riscv/lib/smp.c +++ b/arch/riscv/lib/smp.c @@ -54,7 +54,13 @@ static int send_ipi_many(struct ipi_data *ipi, int wait) gd->arch.ipi[reg].arg0 = ipi->arg0; gd->arch.ipi[reg].arg1 = ipi->arg1; - __smp_mb(); + /* + * Ensure valid only becomes set when the IPI parameters are + * set. An IPI may already be pending on other harts, so we + * need a way to signal that the IPI device has been + * initialized, and that it is ok to call the function. + */ + __smp_store_release(&gd->arch.ipi[reg].valid, 1); ret = riscv_send_ipi(reg); if (ret) { @@ -83,7 +89,13 @@ void handle_ipi(ulong hart) if (hart >= CONFIG_NR_CPUS) return; - __smp_mb(); + /* + * If valid is not set, then U-Boot has not requested the IPI. The + * IPI device may not be initialized, so all we can do is wait for + * U-Boot to initialize it and send an IPI + */ + if (!__smp_load_acquire(&gd->arch.ipi[hart].valid)) + return; smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; invalidate_icache_all();
Some IPIs may already be pending when U-Boot is started. This could be a problem if a secondary hart tries to handle an IPI before the boot hart has initialized the IPI device. This commit introduces a valid bit so secondary harts know when and IPI originates from U-boot, and it is safe to use the IPI API. The valid bit is initialized to 0 by board_init_f_init_reserve. Before this, secondary harts wait in wait_for_gd_init. Signed-off-by: Sean Anderson <seanga2@gmail.com> --- Changes in v2: - Use a valid bit instead of addr to validate IPIs arch/riscv/include/asm/smp.h | 7 +++++++ arch/riscv/lib/smp.c | 16 ++++++++++++++-- 2 files changed, 21 insertions(+), 2 deletions(-)