From patchwork Sun Dec 8 22:28:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lukas Auer X-Patchwork-Id: 1205776 X-Patchwork-Delegate: uboot@andestech.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 47WLbZ4YYVz9s4Y for ; Mon, 9 Dec 2019 09:29:30 +1100 (AEDT) Received: from phobos.denx.de (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id DF81E806D0; Sun, 8 Dec 2019 23:29:15 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Received: by phobos.denx.de (Postfix, from userid 109) id E63558066E; Sun, 8 Dec 2019 23:29:12 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=0.4 required=5.0 tests=KHOP_HELO_FCRDNS, SPF_HELO_NONE, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.2 Received: from mail-edgeS23.fraunhofer.de (mail-edges23.fraunhofer.de [153.97.7.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 5939B80661 for ; Sun, 8 Dec 2019 23:29:10 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=none (p=none dis=none) header.from=aisec.fraunhofer.de Authentication-Results: phobos.denx.de; spf=none smtp.mailfrom=lukas.auer@aisec.fraunhofer.de X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2HDBQB/eO1d/xoHYZllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88CoxuhhWdPwkBAQEBAQEBAQEHARMcAQGEQAKCFiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBrGIziQSBSAkBgSyHP2aDcw+BTD+BR4MbijcEjSuCP4hJliYHgURtbgSFTo8aDBuOSYtpqHUCBAIEBQIVgWkjgVgzGiSDO1ARFIxyFxWOD0ABATEBjjCBEAEB X-IPAS-Result: A2HDBQB/eO1d/xoHYZllHAEBAQEBBwEBEQEEBAEBgX6CIIFBAR88CoxuhhWdPwkBAQEBAQEBAQEHARMcAQGEQAKCFiQ4EwIQAQEFAQEBAQEFBAICaYVDhigGJwsBRhBRVwYOBYMigncBrGIziQSBSAkBgSyHP2aDcw+BTD+BR4MbijcEjSuCP4hJliYHgURtbgSFTo8aDBuOSYtpqHUCBAIEBQIVgWkjgVgzGiSDO1ARFIxyFxWOD0ABATEBjjCBEAEB X-IronPort-AV: E=Sophos;i="5.69,293,1571695200"; d="scan'208";a="14768693" Received: from mail-mtas26.fraunhofer.de ([153.97.7.26]) by mail-edgeS23.fraunhofer.de with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Dec 2019 23:29:09 +0100 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A0CIAgDBeO1dfRBhWMBlHAEBAQEBBwEBEQEEBAEBgX6DYQEfPAqMbqNUCQEDAQEBAQEHAS8BAYRAAoI5OBMCEAEBBAEBAQIBBQQUAQEWOoVKhVMDAycLAUYQUVcGDgWDIoJ4rGMziQSBSAkBgSyHP2aEAoFMP4FHgxuKNwSNK4I/iEmWJgeBRG1uBIVOjxoMG45Ji2modQIEAgQFAhWBaSGBWTMaJIM7UBEUjHIXFY4PQAEBATABjjCBEAEB X-IronPort-AV: E=Sophos;i="5.69,293,1571695200"; d="scan'208";a="99476115" Received: from fgdemucivp01ltm.xch.fraunhofer.de (HELO FGDEMUCIMP12EXC.ads.fraunhofer.de) ([192.88.97.16]) by mail-mtaS26.fraunhofer.de with ESMTP/TLS/AES256-SHA; 08 Dec 2019 23:29:07 +0100 Received: from xch-onprem-07.ads.fraunhofer.de (10.225.16.45) by FGDEMUCIMP12EXC.ads.fraunhofer.de (10.80.232.43) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sun, 8 Dec 2019 23:32:22 +0100 Received: from localhost.localdomain (10.225.16.62) by xch-onprem-07.ads.fraunhofer.de (10.225.16.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Sun, 8 Dec 2019 23:29:06 +0100 From: Lukas Auer To: Subject: [PATCH v2 3/4] riscv: add option to wait for ack from secondary harts in smp functions Date: Sun, 8 Dec 2019 23:28:51 +0100 Message-ID: <20191208222852.5506-4-lukas.auer@aisec.fraunhofer.de> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20191208222852.5506-1-lukas.auer@aisec.fraunhofer.de> References: <20191208222852.5506-1-lukas.auer@aisec.fraunhofer.de> MIME-Version: 1.0 X-Originating-IP: [10.225.16.62] X-ClientProxiedBy: xch-onprem-08.ads.fraunhofer.de (10.225.16.40) To xch-onprem-07.ads.fraunhofer.de (10.225.16.45) X-TM-AS-Product-Ver: SMEX-11.0.0.4179-8.200.1013-25090.004 X-TM-AS-Result: No-2.674600-8.000000-31 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.26 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marcus Comstedt Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.101.4 at phobos.denx.de X-Virus-Status: Clean Add a wait option to smp_call_function() to wait for the secondary harts to acknowledge the call-function request. The request is considered to be acknowledged once each secondary hart has cleared the corresponding IPI. As part of the call-function request, the secondary harts invalidate the instruction cache after clearing the IPI. This adds a delay between acknowledgment (clear IPI) and fulfillment (call function) of the request. We want to use the acknowledgment to be able to judge when the request has been completed. Remove the delay by clearing the IPI after cache invalidation and just before calling the function from the request. Signed-off-by: Lukas Auer Reviewed-by: Rick Chen Tested-by: Rick Chen Reviewed-by: Anup Patel --- Changes in v2: None arch/riscv/cpu/start.S | 2 ++ arch/riscv/include/asm/smp.h | 3 ++- arch/riscv/lib/bootm.c | 2 +- arch/riscv/lib/smp.c | 31 ++++++++++++++++++++++--------- arch/riscv/lib/spl.c | 2 +- common/spl/spl_opensbi.c | 2 +- 6 files changed, 29 insertions(+), 13 deletions(-) diff --git a/arch/riscv/cpu/start.S b/arch/riscv/cpu/start.S index 0a2ce6d691..60631638dd 100644 --- a/arch/riscv/cpu/start.S +++ b/arch/riscv/cpu/start.S @@ -197,6 +197,7 @@ spl_secondary_hart_stack_gd_setup: la a0, secondary_hart_relocate mv a1, s0 mv a2, s0 + mv a3, zero jal smp_call_function /* hang if relocation of secondary harts has failed */ @@ -337,6 +338,7 @@ relocate_secondary_harts: mv a1, s2 mv a2, s3 + mv a3, zero jal smp_call_function /* hang if relocation of secondary harts has failed */ diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h index bc863fdbaf..74de92ed13 100644 --- a/arch/riscv/include/asm/smp.h +++ b/arch/riscv/include/asm/smp.h @@ -46,8 +46,9 @@ void handle_ipi(ulong hart); * @addr: Address of function * @arg0: First argument of function * @arg1: Second argument of function + * @wait: Wait for harts to acknowledge request * @return 0 if OK, -ve on error */ -int smp_call_function(ulong addr, ulong arg0, ulong arg1); +int smp_call_function(ulong addr, ulong arg0, ulong arg1, int wait); #endif diff --git a/arch/riscv/lib/bootm.c b/arch/riscv/lib/bootm.c index efbd3e23e7..e96137a50c 100644 --- a/arch/riscv/lib/bootm.c +++ b/arch/riscv/lib/bootm.c @@ -99,7 +99,7 @@ static void boot_jump_linux(bootm_headers_t *images, int flag) if (IMAGE_ENABLE_OF_LIBFDT && images->ft_len) { #ifdef CONFIG_SMP ret = smp_call_function(images->ep, - (ulong)images->ft_addr, 0); + (ulong)images->ft_addr, 0, 0); if (ret) hang(); #endif diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c index 188a7e34bd..17adb35730 100644 --- a/arch/riscv/lib/smp.c +++ b/arch/riscv/lib/smp.c @@ -44,11 +44,11 @@ extern int riscv_clear_ipi(int hart); */ extern int riscv_get_ipi(int hart, int *pending); -static int send_ipi_many(struct ipi_data *ipi) +static int send_ipi_many(struct ipi_data *ipi, int wait) { ofnode node, cpus; u32 reg; - int ret; + int ret, pending; cpus = ofnode_path("/cpus"); if (!ofnode_valid(cpus)) { @@ -91,6 +91,15 @@ static int send_ipi_many(struct ipi_data *ipi) pr_err("Cannot send IPI to hart %d\n", reg); return ret; } + + if (wait) { + pending = 1; + while (pending) { + ret = riscv_get_ipi(reg, &pending); + if (ret) + return ret; + } + } } return 0; @@ -104,21 +113,25 @@ void handle_ipi(ulong hart) if (hart >= CONFIG_NR_CPUS) return; + __smp_mb(); + + smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; + invalidate_icache_all(); + + /* + * Clear the IPI to acknowledge the request before jumping to the + * requested function. + */ ret = riscv_clear_ipi(hart); if (ret) { pr_err("Cannot clear IPI of hart %ld\n", hart); return; } - __smp_mb(); - - smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr; - invalidate_icache_all(); - smp_function(hart, gd->arch.ipi[hart].arg0, gd->arch.ipi[hart].arg1); } -int smp_call_function(ulong addr, ulong arg0, ulong arg1) +int smp_call_function(ulong addr, ulong arg0, ulong arg1, int wait) { int ret = 0; struct ipi_data ipi; @@ -127,7 +140,7 @@ int smp_call_function(ulong addr, ulong arg0, ulong arg1) ipi.arg0 = arg0; ipi.arg1 = arg1; - ret = send_ipi_many(&ipi); + ret = send_ipi_many(&ipi, wait); return ret; } diff --git a/arch/riscv/lib/spl.c b/arch/riscv/lib/spl.c index a544df0a2b..dc7577f751 100644 --- a/arch/riscv/lib/spl.c +++ b/arch/riscv/lib/spl.c @@ -41,7 +41,7 @@ void __noreturn jump_to_image_no_args(struct spl_image_info *spl_image) debug("image entry point: 0x%lX\n", spl_image->entry_point); #ifdef CONFIG_SMP - ret = smp_call_function(spl_image->entry_point, (ulong)fdt_blob, 0); + ret = smp_call_function(spl_image->entry_point, (ulong)fdt_blob, 0, 0); if (ret) hang(); #endif diff --git a/common/spl/spl_opensbi.c b/common/spl/spl_opensbi.c index fed41b1e66..58bf2468ce 100644 --- a/common/spl/spl_opensbi.c +++ b/common/spl/spl_opensbi.c @@ -78,7 +78,7 @@ void spl_invoke_opensbi(struct spl_image_info *spl_image) #ifdef CONFIG_SMP ret = smp_call_function((ulong)spl_image->entry_point, (ulong)spl_image->fdt_addr, - (ulong)&opensbi_info); + (ulong)&opensbi_info, 0); if (ret) hang(); #endif