From patchwork Tue Aug 18 04:07:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 1346604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2001:8b0:10b:1231::1; helo=merlin.infradead.org; envelope-from=opensbi-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=merlin.20170209 header.b=qsP4GOk5; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=wdc.com header.i=@wdc.com header.a=rsa-sha256 header.s=dkim.wdc.com header.b=fUvZdzoC; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.a=rsa-sha256 header.s=selector2-sharedspace-onmicrosoft-com header.b=v29ldclp; dkim-atps=neutral Received: from merlin.infradead.org (merlin.infradead.org [IPv6:2001:8b0:10b:1231::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BVy7j1XtFz9sRK for ; Tue, 18 Aug 2020 14:08:16 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=Yrwq9sSMu7f5jdgmp3ERvtLmcLkWJDfGKkG7135qcUs=; b=qsP4GOk5LakCi9XaWufe3kcYW0 5OZ3rCyS0dJGIR+y4GP2U8BJgjFmIgjpzkcqgvdREi3MuRzqwA5OTJx+jyS685KPrnwIMyfv9x5FG 9laoytAsBA7qP5NXrLUjIdJWST543TqzGBaWtKETqtqSX/KZjrfUdcnaMNmGoxItZneAQXjMxvrTf l9gJVLfueX+2+q1b4o+ICv+Rez4feY/jCVVjsCgp5BXSyGWFdW22TufLPgJVIjd2oIPs44kprSBi0 Vxles1XoBJ9Zf7N1noyoed677PdaVQ+kI5GxQ7YmINRkQPQtT6HBhll5KAy3EwQmhefXZD9jd3X05 rKv/QQow==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7suq-00011a-8T; Tue, 18 Aug 2020 04:08:04 +0000 Received: from esa2.hgst.iphmx.com ([68.232.143.124]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1k7sum-0000k8-Ck for opensbi@lists.infradead.org; Tue, 18 Aug 2020 04:08:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1597723688; x=1629259688; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=9mqXcxvTN9JzshXPyi4ZIDEl+AmOdM4kC9REKw2XuA8=; b=fUvZdzoCRdw0xARH3VjwoCClPVBar8jlenCMjgGrn9BGvpOqNivrG+mn lU44RdsFAtBXz7xGbIxXTdgHuob2TZpyzHU7XbkEUMu65iP/NGqYRD9fr D1Pdt7ynd8wZWsUR6Eq0+/rw3d3Oe3nAKr+vlA1QkZyFZY+FriZQieq2y DO65UKNcGNuEoB0aS2ELSo2OgcSJzvSSd43QztkxGCDm8hQWEhCKdtEhY HmgsAQwcZLgEYnzpjuZ+o/noheQCyV0pGSfPYMFHx4jRvXe2Bbc440o+G 0c13ZHZu95UTyuKdmAigwzPY9spNqAjESH7jCH+14X9gvGr4x+rGhfetI A==; IronPort-SDR: mnyOBmAz+EOmCqJHYOD/NgrDPm94f11i9yOWyOW5Ba7Vh8mXYQS2SPiq2NFLOqu6GdBvzQbuVt Qvjp0tLAcqms/EHa8cdZHcH2jWIEzYGH8/dKmX6DwQmSzd8uCBvFyv+hvhW7KlAMma/NAbdfYU dCkfk2WdepR0wG3i6WYEUeZpnsttFummBsjU8D0NEyDoz09uCIJL5OjapBIYGbeHb0HMV6hfAd bGciTg4ksl/wUE/C7NeiJ+/svI3Kp3UMq3mjCw9FcUNK9Uo6/9cPw95xBxZk8T0EK6lbVb8QWu yeE= X-IronPort-AV: E=Sophos;i="5.76,326,1592841600"; d="scan'208";a="248356611" Received: from mail-dm6nam10lp2102.outbound.protection.outlook.com (HELO NAM10-DM6-obe.outbound.protection.outlook.com) ([104.47.58.102]) by ob1.hgst.iphmx.com with ESMTP; 18 Aug 2020 12:08:03 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nPQBRn6xfnUbtvOhz6+o8M+2JKm7epQoXnSTlNFSCdZefdjXjgt9xMkCtQfXFi7FC1XWICi/i4YLPHHoJPqfK9KzIa92s5Z+lnJLCOorI8g9ZCMcRn8Dqxr8fl1VP6CgaEaJsPhX5pZ5RMwIu9JV8ZJev7X8RmwfzO1BqtBL5VW7kQvW+SOdTSrjYKoGF2UeiNOynZGHzKyurtPK6/Fw5sJ7+JLP1SDUdHFPhgr9f3MGqewVbiGFW5gKX4yXd39UTJWDX1fmGAlPY+e5d6vyDqDqx+sAPNWavUlO/v1OR+M8sXvQnQQp28pUGm6gsEJdGAHbblONwLwqy4mXiIFoww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+es5KpH54ZYZslWgoPW8GvDZDowYlXxq0xc1++j+jRg=; b=WYh+iyukRj2M3VCY6wPL4cIxbt21Hk/E9/yRMAE7m19/qGlMSH2argGjEuVCl55N9IxjnSPT8KGMV7jiNxJfjhdvGtR90L+OtJnSRv32gbF0xsEpsV9ErE/jEa2epG2GK0lKA3G7dPPJZt2jfD4KBU7EzMjFy8HiX2T8xp7zYj8boZWSibRGp03fJPbAH+ZDW16JCWdUAwKbxsGgxz5ZblvEiKTWeb72Btp9/JSeeWYTAPfi5teASMKDfw4LXZxgnEUc34/J3AAJkyNu7qfVlFiAKWTyRtss4AhpMNaOEbGAjRYax1McG+a9qy7I9TPFz1rPKZtCnASgizfOtBLQFw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+es5KpH54ZYZslWgoPW8GvDZDowYlXxq0xc1++j+jRg=; b=v29ldclps2L9JZ/AUthZpbOXGt2TIe0fG+Pm/a8BqurizQ0P3ouLo8wmzUU00TxH22qpgu3042lbvnr9f4UrS//FWs+IymW7QM73xZJB7dY9hbTQFOrUWc76OWFwKh6JlgNZh7FkT+XP6UnjVPwpwdjP8/6I9O4WHmr4y/Gy+/c= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM5PR04MB3772.namprd04.prod.outlook.com (2603:10b6:3:fb::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3283.15; Tue, 18 Aug 2020 04:07:56 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::607a:44ed:1477:83e]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::607a:44ed:1477:83e%7]) with mapi id 15.20.3283.028; Tue, 18 Aug 2020 04:07:56 +0000 From: Anup Patel To: Atish Patra , Alistair Francis Subject: [PATCH v3 1/2] lib: sbi_init: Avoid thundering hurd problem with coldboot_lock Date: Tue, 18 Aug 2020 09:37:27 +0530 Message-Id: <20200818040728.528426-1-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 X-ClientProxiedBy: BM1PR0101CA0015.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:18::25) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (103.20.29.9) by BM1PR0101CA0015.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:18::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3283.15 via Frontend Transport; Tue, 18 Aug 2020 04:07:53 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [103.20.29.9] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 4ada2bbe-df56-4f19-baa2-08d8432c4962 X-MS-TrafficTypeDiagnostic: DM5PR04MB3772: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4VnZ7SwKWrNMtRVEaEJabeHGyM/nW7/vg8gSgo9rrLSgYCGNj4jDMIL9x56sjph4mNJp+zTC871sLuGb/VxHICHhwTYs6/PNBrG4nbRkONKiIRUykYpd7TAywxgm58Ki3atKNcxuGwezFR/3e8ypdAM2I6a2JBSgTa/O85obpFqXcMNde9rtYgOrmJ5UdTloBCyMA7u2Cj+0s6EUBMDlrQ21zV8wMx0LYFrak9hIS6TDYlGdyWvI6bfg/Uh1hnE0RLT9KDYmAuekE1N+uOWj9Z6xwWcugMfgHnZrkr6tSFhDivdj7yAVDIoa/g3BNwJ5 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(366004)(346002)(396003)(376002)(4326008)(6666004)(83380400001)(1076003)(8886007)(44832011)(186003)(66556008)(66476007)(2906002)(66946007)(8676002)(8936002)(86362001)(55016002)(7696005)(16526019)(5660300002)(316002)(26005)(2616005)(54906003)(36756003)(6636002)(956004)(52116002)(110136005)(478600001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: pjdQjDD1xyRrG2VcNd0zb7VvUdp9RHfk6ZXiZxq+JnH/MyyG/p8XY3N288OqSlVuNNQxoRANcIcgGIgZQeF0gA6r59OmmzGiHv+Hu9FodoDXOzrkQuHFOXyar0qDh2paxySb1G9Z8GnDz/YePysjeNeM79PWVxnNAezmlUU6HUZIqXQcwP1nFAwset+bCMlnVBZwPDeSZKITcMphCJ42/dfBPcT2XkLCT9IGwIYXQt49DV0M1cU5pXMr8e60kJHX2B9TqyWlHU9HtWV6W1mTRDU3VrdhubQeA5AYLlQFHj5jg5Bsmuv8/Xt8HXpqwfugE1YKohPlC/MZ050DcQModjRke7InpoMJ0uYnX6UlF74Di13MjRyys9VVbHjY1NJ1wDeiIFGN4bT3/bgY2GvfFhPCgGxVpTEwAdOU0xNJIoEW8X0JNcQsQI3bS+f82wKCKCq4hu6MBLsblkDuXNwD4HoB9c+EMJA9mqEJzlhIWJhYG33PyB8qT9Y2nlAZugfkkx8k7FxhUtUP/ui2ELQdxko9z+2KzpuQeIJmDsp985/sLuJVKKJyvr8IoNR/8qObpE4VwoKKM/IFGbgn1vPePGxBlR9ecqq+hgfXYhBPXmk1qh/UPQDG9wfR4hnJzotNJ42qYZR+W4ZN74Ig/Goh3Q== X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4ada2bbe-df56-4f19-baa2-08d8432c4962 X-MS-Exchange-CrossTenant-AuthSource: DM6PR04MB6201.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2020 04:07:56.3133 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aZ1JQ/a58gyHLFXK9AlBZjis9sAjxRsoAmFlFj58SFnA8gGAs+MgbiUN7aO12EhKSjB28DwQYVFDlrCanOEaBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR04MB3772 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200818_000800_644702_5E0AB9CF X-CRM114-Status: GOOD ( 16.58 ) X-Spam-Score: -2.5 (--) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-2.5 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [68.232.143.124 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 SPF_HELO_PASS SPF: HELO matches SPF record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.0 MSGID_FROM_MTA_HEADER Message-Id was added by a relay X-BeenThere: opensbi@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Heinrich Schuchardt , Anup Patel , opensbi@lists.infradead.org, Anup Patel Sender: "opensbi" Errors-To: opensbi-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org We can have thundering hurd problem with coldboot_lock where the boot HART can potentially starve trying to acquire coldboot_lock because some of the non-boot HARTs are continuously acquiring and releasing coldboot_lock. This can happen if MIP.MSIP bit is already set for some of the non-boot HARTs. To avoid thundering hurd problem for coldboot_lock, we use the __smp_load_acquire() and __smp_store_release() for coldboot_done flag and use coldboot_lock only for coldboot_wait_hmask. Signed-off-by: Anup Patel Tested-by: Heinrich Schuchardt Reviewed-by: Atish Patra Reviewed-by: Bin Meng --- lib/sbi/sbi_init.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/lib/sbi/sbi_init.c b/lib/sbi/sbi_init.c index a7fb848..c438eaa 100644 --- a/lib/sbi/sbi_init.c +++ b/lib/sbi/sbi_init.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -85,9 +86,10 @@ static void sbi_boot_prints(struct sbi_scratch *scratch, u32 hartid) } static spinlock_t coldboot_lock = SPIN_LOCK_INITIALIZER; -static unsigned long coldboot_done = 0; static struct sbi_hartmask coldboot_wait_hmask = { 0 }; +static unsigned long coldboot_done; + static void wait_for_coldboot(struct sbi_scratch *scratch, u32 hartid) { unsigned long saved_mie, cmip; @@ -105,16 +107,20 @@ static void wait_for_coldboot(struct sbi_scratch *scratch, u32 hartid) /* Mark current HART as waiting */ sbi_hartmask_set_hart(hartid, &coldboot_wait_hmask); + /* Release coldboot lock */ + spin_unlock(&coldboot_lock); + /* Wait for coldboot to finish using WFI */ - while (!coldboot_done) { - spin_unlock(&coldboot_lock); + while (!__smp_load_acquire(&coldboot_done)) { do { wfi(); cmip = csr_read(CSR_MIP); } while (!(cmip & MIP_MSIP)); - spin_lock(&coldboot_lock); }; + /* Acquire coldboot lock */ + spin_lock(&coldboot_lock); + /* Unmark current HART as waiting */ sbi_hartmask_clear_hart(hartid, &coldboot_wait_hmask); @@ -132,12 +138,12 @@ static void wake_coldboot_harts(struct sbi_scratch *scratch, u32 hartid) { const struct sbi_platform *plat = sbi_platform_ptr(scratch); + /* Mark coldboot done */ + __smp_store_release(&coldboot_done, 1); + /* Acquire coldboot lock */ spin_lock(&coldboot_lock); - /* Mark coldboot done */ - coldboot_done = 1; - /* Send an IPI to all HARTs waiting for coldboot */ for (int i = 0; i <= sbi_scratch_last_hartid(); i++) { if ((i != hartid) &&