From patchwork Tue May 13 10:06:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084943 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3557-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXJM54wwz1yYN for ; Tue, 13 May 2025 20:09:23 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 260E61B43E57 for ; Tue, 13 May 2025 10:09:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C4CED244672; Tue, 13 May 2025 10:07:48 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 26342242D7B; Tue, 13 May 2025 10:07:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; cv=none; b=Nut5jM5jf1E/FCR+w/lwqoBprR9wBk7coSBoLdKzFM+Mu5fH4lw3MbAxSBaF5k+z9lgUGJGFVpCllx+PLdjUGloE39cO2uM5dgq03XoXiFsP3GS4FS3UY+AndFUFiGLuX2zjyoV1OxhugKUotxgBFNOdWXGlzPIfpOPjo8tmTCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; c=relaxed/simple; bh=tb6fnFpucKzItJPP15tgDT3weF1MGwlqqKLWWoBgkgc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=HamhEoDf2AJzk8CvJ1w8ivxTgcitpAi5znC4wFcXFWuLAMWjQORXnVfIrAmQl15S4yE09pOifJ7GNaydGnz1yQ3mdEJchvPdGWnie9OSN5p6b5DokMxGLPYvFkEl0/YfEU8BdfdKrN+qXT5J00lXjHjtWUlsczgmeCpq6eypD/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-d8-682319edcf23 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 01/43] llist: move llist_{head,node} definition to types.h Date: Tue, 13 May 2025 19:06:48 +0900 Message-Id: <20250513100730.12664-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTdxTG/b93qjWvHdFX0OBqjJlmKt5yCGLwFv/7sEWiUbO5bM14Y6tQ sQUUE02LxSBYREwBQbFWUggUwWIydNYhBASJiIAVFFGqorVUEqCNVby0GL+c/PI8z3nOl8OR sjt0BKdSp4oatSJJzkgoiXeG5Wfv3IXKFeUeOfgmsik4X2tjoOtKNQLbNT0B7pat8Mg/guDj vfskFJm6EFwaekrCtdZBBI7KTAZ6Xs6EXt8oA+2mXAaOX65l4IFnkoCBwgICqu2/wjPrMAUd +RYCitwMlBYdJ4LjDQEBaxULVt0icFWWsDA5FA3tg04aHI+XwrmyAQZuOtopaG1wEdBz4zwD g7YvNHS0tlHgz4uErjNGGmreWRjw+K0kWH2jLHQ3mgloNc+GOkOw8MT4ZxruGBsJOFF+lYDe /v8Q3Mp+ToDd5mSg2TdCQL3dRMKHihYErjwvC1mnAiyU6vMQ5GYVUmAYWAMf3wcvX5iIBv3F OgpqPjlRfBy2ldkQbh4ZJbGh/hD+4HvIYIffTOG7FgFfL3nKYsOtxyw229NwfeUSfPmmm8CX xnw0tledZLB9rIDFOd5eAr/r7GS3zftdsi5RTFKli5rl6/+WKCva9GxKseRwrc5C6dANLgdx nMCvFspf/ZKDwqbQ3d1Oh5jhFwt9fQEyxOH8AqHeOBzUJRzJO6cLjy70o5DxA/+b8KLbQYWY 4hcJ92wmNsRSfo1QUFpHfiuNEqrrGqc4jF8rfKronMrLgpl8czUVKhX4s2FCS+Zd9tvCXOF2 ZR+Vj6RmNK0KyVTq9GSFKmn1MmWGWnV42T8Hku0o+F7Wo5N/NKCxru1NiOeQfIa0zf2jUkYr 0rUZyU1I4Eh5uFT/b1CSJioyjoiaA39p0pJEbROK5Cj5HOlK/6FEGb9XkSruF8UUUfPdJbiw CB3amzB69fXppSXj468uBp4TEc9m+beO57oC6ifNO/3FE8d29eQZd/Qe3Bijznatjd2X3PDl //7oPfOj5jtnp76tMJYlzNoUnmWYGRvTZooaHkrbTZnObZj+ZE9Gc4rsdqYkviPOe/pMzebr xdt+wgnxpi2FnrO0bqBAgx1/rtI2HqU9ckqrVEQvITVaxVfKGFVYWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+/6e+3H2k/DDPJ2ap3nIHB9j+Eu/eRqThRlu9Zu7qavdVZyN 3amMKIlET66jK3eXztU8pLNWpNPk9CRJiJlTqXHXROKO+eez197v996ff94MHmQmpzJKVYKo VsljpBRLsFtXJy/qnzJHsdTlGQtezykC8sutFLhuWhBYK/UYuB+Fw4uhPgQ/nz7DISfbhaDo 3WscKuu7EThKT1DQ8mEctHoHKHBmn6Eg+Vo5Bc97RzDoupSFgcW+Bd6YPhLQmGnEIMdNQV5O MuY7nzAYNplpMOlCoac0l4aRd2Hg7G4noa7ASYKjcyFcKeyioNrhJKD+bg8GLVX5FHRbf5PQ WN9AwFDGNHCdTyeh7IuRgt4hEw4m7wANzTUGDOoNk8CW4ms9+W2UhMfpNRicvH4Lg9aX9xE8 OPUWA7u1nYI6bx8GFfZsHH6UPELQk9FPQ+rZYRry9BkIzqReIiClSwY/v/s+F3jCQH/VRkDZ r3a0fq1gLbQioa5vABdSKg4LP7xtlOAYMhDCEyMv3Mt9TQspDzppwWBPFCpKFwjXqt2YUPTV Swp282lKsH/NooW0/lZM+NLURG+bvoddEy3GKJNE9ZK1B1hFSYOejr/MHinXGQkdqmLSUCDD c8t5d7OT9DPFzeU7OoZxPwdzs/iK9I8+nWVwrn0M/6LgJfIbE7it/PtmB+Fnggvln1qzaT9L OBmflWfD/5XO5C22mr8cyK3gf5U0/c0H+TKZBguRiVgDCjCjYKUqKVaujJEt1hxSaFXKI4uj 4mLtyLcg07GR83eRpyW8FnEMko6VNLhnK4JIeZJGG1uLeAaXBkv0d3ySJFquPSqq4/arE2NE TS2axhDSyZKNkeKBIO6gPEE8JIrxovq/izGBU3VI/up4efSyeQONUYMBi5bWGkNWy/ZJZs4N KWr9XHxxe8b4NXtD2Yljvn0/t/vCeodkU650tnMwYKUrf3TzKll1Q+fDyPm735jjDTuqbhco 89PSl+i0VqNHVTQxorvjctmGQrYtIswyY11qf9SuSu3gWXHG1eLltnm5zybsC+94uNC584aU 0CjkYQtwtUb+ByZCBgM9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: llist_head and llist_node can be used by very primitives. For example, dept for tracking dependencies uses llist in its header. To avoid header dependency, move those to types.h. Signed-off-by: Byungchul Park --- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/llist.h b/include/linux/llist.h index 2c982ff7475a..3ac071857612 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -53,14 +53,6 @@ #include #include -struct llist_head { - struct llist_node *first; -}; - -struct llist_node { - struct llist_node *next; -}; - #define LLIST_HEAD_INIT(name) { NULL } #define LLIST_HEAD(name) struct llist_head name = LLIST_HEAD_INIT(name) diff --git a/include/linux/types.h b/include/linux/types.h index 49b79c8bb1a9..c727cc2249e8 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -204,6 +204,14 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +struct llist_head { + struct llist_node *first; +}; + +struct llist_node { + struct llist_node *next; +}; + struct ustat { __kernel_daddr_t f_tfree; #ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE From patchwork Tue May 13 10:06:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084966 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ide+bounces-3561-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXNT6Z2Wz1yPv for ; Tue, 13 May 2025 20:12:57 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 2458C7B7C89 for ; Tue, 13 May 2025 10:09:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B870D2512D5; Tue, 13 May 2025 10:07:52 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 23796242D7A; Tue, 13 May 2025 10:07:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130872; cv=none; b=IfikUk3KwkoDB8yhW03y6xcJXUofr1O7PoHEWt9Yc0BtijpuB0/hSDQNBXae1bgCTEY7RXrBJF7sYJ63u2U9WK4siukrLkv/BQnOzCmnNWb4vNUzOgdBQyMWp85aMIC8WYKwPcWOJC+kM1F5KavXiNs0x8PKVt6QTZyUgAOirVQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130872; c=relaxed/simple; bh=r9AOmCuG9olXiJLIoeJcAKKVSNJf9gWYAwWvkyMyvIU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=af1ldlo9qo8GoQx6XMCnvmIYjB7xlBIenNUweYgG3crtLtFn7NqjQ4J/Z8koWKtbpMpKSM7k5/cRN9AOjVCPgtoFalPuu+iO6/6iIzgsYm2vvNZWkCKH8n0NqesA01Iq2HFO4fh6YiGqbtq+7OtPUR6v3T4Xsfmg9+kkxY2f4vY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-e9-682319ed136a From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 02/43] dept: implement DEPT(DEPendency Tracker) Date: Tue, 13 May 2025 19:06:49 +0900 Message-Id: <20250513100730.12664-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+/3u3b3bcnFZUTczsoUFRpaRcaIHkUWXIOhBQQ+okbc2nBZT ZxbFzCWmaSaopZlr1pQ90qaVlZaZmVMyK9+ppUU0cmrqRjp7rFX/HD58z/ec7/njCAlpg8Bf qIyO5dXRcpWMEpNip59hmXPuIsWKqrtrwTWeQsK1UgsFLbfNCCwViRgcz7dCh3sQgeflKwJy s1sQ3OjvJaCivg9Bdck5Ct5+mgGtrmEK7NlpFCQVlVLw+usUhp6cLAxm23Z4b/xMQlOmAUOu g4L83CTsLV8wTBhNNBi1QTBQkkfDVH8o2PvaBVDdvRSuXu+hoKraTkJ95QCGtw+vUdBn+SWA pvoGEtwZ86DlcroArEMGCr66jQQYXcM0vKnRY6jXz4YynXdh8thPAbxIr8GQfPMOhtauRwge p3zAYLO0U/DMNYih3JZNwGTxcwQDGU4azl+coCE/MQNB2vkcEnQ9YeD57k0uGA+FxMIyEqw/ 2tHG9ZzlugVxzwaHCU5XHs9NutoortqtJ7lGA8s9yOulOd3jbprT2+K48pJgrqjKgbkboy4B ZzNdoDjbaBbNpTpbMTfU3EzvCNgvXhfBq5QaXr18w2GxYsyx5IRdS5003KrDWmRuI1ORSMgy q9gGTwf6z3UjWvoPU8wStrNzgvjDs5hAtjz9syAViYUE0z6d7Sjo8g3MZMLZXJPBZyKZIPaj NcmnS5gwtmbETPxduoA1l9X4WMSsZn8UN/uCpV5Ppt7874hLIvb1O8Vfnss+LekkM5FEj6aZ kFQZrYmSK1WrQhQJ0cqTIUeOR9mQ97+MZ6YOVKLRlt21iBEimZ+kwbFQIRXINTEJUbWIFRKy WZLE+15JEiFPOMWrjx9Sx6n4mFo0T0jK5khWuuMjpMwxeSwfyfMnePX/LhaK/LWISHNA82Cs f1ZOYM9BzbeRpPHJGfjqmcqf4fkFst2iRr/5jzx9u05vt2pFks1xoXkbrKbkfc6wcdOOpk0a +8PS6YWBa+ek7QwfUHfHVtVGNhboj2oXq1Qpe+/Mf3LFBv2zdW8C69bsgZCAxZ1nLepDQ9vu fXnqWf5qi7N44VgAESkjYxTy0GBCHSP/DYvpgTJbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/fXc4eaTzCcGZNhtrEZ2qtMeuZLb83htGNh7tVhzsim+1O XaOUq8mF4hxOuovcZZIup5SOJK5CO1HMHKVUF+kHV+afz157v997v//5MHiAmQxi5IpDglIh TZBQIkK0NiJlUdf0ebJQm44Db/9JAvJvWyhovGVGYCnVYOCpiYHXA50Ihp6/wEGf24jgSvs7 HEpr2xDYC09Q4Po0CZq83RQ4czMoSLl6m4KX34YxcJ/LwcBsjYX3ps8EPNMZMdB7KLioT8F8 5wsGg6YiGkzq+dBReIGG4fYwcLa1kFBd4CTB3roQzl9yU1BhdxJQW9aBgas8n4I2yx8SntXW ETCQNQMaszNJKP5upODbgAkHk7ebhlcOAwa1hqlQkuprTesbJeFJpgODtGt3MGh6+wBB5ckP GFgtLRRUezsxsFlzcfh9owZBR1YXDdrTgzRc1GQhyNCeIyDVHQ5Dv3zLBf1hoLlcQkDxSAuK juItlyyIr+7sxvlU2xH+t7eZ4u0DBoJ/auT4+xfe0XxqZSvNG6yHeVthCH+1woPxV3q9JG8t OkXx1t4cmk/vasL47w0N9PpZ20SRe4QEeZKgXBIVJ5L1eYIPONXUUeP1x5gamZuJdOTHcOxS 7nGPmh5jig3m3rwZxMc4kJ3D2TI/k+lIxOBsiz/3uuAtGjOmsKs4fZFxPESw87mPxSnjupgN 5xw9Zvxf6WzOXOIYZz92GTdyo2F8LMCX0RnMhA6JDGhCEQqUK5ISpfKE8MWqeFmyQn508e79 iVbkeyHT8eHsMtTviqlCLIMkE8V1nrmyAFKapEpOrEIcg0sCxZp7Pkm8R5p8TFDu36U8nCCo qtAMhpBME6/ZIsQFsPukh4R4QTggKP+7GOMXpEaEI9oOumuush0v89yR+tNnN669qQ2lJ/uT wX1fY/MNjtJbWyNXVfx0h+RtPmNTssOVK8vDZ9bXNW7ftzv+3rzmiDxv9Oon5b+Y0a6DtLae Wjfk6Vn+KEazYEX9pvLINfbWHwUKRc1eY1z73YexWtjsyi+sH82MCrr7Z2cosYPcICFUMmlY CK5USf8CnUytDT4DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: CURRENT STATUS -------------- lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account. lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design. PROBLEM ------- *Waits* and their *events* that never reach eventually cause deadlock. However, lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock. Even worse, no one likes lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to lockdep's false positive. Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. lockdep is no longer a good tool for that purpose. SOLUTION -------- Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, DEPT(DEPendency Tracker), focuses on waits and events themselves. dept tracks waits and events and report it if any event would be never reachable. dept does: . Works with read lock in the right way. . Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs. . Does exactly what dependency checker should do. Q & A ----- Q. Is this the first try ever to address the problem? A. No, cross-release feature (b09be676e0ff2 locking/lockdep: Implement the 'crossrelease' feature) addressed it that was a lockdep extension and merged but reverted shortly because: cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes lockdep's false positive reports since it makes lockdep stop, preventing reporting further real problems. Q. Why not dept was developed as an extension of lockdep? A. lockdep definitely includes all the efforts great developers have made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following: 1) lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model. 2) lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection. Q. Do you intend to totally replace lockdep? A. No, lockdep also checks if lock usage is correct. Of course, the dependency check routine should be replaced but the other functions should be still there. Q. Do you mean the dependency check routine should be replaced right away? A. No, I admit lockdep is stable enough thanks to great efforts kernel developers have made. lockdep and dept, both should be in the kernel until dept gets considered stable. Q. Stronger detection capability would give more false positive report. Which was a big problem when cross-release was introduced. Is it ok with dept? A. It's ok. dept allows multiple reporting thanks to simple and quite generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was. Signed-off-by: Byungchul Park Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Gwan-gyeong Mun Tested-by: Yeoreum Yun Tested-by: Yunseong Kim --- include/linux/dept.h | 445 +++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 110 +- init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 3002 +++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 4 + kernel/sched/core.c | 8 + lib/Kconfig.debug | 26 + lib/locking-selftest.c | 2 + 16 files changed, 3632 insertions(+), 2 deletions(-) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h diff --git a/include/linux/dept.h b/include/linux/dept.h new file mode 100644 index 000000000000..dc50fa9d388b --- /dev/null +++ b/include/linux/dept.h @@ -0,0 +1,445 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DEPT(DEPendency Tracker) - runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include + +struct task_struct; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt; +struct dept_iecxt { + struct dept_ecxt *ecxt; + int enirq; + /* + * for preventing to add a new ecxt + */ + bool staled; +}; + +struct dept_wait; +struct dept_iwait { + struct dept_wait *wait; + int irq; + /* + * for preventing to add a new wait + */ + bool staled; + bool touched; +}; + +struct dept_class { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * unique information about the class + */ + const char *name; + unsigned long key; + int sub_id; + + /* + * for BFS + */ + unsigned int bfs_gen; + struct dept_class *bfs_parent; + struct list_head bfs_node; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking all classes + */ + struct list_head all_node; + + /* + * for associating its dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for tracking IRQ dependencies + */ + struct dept_iecxt iecxt[DEPT_IRQS_NR]; + struct dept_iwait iwait[DEPT_IRQS_NR]; + + /* + * classified by a map embedded in task_struct, + * not an explicit map + */ + bool sched_map; + }; + }; +}; + +struct dept_key { + union { + /* + * Each byte-wise address will be used as its key. + */ + char base[DEPT_MAX_SUBCLASSES]; + + /* + * for caching the main class pointer + */ + struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE]; + }; +}; + +struct dept_map { + const char *name; + struct dept_key *keys; + + /* + * subclass that can be set from user + */ + int sub_u; + + /* + * It's local copy for fast access to the associated classes. + * Also used for dept_key for static maps. + */ + struct dept_key map_key; + + /* + * wait timestamp associated to this map + */ + unsigned int wgen; + + /* + * whether this map should be going to be checked or not + */ + bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER(n, k) \ +{ \ + .name = #n, \ + .keys = (struct dept_key *)(k), \ + .sub_u = 0, \ + .map_key = { .classes = { NULL, } }, \ + .wgen = 0U, \ + .nocheck = false, \ +} + +struct dept_stack { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * backtrace entries + */ + unsigned long raw[DEPT_MAX_STACK_ENTRY]; + int nr; + }; + }; +}; + +struct dept_ecxt { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function that entered to this ecxt + */ + const char *ecxt_fn; + + /* + * event function + */ + const char *event_fn; + + /* + * associated class + */ + struct dept_class *class; + + /* + * flag indicating which IRQ has been + * enabled within the event context + */ + unsigned long enirqf; + + /* + * where the IRQ-enabled happened + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + + /* + * where the event context started + */ + unsigned long ecxt_ip; + struct dept_stack *ecxt_stack; + + /* + * where the event triggered + */ + unsigned long event_ip; + struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function causing this wait + */ + const char *wait_fn; + + /* + * the associated class + */ + struct dept_class *class; + + /* + * which IRQ the wait was placed in + */ + unsigned long irqf; + + /* + * where the IRQ wait happened + */ + unsigned long irq_ip[DEPT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_IRQS_NR]; + + /* + * where the wait happened + */ + unsigned long wait_ip; + struct dept_stack *wait_stack; + + /* + * whether this wait is for commit in scheduler + */ + bool sched_sleep; + }; + }; +}; + +struct dept_dep { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * key data of dependency + */ + struct dept_ecxt *ecxt; + struct dept_wait *wait; + + /* + * This object can be referred without dept_lock + * held but with IRQ disabled, e.g. for hash + * lookup. So deferred deletion is needed. + */ + struct rcu_head rh; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking to a class object + */ + struct list_head dep_node; + struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { + /* + * hash table + */ + struct hlist_head *table; + + /* + * size of the table e.i. 2^bits + */ + int bits; +}; + +struct dept_ecxt_held { + /* + * associated event context + */ + struct dept_ecxt *ecxt; + + /* + * unique key for this dept_ecxt_held + */ + struct dept_map *map; + + /* + * class of the ecxt of this dept_ecxt_held + */ + struct dept_class *class; + + /* + * the wgen when the event context started + */ + unsigned int wgen; + + /* + * subclass that only works in the local context + */ + int sub_l; +}; + +struct dept_wait_hist { + /* + * associated wait + */ + struct dept_wait *wait; + + /* + * unique id of all waits system-wise until wrapped + */ + unsigned int wgen; + + /* + * local context id to identify IRQ context + */ + unsigned int ctxt_id; +}; + +extern void dept_on(void); +extern void dept_off(void); +extern void dept_init(void); +extern void dept_task_init(struct task_struct *t); +extern void dept_task_exit(struct task_struct *t); +extern void dept_free_range(void *start, unsigned int sz); + +extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_copy(struct dept_map *to, struct dept_map *from); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_request_event_wait_commit(void); +extern void dept_clean_stage(void); +extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); +extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); +extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); +extern void dept_request_event(struct dept_map *m); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); +extern void dept_sched_enter(void); +extern void dept_sched_exit(void); + +static inline void dept_ecxt_enter_nokeep(struct dept_map *m) +{ + dept_ecxt_enter(m, 0UL, 0UL, NULL, NULL, 0); +} + +/* + * for users who want to manage external keys + */ +extern void dept_key_init(struct dept_key *k); +extern void dept_key_destroy(struct dept_key *k); +extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, struct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const char *new_c_fn, const char *new_e_fn, int new_sub_l); + +extern void dept_softirq_enter(void); +extern void dept_hardirq_enter(void); +extern void dept_softirqs_on_ip(unsigned long ip); +extern void dept_hardirqs_on(void); +extern void dept_softirqs_off(void); +extern void dept_hardirqs_off(void); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; + +#define DEPT_MAP_INITIALIZER(n, k) { } + +#define dept_on() do { } while (0) +#define dept_off() do { } while (0) +#define dept_init() do { } while (0) +#define dept_task_init(t) do { } while (0) +#define dept_task_exit(t) do { } while (0) +#define dept_free_range(s, sz) do { } while (0) + +#define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_copy(t, f) do { } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_request_event_wait_commit() do { } while (0) +#define dept_clean_stage() do { } while (0) +#define dept_ttwu_stage_wait(t, ip) do { } while (0) +#define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) +#define dept_ecxt_holding(m, e_f) false +#define dept_request_event(m) do { } while (0) +#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit(m, e_f, ip) do { } while (0) +#define dept_sched_enter() do { } while (0) +#define dept_sched_exit() do { } while (0) +#define dept_ecxt_enter_nokeep(m) do { } while (0) +#define dept_key_init(k) do { (void)(k); } while (0) +#define dept_key_destroy(k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter() do { } while (0) +#define dept_hardirq_enter() do { } while (0) +#define dept_softirqs_on_ip(ip) do { } while (0) +#define dept_hardirqs_on() do { } while (0) +#define dept_softirqs_off() do { } while (0) +#define dept_hardirqs_off() do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */ diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4d4c06..bb279dbbe748 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ void irq_exit_rcu(void); */ #define __nmi_enter() \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() == NMI_MASK); \ @@ -128,6 +130,7 @@ void irq_exit_rcu(void); __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ arch_nmi_exit(); \ lockdep_on(); \ + dept_on(); \ } while (0) #define nmi_exit() \ diff --git a/include/linux/sched.h b/include/linux/sched.h index f96ac1982893..a1924b40feb5 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -48,6 +48,8 @@ #include #include #include +#include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -810,6 +812,110 @@ struct kmap_ctrl { #endif }; +#ifdef CONFIG_DEPT +struct dept_task { + /* + * all event contexts that have entered and before exiting + */ + struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD]; + int ecxt_held_pos; + + /* + * ring buffer holding all waits that have happened + */ + struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST]; + int wait_hist_pos; + + /* + * sequential id to identify each IRQ context + */ + unsigned int irq_id[DEPT_IRQS_NR]; + + /* + * for tracking IRQ-enabled points with cross-event + */ + unsigned int wgen_enirq[DEPT_IRQS_NR]; + + /* + * for keeping up-to-date IRQ-enabled points + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + + /* + * for reserving a current stack instance at each operation + */ + struct dept_stack *stack; + + /* + * for preventing recursive call into DEPT engine + */ + int recursive; + + /* + * for preventing reentrance to WARN*() while warning + */ + int in_warning; + + /* + * for staging data to commit a wait + */ + struct dept_map stage_m; + struct dept_map *stage_real_m; + bool stage_sched_map; + const char *stage_w_fn; + unsigned long stage_ip; + arch_spinlock_t stage_lock; + + /* + * the number of missing ecxts + */ + int missing_ecxt; + + /* + * for tracking IRQ-enable state + */ + bool hardirqs_enabled; + bool softirqs_enabled; + + /* + * whether the current is on do_exit() + */ + bool task_exit; + + /* + * whether the current is running __schedule() + */ + bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER(t) \ +{ \ + .wait_hist = { { .wait = NULL, } }, \ + .ecxt_held_pos = 0, \ + .wait_hist_pos = 0, \ + .irq_id = { 0U }, \ + .wgen_enirq = { 0U }, \ + .enirq_ip = { 0UL }, \ + .stack = NULL, \ + .recursive = 0, \ + .in_warning = 0, \ + .stage_m = DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_real_m = NULL, \ + .stage_sched_map = false, \ + .stage_w_fn = NULL, \ + .stage_ip = 0UL, \ + .stage_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ + .missing_ecxt = 0, \ + .hardirqs_enabled = false, \ + .softirqs_enabled = false, \ + .task_exit = false, \ + .in_sched = false, \ +} +#else +struct dept_task { }; +#define DEPT_TASK_INITIALIZER(t) { } +#endif + struct task_struct { #ifdef CONFIG_THREAD_INFO_IN_TASK /* @@ -1267,6 +1373,8 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; #endif + struct dept_task dept_task; + #if defined(CONFIG_UBSAN) && !defined(CONFIG_UBSAN_TRAP) unsigned int in_ubsan; #endif @@ -2210,8 +2318,6 @@ extern bool sched_task_on_rq(struct task_struct *p); extern unsigned long get_wchan(struct task_struct *p); extern struct task_struct *cpu_curr_snapshot(int cpu); -#include - /* * In order to reduce various lock holder preemption latencies provide an * interface to see if a vCPU is currently running or not. diff --git a/init/init_task.c b/init/init_task.c index e557f622bd90..84da2464c390 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -14,6 +14,7 @@ #include #include #include +#include #include @@ -204,6 +205,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = { .curr_chain_key = INITIAL_CHAIN_KEY, .lockdep_recursion = 0, #endif + .dept_task = DEPT_TASK_INITIALIZER(init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack = NULL, .tracing_graph_pause = ATOMIC_INIT(0), diff --git a/init/main.c b/init/main.c index 7f0a2a3dbd29..9b91f4d3b518 100644 --- a/init/main.c +++ b/init/main.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -1037,6 +1038,7 @@ void start_kernel(void) panic_param); lockdep_init(); + dept_init(); /* * Need to run this when irqs are enabled, because it wants diff --git a/kernel/Makefile b/kernel/Makefile index 434929de17ef..ee83774d4c73 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -56,6 +56,7 @@ obj-y += livepatch/ obj-y += dma/ obj-y += entry/ obj-$(CONFIG_MODULES) += module/ +obj-y += dependency/ obj-$(CONFIG_KCMP) += kcmp.o obj-$(CONFIG_FREEZER) += freezer.o diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile new file mode 100644 index 000000000000..b5cfb8a03c0c --- /dev/null +++ b/kernel/dependency/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) += dept.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c new file mode 100644 index 000000000000..9a7a8693a738 --- /dev/null +++ b/kernel/dependency/dept.c @@ -0,0 +1,3002 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DEPT(DEPendency Tracker) - Runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * + * DEPT provides a general way to detect deadlock possibility in runtime + * and the interest is not limited to typical lock but to every + * syncronization primitives. + * + * The following ideas were borrowed from LOCKDEP: + * + * 1) Use a graph to track relationship between classes. + * 2) Prevent performance regression using hash. + * + * The following items were enhanced from LOCKDEP: + * + * 1) Cover more deadlock cases. + * 2) Allow muliple reports. + * + * TODO: Both LOCKDEP and DEPT should co-exist until DEPT is considered + * stable. Then the dependency check routine should be replaced with + * DEPT after. It should finally look like: + * + * + * + * As is: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | +-------------------------------------+ | + * | | Dependency check | | + * | | (by tracking lock acquisition order)| | + * | +-------------------------------------+ | + * | | + * +-----------------------------------------+ + * + * DEPT + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + * + * + * + * To be: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | (Request dependency check) | + * | T | + * +--------------------|--------------------+ + * | + * DEPT V + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static int dept_stop; +static int dept_per_cpu_ready; + +static inline struct dept_task *dept_task(void) +{ + return ¤t->dept_task; +} + +#define DEPT_READY_WARN (!oops_in_progress && !dept_task()->in_warning) + +/* + * Make all operations using DEPT_WARN_ON() fail on oops_in_progress and + * prevent warning message. + */ +#define DEPT_WARN_ON_ONCE(c) \ + ({ \ + int __ret = !!(c); \ + \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN_ONCE(c, "DEPT_WARN_ON_ONCE: " #c); \ + --dept_task()->in_warning; \ + } \ + __ret; \ + }) + +#define DEPT_WARN_ONCE(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN_ONCE(1, "DEPT_WARN_ONCE: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_WARN_ON(c) \ + ({ \ + int __ret = !!(c); \ + \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(c, "DEPT_WARN_ON: " #c); \ + --dept_task()->in_warning; \ + } \ + __ret; \ + }) + +#define DEPT_WARN(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(1, "DEPT_WARN: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_STOP(s...) \ + ({ \ + WRITE_ONCE(dept_stop, 1); \ + if (likely(DEPT_READY_WARN)) { \ + ++dept_task()->in_warning; \ + WARN(1, "DEPT_STOP: " s); \ + --dept_task()->in_warning; \ + } \ + }) + +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) + +static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; + +/* + * DEPT internal engine should be careful in using outside functions + * e.g. printk at reporting since that kind of usage might cause + * untrackable deadlock. + */ +static atomic_t dept_outworld = ATOMIC_INIT(0); + +static void dept_outworld_enter(void) +{ + atomic_inc(&dept_outworld); +} + +static void dept_outworld_exit(void) +{ + atomic_dec(&dept_outworld); +} + +static bool dept_outworld_entered(void) +{ + return atomic_read(&dept_outworld); +} + +static bool dept_lock(void) +{ + while (!arch_spin_trylock(&dept_spin)) + if (unlikely(dept_outworld_entered())) + return false; + return true; +} + +static void dept_unlock(void) +{ + arch_spin_unlock(&dept_spin); +} + +enum bfs_ret { + BFS_CONTINUE, + BFS_DONE, + BFS_SKIP, +}; + +static bool before(unsigned int a, unsigned int b) +{ + return (int)(a - b) < 0; +} + +static bool valid_stack(struct dept_stack *s) +{ + return s && s->nr > 0; +} + +static bool valid_class(struct dept_class *c) +{ + return c->key; +} + +static void invalidate_class(struct dept_class *c) +{ + c->key = 0UL; +} + +static struct dept_ecxt *dep_e(struct dept_dep *d) +{ + return d->ecxt; +} + +static struct dept_wait *dep_w(struct dept_dep *d) +{ + return d->wait; +} + +static struct dept_class *dep_fc(struct dept_dep *d) +{ + return dep_e(d)->class; +} + +static struct dept_class *dep_tc(struct dept_dep *d) +{ + return dep_w(d)->class; +} + +static const char *irq_str(int irq) +{ + if (irq == DEPT_SIRQ) + return "softirq"; + if (irq == DEPT_HIRQ) + return "hardirq"; + return "(unknown)"; +} + +/* + * Dept doesn't work either when it's stopped by DEPT_STOP() or in a nmi + * context. + */ +static bool dept_working(void) +{ + return !READ_ONCE(dept_stop) && !in_nmi(); +} + +/* + * Even k == NULL is considered as a valid key because it would use + * &->map_key as the key in that case. + */ +struct dept_key __dept_no_validate__; +static bool valid_key(struct dept_key *k) +{ + return &__dept_no_validate__ != k; +} + +/* + * Pool + * ===================================================================== + * DEPT maintains pools to provide objects in a safe way. + * + * 1) Static pool is used at the beginning of booting time. + * 2) Local pool is tried first before the static pool. Objects that + * have been freed will be placed. + */ + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +#define OBJECT(id, nr) \ +static struct dept_##id spool_##id[nr]; \ +static DEFINE_PER_CPU(struct llist_head, lpool_##id); + #include "dept_object.h" +#undef OBJECT + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +static struct dept_pool pool[OBJECT_NR] = { +#define OBJECT(id, nr) { \ + .name = #id, \ + .obj_sz = sizeof(struct dept_##id), \ + .obj_nr = ATOMIC_INIT(nr), \ + .node_off = offsetof(struct dept_##id, pool_node), \ + .spool = spool_##id, \ + .lpool = &lpool_##id, }, + #include "dept_object.h" +#undef OBJECT +}; + +/* + * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is + * enabled or not because NMI and other contexts in the same CPU never + * run inside of DEPT concurrently by preventing reentrance. + */ +static void *from_pool(enum object_t t) +{ + struct dept_pool *p; + struct llist_head *h; + struct llist_node *n; + + /* + * llist_del_first() doesn't allow concurrent access e.g. + * between process and IRQ context. + */ + if (DEPT_WARN_ON(!irqs_disabled())) + return NULL; + + p = &pool[t]; + + /* + * Try local pool first. + */ + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + n = llist_del_first(h); + if (n) + return (void *)n - p->node_off; + + /* + * Try static pool. + */ + if (atomic_read(&p->obj_nr) > 0) { + int idx = atomic_dec_return(&p->obj_nr); + + if (idx >= 0) + return p->spool + (idx * p->obj_sz); + } + + DEPT_INFO_ONCE("---------------------------------------------\n" + " Some of Dept internal resources are run out.\n" + " Dept might still work if the resources get freed.\n" + " However, the chances are Dept will suffer from\n" + " the lack from now. Needs to extend the internal\n" + " resource pools. Ask max.byungchul.park@gmail.com\n"); + return NULL; +} + +static void to_pool(void *o, enum object_t t) +{ + struct dept_pool *p = &pool[t]; + struct llist_head *h; + + preempt_disable(); + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + llist_add(o + p->node_off, h); + preempt_enable(); +} + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); \ +static void (*dtor_##id)(struct dept_##id *a); \ +static struct dept_##id *new_##id(void) \ +{ \ + struct dept_##id *a; \ + \ + a = (struct dept_##id *)from_pool(OBJECT_##id); \ + if (unlikely(!a)) \ + return NULL; \ + \ + atomic_set(&a->ref, 1); \ + \ + if (ctor_##id) \ + ctor_##id(a); \ + \ + return a; \ +} \ + \ +static struct dept_##id *get_##id(struct dept_##id *a) \ +{ \ + atomic_inc(&a->ref); \ + return a; \ +} \ + \ +static void put_##id(struct dept_##id *a) \ +{ \ + if (!atomic_dec_return(&a->ref)) { \ + if (dtor_##id) \ + dtor_##id(a); \ + to_pool(a, OBJECT_##id); \ + } \ +} \ + \ +static void del_##id(struct dept_##id *a) \ +{ \ + put_##id(a); \ +} \ + \ +static bool __maybe_unused id##_consumed(struct dept_##id *a) \ +{ \ + return a && atomic_read(&a->ref) > 1; \ +} +#include "dept_object.h" +#undef OBJECT + +#define SET_CONSTRUCTOR(id, f) \ +static void (*ctor_##id)(struct dept_##id *a) = f + +static void initialize_dep(struct dept_dep *d) +{ + INIT_LIST_HEAD(&d->dep_node); + INIT_LIST_HEAD(&d->dep_rev_node); +} +SET_CONSTRUCTOR(dep, initialize_dep); + +static void initialize_class(struct dept_class *c) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iecxt *ie = &c->iecxt[i]; + struct dept_iwait *iw = &c->iwait[i]; + + ie->ecxt = NULL; + ie->enirq = i; + ie->staled = false; + + iw->wait = NULL; + iw->irq = i; + iw->staled = false; + iw->touched = false; + } + c->bfs_gen = 0U; + + INIT_LIST_HEAD(&c->all_node); + INIT_LIST_HEAD(&c->dep_head); + INIT_LIST_HEAD(&c->dep_rev_head); + INIT_LIST_HEAD(&c->bfs_node); +} +SET_CONSTRUCTOR(class, initialize_class); + +static void initialize_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + e->enirq_stack[i] = NULL; + e->enirq_ip[i] = 0UL; + } + e->ecxt_ip = 0UL; + e->ecxt_stack = NULL; + e->enirqf = 0UL; + e->event_ip = 0UL; + e->event_stack = NULL; +} +SET_CONSTRUCTOR(ecxt, initialize_ecxt); + +static void initialize_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + w->irq_stack[i] = NULL; + w->irq_ip[i] = 0UL; + } + w->wait_ip = 0UL; + w->wait_stack = NULL; + w->irqf = 0UL; +} +SET_CONSTRUCTOR(wait, initialize_wait); + +static void initialize_stack(struct dept_stack *s) +{ + s->nr = 0; +} +SET_CONSTRUCTOR(stack, initialize_stack); + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_CONSTRUCTOR + +#define SET_DESTRUCTOR(id, f) \ +static void (*dtor_##id)(struct dept_##id *a) = f + +static void destroy_dep(struct dept_dep *d) +{ + if (dep_e(d)) + put_ecxt(dep_e(d)); + if (dep_w(d)) + put_wait(dep_w(d)); +} +SET_DESTRUCTOR(dep, destroy_dep); + +static void destroy_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (e->enirq_stack[i]) + put_stack(e->enirq_stack[i]); + if (e->class) + put_class(e->class); + if (e->ecxt_stack) + put_stack(e->ecxt_stack); + if (e->event_stack) + put_stack(e->event_stack); +} +SET_DESTRUCTOR(ecxt, destroy_ecxt); + +static void destroy_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (w->irq_stack[i]) + put_stack(w->irq_stack[i]); + if (w->class) + put_class(w->class); + if (w->wait_stack) + put_stack(w->wait_stack); +} +SET_DESTRUCTOR(wait, destroy_wait); + +#define OBJECT(id, nr) \ +static void (*dtor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_DESTRUCTOR + +/* + * Caching and hashing + * ===================================================================== + * DEPT makes use of caching and hashing to improve performance. Each + * object can be obtained in O(1) with its key. + * + * NOTE: Currently we assume all the objects in the hashs will never be + * removed. Implement it when needed. + */ + +/* + * Some information might be lost but it's only for hashing key. + */ +static unsigned long mix(unsigned long a, unsigned long b) +{ + int halfbits = sizeof(unsigned long) * 8 / 2; + unsigned long halfmask = (1UL << halfbits) - 1UL; + + return (a << halfbits) | (b & halfmask); +} + +static bool cmp_dep(struct dept_dep *d1, struct dept_dep *d2) +{ + return dep_fc(d1)->key == dep_fc(d2)->key && + dep_tc(d1)->key == dep_tc(d2)->key; +} + +static unsigned long key_dep(struct dept_dep *d) +{ + return mix(dep_fc(d)->key, dep_tc(d)->key); +} + +static bool cmp_class(struct dept_class *c1, struct dept_class *c2) +{ + return c1->key == c2->key; +} + +static unsigned long key_class(struct dept_class *c) +{ + return c->key; +} + +#define HASH(id, bits) \ +static struct hlist_head table_##id[1 << (bits)]; \ + \ +static struct hlist_head *head_##id(struct dept_##id *a) \ +{ \ + return table_##id + hash_long(key_##id(a), bits); \ +} \ + \ +static struct dept_##id *hash_lookup_##id(struct dept_##id *a) \ +{ \ + struct dept_##id *b; \ + \ + hlist_for_each_entry_rcu(b, head_##id(a), hash_node) \ + if (cmp_##id(a, b)) \ + return b; \ + return NULL; \ +} \ + \ +static void hash_add_##id(struct dept_##id *a) \ +{ \ + get_##id(a); \ + hlist_add_head_rcu(&a->hash_node, head_##id(a)); \ +} \ + \ +static void hash_del_##id(struct dept_##id *a) \ +{ \ + hlist_del_rcu(&a->hash_node); \ + put_##id(a); \ +} +#include "dept_hash.h" +#undef HASH + +static struct dept_dep *lookup_dep(struct dept_class *fc, + struct dept_class *tc) +{ + struct dept_ecxt onetime_e = { .class = fc }; + struct dept_wait onetime_w = { .class = tc }; + struct dept_dep onetime_d = { .ecxt = &onetime_e, + .wait = &onetime_w }; + return hash_lookup_dep(&onetime_d); +} + +static struct dept_class *lookup_class(unsigned long key) +{ + struct dept_class onetime_c = { .key = key }; + + return hash_lookup_class(&onetime_c); +} + +/* + * Report + * ===================================================================== + * DEPT prints useful information to help debuging on detection of + * problematic dependency. + */ + +static void print_ip_stack(unsigned long ip, struct dept_stack *s) +{ + if (ip) + print_ip_sym(KERN_WARNING, ip); + +#ifdef CONFIG_DEPT_DEBUG + if (!s) + pr_warn("stack is NULL.\n"); + else if (!s->nr) + pr_warn("stack->nr is 0.\n"); + if (s) + pr_warn("stack ref is %d.\n", atomic_read(&s->ref)); +#endif + + if (valid_stack(s)) { + pr_warn("stacktrace:\n"); + stack_trace_print(s->raw, s->nr, 5); + } + + if (!ip && !valid_stack(s)) + pr_warn("(N/A)\n"); +} + +#define print_spc(spc, fmt, ...) \ + pr_warn("%*c" fmt, (spc) * 3, ' ', ##__VA_ARGS__) + +static void print_diagram(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + bool firstline = true; + int spc = 1; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (!firstline) + pr_warn("\nor\n\n"); + firstline = false; + + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, " <%s interrupt>\n", irq_str(irq)); + print_spc(spc + 1, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } + + if (!irqf) { + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } +} + +static void print_dep(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + pr_warn("%s has been enabled:\n", irq_str(irq)); + print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); + pr_warn("\n"); + + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d) in %s context:\n", + w_fn, tc_n, tc->sub_id, irq_str(irq)); + print_ip_stack(w->irq_ip[irq], w->irq_stack[irq]); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } + + if (!irqf) { + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d):\n", w_fn, tc_n, tc->sub_id); + print_ip_stack(w->wait_ip, w->wait_stack); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } +} + +static void save_current_stack(int skip); + +/* + * Print all classes in a circle. + */ +static void print_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + int i; + + dept_outworld_enter(); + save_current_stack(6); + + pr_warn("===================================================\n"); + pr_warn("DEPT: Circular dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + pr_warn("summary\n"); + pr_warn("---------------------------------------------------\n"); + + if (fc == tc) + pr_warn("*** AA DEADLOCK ***\n\n"); + else + pr_warn("*** DEADLOCK ***\n\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + if (fc != c) + pr_warn("\n"); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("\n"); + pr_warn("[S]: start of the event context\n"); + pr_warn("[W]: the wait blocked\n"); + pr_warn("[E]: the event not reachable\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c's detail\n", 'A' + i); + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + pr_warn("\n"); + print_dep(d); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +/* + * BFS(Breadth First Search) + * ===================================================================== + * Whenever a new dependency is added into the graph, search the graph + * for a new circular dependency. + */ + +struct bfs_ops { + void (*bfs_init)(void *, void *, void **); + void (*extend)(struct list_head *, void *); + void *(*dequeue)(struct list_head *); + enum bfs_ret (*callback)(void *, void *, void **); +}; + +static unsigned int bfs_gen; + +/* + * NOTE: Must be called with dept_lock held. + */ +static void bfs(void *root, struct bfs_ops *ops, void *in, void **out) +{ + LIST_HEAD(q); + enum bfs_ret ret; + + if (DEPT_WARN_ON(!ops || !ops->bfs_init || !ops->extend || + !ops->dequeue || !ops->callback)) + return; + + /* + * Avoid zero bfs_gen. + */ + bfs_gen = bfs_gen + 1 ?: 1; + ops->bfs_init(root, in, out); + + ret = ops->callback(root, in, out); + if (ret != BFS_CONTINUE) + return; + + ops->extend(&q, root); + while (!list_empty(&q)) { + void *node = ops->dequeue(&q); + + if (ret == BFS_DONE) + continue; + + ret = ops->callback(node, in, out); + if (ret == BFS_CONTINUE) + ops->extend(&q, node); + } +} + +/* + * Main operations + * ===================================================================== + * Add dependencies - Each new dependency is added into the graph and + * checked if it forms a circular dependency. + * + * Track waits - Waits are queued into the ring buffer for later use to + * generate appropriate dependencies with cross-event. + * + * Track event contexts(ecxt) - Event contexts are pushed into local + * stack for later use to generate appropriate dependencies with waits. + */ + +static unsigned long cur_enirqf(void); +static int cur_irq(void); +static unsigned int cur_ctxt_id(void); + +static struct dept_iecxt *iecxt(struct dept_class *c, int irq) +{ + return &c->iecxt[irq]; +} + +static struct dept_iwait *iwait(struct dept_class *c, int irq) +{ + return &c->iwait[irq]; +} + +static void stale_iecxt(struct dept_iecxt *ie) +{ + if (ie->ecxt) + put_ecxt(ie->ecxt); + + WRITE_ONCE(ie->ecxt, NULL); + WRITE_ONCE(ie->staled, true); +} + +static void set_iecxt(struct dept_iecxt *ie, struct dept_ecxt *e) +{ + /* + * ->ecxt will never be updated once getting set until the class + * gets removed. + */ + if (ie->ecxt) + DEPT_WARN_ON(1); + else + WRITE_ONCE(ie->ecxt, get_ecxt(e)); +} + +static void stale_iwait(struct dept_iwait *iw) +{ + if (iw->wait) + put_wait(iw->wait); + + WRITE_ONCE(iw->wait, NULL); + WRITE_ONCE(iw->staled, true); +} + +static void set_iwait(struct dept_iwait *iw, struct dept_wait *w) +{ + /* + * ->wait will never be updated once getting set until the class + * gets removed. + */ + if (iw->wait) + DEPT_WARN_ON(1); + else + WRITE_ONCE(iw->wait, get_wait(w)); + + iw->touched = true; +} + +static void touch_iwait(struct dept_iwait *iw) +{ + iw->touched = true; +} + +static void untouch_iwait(struct dept_iwait *iw) +{ + iw->touched = false; +} + +static struct dept_stack *get_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + return s ? get_stack(s) : NULL; +} + +static void prepare_current_stack(void) +{ + DEPT_WARN_ON(dept_task()->stack); + + dept_task()->stack = new_stack(); +} + +static void save_current_stack(int skip) +{ + struct dept_stack *s = dept_task()->stack; + + if (!s) + return; + + if (valid_stack(s)) + return; + + s->nr = stack_trace_save(s->raw, DEPT_MAX_STACK_ENTRY, skip); +} + +static void finish_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + /* + * Fill the struct dept_stack with a valid stracktrace if it has + * been referred at least once. + */ + if (stack_consumed(s)) + save_current_stack(2); + + dept_task()->stack = NULL; + + /* + * Actual deletion will happen at put_stack() if the stack has + * been referred. + */ + if (s) + del_stack(s); +} + +/* + * FIXME: For now, disable LOCKDEP while DEPT is working. + * + * Both LOCKDEP and DEPT report it on a deadlock detection using + * printk taking the risk of another deadlock that might be caused by + * locks of console or printk between inside and outside of them. + * + * For DEPT, it's no problem since multiple reports are allowed. But it + * would be a bad idea for LOCKDEP since it will stop even on a singe + * report. So we need to prevent LOCKDEP from its reporting the risk + * DEPT would take when reporting something. + */ +#include + +void noinstr dept_off(void) +{ + dept_task()->recursive++; + lockdep_off(); +} + +void noinstr dept_on(void) +{ + lockdep_on(); + dept_task()->recursive--; +} + +static unsigned long dept_enter(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + dept_off(); + prepare_current_stack(); + return flags; +} + +static void dept_exit(unsigned long flags) +{ + finish_current_stack(); + dept_on(); + arch_local_irq_restore(flags); +} + +static unsigned long dept_enter_recursive(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + return flags; +} + +static void dept_exit_recursive(unsigned long flags) +{ + arch_local_irq_restore(flags); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static struct dept_dep *__add_dep(struct dept_ecxt *e, + struct dept_wait *w) +{ + struct dept_dep *d; + + if (DEPT_WARN_ON(!valid_class(e->class))) + return NULL; + + if (DEPT_WARN_ON(!valid_class(w->class))) + return NULL; + + if (lookup_dep(e->class, w->class)) + return NULL; + + d = new_dep(); + if (unlikely(!d)) + return NULL; + + d->ecxt = get_ecxt(e); + d->wait = get_wait(w); + + /* + * Add the dependency into hash and graph. + */ + hash_add_dep(d); + list_add(&d->dep_node, &dep_fc(d)->dep_head); + list_add(&d->dep_rev_node, &dep_tc(d)->dep_rev_head); + return d; +} + +static void bfs_init_check_dl(void *node, void *in, void **out) +{ + struct dept_class *root = (struct dept_class *)node; + struct dept_dep *new = (struct dept_dep *)in; + + root->bfs_gen = bfs_gen; + dep_tc(new)->bfs_parent = dep_fc(new); +} + +static void bfs_extend_dep(struct list_head *h, void *node) +{ + struct dept_class *cur = (struct dept_class *)node; + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_head, dep_node) { + struct dept_class *next = dep_tc(d); + + if (bfs_gen == next->bfs_gen) + continue; + next->bfs_parent = cur; + next->bfs_gen = bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static void *bfs_dequeue_dep(struct list_head *h) +{ + struct dept_class *c; + + DEPT_WARN_ON(list_empty(h)); + + c = list_first_entry(h, struct dept_class, bfs_node); + list_del(&c->bfs_node); + return c; +} + +static enum bfs_ret cb_check_dl(void *node, void *in, void **out) +{ + struct dept_class *cur = (struct dept_class *)node; + struct dept_dep *new = (struct dept_dep *)in; + + if (cur == dep_fc(new)) { + print_circle(dep_tc(new)); + return BFS_DONE; + } + + return BFS_CONTINUE; +} + +/* + * This function is actually in charge of reporting. + */ +static void check_dl_bfs(struct dept_dep *d) +{ + struct bfs_ops ops = { + .bfs_init = bfs_init_check_dl, + .extend = bfs_extend_dep, + .dequeue = bfs_dequeue_dep, + .callback = cb_check_dl, + }; + + bfs((void *)dep_tc(d), &ops, (void *)d, NULL); +} + +static void bfs_init_dep(void *node, void *in, void **out) +{ + struct dept_class *root = (struct dept_class *)node; + + root->bfs_gen = bfs_gen; +} + +static void bfs_extend_dep_rev(struct list_head *h, void *node) +{ + struct dept_class *cur = (struct dept_class *)node; + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_rev_head, dep_rev_node) { + struct dept_class *next = dep_fc(d); + + if (bfs_gen == next->bfs_gen) + continue; + next->bfs_parent = cur; + next->bfs_gen = bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static enum bfs_ret cb_find_iw(void *node, void *in, void **out) +{ + struct dept_class *cur = (struct dept_class *)node; + int irq = *(int *)in; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + iw = iwait(cur, irq); + + /* + * If any parent's ->wait was set, then the children would've + * been touched. + */ + if (!iw->touched) + return BFS_SKIP; + + if (!iw->wait) + return BFS_CONTINUE; + + *out = iw; + return BFS_DONE; +} + +static struct dept_iwait *find_iw_bfs(struct dept_class *c, int irq) +{ + struct dept_iwait *iw = iwait(c, irq); + struct dept_iwait *found = NULL; + struct bfs_ops ops = { + .bfs_init = bfs_init_dep, + .extend = bfs_extend_dep_rev, + .dequeue = bfs_dequeue_dep, + .callback = cb_find_iw, + }; + + bfs((void *)c, &ops, (void *)&irq, (void **)&found); + + if (found) + return found; + + untouch_iwait(iw); + return NULL; +} + +static enum bfs_ret cb_touch_iw_find_ie(void *node, void *in, void **out) +{ + struct dept_class *cur = (struct dept_class *)node; + int irq = *(int *)in; + struct dept_iecxt *ie = iecxt(cur, irq); + struct dept_iwait *iw = iwait(cur, irq); + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + touch_iwait(iw); + + if (!ie->ecxt) + return BFS_CONTINUE; + if (!*out) + *out = ie; + + /* + * Do touch_iwait() all the way. + */ + return BFS_CONTINUE; +} + +static struct dept_iecxt *touch_iw_find_ie_bfs(struct dept_class *c, + int irq) +{ + struct dept_iecxt *found = NULL; + struct bfs_ops ops = { + .bfs_init = bfs_init_dep, + .extend = bfs_extend_dep, + .dequeue = bfs_dequeue_dep, + .callback = cb_touch_iw_find_ie, + }; + + bfs((void *)c, &ops, (void *)&irq, (void **)&found); + return found; +} + +/* + * Should be called with dept_lock held. + */ +static void __add_idep(struct dept_iecxt *ie, struct dept_iwait *iw) +{ + struct dept_dep *new; + + /* + * There's nothing to do. + */ + if (!ie || !iw || !ie->ecxt || !iw->wait) + return; + + new = __add_dep(ie->ecxt, iw->wait); + + /* + * Deadlock detected. Let check_dl_bfs() report it. + */ + if (new) { + check_dl_bfs(new); + stale_iecxt(ie); + stale_iwait(iw); + } + + /* + * If !new, it would be the case of lack of object resource. + * Just let it go and get checked by other chances. Retrying is + * meaningless in that case. + */ +} + +static void set_check_iecxt(struct dept_class *c, int irq, + struct dept_ecxt *e) +{ + struct dept_iecxt *ie = iecxt(c, irq); + + set_iecxt(ie, e); + __add_idep(ie, find_iw_bfs(c, irq)); +} + +static void set_check_iwait(struct dept_class *c, int irq, + struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + set_iwait(iw, w); + __add_idep(touch_iw_find_ie_bfs(c, irq), iw); +} + +static void add_iecxt(struct dept_class *c, int irq, struct dept_ecxt *e, + bool stack) +{ + /* + * This access is safe since we ensure e->class has set locally. + */ + struct dept_task *dt = dept_task(); + struct dept_iecxt *ie = iecxt(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(ie->staled))) + return; + + /* + * Skip add_iecxt() if ie->ecxt has ever been set at least once. + * Which means it has a valid ->ecxt or been staled. + */ + if (READ_ONCE(ie->ecxt)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(ie->staled)) + goto unlock; + if (ie->ecxt) + goto unlock; + + e->enirqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * enirq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(e->enirq_ip[irq]); + DEPT_WARN_ON(e->enirq_stack[irq]); + + e->enirq_ip[irq] = dt->enirq_ip[irq]; + e->enirq_stack[irq] = stack ? get_current_stack() : NULL; + + set_check_iecxt(c, irq, e); +unlock: + dept_unlock(); +} + +static void add_iwait(struct dept_class *c, int irq, struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(iw->staled))) + return; + + /* + * Skip add_iwait() if iw->wait has ever been set at least once. + * Which means it has a valid ->wait or been staled. + */ + if (READ_ONCE(iw->wait)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(iw->staled)) + goto unlock; + if (iw->wait) + goto unlock; + + w->irqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * irq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(w->irq_ip[irq]); + DEPT_WARN_ON(w->irq_stack[irq]); + + w->irq_ip[irq] = w->wait_ip; + w->irq_stack[irq] = get_current_stack(); + + set_check_iwait(c, irq, w); +unlock: + dept_unlock(); +} + +static struct dept_wait_hist *hist(int pos) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist + (pos % DEPT_MAX_WAIT_HIST); +} + +static int hist_pos_next(void) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist_pos % DEPT_MAX_WAIT_HIST; +} + +static void hist_advance(void) +{ + struct dept_task *dt = dept_task(); + + dt->wait_hist_pos++; + dt->wait_hist_pos %= DEPT_MAX_WAIT_HIST; +} + +static struct dept_wait_hist *new_hist(void) +{ + struct dept_wait_hist *wh = hist(hist_pos_next()); + + hist_advance(); + return wh; +} + +static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) +{ + struct dept_wait_hist *wh = new_hist(); + + if (likely(wh->wait)) + put_wait(wh->wait); + + wh->wait = get_wait(w); + wh->wgen = wg; + wh->ctxt_id = ctxt_id; +} + +/* + * Should be called after setting up e's iecxt and w's iwait. + */ +static void add_dep(struct dept_ecxt *e, struct dept_wait *w) +{ + struct dept_class *fc = e->class; + struct dept_class *tc = w->class; + struct dept_dep *d; + int i; + + if (lookup_dep(fc, tc)) + return; + + if (unlikely(!dept_lock())) + return; + + /* + * __add_dep() will lookup_dep() again with lock held. + */ + d = __add_dep(e, w); + if (d) { + check_dl_bfs(d); + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iwait *fiw = iwait(fc, i); + struct dept_iecxt *found_ie; + struct dept_iwait *found_iw; + + /* + * '->touched == false' guarantees there's no + * parent that has been set ->wait. + */ + if (!fiw->touched) + continue; + + /* + * find_iw_bfs() will untouch the iwait if + * not found. + */ + found_iw = find_iw_bfs(fc, i); + + if (!found_iw) + continue; + + found_ie = touch_iw_find_ie_bfs(tc, i); + __add_idep(found_ie, found_iw); + } + } + dept_unlock(); +} + +static atomic_t wgen = ATOMIC_INIT(1); + +static int next_wgen(void) +{ + /* + * Avoid zero wgen. + */ + return atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); +} + +static void add_wait(struct dept_class *c, unsigned long ip, + const char *w_fn, int sub_l, bool sched_sleep) +{ + struct dept_task *dt = dept_task(); + struct dept_wait *w; + unsigned int wg; + int irq; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + w = new_wait(); + if (unlikely(!w)) + return; + + WRITE_ONCE(w->class, get_class(c)); + w->wait_ip = ip; + w->wait_fn = w_fn; + w->wait_stack = get_current_stack(); + w->sched_sleep = sched_sleep; + + irq = cur_irq(); + if (irq < DEPT_IRQS_NR) + add_iwait(c, irq, w); + + /* + * Avoid adding dependency between user aware nested ecxt and + * wait. + */ + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + + /* + * the case of invalid key'ed one + */ + if (!eh->ecxt) + continue; + + if (eh->ecxt->class != c || eh->sub_l == sub_l) + add_dep(eh->ecxt, w); + } + + wg = next_wgen(); + add_hist(w, wg, cur_ctxt_id()); + + del_wait(w); +} + +static struct dept_ecxt_held *add_ecxt(struct dept_map *m, + struct dept_class *c, unsigned long ip, const char *c_fn, + const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + unsigned long irqf; + unsigned int wg; + int irq; + + if (DEPT_WARN_ON(!valid_class(c))) + return NULL; + + if (DEPT_WARN_ON_ONCE(dt->ecxt_held_pos >= DEPT_MAX_ECXT_HELD)) + return NULL; + + wg = next_wgen(); + if (m->nocheck) { + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = NULL; + eh->map = m; + eh->class = get_class(c); + eh->wgen = wg; + eh->sub_l = sub_l; + + return eh; + } + + e = new_ecxt(); + if (unlikely(!e)) + return NULL; + + e->class = get_class(c); + e->ecxt_ip = ip; + e->ecxt_stack = ip ? get_current_stack() : NULL; + e->event_fn = e_fn; + e->ecxt_fn = c_fn; + + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = get_ecxt(e); + eh->map = m; + eh->class = get_class(c); + eh->wgen = wg; + eh->sub_l = sub_l; + + irqf = cur_enirqf(); + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + add_iecxt(c, irq, e, false); + + del_ecxt(e); + return eh; +} + +static int find_ecxt_pos(struct dept_map *m, struct dept_class *c, + bool newfirst) +{ + struct dept_task *dt = dept_task(); + int i; + + if (newfirst) { + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } else { + for (i = 0; i < dt->ecxt_held_pos; i++) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } + return -1; +} + +static bool pop_ecxt(struct dept_map *m, struct dept_class *c) +{ + struct dept_task *dt = dept_task(); + int pos; + int i; + + pos = find_ecxt_pos(m, c, true); + if (pos == -1) + return false; + + if (dt->ecxt_held[pos].class) + put_class(dt->ecxt_held[pos].class); + + if (dt->ecxt_held[pos].ecxt) + put_ecxt(dt->ecxt_held[pos].ecxt); + + dt->ecxt_held_pos--; + + for (i = pos; i < dt->ecxt_held_pos; i++) + dt->ecxt_held[i] = dt->ecxt_held[i + 1]; + return true; +} + +static bool good_hist(struct dept_wait_hist *wh, unsigned int wg) +{ + return wh->wait != NULL && before(wg, wh->wgen); +} + +/* + * Binary-search the ring buffer for the earliest valid wait. + */ +static int find_hist_pos(unsigned int wg) +{ + int oldest; + int l; + int r; + int pos; + + oldest = hist_pos_next(); + if (unlikely(good_hist(hist(oldest), wg))) { + DEPT_INFO_ONCE("Need to expand the ring buffer.\n"); + return oldest; + } + + l = oldest + 1; + r = oldest + DEPT_MAX_WAIT_HIST - 1; + for (pos = (l + r) / 2; l <= r; pos = (l + r) / 2) { + struct dept_wait_hist *p = hist(pos - 1); + struct dept_wait_hist *wh = hist(pos); + + if (!good_hist(p, wg) && good_hist(wh, wg)) + return pos % DEPT_MAX_WAIT_HIST; + if (good_hist(wh, wg)) + r = pos - 1; + else + l = pos + 1; + } + return -1; +} + +static void do_event(struct dept_map *m, struct dept_map *real_m, + struct dept_class *c, unsigned int wg, unsigned long ip, + const char *e_fn) +{ + struct dept_task *dt = dept_task(); + struct dept_wait_hist *wh; + struct dept_ecxt_held *eh; + unsigned int ctxt_id; + int end; + int pos; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (m->nocheck) + return; + + /* + * The event was triggered before wait. + */ + if (!wg) + return; + + /* + * If an ecxt for this map exists, let the ecxt work for this + * event and do not proceed it in do_event(). + */ + if (find_ecxt_pos(real_m, c, false) != -1) + return; + eh = add_ecxt(m, c, 0UL, NULL, e_fn, 0); + + if (!eh) + return; + + if (DEPT_WARN_ON(!eh->ecxt)) + goto out; + + eh->ecxt->event_ip = ip; + eh->ecxt->event_stack = get_current_stack(); + + pos = find_hist_pos(wg); + if (pos == -1) + goto out; + + ctxt_id = cur_ctxt_id(); + end = hist_pos_next(); + end = end > pos ? end : end + DEPT_MAX_WAIT_HIST; + for (wh = hist(pos); pos < end; wh = hist(++pos)) { + if (dt->in_sched && wh->wait->sched_sleep) + continue; + + if (wh->ctxt_id == ctxt_id) + add_dep(eh->ecxt, wh->wait); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_ecxt *e; + + if (before(dt->wgen_enirq[i], wg)) + continue; + + e = eh->ecxt; + add_iecxt(e->class, i, e, false); + } +out: + /* + * Pop ecxt that temporarily has been added to handle this event. + */ + pop_ecxt(m, c); +} + +static void del_dep_rcu(struct rcu_head *rh) +{ + struct dept_dep *d = container_of(rh, struct dept_dep, rh); + + preempt_disable(); + del_dep(d); + preempt_enable(); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_class(struct dept_class *c) +{ + struct dept_dep *d, *n; + int i; + + list_for_each_entry_safe(d, n, &c->dep_head, dep_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + list_for_each_entry_safe(d, n, &c->dep_rev_head, dep_rev_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + stale_iecxt(iecxt(c, i)); + stale_iwait(iwait(c, i)); + } +} + +/* + * Context control + * ===================================================================== + * Whether a wait is in {hard,soft}-IRQ context or whether + * {hard,soft}-IRQ has been enabled on the way to an event is very + * important to check dependency. All those things should be tracked. + */ + +static unsigned long cur_enirqf(void) +{ + struct dept_task *dt = dept_task(); + int he = dt->hardirqs_enabled; + int se = dt->softirqs_enabled; + + if (he) + return DEPT_HIRQF | (se ? DEPT_SIRQF : 0UL); + return 0UL; +} + +static int cur_irq(void) +{ + if (lockdep_softirq_context(current)) + return DEPT_SIRQ; + if (lockdep_hardirq_context()) + return DEPT_HIRQ; + return DEPT_IRQS_NR; +} + +static unsigned int cur_ctxt_id(void) +{ + struct dept_task *dt = dept_task(); + int irq = cur_irq(); + + /* + * Normal process context + */ + if (irq == DEPT_IRQS_NR) + return 0U; + + return dt->irq_id[irq] | (1UL << irq); +} + +static void enirq_transition(int irq) +{ + struct dept_task *dt = dept_task(); + int i; + + /* + * IRQ can cut in on the way to the event. Used for cross-event + * detection. + * + * wait context event context(ecxt) + * ------------ ------------------- + * wait event + * UPDATE wgen + * observe IRQ enabled + * UPDATE wgen + * keep the wgen locally + * + * on the event + * check the wgen kept + */ + + dt->wgen_enirq[irq] = next_wgen(); + + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + + eh = dt->ecxt_held + i; + e = eh->ecxt; + if (e) + add_iecxt(e->class, irq, e, true); + } +} + +static void dept_enirq(unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long irqf = cur_enirqf(); + int irq; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * IRQ ON/OFF transition might happen while Dept is working. + * We cannot handle recursive entrance. Just ingnore it. + * Only transitions outside of Dept will be considered. + */ + if (dt->recursive) + return; + + flags = dept_enter(); + + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + dt->enirq_ip[irq] = ip; + enirq_transition(irq); + } + + dept_exit(flags); +} + +void dept_softirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = true; + dept_enirq(ip); +} + +void dept_hardirqs_on(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = true; + dept_enirq(_RET_IP_); +} + +void dept_softirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = false; +} + +void dept_hardirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = false; +} + +/* + * Ensure it's the outmost softirq context. + */ +void dept_softirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; +} + +/* + * Ensure it's the outmost hardirq context. + */ +void dept_hardirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; +} + +void dept_sched_enter(void) +{ + dept_task()->in_sched = true; +} + +void dept_sched_exit(void) +{ + dept_task()->in_sched = false; +} + +/* + * Exposed APIs + * ===================================================================== + */ + +static void clean_classes_cache(struct dept_key *k) +{ + int i; + + for (i = 0; i < DEPT_MAX_SUBCLASSES_CACHE; i++) { + if (!READ_ONCE(k->classes[i])) + continue; + + WRITE_ONCE(k->classes[i], NULL); + } +} + +/* + * Assume we don't have to consider race with the map when + * dept_map_init() is called. + */ +void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u < 0)) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u >= DEPT_MAX_SUBCLASSES_USR)) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + clean_classes_cache(&m->map_key); + + m->keys = k; + m->sub_u = sub_u; + m->name = n; + m->wgen = 0U; + m->nocheck = !valid_key(k); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_init); + +/* + * Assume we don't have to consider race with the map when + * dept_map_reinit() is called. + */ +void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + if (k) { + clean_classes_cache(&m->map_key); + m->keys = k; + m->nocheck = !valid_key(k); + } + + if (sub_u >= 0 && sub_u < DEPT_MAX_SUBCLASSES_USR) + m->sub_u = sub_u; + + if (n) + m->name = n; + + m->wgen = 0U; + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_reinit); + +void dept_map_copy(struct dept_map *to, struct dept_map *from) +{ + if (unlikely(!dept_working())) { + to->nocheck = true; + return; + } + + *to = *from; + + /* + * XXX: 'to' might be in a stack or something. Using the address + * in a stack segment as a key is meaningless. Just ignore the + * case for now. + */ + if (!to->keys) { + to->nocheck = true; + return; + } + + /* + * Since the class cache can be modified concurrently we could + * observe half pointers (64bit arch using 32bit copy insns). + * Therefore clear the caches and take the performance hit. + * + * XXX: Doesn't work well with lockdep_set_class_and_subclass() + * since that relies on cache abuse. + */ + clean_classes_cache(&to->map_key); +} + +static LIST_HEAD(classes); + +static bool within(const void *addr, void *start, unsigned long size) +{ + return addr >= start && addr < start + size; +} + +void dept_free_range(void *start, unsigned int sz) +{ + struct dept_task *dt = dept_task(); + struct dept_class *c, *n; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Failed to successfully free Dept objects.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_free_range() should not fail. + * + * FIXME: Should be fixed if dept_free_range() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + list_for_each_entry_safe(c, n, &classes, all_node) { + if (!within((void *)c->key, start, sz) && + !within(c->name, start, sz)) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} + +static int sub_id(struct dept_map *m, int e) +{ + return (m ? m->sub_u : 0) + e * DEPT_MAX_SUBCLASSES_USR; +} + +static struct dept_class *check_new_class(struct dept_key *local, + struct dept_key *k, int sub_id, + const char *n, bool sched_map) +{ + struct dept_class *c = NULL; + + if (DEPT_WARN_ON(sub_id >= DEPT_MAX_SUBCLASSES)) + return NULL; + + if (DEPT_WARN_ON(!k)) + return NULL; + + /* + * XXX: Assume that users prevent the map from using if any of + * the cached keys has been invalidated. If not, the cache, + * local->classes should not be used because it would be racy + * with class deletion. + */ + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + c = READ_ONCE(local->classes[sub_id]); + + if (c) + return c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (c) + goto caching; + + if (unlikely(!dept_lock())) + return NULL; + + c = lookup_class((unsigned long)k->base + sub_id); + if (unlikely(c)) + goto unlock; + + c = new_class(); + if (unlikely(!c)) + goto unlock; + + c->name = n; + c->sched_map = sched_map; + c->sub_id = sub_id; + c->key = (unsigned long)(k->base + sub_id); + hash_add_class(c); + list_add(&c->all_node, &classes); +unlock: + dept_unlock(); +caching: + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + WRITE_ONCE(local->classes[sub_id], c); + + return c; +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool sched_map) +{ + int e; + + /* + * Be as conservative as possible. In case of mulitple waits for + * a single dept_map, we are going to keep only the last wait's + * wgen for simplicity - keeping all wgens seems overengineering. + * + * Of course, it might cause missing some dependencies that + * would rarely, probabily never, happen but it helps avoid + * false positive report. + */ + for_each_set_bit(e, &w_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, sched_map); + if (!c) + continue; + + add_wait(c, ip, w_fn, sub_l, sched_sleep); + } +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_event(struct dept_map *m, struct dept_map *real_m, + unsigned long e_f, unsigned long ip, const char *e_fn, + bool sched_map) +{ + struct dept_class *c; + struct dept_key *k; + int e; + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (DEPT_WARN_ON(e >= DEPT_MAX_SUBCLASSES_EVT)) + return; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); + + if (c) + do_event(m, real_m, c, READ_ONCE(m->wgen), ip, e_fn); +} + +void dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (m->nocheck) + return; + + flags = dept_enter(); + + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_wait); + +void dept_stage_wait(struct dept_map *m, struct dept_key *k, + unsigned long ip, const char *w_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m && m->nocheck) + return; + + /* + * Either m or k should be passed. Which means Dept relies on + * either its own map or the caller's position in the code when + * determining its class. + */ + if (DEPT_WARN_ON(!m && !k)) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Ensure the outmost dept_stage_wait() works. + */ + if (dt->stage_m.keys) + goto exit; + + arch_spin_lock(&dt->stage_lock); + if (m) { + dt->stage_m = *m; + dt->stage_real_m = m; + + /* + * Ensure dt->stage_m.keys != NULL and it works with the + * map's map_key, not stage_m's one when ->keys == NULL. + */ + if (!m->keys) + dt->stage_m.keys = &m->map_key; + } else { + dt->stage_m.name = w_fn; + dt->stage_sched_map = true; + dt->stage_real_m = &dt->stage_m; + } + + /* + * dept_map_reinit() includes WRITE_ONCE(->wgen, 0U) that + * effectively disables the map just in case real sleep won't + * happen. dept_request_event_wait_commit() will enable it. + */ + dept_map_reinit(&dt->stage_m, k, -1, NULL); + + dt->stage_w_fn = w_fn; + dt->stage_ip = ip; + arch_spin_unlock(&dt->stage_lock); +exit: + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_stage_wait); + +static void __dept_clean_stage(struct dept_task *dt) +{ + memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); + dt->stage_real_m = NULL; + dt->stage_sched_map = false; + dt->stage_w_fn = NULL; + dt->stage_ip = 0UL; +} + +void dept_clean_stage(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + arch_spin_lock(&dt->stage_lock); + __dept_clean_stage(dt); + arch_spin_unlock(&dt->stage_lock); + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_clean_stage); + +/* + * Always called from __schedule(). + */ +void dept_request_event_wait_commit(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + unsigned int wg; + unsigned long ip; + const char *w_fn; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + /* + * It's impossible that __schedule() is called while Dept is + * working that already disabled IRQ at the entrance. + */ + if (DEPT_WARN_ON(dt->recursive)) + return; + + flags = dept_enter(); + + arch_spin_lock(&dt->stage_lock); + + /* + * Checks if current has staged a wait. + */ + if (!dt->stage_m.keys) { + arch_spin_unlock(&dt->stage_lock); + goto exit; + } + + w_fn = dt->stage_w_fn; + ip = dt->stage_ip; + sched_map = dt->stage_sched_map; + + wg = next_wgen(); + WRITE_ONCE(dt->stage_m.wgen, wg); + arch_spin_unlock(&dt->stage_lock); + + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); +exit: + dept_exit(flags); +} + +/* + * Always called from try_to_wake_up(). + */ +void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_task *dt_req = &requestor->dept_task; + unsigned long flags; + struct dept_map m; + struct dept_map *real_m; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + flags = dept_enter(); + + arch_spin_lock(&dt_req->stage_lock); + + /* + * Serializing is unnecessary as long as it always comes from + * try_to_wake_up(). + */ + m = dt_req->stage_m; + sched_map = dt_req->stage_sched_map; + real_m = dt_req->stage_real_m; + __dept_clean_stage(dt_req); + arch_spin_unlock(&dt_req->stage_lock); + + /* + * ->stage_m.keys should not be NULL if it's in use. Should + * make sure that it's not NULL when staging a valid map. + */ + if (!m.keys) + goto exit; + + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map); +exit: + dept_exit(flags); +} + +/* + * Modifies the latest ecxt corresponding to m and e_f. + */ +void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, + struct dept_key *new_k, unsigned long new_e_f, + unsigned long new_ip, const char *new_c_fn, + const char *new_e_fn, int new_sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_class *c; + struct dept_key *k; + unsigned long flags; + int pos = -1; + int new_e; + int e; + + if (unlikely(!dept_working())) + return; + + /* + * XXX: Couldn't handle re-enterance cases. Ingore it for now. + */ + if (dt->recursive) + return; + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + pos = find_ecxt_pos(m, c, true); + if (pos != -1) + break; + } + + if (unlikely(pos == -1)) + goto exit; + + eh = dt->ecxt_held + pos; + new_sub_l = new_sub_l >= 0 ? new_sub_l : eh->sub_l; + + new_e = find_first_bit(&new_e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (new_e < DEPT_MAX_SUBCLASSES_EVT) + /* + * Let it work with the first bit anyway. + */ + DEPT_WARN_ON(1UL << new_e != new_e_f); + else + new_e = e; + + pop_ecxt(m, c); + + /* + * Apply the key to the map. + */ + if (new_k) + dept_map_reinit(m, new_k, -1, NULL); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); + + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + goto exit; + + /* + * Successfully pop_ecxt()ed but failed to add_ecxt(). + */ + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_map_ecxt_modify); + +void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, + const char *c_fn, const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + struct dept_class *c; + struct dept_key *k; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt++; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (e >= DEPT_MAX_SUBCLASSES_EVT) + goto missing_ecxt; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); + + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + goto exit; +missing_ecxt: + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_enter); + +bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + bool ret = false; + int e; + + if (unlikely(!dept_working())) + return false; + + if (dt->recursive) + return false; + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + if (find_ecxt_pos(m, c, true) != -1) { + ret = true; + break; + } + } + + dept_exit(flags); + + return ret; +} +EXPORT_SYMBOL_GPL(dept_ecxt_holding); + +void dept_request_event(struct dept_map *m) +{ + unsigned long flags; + unsigned int wg; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + wg = next_wgen(); + WRITE_ONCE(m->wgen, wg); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_request_event); + +void dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + if (dt->recursive) { + /* + * Dept won't work with this even though an event + * context has been asked. Don't make it confused at + * handling the event. Disable it until the next. + */ + WRITE_ONCE(m->wgen, 0U); + return; + } + + flags = dept_enter(); + + __dept_event(m, m, e_f, ip, e_fn, false); + + /* + * Keep the map diabled until the next sleep. + */ + WRITE_ONCE(m->wgen, 0U); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_event); + +void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, + unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt--; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + if (pop_ecxt(m, c)) + goto exit; + } + + dt->missing_ecxt--; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_exit); + +void dept_task_exit(struct task_struct *t) +{ + struct dept_task *dt = &t->dept_task; + int i; + + if (unlikely(!dept_working())) + return; + + raw_local_irq_disable(); + + if (dt->stack) { + put_stack(dt->stack); + dt->stack = NULL; + } + + for (i = 0; i < dt->ecxt_held_pos; i++) { + if (dt->ecxt_held[i].class) { + put_class(dt->ecxt_held[i].class); + dt->ecxt_held[i].class = NULL; + } + if (dt->ecxt_held[i].ecxt) { + put_ecxt(dt->ecxt_held[i].ecxt); + dt->ecxt_held[i].ecxt = NULL; + } + } + + for (i = 0; i < DEPT_MAX_WAIT_HIST; i++) { + if (dt->wait_hist[i].wait) { + put_wait(dt->wait_hist[i].wait); + dt->wait_hist[i].wait = NULL; + } + } + + dt->task_exit = true; + dept_off(); + + raw_local_irq_enable(); +} + +void dept_task_init(struct task_struct *t) +{ + memset(&t->dept_task, 0x0, sizeof(struct dept_task)); + t->dept_task.stage_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +} + +void dept_key_init(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Key initialization fails.\n"); + return; + } + + flags = dept_enter(); + + clean_classes_cache(k); + + /* + * dept_key_init() should not fail. + * + * FIXME: Should be fixed if dept_key_init() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + DEPT_STOP("The class(%s/%d) has not been removed.\n", + c->name, sub_id); + break; + } + + dept_unlock(); + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_key_init); + +void dept_key_destroy(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive == 1 && dt->task_exit) { + /* + * Need to allow to go ahead in this case where + * ->recursive has been set to 1 by dept_off() in + * dept_task_exit() and ->task_exit has been set to + * true in dept_task_exit(). + */ + } else if (dt->recursive) { + DEPT_STOP("Key destroying fails.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_key_destroy() should not fail. + * + * FIXME: Should be fixed if dept_key_destroy() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(dept_key_destroy); + +static void move_llist(struct llist_head *to, struct llist_head *from) +{ + struct llist_node *first = llist_del_all(from); + struct llist_node *last; + + if (!first) + return; + + for (last = first; last->next; last = last->next); + llist_add_batch(first, last, to); +} + +static void migrate_per_cpu_pool(void) +{ + const int boot_cpu = 0; + int i; + + /* + * The boot CPU has been using the temperal local pool so far. + * From now on that per_cpu areas have been ready, use the + * per_cpu local pool instead. + */ + DEPT_WARN_ON(smp_processor_id() != boot_cpu); + for (i = 0; i < OBJECT_NR; i++) { + struct llist_head *from; + struct llist_head *to; + + from = &pool[i].boot_pool; + to = per_cpu_ptr(pool[i].lpool, boot_cpu); + move_llist(to, from); + } +} + +#define B2KB(B) ((B) / 1024) + +/* + * Should be called after setup_per_cpu_areas() and before no non-boot + * CPUs have been on. + */ +void __init dept_init(void) +{ + size_t mem_total = 0; + + local_irq_disable(); + dept_per_cpu_ready = 1; + migrate_per_cpu_pool(); + local_irq_enable(); + +#define HASH(id, bits) BUILD_BUG_ON(1 << (bits) <= 0); + #include "dept_hash.h" +#undef HASH +#define OBJECT(id, nr) mem_total += sizeof(struct dept_##id) * nr; + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) mem_total += sizeof(struct hlist_head) * (1 << (bits)); + #include "dept_hash.h" +#undef HASH + + pr_info("DEPendency Tracker: Copyright (c) 2020 LG Electronics, Inc., Byungchul Park\n"); + pr_info("... DEPT_MAX_STACK_ENTRY: %d\n", DEPT_MAX_STACK_ENTRY); + pr_info("... DEPT_MAX_WAIT_HIST : %d\n", DEPT_MAX_WAIT_HIST); + pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); + pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); +#define OBJECT(id, nr) \ + pr_info("... memory used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct dept_##id) * nr)); + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) \ + pr_info("... hash list head used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); + #include "dept_hash.h" +#undef HASH + pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); +} diff --git a/kernel/dependency/dept_hash.h b/kernel/dependency/dept_hash.h new file mode 100644 index 000000000000..fd85aab1fdfb --- /dev/null +++ b/kernel/dependency/dept_hash.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * HASH(id, bits) + * + * id : Id for the object of struct dept_##id. + * bits: 1UL << bits is the hash table size. + */ + +HASH(dep, 12) +HASH(class, 12) diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h new file mode 100644 index 000000000000..0b7eb16fe9fb --- /dev/null +++ b/kernel/dependency/dept_object.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * OBJECT(id, nr) + * + * id: Id for the object of struct dept_##id. + * nr: # of the object that should be kept in the pool. + */ + +OBJECT(dep, 1024 * 8) +OBJECT(class, 1024 * 8) +OBJECT(stack, 1024 * 32) +OBJECT(ecxt, 1024 * 16) +OBJECT(wait, 1024 * 32) diff --git a/kernel/exit.c b/kernel/exit.c index 1b51dc099f1e..5596de5a8349 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -1001,6 +1001,7 @@ void __noreturn do_exit(long code) exit_tasks_rcu_finish(); lockdep_free_task(tsk); + dept_task_exit(tsk); do_task_dead(); } diff --git a/kernel/fork.c b/kernel/fork.c index c4b26cd8998b..9630060a5005 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -105,6 +105,7 @@ #include #include #include +#include #include #include @@ -2382,6 +2383,7 @@ __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_LOCKDEP lockdep_init_task(p); #endif + dept_task_init(p); #ifdef CONFIG_DEBUG_MUTEXES p->blocked_on = NULL; /* not blocked yet */ diff --git a/kernel/module/main.c b/kernel/module/main.c index a2859dc3eea6..7e569e1b4db5 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1286,12 +1286,14 @@ static void free_mod_mem(struct module *mod) /* Free lock-classes; relies on the preceding sync_rcu(). */ lockdep_free_key_range(mod_mem->base, mod_mem->size); + dept_free_range(mod_mem->base, mod_mem->size); if (mod_mem->size) module_memory_free(mod, type); } /* MOD_DATA hosts mod, so free it at last */ lockdep_free_key_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); + dept_free_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); module_memory_free(mod, MOD_DATA); } @@ -3448,6 +3450,8 @@ static int load_module(struct load_info *info, const char __user *uargs, for_class_mod_mem_type(type, core_data) { lockdep_free_key_range(mod->mem[type].base, mod->mem[type].size); + dept_free_range(mod->mem[type].base, + mod->mem[type].size); } module_memory_restore_rox(mod); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c81cf642dba0..d5c90fa93776 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -66,6 +66,7 @@ #include #include #include +#include #ifdef CONFIG_PREEMPT_DYNAMIC # ifdef CONFIG_GENERIC_ENTRY @@ -4242,6 +4243,7 @@ int try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) if (READ_ONCE(p->on_rq) && ttwu_runnable(p, wake_flags)) break; + dept_ttwu_stage_wait(p, _RET_IP_); #ifdef CONFIG_SMP /* * Ensure we load p->on_cpu _after_ p->on_rq, otherwise it would be @@ -6663,6 +6665,11 @@ static void __sched notrace __schedule(int sched_mode) rq = cpu_rq(cpu); prev = rq->curr; + prev_state = READ_ONCE(prev->__state); + if (sched_mode != SM_PREEMPT && prev_state & TASK_NORMAL) + dept_request_event_wait_commit(); + + dept_sched_enter(); schedule_debug(prev, preempt); if (sched_feat(HRTICK) || sched_feat(HRTICK_DL)) @@ -6771,6 +6778,7 @@ static void __sched notrace __schedule(int sched_mode) raw_spin_rq_unlock_irq(rq); } trace_sched_exit_tp(is_switch, CALLER_ADDR0); + dept_sched_exit(); } void __noreturn do_task_dead(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index f9051ab610d5..d1aeb206b19a 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1365,6 +1365,32 @@ config DEBUG_PREEMPT menu "Lock Debugging (spinlocks, mutexes, etc...)" +config DEPT + bool "Dependency tracking (EXPERIMENTAL)" + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + select DEBUG_SPINLOCK + select DEBUG_MUTEXES + select DEBUG_RT_MUTEXES if RT_MUTEXES + select DEBUG_RWSEMS + select DEBUG_WW_MUTEX_SLOWPATH + select DEBUG_LOCK_ALLOC + select TRACE_IRQFLAGS + select STACKTRACE + select KALLSYMS + select KALLSYMS_ALL + select PROVE_LOCKING + default n + help + Check dependencies between wait and event and report it if + deadlock possibility has been detected. Multiple reports are + allowed if there are more than a single problem. + + This feature is considered EXPERIMENTAL that might produce + false positive reports because new dependencies start to be + tracked, that have never been tracked before. It's worth + noting, to mitigate the impact by the false positives, multi + reporting has been supported. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c index ed99344317f5..18228afccea5 100644 --- a/lib/locking-selftest.c +++ b/lib/locking-selftest.c @@ -1398,6 +1398,8 @@ static void reset_locks(void) local_irq_disable(); lockdep_free_key_range(&ww_lockdep.acquire_key, 1); lockdep_free_key_range(&ww_lockdep.mutex_key, 1); + dept_free_range(&ww_lockdep.acquire_key, 1); + dept_free_range(&ww_lockdep.mutex_key, 1); I1(A); I1(B); I1(C); I1(D); I1(X1); I1(X2); I1(Y1); I1(Y2); I1(Z1); I1(Z2); From patchwork Tue May 13 10:06:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084950 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ide+bounces-3556-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXKX4t5xz1yPv for ; Tue, 13 May 2025 20:10:24 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id E0D437AD973 for ; Tue, 13 May 2025 10:08:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BAA3124467A; Tue, 13 May 2025 10:07:48 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2369C242D77; Tue, 13 May 2025 10:07:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; cv=none; b=h38qKvHnO+IhMrbKVl05YpeJDRAdvC6YU9ynDSVHlMd4dRF/GglaWGiNWVHrgqAKeHybFeFxbiqZhel4YYjuP1L8IYpgH4dOb3cBgyP1/9K4ZpLe9cDvECCpRCbROUwo3RutY+MfCS1T7Kx1dKDbqzA7VaVBQzA/QDqMcTBX6QM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; c=relaxed/simple; bh=QBX5hyho5OMfGnKSjp0Bq0FcSRd46hiVm1ArD8Nb7ms=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=WbqRY0D1F4ALJlC66hCrx26aSXwR8iUcj3zAkfEYl0cr2hzfsP+T/cKRspDsayb6omer2UU/EiHXaIzZcfEM5splZemEjJhbCbiuglvlY2XUSkbizq8I/4ls9fjhdmkBTH7HItrb0cqhbiELWN6BuRhVOzGKlF1dhLfI6beUf90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-ea-682319edb95d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 03/43] dept: add single event dependency tracker APIs Date: Tue, 13 May 2025 19:06:50 +0900 Message-Id: <20250513100730.12664-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxSHfd/7SeWSazV60Tm1xpihIhg0x0SNWTS7M36QLEHFmNnJja0r YIqiNS6h8jGHQpgbNApoKUttaCtQyMaQEqilgs5aFbGYgkqcGeFroG1EYFvL9J+TJ+d3znP+ OSwh76QWs+qMk5I2Q6lR0DJSNhJtWjcSu1KV0JS7GYJvL5BQUWujwXfTisDWqMcw2PEFPA0N I5i6/4AAQ6kPQdXLPgIaPf0InJbzNDx+FQPdwTEaukov0pBbXUvDw6FpDIGyyxisjj3w3Pya hHslJgyGQRrKDbk4XP7CMGmuYcCcswoGLFcZmH6ZCF39PRQ4n62BK9cCNLQ4u0jwNA1geNxc QUO/7V8K7nk6SQgVLwHfj0UU2EdNNAyFzASYg2MMPGozYvAYF0JdXlhY8OYfCu4UtWEo+KUe Q3fvLQStF15gcNh6aLgdHMbQ4Cgl4P2NDgQDxSMM5F+aZKBcX4zgYn4ZCXmBjTD1Lny58m0i 6K/XkWCf6UHbt4q2azYk3h4eI8S8htPi++ATWnSGjKR41ySIv1/tY8S81meMaHScEhsscWJ1 yyAWqyaClOio+YEWHROXGbFwpBuLo14vk/xJqmxLmqRRZ0va9duOyFR9MwXUiZ+FM67RKSIH fT+/EEWxAp8k+P9oZz7yT7kTs0zzqwW/f5KI8AJ+udBQ9JoqRDKW4HvmCk8re1EkmM9/KdRX dMwyya8SagNuHGGO3yh4csrp/6XLBGtd26woit8kzNzwkhGWh2dKjFYyIhX48ihh9M/rHxZi hXaLnyxBnBHNqUFydUZ2ulKtSYpX6TLUZ+KPZqY7UPi/zN9NH2pCE76vXIhnkSKa6xxcoZJT yuwsXboLCSyhWMDpfwu3uDSl7qykzfxae0ojZbnQEpZULOI2hE6nyfljypPSt5J0QtJ+TDEb tTgHofzjBrXbObNfu/zI0rLtyQkrtumqvrlzfB7mfk1J3RnHJdcEljXX6w/0ZrcMKfPtjayq Iu2crrX9vpE77PEaPk1waSqTPnv+Zvfaz1fGH0zxxZTtG48mz+rcfycWmOy2Zn9sjNc6xzK+ f1xX7UthYoaS7O6mVPfeg5m32nYFdijILJUyMY7QZin/AyHKe6xbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ73Vio1r0jmq8ZbjdFodKJAjnGSfdJHvH8gRoNKI6+2EZC0 gjI00lGYq9LgBVEoWAEroVWgaESkhhSpMCaiICABFKJklZsCbcbNrXXZl5Nfzv/kd778JVRA CbNQooo7LarjFDFyVkpL92xJXTe0YIVyw+Vvi8E9fpEGY6mVheYHFgTWh1oMrrrt0O4ZRDD1 8hUF2VnNCO70dlPw0NmDwF78KwstH+dAq3uEhYasSyykFpay8HpgGkPXjasYLLbd8N7cT0Nj ZgGGbBcLudmp2Dv+wjBhLuHAnLIS+opzOJjuDYKGnjYGavMaGLB3roVb+V0sVNsbaHBW9mFo qTKy0GP9h4FGZz0NHsMiaL6SwcD94QIWBjxmCszuEQ7e1JgwOE0/QJnOa00f+8bAi4waDOlF 5Rha3z1F8OziBww2axsLte5BDBW2LAom79Uh6DMMcZB2eYKDXK0BwaW0GzToukJg6m/v57zx INDeLqPh/kwb+jmMWPOtiNQOjlBEV3GGTLrfssTuMdHkjwKBPMnp5ojuWSdHTLYEUlG8hhRW uzC5M+pmiK3kd5bYRq9yRD/UislwUxO3b/Eh6U/RYowqUVT/GBYlVXbPpDPx14WzjuEpKgX9 Nk+P/CQCHyxcSx3lfMzyq4SOjgnKx4H8MqEio5/RI6mE4ttmC+1575AvmMfvEMqNdd+Z5lcK pV3PsY9lfIjgTMll/5MuFSxlNd9FfnyoMHOvifZxgPcm02ShM5HUhGaVoEBVXGKsQhUTsl5z UpkUpzq7/tipWBvyNsh8fvpKJRpv2e5AvATJ/WX1ruXKAEaRqEmKdSBBQskDZdrH3pUsWpH0 i6g+dVSdECNqHGiRhJbPl4UfEKMC+BOK0+JJUYwX1f+nWOK3MAWV7w9uP1wUjncnhX96ZDSE 6bY5jic3fZ06uFr/yWo4pHCV+28rHDMmb22srzvXv2Tdha9BkeaIN9zcTb32kMhBsjNN1Ebx uzbnD0Rv9A/uqN9q+ayv6jmo/fLxyJL4u8mhEZWTVXvl1WOe5aF+vQ7yYtoQ/2rfgT+RZsGG SPvND4VyWqNUBK2h1BrFv1Hq/mE9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises. How to annotate: 1. Initaialize a map for the interesting wait. /* * Place along with the wait instance. */ struct dept_map my_wait; /* * Place in the initialization code. */ sdt_map_init(&my_wait); 2. Place the following at the wait code. sdt_wait(&my_wait); 3. Place the following at the event code. sdt_event(&my_wait); That's it! Signed-off-by: Byungchul Park --- include/linux/dept_sdt.h | 64 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 include/linux/dept_sdt.h diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h new file mode 100644 index 000000000000..93d772c71905 --- /dev/null +++ b/include/linux/dept_sdt.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Single-event Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include +#include + +#ifdef CONFIG_DEPT +#define sdt_map_init(m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait(m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* + * sdt_might_sleep() and its family will be committed in __schedule() + * when it actually gets to __schedule(). Both dept_request_event() and + * dept_wait() will be performed on the commit. + */ + +/* + * Use the code location as the class key if an explicit map is not used. + */ +#define sdt_might_sleep_start(m) \ + do { \ + struct dept_map *__m = m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end() dept_clean_stage() + +#define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#define sdt_request_event(m) dept_request_event(m) +#else /* !CONFIG_DEPT */ +#define sdt_map_init(m) do { } while (0) +#define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start(m) do { } while (0) +#define sdt_might_sleep_end() do { } while (0) +#define sdt_ecxt_enter(m) do { } while (0) +#define sdt_event(m) do { } while (0) +#define sdt_ecxt_exit(m) do { } while (0) +#define sdt_request_event(m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */ From patchwork Tue May 13 10:06:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084961 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3560-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXMX2ckxz1yPv for ; Tue, 13 May 2025 20:12:08 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2AE0A8C39D6 for ; Tue, 13 May 2025 10:09:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A1031246791; Tue, 13 May 2025 10:07:49 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7FA17221282; Tue, 13 May 2025 10:07:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130869; cv=none; b=qtMRyrpoh2fqG9Fay1toaIxLcPt35/7atVYNy6b7GGnauT496IYeH/wpi4mDOgcHt2ZRmBqtsCkZoR33suqdNkcy1cby97pFNTlpGws6DTL5DeQz5ivTN9n7mwJlgzVvWBMfEumUWTzeojaZR6Gz4wc0z9jH/n4a97pIYKiVoKI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130869; c=relaxed/simple; bh=kvYUJhkWNXGz3XMq3BkQN4+iBe4C0wrSkzgpQBtg3O0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=MsZdC49Cr8sMZOOGjLvgv5QMEUfQxKJIoUBxv4e1thdCKsr5F0kfLElh5vrCt8zIkOpWhIDExUfFilPvk/ynlPhftI2UZ58D55uUcHnj2TIRcTz35dBYOW76NScoNmpoJEsGi6i8xJVvIGEiLP3t8ILNc1/ydZ5uo/mO7a5lD08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-fb-682319ed8f02 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 04/43] dept: add lock dependency tracker APIs Date: Tue, 13 May 2025 19:06:51 +0900 Message-Id: <20250513100730.12664-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUiTaxjGfZ73a652ztuSfMvgnDMxSenDsroJCyM4PVBBUP1TdGrlixtO jVmWQaDNzD40K9RO62POmjvbKs80tQ/TVmkmmaWZipqKSZJOW240M2tL+ufmx3VdXPc/l4SS 1zPzJOqkA6I2SalRsFJaOjKzaJFzbqhq6YVKKbjHs2m4fNvGQvMtKwJbeQaGoacb4K1nGMHX Fy8pKMxvRlDU101BeV0PgmrzMRZaBn6DVvcoCw35p1nQFd9m4dXHSQxdBecxWO2b4Z1pkIbG PCOGwiEW9IU67DsfMHhNFg5M6WHQb77EwWRfFDT0tDFQ3RkJ/17tYuFBdQMNdVX9GFruXWah x/adgca6ZzR4ckOg+VwOAzedRhY+ekwUmNyjHLyuNWCoM8yB0kxfYdbnKQbqc2oxZF3/H0Nr x30ED7N7MdhtbSw8dg9jKLPnUzBR8hRBf+4IB8fPeDnQZ+QiOH28gIbMrhXw9Yvv85XxKMi4 VkrDzW9tKHYNsV21IfJ4eJQimWWHyIT7DUuqPQaaPDcK5O6lbo5kPuzkiMF+kJSZI0jxgyFM ilxuhtgtJ1lid53nyKmRVkycTU3clvk7pDFxokadKmqXrN0jVV1z6Zj974MPDzS9xOnILD+F AiUCHy3UtJzAv/j11AvGzywfLrS3eyk/B/F/CmU5gz5dKqH4thnC2ysdyG/M5tcJA531P5nm w4SBjl7WzzJ+hTBR04imS/8QrKW1P4sC+ZXCt5Im2s9yXybPYKWnM2cDBde51dM8V3hkbqfz kMyAAixIrk5KTVSqNdGLVWlJ6sOL9yUn2pFvXqajkzurkKt5qwPxEqSYKXs29JdKzihTU9IS HUiQUIogWUalT5LFKdOOiNrk3dqDGjHFgUIktCJYtsxzKE7OxysPiAmiuF/U/nKxJHBeOrrh NNLfNXfGyvWhEdvtZ9f/3RcUE7N27+pI9Pv8ilDbmO7GpuTYjYb3E+HFlqwqXS8X8I8yYUl4 Qk4ABBdl/2fauCA+mhQsDZtcZeFbGvZsn+X4bP0wFlaxd/Pu2C8V2wYveD9ddDj1kfeNJSGt Pd2WXdSmquDlU3Htnd6F+qO3nijoFJUyKoLSpih/AP7eDKJaAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHfd67q8XLknxLonojIstKaHKgkPpQPt0vBFEEOfSlLa9saRoU W2qUpphgUmbOWdPmLJ1BZs5E0VriNDVLUTOzi+WUzFmaVlvRl8OP///wO18ORyos9BJOE3ta 0saqokVGRsn2bU4Jci1eqd74vSMI3JOXKLh538pA+70yBNYHBgJGmsLg1dQogp+tbSTk5bYj KHrbT8KD5gEE9tILDHQOL4Au9zgDjtwMBlKK7zPw4sssAX3Xcggos+2FN+YPFLRkmwjIG2Eg Py+F8IxPBEybLSyY9atgqPQGC7Nvg8Ex0E1DY4GDBnvvWrh+q4+BWruDgubqIQI6a24yMGD9 TUNL8zMKprICoP1qJg3lYyYGvkyZSTC7x1noqDcS0GxcBBWpHuvFb79oeJpZT8DF25UEdPU8 RlB3aZAAm7WbgUb3KAFVtlwSZkqaEAxluVhIuzLNQr4hC0FG2jUKUvuU8POH53LBZDAYCiso KJ/rRltDsfWWFeHG0XESp1adwTPulwy2Txkp/Nwk4Ec3+lmcWtfLYqMtAVeVBuLi2hECF024 aWyzXGawbSKHxemuLgKPOZ3sgaXHZFsipWhNoqTdEBouUxdOpNDx7/2Thp1thB6VKtKRLyfw m4SOX620lxl+tfD69TTpZT9+uVCV+cGTyziS754nvCroQd5iIb9NGO59+pcpfpUw3DPIeFnO K4WZJy3on3SZUFZR/1fky4cIcyVOyssKz062sYzKRjIj8rEgP01sYoxKE61cr4tSJ8dqktZH xMXYkOeDzOdmr1ajyc6wBsRzSJwvfzayQq2gVYm65JgGJHCk6Cc3PPRE8khV8llJG3dCmxAt 6RpQAEeJ/vJdR6RwBX9SdVqKkqR4Sfu/JTjfJXoEhyLy11gKk9J2H91i32UK2yyfd/a4abC8 sjJ9B37HNmQGiFFFG1yHa7a5fT6u2ytmfA1afcdxnusItPS7Tk2KuGQHN9sWrt+/P35s6Tkx PURvDXX4UON65efI4qCEPXevhPrvdAR0Gt4UsOUh0vY775QHTyWcWPm8ds6Zc73aJVI6tSo4 kNTqVH8AJbTJYz0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrap the base APIs for easier annotation on typical lock. Signed-off-by: Byungchul Park --- include/linux/dept_ldt.h | 77 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 include/linux/dept_ldt.h diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h new file mode 100644 index 000000000000..062613e89fc3 --- /dev/null +++ b/include/linux/dept_ldt.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Lock Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init(m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock(m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock(m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade(m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "read_unlock", -1);\ + } while (0) + +#define ldt_set_class(m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL, k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init(m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock(m, sl, t, n, i) do { } while (0) +#define ldt_rlock(m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock(m, sl, t, n, i) do { } while (0) +#define ldt_unlock(m, i) do { } while (0) +#define ldt_downgrade(m, i) do { } while (0) +#define ldt_set_class(m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */ From patchwork Tue May 13 10:06:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084974 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ide+bounces-3558-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXQ10sdgz1yYx for ; Tue, 13 May 2025 20:14:17 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 09CA57B511A for ; Tue, 13 May 2025 10:08:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 624B12459D9; Tue, 13 May 2025 10:07:49 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7F9ACE545; Tue, 13 May 2025 10:07:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; cv=none; b=ds2rJa14dflaKmHnmIScCpXDywyPK1ZtXcCmmOusqNiC57BSs/05/gkHIG73ozKC+Rv+XE7trSrwP21fCcd/omVWzOWtEd/fTyUQyyZbMbKtzCqXT+XHNfjRy0mqzc9/sd94UjrHrL+BQiqPtl6HITESYVUs8qFy57YxEf7rmfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130868; c=relaxed/simple; bh=YOc2u1cQQ5Heil7x4uwqHeVR/uDF/WcqkEhWMsmeFt0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sZ0Krm4CFi42z4Nu5Wlk2gQAGwu+lGfFxDV4p4LsSu0QCKYEdNH0wE+2NRq5q4pMvbFsJq4cRfJNZArUlWJwQKnW7qWp5qgLYCfMi8uFM+fZNA4jSx7B/roCvUBJksrJStLG/u5lquhR8lNKlwt4+HFkcn5JWngGe5QcOsMWyk8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-0a-682319ee3c27 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 05/43] dept: tie to lockdep and IRQ tracing Date: Tue, 13 May 2025 19:06:52 +0900 Message-Id: <20250513100730.12664-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0ybZRSHeb9rqdR8K+g+wGxaRBPcRcwgJ0ZRF7K9zixZRozLjI4K39Yq l1mgwIhJO9gyrkOwaxyXlYtdpeViOyMgbMA2blPWjcpNQMBKRgZUu7XKALWQ7Z+TJ+ec33P+ OSJS2k+HiJQp6YIqRZ4kY8SUeCmgdudycJjiVa9LCp6H5yiobLYwYG8yI7Bc0RKwcHM/jHoX Eaz+fJsEvc6OoGZ2ioQrvdMIOk2nGRh2Pg0Oj4uBAV0hA7l1zQzcub9GwOSFMgLM1oPwm3Ge glultQToFxio0OcSvnKPgBVjAwtGTTjMmS6ysDYbCQPTIzR0TrwCX1dPMtDROUBBb+scAcPt lQxMW/6j4VZvPwXeklCwf1lMQ+NyLQP3vUYSjB4XC3e7DAT0Gp6Fljyf8OyDf2noK+4i4Gz9 dwQ4xn9EcPXcDAFWywgD1z2LBNisOhIeXb6JYK5kiYUzRSssVGhLEBSeuUBB3mQUrP7ju1z1 MBK0l1ooaFwfQW+/iS3VFoSvL7pInGfLxI88vzC402ug8GAtj9suTrE47+oEiw3WDGwzReC6 jgUC17g9NLY25DPY6i5jccGSg8DLQ0PsoeeOit9IFJKUakG1OyZerMi3/UqfvJuT1e/6g9ag bz8pQP4intvDF2ma0BMedFaTG8xwL/NjYyubHMQ9z9uK5+kCJBaR3MhT/GjV+GYgkHuLX3Va NpniwnnnX+vEBku4KH6pvP2xdDtvbunaFPlz0fz65SFqg6W+nVKDmdqQ8ly5P+9e63kcCOa7 TWNUKZIYkF8DkipT1MlyZdKeXYrsFGXWroTUZCvy/Zfxi7UPW5HbHteDOBGSBUj6F15QSGm5 Oi07uQfxIlIWJNH+4GtJEuXZpwRV6jFVRpKQ1oNCRZRsq+Q1b2ailDshTxc+E4STgurJlBD5 h2iQ7vXZtvQt0zGngjSsPvabbSHS4/c+2Ot+vzvA2hh7tMwUNxl/rLBxn+OA+trhsE+/+ntG W//TM+/2pa7//ufpI9+/GPBO9/7PCz3I3ObnzG+q028JHJee38evbJ1R7tCO5txICK853lzZ fkAznBM99ZL9iPu9Ox/pHLn1t2W71TfcHz/AMipNIY+MIFVp8v8BmgeGMVsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/p+s4fkv4yTwdZstjm/ggMUM/No9/aLOZTn5zNxW7I6Vp XU/rQS0sTemc0lV3R7nLSirtTuk8JEqFJO2YoyS6W6lwZf757LXP+7PX55+3iPDUU94iRfhp QRkuC5XSYlK8Z2P8ir7Zi+Srq8vXgnMwmYRrpUYamm8bEBjL1Rgc9YHQ7upFMPLsOQHZWc0I bnx4R0B5QxeCmuI4GlrsU6HV2U+DLSuNhviCUhpefB3F0HnlEgaDaTe8130i4UlmPoZsBw25 2fHYPT5jGNbpGdDFLoGe4hwGRj/4gq2rjQJrno2CmjfL4Kqmk4bqGhsJDZU9GFqqrtHQZfxD wZOGRhJcGXOg+WI6Bbe+5dPw1aUjQOfsZ+BlnRZDg3YmlCW4rUk/f1PwKL0OQ9LNOxhaX99H UJvcjcFkbKPB6uzFYDZlEfCrqB5BT0YfA4kXhhnIVWcgSEu8QkJCpx+MDLk/5w36gvp6GQm3 xtrQlgDeqDEi3trbT/AJ5rP8L+crmq9xaUn+cT7H38t5x/AJtW8YXms6w5uLffiCagfmb/xw UrxJn0Lzph+XGD61rxXz35qamH1zD4n9jwmhighBuSogWCxPMb+lTr2Mjmzs/0jFopKjqchD xLFruMd2DTHONLuU6+gYnmAvdgFnTv9EpSKxiGDbJnPtea/ReDCd3cyN2I0TTLJLOPvAGB5n CevH9V2uQv+k8zlDWd2EyINdy40VNZHj7Om+ydQayEwk1qJJeuSlCI8IkylC/VaqTsijwhWR K0NOhpmQu0G686MXK9FgS6AFsSIknSJpdCyUe1KyCFVUmAVxIkLqJVFXuFeSY7Koc4Ly5BHl mVBBZUFzRKR0lmRXkBDsyR6XnRZOCMIpQfk/xSIP71hUaFlm83MVSjTnNlR9HwLrl70DISX7 F89Yr9x6KK4+KOTosMWfyxrqiJZ267fHpN2sTzdop82IUgy0xMXuVO3wx97r7vq82BZWEdO+ KbNw+bzSs+UP7bWHWwclVq/o5/aCSod0aqT+dlC32pT84MAsGt9ZtD2xx//IwS9JMWbiKS0l VXKZrw+hVMn+AhKOp7w9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: How to place dept this way looks so ugly. However, it's inevitable for now. The way should be enhanced gradually. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 7 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++------ include/linux/lockdep_types.h | 3 + include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 8 +-- kernel/locking/lockdep.c | 22 ++++++ 14 files changed, 127 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 57b074e0cfbb..d8b9cf093f83 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -55,8 +56,10 @@ extern void trace_hardirqs_off(void); # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -131,6 +134,8 @@ do { \ # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock_internal.h index 8d5ac16a9b17..ec3ff5931aa6 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -27,6 +27,7 @@ typedef struct { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL, diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 67964dc4db95..ef03d8808c10 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ #include #include +#include #include struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *to, */ for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] = NULL; + + dept_map_copy(&to->dmap, &from->dmap); } /* @@ -428,7 +431,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name = (_name), .key = (void *)(_key), } + { .name = (_name), .key = (void *)(_key), \ + .dmap = DEPT_MAP_INITIALIZER(_name, _key) } static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -510,33 +514,89 @@ extern bool read_lock_is_recursive(void); #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n, i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t, 2, 1, n, i) -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l, s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL, i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_IP_) #define lock_map_acquire_try(l) lock_acquire_exclusive(l, 0, 1, NULL, _THIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 9f361d3ab9d9..6ef4dc67efb3 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H #include +#include #define MAX_LOCKDEP_SUBCLASSES 8UL @@ -77,6 +78,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; extern struct lock_class_key __lockdep_no_validate__; @@ -195,6 +197,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index 2143d05116be..f3ae3b11e7af 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -29,6 +29,7 @@ struct device; , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index af7d75ede619..857b0d46f6f1 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -22,7 +22,7 @@ struct percpu_rw_semaphore { }; #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d049883a08a..35889ac5eeae 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ do { \ .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442e7750..6e58dfc84997 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index c8b543d428b0..2540b18e3489 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -22,6 +22,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 5ce48eab7a2a..5f3447449fe0 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -51,7 +51,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) } /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_types_raw.h index 91cb36b65a17..3dcc551ded25 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ typedef struct raw_spinlock { .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ typedef struct raw_spinlock { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 900b0d5c05f5..f2903bdc8179 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 9a7a8693a738..24ac20b9bb16 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -248,10 +248,10 @@ static bool dept_working(void) * Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ != k; + return &__lockdep_no_validate__.dkey != k; } /* @@ -1945,7 +1945,7 @@ void dept_softirqs_off(void) dept_task()->softirqs_enabled = false; } -void dept_hardirqs_off(void) +void noinstr dept_hardirqs_off(void) { /* * Assumes that it's called with IRQ disabled so that accessing @@ -1967,7 +1967,7 @@ void dept_softirq_enter(void) /* * Ensure it's the outmost hardirq context. */ -void dept_hardirq_enter(void) +void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 58d78a33ac65..6c984a55d5ed 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1226,6 +1226,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head = keyhashentry(key); @@ -4362,6 +4364,8 @@ static void __trace_hardirqs_on_caller(void) */ void lockdep_hardirqs_on_prepare(void) { + dept_hardirqs_on(); + if (unlikely(!debug_locks)) return; @@ -4482,6 +4486,8 @@ EXPORT_SYMBOL_GPL(lockdep_hardirqs_on); */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off(); + if (unlikely(!debug_locks)) return; @@ -4526,6 +4532,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4564,6 +4572,8 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + dept_softirqs_off(); + if (unlikely(!lockdep_enabled())) return; @@ -4941,6 +4951,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, { int i; + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] = NULL; @@ -5734,6 +5746,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, { unsigned long flags; + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; @@ -5751,6 +5769,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) { unsigned long flags; + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; @@ -6586,6 +6606,8 @@ void lockdep_unregister_key(struct lock_class_key *key) bool found = false; bool need_callback = false; + dept_key_destroy(&key->dkey); + might_sleep(); if (WARN_ON_ONCE(static_obj(key))) From patchwork Tue May 13 10:06:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084969 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3562-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXPW62Jrz1yPv for ; Tue, 13 May 2025 20:13:51 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E683D3A46AC for ; Tue, 13 May 2025 10:11:06 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4B2C12517BC; Tue, 13 May 2025 10:07:53 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC7F0242D9C; Tue, 13 May 2025 10:07:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130872; cv=none; b=rjjv7153wcxHhM9IevjHi+WqIoOJ7ccnxK+1Oh4f/pTMw0YZxyvKF1D8a14pnztq8Rd9YdAvH3HVlp4oBrGoIIpBazQjqqyP/WNDvtf7O17JWxgR+w60vyXSaniP5fk85oVrDZ6+3komKCCa02vDLjdsQrbEudY2UJ1euN8BS1k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130872; c=relaxed/simple; bh=omTSwuVDldZZD4bvvYyVfYTOKAgDOm2MH8ZuGlfhiGw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Gwyy0ic68lq4rtwgejKSHD7CobPcdtqM90V8niX1P64gKWY+ViwHKspfFeWI4w4mKA0EXGMUTz5lIFiCkb6ZS+Xs+aJWMALXcNoGvMW4aS0jR2L2zyHSJIebBfALLGo/9gQ3iFGj05R8Lx+OUygyQgGhYf6qRoznPqfDEXCxW2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-1a-682319eec4fb From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 06/43] dept: add proc knobs to show stats and dependency graph Date: Tue, 13 May 2025 19:06:53 +0900 Message-Id: <20250513100730.12664-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfd47lZp3lbnXilFqcAYjA4Pb+cAWs2TuiUbj7cN0WVgjb2y1 FGwRRbKFSnEKloERiKhQwFRCi0BLMieWIEgFRUSotVydzDlRLhNtMwRxLWRfTn45/3N+58vh SJmLlnNqbaqo0yo1CkZCScZDyzdMLF+jipnyy8H39jQFl2ptDHRfsyKwNRgIGG37Fh77xxDM 3H9AQnFhN4Lyp0MkNLiGETirTjLQ+2wJuH2TDHQU5jKQVVnLwMNXswQMFp0jwGrfDk8szym4 l19BQPEoAxeLs4hAeUHAtKWaBUtmJIxUlbAw+zQWOoY9NDj718OF0kEGbjo7KHBdHyGg98Yl BoZtH2i452qnwJ+3AroLTDTUTFQw8MpvIcHim2Shp9lMgMu8DOqMAeGpN3M03DE1E3DqSj0B 7r5GBE2n/yDAbvMw0OobI8BhLyTh3dU2BCN54yxkn51m4aIhD0FudhEFxsFNMPNv4PLlt7Fg KKujoOa9B23+EttKbQi3jk2S2Og4ht/5HjHY6TdT+G6FgH8vGWKxsamfxWb7UeyoisKVN0cJ XD7lo7G9+gyD7VPnWJwz7ibwRFcXuzN8vyQ+UdSo00TdZ1/9KFGVBNwp/8Qfr3wQk4kGYnIQ xwl8nPDMzeagkHks6Cwng8zwnwpe7/Q8h/GrBYfpOZ2DJBzJexYLjy/3oWCwlN8t1GcXEEGm +Eih2dHDBFnKbxIaz3eSC9JVgrWueZ5D+M+F91e7qCDLAjP5ZisVlAr8ryHCcMkv9MLCcuFW lZfKR1IzWlSNZGptWpJSrYmLVqVr1cejDyQn2VHguyw/zX5/HU1172lBPIcUodL20QiVjFam 6dOTWpDAkYowqeG3QEuaqEw/IeqSE3RHNaK+Ba3gKMUn0o3+Y4ky/qAyVTwsiimi7v+U4ELk mSi1dl/EFbnJMuhedGfXGWu7w7i9rLZUWOucMyRHj2s/dC31DAx8fcSGD8gTEnozwi/0Pvpz ZW6ZO5Warmv7+yPvF1tLC+MPbYvUx+2oMZlz1/6wr0ET+Y13ccbIttjz9UVbvvv5ZVjG3s7X ZdaIv+LedMyErZuTM7cdQyc+lm3YzD5ZraD0KmVsFKnTK/8DwWhxgVkDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfb4/r+PsK40v5tf5tSFq1B6cZvNHHzY/N4zZuOk7d9Rpd6SM KV3JpVYmzRFX2fXrqO5sjhytVjpxReekrkPMRF12uojEHfPPs9ee97PX889bRIZW0NNFStVR Qa2Sx0sZMSXevCY93DttniLicQYJ/qEsCq5Wmxhov1WFwHQ7jYC+plh4OdyP4OfTNhIKC9oR FL/tIeF2sweBrfwMAx3vJ4LTP8iAvSCbgfTSagaefR4lwH3pAgFV5k3w2viBgta8EgIK+xi4 UphOBMZHAkaMlSwYUxdAb7mehdG3kWD3uGhoLLLTYOtaApevuRm4b7NT0GztJaDj3lUGPKbf NLQ2t1AwnDsD2vNzaLjpLWHg87CRBKN/kIXn9QYCmg1ToEYbsGZ+HaPhUU49AZk3aglwvqpD 8CDrDQFmk4uBRn8/ARZzAQk/ypoQ9OYOsJBxfoSFK2m5CLIzLlGgdUfBz++Bz0VDkZB2vYaC m79caF0MNl0zIdzYP0hireU4/uF/wWDbsIHCj0t4fFffw2Ltgy4WG8zHsKV8MS6930fgYp+f xubKcww2+y6wWDfgJLDX4WC3ztwjlsUJ8cokQb08Zr9YoQ+4E7/IkkvbIlJRd4QOhYh4biWf /6SYDDLDLeI7O0f+chg3h7fkfKB1SCwiOdd4/mXRKxQMJnPb+dqMfCLIFLeAr7c8Z4Is4aL4 uotPyH/S2XxVTf1fDuGi+V9lDirIoYGbPEMVlYfEBjSuEoUpVUkJcmV81DLNYUWKSpm87MCR BDMKFMh4ajTfioY6YhsQJ0LSCZKWvrmKUFqepElJaEC8iJSGSdLuBFaSOHnKCUF9ZJ/6WLyg aUAzRJR0qmTjLmF/KHdQflQ4LAiJgvp/SohCpqeiyt2rcGv0p5MyfV7hZp310JbZh6pXdH/T rZ91q86L5XVL3rt8nVs8+u/zR1pVBz27PTKf/ayjvVbbwnel7vRtazoZbVUnb187tsHmdbb1 vJPmhid2vpjj1O1KfLp3sO1RbEX16hBrjts1qXvH8WS34SH57rTZIVs64EhfGBcec11KaRTy yMWkWiP/Axfy9x88AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It'd be useful to show dept internal stats and dependency graph on runtime via proc for better information. Introduce the knobs. Signed-off-by: Byungchul Park --- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 50 +++------------- kernel/dependency/dept_internal.h | 53 +++++++++++++++++ kernel/dependency/dept_proc.c | 95 +++++++++++++++++++++++++++++++ 4 files changed, 158 insertions(+), 41 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index b5cfb8a03c0c..92f165400187 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DEPT) += dept.o +obj-$(CONFIG_DEPT) += dept_proc.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 24ac20b9bb16..882f29a93483 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,7 @@ #include #include #include +#include "dept_internal.h" static int dept_stop; static int dept_per_cpu_ready; @@ -264,46 +265,13 @@ static bool valid_key(struct dept_key *k) * have been freed will be placed. */ -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, - #include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; - #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT -struct dept_pool { - const char *name; - - /* - * object size - */ - size_t obj_sz; - - /* - * the number of the static array - */ - atomic_t obj_nr; - - /* - * offset of ->pool_node - */ - size_t node_off; - - /* - * pointer to the pool - */ - void *spool; - struct llist_head boot_pool; - struct llist_head __percpu *lpool; -}; - -static struct dept_pool pool[OBJECT_NR] = { +struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ @@ -333,7 +301,7 @@ static void *from_pool(enum object_t t) if (DEPT_WARN_ON(!irqs_disabled())) return NULL; - p = &pool[t]; + p = &dept_pool[t]; /* * Try local pool first. @@ -368,7 +336,7 @@ static void *from_pool(enum object_t t) static void to_pool(void *o, enum object_t t) { - struct dept_pool *p = &pool[t]; + struct dept_pool *p = &dept_pool[t]; struct llist_head *h; preempt_disable(); @@ -2109,7 +2077,7 @@ void dept_map_copy(struct dept_map *to, struct dept_map *from) clean_classes_cache(&to->map_key); } -static LIST_HEAD(classes); +LIST_HEAD(dept_classes); static bool within(const void *addr, void *start, unsigned long size) { @@ -2141,7 +2109,7 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); - list_for_each_entry_safe(c, n, &classes, all_node) { + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) continue; @@ -2217,7 +2185,7 @@ static struct dept_class *check_new_class(struct dept_key *local, c->sub_id = sub_id; c->key = (unsigned long)(k->base + sub_id); hash_add_class(c); - list_add(&c->all_node, &classes); + list_add(&c->all_node, &dept_classes); unlock: dept_unlock(); caching: @@ -2951,8 +2919,8 @@ static void migrate_per_cpu_pool(void) struct llist_head *from; struct llist_head *to; - from = &pool[i].boot_pool; - to = per_cpu_ptr(pool[i].lpool, boot_cpu); + from = &dept_pool[i].boot_pool; + to = per_cpu_ptr(dept_pool[i].lpool, boot_cpu); move_llist(to, from); } } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_internal.h new file mode 100644 index 000000000000..187a9b21f744 --- /dev/null +++ b/kernel/dependency/dept_internal.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dept(DEPendency Tracker) - runtime dependency tracker internal header + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT +#include + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */ diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c new file mode 100644 index 000000000000..7d61dfbc5865 --- /dev/null +++ b/kernel/dependency/dept_proc.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Procfs knobs for Dept(DEPendency Tracker) + * + * Started by Byungchul Park : + * + * Copyright (C) 2021 LG Electronics, Inc. , Byungchul Park + */ +#include +#include +#include +#include "dept_internal.h" + +static void *l_next(struct seq_file *m, void *v, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_next(v, &dept_classes, pos); +} + +static void *l_start(struct seq_file *m, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_start_head(&dept_classes, *pos); +} + +static void l_stop(struct seq_file *m, void *v) +{ +} + +static int l_show(struct seq_file *m, void *v) +{ + struct dept_class *fc = list_entry(v, struct dept_class, all_node); + struct dept_dep *d; + const char *prefix; + + if (v == &dept_classes) { + seq_puts(m, "All classes:\n\n"); + return 0; + } + + prefix = fc->sched_map ? " " : ""; + seq_printf(m, "[%p] %s%s\n", (void *)fc->key, prefix, fc->name); + + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + list_for_each_entry(d, &fc->dep_head, dep_node) { + struct dept_class *tc = d->wait->class; + + prefix = tc->sched_map ? " " : ""; + seq_printf(m, " -> [%p] %s%s\n", (void *)tc->key, prefix, tc->name); + } + seq_puts(m, "\n"); + + return 0; +} + +static const struct seq_operations dept_deps_ops = { + .start = l_start, + .next = l_next, + .stop = l_stop, + .show = l_show, +}; + +static int dept_stats_show(struct seq_file *m, void *v) +{ + int r; + + seq_puts(m, "Availability in the static pools:\n\n"); +#define OBJECT(id, nr) \ + r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ + if (r < 0) \ + r = 0; \ + seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + #include "dept_object.h" +#undef OBJECT + + return 0; +} + +static int __init dept_proc_init(void) +{ + proc_create_seq("dept_deps", S_IRUSR, NULL, &dept_deps_ops); + proc_create_single("dept_stats", S_IRUSR, NULL, dept_stats_show); + return 0; +} + +__initcall(dept_proc_init); From patchwork Tue May 13 10:06:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084960 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3565-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [147.75.80.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXMV3QG6z1yPv for ; Tue, 13 May 2025 20:12:06 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 957BA1881B04 for ; Tue, 13 May 2025 10:12:09 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D8B5A25392E; Tue, 13 May 2025 10:07:53 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D77A3244691; Tue, 13 May 2025 10:07:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; cv=none; b=CkixV3aKVKPSAc11S5ybn//vFsz9w6/7nUoGqk2T1YVRhqaB3EF+G4HcZdpHPCBkJTyZgFuZzXlzjzeMngcESFdBveyCwgxhSPU4W2rYeLLdB8X9JtA0vP2yMBg1e9dH3Bde3/F2MVKxpeAHZ3gI31NS5vWN5f2bOeYP6CMxik0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; c=relaxed/simple; bh=gP8tQf8HzUzkOl0/4uKU2DTMSmlUgz+LzrjzVdAxmXI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jRtOymb4CyW8agWG1QLJW6fNFJ2744YkRb+odvZNIWzB1FdcGyNewghVJKVbPcij7SVioThM8IDy8IYlBTduc7FhkzSWmKZxkXW/NVodQ2S0YESzYyeOSMAIFUjJmNVihuG9Qt6P5gLjavLDB8YtQmH34t3YHJN3I0PPfy+Pk4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-2a-682319ee2b9f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 07/43] dept: distinguish each kernel context from another Date: Tue, 13 May 2025 19:06:54 +0900 Message-Id: <20250513100730.12664-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTaRSG+b6Z+WZarZnU24hGsWowGi81ao6Ju9FE13GjidEYjfzQRia2 EYppFcVbqCBhixAWUnC9VpBSoSgWoyhWK2gRjIiIXAxUQaMQbsraIhfRUuOfkyfve973/Dkc paxkQjmd/qBk0GuiVEROy3vG5yzqnTZHuzStYj74vibTcOGGg0Dt9UIEjlsmDJ1PNkCjvxvB 8PMXFGRbahFcaWul4JbHi8BlP0Xg1YcJUO/rI1BlSSGQkHuDwMuuEQwtWRkYCp2b4a3tIw3P 0nMwZHcSOJ+dgAOjA8OgrYAFW/w8aLefY2GkTQ1V3gYGXG8Wwn+XWgjcd1XR4Cltx/Dq3gUC XscPBp55ntLgT5sOtf+mMlDUm0Ogy2+jwObrY6HObcXgsU6B4sRAYdL/owxUproxJF29iaG+ uQzBg+R3GJyOBgIVvm4MJU4LBUP5TxC0p/WwcPrMIAvnTWkIUk5n0ZDYsgKGvwUuX/yqBtPl YhqKvjegNX+IjksOJFZ091FiYslhccj3moguv5UWq3ME8e65VlZMfPCGFa3OQ2KJfYGYe78T i1f6fYzoLPiHiM7+DFY099Rjsbemht0yY5d8daQUpYuVDEv+3CPX5rpH8YG6HUcyBxJwPDq7 3ow4TuCXC64iMCNZEFvf2+kxJny40NQ0SI3xJD5MKEn9yJiRnKP4hnFC48VmNGZM5DcLeW3V wQDNzxPMmT1BVvArBNe1LPpX6SyhsNgdLJLxK4Xv+TVBXRnYSbcW0mOlAp8tE5yVzeRXYJrw yN5EpyOFFYUUIKVOHxut0UUtX6yN0+uOLN4bE+1EgfeynRiJKEX9tdvKEc8h1XjF087ZWiWj iTXGRZcjgaNUkxSmOwFJEamJOyoZYnYbDkVJxnI0naNVUxXL/Icjlfw+zUFpvyQdkAy/XczJ QuPR1dIImXpto3a9ZebGnV9e5x872V5VOjkjb1zYptH6mKQ8t+PshpQIj8Vr6BraWgcDZRtn 7T/e+35X9aBXpp27qvzx2vCHA4tuI9W2b2vK4oyf1cN/7Qwpq57LTNyecDOXPN+yqmN7zBJ/ yN+Z4Z51c2aE1bX6R1qSyaZPHaEWOklvUtFGrUa9gDIYNT8BgHO97loDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTYRjG/b5zzne22eJkVicrs0VUdpUyXiiioPBUFhVBEJIOPbXhNNnM NAi0LS1L0UTtojUvTdNZcwpZOREl80JmadZCLe1CIy9Um2XNahr98/Ljed73ef55JZRPBeMn UcfGi9pYpUZBZLRs32b9mtH5S1Xrc9Nmgst5noaCu2YCXXcqEZhrUzA4HoXAy/FhBL+ePKUg P7cLQdFgPwW1LQMIbOVnCXS/nwk9rjECbbkXCehL7hJ49tmNoS/vMoZK6154Y/pIQ0dWMYZ8 B4Hr+XrsGZ8wTJgqWDAlL4Oh8mssuAeDoG2gl4HmwjYGbK9XwdUbfQTqbW00tNQNYeh+UEBg wPyHgY6WVhrGMxdAV3YGA1WjxQQ+j5soMLnGWHjeaMTQYpwLFoMnNfXbbwYeZzRiSC2txtBj f4ig4fxbDFZzL4Fm1zCGGmsuBT/LHiEYyhxh4dylCRaup2QiuHgujwZDXzD8+uFpLnQGQcpN Cw1Vk71o21bBfMOMhObhMUow1JwSfrpeEME2bqSF9mJeuH+tnxUMDa9ZwWg9KdSUBwol9Q4s FH11MYK14gIRrF8vs0L6SA8WRjs72f2Ljsi2RIkadYKoXbc1QqYqafyN454fTsz5rsfJ6MrO dCSV8NxGvv9dOT3FhFvOv3o1QU2xLxfA12R8ZNKRTEJxvd78y0I7mjJmc3v5W4Pt0wc0t4xP zxmZZjkXzNtu59H/QhfzlZbG6SApt4mfLOuc1n08O1nGSjoLyYzIqwL5qmMTYpRqTfBaXbQq KVaduDbyRIwVeT7IdMadXYec3SFNiJMgxQx5q2OJyodRJuiSYpoQL6EUvvKUex5JHqVMOi1q T4RrT2pEXRNaIKEV8+S7D4sRPtxxZbwYLYpxova/iyVSv2R0XDrLf56yqkPXcTXCKzvA5jzg VZS65dscVbX4peGYY2Noon2yKZQsz97z5cPSBLt9x5FVOW7TSuf67er4FeFPH8d17ltoCdtz +2D4iNNgSUrTju4ivrXuwrAZUmqDpq756OpWx6JoQ9rO+OH6dv0T73t+3m5y4VBgqUYaWTIW 56+gdSplUCCl1Sn/AoqGLeo9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Each unique kernel context, in dept's point of view, should be identified on every entrance to kernel mode e.g. system call or user oriented fault. Otherwise, dept may track meaningless dependencies across different kernel context. Plus, in order to update kernel context id at the very beginning of each entrance, arch code support is required, that could be configured by CONFIG_ARCH_HAS_DEPT_SUPPORT. Signed-off-by: Byungchul Park --- include/linux/dept.h | 29 ++++++++++------- include/linux/sched.h | 10 +++--- kernel/dependency/dept.c | 67 ++++++++++++++++++++-------------------- lib/Kconfig.debug | 5 ++- 4 files changed, 61 insertions(+), 50 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index dc50fa9d388b..138106869494 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -25,11 +25,16 @@ struct task_struct; #define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) #define DEPT_MAX_SUBCLASSES_CACHE 2 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { + DEPT_CXT_SIRQ = 0, + DEPT_CXT_HIRQ, + DEPT_CXT_IRQS_NR, + DEPT_CXT_PROCESS = DEPT_CXT_IRQS_NR, + DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ) struct dept_ecxt; struct dept_iecxt { @@ -94,8 +99,8 @@ struct dept_class { /* * for tracking IRQ dependencies */ - struct dept_iecxt iecxt[DEPT_IRQS_NR]; - struct dept_iwait iwait[DEPT_IRQS_NR]; + struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR]; + struct dept_iwait iwait[DEPT_CXT_IRQS_NR]; /* * classified by a map embedded in task_struct, @@ -207,8 +212,8 @@ struct dept_ecxt { /* * where the IRQ-enabled happened */ - unsigned long enirq_ip[DEPT_IRQS_NR]; - struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR]; /* * where the event context started @@ -252,8 +257,8 @@ struct dept_wait { /* * where the IRQ wait happened */ - unsigned long irq_ip[DEPT_IRQS_NR]; - struct dept_stack *irq_stack[DEPT_IRQS_NR]; + unsigned long irq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR]; /* * where the wait happened @@ -383,6 +388,7 @@ extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); +extern void dept_update_cxt(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -430,6 +436,7 @@ struct dept_map { }; #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) +#define dept_update_cxt() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index a1924b40feb5..802fca4d99b3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -827,19 +827,19 @@ struct dept_task { int wait_hist_pos; /* - * sequential id to identify each IRQ context + * sequential id to identify each context */ - unsigned int irq_id[DEPT_IRQS_NR]; + unsigned int cxt_id[DEPT_CXTS_NR]; /* * for tracking IRQ-enabled points with cross-event */ - unsigned int wgen_enirq[DEPT_IRQS_NR]; + unsigned int wgen_enirq[DEPT_CXT_IRQS_NR]; /* * for keeping up-to-date IRQ-enabled points */ - unsigned long enirq_ip[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; /* * for reserving a current stack instance at each operation @@ -893,7 +893,7 @@ struct dept_task { .wait_hist = { { .wait = NULL, } }, \ .ecxt_held_pos = 0, \ .wait_hist_pos = 0, \ - .irq_id = { 0U }, \ + .cxt_id = { 0U }, \ .wgen_enirq = { 0U }, \ .enirq_ip = { 0UL }, \ .stack = NULL, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 882f29a93483..a936a8d831c5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -229,9 +229,9 @@ static struct dept_class *dep_tc(struct dept_dep *d) static const char *irq_str(int irq) { - if (irq == DEPT_SIRQ) + if (irq == DEPT_CXT_SIRQ) return "softirq"; - if (irq == DEPT_HIRQ) + if (irq == DEPT_CXT_HIRQ) return "hardirq"; return "(unknown)"; } @@ -409,7 +409,7 @@ static void initialize_class(struct dept_class *c) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iecxt *ie = &c->iecxt[i]; struct dept_iwait *iw = &c->iwait[i]; @@ -435,7 +435,7 @@ static void initialize_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { e->enirq_stack[i] = NULL; e->enirq_ip[i] = 0UL; } @@ -451,7 +451,7 @@ static void initialize_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { w->irq_stack[i] = NULL; w->irq_ip[i] = 0UL; } @@ -490,7 +490,7 @@ static void destroy_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (e->enirq_stack[i]) put_stack(e->enirq_stack[i]); if (e->class) @@ -506,7 +506,7 @@ static void destroy_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (w->irq_stack[i]) put_stack(w->irq_stack[i]); if (w->class) @@ -664,7 +664,7 @@ static void print_diagram(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (!firstline) pr_warn("\nor\n\n"); firstline = false; @@ -697,7 +697,7 @@ static void print_dep(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { pr_warn("%s has been enabled:\n", irq_str(irq)); print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); pr_warn("\n"); @@ -865,7 +865,7 @@ static void bfs(void *root, struct bfs_ops *ops, void *in, void **out) */ static unsigned long cur_enirqf(void); -static int cur_irq(void); +static int cur_cxt(void); static unsigned int cur_ctxt_id(void); static struct dept_iecxt *iecxt(struct dept_class *c, int irq) @@ -1442,7 +1442,7 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) if (d) { check_dl_bfs(d); - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iwait *fiw = iwait(fc, i); struct dept_iecxt *found_ie; struct dept_iwait *found_iw; @@ -1486,7 +1486,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, struct dept_task *dt = dept_task(); struct dept_wait *w; unsigned int wg; - int irq; + int cxt; int i; if (DEPT_WARN_ON(!valid_class(c))) @@ -1502,9 +1502,9 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; - irq = cur_irq(); - if (irq < DEPT_IRQS_NR) - add_iwait(c, irq, w); + cxt = cur_cxt(); + if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) + add_iwait(c, cxt, w); /* * Avoid adding dependency between user aware nested ecxt and @@ -1578,7 +1578,7 @@ static struct dept_ecxt_held *add_ecxt(struct dept_map *m, eh->sub_l = sub_l; irqf = cur_enirqf(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) add_iecxt(c, irq, e, false); del_ecxt(e); @@ -1727,7 +1727,7 @@ static void do_event(struct dept_map *m, struct dept_map *real_m, add_dep(eh->ecxt, wh->wait); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_ecxt *e; if (before(dt->wgen_enirq[i], wg)) @@ -1774,7 +1774,7 @@ static void disconnect_class(struct dept_class *c) call_rcu(&d->rh, del_dep_rcu); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { stale_iecxt(iecxt(c, i)); stale_iwait(iwait(c, i)); } @@ -1799,27 +1799,21 @@ static unsigned long cur_enirqf(void) return 0UL; } -static int cur_irq(void) +static int cur_cxt(void) { if (lockdep_softirq_context(current)) - return DEPT_SIRQ; + return DEPT_CXT_SIRQ; if (lockdep_hardirq_context()) - return DEPT_HIRQ; - return DEPT_IRQS_NR; + return DEPT_CXT_HIRQ; + return DEPT_CXT_PROCESS; } static unsigned int cur_ctxt_id(void) { struct dept_task *dt = dept_task(); - int irq = cur_irq(); + int cxt = cur_cxt(); - /* - * Normal process context - */ - if (irq == DEPT_IRQS_NR) - return 0U; - - return dt->irq_id[irq] | (1UL << irq); + return dt->cxt_id[cxt] | (1UL << cxt); } static void enirq_transition(int irq) @@ -1876,7 +1870,7 @@ static void dept_enirq(unsigned long ip) flags = dept_enter(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { dt->enirq_ip[irq] = ip; enirq_transition(irq); } @@ -1922,6 +1916,13 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled = false; } +void noinstr dept_update_cxt(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + /* * Ensure it's the outmost softirq context. */ @@ -1929,7 +1930,7 @@ void dept_softirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_SIRQ] += 1UL << DEPT_CXTS_NR; } /* @@ -1939,7 +1940,7 @@ void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_HIRQ] += 1UL << DEPT_CXTS_NR; } void dept_sched_enter(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index d1aeb206b19a..a626631f6bec 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1365,9 +1365,12 @@ config DEBUG_PREEMPT menu "Lock Debugging (spinlocks, mutexes, etc...)" +config ARCH_HAS_DEPT_SUPPORT + bool + config DEPT bool "Dependency tracking (EXPERIMENTAL)" - depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT && ARCH_HAS_DEPT_SUPPORT select DEBUG_SPINLOCK select DEBUG_MUTEXES select DEBUG_RT_MUTEXES if RT_MUTEXES From patchwork Tue May 13 10:06:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084965 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3563-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXNK10Y6z1yPv for ; Tue, 13 May 2025 20:12:49 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4B19616AE90 for ; Tue, 13 May 2025 10:11:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 98A382528F0; Tue, 13 May 2025 10:07:53 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7B04E2459F9; Tue, 13 May 2025 10:07:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; cv=none; b=RidrudEs4mygV3ub85GrONb6nOBegA7euC6NMZz4tJaPIHk2Djgv8pizSr0ZBQL4TqA1io8SFl70jCp2SMTD5s8USuK+cXz/LNIQd5itGIQq9RyfheQgbDZKkAUD4eXO81KbDfim8TfT/hSk4CCdF6BW51CHSB6QB5ynt5hQnUs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; c=relaxed/simple; bh=8SxiXT7aMr/Jndlc0Cvu8YV+kxqp+XaubTMN75I/AOI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=s/XujMkl7vYkARKsDXnclsC95OaUaVlEn89AEmbsJEx+E4dNO4kk4aWyiLgZ1Onbl8h0lYlcKL6wOORL7/VFOy9Cute4OerJdsCtED8NlO9+InwF6AjExpFOvvUumVXIunY2DqO8s2iXCt3F9cmxDXOxXZsxOoeyLdXRE1hJe8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-3a-682319ee2425 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 08/43] x86_64, dept: add support CONFIG_ARCH_HAS_DEPT_SUPPORT to x86_64 Date: Tue, 13 May 2025 19:06:55 +0900 Message-Id: <20250513100730.12664-9-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/nued5Op09O60eMj/O7yjOwmeG+TlPYtPMJmXc9NTdVHKX fthYp6McJaaiX65wpS7qyuRHLZnTj8nRqSShEamrKXeTfnDV/PPZa+/3e+/PP28aF5sEM2lF eCSvDJeFSkghIbQ653r2zZgvX/k1eznYfiUSkHXPQIL5bhECQ7kag+7nO6DF3otg+OUrHNJT zQhyP3/AodzUgaCy4AwJTV+mgcXWT0Jd6gUS4m/eI+F1zwgG7WlXMCgy7oaP+i4CGlLyMEjv JiEzPR5znO8YDOkLKdDHLYTOggwKRj5Loa6jWQCVbcvgek47CU8q6wgwVXRi0PQoi4QOw18B NJhqCbAnu4P5cpIAivvySOix63HQ2/opeFOtw8Ckc4USjaPw3OCYAF4kVWNw7lYpBpZ3jxFU JX7CwGhoJuGZrReDMmMqDn/ynyPoTLZScPbiEAWZ6mQEF86mEaBpXw3Dvx2fs39JQX2jhIDi 0Wa0aQNnyDEg7llvP85pyqK5P7a3JFdp1xFcfR7LPcz4QHGaqjaK0xlPcGUFHtzNJ90Ylztg E3DGwvMkZxy4QnFaqwXj+hobqT2zDgjXB/GhiiheuWLjYaH8fbaViKh3iRmOv03EoR+MFjnR LOPNjlVU4FpET/DVDNW4TDKL2dbWIXycXZi5bFlSl0CLhDTONE9lW7LfoXFjOhPAtr+6PMEE s5AdffuGGGcRs5rV3C8kJ/vnsEUl1RNFTswadjS/cSIjdmRSdEXEZCbdidVqAid5Bvu0oJVI QSIdmlKIxIrwqDCZItTbSx4brojxOnIszIgc69KfGgmoQAPmvTWIoZHEWVTbPU8uFsiiVLFh NYilcYmLSP3AIYmCZLEneeWxQ8oTobyqBrnThMRNtMoeHSRmQmSR/FGej+CV/12MdpoZhzbn otYd0cHO/KVFWRFyc+m6+VcX+H3p2pU2drg3sC/zdv3swaUtu65vOe0rjftWXrI1MWrfwYQx ssU9xI+S7p7+LdBS5W89Xmq9FlwTktCY8D0ksmrY37s4z37np327xc0n2H9n24G1SzLNXU3B i7b2bHztk+/rrMY9Xfm95/3o/dskhEouk3rgSpXsH/5nLBBZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+/6eO85+O+GHeegsNs1DKJ8ta4bNz0NiszE2+uE3d1yX3Sll Q7nTuJSKUyKucKUOuSvloZxaqVDRM+eomUlPG91NKrna/PPea+/3e+/PPx8Gl90jZzFK9XFR oxZUckpCSLYF65b0z1ygWP7y41RwDZ4n4MZDCwWNDwoQWIriMeiu2ght7l4Ew28bcEg3NiLI 7vyEQ1G1E0FZ3lkKmr5OgWbXAAW1xkQKdLcfUvCuZwQDx9U0DAqsofDZ/I2A1yk5GKR3U3A9 XYd55DsGQ+Z8GsxxftCVl0nDSGcA1DpbSajMqiWh7IM/XLvpoOB5WS0B1aVdGDQ9vUGB0zJG wuvqGgLcybOhMTWJhPv9ORT0uM04mF0DNLy3mzCoNk2HQr1nNeHXXxJeJdkxSLjzCIPmjmcI ys9/wcBqaaWg0tWLgc1qxOFPbhWCruQ+Gs5dHKLhenwygsRzVwnQOwJh+LfnctZgAMTfKiTg /mgrWhvCW25aEF/ZO4DzetsJ/o+rheLL3CaCr8vh+CeZn2heX/6B5k3WKN6Wt5i//bwb47N/ ukjemn+B4q0/02je0NeM8f319fT2OXskaw6JKmW0qFkWEi5RfMzqI47V+cQM6+4ScegHa0AM w7GruCuZWgPyZih2EdfePoSPsw87n7MlfSMNSMLgbOskri2rA40HU9m9nKMhdYIJ1o8bbXlP jLOUDeT0xfnUOHPsPK6g0D4x5M0GcaO59RMdmaeTYiogUpDEhLzykY9SHR0hKFWBS7VHFbFq ZczSg5ERVuR5IPOpkdRSNNi0sQKxDJJPltZ0+ypkpBCtjY2oQByDy32k8SUeS3pIiD0paiL3 a6JUorYCzWYI+Qzp5l1iuIw9LBwXj4riMVHzP8UY71lx6G34iqQx4U1iKCl2SMYafgcFdBaf znGE+Knc61cnvBLCDGe3rseJfQvb645sSF/rP6YxFmUoHkdNNr/w3Tv3R+i9B7kzttXonU5Z VUOYXT3lUm4LOcfrb3ByYqqC3Gn0kqbp/M94hx6YtmTHyjbXulLp1y12v9X2gchN5ZdtGSW7 5YRWIQQsxjVa4R+GEGaMPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: dept needs to notice every entrance from user to kernel mode to treat every kernel context independently when tracking wait-event dependencies. Roughly, system call and user oriented fault are the cases. Make dept aware of the entrances of x86_64 and add support CONFIG_ARCH_HAS_DEPT_SUPPORT to x86_64. Signed-off-by: Byungchul Park --- arch/x86/Kconfig | 1 + arch/x86/entry/syscall_64.c | 7 +++++++ arch/x86/mm/fault.c | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 5873c9e39919..4b6d9e59d96c 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -38,6 +38,7 @@ config X86_64 select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 select EXECMEM if DYNAMIC_FTRACE + select ARCH_HAS_DEPT_SUPPORT config FORCE_DYNAMIC_FTRACE def_bool y diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c index b6e68ea98b83..66bd5af5aff1 100644 --- a/arch/x86/entry/syscall_64.c +++ b/arch/x86/entry/syscall_64.c @@ -8,6 +8,7 @@ #include #include #include +#include #define __SYSCALL(nr, sym) extern long __x64_##sym(const struct pt_regs *); #define __SYSCALL_NORETURN(nr, sym) extern long __noreturn __x64_##sym(const struct pt_regs *); @@ -86,6 +87,12 @@ static __always_inline bool do_syscall_x32(struct pt_regs *regs, int nr) /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { + /* + * This is a system call from user mode. Make dept work with a + * new kernel mode context. + */ + dept_update_cxt(); + add_random_kstack_offset(); nr = syscall_enter_from_user_mode(regs, nr); diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 296d294142c8..241537ce47fe 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -20,6 +20,7 @@ #include #include /* find_and_lock_vma() */ #include +#include #include /* boot_cpu_has, ... */ #include /* dotraplinkage, ... */ @@ -1220,6 +1221,12 @@ void do_user_addr_fault(struct pt_regs *regs, tsk = current; mm = tsk->mm; + /* + * This fault comes from user mode. Make dept work with a new + * kernel mode context. + */ + dept_update_cxt(); + if (unlikely((error_code & (X86_PF_USER | X86_PF_INSTR)) == X86_PF_INSTR)) { /* * Whoops, this is kernel mode code trying to execute from From patchwork Tue May 13 10:06:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084984 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3566-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXR83WF7z1yXB for ; Tue, 13 May 2025 20:15:16 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3F8F1862DE6 for ; Tue, 13 May 2025 10:12:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D5DB0253F39; Tue, 13 May 2025 10:07:54 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 996F024677D; Tue, 13 May 2025 10:07:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130874; cv=none; b=f+kWosv6zNu9wDSKh1XfcBUlKQmpvs1rkuro8wFMH2aozFMxYEDiPj3kAPq09nk0oqILGeDp2+CNnESdV+VTS3thHDqv9ao0jNooMhuSN/Kbr2qwzVPO861EpRg5w8xM26Vm9E5VYKBaWHMwbVYXbiH+Um5p79fYGej6G1JI1Nw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130874; c=relaxed/simple; bh=BG7AZBSjSmF6NrOFDpw8Rk4CMPwauNC98FDvthYcPSo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=fjKNTNcdNNikV1N/rGyKOc0I8apB63v4aY/FacTQm/f9mpVpMY4CmH0CWH7myjw6P/zRRHLqYpt6hbuxOA2ZMuRlMpNV/hoZ7pGbX4t3Yq4Sq69tZVvuwqxC/He3Ma9MBAE169CAitZIAf0dKG3uuTP+vDHUVf+t7HZbzlR+JpU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-4a-682319eeafcf From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 09/43] arm64, dept: add support CONFIG_ARCH_HAS_DEPT_SUPPORT to arm64 Date: Tue, 13 May 2025 19:06:56 +0900 Message-Id: <20250513100730.12664-10-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSXUxTdxjG/f/P6TmHatlJZ7YDGKc14KbIZBHzXuhivPE/P6IJ8WLuYjZy snaWjxX5mjOCogGxBGsQsaAVtVRaKrbq2AaEYai0OtYhK9jwMcgGNnw0QVpFENdivHnzy/Pk ed6bh6PkXZJYTp1xTNRmKDUKRkpLp1bUbZqOWafa3OPdCsHZEhpq7lgZ8NgsCKz3ijD4O3dB X2gSwfwff1JQVelBcH1kkIJ7ziEEreZTDDz9Nxp6gwEGXJVlDJy+cYeBvyYWMAxc0mOw2PfB sGmMhscVdRiq/AwYqk7j8HmOYc7UwIKpMB5GzVdYWBhJBteQVwKtvo1QfXWAgZZWFw3O5lEM T3+tYWDI+lYCj51dNITK48BzQSeBxuk6BiZCJgpMwQALPe1GDE7jR9BUHC48+2JRAo907RjO 3ryLoffZbwjaSv7BYLd6GXgYnMTgsFdS8Lq+E8Fo+RQLZ87PsWAoKkdQduYSDcUDKTD/Kvy5 djYZiq410dD4xot2bCfWq1ZEHk4GKFLsyCOvg38zpDVkpIm7TiC/XBlkSXGbjyVGew5xmDeQ Gy1+TK7PBCXE3lDKEPuMniXnpnoxme7uZg+sOiTdliZq1Lmi9vMvD0tVj2YbcZZLnj/epseF qPyDcyiKE/gtwpOq+5L37K/2LTHDrxf6++eoCK/k1wgO3VhYl3IU710u9NU+QxHjQ/5rYcw/ sRSg+XjBrXPjCMv4rcJtt416V/qJYGlqX+KosP6mvpuOsJxPESqMFjpSKvCGKKF2McC+C8QI v5v76QokM6JlDUiuzshNV6o1W5JUBRnq/KQjmel2FN6X6cTCN81oxpPagXgOKVbIuvxrVXKJ Mje7IL0DCRylWCkr+jksydKUBT+K2sxvtTkaMbsDxXG04mPZF6G8NDn/nfKYeFQUs0Ttexdz UbGFqPTmQH7Z/ltNr2wJjlDO3a/aFLXeRN7wsiSlRIjbhOoPG+KHO/sCR3fquNjS5FRdQqpX 7Y9Vm6T64xcX9+TPW+4ntgQTPws4Nfs+Pb76ZHRSjbPZ5llHNJfdjZsP+nf7EsYeNO/+/r8K s/xA994037WTNv1zy0/VPzxoCY5nuqIVdLZKmbyB0mYr/wde5NToWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfZ77VuoqN5XAVbOw1VTju8RhTrLFmPnBZ0u2LNkHlX2QRi+2 aUHXKhN1EWw1KoMVNiAqaC1aGC2KF40v0IYURVsiQ6mgCCjE4BoQDOvtBKquddmXk19+/5z/ +XIUlLqBWagw5O0VzXk6k4ZV0spvP7eumliwWL+280I6yJHjNFRf9rDQfcmNwHO1CEP4zmbo i44jmL3/JwVVFd0Izg8PUnC1YwiBt/4ICz0v5kFInmQhUFHMgrX2MgsPxmIYBirLMbilb+CZ a5SGTrsTQ1WYhTNVVhwff2GYdjVw4CrUwkj9aQ5iwxkQGOploL0mwIC3fwWcOjvAQqs3QEPH jREMPbeqWRjyvGegs+MeDdHSRdBdVsJA44SThbGoiwKXPMnBwzYHhg5HKjTZ4q3H/n7HwN2S NgzHLlzBEHrSgsB3/DkGydPLQrs8jqFZqqBgpu4OgpHSVxwc/WWagzNFpQiKj1bSYBvIhNk3 8cs1kQwoOtdEQ+PbXrRxA/Gc9SDSPj5JEVvzT2RGfsQSb9RBk6BTIDdPD3LE5uvniEPaR5rr l5Pa1jAm56dkhkgNJ1giTZVz5OSrECYTXV3cdx9nKb/YKZoM+aJ5zYZspf5upBHvCaj3v/SV 40JUmnwSJSkE/jMhfKqfSTDLLxUeP56mEpzCfyI0l4zGvVJB8b1zhb6aJygRzOe3CaPhsQ8L NK8VgiVBnGAVv174I3iJ+q80XXA3tX3gpLh/W9dFJ1jNZwp2h5u2I6UDzWlAKYa8/FydwZS5 2mLUF+QZ9q/esTtXQvEPcv0cK7uBIj2b/YhXIM1HqnvhT/VqRpdvKcj1I0FBaVJURdfjSrVT V3BANO/ebt5nEi1+tEhBa9JUX28Rs9X8Lt1e0SiKe0Tz/ylWJC0sRKllOVqb1R6Sd1QbWyTn uf7gVjk9+cWCylrr6MybLOO0ZcKdbW/duCZ6KCeNDGuviWkH5m6Sbr8v9ldkBZfEvj+Y7E/7 6ktlrM6W80+bpZZZ96vu9dOth5nfQibfyvynqc6LwqBN1TJ5c3YqtS9S4zP+OBZa553R/r5p 2fMr8g8a2qLXZSynzBbdvxjrewk9AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: dept needs to notice every entrance from user to kernel mode to treat every kernel context independently when tracking wait-event dependencies. Roughly, system call and user oriented fault are the cases. Make dept aware of the entrances of arm64 and add support CONFIG_ARCH_HAS_DEPT_SUPPORT to arm64. Signed-off-by: Byungchul Park --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/syscall.c | 7 +++++++ arch/arm64/mm/fault.c | 7 +++++++ 3 files changed, 15 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a182295e6f08..6c69598a6423 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -279,6 +279,7 @@ config ARM64 select HAVE_SOFTIRQ_ON_OWN_STACK select USER_STACKTRACE_SUPPORT select VDSO_GETRANDOM + select ARCH_HAS_DEPT_SUPPORT help ARM 64-bit (AArch64) Linux support. diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index c442fcec6b9e..bbd306335179 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -96,6 +97,12 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, * (Similarly for HVC and SMC elsewhere.) */ + /* + * This is a system call from user mode. Make dept work with a + * new kernel mode context. + */ + dept_update_cxt(); + if (flags & _TIF_MTE_ASYNC_FAULT) { /* * Process the asynchronous tag check fault before the actual diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ec0a337891dd..0fcc3dc9c2a9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -616,6 +617,12 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, if (!(mm_flags & FAULT_FLAG_USER)) goto lock_mmap; + /* + * This fault comes from user mode. Make dept work with a new + * kernel mode context. + */ + dept_update_cxt(); + vma = lock_vma_under_rcu(mm, addr); if (!vma) goto lock_mmap; From patchwork Tue May 13 10:06:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084968 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ide+bounces-3564-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXP753lZz1yPv for ; Tue, 13 May 2025 20:13:31 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 9BE047A3DF9 for ; Tue, 13 May 2025 10:10:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9C744252912; Tue, 13 May 2025 10:07:53 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B48CA24728A; Tue, 13 May 2025 10:07:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; cv=none; b=cTBNgD9xKaXYBkIPaeHKutf0bHHn7rLebH9EY/Aq8nrYf/xZyyJdR9QCBF+WBRf9NDXEpGGu5atkXg1xWC5etyVTfNIXG5BpLyHp2mj7Z0n8LD4ES+RsBuPySB0FZTEDlvE4xt9JhUBsPounXwI1bcft3Xz47qMEXrSyqJcw7Vc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130873; c=relaxed/simple; bh=KlggbR8P/n36oRZlJZsBth/VG91g9Dz5bdB/P0jqAy0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bjDvnLVSqvysT3sRoxA1+MY0Hz3ZhtT+F4NXP61WrsntuQxoqUht6kNpVFmgJV3lLyxSU5wnPYNIhzgiRxGfb4x2QQVd8DWOzapabh4YzA7qaF9btqhb6h5i7BYuLBX5NE9sguDtI3tieM/Kq2ZLpUpsZ7To+1T0WjCBpPDBaUY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-5b-682319ee0be8 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 10/43] dept: distinguish each work from another Date: Tue, 13 May 2025 19:06:57 +0900 Message-Id: <20250513100730.12664-11-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTdxjG9z/n9H8Olc6TejvWTbFIjExQprLXxJFFP/jng4nGgIox2MjZ WrlaoMimC53FOBwNYhDBC+WSUqEKFo3gwDHACqiIglAN4sDFrOGmnW1A8NJi/PLml+d53+f5 8nK03C5RcJqkNFGbpEpQYikjHfMvDZlYHKheO2nE4H5zgoHzNVYM3VeqEViv6Slw3t4K/Z5R BNP3H9BQWNCNoHToGQ3X7IMImiy/Yej590vodU9g6Cg4ieFYeQ2GhyMzFAycyaeg2rYNnptf MnA3r4yCQieGc4XHKO/4j4IpcxUL5qwgGLYUszAzFAYdg30SaHr6DRRdHMDQ2NTBgL1+mIKe m+cxDFo/SOCuvZ0Bj3EJdJ/KlcDl8TIMIx4zDWb3BAuPmk0U2E0LodbgDTz+/3sJ3MltpuB4 xVUKep/8ieDWiX8osFn7MLS6RymosxXQ8LbyNoJh4xgL2X9MsXBOb0RwMvsMA4aBDTA96W2+ 8CYM9CW1DFx+14d++J5YL1oRaR2doImhLoO8dT/GpMljYkhnmUAaip+xxHDrKUtMtnRSZwkm 5Y1OipS63BJiq/odE5srnyU5Y70UGe/qYrd/FSPdFCcmaHSidk3Efqm6NluXMs4ermh5RWWh RpyD/DiBXy8M1/TTOYib5Uud6T4Z8ysFh2OK9vF8PkCoy30pyUFSjub75gj9F54gnzGP3yIY W7MYHzN8kHDvZsksy/hwwZHfS3/KXyZU1zbPsp9Xf1fZNbsj5zcIeaZqxhcq8Kf9BL1hnPl0 sFj42+Jg8pDMhL6oQnJNki5RpUlYH6rOTNIcDj2QnGhD3u8yH53ZW49c3TtbEM8hpb+s3blc LZeodKmZiS1I4GjlfJn+hleSxakyfxa1ybHa9AQxtQUt4RjlItm3now4Of+TKk2MF8UUUfvZ pTg/RRbSrUxb8MKFi2JTmPplI/bvkn85FGCJj/1V8XBV1KXl6q8PrtvXVkGmw44qUGjDgbP0 5rYdR2RzN/V01qwIWR2c4hh67oxeuita2YYiwq83GKObY6I2NkRGf3i9elK7W9FunRMU9JdF meEfGFPiKS7fG7H7CBe4JzfSODX0Y7i65FCokklVq8KCaW2q6iN1Dzi1WQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfSzUcRzH+/6eXV37Tapfer7Qes6W+mxatfrDb22sh5Wtf3Lxq7t5qN2h znoglIjQEFJHdeROOJQ8lLl1dSkUnYdQzMQ8tbiLiI6tfz577f3+vD/vfz4Mbp9HOjLyoGBB ESQNkFAiQuTlHrltZMUG2c4fY5vAMh5DwP1CHQWNz7QIdKURGAy88YAW6xCCqY8NOKSlNCLI 7u7EodTYhaA67zoFTb2LodkySoEpJY6CyEeFFHwanMagIzUZA63eE75p+gioS8zBIG2Agsy0 SMw2+jGY1OTToAl3hp68DBqmu13B1GUmwZBlIqG6fQukP+igoKraRICxvAeDpor7FHTpZkmo M74jwJqwEhqT4kkoGMmhYNCqwUFjGaXhc40aA6N6GRRF2a7eGJsh4W18DQY3Hhdj0NxWieBV zHcM9DozBQbLEAYl+hQc/uS+QdCTMExD9O1JGjIjEhDERacSENXhBlMTtuascVeIeFhEQMFf Mzqwj9c90CHeMDSK81ElF/k/li8UX21VE/z7HI5/mdFJ81Gv2mlerQ/hS/I284+qBjA++5eF 5PX5tyhe/yuZ5mOHmzF+pL6ePrL6lGivnxAgDxUUO/b5iGRF0aEXRuhLj2t/YuGoiopFDMOx u7in70NikR1DsRu51tZJfI4d2HVcSXwfGYtEDM6aF3ItWW1ozljCHuISDOHEHBOsM/eh4uE8 i9ndXGty83yYY9dy2qKaebaz6X9z6+d37Fk3LlGtJRKRSI0W5CMHeVBooFQe4LZd6S9TBckv bfc9H6hHtv/RXJlOKkfjTR61iGWQZJH43cB6mT0pDVWqAmsRx+ASB3HEC5sk9pOqwgTF+dOK kABBWYtWMoRkufiwt+Bjz56TBgv+gnBBUPx3McbOMRwVutet0p1Msha7HDte37ZfhaeW3Szr 9bzp4uTsEtcNCi+t/5rfo7Izn9OHm5K82cvs0rRpS9escSJr/9djpYwiX/W6d6Kh5vnEjNDe 8Lpgx70M87P+03uuMqb4mF2EqMf3YOW2UieHO0enrj2p6zfc3brQaWvu2RPnGrON7clhWm8J oZRJXTfjCqX0H7RVwqA7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Workqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context. So let dept assign a different context id to each work. Signed-off-by: Byungchul Park --- kernel/workqueue.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index cf6203282737..5a6ab354c416 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -55,6 +55,7 @@ #include #include #include +#include #include "workqueue_internal.h" @@ -3155,6 +3156,8 @@ __acquires(&pool->lock) lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + dept_update_cxt(); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); From patchwork Tue May 13 10:06:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084987 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3567-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXSF66L3z1yXB for ; Tue, 13 May 2025 20:16:13 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 53AFB3BFBD0 for ; Tue, 13 May 2025 10:13:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 740FD257AEC; Tue, 13 May 2025 10:07:57 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D01C92512E0; Tue, 13 May 2025 10:07:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130876; cv=none; b=DKeE9Upt9bigr9DGlt+XwaBJdVvTTzDjkcDFhQ0vt+Zfa0ZyWkblQpNE5tRkFpcLA4faT94E6QkAK2SjlYxkZ1ETqz7EbqCZj03TZdolehvMUlZeoN4cSv/qWjCmfyVRT23bX9aJEXQgOXDwhKu5MjxX41Cwh2POagiEgYZglRw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130876; c=relaxed/simple; bh=4A9mulUq0DyLsLDdCQ4U5LwuKs+mN67d32ljZcRTlfc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=AdcUkDUt5EKHUA0S3W/IYgZiW3As06cAloaV/cL90WbLMwDxlB9b2wqe0ve0gsyIzxiRWQJw8yQoDnVLeyEDmcg6TYpRqZIehKmvI79TR8aKRXVtrGcBE6XDLtmTwCjlr+do1CM/rzjkoskbrmTeXvYKXbSA536ukfS7uE9hJk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-6c-682319eefd7d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 11/43] dept: add a mechanism to refill the internal memory pools on running out Date: Tue, 13 May 2025 19:06:58 +0900 Message-Id: <20250513100730.12664-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2xLYRjHve85PaetdTmpmYOJrQyZuMxleQi2+eJ8EYKEkKCxM21snbTb mERsdDJsNWIaw3Ql3bJ2Np3EtdQ2tblMWe3CNoyx2qWyaWOso3X58uSX/z/5Pc+HR0hIGwRT hEpVOq9WyVNklJgUD4QY53kmz1AsdOhDwfstj4QLVRYKnFfNCCzXczC4H66BVl8/gp/PnhOg L3IiKH3fScB1RxcCW/lhCpo/hoLL66GgsegEBUcuV1Hwom8UQ8fZ0xjM1rXw1vSJhCeFRgx6 NwXn9UdwYPRiGDFV0GDKjobu8mIaRt/HQmNXiwBsr+fCuZIOCu7aGklw3OzG0Hz7AgVdll8C eOJoIMGnmwrOUwUCqBw0UtDnMxFg8npoeGk3YHAYwqFaGxAeHR4TwKMCO4ajV65hcLXfQXAv 7x0Gq6WFgjpvP4YaaxEBP8oeIujWDdCQmz9Cw/kcHYITuWdJ0HYshZ/fA5svfouFnEvVJFT6 W1DCSs5SYkFcXb+H4LQ1+7gf3lcUZ/MZSO6xkeVuFXfSnPbea5ozWDO4mvIY7vJdN+ZKh7wC zlpxjOKsQ6dp7viAC3ODTU30+oit4hVJfIoyk1cvWLVTrGh/PoD3vkjYX/VFi7NR5eLjSCRk mSWss8kg+M+mc6Y/TDGz2ba2ESLIYUwkW1PwKZCLhQTTMp5tvdiOgsUEZhebXVBMB5lkotmv H404yBImjh08NUz/lU5nzdX2PyJRIPeXNZFBljJL2UKDmQxKWeakiO3zm/9dMZl9UN5GFiKJ AY2rQFKlKjNVrkxZMl+RpVLun78rLdWKAv9lOji67SYacm6sRYwQyUIkDe4ohVQgz9RkpdYi VkjIwiQ5NwKRJEmedYBXp+1QZ6Twmlo0VUjKJkkW+fYlSZnd8nR+D8/v5dX/WywUTclGOrWn zJbvdz2a1uzsQZvotC3inolZ450feutz8+zpnsfudRH5syJvf8ZjCcmhvOtNcufM6J772+f0 RrzZvnn508T4sZHwacsSb+i7vDs3x8WrQuo0G+KHv8qGEp1kZnHGXOme8Kv2wtb60isl0rgv q39ZxkVFRYp6/GcW5B3SzYmSkRqFPDaGUGvkvwHIBvbIWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzH+36fX3fH5XEaD2Y4a4yJQ3zM75n1nY3xD+YfPfJwl37YHZGN 3bnL/KhWJpHSiV3tupS7RiFrteK0Uqojq1MxRF1GF6Vwx/zz2Wuv9/Z+//ORUSo7M0OmSzgq 6RPEODWroBXb15gX+6bP0y5tH18G/qFzNOSWOlhovlOMwFFuwtBXFwUvh/sR/Gx8TkF2VjOC mz1dFJTXexFUFZ1hofVdKLT5B1lwZ11kwXyrlIWWz2MYOq9cwlDs3AZvbO9paMgowJDdx8L1 bDMOnI8YRmx2DmzGcOgtyuFgrEcDbq+Hgdo8NwNVrxfBtRudLDyqctNQX9GLofVBLgtex28G Guqf0jCcPhOaM9MYKPEVsPB52EaBzT/IwYtqK4Z661QoswRaz377xcCTtGoMZ2/fxdDW8RDB 43PdGJwODwu1/n4MLmcWBaOFdQh60wc4SEkd4eC6KR3BxZQrNFg6I+Hnj8By3pAGTPllNJSM e9DG9cRxw4FIbf8gRSyu42TU386SqmErTZ4VCKQyp4sjlsevOWJ1HiOuooXk1qM+TG5+9TPE aT/PEufXSxy5MNCGia+pidsxa69i7QEpTpck6Zesj1ZoO54P4CMtG0+UfrJgIypZfgHJZQK/ QrBdszFBZvn5wqtXI1SQw/g5givtfcArZBTvmSC8zOtAwWAKHyMY03K4INN8uPDlXQEOspJf Kfgyv3H/SmcLxWXVf4vkAT9e2EQHWcVHChnWYjoDKawoxI7CdAlJ8aIuLjLCcFibnKA7ERGT GO9EgQ+ynRrLrEBDrVE1iJch9UTl0765WhUjJhmS42uQIKPUYUrT/YBSHhCTT0r6xH36Y3GS oQbNlNHqacqtu6VoFX9IPCodlqQjkv5/imXyGUZ0ryLGlyyqyvPS6yrxh8mX2yRXDlEYjB89 saMKS7Ry2sGyVfLUrh/duT3fo9amxu73hsu9YxGVhRacb859u82Vstrd1L6OD72KY0/pTY12 a/7Ojilyc8jmzE2NhV8iN+zfo5m0q7Gz+0xq6O0dKa0Np30LQloSt+gqH2iUS8RlatqgFTUL Kb1B/AN/tU/BPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: dept engine works in a constrained environment. For example, dept cannot make use of dynamic allocation e.g. kmalloc(). So dept has been using static pools to keep memory chunks dept uses. However, dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack, using irq work and workqueue that fits on the contrained environment. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 108 +++++++++++++++++++++++++----- kernel/dependency/dept_internal.h | 19 ++++-- kernel/dependency/dept_object.h | 10 +-- kernel/dependency/dept_proc.c | 8 +-- 4 files changed, 116 insertions(+), 29 deletions(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a936a8d831c5..fc1d9e8b28f9 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,9 @@ #include #include #include +#include +#include +#include #include "dept_internal.h" static int dept_stop; @@ -142,9 +145,11 @@ static inline struct dept_task *dept_task(void) } \ }) -#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* * DEPT internal engine should be careful in using outside functions @@ -267,6 +272,7 @@ static bool valid_key(struct dept_key *k) #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ +static struct dept_##id rpool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT @@ -275,14 +281,74 @@ struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ - .obj_nr = ATOMIC_INIT(nr), \ + .obj_nr = nr, \ + .tot_nr = nr, \ + .acc_sz = ATOMIC_INIT(sizeof(spool_##id) + sizeof(rpool_##id)), \ .node_off = offsetof(struct dept_##id, pool_node), \ .spool = spool_##id, \ + .rpool = rpool_##id, \ .lpool = &lpool_##id, }, #include "dept_object.h" #undef OBJECT }; +static void dept_wq_work_fn(struct work_struct *work) +{ + int i; + + for (i = 0; i < OBJECT_NR; i++) { + struct dept_pool *p = dept_pool + i; + int sz = p->tot_nr * p->obj_sz; + void *rpool; + bool need; + + local_irq_disable(); + arch_spin_lock(&dept_pool_spin); + need = !p->rpool; + arch_spin_unlock(&dept_pool_spin); + local_irq_enable(); + + if (!need) + continue; + + rpool = vmalloc(sz); + + if (!rpool) { + DEPT_STOP("Failed to extend internal resources.\n"); + break; + } + + local_irq_disable(); + arch_spin_lock(&dept_pool_spin); + if (!p->rpool) { + p->rpool = rpool; + rpool = NULL; + atomic_add(sz, &p->acc_sz); + } + arch_spin_unlock(&dept_pool_spin); + local_irq_enable(); + + if (rpool) + vfree(rpool); + else + DEPT_INFO("Dept object(%s) just got refilled successfully.\n", p->name); + } +} + +static DECLARE_WORK(dept_wq_work, dept_wq_work_fn); + +static void dept_irq_work_fn(struct irq_work *w) +{ + schedule_work(&dept_wq_work); +} + +static DEFINE_IRQ_WORK(dept_irq_work, dept_irq_work_fn); + +static void request_rpool_refill(void) +{ + irq_work_queue(&dept_irq_work); +} + /* * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is * enabled or not because NMI and other contexts in the same CPU never @@ -318,19 +384,31 @@ static void *from_pool(enum object_t t) /* * Try static pool. */ - if (atomic_read(&p->obj_nr) > 0) { - int idx = atomic_dec_return(&p->obj_nr); + arch_spin_lock(&dept_pool_spin); + + if (!p->obj_nr) { + p->spool = p->rpool; + p->obj_nr = p->rpool ? p->tot_nr : 0; + p->rpool = NULL; + request_rpool_refill(); + } + + if (p->obj_nr) { + void *ret; + + p->obj_nr--; + ret = p->spool + (p->obj_nr * p->obj_sz); + arch_spin_unlock(&dept_pool_spin); - if (idx >= 0) - return p->spool + (idx * p->obj_sz); + return ret; } + arch_spin_unlock(&dept_pool_spin); - DEPT_INFO_ONCE("---------------------------------------------\n" - " Some of Dept internal resources are run out.\n" - " Dept might still work if the resources get freed.\n" - " However, the chances are Dept will suffer from\n" - " the lack from now. Needs to extend the internal\n" - " resource pools. Ask max.byungchul.park@gmail.com\n"); + DEPT_INFO("------------------------------------------\n" + " Dept object(%s) is run out.\n" + " Dept is trying to refill the object.\n" + " Nevertheless, if it fails, Dept will stop.\n", + p->name); return NULL; } @@ -2957,8 +3035,8 @@ void __init dept_init(void) pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); #define OBJECT(id, nr) \ - pr_info("... memory used by %s: %zu KB\n", \ - #id, B2KB(sizeof(struct dept_##id) * nr)); + pr_info("... memory initially used by %s: %zu KB\n", \ + #id, B2KB(sizeof(spool_##id) + sizeof(rpool_##id))); #include "dept_object.h" #undef OBJECT #define HASH(id, bits) \ @@ -2966,6 +3044,6 @@ void __init dept_init(void) #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); #include "dept_hash.h" #undef HASH - pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... total memory initially used by objects and hashs: %zu KB\n", B2KB(mem_total)); pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_internal.h index 187a9b21f744..80a7b90df37b 100644 --- a/kernel/dependency/dept_internal.h +++ b/kernel/dependency/dept_internal.h @@ -22,9 +22,19 @@ struct dept_pool { size_t obj_sz; /* - * the number of the static array + * the remaining number of the object in spool */ - atomic_t obj_nr; + int obj_nr; + + /* + * the number of the object in spool + */ + int tot_nr; + + /* + * accumulated amount of memory used by the object in byte + */ + atomic_t acc_sz; /* * offset of ->pool_node @@ -34,9 +44,10 @@ struct dept_pool { /* * pointer to the pool */ - void *spool; + void *spool; /* static pool */ + void *rpool; /* reserved pool */ struct llist_head boot_pool; - struct llist_head __percpu *lpool; + struct llist_head __percpu *lpool; /* local pool */ }; enum object_t { diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h index 0b7eb16fe9fb..4f936adfa8ee 100644 --- a/kernel/dependency/dept_object.h +++ b/kernel/dependency/dept_object.h @@ -6,8 +6,8 @@ * nr: # of the object that should be kept in the pool. */ -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT(wait, 1024 * 32) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT(wait, 1024 * 4 * 4) diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c index 7d61dfbc5865..f07a512b203f 100644 --- a/kernel/dependency/dept_proc.c +++ b/kernel/dependency/dept_proc.c @@ -73,12 +73,10 @@ static int dept_stats_show(struct seq_file *m, void *v) { int r; - seq_puts(m, "Availability in the static pools:\n\n"); + seq_puts(m, "Accumulated amount of memory used by pools:\n\n"); #define OBJECT(id, nr) \ - r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r = 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r = atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024); #include "dept_object.h" #undef OBJECT From patchwork Tue May 13 10:06:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084994 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3571-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXTP5VYMz1yXB for ; Tue, 13 May 2025 20:17:13 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B5B843A8C74 for ; Tue, 13 May 2025 10:14:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 40FE7267F43; Tue, 13 May 2025 10:07:59 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4E59025228E; Tue, 13 May 2025 10:07:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130878; cv=none; b=tyOYIt5NFtMzLbwwCWH1jlkqv+vwOXuhwv130gxzPz5Ofa0Yhz1EZIkPjJM2TrG+lDm1BMiIazl656N9m6a5iF4BsF3UDVou+WZDqzkYEyiTdS4APBXY+ukbCoDMYEZ+b/rDPynHW4I87uBHtUjkqbon8K+nHpoGUmu1RKQ/9/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130878; c=relaxed/simple; bh=Tt5fY0uAKrgzU+Mw+IpqvXF4Smt9wx8k6PcTCp/EFPY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jKcP6wVIRY1eMOG/PKALfDjN0jPrwZJazc6X2iU98HlkIa+uhmtSk/Y8w3LNN3/Xx/2sg3IcwoDEPotuVyyj1+FWt8pEZt6LC7oTuHl6Lr5rFWSRm1+reGHK1ci1YB/0QO79dVtT0iGO27RRiSK5C24P5vwq/t6OlvJTLyvnAWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-7d-682319ef0224 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 12/43] dept: record the latest one out of consecutive waits of the same class Date: Tue, 13 May 2025 19:06:59 +0900 Message-Id: <20250513100730.12664-13-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTdxTG/b/3VmvedERfwIxZQ4wsOCtizvCSxX3Yf94TEzMhuHXjdW0G BYuAmJCAAoNyGbIAys2CriAt0hWWoROHRYuMDEFKRYcobBoJlyaFVlGYthK/nPzynOc8z5fD kXI7HcRptCdEnVYVr2CklHR6RV24K3CdelP+7CrwzOVRUN1iZqD/igmBuS2LgInbX8B97xSC 13/fJaGirB9B3dgjEtrsowg6Gk8zMPjfSnB4XAz0lBUwcOZiCwMDkwsEjJSXEmCy7oPHxmcU 9JbUE1AxwUBVxRnCN54TMG9sYsGYGQrjjZUsLIwpoWfUSUPHw4/hfO0IA9c7eiiwt48TMHit moFR8xsaeu13KPAWB0P/2SIammfqGZj0Gkkwelws3Os0EGA3rAJLti8wd/Z/GrqLOgnIvfQr AY4HfyC4kfeEAKvZyUCXZ4qAVmsZCa8abiMYL55mIadwnoWqrGIEBTnlFGSPRMLrl77mmjkl ZF2wUNC86ESf7cDmWjPCXVMuEme3puFXniEGd3gNFP6rXsBXKx+xOPvGQxYbrCm4tTEMX7w+ QeA6t4fG1qZ8BlvdpSzWTzsIPNPXxx5cEy3dHifGa1JF3Sc7v5GqHS0vqKSny08aSg10Jrol 0SMJJ/BbhK7CXlaPuHfc5Irxywy/Xhgenif9HMB/JLQWPaP1SMqRvHO5cL/mAfL7P+BVwmR5 gt9D8aFC6dlO1s8yfqsw731CLsWHCCZL5zuW+PTFhj7Kz3I+UigxmCh/psBXSISSXyzM0kGg cLNxmCpBMgNa1oTkGm1qgkoTv2WjOl2rObnxu8QEK/J9lzFjIaYdufsP2RDPIcUK2Z2JtWo5 rUpNTk+wIYEjFQGyrN99kixOlX5K1CV+rUuJF5NtKJijFKtlm71pcXL+e9UJ8QdRTBJ177cE JwnKRFzwalssjtj2U9Wu8L603Nh/a2sy8ocirT2zx5ybo45c/q078elYQ8g1xzm9Z++6w5Vh OfuVtkD38bWCfmGg200OxFTu0XS9nEkp/HP77ox/juY2B0XcNR346tPFxGPfbogWf771uUW7 Lapg64fBX1aPJT1ec2Su7SqvbJOdz/8xPK8gQkElq1XKMFKXrHoLkMqe5lkDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/fXY5HGg+t4Sy2rJKpfbZ+zH89szEzZssfeqaHu1WHO1KN KZUfUauo0A9XcdrdqdxFibN0ipiTStEqFTWtH2d0j37izuafz157f97vz/ufD4N7VZFrGZX6 pKhRC/EKSkbIdoelB0yt2ajcWjESDs7pSwSU1JgoaK82IjDVpWEw1hIFPdIEgvm373AoKmhH UD7Uj0Nd6wACa9V5Cjq/LoMup4OCtoIrFKRX1lDwfnwBg77CfAyM5l3wWT9KwJvcCgyKxigo LkrHXOMbBrN6Aw36VD8YrrpFw8JQMLQNdJNgK20jwdq7BW6W9VHw1NpGQGvDMAadjSUUDJj+ kPCm9RUBUo4PtOdlk3B/qoKCcUmPg97poKGjSYdBq24V1Ga4rl74+ZuEl9lNGFy48wCDrk9P EDy7NIiB2dRNgc05gYHFXIDD3L0WBMM5kzRkXp2loTgtB8GVzEICMvpCYH7G1Vw6HQxpt2sJ uL/YjXZE8qYyE+JtEw6cz7Cc5uecHyjeKukI/nUFxz++1U/zGc96aV5nPsVbqvz5yqdjGF/+ w0nyZsNlijf/yKf5rMkujJ+y2+k9vtGy8FgxXpUoaoIiY2TKrppfxPERzyRdvo5MRS88shDD cOx2zuA4mIU8GIrdzH38OIu72Ztdz1myR8ksJGNwttuT6yn9hNz+lazAjRcmuD0E68fl5zXR bpazodysNPgvy7HrOGNt0z/2cOmL9+yEm73YEC5XZyRykUyHlhiQt0qdmCCo4kMCtXHKZLUq KfDwsQQzcv2P/uxCXgOa7oxqRiyDFEvlr8Y2KL1IIVGbnNCMOAZXeMvT6l2SPFZIThE1xw5p TsWL2mbkwxCK1fKdB8QYL/aocFKME8Xjoub/FmM81qYiQ4qU97qeupbZ7BMQfuSxtOLMw/69 P4uNOfqgM8v7cYctImizkSDv+l6XXm6I3rutJ/NLh7Av4ITd8J69GLNPG2Zv2LG43DfsyZBn UuPz6pLJ9MCoVXEfyizPt288MRKxOtSYYpuZk4YGz+0vuFFvNfSov2/q2PJoYKGypbyvt2aJ gtAqhWB/XKMV/gL5TXZiOwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The current code records all the waits for later use to track relation between waits and events within each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class. Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one when the same class'ed waits consecutively appear. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index fc1d9e8b28f9..211bafffc980 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1485,9 +1485,28 @@ static struct dept_wait_hist *new_hist(void) return wh; } +static struct dept_wait_hist *last_hist(void) +{ + int pos_n = hist_pos_next(); + struct dept_wait_hist *wh_n = hist(pos_n); + + /* + * This is the first try. + */ + if (!pos_n && !wh_n->wait) + return NULL; + + return hist(pos_n + DEPT_MAX_WAIT_HIST - 1); +} + static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) { - struct dept_wait_hist *wh = new_hist(); + struct dept_wait_hist *wh; + + wh = last_hist(); + + if (!wh || wh->wait->class != w->class || wh->ctxt_id != ctxt_id) + wh = new_hist(); if (likely(wh->wait)) put_wait(wh->wait); From patchwork Tue May 13 10:07:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084970 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3568-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXPn2XMYz1yPv for ; Tue, 13 May 2025 20:14:05 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D03471B45C75 for ; Tue, 13 May 2025 10:13:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A461B2566D3; Tue, 13 May 2025 10:07:57 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6B5662522B2; Tue, 13 May 2025 10:07:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; cv=none; b=aDAp/57gUKvGRdim4WeyhWHa5JipZFo4poSlCjAVETVD+5tiNUO64uKkUnzVJHNMPeAu9HsftPWZU6Hog/sXJEaPJH3kNJpODiIQLjPZtoMrfr2IcAqudciOV4cBCup991TBOD+lnGCN6meWk76XHybz7tkXbwRIJ5cDmAgQpWo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; c=relaxed/simple; bh=mVczWw5AJvTGaUa2k+kGuAdITLqzYAqLiO2CNQ+CNGE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=EhJSowfW9Q7Im0Xbub+cVqTdtO3XfUsGL3vOYSv6bLjFuTZsLrnwNffYeiA8yymTRLX9gx96rwwyMqzOnP0fProm/jPRy0afBFG+Z8Mm517D1jnwqBSlMT73QbxgJelKBpGv8cp3raP+3rTgB8u/MHM6SlUJOo3jeoUTKhm6KAk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-8c-682319ef1729 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 13/43] dept: apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Date: Tue, 13 May 2025 19:07:00 +0900 Message-Id: <20250513100730.12664-14-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUxTaRTH/b67tkPJtSBz3aJTYkwgLhCdHBPH6IPxPmjU+OaYaCPXaR1A LViExIQCEgcFEQVcQAtqrbQMnbaJjtIGyrAJYpVFVEApE0MHsIq2Y1nUVuLLyS+//8n/vByW kLdSi1h1arqoSVUmK2gpKZ2IqF7lWxirWvugZin4P54moaLOTIP7TxMCs12Hwdu8DZ4FxhFM P3pMQHmpG0HV8CAB9pYhBA5jDg3d/0ZCj99HQ3vpGRpyb9TR8GRsBsNAWQkGk3UHvDK8IaGj uBpDuZeGq+W5ODRGMQQNNQwYsleAx3iFgZnhBGgf6qPA8SIeLl8boKHe0U5Cyz0Phu77FTQM mb9Q0NHSRkKgaDG4zxdSUPu2moaxgIEAg9/HwNMGPYYWfQxY8kKF+R8+U9Ba2IAh/+ZfGHqe P0DgPP0ag9XcR0OTfxyDzVpKwNTtZgSeogkGTp0NMnBVV4TgzKkyEvIG1sP0p9Dlyo8JoLtu IaF2tg9t/kUwXzMjoWncRwh5tgxhyt9LC46AnhQeVvPC31cGGSHP+YIR9Nbjgs0YJ9yo92Kh atJPCdaaP2jBOlnCCAUTPVh429XF7FqyV7oxSUxWa0XNmk0HpKppVwE+6ph/YrTXjbKRLbIA SVieW8fPDjbT37muuxOHmeZW8v39QSLM0dxy3lb4hipAUpbg+n7gn1U+R+EgilPxOleQCjPJ reA7s///5mXcz/yrW5+pudJlvMnS8K1IEvKzt7vIMMu59Xyx3kTO7ZRLePudiDleyDca+8li JNOjeTVIrk7VpijVyetWqzJT1SdWHzySYkWh9zKcnPn1Hpp073EhjkWKCFmb9yeVnFJq0zJT XIhnCUW0THc3pGRJyswsUXNkv+Z4spjmQotZUvGjLDGQkSTnflOmi7+L4lFR8z3FrGRRNjq3 cuJQ7zThJEbRlljJtovR/9TvWGaO3v5uY3pZpGf4U0VMTFPQaI/cp21c4O7wZXg4Jh857fLE qJdbuw7+1xln0OuzanOkWs31L5dyRpipSq+rZPeGiCd31jRaH12wECWHx95XmdbGjlxMjKfY 7p2DufYDd4ecWcfGRyy9jgUKMk2lTIgjNGnKrw3Z7yFaAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUiTaxjHu+/n1dXsycSetKwW0Ru9GGdxQYc6Hw50ExVFQdGHctRTG+mM rUwLO1u6MHVigUm+1LSapjN1E7NyIZPM9WLWnGWZlpQovkE5a2nrbB3Ol4sfvz/8/18unoqo ZKJ5jfakpNOqEhSsjJbt3JS+Znz+UvV6bx8DvolMGoprbCx03KlCYKs3Yhh6tBVeT44gmHr+ goKC/A4EpR/fU1Df2ovAWXGeBc+ncOj0jbPgzs9mIf1GDQsvh6cx9Fy5jKHKvgP6rAM0PM0r w1AwxEJRQToOnkEMfmslB1bDMuivKORg+mMcuHu7GGgpcTPgfLsarl7rYaHJ6aahtbEfg+d+ MQu9tl8MPG1to2EyNwY6LpkZqB4rY2F40kqB1TfOwatmC4ZWSxTUZgRbL3wNMPDY3Izhws06 DJ3dDxA8zPyAwW7rYqHFN4LBYc+n4Ef5IwT9uaMcmHL8HBQZcxFkm67QkNGjhKnvweWSiTgw Xq+lofpnF/prM7FdsyHSMjJOkQzHafLD52WJc9JCkydlIrlX+J4jGQ/fcsRiP0UcFavIjaYh TEq/+Bhir7zIEvuXyxzJGu3EZKy9ndu18IDszyNSgiZZ0q3bHC9TT7my8AnnnJRBbwcyIEd4 FgrjReEPscbzDIeYFZaLb974qRBHCotFh3mAyUIynhK6ZoqvS7pRKJgrqEWjy8+EmBaWic8M 3357ubBR7LsVYP4rXSRW1Tb/LgoL+p/l7XSIIwSlmGepovOQzIJmVKJIjTY5UaVJUK7VH1en ajUpaw8nJdpR8IOsadOXGtGEZ6sLCTxSzJK3DS1RRzCqZH1qoguJPKWIlBvvBpX8iCr1jKRL OqQ7lSDpXSiGpxXz5Nv2SfERwjHVSem4JJ2QdP+nmA+LNqCyBf7uxsPNB9sG238Nh11VGgsM 0VHaOsrkUULMudjBFdmHineWxzxJ227e85lPy4xdeDfnXVQ85TYlbXcH+AYdPtdUXHQndtQ7 dn+NN67001mT7+I/O3JI0krHlin3frah+nqfVBd+OzCwF5PZKUfXB5o2vMtd7jIbvH+/2l2o oPVqVdwqSqdX/QsZd9PJPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Make dept able to track dependencies by wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index fb2915676574..bd2c207481d6 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -10,6 +10,7 @@ */ #include +#include /* * struct completion - structure used to maintain state for a "completion" @@ -26,14 +27,33 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map dmap; }; +#define init_completion(x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* + * XXX: No use cases for now. Fill the body when needed. + */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) {} -static inline void complete_release(struct completion *x) {} + +static inline void complete_acquire(struct completion *x) +{ + sdt_might_sleep_start(&x->dmap); +} + +static inline void complete_release(struct completion *x) +{ + sdt_might_sleep_end(); +} #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap = DEPT_MAP_INITIALIZER(work, NULL), } #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -75,13 +95,13 @@ static inline void complete_release(struct completion *x) {} #endif /** - * init_completion - Initialize a dynamically allocated completion + * __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion(struct completion *x) +static inline void __init_completion(struct completion *x) { x->done = 0; init_swait_queue_head(&x->wait); From patchwork Tue May 13 10:07:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084991 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3570-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXSn2qWDz1yYx for ; Tue, 13 May 2025 20:16:41 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E48828637CD for ; Tue, 13 May 2025 10:13:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7068725D534; Tue, 13 May 2025 10:07:58 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 99D5E24DFE6; Tue, 13 May 2025 10:07:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; cv=none; b=bWNgyxfjw4pzP9h4ncZV8XH3bfKBdMTVWBQ0Sao7/mmXG4huINZiQgjADuWPOb1wbAoc+i8U/pjEA84+l85MLliA7XXcoHBDicSs98xRmEFQYVs28FL+Am+MGisfEsZRxFJxvsea/WEEmytOvWwK/ZzJxsaNR9XNFhl6P+IHjAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; c=relaxed/simple; bh=pLUSXjgVOyB6ewzDMD2B/aTj2n2b3ii/EXLHyurcTzs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=aEKg47bPDRHEMBbuaYoAdIHdFuU0ovOLzuxc94C11Ij4kl/laJnJc/dnsFVHj49c2kK8X5Amr19IKJYEgSJc/+9TzFLSYhPr5TRZ+/DduscMpCUSU7H/dKd9eRYL8xBi5+t1h6dKLizF5FNmRZN0oIjQg2c/O/VQyKs5njGorrQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-9c-682319ef5bc7 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 14/43] dept: apply sdt_might_sleep_{start,end}() to swait Date: Tue, 13 May 2025 19:07:01 +0900 Message-Id: <20250513100730.12664-15-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX8P39+v4/g5Lb88TK41EyJP+zBPY+OHMdOYeRg395u7dZVd KjW2jvIQPXg4TZTrypU6ymVDLlK6VEopidVRLG5KxJ1SHu6Yfz577f35fF6ffz4sKauiJ7Dq 8AOiNlyhkWMJJekdZZzV5+uvmmOv9gPntxMUXC4yY2i8UYjAfEtHgKNqDbxw9SAYqn9KQrq+ EUF2ZwcJt2x2BGX5RzA0vxsNLc4+DDX6UxiO5hRhaPo4TED7hbMEFFo2wGtTNwV1aUYC0h0Y LqUfJdzlAwGDpgIGTPEB0JWfwcBwZzDU2FtpKHs1Ay5mtWOwltVQYLvTRUBz6WUMdvNvGups jylwpUyExjPJNFz/ZMTw0WUiweTsY+BZuYEAm8EHihPcwmNff9FQnVxOwLHcmwS0vLyH4P6J NwRYzK0YKp09BJRY9CT8yKtC0JXSy0Di6UEGLulSEJxKvEBBQvsCGBpwX878Fgy6K8UUXP/Z ilYsFcxZZiRU9vSRQkJJjPDD+RwLZS4DJdQaeeFuRgcjJNx/xQgGS5RQkh8o5FgdhJDd76QF S8FJLFj6zzJCUm8LIXxqaGA2TdouWaIUNepoUTt72R6Jqr/zOb2/ij3YdrWUiUcNOAmxLM/N 581GvyTk9Re/5BXRHsbcNL6tbZD0sDfnx5ckd7tzCUtyrSP5F5kvkWd3HLeBz6rb50GKC+Cv HgHPuJRbyDdV96B/yil8YXH5X42XO/+Z10B5WMYt4NMMhZRHyXPnvPi3uan0vwVf/mF+G5WG pAY0ogDJ1OHRYQq1Zn6QKjZcfTBob0SYBbmfy3R4eMcd1N8YUoE4FslHSR87pqpktCI6Mjas AvEsKfeW6m67I6lSERsnaiN2a6M0YmQFmshS8vHSua4YpYzbpzgghoriflH7v0uwXhPiUdwW +6OdnH/iuvdrH1Z0L7q2efJT7F23LDQkcCBTb5WPvbJeiWqzlB0Ztc9US3xDqCeWjQPT9m6a XanTpJYuDZ7y/YNkSK9XkSN9DuUWnZyXnX7ejx/zeVuTNS4oB6+a7nP7s23rakfBqmH/lbsW xhhmpi4vj/31oF4NUQFdh6yLj8upSJUiOJDURir+ABpoeWlYAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX8P39+vc2e/JfyI4cxsTLSJzzy0Nhs/kvGXeZo7+q07rsMd kc30cCylhCVPnRNdqcvlzibpcruU0qRUJ7f0NENcmnRH5OGO+eez117vz96ffz4sGVpCT2PV 2kOiTqvUyLGEkmxckb5waOoc1WLPBSn4RjIouGa1YGi5U4bAci+VgIG6tfDS70Xw49lzEvLz WhDc6HtNwr36bgSOkjQMbW8mQLtvCENjXhaG9JtWDK0fxwjouniegDJbHPSY31LQlFtIQP4A hqv56URgvCdg1FzKgDllLvSXXGFgrC8SGrvdNNQWNNLg8CyAy8YuDNWORgrqK/sJaKu6hqHb 8puGpvoGCvw54dByLpuG8k+FGD76zSSYfUMMvHCaCKg3TYYKQ6D11JdfNDzJdhJw6tZdAtpf PURQk9FLgM3ixlDr8xJgt+WR8L24DkF/ziADJ8+MMnA1NQdB1smLFBi6ouDHt8DlgpFISL1e QUH5TzeKiRYsRgsSar1DpGCwHxG++zqw4PCbKOFpIS88uPKaEQw1HkYw2Q4L9pL5ws3qAUK4 MeyjBVvpaSzYhs8zQuZgOyF8am5mNs3YJlkZL2rUSaJuUbRCohru66AP1LFHO4uqmBTUjDNR CMtzS/jPxVY6yJibx3d2jpJBDuNm8fbstwEvYUnOPZ5/WfAKZSKWncjF8camhCBS3Fy+KA2C 6zJuKd/6xIv+Vc7kyyqcf2tCAv5ncTMV5FAuis81lVG5SGJC40pRmFqblKhUa6Ii9PtUyVr1 0Yg9+xNtKPA/5uNj5yrRSNtaF+JYJJfKGgZmq0JpZZI+OdGFeJaUh8lS7weULF6ZfEzU7d+l O6wR9S4UzlLyKbL1W0RFKJegPCTuE8UDou5/SrAh01JQzSpFe/huXYbz0YQW1+r4rfkHz8g8 pGyl9HeVQbFsDdYqLjXEvstLk9r81qLexfqdk25XfN2YEBMR6z6b6bnT4Ynd/jgqYfPzZ8Zf nk29neXuMFdPT/SGvuQZrZVuLFu+V9M2vTDrxOA6e9dMb/Wlmjj9h5gjzu5vO6TpDuvUfqOc 0quUkfNJnV75B8OURQw7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Make dept able to track dependencies by swaits. Signed-off-by: Byungchul Park --- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/swait.h b/include/linux/swait.h index d324419482a0..277ac74f61c3 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -6,6 +6,7 @@ #include #include #include +#include #include /* @@ -161,6 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ + sdt_might_sleep_start(NULL); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ @@ -176,6 +178,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); cmd; \ } \ finish_swait(&wq, &__wait); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue May 13 10:07:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084976 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3569-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXQ95Wqfz1yPv for ; Tue, 13 May 2025 20:14:25 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id BECF44A50DD for ; Tue, 13 May 2025 10:13:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 20C9825C6E3; Tue, 13 May 2025 10:07:58 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BFFFE221282; Tue, 13 May 2025 10:07:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; cv=none; b=IsIgIshJMUgOVWxFU0F73Eeva76Iul7yYMoScVyxxn6Oo6Btf1P2yY514fxVguyw0qi45NpBh9yir0yPLamE6rtPmru2C2ZXLWtNsjE9vjCDopeG5/mE3RWjld/FWl8bmHfCOG78aBybDtgAsciYLCDU2kBzjhb9k6VzLD+xh0s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130877; c=relaxed/simple; bh=l20SQrf5lRl3+sKTvdmaOt5bA5DWEDLDzGURyc0sECY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=CaWYhLbYY4gEy+IQHxoYHUHBEw75hVXkRSf8tGT/6karurJNhopR9ujFU62sDdUPC7V17hglIOWeV/dMI/PzMJ5XPCCQIxtEJkJH3CVvmQ0i0pCJ1z9fBNTqvsmZMfGG+3Gl99lneBgaPan6W1UkSe2AduuGJrKGOovd3XHPFe8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-ac-682319ef66b8 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 15/43] dept: apply sdt_might_sleep_{start,end}() to waitqueue wait Date: Tue, 13 May 2025 19:07:02 +0900 Message-Id: <20250513100730.12664-16-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTdxTG/f/vayt3u+l8uVYWsMaZsPiCQXfi1Czbh93EEJbMKNkStxt7 YxsLmlarmJDBKMahdMyENlqBUkxt2oqskMxNSyqEChIBASsqoBAVCUUSpM1QqLZufjn55TnP eZ4vhyVUYUrN6guPysZCyaChlaRyOq1+w8yqtbrNsd4NEJs7TcLFq34a+hp9CPwtpRgmO76F +/Eogjd3egmwV/chqB8bIaAlPIog6PmVhoGnH8FgbIaGruozNJQ1XKXh7tQChmHbOQy+QC48 dj8nobvKhcE+SYPDXoaT4wWGebeXAXfJOhj3XGBgYSwbukYjFAQffg7na4dpuBHsIiF8bRzD wD8XaRj1v6WgO9xJQty6Gvr+qKTgyksXDVNxNwHu2AwD/SEnhrBzBTRZkoGnXiUouFUZwnDq 0p8YBh9cR9B6+gmGgD9CQ3ssiqE5UE3A68sdCMat0wyUn51nwFFqRXCm3EaCZXgrvPk32Vwz lw2ldU0kXFmMoK92iv5aPxLbozOEaGk+Lr6O3aPFYNxJirddgvj3hRFGtLQ+ZERn4JjY7MkS G25MYrF+NkaJAe9vtBiYPceIFdODWHzZ08N8l/6DcodWNujNsnHTrp+VutYR7RGr4kQ03E+U oEqmAilYgc8RPKFx+gNPTLTjFNP8emFoaJ5I8TI+U2iufE5VICVL8JGlwv2aB6gCsewn/F5h ro1LeUh+nXC+2/rez/HbBNuY4//8DMHXFHqvK5L64uUeMsUqfqtQ5fSRqUyBr1MICXsc/3ew SrjpGSKrEOdES7xIpS80F0h6Q85GXVGh/sTGA4cLAij5Xe7ihR+vodm+79sQzyJNGtc5uUan oiSzqaigDQksoVnGlf6VlDitVHRSNh7+yXjMIJva0GqW1KzktsSPa1X8QemofEiWj8jGD1vM KtQl6GPvna+X7F/hy33ESp/lPcuzNHV7HftMy/fZvslzIXM4XJ1G1/ZPvTggnXXcTHe5O8q2 Le/JyMnibJ+u1K2nLNt71caE7tX87rUTzFjZwBM/VZ4/l6GOLO6PG/J/2ZHQtuw1Z976fZZr KE5/WhyNdn05lZm3JfRFom53eWPnpT0a0qSTsrMIo0l6B3m97mFZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0jTeRjH7/P9vemuL6b2vSSqVYRKmajHEx4VHOWH4OTun4oIauS3tppb bP7IvCOXM7p1GzPQUUtvWkxxq2wL++VMFFcmeda89QO1HGG3c+Zhm2TZjxndPw8vXs+b9/PP w5FJrfRSTqUpEXUahVrOSClpYX71uunvVik33JlMhlj0FAXnr7gZGLrsQuC+ZiAg3FcAj2cj CN4/+IsEW90QgqbxURKu+ccQ+FpPMBB4+S0Mx6YZ6K87zUD1hSsMPJycJ2Ck/gwBLs9P8Nw5 QcGAtZkAW5gBu62aiI9/CJhztrHgrFoDodZzLMyPZ0P/WJCG3oZ+GnzPMuFs4wgDnb5+Cvw3 QgQEbp1nYMz9iYYB/z0KZi1pMFRrpuHS62YGJmedJDhj0yw86nYQ4HekQrsx3nryzUca7pq7 CTh58SoBw09vI+g69YIAjzvIQG8sQoDXU0fCu5Y+BCHLFAs1f8yxYDdYEJyuqafAOJIH79/G LzdEs8HwZzsFlz4E0ZZN2N3oRrg3Mk1io7ccv4v9zWDfrIPC95sFfPPcKIuNXc9Y7PCUYm9r Br7QGSZw00yMxp623xnsmTnDYtPUMIFfDw6yPy/bLf2hSFSrykRd1qZ9UmXXaNERi+RoxP+I rEJm1oQknMDnCq9e9RILzPBrhSdP5sgFTuZXCF7zBG1CUo7kgwnC44anyIQ4bjG/Q4j2yBYy FL9GODtg+ZKX8d8L9eP2r53LBVd79xcvifsPLYPUAifxeYLV4aKsSOpA37ShZJWmrFihUuet 1x9WVmhUR9fv1xZ7UPx/nL/N195A0UBBD+I5JE+U3QuvVCbRijJ9RXEPEjhSniwzXI8rWZGi 4pio0+7VlapFfQ9K4yj5Etn2neK+JP6gokQ8LIpHRN3/W4KTLK1ClYcm7yQE88s7/j1QZd6V sjo/fVtfYcIuX7H8mH1xZeLN1JyOmbRM2KGxaT+m21v+S+mo25jpdVlXdP/SN56F99zfuMhe EGW2V89LDFsrw6sPmVIL1fsjU5utxhxb7UTBp5J1ow8PbF0Zyr3YmPLjcm1JZ5Y7EBgZ2rMk 8Ktaezwkp/RKRXYGqdMrPgMTrqkcOwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Make dept able to track dependencies by waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 965a19809c7e..aae161e500dc 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -7,6 +7,7 @@ #include #include #include +#include #include @@ -303,6 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ @@ -321,6 +323,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); break; \ } \ finish_wait(&wq_head, &__wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue May 13 10:07:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084985 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3572-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [147.75.80.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXRW6gg3z1yXB for ; Tue, 13 May 2025 20:15:35 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id A44D01B46D23 for ; Tue, 13 May 2025 10:15:25 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0334E26A0AB; Tue, 13 May 2025 10:08:01 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E98BA2472A2; Tue, 13 May 2025 10:07:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130880; cv=none; b=WQPMfvypT1oHmf+9mzH92COqC93R3Cob4wUOaeKFTyRBuDVApR2FdPgwWrBelaJ28VC2CPYUmQpzKocmjSbkOO1B72uB5N2YonA4yNjdwxcnLMXZlpii2Nw6D1dYYmcPHLLmX9n7BpbIxR4L0Hh0OXhcU680O0sbas7AHm0gQn0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130880; c=relaxed/simple; bh=+bd9WiorYKiALUTe/ZE9Twmt5uX3p97amk10CfbvR3w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=A+kW/oAnwgiO5a66VsNS0bR3miysfDLgrEyeP7kVYtLlwI0fdBdm2+AVc/1nvL5Hq/TXJT7OaXuEEoBN4201qNzIZuwUfoMkpdY7peVMmsaCxywqq7ryYYaapfO1b1x0ldddQx3tn7dVGJP2dgwmjtVTyDOyUfdKrYhuSv8CTgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-bc-682319efd1c9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 16/43] dept: apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Date: Tue, 13 May 2025 19:07:03 +0900 Message-Id: <20250513100730.12664-17-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbWxLYRTHPc+9vfe2W7mp4dokrCISYsxbTgTxzfVBQiyREKGxS8tWS0ut QqzMwmYzYkb3otusm62zapdg1plJy2ammL3Ziw2jsa1JaZltaL18Ofnlf05+53w4DCF7LIpk VOrDgkatSJBTElIyEl68xDt7vnJZmmcF+L+eJSG/2kKB+1YlAkuNAYPHuRE6AsMIxp89JyA3 x42gaKCXgBpXHwJH+SkKXr2fCm1+LwVNORkUnC6ppuDF5wkMPVcuYai0bYZ+8xAJT7OLMeR6 KMjLPY2D5ROGMXMFDeaUBTBYbqRhYiAWmvraReDoXgzXCnsoqHM0keC6O4jhVW0+BX2WXyJ4 6npCQiArCtwXM0VQNVpMweeAmQCz30vDywYTBpdpJlhTg8K0Lz9F8DizAUPajdsY2rruI6g/ +xaDzdJOwSP/MAa7LYeAH2VOBINZIzScOT9GQ54hC0HGmSskpPasgvHvwc0FX2PBcN1KQtVk O9qwjrcUWhD/aNhL8Kn2o/wP/2uKdwRMJN9czPH3jL00n1rfTfMm2xHeXr6IL6nzYL7I5xfx topzFG/zXaL59JE2zI+2ttJb5uyQrI0XElQ6QbN0/R6JsqblHErqZ5JL3NVECuqn0pGY4diV XH1zM/2fzfkdOMQUu5Dr7BwjQhzBzuPsmUOidCRhCLY9jOso6EKhxnR2F/cx51twiGFIdgFn rFsTiqXsaq529DL665zLVVob/njEwXyyrJUMsYxdxWWbKsmQk2PzxNwnb+m/I2ZzD8s7yWwk NaEpFUimUusSFaqElTFKvVqVHLP3UKINBd/LfGJi513kc29rRCyD5OHSJ55opUyk0Gn1iY2I Ywh5hNRwJxhJ4xX6Y4Lm0G7NkQRB24iiGFI+S7o8cDRexu5XHBYOCkKSoPnfxYw4MgUV+JbH dOOZb23fdq8wLcObwkY/tGie6QXdjO3c1QvjTm1/n6+77rnOaU802Pf3nlpYFmWMc4cZ4yI2 zXtXWhW9xeHYU3U72Vo6YFFPmV94IGMopWjxzshbJ2upaeFOe7TsuLWLQeM7urY+SDp4TO/p cbW8nvXxzU28Txx3efqkS05qlYrYRYRGq/gNiTI3HFoDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSaUxTaRSG/e5OnZorEuZaBpcaYoJRbLTkRI3RH4YvY9x+mWhUGr3SKqBt gQEnZsACCk4ZIFaUzQqmIBTBC4mo1BAQlMGpnQFBHRYlE5SwNKm0yubYavxz8uR5T97z53Bk 8G1awekSk0RDoiZeycgo2d6tpvXu5Wu0G689Cwfv9CUKSuvtDLju1CKwN2UQMNYRA/2+CQRz fz0nocjiQnDz7SAJTZ1DCBzVFxjo+W8J9HrdDHRZLjNgqqxn4O/xeQIGrhYSUCvtgWHbKAXd +RUEFI0xUFJkIvzjPQEzthoWbOkRMFJdzML8WxV0DfXR0F7WRYPj9Tq4Xj7AQIuji4LO5hEC eh6UMjBk/5+G7s6nFPjywsBVYKahbqqCgXGfjQSb183CP61WAjqtodCQ6W/N/vCZhifmVgKy b90loPfVQwSPLr0hQLL3MdDunSCgUbKQMFvVgWAkb5KFrN9nWCjJyENwOesqBZkDapj75L9c Nq2CjBsNFNQt9KEd27G93I5w+4SbxJmNv+BZ7wsGO3xWCv9ZIeD7xYMsznz0msVWKRk3Vkfi ypYxAt/0eGks1eQwWPIUsjh3spfAU04nuz/8kGzbCTFelyIaorbHyrRNz3LQ2WEutdJVT6aj YSYXBXECv1mwlfYTAWb4tcLLlzNkgEP4VUKjeZTORTKO5PsWC/1lr1AgWMYfEd5ZPvqXOI7i I4Tili0BLeejhQdTV9C3zpVCbUPr154gv1+oclIBDubVQr61lspHMitaVINCdIkpCRpdvHqD 8bQ2LVGXuuH4mQQJ+R/Idn6+oBlN98S0IZ5Dyh/kT8dWa4NpTYoxLaENCRypDJFn3PMr+QlN 2jnRcOaYITleNLahMI5S/ij/+aAYG8zHaZLE06J4VjR8TwkuSJGOemP36mN/XVOfE6XYvdNd MJv0+Ld9o58GTbtMSdez1Qqp3FO1Y1P/+cowZ2iJc8483nQy7l/T0j+OvtEYXO/xqfCIuslU H5+1UGg5YNHXqc3RoioibtnhA4rd637Sb5WSFSuizfootapDf3Em9JS+NCbV45mOjvFNpH92 bLrWvVxJGbUaVSRpMGq+AKEJfCU8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Make dept able to track dependencies by hashed-waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 9e29d79fc790..179a616ad245 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -6,6 +6,7 @@ * Linux wait-bit related types and methods: */ #include +#include struct wait_bit_key { unsigned long *flags; @@ -257,6 +258,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ @@ -274,6 +276,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); cmd; \ } \ finish_wait(__wq_head, &__wbq_entry.wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue May 13 10:07:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084990 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3573-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXSm3zJxz1yXB for ; Tue, 13 May 2025 20:16:40 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B732046561B for ; Tue, 13 May 2025 10:15:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 99BF726AA82; Tue, 13 May 2025 10:08:01 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 51FD7256C9E; Tue, 13 May 2025 10:07:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130881; cv=none; b=iX3evxRkt9Y39PpnYLxbJ9WP5zobbOTufYLSKHLc8n4ylnTHfrzCVoxCdNoLMjEEXsIZgEInZJEEl1snNK3NPMppVzJ/46z3kn8fFYZLHYUXgBLIe0aR1Dl4qd6oftBqYNl5rTlKVtVViXNGfs4FNHwCb2Jc+Tiw5rzbufbfIIE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130881; c=relaxed/simple; bh=9Sbtv7YuVMjEizoot53ogfQz7NbvVKio90qc+0mHg80=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=O5YHJLnij5Yyo8kNuskF8lR2OD0QaJsSsOmwwNbCU59tCEXSgGPsol3cMhX9g+YSw20Cr7Aa9u/Oh/E2ZlBdW/oLr1l0rm/MMjP0563onNSb/8UIw/XE7OU0LkW2Q39/OC1MsDJvKNQ19FThuE2ckpThNDLnSUz90gwOuFfiBRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-cc-682319ef77c9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 17/43] dept: apply sdt_might_sleep_{start,end}() to dma fence Date: Tue, 13 May 2025 19:07:04 +0900 Message-Id: <20250513100730.12664-18-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+59zds5xuTitsGMG5SKFIsvQeIOMoA+diMhQKAupUx7ayC2Z ZRlFM9dN81KgonmZs9baltlmN2+Z0sosW7WWhZc0kaSpYW1lLWsz+vLy8P5+PO+Xl8alj0Xz aYXqkKBW8akyUkyIR4Orl38JXSxfWb0KPN/OEVB+00KCo9aMwFKfhcHIo43w1utG8Ov5CxxK ihwIqgd6cai39yFoNp4i4fXQLHB6xknoKMolIbvmJgkvP/sw6Cm+hIHZugX6DcMEdBbqMSgZ IeFySTbmH58wmDSYKDBolsCgsYwC30A0dPS5RND8fhmUVvaQ0NTcQYD93iAGrxvKSeiz/BFB p/0JAd78MHBczBPBjTE9CZ+9BhwMnnEKXrXqMLDrQqBO6xee+Tolgsd5rRicuXILA+e7RgQt 5z5gYLW4SGj3uDGwWYtw+HntEYLB/FEKTl+YpOByVj6C3NPFBGh7YuHXD//lim/RkFVVR8CN 3y60Po6zVFoQ1+4exzmt7Qj30/OG5Jq9OoJ7qme5+2W9FKdteU9xOuthzmZcytU0jWBc9YRH xFlN50nOOnGJ4nJGnRg31tVFxS/YKV6bIqQqMgT1inV7xPLGs1fxtHLJ0U/1BZQG+cQ5KIhm mRg2t18j+s/OxgYswCQTyXZ3T+IBnsssYm15w/6OmMYZ10z2bcU7FAjmMNvY8q+vpksEs4R1 6AvIAEuY1ay7JR/9ky5kzXWt050g//73tS4iwFImli3UmYmAlGWqgtjsqn8ilgllHxq7iUIk 0aEZJiRVqDKUvCI1JkqeqVIcjdp3UGlF/vcynPDtuocmHAltiKGRLFjyZCRcLhXxGemZyjbE 0rhsriTrrn8lSeEzjwnqg7vVh1OF9DYURhOyeZJV3iMpUmY/f0g4IAhpgvp/itFB8zXIUtGw ac1eVeSth+oY2CRxRWlMtbNzooI3xFpX8NeTOhNoe6Jla/hysnBIkeFWJh2fKulMjltrmmgv 3s4MlEYMJ37faJh1J2XHx+SwxMEtm5/t8ginQioi4sac8TW+fZUR4UPKB96n2jy9Kp7PPGB2 2ow/CppOJt+eo9kztaamV0aky/nopbg6nf8L6y132FoDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTaRiF+b6Z+aZ0rRkrwUHjrUKI9bKSWPPGdUV/qBOjRv2hCYmX7jqx jYDaIsImm4BcVJSCJoAoYEGtDVSLA/GGIKFa6ZLtVqgFDaISNRCqEKXNIvXSavzz5sk5J+f9 c2SUsp6ZKdOnZ4iGdG2qishp+Zbf8paMxi3QLTv+NREC4ydoqLLbCHiuNyCwNediGH64AXqD fgST//5HQUWZB0Htq+cUNDsHELRajxHoeT0VvIFRAq6yUwTyLtkJPB4JYegvP4uhQdoMLyxv aegqrcNQMUzgQkUeDp8hDBOWehYsOQkwaD3PQuhVErgGfAw4ql0MtD5bBJU1/QTutbpocN4e xNBzt4rAgO0rA13OThqCplngOVPMwLX3dQRGghYKLIFRFrrbzRic5lhozA+3Fn78wsCj4nYM hZdvYPA+bUHQduIlBsnmI+AI+DE0SWUUfLr6EMGg6R0LBacnWLiQa0JwqqCchvx+DUz+H/5c PZ4EuRcbabj22YfWrBZsNTYkOPyjlJDfdFT4FHhChNagmRb+qeOFO+efs0J+2zNWMEtHhCar Wrh0bxgLtR8CjCDVnySC9OEsKxS982LhvdvNbp2dIl+1T0zVZ4qGX1fvletajl+hDlUpsoaa S9gcFJIXoWgZzy3nvS13cYQJl8j39U1QEY7h5vFNxW+ZIiSXUZzvF763+imKGNO5bXzVx+7v IZpL4D11JSTCCm4F728zoR+lc/mGxvbvmeiw/vmqm46wktPwpeYGuhTJzSiqHsXo0zPTtPpU zVLjAV12uj5r6Z8H0yQUXpDl79CZ22i8Z0MH4mRINUXROTxfp2S0mcbstA7EyyhVjCL3VlhS 7NNm/yUaDu4xHEkVjR1oloxWzVBs3CnuVXL7tRniAVE8JBp+ulgWPTMHqZPHtkt9OyY98ec0 1pUZsZopKb0+NX14bdyI33Hz0RidpTVHJ6waUioTXDc6216Qvpbd6mnr4na4bdZyoz3FHuWI XXC/ZPGeO0kmT6VzU2L/xXj7m4KuPyrlC3cVFhnXuJ50/578INktWXTro7xfxoJMbRfeb50d aifxujkWFW3UaZPUlMGo/QYe7KZkPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Make dept able to track dependencies by dma fence waits and signals. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index f0cdd3e99d36..5d2fd2f6a46d 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -16,6 +16,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -783,6 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); + sdt_might_sleep_start(NULL); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -796,6 +798,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); if (!list_empty(&cb.base.node)) list_del(&cb.base.node); @@ -885,6 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } + sdt_might_sleep_start(NULL); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); @@ -899,6 +903,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); __set_current_state(TASK_RUNNING); From patchwork Tue May 13 10:07:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084996 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3576-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXTx4RS8z1yXB for ; Tue, 13 May 2025 20:17:41 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 8A38A463DF5 for ; Tue, 13 May 2025 10:16:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B0FF5274668; Tue, 13 May 2025 10:08:03 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9227024BD02; Tue, 13 May 2025 10:07:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130883; cv=none; b=EYE4C0Z5xY5sLKk3MZyie4v0jueOSGeR0Pivvvyv6MEenDQjmatGraw/xy/Vuwxlsn7VTKBY1FunA3CCW4bEN/PfEvg9slAgZaBbfsvxfFofc1QlDlzjs2ELb127OYlLr2j3oOY/lafAC9UI/LVRFuvErLJ53ujhQMzMfXzFDo8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130883; c=relaxed/simple; bh=tLuuN6RyLYtBAfkN4cOg2ymc7qjSoDifZjv2z6JCU54=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=lwjpMge0AitUrenOTEFQolgoTlMxEWFk2Ab6d/M4M5iUWvZxMSyMsp/h4fTF0D9bpd4DssMAEBeuKI9kWsMwF3J9MRVpE8WgUGy1w9X9QA9bM4qWtDaRnCTFzpb77Py1mfxvnHipXdx8vOh2f5A2SrI9OFg8NdKhX50b7ZHyakE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-dc-682319ef9228 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 18/43] dept: track timeout waits separately with a new Kconfig Date: Tue, 13 May 2025 19:07:05 +0900 Message-Id: <20250513100730.12664-19-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ73SqXmTXX6KiZsdV7CMhVvO5l4yb7sSYzGbPs0WGYjb9ZO QFMErbfQUYhyE9iwXlALuNq0ZbJCBDdLWAmlqCAKVmqwQmNQFOiCtFsRL63GLye//P45//Pl 8JTCwyziNVn7JW2WKkPJymjZeHzt5/8uXKJe/Sw8G0JTx2movmJnofcPGwJ7kx7DaMfXcD88 huBl920KjFW9CGqGH1LQ5PYjcFp+YaHv8RzoDwVZ6KoqZiG/7goLd57PYBg8VYnB5tgOj8wj NNwsr8VgHGXhnDEfR8dTDBGzlQNz3lIIWM5yMDOcDF1+LwPOB5/BmQuDLFx3dtHgbglg6Pur mgW//Q0DN90eGsJlCdBbUcpA/UQtC8/DZgrMoSAHd9tMGNym+dBgiBYWvnjNQGdpG4bCS39i 6Pf9jaD1+BAGh93LQntoDEOjo4qC6csdCAJl4xwUlEQ4OKcvQ1BccIoGw+B6ePl/9PL5qWTQ X2ygof6VF23dROwX7Ii0jwUpYmg8QKZD91jiDJtocqNWJNfOPuSIofUBR0yOHNJoSSJ110cx qZkMMcRhPcESx2QlR4rG+zGZ6Onhdi7+XpaSLmVociXtqs27ZOpKwwtuX/Hug15fJ85DnTuK UBwvCuvE4F0r+4E9VeNMjFlhuTgwEKFiPE/4WGwsHYl6GU8J3tni/fM+FAvmCt+Izd1tXIxp YanoHDHiGMuFDWKfzUO/L00UbQ1t74riov7V5Z53XiGsF8tNNjpWKgon48Si4SHm/cJC8R/L AF2O5CY0y4oUmqzcTJUmY91KtS5Lc3Dl7r2ZDhT9L/PRmdQWNNn7rQsJPFLGyz2jn6gVjCo3 W5fpQiJPKefJ9c1RJU9X6Q5J2r0/anMypGwXSuBp5QL5mvCBdIXwk2q/tEeS9knaDynm4xbl IdvmYGDImaOom55a+2n94ZIjE4n9l5ZVJ3xZ4db+umVb3Re3LGlp259ohY6vrs7kNf+255FU 2Gr06Qq+07Tf+GFb2o5u/kkkdNrfaanYmu/+qCS+wapekeK787NaH0ipUR4Lp7YkDm1Mcq2W i03E9d+9tdObri339y0rH7Cm/h7RPVPS2WpVchKlzVa9Bf6O3gdbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRiGe9/vNFerD5P6Mjot7GBYGRkPFdKf6KVIOvyIgsiRH215iq08 RIHmKss0M5bkoabGkm2mbUZmLmyWujyWNjXU0lI0tUG5oWWHrejPw8V1w33/eSSUv5EJlKji TovqOEWMnJXS0ohtaSGuRSuVG7ObN4J7Mp2GgnIzC+0PTAjMlakYRl/ugi7POIIfLW0U5Ora ERQN9FFQWd+PwFZ6gYWOT3Oh0+1iwaHLYCGtpJyF12MzGHpv5WAwWfbCe8MwDU3ZxRhyR1nI z03D3jOCYdpg5MCQEgSDpXkczAyEgqPfyUBdoYMB27t1cPtOLws1NgcN9VWDGDqqC1joN/9m oKm+kQZP1mJov5HJQNmXYhbGPAYKDG4XB29q9Rjq9QugQuttvfTtFwMNmbUYLt17iKGz5ymC Z+kfMFjMThbq3OMYrBYdBd/vv0QwmDXBwcVr0xzkp2YhyLh4iwZtbxj8mPIuF06GQurdChrK fjrRjnBivmNGpG7cRRGtNZF8d79lic2jp8mrYoE8yevjiPbZO47oLWeItTSYlNSMYlL01c0Q i/EKSyxfczhydaITky+trdy+JUek26PEGFWCqN4QHilV5mi/cacyjic5expwCmqIuIr8JAK/ WWjUTTA+ZvnVQnf3NOXjAH65YM0c9nqphOKds4Wuwh7kC+bzB4THLbWcj2k+SLAN52Ify/gt Qoepkf5XukwwVdT+LfLz+p/3W/96fz5MyNab6Gwk1aNZRhSgikuIVahiwtZropXJcaqk9cfj Yy3I+0GG8zM3qtBkxy474iVIPkfWOLpC6c8oEjTJsXYkSCh5gCz1sVfJohTJZ0V1/DH1mRhR Y0eLJbR8oWz3ITHSnz+hOC1Gi+IpUf0/xRK/wBQkbB3PsxU8H9pvL8b2nWMRawNl027SP1az 9jxa5Vqmkvek4cSgremZctPsvirZwYEpm8Nx+XB41s3gy+iccaptFtu86rPnYH71o0QqiSm/ 3hVi361cOsJb0YbuNyc/Hr1btie87P0chS5eOhTw1DHvxb5H+eWri9bQyxdd2KRZIqc1SkVo MKXWKP4AYsnRDT0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Waits with valid timeouts don't actually cause deadlocks. However, dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock. However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired. Report these as an information rather than warning DEADLOCK. Plus, introduce CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose. Signed-off-by: Byungchul Park --- include/linux/dept.h | 13 +++++--- include/linux/dept_ldt.h | 6 ++-- include/linux/dept_sdt.h | 13 +++++--- include/linux/sched.h | 2 ++ kernel/dependency/dept.c | 66 ++++++++++++++++++++++++++++++++++------ lib/Kconfig.debug | 10 ++++++ 6 files changed, 89 insertions(+), 21 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 138106869494..58362bd2c4ad 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -270,6 +270,11 @@ struct dept_wait { * whether this wait is for commit in scheduler */ bool sched_sleep; + + /* + * whether a timeout is set + */ + bool timeout; }; }; }; @@ -376,8 +381,8 @@ extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); -extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); -extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn, long timeout); extern void dept_request_event_wait_commit(void); extern void dept_clean_stage(void); extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); @@ -424,8 +429,8 @@ struct dept_map { }; #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_copy(t, f) do { } while (0) -#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0) #define dept_request_event_wait_commit() do { } while (0) #define dept_clean_stage() do { } while (0) #define dept_ttwu_stage_wait(t, ip) do { } while (0) diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h index 062613e89fc3..8adf298dfcb8 100644 --- a/include/linux/dept_ldt.h +++ b/include/linux/dept_ldt.h @@ -27,7 +27,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_wait(m, LDT_EVT_L, i, "lock", sl, false); \ dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ } \ } while (0) @@ -39,7 +39,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ else { \ - dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ } \ } while (0) @@ -51,7 +51,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ } \ } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 93d772c71905..b92bc8c988c9 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -22,11 +22,12 @@ #define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) -#define sdt_wait(m) \ +#define sdt_wait_timeout(m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait(m) sdt_wait_timeout(m, -1L) /* * sdt_might_sleep() and its family will be committed in __schedule() @@ -37,13 +38,13 @@ /* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start(m) \ +#define sdt_might_sleep_start_timeout(m, t) \ do { \ struct dept_map *__m = m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) - +#define sdt_might_sleep_start(m) sdt_might_sleep_start_timeout(m, -1L) #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) @@ -53,7 +54,9 @@ #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout(m, t) do { } while (0) #define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start_timeout(m, t) do { } while (0) #define sdt_might_sleep_start(m) do { } while (0) #define sdt_might_sleep_end() do { } while (0) #define sdt_ecxt_enter(m) do { } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index 802fca4d99b3..541ddacdc3d0 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -864,6 +864,7 @@ struct dept_task { bool stage_sched_map; const char *stage_w_fn; unsigned long stage_ip; + bool stage_timeout; arch_spinlock_t stage_lock; /* @@ -904,6 +905,7 @@ struct dept_task { .stage_sched_map = false, \ .stage_w_fn = NULL, \ .stage_ip = 0UL, \ + .stage_timeout = false, \ .stage_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 211bafffc980..75fe64f86ee5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -756,6 +756,8 @@ static void print_diagram(struct dept_dep *d) if (!irqf) { print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + if (w->timeout) + print_spc(spc, "--------------- >8 timeout ---------------\n"); print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); } } @@ -809,6 +811,24 @@ static void print_dep(struct dept_dep *d) static void save_current_stack(int skip); +static bool is_timeout_wait_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + + do { + struct dept_dep *d = lookup_dep(fc, tc); + + if (d->wait->timeout) + return true; + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + return false; +} + /* * Print all classes in a circle. */ @@ -831,10 +851,14 @@ static void print_circle(struct dept_class *c) pr_warn("summary\n"); pr_warn("---------------------------------------------------\n"); - if (fc == tc) + if (is_timeout_wait_circle(c)) { + pr_warn("NOT A DEADLOCK BUT A CIRCULAR DEPENDENCY\n"); + pr_warn("CHECK IF THE TIMEOUT IS INTENDED\n\n"); + } else if (fc == tc) { pr_warn("*** AA DEADLOCK ***\n\n"); - else + } else { pr_warn("*** DEADLOCK ***\n\n"); + } i = 0; do { @@ -1578,7 +1602,8 @@ static int next_wgen(void) } static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep) + const char *w_fn, int sub_l, bool sched_sleep, + bool timeout) { struct dept_task *dt = dept_task(); struct dept_wait *w; @@ -1598,6 +1623,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_fn = w_fn; w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; + w->timeout = timeout; cxt = cur_cxt(); if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) @@ -2298,7 +2324,7 @@ static struct dept_class *check_new_class(struct dept_key *local, */ static void __dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, - bool sched_sleep, bool sched_map) + bool sched_sleep, bool sched_map, bool timeout) { int e; @@ -2321,7 +2347,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, if (!c) continue; - add_wait(c, ip, w_fn, sub_l, sched_sleep); + add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); } } @@ -2356,14 +2382,23 @@ static void __dept_event(struct dept_map *m, struct dept_map *real_m, } void dept_wait(struct dept_map *m, unsigned long w_f, - unsigned long ip, const char *w_fn, int sub_l) + unsigned long ip, const char *w_fn, int sub_l, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (dt->recursive) return; @@ -2372,21 +2407,30 @@ void dept_wait(struct dept_map *m, unsigned long w_f, flags = dept_enter(); - __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false, timeout); dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_wait); void dept_stage_wait(struct dept_map *m, struct dept_key *k, - unsigned long ip, const char *w_fn) + unsigned long ip, const char *w_fn, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (m && m->nocheck) return; @@ -2435,6 +2479,7 @@ void dept_stage_wait(struct dept_map *m, struct dept_key *k, dt->stage_w_fn = w_fn; dt->stage_ip = ip; + dt->stage_timeout = timeout; arch_spin_unlock(&dt->stage_lock); exit: dept_exit_recursive(flags); @@ -2448,6 +2493,7 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_sched_map = false; dt->stage_w_fn = NULL; dt->stage_ip = 0UL; + dt->stage_timeout = false; } void dept_clean_stage(void) @@ -2480,6 +2526,7 @@ void dept_request_event_wait_commit(void) unsigned long ip; const char *w_fn; bool sched_map; + bool timeout; if (unlikely(!dept_working())) return; @@ -2506,12 +2553,13 @@ void dept_request_event_wait_commit(void) w_fn = dt->stage_w_fn; ip = dt->stage_ip; sched_map = dt->stage_sched_map; + timeout = dt->stage_timeout; wg = next_wgen(); WRITE_ONCE(dt->stage_m.wgen, wg); arch_spin_unlock(&dt->stage_lock); - __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: dept_exit(flags); } diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index a626631f6bec..7c74f92e4cc2 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1394,6 +1394,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT From patchwork Tue May 13 10:07:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085004 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3575-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXWj00drz1yXB for ; Tue, 13 May 2025 20:19:12 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id AD6D18C3169 for ; Tue, 13 May 2025 10:16:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1CCF9270550; Tue, 13 May 2025 10:08:03 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3614225CC69; Tue, 13 May 2025 10:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130882; cv=none; b=OvDROwblmm2+0G2NUCBbdTBIAf7ZvDkSixgegPAVko7h7SChVBvHDaa9ANoFFXr6veDnuZEQ7GBjTroJZEgCZP82lSviCBRUAbLzsZTgJU0hOeQP5rQg0l5aQCkUS61DYtE2NrhgeIXCnn+P3JfR6qIiA4fmsZpN/NxXwApO2rw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130882; c=relaxed/simple; bh=TSPxdh3H9mjhA/wDLLlJUz6YZTY7hYg5ulgpaHBJJSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=M9UkoWFxJb26aK3k0aWyJPdndCx6zqahtjLorU9m6ugz6M5E7/Zo31dDVnddtmsPzBSANFgnrj3xYU16vGp0Q3v5fdI/l8f8Yd++mmGec2x+VL6IvqYq0LK9z+k5b48FIUW+/7HnuacLl3PJApLzCKCG8yJAaZ+hdU6zeL36ByY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-ec-682319f0e72e From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 19/43] dept: apply timeout consideration to wait_for_completion()/complete() Date: Tue, 13 May 2025 19:07:06 +0900 Message-Id: <20250513100730.12664-20-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHe573utXiZXZ5M6lcSGFXI+OQ3agPvhGBXSDKD7XyrY3UZJap EEydlaZmF7WLrc1qmq60LcjKlRkuL6Qz70MtJSrLS1kbaXbZrL4cfvwP/9/5clhCXkP5suro o6ImWhmpoKWkdHCKccnnWfNVy+3lPLi+nSYhv9RMg+NuCQLz/SQM/dWh0O4eQPDjZSMBeTkO BMbebgLu23sQ2IqSaWh+OxVaXMM01OacoSHlRikNTZ/GMXTlnsdQYtkKr03vSKjPLsCQ10/D 1bwU7BkfMIyaihkwaQOgr+gKA+O9QVDb00aBzbkILuu7aKiw1ZJgL+/D0Pwon4Ye828K6u01 JLizZoPjXCYFd4YKaPjkNhFgcg0z8KrSgMFumAFlOo/w5NdfFLzIrMRw8uY9DC2djxE8Of0G g8XcRsNz1wAGqyWHgLHCagR9WYMMpGaMMnA1KQvBmdRcEnRdwfDju+fytW9BkHS9jIQ7P9vQ hrWCWW9GwvOBYULQWY8LY65WWrC5DaRQV8ALD690M4LuiZMRDJZjgrUoULhR0Y8F44iLEizF abRgGTnPCOmDLVgYamhgwvz2SNdEiJHqOFGzbN0+qcr86jITo5fGW9OasBa9YdORhOW5lfxL ox79Z6vzAuVlmlvAd3SMEl6exs3jrZnvPLmUJbi2yXz7tc6Jgg+3jx8zuLGXSS6A/3rhwwTL uFX8l1spxF/pXL6krHKCJZ78Z2ED6WU5F8xnG0pIr5Tnzkr4bmv6v8Is/llRB5mNZAY0qRjJ 1dFxUUp15MqlqoRodfzSA0eiLMjzX6YT4+HlaMSxowpxLFJMkdX0+6vklDIuNiGqCvEsoZgm S3rgiWQRyoREUXNkr+ZYpBhbhWazpGKmbIX7eIScO6Q8Kh4WxRhR83+LWYmvFoWvD2qhHj/T J/uHL4vZtvH7zvnpX1oPuRpfV59afzDxXliGT1jonNUHhjaZtGKFLbHb8bEadyoK2w/m9uwP CFyyeXzh9hlOv/IqWWMI+JlCjVtCMqZf2hWc3BtaX/feFqK9nVH31Odh11bn591zLvmfXSxp 2u+bmlaqi7+4Kvvjdr8CBRmrUgYFEppY5R87WIikWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/Pz46zZ2k8an6dmS0TmexjDBubRxuimc1sOjzcrS48R9Sw 0oWkJEvohxNduU7OXVNSLbVOp0m/JC0nMesmWtwdXflxx/zz2Wuv9/Z+//NhiUAjFcyq44+K UrwyTkHLSNmWVamLR2bOVy3tMK4Et+s8CQX3TTS0V5QjMFWmYHA2b4RXnmEE489fEJCX247g 1rs3BFTaHAjqys7Q0PVhKnS7R2iw52bQkHr7Pg0dnyYw9F/NwVBu2QxvDR9JaM0uxpDnpCE/ LxX7zhCGMYORAUPyAhgsu8HAxLtwsDt6KGgqtFNQ17cIrhf101BbZyfBVj2IoaumgAaH6TcF rbYWEjxZIdB+OZOCe1+KafjkMRBgcI8w0Nmgx2DTTwezztd69tsvCp5mNmA4e+cBhu7XjxHU nx/AYDH10NDkHsZgteQS4C1tRjCY9ZmBtItjDOSnZCHISLtKgq4/AsZ/+JYLXeGQctNMwr2f PWjdGsFUZEJC0/AIIeisxwWv+yUt1Hn0pPCsmBce3XjDCLr6PkbQW44J1rJQ4XatEwu3vrop wWJMpwXL1xxGuPC5Gwtf2tqYqFm7ZKv3i3HqBFFasiZGpjJ1XmcOF8lOWNM7cDIaYC+gAJbn lvPWviuUn2luId/bO0b4OYiby1szP/q8jCW4nsn8q8LXyB9M42J4r96D/UxyC/hvV4b+spxb wY+WpBL/Sufw5eaGvxzg8z9L20g/B3IRfLa+nMxGMj2aZERB6vgEjVIdFxGmjVUlxqtPhO07 pLEg3wcZTk1crkauro2NiGORYoq8xTlPFUgpE7SJmkbEs4QiSJ5S5VPy/crEJFE6tEc6Fidq G1EISypmyCN3ijGB3EHlUTFWFA+L0v8UswHByajm3DWX5GhbPloCWyPnh21twdvlW7J733cl VUmRs5NsmlpvfuOg/Un08eCK8aHHM04vO0Otqv51yhXLpU+ZXRkeLMtMW7tkQ4vZu+mAZnfO w20ZIQnRSQpntNkxc/ORu5cm6faOrd+2o7V2fBSiKgqyBr4XrIyKJNZJFdWhnXNP9ipIrUoZ HkpIWuUfQXCWZD0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index bd2c207481d6..3200b741de28 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -41,9 +41,9 @@ do { \ */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) +static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start(&x->dmap); + sdt_might_sleep_start_timeout(&x->dmap, timeout); } static inline void complete_release(struct completion *x) diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 3561ab533dd4..499b1fee9dc1 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -110,7 +110,7 @@ __wait_for_common(struct completion *x, { might_sleep(); - complete_acquire(x); + complete_acquire(x, timeout); raw_spin_lock_irq(&x->wait.lock); timeout = do_wait_for_common(x, action, timeout, state); From patchwork Tue May 13 10:07:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084992 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3574-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXT84nGFz1yXB for ; Tue, 13 May 2025 20:17:00 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 90A1D16D20C for ; Tue, 13 May 2025 10:16:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D04226E167; Tue, 13 May 2025 10:08:02 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2E386248880; Tue, 13 May 2025 10:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130881; cv=none; b=EAjE+hn700nMwn01DNDwbDQIruQXFN3QYpMjSdJR7HnPLqGw/ZUPKlpNpZKzGjZfil/HbP0EXINYyZZ/4E4pC73q3DLlg+PvE4OSm4rkdz6OpOCkx8xkQGnwhDiH6GL1qGN9KT20RikS/dYC04q7GiMKlzG84f1AChX0qvD/sP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130881; c=relaxed/simple; bh=9o9SI0ONdb1WbiIWXrCSb5nX53qir4UGX5Gt0IQgpYM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=n3Dn1OQcRi3YqqNkKpWhuwgYWc481xv71+npgMwZo34nWPrn8TI7xbBER1/rNYh+S3PtFV+xh0z6Nxrv8upGq9z1YOcEeG7BJCz5M8qUrgu/wzJd0jBV5OSXg/g+3JakzZYvSDoFAIB5QZUlyNPyIM58A2tLHDVSS6DauYDqQkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-fc-682319f015c0 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 20/43] dept: apply timeout consideration to swait Date: Tue, 13 May 2025 19:07:07 +0900 Message-Id: <20250513100730.12664-21-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0zTZxjFff/XUq3+7cj2etcKajAyi7o8H+biF8P7xWSXZHPTRBv5Z21W QAqikJhRBYIIDCGVCKgtkFKhSCle8FJXQe5Y6+iAKrLJjFlHkaXaZkBxKzi/PDk55+T8vjwS Wt7DrpRoUjJEXYpKq+CkjHRySc22v1dsVG/3TayH4JsCBqqbrRy4rzYisF7TU+DrTIThkB/B 7MNHNFQY3AhMz5/RcK1rDIHDcoqDwRdLwROc4qDXcJaD07XNHDyeCFMwer6Mgkb7PvjN/JKB /tIaCip8HFRVnKYi508Kps0NPJhzYmHcUslD+LkSeseGWHA82QoXLo1ycNfRy0BX2zgFg7er ORiz/stCf1cPA6GSVeA+V8xC06saDiZCZhrMwSkefnEaKegyfgi23Mhg/uu3LHQXOynIr2uh wOO9g+Bewe8U2K1DHHQE/RS02g00zNR3IhgvmeQhr2iahyp9CYKzeecZyB3dBbP/RMgX3yhB f9nGQNPcENqzm1gvWRHp8E/RJLf1OJkJ/soRR8jIkL4aTG5VPuNJ7r0nPDHaj5FWSxypveuj iCkQZIm94QxH7IEynhROeijyyuXiP1/9nfTTJFGryRR1H392WKpufqFHR19zJyo669gc1McW oigJFnbiAkse/16fqncw85oTNuORkWl6XkcL63Fr8ctIXyqhhaHFePiiF80HHwh7cXu+YaHE CLF49HLlwpBM+ATrp53/A9bhRptzoRMV8efqXQsAubALlxobmflRLPwUhW2V78hYWIHvW0aY UiQzokUNSK5JyUxWabQ749VZKZoT8UdSk+0o8l/mk+EDbSjg/qodCRKkWCLr8W1Qy1lVZnpW cjvCEloRLdPfjFiyJFVWtqhLPaQ7phXT29EqCaP4SJYQOp4kF75XZYg/iOJRUfc+pSRRK3MQ k2zeT9oKZhNtdJnO1FuYWeS3v73yY3ZV3dMmj39ix/ZFM6Zl5V8+PfnzmfJzf+0wdS+ztnzd MaCbCoXjUnevcXk9iwfEtbXfbvX/YYQ9g+4m5epsblNeWrjfpd3vPhigYoq2XHfI5gwbvd5v Br5wBmKU+EFCRuCGdXlCWmx1j4JJV6uUcbQuXfUfaqoksVsDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5/rXE6OS+qkRbbogpYmZLxQaV/CQ2SEBIEEOfLQlpvGlqZB 5XKJXSZL0pHXpTJlzqZTUquJTZyXylba0jArG9HIS5STVnaZUl9efjzPw/N8eQW42EyGC+SZ Z3lVplQhoYSE8PCegh1f1m6S7ax14eCbLyKg0mqhwHW3CYGlXYOBty8JXi1MI/j59BkOhlIX gjvv3+DQ7pxEYG+8TMGIJwRGfXMUDJZep6CgzkrB88+LGEyUlWDQZEuGt6aPBDzW12Jg8FJQ YSjAAucTBn6TmQZT/maYaiynYfF9HAxOuknorRokwf46Gm5XT1Dw0D5IgLNzCoOR+5UUTFr+ kPDYOUDAQnEEuG7qSGieraXg84IJB5NvjoYXPUYMnMbV0KINtBZ++01Cv64Hg8L6VgxGxx8g 6C56h4HN4qag1zeNQZutFIcfDX0IpopnaLhyw09DhaYYwfUrZQRoJ+Lh5/fActV8HGhqWgho /uVG+xM4S7UFcb3TczinbTvH/fC9pDj7gpHghmpZrqv8Dc1pu1/TnNGWzbU1RnF1D70Yd+er j+Rs5qsUZ/taQnPXZkYxbnZ4mD6yPlW4N51XyHN4VWxCmlBm9WjQmW9UrqGvnsxHQ+Q1FCRg mV3s5QY7scQUs5UdG/PjSxzGRLJtuo+BjFCAM+6V7KuqcbRkrGIOsI7C0uUQwWxmJ2rK6SUW MbtZjb/nX+kGtqmlZzkTFNB/NQwvD4iZeFZvbCL0SGhEK8woTJ6Zo5TKFfEx6gxZXqY8N+Zk ltKGAh9kurB4sxPNjyQ5ECNAkmDRgHejTExKc9R5SgdiBbgkTKTpCEiidGneeV6VdUKVreDV DhQhICRrRAeP8Wli5pT0LJ/B82d41X8XEwSF5yP979R2U/ktb/KlxKjCw+uiY501l7z60/7w 6tlcQ0Ta+WxdjMTZkZOYmHroWKL4g67q+Qvr0ZHmRt/co/7pd7PbQlwplrCu7hXR/bonW+q0 6SXe1j3WcJEnxRzyKfJil2dmLDT4+3aHMuvls+57Es96t5Ldl+FSBBc9uhVadjzKKiHUMmlc FK5SS/8CJQIgkj0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/swait.h b/include/linux/swait.h index 277ac74f61c3..233acdf55e9b 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -162,7 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ From patchwork Tue May 13 10:07:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085010 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3577-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXXy38Kwz1yYN for ; Tue, 13 May 2025 20:20:18 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id B4C573A9633 for ; Tue, 13 May 2025 10:16:34 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B02FE274664; Tue, 13 May 2025 10:08:03 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 40E2F267B90; Tue, 13 May 2025 10:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130883; cv=none; b=g4H8iZ0RmxL/jShZK5sVs95jcNu7DI2O5Ahjb7zCk5mklEBEjeSKc+hfxVnbbcICZdrUrdMhHpbUG5/3Cc1LtT2Eo/v3HPyQb51H/1/PoyplvK5an2MTqAJa5FyTSMzPPABYyUJFTNJdmMXXPyikhu/YW98hU3F80T6bKC85Hts= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130883; c=relaxed/simple; bh=G02jUxrc8GHX4FLfxPcv5jAlvBb9vcSmciPzIKTL9pw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=FT7QyFPqiRyRdlASUGhDBrDwE9YWlHYj3yogd0zpn8cq3GT29j9NKkShZ8+EMdL7IT1XcRzw+15Nn2ZjmV8lz/4Je7yD50XQnWSMkF1MrIILy+lxD93S5WIloYP5PWHz7HmiCuci192qZivnVDfnMhbVCzZWmr3GqPnxLXsEfIE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-669ff7000002311f-0c-682319f0cb72 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 21/43] dept: apply timeout consideration to waitqueue wait Date: Tue, 13 May 2025 19:07:08 +0900 Message-Id: <20250513100730.12664-22-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTcRTG+7/XuVq8rMjXNKrViopKReMQFX4IeimKoE8VVCNf20inzLQm BLOmrYtihkraZVrM6ZaXqWXlzBSXS7KVutSmqYi1vIW6mWmXzerL4cfznPM8X44AF7eQKwUK 5TlepZTFSighIRxbUrj1W9A6eajRvgk80zoC7pSbKXCUmRCYq1MxcDfvgw/eUQRzb97ikJfj QFA40ItDta0PgdV4iYL2oaXQ4ZmgwJ5znYLLD8opeDcyj4ErNxsDk+UgfDIME9CaVYRBnpuC grzLmG98wWDWUEqDQSOFQWM+DfMDYWDvc5Jg7dkCt++5KKiz2gmw1Q5i0P7sDgV95t8ktNpa CPBmBoPjZgYJj8aLKBjxGnAweCZoeN+gx8CmXwEVWl9g+tQvEl5lNGCQ/rASg47u5wjqdf0Y WMxOCpo8oxhUWXJw+FHcjGAwc4yGtBuzNBSkZiK4npZLgNYVCXPffc13p8Mg9X4FAY9+OlHU bs58z4y4ptEJnNNWned+eDopzurVE9zrIpZ7mt9Lc9r6HprTW5K4KuNm7kGdG+MKJz0kZym9 SnGWyWyauzbWgXHjbW304ZBjwl3RfKwimVdt33NKKHdla8mEGvqCqfgSpUFa6hoKELBMBGus nCL/84uyxws6xWxku7pmcT8vZ9awVRnDvh2hAGeci9kPd7uR31jGHGJLa+YWjglGyjpffqb9 LGJ2sAMfs/4VrGZNFQ0LQQE+/WdxG+FnMRPJZulNhD+UZW4FsK4SO/H3IIh9aewispBIjxaV IrFCmRwnU8RGbJOrlYoL207Hx1mQ778MF+eP16JJx5FGxAiQZImoxb1WLiZlyYnquEbECnDJ clHqE58kipapU3hV/ElVUiyf2IiCBYQkUBTuPR8tZs7IzvFneT6BV/13MUHASg3CyRjrhp2O BN1QSkh+0fyvZdLwJsWEqZP/JJ7JWXTqa/xMSJTw4XhoR7070t7XW1I8N7zfsk5DqruV00d1 QXb1AUaXd3JqdeWV3Bhna2BSJja4dm/1gYzckZoBjZRJq2xfRcedaJJuaU73xNcG2/pnvs8O rReHPdtY8LF2TdS4hEiUy8I246pE2R8tt+I4WwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRiGfb/jnC2+TOrrRLWywsgSsh4oovrjS5D0I0iigys/2mpO22xp B3C5xLTpMmyUaXPFtG2lbR3soJni0iyztGW1LKXTyLLSSTY7bFF/Hi7uG677zyMiI230ZJFC lSGoVTKllBFT4sRlOQu+TJolX3S+Wgz+oTwKTlc7GOi4aEfguKwjwNecAE+H+xEEHjwkwVTS gaCi9yUJl909COqqDjHQ+WYsdPkHGGgtKWAg52w1A48+jhLgPVFMgN25Fl5Z31HQZrQQYPIx UGrKIYLnAwEjVhsL1uxo6Ks6xcJobxy09nhoaCprpaHu+Xw4We5l4FZdKwXu2j4COm+cZqDH 8ZuGNncLBcOFU6DjmIGGC58tDHwctpJg9Q+w8LjBTIDbPAFq9EFr7uAvGu4aGgjIPXeJgK5n NxHU570mwOnwMNDk7yfA5Swh4UdlM4K+wk8sHD46wkKprhBBweETFOi98RD4HlwuG4oD3Zka Ci789KCVK7Cj3IFwU/8AifWuvfiH/wmD64bNFL5n4fH1Uy9ZrK9/zmKzcw92VcXgs7d8BK74 5qex03aEwc5vxSzO/9RF4M/t7ey6aRvFy1MEpUIrqBeuSBbLvcV6Ov0Km2mvPMRkIz2Tj8JF PLeYv33x6l9muLl8d/cIGeIobgbvMryj85FYRHKeCP5p2TMUKsZzibztSoAOMcVF854779kQ S7glfO8L4z/pdN5e0/BXFB7Mf1a2UyGO5OJ5o9lOGZHYjMJsKEqh0qbKFMr4WM0ueZZKkRm7 PS3ViYIfZD04eqwWDXUmNCJOhKRjJC2+mfJIWqbVZKU2Il5ESqMkumvBSJIiy9onqNO2qvco BU0jmiKipBMlazYIyZHcDlmGsEsQ0gX1/5YQhU/ORknO44PREZtsuPjqV2XCwGBOUmmRJa+x fqwxq4nQjls3bZXs4VFDScPXjeJk3ZyZieu9LfNi0w5sPtBt01oCLi6hwDSuaGpXxdvlRau3 7t/yJaLUuN33+/3usNmxYerc0dvblNay9g+Bpe6M7pW5bS+Om3bezzR4+q7HaNrKX7ObVFJK I5fFxZBqjewPbbt9GD0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index aae161e500dc..e77344a8160d 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -304,7 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ From patchwork Tue May 13 10:07:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2084998 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3578-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXVd5Pmlz1yXB for ; Tue, 13 May 2025 20:18:17 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B45364A6C8E for ; Tue, 13 May 2025 10:17:26 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A4DA627780C; Tue, 13 May 2025 10:08:04 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9A5E4269D09; Tue, 13 May 2025 10:08:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130884; cv=none; b=haafN84FCufVUTWBvekoCC4HNa4XMCLegmzi49wBtxZuUKLPACqREEZgv8emKpGl4pZ6bxGSMkLuiUY9rTeps7fwjY94IbDY6ZJoE9jNfHYWIedobdhaER0jNveL1K+HkkaFoAkf/tlqzyoN3wjiBYGqjwBxiBCTCwzsGOnf2GU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130884; c=relaxed/simple; bh=sXq6ZG2FCPgGrD/93i05yDVFGWV/kuOXrwRc2Zt22go=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=YeuZK3NtQy3ULV/4NmFruHfVFsB8OqUR46LUSUXpSSTgHtQbj3ESHSPxMMvhtaG7Wn1ZCY3fmqAs+cmdWsDbp/cGPZ0aHsouy7Rp36IJQZB3WjC6FmxvQ1MTp734KJKiU64FE5TuywwbdGu1soNx6rfR5kk3Ugq98W0Ue3VTYHQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-1c-682319f07f07 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 22/43] dept: apply timeout consideration to hashed-waitqueue wait Date: Tue, 13 May 2025 19:07:09 +0900 Message-Id: <20250513100730.12664-23-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRjH/b3ntly8ruttRcdECiXLqHigg4igH1TQSZGRjXxxI7XayjII tKaVtqWS2aG1NNbSeTSje2WKSzOvtKWlplLRcB6YWy212jr+efjw/T7f7/PPIyJlL2i5SB1/ WNDEK2MVjISS9AfmLxiaHqxa1N0ZBu6RMxTklloYaCopQmC5m0yAs3odvPW4EIzWN5KQk92E 4EZPJwl37V0IbOaTDLR8nAit7kEGarPTGThVUMpAc98YAR0Xswgosm6ED6bPFNRl5BOQ42Tg as4pwje+EOA1FbJgSgqBXvMVFsZ6IqC2y0GD7V0YXL7WwcATWy0F9ge9BLQ8ymWgy/KLhjp7 DQUewwxoytTTUDyQz0Cfx0SCyT3IwusKIwF241Qo0/kKU7/+pOGFvoKA1Jt3CGhtf4zg6Zlu AqwWBwNVbhcB5dZsEn7cqkbQa+hnIeWcl4WryQYE6SkXKdB1LIXR777LeSMRkHy9jILicQda vRJbrlkQrnINklhXfhT/cL9hsM1jpPDLfB4/vNLJYt3Tdyw2Wo/gcnMoLnjiJPCNYTeNrYVn GWwdzmJxWn8rgQcaGthNM3dJVkQLseoEQbNw1V6J6lv6xoMG9tinskw6CfXQaUgs4rklvCu3 lv3Phux7f5jh5vFtbV7Sz5O5OXy5/rNvXyIiOccE/m1eO/Ibk7jtfN1oDeNnigvhL7T9DUu5 ZbzRfvtf6Wy+qKziT5HYp4/faqD8LOOW8hnGIspfynPnxXy3d+RfYDr/3NxGZSCpEQUUIpk6 PiFOqY5dEq5KjFcfC993IM6KfO9lOjEW+QANN22tRJwIKQKlNc65KhmtTNAmxlUiXkQqJkuT 7/skabQy8bigORClORIraCvRDBGlmCZd7DkaLeNilIeF/YJwUND8dwmRWJ6EQmat6c1srjih n2KO1uaW1ttcQwhdYj9VdQbIg8RU6Eum+1Dws6H2pILN9bvCnHWB1RtaOmK2ffXOX/detyYg IG4nEZyVsl7u6XuGI7es3RPUOjDe2GjShy2P+vbqcqrj3unmgeMXPj6PqW9I21ESPKjrKg5X uoL2pcij5i9b7tytoLQqZUQoqdEqfwNddHR5WgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/fnY5HGo/McNUME43aZ0Mz/3j8yPKH2fxTz3jmbv2QOyIb 7nQapVs1aeh4Kq6rOzpXC6mW2p3SJEqlJWr6caSMLuoq7ph/Pnvt/f583u9/PgweUEoGMaqk k6I6SUhQUDJCtn9r2oZvy4KVmzLMi8A9cZmAgnIrBW0PLAislToMXI5d0DU5isDz8hUO+Xlt CAr73+NQ6exDUGu+SEH7pwXQ4R6noDkvk4K04nIKXn+ZwaD3ei4GFns0fDANEdCSXYRBvouC W/lpmHeMYDBlKqPBpA2FAfNNGmb6w6G5r5OERmMzCbU96+HG7V4KamqbCXA+HsCgvbqAgj7r bxJanE0ETBqWQ1tOFgn3x4oo+DJpwsHkHqfhTb2EgVNaAja9NzX9xxwJz7PqMUi/+xCDjndP EdRd/oiB3dpJQaN7FIMKex4O0yUOBAOGrzRcujpFwy2dAUHmpesE6HsjwPPL22ycCAfdHRsB 92c70Y4o3nrbivjG0XGc11ec5qfdbym+dlIi+BdFHP/k5nua19f10LxkP8VXmNfxxTUujC/8 7iZ5e9kVird/z6X5jK8dGD/W2krHrDgs23ZUTFCliOqNUXEy5c/M6GQDfWbQlkNqUT+ZgfwY jt3CGfKqaB9T7Bquu3sK93Egu4qryBry7sgYnO2cz3UZ3yGfsZg9yLV4migfE2wod63737Gc jeQkZyn9L3QlZ7HV/w3y8+qzJa2EjwPYCC5bshDZSCaheWUoUJWUkiioEiLCNPHK1CTVmbAj xxPtyPtApnMzOY/RRPuuBsQySOEvb3KtVgaQQoomNbEBcQyuCJTrHnkl+VEh9ayoPh6rPpUg ahrQcoZQLJXvOSTGBbDHhJNivCgmi+r/Lsb4BWmR7pq2arDyhH270ba2KlLRXliwcXN99YqB rkht0KDNP7b00O6R15mekD3PttY4733ev3A6bM3VF8Pn5jadiO6xDOmpp9r0RiadC5kX/1Iq 959ft/TCcKDVodog7ZRX++cKqa6+6mCP2VgS5eBmw87vFX7F5B6IcbyK2bdTNTeWHKwgNEoh fB2u1gh/ACisQUA8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 179a616ad245..9885ac4e1ded 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -258,7 +258,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ From patchwork Tue May 13 10:07:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085000 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3579-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXWT28jgz1yXB for ; Tue, 13 May 2025 20:19:01 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EA13F1B602A3 for ; Tue, 13 May 2025 10:18:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B421C27A446; Tue, 13 May 2025 10:08:05 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 927332459F9; Tue, 13 May 2025 10:08:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130885; cv=none; b=mivew7ygZ6Kh5GzR/nzY3mmu+RR9zrOhBtY3hT8FD1Vx7v1WhNGoycgXDZ5vULV+U7/zKh3gSp8kfsZfJYwqKBMfaCvU4pZ6tDp5vKd1TLa9H7VnQ+FoTu+khYrZoAVPSBxu2iQdUZU85oVNNRmC2WGo8Phro+ouZjxuRGMWmjE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130885; c=relaxed/simple; bh=WB3yty1UxgRrT5j6RAtqcF8Rog/sCePCJmMWqhpGSYE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=MhJkEY8cXqNwFweosvJm3dgaACQ+ZpVy0jS9I7TIBVrC76IyCUi1IomVJ1zpk5C0fMNsHfe/gUehKziByhVAUQrToN5hTdzuxSVbeIQxeDQ3OFP2UJk/snM6q7jTbY5/ktaTiFSZooruUspbeebL9sHoiJrNJ5j43YJsPH3gJuE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-2c-682319f04278 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 23/43] dept: apply timeout consideration to dma fence wait Date: Tue, 13 May 2025 19:07:10 +0900 Message-Id: <20250513100730.12664-24-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUxTaRTH+b67tlBzUwSvYKLWGI1GRozLicsMxgfuPGg0PmjURBu5Q5th MQVZNETQQhCEYGegDiAU0NrQIrVgXAYQQWvByDBDB6ECaiHGKosibURxnNbl5eSX/8n5nfNw WEL+gIpg1UmpoiZJmaCgpaR0IqRmzZuFy1RrZ95FgXcmn4TKRgsNvVfNCCzNORg892PhsW8c wcdHfxGgL+1FUPN8mIBm+wiCVtNpGvrG5oHTO0VDV2khDWfqGmn4+/UchqEyHQazbSc8Nb4g 4WFJLQa9h4YK/RnsLy8xzBrrGTBmLwe3qZyBuefR0DXST0GrazX8UTVEQ0trFwn2m24Mfbcr aRixfKbgod1Bgq84EnrPF1HQMFlLw2ufkQCjd4qBf9oNGOyGcLBq/cK8d/9R8KCoHUPepWsY nIN/ImjLf4bBZumnodM7jqHJVkrAhyv3EbiLJxjIPTfLQEVOMYLC3DIStEMb4ON7/+aLM9GQ U20loeFTP4rZJliqLEjoHJ8iBG1TuvDB+y8ttPoMpNBdywu3yocZQdvmYgSD7bjQZFol1LV4 sFAz7aUEW/1ZWrBN6xihYMKJhcmeHmb3ogPSrXFigjpN1Pzw4xGpanT0LXmsj824PKsjs5GZ KUASlufW85fKG/B3rtBNEgGmuRX8wMDsF57PLeGbil5QBUjKElx/MP/44iAKNEK5XXynedI/ zLIkt5w3nFsZiGXcRt7RZkVfnYt5s7X9i0fizz9d6SEDLOc28CUGMxlw8txvEv59t/vbQQv5 u6YBsgTJDCioHsnVSWmJSnXC+ihVZpI6I+pocqIN+d/LmDV38Caa7t3bgTgWKUJkDs9SlZxS pqVkJnYgniUU82U5N/yRLE6ZeULUJB/WHE8QUzpQJEsqFsjW+dLj5Fy8MlX8VRSPiZrvXcxK IrKR8VDwyeSni8LPx7c0mq7+wjC1++50bw4KDf4p7+egrhh3ljPVXhUW69ldlx52Iu6e5Ul4 TGwyd2pLoeSVayTkbkTkKdeOlb6NmsEyreOlNHdp85N5LXpztX5T1v7te0crr+9p9xS4ZsLW pY1Jbyx2DGfohn/Pt46uueAMXRD/GVNjCjJFpYxeRWhSlP8DjYy1a1oDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5+7q8XBJE+JXRaSJGULrRcs6Ut0utIF6QKVK09tNDU2My0C lxq1nK2BSnlpmm3mVq6tuyni0FyamdqsMCutUNJZ5qyVWbPoy8uP3wPP8+Vl8MBr5GxGkZQi qJJkSgklIkSbYzIXf561QL5UUx0M3tEzBBRVWSlou2FBYL2lwWCgYS10jQ0i+PnkKQ4FeW0I St+9xuFWYw+CmopTFHS8nw6d3mEKXHnnKMi8UkXBs0/jGHTnGzCw2DfBG9NHApr1ZRgUDFBQ WJCJ+U8/Bj5TJQ2mjDDorbhEw/g7Kbh63CQ4i10k1LyKgIsl3RQ8rHER0HivF4OOB0UU9Fh/ k9Dc2ETAWG4ItF3QkXDdU0bBpzETDibvMA3tdUYMGo0zwZblbz39dYKER7o6DE6X38Sg82U1 gtozbzGwW90UOL2DGDjseTj8MDcg6M0doiE7x0dDoSYXwbnsfAKyuqPh53f/cvGoFDSXbQRc /+VGq2N5a4kV8c7BYZzPchzjf3ifU3zNmJHgH5dx/P1Lr2k+q/YVzRvtR3lHxSL+ysMBjC8d 8ZK8vfIsxdtHDDSvHerEeE9rK70ldLdoZYKgVKQKqsjYeJG8r+8LcaSDSbvqMxAZyEJrUQDD sVFcocGDTzLFLuRevPD95SB2HufQfSS1SMTgrHsq11X8Ek0GM9jNnNPiwbSIYQg2jDPmhE9q Mbuca6q1oX+dczmLre5vT4Df/zK3EpMcyEZzeqOF0COREU2pREGKpNREmUIZvUR9WJ6epEhb ciA50Y78D2Q6OX7hHhrtWFuPWAZJpombBubLA0lZqjo9sR5xDC4JEmvu+pU4QZZ+XFAl71Md VQrqehTCEJJg8fodQnwge0iWIhwWhCOC6n+KMQGzM5C5pSTmWZz+fNEXd/4K/bc9t324B4Wc CHb09c65sUC67Xe4q3XDTqdx+8buuBD91lDp4ER5V3hcWnwMFSdxaGOrPyekRPVHVE0otxx8 bo5a1XJ1na59VZ9hmVcYGY54kLdGF7krJ3K/OVQmCW6w7c15O/VO9oehIP06j7uwf+XIXQmh lsuki3CVWvYHX/MN/zwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 5d2fd2f6a46d..a45e5416f2dd 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -784,7 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -888,7 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); From patchwork Tue May 13 10:07:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085007 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3580-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXX55hhCz1yXB for ; Tue, 13 May 2025 20:19:33 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id CC2EB1B60E50 for ; Tue, 13 May 2025 10:18:46 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9D0D427AC45; Tue, 13 May 2025 10:08:06 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D8C126D4CA; Tue, 13 May 2025 10:08:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130886; cv=none; b=KdbXPWe3x5t26pnK4mmd85CC6Cs0ve4kTTLrGYaeck4pbh6rSl160OjsR4ccPK/q9YVu1FgZWCF/eY7dwlIDlPdMeW2rgiypYgKklCnAwJj6z8mXJduRwz3n8PC0oZpNUhWaVulvqYc1tm4I99yhPQuscaErOrS94kss3hYQR2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130886; c=relaxed/simple; bh=LO9R/1OiSZjbabE34wPUvkoIfLujUBBhnvS1gS9s3lM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=SPC+MKnl9Ht2ugmTmckxr+v2bqYPorz21/fZW5C75NI1DjOnvQupwATnLw1XHDjt4B6tK8M+VmE8fvrELuWakK7tgSrqyD7paqmXbBapJmjCpvoHVoRiy6qz4zfWmY12fkoUSPYYLQ9Ap/B5BcIEABj/3FWKktnYoJmwBGPObXY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-3c-682319f0bee8 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 24/43] dept: make dept able to work with an external wgen Date: Tue, 13 May 2025 19:07:11 +0900 Message-Id: <20250513100730.12664-25-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTZxjHfd9z+p5DR+GkEnYUF7ELMbqow1TyJDrDPhhOVBITtw9OE23k bK2WiuVSMdPQUYmrQBgLosilIBYClUsxETbqEEO5BagTsSDCwNsIN1NsN6S4FTa/PPnl/8/z e748LCV3StazGl2KqNeptAoipaWzweXb3qz7VP25rWw3eN9epqG43kbAVVeLwHbHiGGqIw6e +GYQLPUNUFBY4EJQPvGMgjvOMQSO6h8IPHoRAoPeeQLdBVcIZN6sJ/Bw2o9h9Go+hlp7PIxb X9HQm1eBoXCKwI3CTBwYf2JYtNYwYM2IgsnqIgb8E9HQPTYkAcfIZ3C9dJRAq6ObBmfzJIZH vxQTGLP9I4FeZxcNvtwIcP2UI4HbcxUEpn1WCqzeeQZ+b7NgcFrCocEUEGYtvJdAZ04bhqzK RgyDw78iuHf5Dwx22xCBB94ZDE32AgreVXUgmMydZeBS9iIDN4y5CK5cukqDaXQXLP0duFzy NhqMZQ003F4eQrFfCLZSGxIezMxTgqnJILzzPiaCw2ehhZ4KXmgpesYIpnsjjGCxpwpN1VuF m61TWCj3eCWCveZHItg9+Yxgnh3Ewlx/P3NowzfSPQmiVpMm6nfsPSFVL1iOJbXHnpt7uURn oBalGQWxPKfkp3uyyQc2jlvpFSbcZt7tXqRWOIyL5JtyXknMSMpS3NBH/JOSYWRGLLuWi+fv v1710FwUX9XpwSss42L4jLLl/50b+dqGtlVPUCBfrupf9cu5XXyepZZecfLcz0H8w/du9N/C Ov5+tZvOQzILWlOD5BpdWqJKo1VuV6frNOe2nzyTaEeB77Je8B9tRh7X4XbEsUgRLOua2qSW S1RpyemJ7YhnKUWYzHg3EMkSVOnnRf2Z4/pUrZjcjiJYWvGxbKfPkCDnvlOliKdFMUnUf2gx G7Q+A+luXRt53KgdPpu/P7vz6wil7Pu4jnJXZPFu/yfPvWf9fXAxZiIpdZ89lZO3hu/bJjVN 9nXEXXt6aoYYgpvdf2UeyjvwWoXpLUdkvuPKyg17B6QxX2Xl9I62bukJrWfIRF3JwfAWMbay bqDI0/hlisHA3A0ZXzgR6jDHfPvbQrZJQSerVdFbKX2y6l/jpQvDWQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x05nv6XxoyWuNSaiTXw21rLZ+gnN2BhrdNOPu/WAu0qx tjvlKUqxnKe4wpXuKHeZQqSb45hE6UmiRhynrLo4PXBl/vnstfd7n9fnnw9L+JRSM1llUrKo SpInyGgJKYlenrnwx4xAxWKjRgauwaMkXCw30dBw04jAVKnF4HgcCS1DTgTDL14SoCtoQFDU 9Y6ASlsngprSgzQ0fpwCTa4+GuwFx2nIvFJOw6tvIxg6zpzCYDSvg/eGHhKe5xVj0DlouKDL xJ7xBYPbUMaAQRME3aXnGRjpCgV7ZzMF1kI7BTXtwXDuUgcN92vsJNiqujE03r1IQ6fpDwXP bU9JGMr1g4b8HApu9BbT8G3IQIDB1cfA61o9Bpt+GlRkeayHB8YoeJJTi+Hw1VsYmtruIXhw 9AMGs6mZBqvLicFiLiDgd8ljBN253xk4dMLNwAVtLoLjh86QkNURBsO/PJcLB0NBe7mChBuj zSgiXDBdMiHB6uwjhCzLPuG36w0t1AzpSeFZMS9Un3/HCFkP2hlBb04RLKXzhSv3HVgo6ndR grnsGC2Y+08xQvb3Jiz01tcz6/23SlbEiQnKVFG1KDxWohjQx+ypi0jr/TRMalD1kmzkxfLc El773kCOM83N5Vtb3cQ4+3KzeUtOD5WNJCzBNU/mWwrbUDZi2ancOv7R54ldkgviS57043GW ckt5zeVR+p8zgDdW1E54vDz5aEn9hN+HC+Pz9EYyD0n0aFIZ8lUmpSbKlQlhIep4RXqSMi1k x+5EM/L8jyFjJL8KDTZG1iGORTJv6VPHHIUPJU9VpyfWIZ4lZL5S7R1PJI2Tp+8XVbu3q1IS RHUd8mNJ2XRp1GYx1ofbJU8W40Vxj6j632LWa6YGHTCcDrd4fU2ztlUfGbRAFJ46/OjO7cqg kJiVPzUfwV2++tx196qRs30x88KqVkRleOuutWacdG6ZNXv6WHRrhsO6rGTNm4i9ge0BwWuT C1bt9AtyPuyI3rajRSpZ+nZX+Kz4ziqqJ3DDRvuCX3Fvv2z6fDDSv31At7xnS4pvkaLLFiAj 1Qp56HxCpZb/BVRCAJE7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: There is a case where the total map size of waits of a class is so large. For instance, PG_locked is the case if every struct page embeds its regular map for PG_locked. The total size for the maps will be 'the # of pages * sizeof(struct dept_map)', which is too big to accept. Keep the minimum data in the case, timestamp called 'wgen', that dept uses. Make dept able to work with the wgen instead of whole regular map. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 6 +++--- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 38 insertions(+), 16 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 58362bd2c4ad..10536418ab41 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -371,6 +371,13 @@ struct dept_wait_hist { unsigned int ctxt_id; }; +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen { + unsigned int wgen; +}; + extern void dept_on(void); extern void dept_off(void); extern void dept_init(void); @@ -380,6 +387,7 @@ extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn, long timeout); @@ -388,8 +396,8 @@ extern void dept_clean_stage(void); extern void dept_ttwu_stage_wait(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -416,6 +424,7 @@ extern void dept_hardirqs_off(void); #else /* !CONFIG_DEPT */ struct dept_key { }; struct dept_map { }; +struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } @@ -428,6 +437,7 @@ struct dept_map { }; #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) #define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0) @@ -436,8 +446,8 @@ struct dept_map { }; #define dept_ttwu_stage_wait(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index b92bc8c988c9..b7b0f358646f 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -24,7 +24,7 @@ #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -48,9 +48,9 @@ #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) -#define sdt_request_event(m) dept_request_event(m) +#define sdt_request_event(m) dept_request_event(m, NULL) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 75fe64f86ee5..27ca8f723ccc 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2171,6 +2171,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + ewg->wgen = 0U; +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2356,7 +2361,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, struct dept_map *real_m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int wg) { struct dept_class *c; struct dept_key *k; @@ -2378,7 +2383,7 @@ static void __dept_event(struct dept_map *m, struct dept_map *real_m, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c) - do_event(m, real_m, c, READ_ONCE(m->wgen), ip, e_fn); + do_event(m, real_m, c, wg, ip, e_fn); } void dept_wait(struct dept_map *m, unsigned long w_f, @@ -2603,7 +2608,7 @@ void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); exit: dept_exit(flags); } @@ -2782,10 +2787,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2798,18 +2804,22 @@ void dept_request_event(struct dept_map *m) */ flags = dept_enter_recursive(); + wg_p = ewg ? &ewg->wgen : &m->wgen; + wg = next_wgen(); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wg_p, wg); dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt = dept_task(); unsigned long flags; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2817,24 +2827,26 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; + wg_p = ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); return; } flags = dept_enter(); - __dept_event(m, m, e_f, ip, e_fn, false); + __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); dept_exit(flags); } From patchwork Tue May 13 10:07:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085012 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3581-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXY605yFz1yYN for ; Tue, 13 May 2025 20:20:26 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 518121B6052C for ; Tue, 13 May 2025 10:19:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1F76A27EC7D; Tue, 13 May 2025 10:08:08 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 155E727054F; Tue, 13 May 2025 10:08:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130887; cv=none; b=NdovQ3TalsUVzqMNMnHXPM5nAG7IkkHdF1W4Q8ooygKc3OswIhNQU2dkoLrbmuYFgdX8H5BqmYqgAMFR5aW4JSALTPocC+3dwmPRwgqSmObMzz0fT/3ynUK5yPRlGgUy/0caVN/ghAPFS/ArZxVaw6pbXK+OxwsULHMA6Y/gWNk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130887; c=relaxed/simple; bh=tIR4uGZ5TyiPjwbijfaMoNqyjNLvTfkFnTPvc6vFuyA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ohrmNMTz6EIQNpDA0aa21L2f5W7tIryxYV0knYrufvwA9X2QR84e72QQtDtBAJ42MO1c8JNRTRZG7i+d9kZqeDde0cXMWOZ54HC5tTA5OZI12eFzkipggw4V07QE3lG6KO6uDL/jMp2QN3eaffJWbfH9EV9otCermxvgl58P7R0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-4c-682319f04bc3 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 25/43] dept: track PG_locked with dept Date: Tue, 13 May 2025 19:07:12 +0900 Message-Id: <20250513100730.12664-26-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfT7fn52Oryt8/Ri5mI35UWTPDGNmvoZh/svPm766m0q7q6sM uyPph275kRtXuYor11GujChL1imthOrSTnJ+TFOadEcULq1/nr32fp7n9fzzsISsjprJqmLi RHWMIkpOS0hJn3/+km8zgpXLnWcJ8AymkpBTaqOh5XYJAluFHkNP3WZwensR/G56ToAxuwVB /rs3BFQ4uhBUF5+i4dWHSdDq6aehITuDhtOFpTS8+DKMwXX5AoYS+3Z4a/lEQmNWAQZjDw0m 42nsK58xDFmsDFh0C8BdfJWB4Xch0NDVTkF152K4kueioaq6gQTHfTeGVw9yaOiy/aWg0VFP gtcwC1rOZ1Jw62sBDV+8FgIsnn4GXtaYMTjM06As2SdM+f6HgqeZNRhSrt/B0Pr6IYJHqd0Y 7LZ2Gp54ejGU27MJ+FVUh8Bt6GPgzLkhBkx6A4KMM5dJSHaFwe+fvsu5gyGgv1ZGwq2RdrR+ rWDLsyHhSW8/ISSXJwi/PG20UO01k8KzAl6ovPqGEZIfdTKC2R4vlBcvEgqrerCQP+ChBLs1 jRbsAxcYIb2vFQtfm5uZnbPDJWsixCiVVlQvW3dQomwrOhxr2JtoeLBZh+5uTUd+LM+t5NPc Q8Q43/04+J9pbiHf0TGWB3JBfHnmJyodSViCa5/IO3Nfo3TEsgHcar7gZ/joDMkt4O1DJmaU pdwq/mbbc2rMOZcvKav57/Hz5SNFzeQoy7gwPstcQo46ee6iH2/trKDHFmbwj4s7yCwkNaMJ ViRTxWijFaqolUuVSTGqxKWHjkbbke+5LCeG99xHAy27axHHIrm/tL5nnlJGKbSapOhaxLOE PFCqv+eLpBGKpGOi+ugBdXyUqKlFs1hSPl0a6k2IkHGRijjxiCjGiurxLmb9ZurQpf1TIkI3 VU75M4w8+0LrS1OOvfhRNdm11hMQPO1k4rbKhydN7OfvTYlNJhx+Y/YGWpMXpptwYkfmTseu SItsk/59ArZumKNy48CtW7TdamOQMXKJK9Xp3C8LzvXfeKW133ldXdb74eUKbeOkQXb+SPwe S32hYV3s8aycqbqDcQFyUqNUhCwi1BrFP3HkxPlYAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHWc/afxkhrM0NRFln8WwmfWbp9oszGx19ONOV7jjyDyU TqPcLSZNj9fFXeokvzRKWevWkYdKz1mi0KSUh+7WEbmYfz577f3+7PX558PgnhbSm1HGHRXU cXKVjJIQkm2rk5Z+mbNIsdxoWQKO0QsEZN+xUtBUUozAei8Rg4G6UOhwDiH4+aIRh4z0JgT5 va9xuGfvQVBdeI6ClvfTodUxQkF9eioFSQV3KHg5OI5B97UrGBSLW+GNuZ+AZ2kmDDIGKMjK SMLc4yMGLnMRDeYEH+grzKRhvDcQ6nvaSbDl1JNQ/cofrud2U1BVXU+A/UEfBi2V2RT0WCdI eGZ/QoDTMBeaLutJuD1somDQacbB7BihobnGiIHdOAtKdW5r8vffJDzW12CQfOMuBq1dDxE8 uvAWA9HaToHNMYRBmZiOww9LHYI+w2cazl9y0ZCVaECQev4aAbruYPg55r6cMxoIiXmlBNz+ 1Y7Wr+WtuVbE24ZGcF5Xdpz/4Wij+GqnkeCfmji+IvM1zesevaJ5o3iMLyv04wuqBjA+/5uD 5MWiixQvfrtC8ymfWzF+uKGBDp+3W7ImWlAptYJ62dooiaLNsv+wYc8JQ2VoAirfnII8GI4N 4so/jOKTTLG+XGen6y97sQu4Mn0/mYIkDM62T+U6crpQCmKYmWwIZxrbPblDsD6c6MqiJ1nK ruJutTWS/5zzueLSmr8eD3f+y9JATLInG8ylGYuJNCQxoilFyEsZp42VK1XBAZoYRXyc8kTA vkOxInK/j/n0+OUHaLQltBaxDJJNkz4ZWKjwJOVaTXxsLeIYXOYlTbzvjqTR8viTgvpQpPqY StDUorkMIZst3bRTiPJkD8iPCjGCcFhQ/28xxsM7Ab1tjszPNsdoz064/L3oyL16bfOMcd93 U6qWWJOjp5XsstoPDs7a0jtzf0K4RZ9aG3pznS4oKL23fDtpeh7lr2O88y5Fq2xUY8e7vRFf w/wqXF+ViPqUe6Zuw4qNYyMTtiOaiB2LK0OOOLMPhIWH+E09FT4sOgv6G8OulihXOtseywiN Qh7oh6s18j9Xr7hVOgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes dept able to track PG_locked waits and events, which will be useful in practice. See the following link that shows dept worked with PG_locked and detected real issues in practice: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 2 + include/linux/page-flags.h | 125 +++++++++++++++++++++++++++++++++---- include/linux/pagemap.h | 7 ++- mm/filemap.c | 26 ++++++++ mm/mm_init.c | 2 + 5 files changed, 149 insertions(+), 13 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 56d07edd01f9..7e9d63cef28a 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -20,6 +20,7 @@ #include #include #include +#include #include @@ -224,6 +225,7 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif + struct dept_ext_wgen pg_locked_wgen; } _struct_page_alignment; /* diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index e6a21b62dcce..73cb8a1ad4f3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -194,6 +194,61 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H +#ifdef CONFIG_DEPT +#include +#include + +extern struct dept_map pg_locked_map; + +/* + * Place the following annotations in its suitable point in code: + * + * Annotate dept_page_set_bit() around firstly set_bit*() + * Annotate dept_page_clear_bit() around clear_bit*() + * Annotate dept_page_wait_on_bit() around wait_on_bit*() + */ + +static inline void dept_page_set_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_request_event(&pg_locked_map, &p->pg_locked_wgen); +} + +static inline void dept_page_clear_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_event(&pg_locked_map, 1UL, _RET_IP_, __func__, &p->pg_locked_wgen); +} + +static inline void dept_page_wait_on_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_wait(&pg_locked_map, 1UL, _RET_IP_, __func__, 0, -1L); +} + +static inline void dept_folio_set_bit(struct folio *f, int bit_nr) +{ + dept_page_set_bit(&f->page, bit_nr); +} + +static inline void dept_folio_clear_bit(struct folio *f, int bit_nr) +{ + dept_page_clear_bit(&f->page, bit_nr); +} + +static inline void dept_folio_wait_on_bit(struct folio *f, int bit_nr) +{ + dept_page_wait_on_bit(&f->page, bit_nr); +} +#else +#define dept_page_set_bit(p, bit_nr) do { } while (0) +#define dept_page_clear_bit(p, bit_nr) do { } while (0) +#define dept_page_wait_on_bit(p, bit_nr) do { } while (0) +#define dept_folio_set_bit(f, bit_nr) do { } while (0) +#define dept_folio_clear_bit(f, bit_nr) do { } while (0) +#define dept_folio_wait_on_bit(f, bit_nr) do { } while (0) +#endif + #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); @@ -415,27 +470,51 @@ static __always_inline bool folio_test_##name(const struct folio *folio) \ #define FOLIO_SET_FLAG(name, page) \ static __always_inline void folio_set_##name(struct folio *folio) \ -{ set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + set_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_set_bit(folio, PG_##name); \ +} #define FOLIO_CLEAR_FLAG(name, page) \ static __always_inline void folio_clear_##name(struct folio *folio) \ -{ clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + clear_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_clear_bit(folio, PG_##name); \ +} #define __FOLIO_SET_FLAG(name, page) \ static __always_inline void __folio_set_##name(struct folio *folio) \ -{ __set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + __set_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_set_bit(folio, PG_##name); \ +} #define __FOLIO_CLEAR_FLAG(name, page) \ static __always_inline void __folio_clear_##name(struct folio *folio) \ -{ __clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + __clear_bit(PG_##name, folio_flags(folio, page)); \ + dept_folio_clear_bit(folio, PG_##name); \ +} #define FOLIO_TEST_SET_FLAG(name, page) \ static __always_inline bool folio_test_set_##name(struct folio *folio) \ -{ return test_and_set_bit(PG_##name, folio_flags(folio, page)); } +{ \ + bool __ret = test_and_set_bit(PG_##name, folio_flags(folio, page)); \ + \ + if (!__ret) \ + dept_folio_set_bit(folio, PG_##name); \ + return __ret; \ +} #define FOLIO_TEST_CLEAR_FLAG(name, page) \ static __always_inline bool folio_test_clear_##name(struct folio *folio) \ -{ return test_and_clear_bit(PG_##name, folio_flags(folio, page)); } +{ \ + bool __ret = test_and_clear_bit(PG_##name, folio_flags(folio, page)); \ + \ + if (__ret) \ + dept_folio_clear_bit(folio, PG_##name); \ + return __ret; \ +} #define FOLIO_FLAG(name, page) \ FOLIO_TEST_FLAG(name, page) \ @@ -450,32 +529,54 @@ static __always_inline int Page##uname(const struct page *page) \ #define SETPAGEFLAG(uname, lname, policy) \ FOLIO_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline void SetPage##uname(struct page *page) \ -{ set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define CLEARPAGEFLAG(uname, lname, policy) \ FOLIO_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline void ClearPage##uname(struct page *page) \ -{ clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define __SETPAGEFLAG(uname, lname, policy) \ __FOLIO_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline void __SetPage##uname(struct page *page) \ -{ __set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define __CLEARPAGEFLAG(uname, lname, policy) \ __FOLIO_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline void __ClearPage##uname(struct page *page) \ -{ __clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define TESTSETFLAG(uname, lname, policy) \ FOLIO_TEST_SET_FLAG(lname, FOLIO_##policy) \ static __always_inline int TestSetPage##uname(struct page *page) \ -{ return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_set_bit(PG_##lname, &policy(page, 1)->flags);\ + if (!ret) \ + dept_page_set_bit(page, PG_##lname); \ + return ret; \ +} #define TESTCLEARFLAG(uname, lname, policy) \ FOLIO_TEST_CLEAR_FLAG(lname, FOLIO_##policy) \ static __always_inline int TestClearPage##uname(struct page *page) \ -{ return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_clear_bit(PG_##lname, &policy(page, 1)->flags);\ + if (ret) \ + dept_page_clear_bit(page, PG_##lname); \ + return ret; \ +} #define PAGEFLAG(uname, lname, policy) \ TESTPAGEFLAG(uname, lname, policy) \ diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 26baa78f1ca7..f31cd68f2935 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1097,7 +1097,12 @@ void folio_unlock(struct folio *folio); */ static inline bool folio_trylock(struct folio *folio) { - return likely(!test_and_set_bit_lock(PG_locked, folio_flags(folio, 0))); + bool ret = !test_and_set_bit_lock(PG_locked, folio_flags(folio, 0)); + + if (ret) + dept_page_set_bit(&folio->page, PG_locked); + + return likely(ret); } /* diff --git a/mm/filemap.c b/mm/filemap.c index 7b90cbeb4a1a..cab03b41add2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -48,6 +48,7 @@ #include #include #include +#include #include #include #include "internal.h" @@ -1145,6 +1146,7 @@ static int wake_page_function(wait_queue_entry_t *wait, unsigned mode, int sync, if (flags & WQ_FLAG_CUSTOM) { if (test_and_set_bit(key->bit_nr, &key->folio->flags)) return -1; + dept_page_set_bit(&key->folio->page, key->bit_nr); flags |= WQ_FLAG_DONE; } } @@ -1228,6 +1230,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, if (wait->flags & WQ_FLAG_EXCLUSIVE) { if (test_and_set_bit(bit_nr, &folio->flags)) return false; + dept_page_set_bit(&folio->page, bit_nr); } else if (test_bit(bit_nr, &folio->flags)) return false; @@ -1235,6 +1238,9 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, return true; } +struct dept_map __maybe_unused pg_locked_map = DEPT_MAP_INITIALIZER(pg_locked_map, NULL); +EXPORT_SYMBOL(pg_locked_map); + static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) { @@ -1246,6 +1252,8 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, unsigned long pflags; bool in_thrashing; + dept_page_wait_on_bit(&folio->page, bit_nr); + if (bit_nr == PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { delayacct_thrashing_start(&in_thrashing); @@ -1339,6 +1347,23 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, break; } + /* + * dept_page_set_bit() might have been called already in + * folio_trylock_flag(), wake_page_function() or somewhere. + * However, call it again to reset the wgen of dept to ensure + * dept_page_wait_on_bit() is called prior to + * dept_page_set_bit(). + * + * Remind dept considers all the waits between + * dept_page_set_bit() and dept_page_clear_bit() as potential + * event disturbers. Ensure the correct sequence so that dept + * can make correct decisions: + * + * wait -> acquire(set bit) -> release(clear bit) + */ + if (wait->flags & WQ_FLAG_DONE) + dept_page_set_bit(&folio->page, bit_nr); + /* * If a signal happened, this 'finish_wait()' may remove the last * waiter from the wait-queues, but the folio waiters bit will remain @@ -1496,6 +1521,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + dept_page_clear_bit(&folio->page, PG_locked); if (folio_xor_flags_has_waiters(folio, 1 << PG_locked)) folio_wake_bit(folio, PG_locked); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 327764ca0ee4..39cf0bc355ba 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -31,6 +31,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -586,6 +587,7 @@ void __meminit __init_single_page(struct page *page, unsigned long pfn, atomic_set(&page->_mapcount, -1); page_cpupid_reset_last(page); page_kasan_tag_reset(page); + dept_ext_wgen_init(&page->pg_locked_wgen); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL From patchwork Tue May 13 10:07:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085025 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3582-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXc95kHrz1yYN for ; Tue, 13 May 2025 20:23:05 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 000239847B1 for ; Tue, 13 May 2025 10:19:00 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2FBAB27F178; Tue, 13 May 2025 10:08:08 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A54982741DA; Tue, 13 May 2025 10:08:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130887; cv=none; b=JcLG5yG0VF4bzomXbrHNJIuzfozkauDRS2KiACrpS7C3x+wL/Az3Ypg6RnEVxlvxPpGz9UH9fuSVx/cigx1oWrbA9aO/+Q9ur55ljIx39MsY1FYnxsUdCaxZO4Tr/Dpfi612m7XwHFR98c2ro8DYlUxJTOOtYfYfaf0n5eq4m2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130887; c=relaxed/simple; bh=e73Qv2acC8mG3MwWR0Y7e7Ex09qWOpXzcPGl5zF8wRc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=WlSXOhXP9tEjQBT5u9NGHfCwwvRRJr4z9V1aD5lgD7hGoey/6rQJK41tu/bHkZGoyorNiZlY1/98eI9LBDeUB5r0u0IMyZ5JXaAYkpmPDe+VBYFLLYhIZ1pfb7pbvjfPoNEBP16QdmaVqqWcEFo+uPfdMvx8WDmp9iDA3XdSvyc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-5c-682319f01bfd From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 26/43] dept: print staged wait's stacktrace on report Date: Tue, 13 May 2025 19:07:13 +0900 Message-Id: <20250513100730.12664-27-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+917d+/danWbUbcHVCt7kmX2OIFF9E/XIgiEoPqjll7ayGls ZhlEzsxMU6x8lKXNadehU+cmaI/Z0lyaZJY2H5mphbh8kbWRr2or+ufw4XvO+Zx/Do3LXoqW 0KrIaF4TqYiQkxJCMjLHsOnb4lXKLUK/GNw/kgi4X24ioaWsBIGpUoeBq34/tHuGEUy9foND dmYLgvy+jzhUOnoQ2IzxJLR+mQtt7jESGjNTSLhSUE7C26FpDLqzbmFQYjkEn4QBAprSDRhk u0i4l30F85ZBDCaEYgqEOH/oN+ZQMN0XCI09ThHYujbC3bxuEp7aGglwVPdj0Pr4Pgk9pt8i aHI0EOBJWwotN1NFUDpqIGHII+AguMcoeGfXY+DQLwRzgleY+P2XCF6m2jFILKzAoK3zCYKa pF4MLCYnCXXuYQyslkwcJovqEfSnjVBw9cYEBfd0aQhSrmYRkNC9HaZ+ei/n/ggE3QMzAaUz TrR3N2fKMyGubngM5xKs57lJ93uSs3n0BPfKwHKPcj5SXEJNF8XpLec4q3EDV/DUhXH5424R Zym+TnKW8VsUlzzShnGjzc3U4WXHJMHhfIQqhtds3nNSomy3VmBnp3dfKK3zUHHIFZiMxDTL bGOHUozUfy4qa0U+Jpm1bEfHBO7jBcwK1po6IEpGEhpnnLPZ9tzOv0N+TAibpnvoXaZpgvFn y3plvljK7GB/Vj8i/zmXsyVm+1+P2JvPFDUTPpYx29l0fQnhc7LMbTGbMdCA/i0sZp8bO4h0 JNWjWcVIpoqMUStUEdsClLGRqgsBYVFqC/K+l3Bp+ng1Gm8JrUUMjeRzpA2ulUqZSBGjjVXX IpbG5QukuipvJA1XxF7kNVEnNOcieG0tWkoT8kXSrZ7z4TLmtCKaP8PzZ3nN/y5Gi5fEIcN8 v9G+cedkFh1QKL1W2BW/PlQWFbzrYMqRAiIo+E5l7458G1MtO2X+tmgw+9jIgdfpuGC43CBv Lso4U3PUNbTs64dnjozQF2vC9q0zh123xSdSU1XqlQGqfUTQYBP9XVWflzNPOHVYnZs0O2TG z9F2+7F25+DlIPVn+2r7E0mInNAqFYEbcI1W8QfkDbBfWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTcRSH/b93V6uXJfXWkGoR3UgdZRyoJCLqzS70qaKgXPnSlm7JlpZF sKWLshQz1PKSS2vanGZbF60UUTSXpKvZzPBSo6KRlzC3Ulc2i74cHp7f4Xe+HAaX3CUXMCrN SUGrUSTKKBEh2r0+bfW3+UuUUYHCKPCNXSSg6J6VAmd1JQLrAwMG3pZt0O0fRDD5shOH/Fwn glsf+nB40NqPoL7iPAWuj7OgyzdCgSP3MgVpZfcoePU1gEFvXg4GlbZdMGD+TEB7dikG+V4K CvPTsOD4gsG42UKDWb8UPBUFNAQ+yMHR7yahudhBQv27VXDjZi8Fz+odBLTWejBwPSmioN86 RUJ7axsB/iwpOK9mklA1XErBV78ZB7NvhIbXjSYMWk1zoSY92Hrh+28Snmc2YnDh9n0Munqe Imi4+B4Dm9VNQbNvEAO7LReHifIWBJ6sIRqMV8ZpKDRkIbhszCMgvTcaJn8GLxePycFQUkNA 1S832hTDW29aEd88OILz6fZT/ITvDcXX+00E/6KU4+sK+mg+veEdzZtsyby9YiVf9syL8bdG fSRvs1yieNtoDs1nDHVh/HBHB70n/IBoQ7yQqEoRtJExcSJlt/0+lhTYeLqq2U/rkVeegUIZ jl3LlVe70DRT7DLu7dtxfJrD2EWcPfMzmYFEDM66Z3DdxT1/l+aw27kswx06AzEMwS7lqt9L prWYXcf9rK2j/nUu5CprGv/2hAb9r/IOYpolbDSXbaokspHIhEIsKEylSVErVInREboEZapG dTri6Am1DQUfyHwucLUWjbm2NSGWQbKZ4jbvYqWEVKToUtVNiGNwWZjY8DioxPGK1DOC9sRh bXKioGtCUoaQzRPH7hPiJOwxxUkhQRCSBO3/FGNCF+iR4F+2afMwEVMmzdsgd66J7VIP7Ij8 9Can83iJp+WR89VUiHL5jyN49FmPfufE+CzjZITP6eg71ydv2rJxxHh7d9Qld8OiFZtfs1O/ 8VWWpHBARcY8zejW2LRId/n+gzV7CjpD7HH6hwWzLeShnusDlEv4JN17LVktTWkLL24YkhE6 pUK+EtfqFH8ALWISYDwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Currently, print nothing about what event wakes up in report. However, it makes hard to interpret dept's report. Make it print wait's stacktrace that the event wakes up. Signed-off-by: Byungchul Park --- include/linux/dept.h | 5 ++++ include/linux/sched.h | 2 ++ kernel/dependency/dept.c | 59 ++++++++++++++++++++++++++++++++++------ 3 files changed, 57 insertions(+), 9 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 10536418ab41..551168220954 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -226,6 +226,11 @@ struct dept_ecxt { */ unsigned long event_ip; struct dept_stack *event_stack; + + /* + * wait that this event ttwu + */ + struct dept_stack *ewait_stack; }; }; }; diff --git a/include/linux/sched.h b/include/linux/sched.h index 541ddacdc3d0..43927e61921b 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -865,6 +865,7 @@ struct dept_task { const char *stage_w_fn; unsigned long stage_ip; bool stage_timeout; + struct dept_stack *stage_wait_stack; arch_spinlock_t stage_lock; /* @@ -906,6 +907,7 @@ struct dept_task { .stage_w_fn = NULL, \ .stage_ip = 0UL, \ .stage_timeout = false, \ + .stage_wait_stack = NULL, \ .stage_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 27ca8f723ccc..278194093108 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -522,6 +522,7 @@ static void initialize_ecxt(struct dept_ecxt *e) e->enirqf = 0UL; e->event_ip = 0UL; e->event_stack = NULL; + e->ewait_stack = NULL; } SET_CONSTRUCTOR(ecxt, initialize_ecxt); @@ -577,6 +578,8 @@ static void destroy_ecxt(struct dept_ecxt *e) put_stack(e->ecxt_stack); if (e->event_stack) put_stack(e->event_stack); + if (e->ewait_stack) + put_stack(e->ewait_stack); } SET_DESTRUCTOR(ecxt, destroy_ecxt); @@ -793,6 +796,11 @@ static void print_dep(struct dept_dep *d) pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); print_ip_stack(e->event_ip, e->event_stack); + + if (valid_stack(e->ewait_stack)) { + pr_warn("(wait to wake up)\n"); + print_ip_stack(0, e->ewait_stack); + } } if (!irqf) { @@ -806,6 +814,11 @@ static void print_dep(struct dept_dep *d) pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); print_ip_stack(e->event_ip, e->event_stack); + + if (valid_stack(e->ewait_stack)) { + pr_warn("(wait to wake up)\n"); + print_ip_stack(0, e->ewait_stack); + } } } @@ -1656,7 +1669,8 @@ static void add_wait(struct dept_class *c, unsigned long ip, static struct dept_ecxt_held *add_ecxt(struct dept_map *m, struct dept_class *c, unsigned long ip, const char *c_fn, - const char *e_fn, int sub_l) + const char *e_fn, int sub_l, + struct dept_stack *ewait_stack) { struct dept_task *dt = dept_task(); struct dept_ecxt_held *eh; @@ -1690,6 +1704,7 @@ static struct dept_ecxt_held *add_ecxt(struct dept_map *m, e->class = get_class(c); e->ecxt_ip = ip; e->ecxt_stack = ip ? get_current_stack() : NULL; + e->ewait_stack = ewait_stack ? get_stack(ewait_stack) : NULL; e->event_fn = e_fn; e->ecxt_fn = c_fn; @@ -1796,7 +1811,7 @@ static int find_hist_pos(unsigned int wg) static void do_event(struct dept_map *m, struct dept_map *real_m, struct dept_class *c, unsigned int wg, unsigned long ip, - const char *e_fn) + const char *e_fn, struct dept_stack *ewait_stack) { struct dept_task *dt = dept_task(); struct dept_wait_hist *wh; @@ -1824,7 +1839,7 @@ static void do_event(struct dept_map *m, struct dept_map *real_m, */ if (find_ecxt_pos(real_m, c, false) != -1) return; - eh = add_ecxt(m, c, 0UL, NULL, e_fn, 0); + eh = add_ecxt(m, c, 0UL, NULL, e_fn, 0, ewait_stack); if (!eh) return; @@ -2361,7 +2376,8 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, struct dept_map *real_m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map, unsigned int wg) + bool sched_map, unsigned int wg, + struct dept_stack *ewait_stack) { struct dept_class *c; struct dept_key *k; @@ -2383,7 +2399,7 @@ static void __dept_event(struct dept_map *m, struct dept_map *real_m, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c) - do_event(m, real_m, c, wg, ip, e_fn); + do_event(m, real_m, c, wg, ip, e_fn, ewait_stack); } void dept_wait(struct dept_map *m, unsigned long w_f, @@ -2499,6 +2515,9 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_w_fn = NULL; dt->stage_ip = 0UL; dt->stage_timeout = false; + if (dt->stage_wait_stack) + put_stack(dt->stage_wait_stack); + dt->stage_wait_stack = NULL; } void dept_clean_stage(void) @@ -2562,6 +2581,14 @@ void dept_request_event_wait_commit(void) wg = next_wgen(); WRITE_ONCE(dt->stage_m.wgen, wg); + + /* + * __schedule() can be hit multiple times between + * dept_stage_wait() and dept_clean_stage(). In that case, + * keep the first stacktrace only. That's enough. + */ + if (!dt->stage_wait_stack) + dt->stage_wait_stack = get_current_stack(); arch_spin_unlock(&dt->stage_lock); __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); @@ -2580,6 +2607,7 @@ void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) struct dept_map m; struct dept_map *real_m; bool sched_map; + struct dept_stack *ewait_stack; if (unlikely(!dept_working())) return; @@ -2598,6 +2626,10 @@ void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) m = dt_req->stage_m; sched_map = dt_req->stage_sched_map; real_m = dt_req->stage_real_m; + ewait_stack = dt_req->stage_wait_stack; + if (ewait_stack) + get_stack(ewait_stack); + __dept_clean_stage(dt_req); arch_spin_unlock(&dt_req->stage_lock); @@ -2608,8 +2640,12 @@ void dept_ttwu_stage_wait(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); + __dept_event(&m, real_m, 1UL, ip, "try_to_wake_up", sched_map, + m.wgen, ewait_stack); exit: + if (ewait_stack) + put_stack(ewait_stack); + dept_exit(flags); } @@ -2689,7 +2725,7 @@ void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); - if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l, NULL)) goto exit; /* @@ -2741,7 +2777,7 @@ void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); - if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l, NULL)) goto exit; missing_ecxt: dt->missing_ecxt++; @@ -2841,7 +2877,7 @@ void dept_event(struct dept_map *m, unsigned long e_f, flags = dept_enter(); - __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); + __dept_event(m, m, e_f, ip, e_fn, false, READ_ONCE(*wg_p), NULL); /* * Keep the map diabled until the next sleep. @@ -2913,6 +2949,11 @@ void dept_task_exit(struct task_struct *t) dt->stack = NULL; } + if (dt->stage_wait_stack) { + put_stack(dt->stage_wait_stack); + dt->stage_wait_stack = NULL; + } + for (i = 0; i < dt->ecxt_held_pos; i++) { if (dt->ecxt_held[i].class) { put_class(dt->ecxt_held[i].class); From patchwork Tue May 13 10:07:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3584-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXdQ5zL6z1yYN for ; Tue, 13 May 2025 20:24:10 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D673F3B2E32 for ; Tue, 13 May 2025 10:19:42 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 59AB225CC69; Tue, 13 May 2025 10:08:09 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9F980270EBB; Tue, 13 May 2025 10:08:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130888; cv=none; b=ExLJ3Q6dCgFhT73ypmpZhr4IOQPtNoY6Mb8qb0Ok3JN/F1uCY2fA2uxYbEJxEcl7djaZMy5eI7rYybAIQbkLQfjmhAtsCHIUM1sOjK/LwJeDL7IZgWZwrebKUMoYyyUbj8F3SC/XE/OBQEZxEeoddTgQ7xgfB9Eg56ywZKD1oHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130888; c=relaxed/simple; bh=XT7wM+AY+fy9b7R9FSPBT0zn1xc+Ht0DfCIhlHeaS6U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=G+mSbZqZ8gdbO7f0gB2evqiKY44zXH5UIUSTZBwOByyOX9sHiWsc0Ec2Dde8VB7N1ghRp4fZcJKjlt6H0RNBeIHUAKxPmSAUbtUJ21088HzFkQ187NX6XNyKBUj0WQTytMTAdtPq7/uXLLjfxpVsLjQRepRy7nRQ5sCAv0K0sog= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-6c-682319f0f394 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 27/43] locking/lockdep: prevent various lockdep assertions when lockdep_off()'ed Date: Tue, 13 May 2025 19:07:14 +0900 Message-Id: <20250513100730.12664-28-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG/Z9z9j9n0+lpRR2zKBcWFWVGl1fo+iE6UFHRlyiiRju4lTdm mlqB5rSrwwqzMnNaLdF52yq7zUzbakVqacuFWUqNRupI20qzy2b05eXH8zzv8355GVL2RDSV USceEDSJing5llCSgZCyBV/DZ6kW/egNBe+34xRcrjViaK+pQmC8mU2A27oe3vj6Efx80UZC UWE7grLedyTctPUgsFQcxdDxMRQ6vR4M9sJTGHKu1mJ4+WWMgO7zZwmoMm2C9wYXBc8Lygko cmMoLsoh/OMzASOGShoMWVHQV3GJhrHeGLD3OERgeTsfLl7pxvDAYqfAdqePgI57lzH0GP+I 4LntKQU+XQS0n8kXQfVgOYYvPgMJBq+HhldNegJs+slQp/UX5g3/FsGT/CYC8q7VE9DpvI+g 8fgHAkxGB4YWbz8BZlMhCaM3rAj6dAM05J4eoaE4W4fgVO55CrTdS+HnD//lkm8xkF1aR0H1 Lwdas5I3XjEivqXfQ/Ja80F+1Psa8xafnuKflXP83UvvaF7b+Jbm9aZU3lwxj7/6wE3wZUNe EW+qPIF509BZmj850Enwg62t9JZpOyQrlEK8Ok3QRK/aI1FZPbOTnRPSDZ58nIUMYSeRmOHY Jdwnx6joP7urbTjAmJ3DdXWNkAGexM7kzPkuf0bCkKwjmHtT4kQBYyKr5Oxu53iIYqO4PJ11 vEjKLuPMvbfxv9IZXFVd03hG7Nd/3WilAixjl3IF+ioqUMqxpWJOV2gh/y2Ec48quqgCJNWj oEokUyemJSjU8UsWqjIS1ekL9yYlmJD/vQxHxnbeQUPt25oRyyB5iPSpO1IlEynSUjISmhHH kPJJ0uwGvyRVKjIyBU3Sbk1qvJDSjCIYSj5Futh3UClj4xQHhP2CkCxo/rsEI56ahTYr0ovw +61hNbaOmpCcksczcyfQccv/xLlmpRa7uClBLYe2Xo9WN8LG79uKLcbu8qjIpE3O6RdWnTuy OpKp23cscx8OdZpXNsRqS4IOmetnTxu8tWttRKz64ZhbvD08eMO6CP0tTcLkT9KWWG2rVan1 Hr4+PFFtdw1Fz2jLJBvmyqkUlSJmHqlJUfwFNi2KRloDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0xTZxjG/c7lO6VaPSLRo8RbFZdgRIliXrPFGDXhi1HUxcS4LNEzOLMN F02LCMYLjEoQpSIJEhCwwlIQymAt8Q4hsFYqWkHKpVhRCEGrYI2jKIiX1mX/vPnleZ487z+P jA6+zi6SqZOSJU2SmKDEckYe82PmmncLV6jWXZ3cCL7xbAZK6kwYOv6qQWBqyKDAY42G3olR BJ8ePaahsKADwbXBZzQ02AYQNFb9gaFreDY4fV4M9oLzGDIr6jB0vpmmwH05n4Ia8y54bhxh oD2vnIJCD4YrhZmU/7yiYNJYzYExPQyGqoo5mB6MBPtADwutpXYWGvtXQ1GZG8O9RjsDtltD FHTdKcEwYPrKQrutjYEJfSh0XMplofZtOYY3E0YajD4vB0+aDRTYDPOhXudvzfr3Cwv3c5sp yPrzbwqcrrsImrJfUGA29WBo9Y1SYDEX0DBVaUUwpB/j4OyFSQ6uZOgRnD97mQGdOwo+ffR/ Lh2PhIyr9QzUfu5BWzYTU5kJkdZRL010luNkyteNSeOEgSEPygVyu/gZR3RN/RwxmI8RS1U4 qbjnoci19z6WmKvPYWJ+n8+RnDEnRd46HNyexb/If4qTEtQpkmbt5kNyldW76qhrbqrRm4vT kXFODgqSCfwGwVNrwwHG/A9CX98kHeAQfplgyR1hc5BcRvM9M4XeUhcKGPP4OMHucX0PMXyY kKW3sgFW8BsFy+AN/F/pUqGmvvl7Jsivf650MAEO5qOEPEMNk4fkBjSjGoWok1ISRXVCVIQ2 XpWWpE6NiD2SaEb+ARlPTV+6hca7olsQL0PKWYo2z3JVMCumaNMSW5Ago5UhioybfkkRJ6ad kDRHDmqOJUjaFhQqY5QLFDv2S4eC+cNishQvSUclzf8uJQtalI62Rx3I7l4x/OG6bv05Yn9Y uPv16X9KXu9fwxX1Fm/Y1NXtnGGIWXKh01Usth2ItuqHzfFzY71nvm7bY2tKrX8w9djd/8Ln NLVvXfX7SOfxrSsvrs7fHV7WGT4w9tu+tp8pW9Ho3odnfnU77j9dMPtlXKhzaqxhuy/WsX7p ywh+58mF4kwlo1WJkeG0Rit+A3/0fZE8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: lockdep provides APIs for assertion only if lockdep is enabled at the moment asserting to avoid unnecessary confusion, using the following condition, debug_locks && !this_cpu_read(lockdep_recursion). However, lockdep_{off,on}() are also used for disabling and enabling lockdep for a simular purpose. Add !lockdep_recursing(current) that is updated by lockdep_{off,on}() to the condition so that the assertions are aware of !__lockdep_enabled if lockdep_off()'ed. Signed-off-by: Byungchul Park --- include/linux/lockdep.h | 3 ++- kernel/locking/lockdep.c | 10 ++++++++++ 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index ef03d8808c10..c83fe95199db 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -303,6 +303,7 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); lockdep_assert_once(!current->lockdep_depth) #define lockdep_recursing(tsk) ((tsk)->lockdep_recursion) +extern bool lockdep_recursing_current(void); #define lockdep_pin_lock(l) lock_pin_lock(&(l)->dep_map) #define lockdep_repin_lock(l,c) lock_repin_lock(&(l)->dep_map, (c)) @@ -630,7 +631,7 @@ DECLARE_PER_CPU(int, hardirqs_enabled); DECLARE_PER_CPU(int, hardirq_context); DECLARE_PER_CPU(unsigned int, lockdep_recursion); -#define __lockdep_enabled (debug_locks && !this_cpu_read(lockdep_recursion)) +#define __lockdep_enabled (debug_locks && !this_cpu_read(lockdep_recursion) && !lockdep_recursing_current()) #define lockdep_assert_irqs_enabled() \ do { \ diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 6c984a55d5ed..d2805ce250cb 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -6889,3 +6889,13 @@ void lockdep_rcu_suspicious(const char *file, const int line, const char *s) warn_rcu_exit(rcu); } EXPORT_SYMBOL_GPL(lockdep_rcu_suspicious); + +/* + * For avoiding header dependency when using (struct task_struct *)current + * and lockdep_recursing() at the same time. + */ +noinstr bool lockdep_recursing_current(void) +{ + return lockdep_recursing(current); +} +EXPORT_SYMBOL_GPL(lockdep_recursing_current); From patchwork Tue May 13 10:07:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085014 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3583-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXYS6w1mz1yYN for ; Tue, 13 May 2025 20:20:44 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 5E8531B60A6B for ; Tue, 13 May 2025 10:19:50 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B014027FB3D; Tue, 13 May 2025 10:08:08 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 995AD27603A; Tue, 13 May 2025 10:08:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130888; cv=none; b=fjGl9UFs6kxkCP+KpsazbIgVO1xLNAzfJr2TXA+1YxAFeJ5JJivk2264Z9tL176DF5QE7Mkrw6B6jKhreAjxhBSdxvWT8ZVBGmwgGzp350NsemQu20PMJECKx0/z6zZUVySH2ZKatbKT6fMTTC6KBnQvKQTKmmtPslmjuIYcvqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130888; c=relaxed/simple; bh=V9G75luuvDdhIj9lO7cbblDKNYFXdg8m9W2gAiy+KKI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dPpO21+f1DVWiIBbGVeN09WOELxOcttGiqjOSTXdF6RpAEV9VEl7JOJVGdKnKKpctWs6pIEuE09aGASs1YzIwyfQGZx99S7cf/IsGmW4ZyI1lPwp3PYtWQaIfLpus3dPTyb/4mJlnXlCuliYZRfLJnflg1sJWajAW6wxbtlQIdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-7c-682319f13be5 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 28/43] dept: suppress reports with classes that have been already reported Date: Tue, 13 May 2025 19:07:15 +0900 Message-Id: <20250513100730.12664-29-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxSHfd9773tviyXXSvCixI8aYwLRiVFzssxtxkRvDEajy6YumWvk Zu0shbR8iIkpVTSC0iCKBAQsZakNrcqK2xyCQwwFNGIVKB8BNokBiiCKtFkVP1qN/5w8+f1y nvPP4ShlG7OY0+ozJINerVMROS2fml+95mXcSs26uoo4CMyepqHiuouA95oTgeuGGYO/dTv0 BicRvHnwkILSEi+C6idDFNzwDCNochwn0PU0GroD0wQ6Ss4QOFFzncCjZ3MYBi8WY3C6d8K/ 9lEa7hfZMJT6CVwqPYHDYxxDyF7Lgj13FYw4ylmYe5IEHcM+BpoGEqGsapBAY1MHDZ6bIxi6 GioIDLveM3Df005D0LIEvOcKGbj63EbgWdBOgT0wzcLjZisGjzUW6vLCwlOv3jHQVtiM4dRv v2Po7r+F4Pbp/zC4XT4CdwOTGOrdJRS8vtKKYMQyxcLJsyEWLpktCM6cvEhD3uBGePN/+HLl bBKYL9fRcPWtD327WXRVuZB4d3KaEvPqs8XXgR4iNgWttHjPJoh/lw+xYt7tAVa0ujPFekeC WNPox2L1TIAR3bX5RHTPFLNiwVQ3Fp93drK74w/Iv0qRdNosyfDF1z/LNY7KRpQ+pDgyHpxg c9G4vADJOIHfIFw2t+PP3FBrIREm/Gqhry9ERTiGXy7UF44yBUjOUbwvSuit7EeRYiH/k9D7 R094geNofpXQdlwXiRX8JqHhxSvyyblMcNY1f/TIwvnbK510hJX8RqHI6qQjToE/LxNszgvs p4U44Y6jjy5CCiuaV4uUWn1Wqlqr27BWk6PXHll7KC3VjcLvZT829+NNNOPd24J4DqnmK9r9 KzRKRp1lzEltQQJHqWIU5r/CkSJFnXNUMqQdNGTqJGMLWsLRqkWK9cHsFCX/izpDOixJ6ZLh c4s52eJc9M0sS0y7lJb1R9NCiVvfl36XLPwjK+5KjveP9rRPRJUNFM8scHLe1l/TW/Ntt/Yh 46wpetuC7cy6pd+v+TJ7ZZWlbOdYf3JCRo3GFLXFeudpZmLh0MGQSRe7NWuiSj8auzBhDG3m zfFxvuhzbDk+9EOM6Vr+MUrl+XPHfvlY2h4VbdSokxIog1H9AaMSespaAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ8f3+fpOB6n6aH5da3ZGNUmPhuz/KNnjPGXLYaTZ+64u3Kn koVyMUq3MtVcXa6ykzrkCkm1W1FOk1IurJKYleJQd1zy447557PXXu993p9/Piwpu0bPZ1Xa I6JOq1DLsYSSbF1rWPFlXpgy0tJMgGfiLAXFN20YOm9UIbDVZhAw8jAWer1jCH48eUpCYX4n gtI3/STUtg4gaKw4haH73Uzo8bgxOPOzMRjKb2LoGp0ioK/gAgFV9i3w2vqegvbcMgIKRzAU FRoI/xgmwGetZMCaHg5DFSYGpt5EgXPARUOL2UlD46vlcKmkD0NDo5OC1rohArrrizEM2H7T 0N76iAKvMRQ683JouP6pDMOo10qC1eNm4JnDQkCrZS5UZ/pbz4z/oqEtx0HAmSu3COh5eR9B 09lBAuw2F4YWzxgBNfZ8EiavPkQwZPzIwOnzPgaKMowIsk8XUJDZFw0/vvsvmyeiIONyNQXX f7pQzHrBVmJDQsuYmxQya1KESc9zLDR6LZTwuIwX7pn6GSGz6RUjWOxJQk3FMqG8YYQQSr96 aMFeeQ4L9q8XGCHrYw8hfOroYLYtiJOs2y+qVcmiLmL9XomywtyAEvulR4e9H5h0NCzJQkEs z63i6yuNOMCYW8q/eOEjAxzMLeZrct7TWUjCkpxrOt9rfokCwRxuN997+7l/gWUpLpxvO6UO aCm3mq//PI7/dS7iq6odf3uC/P7n1Q4qwDIums+1VFG5SGJB0ypRsEqbrFGo1NEr9YeUqVrV 0ZXxCRo78j+Q9fhUXh2a6I5tRhyL5DOkj0aWKGW0IlmfqmlGPEvKg6UZd/1Kul+RekzUJezR JalFfTMKZSl5iHTTDnGvjDugOCIeEsVEUfc/Jdig+ekIG3wq88aQ0bQTF3cNNqVFFBVsvnTf Ne/c4hzNuGyn4+0i08Hd4ZHF2oRrsa4Erm0ge1AbL+8wFTrSSlakGPLWumPcC7u+pW4ONT0Y dMSsueKbnf3aueGidvVhe9hYX54WJ207uT1Csy4/co9h1uw6Y1PRnRRl+76wOL1OE7nljldO 6ZWKqGWkTq/4A1Zc+8g8AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Each different chain implies its own dependency problem, that is worth being reported. However, generating similar reports mutiple times should be suppressed. To avoid that, suppress reports with classes that have been already reported. Signed-off-by: Byungchul Park --- include/linux/dept.h | 6 ++++++ kernel/dependency/dept.c | 10 ++++++++++ 2 files changed, 16 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 551168220954..b1e5a1ec6763 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -107,6 +107,12 @@ struct dept_class { * not an explicit map */ bool sched_map; + + /* + * for avoiding this class involved in report + * more than once + */ + bool reported; }; }; }; diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 278194093108..8bea64bab379 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -501,6 +501,7 @@ static void initialize_class(struct dept_class *c) iw->touched = false; } c->bfs_gen = 0U; + c->reported = false; INIT_LIST_HEAD(&c->all_node); INIT_LIST_HEAD(&c->dep_head); @@ -913,6 +914,12 @@ static void print_circle(struct dept_class *c) dump_stack(); dept_outworld_exit(); + + do { + tc->reported = true; + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); } /* @@ -1225,6 +1232,9 @@ static enum bfs_ret cb_check_dl(void *node, void *in, void **out) struct dept_class *cur = (struct dept_class *)node; struct dept_dep *new = (struct dept_dep *)in; + if (cur->reported) + return BFS_SKIP; + if (cur == dep_fc(new)) { print_circle(dep_tc(new)); return BFS_DONE; From patchwork Tue May 13 10:07:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085020 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3586-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXb45NyHz1yYN for ; Tue, 13 May 2025 20:22:08 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id B9C5A4A8171 for ; Tue, 13 May 2025 10:21:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4CEE72882DF; Tue, 13 May 2025 10:08:11 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 89D6127A112; Tue, 13 May 2025 10:08:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130891; cv=none; b=LnYjZ98/Ix2/QIKWcE8E6DsvskRVHypYRrM8reIbG1kmwMjp2q3aHFmIH+43bDTJBbjazyXxKVXriuj9bsA6lUNxNwpFYlUKeIC2ToJ3bm/hXpPgqYnOAxGoObv3h/tVRHhQz7JevDaQitWDBx18I4DbRM0y+fqs74yVXWgL4/Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130891; c=relaxed/simple; bh=faITj/Q/isOD+29VNf4M4bgUiNP5RcjvtJE110dADOo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=cUlCx/fuWdhOd5aeKf2T/sOO3cEGzcqmJBiWourN6UjfsFmoKDQ2BI1gfY16i3K3t9n03TfRqp4OSVH8qPwSu/bSqyBE1L+wHTf2/yaz2TSV8/3zrsZHdRgMLFwHbx0qMT7p9t5A4cyRTCh64JqD6ubWG7Krgo97bnJADRcY24c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-8c-682319f1060c From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 29/43] dept: add documentation for dept Date: Tue, 13 May 2025 19:07:16 +0900 Message-Id: <20250513100730.12664-30-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2Sa0yTZxTH97x3Kp3vOoMvamSr0SU4dBidR/H6ac9iXBb5JLpBA+/WTkBS tFoTIxXwwk1GBoiAK6ClgRZYIRsO6iqEYlGRCVbAikIWtHJTpBUUxQLOLye//M85v/PlcKTM QS/hVPGHRHW8IlbOSCjJiH9JyPPAFcqvCgZDwDNxhoKiahMDHVWVCEx1OgLcLd/APe8wgte3 bpOQn9uBoKT/AQl19j4EVuNJBjr/+xi6PGMMOHLTGUguq2bg36FpAlx5OQRUWnbDQ8MgBTey SwnIdzNQmJ9M+MoTAqYMFSwYklbCgPECC9P9oeDoc9Jg7V0NBRddDDRaHRTY6wcI6Py7iIE+ 0wwNN+zXKfBmLYWOXzNpMI+WMjDkNZBg8IyxcMemJ8CuD4CaFJ/w1Iu3NLRm2gg4dekPArp6 GhBcPfOIAIvJyUCzZ5iAWksuCa/KWxAMZI2wkJoxxUKhLgtBemoeBSmuDfB60ne5eCIUdL/X UGB+40Q7tmLTRRPCzcNjJE6pPYJfee4y2OrVU7itVMBXLjxgccrVXhbrLYdxrTEYlzW6CVwy 7qGxpeIsgy3jOSxOG+ki8Gh7O/v9sgjJlhgxVqUR1Wu3RUmU9dabdMLTLnR0wu5mklBrMUpD fpzArxfSX5o/8LP7U+QsM/wXQnf3PC/iPxNqMwfpNCThSN65QLhX3DO38CkfJjxP9TKzTPEr hYzCk8QsS/mvhZGzTmpeGiRU1tjmRH6+/E15+1wu4zcI2frK9zNlfoKxWjvPgcI1YzeVjaR6 9FEFkqniNXEKVez6NUptvOromuiDcRbkezDD8el99Wi8I7wJ8RyS+0uvuz9XymiFJlEb14QE jpQvkur+8kXSGIX2mKg+GKk+HCsmNqGlHCVfLF3nPRIj439WHBIPiGKCqP6/S3B+S5LQjzPn zzl27ta++O3a6b2u5rDwf1xxJ6SNwVxYp/nPb62f7LHvLO2ZTCheEQ2bllUtHOvdqLgTvmqy bZTCJ9qu5JqnW/hQtUPTEJEX7f/I9WXk/r7gmUyhmlxtW67RbX4bdcuxjryc7Mje1Wn5ztYS 8BNyl/3QUPTL9oKg9sfnPTEBcipRqQgNJtWJindDmq5oXAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxSH97733vdeutVcEfVOIpudX0FBMcOchYVosoV3JiybS7ZoYrTK DW340hYQMCblM4qCyAJkQF0BVwiUgYUoKiUEQhHZEAUrKuDKhNgBIo42ojBXMP5z8uT55fzO P0dgfC3cOkEbnyjr4tWxKqJgFd+GZQbNfvyZZmd/qw+4586wUN5gIdD/ex0CS3M6BldXBDzw TCF48+cdBkqK+hFUOEcYaLaPIrDVZBAYeLoCBt0zBHqKzhHIrGogcHdyAcNwcSGGOmskPDFP sNBbUImhxEWgrCQTe8czDPPmWh7Mhk0wVlPKw4IzBHpGHRx0Gns4sD3aBr9cGibQauthwd4y hmHgRjmBUctbDnrtt1jw5PtD/8U8DuqfVxKY9JgZMLtneLjXbsJgN62Bxixva86//3HQndeO IefyFQyDD28iaDvzFwarxUGg0z2FoclaxMDr6i4EY/nTPGSfn+ehLD0fwbnsYhayhkPhzSvv ZeNcCKT/2shC/aID7QmnlksWRDunZhia1XSSvnbfJ9TmMbH0dqVEr5eO8DSr7RFPTdYk2lQT SKtaXZhWvHRz1Fp7llDry0Ke5k4PYvq8r4//bv1BxZdRcqw2WdbtCD+i0LTY/uCO/zOIUubs LmJA3UaUi3wESfxcevF4nlliIm6RhobesZ/4qdSUN8HlIoXAiI4PpQfGh8sLq8QwaTbbQ5aY FTdJ58sy8BIrxd3S9FkH+670E6musX25yMfrF6v7lr2vGCoVmOrYAqQwoQ9qkZ82PjlOrY0N DdbHaFLjtSnBxxLirMj7Q+bTCxdb0NxARAcSBaT6SHnLtUHjy6mT9alxHUgSGJWfMv2aVymj 1Klpsi7hsC4pVtZ3IH+BVa1V7vtJPuIrRqsT5RhZPi7r3qdY8FlnQOPb3wYnRU8eXvv1hf1G y3ybv7K0N0Cz+VTG0AFi2nWCm40Zv998YOvVH8RC4UrJwY3Ozeurvjiqch77+/qKgNXlW36L 9HsVg7c1XEjsClsZNRGwv+Kb7EhDhFox/PPIzPeFiwbDUBBXE0S/2tMavq/YcDnBWOpMC3Is 5igOpe3FP0arWL1GHRLI6PTq/wEM22cwPwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: This document describes the concept and APIs of dept. Signed-off-by: Byungchul Park --- Documentation/dependency/dept.txt | 735 ++++++++++++++++++++++++++ Documentation/dependency/dept_api.txt | 117 ++++ 2 files changed, 852 insertions(+) create mode 100644 Documentation/dependency/dept.txt create mode 100644 Documentation/dependency/dept_api.txt diff --git a/Documentation/dependency/dept.txt b/Documentation/dependency/dept.txt new file mode 100644 index 000000000000..5dd358b96734 --- /dev/null +++ b/Documentation/dependency/dept.txt @@ -0,0 +1,735 @@ +DEPT(DEPendency Tracker) +======================== + +Started by Byungchul Park + +How lockdep works +----------------- + +Lockdep detects a deadlock by checking lock acquisition order. For +example, a graph to track acquisition order built by lockdep might look +like: + + A -> B - + \ + -> E + / + C -> D - + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep keeps adding each new acquisition order into the graph in +runtime. For example, 'E -> C' will be added when the two locks have +been acquired in the order, E and then C. The graph will look like: + + A -> B - + \ + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +This graph contains a subgraph that demonstrates a loop like: + + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep reports it as a deadlock on detection of a loop and stops its +working. + +CONCLUSION + +Lockdep detects a deadlock by checking if a loop has been created after +adding a new acquisition order into the graph. + + +Limitation of lockdep +--------------------- + +Lockdep deals with a deadlock by typical lock e.g. spinlock and mutex, +that are supposed to be released within the acquisition context. However, +when it comes to a deadlock by folio lock that is not supposed to be +released within the acquisition context or other general synchronization +mechanisms, lockdep doesn't work. + +Can lockdep detect the following deadlock? + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +No. What about the following? + + context X context Y + + mutex_lock A + mutex_lock A <- DEADLOCK + wait_for_complete B <- DEADLOCK + complete B + mutex_unlock A + mutex_unlock A + +No. + +CONCLUSION + +Lockdep cannot detect a deadlock by folio lock or other general +synchronization mechanisms. + + +What leads a deadlock +--------------------- + +A deadlock occurs when one or multi contexts are waiting for events that +will never happen. For example: + + context X context Y context Z + + | | | + v | | + 1 wait for A v | + . 2 wait for C v + event C . 3 wait for B + event B . + event A + +Event C cannot be triggered because context X is stuck at 1, event B +cannot be triggered because context Y is stuck at 2, and event A cannot +be triggered because context Z is stuck at 3. All the contexts are stuck. +We call this *deadlock*. + +If an event occurrence is a prerequisite to reaching another event, we +call it *dependency*. In this example: + + Event A occurrence is a prerequisite to reaching event C. + Event C occurrence is a prerequisite to reaching event B. + Event B occurrence is a prerequisite to reaching event A. + +In terms of dependency: + + Event C depends on event A. + Event B depends on event C. + Event A depends on event B. + +Dependency graph reflecting this example will look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +A circular dependency exists. Such a circular dependency leads a +deadlock since no waiters can have desired events triggered. + +CONCLUSION + +A circular dependency of events leads a deadlock. + + +Introduce DEPT +-------------- + +DEPT(DEPendency Tracker) tracks wait and event instead of lock +acquisition order so as to recognize the following situation: + + context X context Y context Z + + | | | + v | | + wait for A v | + . wait for C v + event C . wait for B + event B . + event A + +and builds up a dependency graph in runtime that is similar to lockdep. +The graph might look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT keeps adding each new dependency into the graph in runtime. For +example, 'B -> D' will be added when event D occurrence is a +prerequisite to reaching event B like: + + | + v + wait for D + . + event B + +After the addition, the graph will look like: + + -> D + / + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT is going to report a deadlock on detection of a new loop. + +CONCLUSION + +DEPT works on wait and event so as to theoretically detect all the +potential deadlocks. + + +How DEPT works +-------------- + +Let's take a look how DEPT works with the 1st example in the section +'Limitation of lockdep'. + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +Adding comments to describe DEPT's view in terms of wait and event: + + context X context Y context Z + + mutex_lock A + /* wait for A */ + folio_lock B + /* wait for A */ + /* start event A context */ + + folio_lock B + /* wait for B */ <- DEADLOCK + /* start event B context */ + + mutex_lock A + /* wait for A */ <- DEADLOCK + /* start event A context */ + + folio_unlock B + /* event B */ + folio_unlock B + /* event B */ + + mutex_unlock A + /* event A */ + mutex_unlock A + /* event A */ + +Adding more supplementary comments to describe DEPT's view in detail: + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +Let's build up dependency graph with this example. Firstly, context X: + + context X + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + +There are no events to create dependency. Next, context Y: + + context Y + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 3 */ + + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + +There are two events. For event B, folio_unlock B, since there are no +waits between 3 and the event, event B does not create dependency. For +event A, there is a wait, folio_lock B, between 1 and the event. Which +means event A cannot be triggered if event B does not wake up the wait. +Therefore, we can say event A depends on event B, say, 'A -> B'. The +graph will look like after adding the dependency: + + A -> B + + where 'A -> B' means that event A depends on event B. + +Lastly, context Z: + + context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +There are also two events. For event B, folio_unlock B, there is a +wait, mutex_lock A, between 2 and the event - remind 2 is at a very +start and before the wait in timeline. Which means event B cannot be +triggered if event A does not wake up the wait. Therefore, we can say +event B depends on event A, say, 'B -> A'. The graph will look like +after adding the dependency: + + -> A -> B - + / \ + \ / + ----------- + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock. For +event A, mutex_unlock A, since there are no waits between 4 and the +event, event A does not create dependency. That's it. + +CONCLUSION + +DEPT works well with any general synchronization mechanisms by focusing +on wait, event and its context. + + +Interpret DEPT report +--------------------- + +The following is the example in the section 'How DEPT works'. + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +We can Simplify this by replacing each waiting point with [W], each +point where its event's context starts with [S] and each event with [E]. +This example will look like after the replacement: + + context X context Y context Z + + [W][S] mutex_lock A + [W][S] folio_lock B + [W][S] folio_lock B <- DEADLOCK + + [W][S] mutex_lock A <- DEADLOCK + [E] folio_unlock B + [E] folio_unlock B + [E] mutex_unlock A + [E] mutex_unlock A + +DEPT uses the symbols [W], [S] and [E] in its report as described above. +The following is an example reported by DEPT for a real problem. + + Link: https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SAKURA.ne.jp/#t + Link: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ + + =================================================== + DEPT: Circular dependency has been detected. + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 Not tainted + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + --------------------------------------------------- + context A's detail + --------------------------------------------------- + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + [S] lock(&ni->ni_lock:0): + [] ntfs3_setattr+0x54b/0xd40 + stacktrace: + ntfs3_setattr+0x54b/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [W] folio_wait_bit_common(PG_locked_map:0): + [] truncate_inode_pages_range+0x9b0/0xf20 + stacktrace: + folio_wait_bit_common+0x5e0/0xaf0 + truncate_inode_pages_range+0x9b0/0xf20 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [E] unlock(&ni->ni_lock:0): + (N/A) + --------------------------------------------------- + context B's detail + --------------------------------------------------- + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S] (unknown)(PG_locked_map:0): + (N/A) + + [W] lock(&ni->ni_lock:0): + [] attr_data_get_block+0x32c/0x19f0 + stacktrace: + attr_data_get_block+0x32c/0x19f0 + ntfs_get_block_vbo+0x264/0x1330 + __block_write_begin_int+0x3bd/0x14b0 + block_write_begin+0xb9/0x4d0 + ntfs_write_begin+0x27e/0x480 + generic_perform_write+0x256/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + + [E] folio_unlock(PG_locked_map:0): + [] generic_write_end+0xf2/0x440 + stacktrace: + generic_write_end+0xf2/0x440 + ntfs_write_end+0x42e/0x980 + generic_perform_write+0x316/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + __x64_sys_sendfile64+0x1d0/0x210 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + --------------------------------------------------- + information that might be helpful + --------------------------------------------------- + CPU: 1 PID: 8060 Comm: a.out Not tainted + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), + BIOS Bochs 01/01/2011 + Call Trace: + + dump_stack_lvl+0xf2/0x169 + print_circle.cold+0xca4/0xd28 + ? lookup_dep+0x240/0x240 + ? extend_queue+0x223/0x300 + cb_check_dl+0x1e7/0x260 + bfs+0x27b/0x610 + ? print_circle+0x240/0x240 + ? llist_add_batch+0x180/0x180 + ? extend_queue_rev+0x300/0x300 + ? __add_dep+0x60f/0x810 + add_dep+0x221/0x5b0 + ? __add_idep+0x310/0x310 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + __dept_wait+0x600/0x1490 + ? add_iecxt+0x1bc/0xa60 + ? truncate_inode_pages_range+0x9b0/0xf20 + ? check_new_class+0x790/0x790 + ? dept_enirq_transition+0x519/0x9c0 + dept_wait+0x159/0x3b0 + ? truncate_inode_pages_range+0x9b0/0xf20 + folio_wait_bit_common+0x5e0/0xaf0 + ? filemap_get_folios_contig+0xa30/0xa30 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + ? lock_is_held_type+0x11e/0x160 + truncate_inode_pages_range+0x9b0/0xf20 + ? truncate_inode_partial_folio+0xba0/0xba0 + ? setattr_prepare+0x142/0xc40 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + ? ktime_get_coarse_real_ts64+0x1e5/0x2f0 + ? ntfs_extend+0x5c0/0x5c0 + ? mode_strip_sgid+0x210/0x210 + ? ntfs_extend+0x5c0/0x5c0 + notify_change+0xcb3/0x1430 + ? do_truncate+0x149/0x210 + do_truncate+0x149/0x210 + ? file_open_root+0x430/0x430 + ? process_measurement+0x18c0/0x18c0 + ? ntfs_file_release+0x230/0x230 + path_openat+0x21a3/0x2a90 + ? path_lookupat+0x840/0x840 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + do_filp_open+0x1ba/0x410 + ? may_open_dev+0xf0/0xf0 + ? find_held_lock+0x2d/0x110 + ? lock_release+0x43c/0x830 + ? dept_ecxt_exit+0x31a/0x590 + ? _raw_spin_unlock+0x3b/0x50 + ? alloc_fd+0x2de/0x6e0 + do_sys_openat2+0x16d/0x4e0 + ? __ia32_sys_get_robust_list+0x3b0/0x3b0 + ? build_open_flags+0x6f0/0x6f0 + ? dept_enirq_transition+0x519/0x9c0 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x4e/0x160 + ? lock_is_held_type+0x4e/0x160 + __x64_sys_creat+0xcd/0x120 + ? __x64_compat_sys_openat+0x1f0/0x1f0 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + RIP: 0033:0x7f8b9e4e4469 + Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 + 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> + 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48 + RSP: 002b:00007f8b9eea4ef8 EFLAGS: 00000202 ORIG_RAX: 0000000000000055 + RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f8b9e4e4469 + RDX: 0000000000737562 RSI: 0000000000000000 RDI: 0000000020000000 + RBP: 00007f8b9eea4f20 R08: 0000000000000000 R09: 0000000000000000 + R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa75511ee + R13: 00007fffa75511ef R14: 00007f8b9ee85000 R15: 0000000000000003 + + +Let's take a look at the summary that is the most important part. + + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + +The summary shows the following scenario: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(&f1) + + [E] unlock(&ni->ni_lock:0) + +Adding supplementary comments to describe DEPT's view in detail: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + /* start to take into account context + B heading for folio_unlock(&f1) */ + /* 1 */ + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (= folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +Let's build up dependency graph with this report. Firstly, context A: + + context A + + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (= folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +There is one interesting event, unlock(&ni->ni_lock:0). There is a +wait, folio_lock(&f1), between 2 and the event. Which means +unlock(&ni->ni_lock:0) is not reachable if folio_unlock(&f1) does not +wake up the wait. Therefore, we can say unlock(&ni->ni_lock:0) depends +on folio_unlock(&f1), say, 'unlock(&ni->ni_lock:0) -> folio_unlock(&f1)'. +The graph will look like after adding the dependency: + + unlock(&ni->ni_lock:0) -> folio_unlock(&f1) + + where 'A -> B' means that event A depends on event B. + +Secondly, context B: + + context B + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + +There is also one interesting event, folio_unlock(&f1). There is a +wait, lock(&ni->ni_lock:0), between 1 and the event - remind 1 is at a +very start and before the wait in timeline. Which means folio_unlock(&f1) +is not reachable if unlock(&ni->ni_lock:0) does not wake up the wait. +Therefore, we can say folio_unlock(&f1) depends on unlock(&ni->ni_lock:0), +say, 'folio_unlock(&f1) -> unlock(&ni->ni_lock:0)'. The graph will look +like after adding the dependency: + + -> unlock(&ni->ni_lock:0) -> folio_unlock(&f1) - + / \ + \ / + ------------------------------------------------ + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock! Cool! + +CONCLUSION + +DEPT works awesome! diff --git a/Documentation/dependency/dept_api.txt b/Documentation/dependency/dept_api.txt new file mode 100644 index 000000000000..8e0d5a118a46 --- /dev/null +++ b/Documentation/dependency/dept_api.txt @@ -0,0 +1,117 @@ +DEPT(DEPendency Tracker) APIs +============================= + +Started by Byungchul Park + +SDT(Single-event Dependency Tracker) APIs +----------------------------------------- +Use these APIs to annotate on either wait or event. These have been +already applied into the existing synchronization primitives e.g. +waitqueue, swait, wait_for_completion(), dma fence and so on. The basic +APIs of SDT are: + + /* + * After defining 'struct dept_map map', initialize the instance. + */ + sdt_map_init(map); + + /* + * Place just before the interesting wait. + */ + sdt_wait(map); + + /* + * Place just before the interesting event. + */ + sdt_event(map); + +The advanced APIs of SDT are: + + /* + * After defining 'struct dept_map map', initialize the instance + * using an external key. + */ + sdt_map_init_key(map, key); + + /* + * Place just before the interesting timeout wait. + */ + sdt_wait_timeout(map, time); + + /* + * Use sdt_might_sleep_start() and sdt_might_sleep_end() in pair. + * Place at the start of the interesting section that might enter + * schedule() or its family that needs to be woken up by + * try_to_wake_up(). + */ + sdt_might_sleep_start(map); + + /* + * Use sdt_might_sleep_start_timeout() and sdt_might_sleep_end() in + * pair. Place at the start of the interesting section that might + * enter schedule_timeout() or its family that needs to be woken up + * by try_to_wake_up(). + */ + sdt_might_sleep_start_timeout(map, time); + + /* + * Use sdt_might_sleep_start() and sdt_might_sleep_end() in pair. + * Place at the end of the interesting section that might enter + * schedule(), schedule_timeout() or its family that needs to be + * woken up by try_to_wake_up(). + */ + sdt_might_sleep_end(); + + /* + * Use sdt_ecxt_enter() and sdt_ecxt_exit() in pair. Place at the + * start of the interesting section where the interesting event might + * be triggered. + */ + sdt_ecxt_enter(map); + + /* + * Use sdt_ecxt_enter() and sdt_ecxt_exit() in pair. Place at the + * end of the interesting section where the interesting event might + * be triggered. + */ + sdt_ecxt_exit(map); + + +LDT(Lock Dependency Tracker) APIs +--------------------------------- +Do not use these APIs directly. These are the wrappers for typical +locks, that have been already applied into major locks internally e.g. +spin lock, mutex, rwlock and so on. The APIs of LDT are: + + ldt_init(map, key, sub, name); + ldt_lock(map, sub_local, try, nest, ip); + ldt_rlock(map, sub_local, try, nest, ip, queued); + ldt_wlock(map, sub_local, try, nest, ip); + ldt_unlock(map, ip); + ldt_downgrade(map, ip); + ldt_set_class(map, name, key, sub_local, ip); + + +Raw APIs +-------- +Do not use these APIs directly. The raw APIs of dept are: + + dept_free_range(start, size); + dept_map_init(map, key, sub, name); + dept_map_reinit(map, key, sub, name); + dept_ext_wgen_init(ext_wgen); + dept_map_copy(map_to, map_from); + dept_wait(map, wait_flags, ip, wait_func, sub_local, time); + dept_stage_wait(map, key, ip, wait_func, time); + dept_request_event_wait_commit(); + dept_clean_stage(); + dept_stage_event(task, ip); + dept_ecxt_enter(map, evt_flags, ip, ecxt_func, evt_func, sub_local); + dept_ecxt_holding(map, evt_flags); + dept_request_event(map, ext_wgen); + dept_event(map, evt_flags, ip, evt_func, ext_wgen); + dept_ecxt_exit(map, evt_flags, ip); + dept_ecxt_enter_nokeep(map); + dept_key_init(key); + dept_key_destroy(key); + dept_map_ecxt_modify(map, cur_evt_flags, key, evt_flags, ip, ecxt_func, evt_func, sub_local); From patchwork Tue May 13 10:07:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085029 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3585-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXdP5PLWz1yYN for ; Tue, 13 May 2025 20:24:09 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id DC43B98554A for ; Tue, 13 May 2025 10:20:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B8654286D6E; Tue, 13 May 2025 10:08:10 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D6AEA27B4FC; Tue, 13 May 2025 10:08:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130890; cv=none; b=aifjKGjlAxvSRKGBhqJbZDGuiAK80emQ4105gzlo3nq0hkiLQetsiv9Gi8sLtQeUH9dtbAt19WHt/nE7y41I6nQ7kZ2l5NMNKEispQ/MC7wxP0M6ePuUSuYBQQ2uG/rdg5lvYQOtyeYo4EuV9qMZ5bpHNh96IIjleVe2XfdSD6w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130890; c=relaxed/simple; bh=1IZJyJwmvBeB2aOIzTtB/M01VK39C8cxFBwYkS4jNho=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=cjTr7WcgjAC1U8Y1WgXqdMxi+I5KTRnFfALOsdmXQw6gCnYRRdcmfasPd8/4CbJTf1mcmMg7NA2y5jLwEQ+iR+gueU4YDWLIJde7UfKN/MlPxhtQVvkhDF5Ovb1rxnhaN8aR1j6mjZpf9j7O39nOhetdSCj37yHf9/96IrL0i0o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-9d-682319f12074 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 30/43] cpu/hotplug: use a weaker annotation in AP thread Date: Tue, 13 May 2025 19:07:17 +0900 Message-Id: <20250513100730.12664-31-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2Ta0yTZxTHfZ731lZLXjvnXumyzSrRYMaGQ3eWqDEmuGeJCyZ+8JYozXi1 zbil3ISEBLQQLZcwDKJSakFXESrUUi9zlFQMHZeMgWJBhlDIIhJBMrDNCsjWovPLyT/nf/6/ 8+UcCaXoYMIl2uR0UZesTlSxMlo2varm87/XbdB82Xn/G/C9PkuDscnKQm9jAwKrIx/DZPu3 MOCfQrDw+x8UVFb0IqgZe0aBwz2CwFl3moXHf4VBv2+Ghc6KIhbOXG1ioe/lIobhC+UYGuzf w6jlOQ3dZbUYKidZqKo8g4PlBYaApZ4DS14EjNdd5mBxLBo6RzwMOIe2wCXTMAstzk4a3PfG MTy+b2RhxPovA93uDhr8pUro/amEgZuvall46bdQYPHNcPDIZcbgNq8Fmz4ILJxbYuC3EheG wmu3MPQ//RVB61kvBrvVw8JD3xSGZnsFBfPX2xGMl05zUFAc4KAqvxRBUcEFGvTD22Dhn+Dm 6tfRkH/FRsPNNx60eyexmqyIPJyaoYi+OYvM+56wxOk306SrViC/XH7GEX3rEEfM9gzSXBdJ rrZMYlIz62OIvf4cS+yz5RwxTPdj8qqnh9v/8RHZjgQxUZsp6r7YFS/TtFsWcKpXeurukwmU hxY5A5JKBD5GsDm87/X86A06pFl+kzA4GKBCeg3/mdBc8pwxIJmE4j0rhYHqpyhkfMDvE2r+ dC0HaD5CmFhawiEt57cLrpHBd9BPhQabaxkkDfbfXO9Znlfw24QycwMdggp8lVQoLLYwbwPr hAd1g3QZkpvRinqk0CZnJqm1iTFRmuxk7amoH1KS7Ch4X5bcxaP30GzvgTbES5Bqlbxjcr1G wagz07KT2pAgoVRr5Pl3gy15gjo7R9SlHNdlJIppbUgpoVUfybf6sxIU/El1uvijKKaKuv9d LJGG56HTzpKu7+Juz7U+issZW7leEVZw4I43Z3t1YMG0Om7fZseOk/rajTvxXsPhiy5b7PmA ++C8Sbn10IukzaMTnxTHH7x02HPCEfNVuPxD6R5j1INhU5NyLUrpOzZU1HLFEGHM2pvRyqbH 5ga/JTfrovJrHe/MW3Gtyxg210givXz4zyo6TaOOjqR0aer/AIXIDF5bAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/p7vj9Ntp+smMztN2JjLZx8PMs988zR8e5tmtfutOV7ir VMZKVyNqadJEuoqT6yiXKXR2SsdlUnoQ6lRMokS6pgdxNf757LX3+/N5v//5iAiZifIWqUPD BG2oUiOnJaRky9K4uT8mTVfNTyn0AVfvaRKuFJhpqL6dj8B8NxZDR8V6eN3XiWDwxUsC0tOq EWS3NhNw1+5EYM07RUPtx/FQ5+qmwZF2loa43AIaar4OYWi6mIoh37IZ3hs/kfA8JQdDegcN l9PjsHt8xtBvNDFgjJkJbXkZDAy1+oHD2UBBeaaDAuvbOXDpahMNpVYHCfaSNgy1D67Q4DT/ oeC5/RkJfcmTofp8EgW3vuXQ8LXPSIDR1c3AK5sBg90wEQr17tSEn8MUPE2yYUi4dgdD3ZuH CB6dbsFgMTfQUO7qxFBkSSNg4EYFgrbkLgbiz/UzcDk2GcHZ+Isk6Jv8YfCXuzmz1w9iswpJ uPW7Aa1YzpuvmhFf3tlN8PqiY/yAq57mrX0Gkq/M4fj7Gc0Mr3/0luENlnC+KE/B55Z2YD67 x0XxFtMZmrf0pDJ8Ylcd5r9VVTFbp+yWLAsUNOoIQTtv+UGJqsI4iI+0iCOL69tRDBpiEpFY xLELuYH3N8kRptnZXGNjPzHCnuw0rijpE5WIJCKCbRjLvc58g0aMCewmLvudbfSAZGdy7cPD eISl7CLO5mz8FzqVyy+0jQaJ3frvG1Wj+zLWn0sx5JMpSGJAY0zIUx0aEaJUa/x9dcGqqFB1 pG/A4RALcn+Q8cTQ+RLUW7u+DLEiJB8nfdbho5JRyghdVEgZ4kSE3FMaW+yWpIHKqGhBe/iA Nlwj6MrQZBEp95Ju2CkclLFByjAhWBCOCNr/LhaJvWPQ5ubraw9F/9izfUnW3ujpVUFjepzF S+ru+XadCzDfNodE7tHPsETvTTCt+XJyX9YL264Webi81a/2+MrVXWOffK9X+/zxeHD02mLp QodGfCGsJmpWZYnPu/1ewR5ckv3DyeOPL2x4GjSpPSM1dUfCtrgzioJS7wXW4XWVOxWrFsXH bNmolpM6ldJPQWh1yr8J1dEwPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: cb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread. However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire() as a potential wait and might report a deadlock by the wait. Replace it with trylock version of annotation. Signed-off-by: Byungchul Park --- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index b08bb34b1718..3eecd0a7b5b3 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -537,7 +537,7 @@ int lockdep_is_cpus_held(void) static void lockdep_acquire_cpus_lock(void) { - rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 0, _THIS_IP_); + rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 1, _THIS_IP_); } static void lockdep_release_cpus_lock(void) From patchwork Tue May 13 10:07:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085023 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3587-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXbR2MS3z1yYN for ; Tue, 13 May 2025 20:22:27 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 71BDF1B60EFD for ; Tue, 13 May 2025 10:21:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 78C4828851B; Tue, 13 May 2025 10:08:11 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C7E9027E7C6; Tue, 13 May 2025 10:08:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130891; cv=none; b=BfALdea4hoEeiS+xJi4JhheWiO/WiDpOspVs+mnHBNmABlnc2EW/juVz+WPOqBhngYY/4GsquE4HnpLvs8LR9F7bltfCGLi/JbIyXCzVT1IwTt84y0DZgHRD/PjL1KKQVYpYjPoIOeuWXNxM4+U2qq6smYMj+M3NyEArF2iKCwk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130891; c=relaxed/simple; bh=WLFy2XrzC3m864Tg26cLbN8q8gwXnUddJfJFmRKv6UE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=mqjRAJfybN2Av/zNX/a8uCbUwaEsZA0dg0lsKpaWQCx7gykR/Nr7Rm9O+FByIt1e9ilkbUVdebVFqK7sEuAypi13wZgbdmiAxoc0JuvPFWA4ZoRpthsXbHapSEDkz+QNuz3+RDgPpvLjqU0Ne+aO9/ZqLClwPks8FfNTfFQbhqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-af-682319f15cda From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 31/43] fs/jbd2: use a weaker annotation in journal handling Date: Tue, 13 May 2025 19:07:18 +0900 Message-Id: <20250513100730.12664-32-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfe57qzXXQvSCJtOqcWERhKA5iTL9YOLzxczEt0RdtkbubGNB 0wqKbpEKIhZhlQUIKtKCVkLbyS4uOAeGoaCVWFGQUuVFsXFrKJJU28mrthC/nPzzO7/8z5fD kcp2Op7TZh4T9ZlqnYqRU/LRBda1wbiVmnX+IQShD4UUXLnpYKDrdzsCxy0jAf72beAJBxBM Pn5CQkVZFwLr6wESbnUMImipO8NAt28h9ITGGHCVFTGQV3uTgacjUwT0l5cSYJe2w5DtLQWd 5hoCKvwMXK7IIyLjPwLGbfUs2HJXw3DdJRamXieDa7CXhpYX30Dl1X4GmltcFHTcHiag+84V BgYdn2jo7HhIQbhkKXRdLKbB+a6GgZGwjQRbaIyFZ60WAjosi6EhP1JY8H6GhgfFrQQUXPuD gB7v3wjuFr4iQHL0MnAvFCCgUSojYeJGO4LhklEWzl4YZ+GysQRB0dlyCvL718Pkx8jlqg/J YKxuoMA53Yu2pGHHVQfC9wJjJM5vPI4nQs8Z3BK2UPhRjYD/ujTA4vy7L1hskbJwY10Crm32 E9gaDNFYqj/PYClYymLTaA+B37nd7I5l++Sb0kWdNlvUJ337o1xT1FNJHP04/0TVr26Ui/6X mZCME/hUwTM1SZoQN5tLzYlRzPBrhL6+cTKaY/nlQmPxW9qE5BzJ984XPFVeFPVj+O+EzmZN 1KH41cKM882sr+A3CEPOCXau/ivB3tA6y2URPn3DTUWzkl8vmC12as6plgnN19LmcpzwT10f ZUYKC5pXj5TazOwMtVaXmqjJydSeSDx4JENCkeey/TK1/zYKdu1sQzyHVAsUD/0rNEpanW3I yWhDAkeqYhXGpghSpKtzTor6Iz/os3SioQ0t5SjVEkVK+Hi6kj+kPiYeFsWjov7LluBk8bmI K/d6Zr5PUFTcsfum98Y+s07udkk8XfjT4p+vdyuCiVzgwGbPef0uedP++6v6U7bq1j1etPGw FP/1c/PBc9UuTUxSk+O0yhQ36D1pdtWeyirzahN87Y50HLP530O+gFvjHahMLRhRpzmZJkOR L2Xrb9fziq3b9qQu2/Qy6U9jnIoyaNTJCaTeoP4Mv/yOjFgDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ732mrNO2D6ghq1xjk1MklET7xh/OLjbTPxw7KpkaqvtrFU bBVBo6FSCBZBJEHwgtZqSkeLskIypoAEYrUSOya13gAVlUhEG5GWq2hr4peTf37nl3O+/Hkq ys7E8RrdAUmvU2mVrJyW/7osa8HH2FnqhXmenyHYn0vDhetOFlqvORA4a4wYem6vgUehXgQj 9/+joKS4FcHllx0U1Lg7EdTbj7PQ9noi+IIBFjzFeSxkXbnOwv/vRjG0nynC4HBthOe2bhpa Cq0YSnpYOF+ShcPjLYYhWwUHtszZ0GU/x8HoywTwdPoZaC7zMFD/dD6cvdjOQl29hwZ3bReG thsXWOh0fmGgxX2XhlDBFGg9nc9A5QcrC+9CNgpswQAHDxotGNyWSVBlCl/N+TTGwJ38Rgw5 V//G4HtyE0FD7gsMLqefheZgL4ZqVzEFw+W3EXQVvOcg++QQB+eNBQjyss/QYGpPhJHB8Oey /gQwXqqiofKzH61aSZwXnYg09wYoYqo+RIaDD1lSH7LQ5J5VJP+e6+CIqeEpRyyug6TaPo9c qevB5HJfkCGuihMscfUVccT83ofJB6+X2zTtT/nyXZJWkybpf1mZLFfn+c7i1MHx6WWnvCgT DcjMiOdFYZFYVBhvRjKeFeaIjx8PUZEcI8wQq/O7GTOS85TgHy8+KnuCIn608JvYUqeOOLQw WxyrfPXNVwiLxeeVw1wki8J00VHV+I3LwvxzuZeO5CghUSy0OOhCJLegcRUoRqNLS1FptInx hr3qDJ0mPX7nvhQXCvfHdnT0dC3qb1vThAQeKSco7vbMVEcxqjRDRkoTEnlKGaMw/hNGil2q jMOSft92/UGtZGhCU3haOVmx7ncpOUrYozog7ZWkVEn/fYt5WVwmMjqK12qPm9wNc/0VpWnW e7EjfNIPOW9Wn0p/yK7ocH2MvuW70ZAkLasLJA9uGdhQM9n+046+pHg/mVrg/mtbYIyg6CWa Uln3/DmbpulTpcbyWX/Udr3BP4a2bbTqYreqJe/Aseydm7fHPDuyZXBo9+61J5OV9P64gdzE peuzlh8yK2mDWpUwj9IbVF8BbO0khDsDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: jbd2 journal handling code doesn't want jbd2_might_wait_for_commit() to be placed between start_this_handle() and stop_this_handle(). So it marks the region with rwsem_acquire_read() and rwsem_release(). However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire_read() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire_read() as a potential wait and might report a deadlock by the wait. Replace it with trylock version of annotation. Signed-off-by: Byungchul Park --- fs/jbd2/transaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index cbc4785462f5..7de227a2a6f8 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -441,7 +441,7 @@ static int start_this_handle(journal_t *journal, handle_t *handle, read_unlock(&journal->j_state_lock); current->journal_info = handle; - rwsem_acquire_read(&journal->j_trans_commit_map, 0, 0, _THIS_IP_); + rwsem_acquire_read(&journal->j_trans_commit_map, 0, 1, _THIS_IP_); jbd2_journal_free_transaction(new_transaction); /* * Ensure that no allocations done while the transaction is open are From patchwork Tue May 13 10:07:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3588-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXcl1gb7z1yYN for ; Tue, 13 May 2025 20:23:35 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 648F9188821B for ; Tue, 13 May 2025 10:22:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7A0FD28AB07; Tue, 13 May 2025 10:08:12 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 19F3F248880; Tue, 13 May 2025 10:08:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; cv=none; b=in1/23ix0oya6xk55XYwL5MrMJk2S/ao0YeKbeZT08MerbxpFZMMrfXVxed/QeOXKBwgsLu2Dodo4HuDXsM5uYm/Eq/0H1pPPOwnqMsBMz5r6oplaiMs+J21uVVe03363Y2wvuVRDKL870OcXcDgLDZToOWXXg0HGdnN8/uHl3E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; c=relaxed/simple; bh=YIhasWAI2DFiyLXsGvo/xd87ErP0zWnVDexTxa/J6wA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=PkxqcOiYkADb5j09SB6vAmP9upg0q+fQ96z2V3gLQUHwWRR9/38v0rVFKf75PjHS+AZmNJHV12N0gu7gcVvfkubhMQVBffWKOU7cCtjse+iOerLZIr4BxwcvoAHAwEqe0z+8xQIG4mWYuFlDgArh4ugRoWsDidgkPLHtqc1t+cQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-be-682319f1c0cb From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 32/43] dept: assign dept map to mmu notifier invalidation synchronization Date: Tue, 13 May 2025 19:07:19 +0900 Message-Id: <20250513100730.12664-33-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfb6/uzq+zo0vGTozWyiZeP7AGOa7mY1h5tdy6Tt3U7GLEjPF 5Ue5VjiR4rrsSncqFxOVpUhp6iRXbnXq/Jh0lVV3SifuMv88e+39PM/r+edhcMkrchajjD0m qGLl0TJKRIj6AvRLBmfOVyy1VfiDa/giATklJgosxUYEpofJGPS83ARtbieCsTfNOGRpLQjy ujtxeFhnR1BVeJaCd58nQ6trgIIGbRoF5/JLKHjb68Gg4/oVDIzmLfDR8JWAxgw9Blk9FNzK Ood5yzcMRg1FNBiSFoCjMJsGT3cYNNitJFTZFsHN2x0UVFY1EFBX7sDg3dMcCuymPyQ01tUT 4E4PBEumhoT7/XoKet0GHAyuARpaqnUY1OmmQ6naKzw/NE7CK001BufvPsCg9UMFgmcXuzAw m6wU1LqcGJSZtTj8KniJwJHeR0PK5VEabiWnI0hLuU6AuiMcxka8l3OHwyD5TikB939b0drV vOm2CfG1zgGcV5cl8L9c7ym+yq0j+Nd6jn+S3Unz6mc2mteZj/NlhcF8fmUPxucNukjeXHSJ 4s2DV2g+ta8V4/ubmuits/eIVkUJ0cp4QRW65oBIkd/1Ax1tW3nih+c7mYTsS1KRH8OxyznH 8E8yFTET/GkkwRdT7EKuvX0U97GUnceVab56R0QMzlr9ubbcD8jXmMbu5/S2/Akm2AXc6FAz 6WMxu4Kz5NnJf/65nLG0ekLk581/FzQRPpaw4VyGzkj4pBx71Y+zt2RT/xZmcs8L24kMJNah SUVIooyNj5Ero5eHKBJjlSdCDh6JMSPvexlOe/aWo0HL9hrEMkgWIK7vCVJISHl8XGJMDeIY XCYVJz/2RuIoeeJJQXUkQnU8WoirQYEMIZshXuZOiJKwh+THhMOCcFRQ/e9ijN+sJJQ3J8Z5 qDvJOelUbWD3I4q9Fr5BKr33p35svmTu7I8vIkPX73hzYF+4I2B3hKv5y3hne8S0yPXrtOma zZc7pyqz1z3/snX6rrQWS1eqeuPOxcGnDud62gwhCb2abcU5eytvjqRIjRfUGlKfqbWOV4xP tmk+x/ffqF8d5F/eAlPSzmTKiDiFPCwYV8XJ/wJyrz64WgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRzF+933VsuLjbwVvVZSmD2ErG8UIkH5IygiCEmoXHVpwzllS80g 0NQoy2GRSZq5acyl8zXtneIDTXuoNVMznbWiskwr3eiWPabQP4cP58A5/xyO9L9Bz+e0+mOi Qa/WqRg5Jd+1OW31t3nLNOsuNa8Fz8QZCq5W2hnoqihDYK9NJWC4JQJ6vSMIfj3tJCE3pwuB 5c0gCbWtLgR1tlMMON/Nhm7PGAPtOecYSCuuZODZ50kCBi5fJKDMsROGrO8peJxdREDuMAP5 uWmETz4SIFlLWbCmBILblsfC5JsQaHf10NBc0E5DXf8quHJtgIEHde0UtN5xE+C8d5UBl/0v DY9b2yjwmhZA14UsGspHixj47LWSYPWMsfC8wUxAq3kuVKX7Wk+P/6HhYVYDAaevVxPQ/fI+ gvozrwlw2HsYaPaMEFDjyCHhZ0kLArfpCwsZ5yUW8lNNCM5lXKYgfSAUfv3wLRdMhEBqYRUF 5b97UHgYtl+zI9w8Mkbi9Jok/NPzgsF1XjOFHxUJ+G7eIIvT6/tZbHYk4BpbEC5+MExgy3cP jR2lZxns+H6RxZlfugk82tHB7l4YJd9yRNRpE0XD2rBouab49VcU37vx+NfJT3QKcq3ORBwn 8OuFtz+SMpGMY/gVQl+fRE6xkl8i1GS9pzORnCP5nplCb8FLNBXM4fcLRf3F00zxgYI03klP sYLfIHRZXNMs8IuFsqqG6SKZz/9d0kFNsT8fKmSby6hsJDejGaVIqdUnxqq1utA1xhhNsl57 fM3huFgH8h3IenLywh004YxoQjyHVLMUbcNLNf60OtGYHNuEBI5UKRWpt32W4og6+YRoiDto SNCJxia0gKNUAYodkWK0P39UfUyMEcV40fA/JTjZ/BS058rsmfG2PuXy/G1RWPJwZ/PjbkgJ B+etfxqntJjCnffHwxbt1e8KW6Re2XRTu6nCz/whwBTU6H0FkeHOS/X7DtTHR1/fJA0d2nc4 2C/AUmFTEuYZGY3B1e5OUuveGqmIsO+RuvMqn8u4Z7rtMbcqcgtnFcqkJ7GWZFSVFJ2goowa dUgQaTCq/wFtpqPwPAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Resolved the following false positive by introducing explicit dept map and annotations for dealing with this case: *** DEADLOCK *** context A [S] (unknown)(:0) [W] lock(&mm->mmap_lock:0) [E] try_to_wake_up(:0) context B [S] lock(&mm->mmap_lock:0) [W] mmu_interval_read_begin(:0) [E] unlock(&mm->mmap_lock:0) [S]: start of the event context [W]: the wait blocked [E]: the event not reachable dept already tracks dependencies between scheduler sleep and ttwu based on internal timestamp called wgen. However, in case that more than one event contexts are overwrapped, dept has chance to wrongly guess the start of the event context like the following: context A: lock L context A: mmu_notifier_invalidate_range_start() context B: lock L' context B: mmu_interval_read_begin() : wait <- here is the start of the event context of C. context B: unlock L' context C: lock L'' context C: mmu_notifier_invalidate_range_start() context A: mmu_notifier_invalidate_range_end() context A: unlock L context C: mmu_notifier_invalidate_range_end() : ttwu <- here is the end of the event context of C. dept observes a wait, lock L'' within the event context of C. Which causes a false positive dept report. context C: unlock L'' By explicitly annotating the interesting event context range, make dept work with more precise information like: context A: lock L context A: mmu_notifier_invalidate_range_start() context B: lock L' context B: mmu_interval_read_begin() : wait context B: unlock L' context C: lock L'' context C: mmu_notifier_invalidate_range_start() <- here is the start of the event context of C. context A: mmu_notifier_invalidate_range_end() context A: unlock L context C: mmu_notifier_invalidate_range_end() : ttwu <- here is the end of the event context of C. dept doesn't observe the wait, lock L'' within the event context of C. context C is responsible only for the range delimited by mmu_notifier_invalidate_range_{start,end}(). context C: unlock L'' Signed-off-by: Byungchul Park --- include/linux/mmu_notifier.h | 26 ++++++++++++++++++++++++++ mm/mmu_notifier.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 55 insertions(+), 2 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index bc2402a45741..1e256f5305b7 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -428,6 +428,14 @@ static inline int mmu_notifier_test_young(struct mm_struct *mm, return 0; } +#ifdef CONFIG_DEPT +void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_notifier_range *range); +void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifier_range *range); +#else +static inline void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_notifier_range *range) {} +static inline void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifier_range *range) {} +#endif + static inline void mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) { @@ -439,6 +447,12 @@ mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) __mmu_notifier_invalidate_range_start(range); } lock_map_release(&__mmu_notifier_invalidate_range_start_map); + + /* + * From now on, waiters could be there by this start until + * mmu_notifier_invalidate_range_end(). + */ + mmu_notifier_invalidate_dept_ecxt_start(range); } /* @@ -459,6 +473,12 @@ mmu_notifier_invalidate_range_start_nonblock(struct mmu_notifier_range *range) ret = __mmu_notifier_invalidate_range_start(range); } lock_map_release(&__mmu_notifier_invalidate_range_start_map); + + /* + * From now on, waiters could be there by this start until + * mmu_notifier_invalidate_range_end(). + */ + mmu_notifier_invalidate_dept_ecxt_start(range); return ret; } @@ -470,6 +490,12 @@ mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range) if (mm_has_notifiers(range->mm)) __mmu_notifier_invalidate_range_end(range); + + /* + * The event context that has been started by + * mmu_notifier_invalidate_range_start() ends. + */ + mmu_notifier_invalidate_dept_ecxt_end(range); } static inline void mmu_notifier_arch_invalidate_secondary_tlbs(struct mm_struct *mm, diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index fc18fe274505..850d75952f98 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -46,6 +46,7 @@ struct mmu_notifier_subscriptions { unsigned long active_invalidate_ranges; struct rb_root_cached itree; wait_queue_head_t wq; + struct dept_map dmap; struct hlist_head deferred_list; }; @@ -165,6 +166,25 @@ static void mn_itree_inv_end(struct mmu_notifier_subscriptions *subscriptions) wake_up_all(&subscriptions->wq); } +#ifdef CONFIG_DEPT +void mmu_notifier_invalidate_dept_ecxt_start(struct mmu_notifier_range *range) +{ + struct mmu_notifier_subscriptions *subscriptions = + range->mm->notifier_subscriptions; + + if (subscriptions) + sdt_ecxt_enter(&subscriptions->dmap); +} +void mmu_notifier_invalidate_dept_ecxt_end(struct mmu_notifier_range *range) +{ + struct mmu_notifier_subscriptions *subscriptions = + range->mm->notifier_subscriptions; + + if (subscriptions) + sdt_ecxt_exit(&subscriptions->dmap); +} +#endif + /** * mmu_interval_read_begin - Begin a read side critical section against a VA * range @@ -246,9 +266,12 @@ mmu_interval_read_begin(struct mmu_interval_notifier *interval_sub) */ lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); lock_map_release(&__mmu_notifier_invalidate_range_start_map); - if (is_invalidating) + if (is_invalidating) { + sdt_might_sleep_start(&subscriptions->dmap); wait_event(subscriptions->wq, READ_ONCE(subscriptions->invalidate_seq) != seq); + sdt_might_sleep_end(); + } /* * Notice that mmu_interval_read_retry() can already be true at this @@ -625,6 +648,7 @@ int __mmu_notifier_register(struct mmu_notifier *subscription, INIT_HLIST_HEAD(&subscriptions->list); spin_lock_init(&subscriptions->lock); + sdt_map_init(&subscriptions->dmap); subscriptions->invalidate_seq = 2; subscriptions->itree = RB_ROOT_CACHED; init_waitqueue_head(&subscriptions->wq); @@ -1070,9 +1094,12 @@ void mmu_interval_notifier_remove(struct mmu_interval_notifier *interval_sub) */ lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); lock_map_release(&__mmu_notifier_invalidate_range_start_map); - if (seq) + if (seq) { + sdt_might_sleep_start(&subscriptions->dmap); wait_event(subscriptions->wq, mmu_interval_seq_released(subscriptions, seq)); + sdt_might_sleep_end(); + } /* pairs with mmgrab in mmu_interval_notifier_insert() */ mmdrop(mm); From patchwork Tue May 13 10:07:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085032 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3589-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXdn4xTRz1yYN for ; Tue, 13 May 2025 20:24:29 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id EBBBE189252F for ; Tue, 13 May 2025 10:22:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 44E4128C5DF; Tue, 13 May 2025 10:08:13 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B409F27FD45; Tue, 13 May 2025 10:08:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; cv=none; b=OVIkrH3StDIXG7krVtCmAinNPXMLPQh7PJj2rF2W99ByJFsFiUVFtIbROk7oE02quZttWSR5TnIjezq7ihvgJLzaxdXgzz1f2hrZW3avoH4JMGMSBVuZcuj63qeqEyizzRFELSFZRx9RB9Oh8TrEpAxVJDnegHCgw5/dv81yNMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; c=relaxed/simple; bh=LsYjF8JhNvst04EPgWY8e97ui1aABX9yUv7y65Wkm50=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=uE0HkpFf+bsZICFaalkJ6OYsNM1L56lKQhEVRm+w9nYPG9rlf4KqwNNOU3QbyJ0eyLPzNWS4saabNq/3mPI68j5n0v0Urs4xTc3z35I+1h0rLmVfmhJ7cXIMTLVugvmnoLFfAv817n45dHzC4x9nswLjA4x3/d4qyz97g04eg5k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-ce-682319f1987d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 33/43] dept: assign unique dept_key to each distinct dma fence caller Date: Tue, 13 May 2025 19:07:20 +0900 Message-Id: <20250513100730.12664-34-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ732mrNa+fkRY1KhZCAol28nA9qTMyyN0YTE5cYxVuVN7QT qmkBZQuGykUBIYjWRimsoFaEIlgw4hSGECqMgFWwAkNE4nQNt1htFQtqwfjl5Jf/P+d3vhyW kDuohaxGmyDqtKo4BS0lpaNzSlZ6gperVxd/mAfe92dIMFfZaHDerEBgqzVgcLf8As98Iwj8 HY8IMBmdCEpePieg1jGAoL7sFA1dr+ZCt3echjZjDg1pV6poeDw8iaH/YgGGCvt2eGF9TUJ7 fikGk5uGQlMaDoz/MUxYyxmwpobBUNllBiZfKqFtwEVBfV8kXCrup+F+fRsJjrohDF1/mWkY sH2hoN3RSoIvbxE4z+VSUDlWSsOwz0qA1TvOwJNGCwaHZQFUpweEme8+U/AwtxFD5tVbGLp7 7yFoODOIwW5z0dDsHcFQYzcS8Ol6C4KhvFEGMs5OMFBoyEOQk3GRhPT+teD/GLhc9F4Jhj+r SaiccqHNGwVbsQ0JzSPjhJBec1z45H1KC/U+Cyn8U8oLdy8/Z4T0hj5GsNgThZqyCOHKfTcW SjxeSrCXZ9GC3VPACNmj3VgY6+xkdizeI90QI8ZpkkTdqk0Hpeobvj+ONSlPuB4ayFQ0Fp6N JCzPreGH3a3Ed3b1e5hpprlwvqdnYiafzy3ja3JfU9lIyhKcazb/rKgXTRc/cLv5zFujM0xy YXxO82MyG7GsjFvHtz8J++ZcyldUN854JIF46nonOc1ybi2fb6kgp508d17Cp30sRN8WgvkH ZT1kPpJZ0KxyJNdok+JVmrg1UepkreZE1OGj8XYU+C5rymR0HfI4dzYhjkWKObJWd4haTqmS 9MnxTYhnCcV8meFOIJLFqJJ/F3VHD+gS40R9E1rEkoog2U++4zFyLlaVIB4RxWOi7nuLWcnC VLQ+NfjfzF8bi/asCOXGjCGrfj64+U1x0KBxid7fcW/TMuMH99l8WaQ1usj0oA+H+3uDWrQF 1oRLTm3O4hX+k1NbJVGfI8wnEw+ZL1wL3f8mdGJ9V6y5r/m/4GQ0sGVy0KusSskYzNqm3lv3 949Zp023M6h9u1rbd1GKWSlq3PF23W8KUq9WKSMInV71FUI3GwZZAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5+7q9nBRp0mUc5GYGUKKW90JawORVHR/UO56tTmZdVWlkXk coppml3MbrOlMc3NS1ugXWaykbUsM7VpsqxJRJIXMjeytJpFX15+PA/8ni8vg4eUkVJGpT4s aNSKJBklIkTrFqbP/To1XBnV4ogF31AWATeqLBQ0V5oRWO7pMOh5sgra/b0Ifr58hUNhQTOC W953ONxr6EJgLztNQevHYGjzDVDgKsihIL2kioLXX0Yw8Fy+gIHZuhbemz4R0JhfjEFhDwXX C9OxwPmMwbCpnAZTmhy6y67RMOKNBleXmwSnwUWCvXM2XC3yUPDI7iKgobYbg9YHNyjosvwm obHhGQH+vFBoPp9LQkV/MQVf/CYcTL4BGlrqjRg0GCdDtT5gzfz2i4SnufUYZN6+i0Hb24cI 6rI+YGC1uClw+noxsFkLcPhR+gRBd14fDRlnh2m4rstDkJNxmQC9JwZ+fg8sG4aiQXezmoCK UTdatoS3FFkQ7+wdwHm97Sj/w/eG4u1+I8E/L+b4+9fe0by+rpPmjdYjvK0sgi951IPxtwZ9 JG8tP0Px1sELNJ/d14bx/U1N9PppO0SL9gpJqhRBM29JvEh5x3/ioCP6mPupjkhD/bOyURDD sfM5t2eQHmOKncV1dAzjYyxhZ3C23E9kNhIxOOsez7Ub3qKxYhK7ncu82/eXCVbO5ThfE9mI YcRsLNfYIv/nnM6Zq+v/eoIC8WhpEzHGIWwMl280E/lIZETjypFEpU5JVqiSYiK1icpUtepY 5J4DyVYU+B/TyZHztWiodZUDsQySTRA/6wlThpCKFG1qsgNxDC6TiHU1gUi8V5F6XNAc2KU5 kiRoHSiUIWRTxKu3CvEh7H7FYSFREA4Kmv8txgRJ01BCcORSbFOcdP5nb16oY/OGKG0Jad+2 MkxteBGXWuuqjKuRnohSJPwqWeDNCF98yZry3vNxGDaNjqwplU61xSz7NrFofeLM4NyqrE5S /mBj+LTthx7b5UEnj3t3XqwRP8/f/d1Q0bLx6JWw0C0L9QZzx5zl5zqckvhTCfs6uwx1KyQy QqtUREfgGq3iDxGf1bk7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: dma fence can be used at various points in the code and it's very hard to distinguish dma fences between different usages. Using a single dept_key for all the dma fences could trigger false positive reports. Assign unique dept_key to each distinct dma fence wait to avoid false positive reports. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 12 +++--- include/linux/dma-fence.h | 74 +++++++++++++++++++++++++++++-------- 2 files changed, 65 insertions(+), 21 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index a45e5416f2dd..5215ac4eecfb 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -499,7 +499,7 @@ EXPORT_SYMBOL(dma_fence_signal); * See also dma_fence_wait() and dma_fence_wait_any_timeout(). */ signed long -dma_fence_wait_timeout(struct dma_fence *fence, bool intr, signed long timeout) +__dma_fence_wait_timeout(struct dma_fence *fence, bool intr, signed long timeout) { signed long ret; @@ -520,7 +520,7 @@ dma_fence_wait_timeout(struct dma_fence *fence, bool intr, signed long timeout) trace_dma_fence_wait_end(fence); return ret; } -EXPORT_SYMBOL(dma_fence_wait_timeout); +EXPORT_SYMBOL(__dma_fence_wait_timeout); /** * dma_fence_release - default release function for fences @@ -759,7 +759,7 @@ dma_fence_default_wait_cb(struct dma_fence *fence, struct dma_fence_cb *cb) * functions taking a jiffies timeout. */ signed long -dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) +__dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) { struct default_wait_cb cb; unsigned long flags; @@ -808,7 +808,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) spin_unlock_irqrestore(fence->lock, flags); return ret; } -EXPORT_SYMBOL(dma_fence_default_wait); +EXPORT_SYMBOL(__dma_fence_default_wait); static bool dma_fence_test_signaled_any(struct dma_fence **fences, uint32_t count, @@ -848,7 +848,7 @@ dma_fence_test_signaled_any(struct dma_fence **fences, uint32_t count, * See also dma_fence_wait() and dma_fence_wait_timeout(). */ signed long -dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, +__dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, bool intr, signed long timeout, uint32_t *idx) { struct default_wait_cb *cb; @@ -916,7 +916,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, return ret; } -EXPORT_SYMBOL(dma_fence_wait_any_timeout); +EXPORT_SYMBOL(__dma_fence_wait_any_timeout); /** * DOC: deadline hints diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h index e7ad819962e3..d7dc7dcd213f 100644 --- a/include/linux/dma-fence.h +++ b/include/linux/dma-fence.h @@ -393,8 +393,22 @@ int dma_fence_signal_locked(struct dma_fence *fence); int dma_fence_signal_timestamp(struct dma_fence *fence, ktime_t timestamp); int dma_fence_signal_timestamp_locked(struct dma_fence *fence, ktime_t timestamp); -signed long dma_fence_default_wait(struct dma_fence *fence, +signed long __dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout); + +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_default_wait(f, intr, t) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __dma_fence_default_wait(f, intr, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) + int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb, dma_fence_func_t func); @@ -609,12 +623,37 @@ static inline ktime_t dma_fence_timestamp(struct dma_fence *fence) return fence->timestamp; } -signed long dma_fence_wait_timeout(struct dma_fence *, +signed long __dma_fence_wait_timeout(struct dma_fence *, bool intr, signed long timeout); -signed long dma_fence_wait_any_timeout(struct dma_fence **fences, +signed long __dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, bool intr, signed long timeout, uint32_t *idx); +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_wait_timeout(f, intr, t) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __dma_fence_wait_timeout(f, intr, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) + +/* + * Associate every caller with its own dept map. + */ +#define dma_fence_wait_any_timeout(fpp, count, intr, t, idx) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __dma_fence_wait_any_timeout(fpp, count, intr, t, idx); \ + sdt_might_sleep_end(); \ + __ret; \ +}) /** * dma_fence_wait - sleep until the fence gets signaled @@ -630,19 +669,24 @@ signed long dma_fence_wait_any_timeout(struct dma_fence **fences, * fence might be freed before return, resulting in undefined behavior. * * See also dma_fence_wait_timeout() and dma_fence_wait_any_timeout(). + * + * Associate every caller with its own dept map. */ -static inline signed long dma_fence_wait(struct dma_fence *fence, bool intr) -{ - signed long ret; - - /* Since dma_fence_wait_timeout cannot timeout with - * MAX_SCHEDULE_TIMEOUT, only valid return values are - * -ERESTARTSYS and MAX_SCHEDULE_TIMEOUT. - */ - ret = dma_fence_wait_timeout(fence, intr, MAX_SCHEDULE_TIMEOUT); - - return ret < 0 ? ret : 0; -} +#define dma_fence_wait(f, intr) \ +({ \ + signed long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, MAX_SCHEDULE_TIMEOUT); \ + __ret = __dma_fence_wait_timeout(f, intr, MAX_SCHEDULE_TIMEOUT);\ + sdt_might_sleep_end(); \ + \ + /* \ + * Since dma_fence_wait_timeout cannot timeout with \ + * MAX_SCHEDULE_TIMEOUT, only valid return values are \ + * -ERESTARTSYS and MAX_SCHEDULE_TIMEOUT. \ + */ \ + __ret < 0 ? __ret : 0; \ +}) void dma_fence_set_deadline(struct dma_fence *fence, ktime_t deadline); From patchwork Tue May 13 10:07:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085033 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3590-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXdq3fkJz1yYN for ; Tue, 13 May 2025 20:24:31 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 13FBF1892D1E for ; Tue, 13 May 2025 10:22:45 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 585F4269CF5; Tue, 13 May 2025 10:08:13 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1CFAA280336; Tue, 13 May 2025 10:08:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; cv=none; b=dqUptWBITFNjU8Uwi+pAK5QHbQuYRZln0F6tLTkfth9N0mOcKgFOZuOlg+wmW+qpxRTd5Saumw0Cddm9qfVXVuOaeSUpK1O/GYOwBM7Be7qu0p4RrPmRDXk4COjlZZMmzftMUJbWzU7mO/RNrc1tkEg50e6IHFBbFCHhfaE/+NM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130892; c=relaxed/simple; bh=MolzEkunzKwb0NQHelPzBSMXcYYJU6hdoDltAc6JHsU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=guEYtew2PrbByQX0eO/pU7x4C8fT9MoEVIZF89BW9k1azXiT4q2Kuo2fMxxF7rHmiFCEKDBJsaCOwh6XOlyxN4B8J/eO6ujhWocV+HJ8B34nalpivj7HwZIUHG24/J2pQdL6KOK14fvd5wgZu434ipkvqvfaHyFYK6MXcSdAOno= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-de-682319f29c68 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 34/43] dept: make dept aware of lockdep_set_lock_cmp_fn() annotation Date: Tue, 13 May 2025 19:07:21 +0900 Message-Id: <20250513100730.12664-35-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTcRSH+7/Xbbl4WVGvGV1mF7C0jIoDNQkKer8UXSC6Qa18ayNvzTIN JM1VZmlm6MrrvLBsW2lbkJUrs5yaqcvMNMxSzBo5jeWm5rI2qy+Hh+fH+Z0vR4BLGsi5AmXU SV4VJY+QUiJC5PArDnb6BypWWRwB4BpJJSC/wkiB7a4BgfF+Mgb2ui3wzj2IYKK5FQdNtg1B ce8HHO5bexBYys9R8KZ/BrS7hilozL5MQUppBQWvv3kw6M7JwsBg2gofdQMENGWWYKCxU5Cn ScG84ysG4zo9DbqkJdBXnkuDpzcUGns6SLC8Xw43C7spqLY0EmCt6sPgzaN8CnqMv0losjYQ 4M4IANu1dBLuDJVQ8M2tw0HnGqahrUaLgVU7GyrV3sILPyZJqE+vweBC2T0M2rseI3iS+gkD k7GDgueuQQzMpmwcft6qQ9CX4aDh/JVxGvKSMxBcPp9DgLp7LUyMeS8XjIRCclElAXd+daCN Ms5YaETc88FhnFObT3M/XW8pzuLWEtzLEpZ7mPuB5tRP3tOc1nSKM5cHcaXVdowrdrpIzqS/ RHEmZxbNpTnaMW6opYXePm+faEM4H6GM41Urww6JFEOGARRzY1m8pyiHTEL5C9KQUMAya1jb 7QrsP4/lXqd9TDHL2M7OcdzHs5iFrDl9gExDIgHOdExn3xV0IV8wk9nDqr/0Uz4mmCXs57ae qQUxs45tzukj/pYuYA2VNVNe6PW/brVMeQmzls3UGghfKcvkCVlbu/7fgj/7rLyTyERiLZqm RxJlVFykXBmxJkSREKWMDzkSHWlC3v/SJXr2VyGnbVctYgRI6idusC9SSEh5XGxCZC1iBbh0 ljj5gVeJw+UJZ3hV9EHVqQg+thYFCAjpHPFq9+lwCXNMfpI/zvMxvOp/igmEc5OQyD94PbPN sXM07eHwkaT6On2grMxelsUr0+cnas5OGwybDJMUpD4e237ixTPh5pTmi6GykdsGR130HHrU s+eSvEt2oHHUtTTxyjFZVuF+/VVN/VHr4dk7v29q3VHdJo2cKLq+otQvjNi7e3IgYLFz8TnN oq3ZT82lr141WVf8iN/ULSViFfLQIFwVK/8DnYI+O1sDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTH99zXUq1eEfUqYWIJkviCkog5mUb95hOSOeMXlZjNRq+2ESq2 QqmbsUhVRCFVR+t4Le1WCb0gtia+UBQhorURUbAqVlTimKQIkdFGtIoF45eTX37/nP/5ciRk bD29QKJSHxQ0akWWnJFS0k1rCpePzk9SrrR5CAiNFVFQeVFkoKvRiUC8XEDA4O2N8CQ8hODT /QckWMq6ENS+fkHC5Y4+BC11RxnofjMDekIjDHjLTjFQaL/IwMNghICA+SwBTtfP8NIxQIHP ZCPAMshAhaWQiI63BIw76llwGJKhv66chcjrNPD2+Wlor/LS0NK7FP6qDjDgafFS0HG1n4Du 65UM9IkTNPg67lIQLo2HrjMlNDQM2xgIhh0kOEIjLDxqtRLQYZ0LTcZo6/H/v9Bwp6SVgON/ XyKg51kzghtFrwhwiX4G2kNDBLhdZSR8vHAbQX/pOxaOnR5noaKgFMGpY2YKjIF0+PQherlq LA0KapooaPjsRxvWYbFaRLh9aITERrcOfww9ZnBL2ErhezYeXyt/wWLjjV4WW1252F23BNs9 gwSuHQ3R2FV/ksGu0bMsLn7XQ+Dhzk52c0KmdO1uIUuVJ2hWrNspVQ47B1DO+ZT8SI2ZNqDK hcUoRsJzq/gP5efYSWa4FP7p03FykuO4RN5dMkAXI6mE5PzT+CdVz9BkMJvbxhv/e8NMMsUl 8/8+6ptakHGr+fvmfupb6ULe2dQ65WOi/vOFzikfy6XzJquTMiGpFf1Qj+JU6rxshSorPVW7 T6lXq/JTd+3PdqHoBzkOR85cRWPdG9sQJ0Hy6bK7g4uUsbQiT6vPbkO8hJTHyQquRJVst0J/ SNDs/02TmyVo21C8hJLPk2VsFXbGcnsVB4V9gpAjaL6nhCRmgQF1KytuOUI6X6YvIz5js7Qm 1Zz8py/Y0Nusc3kyb845F7G8/2XvqGFiIt+SsMOxfkcgQbo4eOT3XwMGddI80/PtKWt0Yc8m XaZpg9ruH5gx84+co3pnYu6PpV58+ITtQXrvIjG47HHCrESxGakP/GNoDG7TL63dI563/2TP dW8pklNapSJtCanRKr4CRA6kRT0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: commit eb1cfd09f788e (lockdep: Add lock_set_cmp_fn() annotation) has been added to address the issue that lockdep was not able to detect a true deadlock like the following: https://lore.kernel.org/lkml/20220510232633.GA18445@X58A-UD3R/ The approach is only for lockdep but dept should work being aware of it because the new annotation is already used to avoid false positive of lockdep in the code. Make dept aware of the new lockdep annotation. Signed-off-by: Byungchul Park --- include/linux/dept.h | 10 +++++++++ kernel/dependency/dept.c | 48 +++++++++++++++++++++++++++++++++++----- kernel/locking/lockdep.c | 1 + 3 files changed, 53 insertions(+), 6 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index b1e5a1ec6763..19d72b0b0c4b 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -135,6 +135,11 @@ struct dept_map { const char *name; struct dept_key *keys; + /* + * keep lockdep map to handle lockdep_set_lock_cmp_fn(). + */ + void *lockdep_map; + /* * subclass that can be set from user */ @@ -161,6 +166,7 @@ struct dept_map { { \ .name = #n, \ .keys = (struct dept_key *)(k), \ + .lockdep_map = NULL, \ .sub_u = 0, \ .map_key = { .classes = { NULL, } }, \ .wgen = 0U, \ @@ -432,6 +438,8 @@ extern void dept_softirqs_on_ip(unsigned long ip); extern void dept_hardirqs_on(void); extern void dept_softirqs_off(void); extern void dept_hardirqs_off(void); + +#define dept_set_lockdep_map(m, lockdep_m) ({ (m)->lockdep_map = lockdep_m; }) #else /* !CONFIG_DEPT */ struct dept_key { }; struct dept_map { }; @@ -474,5 +482,7 @@ struct dept_ext_wgen { }; #define dept_hardirqs_on() do { } while (0) #define dept_softirqs_off() do { } while (0) #define dept_hardirqs_off() do { } while (0) + +#define dept_set_lockdep_map(m, lockdep_m) do { } while (0) #endif #endif /* __LINUX_DEPT_H */ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 8bea64bab379..5397167c7031 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1624,9 +1624,39 @@ static int next_wgen(void) return atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); } -static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep, - bool timeout) +/* + * XXX: This is a temporary patch needed until lockdep stops tracking + * dependency in wrong way. lockdep has added an annotation to specify + * a callback to determin whether the given lock aquisition order is + * okay or not in its own way. Even though dept is already working + * correctly with sub class on that issue, it needs to be aware of the + * annotation anyway. + */ +static bool lockdep_cmp_fn(struct dept_map *prev, struct dept_map *next) +{ + /* + * Assumes the cmp_fn thing comes from struct lockdep_map. + */ + struct lockdep_map *p_lock = (struct lockdep_map *)prev->lockdep_map; + struct lockdep_map *n_lock = (struct lockdep_map *)next->lockdep_map; + struct lock_class *p_class = p_lock ? p_lock->class_cache[0] : NULL; + struct lock_class *n_class = n_lock ? n_lock->class_cache[0] : NULL; + + if (!p_class || !n_class) + return false; + + if (p_class != n_class) + return false; + + if (!p_class->cmp_fn) + return false; + + return p_class->cmp_fn(p_lock, n_lock) < 0; +} + +static void add_wait(struct dept_map *m, struct dept_class *c, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool timeout) { struct dept_task *dt = dept_task(); struct dept_wait *w; @@ -1667,8 +1697,13 @@ static void add_wait(struct dept_class *c, unsigned long ip, if (!eh->ecxt) continue; - if (eh->ecxt->class != c || eh->sub_l == sub_l) - add_dep(eh->ecxt, w); + if (eh->ecxt->class == c && eh->sub_l != sub_l) + continue; + + if (i == dt->ecxt_held_pos - 1 && lockdep_cmp_fn(eh->map, m)) + continue; + + add_dep(eh->ecxt, w); } wg = next_wgen(); @@ -2154,6 +2189,7 @@ void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, m->name = n; m->wgen = 0U; m->nocheck = !valid_key(k); + m->lockdep_map = NULL; dept_exit_recursive(flags); } @@ -2377,7 +2413,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, if (!c) continue; - add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); + add_wait(m, c, ip, w_fn, sub_l, sched_sleep, timeout); } } diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index d2805ce250cb..acab023eb015 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -5036,6 +5036,7 @@ void lockdep_set_lock_cmp_fn(struct lockdep_map *lock, lock_cmp_fn cmp_fn, class->print_fn = print_fn; } + dept_set_lockdep_map(&lock->dmap, lock); lockdep_recursion_finish(); raw_local_irq_restore(flags); } From patchwork Tue May 13 10:07:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085038 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ide+bounces-3591-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXft2xCwz1yPv for ; Tue, 13 May 2025 20:25:26 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 14EF01B432EA for ; Tue, 13 May 2025 10:23:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7248228D85D; Tue, 13 May 2025 10:08:15 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C988D28688B; Tue, 13 May 2025 10:08:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130894; cv=none; b=rtXKRR5clrLttBKpwbM8lckTo3QGICuzPUupoJoqFG6n39oRXwyYZoGg0u/FO1Vsn5bgcUrdqJIINyvk53EHMgrEaWJZx3UXHhv1mKVXQT7GXe7JEL6x66x6IWfkfrx61Rr+Sqrv/adZE1MkM2rwY306AYQSSBU/XvRpN3qiSCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130894; c=relaxed/simple; bh=IO4+1SDEb7n9xPoWaW6rA/158UOork4hng4YL+8x6nw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=mqMqYarEwHhxBFUgjxQnk8lOcIGrge/j15Yhdg1w4FwTdhvEG7YONbx9CiDCy+DeLW0N7cPE4tAHrfJvyFaA/sZX58yq39doYBE0XhCHfHzmTA53cSoA6msJrm2EiqlGKD0chWBJod53TBGwTrzytuvlqhs8WkVP10tnSki4kT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-ee-682319f29217 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 35/43] dept: make dept stop from working on debug_locks_off() Date: Tue, 13 May 2025 19:07:22 +0900 Message-Id: <20250513100730.12664-36-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2xLcRjG/c85Pee0UzkrscNcK4sgLpXh/eAW14MIyz65hbJjLbtIN5sR stHJjM5IprFrN0vVVtu0Lhu6zGalLlPMLmyzlqDZZoyW1mZa4subX573eZ73y0vjkgeC8bQy LpFXxcljpKSIEPWNLJ49MG6aYp7aLAX39wwC8iuNJNgryhEYr6dh4GpcC62eXgS/nj7DQZtj R1Ds6MThurULgcVwnISX70dBs7ufBFvOaRJOXKok4XnPIAYdF85jUG7aCG/1Hwh4nF2CgdZF Qp72BOYfnzDw6sso0KeGgdOQS8GgQwa2rhYBWF7PgouFHSTctdgIsFY7MXh5O5+ELuOwAB5b HxLgyQoF+zmNAK5+LiGhx6PHQe/up+BFnQ4Dq24sVKn9hSe//RbAA00dBidLr2HQ3H4HQW1G NwYmYwsJDe5eDMymHBx8lxsROLP6KEg/46UgLy0Lwen0CwSoOxbAr5/+ywXfZZBWVEXA1aEW tHwJZyw0Iq6htx/n1OZkzud+RXIWj47gHpWwXE1uJ8Wpa19TnM50kDMbZnKX7rowrnjALeBM ZadIzjRwnuIy+5ox7nNTE7V5wlbR4ig+RpnEq+Yu3SVSfEs/ix1wiA85XDUoFRUEZSIhzTLh rMZZi/6z7dQPMsAkM51ta/PiAR7DTGHNmg+CTCSicaYliG0taPcHaHo0E8EWakMDHoIJY+8X 2v/6xcxCdvCjj/rXOZktr6r7qwv9+tDlJiLAEmYBm60rJwKdLFMkZBv7PeS/wDj2nqGNyEZi HRpRhiTKuKRYuTImfI4iJU55aM6e+FgT8r+X/ujgtmo0YI+sRwyNpCPFD11TFRKBPCkhJbYe sTQuHSNOu+WXxFHylMO8Kn6n6mAMn1CPQmlCGiKe70mOkjDR8kR+P88f4FX/txgtHJ+KqNHb 33iytOLj7KKgpOiprTdX+qonHl62Kv6ItbsmeHeDTEPf2uJdU7VD+rzC4N3X+fZK+Ka8SbZj 54QrlhYVr55xzzu8/objYv6kSMveLyHBcxPX/5Ax0xdPDHF2+iqG1mk3ZUQ+Obp7eLM6Ylfp BkN7z9cl3fS66HfVpZXJjRFnW3ulRIJCLpuJqxLkfwAJYvXyWgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxSHfd/vSrHmG+L2KW66OkOEiCMRPFGDOqO+6lxcYmJiXLSOL2sj oLaAYKIWKARBrgkS5VZxqYwWxQ/nvYTAZFYU2UCECswSRBsRkkKrQL0UF/85efKcX37nn8NT QX8wC3htQqKkS1DHqVgFrfhpTcZy9/wlmu8L+gPBM5FNQ/llKwsdlywIrFfTMLjuboEn3hEE 0w8fUVBa0oHgvLOfgqutAwhsNeksdA7NgS7PGAv2klwWMi5cZuGfVz4MfWeKMVjkHfCfeZiG tsJqDKUuFspKM7B/vMQwaa7lwGxYCoM15zjwOSPBPtDNQEuFnQGbIxzOVvaxcMdmp6H1xiCG zlvlLAxYPzDQ1nqPBm9+CHQU5TFQN1rNwiuvmQKzZ4yDf5tMGFpNX0K90d+aNf6egb/zmjBk /X4FQ1fvbQSN2c8wyNZuFlo8Ixga5BIKpi7eRTCY/5qDzNOTHJSl5SPIzTxDg7EvCqbf+i9X TERCWlU9DXXvutH6GGKttCLSMjJGEWPDUTLlecwSm9dEk/vVIrl5rp8jxkYHR0xyEmmoCSMX 7rgwOe/2MESuPcUS2V3MkZzXXZiMtrdzO7/eo1gbK8VpkyXdipj9Cs14ZgE+7FSmOF03kQFV BOagAF4UVor2U2/YGWaFULGnZ5Ka4WBhsdiQN8zkIAVPCd2B4pOKXpSDeH6u8LNYWRoyk6GF peJflR2f8kohWvS9mOL+71wkWuqbPvkAv393sZ2e4SAhSiw0WehCpDChWbUoWJuQHK/WxkVF 6A9qUhO0KRG/HoqXkf+BzMd9RTfQROeWZiTwSDVbec/1rSaIUSfrU+ObkchTqmBl2nW/Usaq U49JukP7dElxkr4ZhfC06ivltt3S/iDhN3WidFCSDku6z1vMBywwIEfo87WL1hhsuwZM92Ny Q3/JokKqkLc55mGsa/ecvcuOWTadGLq+kYzXGPgh2Re8eeGge3rXvOLGa7dvCddEoSpi1U5D 74Oq7aM/tLnfCw78tMsRvtUUkW486fzmz0tX5CXhAT736uF1YV98V5einScbdhzJPhu9ITT8 Ry5ddaCtWEXrNerIMEqnV38EzYHV2TwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: For many reasons, debug_locks_off() is called to stop lock debuging feature e.g. on panic(). dept should also stop it in the conditions. Signed-off-by: Byungchul Park --- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 6 ++++++ lib/debug_locks.c | 2 ++ 3 files changed, 10 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 19d72b0b0c4b..a6ff9db9bcf9 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -395,6 +395,7 @@ struct dept_ext_wgen { unsigned int wgen; }; +extern void dept_stop_emerg(void); extern void dept_on(void); extern void dept_off(void); extern void dept_init(void); @@ -447,6 +448,7 @@ struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } +#define dept_stop_emerg() do { } while (0) #define dept_on() do { } while (0) #define dept_off() do { } while (0) #define dept_init() do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 5397167c7031..3313ac4df3a6 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -186,6 +186,12 @@ static void dept_unlock(void) arch_spin_unlock(&dept_spin); } +void dept_stop_emerg(void) +{ + WRITE_ONCE(dept_stop, 1); +} +EXPORT_SYMBOL_GPL(dept_stop_emerg); + enum bfs_ret { BFS_CONTINUE, BFS_DONE, diff --git a/lib/debug_locks.c b/lib/debug_locks.c index a75ee30b77cb..14a965914a8f 100644 --- a/lib/debug_locks.c +++ b/lib/debug_locks.c @@ -38,6 +38,8 @@ EXPORT_SYMBOL_GPL(debug_locks_silent); */ int debug_locks_off(void) { + dept_stop_emerg(); + if (debug_locks && __debug_locks_off()) { if (!debug_locks_silent) { console_verbose(); From patchwork Tue May 13 10:07:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085031 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3592-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXdZ29MXz1yYN for ; Tue, 13 May 2025 20:24:18 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9815F4A8080 for ; Tue, 13 May 2025 10:23:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AD81C28D8E1; Tue, 13 May 2025 10:08:15 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5008D288502; Tue, 13 May 2025 10:08:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130895; cv=none; b=VIw9CyHbjHgxQgxZEyq80tQgNICbIj02ThgdCipzx6i/aQ+nJ/2ouxlkBFnzZ1PH3gDZy1x5aHNHYVLXcUrWWFsmb988e4YcHaSLeM3ITW+MQKOLL/x77ABVRQN7B+nc4zyh22DRRgW8xg7ZJi51NSmMwyR0T1BH/bCNH18cb+M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130895; c=relaxed/simple; bh=oQEw6Eo1ZD4NTR5FBfSUcltAFBYsFRqKtAgptAn2JR8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=k75douOyX2hEMb8iYX8hYP5mia8uatbz44tn8XjseS6aPrg3ooJo9kRC0i8vwfrAd5hH67W/6i9B9aVfKKqpe1KfmSNNWIGpWh1VHv0uHiMBjc9AHj6tjFlpg9ZGYXJ/T+CNxs/VoeFFa6LyepWKIn+2GF+u+83mjW4fRj/vCPA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-fe-682319f2b3f9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 36/43] i2c: rename wait_for_completion callback to wait_for_completion_cb Date: Tue, 13 May 2025 19:07:23 +0900 Message-Id: <20250513100730.12664-37-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUxTWxSG3fuMrRaPlasHNVGrRsXhglNWjAPxxX2jJhoTHyAKjZzQhklb QHCIIOAAUkEvElGwoCkNHKQWgyMGUahccxGVSVNBiKINU4K2EQS11fiy8uX7s/71snhK7WBm 8fq4BMkQp43RsEpaOTilZMVIwAJdkKskENxfTtNwpUpmoeVGBQL5VhoGV8NW6PAMIPj2/3MK CvJbEJT0vKXgVmMXglrrCRZevfeDVvcwC0352SykX6ti4UX/OAbnxfMYKuw7oNvSR8Oz3FIM BS4WLhekY+/4hGHUUs6BJXUR9FoLORjvCYamrnYGat8sg0vFThYe1DbR0HinF8Ore1dY6JJ/ MPCs8SkNHtNsaMnLYaByqJSFfo+FAot7mIOXdWYMjeYZYMvwFp78/J0BR04dhpPXb2JofX0f wcPT7zDY5XYWHrsHMFTb8ykYK2tA0Gsa5CDz7CgHl9NMCLIzL9KQ4VwL3756Lxd9CYa0qzYa KifaUchGIhfLiDweGKZIRvUhMuZuY0mtx0yT/0pFcrfwLUcyHr7hiNmeSKqtgeTaAxcmJSNu htjLz7DEPnKeI1mDrZgMNTdzO+eEKjdESjH6JMnw96YIpa7fJTMHKsVkh1yKU1GPfxZS8KKw RnQ6ZJyF+F/8MVXyaVZYLHZ2jlI+9hfmidU5fUwWUvKU0D5Z7Ch6jXzBdGGvWDhkwz6mhUXi ubF/WR+rhHVizYtx6nf/XLHCVveLFV4/UdZM+1gtrBVzzRW0r1QULijEJ21O7vdCgPjI2knn IpUZTSpHan1cUqxWH7NmpS4lTp+8cn98rB1538tybDzsDhpp2V2PBB5ppqieuubr1Iw2yZgS W49EntL4q9Jue5UqUptyWDLEhxsSYyRjPZrN05qZqlWeQ5FqIUqbIEVL0gHJ8CfFvGJWKqo6 E0ntCPnrQ2XYztFua0d0/vHt2W0hF+b07orffSr2nyNbgv26w9cr8o72v5taM0PZl2lOnFqD /VYUn91sNUY4vv+wRkeEBO1Pd5QbotYHEeTXsNqjz5vWk7MkTA49mHzcZFPaOqKK9k0sDaUT wpe3Jm6rN6XvmXnqflnAxMsjC0FDG3Xa4EDKYNT+BHvcCUVaAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x47jt9z4CcOZ2Ypok32MNX8YvyH8xWYeOvrN3VS4uyIP 253rSJTTShNxHa7TXcoVQpFaJQ+nXFKtojzeSk266EnubP5577X3e5/P+583SwTeooJYVZxW VMcpYuS0hJRsWmlY3D9jvnLpmYdTwTuQTMKVQgcN9bftCBwlegye6nXwbrAHwcir1wRkZdYj yO1sJ6CkpgNBue0kDe5Pk6HR20dDXeZZGgzXC2lo6B7F0HYxHYPdGQnvrV9IeGGyYMjy0HA5 y4B98g3DkDWfAatuAXTZshkY7QyDuo4mCqpy6igobw2BS1fbaCgrryOhprQLg/vhFRo6HOMU vKh5RsJg2kyov5BKQUGvhYbuQSsBVm8fA28qzBhqzNOgKMn39dTPPxTUplZgOHXjDobGlkcI Hid/wOB0NNFQ5e3BUOzMJGA4rxpBV9p3Boznhhi4rE9DcNZ4kYSktnAY+e1rzhkIA/21IhIK xprQ6gjBcdWBhKqePkJIKj4sDHvf0kL5oJkUnlt44UF2OyMkPW5lBLMzXii2BQvXyzxYyO33 UoIz/wwtOPvTGSHleyMWel0uZsvs7ZJV0WKMKkFUL4mIkii7PQ7qYAF/pNZhwTrUKUtBLMtz y/ivOjEFBbA0t5Bvbh4i/Czj5vLFqV+oFCRhCa5pIv8upwX5g6ncTj67twj7meQW8OeHM2g/ S7nl/L2G0X/HPDeHtxdV/OMAnz+W5yL9HMiF8yaznTQhiRlNyEcyVVxCrEIVEx6q2a9MjFMd Cd17INaJfAOynhi9UIoG3OsqEcci+STpM888ZSClSNAkxlYiniXkMqn+vs+SRisSj4rqA7vV 8TGiphLNZEn5dOn6bWJUILdPoRX3i+JBUf0/xWxAkA7d2q4n2yWhi1y2cw3V482m8BMzPt9F 8bNmrYx050XtODrlo9zyZDEcKphvXXRoNqeqythjtHPGtTvHJcFDdpd2w/mQ5Jsryn7dj5eZ OPWSLT8ieo9bdIb0IO0r22St4Zht67TSibUjGWNzuJeWXe7nuRsjk08HGJ9qN0+Y19ypWyMn NUpFWDCh1ij+AhyQGK48AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Functionally no change. This patch is a preparation for DEPT(DEPendency Tracker) to track dependencies related to a scheduler API, wait_for_completion(). Unfortunately, struct i2c_algo_pca_data has a callback member named wait_for_completion, that is the same as the scheduler API, which makes it hard to change the scheduler API to a macro form because of the ambiguity. Add a postfix _cb to the callback member to remove the ambiguity. Signed-off-by: Byungchul Park --- drivers/i2c/algos/i2c-algo-pca.c | 2 +- drivers/i2c/busses/i2c-pca-isa.c | 2 +- drivers/i2c/busses/i2c-pca-platform.c | 2 +- include/linux/i2c-algo-pca.h | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/i2c/algos/i2c-algo-pca.c b/drivers/i2c/algos/i2c-algo-pca.c index 384af88e58ad..3647e4e19360 100644 --- a/drivers/i2c/algos/i2c-algo-pca.c +++ b/drivers/i2c/algos/i2c-algo-pca.c @@ -30,7 +30,7 @@ static int i2c_debug; #define pca_clock(adap) adap->i2c_clock #define pca_set_con(adap, val) pca_outw(adap, I2C_PCA_CON, val) #define pca_get_con(adap) pca_inw(adap, I2C_PCA_CON) -#define pca_wait(adap) adap->wait_for_completion(adap->data) +#define pca_wait(adap) adap->wait_for_completion_cb(adap->data) static void pca_reset(struct i2c_algo_pca_data *adap) { diff --git a/drivers/i2c/busses/i2c-pca-isa.c b/drivers/i2c/busses/i2c-pca-isa.c index 85e8cf58e8bf..0cbf2f509527 100644 --- a/drivers/i2c/busses/i2c-pca-isa.c +++ b/drivers/i2c/busses/i2c-pca-isa.c @@ -95,7 +95,7 @@ static struct i2c_algo_pca_data pca_isa_data = { /* .data intentionally left NULL, not needed with ISA */ .write_byte = pca_isa_writebyte, .read_byte = pca_isa_readbyte, - .wait_for_completion = pca_isa_waitforcompletion, + .wait_for_completion_cb = pca_isa_waitforcompletion, .reset_chip = pca_isa_resetchip, }; diff --git a/drivers/i2c/busses/i2c-pca-platform.c b/drivers/i2c/busses/i2c-pca-platform.c index 87da8241b927..c0f35ebbe37d 100644 --- a/drivers/i2c/busses/i2c-pca-platform.c +++ b/drivers/i2c/busses/i2c-pca-platform.c @@ -180,7 +180,7 @@ static int i2c_pca_pf_probe(struct platform_device *pdev) } i2c->algo_data.data = i2c; - i2c->algo_data.wait_for_completion = i2c_pca_pf_waitforcompletion; + i2c->algo_data.wait_for_completion_cb = i2c_pca_pf_waitforcompletion; if (i2c->gpio) i2c->algo_data.reset_chip = i2c_pca_pf_resetchip; else diff --git a/include/linux/i2c-algo-pca.h b/include/linux/i2c-algo-pca.h index 7c522fdd9ea7..e305bf32e40a 100644 --- a/include/linux/i2c-algo-pca.h +++ b/include/linux/i2c-algo-pca.h @@ -71,7 +71,7 @@ struct i2c_algo_pca_data { void *data; /* private low level data */ void (*write_byte) (void *data, int reg, int val); int (*read_byte) (void *data, int reg); - int (*wait_for_completion) (void *data); + int (*wait_for_completion_cb) (void *data); void (*reset_chip) (void *data); /* For PCA9564, use one of the predefined frequencies: * 330000, 288000, 217000, 146000, 88000, 59000, 44000, 36000 From patchwork Tue May 13 10:07:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085048 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3593-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXhK01SQz1yPv for ; Tue, 13 May 2025 20:26:40 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id E6E2898772E for ; Tue, 13 May 2025 10:23:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 803EB28DEE1; Tue, 13 May 2025 10:08:16 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8E16528853E; Tue, 13 May 2025 10:08:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130896; cv=none; b=iMYdP5okT4K8CdDYTaKSZ5jm1pigRuIB8zNXM+3dgtqa0ubnzu6wfYP6AgnPs9x6bgbCrvd25o9eSbybkgl2rvrf8d7RQXthrj1isBWh3rZBXUzz6cLLxla7qKQYiDPIn/EJMsDTON8GeHGqF+lKmp4ifvVRgR14nzgRXIVh3BM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130896; c=relaxed/simple; bh=YrWBIIAMqnF09k3CiHle1jkWmEk31GBWH5LYnj+OH1k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jVf6wMpqxzyZ5kwLTYhV/8FvfNMftX6nnRsZFuxLm3P3dFvl6UTqo+eo9RLJvgPenJFesJgMPkCKNMbkJjcO7H/eMrHpkd0pmaNbfrbOM6c9ycYHbP/lxwz6ro8v/fQrtuOnw6JQNgjIJpdLwUBEYMgC/p7wLg584aBCxzeJrag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-0e-682319f279ba From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 37/43] dept: assign unique dept_key to each distinct wait_for_completion() caller Date: Tue, 13 May 2025 19:07:24 +0900 Message-Id: <20250513100730.12664-38-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTdxTG+b/XUq2+6by8BZNpBzHTeCkBPTGKuuzy/zCTZUuWBRJnI+9s Y6nkRRBMNFSqMhCGOCSKuAKu3MrAoihISQWpIIooiKC0XGIaG0C00m4IMgvGLydPnvM7zzkf joSUt9MhEq3+sCDq1TolI6WkE4uLN3gVX2g217giwTeVQcGlGgsD3f9UIbBcMxDgafsOnvrH Ecw8eEhCQX43guIRJwnXHC4EtvITDPS8WAK9vkkGOvKzGEgvrWHg0dgsAYPn8wiosu6BIbOb gs7cEgIKPAwUFqQTgfKSgGlzJQvmtHAYLb/IwuyICjpcfTTYnq2HC5cHGWiydVDguDlKQE/j JQZclv9p6HS0U+DPCYXus9k0VL8qYWDMbybB7Jtk4bHdRIDDtAJqjYHAU2/naLibbSfg1JWr BPQO3ELQnDFMgNXSx0Crb5yAOms+Ce/K2hCM5kywcPLMNAuFhhwEWSfPU2AcjIKZ/wKbi6ZU YPirloLq931o1w5suWxBuHV8ksTGuiP4ne8Jg21+E4XvlfC44aKTxcbmZyw2WZNwXfk6XNrk IXCx10dja+XvDLZ681icOdFL4FddXewPq2Kk2+MEnTZZEDdF75Nq2p/fIhIM+hRj+j0qDf0R k4mCJTwXyQ+Nj7GftKHhLJrXDLeW7++fJuf1Mm41X5ftpjORVEJyfYv4p0UDC9BnnMC/vlC6 AFFcON/sHaHmtYzbwjsDV30M/ZyvqrUvMMEB/31Z1wIj56L4XFMVNR/Kc+eC+XpbOvo4oOBv l/dTuUhmQkGVSK7VJ8ertbrIjZpUvTZl4/5D8VYU+C/zsdnYm8jb/VML4iRIuVjW7lmjkdPq 5MTU+BbES0jlMpnhRsCSxalTjwrioV/FJJ2Q2IJCJZRypSzCfyROzh1QHxYOCkKCIH7qEpLg kDS0vP56hCPi3Oa1+1LdvUt2rt7wlU4R8nO+O7pNWbG0VQxzR9M/LvWI99/83WmY29vwTUbB yKpv99/BqsKyKxE3tLH2qw1dFad37al27pXO/qYPb5rKExWKoCB70daaP13bvo+ubFxjd+qH ooaj6uv/bfzyTSET9jpL1Xh8t4yc+foXmZJK1KhV60gxUf0BoHiyA1sDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+35/T9dx9pPGr0IczWOI4uOp2Wz8MGYzbMw4+s3dXMfulGK2 TpdRSjXV0oOruFpdyh2Th0urOZ0m0blEnWq0Wqmka3rClfnns/fen9fen/cfHxHhVUz5ihSq 84JaJVNKaTEp3rc5NnDQZ5F8TZ7TF1xD10jILjPS0HC/BIHxoRZD98ud0DTci2DszVsCMtIa EOS1txLw0OpEYCm6QkPj1xlgd/XTYEtLoCG2oIyGdz3jGFrSUzGUmPbCF0MnCXXJ+RgyumnI yojF7tGFYcRQzIAhJgA6im4zMN4eBDang4KaHBsFlk8rIDO3hYbnFhsJ1ooODI1Ps2lwGv9Q UGetJWE4yQ8aUhIpKO3Lp6Fn2ECAwdXPwPsqPQarfjaU69ypV3/+puBVYhWGq3cfYLA3P0NQ ea0Ng8nooKHG1YvBbEojYLTwJYKOpO8MxN0YYSBLm4QgIS6dBF1LCIz9cl/OGQoC7Z1yEkon HGhbKG/MNSK+pref4HXmC/yo6wPNW4b1JP86n+Of3G5leF3lJ4bXmyJ4c9FyvuB5N+bzBl0U byq+TvOmwVSGj/9ux3xffT2zf94R8ZYwQamIFNSrQ0+I5bWfn+FzWlWULvY1GYNuHolHniKO Dea0T1LQpKbZJdzHjyPEpPZmF3DmxE4qHolFBOuYxjXlNE9Bs1iBG8gsmIJINoCrHGwnJ7WE Xc+1upv8C/XnSsqrphhPtz9RWD/FeLEhXLK+hExGYj3yKEbeClVkuEyhDFmlOSOPVimiVp06 G25C7g8yXB5PqUBDjTurEStC0umS2u6Fci9KFqmJDq9GnIiQeku0j92WJEwWfVFQnz2ujlAK mmrkJyKlcyS7DwsnvNjTsvPCGUE4J6j/b7HI0zcGfcAVeV1d6dKb2TsOFgXOWbzG24YPYPbW wMVvyzYOfLV45I4cOzp/5aJ7Pk7HunsHvwQc2nMyoa62L9//8tz3E37W4PKUXR7z5870N0/T vgi19fxoptcG+B5rCA/8nJVkX+1n317Wf4nesDT11o0IeRVs8omKU9YkhBnaOMujtqbcrVJS I5cFLSfUGtlfXiXIoT0DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: wait_for_completion() can be used at various points in the code and it's very hard to distinguish wait_for_completion()s between different usages. Using a single dept_key for all the wait_for_completion()s could trigger false positive reports. Assign unique dept_key to each distinct wait_for_completion() caller to avoid false positive reports. Signed-off-by: Byungchul Park --- include/linux/completion.h | 100 +++++++++++++++++++++++++++++++------ kernel/sched/completion.c | 60 +++++++++++----------- 2 files changed, 115 insertions(+), 45 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 3200b741de28..4d8fb1d95c0a 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -27,12 +27,10 @@ struct completion { unsigned int done; struct swait_queue_head wait; - struct dept_map dmap; }; #define init_completion(x) \ do { \ - sdt_map_init(&(x)->dmap); \ __init_completion(x); \ } while (0) @@ -43,17 +41,14 @@ do { \ static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start_timeout(&x->dmap, timeout); } static inline void complete_release(struct completion *x) { - sdt_might_sleep_end(); } #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ - .dmap = DEPT_MAP_INITIALIZER(work, NULL), } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), } #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -119,18 +114,18 @@ static inline void reinit_completion(struct completion *x) x->done = 0; } -extern void wait_for_completion(struct completion *); -extern void wait_for_completion_io(struct completion *); -extern int wait_for_completion_interruptible(struct completion *x); -extern int wait_for_completion_killable(struct completion *x); -extern int wait_for_completion_state(struct completion *x, unsigned int state); -extern unsigned long wait_for_completion_timeout(struct completion *x, +extern void __wait_for_completion(struct completion *); +extern void __wait_for_completion_io(struct completion *); +extern int __wait_for_completion_interruptible(struct completion *x); +extern int __wait_for_completion_killable(struct completion *x); +extern int __wait_for_completion_state(struct completion *x, unsigned int state); +extern unsigned long __wait_for_completion_timeout(struct completion *x, unsigned long timeout); -extern unsigned long wait_for_completion_io_timeout(struct completion *x, +extern unsigned long __wait_for_completion_io_timeout(struct completion *x, unsigned long timeout); -extern long wait_for_completion_interruptible_timeout( +extern long __wait_for_completion_interruptible_timeout( struct completion *x, unsigned long timeout); -extern long wait_for_completion_killable_timeout( +extern long __wait_for_completion_killable_timeout( struct completion *x, unsigned long timeout); extern bool try_wait_for_completion(struct completion *x); extern bool completion_done(struct completion *x); @@ -139,4 +134,79 @@ extern void complete(struct completion *); extern void complete_on_current_cpu(struct completion *x); extern void complete_all(struct completion *); +#define wait_for_completion(x) \ +({ \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __wait_for_completion(x); \ + sdt_might_sleep_end(); \ +}) +#define wait_for_completion_io(x) \ +({ \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __wait_for_completion_io(x); \ + sdt_might_sleep_end(); \ +}) +#define wait_for_completion_interruptible(x) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret = __wait_for_completion_interruptible(x); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_killable(x) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret = __wait_for_completion_killable(x); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_state(x, s) \ +({ \ + int __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, -1L); \ + __ret = __wait_for_completion_state(x, s); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_timeout(x, t) \ +({ \ + unsigned long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __wait_for_completion_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_io_timeout(x, t) \ +({ \ + unsigned long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __wait_for_completion_io_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_interruptible_timeout(x, t) \ +({ \ + long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __wait_for_completion_interruptible_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) +#define wait_for_completion_killable_timeout(x, t) \ +({ \ + long __ret; \ + \ + sdt_might_sleep_start_timeout(NULL, t); \ + __ret = __wait_for_completion_killable_timeout(x, t); \ + sdt_might_sleep_end(); \ + __ret; \ +}) #endif diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 499b1fee9dc1..247169b26d81 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -4,7 +4,7 @@ * Generic wait-for-completion handler; * * It differs from semaphores in that their default case is the opposite, - * wait_for_completion default blocks whereas semaphore default non-block. The + * __wait_for_completion default blocks whereas semaphore default non-block. The * interface also makes it easy to 'complete' multiple waiting threads, * something which isn't entirely natural for semaphores. * @@ -37,7 +37,7 @@ void complete_on_current_cpu(struct completion *x) * This will wake up a single thread waiting on this completion. Threads will be * awakened in the same order in which they were queued. * - * See also complete_all(), wait_for_completion() and related routines. + * See also complete_all(), __wait_for_completion() and related routines. * * If this function wakes up a task, it executes a full memory barrier before * accessing the task state. @@ -134,23 +134,23 @@ wait_for_common_io(struct completion *x, long timeout, int state) } /** - * wait_for_completion: - waits for completion of a task + * __wait_for_completion: - waits for completion of a task * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It is NOT * interruptible and there is no timeout. * - * See also similar routines (i.e. wait_for_completion_timeout()) with timeout + * See also similar routines (i.e. __wait_for_completion_timeout()) with timeout * and interrupt capability. Also see complete(). */ -void __sched wait_for_completion(struct completion *x) +void __sched __wait_for_completion(struct completion *x) { wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion); +EXPORT_SYMBOL(__wait_for_completion); /** - * wait_for_completion_timeout: - waits for completion of a task (w/timeout) + * __wait_for_completion_timeout: - waits for completion of a task (w/timeout) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -162,28 +162,28 @@ EXPORT_SYMBOL(wait_for_completion); * till timeout) if completed. */ unsigned long __sched -wait_for_completion_timeout(struct completion *x, unsigned long timeout) +__wait_for_completion_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_timeout); +EXPORT_SYMBOL(__wait_for_completion_timeout); /** - * wait_for_completion_io: - waits for completion of a task + * __wait_for_completion_io: - waits for completion of a task * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It is NOT * interruptible and there is no timeout. The caller is accounted as waiting * for IO (which traditionally means blkio only). */ -void __sched wait_for_completion_io(struct completion *x) +void __sched __wait_for_completion_io(struct completion *x) { wait_for_common_io(x, MAX_SCHEDULE_TIMEOUT, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_io); +EXPORT_SYMBOL(__wait_for_completion_io); /** - * wait_for_completion_io_timeout: - waits for completion of a task (w/timeout) + * __wait_for_completion_io_timeout: - waits for completion of a task (w/timeout) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -196,14 +196,14 @@ EXPORT_SYMBOL(wait_for_completion_io); * till timeout) if completed. */ unsigned long __sched -wait_for_completion_io_timeout(struct completion *x, unsigned long timeout) +__wait_for_completion_io_timeout(struct completion *x, unsigned long timeout) { return wait_for_common_io(x, timeout, TASK_UNINTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_io_timeout); +EXPORT_SYMBOL(__wait_for_completion_io_timeout); /** - * wait_for_completion_interruptible: - waits for completion of a task (w/intr) + * __wait_for_completion_interruptible: - waits for completion of a task (w/intr) * @x: holds the state of this particular completion * * This waits for completion of a specific task to be signaled. It is @@ -211,7 +211,7 @@ EXPORT_SYMBOL(wait_for_completion_io_timeout); * * Return: -ERESTARTSYS if interrupted, 0 if completed. */ -int __sched wait_for_completion_interruptible(struct completion *x) +int __sched __wait_for_completion_interruptible(struct completion *x) { long t = wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE); @@ -219,10 +219,10 @@ int __sched wait_for_completion_interruptible(struct completion *x) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_interruptible); +EXPORT_SYMBOL(__wait_for_completion_interruptible); /** - * wait_for_completion_interruptible_timeout: - waits for completion (w/(to,intr)) + * __wait_for_completion_interruptible_timeout: - waits for completion (w/(to,intr)) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -233,15 +233,15 @@ EXPORT_SYMBOL(wait_for_completion_interruptible); * or number of jiffies left till timeout) if completed. */ long __sched -wait_for_completion_interruptible_timeout(struct completion *x, +__wait_for_completion_interruptible_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_INTERRUPTIBLE); } -EXPORT_SYMBOL(wait_for_completion_interruptible_timeout); +EXPORT_SYMBOL(__wait_for_completion_interruptible_timeout); /** - * wait_for_completion_killable: - waits for completion of a task (killable) + * __wait_for_completion_killable: - waits for completion of a task (killable) * @x: holds the state of this particular completion * * This waits to be signaled for completion of a specific task. It can be @@ -249,7 +249,7 @@ EXPORT_SYMBOL(wait_for_completion_interruptible_timeout); * * Return: -ERESTARTSYS if interrupted, 0 if completed. */ -int __sched wait_for_completion_killable(struct completion *x) +int __sched __wait_for_completion_killable(struct completion *x) { long t = wait_for_common(x, MAX_SCHEDULE_TIMEOUT, TASK_KILLABLE); @@ -257,9 +257,9 @@ int __sched wait_for_completion_killable(struct completion *x) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_killable); +EXPORT_SYMBOL(__wait_for_completion_killable); -int __sched wait_for_completion_state(struct completion *x, unsigned int state) +int __sched __wait_for_completion_state(struct completion *x, unsigned int state) { long t = wait_for_common(x, MAX_SCHEDULE_TIMEOUT, state); @@ -267,10 +267,10 @@ int __sched wait_for_completion_state(struct completion *x, unsigned int state) return t; return 0; } -EXPORT_SYMBOL(wait_for_completion_state); +EXPORT_SYMBOL(__wait_for_completion_state); /** - * wait_for_completion_killable_timeout: - waits for completion of a task (w/(to,killable)) + * __wait_for_completion_killable_timeout: - waits for completion of a task (w/(to,killable)) * @x: holds the state of this particular completion * @timeout: timeout value in jiffies * @@ -282,12 +282,12 @@ EXPORT_SYMBOL(wait_for_completion_state); * or number of jiffies left till timeout) if completed. */ long __sched -wait_for_completion_killable_timeout(struct completion *x, +__wait_for_completion_killable_timeout(struct completion *x, unsigned long timeout) { return wait_for_common(x, timeout, TASK_KILLABLE); } -EXPORT_SYMBOL(wait_for_completion_killable_timeout); +EXPORT_SYMBOL(__wait_for_completion_killable_timeout); /** * try_wait_for_completion - try to decrement a completion without blocking @@ -329,7 +329,7 @@ EXPORT_SYMBOL(try_wait_for_completion); * completion_done - Test to see if a completion has any waiters * @x: completion structure * - * Return: 0 if there are waiters (wait_for_completion() in progress) + * Return: 0 if there are waiters (__wait_for_completion() in progress) * 1 if there are no waiters. * * Note, this will always return true if complete_all() was called on @X. From patchwork Tue May 13 10:07:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085035 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3594-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXfd3j9Hz1yPv for ; Tue, 13 May 2025 20:25:13 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 157FD163551 for ; Tue, 13 May 2025 10:24:07 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0A30D28DF15; Tue, 13 May 2025 10:08:17 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C9B6928B7EA; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130896; cv=none; b=lWYWPM5piMVNXJiLekfmachphGYQ8MX6mPymLDoxKJq8aJMGyySPkpizdljnqOyQOY6H/JmxDCZvL4uXl6KCeV9h5OVzsNDe7mouQx31GSXn1v66EgxWbktF8Zyx16ciEElnQnbADsohuoL/R8ijr4c4MTzZs8Zp3CAkDijCkEE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130896; c=relaxed/simple; bh=314TpUAkZxMvbV2pk3wlciVKMbaC2tgJL1bvCZ48Y0A=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=BeRSUqCWqQUSBXFtSdWvjiPewnGbMz0MLeAdoEdJk8PdLxSyysstM2X4uOjHdVP/PV4FJTQB0pvwwMIcBEBGTBrroB0JwdlPCCnN+qPiOLOst4yT0efEfNhInBi5tcu1WDkDUU4mH7X9y5aDYbf1glhS375EcRFc8DON42sspx8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-1e-682319f26b45 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 38/43] completion, dept: introduce init_completion_dmap() API Date: Tue, 13 May 2025 19:07:25 +0900 Message-Id: <20250513100730.12664-39-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+59zds5xuTjMqKMF1souRpbdeIluHwIPhKB0+VBRrTy00Vyx mWkQzZpm2sQUk9RqzpyXzbQZZJctc7m00FbZNDNLqWjlJaZb6eyyFX15efj9eJ/3y0vj4lZB BC1XpvAqpVQhIYWEcDi0fPlY+ALZSvtm8I5nE1BWbybBecOEwHwrAwN3axx0+4YQ+Due4VBc 5ERQPvAWh1uOfgTW6jMkvPwwA7q8oyS0F+WScLainoTnX6cw6LtUgIHJEg/vjJ8IeJpvwKDY TUJp8VksMD5jMGGspcCoiYLB6hIKpgZiob3fJQBr7zK4fLWPhPvWdgIcTYMYvLxbRkK/+bcA njraCPDlzQHnRZ0A6kYMJHz1GXEwekcpeNGsx8ChnwUN2oAwa+yXAB7rmjHIun4Tg67X9xDY st9jYDG7SLB7hzBotBThMFnVimAwb5iCzAsTFJRm5CHIzbxEgLZvLfh/BC5fGY+FjGsNBNT9 dKEtGznzVTPi7EOjOKdtPMFNel+RnNWnJ7gnBpa7U/KW4rS2XorTW45zjdXRXMV9N8aVe7wC zlJ7nuQsngKKyxnuwriRzk4qYe5u4YYkXiFP5VUrNh0QyirrS8lj1qVpj8bykQZNRuagEJpl 1rD2mxUoB9F/ebxnfTAmmcVsT88EHuSZzDy2UfdJkIOENM64prPdV16jYBHGJLLtlVNUkAkm ir1m+0IEPSJmHav7fvqfPpI1NTT/9YQE4p9VnUSQxcxaNl9vIoJOlikMYTve5KF/C+Hsw+oe Ih+J9GhaLRLLlanJUrliTYwsXSlPizl0NNmCAs9lPDW1pwl5nNtbEEMjSaiozT1fJhZIU9Xp yS2IpXHJTFHG7UAkSpKmn+RVR/erjit4dQuaQxOS2aJVvhNJYuawNIU/wvPHeNX/FqNDIjSo 7IHNs3jXt22/2aKUvSKnf0lHYfySTn/cHnvX97tLKzP98SlKtfbjAl2uOFVxxJRYlWC7Ufdw Z/TY89XT00yyibCGkiw3Tmxl9Oc+RGyo8xpqeg2e+A5fPZ1ow12nT+1IGHF0uzVbS28/aorU 7FtYmB12Z796eNG60CTpQazGISHUMmlsNK5SS/8AUlSKS1gDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHcfP1fipkc4aMg9tss+IGRvf2ZiHNuaPOPrN3VTsjsg0 pUOiWyGhp6s4rS6Xq608XFrldJrEpdIqatacUlvdHZ0Ldzb/fPba67293/98OFJWTgdzqsRT ojpRES9nJJRk14b0lRMLlijX9N4JA5czg4ICk5GBjkeVCIy1aQQ4Xm6Hbvcogl9v3pKQl9uB oGSwn4Ra6wACS/lFBuxfZkOna5wBW+41BtLLTAy8G/ES0Hf7BgGV5p3wyTBMQVt2KQF5Dgby 89IJ3/lKwJShggVDajgMld9jwTsYCbaBLhqaC200WHpXwN2iPgaeW2wUWOuHCLA/LWBgwPiH hjZrKwVuXQh05GTRUDVWysCI20CCwTXOwvtGPQFW/Tyo1vpaL0/+puFVViMBl+8/JqDz4zME DRmfCTAbuxhodo0SUGPOJcHz8CWCId13Fi5dn2IhP02H4Nql2xRo+6Lg10/fcqEzEtKKqymo mu5CmzdhY5ER4ebRcRJra85gj+sDgy1uPYVflwr4yb1+FmsbelmsN5/GNeURuOy5g8AlEy4a myuuMtg8cYPFmd87CTzW3s7uXnhQEh0nxquSRPXqTYclygemfOakZfnZlslslIo8oZmI4wR+ reDsWZ+JAjiGXyr09EyRfg7iFws1WcN0JpJwJN81U+gu/Ij8QSC/R7A98LJ+pvhwobjhG+Xv kfLrhKwfF/xa4EOFyurGfz0BPj39sJ3ys4yPErL1lVQ2kujRjAoUpEpMSlCo4qNWaY4rkxNV Z1cdPZFgRr7/MaR4c+qR0769CfEcks+StjrClDJakaRJTmhCAkfKg6RpdT4ljVMknxPVJw6p T8eLmiYUwlHy+dId+8XDMv6Y4pR4XBRPiur/KcEFBKcifk2Z1XNzsa5+7Ixs+MoO5uhIlDfs ae67IndoRorHkJNxaG5LccyLwPcH+BKTIWb6oj3Y/mr2TMetulmLApe9aBmKiwhp+9M9ke7s aKyl3lSEx0ajW5PwE5vm3FnfGtNzPrSueN8ztSe6fzB225GtHzYGXXXpe7e4E9ZtTNmr03bK KY1SERlBqjWKvz/l+6A7AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Currently, dept uses dept's map embedded in task_struct to track dependencies related to wait_for_completion() and its family. So it doesn't need an explicit map basically. However, for those who want to set the maps with customized class or key, introduce a new API to use external maps. Signed-off-by: Byungchul Park --- include/linux/completion.h | 40 +++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 4d8fb1d95c0a..e50f7d9b4b97 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -27,17 +27,15 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map *dmap; }; -#define init_completion(x) \ -do { \ - __init_completion(x); \ -} while (0) +#define init_completion(x) init_completion_dmap(x, NULL) /* - * XXX: No use cases for now. Fill the body when needed. + * XXX: This usage using lockdep's map should be deprecated. */ -#define init_completion_map(x, m) init_completion(x) +#define init_completion_map(x, m) init_completion_dmap(x, NULL) static inline void complete_acquire(struct completion *x, long timeout) { @@ -48,8 +46,11 @@ static inline void complete_release(struct completion *x) } #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), .dmap = NULL, } +/* + * XXX: This usage using lockdep's map should be deprecated. + */ #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -90,15 +91,18 @@ static inline void complete_release(struct completion *x) #endif /** - * __init_completion - Initialize a dynamically allocated completion + * init_completion_dmap - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized + * @dmap: pointer to external dept's map to be used as a separated map * * This inline function will initialize a dynamically created completion * structure. */ -static inline void __init_completion(struct completion *x) +static inline void init_completion_dmap(struct completion *x, + struct dept_map *dmap) { x->done = 0; + x->dmap = dmap; init_swait_queue_head(&x->wait); } @@ -136,13 +140,13 @@ extern void complete_all(struct completion *); #define wait_for_completion(x) \ ({ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __wait_for_completion(x); \ sdt_might_sleep_end(); \ }) #define wait_for_completion_io(x) \ ({ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __wait_for_completion_io(x); \ sdt_might_sleep_end(); \ }) @@ -150,7 +154,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret = __wait_for_completion_interruptible(x); \ sdt_might_sleep_end(); \ __ret; \ @@ -159,7 +163,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret = __wait_for_completion_killable(x); \ sdt_might_sleep_end(); \ __ret; \ @@ -168,7 +172,7 @@ extern void complete_all(struct completion *); ({ \ int __ret; \ \ - sdt_might_sleep_start_timeout(NULL, -1L); \ + sdt_might_sleep_start_timeout((x)->dmap, -1L); \ __ret = __wait_for_completion_state(x, s); \ sdt_might_sleep_end(); \ __ret; \ @@ -177,7 +181,7 @@ extern void complete_all(struct completion *); ({ \ unsigned long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret = __wait_for_completion_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -186,7 +190,7 @@ extern void complete_all(struct completion *); ({ \ unsigned long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret = __wait_for_completion_io_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -195,7 +199,7 @@ extern void complete_all(struct completion *); ({ \ long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret = __wait_for_completion_interruptible_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ @@ -204,7 +208,7 @@ extern void complete_all(struct completion *); ({ \ long __ret; \ \ - sdt_might_sleep_start_timeout(NULL, t); \ + sdt_might_sleep_start_timeout((x)->dmap, t); \ __ret = __wait_for_completion_killable_timeout(x, t); \ sdt_might_sleep_end(); \ __ret; \ From patchwork Tue May 13 10:07:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085050 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3599-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXhT4JFpz1yYx for ; Tue, 13 May 2025 20:26:49 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4D69B17FA0B for ; Tue, 13 May 2025 10:25:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EEBD127587F; Tue, 13 May 2025 10:08:18 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C996028B51F; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130898; cv=none; b=uhWDtoxUM1K+k+ve8PNp1SRujShZ3XK0Nq7mOgw2gvlYa2wxIuTvXB0chQHyccwWXSJNN19WqS607SeLJg4MhbVFgA/yqWJMcVSr+0lWoHp7fVvTv0BLRWeQCKBXqsBWgfelD3fUTH/Vbv/7lqA+SmuWqbSLpHmgH7BRBWA5rkA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130898; c=relaxed/simple; bh=gEgyg/9rYxrXc1J42HwM7TQpw+QbrmZR0z4Ef2LtTSc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=hab17Seb66PDggGwrcMlj9oAWitUlwwmxQ/8R01cqnVbCmYTXnjug1uxKOgLle5E/Ms3WjPdeUQ9/MgiaDh4XDKPYdlHp4WH1YUovWrp7dBdXJfnfajAsii2Gmd9UFnfyz7nnHsh+lQJ8Gkx3VuS/jlh4Ei6cUX9HaicEoAWlC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-2e-682319f2e398 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 39/43] dept: introduce a new type of dependency tracking between multi event sites Date: Tue, 13 May 2025 19:07:26 +0900 Message-Id: <20250513100730.12664-40-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUyTZxSGfd7vVmpeK5kPYKLWEBPmx9jEnKCYJcb4/NGoxD8ao814tY0F tMinGosgKAqiAYkI0qKptS0fa52CWoIQK+gGKKyCA1SixgpIgrQRQVzL5p+TK/d9zn3/OQKt bGPDBW3SEUmfpNapODkjHw0xrfwUtkzzk+0tA76J0wxU1Nk56Kq1IbDfyqbA+3AzPPePIJj6 q5OGstIuBKbXAzTccg8icFlOctD9Zh70+MY4aC89y0HOtToOng5PU9B/6SIFNscWeGl+x8CT 4moKyrwcXCnLoQLjPQWTZisPZkMkDFnKeZh+HQ3tgx4WXC9+hMtX+zm472pnwN0wREH33QoO Bu3fWHjibmPAXxQBXRcKWaj5WM3BsN9Mg9k3xsOzZiMFbuMPUJ8bCMz7NMPCo8JmCvKu/05B T989BE2nX1HgsHs4aPWNUOB0lNLw5cZDBENFozycOjfJw5XsIgRnT11iILc/BqY+B5orJ6Ih u6qegZqvHvRrHLFftSPSOjJGk1xnOvni+5sjLr+RIY+rMWksH+BJbtMLnhgdqcRpiSLX7nsp Yhr3scRhPcMRx/hFnhSM9lDkY0cHv23RLvn6BEmnTZP0qzfsk2veTw3zh6wJGTWdRtqAjNsL kEzA4hr87EMF+50bz3jpIHPictzbOznLoeIS7Cx8F9iRC7TomYufV/ahoLFA3I+nSx7PMiNG 4k5vORdkhbgWF1hH6f9CF2NbffMsywL61xsdTJCVYgwuNtqYYCgWz8uwyznz/0EYfmDpZYqR wojmWJFSm5SWqNbq1qzSZCZpM1b9lpzoQIH/Mh+f3t2AxrviW5AoIFWIos27VKNk1WkpmYkt CAu0KlSRfScgKRLUmVmSPnmvPlUnpbSgCIFRLVT87E9PUIoH1Eekg5J0SNJ/dylBFm5AJWQZ zOOFFYYxW9g/x6JySv64a+j7M0c70P3IFRc2E1Pldrei9e75dzYejq1Fpk1H8bbYusj4WvL0 lw2X9QsPhyTmrzN13MswHcgKvbljp+f22xlh8YXkExO4N31n077be15ZirM0Wxt3pVZWxcga CkaWLj+Z/iCfDWdDa9ROXayKSdGoo6NofYr6X8Q6CjhbAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSfUzMcRwHcN/fc8fZb2n1w0zOmocobcVnWPgDX8zjP202dPTTHdfD7ohs baWE6FQkeuDn2GnXlfzO5vFopThNRamkQqx1Klm6owcPV+afz177vLf3558PR3pb6BmcNvaQ qI9V61SMglJsWZG6eHD6XM0S+7lAcA2doqDwlpWBhrISBNY7KQQ4q9dDi7sPwejLehLychsQ XPvYQcKdmk4E9uLjDDR+ngpNrgEGHLlnGEi9fouBV71jBLRfzCGgRN4M783dFNRmmQjIczJQ kJdKeEYPAcNmCwvm5ADoKs5nYexjCDg6m2moKnLQYG8LhMtX2hl4ZHdQUHOvi4DGB4UMdFr/ 0FBb85wCt3EmNGRn0lD61cRAr9tMgtk1wMLrComAGskXytM8renff9PwLLOCgPQbtwloevsQ weNTHwiQrc0MVLn6CLDJuSSM3KxG0GXsZ+HE2WEWClKMCM6cuEhBWnsYjP70XC4aCoGUq+UU lP5qRqvDsfWKFeGqvgESp9mO4BHXGwbb3RKFX5gEfD+/g8Vpj9tYLMmHsa14Ib7+yEnga4Mu GsuW0wyWB3NYnNHfROCvdXXstlk7FSujRJ02QdQHh0cqND2jvWy8Jepoab1EJiNpewby4gQ+ VLh/2kmOm+HnCa2twxP24f0FW2Y3nYEUHMk3TxZait6i8WAav18Yu/BiwhQfINQ785lxK/ml Qoaln/xXOlsoKa+YsJdn/+tmHTVubz5MyJJKqCykkNAkC/LRxibEqLW6sCDDQU1irPZo0L64 GBl5PsicNJZ9Dw01rq9EPIdUU5TPnXM03rQ6wZAYU4kEjlT5KFPuelbKKHXiMVEft0d/WCca KtFMjlL5KTdGiJHefLT6kHhQFONF/f+U4LxmJKNPa4O2Sq6t60Z0/nhZYGOM6eke/1WmA7ME 2BHOXvqt4CMNT3RyaNuudwte71X6zpeO8CHpET2aONv5V0nZG1pOLgr2E93pL0eSpGhNvTFn le/ubcihfdgz2S5vWuMz7VuApcPYVlabLE8t7LZennI2NPTH8vzjfV/uEn67z3PBKsqgUYcs JPUG9V/WobTYPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It's worth reporting wait-event circular dependency even if it doesn't lead to an actual deadlock, because it's a good information about a circular dependency anyway. However, it should be suppressed once turning out it doesn't lead an actual deadlock, for instance, there are other wake-up(or event) paths. The report needs to be suppressed by annotating that an event can be recovered by other sites triggering the desired wake-up, using a newly introduced API, dept_recover_event() specifying an event site and its recover site. By the introduction, need of a new type of dependency tracking arises since a loop of recover dependency could trigger another type of deadlock. So implement a logic to track the new type of dependency between multi event sites for a single wait. Lastly, to make sure that recover sites must be used in code, introduce a section '.dept.event_sites' to mark it as 'used' only if used in code, and warn it if dept_recover_event()s are annotated with recover sites, not used in code. Signed-off-by: Byungchul Park --- include/asm-generic/vmlinux.lds.h | 13 +- include/linux/dept.h | 91 ++++++++++++++ kernel/dependency/dept.c | 197 ++++++++++++++++++++++++++++++ 3 files changed, 300 insertions(+), 1 deletion(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 58a635a6d5bd..fd24b35fb379 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -699,6 +699,16 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG) #define KERNEL_CTORS() #endif +#ifdef CONFIG_DEPT +#define DEPT_EVNET_SITES_USED() \ + . = ALIGN(8); \ + __dept_event_sites_start = .; \ + KEEP(*(.dept.event_sites)) \ + __dept_event_sites_end = .; +#else +#define DEPT_EVNET_SITES_USED() +#endif + /* init and exit section handling */ #define INIT_DATA \ KEEP(*(SORT(___kentry+*))) \ @@ -723,7 +733,8 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG) EARLYCON_TABLE() \ LSM_TABLE() \ EARLY_LSM_TABLE() \ - KUNIT_INIT_TABLE() + KUNIT_INIT_TABLE() \ + DEPT_EVNET_SITES_USED() #define INIT_TEXT \ *(.init.text .init.text.*) \ diff --git a/include/linux/dept.h b/include/linux/dept.h index a6ff9db9bcf9..b933f9d98c69 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -395,6 +395,82 @@ struct dept_ext_wgen { unsigned int wgen; }; +struct dept_event_site { + /* + * event site name + */ + const char *name; + + /* + * function name where the event is triggered in + */ + const char *func_name; + + /* + * for associating its recover dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for BFS + */ + unsigned int bfs_gen; + struct dept_event_site *bfs_parent; + struct list_head bfs_node; + + /* + * flag indicating the event is not only declared but also + * actually used in code + */ + bool used; +}; + +struct dept_event_site_dep { + struct dept_event_site *evt_site; + struct dept_event_site *recover_site; + + /* + * for linking to dept_event objects + */ + struct list_head dep_node; + struct list_head dep_rev_node; +}; + +#define DEPT_EVENT_SITE_INITIALIZER(es) \ +{ \ + .name = #es, \ + .func_name = NULL, \ + .dep_head = LIST_HEAD_INIT((es).dep_head), \ + .dep_rev_head = LIST_HEAD_INIT((es).dep_rev_head), \ + .bfs_gen = 0, \ + .bfs_parent = NULL, \ + .bfs_node = LIST_HEAD_INIT((es).bfs_node), \ + .used = false, \ +} + +#define DEPT_EVENT_SITE_DEP_INITIALIZER(esd) \ +{ \ + .evt_site = NULL, \ + .recover_site = NULL, \ + .dep_node = LIST_HEAD_INIT((esd).dep_node), \ + .dep_rev_node = LIST_HEAD_INIT((esd).dep_rev_node), \ +} + +struct dept_event_site_init { + struct dept_event_site *evt_site; + const char *func_name; +}; + +#define dept_event_site_used(es) \ +do { \ + static struct dept_event_site_init _evtinit __initdata = \ + { .evt_site = (es), .func_name = __func__ }; \ + static struct dept_event_site_init *_evtinitp __used \ + __attribute__((__section__(".dept.event_sites"))) = \ + &_evtinit; \ +} while (0) + extern void dept_stop_emerg(void); extern void dept_on(void); extern void dept_off(void); @@ -432,6 +508,14 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) extern void dept_key_init(struct dept_key *k); extern void dept_key_destroy(struct dept_key *k); extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, struct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const char *new_c_fn, const char *new_e_fn, int new_sub_l); +extern void __dept_recover_event(struct dept_event_site_dep *esd, struct dept_event_site *es, struct dept_event_site *rs); + +#define dept_recover_event(es, rs) \ +do { \ + static struct dept_event_site_dep _esd = DEPT_EVENT_SITE_DEP_INITIALIZER(_esd);\ + \ + __dept_recover_event(&_esd, es, rs); \ +} while (0) extern void dept_softirq_enter(void); extern void dept_hardirq_enter(void); @@ -445,8 +529,10 @@ extern void dept_hardirqs_off(void); struct dept_key { }; struct dept_map { }; struct dept_ext_wgen { }; +struct dept_event_site { }; #define DEPT_MAP_INITIALIZER(n, k) { } +#define DEPT_EVENT_SITE_INITIALIZER(es) { } #define dept_stop_emerg() do { } while (0) #define dept_on() do { } while (0) @@ -477,6 +563,7 @@ struct dept_ext_wgen { }; #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) #define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) +#define dept_recover_event(es, rs) do { } while (0) #define dept_softirq_enter() do { } while (0) #define dept_hardirq_enter() do { } while (0) @@ -487,4 +574,8 @@ struct dept_ext_wgen { }; #define dept_set_lockdep_map(m, lockdep_m) do { } while (0) #endif + +#define DECLARE_DEPT_EVENT_SITE(es) extern struct dept_event_site (es) +#define DEFINE_DEPT_EVENT_SITE(es) struct dept_event_site (es) = DEPT_EVENT_SITE_INITIALIZER(es) + #endif /* __LINUX_DEPT_H */ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 3313ac4df3a6..b7167ce4f647 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -979,6 +979,118 @@ static void bfs(void *root, struct bfs_ops *ops, void *in, void **out) } } +/* + * Recover dependency between event sites + * ===================================================================== + * Even though an event is in a chain of wait-event circular dependency, + * the corresponding wait might be woken up by another site triggering + * the desired event. To reflect that, dept allows to annotate the + * recover relationship between event sites using __dept_recover_event(). + * However, that requires to track a new type of dependency between the + * event sites. + */ + +/* + * Print all events in the circle. + */ +static void print_recover_circle(struct dept_event_site *es) +{ + struct dept_event_site *from = es->bfs_parent; + struct dept_event_site *to = es; + + dept_outworld_enter(); + + pr_warn("===================================================\n"); + pr_warn("DEPT: Circular recover dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + + do { + print_spc(1, "event site(%s@%s)\n", from->name, from->func_name); + print_spc(1, "-> event site(%s@%s)\n", to->name, to->func_name); + to = from; + from = from->bfs_parent; + + if (to != es) + pr_warn("\n"); + } while (to != es); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +static void bfs_init_recover(void *node, void *in, void **out) +{ + struct dept_event_site *root = (struct dept_event_site *)node; + struct dept_event_site_dep *new = (struct dept_event_site_dep *)in; + + root->bfs_gen = bfs_gen; + new->recover_site->bfs_parent = new->evt_site; +} + +static void bfs_extend_recover(struct list_head *h, void *node) +{ + struct dept_event_site *cur = (struct dept_event_site *)node; + struct dept_event_site_dep *esd; + + list_for_each_entry(esd, &cur->dep_head, dep_node) { + struct dept_event_site *next = esd->recover_site; + + if (bfs_gen == next->bfs_gen) + continue; + next->bfs_parent = cur; + next->bfs_gen = bfs_gen; + list_add_tail(&next->bfs_node, h); + } +} + +static void *bfs_dequeue_recover(struct list_head *h) +{ + struct dept_event_site *es; + + DEPT_WARN_ON(list_empty(h)); + + es = list_first_entry(h, struct dept_event_site, bfs_node); + list_del(&es->bfs_node); + return es; +} + +static enum bfs_ret cb_check_recover_dl(void *node, void *in, void **out) +{ + struct dept_event_site *cur = (struct dept_event_site *)node; + struct dept_event_site_dep *new = (struct dept_event_site_dep *)in; + + if (cur == new->evt_site) { + print_recover_circle(new->recover_site); + dept_stop_emerg(); + return BFS_DONE; + } + + return BFS_CONTINUE; +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void check_recover_dl_bfs(struct dept_event_site_dep *esd) +{ + struct bfs_ops ops = { + .bfs_init = bfs_init_recover, + .extend = bfs_extend_recover, + .dequeue = bfs_dequeue_recover, + .callback = cb_check_recover_dl, + }; + + bfs((void *)esd->recover_site, &ops, (void *)esd, NULL); +} + /* * Main operations * ===================================================================== @@ -3176,8 +3288,78 @@ static void migrate_per_cpu_pool(void) } } +static bool dept_recover_ready; + +void __dept_recover_event(struct dept_event_site_dep *esd, + struct dept_event_site *es, struct dept_event_site *rs) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (!esd || !es || !rs) { + DEPT_WARN_ONCE("All the parameters should be !NULL.\n"); + return; + } + + /* + * Check locklessly if another already has done it for us. + */ + if (READ_ONCE(esd->evt_site)) + return; + + if (!dept_recover_ready) { + DEPT_WARN("Should be called once dept_recover_ready.\n"); + return; + } + + flags = dept_enter(); + if (unlikely(!dept_lock())) + goto exit; + + /* + * Check if another already has done it for us with lock held. + */ + if (esd->evt_site) + goto unlock; + + /* + * Can be used as an indicator of whether this + * __dept_recover_event() has been processed or not as well as + * for storing its associated events. + */ + WRITE_ONCE(esd->evt_site, es); + esd->recover_site = rs; + + if (!es->used || !rs->used) { + if (!es->used) + DEPT_INFO("dept_event_site %s has never been used.\n", es->name); + if (!rs->used) + DEPT_INFO("dept_event_site %s has never been used.\n", rs->name); + + DEPT_WARN("Cannot track recover dependency with events that never used.\n"); + goto unlock; + } + + list_add(&esd->dep_node, &es->dep_head); + list_add(&esd->dep_rev_node, &rs->dep_rev_head); + check_recover_dl_bfs(esd); +unlock: + dept_unlock(); +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(__dept_recover_event); + #define B2KB(B) ((B) / 1024) +extern char __dept_event_sites_start[], __dept_event_sites_end[]; + /* * Should be called after setup_per_cpu_areas() and before no non-boot * CPUs have been on. @@ -3185,6 +3367,21 @@ static void migrate_per_cpu_pool(void) void __init dept_init(void) { size_t mem_total = 0; + struct dept_event_site_init **evtinitpp; + + /* + * dept recover dependency tracking works from now on. + */ + for (evtinitpp = (struct dept_event_site_init **)__dept_event_sites_start; + evtinitpp < (struct dept_event_site_init **)__dept_event_sites_end; + evtinitpp++) { + (*evtinitpp)->evt_site->used = true; + (*evtinitpp)->evt_site->func_name = (*evtinitpp)->func_name; + pr_info("dept_event %s@%s is initialized.\n", + (*evtinitpp)->evt_site->name, + (*evtinitpp)->evt_site->func_name); + } + dept_recover_ready = true; local_irq_disable(); dept_per_cpu_ready = 1; From patchwork Tue May 13 10:07:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085041 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3596-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXgk3mcsz1yPv for ; Tue, 13 May 2025 20:26:10 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id EA15016E256 for ; Tue, 13 May 2025 10:24:48 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 14EC328F506; Tue, 13 May 2025 10:08:18 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CA5DC28BA9F; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; cv=none; b=YxIPBulYU6xhK7dSfcwF1Wv+DrGeJHt1FDoDeM0efXWXmXM4mmKkaUbB4lbumiBMCcPvKNf/AW7Xd82t4mqP2zPXtviWai5GlGXlrFyMxFNYMGJQfYch4Dg0y3nEDbDCWwFk6cAdB91iu3USzHIVG8+4mQvIFpNiDx3Wv7kxO9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; c=relaxed/simple; bh=YJ8xcmEE39aGVby1qjH5lpQQhezUmf8G4pJR1ncXYyQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=tUVAwzQKzMyYzg0JCZtU+amFgpYRuqPVOKCkmYY9HenZ/B9W7NLMi8ImApw0gNRxNANOqelG2F/3yuk6VBg6Yzeazm5QjAqPALhPCBzLVbNmwqhTKddtHE+6PDv+mHMu2wpsd4AGpdxtNGlF1vHovcHXXzyJLKyARk0qlPhQal4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-3e-682319f3ed9d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 40/43] dept: add module support for struct dept_event_site and dept_event_site_dep Date: Tue, 13 May 2025 19:07:27 +0900 Message-Id: <20250513100730.12664-41-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTaRSG+b6Z+WZaqU7qbfAS3W6IWY0oBM2JUWNM1MmuJsbVP7qbtVlG WwUkRRDWNbZaTK1AoAYRQS2gpSkVsXWzriuKmFbwgqhQwQAKUSJShKBtRIvYSvxz8uR9T57z 53CU0sPM4rSp+yVdqjpZReS0fDC6fPH7mB81S/X66RD4YKKh7LKTQEtNNQLnVQOGfs8GeBb0 I/j88BEFxUUtCMp7uii46u1GUGc/QuDpq8nQGhgi0FR0gsDRyssEHg+EMHSesmCodm2CF7Y+ Gu4XVGAo7idQWnwUh8cbDKM2Bws2fSz02s+wEOqJh6ZuHwN1zxdByblOAjfqmmjwXuvF8PR6 GYFu5zgD972NNATzZ0NLYR4Dl95VEBgI2iiwBYZYeFJvxeC1zoBaY1h47P0XBu7m1WM4duEK htaO/xHcNL3E4HL6CNwJ+DG4XUUUfKryIOjNH2QhJ3eUhVJDPoITOadoMHYug88fw5fPfogH w/laGi6N+dCaVaLznBOJd/xDlGh0HxA/BdqIWBe00uK9CkH870wXKxpvPmdFqytDdNsXipU3 +rFYPhJgRJfjOBFdIxZWNA+2YvFdczO7ec52+cokKVmbKemWrN4p1+RaXqO0gfVZw9f7sB5d WGFGHCfwicLJkNKMZN/w4oiZRJjwC4T29lEqwtP4+YI7r48xIzlH8b5JwrOzHShSTOV3CaP5 OSjioflY4a49JRIr+OVC43g9mXDOE6pr6795ZOF8rKqZjrCSXyYUWKvpiZ2TMmHIIZvgGOG2 vZ0uQAorinIgpTY1M0WtTU6M02SnarPi/tyX4kLh57IdCu24hkZafm1APIdU0YrG/h80Skad mZ6d0oAEjlJNUxj+DUeKJHX2X5Ju3x+6jGQpvQHN5mjVTEVC8ECSkt+t3i/tlaQ0Sfe9xZxs lh5ZTD0H52ybqa15A3HOtVeKhxcXruqqhCVbFsSZ5m6bEt2gkP9Olfb84/eGosYyb3WfPkxq YrQbkwJvq3xR/KOSwlfjCQkbt/5t/G1P6BdTCT380/q3uPXnnQ+yovnXVUpPbprD7/McPLyj bbCtd53jcceR0nlukkXFxzJordFiUNHpGnX8QkqXrv4KHlBP3FgDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHWc/CT9hOGIyD5n0mTB/GD+P8x/zj275cbeu4k4phk5X y9GplCbiSl2nu5S7RuiSmh48JEqpJRVarYdruWuuB9wx/3z22uu9vd//fBjc+z7py8gjTgvK CKlCQokI0YHg+DWj85fJ1r8wBoHTkUTA7WIzBY0PTAjMpWoM+l/ugtaxQQQTb9/hkJnRiCCn +zMOpTWdCGzGSxQ0fZsJzU47BfUZVyiIv1dMwfuBSQw6bqRhYLLshy+GXgJep+RikNlPwa3M eMx9+jBwGQppMMT5QY8xi4bJ7gCo72whoTq7ngRb+2q4eaeDgnJbPQE1ZT0YND29TUGn+TcJ r2vqCBjTLYDG1GQSioZzKRgYM+BgcNpp+FCpx6BGPxdKNO7WxB+/SKhNrsQgMe8hBs1tzxBU JHVhYDG3UFDtHMTAasnAYbzgJYIe3RANCVddNNxS6xBcSbhBgKYjECZ+upezHQGgvltCQNFU C9q+jTffMSO+etCO8xrrGX7c+ZHibWN6gn+Vy/FPsj7TvKaineb1lijeavTn75X3Y3zOqJPk LYWXKd4ymkbz2qFmjB9uaKAPLjoi2nJMUMijBeW6bSEi2dW07+jkwM6Ykae9WBzK26xFXgzH buTyR7WUhyl2Jffpkwv3sA+7hLMm95JaJGJwtmU615rdhjzBbPY459IluJlhCNaPqzWGe7SY 3cTV/a6k/nUu5kwllX97vNx+qqCB8LA3G8il6E1EChLp0bRC5COPiA6XyhWBa1VhstgIecza 0MhwC3I/kOH8ZGoZcjTtqkIsgyQzxHX9S2XepDRaFRtehTgGl/iI1Y/dSnxMGntWUEYeVUYp BFUVWsAQknniPYeEEG/2hPS0ECYIJwXl/xRjvHzj0JyAnQX7inzLU8MmQjfagzaMH76YrrDO Lf8ic5nK8tl0bdaKqb0dMaeW78uoWOW/ZaGxseFCYXdQ7azi547oN4HD597aIp8cvJ8+gvuc tXU6er7bc/qCraF7Fs6RyKrXN83InpqXkLY96lXX9dYdWfHK3b46+6NSNd2leXNt60r/oq8S QiWTBvjjSpX0DziDbA08AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: struct dept_event_site and struct dept_event_site_dep have been introduced to track dependencies between multi event sites for a single wait, that will be loaded to data segment. Plus, a custom section, '.dept.event_sites', also has been introduced to keep pointers to the objects to make sure all the event sites defined exist in code. dept should work with the section and segment of module. Add the support to handle the section and segment properly whenever modules are loaded and unloaded. Signed-off-by: Byungchul Park --- include/linux/dept.h | 14 +++++++ include/linux/module.h | 5 +++ kernel/dependency/dept.c | 79 +++++++++++++++++++++++++++++++++++----- kernel/module/main.c | 15 ++++++++ 4 files changed, 103 insertions(+), 10 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index b933f9d98c69..a97f34d62468 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -419,6 +419,11 @@ struct dept_event_site { struct dept_event_site *bfs_parent; struct list_head bfs_node; + /* + * for linking all dept_event_site's + */ + struct list_head all_node; + /* * flag indicating the event is not only declared but also * actually used in code @@ -435,6 +440,11 @@ struct dept_event_site_dep { */ struct list_head dep_node; struct list_head dep_rev_node; + + /* + * for linking all dept_event_site_dep's + */ + struct list_head all_node; }; #define DEPT_EVENT_SITE_INITIALIZER(es) \ @@ -446,6 +456,7 @@ struct dept_event_site_dep { .bfs_gen = 0, \ .bfs_parent = NULL, \ .bfs_node = LIST_HEAD_INIT((es).bfs_node), \ + .all_node = LIST_HEAD_INIT((es).all_node), \ .used = false, \ } @@ -455,6 +466,7 @@ struct dept_event_site_dep { .recover_site = NULL, \ .dep_node = LIST_HEAD_INIT((esd).dep_node), \ .dep_rev_node = LIST_HEAD_INIT((esd).dep_rev_node), \ + .all_node = LIST_HEAD_INIT((esd).all_node), \ } struct dept_event_site_init { @@ -478,6 +490,7 @@ extern void dept_init(void); extern void dept_task_init(struct task_struct *t); extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); +extern void dept_mark_event_site_used(void *start, void *end); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); @@ -541,6 +554,7 @@ struct dept_event_site { }; #define dept_task_init(t) do { } while (0) #define dept_task_exit(t) do { } while (0) #define dept_free_range(s, sz) do { } while (0) +#define dept_mark_event_site_used(s, e) do { } while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) diff --git a/include/linux/module.h b/include/linux/module.h index b3329110d668..3da466a2c705 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -28,6 +28,7 @@ #include #include #include +#include #include #include @@ -599,6 +600,10 @@ struct module { #ifdef CONFIG_DYNAMIC_DEBUG_CORE struct _ddebug_info dyndbg_info; #endif +#ifdef CONFIG_DEPT + struct dept_event_site **dept_event_sites; + unsigned int num_dept_event_sites; +#endif } ____cacheline_aligned __randomize_layout; #ifndef MODULE_ARCH_INIT #define MODULE_ARCH_INIT {} diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index b7167ce4f647..bd46c2ce84eb 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -990,6 +990,9 @@ static void bfs(void *root, struct bfs_ops *ops, void *in, void **out) * event sites. */ +static LIST_HEAD(dept_event_sites); +static LIST_HEAD(dept_event_site_deps); + /* * Print all events in the circle. */ @@ -2053,6 +2056,33 @@ static void del_dep_rcu(struct rcu_head *rh) preempt_enable(); } +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_event_site_dep(struct dept_event_site_dep *esd) +{ + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_event_site(struct dept_event_site *es) +{ + struct dept_event_site_dep *esd, *next_esd; + + list_for_each_entry_safe(esd, next_esd, &es->dep_head, dep_node) { + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); + } + + list_for_each_entry_safe(esd, next_esd, &es->dep_rev_head, dep_rev_node) { + list_del_rcu(&esd->dep_node); + list_del_rcu(&esd->dep_rev_node); + } +} + /* * NOTE: Must be called with dept_lock held. */ @@ -2396,6 +2426,8 @@ void dept_free_range(void *start, unsigned int sz) { struct dept_task *dt = dept_task(); struct dept_class *c, *n; + struct dept_event_site_dep *esd, *next_esd; + struct dept_event_site *es, *next_es; unsigned long flags; if (unlikely(!dept_working())) @@ -2417,6 +2449,24 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); + list_for_each_entry_safe(esd, next_esd, &dept_event_site_deps, all_node) { + if (!within((void *)esd, start, sz)) + continue; + + disconnect_event_site_dep(esd); + list_del(&esd->all_node); + } + + list_for_each_entry_safe(es, next_es, &dept_event_sites, all_node) { + if (!within((void *)es, start, sz) && + !within(es->name, start, sz) && + !within(es->func_name, start, sz)) + continue; + + disconnect_event_site(es); + list_del(&es->all_node); + } + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) @@ -3348,6 +3398,7 @@ void __dept_recover_event(struct dept_event_site_dep *esd, list_add(&esd->dep_node, &es->dep_head); list_add(&esd->dep_rev_node, &rs->dep_rev_head); + list_add(&esd->all_node, &dept_event_site_deps); check_recover_dl_bfs(esd); unlock: dept_unlock(); @@ -3358,6 +3409,23 @@ EXPORT_SYMBOL_GPL(__dept_recover_event); #define B2KB(B) ((B) / 1024) +void dept_mark_event_site_used(void *start, void *end) +{ + struct dept_event_site_init **evtinitpp; + + for (evtinitpp = (struct dept_event_site_init **)start; + evtinitpp < (struct dept_event_site_init **)end; + evtinitpp++) { + (*evtinitpp)->evt_site->used = true; + (*evtinitpp)->evt_site->func_name = (*evtinitpp)->func_name; + list_add(&(*evtinitpp)->evt_site->all_node, &dept_event_sites); + + pr_info("dept_event_site %s@%s is initialized.\n", + (*evtinitpp)->evt_site->name, + (*evtinitpp)->evt_site->func_name); + } +} + extern char __dept_event_sites_start[], __dept_event_sites_end[]; /* @@ -3367,20 +3435,11 @@ extern char __dept_event_sites_start[], __dept_event_sites_end[]; void __init dept_init(void) { size_t mem_total = 0; - struct dept_event_site_init **evtinitpp; /* * dept recover dependency tracking works from now on. */ - for (evtinitpp = (struct dept_event_site_init **)__dept_event_sites_start; - evtinitpp < (struct dept_event_site_init **)__dept_event_sites_end; - evtinitpp++) { - (*evtinitpp)->evt_site->used = true; - (*evtinitpp)->evt_site->func_name = (*evtinitpp)->func_name; - pr_info("dept_event %s@%s is initialized.\n", - (*evtinitpp)->evt_site->name, - (*evtinitpp)->evt_site->func_name); - } + dept_mark_event_site_used(__dept_event_sites_start, __dept_event_sites_end); dept_recover_ready = true; local_irq_disable(); diff --git a/kernel/module/main.c b/kernel/module/main.c index 7e569e1b4db5..1c439ed8c9ed 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2611,6 +2611,11 @@ static int find_module_sections(struct module *mod, struct load_info *info) &mod->dyndbg_info.num_classes); #endif +#ifdef CONFIG_DEPT + mod->dept_event_sites = section_objs(info, ".dept.event_sites", + sizeof(*mod->dept_event_sites), + &mod->num_dept_event_sites); +#endif return 0; } @@ -3249,6 +3254,14 @@ static int early_mod_check(struct load_info *info, int flags) return err; } +static void dept_mark_event_site_used_module(struct module *mod) +{ +#ifdef CONFIG_DEPT + dept_mark_event_site_used(mod->dept_event_sites, + mod->dept_event_sites + mod->num_dept_event_sites); +#endif +} + /* * Allocate and load the module: note that size of section 0 is always * zero, and we rely on this for optional sections. @@ -3408,6 +3421,8 @@ static int load_module(struct load_info *info, const char __user *uargs, /* Done! */ trace_module_load(mod); + dept_mark_event_site_used_module(mod); + return do_init_module(mod); sysfs_cleanup: From patchwork Tue May 13 10:07:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085055 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ide+bounces-3597-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXqD21XNz1yXB for ; Tue, 13 May 2025 20:32:40 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 5662F987EC1 for ; Tue, 13 May 2025 10:24:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 21D0C28F936; Tue, 13 May 2025 10:08:18 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C9BE328B7EF; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; cv=none; b=YggN7nFSnxsnZTXtW8G1bGn48mU+sWLjYdSwB+38aMdGdx3X1Qw+kGp3TqrmJ3dWm5YPKlT7yzoqFn0vGS25PBLpUgO2Cccva2McGSP2rKYZPOc3kL8uK1DwdFU+YyuCosuML0Gbj1gRJaU7aU1oJofsT+y13L8dsQDJzd+8tVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; c=relaxed/simple; bh=tQHWVNGGq5DZ2nAmsfnbBPxlhkvZFZVsn98qF7iS0Ug=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Tm14gyLa69YXfiuDsEgSKpEpQJBVIYfsxwSc7+aONkii7pfyiBKRkAGx/87a9Wud8NCPBc1eXl9Qer3kS5CM1fHFd+Hy3QXT4I/UKAiuIoqXoYECm41gvaIcN1sLLgkU2Qj0YlmLsRFerP+23PVVHIo6RpN+t9v5kwk9Bg1sBQg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-4e-682319f31c84 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 41/43] dept: introduce event_site() to disable event tracking if it's recoverable Date: Tue, 13 May 2025 19:07:28 +0900 Message-Id: <20250513100730.12664-42-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTZxjF97739t7bji6XzswXcbh1IUbmBxidz5JtIdkS3xk1S/bPpkZt 7I1thKItIhiNVNCwMhjrVsn8wIK2NlCktGw6ZxFLrDAyhpMVqICCBodSSNA2ovhRuu2fJ7+c k3Oef47AqK7LFgh6Q75kNGhy1JyCVUSSapc9SnlPl1keToHo4zIWTja5Oeg534DA3WLGMH5t LfTFJhA8++NPBqptPQhqR4YYaAkOI/C7DnNw894b0Bud4qDTVs5ByZkmDm48nMUweMyKocG7 AW47x1joqqrDUD3OwYnqEhw//2CYcdbz4CxOh1HXcR5mR7KgczgkA3/4ffipZpCDy/5OFoIX RzHcvHSSg2H3Sxl0BTtYiFWmQs/3FTJonKzj4GHMyYAzOsXDX212DEH7W+ApjRceffRCBtcr 2jAcPduMoXfgNwStZXcweN0hDtqjExh8XhsDT89dQzBaGeHhyLczPJwwVyIoP3KMhdLB1fDs SfzzqcdZYD7tYaHxeQhlf0zdNW5E2yemGFrq20efRv/mqD9mZ+nvdYT+enyIp6WtYZ7avXup z5VBz1wex7R2Oiqj3vpvOOqdtvLUEunFdLK7m/9i4SbFR1opR18gGVd8sl2hC5dcYnffTSuM dJQxxchBLEgQiLiKdDVnW5A8gSNNIWaOOXEx6e+fSfA88R3iqxiTWZBCYMTQ66Tv1ACaM94U JfKi5UGCWTGdlHgciYBS/ICMBQ/z/5YuIg2etoQuj+vPz3Wzc6wSV5MqewM7V0rEH+TklsX6 XyCFXHX1s1VIaUev1SOV3lCQq9HnrFquKzLoC5fvyMv1ovi8nAdnN19E0z1fBpAoIHWSsmP8 XZ1KpikwFeUGEBEY9Tyl+UJcUmo1RfslY942494cyRRAqQKrnq9cGdunVYk7NfnSLknaLRn/ d7EgX1CMFmZq021rDtXs+c7qup+2J3Nn2afrypsl7VL5CnPBorMrqx78yGccWk92eEN24kkO Ce6v/N2+sJWxrU3bNoRNmy98GIokK7NTJx0DlNmSbMG3HZ8bfmntWtKXH/Ql7Wrs2Bro/Xpd 05NhW4Xis/af118ZK3w7sGmj1HzDkZdmPqBmTTpNVgZjNGleAQftGd9aAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUhTcRSH+9/XuZpcltjN6G1hRaFmaJwoovriJejFCgqll0vd2nDOsZVl IWhTMUsxQaUXbVlNczPXppSVZopLs8xyLR1qaWWtNEPdyprWNPpyeHh+nN/5ckS4tJwMEilU xwSNilfKKDEh3rZOFzI6d4l8Vd0oAe6xTAKuVJooaL9tRGCqSsXA1RQFbzyDCH4/f4FDYX47 gmt9PThU2XoR1JadoaDjgz/Y3cMUtOSfo0B3vZKCl1+9GHQX5GFgtGyFt4YBAlpzSzAodFFw uVCH+cZnDMYN5TQYUoKhv+wSDd6+cGjpdZDQWNRCQq1zJVws7qbgYW0LAbZ7/Rh03L9CQa/p DwmttmYCPDnzoP1CNgkV30oo+Oox4GBwD9Pwql6PgU0fCOY0X2vG6CQJT7LrMci4cQcDe9cD BHWZ7zCwmBwUNLoHMbBa8nH4VdqEoD9niIb08+M0XE7NQXAuvYCAtO5I+P3Td7loLBxSr5oJ qJhwoI0bOFOxCXGNg8M4l2Y9wf1yv6a4Wo+e4J6WsFzNpR6aS6tz0pzecpyzlq3grj90Ydy1 ETfJWcrPUpxlJI/msobsGPetrY3eMT9GvP6woFQkCpqwDQfFcqfuPqF+v+DkUHMmnoJuslnI T8QyEWxfpQOfYopZxnZ2jk9zALOItWYPkFlILMIZx0z2TVEXmgpmMwI7WfVlmgkmmNWZb04v SJg17IDtDP2vdCFrNNdPez+fnyhtI6ZYykSyuXojkYvEejSjHAUoVInxvEIZGaqNkyepFCdD DyXEW5DvgwzJ3gv30FhHVANiREg2S9LsWiyXknyiNim+AbEiXBYgSb3rU5LDfNIpQZNwQHNc KWgb0DwRIZsj2bJHOChljvLHhDhBUAua/ykm8gtKQXudF5VNMdGns6010tjlCUtCkPqI8a7p RVdYE7svPekRtqs6omc+b+oMq1mtWZQYFRhitxd4S9XS8Oi813G3tjz++OMZUT/HtRbt/LR/ b8buyJrqQxtj/CNGNvc4k2Wb1GPf82M7mISz3qV89Z/hXUOqtYF9k9Fo+7pPwZZZE+lmGaGV 8+ErcI2W/wtD0xxuPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: With multi event sites for a single wait, dept allows to skip tracking an event that is recoverable by other recover paths. Introduce an API, event_site(), to skip tracking the event in the case. Signed-off-by: Byungchul Park --- include/linux/dept.h | 30 ++++++++++++++++++++++++++++++ include/linux/sched.h | 6 ++++++ kernel/dependency/dept.c | 20 ++++++++++++++++++++ 3 files changed, 56 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index a97f34d62468..b2292587c1cc 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -492,6 +492,31 @@ extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); extern void dept_mark_event_site_used(void *start, void *end); +extern void disable_event_track(void); +extern void enable_event_track(void); + +#define event_site(es, evt_func, ...) \ +do { \ + unsigned long _flags; \ + bool _disable; \ + \ + local_irq_save(_flags); \ + dept_event_site_used(es); \ + /* \ + * If !list_empty(&(es)->dept_head), the event site can be \ + * recovered by others. Do not track event dependency if so. \ + */ \ + _disable = !list_empty(&(es)->dep_head); \ + if (_disable) \ + disable_event_track(); \ + \ + evt_func(__VA_ARGS__); \ + \ + if (_disable) \ + enable_event_track(); \ + local_irq_restore(_flags); \ +} while (0) + extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); @@ -555,6 +580,11 @@ struct dept_event_site { }; #define dept_task_exit(t) do { } while (0) #define dept_free_range(s, sz) do { } while (0) #define dept_mark_event_site_used(s, e) do { } while (0) +#define event_site(es, evt_func, ...) \ +do { \ + (void)(es); \ + evt_func(__VA_ARGS__); \ +} while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) diff --git a/include/linux/sched.h b/include/linux/sched.h index 43927e61921b..44a77b7116b7 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -873,6 +873,11 @@ struct dept_task { */ int missing_ecxt; + /* + * not to track events + */ + int disable_event_track_cnt; + /* * for tracking IRQ-enable state */ @@ -910,6 +915,7 @@ struct dept_task { .stage_wait_stack = NULL, \ .stage_lock = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED,\ .missing_ecxt = 0, \ + .disable_event_track_cnt = 0, \ .hardirqs_enabled = false, \ .softirqs_enabled = false, \ .task_exit = false, \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index bd46c2ce84eb..03d6c057cdc5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2585,6 +2585,23 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, } } +void disable_event_track(void) +{ + dept_task()->disable_event_track_cnt++; +} +EXPORT_SYMBOL_GPL(disable_event_track); + +void enable_event_track(void) +{ + dept_task()->disable_event_track_cnt--; +} +EXPORT_SYMBOL_GPL(enable_event_track); + +static bool event_track_disabled(void) +{ + return !!dept_task()->disable_event_track_cnt; +} + /* * Called between dept_enter() and dept_exit(). */ @@ -2597,6 +2614,9 @@ static void __dept_event(struct dept_map *m, struct dept_map *real_m, struct dept_key *k; int e; + if (event_track_disabled()) + return; + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); if (DEPT_WARN_ON(e >= DEPT_MAX_SUBCLASSES_EVT)) From patchwork Tue May 13 10:07:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085046 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3598-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXhC3vgcz1yPv for ; Tue, 13 May 2025 20:26:35 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id AEF4617A7EE for ; Tue, 13 May 2025 10:25:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B2D9529291C; Tue, 13 May 2025 10:08:18 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C9CC528BA86; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130898; cv=none; b=INAhoK6RXEuh79uGpqtxhCG//4RGc6XkGGLqMIzihosPLjSz03pOyvfdZIW4YrtQO31LdiycpSgLCi2xG+pCD37Nfr7hlqrt54Mx5qrTYPA1GLrpljmyHTDIzWZ4HQAL6gq35DYQEJtG00tdjMpON6Co16SZZNIo0JjjvhgXyeE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130898; c=relaxed/simple; bh=s958f0RG0z2GBnnNwp7CQVbHsfxq66tZewU+8+MIME8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=G3hq6IS3ECkIjabM/XOuf504U2eMYh593okw/7C6ZHJbBRa8u6sMGmJZdq7jHrOe2e8vA+2iZg5+lrS2lOqvX6bE3dDQpsoPGZLKzscNc30mVqUgn4t1qj77RxxKB8XPGCIkjTOZtfpNUXFJvbkdlht+0MbkV+AnWlaftQm02eY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-5e-682319f38f86 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 42/43] dept: implement a basic unit test for dept Date: Tue, 13 May 2025 19:07:29 +0900 Message-Id: <20250513100730.12664-43-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x46zn2P8PMzDmbWFFLHP8jAzD7+ZPIz5gxlHv7mbLrke FLPd6UouJVk1eboru1pd5PIQuiQ60kr0nAqNSKUpd+tUuKv557PX3p/35/3+58MSsgpqFqsK jRA1oYoQOS0hJX2TTEsHZy5U+mXqvMHxK4GEa3csNNTezkdguafD0F2xBZqcvQiGq98QkJFW i8D0qZ2Ae/YOBLbcszTUfZ4M9Y5+GirTEmmIzb5Dw9ueEQxt6akY8q1B8MHcRUJVShaGjG4a rmbEYvf4hsFlzmPArF0EnbmZDIx88ofKjkYKbK2L4cqNNhpKbJUk2Is7MdQ9vkZDh+UvBVX2 VyQ4k2dD7aUkCgp+ZNHQ4zQTYHb0M/CuzIjBbpwOhXp3YPzgHwpeJpVhiL91F0N9yxMEpQkf MVgtjTQ8d/RiKLKmEfA7pwJBZ3IfA3EXXAxc1SUjSIxLJ0HfthKGh9zN13/5g+5mIQkFo41o /VrBcsOChOe9/YSgLzop/HY00ILNaSSF11m88CiznRH0pa2MYLRGCkW5PkJ2STcWTAMOSrDm nacF60AqIxj66rHwo6aG2Tlnn2RNsBiiihI1y9Ydkii/Zptw2M/N0U1fLjBaZFptQCzLcwH8 g/glBuQ1hp32QuRhmvPmm5tdhIencfP5oqQuyoAkLME1TuSbrrcgz+1UbhP/rfi0x0Nyi/hz scOkh6XcKt5lN9DjmfP4/MKysRwvtz6aUzPmkXEr+RRjPjnuuezFn29dM84z+We5zWQKkhrR hDwkU4VGqRWqkABfZUyoKtr3yHG1Fbmfy3xmZH8xGqjdXY44FsknSV91L1DKKEVUeIy6HPEs IZ8m1T10S9JgRcwpUXP8oCYyRAwvR7NZUj5Dutx5MljGHVVEiMdEMUzU/N9i1muWFm03O05X ldweqj5TP0rsG9Srn2ovlnjviva9e27PksOfL7WceEEc7fWPDlwx1FB238d7w/wXXffnlp5I bT84URE/A+ms2+Kyb0llgWqjX8KBFUwkVZDzMKIpKJ3yw9UN3wPFmyZJD6ENWhWQvPFR3t6d g1vfT0mcvtXmetm+I+jQ9vVyMlyp8PchNOGKf/fDXXFYAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yScRSH/b93KOidWb3lutF1tS62rLN1/WK+a2V+616yegumUgM1sdo0 rczSUZvZRQ20kQkWYVtU4pwuylhmoXiZWlJZTNNlQnmhwlpfzp79zrPf+XIYPPQuOYNRqpIE tUqeIKPEhDhmXeay79PnKVb2uqXgG8omoPC+mYLGeyYE5ocZGHifRUOLvw/B6KvXOBTkNyIw dHfi8NDRhcBedoYC10cpNPkGKKjPv0hBZul9Ct70jmHQcfUKBibrdnhn7CHAqSvBoMBLwc2C TCw4vmAwbCynwZi+ADxlN2gY646A+i43CXVF9STY25fC9eIOCqrs9QQ4bB4MXE8KKegy/ybB 6XhBgD8vHBov55JQ0V9CQa/fiIPRN0DD2xo9Bg79VLBkBVvPff9FwvPcGgzO3X6AQVPbUwTV 2e8xsJrdFNT5+jCotObjMHLnGQJP3lcazl4apuFmRh6Ci2evEpDVEQmjP4OXi4YiIOOWhYCK gBtt3sibi82Ir+sbwPmsyhP8iK+Z4u1+PcG/LOH4xzc6aT6rup3m9dZkvrJsCV9a5cV4w6CP 5K3lFyjeOniF5nO+NmF8f0MDHTtzj3j9YSFBmSKoV2yMEys+lxqw49+2pLZ8ukSnI8O6HCRi OHY153FY0DhT7CKutXUYH+cwdg5XmdtD5iAxg7PuCVxLUVtQYpjJbBT3xXZy3CHYBdz5zFFi nCXsGm7YkUP965zNmSw1f3tEwTxwp+GvE8pGcjq9idAhsR6FlKMwpSolUa5MiFyuiVdoVcrU 5YeOJVpR8IGMp8cu29CQK7oWsQySTZS88M5VhJLyFI02sRZxDC4Lk2Q8CkaSw3JtmqA+dlCd nCBoalE4Q8imSbbuFOJC2aPyJCFeEI4L6v9bjBHNSEcWbXVU1KTy2IBo9/7YkFXZuvZZaRXz 1Rqvs+PINVvq3vCR5tbegFR10B+3I37t4p6FgeiQ7jSp1/mzUJe97cPRux7zNOeFna7FkpX4 ppilVZ25tzyfB60bdsU0Ndv2iX5M7G+jkqIOSItN11I6l7UoFmkr8qecsocZ2FUuX3LaLhmh UcgjluBqjfwPYpXPqTwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement CONFIG_DEPT_UNIT_TEST introducing a kernel module that runs basic unit test for dept. Signed-off-by: Byungchul Park --- include/linux/dept_unit_test.h | 67 +++++++++++ kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 12 ++ kernel/dependency/dept_unit_test.c | 173 +++++++++++++++++++++++++++++ lib/Kconfig.debug | 12 ++ 5 files changed, 265 insertions(+) create mode 100644 include/linux/dept_unit_test.h create mode 100644 kernel/dependency/dept_unit_test.c diff --git a/include/linux/dept_unit_test.h b/include/linux/dept_unit_test.h new file mode 100644 index 000000000000..51660f534104 --- /dev/null +++ b/include/linux/dept_unit_test.h @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * DEPT unit test + * + * Copyright (C) SK hynix, 2025 + * + * Authors: Byungchul Park + */ + +#ifndef __LINUX_DEPT_UNIT_TEST_H +#define __LINUX_DEPT_UNIT_TEST_H + +#if defined(CONFIG_DEPT_UNIT_TEST) || defined(CONFIG_DEPT_UNIT_TEST_MODULE) +struct dept_ut { + bool circle_detected; + bool recover_circle_detected; + + int ecxt_stack_total_cnt; + int wait_stack_total_cnt; + int evnt_stack_total_cnt; + int ecxt_stack_valid_cnt; + int wait_stack_valid_cnt; + int evnt_stack_valid_cnt; +}; + +extern struct dept_ut dept_ut_results; + +static inline void dept_ut_circle_detect(void) +{ + dept_ut_results.circle_detected = true; +} +static inline void dept_ut_recover_circle_detect(void) +{ + dept_ut_results.recover_circle_detected = true; +} +static inline void dept_ut_ecxt_stack_account(bool valid) +{ + dept_ut_results.ecxt_stack_total_cnt++; + + if (valid) + dept_ut_results.ecxt_stack_valid_cnt++; +} +static inline void dept_ut_wait_stack_account(bool valid) +{ + dept_ut_results.wait_stack_total_cnt++; + + if (valid) + dept_ut_results.wait_stack_valid_cnt++; +} +static inline void dept_ut_evnt_stack_account(bool valid) +{ + dept_ut_results.evnt_stack_total_cnt++; + + if (valid) + dept_ut_results.evnt_stack_valid_cnt++; +} +#else +struct dept_ut {}; + +#define dept_ut_circle_detect() do { } while (0) +#define dept_ut_recover_circle_detect() do { } while (0) +#define dept_ut_ecxt_stack_account(v) do { } while (0) +#define dept_ut_wait_stack_account(v) do { } while (0) +#define dept_ut_evnt_stack_account(v) do { } while (0) + +#endif +#endif /* __LINUX_DEPT_UNIT_TEST_H */ diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index 92f165400187..fc584ca87124 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -2,3 +2,4 @@ obj-$(CONFIG_DEPT) += dept.o obj-$(CONFIG_DEPT) += dept_proc.o +obj-$(CONFIG_DEPT_UNIT_TEST) += dept_unit_test.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 03d6c057cdc5..50ba3e1c3fd5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -77,8 +77,12 @@ #include #include #include +#include #include "dept_internal.h" +struct dept_ut dept_ut_results; +EXPORT_SYMBOL_GPL(dept_ut_results); + static int dept_stop; static int dept_per_cpu_ready; @@ -826,6 +830,10 @@ static void print_dep(struct dept_dep *d) pr_warn("(wait to wake up)\n"); print_ip_stack(0, e->ewait_stack); } + + dept_ut_ecxt_stack_account(valid_stack(e->ecxt_stack)); + dept_ut_wait_stack_account(valid_stack(w->wait_stack)); + dept_ut_evnt_stack_account(valid_stack(e->event_stack)); } } @@ -926,6 +934,8 @@ static void print_circle(struct dept_class *c) tc = fc; fc = fc->bfs_parent; } while (tc != c); + + dept_ut_circle_detect(); } /* @@ -1027,6 +1037,8 @@ static void print_recover_circle(struct dept_event_site *es) dump_stack(); dept_outworld_exit(); + + dept_ut_recover_circle_detect(); } static void bfs_init_recover(void *node, void *in, void **out) diff --git a/kernel/dependency/dept_unit_test.c b/kernel/dependency/dept_unit_test.c new file mode 100644 index 000000000000..489a7870f2c4 --- /dev/null +++ b/kernel/dependency/dept_unit_test.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * DEPT unit test + * + * Copyright (C) SK hynix, 2025 + * + * Authors: Byungchul Park + */ + +#include +#include +#include +#include +#include + +MODULE_DESCRIPTION("DEPT unit test"); +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Byungchul Park "); + +struct unit { + const char *name; + bool (*func)(void); + bool result; +}; + +static DEFINE_SPINLOCK(s1); +static DEFINE_SPINLOCK(s2); +static bool test_spin_lock_deadlock(void) +{ + dept_ut_results.circle_detected = false; + + spin_lock(&s1); + spin_lock(&s2); + spin_unlock(&s2); + spin_unlock(&s1); + + spin_lock(&s2); + spin_lock(&s1); + spin_unlock(&s1); + spin_unlock(&s2); + + return dept_ut_results.circle_detected; +} + +static DEFINE_MUTEX(m1); +static DEFINE_MUTEX(m2); +static bool test_mutex_lock_deadlock(void) +{ + dept_ut_results.circle_detected = false; + + mutex_lock(&m1); + mutex_lock(&m2); + mutex_unlock(&m2); + mutex_unlock(&m1); + + mutex_lock(&m2); + mutex_lock(&m1); + mutex_unlock(&m1); + mutex_unlock(&m2); + + return dept_ut_results.circle_detected; +} + +static bool test_wait_event_deadlock(void) +{ + struct dept_map dmap1; + struct dept_map dmap2; + + sdt_map_init(&dmap1); + sdt_map_init(&dmap2); + + dept_ut_results.circle_detected = false; + + sdt_request_event(&dmap1); /* [S] */ + sdt_wait(&dmap2); /* [W] */ + sdt_event(&dmap1); /* [E] */ + + sdt_request_event(&dmap2); /* [S] */ + sdt_wait(&dmap1); /* [W] */ + sdt_event(&dmap2); /* [E] */ + + return dept_ut_results.circle_detected; +} + +static void dummy_event(void) +{ + /* Do nothing. */ +} + +static DEFINE_DEPT_EVENT_SITE(es1); +static DEFINE_DEPT_EVENT_SITE(es2); +static bool test_recover_deadlock(void) +{ + dept_ut_results.recover_circle_detected = false; + + dept_recover_event(&es1, &es2); + dept_recover_event(&es2, &es1); + + event_site(&es1, dummy_event); + event_site(&es2, dummy_event); + + return dept_ut_results.recover_circle_detected; +} + +static struct unit units[] = { + { + .name = "spin lock deadlock test", + .func = test_spin_lock_deadlock, + }, + { + .name = "mutex lock deadlock test", + .func = test_mutex_lock_deadlock, + }, + { + .name = "wait event deadlock test", + .func = test_wait_event_deadlock, + }, + { + .name = "event recover deadlock test", + .func = test_recover_deadlock, + }, +}; + +static int __init dept_ut_init(void) +{ + int i; + + lockdep_off(); + + dept_ut_results.ecxt_stack_valid_cnt = 0; + dept_ut_results.ecxt_stack_total_cnt = 0; + dept_ut_results.wait_stack_valid_cnt = 0; + dept_ut_results.wait_stack_total_cnt = 0; + dept_ut_results.evnt_stack_valid_cnt = 0; + dept_ut_results.evnt_stack_total_cnt = 0; + + for (i = 0; i < ARRAY_SIZE(units); i++) + units[i].result = units[i].func(); + + pr_info("\n"); + pr_info("******************************************\n"); + pr_info("DEPT unit test results\n"); + pr_info("******************************************\n"); + for (i = 0; i < ARRAY_SIZE(units); i++) { + pr_info("(%s) %s\n", units[i].result ? "pass" : "fail", + units[i].name); + } + pr_info("ecxt stack valid count = %d/%d\n", + dept_ut_results.ecxt_stack_valid_cnt, + dept_ut_results.ecxt_stack_total_cnt); + pr_info("wait stack valid count = %d/%d\n", + dept_ut_results.wait_stack_valid_cnt, + dept_ut_results.wait_stack_total_cnt); + pr_info("event stack valid count = %d/%d\n", + dept_ut_results.evnt_stack_valid_cnt, + dept_ut_results.evnt_stack_total_cnt); + pr_info("******************************************\n"); + pr_info("\n"); + + lockdep_on(); + + return 0; +} + +static void dept_ut_cleanup(void) +{ + /* + * Do nothing for now. + */ +} + +module_init(dept_ut_init); +module_exit(dept_ut_cleanup); diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 7c74f92e4cc2..65f867e35be8 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1404,6 +1404,18 @@ config DEPT_AGGRESSIVE_TIMEOUT_WAIT that timeout is used to avoid a deadlock. Say N if you'd like to avoid verbose reports. +config DEPT_UNIT_TEST + tristate "unit test for DEPT" + depends on DEBUG_KERNEL && DEPT + default n + help + This option provides a kernel module that runs unit test for + DEPT. + + Say Y if you want DEPT unit test to be built into the kernel. + Say M if you want DEPT unit test to build as a module. + Say N if you are unsure. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT From patchwork Tue May 13 10:07:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 2085040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.199.223; helo=ny.mirrors.kernel.org; envelope-from=linux-ide+bounces-3595-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [147.75.199.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4ZxXgJ3drvz1yPv for ; Tue, 13 May 2025 20:25:48 +1000 (AEST) Received: from smtp.subspace.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 9EE61169718 for ; Tue, 13 May 2025 10:24:30 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AEF8C28E5F1; Tue, 13 May 2025 10:08:17 +0000 (UTC) X-Original-To: linux-ide@vger.kernel.org Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9512A28B4E2; Tue, 13 May 2025 10:08:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; cv=none; b=EoHXwnDi80QGHF9ensY24P6bbNFNCx0twTB0Cd/0mFhcEi1Ch3DKXECrQPKm6BSVErljQr9h0EnXhbdB/dPebGyP21X9Pm59Erb2c+CNnP2StQC6eaReXaPMOBx9C7OTrnhTQYrcM3W2+65OCDPmGbgSP6VT1Km0eW795NO4/rY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747130897; c=relaxed/simple; bh=QsfbRnxCWDXHvX6Jz9hxwcCFfOh5Fy5Hk/37cuZHV9I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=oTXfSwTGg4GfCv2KbwcxdYhSVKdLzJ472Sj7mmsYug1FSBVd9ragx5QLwDFn5dUBp35SAgtnKyDkDJez7XEHDmKLwBxDzfu/WORJMw9kx86yVM0/n4Zp/dq8+GzQfCcixXszNu1dFPFElXWC0uO5eEetW4B9vilbOSmoAt4oDGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-681ff7000002311f-6e-682319f3d65e From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, harry.yoo@oracle.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, yskelg@gmail.com, yunseong.kim@ericsson.com, yeoreum.yun@arm.com, netdev@vger.kernel.org, matthew.brost@intel.com, her0gyugyu@gmail.com Subject: [PATCH v15 43/43] dept: call dept_hardirqs_off() in local_irq_*() regardless of irq state Date: Tue, 13 May 2025 19:07:30 +0900 Message-Id: <20250513100730.12664-44-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250513100730.12664-1-byungchul@sk.com> References: <20250513100730.12664-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0xTaRSF+f9zK9XKSb0d1HjpSDR4xajZkwxqMonz+2AyUTMPo4ke5Wgr V4simEjogAZBCDgDjVycUpjSQLlYTASlihAQRAG1IhJAQTFUihiwpQjoFIwvOyvf2mvtly2j lM3MMpkm4qykjRDDVKyclo/ML9g07v+Teuvjxg3g+pxMQ16FhYWO8lIElls6DI7G3+Cl24lg 6kk7BfqsDgQF/b0U3GrqQ2Az/8XC83cLwO4aZaElK5WFxMIKFp4OT2Poyb6GodS6H16b3tPQ mmHEoHewkKtPxN4xhGHSVMKBKSEABsw5HEz3B0FLXycDtu4NcP1GDwu1thYamqoHMDy/k8dC n+UbA61NzTS405dDR2YaA2UfjSwMu00UmFyjHDyrM2BoMiyByiRv4eXxrww8TKvDcLnoJgb7 q7sI7iW/wWC1dLLQ4HJiqLJmUfCluBHBQPoIB5euTnKQq0tHkHopm4aknh0w5fFezv8cBLp/ K2kom+lEe4KJ5YYFkQbnKEWSqs6TL64XLLG5DTR5ZBRITU4vR5LudXPEYD1HqsyBpLDWgUnB mIsh1pIrLLGOXeNIyogdk49tbdzvK/6U/xIihWliJO2WXcfk6v72QRyl84t9O+niEtDIvBTk KxP47YIz8yvzQw8WDONZzfLrhK6uSWpWL+JXC1Vp7707chnFd84TXua/QrPGQv64UNQ3NRem +QDB6LHOBRT8TkFvv09/L10llFbWzXFfL58pbpvjSn6HkGEopWdLBf5vX+G/ijfU94C/8MDc RWcghQH5lCClJiImXNSEbd+sjovQxG4+ERluRd7/Ml2cPlyNxjoO1iNehlTzFc2ONWolI8ZE x4XXI0FGqRYpdLe9SBEixl2QtJFHtefCpOh6tFxGq5YqtrnPhyj5U+JZKVSSoiTtDxfLfJcl oCKz32Cau2Fo64d/YvSP0RFnfPvRjNvm0J89r9NX6JaqHMHB6xdPlLean6SKsa1lgZHjVyZ8 svwPGgNWRi3+lHgg9NDeBYE5Z6ZnBu9PvLUcymcy/wj/JsYPVfvYi9UnN/oVxqXk1cR7ds+U rfUk55avFk9n13ssv9r2kcOgYYaiVHS0WgwKpLTR4v9PDQHqWwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHe5736mz1tsTebpgLCZQuQsbBrkTSS1D4qSiKXPXSVrpiM80i cGlhmrICtTJrmkxxq9kmZRdNXG0zy1mulWGmVtLw1sVJ81JNoy+HH78/53++HJaQVVMLWJU6 VdSoFclyWkJKdqzNWv5z/lLlql53OPhHcki4bjHT0HbHhMBcq8Pge7YV3o4OIBh/6SaguLAN QVnPBwJqHV0I6qvO0tD+eRZ4/MM0NBfm0ZB1y0LDq/4JDJ1FlzGYrNvho7GPhBZ9OYZiHw0l xVk4OL5iCBirGTBmRkFv1TUGJnpiobnLS4G9tJmC+vcxcPVGJw2P65tJcNT1Ymh/eJ2GLvMf ClocLhJGCxZC26V8Cm4PldPQP2okwOgfZuB1owGDwxAONdnB1vM/f1PgzG/EcL7iLgZPxyME DTndGKxmLw12/wAGm7WQgLHKZwh6CwYZOHcxwECJrgBB3rkiErI742D8V/By6Ugs6G7WkHB7 0os2bRDMN8xIsA8ME0K2LV0Y87+hhfpRAyk8L+eFB9c+MEJ2w3tGMFhPCLaqaOHWYx8Wyn74 KcFafYEWrD8uM0LuoAcLQ62tTOLiPZJ1h8RkVZqoWbkhSaLscX/Bx3WzT34K+JlMNBiai0JY nlvNfynrx1NMc8v4d+8CxBSHcUt4W34flYskLMF5Q/m3pR1oKpjLHeArusapKSa5KL78l3V6 Qcqt4Ys9T8h/pRG8qaZx2ocE/WRl67SXcXG83mAi9UhiQDOqUZhKnZaiUCXHrdAeVWaoVSdX HDyWYkXBDzKembhUh0batzYhjkXymVKXL1IpoxRp2oyUJsSzhDxMqrsfVNJDioxToubYfs2J ZFHbhBaypHyedNsuMUnGHVakikdF8bio+Z9iNmRBJqpcNvFSt0a3c16dy9GoutfntK1vSYiP UEckvAqtTX+60Xl6bmfDi83606v5xMNG92tf3aRsUUJ3zEf07VRhIPJM+N5V+/6MHYzscMXE q2u2zDENuY+4LNvWl0WzOZbdbZaO31j3Iu/h0gsl8fYrTkV34rhnssLt7Cmyf5+dqk/3ykmt UhEbTWi0ir8OZ96uPQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-ide@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: For dept to function properly, dept_task()->hardirqs_enabled must be set correctly. If it fails to set this value to false, for example, dept may mistakenly think irq is still enabled even when it's not. Do dept_hardirqs_off() regardless of irq state not to miss any unexpected cases by any chance e.g. changes of the state by asm code. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 14 ++++++++++++++ kernel/dependency/dept.c | 1 + 2 files changed, 15 insertions(+) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index d8b9cf093f83..586f5bad4da7 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -214,6 +214,13 @@ extern void warn_bogus_irq_restore(void); raw_local_irq_disable(); \ if (!was_disabled) \ trace_hardirqs_off(); \ + /* \ + * Just in case that C code has missed \ + * trace_hardirqs_off() at the first \ + * place e.g. disabling irq at asm code.\ + */ \ + else \ + dept_hardirqs_off(); \ } while (0) #define local_irq_save(flags) \ @@ -221,6 +228,13 @@ extern void warn_bogus_irq_restore(void); raw_local_irq_save(flags); \ if (!raw_irqs_disabled_flags(flags)) \ trace_hardirqs_off(); \ + /* \ + * Just in case that C code has missed \ + * trace_hardirqs_off() at the first \ + * place e.g. disabling irq at asm code.\ + */ \ + else \ + dept_hardirqs_off(); \ } while (0) #define local_irq_restore(flags) \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 50ba3e1c3fd5..9993222af73c 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2258,6 +2258,7 @@ void noinstr dept_hardirqs_off(void) */ dept_task()->hardirqs_enabled = false; } +EXPORT_SYMBOL_GPL(dept_hardirqs_off); void noinstr dept_update_cxt(void) {