From patchwork Mon Nov 7 12:20:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700548 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=f1kbEnXF; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vjg5b4Zz23lT for ; Mon, 7 Nov 2022 23:22:31 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjg5Mmgz4xG8 for ; Mon, 7 Nov 2022 23:22:31 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vjg5KWBz4xGT; Mon, 7 Nov 2022 23:22:31 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=f1kbEnXF; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjg5FFRz4xG8 for ; Mon, 7 Nov 2022 23:22:31 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232125AbiKGMWb (ORCPT ); Mon, 7 Nov 2022 07:22:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMW3 (ORCPT ); Mon, 7 Nov 2022 07:22:29 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E16A9D62 for ; Mon, 7 Nov 2022 04:22:28 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id p21so10895215plr.7 for ; Mon, 07 Nov 2022 04:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zhu5N1/WsdaDuDfEuZec2Dmbs5lsbYY8/HuylV78JBE=; b=f1kbEnXFkza1ZfjT//Sm6JsQuhulqyDqiViAeQiaP5NpQlIQwBF+f6/WeUMSziWs0T FozY01LRiRpUOGsoDoKZTADs2ijFoEX+6jyc/N+P+u5ZcrbJKBFHoNnVtYmf9uxjrRL0 t6b5Dv1Td78V9k8dVgBn06xQGcE8d/IMOrZQn9kTf+ZZBzUsSkXb+R/lJcCTGCkFPeA2 BJy2YFJONQAqovoXMhkMANi6bDOWhb26Juzpz3ijrHzIlZr95LqEPwmuo7P0agOe3fYk upcG+v0adxeX3ki7clYaCG1GkxkQFW+9clMpDIjFg8R+GwoRcZnVbKG2wT4ydqPecLge wQQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zhu5N1/WsdaDuDfEuZec2Dmbs5lsbYY8/HuylV78JBE=; b=vnaNy+TgVwUS6LCCLtaZ62OWBboK1obbeijRsh44ykYWFwOGyVyo/q3iEclOyDH3rx mvFRJoAmwpg9j2qmYu1pmAZzqPk5vYjTGXXblit88bwhd/oZdo4V6mnAj2V4k2g0xXtn pVJatgg0en8sgxP8yUK2mOoqUmOCpfqxD/6ccnifiUSjKRQQqpdsakq7zpD2Iin3jsek tv7taabB3FzPvUrtEVQyGZdLTnOb84gTN3ptIGEOQpsVT53cJ3PX3R8d8oZj4C7iRViY 39x1OxsYU6QaCryAFqo7tTK30LBlnMcKoalO12epQyVTKu/GUa+pJ9CexCOoZrgcSIg0 SXjw== X-Gm-Message-State: ANoB5plieEiAIxvVVxGiXYefZr6muakzrcvX8d5/4Kffx2EP7Da4+NG1 5gRnw1wRkCfI0wXPXj1MTMs= X-Google-Smtp-Source: AA0mqf7reBytkkV3J7RmlEHMt7C4DFq90e8EXXpjmhQf1+xRHIF6PqGbBd7FYxREQWLIQnDXFi0dsA== X-Received: by 2002:a17:902:9042:b0:188:6fc9:1da with SMTP id w2-20020a170902904200b001886fc901damr13818756plz.162.1667823748458; Mon, 07 Nov 2022 04:22:28 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id a10-20020a63cd4a000000b0043941566481sm4128203pgj.39.2022.11.07.04.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:27 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 01/72] e2fsck: Fix unbalanced mutex unlock for BOUNCE_MTX Date: Mon, 7 Nov 2022 17:50:49 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org f_crashdisk test failed with UNIX_IO_FORCE_BOUNCE=yes due to unbalanced mutex unlock in below path. This patch fixes it. Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/unix_io.c | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c index e53db333..5b894826 100644 --- a/lib/ext2fs/unix_io.c +++ b/lib/ext2fs/unix_io.c @@ -305,7 +305,6 @@ bounce_read: while (size > 0) { actual = read(data->dev, data->bounce, align_size); if (actual != align_size) { - mutex_unlock(data, BOUNCE_MTX); actual = really_read; buf -= really_read; size += really_read; From patchwork Mon Nov 7 12:20:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700549 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=AIi2DCcH; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vjq1f9Qz23lT for ; Mon, 7 Nov 2022 23:22:39 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjn320qz4xG8 for ; Mon, 7 Nov 2022 23:22:37 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vjn2zGSz4xGT; Mon, 7 Nov 2022 23:22:37 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=AIi2DCcH; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjn2tXLz4xG8 for ; Mon, 7 Nov 2022 23:22:37 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232130AbiKGMWg (ORCPT ); Mon, 7 Nov 2022 07:22:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMWf (ORCPT ); Mon, 7 Nov 2022 07:22:35 -0500 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF729D62 for ; Mon, 7 Nov 2022 04:22:34 -0800 (PST) Received: by mail-pl1-x62a.google.com with SMTP id c2so10879430plz.11 for ; Mon, 07 Nov 2022 04:22:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PeguO5OJQkOMV7A3vjAm+yi2UDth+yAgHrBJt9W7x20=; b=AIi2DCcHrqUqsdTtRq6K37rKo+F7IPFgqVwj1RGbeBIS2Ljqc4aKaNBCOwY7cLB4kY Sv0rdX1ZROE6jVBY9iS7+a626GaljeZFvJEvsKUrZvS8J+rF+/zTHogCLQaESGDOCYNG vbkls6mAObm3pTFT8nonK2/p02aygel4xgWVwuHNTpFFJc+XNZyi501fmmSsOtXMqHDI Ha6lmR1zVBknimeCAdOzZsKiJ77y/Etz2R+VR07t46/OGwvRb5racnqXV88Z7TgrKtpV EkGQWiANHuMK5HjNOrz16lMz5Mhv1zJzoVdXhDQuw240nWG00GKqpTxwDaktXMOCGD2x 6BYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PeguO5OJQkOMV7A3vjAm+yi2UDth+yAgHrBJt9W7x20=; b=jhPRCdxyPp08YX5uf36UirpPckR4j0i6cGPq+VSm2goRA7E2MB4PweGeK5uzTxAutA P7WDc+ztG39ZFBqx5EPUANPBVXSjxTrq8mxXZlnEVZWZ/p3zrwnQNu7dh37nk/FN76JY 6yXNNOmJs0ZD64GxSnIovz9UNrCmlCszgI315HooKNVgY3y6UxBn/WUKyPTYJGsmHQTn J5URdk8P82hsctb8bABwD1kVjkJcGKsIki6D69Ew48xcQNjCXsISn0ka8ypcNqIPvWSB a2U7KFOr7irn11V0ndYxwlYQfKb/ivrd+tp6eUyTCMoYKsVj+yZgr5Zj9aWFaHI/ZqUF BaZA== X-Gm-Message-State: ACrzQf3AWG0KEJae7DdUnadyzicxnyH7NwKI3UqAgPRXQCwGAg8jai6r UJ0ZyccBOJ6oaLfTWJs00Kg= X-Google-Smtp-Source: AMsMyM5cbMZ/iNcTx7Axe+/J1M2V3Jx2DcE5804iD3BK7PHF5DSC8FwAWkqJGp6nYyq9Vd2fcmAi9g== X-Received: by 2002:a17:902:e74a:b0:186:a094:1d3 with SMTP id p10-20020a170902e74a00b00186a09401d3mr48851963plf.153.1667823754480; Mon, 07 Nov 2022 04:22:34 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y23-20020a17090264d700b0017f92246e4dsm4824708pli.181.2022.11.07.04.22.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:33 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 02/72] gen_bitmaps: Fix ext2fs_compare_generic_bmap/bitmap logic Date: Mon, 7 Nov 2022 17:50:50 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Currently this function was not correctly comparing against the right length of the bitmap. Also when we compare bitarray v/s rbtree bitmap the value returned by ext2fs_test_generic_bmap() could be different in these two implementations. Hence only check against boolean value. Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/gen_bitmap.c | 9 ++++++--- lib/ext2fs/gen_bitmap64.c | 10 +++++++--- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/lib/ext2fs/gen_bitmap.c b/lib/ext2fs/gen_bitmap.c index 1536d4b3..f7764fca 100644 --- a/lib/ext2fs/gen_bitmap.c +++ b/lib/ext2fs/gen_bitmap.c @@ -385,10 +385,13 @@ errcode_t ext2fs_compare_generic_bitmap(errcode_t magic, errcode_t neq, (size_t) (bm1->end - bm1->start)/8))) return neq; - for (i = bm1->end - ((bm1->end - bm1->start) % 8); i <= bm1->end; i++) - if (ext2fs_fast_test_block_bitmap(gen_bm1, i) != - ext2fs_fast_test_block_bitmap(gen_bm2, i)) + for (i = bm1->start; i <= bm1->end; i++) { + int ret1, ret2; + ret1 = !!ext2fs_fast_test_block_bitmap(gen_bm1, i); + ret2 = !!ext2fs_fast_test_block_bitmap(gen_bm2, i); + if (ret1 != ret2) return neq; + } return 0; } diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c index c860c10e..f7710afd 100644 --- a/lib/ext2fs/gen_bitmap64.c +++ b/lib/ext2fs/gen_bitmap64.c @@ -629,10 +629,14 @@ errcode_t ext2fs_compare_generic_bmap(errcode_t neq, (bm1->end != bm2->end)) return neq; - for (i = bm1->end - ((bm1->end - bm1->start) % 8); i <= bm1->end; i++) - if (ext2fs_test_generic_bmap(gen_bm1, i) != - ext2fs_test_generic_bmap(gen_bm2, i)) + for (i = bm1->start; i < bm1->end; i++) { + int ret1, ret2; + ret1 = !!ext2fs_test_generic_bmap(gen_bm1, i); + ret2 = !!ext2fs_test_generic_bmap(gen_bm2, i); + if (ret1 != ret2) { return neq; + } + } return 0; } From patchwork Mon Nov 7 12:20:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700550 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=fgD/iZ6l; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vjv2xVnz23lT for ; Mon, 7 Nov 2022 23:22:43 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjv2VyHz4xG8 for ; Mon, 7 Nov 2022 23:22:43 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vjv2SB7z4xGT; Mon, 7 Nov 2022 23:22:43 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=fgD/iZ6l; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vjv2LvFz4xG8 for ; Mon, 7 Nov 2022 23:22:43 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232132AbiKGMWm (ORCPT ); Mon, 7 Nov 2022 07:22:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMWm (ORCPT ); Mon, 7 Nov 2022 07:22:42 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4740B6365 for ; Mon, 7 Nov 2022 04:22:41 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id y203so10448762pfb.4 for ; Mon, 07 Nov 2022 04:22:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jXoIoNClKEmqFxAE6Ld3vjNDtgFGvZES84QuwnIxgtE=; b=fgD/iZ6lwmpoNyZrGPWMFTQ99OfZAwkMQbg8bIoVreTE/nu/4FcS+qGGUF1FXxoOuW Xm+LVfpShldCa3ifyQxrDibMCAoZxUcVFy/VDBEpSkTaPBwJ7UQGDRBx1ICdCGFGZ9qd Xb9zmxl9Zpv4LinHGZM4+MBFgzQ2W8K2/Bt6FwTNAtDnl3n4bQ3CUSLIJBnR5kWdomeE C3Mb0DhecA2wF2lAxLIiAlGyivIbZHQgbgigoxQxqewDwRpy+l2vXqMMTUwe9vzoBuXh 7YZLb7u93HxpEalsYHH9Av/BQPzjtSJG0pw++qBRQSaOCuBvZ9cZ8zKHGxq+kje0kN3Q F7Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jXoIoNClKEmqFxAE6Ld3vjNDtgFGvZES84QuwnIxgtE=; b=ENpIgxD2H4Qg6K/8AO8sQCX70DpSDpgGDy+L5i+6NgdQe6HwRSIHYNvzEByCZGTbBT EMQlVUjWt/rfio/5ISI1QdC+wEFea+Opu+UJC4jQBDQ+sk9OWk6Bl+gnOvfyjD7XK9NG AuUBVqJqzqoeCMBhOTuKGyTWtCqtAkh/dLWWmkgACidUgNm4LFQ8bya+3Z5urOqAMwU7 H99XlKRBsPAbWCCiks7s5dDr3lar2XI6ZY3+K/UiWFIrPYEbX3T5YSeqXayDxjYPrib5 FJO9dWmMnHkQMigbE+484pLcQGZCoYG1xpFgeus+4ybjefyvZftuzOPMZtErys5VchiU 3CQw== X-Gm-Message-State: ACrzQf0bROZQ1AFLJ5DGToVq7btFJYvw8uJhJEK7bxvaByLi9YZZy5Jf EKwM93TTi398FLV4SjmYGGuyVn4w8Po= X-Google-Smtp-Source: AMsMyM54tzpbro4An/Ujti04W2iNDtPqm/1f3EYwYtrcSEPzjXlaTRmzIAf0D1J5rCjzQxytMBfBFQ== X-Received: by 2002:a62:1d52:0:b0:56b:f472:55e7 with SMTP id d79-20020a621d52000000b0056bf47255e7mr50552192pfd.63.1667823760806; Mon, 07 Nov 2022 04:22:40 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id e18-20020a63e012000000b00470275c8d6dsm4034156pgh.10.2022.11.07.04.22.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:40 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 03/72] blkmap64_ba: Add common helper for bits size calculation Date: Mon, 7 Nov 2022 17:50:51 +0530 Message-Id: <728edcaf0eb7fca7e347183799b5dca743236db0.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Just a quick common helper for bits size calculation. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/blkmap64_ba.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/lib/ext2fs/blkmap64_ba.c b/lib/ext2fs/blkmap64_ba.c index 5d8f1548..4e7007f0 100644 --- a/lib/ext2fs/blkmap64_ba.c +++ b/lib/ext2fs/blkmap64_ba.c @@ -40,6 +40,13 @@ struct ext2fs_ba_private_struct { typedef struct ext2fs_ba_private_struct *ext2fs_ba_private; +#define ba_bits_size(start, end) ((((end) - (start)) / 8 + 1)) + +static size_t ba_bitmap_size(ext2fs_generic_bitmap_64 bitmap) +{ + return (size_t) ba_bits_size(bitmap->start, bitmap->real_end); +} + static errcode_t ba_alloc_private_data (ext2fs_generic_bitmap_64 bitmap) { ext2fs_ba_private bp; @@ -56,7 +63,7 @@ static errcode_t ba_alloc_private_data (ext2fs_generic_bitmap_64 bitmap) if (retval) return retval; - size = (size_t) (((bitmap->real_end - bitmap->start) / 8) + 1); + size = ba_bitmap_size(bitmap); retval = ext2fs_get_mem(size, &bp->bitarray); if (retval) { @@ -80,7 +87,7 @@ static errcode_t ba_new_bmap(ext2_filsys fs EXT2FS_ATTR((unused)), return retval; bp = (ext2fs_ba_private) bitmap->private; - size = (size_t) (((bitmap->real_end - bitmap->start) / 8) + 1); + size = ba_bitmap_size(bitmap); memset(bp->bitarray, 0, size); return 0; @@ -115,7 +122,7 @@ static errcode_t ba_copy_bmap(ext2fs_generic_bitmap_64 src, dest_bp = (ext2fs_ba_private) dest->private; - size = (size_t) (((src->real_end - src->start) / 8) + 1); + size = ba_bitmap_size(src); memcpy (dest_bp->bitarray, src_bp->bitarray, size); return 0; @@ -145,8 +152,8 @@ static errcode_t ba_resize_bmap(ext2fs_generic_bitmap_64 bmap, return 0; } - size = ((bmap->real_end - bmap->start) / 8) + 1; - new_size = ((new_real_end - bmap->start) / 8) + 1; + size = ba_bitmap_size(bmap); + new_size = ba_bits_size(new_real_end, bmap->start); if (size != new_size) { retval = ext2fs_resize_mem(size, new_size, &bp->bitarray); @@ -306,8 +313,7 @@ static void ba_clear_bmap(ext2fs_generic_bitmap_64 bitmap) { ext2fs_ba_private bp = (ext2fs_ba_private) bitmap->private; - memset(bp->bitarray, 0, - (size_t) (((bitmap->real_end - bitmap->start) / 8) + 1)); + memset(bp->bitarray, 0, ba_bitmap_size(bitmap)); } #ifdef ENABLE_BMAP_STATS From patchwork Mon Nov 7 12:20:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700551 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=S/Z5s8P8; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vk13vZyz23lT for ; Mon, 7 Nov 2022 23:22:49 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vk13SZpz4xGT for ; Mon, 7 Nov 2022 23:22:49 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vk13Pvhz4xZ3; Mon, 7 Nov 2022 23:22:49 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=S/Z5s8P8; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vk13GdBz4xGT for ; Mon, 7 Nov 2022 23:22:49 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232136AbiKGMWs (ORCPT ); Mon, 7 Nov 2022 07:22:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMWs (ORCPT ); Mon, 7 Nov 2022 07:22:48 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95076633D for ; Mon, 7 Nov 2022 04:22:47 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id p12so5404068plq.4 for ; Mon, 07 Nov 2022 04:22:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ymBX60LUOuBsP+vRhvxq6pMPtRa4aOay12HNgYBIPk8=; b=S/Z5s8P8T5uiK4XbX4rCHctiscfLR0VRkfVa8d/UjKp4EGl/YLo7qlEmhcS0/sed7c TDZaTDoUC0Yk0MEyLxMv9cwVzIszQAZ9xvMqtpb1Vd2BC6vSp10KqA/E3jj9OBZ8V0rv qD/RXLMeRr15dwdiXgCJi+UgutxS5NuOXI5+F59/CAofV4Pd6AorhFx3Zm8aBlPSVnFi 4ggcjjtnHbIItdYxv5yy8Pp8jp/4/c4RF31WSlW2E7kdTCp5peYN6/Cg8d6YbokZRlWI gmZo0uNHxCnsD71h1eZ0BV70T2hoo0wDxXCvNCQj7xAWEOEq49wrLfquuC4yrnCgynvj LdsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ymBX60LUOuBsP+vRhvxq6pMPtRa4aOay12HNgYBIPk8=; b=IfmQc/N7p2W+44/A+3IKXfPmENQLE5wbmJnsL4aop3AVkyqDpeZW794jVN0Tf5uEKG +TgTjcq7zr574/AJXv84wjAO0HfOfsmB6RF/QVOBO7OtvCi/+38q3R1i6nsrqBpaEtKj DJfSxj8uH4NWoM5UzhNtpekkI3rOlzub+NmSwkGAfP3YRTUq3naYsVwThQ+i3zz2rf+q v8DA0aCIb66LfGINJf5ReCwYtbXy3HDEjtZKnRtce2BLJOjIKeineu/TtDr6boPEshFa 3h7X30pwaCVCkzuDeV7y2w8X7PjqI1ez1mJqv8Rz35+EhqiFHxw6UDsqIzMHvbbpf4hV xCng== X-Gm-Message-State: ACrzQf2/ig9PoVr83/CBKo+C7FE7SDh7GfUGN3HyTOU9xtqU6PcG2d4t Nh512YxxhntPcTOwegVfXyk= X-Google-Smtp-Source: AMsMyM5EZL5GsQknopktkzGmJCun0mSTKCL8lq2xeFwr0jRo84n1Q1L8/dn4GReV6pnfYEjCyxU08w== X-Received: by 2002:a17:90b:30d8:b0:213:fd7d:6a7b with SMTP id hi24-20020a17090b30d800b00213fd7d6a7bmr37298919pjb.29.1667823767066; Mon, 07 Nov 2022 04:22:47 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id s23-20020a632157000000b0046ec0ef4a7esm4104967pgm.78.2022.11.07.04.22.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:46 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 04/72] badblocks: Remove unused badblocks_flags Date: Mon, 7 Nov 2022 17:50:52 +0530 Message-Id: <3b00feafe3b640cc230582eeff4a194a5ba5838b.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org badblocks_flags is not used anywhere. So just remove it. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/badblocks.c | 6 +----- lib/ext2fs/ext2fsP.h | 1 - 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/lib/ext2fs/badblocks.c b/lib/ext2fs/badblocks.c index a306bc06..345168e0 100644 --- a/lib/ext2fs/badblocks.c +++ b/lib/ext2fs/badblocks.c @@ -81,11 +81,7 @@ errcode_t ext2fs_u32_copy(ext2_u32_list src, ext2_u32_list *dest) { errcode_t retval; - retval = make_u32_list(src->size, src->num, src->list, dest); - if (retval) - return retval; - (*dest)->badblocks_flags = src->badblocks_flags; - return 0; + return make_u32_list(src->size, src->num, src->list, dest); } errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src, diff --git a/lib/ext2fs/ext2fsP.h b/lib/ext2fs/ext2fsP.h index a20a0502..d2045af8 100644 --- a/lib/ext2fs/ext2fsP.h +++ b/lib/ext2fs/ext2fsP.h @@ -34,7 +34,6 @@ struct ext2_struct_u32_list { int num; int size; __u32 *list; - int badblocks_flags; }; struct ext2_struct_u32_iterate { From patchwork Mon Nov 7 12:20:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700552 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=GFaxAb9X; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vk82KMhz23lT for ; Mon, 7 Nov 2022 23:22:56 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vk81t4cz4xG8 for ; Mon, 7 Nov 2022 23:22:56 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vk81qNRz4xGT; Mon, 7 Nov 2022 23:22:56 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=GFaxAb9X; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vk81T4Tz4xG8 for ; Mon, 7 Nov 2022 23:22:56 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232134AbiKGMWz (ORCPT ); Mon, 7 Nov 2022 07:22:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMWy (ORCPT ); Mon, 7 Nov 2022 07:22:54 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A42062DF for ; Mon, 7 Nov 2022 04:22:53 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id h14so10398897pjv.4 for ; Mon, 07 Nov 2022 04:22:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c76ez1bFQDKYbcoz3xWef+fbR+qhw1r0UYUFuSqVoPo=; b=GFaxAb9XkY1c+KBmUpJ+pPUmmsouAFx+McR2BvbSGEDyUilRvu4yfc07Wzs5af0i8+ 7f5ueLIV5y/PKVH+nIRk65loaNR7KzZhmf6QJfcsox/5EbfGCshT8ugLC1uVY2xB7m/J f8H+Tx4Z2ImM6xARbPtpwP9k7HomkcpoWDceolajkZOqGk/ByBtLafVGWQJdjb3hxQt0 e0JkPKLcCfYDzI4KSuR9JmaKci0Op24TIq1+/KwEyEf0WVTjxAQ3C6+9WGcFHJ+L9rNK jLFwvnbJ5/JmemyLYu3QrQrdHTM1q19pGoKZcMQ+t898DhP9EGKaCD/nNljuLrMQKS9C j7jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c76ez1bFQDKYbcoz3xWef+fbR+qhw1r0UYUFuSqVoPo=; b=lWTcvUoDfD9kYw2IcqPmP2s/66W9QszHCGOnmiWFB6ktgNBBvyfzSeIftbEMxb+0lC 36eZkGuKkWyY8XDj1r2DpU+DqxJMv++4qhnUo22Yyfx7f+69Beij0xgI4EaG04TM2Cl1 aQAVDen5QJki825Xki+/BeHuDWh2dJxILG3as1/JCyZNiubxj3jn92A/QywjpHqt/zkH YcZb7a6mXcyuiYABQnzHTtcfMvD1Jqz258tanQlNViGFe6GqLfSjbwMgHFIBtUvIuGi3 RTCoRddaseYQghYXE72qzg3UDTzul60FUQ10IDgtdq8PxRBOgVvBdsdQ7M+k7YICttOq 97Jg== X-Gm-Message-State: ACrzQf15MmbUKnbLhtpdbVTbpjShrKzQbLrTo47Tlr/9/fUxBrKNHuP4 vax/btNk/v9+5q1de7efVSY= X-Google-Smtp-Source: AMsMyM7m8YTC2akZlBM+r+U5VbWuYmjJzbVONQfIxbSBv0EpttW5UE9uUoa9ep6lrZebjKdmGVX04Q== X-Received: by 2002:a17:903:1109:b0:179:d220:1f55 with SMTP id n9-20020a170903110900b00179d2201f55mr33147531plh.42.1667823773011; Mon, 07 Nov 2022 04:22:53 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id b17-20020a621b11000000b0053e4baecc14sm4365980pfb.108.2022.11.07.04.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:52 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 05/72] badblocks: Add badblocks merge logic Date: Mon, 7 Nov 2022 17:50:53 +0530 Message-Id: <01e72f626dcebdb8f0a78b53e4cd093357d82787.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Add badblocks merge logic Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/badblocks.c | 75 ++++++++++++++++++++++++++++++++++++++++++ lib/ext2fs/ext2fs.h | 2 ++ 2 files changed, 77 insertions(+) diff --git a/lib/ext2fs/badblocks.c b/lib/ext2fs/badblocks.c index 345168e0..36794e69 100644 --- a/lib/ext2fs/badblocks.c +++ b/lib/ext2fs/badblocks.c @@ -56,6 +56,74 @@ static errcode_t make_u32_list(int size, int num, __u32 *list, return 0; } +static inline int insert_ok(blk_t *array, int cnt, blk_t new) +{ + return (cnt == 0 || array[cnt - 1] != new); +} + +/* + * Merge list from src to dest + */ +static errcode_t merge_u32_list(ext2_u32_list src, ext2_u32_list dest) +{ + errcode_t retval; + int src_count = src->num; + int dest_count = dest->num; + int size = src_count + dest_count; + int size_entry = sizeof(blk_t); + blk_t *array; + blk_t *src_array = src->list; + blk_t *dest_array = dest->list; + int src_index = 0; + int dest_index = 0; + int uniq_cnt = 0; + + if (src->num == 0) + return 0; + + retval = ext2fs_get_array(size, size_entry, &array); + if (retval) + return retval; + + /* + * It is possible that src list and dest list could be + * duplicated when merging badblocks. + */ + while (src_index < src_count || dest_index < dest_count) { + if (src_index >= src_count) { + for (; dest_index < dest_count; dest_index++) + if (insert_ok(array, uniq_cnt, dest_array[dest_index])) + array[uniq_cnt++] = dest_array[dest_index]; + break; + } + if (dest_index >= dest_count) { + for (; src_index < src_count; src_index++) + if (insert_ok(array, uniq_cnt, src_array[src_index])) + array[uniq_cnt++] = src_array[src_index]; + break; + } + if (src_array[src_index] < dest_array[dest_index]) { + if (insert_ok(array, uniq_cnt, src_array[src_index])) + array[uniq_cnt++] = src_array[src_index]; + src_index++; + } else if (src_array[src_index] > dest_array[dest_index]) { + if (insert_ok(array, uniq_cnt, dest_array[dest_index])) + array[uniq_cnt++] = dest_array[dest_index]; + dest_index++; + } else { + if (insert_ok(array, uniq_cnt, dest_array[dest_index])) + array[uniq_cnt++] = dest_array[dest_index]; + src_index++; + dest_index++; + } + } + + ext2fs_free_mem(&dest->list); + dest->list = array; + dest->num = uniq_cnt; + dest->size = size; + return 0; +} /* * This procedure creates an empty u32 list. @@ -91,6 +159,13 @@ errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src, (ext2_u32_list *) dest); } +errcode_t ext2fs_badblocks_merge(ext2_badblocks_list src, + ext2_badblocks_list dest) +{ + return merge_u32_list((ext2_u32_list) src, + (ext2_u32_list) dest); +} + /* * This procedure frees a badblocks list. * diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 9cc994b1..18dddc2c 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -845,6 +845,8 @@ extern int ext2fs_badblocks_list_iterate(ext2_badblocks_iterate iter, extern void ext2fs_badblocks_list_iterate_end(ext2_badblocks_iterate iter); extern errcode_t ext2fs_badblocks_copy(ext2_badblocks_list src, ext2_badblocks_list *dest); +extern errcode_t ext2fs_badblocks_merge(ext2_badblocks_list src, + ext2_badblocks_list dest); extern int ext2fs_badblocks_equal(ext2_badblocks_list bb1, ext2_badblocks_list bb2); extern int ext2fs_u32_list_count(ext2_u32_list bb); From patchwork Mon Nov 7 12:20:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700553 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=VHnxedjw; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VkG3CB0z23lT for ; Mon, 7 Nov 2022 23:23:02 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkG1VbYz4xG8 for ; Mon, 7 Nov 2022 23:23:02 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VkG1SZRz4xGT; Mon, 7 Nov 2022 23:23:02 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=VHnxedjw; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkG1NBZz4xG8 for ; Mon, 7 Nov 2022 23:23:02 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232143AbiKGMXB (ORCPT ); Mon, 7 Nov 2022 07:23:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49348 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232070AbiKGMXA (ORCPT ); Mon, 7 Nov 2022 07:23:00 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D57D1633D for ; Mon, 7 Nov 2022 04:22:59 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id b1-20020a17090a7ac100b00213fde52d49so10121364pjl.3 for ; Mon, 07 Nov 2022 04:22:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S4wVAqt8eOXFjCWV8thKlgvNZ26hyT3uO6MwWqI24NA=; b=VHnxedjwPZv1GI0Wgl40+f3O/4PJn8vc2zKbiqdic5jC3UCpxjMk8JF9HaNjUUMTWl UHLsBiThTwBtJrlIw4/krc1RVu3SZNqssFbT8q62DEXvPwjo9jgkqFU/BEeK0jnTAZ4Q pp464XtXMsvi+9vfcQbDXXdnzVrTFgIx2mgTm7iB0qeleKW1AkQ9PTiJapk5okWK4MrB 0hmT0SzPGeYLEOPM3HBZLXrqmoYnf/q5cUVOXhFLk2wOJCHoAkAZ8AUGp3tkdqvjamX/ h9gNEZ86KwonSKrZhl0Tt92JNPIkFxIVJAgqelUsPAETDrEpd4VmxusbWyqEuEztLWH9 vOPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S4wVAqt8eOXFjCWV8thKlgvNZ26hyT3uO6MwWqI24NA=; b=Tjobm/GsRSKJ+vwE4IcpBe5ZnA98U4CCv/ynJSUKX3vlBnCl0sF48cdjSbk4geeiBt fkFOZjluKTN3JAPmch5ySG6URkWdS+ifafa5G8o/rToA4AQmbHH03ePTXZVDb85o3M44 tHbtUx0UzhpscobM897fjSThgbpfcJf6hrFglTmo5jXXjc0OTHSSejk9uNpqNGDn0NbK wb6HmmvJawzzr3MeUQPrFmZfbl3Fq4qTRu3G8MK+kn4rRB1KgXy1++Ahx2tP4Uxq2Qpe u929vzUUXJ9ZogvWnr1FBWRUZ1kqApf4GTu6c7uTPS+97iamymDcGrIgWwWcoOJY5w/o wimQ== X-Gm-Message-State: ACrzQf1ZaLbUZnaRfF0fC/jbnqm5nyr51eOLKnY34WqrOpwq3ZaCr+hq cVCLEGp6oYkq0TD2l3eZaMidzRgNh2o= X-Google-Smtp-Source: AMsMyM5khg5zdqQlbQ1ind3anwN6NIoBKoUzFN36VXzDIWRlVgbX/VwG/MxsLLBD5nvzBWkfBzJ0cw== X-Received: by 2002:a17:90b:3901:b0:213:dfd5:a75f with SMTP id ob1-20020a17090b390100b00213dfd5a75fmr42776009pjb.233.1667823779318; Mon, 07 Nov 2022 04:22:59 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y15-20020a62ce0f000000b0056d2e716e01sm4369131pfg.139.2022.11.07.04.22.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:22:58 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 06/72] dblist: add dblist merge logic Date: Mon, 7 Nov 2022 17:50:54 +0530 Message-Id: <99ecce2af8609cc7bbe79de273472dcb42cca8f5.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi This adds dblist merge logic. TODO: Add a unit test for core operations of dblist. Currently there is no such test for this. Signed-off-by: Li Xi Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/dblist.c | 36 ++++++++++++++++++++++++++++++++++++ lib/ext2fs/ext2fs.h | 1 + 2 files changed, 37 insertions(+) diff --git a/lib/ext2fs/dblist.c b/lib/ext2fs/dblist.c index bbdb221d..5568b8ec 100644 --- a/lib/ext2fs/dblist.c +++ b/lib/ext2fs/dblist.c @@ -119,6 +119,42 @@ errcode_t ext2fs_copy_dblist(ext2_dblist src, ext2_dblist *dest) return 0; } +/* + * Merge a directory block list @src to @dest + */ +errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest) +{ + unsigned long long src_count = src->count; + unsigned long long dest_count = dest->count; + unsigned long long size = src_count + dest_count; + size_t size_entry = sizeof(struct ext2_db_entry2); + struct ext2_db_entry2 *array, *array2; + errcode_t retval; + + if (src_count == 0) + return 0; + + if (src->sorted || (dest->sorted && dest_count != 0)) + return EINVAL; + + retval = ext2fs_get_array(size, size_entry, &array); + if (retval) + return retval; + + array2 = array; + memcpy(array, src->list, src_count * size_entry); + array += src_count; + memcpy(array, dest->list, dest_count * size_entry); + ext2fs_free_mem(&dest->list); + + dest->list = array2; + dest->count = src_count + dest_count; + dest->size = size; + dest->sorted = 0; + + return 0; +} + /* * Close a directory block list * diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 18dddc2c..443f93d2 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -1143,6 +1143,7 @@ extern errcode_t ext2fs_add_dir_block(ext2_dblist dblist, ext2_ino_t ino, blk_t blk, int blockcnt); extern errcode_t ext2fs_add_dir_block2(ext2_dblist dblist, ext2_ino_t ino, blk64_t blk, e2_blkcnt_t blockcnt); +extern errcode_t ext2fs_merge_dblist(ext2_dblist src, ext2_dblist dest); extern void ext2fs_dblist_sort(ext2_dblist dblist, EXT2_QSORT_TYPE (*sortfunc)(const void *, const void *)); From patchwork Mon Nov 7 12:20:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700554 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RU74Gbrm; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VkN3GCvz23lT for ; Mon, 7 Nov 2022 23:23:08 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkN2ms2z4xG8 for ; Mon, 7 Nov 2022 23:23:08 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VkN2F5dz4xGT; Mon, 7 Nov 2022 23:23:08 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RU74Gbrm; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkN1ldqz4xG8 for ; Mon, 7 Nov 2022 23:23:08 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232144AbiKGMXH (ORCPT ); Mon, 7 Nov 2022 07:23:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232128AbiKGMXG (ORCPT ); Mon, 7 Nov 2022 07:23:06 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DEA5633D for ; Mon, 7 Nov 2022 04:23:06 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id q1-20020a17090a750100b002139ec1e999so10133320pjk.1 for ; Mon, 07 Nov 2022 04:23:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ohcYnALKAlw5VG/FCKGNphojrvBOufLF01HC/abJirE=; b=RU74Gbrm8UAIvnuRDq+dwgTB1o/kvDFIhOIumAj1Wo34vKavre1HCd8z7An7aCQJvF O6TkLnBfYpfEgULi0DdeIoNLRaekBA/Wlj2oAoOnJ9oVwT3G9PgFyHK2HsWKHIGyYw75 EgTR2eltYagv0Fu2kNn/P5rpW96EtRHaCrVIXcmwGxyMsQHY2mDTs47dMTfqJcm80ygP LcJv2qzHNbcYc2k/vo6YGL8HZ+C4MN/+xZOOlngypCEKbVJn4DdeCq45lNS3oDfEvWzd DUDSUeeej/2WWDxB3XwaXZSHkv7cjnHvHMgX4RQNI+Sk/pfC9TkLp/y4uzHlxdRLX8gu ZIeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ohcYnALKAlw5VG/FCKGNphojrvBOufLF01HC/abJirE=; b=AytWeNgcGwfT7BsI5km1oMvJ7OKVsv11QGOfSpn8kKigsZCEM6V4xsRjKsKhUJmTMu pp626F3pmAYEsfKZa3tHnrBC+2oxB2CeGlL2xmxm8exmDcKv/F/G31QPSfRoRCzuAT5F KeGSj83fuoxubvf9og3cKqLDkafRhekHRuyfT9iwtJxDipauBH7hm8nwfcK5/y07fiE5 b/Gm1Htj3MGymTlmA1mkJaxB1yA25chteUy/67YYf0hea+QNc+BZ+SUPD7OTCqAdupLY ZCPRZLfLp3/p9D8Ohe8+DrboJyWtHJsZKYVfpW2bP55eV6BuZ0hLCOSwujm/F2PrqBp9 Sehg== X-Gm-Message-State: ACrzQf1K/rZDaYxLEqCZ+V9XEGUERdU9cxMCaxiaxVZS9MjBFdyUa75j QTgTqP63giDVxheVnj9BF20= X-Google-Smtp-Source: AMsMyM59oazQiin2a7AWOtpIHky1oStc8SX/uJtMsoCiVcCYpwGicgi8mzf1czK/qKsyjyGfsYccLw== X-Received: by 2002:a17:90b:1113:b0:216:616c:5fa0 with SMTP id gi19-20020a17090b111300b00216616c5fa0mr21327182pjb.225.1667823785568; Mon, 07 Nov 2022 04:23:05 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id nu14-20020a17090b1b0e00b0020d9c2f6c39sm6095431pjb.34.2022.11.07.04.23.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:04 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 07/72] libext2fs: Add rbtree bitmap merge logic Date: Mon, 7 Nov 2022 17:50:55 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong This adds rbtree bitmap merge logic. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/blkmap64_rb.c | 65 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/lib/ext2fs/blkmap64_rb.c b/lib/ext2fs/blkmap64_rb.c index 0df58dc7..d7c88aef 100644 --- a/lib/ext2fs/blkmap64_rb.c +++ b/lib/ext2fs/blkmap64_rb.c @@ -977,11 +977,76 @@ static void rb_print_stats(ext2fs_generic_bitmap_64 bitmap EXT2FS_ATTR((unused)) } #endif +static errcode_t rb_merge_bmap(ext2fs_generic_bitmap_64 src, + ext2fs_generic_bitmap_64 dest, + ext2fs_generic_bitmap_64 dup, + ext2fs_generic_bitmap_64 dup_allowed) +{ + struct ext2fs_rb_private *src_bp, *dest_bp, *dup_bp = NULL; + struct bmap_rb_extent *src_ext; + struct rb_node *src_node; + errcode_t retval = 0; + int dup_found = 0; + __u64 i; + + src_bp = (struct ext2fs_rb_private *) src->private; + dest_bp = (struct ext2fs_rb_private *) dest->private; + if (dup) + dup_bp = (struct ext2fs_rb_private *)dup->private; + src_bp->rcursor = NULL; + dest_bp->rcursor = NULL; + + src_node = ext2fs_rb_first(&src_bp->root); + while (src_node) { + src_ext = node_to_extent(src_node); + retval = rb_test_clear_bmap_extent(dest, + src_ext->start + src->start, + src_ext->count); + if (retval) { + rb_insert_extent(src_ext->start, src_ext->count, + dest_bp); + goto next; + } + + /* unlikely case, do it one by one block */ + for (i = src_ext->start; + i < src_ext->start + src_ext->count; i++) { + retval = rb_test_clear_bmap_extent(dest, i + src->start, 1); + if (retval) { + rb_insert_extent(i, 1, dest_bp); + continue; + } + if (dup_allowed) { + retval = rb_test_clear_bmap_extent(dup_allowed, + i + src->start, 1); + /* not existed in dup_allowed */ + if (retval) { + dup_found = 1; + if (dup_bp) + rb_insert_extent(i, 1, dup_bp); + } /* else we conside it not duplicated */ + } else { + if (dup_bp) + rb_insert_extent(i, 1, dup_bp); + dup_found = 1; + } + } +next: + src_node = ext2fs_rb_next(src_node); + } + + if (dup_found && dup) + return EEXIST; + + return 0; +} + struct ext2_bitmap_ops ext2fs_blkmap64_rbtree = { .type = EXT2FS_BMAP64_RBTREE, .new_bmap = rb_new_bmap, .free_bmap = rb_free_bmap, .copy_bmap = rb_copy_bmap, + .merge_bmap = rb_merge_bmap, .resize_bmap = rb_resize_bmap, .mark_bmap = rb_mark_bmap, .unmark_bmap = rb_unmark_bmap, From patchwork Mon Nov 7 12:20:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700555 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jJ2VwZ0Y; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VkW4cSkz23lT for ; Mon, 7 Nov 2022 23:23:15 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkW4BmNz4xG8 for ; Mon, 7 Nov 2022 23:23:15 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VkW48mFz4xGT; Mon, 7 Nov 2022 23:23:15 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jJ2VwZ0Y; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VkW44Xlz4xG8 for ; Mon, 7 Nov 2022 23:23:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232146AbiKGMXO (ORCPT ); Mon, 7 Nov 2022 07:23:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232128AbiKGMXN (ORCPT ); Mon, 7 Nov 2022 07:23:13 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09FF462DF for ; Mon, 7 Nov 2022 04:23:12 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id f5-20020a17090a4a8500b002131bb59d61so12428563pjh.1 for ; Mon, 07 Nov 2022 04:23:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rg7HbLWHsPCyjYPcF7Zv5a4qGG6YFqdd/e9CXoAgVFc=; b=jJ2VwZ0YE3FKUjOuKJ9NmbY/ou6xGILHaN2PLnKiANwK18oGQeUIdu9BT2HLOJR7to L+pboosRbflaig8h3OT746kEXycAuNs0Ow6Cf3ShP6DhkXzIbTDasd93Je9KUbABorvm dGHLHSSzE3bIh/vUIMzi4X8kAnDiI70vcXUwEukomIvLN+8Plj+X4qGNNh9sAbZkXadw jIeuCBZSK3RE8hP9PHSbuIzG9vKx7FhGEoYLlL2a7Br7TA/Y58/OfgprUwomMtoHU4Fz HHC6UP05tdcA6k9UC3z1UHlO4ye9sL+q7LFt9Ocdm9Ugb/mKlmQsuh6dAYWPqV7tLXex srYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rg7HbLWHsPCyjYPcF7Zv5a4qGG6YFqdd/e9CXoAgVFc=; b=2C6GsjvywTW8CV/KF/EP0CfkZZoJqbnt1FMIQ8Ti6QVCWYL8ciAKe75FjUOTCj9l/6 LuwoBkILZB3inwhnZfs+a+yQjZMYUoYrF48K2FTlMjyS6faqaoK5ZzsiXBaOG5fspUjo bhjgKn3wxGBa+qXYZ6y59Dn4YvZ0FRv42h1a+PN8kpx4oxMnpIp0m/rfpKL3GEFGsAgr YdZlK8+DdtS5ApwY82502mKldiUQqpGOwu+U19yReTdzuVfvSyqjcZwNFTDMZd8fOqhI IJS1WtRw6W1KxYvLzmgUQZGXt3RZozEFHjAv20FSH+e77hS0nA0awFD1eSdaM2YtzOYO gyHA== X-Gm-Message-State: ACrzQf0ui56Z6elyoR2Z5VsQD3OKXd/nizYhYdE77zyrvALchiC2peCV Y6jKs9ROJY5iqtYFiurTR18= X-Google-Smtp-Source: AMsMyM4Z3WUwdoC1VHoeUnZXo7H9+0QvQAixSHnP+LjwooRW0RHa+1Won4KBDFOkN7RC1TmOCIkt5Q== X-Received: by 2002:a17:90b:a53:b0:215:d767:4863 with SMTP id gw19-20020a17090b0a5300b00215d7674863mr600683pjb.233.1667823791526; Mon, 07 Nov 2022 04:23:11 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id b5-20020a170902e94500b00176d218889esm4857443pll.228.2022.11.07.04.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:10 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 08/72] libext2fs: Add bitmaps merge ops Date: Mon, 7 Nov 2022 17:50:56 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Signed-off-by: Li Xi Signed-off-by: Wang Shilong [Note: splitted rb merge tree logic patch such that we seperate out libext2fs changes from e2fsck specific changes] Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/bitmaps.c | 10 ++++++++++ lib/ext2fs/bmap64.h | 4 ++++ lib/ext2fs/ext2fs.h | 8 ++++++++ lib/ext2fs/gen_bitmap64.c | 29 +++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+) diff --git a/lib/ext2fs/bitmaps.c b/lib/ext2fs/bitmaps.c index 8bfa24b1..9437331e 100644 --- a/lib/ext2fs/bitmaps.c +++ b/lib/ext2fs/bitmaps.c @@ -45,6 +45,16 @@ errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src, { return (ext2fs_copy_generic_bmap(src, dest)); } + +errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src, + ext2fs_generic_bitmap dest, + ext2fs_generic_bitmap dup, + ext2fs_generic_bitmap dup_allowed) +{ + return ext2fs_merge_generic_bmap(src, dest, dup, + dup_allowed); +} + void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map) { ext2fs_set_generic_bmap_padding(map); diff --git a/lib/ext2fs/bmap64.h b/lib/ext2fs/bmap64.h index de334548..555193ee 100644 --- a/lib/ext2fs/bmap64.h +++ b/lib/ext2fs/bmap64.h @@ -72,6 +72,10 @@ struct ext2_bitmap_ops { void (*free_bmap)(ext2fs_generic_bitmap_64 bitmap); errcode_t (*copy_bmap)(ext2fs_generic_bitmap_64 src, ext2fs_generic_bitmap_64 dest); + errcode_t (*merge_bmap)(ext2fs_generic_bitmap_64 src, + ext2fs_generic_bitmap_64 dest, + ext2fs_generic_bitmap_64 dup, + ext2fs_generic_bitmap_64 dup_allowed); errcode_t (*resize_bmap)(ext2fs_generic_bitmap_64 bitmap, __u64 new_end, __u64 new_real_end); diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 443f93d2..54aed5d1 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -870,6 +870,10 @@ extern void ext2fs_free_block_bitmap(ext2fs_block_bitmap bitmap); extern void ext2fs_free_inode_bitmap(ext2fs_inode_bitmap bitmap); extern errcode_t ext2fs_copy_bitmap(ext2fs_generic_bitmap src, ext2fs_generic_bitmap *dest); +extern errcode_t ext2fs_merge_bitmap(ext2fs_generic_bitmap src, + ext2fs_generic_bitmap dest, + ext2fs_generic_bitmap dup, + ext2fs_generic_bitmap dup_allowed); extern errcode_t ext2fs_allocate_block_bitmap(ext2_filsys fs, const char *descr, ext2fs_block_bitmap *ret); @@ -1467,6 +1471,10 @@ void ext2fs_set_generic_bmap_padding(ext2fs_generic_bitmap bmap); errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap bmap, __u64 new_end, __u64 new_real_end); +errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src, + ext2fs_generic_bitmap gen_dest, + ext2fs_generic_bitmap gen_dup, + ext2fs_generic_bitmap dup_allowed); errcode_t ext2fs_compare_generic_bmap(errcode_t neq, ext2fs_generic_bitmap bm1, ext2fs_generic_bitmap bm2); diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c index f7710afd..c31f942f 100644 --- a/lib/ext2fs/gen_bitmap64.c +++ b/lib/ext2fs/gen_bitmap64.c @@ -346,6 +346,35 @@ errcode_t ext2fs_copy_generic_bmap(ext2fs_generic_bitmap gen_src, return 0; } +errcode_t ext2fs_merge_generic_bmap(ext2fs_generic_bitmap gen_src, + ext2fs_generic_bitmap gen_dest, + ext2fs_generic_bitmap gen_dup, + ext2fs_generic_bitmap gen_dup_allowed) +{ + ext2fs_generic_bitmap_64 src = (ext2fs_generic_bitmap_64)gen_src; + ext2fs_generic_bitmap_64 dest = (ext2fs_generic_bitmap_64)gen_dest; + ext2fs_generic_bitmap_64 dup = (ext2fs_generic_bitmap_64)gen_dup; + ext2fs_generic_bitmap_64 dup_allowed = (ext2fs_generic_bitmap_64)gen_dup_allowed; + + if (!src || !dest) + return EINVAL; + + if (!EXT2FS_IS_64_BITMAP(src) || !EXT2FS_IS_64_BITMAP(dest) || + (dup && !EXT2FS_IS_64_BITMAP(dup)) || + (dup_allowed && !EXT2FS_IS_64_BITMAP(dup_allowed))) + return EINVAL; + + if (src->bitmap_ops != dest->bitmap_ops || + (dup && src->bitmap_ops != dup->bitmap_ops) || + (dup_allowed && src->bitmap_ops != dup_allowed->bitmap_ops)) + return EINVAL; + + if (src->bitmap_ops->merge_bmap == NULL) + return EOPNOTSUPP; + + return src->bitmap_ops->merge_bmap(src, dest, dup, dup_allowed); +} + errcode_t ext2fs_resize_generic_bmap(ext2fs_generic_bitmap gen_bmap, __u64 new_end, __u64 new_real_end) From patchwork Mon Nov 7 12:20:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700556 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MT9bzENg; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vkd1WRyz23lT for ; Mon, 7 Nov 2022 23:23:21 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkd14QRz4xG8 for ; Mon, 7 Nov 2022 23:23:21 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vkd12Dhz4xGT; Mon, 7 Nov 2022 23:23:21 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MT9bzENg; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkd0y3fz4xG8 for ; Mon, 7 Nov 2022 23:23:21 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232148AbiKGMXU (ORCPT ); Mon, 7 Nov 2022 07:23:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232128AbiKGMXS (ORCPT ); Mon, 7 Nov 2022 07:23:18 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 188F06365 for ; Mon, 7 Nov 2022 04:23:18 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id d59-20020a17090a6f4100b00213202d77e1so14419821pjk.2 for ; Mon, 07 Nov 2022 04:23:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=irSzPw79BRKUVrHVl4f23XokUTckSF260z11EDuYM7Y=; b=MT9bzENgyufshjZCrjPnVVQY7lA+rvJ7Nij555u8DnnEbz8phJpGApqyMwdWcCm8Ls QSarOJXQWAB8lxXcZttXL8xXoDE2u7NWttdjxNM5G1VaRrH3PoTIKAMzajGJYavcn0y2 lfpogVkB8lS2xZbDF9shCvHTjX1xKYN6MviIPtXQIIgbOc1b4dZQP/Thh75q/T3IPD3f m+gRcNig/+2QZvlmmw5zk8uENPq6HaPDfW8UuhwXki6F/VSA3BzxxCuH44bJfABIUtyC siJ2Wg/qwVY83NFoDfGj1drsZxOcTbDbT/DLU2IVb4JN8ExiKX0F03rlJPwMgt3yDFR4 VPPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=irSzPw79BRKUVrHVl4f23XokUTckSF260z11EDuYM7Y=; b=oPa4V7UbYJe+yqPkVfuEd5iWfs87nMZsTZE2cTX7nyFHl7jChzAeu1+Vm7X3foI7R8 fLwwzwde+LTn6BJU48HgEAbpkr8HWC8VLyt6/7KIkm5DQd7LlZK+Qfijzu2Xu5M/K8Tr tBbhl6+Sr94FsV1uWzhCGKxZgCCs/2uKHbsmYnPVDKlrS1BzVwUmurNsAHU95mMysx2T LcBUgJiLbhp6bwl51N1X77hVe2DKPNhzO7aVIX6H3Yt+5vcpap8R/Zaltni3Vr6NJbPL 0+atTVrHc+3YrOMogt4QgYXyN+Fcm0+oksKgTyUYKrBiC2sIE3sojhKzpSqRH2A7wRja 74nw== X-Gm-Message-State: ACrzQf0uAe3SlqoQn05bI+PtbG7sS/TYTyIjAurPQKCa2hPmcAeFkGxT J/JbM6dh1dqgpjqymJXOWsQ= X-Google-Smtp-Source: AMsMyM5TQIXJHahPVWTVPL/9YGnXOogquy5Rzb4cnsUrmYxeVOO0ngFlD6sbUO2GptAwQ3oVDLXhVQ== X-Received: by 2002:a17:902:b281:b0:186:9596:742f with SMTP id u1-20020a170902b28100b001869596742fmr49856214plr.49.1667823797587; Mon, 07 Nov 2022 04:23:17 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id p1-20020a170902a40100b00177f4ef7970sm4896664plq.11.2022.11.07.04.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:16 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 09/72] libext2fs: Add flush cleanup API Date: Mon, 7 Nov 2022 17:50:57 +0530 Message-Id: <81451e5fcf02b502164e0e9f049df940b07de715.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/ext2_io.h | 2 ++ lib/ext2fs/undo_io.c | 19 +++++++++++++++++++ lib/ext2fs/unix_io.c | 24 +++++++++++++++++++++--- 3 files changed, 42 insertions(+), 3 deletions(-) diff --git a/lib/ext2fs/ext2_io.h b/lib/ext2fs/ext2_io.h index 8fe5b323..8cc355be 100644 --- a/lib/ext2fs/ext2_io.h +++ b/lib/ext2fs/ext2_io.h @@ -82,6 +82,7 @@ struct struct_io_manager { errcode_t (*write_blk)(io_channel channel, unsigned long block, int count, const void *data); errcode_t (*flush)(io_channel channel); + errcode_t (*flush_cleanup)(io_channel channel); errcode_t (*write_byte)(io_channel channel, unsigned long offset, int count, const void *data); errcode_t (*set_option)(io_channel channel, const char *option, @@ -116,6 +117,7 @@ struct struct_io_manager { #define io_channel_read_blk(c,b,n,d) ((c)->manager->read_blk((c),b,n,d)) #define io_channel_write_blk(c,b,n,d) ((c)->manager->write_blk((c),b,n,d)) #define io_channel_flush(c) ((c)->manager->flush((c))) +#define io_channel_flush_cleanup(c) ((c)->manager->flush_cleanup((c))) #define io_channel_bumpcount(c) ((c)->refcount++) /* io_manager.c */ diff --git a/lib/ext2fs/undo_io.c b/lib/ext2fs/undo_io.c index f4a6d526..678ff421 100644 --- a/lib/ext2fs/undo_io.c +++ b/lib/ext2fs/undo_io.c @@ -1024,6 +1024,24 @@ static errcode_t undo_flush(io_channel channel) return retval; } +/* + * Flush data buffers to disk and cleanup the cache. + */ +static errcode_t undo_flush_cleanup(io_channel channel) +{ + errcode_t retval = 0; + struct undo_private_data *data; + + EXT2_CHECK_MAGIC(channel, EXT2_ET_MAGIC_IO_CHANNEL); + data = (struct undo_private_data *) channel->private_data; + EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL); + + if (data->real) + retval = io_channel_flush_cleanup(data->real); + + return retval; +} + static errcode_t undo_set_option(io_channel channel, const char *option, const char *arg) { @@ -1095,6 +1113,7 @@ static struct struct_io_manager struct_undo_manager = { .read_blk = undo_read_blk, .write_blk = undo_write_blk, .flush = undo_flush, + .flush_cleanup = undo_flush_cleanup, .write_byte = undo_write_byte, .set_option = undo_set_option, .get_stats = undo_get_stats, diff --git a/lib/ext2fs/unix_io.c b/lib/ext2fs/unix_io.c index 5b894826..8f8118a3 100644 --- a/lib/ext2fs/unix_io.c +++ b/lib/ext2fs/unix_io.c @@ -1173,9 +1173,9 @@ static errcode_t unix_write_byte(io_channel channel, unsigned long offset, } /* - * Flush data buffers to disk. + * Flush data buffers to disk and invalidate cache if needed */ -static errcode_t unix_flush(io_channel channel) +static errcode_t _unix_flush(io_channel channel, int invalidate) { struct unix_private_data *data; errcode_t retval = 0; @@ -1185,7 +1185,7 @@ static errcode_t unix_flush(io_channel channel) EXT2_CHECK_MAGIC(data, EXT2_ET_MAGIC_UNIX_IO_CHANNEL); #ifndef NO_IO_CACHE - retval = flush_cached_blocks(channel, data, 0); + retval = flush_cached_blocks(channel, data, invalidate); #endif #ifdef HAVE_FSYNC if (!retval && fsync(data->dev) != 0) @@ -1194,6 +1194,22 @@ static errcode_t unix_flush(io_channel channel) return retval; } +/* + * Flush data buffers to disk. + */ +static errcode_t unix_flush(io_channel channel) +{ + return _unix_flush(channel, 0); +} + +/* + * Flush data buffers to disk and invalidate cache. + */ +static errcode_t unix_flush_cleanup(io_channel channel) +{ + return _unix_flush(channel, 1); +} + static errcode_t unix_set_option(io_channel channel, const char *option, const char *arg) { @@ -1383,6 +1399,7 @@ static struct struct_io_manager struct_unix_manager = { .discard = unix_discard, .cache_readahead = unix_cache_readahead, .zeroout = unix_zeroout, + .flush_cleanup = unix_flush_cleanup, }; io_manager unix_io_manager = &struct_unix_manager; @@ -1404,6 +1421,7 @@ static struct struct_io_manager struct_unixfd_manager = { .discard = unix_discard, .cache_readahead = unix_cache_readahead, .zeroout = unix_zeroout, + .flush_cleanup = unix_flush_cleanup, }; io_manager unixfd_io_manager = &struct_unixfd_manager; From patchwork Mon Nov 7 12:20:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700557 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=J2bSlmaG; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vkn3mTfz23lT for ; Mon, 7 Nov 2022 23:23:29 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkn3Jsnz4xG8 for ; Mon, 7 Nov 2022 23:23:29 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vkn3G6cz4xGT; Mon, 7 Nov 2022 23:23:29 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=J2bSlmaG; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkn396pz4xG8 for ; Mon, 7 Nov 2022 23:23:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232153AbiKGMX1 (ORCPT ); Mon, 7 Nov 2022 07:23:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49598 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232151AbiKGMX0 (ORCPT ); Mon, 7 Nov 2022 07:23:26 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73070E7E for ; Mon, 7 Nov 2022 04:23:24 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id h193so10280039pgc.10 for ; Mon, 07 Nov 2022 04:23:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LtwaF0H7hgt2Q37P05Dj3KxwH8awOYacDLb4m7drUn4=; b=J2bSlmaG0k+Xf8UMmNJzKcT4kn2LlJB5mxhKJMBt8XAhcPMaejl9DL2WFIsXo4lzlX SiHFjVExOiV2qZA5KJtzWnplQqZ7eWGq+VYE2v45VNHYrLTUFiXLvsN2/+fPjFmA904a P+iMz5nYuZ8ZMlqSJQi/Zqvi5K5ftAeyC31mPB9myfSMn4BRyjjNeh+6ek6vrY64Tzpe Wd8qkJo1ti8jZ2ajF04yAsjX92z5vtw5HO6Rdupz+vwuFU7RTvNj7h86AqKPmsdaKjr5 dvT1LGnR5vShfuoHlZAiLHI1iSZoRBmVjLE6bgYjjQc99Ew2lMmXvdKKG+w3uMQ6vlMX tOUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LtwaF0H7hgt2Q37P05Dj3KxwH8awOYacDLb4m7drUn4=; b=ZLOUvCai6sPP32oQaetsI7AJWry5zPeuuTayY//VElrw6Yigjx/tbTG6+dFwwcNYCa bDar0Xa5tdiEUYy3m4gcpIUp6D8qvW9vUs10/lzQyaxRVbWCYGqJltrVSAmQcASUtn0U UJqGj/aYGkyGkUZy6ZB/5uaxWsod9o50LAHsk/gJXRVhAXu9zLv0KAYCYhE2iBsHIYS7 qQyhX7bf9vrMTnv7jmAT/tf/R2MDzfXHPBLCXAenQ/5R64cGsNFU0+WwAbxFwKq2sEBz XvDgcY7A+smED8Pur5hRX5uPbp1j43P1ODKa1+THyRWhrNw7AaDzNW/3aEtkTTdDdvR2 t7Jg== X-Gm-Message-State: ANoB5pkO/bNw+sKeU8vZhRCpAjljpGhePwjaIhZPGHS1MB3I+ZSwArzL RU1Qqo0TiqAuCihPp7A1hz0= X-Google-Smtp-Source: AA0mqf5BnQh3CROdu4Et5Cgbq3JygdUnh3V0bWL+vCJHKZ2i3WAqqpkLqfuKJxrdvBkTZzh+kWw5vw== X-Received: by 2002:a05:6a00:168b:b0:56e:d7f4:3aaf with SMTP id k11-20020a056a00168b00b0056ed7f43aafmr10849337pfc.81.1667823803974; Mon, 07 Nov 2022 04:23:23 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id v4-20020a622f04000000b0056c6e59fb69sm4352877pfv.83.2022.11.07.04.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:23 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 10/72] libext2fs: merge icounts after thread finishes Date: Mon, 7 Nov 2022 17:50:58 +0530 Message-Id: <817a806b1c7ed161cb76da5f163a9aade60c8dfd.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Merge inode_count and inode_link_info properly after threads finish. Signed-off-by: Li Xi Signed-off-by: Wang Shilong [Note: splitted the patch to seperate libext2fs changes from e2fsck] Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/ext2fs.h | 1 + lib/ext2fs/icount.c | 103 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 104 insertions(+) diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 54aed5d1..139a25fc 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -1546,6 +1546,7 @@ extern errcode_t ext2fs_icount_decrement(ext2_icount_t icount, ext2_ino_t ino, __u16 *ret); extern errcode_t ext2fs_icount_store(ext2_icount_t icount, ext2_ino_t ino, __u16 count); +extern errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest); extern ext2_ino_t ext2fs_get_icount_size(ext2_icount_t icount); errcode_t ext2fs_icount_validate(ext2_icount_t icount, FILE *); diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c index 888a90b2..766eccca 100644 --- a/lib/ext2fs/icount.c +++ b/lib/ext2fs/icount.c @@ -13,6 +13,7 @@ #if HAVE_UNISTD_H #include #endif +#include #include #include #include @@ -701,6 +702,108 @@ errcode_t ext2fs_icount_store(ext2_icount_t icount, ext2_ino_t ino, return 0; } +errcode_t ext2fs_icount_merge_full_map(ext2_icount_t src, ext2_icount_t dest) +{ + /* TODO: add the support for full map */ + return EOPNOTSUPP; +} + +errcode_t ext2fs_icount_merge_el(ext2_icount_t src, ext2_icount_t dest) +{ + int src_count = src->count; + int dest_count = dest->count; + int size = src_count + dest_count; + int size_entry = sizeof(struct ext2_icount_el); + struct ext2_icount_el *array; + struct ext2_icount_el *array_ptr; + struct ext2_icount_el *src_array = src->list; + struct ext2_icount_el *dest_array = dest->list; + int src_index = 0; + int dest_index = 0; + errcode_t retval; + + if (src_count == 0) + return 0; + + retval = ext2fs_get_array(size, size_entry, &array); + if (retval) + return retval; + + array_ptr = array; + /* + * This can be improved by binary search and memcpy, but codes + * would be more complex. And if number of bad blocks is small, + * the optimization won't improve performance a lot. + */ + while (src_index < src_count || dest_index < dest_count) { + if (src_index >= src_count) { + memcpy(array_ptr, &dest_array[dest_index], + (dest_count - dest_index) * size_entry); + break; + } + if (dest_index >= dest_count) { + memcpy(array_ptr, &src_array[src_index], + (src_count - src_index) * size_entry); + break; + } + if (src_array[src_index].ino < dest_array[dest_index].ino) { + *array_ptr = src_array[src_index]; + src_index++; + } else { + assert(src_array[src_index].ino > + dest_array[dest_index].ino); + *array_ptr = dest_array[dest_index]; + dest_index++; + } + array_ptr++; + } + + ext2fs_free_mem(&dest->list); + dest->list = array; + dest->count = src_count + dest_count; + dest->size = size; + dest->last_lookup = NULL; + return 0; +} + +errcode_t ext2fs_icount_merge(ext2_icount_t src, ext2_icount_t dest) +{ + errcode_t retval; + + if (src->fullmap && !dest->fullmap) + return EINVAL; + + if (!src->fullmap && dest->fullmap) + return EINVAL; + + if (src->multiple && !dest->multiple) + return EINVAL; + + if (!src->multiple && dest->multiple) + return EINVAL; + + if (src->fullmap) + return ext2fs_icount_merge_full_map(src, dest); + + retval = ext2fs_merge_bitmap(src->single, dest->single, NULL, + NULL); + if (retval) + return retval; + + if (src->multiple) { + retval = ext2fs_merge_bitmap(src->multiple, dest->multiple, + NULL, NULL); + if (retval) + return retval; + } + + retval = ext2fs_icount_merge_el(src, dest); + if (retval) + return retval; + + return 0; +} + ext2_ino_t ext2fs_get_icount_size(ext2_icount_t icount) { if (!icount || icount->magic != EXT2_ET_MAGIC_ICOUNT) From patchwork Mon Nov 7 12:20:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700558 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=dzaEh4ra; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vkv5dvlz23lx for ; Mon, 7 Nov 2022 23:23:35 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkv58mYz4xGT for ; Mon, 7 Nov 2022 23:23:35 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vkv56Pmz4xZ3; Mon, 7 Nov 2022 23:23:35 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=dzaEh4ra; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vkv52mRz4xGT for ; Mon, 7 Nov 2022 23:23:35 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231831AbiKGMXe (ORCPT ); Mon, 7 Nov 2022 07:23:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232151AbiKGMXc (ORCPT ); Mon, 7 Nov 2022 07:23:32 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 493966247 for ; Mon, 7 Nov 2022 04:23:30 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id b62so10338460pgc.0 for ; Mon, 07 Nov 2022 04:23:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yNH/JRMMbNBjJGEyAIMaU9Rbue7IVZktvVu0mJ7rLNc=; b=dzaEh4raQcQr2hBzOZqZPhKf4hDCUSrlW+7rTUcXfd1iMwnBUa/LOpM2hzOMNLJfFs 6GXZAalYTp6sX+9rykYWWj0q+GssuY6Bdaopue+YlCq9uYQEhPZ7xw3+/Xqf3le786/S 0hB7sPXaP4SdoVj8cduB9XbQwD1/vQrDL7erf5G4PWfW0dRBjxXaVVcLn6Jca/sueYhP TlelrCxhfh/eZ+mYj/+JQSgYAd2/LahF040N+51Av7fN3QW/oLq55gJTenN8XuXu+dW8 v4hNV1sK/7yfk3yyySZShEdWd9sfrMqcxqGMAsIsMBt8zurZSgnkicYbloAHJCaegQg2 vFlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yNH/JRMMbNBjJGEyAIMaU9Rbue7IVZktvVu0mJ7rLNc=; b=UostMGbveqRjXmzKVK2DB9L7aA3yuyf3N42GJJ8te5JuGotsS/aZxqPeP6eHSLrhJF myKUwD/j1YGgZ4JkZy0GbBDKVJubmn+agQAq/GJQ31BZnkqV+cWntFijOr2DJu9TK/NE g3wxQ2NupVhvoGvSmRAA/7pWZLl8yMwGNpjEPJNaMoy4X0lwmmJRjrf0VKJFfQaxGBtQ A4eUp9VlwYygwZNUaTIWSW1gS5Pl+Uf247qJUJ3vMHsev9iCa6uLpn0UHrwHJFQwILhZ z3uPb1QRW2deYYVz+FBBntQmQoQ5l4ZSP4Z2YtWQjOpoOGoMfKVPl+IS/IzWepRaaY5m PyMg== X-Gm-Message-State: ACrzQf118dt5Z7jrDxkEJSJNvZWBbRjaXEFhdr+xdxxvxITPWmVtmn0S Suva02MG5cHAcWQPCknKy7M= X-Google-Smtp-Source: AMsMyM4MdcYpDheORHYmCYatZ5QkxfB8DhbHv8BdWDOHCODe4wq8AEa4fuO+tGpJZTs2mAL5TD21EA== X-Received: by 2002:a63:5123:0:b0:46f:f329:c013 with SMTP id f35-20020a635123000000b0046ff329c013mr27173500pgb.428.1667823809762; Mon, 07 Nov 2022 04:23:29 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id b12-20020a1709027e0c00b00186f608c543sm4834064plm.304.2022.11.07.04.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:29 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 11/72] libext2fs: merge quota context after threads finish Date: Mon, 7 Nov 2022 17:50:59 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Every threads calculate its own quota accounting, merge them after threads finish. Signed-off-by: Wang Shilong [Note: splitted the patch to seperate libext2fs changes from e2fsck] Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/support/mkquota.c | 39 +++++++++++++++++++++++++++++++++++++++ lib/support/quotaio.h | 3 +++ 2 files changed, 42 insertions(+) diff --git a/lib/support/mkquota.c b/lib/support/mkquota.c index 9339c994..ce1ab4cd 100644 --- a/lib/support/mkquota.c +++ b/lib/support/mkquota.c @@ -618,6 +618,45 @@ out: return err; } +static errcode_t merge_usage(dict_t *dest, dict_t *src) +{ + dnode_t *n; + struct dquot *src_dq, *dest_dq; + + for (n = dict_first(src); n; n = dict_next(src, n)) { + src_dq = dnode_get(n); + if (!src_dq) + continue; + dest_dq = get_dq(dest, src_dq->dq_id); + if (dest_dq == NULL) + return -ENOMEM; + dest_dq->dq_dqb.dqb_curspace += src_dq->dq_dqb.dqb_curspace; + dest_dq->dq_dqb.dqb_curinodes += src_dq->dq_dqb.dqb_curinodes; + } + + return 0; +} + + +errcode_t quota_merge_and_update_usage(quota_ctx_t dest_qctx, + quota_ctx_t src_qctx) +{ + dict_t *dict; + enum quota_type qtype; + errcode_t retval = 0; + + for (qtype = 0; qtype < MAXQUOTAS; qtype++) { + dict = src_qctx->quota_dict[qtype]; + if (!dict) + continue; + retval = merge_usage(dest_qctx->quota_dict[qtype], dict); + if (retval) + break; + } + + return retval; +} + /* * Compares the measured quota in qctx->quota_dict with that in the quota inode * on disk and updates the limits in qctx->quota_dict. 'usage_inconsistent' is diff --git a/lib/support/quotaio.h b/lib/support/quotaio.h index 84fac35d..240a0762 100644 --- a/lib/support/quotaio.h +++ b/lib/support/quotaio.h @@ -40,6 +40,7 @@ #include "ext2fs/ext2_fs.h" #include "ext2fs/ext2fs.h" #include "dqblk_v2.h" +#include "support/dict.h" typedef int64_t qsize_t; /* Type in which we store size limitations */ @@ -236,6 +237,8 @@ int quota_file_exists(ext2_filsys fs, enum quota_type qtype); void quota_set_sb_inum(ext2_filsys fs, ext2_ino_t ino, enum quota_type qtype); errcode_t quota_compare_and_update(quota_ctx_t qctx, enum quota_type qtype, int *usage_inconsistent); +errcode_t quota_merge_and_update_usage(quota_ctx_t dest_qctx, + quota_ctx_t src_qctx); int parse_quota_opts(const char *opts, int (*func)(char *)); /* parse_qtype.c */ From patchwork Mon Nov 7 12:21:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700559 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=odGGP31j; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vl14tDqz23lT for ; Mon, 7 Nov 2022 23:23:41 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vl14RcZz4xG8 for ; Mon, 7 Nov 2022 23:23:41 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vl14PBXz4xGT; Mon, 7 Nov 2022 23:23:41 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=odGGP31j; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vl14Jr4z4xG8 for ; Mon, 7 Nov 2022 23:23:41 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232154AbiKGMXk (ORCPT ); Mon, 7 Nov 2022 07:23:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232018AbiKGMXh (ORCPT ); Mon, 7 Nov 2022 07:23:37 -0500 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3E80E63D5 for ; Mon, 7 Nov 2022 04:23:36 -0800 (PST) Received: by mail-pl1-x62f.google.com with SMTP id g24so10914548plq.3 for ; Mon, 07 Nov 2022 04:23:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lvTW6jamv93zZLHw0sL+JvIG/eGiiBZQA/Z4AIhsJ3o=; b=odGGP31j3pmLdnvIeQWAvG6AuTassx+sRxlS3x0ci1plwjWl6w6gxpaM5sG9nBVMQu SpdZTEDlu27uxgCLDVyHAn4R5JIym/jynpSo5mjVy1kUdh1A1zpyVH8YUDvxbLnPzQWR RHlh+UA93awQlWbADjfxPQ/8nRhZz6/PwDyV9ln/JIj6xkRT3/g6eLhdwoW3EqyKsyYE qlK6HkSgpD2iJr123YhHIyn6VvzTIj6Wif+tlZb6O1kBHgyGC93uoV31MmGHzVHgUjXI Adftpouogfrc94K9rj5AQ5O0dOgfnz6LEfQMpqLNZIHl+FuEv2tN47WiDMGnU5S3CBRv V/Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lvTW6jamv93zZLHw0sL+JvIG/eGiiBZQA/Z4AIhsJ3o=; b=tmwj9BiDrCkImHI28bKjjned/HFD5LuFTJH6h9seJ+ll1dylULACSWUeS0w0QPkENs Pj0/SDZdPmMX8RT5w4HYYMgdPMWLrnn3Z6phXlTMGoUY+4UZDT9e3lg3eP5fakbgkSPf PpT0s0z2DvO1Ddag8x0L6qA7UqrjgmqJXgZLRQUQVwe8s1WBhUP/OeemS9tIMolADpV6 O3/tFPf81Agdgfw+9lp+i1KjgCgsCHgvvR2rFAFD4mWT1sdKMLU3AuGcUskys8CJxnHC Ci8P1SggVzL580POtSAiLEYl2Pfb+h+OXm+5v62DGJURNrurFDbFhLuaDe3/UnzqTOiI Q7DA== X-Gm-Message-State: ACrzQf1OxWzSGX5CE/cneeEtfTWkhCpo8KR2UjN86pUQIBTYaP/uPJNO ps5SmI5WyYuOCgU/elLziKU= X-Google-Smtp-Source: AMsMyM5NjqEgDyyV1fO1zckTVtif+gXHKM3W7Y3bzzCZeYpnxBRu6YE42v7G1gZev9kiIRq01kfU/A== X-Received: by 2002:a17:90a:d084:b0:213:8cf1:4d34 with SMTP id k4-20020a17090ad08400b002138cf14d34mr52416138pju.150.1667823815701; Mon, 07 Nov 2022 04:23:35 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id e4-20020a170902784400b001871acf245csm4867010pln.37.2022.11.07.04.23.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:35 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Saranya Muruganandam , Ritesh Harjani Subject: [RFCv1 12/72] libext2fs: dupfs: Add fs clone & merge api Date: Mon, 7 Nov 2022 17:51:00 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Saranya Muruganandam This patch mainly adds "parent" & "clone_flags" member in ext2_filsys struct for enabling multi-threading. Based on what CLONE flags will be passed from the client of libext2fs down to ext2fs_clone_fs(), those structures/bitmaps will be cloned (thread-aware child copy) and rest will be shared with the parent fs. The same flags will also help to merge those cloned bitmap structures back into the parent bitmaps when ext2fs_merge_fs() will be called with childfs struct. Signed-off-by: Saranya Muruganandam Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/dupfs.c | 183 ++++++++++++++++++++++++++++++++++++++++++++ lib/ext2fs/ext2fs.h | 23 ++++++ 2 files changed, 206 insertions(+) diff --git a/lib/ext2fs/dupfs.c b/lib/ext2fs/dupfs.c index 02721e1a..ecc57cf7 100644 --- a/lib/ext2fs/dupfs.c +++ b/lib/ext2fs/dupfs.c @@ -14,8 +14,12 @@ #if HAVE_UNISTD_H #include #endif +#if HAVE_PTHREAD_H +#include +#endif #include #include +#include #include "ext2_fs.h" #include "ext2fsP.h" @@ -120,3 +124,182 @@ errout: } +#ifdef HAVE_PTHREAD +errcode_t ext2fs_clone_fs(ext2_filsys fs, ext2_filsys *dest, unsigned int flags) +{ + errcode_t retval; + ext2_filsys childfs; + + EXT2_CHECK_MAGIC(fs, EXT2_ET_MAGIC_EXT2FS_FILSYS); + + retval = ext2fs_get_mem(sizeof(struct struct_ext2_filsys), &childfs); + if (retval) + return retval; + + /* make an exact copy implying lists and memory structures are shared */ + memcpy(childfs, fs, sizeof(struct struct_ext2_filsys)); + childfs->inode_map = NULL; + childfs->block_map = NULL; + childfs->badblocks = NULL; + childfs->dblist = NULL; + + pthread_mutex_lock(&fs->refcount_mutex); + fs->refcount++; + pthread_mutex_unlock(&fs->refcount_mutex); + + if ((flags & EXT2FS_CLONE_INODE) && fs->inode_map) { + retval = ext2fs_copy_bitmap(fs->inode_map, &childfs->inode_map); + if (retval) + return retval; + childfs->inode_map->fs = childfs; + } + + if ((flags & EXT2FS_CLONE_BLOCK) && fs->block_map) { + retval = ext2fs_copy_bitmap(fs->block_map, &childfs->block_map); + if (retval) + return retval; + childfs->block_map->fs = childfs; + } + + if ((flags & EXT2FS_CLONE_BADBLOCKS) && fs->badblocks) { + retval = ext2fs_badblocks_copy(fs->badblocks, &childfs->badblocks); + if (retval) + return retval; + } + + if ((flags & EXT2FS_CLONE_DBLIST) && fs->dblist) { + retval = ext2fs_copy_dblist(fs->dblist, &childfs->dblist); + if (retval) + return retval; + childfs->dblist->fs = childfs; + } + + /* icache when NULL will be rebuilt if needed */ + childfs->icache = NULL; + + childfs->clone_flags = flags; + childfs->parent = fs; + *dest = childfs; + + return 0; +} + +errcode_t ext2fs_merge_fs(ext2_filsys *thread_fs) +{ + ext2_filsys fs = *thread_fs; + errcode_t retval = 0; + ext2_filsys dest = fs->parent; + ext2_filsys src = fs; + unsigned int flags = fs->clone_flags; + struct ext2_inode_cache *icache; + io_channel dest_io; + io_channel dest_image_io; + ext2fs_inode_bitmap inode_map; + ext2fs_block_bitmap block_map; + ext2_badblocks_list badblocks; + ext2_dblist dblist; + void *priv_data; + int fsflags; + + pthread_mutex_lock(&fs->refcount_mutex); + fs->refcount--; + assert(fs->refcount >= 0); + pthread_mutex_unlock(&fs->refcount_mutex); + + icache = dest->icache; + dest_io = dest->io; + dest_image_io = dest->image_io; + inode_map = dest->inode_map; + block_map = dest->block_map; + badblocks = dest->badblocks; + dblist = dest->dblist; + priv_data = dest->priv_data; + fsflags = dest->flags; + + memcpy(dest, src, sizeof(struct struct_ext2_filsys)); + + dest->io = dest_io; + dest->image_io = dest_image_io; + dest->icache = icache; + dest->inode_map = inode_map; + dest->block_map = block_map; + dest->badblocks = badblocks; + dest->dblist = dblist; + dest->priv_data = priv_data; + if (dest->dblist) + dest->dblist->fs = dest; + dest->flags = src->flags | fsflags; + if (!(src->flags & EXT2_FLAG_VALID) || !(dest->flags & EXT2_FLAG_VALID)) + ext2fs_unmark_valid(dest); + + if ((flags & EXT2FS_CLONE_INODE) && src->inode_map) { + if (dest->inode_map == NULL) { + dest->inode_map = src->inode_map; + src->inode_map = NULL; + } else { + retval = ext2fs_merge_bitmap(src->inode_map, dest->inode_map, NULL, NULL); + if (retval) + goto out; + } + dest->inode_map->fs = dest; + } + + if ((flags & EXT2FS_CLONE_BLOCK) && src->block_map) { + if (dest->block_map == NULL) { + dest->block_map = src->block_map; + src->block_map = NULL; + } else { + retval = ext2fs_merge_bitmap(src->block_map, dest->block_map, NULL, NULL); + if (retval) + goto out; + } + dest->block_map->fs = dest; + } + + if ((flags & EXT2FS_CLONE_BADBLOCKS) && src->badblocks) { + if (dest->badblocks == NULL) + retval = ext2fs_badblocks_copy(src->badblocks, &dest->badblocks); + else + retval = ext2fs_badblocks_merge(src->badblocks, dest->badblocks); + if (retval) + goto out; + } + + if ((flags & EXT2FS_CLONE_DBLIST) && src->dblist) { + if (dest->dblist == NULL) { + dest->dblist = src->dblist; + src->dblist = NULL; + } else { + retval = ext2fs_merge_dblist(src->dblist, dest->dblist); + if (retval) + goto out; + } + dest->dblist->fs = dest; + } + + if (src->icache) { + ext2fs_free_inode_cache(src->icache); + src->icache = NULL; + } + +out: + if (src->io) + io_channel_close(src->io); + + if ((flags & EXT2FS_CLONE_INODE) && src->inode_map) + ext2fs_free_generic_bmap(src->inode_map); + if ((flags & EXT2FS_CLONE_BLOCK) && src->block_map) + ext2fs_free_generic_bmap(src->block_map); + if ((flags & EXT2FS_CLONE_BADBLOCKS) && src->badblocks) + ext2fs_badblocks_list_free(src->badblocks); + if ((flags & EXT2FS_CLONE_DBLIST) && src->dblist) { + ext2fs_free_dblist(src->dblist); + src->dblist = NULL; + } + + ext2fs_free_mem(&src); + *thread_fs = NULL; + + return retval; +} +#endif diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 139a25fc..b1505f95 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -12,6 +12,10 @@ #ifndef _EXT2FS_EXT2FS_H #define _EXT2FS_EXT2FS_H +#ifdef HAVE_PTHREAD_H +#include +#endif + #ifdef __GNUC__ #define EXT2FS_ATTR(x) __attribute__(x) #else @@ -331,6 +335,13 @@ struct struct_ext2_filsys { struct ext2fs_hashmap* block_sha_map; const struct ext2fs_nls_table *encoding; + +#ifdef HAVE_PTHREAD + struct struct_ext2_filsys *parent; + size_t refcount; + pthread_mutex_t refcount_mutex; + unsigned int clone_flags; +#endif }; #if EXT2_FLAT_INCLUDES @@ -1057,6 +1068,18 @@ extern errcode_t ext2fs_move_blocks(ext2_filsys fs, /* check_desc.c */ extern errcode_t ext2fs_check_desc(ext2_filsys fs); +#ifdef HAVE_PTHREAD +/* flags for ext2fs_clone_fs */ +#define EXT2FS_CLONE_BLOCK 0x0001 +#define EXT2FS_CLONE_INODE 0x0002 +#define EXT2FS_CLONE_BADBLOCKS 0x0004 +#define EXT2FS_CLONE_DBLIST 0x0008 + +extern errcode_t ext2fs_clone_fs(ext2_filsys fs, ext2_filsys *dest, + unsigned int flags); +extern errcode_t ext2fs_merge_fs(ext2_filsys *fs); +#endif + /* closefs.c */ extern errcode_t ext2fs_close(ext2_filsys fs); extern errcode_t ext2fs_close2(ext2_filsys fs, int flags); From patchwork Mon Nov 7 12:21:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700560 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ePuwIJfO; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vl91RTnz23lT for ; Mon, 7 Nov 2022 23:23:49 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vl90zFFz4xwy for ; Mon, 7 Nov 2022 23:23:49 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vl90wMWz4xx0; Mon, 7 Nov 2022 23:23:49 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ePuwIJfO; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vl90qxmz4xwy for ; Mon, 7 Nov 2022 23:23:49 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232123AbiKGMXs (ORCPT ); Mon, 7 Nov 2022 07:23:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232159AbiKGMXn (ORCPT ); Mon, 7 Nov 2022 07:23:43 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21D12262D for ; Mon, 7 Nov 2022 04:23:42 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id f63so10319692pgc.2 for ; Mon, 07 Nov 2022 04:23:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Cdp2EtBFym3wXOJtzcxo7hXulJYdnGAKwekrjG+zUb0=; b=ePuwIJfOiqNjLwnrs8fQXia1tSkKaWqHiYhh5rB4yQ12sIGpVnyt3zmvbyRgDsAdNf z0YBQL+b7ju2XA26nK5qoPPMu+184dsHxwI51AYPi53P8u+d+XR3cQnWmm4btU9DfSxs mc/lX+ORbQFAkT883VHP3oL9KzDOqj8FLwRSjOcxgkVba1+LBfAGU6dXQAFgQzsvtnlW +YR+nAonD1ZoNmAghRJuHuxUCVDMxZsloUPtVMO2t+51wnMetH/WN74QvjW+3X0m0E+t aaUWaOckzANtSBN3QPSptmGcXJJxSiKWE6KWi3gdsvur0hjlChT3A9Wwu3FoDsQLbXDE zW4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Cdp2EtBFym3wXOJtzcxo7hXulJYdnGAKwekrjG+zUb0=; b=nRjU2Geu5IEvK/p98yx2G1bqDUnG3jglWP/sSwJWMRcVVwSn0Ug6Dv6kO6iX/frvjC HD1iPvtkywW1OAIZaiUQg4I4+lHtNf1sMpQBcn7P1iWzcH9sINOuvTUkNby1wlRI7iPZ tblm+AE7MRA6mOe0VJ1fmW4a9PsleF915XZDtHgra0zhkd+vem2TJnO7U4TkZUGEr1DA cbu4qV+rZeNkhgLu7xbYUAEFb3g5Jny9KJqYUPBxwdb+DzxpUPADuZqeyQXLu7n4eO6M cKdq1ze++8El9JN7b35ipCdCpqZZiXyjIY+XAmr1m8YraR9boQJxj/ax/dfMacTJJqij Tjvw== X-Gm-Message-State: ACrzQf2+3SnDyNbBci3WJru5FIEQqoXbv74o2WwP+PhjrkNRsqlnIoKm GFRz7kvnEChbKt9N0/eRHAs= X-Google-Smtp-Source: AMsMyM7SiBE5gw5JRh8TXYttBoUlkk2cbbidTQZvuulWna9bPnulNpy9pSH+x1TLd4oxmj865c2LQg== X-Received: by 2002:a63:f84c:0:b0:470:f04:5c81 with SMTP id v12-20020a63f84c000000b004700f045c81mr22790213pgj.143.1667823821654; Mon, 07 Nov 2022 04:23:41 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id ci11-20020a17090afc8b00b0020de216d0f7sm4223869pjb.18.2022.11.07.04.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:40 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 13/72] tst_badblocks: Add unit test to verify badblocks list merge api Date: Mon, 7 Nov 2022 17:51:01 +0530 Message-Id: <9442bf260312e78236353ce1976ea7edc519d1cf.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Add unit test to verify badblocks list merge api i.e. ext2fs_badblocks_merge() Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/tst_badblocks.c | 61 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/lib/ext2fs/tst_badblocks.c b/lib/ext2fs/tst_badblocks.c index b6e766ab..946de0ae 100644 --- a/lib/ext2fs/tst_badblocks.c +++ b/lib/ext2fs/tst_badblocks.c @@ -119,6 +119,40 @@ static void print_list(badblocks_list bb, int verify) } } +static void do_list_merge_verify(badblocks_list bb, badblocks_list bbm, int verify) +{ + errcode_t retval; + badblocks_iterate iter; + blk_t blk; + int i, ok; + + retval = ext2fs_badblocks_merge(bb, bbm); + if (retval) { + com_err("do_list_merge_verify", retval, "while doing list merge"); + return; + } + + if (!verify) + return; + + retval = ext2fs_badblocks_list_iterate_begin(bb, &iter); + if (retval) { + com_err("do_list_merge_verify", retval, "while setting up iterator"); + return; + } + + while (ext2fs_badblocks_list_iterate(iter, &blk)) { + retval = ext2fs_badblocks_list_test(bbm, blk); + if (retval == 0) { + printf(" --- NOT OK\n"); + test_fail++; + return; + } + } + ext2fs_badblocks_list_iterate_end(iter); + printf(" --- OK\n"); +} + static void validate_test_seq(badblocks_list bb, blk_t *vec) { int i, match, ok; @@ -275,13 +309,13 @@ out: int main(int argc, char **argv) { - badblocks_list bb1, bb2, bb3, bb4, bb5; + badblocks_list bb1, bb2, bb3, bb4, bb5, bbm; int equal; errcode_t retval; add_error_table(&et_ext2_error_table); - bb1 = bb2 = bb3 = bb4 = bb5 = 0; + bb1 = bb2 = bb3 = bb4 = bb5 = bbm = 0; printf("test1: "); retval = create_test_list(test1, &bb1); @@ -346,6 +380,27 @@ int main(int argc, char **argv) printf("\n"); } + printf("Create merge bb list\n"); + retval = ext2fs_badblocks_list_create(&bbm, 5); + if (retval) { + com_err("ext2fs_badblocks_list_create", retval, "while creating list"); + test_fail++; + } + + printf("Merge & Verify all bb{1..5} into bbm\n"); + if (bb1 && bb2 && bb3 && bb4 && bb5 && bbm) { + printf("Merge bb1 into bbm"); + do_list_merge_verify(bb1, bbm, 1); + printf("Merge bb2 into bbm"); + do_list_merge_verify(bb2, bbm, 1); + printf("Merge bb3 into bbm"); + do_list_merge_verify(bb3, bbm, 1); + printf("Merge bb4 into bbm"); + do_list_merge_verify(bb4, bbm, 1); + printf("Merge bb5 into bbm"); + do_list_merge_verify(bb5, bbm, 1); + } + file_test(bb4); file_test_invalid(bb4); @@ -363,6 +418,8 @@ int main(int argc, char **argv) ext2fs_badblocks_list_free(bb4); if (bb5) ext2fs_badblocks_list_free(bb5); + if (bbm) + ext2fs_badblocks_list_free(bbm); return test_fail; From patchwork Mon Nov 7 12:21:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700561 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jZ2DurGX; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VlN25PBz23lT for ; Mon, 7 Nov 2022 23:24:00 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VlN1YhBz4xG8 for ; Mon, 7 Nov 2022 23:24:00 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VlN1Vqbz4xGT; Mon, 7 Nov 2022 23:24:00 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jZ2DurGX; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VlN1PYTz4xG8 for ; Mon, 7 Nov 2022 23:24:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231995AbiKGMX6 (ORCPT ); Mon, 7 Nov 2022 07:23:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49980 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232161AbiKGMX4 (ORCPT ); Mon, 7 Nov 2022 07:23:56 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35331178A7 for ; Mon, 7 Nov 2022 04:23:48 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id c15-20020a17090a1d0f00b0021365864446so10111945pjd.4 for ; Mon, 07 Nov 2022 04:23:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tiYVQActMABdUDZrhshgyV59Yo6jiV7QqDiI8hnUXwg=; b=jZ2DurGX2lQttWFmXB1HBZL0X/DvauieS2J7nPiOSgOWZwHMkZdNK36c4KEseZ1tLE +fw2upmdljylKN5DuYGPwh3SoCuUPLTWunOOinCph5oSrPgRAZiqkB6tJ1nEtJrcVdSq D9gKs5xthe+cTlsrrUdrkJZyz2hYLZYCEmOCTyhxgDdI8tmsn4Yn+vAdvIDJ2KL45/7L nUh0ldNjp/Lvii5n9p31kj250YjoEVohQb8CLMDcR9tc/vYqn329ZrDqlqdQScQdLoe+ MAUwlJbb00uS/IjW8jlUsgdwKdl+J2VzQ/c//XRFUEKA+YLIbEoJHnBcUkypprsXWOK/ EErQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tiYVQActMABdUDZrhshgyV59Yo6jiV7QqDiI8hnUXwg=; b=SxxIU8V7Ce0V8bXWHSk/d5PKvCqtuBgV56n+5ts/I+kuMBCnTONtMLbI6sDuG6kh9u x7spWTVYgmajyLmrmdlE4y3s/HeBO6+V13lknMevIdMhigPsEYE7Y4ndQEl/B2NHdey9 rScqqVvnlkTpdITlyhT+30dfNDRcWKHkJjsz4JSDTuNuvZ0hCYwqZPR2Z+miC1suHCOC 6EGJ8AK3RxoAtKf+LJZltxqKC0nvjxZ9aMcaSzOeggA4TVgkgp2IJkzv/aaFTpYJkKLP xlTn/YELNzz9g41ECDUvU/1k/Hpr+lvVTuh8CeXFgSUJaPykw+YnT5mJk7CYM9tZngaJ KMAg== X-Gm-Message-State: ANoB5plI5cTeG4ehCd0SUKWBFEimURlZO4kQQ+NoXgAd0AKGHDminh5b ogxf+zllTjtkBNtV4bYjILLxtj1cSbk= X-Google-Smtp-Source: AA0mqf4b8XDCtzmpNuUF5esJP/NE349i4G5rAzXFq7iYqW6PbeSpSxzQI+b9EdLBOtqi9lLyGqBeXQ== X-Received: by 2002:a17:90a:1911:b0:217:3ddc:3d4d with SMTP id 17-20020a17090a191100b002173ddc3d4dmr16806960pjg.10.1667823827638; Mon, 07 Nov 2022 04:23:47 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id i17-20020aa796f1000000b0056da073b2b7sm4374788pfq.210.2022.11.07.04.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:47 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 14/72] tst_bitmaps_standalone: Add copy and merge bitmaps test Date: Mon, 7 Nov 2022 17:51:02 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This adds a basic copy and merge api test for rbtree bitmap type. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/Makefile.in | 25 +++- lib/ext2fs/tst_bitmaps_standalone.c | 170 ++++++++++++++++++++++++++++ 2 files changed, 189 insertions(+), 6 deletions(-) create mode 100644 lib/ext2fs/tst_bitmaps_standalone.c diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in index f6a050a2..1692500e 100644 --- a/lib/ext2fs/Makefile.in +++ b/lib/ext2fs/Makefile.in @@ -227,6 +227,7 @@ SRCS= ext2_err.c \ $(srcdir)/write_bb_file.c \ $(srcdir)/rbtree.c \ $(srcdir)/tst_libext2fs.c \ + $(srcdir)/tst_bitmaps_standalone.c \ $(DEBUG_SRCS) HFILES= bitops.h ext2fs.h ext2_io.h ext2_fs.h ext2_ext_attr.h ext3_extents.h \ @@ -328,9 +329,9 @@ tst_getsectsize: tst_getsectsize.o getsectsize.o $(STATIC_LIBEXT2FS) \ $(ALL_LDFLAGS) $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) \ $(SYSLIBS) -tst_types.o: $(srcdir)/tst_types.c ext2_types.h +tst_types.o: $(srcdir)/tst_types.c ext2_types.h -tst_types: tst_types.o ext2_types.h +tst_types: tst_types.o ext2_types.h $(E) " LD $@" $(Q) $(CC) -o tst_types tst_types.o $(ALL_LDFLAGS) $(SYSLIBS) @@ -362,6 +363,11 @@ tst_sha512: $(srcdir)/sha512.c $(srcdir)/ext2_fs.h $(Q) $(CC) $(ALL_LDFLAGS) $(ALL_CFLAGS) -o tst_sha512 \ $(srcdir)/sha512.c -DUNITTEST $(SYSLIBS) +tst_bitmaps_standalone: tst_bitmaps_standalone.o $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCOM_ERR) + $(E) " LD $@" + $(Q) $(CC) -o tst_bitmaps_standalone tst_bitmaps_standalone.o $(ALL_LDFLAGS) \ + $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) $(SYSLIBS) + ext2_tdbtool: tdbtool.o $(E) " LD $@" $(Q) $(CC) -o ext2_tdbtool tdbtool.o tdb.o $(ALL_LDFLAGS) $(SYSLIBS) @@ -533,7 +539,7 @@ mkjournal: mkjournal.c $(STATIC_LIBEXT2FS) $(DEPLIBCOM_ERR) fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ tst_super_size tst_types tst_inode_size tst_csum tst_crc32c tst_bitmaps \ tst_inline tst_inline_data tst_libext2fs tst_sha256 tst_sha512 \ - tst_digest_encode tst_getsize tst_getsectsize + tst_digest_encode tst_getsize tst_getsectsize tst_bitmaps_standalone $(TESTENV) ./tst_bitops $(TESTENV) ./tst_badblocks $(TESTENV) ./tst_iscan @@ -556,6 +562,7 @@ fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ $(TESTENV) ./tst_bitmaps -l -f $(srcdir)/tst_bitmaps_cmds > tst_bitmaps_out diff $(srcdir)/tst_bitmaps_exp tst_bitmaps_out $(TESTENV) ./tst_digest_encode + $(TESTENV) ./tst_bitmaps_standalone installdirs:: $(E) " MKDIR_P $(libdir) $(includedir)/ext2fs" @@ -581,7 +588,7 @@ install:: all $(HFILES) $(HFILES_IN) installdirs ext2fs.pc uninstall:: $(RM) -f $(DESTDIR)$(libdir)/libext2fs.a \ $(DESTDIR)$(pkgconfigdir)/ext2fs.pc - $(RM) -rf $(DESTDIR)$(includedir)/ext2fs + $(RM) -rf $(DESTDIR)$(includedir)/ext2fs clean:: $(RM) -f \#* *.s *.o *.a *~ *.bak core profiled/* \ @@ -590,7 +597,7 @@ clean:: tst_bitops tst_types tst_icount tst_super_size tst_csum \ tst_bitmaps tst_bitmaps_out tst_extents tst_inline \ tst_inline_data tst_inode_size tst_bitmaps_cmd.c \ - tst_digest_encode tst_sha256 tst_sha512 \ + tst_digest_encode tst_sha256 tst_sha512 tst_bitmaps_standalone \ ext2_tdbtool mkjournal debug_cmds.c tst_cmds.c extent_cmds.c \ ../libext2fs.a ../libext2fs_p.a ../libext2fs_chk.a \ crc32c_table.h gen_crc32ctable tst_crc32c tst_libext2fs \ @@ -646,7 +653,7 @@ windows_io.o: $(srcdir)/windows_io.c $(top_builddir)/lib/config.h \ $(srcdir)/ext2_ext_attr.h $(srcdir)/bitops.h $(srcdir)/ext2fsP.h # +++ Dependency line eater +++ -# +# # Makefile dependencies follow. This must be the last section in # the Makefile.in file # @@ -1156,6 +1163,12 @@ tst_iscan.o: $(srcdir)/tst_iscan.c $(top_builddir)/lib/config.h \ $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h +tst_bitmaps_standalone.o: $(srcdir)/tst_bitmaps_standalone.c $(top_builddir)/lib/config.h \ + $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ + $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ + $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ + $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ + $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h undo_io.o: $(srcdir)/undo_io.c $(top_builddir)/lib/config.h \ $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ diff --git a/lib/ext2fs/tst_bitmaps_standalone.c b/lib/ext2fs/tst_bitmaps_standalone.c new file mode 100644 index 00000000..68b598a8 --- /dev/null +++ b/lib/ext2fs/tst_bitmaps_standalone.c @@ -0,0 +1,170 @@ +#include "config.h" +#include +#include +#include +#if HAVE_UNISTD_H +#include +#endif +#include +#include +#include +#include +#if HAVE_ERRNO_H +#include +#endif + +#include "ext2_fs.h" +#include "ext2fs.h" +#include "bmap64.h" + +ext2_filsys test_fs; +ext2fs_block_bitmap block_map_1; +ext2fs_block_bitmap block_map_2; +ext2fs_block_bitmap block_map; + +static int test_fail = 0; + +void dump_bitmap(ext2fs_generic_bitmap bmap, unsigned int start, unsigned num) +{ + unsigned char *buf; + errcode_t retval; + int i, len = (num - start + 7) / 8; + + buf = malloc(len); + if (!buf) { + com_err("dump_bitmap", 0, "couldn't allocate buffer"); + return; + } + memset(buf, 0, len); + retval = ext2fs_get_generic_bmap_range(bmap, (__u64) start, num, buf); + if (retval) { + com_err("dump_bitmap", retval, + "while calling ext2fs_generic_bmap_range"); + free(buf); + return; + } + for (i=len-1; i >= 0; i--) + printf("%02x ", buf[i]); + printf("\n"); + printf("bits set: %u\n", ext2fs_bitcount(buf, len)); + free(buf); +} + +static void test_copy_run() +{ + int blocks[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 21, 23, 26, 29, 33, 37, 38}; + errcode_t ret; + char *buf_map = NULL; + char *buf_copy_map = NULL; + + assert(ext2fs_allocate_block_bitmap(test_fs, "block bitmap", &block_map_1) == 0); + + for (int i = 0; i < sizeof(blocks)/sizeof(blocks[0]); i++) { + ext2fs_mark_block_bitmap2(block_map_1, blocks[i]); + } + + assert(ext2fs_copy_bitmap(block_map_1, &block_map) == 0); + + if (ext2fs_compare_block_bitmap(block_map_1, block_map) != 0) { + printf("block bitmap copy test failed\n"); + test_fail++; + + dump_bitmap(block_map_1, test_fs->super->s_first_data_block, + test_fs->super->s_blocks_count); + + dump_bitmap(block_map, test_fs->super->s_first_data_block, + test_fs->super->s_blocks_count); + } + + ext2fs_free_block_bitmap(block_map_1); + ext2fs_free_block_bitmap(block_map); +} + +void test_merge_run() +{ + int blocks_odd[] = {1, 3, 5, 7, 9, 21, 23, 29, 33, 37}; + int blocks_even[] = {2, 4, 6, 8, 10, 26, 38}; + ext2fs_generic_bitmap_64 tmp_map; + + assert(ext2fs_allocate_block_bitmap(test_fs, "block bitmap 1", &block_map_1) == 0); + assert(ext2fs_allocate_block_bitmap(test_fs, "block bitmap 2", &block_map_2) == 0); + assert(ext2fs_allocate_block_bitmap(test_fs, "block bitmap 2", &block_map) == 0); + + for (int i = 0; i < sizeof(blocks_odd) / sizeof(blocks_odd[0]); i++) { + ext2fs_mark_block_bitmap2(block_map_1, blocks_odd[i]); + ext2fs_mark_block_bitmap2(block_map, blocks_odd[i]); + } + + for (int i = 0; i < sizeof(blocks_even) / sizeof(blocks_even[0]); i++) { + ext2fs_mark_block_bitmap2(block_map_2, blocks_even[i]); + ext2fs_mark_block_bitmap2(block_map, blocks_even[i]); + } + + assert(ext2fs_merge_bitmap(block_map_2, block_map_1, NULL, NULL) == 0); + if (ext2fs_compare_block_bitmap(block_map_1, block_map) != 0) { + printf("block bitmap merge test failed\n"); + test_fail++; + + dump_bitmap(block_map_1, test_fs->super->s_first_data_block, + test_fs->super->s_blocks_count); + + dump_bitmap(block_map, test_fs->super->s_first_data_block, + test_fs->super->s_blocks_count); + } + + ext2fs_free_block_bitmap(block_map_1); + ext2fs_free_block_bitmap(block_map_2); + ext2fs_free_block_bitmap(block_map); +} + +static void setup_filesystem(const char *name, unsigned int blocks, + unsigned int inodes, unsigned int type, + unsigned int flags) +{ + struct ext2_super_block param; + errcode_t ret; + + memset(¶m, 0, sizeof(param)); + ext2fs_blocks_count_set(¶m, blocks); + param.s_inodes_count = inodes; + + ret = ext2fs_initialize(name, flags, ¶m, test_io_manager, + &test_fs); + if (ret) { + com_err(name, ret, "while initializing filesystem"); + return; + } + + test_fs->default_bitmap_type = type; + + ext2fs_free_block_bitmap(test_fs->block_map); + ext2fs_free_block_bitmap(test_fs->inode_map); + + return; +errout: + ext2fs_close_free(&test_fs); +} + +int main(int argc, char **argv) +{ + unsigned int blocks = 127; + unsigned int inodes = 0; + unsigned int type = EXT2FS_BMAP64_RBTREE; + unsigned int flags = EXT2_FLAG_64BITS; + char *buf = NULL; + + setup_filesystem(argv[0], blocks, inodes, type, flags); + + /* test for EXT2FS_BMAP64_RBTREE */ + test_copy_run(); + test_merge_run(); + + /* TODO: test for EXT2FS_BMAP64_BITARRAY */ + + if (test_fail) + printf("%s: Test copy & merge bitmaps -- NOT OK\n", argv[0]); + else + printf("%s: Test copy & merge bitmaps -- OK\n", argv[0]); + + return test_fail; +} From patchwork Mon Nov 7 12:21:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700562 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BPBcZDLD; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VlP0vnzz23lx for ; Mon, 7 Nov 2022 23:24:01 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VlP0WHGz4xG8 for ; Mon, 7 Nov 2022 23:24:01 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VlP0SQsz4xGT; Mon, 7 Nov 2022 23:24:01 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BPBcZDLD; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VlN5yktz4xG8 for ; Mon, 7 Nov 2022 23:24:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232164AbiKGMYA (ORCPT ); Mon, 7 Nov 2022 07:24:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231373AbiKGMX5 (ORCPT ); Mon, 7 Nov 2022 07:23:57 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD11D1B78D for ; Mon, 7 Nov 2022 04:23:54 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id k15so10456602pfg.2 for ; Mon, 07 Nov 2022 04:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Mn1lRc0cP2jFlAAalzrANSO9b0KSj41EtzHe7T6nSJg=; b=BPBcZDLDBLoJsxLMHKXQlNUlf1aRgCgy0M9pSyoXyEOuy5Qo26tqVAoI+CDz8nYj8T y0/YbQ5wtJzwhbPOVJn3O8f546//pk0Qs6UZAZ8/CvVtUqehG122cpy9F0QluZHdK7Fy hwBrOWRjRXYGrEdF0OERcpt/DDu4JI2TOvTw/Q2qC1ePap9e717ugjMoEAZYM+jKKhWs cBOZMy0z9RJ5iQiHUTyfesEEMmApnXyIKTrznCjDNHtaMsdX0s0HS4COhL9OJMTwVgVg IjEAOGpmFdJKRcib2lJjK7/D1mVpT6lBOvQVvRUVpVgPCU+2fg1ucKB3bSEgVOy5yqyQ /Rtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mn1lRc0cP2jFlAAalzrANSO9b0KSj41EtzHe7T6nSJg=; b=u3BPAWz2RhRTrGn3zIu4V/Tijv6WNQRcSZ3e1LhofmTIDgtvW7cCdolbtXZqcy4Oxz dq0cYCHB7Hh4xn2xSsp9IGrqKxDk8bDjntzsAfGV5h4YBiEu//CT6oVcrLJovnv+jnUJ cx4LLDNClM0IxKNMk1tc40qMEU9/txu0gA8izAa/X+vpe6cUZ4cs0fUjlqNxin5vimKB B5b7PynyenDk64Ryp2Xm712ekpqUalyjo0bcgfbixZN30bq0fMrJKZOHfN05IdrUBbzl LQoC8LoAeeSaIYg99LlwVELDgL39peui9uy0M8Ne9MySUoXC+q6T/VndFndNnHc8xale xgMQ== X-Gm-Message-State: ACrzQf0RuZMnuKKyJhKVJvVXLJQc7bv6NFQeEqeYyzX0/oQCtWPLcMxo NwM2ShpiW8XsJRWOjLzfYpg= X-Google-Smtp-Source: AMsMyM7rUY1YQGqPRnvlSrFd3THVdgoLiBKiHirpWqGqPfDqi2fFEI2Ug/ort5PFbdjez7FLhtbxzw== X-Received: by 2002:a63:6a48:0:b0:43a:18ce:4e08 with SMTP id f69-20020a636a48000000b0043a18ce4e08mr43954917pgc.432.1667823833835; Mon, 07 Nov 2022 04:23:53 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id j5-20020a170903024500b001785a72d285sm4930509plh.48.2022.11.07.04.23.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:53 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 15/72] tst_bitmaps_pthread: Add merge bitmaps test using pthreads Date: Mon, 7 Nov 2022 17:51:03 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This patch adds a test to verify the core bitmaps merge APIs for rbtree bitmap type. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/Makefile.in | 17 ++- lib/ext2fs/tst_bitmaps_pthread.c | 247 +++++++++++++++++++++++++++++++ 2 files changed, 263 insertions(+), 1 deletion(-) create mode 100644 lib/ext2fs/tst_bitmaps_pthread.c diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in index 1692500e..c0694175 100644 --- a/lib/ext2fs/Makefile.in +++ b/lib/ext2fs/Makefile.in @@ -228,6 +228,7 @@ SRCS= ext2_err.c \ $(srcdir)/rbtree.c \ $(srcdir)/tst_libext2fs.c \ $(srcdir)/tst_bitmaps_standalone.c \ + $(srcdir)/tst_bitmaps_pthread.c \ $(DEBUG_SRCS) HFILES= bitops.h ext2fs.h ext2_io.h ext2_fs.h ext2_ext_attr.h ext3_extents.h \ @@ -368,6 +369,11 @@ tst_bitmaps_standalone: tst_bitmaps_standalone.o $(STATIC_LIBEXT2FS) $(DEPSTATIC $(Q) $(CC) -o tst_bitmaps_standalone tst_bitmaps_standalone.o $(ALL_LDFLAGS) \ $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) $(SYSLIBS) +tst_bitmaps_pthread: tst_bitmaps_pthread.o $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCOM_ERR) + $(E) " LD $@" + $(Q) $(CC) -o tst_bitmaps_pthread tst_bitmaps_pthread.o $(ALL_LDFLAGS) \ + $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) $(SYSLIBS) + ext2_tdbtool: tdbtool.o $(E) " LD $@" $(Q) $(CC) -o ext2_tdbtool tdbtool.o tdb.o $(ALL_LDFLAGS) $(SYSLIBS) @@ -539,7 +545,8 @@ mkjournal: mkjournal.c $(STATIC_LIBEXT2FS) $(DEPLIBCOM_ERR) fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ tst_super_size tst_types tst_inode_size tst_csum tst_crc32c tst_bitmaps \ tst_inline tst_inline_data tst_libext2fs tst_sha256 tst_sha512 \ - tst_digest_encode tst_getsize tst_getsectsize tst_bitmaps_standalone + tst_digest_encode tst_getsize tst_getsectsize tst_bitmaps_standalone \ + tst_bitmaps_pthread $(TESTENV) ./tst_bitops $(TESTENV) ./tst_badblocks $(TESTENV) ./tst_iscan @@ -563,6 +570,7 @@ fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ diff $(srcdir)/tst_bitmaps_exp tst_bitmaps_out $(TESTENV) ./tst_digest_encode $(TESTENV) ./tst_bitmaps_standalone + $(TESTENV) ./tst_bitmaps_pthread installdirs:: $(E) " MKDIR_P $(libdir) $(includedir)/ext2fs" @@ -598,6 +606,7 @@ clean:: tst_bitmaps tst_bitmaps_out tst_extents tst_inline \ tst_inline_data tst_inode_size tst_bitmaps_cmd.c \ tst_digest_encode tst_sha256 tst_sha512 tst_bitmaps_standalone \ + tst_bitmaps_pthread \ ext2_tdbtool mkjournal debug_cmds.c tst_cmds.c extent_cmds.c \ ../libext2fs.a ../libext2fs_p.a ../libext2fs_chk.a \ crc32c_table.h gen_crc32ctable tst_crc32c tst_libext2fs \ @@ -1169,6 +1178,12 @@ tst_bitmaps_standalone.o: $(srcdir)/tst_bitmaps_standalone.c $(top_builddir)/lib $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h +tst_bitmaps_pthread.o: $(srcdir)/tst_bitmaps_pthread.c $(top_builddir)/lib/config.h \ + $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ + $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ + $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ + $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ + $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h undo_io.o: $(srcdir)/undo_io.c $(top_builddir)/lib/config.h \ $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ diff --git a/lib/ext2fs/tst_bitmaps_pthread.c b/lib/ext2fs/tst_bitmaps_pthread.c new file mode 100644 index 00000000..243810ca --- /dev/null +++ b/lib/ext2fs/tst_bitmaps_pthread.c @@ -0,0 +1,247 @@ +#include "config.h" +#include +#include +#include +#if HAVE_UNISTD_H +#include +#endif +#include +#include +#include +#include +#if HAVE_ERRNO_H +#include +#endif +#if HAVE_PTHREAD +#include +#endif + +#include "ext2_fs.h" +#include "ext2fs.h" + +/* + * In this test we first setup used_bitmap by setting some random bits. + * This used_bitmap is then scanned in parallel by two threads, each scanning + * upto nr_bits/2 and setting their respective child_bitmap. + * Then once both threads finishes, we merge the child_bitmap_1/2 into + * parent_bitmap which then is used to compare against used_bitmap. + * In the end used_bitmap bits should match with parent_bitmap. + * + * Note we use EXT2FS_BMAP64_BITARRAY always for used_bitmap, this is because + * EXT2FS_BMAP64_RBTREE does not support parallel scan due to rcursor + * optimization. + */ + +int test_fail = 0; +ext2fs_generic_bitmap child_bitmap1, child_bitmap2, parent_bitmap; +ext2fs_generic_bitmap used_bitmap; +pthread_t pthread_infos[2]; + +#define nr_bits 8192 +int nr_threads = 2; +int bitmap_type[1] = {EXT2FS_BMAP64_RBTREE}; + +void dump_bitmap(ext2fs_generic_bitmap bmap, unsigned int start, unsigned num) +{ + unsigned char *buf; + errcode_t retval; + int i, len = (num - start + 7) / 8; + + buf = malloc(len); + if (!buf) { + com_err("dump_bitmap", 0, "couldn't allocate buffer"); + return; + } + memset(buf, 0, len); + retval = ext2fs_get_generic_bmap_range(bmap, (__u64) start, num, buf); + if (retval) { + com_err("dump_bitmap", retval, + "while calling ext2fs_generic_bmap_range"); + free(buf); + return; + } + for (i=len-1; i >= 0; i--) + printf("%02x ", buf[i]); + printf("\n"); + printf("bits set: %u\n", ext2fs_bitcount(buf, len)); + free(buf); +} + +int should_mark_bit() +{ + return rand() % 2 == 0; +} + +void alloc_bitmaps(int type) +{ + errcode_t retval; + + retval = ext2fs_alloc_generic_bmap(NULL, EXT2_ET_MAGIC_GENERIC_BITMAP64, + type, 0, nr_bits, nr_bits, + "child bitmap1", &child_bitmap1); + if (retval) + goto out; + + retval = ext2fs_alloc_generic_bmap(NULL, EXT2_ET_MAGIC_GENERIC_BITMAP64, + type, 0, nr_bits, nr_bits, + "child bitmap2", &child_bitmap2); + if (retval) + goto out; + + retval = ext2fs_alloc_generic_bmap(NULL, EXT2_ET_MAGIC_GENERIC_BITMAP64, + type, 0, nr_bits, nr_bits, + "parent bitmap", &parent_bitmap); + if (retval) + goto out; + + /* + * Note that EXT2FS_BMAP64_RBTREE doesn't support parallel read. + * this is due to a optimization of maintaining a read cursor within + * rbtree bitmap implementation. + */ + retval = ext2fs_alloc_generic_bmap(NULL, EXT2_ET_MAGIC_GENERIC_BITMAP64, + EXT2FS_BMAP64_BITARRAY, 0, nr_bits, nr_bits, + "used bitmap", &used_bitmap); + if (retval) + goto out; + + return; +out: + com_err("alloc_bitmaps", retval, "while allocating bitmaps\n"); + exit(1); +} + +void setup_bitmaps() +{ + int i = 0; + errcode_t retval; + + /* + * Note we cannot setup used_bitmap in parallel w/o locking. + * Hence setting up the used_bitmap (random bits) here before + * starting pthreads. + */ + for (i = 0; i < nr_bits; i++) { + if (should_mark_bit()) + ext2fs_mark_generic_bmap(used_bitmap, i); + } +} + +static void *run_pthread(void *arg) +{ + int i = 0, j = 0, start, end; + ext2fs_generic_bitmap test_bitmap; + errcode_t retval = 0; + pthread_t id = pthread_self(); + + if (pthread_equal(pthread_infos[0], id)) { + start = 0; + end = nr_bits/2; + test_bitmap = child_bitmap1; + } else { + start = nr_bits / 2 + 1;; + end = nr_bits - 1; + test_bitmap = child_bitmap2; + } + + for (i = start; i <= end; i++) { + if (ext2fs_test_generic_bmap(used_bitmap, i)) { + retval = ext2fs_mark_generic_bmap(test_bitmap, i); + if (retval) { + com_err("run_pthread", retval, "while marking child bitmaps %d\n", i); + test_fail++; + pthread_exit(&retval); + } + } + } + return NULL; +} + +void run_pthreads() +{ + errcode_t retval; + void *retp[2]; + int i; + + for (i = 0; i < nr_threads; i++) { + printf("Starting thread (%d)\n", i); + retval = pthread_create(&pthread_infos[i], NULL, &run_pthread, NULL); + if (retval) { + com_err("run_pthreads", retval, "while pthread_create"); + exit(1); + } + } + + for (i = 0; i < nr_threads; i++) { + void *status; + int ret; + retval = pthread_join(pthread_infos[i], &status); + if (retval) { + com_err("run_pthreads", retval, "while joining pthreads"); + exit(1); + + } + ret = status == NULL ? 0 : *(int*)status; + if (ret) { + com_err("run_pthreads", ret, "pthread returned error"); + test_fail++; + } + + printf("Closing thread (%d), ret(%d)\n", i, ret); + } + + assert(ext2fs_merge_bitmap(child_bitmap1, parent_bitmap, NULL, NULL) == 0); + assert(ext2fs_merge_bitmap(child_bitmap2, parent_bitmap, NULL, NULL) == 0); +} + +void test_bitmaps(int type) +{ + errcode_t retval; + retval = ext2fs_compare_generic_bmap(EXT2_ET_NEQ_BLOCK_BITMAP, parent_bitmap, + used_bitmap); + if (retval) { + test_fail++; + printf("Bitmaps compare failed for bitmap type %d err %ld\n", type, retval); + dump_bitmap(parent_bitmap, 0, nr_bits); + dump_bitmap(used_bitmap, 0, nr_bits); + } +} + +void free_bitmaps() +{ + ext2fs_free_generic_bmap(child_bitmap1); + ext2fs_free_generic_bmap(child_bitmap2); + ext2fs_free_generic_bmap(parent_bitmap); + ext2fs_free_generic_bmap(used_bitmap); +} + +int main(int argc, char *argv[]) +{ + int i; + int ret = 0; + +#ifndef HAVE_PTHREAD + printf("No PTHREAD support, exiting...\n"); + return ret; +#endif + + srand(time(0)); + + /* loop to test for bitmap types */ + for (i = 0; i < 1; i++) { + test_fail = 0; + alloc_bitmaps(bitmap_type[i]); + setup_bitmaps(); + run_pthreads(); + test_bitmaps(bitmap_type[i]); + free_bitmaps(); + + if (test_fail) + printf("%s: Test with bitmap (%d) NOT OK!!\n", argv[0], bitmap_type[i]); + else + printf("%s: Test with bitmap (%d) OK!!\n", argv[0], bitmap_type[i]); + ret |= test_fail; + } + + return ret; +} From patchwork Mon Nov 7 12:21:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700563 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=M9uJgPvK; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vlg3QxPz23lT for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlg2yZdz4xZ3 for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vlg2w9wz4xwy; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=M9uJgPvK; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlg1MYjz4xZ3 for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232166AbiKGMYM (ORCPT ); Mon, 7 Nov 2022 07:24:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232168AbiKGMYK (ORCPT ); Mon, 7 Nov 2022 07:24:10 -0500 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A37014D05 for ; Mon, 7 Nov 2022 04:24:01 -0800 (PST) Received: by mail-pg1-x52f.google.com with SMTP id f63so10320329pgc.2 for ; Mon, 07 Nov 2022 04:24:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yBO+k8MBTIYwv4kfPgNFOduT2kfUiX2rHgsZ+dr5Chg=; b=M9uJgPvKATDKnoTVARZLfh9PS41Q8850EGdkB6YMTLsDmVeqi3TML04NkRxbua5jqO P7ksWidPnWx0PFCK4ej1/v7SVdaB3sAmsXZ57H1aO69mbxW6uUxs9NYulHrnD/gh45xR FuhnLxV+z8xLfUUhUU+SXX3tjmYDjVKF+qkWh847+mwh0ajOV0AUc1e6QaJ0wjE+Svz2 Gr5MUN2MBlcmb4thehShPP5H9ton1QXYbzjk8wLOc7me6m8r/MY/N6mNxJR8WKiKOVje j36SN12KeQdnvuJiPi+DqiQbNPsM09KleYpC8ROtwOph80+KmLqBimYZdVXE/hggnP2j jQ4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yBO+k8MBTIYwv4kfPgNFOduT2kfUiX2rHgsZ+dr5Chg=; b=e4Q7+XUcbSLz+y8MA5DQt4PowPorIM1kYj5onKFOXBG4xifM8OKA/G5pfEVL01R1Lv dXmb+vzeWsi1L3vGt3ibdEeXcn3N57hs89ifWJ5M0OL3Fj64AtQLlsRzWa/3kzvaXgQi klWMb8ubAiVrcEZI66G1KfpzvUB+30IndRdwOjbnV4rWap0xWI4Kwmqs8B4/8DAJMYus AMNXT0yKUJcjjh/ZETU1Y6XDaHWve29qH522RgCtNV/+z6unvRwO2R7x5ld9qywdEP0h CaJFFSKByevN7ADCjYw/emUNag9ye/p3X+XMfbXXtIl7JmRAhQUVE87bRBuRwepvBTlK 92eA== X-Gm-Message-State: ACrzQf0HWxi1CQ+xkgSpaxzzkYw+oVPv/qPABji5yWJRM9q5rCbXnIpE aromfrIDYDPCYWP75q5dXm8= X-Google-Smtp-Source: AMsMyM665UF5aeWuxHw1HW76LTFVpGlwkrB48xqt4sLVDEBenDVh8pkFyx2umgtvvEKYR7nAwDD+rg== X-Received: by 2002:a65:6404:0:b0:46f:a711:c481 with SMTP id a4-20020a656404000000b0046fa711c481mr37625397pgv.262.1667823840415; Mon, 07 Nov 2022 04:24:00 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id j11-20020a170903024b00b001708c4ebbaesm4827881plh.309.2022.11.07.04.23.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:23:59 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 16/72] tst_libext2fs_pthread: Add libext2fs merge/clone unit tests Date: Mon, 7 Nov 2022 17:51:04 +0530 Message-Id: <97f434ef290e793ef050cb5348bda7dd955be937.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This adds a unit tests for libext2fs merge/clone apis and uses pthreads to test the functionality correctly. TODO: We can also add EXT2FS_CLONE_BADBLOCKS and EXT2FS_CLONE_DBLIST test as well into it. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/Makefile.in | 17 +- lib/ext2fs/tst_libext2fs_pthread.c | 315 +++++++++++++++++++++++++++++ 2 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 lib/ext2fs/tst_libext2fs_pthread.c diff --git a/lib/ext2fs/Makefile.in b/lib/ext2fs/Makefile.in index c0694175..5fde9900 100644 --- a/lib/ext2fs/Makefile.in +++ b/lib/ext2fs/Makefile.in @@ -229,6 +229,7 @@ SRCS= ext2_err.c \ $(srcdir)/tst_libext2fs.c \ $(srcdir)/tst_bitmaps_standalone.c \ $(srcdir)/tst_bitmaps_pthread.c \ + $(srcdir)/tst_libext2fs_pthread.c \ $(DEBUG_SRCS) HFILES= bitops.h ext2fs.h ext2_io.h ext2_fs.h ext2_ext_attr.h ext3_extents.h \ @@ -374,6 +375,11 @@ tst_bitmaps_pthread: tst_bitmaps_pthread.o $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCO $(Q) $(CC) -o tst_bitmaps_pthread tst_bitmaps_pthread.o $(ALL_LDFLAGS) \ $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) $(SYSLIBS) +tst_libext2fs_pthread: tst_libext2fs_pthread.o $(STATIC_LIBEXT2FS) $(DEPSTATIC_LIBCOM_ERR) + $(E) " LD $@" + $(Q) $(CC) -o tst_libext2fs_pthread tst_libext2fs_pthread.o $(ALL_LDFLAGS) \ + $(STATIC_LIBEXT2FS) $(STATIC_LIBCOM_ERR) $(SYSLIBS) + ext2_tdbtool: tdbtool.o $(E) " LD $@" $(Q) $(CC) -o ext2_tdbtool tdbtool.o tdb.o $(ALL_LDFLAGS) $(SYSLIBS) @@ -546,7 +552,7 @@ fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ tst_super_size tst_types tst_inode_size tst_csum tst_crc32c tst_bitmaps \ tst_inline tst_inline_data tst_libext2fs tst_sha256 tst_sha512 \ tst_digest_encode tst_getsize tst_getsectsize tst_bitmaps_standalone \ - tst_bitmaps_pthread + tst_bitmaps_pthread tst_libext2fs_pthread $(TESTENV) ./tst_bitops $(TESTENV) ./tst_badblocks $(TESTENV) ./tst_iscan @@ -571,6 +577,7 @@ fullcheck check:: tst_bitops tst_badblocks tst_iscan tst_types tst_icount \ $(TESTENV) ./tst_digest_encode $(TESTENV) ./tst_bitmaps_standalone $(TESTENV) ./tst_bitmaps_pthread + $(TESTENV) ./tst_libext2fs_pthread installdirs:: $(E) " MKDIR_P $(libdir) $(includedir)/ext2fs" @@ -606,7 +613,7 @@ clean:: tst_bitmaps tst_bitmaps_out tst_extents tst_inline \ tst_inline_data tst_inode_size tst_bitmaps_cmd.c \ tst_digest_encode tst_sha256 tst_sha512 tst_bitmaps_standalone \ - tst_bitmaps_pthread \ + tst_bitmaps_pthread tst_libext2fs_pthread \ ext2_tdbtool mkjournal debug_cmds.c tst_cmds.c extent_cmds.c \ ../libext2fs.a ../libext2fs_p.a ../libext2fs_chk.a \ crc32c_table.h gen_crc32ctable tst_crc32c tst_libext2fs \ @@ -1184,6 +1191,12 @@ tst_bitmaps_pthread.o: $(srcdir)/tst_bitmaps_pthread.c $(top_builddir)/lib/confi $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h +tst_libext2fs_pthread.o: $(srcdir)/tst_libext2fs_pthread.c $(top_builddir)/lib/config.h \ + $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ + $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ + $(srcdir)/ext2_fs.h $(srcdir)/ext3_extents.h $(top_srcdir)/lib/et/com_err.h \ + $(srcdir)/ext2_io.h $(top_builddir)/lib/ext2fs/ext2_err.h \ + $(srcdir)/ext2_ext_attr.h $(srcdir)/hashmap.h $(srcdir)/bitops.h undo_io.o: $(srcdir)/undo_io.c $(top_builddir)/lib/config.h \ $(top_builddir)/lib/dirpaths.h $(srcdir)/ext2_fs.h \ $(top_builddir)/lib/ext2fs/ext2_types.h $(srcdir)/ext2fs.h \ diff --git a/lib/ext2fs/tst_libext2fs_pthread.c b/lib/ext2fs/tst_libext2fs_pthread.c new file mode 100644 index 00000000..a5bb6fcd --- /dev/null +++ b/lib/ext2fs/tst_libext2fs_pthread.c @@ -0,0 +1,315 @@ +#include "config.h" +#include +#include +#include +#if HAVE_UNISTD_H +#include +#endif +#include +#include +#include +#include +#if HAVE_ERRNO_H +#include +#endif +#if HAVE_PTHREAD +#include +#endif + +#include "ext2_fs.h" +#include "ext2fs.h" + +#ifdef HAVE_PTHREAD +int test_fail = 0; +ext2_filsys testfs; +ext2fs_inode_bitmap inode_used_map; +ext2fs_block_bitmap block_used_map; +ext2_filsys childfs[2]; +pthread_t pthread_infos[2]; + +#define nr_bits 16384 +int nr_threads = 2; + +int should_mark_bit() +{ + return rand() % 2 == 0; +} + +void setupfs() +{ + errcode_t retval; + struct ext2_super_block param; + + initialize_ext2_error_table(); + + memset(¶m, 0, sizeof(param)); + ext2fs_blocks_count_set(¶m, nr_bits); + retval = ext2fs_initialize("test fs", EXT2_FLAG_64BITS, ¶m, + test_io_manager, &testfs); + if (retval) { + com_err("setup", retval, "while initializing filesystem"); + exit(1); + } + + retval = ext2fs_allocate_tables(testfs); + if (retval) { + com_err("setup", retval, "while allocating tables for testfs"); + exit(1); + } +} + +void setup_used_bitmaps() +{ + int saved_type = testfs->default_bitmap_type; + ext2_inode_scan scan; + struct ext2_inode inode; + ext2_ino_t ino; + errcode_t retval; + int i; + + testfs->default_bitmap_type = EXT2FS_BMAP64_BITARRAY; + + /* allocate block and inode used bitmaps */ + retval = ext2fs_allocate_block_bitmap(testfs, "block used map", &block_used_map); + if (retval) + goto out; + + retval = ext2fs_allocate_inode_bitmap(testfs, "inode used map", &inode_used_map); + if (retval) + goto out; + + /* setup block and inode used bitmaps */ + for (i = 1; i < nr_bits; i++) { + /* + * we check for testfs->block_map as well since there could be some + * blocks already set as part of the FS metadata. + */ + if (should_mark_bit() || ext2fs_test_block_bitmap2(testfs->block_map, i)) { + ext2fs_mark_block_bitmap2(block_used_map, i); + } + } + + retval = ext2fs_open_inode_scan(testfs, 8, &scan); + if (retval) { + com_err("setup_inode_map", retval, "while open inode scan"); + exit(1); + } + + retval = ext2fs_get_next_inode(scan, &ino, &inode); + if (retval) { + com_err("setup_inode_map", retval, "while getting next inode"); + exit(1); + } + + while (ino) { + if (should_mark_bit()) + ext2fs_mark_inode_bitmap2(inode_used_map, ino); + + retval = ext2fs_get_next_inode(scan, &ino, &inode); + if (retval) { + com_err("setup_inode_map", retval, "while getting next inode"); + exit(1); + } + } + ext2fs_close_inode_scan(scan); + + testfs->default_bitmap_type = saved_type; + return; +out: + com_err("setup_used_bitmaps", retval, "while setting up bitmaps\n"); + exit(1); +} + +void setup_childfs() +{ + errcode_t retval; + int i; + + for (i = 0; i < nr_threads; i++) { + retval = ext2fs_clone_fs(testfs, &childfs[i], EXT2FS_CLONE_INODE | EXT2FS_CLONE_BLOCK); + if (retval) { + com_err("setup_childfs", retval, "while clone testfs for childfs"); + exit(1); + } + + retval = childfs[i]->io->manager->open(childfs[i]->device_name, + IO_FLAG_THREADS | IO_FLAG_NOCACHE, &childfs[i]->io); + if (retval) { + com_err("setup_pthread", retval, "while opening childfs"); + exit(1); + } + assert(childfs[i]->parent == testfs); + } +} + +static errcode_t scan_callback(ext2_filsys fs, + ext2_inode_scan scan EXT2FS_ATTR((unused)), + dgrp_t group, void *priv_data) +{ + pthread_t id = *((pthread_t *)priv_data); + + printf("%s: Called for group %d via thread %d\n", __func__, group, + pthread_equal(pthread_infos[1], id)); + if (pthread_equal(pthread_infos[0], id)) { + if (group >= fs->group_desc_count / 2 - 1) + return 1; + } + return 0; +} + +static void *run_pthread(void *arg) +{ + errcode_t retval = 0; + int i = 0, start, end; + ext2fs_block_bitmap test_block_bitmap; + ext2fs_inode_bitmap test_inode_bitmap; + ext2_inode_scan scan; + struct ext2_inode inode; + ext2_ino_t ino; + pthread_t id = pthread_self(); + + if (pthread_equal(pthread_infos[0], id)) { + start = 1; + end = nr_bits/2; + test_block_bitmap = childfs[0]->block_map; + test_inode_bitmap = childfs[0]->inode_map; + + retval = ext2fs_open_inode_scan(childfs[0], 8, &scan); + if (retval) { + com_err("setup_inode_map", retval, "while open inode scan"); + exit(1); + } + + } else { + start = nr_bits / 2 + 1;; + end = nr_bits - 1; + test_block_bitmap = childfs[1]->block_map; + test_inode_bitmap = childfs[1]->inode_map; + + retval = ext2fs_open_inode_scan(childfs[1], 8, &scan); + if (retval) { + com_err("setup_inode_map", retval, "while open inode scan"); + exit(1); + } + ext2fs_inode_scan_goto_blockgroup(scan, testfs->group_desc_count/2); + } + + ext2fs_set_inode_callback(scan, scan_callback, &id); + + /* blocks scan */ + for (i = start; i <= end; i++) { + if (ext2fs_test_block_bitmap2(block_used_map, i)) { + ext2fs_mark_block_bitmap2(test_block_bitmap, i); + } + } + + /* inodes scan */ + retval = ext2fs_get_next_inode(scan, &ino, &inode); + if (retval) { + com_err("setup_inode_map", retval, "while getting next inode"); + exit(1); + } + + while (ino) { + if (ext2fs_test_inode_bitmap2(inode_used_map, ino)) { + ext2fs_mark_inode_bitmap2(test_inode_bitmap, ino); + } + + retval = ext2fs_get_next_inode(scan, &ino, &inode); + if (retval) + break; + } + ext2fs_close_inode_scan(scan); + return NULL; +} + +void run_pthreads() +{ + errcode_t retval; + int i; + + for (i = 0; i < nr_threads; i++) { + printf("Starting thread (%d)\n", i); + retval = pthread_create(&pthread_infos[i], NULL, &run_pthread, NULL); + if (retval) { + com_err("run_pthreads", retval, "while pthread_create"); + exit(1); + } + } + + for (i = 0; i < nr_threads; i++) { + void *status; + int ret; + retval = pthread_join(pthread_infos[i], &status); + if (retval) { + com_err("run_pthreads", retval, "while joining pthreads"); + exit(1); + + } + ret = status == NULL ? 0 : *(int*)status; + if (ret) { + com_err("run_pthreads", ret, "pthread returned error"); + test_fail++; + } + + printf("Closing thread (%d), ret(%d)\n", i, ret); + } + + assert(ext2fs_merge_fs(&childfs[0]) == 0); + assert(ext2fs_merge_fs(&childfs[1]) == 0); +} + +void test_bitmaps() +{ + errcode_t retval; + retval = ext2fs_compare_block_bitmap(testfs->block_map, block_used_map); + if (retval) { + printf("Block bitmap compare -- NOT OK!! (%ld)\n", retval); + test_fail++; + } + + printf("Block compare bitmap -- OK!!\n"); + retval = ext2fs_compare_inode_bitmap(testfs->inode_map, inode_used_map); + if (retval) { + printf("Inode bitmap compare -- NOT OK!! (%ld)\n", retval); + test_fail++; + } + printf("Inode compare bitmap -- OK!!\n"); +} + +void free_used_bitmaps() +{ + ext2fs_free_block_bitmap(block_used_map); + ext2fs_free_inode_bitmap(inode_used_map); +} + +#endif + +int main(int argc, char *argv[]) +{ + int i; + +#ifndef HAVE_PTHREAD + printf("No PTHREAD support, exiting...\n"); + return 0; +#else + + srand(time(0)); + + setupfs(); + setup_used_bitmaps(); + + setup_childfs(); + run_pthreads(); + test_bitmaps(i); + + if (test_fail) + printf("%s: Test libext2fs clone/merge with pthreads NOT OK!!\n", argv[0]); + else + printf("%s: Test libext2fs clone/merge with pthreads OK!!\n", argv[0]); + free_used_bitmaps(); + ext2fs_free(testfs); + + return test_fail; +#endif +} From patchwork Mon Nov 7 12:21:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700564 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=azffTayY; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vlg55YJz23lx for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlg4dDYz4xZ3 for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vlg4b3Bz4xwy; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=azffTayY; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlg4Vb0z4xZ3 for ; Mon, 7 Nov 2022 23:24:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232171AbiKGMYO (ORCPT ); Mon, 7 Nov 2022 07:24:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232174AbiKGMYK (ORCPT ); Mon, 7 Nov 2022 07:24:10 -0500 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BC7E1AD9C for ; Mon, 7 Nov 2022 04:24:07 -0800 (PST) Received: by mail-pg1-x533.google.com with SMTP id v3so10308774pgh.4 for ; Mon, 07 Nov 2022 04:24:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cMMzTstL0mOBrRKHUSn6nbcgtNZIGsnpHy2V81lrAT0=; b=azffTayYehRK4m66YqKjjkG9YPDNTT/hTpl6ajfD/tNP4GT1wWNy5BBhIhm8IEhPIf eJCbq1Lk8BUgsZEdP55RxhiXZPrXogNekXG9popw44LAwFhSxXZOlvwp6kNh+sbikpoY IlFkm4JkqAOk0JHeNF4Bj8YWPH0Iu3a9tmCUl68gILZloOZSuK9cFFS2kGOeH3CM4LDs 1qo9Wlp50xVpYNU1fVzE4LQ/nKEWqt/3UlZBFz36v/jkWIZDCgDFfwexo7nMExsGDFRU CgqLuLqoQzIizOV7uP47/TWh1rV41k9DzYpAzuFDg8iJNkakgT4dAEftUauiKs3lH1No emoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cMMzTstL0mOBrRKHUSn6nbcgtNZIGsnpHy2V81lrAT0=; b=oURjR/g0QAD/PH10iS5jLbQKtijYcBci+dGj3eaVGX9doa/soZTij8BvvkmClVXanc Qxh7E+Kmbi9XwRc3gEGqDYnode2eltMx6Taq+LgNhxWO6ymCUf23CI3RSs4alN7bnxLc vBmXymyy3Yrd+g2QHZJ3BsLLv1DO9+iK40CejXbA893FFyllhTDpYPGHQPqq7sKn3rLT 9DaTBQla5aGFGiTNubvd5dbTcJfWSPnf+arLO+3f1aK5Ql4PGgCR0aSF3NwoRjHAzFIO F7LMFFehKsnCmmWh7SyaYUCmmthn2iLCykdY5oU4n5gh8vj4DPAodfxmMpyryy9tz2Ae v5IQ== X-Gm-Message-State: ACrzQf12Vo8rSuT8BkZC+D0syqAofWJbEb3V0XWiPYNt7SJ4IPxAXXfB sexRtsNe4R6LJ9g+1fUBcOI= X-Google-Smtp-Source: AMsMyM4JzDscFprK/IIqtU/7byc++IuGcQBdOnzNhlCxjYWGCre4TQPz3tEgg/xS6qc6W9penG+HRw== X-Received: by 2002:a05:6a00:248c:b0:56d:b13b:e672 with SMTP id c12-20020a056a00248c00b0056db13be672mr35312925pfv.47.1667823846543; Mon, 07 Nov 2022 04:24:06 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id l15-20020a170903244f00b001869d71228bsm4884357pls.170.2022.11.07.04.24.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:05 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 17/72] libext2fs: Add support for ext2fs_test_block_bitmap_range2_valid() Date: Mon, 7 Nov 2022 17:51:05 +0530 Message-Id: <9015303c19c1b3474d880409db60627b0a9de37f.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong This adds the support in libext2fs to query whether the block range is valid or not (within range) given the block bitmap. Also to avoid duplicate warning messages in case of invalid blocks. This will be later used in pass1 of e2fsck is_blocks_used() function to check whether the given block range is valid or not to avoid duplicate warning resulting from ext2fs_test_block_bitmap_range2() Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/bitops.h | 2 ++ lib/ext2fs/gen_bitmap64.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/lib/ext2fs/bitops.h b/lib/ext2fs/bitops.h index 505b3c9c..1facc8dd 100644 --- a/lib/ext2fs/bitops.h +++ b/lib/ext2fs/bitops.h @@ -120,6 +120,8 @@ extern int ext2fs_test_block_bitmap_range(ext2fs_block_bitmap bitmap, extern void ext2fs_set_bitmap_padding(ext2fs_generic_bitmap map); extern __u32 ext2fs_get_generic_bitmap_start(ext2fs_generic_bitmap bitmap); extern __u32 ext2fs_get_generic_bitmap_end(ext2fs_generic_bitmap bitmap); +extern int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap, + blk64_t block, unsigned int num); /* 64-bit versions */ diff --git a/lib/ext2fs/gen_bitmap64.c b/lib/ext2fs/gen_bitmap64.c index c31f942f..a9637cb5 100644 --- a/lib/ext2fs/gen_bitmap64.c +++ b/lib/ext2fs/gen_bitmap64.c @@ -731,6 +731,39 @@ int ext2fs_test_block_bitmap_range2(ext2fs_block_bitmap gen_bmap, return bmap->bitmap_ops->test_clear_bmap_extent(bmap, block, num); } +int ext2fs_test_block_bitmap_range2_valid(ext2fs_block_bitmap bitmap, + blk64_t block, unsigned int num) +{ + ext2fs_generic_bitmap_64 bmap = (ext2fs_generic_bitmap_64)bitmap; + __u64 end = block + num; + + if (!bmap) + return 0; + + if (EXT2FS_IS_32_BITMAP(bmap)) { + if ((block & ~0xffffffffULL) || + ((block+num-1) & ~0xffffffffULL)) { + return 0; + } + } + + if (!EXT2FS_IS_64_BITMAP(bmap)) + return 0; + + /* convert to clusters if necessary */ + block >>= bmap->cluster_bits; + end += (1 << bmap->cluster_bits) - 1; + end >>= bmap->cluster_bits; + num = end - block; + + if ((block < bmap->start) || (block > bmap->end) || + (block+num-1 > bmap->end)) + return 0; + + return 1; +} + + void ext2fs_mark_block_bitmap_range2(ext2fs_block_bitmap gen_bmap, blk64_t block, unsigned int num) { From patchwork Mon Nov 7 12:21:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700565 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=L9zXQk7s; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vlk3tlhz23lT for ; Mon, 7 Nov 2022 23:24:18 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlk3P3Gz4xG8 for ; Mon, 7 Nov 2022 23:24:18 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vlk3Jgvz4xZ3; Mon, 7 Nov 2022 23:24:18 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=L9zXQk7s; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlk3D57z4xG8 for ; Mon, 7 Nov 2022 23:24:18 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232177AbiKGMYR (ORCPT ); Mon, 7 Nov 2022 07:24:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232169AbiKGMYN (ORCPT ); Mon, 7 Nov 2022 07:24:13 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 428806247 for ; Mon, 7 Nov 2022 04:24:13 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id k22so10448240pfd.3 for ; Mon, 07 Nov 2022 04:24:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/wJhWS2aGvLD1Rzy6m9shGLETN6RBjRGDzQAAEc0ebw=; b=L9zXQk7s1hKr7oC1W6uzkSPg+65h4hWVhusIgbbV3utkHI7OGzUUibHE2tm6yZjxm6 K1q/fsDBZxA+xUQxIIGrEFJ/ULylIVupMqvrqkX6seOQ8CQ5G0lAODPThekk7giV+ult kjs0cCcUNpimNfL1pEH1iYNNKHqHS52zeSMm5d50wuxT0zG8JwJLLVb9Dobi9byBmxh3 ovhOHqoiB3FEcAD7MiZNvyx/UebBP+0Ar11OD3krGfRyjJNGGNaedZ/aCNtuXk26C/vT FdSLzXz1c4GJio0iQdtM3cuIkahC+0poJZJ+rWw0xB1GR68LDbDTcrkbjHSFBuOkYuUR +zqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/wJhWS2aGvLD1Rzy6m9shGLETN6RBjRGDzQAAEc0ebw=; b=CAeJzfgBxUfhd7zJu0Me87he4/C9GcGhXUXY68+3GddBFW45qLpMlQUcIypGraw7PB o8pQwtovnFF/EdpbNcObuR7S1QtjS6KLEEUYOlwsH+hBBqr3Bn2215KmoV2oUPy90TBN f0csjjijG/d8hDokt0L12oAB+F8aZmoCX1lJ5cNLKDVC/IqcLZzpQbsacAKbODvhoZCX 8rOfc0IO7wMvmCRiDmvt2PERMLSMbgYlnwkAxZ65/n4C9R9mYTGVu06RFkeEUYDqtQ1J 3rGcjVBW2H7alIWvbk0Uk2TRdC+bMtD+C/Td3G5GkACAbORdi/cu4NBoKUsjz/xyx7d2 RyFw== X-Gm-Message-State: ACrzQf1tx4iTUsnKmKQn6ExYBAyN824NMLOcFTB042Qw23emBaOaJaHR nW+uaYyMmqz/PzUirxbTucQ= X-Google-Smtp-Source: AMsMyM6Gyd/1MaafB2gAV1PG0XNj6aF/w4NBWJ7iJ/3C/ea7Jyd/uo774MPJuKwj2rluqYmT2Wvwyg== X-Received: by 2002:a65:4d07:0:b0:46f:ed91:a50d with SMTP id i7-20020a654d07000000b0046fed91a50dmr27395440pgt.343.1667823852789; Mon, 07 Nov 2022 04:24:12 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id q66-20020a17090a17c800b0021282014066sm6103510pja.9.2022.11.07.04.24.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:12 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 18/72] libext2fs: Add support to get average group count Date: Mon, 7 Nov 2022 17:51:06 +0530 Message-Id: <7e4f563719aee1970dd1058ca45b0609ae4c7c5f.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong number of threads in pfsck should not exceed flex bg numbers. This patch adds the support in libext2fs to calculate ext2fs_get_avg_group() which returns an average group count which each thread has to scan. fs->fs_num_threads will be set by the client, in this case e2fsck. No. of threads will be passed along with -m option while running e2fsck. That will also set fs->fs_num_threads, which will help in controlling the amount of memory consumed to maintain in memory data structures (per thread) in case of multiple parallel threads (pfsck) to avoid oom. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/ext2fs.h | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index b1505f95..6b4926ce 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -279,10 +279,11 @@ struct struct_ext2_filsys { int cluster_ratio_bits; __u16 default_bitmap_type; __u16 pad; + __u32 fs_num_threads; /* * Reserved for future expansion */ - __u32 reserved[5]; + __u32 reserved[4]; /* * Reserved for the use of the calling application. @@ -2231,6 +2232,35 @@ ext2fs_orphan_block_tail(ext2_filsys fs, char *buf) sizeof(struct ext4_orphan_block_tail)); } +static dgrp_t ext2fs_get_avg_group(ext2_filsys fs) +{ +#ifdef HAVE_PTHREAD + dgrp_t average_group; + unsigned flexbg_size; + + if (fs->fs_num_threads <= 1) + return fs->group_desc_count; + + average_group = fs->group_desc_count / fs->fs_num_threads; + if (average_group <= 1) + return 1; + + if (ext2fs_has_feature_flex_bg(fs->super)) { + int times = 1; + + flexbg_size = 1 << fs->super->s_log_groups_per_flex; + if (average_group % flexbg_size) { + times = average_group / flexbg_size; + average_group = times * flexbg_size; + } + } + + return average_group; +#else + return fs->group_desc_count; +#endif +} + #undef _INLINE_ #endif From patchwork Mon Nov 7 12:21:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700566 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jegXKvA1; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vlw16Pqz23lT for ; Mon, 7 Nov 2022 23:24:28 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlw0h2tz4xx0 for ; Mon, 7 Nov 2022 23:24:28 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vlw0dDQz4xx3; Mon, 7 Nov 2022 23:24:28 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jegXKvA1; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vlw0WJ5z4xx0 for ; Mon, 7 Nov 2022 23:24:28 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232197AbiKGMYX (ORCPT ); Mon, 7 Nov 2022 07:24:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232169AbiKGMYT (ORCPT ); Mon, 7 Nov 2022 07:24:19 -0500 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 236DF63C2 for ; Mon, 7 Nov 2022 04:24:19 -0800 (PST) Received: by mail-pg1-x52f.google.com with SMTP id 64so10309340pgc.5 for ; Mon, 07 Nov 2022 04:24:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UZ3j7EdKBWVYJUuDDkl+rN6c4DTW2MiR6s2rRB1jtXM=; b=jegXKvA1zOPiYEuuHil8v1JcZvcIoj9uWWdoNHpCuybmAhyzrHMestZ8w/URCdCQdi YEj1RhFO9MDqvR73HLYMRMXQ5GrfTFPbhQojS1tHfCZYkMz6XSvfmhehJ1lmM1Noz1YV OsiqCuCg6sOrm7kWS0+Dd5L0TOwPRfGEdC1p9xhTqeruUl5w/9f2eGU27FpgZEXWK5h8 VjWYoAJ+nZchesaPzImiBb7IcYxunFScMNVAY8hdY/jijsQxJZXVRR9Kxakcw5NK9Ut8 Qhk/uRhdYxA33j3CR6WyvMqKy9si3NilUGLtQNyNB1kSbiuNhAWgSuKHqZw+44lXbqx7 8gdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UZ3j7EdKBWVYJUuDDkl+rN6c4DTW2MiR6s2rRB1jtXM=; b=dwDlN63ArbDNP760TOsdchwAjow+4Y64kzOLryzhG9LyUFNRDFhxu9XtXfaOoBJW86 7MmSqy7ORz+nJWP2T6pKRe2CPxNI0nTrfymqJ8XUR4QkQK2XxGz7/OHcP4OtE/cYUaUp bJMjl7GowyRoRIBXGc76UJ2NHEJi1VNS44wuBAta0HWiqqWuXesZdDaGdlBH2dMguGPI TNGaEsuOvR/HRXrvMR5Z0MJmDzVAh8QcLL1GaCTPYargyhXxNej7uaXlhAL1eyvSXmMH TR1G+0DilhwukImSuqFsH2jQYPE0ZeUVvhgSU3iPHrxO7Lue9mh+plC2CEDHGhG6LTyq vnIg== X-Gm-Message-State: ACrzQf3d6so81KGFVqZu1rfzeCMF4uPtBLlDL4E8PQGw3RZ+FBn6LMrY qa6Lqtn6yTASChIv9PaOIs4= X-Google-Smtp-Source: AMsMyM6sqRdScPwmvX7UxBes8LBeI0whAdz6owYLSSxdaVH+K4dE3EBwYNxJ/wjtRzO9koys38oLTw== X-Received: by 2002:a65:4bc3:0:b0:439:103b:fc35 with SMTP id p3-20020a654bc3000000b00439103bfc35mr42555258pgr.248.1667823858666; Mon, 07 Nov 2022 04:24:18 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id t10-20020a63224a000000b00460d89df1f1sm4079017pgm.57.2022.11.07.04.24.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:18 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Andreas Dilger , Ritesh Harjani Subject: [RFCv1 19/72] libext2fs: Misc fixes for struct_ext2_filsys Date: Mon, 7 Nov 2022 17:51:07 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Andreas Dilger Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field to avoid consuming one of the few remaining __u32 reserved fields. Signed-off-by: Andreas Dilger Signed-off-by: Ritesh Harjani (IBM) --- lib/ext2fs/ext2fs.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h index 6b4926ce..950ab042 100644 --- a/lib/ext2fs/ext2fs.h +++ b/lib/ext2fs/ext2fs.h @@ -278,12 +278,11 @@ struct struct_ext2_filsys { time_t now; int cluster_ratio_bits; __u16 default_bitmap_type; - __u16 pad; - __u32 fs_num_threads; + __u16 fs_num_threads; /* * Reserved for future expansion */ - __u32 reserved[4]; + __u32 reserved[5]; /* * Reserved for the use of the calling application. From patchwork Mon Nov 7 12:21:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700567 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=qaICZGkG; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vm76lSyz23lT for ; Mon, 7 Nov 2022 23:24:39 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vm76Lk6z4xG8 for ; Mon, 7 Nov 2022 23:24:39 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vm76JJrz4xGT; Mon, 7 Nov 2022 23:24:39 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=qaICZGkG; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vm76DJ9z4xG8 for ; Mon, 7 Nov 2022 23:24:39 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232183AbiKGMYi (ORCPT ); Mon, 7 Nov 2022 07:24:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232202AbiKGMY0 (ORCPT ); Mon, 7 Nov 2022 07:24:26 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85E826405 for ; Mon, 7 Nov 2022 04:24:25 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id m6so10485950pfb.0 for ; Mon, 07 Nov 2022 04:24:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FZkG8P7csv70+FWzJ0hhPsFK2Mt9vIPNTt8Dycvfo50=; b=qaICZGkGuVj5BkgAM3pzst4KYS/Dl+SCU+jpAseiLBv2P/c/0ne73badfeEF8JALBY 77u79sBcJN0A+QSq6fBIpgUk4tTYNNp//Yjoq3iVDqIPU2pde+ba7g+6VwY0A8yDPHce GZs4eMUinXZV6Adn+I8oWM5XcF+QhtAYZkRTrArbjP287e3KiTWaQ8l0+udMk/E4/a+Z a1tRebCNSUs/7O7NuyClSogxV26N0Js2wESHdS05QQEtYPRpsYrmslzWndFJDxGDJOQe gR47zQmaeBB5NXXOSrGhAPZh5jhox5ECRnRneRMYFR5/0mjdTPvNNqXNqdaJaHaTihZM xm3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FZkG8P7csv70+FWzJ0hhPsFK2Mt9vIPNTt8Dycvfo50=; b=n2tgowRVVOQsNjYeq/gRrlDmuR7lxr/nfjzY2Wz3evKVAahjWe0JDpWqy+1b2MPZ7B dHQaF6beoShXfBV2YL5O6doUUgpigN+b6RGGEFhUcm4IrQGdDx1Pdx15M479bGxUCgAI G/9IhDkOLKTf3N1KWlU2xkPWFZQm8q7o3HwahZYU3joQindJP4731WW2xWxc/K7/JY1r jHfNi5WZ6phieXbVAZyqe2byGm64E5fLuuvtUwSTgd51Ro/DsXNkQQuvWAEH0mVK+SXl s3x3U6oveLn9c8udxuvoSu4eLU+3LlcsNxp9t7YeEn7Muz+4gY6nV2DJrFkLEYJBAhAv SoUA== X-Gm-Message-State: ACrzQf0vZ+VRF81V02MFUW2Kd1kUyaQrcoWN2A7lKK+mZh9DVhSO7gtF 5od0/BkgtSUBTpEgcKMVcxU5wW6pifM= X-Google-Smtp-Source: AMsMyM6LP/Yynqsmv/IVMrdYDfqLHqcoIXJPUFF/sN2JoOHtOZymEZ7L2xeH4HzvDnplmRzzbgJn+g== X-Received: by 2002:a05:6a00:a15:b0:56e:9b2b:60dd with SMTP id p21-20020a056a000a1500b0056e9b2b60ddmr15952814pfh.35.1667823864951; Mon, 07 Nov 2022 04:24:24 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id a72-20020a621a4b000000b0056bbeaa82b9sm4348525pfa.113.2022.11.07.04.24.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:24 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 20/72] libext2fs: avoid too much memory allocation in case fs_num_threads Date: Mon, 7 Nov 2022 17:51:08 +0530 Message-Id: <5ae4498b906ea4adffcca5546e2c9deba39dd05a.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong e2fsck init memory according to filesystem inodes/dir numbers recorded in the superblock, this should be aware of filesystem number of threads, otherwise, oom can happen. So in case of fs->fs_num_threads, this patch controls the amount of memory consumed for running multiple threads in e2fsck. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- lib/ext2fs/dblist.c | 2 ++ lib/ext2fs/icount.c | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/lib/ext2fs/dblist.c b/lib/ext2fs/dblist.c index 5568b8ec..c19e17bc 100644 --- a/lib/ext2fs/dblist.c +++ b/lib/ext2fs/dblist.c @@ -58,6 +58,8 @@ static errcode_t make_dblist(ext2_filsys fs, ext2_ino_t size, if (retval) goto cleanup; dblist->size = (num_dirs * 2) + 12; + if (fs->fs_num_threads) + dblist->size /= fs->fs_num_threads; } len = (size_t) sizeof(struct ext2_db_entry2) * dblist->size; dblist->count = count; diff --git a/lib/ext2fs/icount.c b/lib/ext2fs/icount.c index 766eccca..48665c7e 100644 --- a/lib/ext2fs/icount.c +++ b/lib/ext2fs/icount.c @@ -237,6 +237,8 @@ errcode_t ext2fs_create_icount_tdb(ext2_filsys fs EXT2FS_NO_TDB_UNUSED, * value. */ num_inodes = fs->super->s_inodes_count - fs->super->s_free_inodes_count; + if (fs->fs_num_threads) + num_inodes /= fs->fs_num_threads; icount->tdb = tdb_open(fn, num_inodes, TDB_NOLOCK | TDB_NOSYNC, O_RDWR | O_CREAT | O_TRUNC, 0600); @@ -288,6 +290,8 @@ errcode_t ext2fs_create_icount2(ext2_filsys fs, int flags, unsigned int size, if (retval) goto errout; icount->size += fs->super->s_inodes_count / 50; + if (fs->fs_num_threads) + icount->size /= fs->fs_num_threads; } bytes = (size_t) (icount->size * sizeof(struct ext2_icount_el)); From patchwork Mon Nov 7 12:21:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700568 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=S1RA6xkU; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vm850B8z23lT for ; Mon, 7 Nov 2022 23:24:40 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vm84Wkpz4xG8 for ; Mon, 7 Nov 2022 23:24:40 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vm84TnCz4xGT; Mon, 7 Nov 2022 23:24:40 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=S1RA6xkU; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vm82vGMz4xG8 for ; Mon, 7 Nov 2022 23:24:40 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232201AbiKGMYj (ORCPT ); Mon, 7 Nov 2022 07:24:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232179AbiKGMYe (ORCPT ); Mon, 7 Nov 2022 07:24:34 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8082E1B9D0 for ; Mon, 7 Nov 2022 04:24:31 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id io19so10896478plb.8 for ; Mon, 07 Nov 2022 04:24:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CfUADwEhy+K71Db3G/GggakC2dWpzFOKZulNS3cjYSw=; b=S1RA6xkUdWL02xB65iASd324smkrjde7/wOIjf1MCIExys3qB1lFHJ5NjNcMWZoG66 +coNBA0zzksK34jlRYKLBlwLejYhKer6bgHNhsYFkYegbQZ71Hq3oprLlTZh0pkG0Aki Q1hFOr5SI1EYDhAPJsH2x1pvHHO0DZHpfQ+AWnAalkLylfQ3+mSKHqD6ZwM1vycpGFHE smkuLCXmbHivKUNkYiUlw/LDtOGXT6XhnZXMmN4p1etDtGn3q2xUwuAh60k7yeCyj4kg cKmI+f6K/2zS4hERr7er12CNIjDo3VQyFSFAO+T6WBbzNazibh/iHjOAe5fSQhE48G86 s4Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CfUADwEhy+K71Db3G/GggakC2dWpzFOKZulNS3cjYSw=; b=zXeJOdnh84ZNCLhqXHubtx/vKfVDz7cV17YMNRIIY4Ac2aE6l2JLClghpCVcU6e6q7 zEeXz2oIs9IOG3Ot9qBgx2qxXqqH1wcA4QPDhssHBCTF6E8LgYff+DgpNPgZrwZX5Uwd htSTlZa0lBV2MjTPSTta+bRG3lYP0niKz52/XI1AIFxttm+WPSs1Y8PQrtkJwBBButJF itEUvRjuvwzwNomANrlrWTnBP1sZggJa8/GBoIT4uAATQjJqVK8lMUD2+7OuC+iesMHL +j2QCoNGq0Pju6nhOcp6omSgSDHpZIVfjIURY7uSHXKQH8CynQTGOjkvd8hnP2Kt0AQy F0IQ== X-Gm-Message-State: ANoB5pmyBPjkSV2ccCWztVFxF9rDRuslcnuwvE9GyMiYldEzrODf4t6G 2fmhtryrLqSrZbM2rR2bqxI= X-Google-Smtp-Source: AA0mqf7A8b1uq1V6tYI+0mK7z3DskE74WRWoJrOPEy1HU4udxl1i70iMkWwQ66fliOOblgMqT6fEVw== X-Received: by 2002:a17:902:9042:b0:188:6fc9:1da with SMTP id w2-20020a170902904200b001886fc901damr13826968plz.162.1667823870910; Mon, 07 Nov 2022 04:24:30 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id b6-20020a170903228600b00186b0ac12c5sm4870140plh.172.2022.11.07.04.24.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:30 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 21/72] e2fsck: add -m option for multithread Date: Mon, 7 Nov 2022 17:51:09 +0530 Message-Id: <935d3652155338e793325acd5f8b900728a56bd9.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi -m option is added but no actual functionality is added. This patch only adds the logic that when -m is specified, one of -p/-y/-n options should be specified. And when -m is specified, -C shouldn't be specified and the completion progress report won't be triggered by sending SIGUSR1/SIGUSR2 signals. This simplifies the implementation of multi-thread fsck in the future. Completion progress support with multi-thread fsck will be added back after multi-thread fsck implementation is finished. Right now, disable it to simplify the implementation of multi-thread fsck. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Andreas Dilger --- e2fsck/e2fsck.h | 1 + e2fsck/unix.c | 31 ++++++++++++++++++++----- tests/f_multithread/expect.1 | 23 ++++++++++++++++++ tests/f_multithread/expect.2 | 7 ++++++ tests/f_multithread/image.gz | 1 + tests/f_multithread/name | 1 + tests/f_multithread/script | 4 ++++ tests/f_multithread_completion/expect.1 | 2 ++ tests/f_multithread_completion/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_completion/image.gz | 1 + tests/f_multithread_completion/name | 1 + tests/f_multithread_completion/script | 4 ++++ tests/f_multithread_no/expect.1 | 24 +++++++++++++++++++ tests/f_multithread_no/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_no/image.gz | 1 + tests/f_multithread_no/name | 1 + tests/f_multithread_no/script | 4 ++++ tests/f_multithread_preen/expect.1 | 11 +++++++++ tests/f_multithread_preen/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_preen/image.gz | 1 + tests/f_multithread_preen/name | 1 + tests/f_multithread_preen/script | 4 ++++ tests/f_multithread_yes/expect.1 | 2 ++ tests/f_multithread_yes/expect.2 | 23 ++++++++++++++++++ tests/f_multithread_yes/image.gz | 1 + tests/f_multithread_yes/name | 1 + tests/f_multithread_yes/script | 4 ++++ 27 files changed, 217 insertions(+), 6 deletions(-) create mode 100644 tests/f_multithread/expect.1 create mode 100644 tests/f_multithread/expect.2 create mode 120000 tests/f_multithread/image.gz create mode 100644 tests/f_multithread/name create mode 100644 tests/f_multithread/script create mode 100644 tests/f_multithread_completion/expect.1 create mode 100644 tests/f_multithread_completion/expect.2 create mode 120000 tests/f_multithread_completion/image.gz create mode 100644 tests/f_multithread_completion/name create mode 100644 tests/f_multithread_completion/script create mode 100644 tests/f_multithread_no/expect.1 create mode 100644 tests/f_multithread_no/expect.2 create mode 120000 tests/f_multithread_no/image.gz create mode 100644 tests/f_multithread_no/name create mode 100644 tests/f_multithread_no/script create mode 100644 tests/f_multithread_preen/expect.1 create mode 100644 tests/f_multithread_preen/expect.2 create mode 120000 tests/f_multithread_preen/image.gz create mode 100644 tests/f_multithread_preen/name create mode 100644 tests/f_multithread_preen/script create mode 100644 tests/f_multithread_yes/expect.1 create mode 100644 tests/f_multithread_yes/expect.2 create mode 120000 tests/f_multithread_yes/image.gz create mode 100644 tests/f_multithread_yes/name create mode 100644 tests/f_multithread_yes/script diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 252a17db..35428717 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -186,6 +186,7 @@ struct resource_track { #define E2F_OPT_UNSHARE_BLOCKS 0x40000 #define E2F_OPT_CLEAR_UNINIT 0x80000 /* Hack to clear the uninit bit */ #define E2F_OPT_CHECK_ENCODING 0x100000 /* Force verification of encoded filenames */ +#define E2F_OPT_MULTITHREAD 0x200000 /* Use multiple threads to speedup */ /* * E2fsck flags diff --git a/e2fsck/unix.c b/e2fsck/unix.c index e5b672a2..1ee27f6a 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -76,13 +76,14 @@ int journal_enable_debug = -1; static void usage(e2fsck_t ctx) { fprintf(stderr, - _("Usage: %s [-panyrcdfktvDFV] [-b superblock] [-B blocksize]\n" + _("Usage: %s [-pamnyrcdfktvDFV] [-b superblock] [-B blocksize]\n" "\t\t[-l|-L bad_blocks_file] [-C fd] [-j external_journal]\n" "\t\t[-E extended-options] [-z undo_file] device\n"), ctx->program_name ? ctx->program_name : "e2fsck"); fprintf(stderr, "%s", _("\nEmergency help:\n" " -p Automatic repair (no questions)\n" + " -m multiple threads to speedup fsck\n" " -n Make no changes to the filesystem\n" " -y Assume \"yes\" to all questions\n" " -c Check for bad blocks and add them to the badblock list\n" @@ -854,7 +855,8 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) phys_mem_kb = get_memory_size() / 1024; ctx->readahead_kb = ~0ULL; - while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) + + while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) switch (c) { case 'C': ctx->progress = e2fsck_update_progress; @@ -895,6 +897,9 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) } ctx->options |= E2F_OPT_PREEN; break; + case 'm': + ctx->options |= E2F_OPT_MULTITHREAD; + break; case 'n': if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN)) goto conflict_opt; @@ -1014,6 +1019,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("The -n and -l/-L options are incompatible.")); fatal_error(ctx, 0); } + if (ctx->options & E2F_OPT_MULTITHREAD) { + if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) { + com_err(ctx->program_name, 0, "%s", + _("The -m option should be used together with one of -p/-y/-n options.")); + fatal_error(ctx, 0); + } + if (ctx->progress) { + com_err(ctx->program_name, 0, "%s", + _("Only one of the options -C or -m may be specified.")); + fatal_error(ctx, 0); + } + } if (ctx->options & E2F_OPT_NO) ctx->options |= E2F_OPT_READONLY; @@ -1120,10 +1137,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) #ifdef SA_RESTART sa.sa_flags = SA_RESTART; #endif - sa.sa_handler = signal_progress_on; - sigaction(SIGUSR1, &sa, 0); - sa.sa_handler = signal_progress_off; - sigaction(SIGUSR2, &sa, 0); + if ((ctx->options & E2F_OPT_MULTITHREAD) == 0) { + sa.sa_handler = signal_progress_on; + sigaction(SIGUSR1, &sa, 0); + sa.sa_handler = signal_progress_off; + sigaction(SIGUSR2, &sa, 0); + } #endif /* Update our PATH to include /sbin if we need to run badblocks */ diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread/expect.1 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread/expect.2 b/tests/f_multithread/expect.2 new file mode 100644 index 00000000..a833aefc --- /dev/null +++ b/tests/f_multithread/expect.2 @@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 0 diff --git a/tests/f_multithread/image.gz b/tests/f_multithread/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread/name b/tests/f_multithread/name new file mode 100644 index 00000000..df838ea6 --- /dev/null +++ b/tests/f_multithread/name @@ -0,0 +1 @@ +test "e2fsck -m" option \ No newline at end of file diff --git a/tests/f_multithread/script b/tests/f_multithread/script new file mode 100644 index 00000000..0fe96cd0 --- /dev/null +++ b/tests/f_multithread/script @@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_completion/expect.1 b/tests/f_multithread_completion/expect.1 new file mode 100644 index 00000000..61cac9bb --- /dev/null +++ b/tests/f_multithread_completion/expect.1 @@ -0,0 +1,2 @@ +../e2fsck/e2fsck: Only one of the options -C or -m may be specified. +Exit status is 8 diff --git a/tests/f_multithread_completion/expect.2 b/tests/f_multithread_completion/expect.2 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread_completion/expect.2 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread_completion/image.gz b/tests/f_multithread_completion/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread_completion/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread_completion/name b/tests/f_multithread_completion/name new file mode 100644 index 00000000..a959045d --- /dev/null +++ b/tests/f_multithread_completion/name @@ -0,0 +1 @@ +test "e2fsck -m" option conflicts with "-C" \ No newline at end of file diff --git a/tests/f_multithread_completion/script b/tests/f_multithread_completion/script new file mode 100644 index 00000000..bf23cd61 --- /dev/null +++ b/tests/f_multithread_completion/script @@ -0,0 +1,4 @@ +FSCK_OPT="-fy -m -C 1" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1 new file mode 100644 index 00000000..d14c4083 --- /dev/null +++ b/tests/f_multithread_no/expect.1 @@ -0,0 +1,24 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? no + +Free blocks count wrong (11602, counted=11597). +Fix? no + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? no + +Free inodes count wrong (2997, counted=2992). +Fix? no + + +test_filesys: ********** WARNING: Filesystem still has errors ********** + +test_filesys: 11/3008 files (0.0% non-contiguous), 398/12000 blocks +Exit status is 4 diff --git a/tests/f_multithread_no/expect.2 b/tests/f_multithread_no/expect.2 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread_no/expect.2 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread_no/image.gz b/tests/f_multithread_no/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread_no/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread_no/name b/tests/f_multithread_no/name new file mode 100644 index 00000000..fa49692e --- /dev/null +++ b/tests/f_multithread_no/name @@ -0,0 +1 @@ +test "e2fsck -m" option works with "-n" \ No newline at end of file diff --git a/tests/f_multithread_no/script b/tests/f_multithread_no/script new file mode 100644 index 00000000..b93deb3a --- /dev/null +++ b/tests/f_multithread_no/script @@ -0,0 +1,4 @@ +FSCK_OPT="-fn -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_preen/expect.1 b/tests/f_multithread_preen/expect.1 new file mode 100644 index 00000000..b4b0cd9a --- /dev/null +++ b/tests/f_multithread_preen/expect.1 @@ -0,0 +1,11 @@ +../e2fsck/e2fsck: Bad magic number in super-block while trying to open test.img +test_filesys: +The superblock could not be read or does not describe a valid ext2/ext3/ext4 +filesystem. If the device is valid and it really contains an ext2/ext3/ext4 +filesystem (and not swap or ufs or something else), then the superblock +is corrupt, and you might try running e2fsck with an alternate superblock: + e2fsck -b 8193 + or + e2fsck -b 32768 + +Exit status is 8 diff --git a/tests/f_multithread_preen/expect.2 b/tests/f_multithread_preen/expect.2 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread_preen/expect.2 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread_preen/image.gz b/tests/f_multithread_preen/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread_preen/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread_preen/name b/tests/f_multithread_preen/name new file mode 100644 index 00000000..90d199df --- /dev/null +++ b/tests/f_multithread_preen/name @@ -0,0 +1 @@ +test "e2fsck -m" option works with "-p" \ No newline at end of file diff --git a/tests/f_multithread_preen/script b/tests/f_multithread_preen/script new file mode 100644 index 00000000..ecb79cd6 --- /dev/null +++ b/tests/f_multithread_preen/script @@ -0,0 +1,4 @@ +FSCK_OPT="-fp -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_yes/expect.1 b/tests/f_multithread_yes/expect.1 new file mode 100644 index 00000000..8b780ecf --- /dev/null +++ b/tests/f_multithread_yes/expect.1 @@ -0,0 +1,2 @@ +../e2fsck/e2fsck: The -m option should be used together with one of -p/-y/-n options. +Exit status is 8 diff --git a/tests/f_multithread_yes/expect.2 b/tests/f_multithread_yes/expect.2 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread_yes/expect.2 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread_yes/image.gz b/tests/f_multithread_yes/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread_yes/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread_yes/name b/tests/f_multithread_yes/name new file mode 100644 index 00000000..3a703195 --- /dev/null +++ b/tests/f_multithread_yes/name @@ -0,0 +1 @@ +test "e2fsck -m" option works with "-y" \ No newline at end of file diff --git a/tests/f_multithread_yes/script b/tests/f_multithread_yes/script new file mode 100644 index 00000000..38891f6a --- /dev/null +++ b/tests/f_multithread_yes/script @@ -0,0 +1,4 @@ +FSCK_OPT="-f -m" +SECOND_FSCK_OPT=-yf + +. $cmd_dir/run_e2fsck From patchwork Mon Nov 7 12:21:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700569 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=aT/c3BpX; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VmP3Kqfz23lT for ; Mon, 7 Nov 2022 23:24:53 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VmP2rsxz4xGT for ; Mon, 7 Nov 2022 23:24:53 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VmP2mkWz4xZ3; Mon, 7 Nov 2022 23:24:53 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=aT/c3BpX; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VmP2cq6z4xGT for ; Mon, 7 Nov 2022 23:24:53 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232169AbiKGMYw (ORCPT ); Mon, 7 Nov 2022 07:24:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232174AbiKGMYi (ORCPT ); Mon, 7 Nov 2022 07:24:38 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39B681A817 for ; Mon, 7 Nov 2022 04:24:37 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id c15-20020a17090a1d0f00b0021365864446so10113965pjd.4 for ; Mon, 07 Nov 2022 04:24:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=75ylSXVoMs4uFH9hMkzh6UY1bNXGiCN2KHMFQvCqWvg=; b=aT/c3BpX3UY369B1qIAR6AeljKxd3qlPqTp3QEAADAag+pJqGuLXUdIRU/Q2UD4fYi yXRY9SDrGdAOVsZXjMgJpONXpnaijOOd6YZ6BggKSFWgEoq5ri+RxjOldNhBv+HMoWvv AZ3N+Gr5x0qQuSKCj9TMxbMdwaTfvyaTnGK3VcHTmcCBn5meV0MuBtcAMuqa+jDlcuvJ n4LPD/OYzXTxpe1FVjavYu8YrjE40T63m3+qlspktcS2w72BzpVRl9VJhgkOE1Qo+3me BBnEtvnSPP82lwZKr2tk7og3E047ubs6fpc+vGWi3aFWmrVWlLKba3CcqF+J4nRertfz H48A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=75ylSXVoMs4uFH9hMkzh6UY1bNXGiCN2KHMFQvCqWvg=; b=zqVQmm3SvSsRrQoQKi1tpTSw0S0SrjhhSikbF7nQmH4oziU5EAANwBkMoxc8oDDknB NTiXrPV/nI6ud2zX98PMdREfyrPq7wJ3QsA813iGKeS7xh7NZcr99LXuEv8qnapK+/Hp z6M4zdkw7KEfsSVmLgHXcGLJ6auXksm6B+RvdzIwMAClzCK+ffIqBi9rp/jEh1IGN61L D0svlahAL5B5PvqKrGX2jRuKPnQRGd31jACkNf9NHkPVjwqM5HhdOXUWEXhK7MErTuu1 fzmi1XbE1ga4RdKJLvld9tCcaqANnEHByn/eWjc0Mcvn8iOTcl5zlrC0zx51Ts6Hie5d 4wkA== X-Gm-Message-State: ACrzQf0HdWPG+ZpRruRZ0oQ6T54mAKvjtXMDwu7idTVxIhrGRwc6L5Jh q9dK6PuIULSMng2pkmQnPbQ= X-Google-Smtp-Source: AMsMyM4x+79ZcF3td5CkK6gg9mXRgv/wqJ8mezQNpndQs12plg2mHJzqwVWoPUAJFyy3FFikJ3QfDQ== X-Received: by 2002:a17:90b:1a84:b0:213:e8b5:2d16 with SMTP id ng4-20020a17090b1a8400b00213e8b52d16mr41332943pjb.9.1667823876783; Mon, 07 Nov 2022 04:24:36 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y19-20020a170902ed5300b001870dc3b4c0sm4871106plb.74.2022.11.07.04.24.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:36 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 22/72] e2fsck: copy context when using multi-thread fsck Date: Mon, 7 Nov 2022 17:51:10 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi This patch only copy the context to a new one when -m is enabled. It doesn't actually start any thread. When pass1 test finishes, the new context is copied back to the original context. Since the signal handler only changes the original context, so add global_ctx in "struct e2fsck_struct" and use that to check whether there is any signal of canceling. This patch handles the long jump properly so that all the existing tests can be passed even the context has been copied. Otherwise, test f_expisize_ea_del would fail when aborting. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 2 + e2fsck/pass1.c | 117 ++++++++++++++++++++++++++++++++++++++++++++---- e2fsck/unix.c | 1 + 3 files changed, 112 insertions(+), 8 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 35428717..f1259728 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -262,6 +262,8 @@ struct e2fsck_fc_replay_state { }; struct e2fsck_struct { + /* Global context to get the cancel flag */ + e2fsck_t global_ctx; ext2_filsys fs; const char *program_name; char *filesystem_name; diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 73909c39..972265b8 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1151,7 +1151,22 @@ static int quota_inum_is_reserved(ext2_filsys fs, ext2_ino_t ino) return 0; } -void e2fsck_pass1(e2fsck_t ctx) +static int e2fsck_should_abort(e2fsck_t ctx) +{ + e2fsck_t global_ctx; + + if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + return 1; + + if (ctx->global_ctx) { + global_ctx = ctx->global_ctx; + if (global_ctx->flags & E2F_FLAG_SIGNAL_MASK) + return 1; + } + return 0; +} + +void e2fsck_pass1_thread(e2fsck_t ctx) { int i; __u64 max_sizes; @@ -1381,7 +1396,7 @@ void e2fsck_pass1(e2fsck_t ctx) if (ino > ino_threshold) pass1_readahead(ctx, &ra_group, &ino_threshold); ehandler_operation(old_op); - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) goto endit; if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) { /* @@ -2008,7 +2023,7 @@ void e2fsck_pass1(e2fsck_t ctx) if (process_inode_count >= ctx->process_inode_size) { process_inodes(ctx, block_buf); - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) goto endit; } } @@ -2134,6 +2149,92 @@ endit: else ctx->invalid_bitmaps++; } + +static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx) +{ + errcode_t retval; + e2fsck_t thread_context; + + retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context); + if (retval) { + com_err(global_ctx->program_name, retval, "while allocating memory"); + return retval; + } + memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct)); + thread_context->fs->priv_data = thread_context; + thread_context->global_ctx = global_ctx; + + *thread_ctx = thread_context; + return 0; +} + +static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + int flags = global_ctx->flags; +#ifdef HAVE_SETJMP_H + jmp_buf old_jmp; + + memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf)); +#endif + memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct)); +#ifdef HAVE_SETJMP_H + memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf)); +#endif + /* Keep the global singal flags*/ + global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | + (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); + + global_ctx->fs->priv_data = global_ctx; + ext2fs_free_mem(&thread_ctx); + return 0; +} + +void e2fsck_pass1_multithread(e2fsck_t ctx) +{ + errcode_t retval; + e2fsck_t thread_ctx; + + retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx); + if (retval) { + com_err(ctx->program_name, 0, + _("while preparing pass1 thread\n")); + ctx->flags |= E2F_FLAG_ABORT; + return; + } + +#ifdef HAVE_SETJMP_H + /* + * When fatal_error() happens, jump to here. The thread + * context's flags will be saved, but its abort_loc will + * be overwritten by original jump buffer for the later + * tests. + */ + if (setjmp(thread_ctx->abort_loc)) { + thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK; + e2fsck_pass1_thread_join(ctx, thread_ctx); + return; + } + thread_ctx->flags |= E2F_FLAG_SETJMP_OK; +#endif + + e2fsck_pass1_thread(thread_ctx); + retval = e2fsck_pass1_thread_join(ctx, thread_ctx); + if (retval) { + com_err(ctx->program_name, 0, + _("while joining pass1 thread\n")); + ctx->flags |= E2F_FLAG_ABORT; + return; + } +} + +void e2fsck_pass1(e2fsck_t ctx) +{ + if (ctx->options & E2F_OPT_MULTITHREAD) + e2fsck_pass1_multithread(ctx); + else + e2fsck_pass1_thread(ctx); +} + #undef FINISH_INODE_LOOP /* @@ -2196,7 +2297,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf) ehandler_operation(buf); check_blocks(ctx, &pctx, block_buf, &inodes_to_process[i].ea_ibody_quota); - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) break; } ctx->stashed_inode = old_stashed_inode; @@ -3424,7 +3525,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, inlinedata_fs = ext2fs_has_feature_inline_data(ctx->fs->super); if (check_ext_attr(ctx, pctx, block_buf, &ea_block_quota)) { - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) goto out; pb.num_blocks += EXT2FS_B2C(ctx->fs, ea_block_quota.blocks); } @@ -3479,7 +3580,7 @@ static void check_blocks(e2fsck_t ctx, struct problem_context *pctx, } end_problem_latch(ctx, PR_LATCH_BLOCK); end_problem_latch(ctx, PR_LATCH_TOOBIG); - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) goto out; if (pctx->errcode) fix_problem(ctx, PR_1_BLOCK_ITERATE, pctx); @@ -3961,7 +4062,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr = 0; return BLOCK_CHANGED; } - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) return BLOCK_ABORT; } else mark_block_used(ctx, blk); @@ -4058,7 +4159,7 @@ static int process_bad_block(ext2_filsys fs, *block_nr = 0; return BLOCK_CHANGED; } - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) return BLOCK_ABORT; return 0; } diff --git a/e2fsck/unix.c b/e2fsck/unix.c index 1ee27f6a..a77041b0 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -1461,6 +1461,7 @@ int main (int argc, char *argv[]) } reserve_stdio_fds(); + ctx->global_ctx = NULL; set_up_logging(ctx); if (ctx->logf) { int i; From patchwork Mon Nov 7 12:21:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700570 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ix8FHeEm; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VmX75MGz23lT for ; Mon, 7 Nov 2022 23:25:00 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VmX6fSMz4xG8 for ; Mon, 7 Nov 2022 23:25:00 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VmX6bt7z4xGT; Mon, 7 Nov 2022 23:25:00 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ix8FHeEm; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VmX6Xzgz4xG8 for ; Mon, 7 Nov 2022 23:25:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232214AbiKGMZA (ORCPT ); Mon, 7 Nov 2022 07:25:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232212AbiKGMYr (ORCPT ); Mon, 7 Nov 2022 07:24:47 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A32031B7AE for ; Mon, 7 Nov 2022 04:24:46 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id u6so10881021plq.12 for ; Mon, 07 Nov 2022 04:24:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0NuUxgcrcbyLEt+4D5wU+GVX4W+ksOY5RfbdK0CPaRk=; b=Ix8FHeEmnUoyT5Wiff1YiUOBbkH2ITNogZrk4cgWYfoLnR2RPo2nENzFDS71+kFc2y Qg86ALET1edqF9ZHIy0SYi8cAyqIpVpBoyrIgI47lxS7SgC0/F5UEPOC7aTqA0IfysMb ao1ehlxwodN6taZx0fMxwxb9UXoKKyYGXzevtazfJd1kLVwBkoziV+XQN6JZFJnGVW5c BdS7dcGoBNnL+NQoM9idcmHEgEJeVC0eC02RHGAX4hLlks1G2mTJ9lki7FgGyi5rFyU9 WykcCFX3+1Klwote+MFeQ3zfRK8u7EDemrZkO/Ge0DEO07z05lgeUDc2cAlEBnQ3tNSv CVWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0NuUxgcrcbyLEt+4D5wU+GVX4W+ksOY5RfbdK0CPaRk=; b=A/qnvWryOhNsHDyXReNOl9dSTjPnGhhdhuZpvoBqxZKTkbH35GZNFWVH4l21OB9ukR Q3tLMquo2snh+tamHC7ey2EjrvXjIB5MuLAze2GONCc1eaB5Zmh35E3s4W9OSexPX6Dq HEoLeY4YZoR00NdG+uTZd33v0MDowiRzSIpPQMoPuZGKm1k8SRbkWuZJ6bL+j+M2nvm8 0ftC9WrJU8KVUYL9nIuTgE3vOTJqPM+OmdQ8nagupa+dO/KeAz7ZOu8dqLOCF8zmFMP4 hRx7OAz7oJ66CtXSgRfFK/MbCQ8/cDV23XZZtQXpgiwYiNePhZ/40WMmq8ZZLsJxAkOt Ce3g== X-Gm-Message-State: ACrzQf3n5Ww1HFdf9+VrZgchBC66cAp9fNFHuur/1+3F/pziy/kETepR 5zMVK73l1NcL3jl39NCw068= X-Google-Smtp-Source: AMsMyM7Xc45YVUVrIbKj33wGGxzZPTVQQLYRkmDpUeF4MD46jIq2xNQeqqQBRg4Rtwr9kvXcQCnoTA== X-Received: by 2002:a17:90a:d80a:b0:213:1442:24be with SMTP id a10-20020a17090ad80a00b00213144224bemr751410pjv.15.1667823886170; Mon, 07 Nov 2022 04:24:46 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id l5-20020a622505000000b0056be1581126sm4525205pfl.143.2022.11.07.04.24.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:45 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 23/72] e2fsck: create logs for multi-threads Date: Mon, 7 Nov 2022 17:51:11 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi When multi-threads are used, different logs should be created for different threads. Each thread has log files with suffix of ".$THREAD_INDEX". And this patch adds f_multithread_logfile test case. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 6 ++++-- e2fsck/logfile.c | 10 ++++++++- e2fsck/pass1.c | 12 +++++++++++ tests/f_multithread_logfile/expect.1 | 23 ++++++++++++++++++++ tests/f_multithread_logfile/image.gz | 1 + tests/f_multithread_logfile/name | 1 + tests/f_multithread_logfile/script | 32 ++++++++++++++++++++++++++++ 7 files changed, 82 insertions(+), 3 deletions(-) create mode 100644 tests/f_multithread_logfile/expect.1 create mode 120000 tests/f_multithread_logfile/image.gz create mode 100644 tests/f_multithread_logfile/name create mode 100644 tests/f_multithread_logfile/script diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index f1259728..284bc52e 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -342,8 +342,10 @@ struct e2fsck_struct { /* * For pass1_check_directory and pass1_get_blocks */ - ext2_ino_t stashed_ino; - struct ext2_inode *stashed_inode; + ext2_ino_t stashed_ino; + struct ext2_inode *stashed_inode; + /* Thread index, if global_ctx is null, this field is unused */ + int thread_index; /* * Location of the lost and found directory diff --git a/e2fsck/logfile.c b/e2fsck/logfile.c index 9d79eed2..83dbceff 100644 --- a/e2fsck/logfile.c +++ b/e2fsck/logfile.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "e2fsck.h" #include @@ -294,6 +295,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) struct string s, s1, s2; char *s0 = 0, *log_dir = 0, *log_fn = 0; int log_dir_wait = 0; + int string_size; + char string_index[10]; s.s = s1.s = s2.s = 0; @@ -310,6 +313,12 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) goto out; expand_logfn(ctx, log_fn, &s); + if (ctx->global_ctx) { + sprintf(string_index, "%d", ctx->thread_index); + append_string(&s, ".", 1); + append_string(&s, string_index, 0); + } + if ((log_fn[0] == '/') || !log_dir || !log_dir[0]) s0 = s.s; @@ -328,7 +337,6 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) append_string(&s2, log_dir, 0); append_string(&s2, "/", 1); append_string(&s2, s.s, 0); - printf("%s\n", s2.s); } if (s0) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 972265b8..2d4e62ca 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2164,6 +2164,8 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre thread_context->fs->priv_data = thread_context; thread_context->global_ctx = global_ctx; + thread_context->thread_index = 0; + set_up_logging(thread_context); *thread_ctx = thread_context; return 0; } @@ -2171,6 +2173,8 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) { int flags = global_ctx->flags; + FILE *global_logf = global_ctx->logf; + FILE *global_problem_logf = global_ctx->problem_logf; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2185,6 +2189,14 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); global_ctx->fs->priv_data = global_ctx; + global_ctx->logf = global_logf; + global_ctx->problem_logf = global_problem_logf; + if (thread_ctx->logf) + fclose(thread_ctx->logf); + if (thread_ctx->problem_logf) { + fputs("\n", thread_ctx->problem_logf); + fclose(thread_ctx->problem_logf); + } ext2fs_free_mem(&thread_ctx); return 0; } diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1 new file mode 100644 index 00000000..e2b954d0 --- /dev/null +++ b/tests/f_multithread_logfile/expect.1 @@ -0,0 +1,23 @@ +ext2fs_open2: Bad magic number in super-block +../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +Free blocks count wrong for group #0 (7987, counted=7982). +Fix? yes + +Free blocks count wrong (11602, counted=11597). +Fix? yes + +Free inodes count wrong for group #0 (1493, counted=1488). +Fix? yes + +Free inodes count wrong (2997, counted=2992). +Fix? yes + + +test_filesys: ***** FILE SYSTEM WAS MODIFIED ***** +test_filesys: 16/3008 files (0.0% non-contiguous), 403/12000 blocks +Exit status is 1 diff --git a/tests/f_multithread_logfile/image.gz b/tests/f_multithread_logfile/image.gz new file mode 120000 index 00000000..0fd40018 --- /dev/null +++ b/tests/f_multithread_logfile/image.gz @@ -0,0 +1 @@ +../f_zero_super/image.gz \ No newline at end of file diff --git a/tests/f_multithread_logfile/name b/tests/f_multithread_logfile/name new file mode 100644 index 00000000..faaabc3b --- /dev/null +++ b/tests/f_multithread_logfile/name @@ -0,0 +1 @@ +test "e2fsck -m" option works with "-E log_filename=" diff --git a/tests/f_multithread_logfile/script b/tests/f_multithread_logfile/script new file mode 100644 index 00000000..4f9ca6f8 --- /dev/null +++ b/tests/f_multithread_logfile/script @@ -0,0 +1,32 @@ +LOG_FNAME="f_multithread_logfile_xxx" +FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME" +SKIP_VERIFY="true" +ONE_PASS_ONLY="true" +SKIP_CLEANUP="true" + +rm -f $LOG_FNAME.* $LOG_FNAME + +. $cmd_dir/run_e2fsck + +rm -f $test_name.ok $test_name.failed +cmp -s $OUT1 $EXP1 +status1=$? + +if [ "$status1" -eq 0 ]; then + if [ ! -f $LOG_FNAME -o ! -f $LOG_FNAME.0 ]; then + echo "$LOG_FNAME or $LOG_FNAME.0 is not created" > $test_name.failed + echo "$test_name: $test_description: failed" + else + echo "$test_name: $test_description: ok" + touch $test_name.ok + fi +else + diff $DIFF_OPTS $test_dir/expect.1 \ + $test_name.1.log >> $test_name.failed + echo "$test_name: $test_description: failed" +fi + +unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2 +unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD +unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO +unset LOG_FINAME From patchwork Mon Nov 7 12:21:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700571 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=PpRIw9C9; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vmh513fz1yqS for ; Mon, 7 Nov 2022 23:25:08 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vmh4Wr6z4xGT for ; Mon, 7 Nov 2022 23:25:08 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vmh4T59z4xGj; Mon, 7 Nov 2022 23:25:08 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=PpRIw9C9; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vmh4N8gz4xGT for ; Mon, 7 Nov 2022 23:25:08 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231643AbiKGMZG (ORCPT ); Mon, 7 Nov 2022 07:25:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232172AbiKGMYx (ORCPT ); Mon, 7 Nov 2022 07:24:53 -0500 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B292F140F6 for ; Mon, 7 Nov 2022 04:24:52 -0800 (PST) Received: by mail-pl1-x630.google.com with SMTP id 4so10952969pli.0 for ; Mon, 07 Nov 2022 04:24:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KMUAlTv0xFqkr4yBd3hOL6zg1L92K891HA8xH93YHYc=; b=PpRIw9C97by405j/e3btrx6VItpdNXtv2DAZPfa/VaRb8qcjyb30blV4S7KIyXxzV6 t0qvnH3OteQkerC2MWG17nT7GC9cXU49uZwdjgRJKAYJt47EiKuSC28f1pv7nDy63rro m9gPFhHcWtGxxnpvbLEHENdepJPvRDJPTr2ddtayaBZZk3YQnbzHMD7jy+dIqnlYV1d4 rFMhK96O8TiwrVaBbQR43g1jjvHvXrYypenR1/0gEJDDfZe0ufDoniUuMnK2tOuvR93U C9So8L0g3X+t2y+XFzyFsYzEfiFK9DtxR2JX9iCUHCILg5sUSTS7F6mk1xByg/lJq6XG AAcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KMUAlTv0xFqkr4yBd3hOL6zg1L92K891HA8xH93YHYc=; b=diw+Bp6njgQQs7o/ReWMLT4AJwIKnVFeIvFzw9iLQGCD5n/IjGHP5nK8hC5+3x3tQ0 hQhpqhPT6Ox17hQSu/Wb2zSe2yEYBMbBJdGuvtkKnW26JmbyzlzalOnZ89JvXnrcg4cD eszs1Lo8QA40pQK3BMl5/UFYezJ+FlqqJakLI8amedsax//x9n0cLlF15xSDeQ9f5/N5 wpiL/JxWb8hp/s2DyaEUaRqje4VPdEKoHpGCcA0lgz8cgMMAklGmq+ijM/cKlZYvexdl PvDw+3HFzWeUOs2mupc4mIfe3NSyjmR3lg/d6YRpkX6vOaDesX8pwMuV1RleDOTe3JkM cK/g== X-Gm-Message-State: ACrzQf3aQiPdxeXk0wjxqRr8KZyINBzyScVFsuks6jQi1iKFVc+hJW+/ djhdu9+8o1f8ewiUeXYfjdc= X-Google-Smtp-Source: AMsMyM5qrnMtyEVlyQajWbXzA6J6jDqMInTe7IBqb4yK3Lt3v2T5szpZ9L34KeWlYLONM/MKj+QRng== X-Received: by 2002:a17:903:1381:b0:188:602c:3332 with SMTP id jx1-20020a170903138100b00188602c3332mr19711924plb.122.1667823892062; Mon, 07 Nov 2022 04:24:52 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id v2-20020aa799c2000000b00561dcfa700asm4358141pfi.107.2022.11.07.04.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:51 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 24/72] e2fsck: configure one pfsck thread Date: Mon, 7 Nov 2022 17:51:12 +0530 Message-Id: <0bd43001b7c2d1cb5551564b83baf02bc771dcb6.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi This patch creates only one thread to do pass1 check if pthreads are enabled. The same codes can be used to create multiple threads, but other functions need to be modified to get ready for that. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 13 ++++ e2fsck/logfile.c | 2 + e2fsck/pass1.c | 152 ++++++++++++++++++++++++++++++++++++++++------ e2fsck/unix.c | 10 +++ tests/test_one.in | 8 +++ 5 files changed, 166 insertions(+), 19 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 284bc52e..5bc24c3f 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -465,6 +465,19 @@ struct e2fsck_struct { struct e2fsck_fc_replay_state fc_replay_state; }; +#ifdef HAVE_PTHREAD +struct e2fsck_thread_info { + /* ID returned by pthread_create() */ + pthread_t eti_thread_id; + /* Application-defined thread index */ + int eti_thread_index; + /* Thread has been started */ + int eti_started; + /* Context used for this thread */ + e2fsck_t eti_thread_ctx; +}; +#endif + /* Data structures to evaluate whether an extent tree needs rebuilding. */ struct extent_tree_level { unsigned int num_extents; diff --git a/e2fsck/logfile.c b/e2fsck/logfile.c index 83dbceff..74781f80 100644 --- a/e2fsck/logfile.c +++ b/e2fsck/logfile.c @@ -313,11 +313,13 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) goto out; expand_logfn(ctx, log_fn, &s); +#ifdef HAVE_PTHREAD if (ctx->global_ctx) { sprintf(string_index, "%d", ctx->thread_index); append_string(&s, ".", 1); append_string(&s, string_index, 0); } +#endif if ((log_fn[0] == '/') || !log_dir || !log_dir[0]) s0 = s.s; diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 2d4e62ca..596096d1 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -47,6 +47,9 @@ #ifdef HAVE_ERRNO_H #include #endif +#ifdef HAVE_PTHREAD +#include +#endif #include "e2fsck.h" #include @@ -1166,7 +1169,7 @@ static int e2fsck_should_abort(e2fsck_t ctx) return 0; } -void e2fsck_pass1_thread(e2fsck_t ctx) +void e2fsck_pass1_run(e2fsck_t ctx) { int i; __u64 max_sizes; @@ -2150,6 +2153,7 @@ endit: ctx->invalid_bitmaps++; } +#ifdef HAVE_PTHREAD static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx) { errcode_t retval; @@ -2201,18 +2205,38 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) return 0; } -void e2fsck_pass1_multithread(e2fsck_t ctx) +static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, + int num_threads, e2fsck_t global_ctx) { - errcode_t retval; - e2fsck_t thread_ctx; + errcode_t rc; + errcode_t ret = 0; + int i; + struct e2fsck_thread_info *pinfo; - retval = e2fsck_pass1_thread_prepare(ctx, &thread_ctx); - if (retval) { - com_err(ctx->program_name, 0, - _("while preparing pass1 thread\n")); - ctx->flags |= E2F_FLAG_ABORT; - return; + for (i = 0; i < num_threads; i++) { + pinfo = &infos[i]; + + if (!pinfo->eti_started) + continue; + + rc = pthread_join(pinfo->eti_thread_id, NULL); + if (rc) { + com_err(global_ctx->program_name, rc, + _("while joining thread\n")); + if (ret == 0) + ret = rc; + } + e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx); } + free(infos); + + return ret; +} + +static void *e2fsck_pass1_thread(void *arg) +{ + struct e2fsck_thread_info *info = arg; + e2fsck_t thread_ctx = info->eti_thread_ctx; #ifdef HAVE_SETJMP_H /* @@ -2223,28 +2247,118 @@ void e2fsck_pass1_multithread(e2fsck_t ctx) */ if (setjmp(thread_ctx->abort_loc)) { thread_ctx->flags &= ~E2F_FLAG_SETJMP_OK; - e2fsck_pass1_thread_join(ctx, thread_ctx); - return; + goto out; } thread_ctx->flags |= E2F_FLAG_SETJMP_OK; #endif - e2fsck_pass1_thread(thread_ctx); - retval = e2fsck_pass1_thread_join(ctx, thread_ctx); + e2fsck_pass1_run(thread_ctx); + +out: + return NULL; +} + +static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, + int num_threads, e2fsck_t global_ctx) +{ + struct e2fsck_thread_info *infos; + pthread_attr_t attr; + errcode_t retval; + errcode_t ret; + struct e2fsck_thread_info *tmp_pinfo; + int i; + e2fsck_t thread_ctx; + + retval = pthread_attr_init(&attr); if (retval) { - com_err(ctx->program_name, 0, - _("while joining pass1 thread\n")); - ctx->flags |= E2F_FLAG_ABORT; - return; + com_err(global_ctx->program_name, retval, + _("while setting pthread attribute\n")); + return retval; + } + + infos = calloc(num_threads, sizeof(struct e2fsck_thread_info)); + if (infos == NULL) { + retval = -ENOMEM; + com_err(global_ctx->program_name, retval, + _("while allocating memory for threads\n")); + pthread_attr_destroy(&attr); + return retval; + } + + for (i = 0; i < num_threads; i++) { + tmp_pinfo = &infos[i]; + tmp_pinfo->eti_thread_index = i; + retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx); + if (retval) { + com_err(global_ctx->program_name, retval, + _("while preparing pass1 thread\n")); + break; + } + tmp_pinfo->eti_thread_ctx = thread_ctx; + + retval = pthread_create(&tmp_pinfo->eti_thread_id, &attr, + &e2fsck_pass1_thread, tmp_pinfo); + if (retval) { + com_err(global_ctx->program_name, retval, + _("while creating thread\n")); + e2fsck_pass1_thread_join(global_ctx, thread_ctx); + break; + } + + tmp_pinfo->eti_started = 1; + } + + /* destroy the thread attribute object, since it is no longer needed */ + ret = pthread_attr_destroy(&attr); + if (ret) { + com_err(global_ctx->program_name, ret, + _("while destroying thread attribute\n")); + if (retval == 0) + retval = ret; + } + + if (retval) { + e2fsck_pass1_threads_join(infos, num_threads, global_ctx); + return retval; } + *pinfo = infos; + return 0; } +static void e2fsck_pass1_multithread(e2fsck_t global_ctx) +{ + struct e2fsck_thread_info *infos = NULL; + int num_threads = 1; + errcode_t retval; + + retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); + if (retval) { + com_err(global_ctx->program_name, retval, + _("while starting pass1 threads\n")); + goto out_abort; + } + + retval = e2fsck_pass1_threads_join(infos, num_threads, global_ctx); + if (retval) { + com_err(global_ctx->program_name, retval, + _("while joining pass1 threads\n")); + goto out_abort; + } + return; +out_abort: + global_ctx->flags |= E2F_FLAG_ABORT; + return; +} +#endif + void e2fsck_pass1(e2fsck_t ctx) { +#ifdef HAVE_PTHREAD if (ctx->options & E2F_OPT_MULTITHREAD) e2fsck_pass1_multithread(ctx); else - e2fsck_pass1_thread(ctx); +#endif + e2fsck_pass1_run(ctx); } #undef FINISH_INODE_LOOP diff --git a/e2fsck/unix.c b/e2fsck/unix.c index a77041b0..af2457e3 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -83,7 +83,9 @@ static void usage(e2fsck_t ctx) fprintf(stderr, "%s", _("\nEmergency help:\n" " -p Automatic repair (no questions)\n" +#ifdef HAVE_PTHREAD " -m multiple threads to speedup fsck\n" +#endif " -n Make no changes to the filesystem\n" " -y Assume \"yes\" to all questions\n" " -c Check for bad blocks and add them to the badblock list\n" @@ -856,7 +858,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) phys_mem_kb = get_memory_size() / 1024; ctx->readahead_kb = ~0ULL; +#ifdef HAVE_PTHREAD while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) +#else + while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) +#endif switch (c) { case 'C': ctx->progress = e2fsck_update_progress; @@ -897,9 +903,11 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) } ctx->options |= E2F_OPT_PREEN; break; +#ifdef HAVE_PTHREAD case 'm': ctx->options |= E2F_OPT_MULTITHREAD; break; +#endif case 'n': if (ctx->options & (E2F_OPT_YES|E2F_OPT_PREEN)) goto conflict_opt; @@ -1019,6 +1027,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("The -n and -l/-L options are incompatible.")); fatal_error(ctx, 0); } +#ifdef HAVE_PTHREAD if (ctx->options & E2F_OPT_MULTITHREAD) { if ((ctx->options & (E2F_OPT_YES|E2F_OPT_NO|E2F_OPT_PREEN)) == 0) { com_err(ctx->program_name, 0, "%s", @@ -1031,6 +1040,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) fatal_error(ctx, 0); } } +#endif if (ctx->options & E2F_OPT_NO) ctx->options |= E2F_OPT_READONLY; diff --git a/tests/test_one.in b/tests/test_one.in index 78499ad0..a1d46c9c 100644 --- a/tests/test_one.in +++ b/tests/test_one.in @@ -27,6 +27,7 @@ esac test_dir=$1 cmd_dir=$SRCDIR +pfsck_enabled="no" if test "$TEST_CONFIG"x = x; then TEST_CONFIG=$SRCDIR/test_config @@ -52,6 +53,13 @@ else test_description= fi +$FSCK --help 2>&1 | grep -q -w -- -m && pfsck_enabled=yes +if [ "$pfsck_enabled" != "yes" ] ; then + echo "$test_dir" | grep -q multithread && + echo "$test_name: $test_description: skipped (pfsck disabled)" && + exit 0 +fi + if [ -n "$SKIP_SLOW_TESTS" -a -f $test_dir/is_slow_test ]; then echo "$test_name: $test_description: skipped (slow test)" exit 0 From patchwork Mon Nov 7 12:21:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700572 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=OQoMrQ0i; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vms5xgYz23lW for ; Mon, 7 Nov 2022 23:25:17 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vms5Vhwz4xDK for ; Mon, 7 Nov 2022 23:25:17 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vms5RvFz4xGj; Mon, 7 Nov 2022 23:25:17 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=OQoMrQ0i; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vms5LpDz4xDK for ; Mon, 7 Nov 2022 23:25:17 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232207AbiKGMZQ (ORCPT ); Mon, 7 Nov 2022 07:25:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232231AbiKGMZA (ORCPT ); Mon, 7 Nov 2022 07:25:00 -0500 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 77A1E1B1C1 for ; Mon, 7 Nov 2022 04:24:58 -0800 (PST) Received: by mail-pl1-x629.google.com with SMTP id p21so10900701plr.7 for ; Mon, 07 Nov 2022 04:24:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=caV4R+OWTjHvOPUyveUfpCTXxcmng8ZhQguB5J9grQE=; b=OQoMrQ0i6opj6kTmNXcce/l92K59Fb4sRo0kSWE3wwxHHiTH19bWHOAzC3vVhpeczS AT6C2p/GQWcsA/D2XYVvEEABkevsnMN3gOp4yk3mnGJ6rgoLlZXtQYZGw0u7W2C47arn hqu+nWMC46vHVIso98QTAr2fTgF/bkLnPGZpwe2b/shii0JuKpBZ7bWrGe4cU0YugQOL CWUlXpB4YV8iF8XUSbFR5N6SgFz10/LJQ22SLKS8Maau8cMdaC3TxLMTdHpMaL7Tcax7 KVJhJRcA8kLzNQCLXq+opu2S2rWqGar72zywnOzXSNMpnrpJ/JojL3fmqEKQjvtlUtzS PWOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=caV4R+OWTjHvOPUyveUfpCTXxcmng8ZhQguB5J9grQE=; b=crbQySqcl4MnAQrIU9vaOAiXkY/HqdfMTcHiQnjLgudb5yck4GBgwt5dzGULmmdDag Bz5hcJhNPoEgf2BOdgFJ+qw8iFE7YZq2zFNQWxSStYK2u835pjkK7Ul+WBkpt/XfmOuy JXr/YyPjUEFMtc3OjOLZEvDQk/v9VxpaXFKsP8114hyRKSI0c3f6UkH1ikH6oXfVPrwD SQwLTDGDn6F6JJ1uaD3im3ZVbAATkn44FRONHdp3hbjtCRK7EOthy/O6IRlnqK9ItZ9r Ts0dT7GDN0OgdxNbfnFuZX8ztXhgCQ6Hvmt15q0R1jGVMTI2ngohHGNDK4LST951lmZg sdZQ== X-Gm-Message-State: ANoB5plRoJ1GORDxUCJYakekAjxUP7IJvIQie0t523TTFa+SOi0VWXxm 7rak/tL++MuM2c+9sEA4bEY= X-Google-Smtp-Source: AA0mqf6uQbGc6rfCXRR5e+S0I2IjcCh1khsCtPRs7wW1ZaPhh/olMI+pJNykgHPApT1SuhjQf4ONjQ== X-Received: by 2002:a17:903:247:b0:188:88be:4fb1 with SMTP id j7-20020a170903024700b0018888be4fb1mr220789plh.34.1667823898003; Mon, 07 Nov 2022 04:24:58 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id o16-20020a170902d4d000b0018157b415dbsm4889228plg.63.2022.11.07.04.24.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:24:57 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 25/72] e2fsck: Add e2fsck_pass1_thread_join return value Date: Mon, 7 Nov 2022 17:51:13 +0530 Message-Id: <44270897e6bf20bdb883d3e3cab43a83806e696c.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This adds the return value to e2fsck_pass1_thread_join() to check for any error in pass1 threads join operation. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 596096d1..4b165600 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2174,8 +2174,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre return 0; } -static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) +static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) { + errcode_t retval = 0; int flags = global_ctx->flags; FILE *global_logf = global_ctx->logf; FILE *global_problem_logf = global_ctx->problem_logf; @@ -2195,6 +2196,14 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) global_ctx->fs->priv_data = global_ctx; global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; + return retval; +} + +static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + errcode_t retval; + + retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx); if (thread_ctx->logf) fclose(thread_ctx->logf); if (thread_ctx->problem_logf) { @@ -2202,7 +2211,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) fclose(thread_ctx->problem_logf); } ext2fs_free_mem(&thread_ctx); - return 0; + return retval; } static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, @@ -2226,7 +2235,13 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, if (ret == 0) ret = rc; } - e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx); + rc = e2fsck_pass1_thread_join(global_ctx, infos[i].eti_thread_ctx); + if (rc) { + com_err(global_ctx->program_name, rc, + _("while joining pass1 thread\n")); + if (ret == 0) + ret = rc; + } } free(infos); From patchwork Mon Nov 7 12:21:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700573 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=cyDV/b0L; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vmv5rmJz23lW for ; Mon, 7 Nov 2022 23:25:19 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vmv5PVcz4xDK for ; Mon, 7 Nov 2022 23:25:19 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vmv5JBDz4xGj; Mon, 7 Nov 2022 23:25:19 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=cyDV/b0L; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vmv5CXbz4xDK for ; Mon, 7 Nov 2022 23:25:19 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232180AbiKGMZT (ORCPT ); Mon, 7 Nov 2022 07:25:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbiKGMZF (ORCPT ); Mon, 7 Nov 2022 07:25:05 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD0A163C2 for ; Mon, 7 Nov 2022 04:25:04 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id u8-20020a17090a5e4800b002106dcdd4a0so14430909pji.1 for ; Mon, 07 Nov 2022 04:25:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2PkED8TboI7uLiO3M1yPjVn8xujKFFCJ9w9S0ClkL94=; b=cyDV/b0LX+Q7YRZuVCpIvO9Ov1aoqv8/Kw1PfCvZRyB9PHfNF3uXSABQDAw1WAWVbH yn1sbOXqQPuIbGghaofIiMIunum+wdkG+1+q4AISbpz3xIDNLisOs7l3ZLdkheKpIyy9 0pxZv4YJ9B7BTNj7+5u4bu3eG3bgVmddZ+RlLFsCtNFzU4sNYnfhwdi0F9ucCwKnJRl+ EZgomZNZmrUxXxcQOiyq0LZxViC2/H1jbC/yNs85UDKUA0auH63BMEN33xV//fvmUkIM mOAlwFd9SsNUpSabO6CoD/RPe+bMaAUTgEqKoDox97B0Sd7I0uUAn1bBJtBvHbz9j6mX x0rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2PkED8TboI7uLiO3M1yPjVn8xujKFFCJ9w9S0ClkL94=; b=GzHk3dBCdH702ezBKmPjSXAIcb7zQoBEp09l2/XBfo3CQmq8BBfWmOt6cbeqtb8WlU v2H8vCCWdobB7HU3WEkxy8fyD5fM9LltY7MjCPEe+8sL4TT7HgQDZZRTEK8IjWT86UPJ wKrW+pMP43eSgr3AZLZ/nLdQo7kqfMpyaCNg+uTAfJpKi7cva1MI+Qv0Iw5NFCMATH6r buBbqeU9JZMETEUW+XIW5TsURyhffoKFC7n6yZjlBlDZr6dPIJDcHpR9XANd/sQLD/WC bHnAG11gfwRi/3GQBKvt+43L7ncPlcB/cMkxHZ80egYNIGMTdwCnfIo0hwkoXQO5og7T Hy0g== X-Gm-Message-State: ACrzQf1lyv915v5PrMifiHCYh7Vxa6PVd8uszUOnn4aNpYrFgzOC3Eaf NSSW1CUj0QIvvFmGdKMhKuo= X-Google-Smtp-Source: AMsMyM6wlOQ78wzIAhI9mGLcVr6IRFb+xiRkaYNsDCKcjUno5B/w16gh2bMDKaUofuLxXYKSDMcsuA== X-Received: by 2002:a17:902:db0e:b0:186:9b38:ad26 with SMTP id m14-20020a170902db0e00b001869b38ad26mr51684040plx.43.1667823904261; Mon, 07 Nov 2022 04:25:04 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id f4-20020a170902860400b001868d4600b8sm4894962plo.158.2022.11.07.04.25.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:03 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 26/72] e2fsck: Use merge/clone apis of libext2fs Date: Mon, 7 Nov 2022 17:51:14 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This patch makes use of libext2fs merge/clone apis in e2fsck to first clone the global_fs based on the passed CLONE_FLAGS. Then once the parallel e2fsck operation finishes, it calls for merge fs api which merges it into parent fs. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 55 insertions(+), 3 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 4b165600..040c58ce 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2154,10 +2154,36 @@ endit: } #ifdef HAVE_PTHREAD +static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, ext2_filsys src) +{ + errcode_t retval; + + io_channel_flush_cleanup(src->io); + retval = dest->io->manager->open(dest->device_name, + IO_FLAG_RW | IO_FLAG_THREADS, &dest->io); + if (retval) + return retval; + dest->image_io = dest->io; + dest->io->app_data = dest; + /* Block size might not be default */ + io_channel_set_blksize(dest->io, src->io->block_size); + ehandler_init(dest->io); + + dest->priv_data = dest_context; + dest_context->fs = dest; + /* The data should be written to disk immediately */ + dest->io->flags |= CHANNEL_FLAGS_WRITETHROUGH; + /* icache will be rebuilt if needed, so do not copy from @src */ + src->icache = NULL; + return 0; +} + static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx) { errcode_t retval; e2fsck_t thread_context; + ext2_filsys thread_fs; + ext2_filsys global_fs = global_ctx->fs; retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context); if (retval) { @@ -2165,13 +2191,30 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre return retval; } memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct)); - thread_context->fs->priv_data = thread_context; thread_context->global_ctx = global_ctx; + retval = ext2fs_clone_fs(global_fs, &thread_fs, + EXT2FS_CLONE_BLOCK | EXT2FS_CLONE_INODE | + EXT2FS_CLONE_BADBLOCKS | EXT2FS_CLONE_DBLIST); + if (retval) { + com_err(global_ctx->program_name, retval, "while allocating memory"); + goto out_context; + } + + retval = e2fsck_open_channel_fs(thread_fs, thread_context, global_fs); + if (retval) { + com_err(global_ctx->program_name, retval, "while copying fs"); + goto out_fs; + } thread_context->thread_index = 0; set_up_logging(thread_context); *thread_ctx = thread_context; return 0; +out_fs: + ext2fs_merge_fs(&thread_fs); +out_context: + ext2fs_free_mem(&thread_context); + return retval; } static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) @@ -2180,6 +2223,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx int flags = global_ctx->flags; FILE *global_logf = global_ctx->logf; FILE *global_problem_logf = global_ctx->problem_logf; + ext2_filsys thread_fs = thread_ctx->fs; + ext2_filsys global_fs = global_ctx->fs; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2192,10 +2237,17 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx /* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); - - global_ctx->fs->priv_data = global_ctx; global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; + + global_fs->priv_data = global_ctx; + global_ctx->fs = global_fs; + + retval = ext2fs_merge_fs(&(thread_ctx->fs)); + if (retval) { + com_err(global_ctx->program_name, 0, _("while merging fs\n")); + return retval; + } return retval; } From patchwork Mon Nov 7 12:21:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700574 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=pNMdoONQ; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vn95xVLz23lW for ; Mon, 7 Nov 2022 23:25:33 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vn95VXBz4xDK for ; Mon, 7 Nov 2022 23:25:33 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vn95Ryrz4xGj; Mon, 7 Nov 2022 23:25:33 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=pNMdoONQ; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vn95LV4z4xDK for ; Mon, 7 Nov 2022 23:25:33 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232217AbiKGMZa (ORCPT ); Mon, 7 Nov 2022 07:25:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGMZL (ORCPT ); Mon, 7 Nov 2022 07:25:11 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCF7F140FC for ; Mon, 7 Nov 2022 04:25:10 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id m14-20020a17090a3f8e00b00212dab39bcdso14458400pjc.0 for ; Mon, 07 Nov 2022 04:25:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2G1Y5RYslrYkDHJpJVANf9ho1llYiDGkhLIpEGsIOvQ=; b=pNMdoONQaWLIssCJN0Z7wA0OLZ3vbDWGTtv5cJkcT4+uJGprRm4C9N/XPo3Ff0tAS7 APOftga1RoYUWSXnVxO8Kp6VFCEBTqjkJyviXp9UaPSKX7elBsrf5nAncYe/nGIpJdAM 8KbCYYi6Y3FQkwH25d1wzDjQEXCc1sUxZPFbj6UenjA5VeOh3U9cjY8E8iD3Lvuqt34C X69hlX+gDJ9dvt6CfS5EOjhq9G33iFMZ32X0MjPxYp2Y8P1laMa1j+tqK/0ZFYsc/xYJ nfoZDOzmSXiYGHBfGoPdWKa0vJnv7Rg8HyDoADV+4T0YQA0s60i8rfu2sczgihfT2oAQ T25g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2G1Y5RYslrYkDHJpJVANf9ho1llYiDGkhLIpEGsIOvQ=; b=Ih+YzJrYy5gJ/uvoFQJ+trDdei3pQ+CEAi68ZSFedNL69wvQOrad3OnaJ9I9qPDHPv +8KKZZSDY0X65zKhQ3YinpXVY7ToUISG5nEplBRmU4bhfDpK8A/UBfhgjLi52ZpBomFl lPq+K6iaeXPbHwc0nFSUay2Cxe4zh2kxr04tBHbAQ7QUPIwEo/xfv2dIjnKur20HvI3P fJR5R8tzpmQZD3FTJYlu+aLXEHa3/OHU/Pg1ho6UExM5m/qdbe/5BRup5qDJzg/wL1n4 pwB/8KUjXuq8t55hPNU9k+k629ZWTvBFDpY7lZPN2w4efGw497QO9iSDU59gNzr0z5vJ yPVg== X-Gm-Message-State: ACrzQf1L8JPO+sVfE2GvzR5fIOLm2B152wwczmuBEaT/lAZ4m9U++g08 J5rvS7O3HBNutVpayUONXEiT9Va92ig= X-Google-Smtp-Source: AMsMyM5nEuheg+wky1D63STUoBziZ3VE4hLu3dO7q8F74v3Uw7RFts9IkV8kR27QYMMZM/QFFjF/Pw== X-Received: by 2002:a17:902:ebc5:b0:186:b848:c6a with SMTP id p5-20020a170902ebc500b00186b8480c6amr49963757plg.46.1667823910258; Mon, 07 Nov 2022 04:25:10 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id o16-20020a170902d4d000b0018157b415dbsm4889596plg.63.2022.11.07.04.25.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:09 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 27/72] e2fsck: Add e2fsck_pass1_merge_bitmap() api Date: Mon, 7 Nov 2022 17:51:15 +0530 Message-Id: <0ec6a72242cc188ef8d9b308bf862a310801ce76.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong This adds e2fsck_pass1_merge_bitmap() which uses libext2fs merge api (ext2fs_merge_bitmap()) to merge all the bitmaps after threads finishes. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 040c58ce..9a273515 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -50,6 +50,7 @@ #ifdef HAVE_PTHREAD #include #endif +#include #include "e2fsck.h" #include @@ -2154,6 +2155,26 @@ endit: } #ifdef HAVE_PTHREAD +static errcode_t e2fsck_pass1_merge_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src, + ext2fs_generic_bitmap *dest) +{ + errcode_t ret = 0; + + if (*src) { + if (*dest == NULL) { + *dest = *src; + *src = NULL; + } else { + ret = ext2fs_merge_bitmap(*src, *dest, NULL, NULL); + if (ret) + return ret; + } + (*dest)->fs = fs; + } + + return 0; +} + static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, ext2_filsys src) { errcode_t retval; @@ -2185,6 +2206,19 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre ext2_filsys thread_fs; ext2_filsys global_fs = global_ctx->fs; + assert(global_ctx->inode_used_map == NULL); + assert(global_ctx->inode_dir_map == NULL); + assert(global_ctx->inode_bb_map == NULL); + assert(global_ctx->inode_imagic_map == NULL); + assert(global_ctx->inode_reg_map == NULL); + assert(global_ctx->inodes_to_rebuild == NULL); + + assert(global_ctx->block_found_map == NULL); + assert(global_ctx->block_dup_map == NULL); + assert(global_ctx->block_ea_map == NULL); + assert(global_ctx->block_metadata_map == NULL); + assert(global_ctx->fs->dblist == NULL); + retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context); if (retval) { com_err(global_ctx->program_name, retval, "while allocating memory"); @@ -2225,6 +2259,18 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx FILE *global_problem_logf = global_ctx->problem_logf; ext2_filsys thread_fs = thread_ctx->fs; ext2_filsys global_fs = global_ctx->fs; + ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; + ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; + ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; + ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; + ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; + ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; + ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; + ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; + ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; + ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map; + ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; + #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2234,6 +2280,19 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx #ifdef HAVE_SETJMP_H memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf)); #endif + + global_ctx->inode_used_map = inode_used_map; + global_ctx->inode_bad_map = inode_bad_map; + global_ctx->inode_dir_map = inode_dir_map; + global_ctx->inode_bb_map = inode_bb_map; + global_ctx->inode_imagic_map = inode_imagic_map; + global_ctx->inodes_to_rebuild = inodes_to_rebuild; + global_ctx->inode_reg_map = inode_reg_map; + global_ctx->block_found_map = block_found_map; + global_ctx->block_dup_map = block_dup_map; + global_ctx->block_ea_map = block_ea_map; + global_ctx->block_metadata_map = block_metadata_map; + /* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); @@ -2248,6 +2307,63 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx com_err(global_ctx->program_name, 0, _("while merging fs\n")); return retval; } + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_used_map, + &global_ctx->inode_used_map); + if (retval) + return retval; + + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_bad_map, + &global_ctx->inode_bad_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_dir_map, + &global_ctx->inode_dir_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_bb_map, + &global_ctx->inode_bb_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_imagic_map, + &global_ctx->inode_imagic_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inode_reg_map, + &global_ctx->inode_reg_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->inodes_to_rebuild, + &global_ctx->inodes_to_rebuild); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->block_found_map, + &global_ctx->block_found_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->block_dup_map, + &global_ctx->block_dup_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->block_ea_map, + &global_ctx->block_ea_map); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, + &thread_ctx->block_metadata_map, + &global_ctx->block_metadata_map); + if (retval) + return retval; + return retval; } @@ -2256,6 +2372,18 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) errcode_t retval; retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_used_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_bad_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_dir_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_bb_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map); + e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild); + e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map); + e2fsck_pass1_free_bitmap(&thread_ctx->block_dup_map); + e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); + e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map); + if (thread_ctx->logf) fclose(thread_ctx->logf); if (thread_ctx->problem_logf) { From patchwork Mon Nov 7 12:21:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700575 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Iz1aVJ1n; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VnB2bbJz23lW for ; Mon, 7 Nov 2022 23:25:34 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnB28rTz4xDK for ; Mon, 7 Nov 2022 23:25:34 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VnB26Jdz4xGj; Mon, 7 Nov 2022 23:25:34 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Iz1aVJ1n; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnB22rbz4xDK for ; Mon, 7 Nov 2022 23:25:34 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232199AbiKGMZc (ORCPT ); Mon, 7 Nov 2022 07:25:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232146AbiKGMZR (ORCPT ); Mon, 7 Nov 2022 07:25:17 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E893314D05 for ; Mon, 7 Nov 2022 04:25:16 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id y13so10442838pfp.7 for ; Mon, 07 Nov 2022 04:25:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IfEeEEDfQCyUEF0iTzgW5irU43t9MpNBB/mcyhT3TkI=; b=Iz1aVJ1nXiNshUDdVU9mpVLufYdsHQhBslODYVRTTpwxupIhE5220aToKmjJecIqJF KaN+wuHRUttD9p0cVl/3f83J6Q8sF3WlxPVRmFqtshws/VzPDmVyMfIb/oGLVzcYR88h qwMbaqyxz7x49CgVZNVApPgsJsr9fpi6lS3nG2Kr9pjT7fmXIIxIZJNyIJdmHM8QSQ1P zSc9t31fadDT7cKEmwjAZHHbezu4W91yTEaWN3wXEvmPlq384r8ma29yeeRjWnm9cX7Q KBivDSykk2mXtDLxz66ddu+YNLUBFS/uli6yTrlCyW4LLYmEys7R/CEDqXBXLkhQEJoq vg4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IfEeEEDfQCyUEF0iTzgW5irU43t9MpNBB/mcyhT3TkI=; b=s7fFMxh+4ihw0dHT68WVcPu+fBC63k0lCCMDwaIlzFYOsdKuenma5o/nnVyDzknmQa 4IZJryK6NmD6UFV/pt5AEqah5qRVQpiVCftN8VIClJ//0MQJA+rLxYEJ/obZaa9mUPvQ zrbluPo60u+CU37U41Bx8XlE6DGTwDLUukvt7EFQHnVXl1JOy4MtoHgVQEYy3yfUkf8Y kRAqqqc2cQwK1jDAO9IKFM4DsT8EzFfGzOjRMC/g8NOIeaiDLazqoxzpADKBfzG741dp SD/1f9MFaSn8mPvODofz2KSTQpGQsWq1HDqJHXVThf0P3W/SHs68QqBWbu+PHU86dqDK v7LA== X-Gm-Message-State: ANoB5pmvM5LA2supm40fpHuFdoHZ6kBNfZGe6RaIi3YE4iu/ZfQSr+U8 yPb/YBv+7wk1LhDIZGFMRd0= X-Google-Smtp-Source: AA0mqf47GaxC3ZTW6XixEcicI/nVoTZl7n8X/5ulj5NDxLsxbK/8Wl+GvkqOO8AUPxgl7pTbE5yzOA== X-Received: by 2002:a63:1607:0:b0:470:6934:e853 with SMTP id w7-20020a631607000000b004706934e853mr7151836pgl.237.1667823916334; Mon, 07 Nov 2022 04:25:16 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y28-20020aa79afc000000b005627ddbc7a4sm4364810pfp.191.2022.11.07.04.25.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:15 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 28/72] e2fsck: Add asserts in open_channel_fs Date: Mon, 7 Nov 2022 17:51:16 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Signed-off-by: Li Xi Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 9a273515..ea432ff2 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2190,6 +2190,18 @@ static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, io_channel_set_blksize(dest->io, src->io->block_size); ehandler_init(dest->io); + assert(dest->io->magic == src->io->magic); + assert(dest->io->manager == src->io->manager); + assert(strcmp(dest->io->name, src->io->name) == 0); + assert(dest->io->block_size == src->io->block_size); + assert(dest->io->read_error == src->io->read_error); + assert(dest->io->write_error == src->io->write_error); + assert(dest->io->refcount == src->io->refcount); + assert(dest->io->flags == src->io->flags); + assert(dest->io->app_data == dest); + assert(src->io->app_data == src); + assert(dest->io->align == src->io->align); + dest->priv_data = dest_context; dest_context->fs = dest; /* The data should be written to disk immediately */ From patchwork Mon Nov 7 12:21:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700576 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=EADHTMOH; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VnN2Kzgz23lW for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnN1qJVz4xGj for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VnN1n79z4xZ3; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=EADHTMOH; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnN1hDVz4xGj for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231831AbiKGMZn (ORCPT ); Mon, 7 Nov 2022 07:25:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51330 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232213AbiKGMZZ (ORCPT ); Mon, 7 Nov 2022 07:25:25 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36B966247 for ; Mon, 7 Nov 2022 04:25:23 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id v28so10429543pfi.12 for ; Mon, 07 Nov 2022 04:25:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZzYUNICtcdUfkHGzm3SM3RGET55sqDgTXz+Q60l/yv0=; b=EADHTMOHLN5PEWZanKvGFAMTKeoUEYJn9VJkIzy4YbjQ8lk7eyqu5BGC+mjygwl3Ln aeP8kmUR1tTOIfNK+r/yUQk2SFNUk2bFBeZpd2RYF7OO63rdkRXOF8i5XvHE+3xgx1x8 83lcPbNdvcGKAz9UFbwx+g6En0DFb/eRphqk5dpCo/3jcADZ1oO1U6sa5qZNtsUnWRTD soRjS2l4b275479YioKblfzPMbzDBQUeXR7kuVkSF3uSRybGTtzyFit8D/37iKk/SyIH bNF03Ijb/z+d/ZQ6ODabM3VLfctxtj3blWo9MwOqVHpfHqXzrMlHaZIqzPB4PuzrXhqY rh0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZzYUNICtcdUfkHGzm3SM3RGET55sqDgTXz+Q60l/yv0=; b=jz2+HO2vOyJ0COKvKnD7MiO/JBfDZB51lD8qoYgU3dgOwdS6WOePUkiItgh1+0THNt LRdEYOMgJbudTEH2UOsCmyRl7Km2wyQN1Ehr4yLcxjwei2s5D469GL9xSWiaVPDvgyMU 8OBod5LhXVBHhm1iyxneWzUfx/Zj+3ja6ZBq5wJZltwrABntZky6YgUby8d2O6M2JWiv +sCbrZQ/he/ApXnYcst+chhHth0shF/vfY3ZmGz0HaaNSiuuJmKqfIFqWGu/TUamYnx9 HICGfDtYG3+FNN8olcwhnu1KPFophT1U1TVZobiVxmVNb/n823fqL/OxCkBt7SOzEBbp 0/aw== X-Gm-Message-State: ACrzQf1DiD0OMrn2E0BDAgNQdPg+1cw8lmbV3sav1ChnC7sWku5rLrtK JFCLJ/yILon5c+RxLmDxPDxqd9PKS/A= X-Google-Smtp-Source: AMsMyM4wJMNbpHBUCk1Q6EcN19rSlOUx9ETDX+YFcS4/0LTGQ2Jq5qRtHzu6h75h3SyQnV56E3kZ8w== X-Received: by 2002:a05:6a00:23c9:b0:56c:9f62:3369 with SMTP id g9-20020a056a0023c900b0056c9f623369mr49147365pfc.22.1667823922675; Mon, 07 Nov 2022 04:25:22 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id b5-20020aa78ec5000000b0056bb7d90f0fsm4366888pfr.182.2022.11.07.04.25.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:21 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 29/72] e2fsck: add start/end group for thread Date: Mon, 7 Nov 2022 17:51:17 +0530 Message-Id: <800f926c9314d97f735fb25c40a3a88efcc05beb.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi When multi-threads are used for check, each thread needs to jump to different group in pass1 check. This patch adds the group jumping support. But still, only one thread is used to check. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 23 ++++++++++++++++++-- e2fsck/logfile.c | 3 ++- e2fsck/pass1.c | 44 ++++++++++++++++++++++++++++++++++++--- e2fsck/problem.c | 5 +++++ e2fsck/problem.h | 3 +++ lib/ext2fs/ext2_err.et.in | 3 +++ 6 files changed, 75 insertions(+), 6 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 5bc24c3f..639f4e80 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -261,6 +261,22 @@ struct e2fsck_fc_replay_state { __u16 fc_super_state; }; +#ifdef HAVE_PTHREAD +/* + * Fields that used for multi-thread + */ +struct e2fsck_thread { + /* Thread index */ + int et_thread_index; + /* The start group number for this thread */ + dgrp_t et_group_start; + /* The end (not included) group number for this thread*/ + dgrp_t et_group_end; + /* The next group number to check */ + dgrp_t et_group_next; +}; +#endif + struct e2fsck_struct { /* Global context to get the cancel flag */ e2fsck_t global_ctx; @@ -344,8 +360,11 @@ struct e2fsck_struct { */ ext2_ino_t stashed_ino; struct ext2_inode *stashed_inode; - /* Thread index, if global_ctx is null, this field is unused */ - int thread_index; + + /* if @global_ctx is null, this field is unused */ +#ifdef HAVE_PTHREAD + struct e2fsck_thread thread_info; +#endif /* * Location of the lost and found directory diff --git a/e2fsck/logfile.c b/e2fsck/logfile.c index 74781f80..cc811d5a 100644 --- a/e2fsck/logfile.c +++ b/e2fsck/logfile.c @@ -315,7 +315,8 @@ static FILE *set_up_log_file(e2fsck_t ctx, const char *key, const char *fn) expand_logfn(ctx, log_fn, &s); #ifdef HAVE_PTHREAD if (ctx->global_ctx) { - sprintf(string_index, "%d", ctx->thread_index); + sprintf(string_index, "%d", + ctx->thread_info.et_thread_index); append_string(&s, ".", 1); append_string(&s, string_index, 0); } diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index ea432ff2..3bb87669 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1389,6 +1389,23 @@ void e2fsck_pass1_run(e2fsck_t ctx) /* Set up ctx->lost_and_found if possible */ (void) e2fsck_get_lost_and_found(ctx, 0); +#ifdef HAVE_PTHREAD + if (ctx->global_ctx) { + if (ctx->options & E2F_OPT_DEBUG && + ctx->options & E2F_OPT_MULTITHREAD) + fprintf(stderr, "thread %d jumping to group %d\n", + ctx->thread_info.et_thread_index, + ctx->thread_info.et_group_start); + pctx.errcode = ext2fs_inode_scan_goto_blockgroup(scan, + ctx->thread_info.et_group_start); + if (pctx.errcode) { + fix_problem(ctx, PR_1_PASS_HEADER, &pctx); + ctx->flags |= E2F_FLAG_ABORT; + goto endit; + } + } +#endif + while (1) { if (ino % (fs->super->s_inodes_per_group * 4) == 1) { if (e2fsck_mmp_update(fs)) @@ -1432,6 +1449,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino); continue; } + if (pctx.errcode == EXT2_ET_SCAN_FINISHED) + break; if (pctx.errcode && pctx.errcode != EXT2_ET_INODE_CSUM_INVALID && pctx.errcode != EXT2_ET_INODE_IS_GARBAGE) { @@ -2211,12 +2230,14 @@ static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, return 0; } -static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx) +static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx, + int thread_index) { errcode_t retval; e2fsck_t thread_context; ext2_filsys thread_fs; ext2_filsys global_fs = global_ctx->fs; + struct e2fsck_thread *tinfo; assert(global_ctx->inode_used_map == NULL); assert(global_ctx->inode_dir_map == NULL); @@ -2252,8 +2273,15 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_fs; } - thread_context->thread_index = 0; + thread_context->thread_info.et_thread_index = thread_index; set_up_logging(thread_context); + + assert(thread_index == 0); + tinfo = &thread_context->thread_info; + tinfo->et_group_start = 0; + tinfo->et_group_next = 0; + tinfo->et_group_end = thread_fs->group_desc_count; + *thread_ctx = thread_context; return 0; out_fs: @@ -2495,7 +2523,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, for (i = 0; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; - retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx); + retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i); if (retval) { com_err(global_ctx->program_name, retval, _("while preparing pass1 thread\n")); @@ -2580,6 +2608,7 @@ static errcode_t scan_callback(ext2_filsys fs, { struct scan_callback_struct *scan_struct; e2fsck_t ctx; + struct e2fsck_thread *tinfo; scan_struct = (struct scan_callback_struct *) priv_data; ctx = scan_struct->ctx; @@ -2591,6 +2620,15 @@ static errcode_t scan_callback(ext2_filsys fs, ctx->fs->group_desc_count)) return EXT2_ET_CANCEL_REQUESTED; +#ifdef HAVE_PTHREAD + if (ctx->global_ctx) { + tinfo = &ctx->thread_info; + tinfo->et_group_next++; + if (tinfo->et_group_next >= tinfo->et_group_end) + return EXT2_ET_SCAN_FINISHED; + } +#endif + return 0; } diff --git a/e2fsck/problem.c b/e2fsck/problem.c index 6ad6fb84..d5452441 100644 --- a/e2fsck/problem.c +++ b/e2fsck/problem.c @@ -1309,6 +1309,11 @@ static struct e2fsck_problem problem_table[] = { N_("Orphan file @i %i is not in use, but contains data. "), PROMPT_CLEAR, PR_PREEN_OK }, + /* Failed to goto block group */ + { PR_1_SCAN_GOTO, + N_("failed to goto block group"), + PROMPT_NONE, PR_FATAL, 0, 0, 0 }, + /* Pass 1b errors */ /* Pass 1B: Rescan for duplicate/bad blocks */ diff --git a/e2fsck/problem.h b/e2fsck/problem.h index b47b0c63..a6207428 100644 --- a/e2fsck/problem.h +++ b/e2fsck/problem.h @@ -734,6 +734,9 @@ struct problem_context { /* Orphan file inode is not in use, but contains data */ #define PR_1_ORPHAN_FILE_NOT_CLEAR 0x010090 +/* Failed to goto block group */ +#define PR_1_SCAN_GOTO 0x0100A0 + /* * Pass 1b errors */ diff --git a/lib/ext2fs/ext2_err.et.in b/lib/ext2fs/ext2_err.et.in index de140198..b8f45ae2 100644 --- a/lib/ext2fs/ext2_err.et.in +++ b/lib/ext2fs/ext2_err.et.in @@ -557,4 +557,7 @@ ec EXT2_ET_EXTENT_CYCLE, ec EXT2_ET_EXTERNAL_JOURNAL_NOSUPP, "Operation not supported on an external journal" +ec EXT2_ET_SCAN_FINISHED, + "Scanning finished" + end From patchwork Mon Nov 7 12:21:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700577 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=NNQILbdm; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VnN5mGGz23lx for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnN5L01z4xGj for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VnN5HZnz4xZ3; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=NNQILbdm; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnN5Cldz4xGj for ; Mon, 7 Nov 2022 23:25:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232134AbiKGMZo (ORCPT ); Mon, 7 Nov 2022 07:25:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGMZb (ORCPT ); Mon, 7 Nov 2022 07:25:31 -0500 Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6390E1B7AA for ; Mon, 7 Nov 2022 04:25:29 -0800 (PST) Received: by mail-pg1-x530.google.com with SMTP id s196so10319033pgs.3 for ; Mon, 07 Nov 2022 04:25:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=geMLgKNDbiQNg64oeoK/gVcmbGzDD8oHWu7l5dRAT/E=; b=NNQILbdmsGX5BuTvxqmvMfvJCElYOCI9m3KEDIMg3F2+HRoy3hBn5LFew4knl3XL2x yv19H/vg8JWTvrFnTe83cKhA6PoQoX+SktaP/jANF+OU9tQe5BaPrn+8Sj9k1Dg9EOm9 iKdsXG4zpxWIwHglWtQ8Qo6z30FqFXpMzq/4toDPoh67QnTxdDGz8o3vtUL+443ol7IN b+hnWBSBmNIwQuD0SC406/48xtmTHK9dRC3y5I9qv3URo21QLE/iQVpw4pL9oDWL81aC 90KDpCh4+g2+prisSYoArVj5FATXxYuD0K6BzXouZgHh3/nxPmRe7DJ9S6ezBST7Tnoc GU6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=geMLgKNDbiQNg64oeoK/gVcmbGzDD8oHWu7l5dRAT/E=; b=x2a3v3GAVzRbJiWDGMwLuclrPsNJbUu49axAVLj8RaBrZBEgddKw+ucWBG8R/XuOl7 alDAgPqmCCgztYCvoaMygFeiE8V22A52IoEj5Udt2Q5HwY+8m+6mwrSIjxVzSowqffH7 xpxb9uaQfPWOK5RMveQASteHWLmQSv6nFhyMDmQzS4B96nz6hCtz81kCd6njHM+jBpup 5GUMIEDlbY6dPKzqlM0Z8UYs8wFDKQbi14gkCrp0FkyUQbEBPWofqSUAQhIaQr/LoyIa QymPk+vChyecW3WAtGF6F39Tiuo4xX24ozqpYksNkB4jHCgfDDzbqprUvRQX/MSqFMS/ 7mGQ== X-Gm-Message-State: ACrzQf3tw3Z3VI4gyhigncL63NDDBCzIWp4YC8DAaKVzkKjK7pOL/zl8 AyK2Z9JTnhyDjyh0pnhPm2CUheuhRdI= X-Google-Smtp-Source: AMsMyM5CpFl1sDXt1/cJF8XcZCw1D1nhkzPWKvADeGWWc8qdp4ROJDW+/2EcycIqsw/oVSvHy5n6/A== X-Received: by 2002:a05:6a00:170d:b0:56e:55de:9858 with SMTP id h13-20020a056a00170d00b0056e55de9858mr22360715pfc.17.1667823928924; Mon, 07 Nov 2022 04:25:28 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id i4-20020a056a00004400b0056164b52bd8sm4391441pfk.32.2022.11.07.04.25.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:28 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 30/72] e2fsck: split groups to different threads Date: Mon, 7 Nov 2022 17:51:18 +0530 Message-Id: <42bd4ff9929e4166ad5e7b2e36204c995521c9e4.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi The start/end groups of a thread is calculated according to the thread number. But still, only one thread is used to check. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 3bb87669..3b411b70 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2231,13 +2231,14 @@ static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, } static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx, - int thread_index) + int thread_index, int num_threads) { errcode_t retval; e2fsck_t thread_context; ext2_filsys thread_fs; ext2_filsys global_fs = global_ctx->fs; struct e2fsck_thread *tinfo; + dgrp_t average_group; assert(global_ctx->inode_used_map == NULL); assert(global_ctx->inode_dir_map == NULL); @@ -2276,11 +2277,20 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre thread_context->thread_info.et_thread_index = thread_index; set_up_logging(thread_context); - assert(thread_index == 0); + /* + * Distribute work to multiple threads: + * Each thread work on fs->group_desc_count / nthread groups. + */ tinfo = &thread_context->thread_info; - tinfo->et_group_start = 0; - tinfo->et_group_next = 0; - tinfo->et_group_end = thread_fs->group_desc_count; + average_group = thread_fs->group_desc_count / num_threads; + if (average_group == 0) + average_group = 1; + tinfo->et_group_start = average_group * thread_index; + if (thread_index == num_threads - 1) + tinfo->et_group_end = thread_fs->group_desc_count; + else + tinfo->et_group_end = average_group * (thread_index + 1); + tinfo->et_group_next = tinfo->et_group_start; *thread_ctx = thread_context; return 0; @@ -2523,7 +2533,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, for (i = 0; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; - retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i); + retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, + i, num_threads); if (retval) { com_err(global_ctx->program_name, retval, _("while preparing pass1 thread\n")); From patchwork Mon Nov 7 12:21:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700578 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=cUgZQjqN; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VnY5gdNz23lW for ; Mon, 7 Nov 2022 23:25:53 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnY5DVTz4xDK for ; Mon, 7 Nov 2022 23:25:53 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VnY59hPz4xGj; Mon, 7 Nov 2022 23:25:53 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=cUgZQjqN; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VnY54fSz4xDK for ; Mon, 7 Nov 2022 23:25:53 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232095AbiKGMZw (ORCPT ); Mon, 7 Nov 2022 07:25:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232125AbiKGMZg (ORCPT ); Mon, 7 Nov 2022 07:25:36 -0500 Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A94C63D5 for ; Mon, 7 Nov 2022 04:25:35 -0800 (PST) Received: by mail-pg1-x535.google.com with SMTP id 78so10285819pgb.13 for ; Mon, 07 Nov 2022 04:25:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HsYo/9S6H3NL4uLoiJkCUuAX1SB13ttlqeqEDieKiRM=; b=cUgZQjqNN9O8XYOlAeWMTV1F+j4R1eRgi7XAg1ehNxWbNDMWAMZF2CT22610VByuPR DmM+VX3T7EoH9Mx/cQXf+yoaLPgtZ1Q28IvlVH390oSB/hT6yzc1bpc/Ll7uV5dFt65E tLeVKAc6XkHjZg7BlN3Jr1sRSkRXInH99qLGtfwMUTBftA1RMfXU8CAGJyNCclP2zoWV NUC0bPDLloQOw48IGRhNA+IJ5c5pQq+i92QxNSi5q+aKb0BVOY+VOQrDqGC7/EETK3B/ LKTYFDuOKNtvNDs0851JxkvE12RuTnahye/IsHv3gF2n5Zrd1d4eLl5Jdn1mhY86lwCt E1Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HsYo/9S6H3NL4uLoiJkCUuAX1SB13ttlqeqEDieKiRM=; b=2P1e6X6q29tEydhKr/oktzzxM+LrxVeYf7J9BR1dKOIF837LDsVvjMXj4iXTaMMR+v fQbf92bPtzE+Ae1lUyxwZMZxEPKp6a0fb/PDc7VLcX2OrlMdQBYmJq1NECTkVSxbZMMW zTXY398yGNKLpwRpktb8LCDsKO4JXrUr2MmMVNsj7d86o9Quggz9ClKclMXAlz8JR0uW /u08Mnf698zZ1h2zJKwDSkx+6EvbCWUndqYodfrIw+2KXRCzcVb6BcW4egYofHn1n65H 5RFGGNondaEQmrbcxDUlZV9jKj7O2XjSdXuymmMRPePgTpRGMwIjgweubwknnXlsTwJr Ea8w== X-Gm-Message-State: ACrzQf112uImYbKYzIyLk2nwT3zmfVFHdsYTHjoKRvAdMrPVUySraPNg pwADXBCj4Gm3GHWSZI4j7uUMELidllk= X-Google-Smtp-Source: AMsMyM40dL1HP3mM3LUVOSBIf+tGrObANtTeqF9qo+SDH8N96ikIQVYIl1qNEajUj0D9OpAiPSA8lA== X-Received: by 2002:a63:595:0:b0:470:8b7:255a with SMTP id 143-20020a630595000000b0047008b7255amr24127496pgf.329.1667823934904; Mon, 07 Nov 2022 04:25:34 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y65-20020a623244000000b0056262811c5fsm4361303pfy.59.2022.11.07.04.25.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:34 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 31/72] e2fsck: print thread log properly Date: Mon, 7 Nov 2022 17:51:19 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi When multi-thread fsck is enabled, logs printed from multiple threads could overlap with each other. The overlap sometimes makes the logs unreadable because log_out() is used multiple times for a single line. This patch adds leading [Thread XXX] to each logs if multi-thread is enabed by -m option. This patch also adds message to show the group ranges and inode numbers for each thread, which is useful for debuging multi-thread check. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 4 ++ e2fsck/pass1.c | 21 ++++++++++ e2fsck/problem.c | 6 +++ e2fsck/util.c | 61 ++++++++++++++++++++++++++-- tests/f_multithread/expect.1 | 4 +- tests/f_multithread_logfile/expect.1 | 4 +- tests/f_multithread_no/expect.1 | 4 +- 7 files changed, 98 insertions(+), 6 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 639f4e80..cdd158cc 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -274,6 +274,10 @@ struct e2fsck_thread { dgrp_t et_group_end; /* The next group number to check */ dgrp_t et_group_next; + /* Scanned inode number */ + ext2_ino_t et_inode_number; + char et_log_buf[2048]; + char et_log_length; }; #endif diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 3b411b70..a2c13be5 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1460,6 +1460,10 @@ void e2fsck_pass1_run(e2fsck_t ctx) } if (!ino) break; +#ifdef HAVE_PTHREAD + if (ctx->global_ctx) + ctx->thread_info.et_inode_number++; +#endif pctx.ino = ino; pctx.inode = inode; ctx->stashed_ino = ino; @@ -2292,6 +2296,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre tinfo->et_group_end = average_group * (thread_index + 1); tinfo->et_group_next = tinfo->et_group_start; + tinfo->et_inode_number = 0; + tinfo->et_log_buf[0] = '\0'; + tinfo->et_log_length = 0; + if (thread_context->options & E2F_OPT_MULTITHREAD) + log_out(thread_context, _("Scan group range [%d, %d)\n"), + tinfo->et_group_start, tinfo->et_group_end); *thread_ctx = thread_context; return 0; out_fs: @@ -2348,6 +2358,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; + global_ctx->global_ctx = NULL; global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; @@ -2500,6 +2511,12 @@ static void *e2fsck_pass1_thread(void *arg) e2fsck_pass1_run(thread_ctx); out: + if (thread_ctx->options & E2F_OPT_MULTITHREAD) + log_out(thread_ctx, + _("Scanned group range [%lu, %lu), inodes %lu\n"), + thread_ctx->thread_info.et_group_start, + thread_ctx->thread_info.et_group_end, + thread_ctx->thread_info.et_inode_number); return NULL; } @@ -2635,6 +2652,10 @@ static errcode_t scan_callback(ext2_filsys fs, if (ctx->global_ctx) { tinfo = &ctx->thread_info; tinfo->et_group_next++; + if (ctx->options & E2F_OPT_DEBUG && + ctx->options & E2F_OPT_MULTITHREAD) + log_out(ctx, _("group %d finished\n"), + tinfo->et_group_next); if (tinfo->et_group_next >= tinfo->et_group_end) return EXT2_ET_SCAN_FINISHED; } diff --git a/e2fsck/problem.c b/e2fsck/problem.c index d5452441..9ee3914e 100644 --- a/e2fsck/problem.c +++ b/e2fsck/problem.c @@ -2594,6 +2594,12 @@ int fix_problem(e2fsck_t ctx, problem_t code, struct problem_context *pctx) if (*message) message = _(message); if (!suppress) { +#ifdef HAVE_PTHREAD + if ((ctx->options & E2F_OPT_MULTITHREAD) && ctx->global_ctx) + printf("[Thread %d] ", + ctx->thread_info.et_thread_index); +#endif + if ((ctx->options & E2F_OPT_PREEN) && !(ptr->flags & PR_PREEN_NOHDR)) { printf("%s: ", ctx->device_name ? diff --git a/e2fsck/util.c b/e2fsck/util.c index 42740d9e..254b4d04 100644 --- a/e2fsck/util.c +++ b/e2fsck/util.c @@ -11,6 +11,7 @@ #include "config.h" #include +#include #include #include #include @@ -84,13 +85,67 @@ out: exit(exit_value); } +#ifdef HAVE_PTHREAD +static void thread_log_out(struct e2fsck_thread *tinfo) +{ + printf("[Thread %d] %s", tinfo->et_thread_index, + tinfo->et_log_buf); + tinfo->et_log_length = 0; + tinfo->et_log_buf[0] = '\0'; +} +#endif + void log_out(e2fsck_t ctx, const char *fmt, ...) { va_list pvar; + struct e2fsck_thread *tinfo; + int buf_size; + int msg_size; + int left_size; + int fmt_length = strlen(fmt); + +#ifdef HAVE_PTHREAD + if ((ctx->options & E2F_OPT_MULTITHREAD) && ctx->global_ctx) { + tinfo = &ctx->thread_info; + buf_size = sizeof(tinfo->et_log_buf); + left_size = buf_size - tinfo->et_log_length; + + va_start(pvar, fmt); + msg_size = vsnprintf(tinfo->et_log_buf + tinfo->et_log_length, + left_size, fmt, pvar); + va_end(pvar); + + if (msg_size >= left_size) { + tinfo->et_log_buf[tinfo->et_log_length] = '\0'; + + assert(msg_size < buf_size); + if (msg_size < buf_size) { + thread_log_out(tinfo); + + va_start(pvar, fmt); + msg_size = vsnprintf(tinfo->et_log_buf, buf_size, + fmt, pvar); + va_end(pvar); + + tinfo->et_log_length += msg_size; + tinfo->et_log_buf[tinfo->et_log_length] = '\0'; + } + } else { + tinfo->et_log_length += msg_size; + tinfo->et_log_buf[tinfo->et_log_length] = '\0'; + } + + if (tinfo->et_log_length > 0 && + tinfo->et_log_buf[tinfo->et_log_length - 1] == '\n') + thread_log_out(tinfo); + } else +#endif + { + va_start(pvar, fmt); + vprintf(fmt, pvar); + va_end(pvar); + } - va_start(pvar, fmt); - vprintf(fmt, pvar); - va_end(pvar); if (ctx->logf) { va_start(pvar, fmt); vfprintf(ctx->logf, fmt, pvar); diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1 index e2b954d0..8d2acd2b 100644 --- a/tests/f_multithread/expect.1 +++ b/tests/f_multithread/expect.1 @@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1 index e2b954d0..8d2acd2b 100644 --- a/tests/f_multithread_logfile/expect.1 +++ b/tests/f_multithread_logfile/expect.1 @@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1 index d14c4083..f85a3382 100644 --- a/tests/f_multithread_no/expect.1 +++ b/tests/f_multithread_no/expect.1 @@ -1,6 +1,8 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... -Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 2) +[Thread 0] Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts From patchwork Mon Nov 7 12:21:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700579 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=q6KXSlAb; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vnb6yNyz23lx for ; Mon, 7 Nov 2022 23:25:55 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnb6XVNz4xDK for ; Mon, 7 Nov 2022 23:25:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vnb6TjCz4xGj; Mon, 7 Nov 2022 23:25:55 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=q6KXSlAb; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnb6914z4xDK for ; Mon, 7 Nov 2022 23:25:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232208AbiKGMZy (ORCPT ); Mon, 7 Nov 2022 07:25:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232206AbiKGMZm (ORCPT ); Mon, 7 Nov 2022 07:25:42 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D3771A816 for ; Mon, 7 Nov 2022 04:25:41 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id p21so10902231plr.7 for ; Mon, 07 Nov 2022 04:25:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=prsFAPKtax3VzoeqbbBhMC8ROHOAEyX3jB6qXAVQ/Ww=; b=q6KXSlAbnp/SZymZh8CcnO9paKWchqYLFhBRKGvtbQcU+gpZ01h1eAfMzKkMTwYoLK SDAYVNKzwe4NFc91bw0Flr8aSW9yg1PJ/jSxC3TvPqgLBHKPfCA8M0NwUSwJCIZtW+py X+5hgTugEi2uVkgCDpL+SkPUP37RQ18LlU7hsnB7xWk6b0QvY5umfyFhbhR6vrbkCTCP IOyDCRtpvJjjsFEvTXf3UzFB35anf76tju4c/pVIKUn/HbxYjohxrMD+ahMutZ303Kgc rQplKoPHnWa1mT0hLt23rEd9DibigGpbXg/m2TR8ApDe6TxUFl+s9HCX1fl+UGTo+xBv XfcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=prsFAPKtax3VzoeqbbBhMC8ROHOAEyX3jB6qXAVQ/Ww=; b=uqdnd1L6JsDEvwYJ3WU02fKoLbmExddbkuEoCeaUPIAAHSK3bCmXBHVWxIPdHrRqUH 8Xz7GjW27EdDi6U7H3otxlTBF3CvaA1WeB75KaioKffE2HNwOeZOS3zooqiMbbB9KtM7 rDJ1/b3dCvxkai/iHzN+vSUqkSWgP0BCgZGq2LtMU+iztOOAo3JFsoyO7D2HSIsfYTGK ZV11/+yKFCoF9DMppq/YOaa5RsbUCIIZp/HU8AcgT2zkQVOQV/oQ0TeLRyqdI401Cng1 0stgOWkrOIZgXS2gO1byYAtoKOjEWU718wv/OztM0dNFL5RQ8rVM67O0b21/Q91qmRxi UupQ== X-Gm-Message-State: ACrzQf06qTE5fW9A6qx/Np3K8LK7yN6WEUoSfzBxcDTk3mRSgczOtG79 bhFQWk/oq9N5OqKTj6nJTlc= X-Google-Smtp-Source: AMsMyM7Dugk9K4i1Uxdwd3xIbXa8aHv3y+bc07hynd8wx/KLrBD14oY7IqkqcsAkW6KTgVsauGVLbg== X-Received: by 2002:a17:903:124e:b0:179:da2f:244e with SMTP id u14-20020a170903124e00b00179da2f244emr51059882plh.169.1667823940839; Mon, 07 Nov 2022 04:25:40 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id x11-20020aa7956b000000b0056bcd7e1e04sm4339922pfq.124.2022.11.07.04.25.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:40 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 32/72] e2fsck: do not change global variables Date: Mon, 7 Nov 2022 17:51:20 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Global variables used in pass1 check are changed to local variables in this patch. This will avoid conflict between threads. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 75 +++++++++++++++++++++++++++++++------------------- 1 file changed, 47 insertions(+), 28 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index a2c13be5..d5c01dc7 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -86,7 +86,6 @@ static void alloc_imagic_map(e2fsck_t ctx); static void mark_inode_bad(e2fsck_t ctx, ext2_ino_t ino); static void add_casefolded_dir(e2fsck_t ctx, ext2_ino_t ino); static void handle_fs_bad_blocks(e2fsck_t ctx); -static void process_inodes(e2fsck_t ctx, char *block_buf); static EXT2_QSORT_TYPE process_inode_cmp(const void *a, const void *b); static errcode_t scan_callback(ext2_filsys fs, ext2_inode_scan scan, dgrp_t group, void * priv_data); @@ -121,15 +120,15 @@ struct process_inode_block { }; struct scan_callback_struct { - e2fsck_t ctx; - char *block_buf; + e2fsck_t ctx; + char *block_buf; + struct process_inode_block *inodes_to_process; + int *process_inode_count; }; -/* - * For the inodes to process list. - */ -static struct process_inode_block *inodes_to_process; -static int process_inode_count; +static void process_inodes(e2fsck_t ctx, char *block_buf, + struct process_inode_block *inodes_to_process, + int *process_inode_count); static __u64 ext2_max_sizes[EXT2_MAX_BLOCK_LOG_SIZE - EXT2_MIN_BLOCK_LOG_SIZE + 1]; @@ -1173,7 +1172,6 @@ static int e2fsck_should_abort(e2fsck_t ctx) void e2fsck_pass1_run(e2fsck_t ctx) { int i; - __u64 max_sizes; ext2_filsys fs = ctx->fs; ext2_ino_t ino = 0; struct ext2_inode *inode = NULL; @@ -1196,6 +1194,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) ext2_ino_t ino_threshold = 0; dgrp_t ra_group = 0; struct ea_quota ea_ibody_quota; + struct process_inode_block *inodes_to_process; + int process_inode_count; init_resource_track(&rtrack, ctx->fs->io); clear_problem_context(&pctx); @@ -1220,17 +1220,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) mtrace_print("Pass 1"); #endif -#define EXT2_BPP(bits) (1ULL << ((bits) - 2)) - - for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) { - max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i); - max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i); - max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i); - max_sizes = (max_sizes * (1UL << i)); - ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes; - } -#undef EXT2_BPP - imagic_fs = ext2fs_has_feature_imagic_inodes(sb); extent_fs = ext2fs_has_feature_extents(sb); inlinedata_fs = ext2fs_has_feature_inline_data(sb); @@ -1368,6 +1357,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->stashed_inode = inode; scan_struct.ctx = ctx; scan_struct.block_buf = block_buf; + scan_struct.inodes_to_process = inodes_to_process; + scan_struct.process_inode_count = &process_inode_count; ext2fs_set_inode_callback(scan, scan_callback, &scan_struct); if (ctx->progress && ((ctx->progress)(ctx, 1, 0, ctx->fs->group_desc_count))) @@ -2048,13 +2039,15 @@ void e2fsck_pass1_run(e2fsck_t ctx) goto endit; if (process_inode_count >= ctx->process_inode_size) { - process_inodes(ctx, block_buf); + process_inodes(ctx, block_buf, inodes_to_process, + &process_inode_count); if (e2fsck_should_abort(ctx)) goto endit; } } - process_inodes(ctx, block_buf); + process_inodes(ctx, block_buf, inodes_to_process, + &process_inode_count); ext2fs_close_inode_scan(scan); scan = NULL; @@ -2177,6 +2170,27 @@ endit: ctx->invalid_bitmaps++; } +static void init_ext2_max_sizes() +{ + int i; + __u64 max_sizes; + + /* + * Init ext2_max_sizes which will be immutable and shared between + * threads + */ +#define EXT2_BPP(bits) (1ULL << ((bits) - 2)) + + for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) { + max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i); + max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i); + max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i); + max_sizes = (max_sizes * (1UL << i)); + ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes; + } +#undef EXT2_BPP +} + #ifdef HAVE_PTHREAD static errcode_t e2fsck_pass1_merge_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src, ext2fs_generic_bitmap *dest) @@ -2616,6 +2630,7 @@ out_abort: void e2fsck_pass1(e2fsck_t ctx) { + init_ext2_max_sizes(); #ifdef HAVE_PTHREAD if (ctx->options & E2F_OPT_MULTITHREAD) e2fsck_pass1_multithread(ctx); @@ -2641,7 +2656,9 @@ static errcode_t scan_callback(ext2_filsys fs, scan_struct = (struct scan_callback_struct *) priv_data; ctx = scan_struct->ctx; - process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf); + process_inodes((e2fsck_t) fs->priv_data, scan_struct->block_buf, + scan_struct->inodes_to_process, + scan_struct->process_inode_count); if (ctx->progress) if ((ctx->progress)(ctx, 1, group+1, @@ -2667,7 +2684,9 @@ static errcode_t scan_callback(ext2_filsys fs, /* * Process the inodes in the "inodes to process" list. */ -static void process_inodes(e2fsck_t ctx, char *block_buf) +static void process_inodes(e2fsck_t ctx, char *block_buf, + struct process_inode_block *inodes_to_process, + int *process_inode_count) { int i; struct ext2_inode *old_stashed_inode; @@ -2679,15 +2698,15 @@ static void process_inodes(e2fsck_t ctx, char *block_buf) #if 0 printf("begin process_inodes: "); #endif - if (process_inode_count == 0) + if (*process_inode_count == 0) return; old_operation = ehandler_operation(0); old_stashed_inode = ctx->stashed_inode; old_stashed_ino = ctx->stashed_ino; - qsort(inodes_to_process, process_inode_count, + qsort(inodes_to_process, *process_inode_count, sizeof(struct process_inode_block), process_inode_cmp); clear_problem_context(&pctx); - for (i=0; i < process_inode_count; i++) { + for (i=0; i < *process_inode_count; i++) { pctx.inode = ctx->stashed_inode = (struct ext2_inode *) &inodes_to_process[i].inode; pctx.ino = ctx->stashed_ino = inodes_to_process[i].ino; @@ -2705,7 +2724,7 @@ static void process_inodes(e2fsck_t ctx, char *block_buf) } ctx->stashed_inode = old_stashed_inode; ctx->stashed_ino = old_stashed_ino; - process_inode_count = 0; + *process_inode_count = 0; #if 0 printf("end process inodes\n"); #endif From patchwork Mon Nov 7 12:21:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700580 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=er3aEvYP; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vnh497fz23lW for ; Mon, 7 Nov 2022 23:26:00 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnh3jmNz4xDK for ; Mon, 7 Nov 2022 23:26:00 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vnh3drtz4xGj; Mon, 7 Nov 2022 23:26:00 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=er3aEvYP; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnh3VGSz4xDK for ; Mon, 7 Nov 2022 23:26:00 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232153AbiKGMZ7 (ORCPT ); Mon, 7 Nov 2022 07:25:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232216AbiKGMZs (ORCPT ); Mon, 7 Nov 2022 07:25:48 -0500 Received: from mail-pg1-x52e.google.com (mail-pg1-x52e.google.com [IPv6:2607:f8b0:4864:20::52e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64FE81B7B0 for ; Mon, 7 Nov 2022 04:25:47 -0800 (PST) Received: by mail-pg1-x52e.google.com with SMTP id b62so10343361pgc.0 for ; Mon, 07 Nov 2022 04:25:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mH41vIADdp8GYBWaniI1xQi/8RWi01qwqHeYVy9cqW8=; b=er3aEvYPolTY4NH1Hckj4KznF3Ssal87y5sDo7wT3LkS/E1EbkRkq6EnTKhm3dJdg/ JyMRisY1Ev/92wOv6N/T/BJn0mf6AE0Ia4i0S4hKGokrm4d2vtaf0QxfxqP6Zu10dVRT kdsGMa4otqTA6E8sJDwU5SgUALbdD7nntmheV8ZkYxPukbum0th9O18MriVPV3e5IXuf xN1k80UE7rp/OdyRGV3OAC1PiMaoAT+uFmkxIKYYPpbbMkM1LwG8yRthBwKXaPCAuTJG mSOp4nRu1uhj/8WNT5n3FM2ZyoiA/kQGMy068qJ//0pmAcnYW/uJaGu1Rv/7m1wwqhn9 4nUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mH41vIADdp8GYBWaniI1xQi/8RWi01qwqHeYVy9cqW8=; b=69xzIZTuS6jUBxDNJsUg/o6eNyixz1tlPcoirCzYAdqNDyp0p6E4MNVtjf4r7q4BA2 i4Bru/AZXVvWoGNKX4mbwjbSLc+ueJ8l4TpG4BWeRyns0FaCsFPQhzEgrwV2jazCrHko iIh8cdrdB9kiKCSJ9BppiRn2yPZd0KuE7ezfKoXQNWct9hXU8FgH2fApD0QlsTQ9KAkW RQt6onAoHJywo+VRvfIB5yD7dzX/Igjpl8gowGDt12A8AVPpN2l1hcZMyAG5AjhJqjki gxtEObN5uuBXJQ9NH612Qqvp2qVGMPyn7A0Gx6K9IcFp3gArSJZ3ercFRrYZPIoVzAt6 HctQ== X-Gm-Message-State: ACrzQf2YgD00KPL3dFRyBKg/pp4r2GcLwdpTFl4hZ387CgI8dXQ5qSez eBSQ2/ab9i4PDoX0+FA/AP0= X-Google-Smtp-Source: AMsMyM7CGJfNt0VTMJ1xKpvMWYmvOSVXBzxQU/Wq0cZcqlxf2byfpkhp1kfpfpD4pyijtZfcpuUAZg== X-Received: by 2002:a05:6a00:1749:b0:56e:4586:4bc1 with SMTP id j9-20020a056a00174900b0056e45864bc1mr23091079pfc.41.1667823946802; Mon, 07 Nov 2022 04:25:46 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id z11-20020aa79e4b000000b0056e8eb09d57sm4377459pfq.63.2022.11.07.04.25.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:46 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 33/72] e2fsck: optimize the inserting of dir_info_db Date: Mon, 7 Nov 2022 17:51:21 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Binary search is now used when inserting an dir info to the array. Memmove is now used when moving array. Both of them improves the performance of inserting. This patch is also a prepartion for the merging of two dir db arrays. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/dirinfo.c | 172 ++++++++++++++++++++++++++++++----------------- 1 file changed, 112 insertions(+), 60 deletions(-) diff --git a/e2fsck/dirinfo.c b/e2fsck/dirinfo.c index 49d624c5..5c360a90 100644 --- a/e2fsck/dirinfo.c +++ b/e2fsck/dirinfo.c @@ -7,6 +7,7 @@ #undef DIRINFO_DEBUG +#include #include "config.h" #include "e2fsck.h" #include @@ -122,6 +123,104 @@ static void setup_db(e2fsck_t ctx) "directory map"); } +/* + * Return the min index that has ino larger or equal to @ino + * If not found, return -ENOENT + */ +static int +e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info, + ext2_ino_t ino, ext2_ino_t *index) +{ + ext2_ino_t low = 0; + ext2_ino_t mid, high; + ext2_ino_t tmp_ino; + int found = 0; + + if (dir_info->count == 0) + return -ENOENT; + + high = dir_info->count - 1; + while (low <= high) { + /* sum may overflow, but result will fit into mid again */ + mid = (unsigned long long)(low + high) / 2; + tmp_ino = dir_info->array[mid].ino; + if (ino == tmp_ino) { + *index = mid; + found = 1; + return 0; + } else if (ino < tmp_ino) { + /* + * The mid ino is larger than @ino, remember the index + * here so we won't miss this ino + */ + *index = mid; + found = 1; + if (mid == 0) + break; + high = mid - 1; + } else { + low = mid + 1; + } + } + + if (found) + return 0; + + return -ENOENT; +} + +/* + * + * Insert an inode into the sorted array. The array should have at least one + * free slot. + * + * Normally, add_dir_info is called with each inode in + * sequential order; but once in a while (like when pass 3 + * needs to recreate the root directory or lost+found + * directory) it is called out of order. In those cases, we + * need to move the dir_info entries down to make room, since + * the dir_info array needs to be sorted by inode number for + * get_dir_info()'s sake. + */ +static void e2fsck_insert_dir_info(struct dir_info_db *dir_info, ext2_ino_t ino, ext2_ino_t parent) +{ + ext2_ino_t index; + struct dir_info *dir; + size_t dir_size = sizeof(*dir); + struct dir_info *array = dir_info->array; + ext2_ino_t array_count = dir_info->count; + int err; + + /* + * Removing this check won't break anything. But since seqential ino + * inserting happens a lot, this check avoids binary search. + */ + if (array_count == 0 || array[array_count - 1].ino < ino) { + dir = &array[array_count]; + dir_info->count++; + goto out; + } + + err = e2fsck_dir_info_min_larger_equal(dir_info, ino, &index); + if (err >= 0 && array[index].ino == ino) { + dir = &array[index]; + goto out; + } + if (err < 0) { + dir = &array[array_count]; + dir_info->count++; + goto out; + } + + dir = &array[index]; + memmove((char *)dir + dir_size, dir, dir_size * (array_count - index)); + dir_info->count++; +out: + dir->ino = ino; + dir->dotdot = parent; + dir->parent = parent; +} + /* * This subroutine is called during pass1 to create a directory info * entry. During pass1, the passed-in parent is 0; it will get filled @@ -171,30 +270,7 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent) } #endif - /* - * Normally, add_dir_info is called with each inode in - * sequential order; but once in a while (like when pass 3 - * needs to recreate the root directory or lost+found - * directory) it is called out of order. In those cases, we - * need to move the dir_info entries down to make room, since - * the dir_info array needs to be sorted by inode number for - * get_dir_info()'s sake. - */ - if (ctx->dir_info->count && - ctx->dir_info->array[ctx->dir_info->count-1].ino >= ino) { - for (i = ctx->dir_info->count-1; i > 0; i--) - if (ctx->dir_info->array[i-1].ino < ino) - break; - dir = &ctx->dir_info->array[i]; - if (dir->ino != ino) - for (j = ctx->dir_info->count++; j > i; j--) - ctx->dir_info->array[j] = ctx->dir_info->array[j-1]; - } else - dir = &ctx->dir_info->array[ctx->dir_info->count++]; - - dir->ino = ino; - dir->dotdot = parent; - dir->parent = parent; + e2fsck_insert_dir_info(ctx->dir_info, ino, parent); } /* @@ -204,7 +280,8 @@ void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent) static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino) { struct dir_info_db *db = ctx->dir_info; - ext2_ino_t low, high, mid; + ext2_ino_t index; + int err; if (!db) return 0; @@ -245,44 +322,19 @@ static struct dir_info *e2fsck_get_dir_info(e2fsck_t ctx, ext2_ino_t ino) if (db->last_lookup && db->last_lookup->ino == ino) return db->last_lookup; - low = 0; - high = ctx->dir_info->count - 1; - if (ino == ctx->dir_info->array[low].ino) { + err = e2fsck_dir_info_min_larger_equal(ctx->dir_info, ino, &index); + if (err < 0) + return NULL; + assert(ino <= ctx->dir_info->array[index].ino); + if (ino == ctx->dir_info->array[index].ino) { #ifdef DIRINFO_DEBUG - printf("(%u,%u,%u)\n", ino, - ctx->dir_info->array[low].dotdot, - ctx->dir_info->array[low].parent); + printf("(%d,%d,%d)\n", ino, + ctx->dir_info->array[index].dotdot, + ctx->dir_info->array[index].parent); #endif - return &ctx->dir_info->array[low]; + return &ctx->dir_info->array[index]; } - if (ino == ctx->dir_info->array[high].ino) { -#ifdef DIRINFO_DEBUG - printf("(%u,%u,%u)\n", ino, - ctx->dir_info->array[high].dotdot, - ctx->dir_info->array[high].parent); -#endif - return &ctx->dir_info->array[high]; - } - - while (low < high) { - /* sum may overflow, but result will fit into mid again */ - mid = (unsigned long long)(low + high) / 2; - if (mid == low || mid == high) - break; - if (ino == ctx->dir_info->array[mid].ino) { -#ifdef DIRINFO_DEBUG - printf("(%u,%u,%u)\n", ino, - ctx->dir_info->array[mid].dotdot, - ctx->dir_info->array[mid].parent); -#endif - return &ctx->dir_info->array[mid]; - } - if (ino < ctx->dir_info->array[mid].ino) - high = mid; - else - low = mid; - } - return 0; + return NULL; } static void e2fsck_put_dir_info(e2fsck_t ctx EXT2FS_NO_TDB_UNUSED, From patchwork Mon Nov 7 12:21:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700581 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nIAk90/P; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vnq4d68z23lW for ; Mon, 7 Nov 2022 23:26:07 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnq49hnz4xDK for ; Mon, 7 Nov 2022 23:26:07 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vnq47L8z4xGj; Mon, 7 Nov 2022 23:26:07 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=nIAk90/P; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnq3qNfz4xDK for ; Mon, 7 Nov 2022 23:26:07 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232222AbiKGM0G (ORCPT ); Mon, 7 Nov 2022 07:26:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51518 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232136AbiKGMZx (ORCPT ); Mon, 7 Nov 2022 07:25:53 -0500 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 223701B9E6 for ; Mon, 7 Nov 2022 04:25:53 -0800 (PST) Received: by mail-pj1-x1033.google.com with SMTP id b11so10416344pjp.2 for ; Mon, 07 Nov 2022 04:25:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i0/CK5a9TPR66DPtC901kFn8FeF4wHzokbGENRJGLC8=; b=nIAk90/PYkpri0HOXZgT1CNhCjgup5LEpsA6UrZN+LTLX0fwKeUGICWnj8MVaWrer7 wCCyXXNVBnzegxSRLmGBfEiUH6Cjyy75oLSQEv5Q7Kotg30Zl6MECq0L3dbo5kiYmlhY 5feBcdzp64Skq4yH8v9srdf/nrDHgnE6g61N1udBJm4HGyQK46Edx5R7IwwAQAwPLoK3 NWUo15QicbD3VEMZv+nWBPCUHULK3lkmZenmfYsJkRRXUzirLqRDZtYBCwUBc/+G8rXN CAqjIUhBlDVKnHos35bz4kud4kMWq8xpks966Odog2FfWvG4eiS8IwhRNLIwt1c8vhKr zQhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i0/CK5a9TPR66DPtC901kFn8FeF4wHzokbGENRJGLC8=; b=a3/nvpmN/H+bjXOud0x2CQHJ/8pMTX9K/fnLEI6vIQWegsbi5YytRQRRoBokW+L+n3 WJ4bJF/gqxs9wOg9ZINqHWdf3C1XCLsDKccic1NHGvs0in8cp9rrBCPbZDnWVABvc23J T+G8uG7oG9B6XCy7KBnYtrXynGzIFzFMydW+lDwxG4nUixHTJXV+qHubnQ7JGynnCar3 uPfjAhJrnDzSP/NR/tpFIJyjylJLKtLXEiYFPo1B15tVp0DBtVCITJzAEkKTXlzUQQ/c 4dazt7FaYcFSUZQ8qzNYQ7DRL0HH4ZdriuW/fPoNz8CGcaK9YBi8lcCkcWtEmjQFhu0k MD4w== X-Gm-Message-State: ACrzQf3JpiqdpaATZ4gucjVSsyvVM4hhOyCR15z2T3prhIQ0GIQ50HgH L/wYkmTUL5ScE35zJzWnevM= X-Google-Smtp-Source: AMsMyM5uG9jUJ1g5swt3MnWgTTv4XjVOcqFG9vsA8fYK0Rd3MSKZjM3OkCULkdTyznShj87iLbd74g== X-Received: by 2002:a17:902:e745:b0:187:2033:1832 with SMTP id p5-20020a170902e74500b0018720331832mr42125914plf.119.1667823952630; Mon, 07 Nov 2022 04:25:52 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id k31-20020a635a5f000000b00438834b14a1sm4062783pgm.80.2022.11.07.04.25.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:52 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 34/72] e2fsck: merge dir_info after thread finishes Date: Mon, 7 Nov 2022 17:51:22 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi dir_info need be merged after thread finish. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/dirinfo.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++ e2fsck/e2fsck.h | 2 ++ e2fsck/pass1.c | 53 +++++++++++++++++++++++++++++++++++++- 3 files changed, 120 insertions(+), 1 deletion(-) diff --git a/e2fsck/dirinfo.c b/e2fsck/dirinfo.c index 5c360a90..ee8d8a69 100644 --- a/e2fsck/dirinfo.c +++ b/e2fsck/dirinfo.c @@ -169,6 +169,72 @@ e2fsck_dir_info_min_larger_equal(struct dir_info_db *dir_info, return -ENOENT; } +/* + * Merge two sorted dir info to @dest + */ +void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src, + struct dir_info_db *dest) +{ + size_t size_dir_info = sizeof(struct dir_info); + ext2_ino_t size = dest->size; + struct dir_info *src_array = src->array; + struct dir_info *dest_array = dest->array; + ext2_ino_t src_count = src->count; + ext2_ino_t dest_count = dest->count; + ext2_ino_t total_count = src_count + dest_count; + struct dir_info *tmp_array; + struct dir_info *array_ptr; + ext2_ino_t src_index = 0; + ext2_ino_t dest_index = 0; + + if (src->count == 0) + return; + + if (size < total_count) + size = total_count; + + if (size < src->size) + size = src->size; + + tmp_array = e2fsck_allocate_memory(ctx, size * size_dir_info, + "directory map"); + array_ptr = tmp_array; + /* + * This can be improved by binary search and memcpy, but codes + * would be more complex. And if the groups distributed to each + * thread are strided, this implementation won't be too bad + * comparing to the optimiztion. + */ + while (src_index < src_count || dest_index < dest_count) { + if (src_index >= src_count) { + memcpy(array_ptr, &dest_array[dest_index], + (dest_count - dest_index) * size_dir_info); + break; + } + if (dest_index >= dest_count) { + memcpy(array_ptr, &src_array[src_index], + (src_count - src_index) * size_dir_info); + break; + } + if (src_array[src_index].ino < dest_array[dest_index].ino) { + *array_ptr = src_array[src_index]; + src_index++; + } else { + assert(src_array[src_index].ino > + dest_array[dest_index].ino); + *array_ptr = dest_array[dest_index]; + dest_index++; + } + array_ptr++; + } + + if (dest->array) + ext2fs_free_mem(&dest->array); + dest->array = tmp_array; + dest->size = size; + dest->count = total_count; +} + /* * * Insert an inode into the sorted array. The array should have at least one diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index cdd158cc..2ee37f78 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -546,6 +546,8 @@ extern void read_bad_blocks_file(e2fsck_t ctx, const char *bad_blocks_file, /* dirinfo.c */ extern void e2fsck_add_dir_info(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t parent); +void e2fsck_merge_dir_info(e2fsck_t ctx, struct dir_info_db *src, + struct dir_info_db *dest); extern void e2fsck_free_dir_info(e2fsck_t ctx); extern int e2fsck_get_num_dirinfo(e2fsck_t ctx); extern struct dir_info_iter *e2fsck_dir_info_iter_begin(e2fsck_t ctx); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index d5c01dc7..57003d8c 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2325,6 +2325,21 @@ out_context: return retval; } +static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + if (thread_ctx->dir_info == NULL) + return; + + if (global_ctx->dir_info == NULL) { + global_ctx->dir_info = thread_ctx->dir_info; + thread_ctx->dir_info = NULL; + return; + } + + e2fsck_merge_dir_info(global_ctx, thread_ctx->dir_info, + global_ctx->dir_info); +} + static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) { errcode_t retval = 0; @@ -2334,6 +2349,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_filsys thread_fs = thread_ctx->fs; ext2_filsys global_fs = global_ctx->fs; ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; + struct dir_info_db *dir_info = global_ctx->dir_info; ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; @@ -2366,6 +2382,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->block_dup_map = block_dup_map; global_ctx->block_ea_map = block_ea_map; global_ctx->block_metadata_map = block_metadata_map; + global_ctx->dir_info = dir_info; + e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); /* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | @@ -2458,6 +2476,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_pass1_free_bitmap(&thread_ctx->block_dup_map); e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map); + e2fsck_free_dir_info(thread_ctx); if (thread_ctx->logf) fclose(thread_ctx->logf); @@ -2628,11 +2647,43 @@ out_abort: } #endif +/* TODO: tdb needs to be handled properly for multiple threads*/ +static int multiple_threads_supported(e2fsck_t ctx) +{ +#ifdef CONFIG_TDB + unsigned int threshold; + ext2_ino_t num_dirs; + errcode_t retval; + char *tdb_dir; + int enable; + + profile_get_string(ctx->profile, "scratch_files", "directory", 0, 0, + &tdb_dir); + profile_get_uint(ctx->profile, "scratch_files", + "numdirs_threshold", 0, 0, &threshold); + profile_get_boolean(ctx->profile, "scratch_files", + "icount", 0, 1, &enable); + + retval = ext2fs_get_num_dirs(ctx->fs, &num_dirs); + if (retval) + num_dirs = 1024; /* Guess */ + + /* tdb is unsupported now */ + if (enable && tdb_dir && !access(tdb_dir, W_OK) && + (!threshold || num_dirs > threshold)) { + fprintf(stderr, "Fall through single thread for pass1 " + "because tdb could not handle properly\n"); + return 0; + } + #endif + return 1; +} + void e2fsck_pass1(e2fsck_t ctx) { init_ext2_max_sizes(); #ifdef HAVE_PTHREAD - if (ctx->options & E2F_OPT_MULTITHREAD) + if (ctx->options & E2F_OPT_MULTITHREAD && multiple_threads_supported(ctx)) e2fsck_pass1_multithread(ctx); else #endif From patchwork Mon Nov 7 12:21:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700582 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=hMnHeAnL; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vnw4Fs1z23lW for ; Mon, 7 Nov 2022 23:26:12 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnw3r7Mz4xDK for ; Mon, 7 Nov 2022 23:26:12 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vnw3nw7z4xGj; Mon, 7 Nov 2022 23:26:12 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=hMnHeAnL; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vnw3hWPz4xDK for ; Mon, 7 Nov 2022 23:26:12 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232154AbiKGM0L (ORCPT ); Mon, 7 Nov 2022 07:26:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232212AbiKGM0A (ORCPT ); Mon, 7 Nov 2022 07:26:00 -0500 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0C9763CE for ; Mon, 7 Nov 2022 04:25:58 -0800 (PST) Received: by mail-pj1-x102c.google.com with SMTP id q1-20020a17090a750100b002139ec1e999so10140470pjk.1 for ; Mon, 07 Nov 2022 04:25:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Taw4sY5iZaNdt+izCsMrOPjsHwT7+10DDuDS9/mfrd0=; b=hMnHeAnLb/1T5k3HpFIdAeXr64Ygn9CogYZAJhkTaTxdDj3vOF7PZKbQLNmvbaOWaD WW/h4EAhowzxdZV/N/pT5CoEEf8yTikSGg/crvm7VqSUlbqlv36YedSIMInGGms1DuQy bjsPgRTF1tJ/Ky4sv6oz+jtIhaHbDok2ujHHlq97txgxj/6vuv72mvGKMz2ZJdsIOwlG GyM/oo/XAEL/vpI/CfP8KykjiCGzW6RyRUzMsNscZSb6Cz/QgWHCMxKXkY+bVDKd/ZoP b0TMWlX2SLM0nW2L1HO0h+RfS+AK5ANMeelfYfd89Q0mJabFi7Pj54rEQ3DpoFRlg08C se7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Taw4sY5iZaNdt+izCsMrOPjsHwT7+10DDuDS9/mfrd0=; b=OIxDqChjWeemYATzbsLFG2Y5IeTJ1z6uIaRozz0cYmOyWoFQ7MVulwLNcm9U2BSOLu 8ss6lXZLXMxucb1jrEcUObwF78/ZU3oFbkDtluXu+WF1TdvI467TgR/r9nDP1RRgdVcE uqJ4ZYfaYZOmHukxru6Qv0Q5RKAYwqluZHCj9AVSkr0sF+oYirW/4PAJUSkvcGGxTQnO mF7dveCPZbIQKdLzLCpDjryBUhblanEM+o9JTDvHxccxEfUMUApUMOVe1qwKE91Z+G99 8gheLxdZfoY4EE5DNGDYcykl2RUNSnNBZezIK+cm6BEE9eO8CgG1NpvgQeJRgU8StsUg RXCA== X-Gm-Message-State: ACrzQf13vluevm9lvGDXwCMaSqudVaU9TPFKHK4xmB1MaZZBc8D/7ZHb EoCvDEh7p2x6MBXZ8JJNdFU= X-Google-Smtp-Source: AMsMyM4PJPyVcgI5Pff0U4g6YJSKUyuZR3Tw77jN1PUexor/Erx8vCNri8b7A9eyr8NkcAwEEkzfrA== X-Received: by 2002:a17:903:2445:b0:186:daeb:bc09 with SMTP id l5-20020a170903244500b00186daebbc09mr49875076pls.31.1667823958410; Mon, 07 Nov 2022 04:25:58 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id 21-20020a170902c11500b00174c1855cd9sm4841411pli.267.2022.11.07.04.25.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:25:57 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 35/72] e2fsck: rbtree bitmap for dir Date: Mon, 7 Nov 2022 17:51:23 +0530 Message-Id: <5b724259824b4f395e020389ab698796a1be07dc.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Only rbtree support merge operation now, use it for bitmaps. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 1 + 1 file changed, 1 insertion(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 57003d8c..4d98c467 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1240,6 +1240,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } pctx.errcode = e2fsck_allocate_inode_bitmap(fs, _("directory inode map"), + ctx->global_ctx ? EXT2FS_BMAP64_RBTREE : EXT2FS_BMAP64_AUTODIR, "inode_dir_map", &ctx->inode_dir_map); if (pctx.errcode) { From patchwork Mon Nov 7 12:21:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700583 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Dqmm7yxv; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vny1qnKz23lW for ; Mon, 7 Nov 2022 23:26:14 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vny1N3rz4xDK for ; Mon, 7 Nov 2022 23:26:14 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vny1L91z4xGj; Mon, 7 Nov 2022 23:26:14 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Dqmm7yxv; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vny1FN4z4xDK for ; Mon, 7 Nov 2022 23:26:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232136AbiKGM0N (ORCPT ); Mon, 7 Nov 2022 07:26:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51464 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232221AbiKGM0G (ORCPT ); Mon, 7 Nov 2022 07:26:06 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9FCE1B1C9 for ; Mon, 7 Nov 2022 04:26:04 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id z5-20020a17090a8b8500b00210a3a2364fso12688653pjn.0 for ; Mon, 07 Nov 2022 04:26:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9vO3jPRC7xmlL2kLSJ6JtsnSnyM9UGaAxq/yG02THUg=; b=Dqmm7yxv1BHoOuU66HFITJ4WA5B32+CLnFcyDHNqzQuDfVLqprjPzgOZjzc1wc3WEF p8T1wPTGp8YkLsm/R8oOah8rgp+IF98kun/ZSVLudGPdoQbyCA7zMp5HIlEax/xg6n2X 23sg7cDTj+un4nvMDv1aWm0B7LZp/jvqhds+E+8uF0VSSHJlGP9BjSydlceMMdj4pAQn OLf+bV+aLjTz4G4hQajexsBKdApFlnBfd7trY8BFoRkROzj4wbGcrlgi19+l76DmrdSV 1yqM2EYwTkEti1piO3uMqPivhRQ+GDLYwtu7h1BymefcWl5DxrP8d9Wrks2051O3icmM hadA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9vO3jPRC7xmlL2kLSJ6JtsnSnyM9UGaAxq/yG02THUg=; b=DH0HPHswilq6G2KuA42rmroG/vbqKWIjWuNMUf3JvPm2C/42dP8enOshYkRxRpaX+F 1Zbd/zT/DIxN230HhwnLGao+vtwAgiElAC+/Gl3CiRMlIxTpVyA2i6onf3ymxSO4P9zd G7cKi0URMOfqGvTew/Qgv3iWBQ4YuxABpHYibGZk5NHZSJKiw3HrrtS1lzfTNtICI0YB kMOLiyyopAH94hTfhMr8dU+4c6crYLQkE+ViiH+lNYLNaiZgo/j4C0XZBgxNbunWKHuX rmFT+15vyvzfnShqDxc1usyQWKHUtbKwCKsAYK1BQik30LOf+YOemW6drOOVTcCu5MQ/ nV0g== X-Gm-Message-State: ACrzQf02m16IYm1rC6z3tDE9UdtNUWP5Z6XbIH4e6xDAnA2Zlwxhz0cz kqKVWNtMhImpcNMH6MTNZctwBpjKWCU= X-Google-Smtp-Source: AMsMyM7ldy9tgfl3RdyA0NIqP0FnD2qY+N0BWbbnCJ0QEvS9HsXFzC1LaZ/1kCwUkvq14MhFCSaVbg== X-Received: by 2002:a17:90a:6c41:b0:212:fdaf:d79c with SMTP id x59-20020a17090a6c4100b00212fdafd79cmr50302729pjj.134.1667823964465; Mon, 07 Nov 2022 04:26:04 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id u8-20020a1709026e0800b0017bb38e4588sm4861702plk.135.2022.11.07.04.26.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:03 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 36/72] e2fsck: merge icounts after thread finishes Date: Mon, 7 Nov 2022 17:51:24 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi Merge inode_count and inode_link_info properly after threads finish. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 4d98c467..18bf7efd 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2341,6 +2341,41 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->dir_info); } +static inline errcode_t +e2fsck_pass1_merge_icount(ext2_icount_t *dest_icount, + ext2_icount_t *src_icount) +{ + if (*src_icount) { + if (*dest_icount == NULL) { + *dest_icount = *src_icount; + *src_icount = NULL; + } else { + errcode_t ret; + + ret = ext2fs_icount_merge(*src_icount, + *dest_icount); + if (ret) + return ret; + } + } + + return 0; +} + +static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + errcode_t ret; + + ret = e2fsck_pass1_merge_icount(&global_ctx->inode_count, + &thread_ctx->inode_count); + if (ret) + return ret; + ret = e2fsck_pass1_merge_icount(&global_ctx->inode_link_info, + &thread_ctx->inode_link_info); + + return ret; +} + static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) { errcode_t retval = 0; @@ -2361,6 +2396,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map; ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; + ext2_icount_t inode_count = global_ctx->inode_count; + ext2_icount_t inode_link_info = global_ctx->inode_link_info; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2385,6 +2422,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->block_metadata_map = block_metadata_map; global_ctx->dir_info = dir_info; e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); + global_ctx->inode_count = inode_count; + global_ctx->inode_link_info = inode_link_info; /* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | @@ -2396,6 +2435,13 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; + retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx); + if (retval) { + com_err(global_ctx->program_name, 0, + _("while merging icounts\n")); + return retval; + } + retval = ext2fs_merge_fs(&(thread_ctx->fs)); if (retval) { com_err(global_ctx->program_name, 0, _("while merging fs\n")); @@ -2478,6 +2524,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map); e2fsck_free_dir_info(thread_ctx); + ext2fs_free_icount(thread_ctx->inode_count); + ext2fs_free_icount(thread_ctx->inode_link_info); if (thread_ctx->logf) fclose(thread_ctx->logf); From patchwork Mon Nov 7 12:21:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700584 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jGKXN5Jc; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vp61Ht2z23lW for ; Mon, 7 Nov 2022 23:26:22 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vp60r2jz4xDK for ; Mon, 7 Nov 2022 23:26:22 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vp60nFqz4xGj; Mon, 7 Nov 2022 23:26:22 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=jGKXN5Jc; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vp60hRsz4xDK for ; Mon, 7 Nov 2022 23:26:22 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231419AbiKGM0V (ORCPT ); Mon, 7 Nov 2022 07:26:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232158AbiKGM0L (ORCPT ); Mon, 7 Nov 2022 07:26:11 -0500 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1AA363CE for ; Mon, 7 Nov 2022 04:26:10 -0800 (PST) Received: by mail-pj1-x1035.google.com with SMTP id f5-20020a17090a4a8500b002131bb59d61so12433023pjh.1 for ; Mon, 07 Nov 2022 04:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QxVl6aa3TBLVMV6wzcR2mNLIXsGqjOYiHHOZUywWNRk=; b=jGKXN5JcBnInGg6Z7bNkQ7i1SFQsEHPjzkF40ykWe/sU9Q3dDjk2WOzVIB2aKQrHRL XZFKc08QtHyupk1JqKXJp8YeBvNg2zc/YqEwXPFl059HWegv4FhsZ2NeMCyMHsQYym7w TEVjbL46r6b+En09kbNIZTfJPUQDP/Gz9a6ZnkMnbIeZrCpuwGe6R9hAg2Ejl3jCxJAg e+lYbmhm3P6vEfgJi0PEKPjAwKFRKhmOOEIBIssGd4YaRSHZ/yCDTXgUdpaCbEeCFDyN 8GxqYoaIFYiVzwasB0fHuWkPd7qiYqN7OsSf45rRTB4MEx3FqDRs48dlNvr3HXCgJou0 veNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QxVl6aa3TBLVMV6wzcR2mNLIXsGqjOYiHHOZUywWNRk=; b=kFpkY55JyzMgcEm4o2QOp3CfqGc+q5RvOU3w2kYbJSS74DzkCBRV+Ft5ONfY/Iz4pn WZNk1kmAA6McREPiQD/b/7RjRNc3WwQXCOL1hPXozgwcJXI1osJg4j/wlIGi5FQ0ZSE2 Mb+coNYm5bjV9sW8ZH90OUwCRHQu1n9p2yLAqn/+rS2FWccYzuwbLMYMsMJyG50sojTN YaLmKjg/8HnYPx4TlXjgdDru0DU17H2yVU6wE7tFyRD7jS/IahYupgxgSwGYux3tJIyE KHuEuBFr/l7aq0K8OryW04omnt35wmGb/55LpnYLsMm4SPh9FUMR/pIESTkSaAX/hvgq jVIg== X-Gm-Message-State: ACrzQf3dFBV/NUj6VgQp3JkmQFt4250Rbr883EPmPfK/VH26N0IUrvtN ejfAsZoiabmPcQj6NeNLHTQ= X-Google-Smtp-Source: AMsMyM7Pw96uTie9GX7uz2tqAQfJRNKauGBrRK/bLBvU7wI0FXWhIbzoejKK36zfVVviVA+7NF14Tw== X-Received: by 2002:a17:902:b708:b0:184:3921:df30 with SMTP id d8-20020a170902b70800b001843921df30mr51240748pls.43.1667823970427; Mon, 07 Nov 2022 04:26:10 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id t11-20020a170902e84b00b00186c41bd213sm4870881plg.177.2022.11.07.04.26.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:09 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 37/72] e2fsck: add debug codes for multiple threads Date: Mon, 7 Nov 2022 17:51:25 +0530 Message-Id: <6034e7dc1304ad2eea9ffabff96cb1604336d3e9.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Li Xi These debug codes are added to run the multiple pass1 check thread one by one in order. If all the codes are correct, fsck of multiple threads should have exactly the same outcome with single thread. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 16 ++++++++++++++++ e2fsck/pass1.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 2ee37f78..9b0f5067 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -489,6 +489,18 @@ struct e2fsck_struct { }; #ifdef HAVE_PTHREAD +#ifdef DEBUG_THREADS +/* + * Enabling DEBUG_THREADS would cause the parallel + * fsck threads run sequentially. + */ +struct e2fsck_thread_debug { + pthread_mutex_t etd_mutex; + pthread_cond_t etd_cond; + int etd_finished_threads; +}; +#endif + struct e2fsck_thread_info { /* ID returned by pthread_create() */ pthread_t eti_thread_id; @@ -498,7 +510,11 @@ struct e2fsck_thread_info { int eti_started; /* Context used for this thread */ e2fsck_t eti_thread_ctx; +#ifdef DEBUG_THREADS + struct e2fsck_thread_debug *eti_debug; +#endif }; + #endif /* Data structures to evaluate whether an extent tree needs rebuilding. */ diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 18bf7efd..752dca03 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2575,6 +2575,18 @@ static void *e2fsck_pass1_thread(void *arg) { struct e2fsck_thread_info *info = arg; e2fsck_t thread_ctx = info->eti_thread_ctx; +#ifdef DEBUG_THREADS + struct e2fsck_thread_debug *thread_debug = info->eti_debug; +#endif + +#ifdef DEBUG_THREADS + pthread_mutex_lock(&thread_debug->etd_mutex); + while (info->eti_thread_index > thread_debug->etd_finished_threads) { + pthread_cond_wait(&thread_debug->etd_cond, + &thread_debug->etd_mutex); + } + pthread_mutex_unlock(&thread_debug->etd_mutex); +#endif #ifdef HAVE_SETJMP_H /* @@ -2599,6 +2611,14 @@ out: thread_ctx->thread_info.et_group_start, thread_ctx->thread_info.et_group_end, thread_ctx->thread_info.et_inode_number); + +#ifdef DEBUG_THREADS + pthread_mutex_lock(&thread_debug->etd_mutex); + thread_debug->etd_finished_threads++; + pthread_cond_broadcast(&thread_debug->etd_cond); + pthread_mutex_unlock(&thread_debug->etd_mutex); +#endif + return NULL; } @@ -2612,6 +2632,12 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, struct e2fsck_thread_info *tmp_pinfo; int i; e2fsck_t thread_ctx; +#ifdef DEBUG_THREADS + struct e2fsck_thread_debug thread_debug = + {PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0}; + + thread_debug.etd_finished_threads = 0; +#endif retval = pthread_attr_init(&attr); if (retval) { @@ -2632,6 +2658,9 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, for (i = 0; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; +#ifdef DEBUG_THREADS + tmp_pinfo->eti_debug = &thread_debug; +#endif retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, i, num_threads); if (retval) { From patchwork Mon Nov 7 12:21:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700585 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ppGOdpzF; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vp74Bvjz23lW for ; Mon, 7 Nov 2022 23:26:23 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vp73mBGz4xDK for ; Mon, 7 Nov 2022 23:26:23 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vp73jfNz4xGj; Mon, 7 Nov 2022 23:26:23 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ppGOdpzF; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vp73dg2z4xDK for ; Mon, 7 Nov 2022 23:26:23 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231935AbiKGM0W (ORCPT ); Mon, 7 Nov 2022 07:26:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51988 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232181AbiKGM0R (ORCPT ); Mon, 7 Nov 2022 07:26:17 -0500 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 324E063C2 for ; Mon, 7 Nov 2022 04:26:17 -0800 (PST) Received: by mail-pl1-x630.google.com with SMTP id c2so10887392plz.11 for ; Mon, 07 Nov 2022 04:26:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TO3EAnoyooTXPluco+EV2CzZby3gnXHpq0yhCfmKuV0=; b=ppGOdpzF9PUS/kWQEid5qsV8N6gFSbQr0lT208UCSAbJeRXFxXXiJ/uej5nyFvWiQ6 iZTGrDTxJhMq1HO+sF+fFv62ExMZPL29MO7vCw7rjEe3aCTSwucbHcQBkh/IPS54Lwf+ j6BIpsz3AzH7OWBgfCr1zElfduP7YRWt0vncBsOHw+uw7dC8OtUq4mcvWvRz900buY2b Gxkj0ndO6es9xZmiQ89qwGGGJ9O2V7MlZnT2AxIR1zGqmhn5TO8Pt+512gybS9X/2Qqk tF6JRYYCTMvtLJ+QyaC/k6L7ScLAsCh9mzuBcsvzy97OaliOy+jJAjpCzE2+uCOYphtb nj9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TO3EAnoyooTXPluco+EV2CzZby3gnXHpq0yhCfmKuV0=; b=D7+29AVVySo+WnosjzQdzY0X5TvD93U5etLxiRWpWWJpP9XfWQ6aNtCF50XCkOunV7 IuA0430GiAibBb52DEH0oM2pq5GlY5qx4Q7YzApR7LrUGEKpHojFhBUM4rp81WAOPWpI 2mvaYuRK+fZmkXxLbP8OD17PBd+4hRtjofBB91DmbvqLHpI6MlKTenfGPER/eaoH6GO4 yQ8PWclXroJt8XYgLReA9bz0ncOQKoQVGo6Kz0NT7M4dYOBYIGx2yX6m8FSuBe0Zp7k0 I9Zfto/pkVDKShYr3SnsYASMnXIcLcti2OljoYEiArxsjWaqd08RWhKnC00SNY6OqLxR eTWw== X-Gm-Message-State: ACrzQf2/ERbavCDfs/sfASAi7SBMQ2ojHVw1bYvDAgxB/2YmoXD11BhN xXoh578S78Rgs42eGwYnEO0= X-Google-Smtp-Source: AMsMyM7hAPPxEy9zSFaMI+1sxuomge6Cwkh4qmBHv+jj2oAlb7feQvXL8JrePJmlyBnkXZOAblNMPA== X-Received: by 2002:a17:902:e88d:b0:186:f9d4:3fc5 with SMTP id w13-20020a170902e88d00b00186f9d43fc5mr50765294plg.116.1667823976645; Mon, 07 Nov 2022 04:26:16 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id 4-20020a170902c10400b00186c3af9644sm4816035pli.273.2022.11.07.04.26.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:16 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 38/72] e2fsck: merge counts after threads finish Date: Mon, 7 Nov 2022 17:51:26 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Merge counts properly. Signed-off-by: Li Xi Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 752dca03..8b502307 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2398,6 +2398,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; ext2_icount_t inode_count = global_ctx->inode_count; ext2_icount_t inode_link_info = global_ctx->inode_link_info; + __u32 fs_directory_count = global_ctx->fs_directory_count; + __u32 fs_regular_count = global_ctx->fs_regular_count; + __u32 fs_blockdev_count = global_ctx->fs_blockdev_count; + __u32 fs_chardev_count = global_ctx->fs_chardev_count; + __u32 fs_links_count = global_ctx->fs_links_count; + __u32 fs_symlinks_count = global_ctx->fs_symlinks_count; + __u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count; + __u32 fs_fifo_count = global_ctx->fs_fifo_count; + __u32 fs_total_count = global_ctx->fs_total_count; + __u32 fs_badblocks_count = global_ctx->fs_badblocks_count; + __u32 fs_sockets_count = global_ctx->fs_sockets_count; + __u32 fs_ind_count = global_ctx->fs_ind_count; + __u32 fs_dind_count = global_ctx->fs_dind_count; + __u32 fs_tind_count = global_ctx->fs_tind_count; + __u32 fs_fragmented = global_ctx->fs_fragmented; + __u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir; + __u32 large_files = global_ctx->large_files; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2424,6 +2441,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); global_ctx->inode_count = inode_count; global_ctx->inode_link_info = inode_link_info; + global_ctx->fs_directory_count += fs_directory_count; + global_ctx->fs_regular_count += fs_regular_count; + global_ctx->fs_blockdev_count += fs_blockdev_count; + global_ctx->fs_chardev_count += fs_chardev_count; + global_ctx->fs_links_count += fs_links_count; + global_ctx->fs_symlinks_count += fs_symlinks_count; + global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count; + global_ctx->fs_fifo_count += fs_fifo_count; + global_ctx->fs_total_count += fs_total_count; + global_ctx->fs_badblocks_count += fs_badblocks_count; + global_ctx->fs_sockets_count += fs_sockets_count; + global_ctx->fs_ind_count += fs_ind_count; + global_ctx->fs_dind_count += fs_dind_count; + global_ctx->fs_tind_count += fs_tind_count; + global_ctx->fs_fragmented += fs_fragmented; + global_ctx->fs_fragmented_dir += fs_fragmented_dir; + global_ctx->large_files += large_files; /* Keep the global singal flags*/ global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | From patchwork Mon Nov 7 12:21:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700586 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ihAmjaPG; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VpG4cLmz23lW for ; Mon, 7 Nov 2022 23:26:30 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpG47r8z4xDK for ; Mon, 7 Nov 2022 23:26:30 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VpG44kDz4xGj; Mon, 7 Nov 2022 23:26:30 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ihAmjaPG; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpG40d8z4xDK for ; Mon, 7 Nov 2022 23:26:30 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232158AbiKGM03 (ORCPT ); Mon, 7 Nov 2022 07:26:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232125AbiKGM0Z (ORCPT ); Mon, 7 Nov 2022 07:26:25 -0500 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95BD0140F6 for ; Mon, 7 Nov 2022 04:26:23 -0800 (PST) Received: by mail-pl1-x633.google.com with SMTP id 4so10956110pli.0 for ; Mon, 07 Nov 2022 04:26:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V+HeHSxpZtZ5nOaOUqlqkZGRWAIsUwzBESOK8Dchek0=; b=ihAmjaPGcQDy9redWad3uoZ42ppMe4TMvyZ4OW8QDKNJSOBgnC2j4XtnulMWuj3BBo zu5qijV1MGnf+R4gzpqpwV0fhFBNNhQZrF53OQ1BSQ9JzneWaHytXfSakFC0SBVaOSal wI/vB2Nfc/2BAfSqs1gq9f0+uCSsW02grAJKgiIzRWJH1LBNIn+oB12qR+O958KEootg 9OZgJsgd1h5we6KRi/xFsL63aL58oatXRceQTlyXJPOZamxvT4CigH+s6qli2DjBjiwJ smnMagNdblxP47DZy4OThyNkeB2yodFK00dg/TP6PZpO09YnQrXM1qnj4otbPQn+Ak8Q rVyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V+HeHSxpZtZ5nOaOUqlqkZGRWAIsUwzBESOK8Dchek0=; b=8JIyk7ZJ4j6kbCZ2UCZycJr5eNY8sp+IoxoFK88c/oCzV9p1hriiFmQcD3TsEOor+f HLkCyfsr+4bLMS5lwQttBGN71ZeOv3WFU5lXihjvdGIXBBtHsFcdb88of5ZKfTPmFbb0 uxtDQEbKolh67zmDuOVZpGlznq2HkFqkGzxWcxbpPQfgHhOst7OQo3qcrGLZ52k++xCB UILvym2baSSScdpnElhnYe6XNqfmlOSfOcwpOKiCBGqsAOl9r+RO1Xe8bcC/O+klxcYm 9wD21kT2BwQYREcwUWCrceGWOIPTUi8R99U7auqwUlbdDtpYu+GKm9yoNJayBMfEld/k u4Nw== X-Gm-Message-State: ACrzQf3Vak+DaVHFdV0Rq0/spAEVQvQaQu0Nq56XiBmP3s+V+VZEtWcx U8fHvz9TruD8wJfHXIV+tlY= X-Google-Smtp-Source: AMsMyM4jp6GgnjrfuAxRCjvRbirYJwchwVsVkdGwYv03eFcfNkM1k267uFU/NuEJyfFB6PpSdvQl1g== X-Received: by 2002:a17:90b:3696:b0:214:1611:a497 with SMTP id mj22-20020a17090b369600b002141611a497mr33643779pjb.78.1667823983053; Mon, 07 Nov 2022 04:26:23 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id i184-20020a6254c1000000b0053e468a78a8sm4344955pfb.158.2022.11.07.04.26.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:22 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 39/72] e2fsck: merge dx_dir_info after threads finish Date: Mon, 7 Nov 2022 17:51:27 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Merge properly. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/dx_dirinfo.c | 64 +++++++++++++++++++++++++++++++++++++++++++++ e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 23 ++++++++++++++++ 3 files changed, 88 insertions(+) diff --git a/e2fsck/dx_dirinfo.c b/e2fsck/dx_dirinfo.c index caca3e30..91954572 100644 --- a/e2fsck/dx_dirinfo.c +++ b/e2fsck/dx_dirinfo.c @@ -5,6 +5,7 @@ * under the terms of the GNU Public License. */ +#include #include "config.h" #include "e2fsck.h" @@ -79,6 +80,69 @@ void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino, struct ext2_inode *inode, "dx_block info array"); } +/* + * Merge two sorted dir info to @dest + */ +void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + struct dx_dir_info *src_array = thread_ctx->dx_dir_info; + struct dx_dir_info *dest_array = global_ctx->dx_dir_info; + size_t size_dx_info = sizeof(struct dx_dir_info); + ext2_ino_t size = global_ctx->dx_dir_info_size; + ext2_ino_t src_count = thread_ctx->dx_dir_info_count; + ext2_ino_t dest_count = global_ctx->dx_dir_info_count; + ext2_ino_t total_count = src_count + dest_count; + struct dx_dir_info *array; + struct dx_dir_info *array_ptr; + ext2_ino_t src_index = 0, dest_index = 0; + + if (thread_ctx->dx_dir_info_count == 0) + return; + + if (size < total_count) + size = total_count; + + array = e2fsck_allocate_memory(global_ctx, size * size_dx_info, + "directory map"); + array_ptr = array; + /* + * This can be improved by binary search and memcpy, but codes + * would be more complex. And if the groups distributed to each + * thread are strided, this implementation won't be too bad + * comparing to the optimiztion. + */ + while (src_index < src_count || dest_index < dest_count) { + if (src_index >= src_count) { + memcpy(array_ptr, &dest_array[dest_index], + (dest_count - dest_index) * size_dx_info); + break; + } + if (dest_index >= dest_count) { + memcpy(array_ptr, &src_array[src_index], + (src_count - src_index) * size_dx_info); + break; + } + if (src_array[src_index].ino < dest_array[dest_index].ino) { + *array_ptr = src_array[src_index]; + src_index++; + } else { + assert(src_array[src_index].ino > + dest_array[dest_index].ino); + *array_ptr = dest_array[dest_index]; + dest_index++; + } + array_ptr++; + } + + if (global_ctx->dx_dir_info) + ext2fs_free_mem(&global_ctx->dx_dir_info); + if (thread_ctx->dx_dir_info) + ext2fs_free_mem(&thread_ctx->dx_dir_info); + global_ctx->dx_dir_info = array; + global_ctx->dx_dir_info_size = size; + global_ctx->dx_dir_info_count = total_count; +} + /* * get_dx_dir_info() --- given an inode number, try to find the directory * information entry for it. diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 9b0f5067..26c3b8a5 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -578,6 +578,7 @@ extern int e2fsck_dir_info_get_parent(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t *parent); extern int e2fsck_dir_info_get_dotdot(e2fsck_t ctx, ext2_ino_t ino, ext2_ino_t *dotdot); +extern void e2fsck_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx); /* dx_dirinfo.c */ extern void e2fsck_add_dx_dir(e2fsck_t ctx, ext2_ino_t ino, diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 8b502307..f998590e 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2341,6 +2341,22 @@ static void e2fsck_pass1_merge_dir_info(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->dir_info); } +static void e2fsck_pass1_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx) +{ + if (thread_ctx->dx_dir_info == NULL) + return; + + if (global_ctx->dx_dir_info == NULL) { + global_ctx->dx_dir_info = thread_ctx->dx_dir_info; + global_ctx->dx_dir_info_size = thread_ctx->dx_dir_info_size; + global_ctx->dx_dir_info_count = thread_ctx->dx_dir_info_count; + thread_ctx->dx_dir_info = NULL; + return; + } + + e2fsck_merge_dx_dir(global_ctx, thread_ctx); +} + static inline errcode_t e2fsck_pass1_merge_icount(ext2_icount_t *dest_icount, ext2_icount_t *src_icount) @@ -2386,6 +2402,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_filsys global_fs = global_ctx->fs; ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; struct dir_info_db *dir_info = global_ctx->dir_info; + struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info; ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; @@ -2415,6 +2432,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx __u32 fs_fragmented = global_ctx->fs_fragmented; __u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir; __u32 large_files = global_ctx->large_files; + ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; + ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2439,6 +2458,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->block_metadata_map = block_metadata_map; global_ctx->dir_info = dir_info; e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); + global_ctx->dx_dir_info = dx_dir_info; + global_ctx->dx_dir_info_count = dx_dir_info_count; + global_ctx->dx_dir_info_size = dx_dir_info_size; + e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx); global_ctx->inode_count = inode_count; global_ctx->inode_link_info = inode_link_info; global_ctx->fs_directory_count += fs_directory_count; From patchwork Mon Nov 7 12:21:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700587 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=EUnB7qhi; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VpL2N8kz23lW for ; Mon, 7 Nov 2022 23:26:34 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpL1wcnz4xDK for ; Mon, 7 Nov 2022 23:26:34 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VpL1tFrz4xGj; Mon, 7 Nov 2022 23:26:34 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=EUnB7qhi; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpL1mgbz4xDK for ; Mon, 7 Nov 2022 23:26:34 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232164AbiKGM0d (ORCPT ); Mon, 7 Nov 2022 07:26:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52050 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGM0a (ORCPT ); Mon, 7 Nov 2022 07:26:30 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB94863CE for ; Mon, 7 Nov 2022 04:26:29 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id l2so10877941pld.13 for ; Mon, 07 Nov 2022 04:26:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zMGXvG1b324hpwzPRf+YBXbWO6uEyoN1PX4ENEuZ1So=; b=EUnB7qhirjezjPUL+3O+R6BraDB1jOsINu1bfn7cFMcI70xWZfnRynr3GYojz0XptY 8E100U5sFJWYZeJOpOTWc6w4+/WfhoAJjjYzHpmQ7GVT/a7WGE4klaAOS2Fv/tKipZBN QUQ7tu+/FBNEpIGii0qQVX9R3gVWIkI5woRWoC1ikUGtlEfceKDt/djyJlBXC/vmK1Ol lR6mb4dJumiyoRCfMonCg7wxl6RQRHpQNmMZJILfG3RzTdaitMn4tjUjIw2ju6u3MspN ngqfgCBXs9TSSMg19hC0oIJEJ+ZE7EFeErCgJ3/8vS0MgwTqKQ6R9CryFkKfx7/7gXhC HHhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zMGXvG1b324hpwzPRf+YBXbWO6uEyoN1PX4ENEuZ1So=; b=BtBRSGf3WbV/CyYubMw31G+kcnHlGt6RpiJzP58bshocLSGvd22DjkSGpJisg7ByU0 VVS5vR2JdKpUf6u77Rm9bP4qdCvTIeCeYHdnifTzbcKpvyG+QB+U8zZPPcD+V0k4MgUi 4ppX54H3e/ezIixa9CYqbbXAkAWwP2UzE3vLIYS56vCsuqSjJK8xXqHc/wC4+kB35YCY HQ9AuIv7CwXnHut71JVnUTfny+151ylaanEAbPpE8JHQLerSMehQTzYnI9lvkeLxpegz AwKDB4fC1amsze+/GXM/LDVjsofeO8v5wMQnQFSyf2/00/S6l3LJOwZl58qOygn2Ayky x3qA== X-Gm-Message-State: ACrzQf3OvbnUAJzzdcGFwFRv74L6iPcy55dEJ1iGYzQLyuP7HNbt0nXD t45kPOr9YxLb032IGFkqw2w= X-Google-Smtp-Source: AMsMyM57vICXvl+iGs4xiQSQiDX+aZiJAG1+Fik+37QICwgfWrreVqoGpn0WIvzPVKC/7XntkBjgkA== X-Received: by 2002:a17:90a:c78a:b0:212:e56b:2b17 with SMTP id gn10-20020a17090ac78a00b00212e56b2b17mr51603830pjb.51.1667823989453; Mon, 07 Nov 2022 04:26:29 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id l5-20020a170903120500b0018863dbf3b0sm372713plh.45.2022.11.07.04.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:28 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 40/72] e2fsck: merge dirs_to_hash when threads finish Date: Mon, 7 Nov 2022 17:51:28 +0530 Message-Id: <12bb27f0b59e176a599fd9f065f7f25633d8571e.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong @dirs_to_hash list need be merged after threads finish, test covered by t_dangerous. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index f998590e..a3f13402 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2392,6 +2392,25 @@ static errcode_t e2fsck_pass1_merge_icounts(e2fsck_t global_ctx, e2fsck_t thread return ret; } +static errcode_t e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + errcode_t retval = 0; + + if (!thread_ctx->dirs_to_hash) + return 0; + + if (!global_ctx->dirs_to_hash) + retval = ext2fs_badblocks_copy(thread_ctx->dirs_to_hash, + &global_ctx->dirs_to_hash); + else + retval = ext2fs_badblocks_merge(thread_ctx->dirs_to_hash, + global_ctx->dirs_to_hash); + + return retval; +} + + static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) { errcode_t retval = 0; @@ -2434,6 +2453,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx __u32 large_files = global_ctx->large_files; ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; + ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2504,6 +2524,14 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx com_err(global_ctx->program_name, 0, _("while merging fs\n")); return retval; } + global_ctx->dirs_to_hash = dirs_to_hash; + retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx); + if (retval) { + com_err(global_ctx->program_name, 0, + _("while merging dirs to hash\n")); + return retval; + } + retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map, &global_ctx->inode_used_map); @@ -2583,6 +2611,8 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_free_dir_info(thread_ctx); ext2fs_free_icount(thread_ctx->inode_count); ext2fs_free_icount(thread_ctx->inode_link_info); + if (thread_ctx->dirs_to_hash) + ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash); if (thread_ctx->logf) fclose(thread_ctx->logf); From patchwork Mon Nov 7 12:21:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700588 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Hizxj3SP; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VpQ2ZKFz23lW for ; Mon, 7 Nov 2022 23:26:38 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpQ26D1z4xDK for ; Mon, 7 Nov 2022 23:26:38 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VpQ242Cz4xGj; Mon, 7 Nov 2022 23:26:38 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Hizxj3SP; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpQ1zltz4xDK for ; Mon, 7 Nov 2022 23:26:38 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232172AbiKGM0h (ORCPT ); Mon, 7 Nov 2022 07:26:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGM0g (ORCPT ); Mon, 7 Nov 2022 07:26:36 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A68F63D5 for ; Mon, 7 Nov 2022 04:26:36 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id b11so10418304pjp.2 for ; Mon, 07 Nov 2022 04:26:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3GiE7/SSqwiXyofETS8k0nS3VBbJtMpqt7PMfrdEg6o=; b=Hizxj3SPxzgX8XTngOYeCurItcgtV8ptFWuPbMpr+VHoZiBoklVadqx0g27S+RVoJV nuP4sQdblC4yG1zWMtL67c+lwJ5rLa/1/Mnwx5fIUZSiS/1ptdCcM7PtXoM/r6dT7LF5 aDTno8+zuiIBAWlX/GyLkNyWxAAAXjnFgXadcn16TPWdmese4MdTaDIEGac0IP8DZheK p7rJ6TETuqnamCUgeAlwrZLBVAGzI3rHqA7Z+XzRnjPUaOV2DFa1CRGxyUuJ3IKmEbOn KBEJajj3PVd5CADiJIzBCoaPu1PN1x+lI+cbm0ei4DW6Qym/jEPp2O+fOgdrJIgtfj3u PKYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3GiE7/SSqwiXyofETS8k0nS3VBbJtMpqt7PMfrdEg6o=; b=74d9nZua0rxyWdeVK8u0YaE0iYMFyjeChbp6TSj5GZuF3QgQYClTfH6KacbHVNTx5Z Dw5fuW3WVSs3FI5HiZ4t6yxuYVEn52wOoZUVH8YOtJkdFCitIBV0gC6eP9+EgfBVSqRl 0gDHzGt3fxV0tKO+YBTqAJ0pXMkXgTuLSekhnik7yk3FicpPz/M2y6Uq9VDtRNvGN5je PREwqTEnaigZlLciX1i2EyShXkQtQyF0XWxxJ7kGO1jnd/J8+T6DCTAu+5ktO86GvPCU pMgc8V2V7YtFeWw89ALxyjjJAj8esqB1w4xRufM3DTZoa9aAMr1KrAbGA/PhEa+9qcfO 1bnA== X-Gm-Message-State: ACrzQf1Q3QkjE7aakRinkmNIJeolz5U3KyTYPv4uXquAekgqgthJ9CFk HHahA/DwPNpswvBD7cIL0GyjULEtwAs= X-Google-Smtp-Source: AMsMyM7teKx7x97UysYRyCYTlrhFi0/u0BakBtEPxJ/GffP69Bx78+hO9Sn+7ppRDXDnQYIXX9xCOw== X-Received: by 2002:a17:902:ec01:b0:186:748f:e8b6 with SMTP id l1-20020a170902ec0100b00186748fe8b6mr49851371pld.131.1667823995643; Mon, 07 Nov 2022 04:26:35 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id q18-20020a17090311d200b00179f370dbe7sm4821165plh.287.2022.11.07.04.26.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:34 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 41/72] e2fsck: merge context flags properly Date: Mon, 7 Nov 2022 17:51:29 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong e2fsck might restart after pass1, so we should keep flags if possible, this patch try to fix f_illitable_flexbg failure Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index a3f13402..7e167189 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2502,9 +2502,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs_fragmented_dir += fs_fragmented_dir; global_ctx->large_files += large_files; - /* Keep the global singal flags*/ - global_ctx->flags |= (flags & E2F_FLAG_SIGNAL_MASK) | - (global_ctx->flags & E2F_FLAG_SIGNAL_MASK); + global_ctx->flags |= flags; global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; global_ctx->global_ctx = NULL; From patchwork Mon Nov 7 12:21:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700589 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JUhJRCd5; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VpX5s85z23lW for ; Mon, 7 Nov 2022 23:26:44 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpX4JxQz4xDK for ; Mon, 7 Nov 2022 23:26:44 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VpX4Gvhz4xGj; Mon, 7 Nov 2022 23:26:44 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JUhJRCd5; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VpX4B8yz4xDK for ; Mon, 7 Nov 2022 23:26:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232181AbiKGM0n (ORCPT ); Mon, 7 Nov 2022 07:26:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGM0m (ORCPT ); Mon, 7 Nov 2022 07:26:42 -0500 Received: from mail-pg1-x52a.google.com (mail-pg1-x52a.google.com [IPv6:2607:f8b0:4864:20::52a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1EFB863D5 for ; Mon, 7 Nov 2022 04:26:42 -0800 (PST) Received: by mail-pg1-x52a.google.com with SMTP id e129so10303432pgc.9 for ; Mon, 07 Nov 2022 04:26:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=djX6kG4sb5dSN1Jo9voKc/t70N6XAj7Y4LtlOcTTGEI=; b=JUhJRCd5DAjnwvACzAeMR1dtwzdkXflnrkpW0fMdisLYmtuZkezcnQyBt3CF1j5PjO B2N6pcwKH6/MfMNgzVt40q7oImNYyQb5B37k2raftfpD42Xi+H+FFV95Xqhjz8Xzrsp8 73Oz5542gfL3K+qOFcBiw45rThLoqEYTOQwVrrExh7ZvN+EfN3ZqZnk8Iw4C7xbD/MWA ShBsy4ug5HOOGS/Kg8CsF8o7yAl90YqUSDyQCyIbJPTNfsV02HZYTRLnIM9Uj5NTXQDE S7Pvtj4BsQCNwk/MXKI2efaVjE1kWiwIzmUP70rllBCa/4Xoil7WZQC07b4e4VIcUEoV lxVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=djX6kG4sb5dSN1Jo9voKc/t70N6XAj7Y4LtlOcTTGEI=; b=nhyKec9cR/dEbTAcd/rpIeutE8O3Kuo5rdwooHmtPMMM6gZz09NEnax3eHx2eIi3N3 MKO7wtnXqrVrIBJNwScc/DHf2phDkWHqWracEnqINIxI7GfBnEWUfsenkv/wWI7BeB09 OvYn6h683QaYv3UW8ua7ULHcE3gEjcVIU/FT271eQfJURdVteJWskiJXrSrBw4VqfoRb 5JJ2uVzYR1QSigJMuCyPX0Ban1u+HX+t8EkQsSc56FdUrNJjAn6zYXBR9a9fSGQHNwaQ J65rK4qvobHkeFnUp63Du5e/Hz1hb/UEowIFVA/BjjZaJyWyZa3xsZD9tMMby0iZZ3a2 bsvA== X-Gm-Message-State: ACrzQf0dM691NYYLzVHNCF9Qm05Z0IAz1h8XR7MrbdUzqzPRjWOzvWP2 SN2frM+CqobWwGLeDC/aHb8= X-Google-Smtp-Source: AMsMyM4g6es8Govex4KSfeE1vzCpofC57TUWx2HAIkCgq10jmLHUuvApxHaiwSAf7ihsaf74Qw0Y4Q== X-Received: by 2002:a62:ea09:0:b0:562:a86f:63af with SMTP id t9-20020a62ea09000000b00562a86f63afmr51018117pfh.71.1667824001646; Mon, 07 Nov 2022 04:26:41 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id p1-20020a170902ebc100b00179c9219195sm4881685plg.16.2022.11.07.04.26.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:40 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 42/72] e2fsck: merge quota context after threads finish Date: Mon, 7 Nov 2022 17:51:30 +0530 Message-Id: <31672b4c6e497a62c5f4fe0650cf0acde6c0b607.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Every threads calculate its own quota accounting, merge them after threads finish. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 7e167189..213c1a51 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2317,6 +2317,12 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre if (thread_context->options & E2F_OPT_MULTITHREAD) log_out(thread_context, _("Scan group range [%d, %d)\n"), tinfo->et_group_start, tinfo->et_group_end); + retval = quota_init_context(&thread_context->qctx, thread_fs, 0); + if (retval) { + com_err(global_ctx->program_name, retval, + "while init quota context"); + goto out_fs; + } *thread_ctx = thread_context; return 0; out_fs: @@ -2454,6 +2460,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; + quota_ctx_t qctx = global_ctx->qctx; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2530,6 +2537,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx return retval; } + global_ctx->qctx = qctx; + retval = quota_merge_and_update_usage(global_ctx->qctx, + thread_ctx->qctx); + if (retval) + return retval; + retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map, &global_ctx->inode_used_map); @@ -2611,6 +2624,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) ext2fs_free_icount(thread_ctx->inode_link_info); if (thread_ctx->dirs_to_hash) ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash); + quota_release_context(&thread_ctx->qctx); if (thread_ctx->logf) fclose(thread_ctx->logf); From patchwork Mon Nov 7 12:21:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700590 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=T5rSjtWs; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vpg3tKVz23lW for ; Mon, 7 Nov 2022 23:26:51 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpg3G3Pz4xDK for ; Mon, 7 Nov 2022 23:26:51 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vpg3Bn8z4xGj; Mon, 7 Nov 2022 23:26:51 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=T5rSjtWs; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpg1MWcz4xDK for ; Mon, 7 Nov 2022 23:26:51 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232197AbiKGM0u (ORCPT ); Mon, 7 Nov 2022 07:26:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52162 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232106AbiKGM0t (ORCPT ); Mon, 7 Nov 2022 07:26:49 -0500 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F20D6247 for ; Mon, 7 Nov 2022 04:26:48 -0800 (PST) Received: by mail-pj1-x1032.google.com with SMTP id d13-20020a17090a3b0d00b00213519dfe4aso10141430pjc.2 for ; Mon, 07 Nov 2022 04:26:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CGtoRKYzYuzYnA7xUwO89/4tGL6NcueUvPqXpKALfxU=; b=T5rSjtWsO/lTSD/gr2eQefYkQtUNMg/m1K4khrTQldP74PgtuvONrM2zxVA2gtSDlN r1LnFQ/gUiqVtsw3eefpHI7O336LYqnhOHFwAe7ZYAwu4edtxzQrGHSpaQmSC3WzAMD/ JDFCcA3KnAocM5C+10tFNCOMUl/NTt8IGP6Q8GKcPTuxdcfudCn7sS7rsSkLR3qi94ty dRsvWAK0Cu4CP5rFc6N8SztsOj3inZUyuJKb8TJo4AfSst1N4hiQtU+NrGq89J55HoZk gvuZjXklboCLFHMqbml/nAa+figxusoTjTCALQNaLIFIeppxwq9NyXZw9xKMH0YNaHb7 EO3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CGtoRKYzYuzYnA7xUwO89/4tGL6NcueUvPqXpKALfxU=; b=jg/NpWE0DtrogtoNR7s4D/rqc9dbBKPbtpT7uK2VvS1H8m+gTUmB0g508bd1lcLFcA 59rRXuxqkqMcR7v3g+r5XUoTIn52V7AVPae2h1+c1bNBJVy68Mcmmp10x6UCHMD8IWip Dkfm9n7xfPDeQ1eNvQx35X62/SbDIH8Y8x1eCFNf1Up4+d0dBHhSHUfT/93WlODrl3jc ysEU6oTelwmsIo6UL7jjCG66IxV651u4ei41vjcGylOH1dCAj7tmtGtKqasMwGBBqWrH K/+qnGNhxixOUREpc/aB4P6Om7ZAc2fp72ddBVLfimyXDt3LnTAVNCj4pmRGRKE2gDOv TmoA== X-Gm-Message-State: ACrzQf36DA+b7Ixe9+P6iPmBR39+Y9HJ6ZPK9mwbRGJCinRLl5jx2kJJ Xgji4IDwjNg7dBk99qvfpWY= X-Google-Smtp-Source: AMsMyM70CV76IQmMHdmyWL2Z+QV86uQ1ggRWmYQmmr2f3RWu/faFLJ96mGGQcgtI/bTqLFSNBFsj8Q== X-Received: by 2002:a17:90b:4b09:b0:213:655c:158b with SMTP id lx9-20020a17090b4b0900b00213655c158bmr51721479pjb.119.1667824007570; Mon, 07 Nov 2022 04:26:47 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id lt20-20020a17090b355400b0020d48bc6661sm6093042pjb.31.2022.11.07.04.26.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:47 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 43/72] e2fsck: serialize fix operations Date: Mon, 7 Nov 2022 17:51:31 +0530 Message-Id: <2cbf760ca0510593a005f4955abd333ddcbf571f.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Allow different threads to fix at the same time could be dangerous and error-prone now, and most of time parallel scanning and checking is important. So this patch adds a mutex to serialize fix operations during pass1. And the good benefit of this, we don't need block allocations and free, superblock updates protection any more, since only fix operations during pass1 could touch them. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 6 +++++ e2fsck/pass1.c | 63 +++++++++++++++++++++++++++++++++++++++++++------ e2fsck/util.c | 38 +++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 7 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 26c3b8a5..5356e172 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -484,6 +484,10 @@ struct e2fsck_struct { /* Undo file */ char *undo_file; +#ifdef HAVE_PTHREAD + /* serialize fix operation for multiple threads */ + pthread_mutex_t fs_fix_mutex; +#endif /* Fast commit replay state */ struct e2fsck_fc_replay_state fc_replay_state; }; @@ -787,6 +791,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs, const char *profile_name, ext2fs_block_bitmap *ret); unsigned long long get_memory_size(void); +extern void e2fsck_pass1_fix_lock(e2fsck_t ctx); +extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx); /* unix.c */ extern void e2fsck_clear_progbar(e2fsck_t ctx); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 213c1a51..c68e6957 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -380,8 +380,10 @@ static problem_t check_large_ea_inode(e2fsck_t ctx, pctx->num = entry->e_value_inum; if (fix_problem(ctx, PR_1_ATTR_SET_EA_INODE_FL, pctx)) { inode.i_flags |= EXT4_EA_INODE_FL; + e2fsck_pass1_fix_lock(ctx); ext2fs_write_inode(ctx->fs, entry->e_value_inum, &inode); + e2fsck_pass1_fix_unlock(ctx); } else { return PR_1_ATTR_NO_EA_INODE_FL; } @@ -875,8 +877,11 @@ static errcode_t recheck_bad_inode_checksum(ext2_filsys fs, ext2_ino_t ino, if (!fix_problem(ctx, PR_1_INODE_ONLY_CSUM_INVALID, pctx)) return 0; + + e2fsck_pass1_fix_lock(ctx); retval = ext2fs_write_inode_full(fs, ino, (struct ext2_inode *)&inode, sizeof(inode)); + e2fsck_pass1_fix_unlock(ctx); return retval; } @@ -886,15 +891,19 @@ static void reserve_block_for_root_repair(e2fsck_t ctx) errcode_t err; ext2_filsys fs = ctx->fs; + e2fsck_pass1_fix_lock(ctx); ctx->root_repair_block = 0; if (ext2fs_test_inode_bitmap2(ctx->inode_used_map, EXT2_ROOT_INO)) - return; + goto out; err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk); if (err) - return; + goto out; ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); ctx->root_repair_block = blk; +out: + e2fsck_pass1_fix_unlock(ctx); + return; } static void reserve_block_for_lnf_repair(e2fsck_t ctx) @@ -905,15 +914,19 @@ static void reserve_block_for_lnf_repair(e2fsck_t ctx) static const char name[] = "lost+found"; ext2_ino_t ino; + e2fsck_pass1_fix_lock(ctx); ctx->lnf_repair_block = 0; if (!ext2fs_lookup(fs, EXT2_ROOT_INO, name, sizeof(name)-1, 0, &ino)) - return; + goto out; err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk); if (err) - return; + goto out; ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); ctx->lnf_repair_block = blk; +out: + e2fsck_pass1_fix_unlock(ctx); + return; } static errcode_t get_inline_data_ea_size(ext2_filsys fs, ext2_ino_t ino, @@ -1551,8 +1564,10 @@ void e2fsck_pass1_run(e2fsck_t ctx) &size); if (!pctx.errcode && fix_problem(ctx, PR_1_INLINE_DATA_FEATURE, &pctx)) { + e2fsck_pass1_fix_lock(ctx); ext2fs_set_feature_inline_data(sb); ext2fs_mark_super_dirty(fs); + e2fsck_pass1_fix_unlock(ctx); inlinedata_fs = 1; } else if (fix_problem(ctx, PR_1_INLINE_DATA_SET, &pctx)) { e2fsck_clear_inode(ctx, ino, inode, 0, "pass1"); @@ -1640,9 +1655,11 @@ void e2fsck_pass1_run(e2fsck_t ctx) if ((ext2fs_extent_header_verify(inode->i_block, sizeof(inode->i_block)) == 0) && fix_problem(ctx, PR_1_EXTENT_FEATURE, &pctx)) { + e2fsck_pass1_fix_lock(ctx); ext2fs_set_feature_extents(sb); ext2fs_mark_super_dirty(fs); extent_fs = 1; + e2fsck_pass1_fix_unlock(ctx); } else if (fix_problem(ctx, PR_1_EXTENTS_SET, &pctx)) { clear_inode: e2fsck_clear_inode(ctx, ino, inode, 0, "pass1"); @@ -2082,8 +2099,11 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->ea_block_quota_inodes = 0; } - if (ctx->invalid_bitmaps) + if (ctx->invalid_bitmaps) { + e2fsck_pass1_fix_lock(ctx); handle_fs_bad_blocks(ctx); + e2fsck_pass1_fix_unlock(ctx); + } /* We don't need the block_ea_map any more */ if (ctx->block_ea_map) { @@ -2096,7 +2116,9 @@ void e2fsck_pass1_run(e2fsck_t ctx) if (ctx->flags & E2F_FLAG_RESIZE_INODE) { clear_problem_context(&pctx); + e2fsck_pass1_fix_lock(ctx); pctx.errcode = ext2fs_create_resize_inode(fs); + e2fsck_pass1_fix_unlock(ctx); if (pctx.errcode) { if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE, &pctx)) { @@ -2804,6 +2826,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx) int num_threads = 1; errcode_t retval; + pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL); retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); if (retval) { com_err(global_ctx->program_name, retval, @@ -3119,6 +3142,18 @@ static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block, } } +static errcode_t _INLINE_ e2fsck_write_ext_attr3(e2fsck_t ctx, blk64_t block, + void *inbuf, ext2_ino_t inum) +{ + errcode_t retval; + ext2_filsys fs = ctx->fs; + + e2fsck_pass1_fix_lock(ctx); + retval = ext2fs_write_ext_attr3(fs, block, inbuf, inum); + e2fsck_pass1_fix_unlock(ctx); + + return retval; +} /* * Adjust the extended attribute block's reference counts at the end * of pass 1, either by subtracting out references for EA blocks that @@ -3155,7 +3190,7 @@ static void adjust_extattr_refcount(e2fsck_t ctx, ext2_refcount_t refcount, pctx.num = should_be; if (fix_problem(ctx, PR_1_EXTATTR_REFCOUNT, &pctx)) { header->h_refcount = should_be; - pctx.errcode = ext2fs_write_ext_attr3(fs, blk, + pctx.errcode = e2fsck_write_ext_attr3(ctx, blk, block_buf, pctx.ino); if (pctx.errcode) { @@ -3387,7 +3422,7 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx, */ if (failed_csum && fix_problem(ctx, PR_1_EA_BLOCK_ONLY_CSUM_INVALID, pctx)) { - pctx->errcode = ext2fs_write_ext_attr3(fs, blk, block_buf, + pctx->errcode = e2fsck_write_ext_attr3(ctx, blk, block_buf, pctx->ino); if (pctx->errcode) return 0; @@ -3699,10 +3734,12 @@ static void scan_extent_node(e2fsck_t ctx, struct problem_context *pctx, if (try_repairs && is_dir && problem == 0 && (extent.e_flags & EXT2_EXTENT_FLAGS_UNINIT) && fix_problem(ctx, PR_1_UNINIT_DBLOCK, pctx)) { + e2fsck_pass1_fix_lock(ctx); extent.e_flags &= ~EXT2_EXTENT_FLAGS_UNINIT; pb->inode_modified = 1; pctx->errcode = ext2fs_extent_replace(ehandle, 0, &extent); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) return; failed_csum = 0; @@ -3744,15 +3781,19 @@ report_problem: } continue; } + e2fsck_pass1_fix_lock(ctx); e2fsck_read_bitmaps(ctx); pb->inode_modified = 1; pctx->errcode = ext2fs_extent_delete(ehandle, 0); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) { pctx->str = "ext2fs_extent_delete"; return; } + e2fsck_pass1_fix_lock(ctx); pctx->errcode = ext2fs_extent_fix_parents(ehandle); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode && pctx->errcode != EXT2_ET_NO_CURRENT_NODE) { pctx->str = "ext2fs_extent_fix_parents"; @@ -3821,9 +3862,11 @@ report_problem: pctx->num = e_info.curr_level - 1; problem = PR_1_EXTENT_INDEX_START_INVALID; if (fix_problem(ctx, problem, pctx)) { + e2fsck_pass1_fix_lock(ctx); pb->inode_modified = 1; pctx->errcode = ext2fs_extent_fix_parents(ehandle); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) { pctx->str = "ext2fs_extent_fix_parents"; return; @@ -3887,15 +3930,19 @@ report_problem: pctx->blk = extent.e_lblk; pctx->blk2 = new_lblk; if (fix_problem(ctx, PR_1_COLLAPSE_DBLOCK, pctx)) { + e2fsck_pass1_fix_lock(ctx); extent.e_lblk = new_lblk; pb->inode_modified = 1; pctx->errcode = ext2fs_extent_replace(ehandle, 0, &extent); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) { pctx->errcode = 0; goto alloc_later; } + e2fsck_pass1_fix_lock(ctx); pctx->errcode = ext2fs_extent_fix_parents(ehandle); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) goto failed_add_dir_block; pctx->errcode = ext2fs_extent_goto(ehandle, @@ -3991,8 +4038,10 @@ alloc_later: /* Failed csum but passes checks? Ask to fix checksum. */ if (failed_csum && fix_problem(ctx, PR_1_EXTENT_ONLY_CSUM_INVALID, pctx)) { + e2fsck_pass1_fix_lock(ctx); pb->inode_modified = 1; pctx->errcode = ext2fs_extent_replace(ehandle, 0, &extent); + e2fsck_pass1_fix_unlock(ctx); if (pctx->errcode) return; } diff --git a/e2fsck/util.c b/e2fsck/util.c index 254b4d04..9470068f 100644 --- a/e2fsck/util.c +++ b/e2fsck/util.c @@ -38,6 +38,10 @@ #include #endif +#ifdef HAVE_PTHREAD +#include +#endif + #include "e2fsck.h" extern e2fsck_t e2fsck_global_ctx; /* Try your very best not to use this! */ @@ -570,13 +574,45 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino, } } +#ifdef HAVE_PTHREAD +void e2fsck_pass1_fix_lock(e2fsck_t ctx) +{ + e2fsck_t global_ctx = ctx->global_ctx; + if (!global_ctx) + global_ctx = ctx; + + pthread_mutex_lock(&global_ctx->fs_fix_mutex); +} + +void e2fsck_pass1_fix_unlock(e2fsck_t ctx) +{ + e2fsck_t global_ctx = ctx->global_ctx; + if (!global_ctx) + global_ctx = ctx; + + pthread_mutex_unlock(&global_ctx->fs_fix_mutex); +} +#else +void e2fsck_pass1_fix_lock(e2fsck_t ctx) +{ + +} + +void e2fsck_pass1_fix_unlock(e2fsck_t ctx) +{ + +} +#endif + void e2fsck_write_inode_full(e2fsck_t ctx, unsigned long ino, struct ext2_inode * inode, int bufsize, const char *proc) { errcode_t retval; + e2fsck_pass1_fix_lock(ctx); retval = ext2fs_write_inode_full(ctx->fs, ino, inode, bufsize); + e2fsck_pass1_fix_unlock(ctx); if (retval) { com_err("ext2fs_write_inode", retval, _("while writing inode %lu in %s"), ino, proc); @@ -589,7 +625,9 @@ void e2fsck_write_inode(e2fsck_t ctx, unsigned long ino, { errcode_t retval; + e2fsck_pass1_fix_lock(ctx); retval = ext2fs_write_inode(ctx->fs, ino, inode); + e2fsck_pass1_fix_unlock(ctx); if (retval) { com_err("ext2fs_write_inode", retval, _("while writing inode %lu in %s"), ino, proc); From patchwork Mon Nov 7 12:21:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700591 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=hj2uLpdE; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vpn73QJz23lW for ; Mon, 7 Nov 2022 23:26:57 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpn6cXjz4xDK for ; Mon, 7 Nov 2022 23:26:57 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vpn6Y3Zz4xGj; Mon, 7 Nov 2022 23:26:57 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=hj2uLpdE; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpn4zzcz4xDK for ; Mon, 7 Nov 2022 23:26:57 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232206AbiKGM05 (ORCPT ); Mon, 7 Nov 2022 07:26:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232202AbiKGM04 (ORCPT ); Mon, 7 Nov 2022 07:26:56 -0500 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B97163CE for ; Mon, 7 Nov 2022 04:26:54 -0800 (PST) Received: by mail-pj1-x1035.google.com with SMTP id k5so10402893pjo.5 for ; Mon, 07 Nov 2022 04:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mjBaMoNNSebassHxFOgfxmYP0vCsBMKg9gIvBPY0WPI=; b=hj2uLpdEYwt7ftBn64hCcUFlTftzwNmk6J+aPNnBAHPURNCcTecCYP0LW+GWRkwDM4 402zHtjBDJZPrKxzIyqL1Dv19X4QFDTdfgWowRpWIhmgcxyCcROqAosfDoyFuWsYSa92 HEjd6K0p0fkWIg6nx5Rd6tipXlrnUyUeXmIJOEpx24pDMovOv6O2Blet4I0XT+j0JvDb Nxzvt2sHhJZUBs0+xLtbhoQurS58qBBKFeTSl1YWzqnjDIoIo++irqDpJDHSY0ylhUpG liESa5PuwZCibQpF+37JGJxz6V4sLIzMzRzf4fa5I6RCDMpvydR78MqPap1aHlIKfzmZ 95Pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mjBaMoNNSebassHxFOgfxmYP0vCsBMKg9gIvBPY0WPI=; b=K/EuDCA/ikY1wYtRJGHbFALbIV9gISP8eF20zqMTJmcz9kKfgsDvWTf6JZycSGKCp9 QTc+Dc+L9aEXL+IUSI+T5S8irMiS762J93toCINE6FvNRJRElT/+sX/bszwQEbSjbVc9 7I7kN/CsaNIvwJQAQSUcopqJtmPJbXclWjB7+G8HJvWCt9kfeP8qQczL9GIE/ECpBntd aVgUoy2CObaw3MCCE9y1Jp9yerI+MauJGXwg6dKyfTEeahID1Ml9BJIU9XTuZO9yZ15x 8HpLnJqWWiAvZPR9FfDyf/4sB+EruAskWRk9QcL0aAWrTmsA7zi0JuBQB3c1KNvsm5ul 93/w== X-Gm-Message-State: ACrzQf0myeXO14Vo3yom9K79bXcCx3sR0upA/kw5gv4oLH2Xkr4d2N1S JnpvG1P1jMOV+cMW8ecYOiI= X-Google-Smtp-Source: AMsMyM4QqoxOaILXD/Pr2DlOiohck/7wPeCYbq/zo0sDUzLKv+rPsUX30JENmDuwamYiOP7KDZR0qQ== X-Received: by 2002:a17:90b:524f:b0:212:c22f:fbd1 with SMTP id sh15-20020a17090b524f00b00212c22ffbd1mr52483364pjb.155.1667824013974; Mon, 07 Nov 2022 04:26:53 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id g20-20020a635214000000b0043b565cb57csm4059468pgb.73.2022.11.07.04.26.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:53 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 44/72] e2fsck: move some fixes out of parallel pthreads Date: Mon, 7 Nov 2022 17:51:32 +0530 Message-Id: <0c454e13b21aadde5c5db1be3f76f2a87016a0bc.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong We could only use @found_map_block to find free blocks after we have collectd all used blocks, so something like handle_fs_bad_blocks(), ext2fs_create_resize_inode(), e2fsck_pass1_dupblocks() really should be handled after all threads has been finished. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 4 + e2fsck/pass1.c | 296 ++++++++++++++++----------- e2fsck/util.c | 36 +++- tests/f_multithread/expect.1 | 2 +- tests/f_multithread_logfile/expect.1 | 2 +- tests/f_multithread_no/expect.1 | 2 +- 6 files changed, 210 insertions(+), 132 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 5356e172..55f75042 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -487,6 +487,8 @@ struct e2fsck_struct { #ifdef HAVE_PTHREAD /* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex; + /* protect block_found_map, block_dup_map */ + pthread_mutex_t fs_block_map_mutex; #endif /* Fast commit replay state */ struct e2fsck_fc_replay_state fc_replay_state; @@ -793,6 +795,8 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs, unsigned long long get_memory_size(void); extern void e2fsck_pass1_fix_lock(e2fsck_t ctx); extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_lock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_unlock(e2fsck_t ctx); /* unix.c */ extern void e2fsck_clear_progbar(e2fsck_t ctx); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index c68e6957..f156d4e1 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -753,11 +753,15 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx, if (i >= 4) not_device++; + e2fsck_pass1_block_map_lock(ctx); if (blk < ctx->fs->super->s_first_data_block || blk >= ext2fs_blocks_count(ctx->fs->super) || ext2fs_fast_test_block_bitmap2(ctx->block_found_map, - blk)) + blk)) { + e2fsck_pass1_block_map_unlock(ctx); return; /* Invalid block, can't be dir */ + } + e2fsck_pass1_block_map_unlock(ctx); } blk = inode->i_block[0]; } @@ -891,19 +895,15 @@ static void reserve_block_for_root_repair(e2fsck_t ctx) errcode_t err; ext2_filsys fs = ctx->fs; - e2fsck_pass1_fix_lock(ctx); ctx->root_repair_block = 0; if (ext2fs_test_inode_bitmap2(ctx->inode_used_map, EXT2_ROOT_INO)) - goto out; + return; err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk); if (err) - goto out; + return; ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); ctx->root_repair_block = blk; -out: - e2fsck_pass1_fix_unlock(ctx); - return; } static void reserve_block_for_lnf_repair(e2fsck_t ctx) @@ -914,18 +914,15 @@ static void reserve_block_for_lnf_repair(e2fsck_t ctx) static const char name[] = "lost+found"; ext2_ino_t ino; - e2fsck_pass1_fix_lock(ctx); ctx->lnf_repair_block = 0; if (!ext2fs_lookup(fs, EXT2_ROOT_INO, name, sizeof(name)-1, 0, &ino)) - goto out; + return; err = ext2fs_new_block2(fs, 0, ctx->block_found_map, &blk); if (err) - goto out; + return; ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); ctx->lnf_repair_block = blk; -out: - e2fsck_pass1_fix_unlock(ctx); return; } @@ -1182,6 +1179,118 @@ static int e2fsck_should_abort(e2fsck_t ctx) return 0; } +/* + * We need call mark_table_blocks() before multiple + * thread start, since all known system blocks should be + * marked and checked later. + */ +static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) +{ + struct problem_context pctx; + ext2_filsys fs = ctx->fs; + + clear_problem_context(&pctx); + if (!(ctx->options & E2F_OPT_PREEN)) + fix_problem(ctx, PR_1_PASS_HEADER, &pctx); + + pctx.errcode = e2fsck_allocate_subcluster_bitmap(ctx->fs, + _("in-use block map"), EXT2FS_BMAP64_RBTREE, + "block_found_map", &ctx->block_found_map); + if (pctx.errcode) { + pctx.num = 1; + fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); + ctx->flags |= E2F_FLAG_ABORT; + return pctx.errcode; + } + pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs, + _("metadata block map"), EXT2FS_BMAP64_RBTREE, + "block_metadata_map", &ctx->block_metadata_map); + if (pctx.errcode) { + pctx.num = 1; + fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); + ctx->flags |= E2F_FLAG_ABORT; + return pctx.errcode; + } + + mark_table_blocks(ctx); + pctx.errcode = ext2fs_convert_subcluster_bitmap(ctx->fs, + &ctx->block_found_map); + if (pctx.errcode) { + fix_problem(ctx, PR_1_CONVERT_SUBCLUSTER, &pctx); + ctx->flags |= E2F_FLAG_ABORT; + return pctx.errcode; + } + + if (ext2fs_has_feature_mmp(fs->super) && + fs->super->s_mmp_block > fs->super->s_first_data_block && + fs->super->s_mmp_block < ext2fs_blocks_count(fs->super)) + ext2fs_mark_block_bitmap2(ctx->block_found_map, + fs->super->s_mmp_block); + + return 0; +} + +static void e2fsck_pass1_post(e2fsck_t ctx) +{ + struct problem_context pctx; + ext2_filsys fs = ctx->fs; + char *block_buf; + + reserve_block_for_root_repair(ctx); + reserve_block_for_lnf_repair(ctx); + + if (ctx->invalid_bitmaps) + handle_fs_bad_blocks(ctx); + + if (ctx->flags & E2F_FLAG_RESIZE_INODE) { + struct ext2_inode *inode; + int inode_size = EXT2_INODE_SIZE(fs->super); + inode = e2fsck_allocate_memory(ctx, inode_size, + "scratch inode"); + + clear_problem_context(&pctx); + pctx.errcode = ext2fs_create_resize_inode(fs); + if (pctx.errcode) { + if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE, + &pctx)) { + ctx->flags |= E2F_FLAG_ABORT; + ext2fs_free_mem(&inode); + ext2fs_free_mem(&block_buf); + return; + } + pctx.errcode = 0; + } + if (!pctx.errcode) { + e2fsck_read_inode(ctx, EXT2_RESIZE_INO, inode, + "recreate inode"); + inode->i_mtime = ctx->now; + e2fsck_write_inode(ctx, EXT2_RESIZE_INO, inode, + "recreate inode"); + } + ctx->flags &= ~E2F_FLAG_RESIZE_INODE; + ext2fs_free_mem(&inode); + } + + if (ctx->flags & E2F_FLAG_RESTART) { + ext2fs_free_mem(&block_buf); + return; + } + + if (ctx->block_dup_map) { + if (ctx->options & E2F_OPT_PREEN) { + clear_problem_context(&pctx); + fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx); + } + block_buf = + (char *)e2fsck_allocate_memory(ctx, + ctx->fs->blocksize * 3, + "block interate buffer"); + e2fsck_pass1_dupblocks(ctx, block_buf); + ext2fs_free_mem(&block_buf); + } +} + + void e2fsck_pass1_run(e2fsck_t ctx) { int i; @@ -1220,9 +1329,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs); pass1_readahead(ctx, &ra_group, &ino_threshold); - if (!(ctx->options & E2F_OPT_PREEN)) - fix_problem(ctx, PR_1_PASS_HEADER, &pctx); - if (ext2fs_has_feature_dir_index(fs->super) && !(ctx->options & E2F_OPT_NO)) { if (ext2fs_u32_list_create(&ctx->dirs_to_hash, 50)) @@ -1271,24 +1377,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->flags |= E2F_FLAG_ABORT; return; } - pctx.errcode = e2fsck_allocate_subcluster_bitmap(fs, - _("in-use block map"), EXT2FS_BMAP64_RBTREE, - "block_found_map", &ctx->block_found_map); - if (pctx.errcode) { - pctx.num = 1; - fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); - ctx->flags |= E2F_FLAG_ABORT; - return; - } - pctx.errcode = e2fsck_allocate_block_bitmap(fs, - _("metadata block map"), EXT2FS_BMAP64_RBTREE, - "block_metadata_map", &ctx->block_metadata_map); - if (pctx.errcode) { - pctx.num = 1; - fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); - ctx->flags |= E2F_FLAG_ABORT; - return; - } if (casefold_fs) { pctx.errcode = e2fsck_allocate_inode_bitmap(fs, @@ -1344,14 +1432,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) } } - mark_table_blocks(ctx); - pctx.errcode = ext2fs_convert_subcluster_bitmap(fs, - &ctx->block_found_map); - if (pctx.errcode) { - fix_problem(ctx, PR_1_CONVERT_SUBCLUSTER, &pctx); - ctx->flags |= E2F_FLAG_ABORT; - goto endit; - } block_buf = (char *) e2fsck_allocate_memory(ctx, fs->blocksize * 3, "block interate buffer"); if (EXT2_INODE_SIZE(fs->super) == EXT2_GOOD_OLD_INODE_SIZE) @@ -1385,12 +1465,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) fs->super->s_mkfs_time < fs->super->s_inodes_count)) low_dtime_check = 0; - if (ext2fs_has_feature_mmp(fs->super) && - fs->super->s_mmp_block > fs->super->s_first_data_block && - fs->super->s_mmp_block < ext2fs_blocks_count(fs->super)) - ext2fs_mark_block_bitmap2(ctx->block_found_map, - fs->super->s_mmp_block); - /* Set up ctx->lost_and_found if possible */ (void) e2fsck_get_lost_and_found(ctx, 0); @@ -1721,8 +1795,10 @@ void e2fsck_pass1_run(e2fsck_t ctx) failed_csum = 0; } + e2fsck_pass1_block_map_lock(ctx); pctx.errcode = ext2fs_copy_bitmap(ctx->block_found_map, &pb.fs_meta_blocks); + e2fsck_pass1_block_map_unlock(ctx); if (pctx.errcode) { pctx.num = 4; fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); @@ -2069,9 +2145,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ext2fs_close_inode_scan(scan); scan = NULL; - reserve_block_for_root_repair(ctx); - reserve_block_for_lnf_repair(ctx); - /* * If any extended attribute blocks' reference counts need to * be adjusted, either up (ctx->refcount_extra), or down @@ -2099,11 +2172,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->ea_block_quota_inodes = 0; } - if (ctx->invalid_bitmaps) { - e2fsck_pass1_fix_lock(ctx); - handle_fs_bad_blocks(ctx); - e2fsck_pass1_fix_unlock(ctx); - } /* We don't need the block_ea_map any more */ if (ctx->block_ea_map) { @@ -2114,29 +2182,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) /* We don't need the encryption policy => ID map any more */ destroy_encryption_policy_map(ctx); - if (ctx->flags & E2F_FLAG_RESIZE_INODE) { - clear_problem_context(&pctx); - e2fsck_pass1_fix_lock(ctx); - pctx.errcode = ext2fs_create_resize_inode(fs); - e2fsck_pass1_fix_unlock(ctx); - if (pctx.errcode) { - if (!fix_problem(ctx, PR_1_RESIZE_INODE_CREATE, - &pctx)) { - ctx->flags |= E2F_FLAG_ABORT; - goto endit; - } - pctx.errcode = 0; - } - if (!pctx.errcode) { - e2fsck_read_inode(ctx, EXT2_RESIZE_INO, inode, - "recreate inode"); - inode->i_mtime = ctx->now; - e2fsck_write_inode(ctx, EXT2_RESIZE_INO, inode, - "recreate inode"); - } - ctx->flags &= ~E2F_FLAG_RESIZE_INODE; - } - if (ctx->flags & E2F_FLAG_RESTART) { /* * Only the master copy of the superblock and block @@ -2161,13 +2206,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) } } - if (ctx->block_dup_map) { - if (ctx->options & E2F_OPT_PREEN) { - clear_problem_context(&pctx); - fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx); - } - e2fsck_pass1_dupblocks(ctx, block_buf); - } ctx->flags |= E2F_FLAG_ALLOC_OK; endit: e2fsck_use_inode_shortcuts(ctx, 0); @@ -2288,10 +2326,10 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre assert(global_ctx->inode_reg_map == NULL); assert(global_ctx->inodes_to_rebuild == NULL); - assert(global_ctx->block_found_map == NULL); + assert(global_ctx->block_found_map != NULL); + assert(global_ctx->block_metadata_map != NULL); assert(global_ctx->block_dup_map == NULL); assert(global_ctx->block_ea_map == NULL); - assert(global_ctx->block_metadata_map == NULL); assert(global_ctx->fs->dblist == NULL); retval = ext2fs_get_mem(sizeof(struct e2fsck_struct), &thread_context); @@ -2455,10 +2493,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; - ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; - ext2fs_block_bitmap block_metadata_map = global_ctx->block_metadata_map; ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; ext2_icount_t inode_count = global_ctx->inode_count; ext2_icount_t inode_link_info = global_ctx->inode_link_info; @@ -2501,10 +2537,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->inode_imagic_map = inode_imagic_map; global_ctx->inodes_to_rebuild = inodes_to_rebuild; global_ctx->inode_reg_map = inode_reg_map; - global_ctx->block_found_map = block_found_map; - global_ctx->block_dup_map = block_dup_map; global_ctx->block_ea_map = block_ea_map; - global_ctx->block_metadata_map = block_metadata_map; + global_ctx->block_dup_map = block_dup_map; global_ctx->dir_info = dir_info; e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); global_ctx->dx_dir_info = dx_dir_info; @@ -2601,26 +2635,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx &global_ctx->inodes_to_rebuild); if (retval) return retval; - retval = e2fsck_pass1_merge_bitmap(global_fs, - &thread_ctx->block_found_map, - &global_ctx->block_found_map); - if (retval) - return retval; - retval = e2fsck_pass1_merge_bitmap(global_fs, - &thread_ctx->block_dup_map, - &global_ctx->block_dup_map); - if (retval) - return retval; retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->block_ea_map, &global_ctx->block_ea_map); if (retval) return retval; - retval = e2fsck_pass1_merge_bitmap(global_fs, - &thread_ctx->block_metadata_map, - &global_ctx->block_metadata_map); - if (retval) - return retval; return retval; } @@ -2637,10 +2656,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map); e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map); e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild); - e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map); - e2fsck_pass1_free_bitmap(&thread_ctx->block_dup_map); e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); - e2fsck_pass1_free_bitmap(&thread_ctx->block_metadata_map); e2fsck_free_dir_info(thread_ctx); ext2fs_free_icount(thread_ctx->inode_count); ext2fs_free_icount(thread_ctx->inode_link_info); @@ -2827,6 +2843,7 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx) errcode_t retval; pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL); + pthread_mutex_init(&global_ctx->fs_block_map_mutex, NULL); retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); if (retval) { com_err(global_ctx->program_name, retval, @@ -2881,13 +2898,19 @@ static int multiple_threads_supported(e2fsck_t ctx) void e2fsck_pass1(e2fsck_t ctx) { + errcode_t retval; + init_ext2_max_sizes(); + retval = e2fsck_pass1_prepare(ctx); + if (retval) + return; #ifdef HAVE_PTHREAD if (ctx->options & E2F_OPT_MULTITHREAD && multiple_threads_supported(ctx)) e2fsck_pass1_multithread(ctx); else #endif e2fsck_pass1_run(ctx); + e2fsck_pass1_post(ctx); } #undef FINISH_INODE_LOOP @@ -3094,9 +3117,14 @@ static void alloc_imagic_map(e2fsck_t ctx) * WARNING: Assumes checks have already been done to make sure block * is valid. This is true in both process_block and process_bad_block. */ -static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) +static _INLINE_ void mark_block_used_unlocked(e2fsck_t ctx, blk64_t block) { - struct problem_context pctx; + struct problem_context pctx; + e2fsck_t global_ctx; + + global_ctx = ctx->global_ctx; + if (!global_ctx) + global_ctx = ctx; clear_problem_context(&pctx); @@ -3105,11 +3133,15 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) !(ctx->options & E2F_OPT_UNSHARE_BLOCKS)) { return; } - if (!ctx->block_dup_map) { + /** + * this should be safe because this operation has + * been serialized by mutex. + */ + if (!global_ctx->block_dup_map) { pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs, _("multiply claimed block map"), EXT2FS_BMAP64_RBTREE, "block_dup_map", - &ctx->block_dup_map); + &global_ctx->block_dup_map); if (pctx.errcode) { pctx.num = 3; fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, @@ -3119,12 +3151,20 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) return; } } - ext2fs_fast_mark_block_bitmap2(ctx->block_dup_map, block); + ext2fs_fast_mark_block_bitmap2(global_ctx->block_dup_map, block); } else { ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, block); } } +static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) +{ + e2fsck_pass1_block_map_lock(ctx); + mark_block_used_unlocked(ctx, block); + e2fsck_pass1_block_map_unlock(ctx); + +} + /* * When cluster size is greater than one block, it is caller's responsibility * to make sure block parameter starts at a cluster boundary. @@ -3132,14 +3172,16 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block, unsigned int num) { - if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num)) + e2fsck_pass1_block_map_lock(ctx); + if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num)) { ext2fs_mark_block_bitmap_range2(ctx->block_found_map, block, num); - else { + } else { unsigned int i; for (i = 0; i < num; i += EXT2FS_CLUSTER_RATIO(ctx->fs)) - mark_block_used(ctx, block + i); + mark_block_used_unlocked(ctx, block + i); } + e2fsck_pass1_block_map_unlock(ctx); } static errcode_t _INLINE_ e2fsck_write_ext_attr3(e2fsck_t ctx, blk64_t block, @@ -4747,10 +4789,12 @@ static int process_bad_block(ext2_filsys fs, } if (blockcnt < 0) { + e2fsck_pass1_block_map_lock(ctx); if (ext2fs_test_block_bitmap2(p->fs_meta_blocks, blk)) { p->bbcheck = 1; if (fix_problem(ctx, PR_1_BB_FS_BLOCK, pctx)) { *block_nr = 0; + e2fsck_pass1_block_map_unlock(ctx); return BLOCK_CHANGED; } } else if (ext2fs_test_block_bitmap2(ctx->block_found_map, @@ -4759,12 +4803,17 @@ static int process_bad_block(ext2_filsys fs, if (fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK, pctx)) { *block_nr = 0; + e2fsck_pass1_block_map_unlock(ctx); return BLOCK_CHANGED; } - if (e2fsck_should_abort(ctx)) + if (e2fsck_should_abort(ctx)) { + e2fsck_pass1_block_map_unlock(ctx); return BLOCK_ABORT; - } else - mark_block_used(ctx, blk); + } + } else { + mark_block_used_unlocked(ctx, blk); + } + e2fsck_pass1_block_map_unlock(ctx); return 0; } #if 0 @@ -4777,10 +4826,13 @@ static int process_bad_block(ext2_filsys fs, * there's an overlap between the filesystem table blocks * (bitmaps and inode table) and the bad block list. */ + e2fsck_pass1_block_map_lock(ctx); if (!ext2fs_test_block_bitmap2(ctx->block_found_map, blk)) { ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); + e2fsck_pass1_block_map_unlock(ctx); return 0; } + e2fsck_pass1_block_map_unlock(ctx); /* * Try to find the where the filesystem block was used... */ @@ -4935,6 +4987,7 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group, fix_problem(ctx, (old_block ? PR_1_RELOC_FROM_TO : PR_1_RELOC_TO), &pctx); pctx.blk2 = 0; + e2fsck_pass1_block_map_lock(ctx); for (i = 0; i < num; i++) { pctx.blk = i; ext2fs_mark_block_bitmap2(ctx->block_found_map, (*new_block)+i); @@ -4955,6 +5008,7 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group, if (pctx.errcode) fix_problem(ctx, PR_1_RELOC_WRITE_ERR, &pctx); } + e2fsck_pass1_block_map_unlock(ctx); ext2fs_free_mem(&buf); } diff --git a/e2fsck/util.c b/e2fsck/util.c index 9470068f..93cd96c7 100644 --- a/e2fsck/util.c +++ b/e2fsck/util.c @@ -575,23 +575,34 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino, } #ifdef HAVE_PTHREAD +#define e2fsck_get_lock_context(ctx) \ + e2fsck_t global_ctx = ctx->global_ctx; \ + if (!global_ctx) \ + global_ctx = ctx; \ + void e2fsck_pass1_fix_lock(e2fsck_t ctx) { - e2fsck_t global_ctx = ctx->global_ctx; - if (!global_ctx) - global_ctx = ctx; - + e2fsck_get_lock_context(ctx); pthread_mutex_lock(&global_ctx->fs_fix_mutex); } void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { - e2fsck_t global_ctx = ctx->global_ctx; - if (!global_ctx) - global_ctx = ctx; - + e2fsck_get_lock_context(ctx); pthread_mutex_unlock(&global_ctx->fs_fix_mutex); } + +void e2fsck_pass1_block_map_lock(e2fsck_t ctx) +{ + e2fsck_get_lock_context(ctx); + pthread_mutex_lock(&global_ctx->fs_block_map_mutex); +} + +void e2fsck_pass1_block_map_unlock(e2fsck_t ctx) +{ + e2fsck_get_lock_context(ctx); + pthread_mutex_unlock(&global_ctx->fs_block_map_mutex); +} #else void e2fsck_pass1_fix_lock(e2fsck_t ctx) { @@ -601,6 +612,15 @@ void e2fsck_pass1_fix_lock(e2fsck_t ctx) void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { +} + +void e2fsck_pass1_block_map_lock(e2fsck_t ctx) +{ + +} + +void e2fsck_pass1_block_map_unlock(e2fsck_t ctx) +{ } #endif diff --git a/tests/f_multithread/expect.1 b/tests/f_multithread/expect.1 index 8d2acd2b..4db68d9e 100644 --- a/tests/f_multithread/expect.1 +++ b/tests/f_multithread/expect.1 @@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity diff --git a/tests/f_multithread_logfile/expect.1 b/tests/f_multithread_logfile/expect.1 index 8d2acd2b..4db68d9e 100644 --- a/tests/f_multithread_logfile/expect.1 +++ b/tests/f_multithread_logfile/expect.1 @@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity diff --git a/tests/f_multithread_no/expect.1 b/tests/f_multithread_no/expect.1 index f85a3382..eda2fcac 100644 --- a/tests/f_multithread_no/expect.1 +++ b/tests/f_multithread_no/expect.1 @@ -1,7 +1,7 @@ ext2fs_open2: Bad magic number in super-block ../e2fsck/e2fsck: Superblock invalid, trying backup blocks... +Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scan group range [0, 2) -[Thread 0] Pass 1: Checking inodes, blocks, and sizes [Thread 0] Scanned group range [0, 2), inodes 3008 Pass 2: Checking directory structure Pass 3: Checking directory connectivity From patchwork Mon Nov 7 12:21:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700592 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=n9m30imD; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vpz3Z8Hz23lW for ; Mon, 7 Nov 2022 23:27:07 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpz37Hlz4xDK for ; Mon, 7 Nov 2022 23:27:07 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vpz35QCz4xGj; Mon, 7 Nov 2022 23:27:07 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=n9m30imD; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vpz31Rmz4xDK for ; Mon, 7 Nov 2022 23:27:07 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232201AbiKGM1G (ORCPT ); Mon, 7 Nov 2022 07:27:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232216AbiKGM1B (ORCPT ); Mon, 7 Nov 2022 07:27:01 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6334C6405 for ; Mon, 7 Nov 2022 04:27:00 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id l22-20020a17090a3f1600b00212fbbcfb78so14408467pjc.3 for ; Mon, 07 Nov 2022 04:27:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=333vC0dW8Rw1b9p+EPFXewePT2jZKqjSdzIZxgfazzE=; b=n9m30imDnkpTUetumSqddSfkilD4PFWe0NK9k2bpqWTGPGf0PBcJSCdocXdDTiQ1cO MTuJlMssBLF3ozxr/A6Xc2u19iDxvbs5wCer4Me7Ofp9lUbaZHuWRz5v54VMCyAUBmiJ SqlK+/m8dbhOVWDOTEsNKH8VWgsYoxVLnXnGHNfvdGvgnp1SRXJ3iJ3VL0Eu0XY/GVQ9 ksRPsa6rvbQQrCLUxG+MkgQ89tVdo057Pne7XQU3TA8u/FOkAjUil18DDAOIllPk2QM0 ySQmAteE0eZjS+ai/IR+L8b8yuiFOO4ML/0TjtV/UTeMev7xLFAPOGCIHKcF23dLy/66 g4DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=333vC0dW8Rw1b9p+EPFXewePT2jZKqjSdzIZxgfazzE=; b=Vf30AHxFJ/DnOcnIOy+/TbRv2MkdKtXf64Imbimpo4crI3EDJiSxfLX8Hd8zR5PoYj L6eGw7HNZKdJFEF9Ftqhn12kFfz5wONYWA/XRLGdsn7uTuhEDQK+4kI1jRPJ5VpbcS4h b9wsgZ/JtCsNuTaAg6EMsFJpt3sosEoz8PC2L2emv9j9kYxAD6R5syrJRgJ+nSfqoA6d YhCAhjw9WJyLqRgW55oXnXxDREmhpTbuYZAzD3TkPJkPjZlKiZRSkcgNkS5LeXJi/lOI mQ41X//bnh3B3tSXH2KNqIqx+SvtFOFQHCQowFivAzq0mzU+Cevwq5ewuqjAnPd7zpVK yEOQ== X-Gm-Message-State: ACrzQf0uboTx9X6+yMNppOE6j5fzO/1M4ol1aMQ53duNsqDdMcuqLXb1 rcB05Z/l8hdWoQDfq5dG8Ls= X-Google-Smtp-Source: AMsMyM7O/8CwMQC1Bov0068xZwKQ69YaLvDQ/zDcAIaFv25PBTwMb/xbU5A0Z2rgzsCtkV0nQ4n9YA== X-Received: by 2002:a17:90a:6286:b0:213:b01e:4290 with SMTP id d6-20020a17090a628600b00213b01e4290mr48134128pjj.42.1667824019901; Mon, 07 Nov 2022 04:26:59 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id d7-20020a170902cec700b00186ff402525sm4892045plg.213.2022.11.07.04.26.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:26:59 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 45/72] e2fsck: split and merge invalid bitmaps Date: Mon, 7 Nov 2022 17:51:33 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Invalid bitmaps are splitted per thread, and we should merge them after thread finish. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index f156d4e1..e268441a 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2309,6 +2309,62 @@ static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, return 0; } +static void e2fsck_pass1_copy_invalid_bitmaps(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + dgrp_t i, j; + dgrp_t grp_start = thread_ctx->thread_info.et_group_start; + dgrp_t grp_end = thread_ctx->thread_info.et_group_end; + dgrp_t total = grp_end - grp_start; + + thread_ctx->invalid_inode_bitmap_flag = + e2fsck_allocate_memory(global_ctx, sizeof(int) * total, + "invalid_inode_bitmap"); + thread_ctx->invalid_block_bitmap_flag = + e2fsck_allocate_memory(global_ctx, sizeof(int) * total, + "invalid_block_bitmap"); + thread_ctx->invalid_inode_table_flag = + e2fsck_allocate_memory(global_ctx, sizeof(int) * total, + "invalid_inode_table"); + + memcpy(thread_ctx->invalid_block_bitmap_flag, + &global_ctx->invalid_block_bitmap_flag[grp_start], + total * sizeof(int)); + memcpy(thread_ctx->invalid_inode_bitmap_flag, + &global_ctx->invalid_inode_bitmap_flag[grp_start], + total * sizeof(int)); + memcpy(thread_ctx->invalid_inode_table_flag, + &global_ctx->invalid_inode_table_flag[grp_start], + total * sizeof(int)); + + thread_ctx->invalid_bitmaps = 0; + for (i = grp_start, j = 0; i < grp_end; i++, j++) { + if (thread_ctx->invalid_block_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; + if (thread_ctx->invalid_inode_bitmap_flag[j]) + thread_ctx->invalid_bitmaps++; + if (thread_ctx->invalid_inode_table_flag[j]) + thread_ctx->invalid_bitmaps++; + } +} + +static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + dgrp_t i, j; + dgrp_t grp_start = thread_ctx->thread_info.et_group_start; + dgrp_t grp_end = thread_ctx->thread_info.et_group_end; + dgrp_t total = grp_end - grp_start; + + memcpy(&global_ctx->invalid_block_bitmap_flag[grp_start], + thread_ctx->invalid_block_bitmap_flag, total * sizeof(int)); + memcpy(&global_ctx->invalid_inode_bitmap_flag[grp_start], + thread_ctx->invalid_inode_bitmap_flag, total * sizeof(int)); + memcpy(&global_ctx->invalid_inode_table_flag[grp_start], + thread_ctx->invalid_inode_table_flag, total * sizeof(int)); + global_ctx->invalid_bitmaps += thread_ctx->invalid_bitmaps; +} + static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx, int thread_index, int num_threads) { @@ -2384,6 +2440,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre goto out_fs; } *thread_ctx = thread_context; + e2fsck_pass1_copy_invalid_bitmaps(global_ctx, thread_context); return 0; out_fs: ext2fs_merge_fs(&thread_fs); @@ -2519,6 +2576,10 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; quota_ctx_t qctx = global_ctx->qctx; + int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; + int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; + int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; + int invalid_bitmaps = global_ctx->invalid_bitmaps; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2598,6 +2659,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx); if (retval) return retval; + global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; + global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; + global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; + global_ctx->invalid_bitmaps = invalid_bitmaps; + e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, &thread_ctx->inode_used_map, @@ -2663,6 +2729,9 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) if (thread_ctx->dirs_to_hash) ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash); quota_release_context(&thread_ctx->qctx); + ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag); + ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag); + ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag); if (thread_ctx->logf) fclose(thread_ctx->logf); @@ -2682,6 +2751,8 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, int i; struct e2fsck_thread_info *pinfo; + /* merge invalid bitmaps will recalculate it */ + global_ctx->invalid_bitmaps = 0; for (i = 0; i < num_threads; i++) { pinfo = &infos[i]; From patchwork Mon Nov 7 12:21:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700593 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Y2Vcq8aJ; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vq36t2hz23lW for ; Mon, 7 Nov 2022 23:27:11 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vq36Q68z4xDK for ; Mon, 7 Nov 2022 23:27:11 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vq36MY3z4xGj; Mon, 7 Nov 2022 23:27:11 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Y2Vcq8aJ; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vq34pBBz4xDK for ; Mon, 7 Nov 2022 23:27:11 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232237AbiKGM1L (ORCPT ); Mon, 7 Nov 2022 07:27:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232210AbiKGM1H (ORCPT ); Mon, 7 Nov 2022 07:27:07 -0500 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 454616247 for ; Mon, 7 Nov 2022 04:27:06 -0800 (PST) Received: by mail-pg1-x531.google.com with SMTP id r18so10296258pgr.12 for ; Mon, 07 Nov 2022 04:27:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CZD2Jr2yW63XIeTFZ1Y0HbzT/Is6Ul87Qe/nsH446CI=; b=Y2Vcq8aJzbmapYxHsFQszOfCoQLKuycjBL5CHvWpUW1zNE/nf9NwtGOg7rELVSnag0 X6ENNHfuWjzy22RNhVAFVPYInAJLP1o+B4idblrABtwUIdAVYVc0jtCGMqsdinxro0k3 P+RDiT2waSY5dchAyAkfJRZu4yEmvGnl8smlnvwMDjdqEN9l+P548guA/t8Dv3YhR/kz IHrh9INgSLt36li124AP6hkY+DY9sows7Af113s1I0CVedjXp25X9e7czoWh3GtzNeLM A7Hk/yog9nu5dZ5snmuXWBd14oV6dMiDv9Cbhqs83BpoHs20tKz4a3KHXSESZNEOXyEK hfmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CZD2Jr2yW63XIeTFZ1Y0HbzT/Is6Ul87Qe/nsH446CI=; b=gdGw4F4AXtiAHYryJPk1Fj3gp70cU+gJqQaP9eS2hBb+yGSsLnaXS8dVlSurzT8HKv tLaFO+i2kl1h3H24aM2mF9zA+4QWd9IRbE7xs/aoRRx2tSMqrkQsnUa3sP3p3NHk5JvK nw33HRXeJfZp4aAg0AQNLUE2lW3Yb0bIozNWYHqTXnydFaMKy51gy4HdjlwFEZLmSN+D JuMaxdhNsppgQK5TvCSd588GQ3L9WZEoyFzuYMKV4/xJLHtd92plNlFJbzdOQ5woV4d9 mF6Kab8jCUIUJggrb3bDjqOYVnZN6w03FmAYJffn15AWj1xg3lJNlUQYZmM19QOr0NpJ 7ASQ== X-Gm-Message-State: ACrzQf3oJYMF1cHZbO9S3WhpGNsUjRvd72NcmIKIHFCRQHeI2M4o+LQj VPPOGybLNoEZ6qeXVNRcSNw= X-Google-Smtp-Source: AMsMyM4RpHOh5TKd7rsXDzM2Fm3AdPk74wBz2FP+8M1o5Vt2PhO+iukD+/p1J2jargyDmrB6lE9Ctg== X-Received: by 2002:a65:5942:0:b0:46f:f741:8adc with SMTP id g2-20020a655942000000b0046ff7418adcmr27026815pgu.265.1667824025611; Mon, 07 Nov 2022 04:27:05 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id ei15-20020a17090ae54f00b00213d08fa459sm4260093pjb.17.2022.11.07.04.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:05 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 46/72] e2fsck: merge EA blocks properly Date: Mon, 7 Nov 2022 17:51:34 +0530 Message-Id: <554875d57922b15947ee874e2588d174ec2e6284.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong EA blocks might be shared, merge them carefully. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 244 +++++++++++++++++++++++++++++++++++++++++------- 2 files changed, 212 insertions(+), 33 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 55f75042..beac7054 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -333,6 +333,7 @@ struct e2fsck_struct { ext2_refcount_t refcount; ext2_refcount_t refcount_extra; + ext2_refcount_t refcount_orig; /* * Quota blocks and inodes to be charged for each ea block. diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index e268441a..06306a17 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -680,14 +680,14 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx, LINUX_S_ISLNK(inode->i_mode) || inode->i_block[0] == 0) return; - /* + /* * Check the block numbers in the i_block array for validity: * zero blocks are skipped (but the first one cannot be zero - * see above), other blocks are checked against the first and * max data blocks (from the the superblock) and against the * block bitmap. Any invalid block found means this cannot be * a directory. - * + * * If there are non-zero blocks past the fourth entry, then * this cannot be a device file: we remember that for the next * check. @@ -1234,14 +1234,39 @@ static void e2fsck_pass1_post(e2fsck_t ctx) { struct problem_context pctx; ext2_filsys fs = ctx->fs; - char *block_buf; + char *block_buf = + (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3, + "block interate buffer"); reserve_block_for_root_repair(ctx); reserve_block_for_lnf_repair(ctx); + /* + * If any extended attribute blocks' reference counts need to + * be adjusted, either up (ctx->refcount_extra), or down + * (ctx->refcount), then fix them. + */ + if (ctx->refcount) { + adjust_extattr_refcount(ctx, ctx->refcount, block_buf, -1); + ea_refcount_free(ctx->refcount); + ctx->refcount = 0; + } + if (ctx->refcount_extra) { + adjust_extattr_refcount(ctx, ctx->refcount_extra, + block_buf, +1); + ea_refcount_free(ctx->refcount_extra); + ctx->refcount_extra = 0; + } + if (ctx->invalid_bitmaps) handle_fs_bad_blocks(ctx); + /* We don't need the block_ea_map any more */ + if (ctx->block_ea_map) { + ext2fs_free_block_bitmap(ctx->block_ea_map); + ctx->block_ea_map = 0; + } + if (ctx->flags & E2F_FLAG_RESIZE_INODE) { struct ext2_inode *inode; int inode_size = EXT2_INODE_SIZE(fs->super); @@ -1281,10 +1306,6 @@ static void e2fsck_pass1_post(e2fsck_t ctx) clear_problem_context(&pctx); fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx); } - block_buf = - (char *)e2fsck_allocate_memory(ctx, - ctx->fs->blocksize * 3, - "block interate buffer"); e2fsck_pass1_dupblocks(ctx, block_buf); ext2fs_free_mem(&block_buf); } @@ -2145,23 +2166,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ext2fs_close_inode_scan(scan); scan = NULL; - /* - * If any extended attribute blocks' reference counts need to - * be adjusted, either up (ctx->refcount_extra), or down - * (ctx->refcount), then fix them. - */ - if (ctx->refcount) { - adjust_extattr_refcount(ctx, ctx->refcount, block_buf, -1); - ea_refcount_free(ctx->refcount); - ctx->refcount = 0; - } - if (ctx->refcount_extra) { - adjust_extattr_refcount(ctx, ctx->refcount_extra, - block_buf, +1); - ea_refcount_free(ctx->refcount_extra); - ctx->refcount_extra = 0; - } - if (ctx->ea_block_quota_blocks) { ea_refcount_free(ctx->ea_block_quota_blocks); ctx->ea_block_quota_blocks = 0; @@ -2172,13 +2176,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->ea_block_quota_inodes = 0; } - - /* We don't need the block_ea_map any more */ - if (ctx->block_ea_map) { - ext2fs_free_block_bitmap(ctx->block_ea_map); - ctx->block_ea_map = 0; - } - /* We don't need the encryption policy => ID map any more */ destroy_encryption_policy_map(ctx); @@ -2533,6 +2530,155 @@ static errcode_t e2fsck_pass1_merge_dirs_to_hash(e2fsck_t global_ctx, return retval; } +static errcode_t e2fsck_pass1_merge_ea_inode_refs(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + ea_value_t count; + blk64_t blk; + errcode_t retval; + + if (!thread_ctx->ea_inode_refs) + return 0; + + if (!global_ctx->ea_inode_refs) { + global_ctx->ea_inode_refs = thread_ctx->ea_inode_refs; + thread_ctx->ea_inode_refs = NULL; + return 0; + } + + ea_refcount_intr_begin(thread_ctx->ea_inode_refs); + while (1) { + if ((blk = ea_refcount_intr_next(thread_ctx->ea_inode_refs, + &count)) == 0) + break; + if (!global_ctx->block_ea_map || + !ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map, + blk)) { + retval = ea_refcount_store(global_ctx->ea_inode_refs, + blk, count); + if (retval) + return retval; + } + } + + return retval; +} + +static ea_value_t ea_refcount_usage(e2fsck_t ctx, blk64_t blk, + ea_value_t *orig) +{ + ea_value_t count_cur; + ea_value_t count_extra = 0; + ea_value_t count_orig; + + ea_refcount_fetch(ctx->refcount_orig, blk, &count_orig); + ea_refcount_fetch(ctx->refcount, blk, &count_cur); + /* most of time this is not needed */ + if (ctx->refcount_extra && count_cur == 0) + ea_refcount_fetch(ctx->refcount_extra, blk, &count_extra); + + if (!count_orig) + count_orig = *orig; + else if (orig) + *orig = count_orig; + + return count_orig + count_extra - count_cur; +} + +static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + ea_value_t count; + blk64_t blk; + errcode_t retval = 0; + + if (!thread_ctx->refcount) + return 0; + + if (!global_ctx->refcount) { + global_ctx->refcount = thread_ctx->refcount; + thread_ctx->refcount = NULL; + global_ctx->refcount_extra = thread_ctx->refcount; + thread_ctx->refcount_extra = NULL; + return 0; + } + + ea_refcount_intr_begin(thread_ctx->refcount); + while (1) { + if ((blk = ea_refcount_intr_next(thread_ctx->refcount, + &count)) == 0) + break; + /** + * this EA has never seen before, so just store its + * refcount and refcount_extra into global_ctx if needed. + */ + if (!global_ctx->block_ea_map || + !ext2fs_fast_test_block_bitmap2(global_ctx->block_ea_map, + blk)) { + ea_value_t extra; + + retval = ea_refcount_store(global_ctx->refcount, + blk, count); + if (retval) + return retval; + + if (count > 0 || !thread_ctx->refcount_extra) + continue; + ea_refcount_fetch(thread_ctx->refcount_extra, blk, + &extra); + if (extra == 0) + continue; + + if (!global_ctx->refcount_extra) { + retval = ea_refcount_create(0, + &global_ctx->refcount_extra); + if (retval) + return retval; + } + retval = ea_refcount_store(global_ctx->refcount_extra, + blk, extra); + if (retval) + return retval; + } else { + ea_value_t orig; + ea_value_t thread_usage; + ea_value_t global_usage; + ea_value_t new; + + thread_usage = ea_refcount_usage(thread_ctx, + blk, &orig); + global_usage = ea_refcount_usage(global_ctx, + blk, &orig); + if (thread_usage + global_usage <= orig) { + new = orig - thread_usage - global_usage; + retval = ea_refcount_store(global_ctx->refcount, + blk, new); + if (retval) + return retval; + continue; + } + /* update it is as zero */ + retval = ea_refcount_store(global_ctx->refcount, + blk, 0); + if (retval) + return retval; + /* Ooops, this EA was referenced more than it stated */ + if (!global_ctx->refcount_extra) { + retval = ea_refcount_create(0, + &global_ctx->refcount_extra); + if (retval) + return retval; + } + new = global_usage + thread_usage - orig; + retval = ea_refcount_store(global_ctx->refcount_extra, + blk, new); + if (retval) + return retval; + } + } + + return retval; +} static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) { @@ -2551,7 +2697,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; - ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; ext2_icount_t inode_count = global_ctx->inode_count; ext2_icount_t inode_link_info = global_ctx->inode_link_info; @@ -2580,6 +2725,13 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; int invalid_bitmaps = global_ctx->invalid_bitmaps; + ext2_refcount_t refcount = global_ctx->refcount; + ext2_refcount_t refcount_extra = global_ctx->refcount_extra; + ext2_refcount_t refcount_orig = global_ctx->refcount_orig; + ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks; + ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes; + ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; + ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2598,7 +2750,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->inode_imagic_map = inode_imagic_map; global_ctx->inodes_to_rebuild = inodes_to_rebuild; global_ctx->inode_reg_map = inode_reg_map; - global_ctx->block_ea_map = block_ea_map; global_ctx->block_dup_map = block_dup_map; global_ctx->dir_info = dir_info; e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); @@ -2608,6 +2759,13 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx); global_ctx->inode_count = inode_count; global_ctx->inode_link_info = inode_link_info; + global_ctx->refcount = refcount; + global_ctx->refcount_extra = refcount_extra; + global_ctx->refcount_orig = refcount_orig; + global_ctx->ea_block_quota_blocks = ea_block_quota_blocks; + global_ctx->ea_block_quota_inodes = ea_block_quota_inodes; + global_ctx->block_ea_map = block_ea_map; + global_ctx->ea_inode_refs = ea_inode_refs; global_ctx->fs_directory_count += fs_directory_count; global_ctx->fs_regular_count += fs_regular_count; global_ctx->fs_blockdev_count += fs_blockdev_count; @@ -2654,6 +2812,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx return retval; } + e2fsck_pass1_merge_ea_inode_refs(global_ctx, thread_ctx); + e2fsck_pass1_merge_ea_refcount(global_ctx, thread_ctx); global_ctx->qctx = qctx; retval = quota_merge_and_update_usage(global_ctx->qctx, thread_ctx->qctx); @@ -2723,6 +2883,14 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map); e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild); e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); + if (thread_ctx->refcount) + ea_refcount_free(thread_ctx->refcount); + if (thread_ctx->refcount_extra) + ea_refcount_free(thread_ctx->refcount_extra); + if (thread_ctx->ea_inode_refs) + ea_refcount_free(thread_ctx->ea_inode_refs); + if (thread_ctx->refcount_orig) + ea_refcount_free(thread_ctx->refcount_orig); e2fsck_free_dir_info(thread_ctx); ext2fs_free_icount(thread_ctx->inode_count); ext2fs_free_icount(thread_ctx->inode_link_info); @@ -3370,6 +3538,15 @@ static int check_ext_attr(e2fsck_t ctx, struct problem_context *pctx, /* Create the EA refcount structure if necessary */ if (!ctx->refcount) { + pctx->errcode = ea_refcount_create(0, + &ctx->refcount_orig); + if (pctx->errcode) { + pctx->num = 1; + fix_problem(ctx, PR_1_ALLOCATE_REFCOUNT, pctx); + ctx->flags |= E2F_FLAG_ABORT; + return 0; + } + pctx->errcode = ea_refcount_create(0, &ctx->refcount); if (pctx->errcode) { pctx->num = 1; @@ -3575,6 +3752,7 @@ refcount_fail: inc_ea_inode_refs(ctx, pctx, first, end); ea_refcount_store(ctx->refcount, blk, header->h_refcount - 1); + ea_refcount_store(ctx->refcount_orig, blk, header->h_refcount); mark_block_used(ctx, blk); ext2fs_fast_mark_block_bitmap2(ctx->block_ea_map, blk); return 1; From patchwork Mon Nov 7 12:21:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700594 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RUob6gVC; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vq80FnCz23lW for ; Mon, 7 Nov 2022 23:27:16 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vq76yClz4xDK for ; Mon, 7 Nov 2022 23:27:15 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vq76tvwz4xGj; Mon, 7 Nov 2022 23:27:15 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=RUob6gVC; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vq75KC3z4xDK for ; Mon, 7 Nov 2022 23:27:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232254AbiKGM1P (ORCPT ); Mon, 7 Nov 2022 07:27:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232210AbiKGM1N (ORCPT ); Mon, 7 Nov 2022 07:27:13 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B40A6405 for ; Mon, 7 Nov 2022 04:27:12 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id k7so10909962pll.6 for ; Mon, 07 Nov 2022 04:27:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9DepVf17ZBIX728Wkzehxp6NgK34Qut61oRb1OEKT4E=; b=RUob6gVCqPBfVNTmNRQn2N1gMth/+gik+bW6iZG7/J5SB2DEXuQrEsUf7I/RBqXNW2 5kl18IJb/qBfBoV6BmaBhnnAA8x2UJk0XRQIDKYvsWHloyWNcsyNK06FgdjOceyrPq8E Dp+zKz7zHa7gZbmH/mgboo8mveeBibo04hWXGQGYljt7WcP+K1YUFDExFoId9786dydx f6uQexduQW6gZ+k6cl6j0Z+OgbyXXqUWtzyCcbiSv6nD/yFTn1CYb9DEK2OMlMbTQnNi k+cN3fgTdf2Z1SMxqv0HNPN7NhyNp4PO+wa11H4DFPFfcxv6ORg61VS3dGoTZyYcPnUm cjtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9DepVf17ZBIX728Wkzehxp6NgK34Qut61oRb1OEKT4E=; b=BOSmfA1VxTG42D/hEihmWMG7gT54vEmwf1mS3TW2oBbH1wljd7BMukK8hAstwNOAOP B6NCoJj97LNZnzYNixegfn6J5C+PUMaVBlfixjcU1n6MSf09MyMUylaFETEHHEa+eojM PDiYpQZ+2JgYkzmUzgG6yHfDZbG8dHvZcA0C99C1ABt3JNEgYdnrXJeCOM3BaQf/Y9nX wqyQLEs7WB5UJd8PvvSvL9x4H+T08MLrigYEp+0rOqCXouuE/sRWl43deFCn1RHDV+tN hM5lh4KXCiv8qXrFHAunNndPLXL4Dh4gSyNeKVGchGBuggSdXfZGegu2YlMr2SyR/b0s /xpg== X-Gm-Message-State: ACrzQf3OGBC2FxrlT884RaKoUHPsU1Bwq94DdWkW3oZPxw/XjED1i9Hx Y8Zfrl/IFQLWx+rHDISZrEM= X-Google-Smtp-Source: AMsMyM72mTsCa66htpagyyqOCH4zKwarIwYY/xR5XoyhEvlkGy3UrXQDVM0dkK5sabZI85hMeB7bhQ== X-Received: by 2002:a17:90a:cf06:b0:212:d9ab:811b with SMTP id h6-20020a17090acf0600b00212d9ab811bmr52187883pju.65.1667824031764; Mon, 07 Nov 2022 04:27:11 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id c195-20020a621ccc000000b00550724f8ea0sm4326209pfc.128.2022.11.07.04.27.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:11 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 47/72] e2fsck: kickoff mutex lock for block found map Date: Mon, 7 Nov 2022 17:51:35 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Now @block_found_map is no longer shared by multiple threads, and @block_dup_map need be checked again after threads finish. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 9 +- e2fsck/pass1.c | 171 ++++++++++++++++++------------ e2fsck/util.c | 34 ++++-- tests/f_itable_collision/expect.1 | 3 - 4 files changed, 137 insertions(+), 80 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index beac7054..23e8d2ed 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -211,6 +211,7 @@ struct resource_track { #define E2F_FLAG_TIME_INSANE 0x2000 /* Time is insane */ #define E2F_FLAG_PROBLEMS_FIXED 0x4000 /* At least one problem was fixed */ #define E2F_FLAG_ALLOC_OK 0x8000 /* Can we allocate blocks? */ +#define E2F_FLAG_DUP_BLOCK 0x20000 /* dup block found during pass1 */ #define E2F_RESET_FLAGS (E2F_FLAG_TIME_INSANE | E2F_FLAG_PROBLEMS_FIXED) @@ -489,7 +490,7 @@ struct e2fsck_struct { /* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex; /* protect block_found_map, block_dup_map */ - pthread_mutex_t fs_block_map_mutex; + pthread_rwlock_t fs_block_map_rwlock; #endif /* Fast commit replay state */ struct e2fsck_fc_replay_state fc_replay_state; @@ -796,8 +797,10 @@ extern errcode_t e2fsck_allocate_subcluster_bitmap(ext2_filsys fs, unsigned long long get_memory_size(void); extern void e2fsck_pass1_fix_lock(e2fsck_t ctx); extern void e2fsck_pass1_fix_unlock(e2fsck_t ctx); -extern void e2fsck_pass1_block_map_lock(e2fsck_t ctx); -extern void e2fsck_pass1_block_map_unlock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx); +extern void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx); /* unix.c */ extern void e2fsck_clear_progbar(e2fsck_t ctx); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 06306a17..fdd1f3d6 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -650,6 +650,31 @@ static void check_inode_extra_space(e2fsck_t ctx, struct problem_context *pctx, } +static _INLINE_ int is_blocks_used(e2fsck_t ctx, blk64_t block, + unsigned int num) +{ + int retval; + + /* used to avoid duplicate output from below */ + retval = ext2fs_test_block_bitmap_range2_valid(ctx->block_found_map, + block, num); + if (!retval) + return 0; + + retval = ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num); + if (retval) { + e2fsck_pass1_block_map_r_lock(ctx); + if (ctx->global_ctx) + retval = ext2fs_test_block_bitmap_range2( + ctx->global_ctx->block_found_map, block, num); + e2fsck_pass1_block_map_r_unlock(ctx); + if (retval) + return 0; + } + + return 1; +} + /* * Check to see if the inode might really be a directory, despite i_mode * @@ -753,15 +778,10 @@ static void check_is_really_dir(e2fsck_t ctx, struct problem_context *pctx, if (i >= 4) not_device++; - e2fsck_pass1_block_map_lock(ctx); if (blk < ctx->fs->super->s_first_data_block || blk >= ext2fs_blocks_count(ctx->fs->super) || - ext2fs_fast_test_block_bitmap2(ctx->block_found_map, - blk)) { - e2fsck_pass1_block_map_unlock(ctx); + is_blocks_used(ctx, blk, 1)) return; /* Invalid block, can't be dir */ - } - e2fsck_pass1_block_map_unlock(ctx); } blk = inode->i_block[0]; } @@ -1221,11 +1241,28 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) return pctx.errcode; } + pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs, + _("multiply claimed block map"), + EXT2FS_BMAP64_RBTREE, "block_dup_map", + &ctx->block_dup_map); + if (pctx.errcode) { + pctx.num = 3; + fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, + &pctx); + /* Should never get here */ + ctx->flags |= E2F_FLAG_ABORT; + return pctx.errcode; + } + if (ext2fs_has_feature_mmp(fs->super) && fs->super->s_mmp_block > fs->super->s_first_data_block && fs->super->s_mmp_block < ext2fs_blocks_count(fs->super)) ext2fs_mark_block_bitmap2(ctx->block_found_map, fs->super->s_mmp_block); +#ifdef HAVE_PTHREAD + pthread_mutex_init(&ctx->fs_fix_mutex, NULL); + pthread_rwlock_init(&ctx->fs_block_map_rwlock, NULL); +#endif return 0; } @@ -1302,12 +1339,17 @@ static void e2fsck_pass1_post(e2fsck_t ctx) } if (ctx->block_dup_map) { + if (!(ctx->flags & E2F_FLAG_DUP_BLOCK)) { + ext2fs_free_mem(&block_buf); + return; + } if (ctx->options & E2F_OPT_PREEN) { clear_problem_context(&pctx); fix_problem(ctx, PR_1_DUP_BLOCKS_PREENSTOP, &pctx); } e2fsck_pass1_dupblocks(ctx, block_buf); ext2fs_free_mem(&block_buf); + ctx->flags &= ~E2F_FLAG_DUP_BLOCK; } } @@ -1816,10 +1858,11 @@ void e2fsck_pass1_run(e2fsck_t ctx) failed_csum = 0; } - e2fsck_pass1_block_map_lock(ctx); - pctx.errcode = ext2fs_copy_bitmap(ctx->block_found_map, - &pb.fs_meta_blocks); - e2fsck_pass1_block_map_unlock(ctx); + e2fsck_pass1_block_map_r_lock(ctx); + pctx.errcode = ext2fs_copy_bitmap(ctx->global_ctx ? + ctx->global_ctx->block_found_map : + ctx->block_found_map, &pb.fs_meta_blocks); + e2fsck_pass1_block_map_r_unlock(ctx); if (pctx.errcode) { pctx.num = 4; fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); @@ -2381,7 +2424,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre assert(global_ctx->block_found_map != NULL); assert(global_ctx->block_metadata_map != NULL); - assert(global_ctx->block_dup_map == NULL); + assert(global_ctx->block_dup_map != NULL); assert(global_ctx->block_ea_map == NULL); assert(global_ctx->fs->dblist == NULL); @@ -2391,6 +2434,14 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre return retval; } memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct)); + thread_context->block_dup_map = NULL; + + retval = e2fsck_allocate_block_bitmap(global_ctx->fs, + _("in-use block map"), EXT2FS_BMAP64_RBTREE, + "block_found_map", &thread_context->block_found_map); + if (retval) + goto out_context; + thread_context->global_ctx = global_ctx; retval = ext2fs_clone_fs(global_fs, &thread_fs, EXT2FS_CLONE_BLOCK | EXT2FS_CLONE_INODE | @@ -2442,6 +2493,8 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre out_fs: ext2fs_merge_fs(&thread_fs); out_context: + if (thread_context->block_found_map) + ext2fs_free_mem(&thread_context->block_found_map); ext2fs_free_mem(&thread_context); return retval; } @@ -2696,7 +2749,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; - ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; ext2_icount_t inode_count = global_ctx->inode_count; ext2_icount_t inode_link_info = global_ctx->inode_link_info; @@ -2732,6 +2784,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes; ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; + ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; + ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2751,6 +2805,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->inodes_to_rebuild = inodes_to_rebuild; global_ctx->inode_reg_map = inode_reg_map; global_ctx->block_dup_map = block_dup_map; + global_ctx->block_found_map = block_found_map; global_ctx->dir_info = dir_info; e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); global_ctx->dx_dir_info = dx_dir_info; @@ -2867,6 +2922,23 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx if (retval) return retval; + if (ext2fs_has_feature_shared_blocks(global_fs->super) && + !(global_ctx->options & E2F_OPT_UNSHARE_BLOCKS)) + return 0; + /* + * This need be done after merging block_ea_map + * because ea block might be shared, we need exclude + * them from dup blocks. + */ + e2fsck_pass1_block_map_w_lock(thread_ctx); + retval = ext2fs_merge_bitmap(thread_ctx->block_found_map, + global_ctx->block_found_map, + global_ctx->block_dup_map, + global_ctx->block_ea_map); + e2fsck_pass1_block_map_w_unlock(thread_ctx); + if (retval == EEXIST) + global_ctx->flags |= E2F_FLAG_DUP_BLOCK; + return retval; } @@ -2882,6 +2954,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map); e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map); e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild); + e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map); e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); if (thread_ctx->refcount) ea_refcount_free(thread_ctx->refcount); @@ -3081,8 +3154,6 @@ static void e2fsck_pass1_multithread(e2fsck_t global_ctx) int num_threads = 1; errcode_t retval; - pthread_mutex_init(&global_ctx->fs_fix_mutex, NULL); - pthread_mutex_init(&global_ctx->fs_block_map_mutex, NULL); retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); if (retval) { com_err(global_ctx->program_name, retval, @@ -3356,54 +3427,27 @@ static void alloc_imagic_map(e2fsck_t ctx) * WARNING: Assumes checks have already been done to make sure block * is valid. This is true in both process_block and process_bad_block. */ -static _INLINE_ void mark_block_used_unlocked(e2fsck_t ctx, blk64_t block) +static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) { struct problem_context pctx; - e2fsck_t global_ctx; - - global_ctx = ctx->global_ctx; - if (!global_ctx) - global_ctx = ctx; + e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx; clear_problem_context(&pctx); - if (ext2fs_fast_test_block_bitmap2(ctx->block_found_map, block)) { + if (is_blocks_used(ctx, block, 1)) { if (ext2fs_has_feature_shared_blocks(ctx->fs->super) && !(ctx->options & E2F_OPT_UNSHARE_BLOCKS)) { return; } - /** - * this should be safe because this operation has - * been serialized by mutex. - */ - if (!global_ctx->block_dup_map) { - pctx.errcode = e2fsck_allocate_block_bitmap(ctx->fs, - _("multiply claimed block map"), - EXT2FS_BMAP64_RBTREE, "block_dup_map", - &global_ctx->block_dup_map); - if (pctx.errcode) { - pctx.num = 3; - fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, - &pctx); - /* Should never get here */ - ctx->flags |= E2F_FLAG_ABORT; - return; - } - } + ctx->flags |= E2F_FLAG_DUP_BLOCK; + e2fsck_pass1_block_map_w_lock(ctx); ext2fs_fast_mark_block_bitmap2(global_ctx->block_dup_map, block); + e2fsck_pass1_block_map_w_unlock(ctx); } else { ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, block); } } -static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) -{ - e2fsck_pass1_block_map_lock(ctx); - mark_block_used_unlocked(ctx, block); - e2fsck_pass1_block_map_unlock(ctx); - -} - /* * When cluster size is greater than one block, it is caller's responsibility * to make sure block parameter starts at a cluster boundary. @@ -3411,16 +3455,14 @@ static _INLINE_ void mark_block_used(e2fsck_t ctx, blk64_t block) static _INLINE_ void mark_blocks_used(e2fsck_t ctx, blk64_t block, unsigned int num) { - e2fsck_pass1_block_map_lock(ctx); - if (ext2fs_test_block_bitmap_range2(ctx->block_found_map, block, num)) { + if (!is_blocks_used(ctx, block, num)) { ext2fs_mark_block_bitmap_range2(ctx->block_found_map, block, num); } else { unsigned int i; for (i = 0; i < num; i += EXT2FS_CLUSTER_RATIO(ctx->fs)) - mark_block_used_unlocked(ctx, block + i); + mark_block_used(ctx, block + i); } - e2fsck_pass1_block_map_unlock(ctx); } static errcode_t _INLINE_ e2fsck_write_ext_attr3(e2fsck_t ctx, blk64_t block, @@ -3753,7 +3795,12 @@ refcount_fail: inc_ea_inode_refs(ctx, pctx, first, end); ea_refcount_store(ctx->refcount, blk, header->h_refcount - 1); ea_refcount_store(ctx->refcount_orig, blk, header->h_refcount); - mark_block_used(ctx, blk); + /** + * It might be racy that this block has been merged in the + * global found map. + */ + if (!is_blocks_used(ctx, blk, 1)) + ext2fs_fast_mark_block_bitmap2(ctx->block_found_map, blk); ext2fs_fast_mark_block_bitmap2(ctx->block_ea_map, blk); return 1; @@ -5038,31 +5085,24 @@ static int process_bad_block(ext2_filsys fs, } if (blockcnt < 0) { - e2fsck_pass1_block_map_lock(ctx); if (ext2fs_test_block_bitmap2(p->fs_meta_blocks, blk)) { p->bbcheck = 1; if (fix_problem(ctx, PR_1_BB_FS_BLOCK, pctx)) { *block_nr = 0; - e2fsck_pass1_block_map_unlock(ctx); return BLOCK_CHANGED; } - } else if (ext2fs_test_block_bitmap2(ctx->block_found_map, - blk)) { + } else if (is_blocks_used(ctx, blk, 1)) { p->bbcheck = 1; if (fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK, pctx)) { *block_nr = 0; - e2fsck_pass1_block_map_unlock(ctx); return BLOCK_CHANGED; } - if (e2fsck_should_abort(ctx)) { - e2fsck_pass1_block_map_unlock(ctx); + if (e2fsck_should_abort(ctx)) return BLOCK_ABORT; - } } else { - mark_block_used_unlocked(ctx, blk); + mark_block_used(ctx, blk); } - e2fsck_pass1_block_map_unlock(ctx); return 0; } #if 0 @@ -5075,13 +5115,10 @@ static int process_bad_block(ext2_filsys fs, * there's an overlap between the filesystem table blocks * (bitmaps and inode table) and the bad block list. */ - e2fsck_pass1_block_map_lock(ctx); - if (!ext2fs_test_block_bitmap2(ctx->block_found_map, blk)) { + if (!is_blocks_used(ctx, blk, 1)) { ext2fs_mark_block_bitmap2(ctx->block_found_map, blk); - e2fsck_pass1_block_map_unlock(ctx); return 0; } - e2fsck_pass1_block_map_unlock(ctx); /* * Try to find the where the filesystem block was used... */ @@ -5236,7 +5273,6 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group, fix_problem(ctx, (old_block ? PR_1_RELOC_FROM_TO : PR_1_RELOC_TO), &pctx); pctx.blk2 = 0; - e2fsck_pass1_block_map_lock(ctx); for (i = 0; i < num; i++) { pctx.blk = i; ext2fs_mark_block_bitmap2(ctx->block_found_map, (*new_block)+i); @@ -5257,7 +5293,6 @@ static void new_table_block(e2fsck_t ctx, blk64_t first_block, dgrp_t group, if (pctx.errcode) fix_problem(ctx, PR_1_RELOC_WRITE_ERR, &pctx); } - e2fsck_pass1_block_map_unlock(ctx); ext2fs_free_mem(&buf); } diff --git a/e2fsck/util.c b/e2fsck/util.c index 93cd96c7..5714576a 100644 --- a/e2fsck/util.c +++ b/e2fsck/util.c @@ -592,17 +592,29 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx) pthread_mutex_unlock(&global_ctx->fs_fix_mutex); } -void e2fsck_pass1_block_map_lock(e2fsck_t ctx) +void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx) { e2fsck_get_lock_context(ctx); - pthread_mutex_lock(&global_ctx->fs_block_map_mutex); + pthread_rwlock_wrlock(&global_ctx->fs_block_map_rwlock); } -void e2fsck_pass1_block_map_unlock(e2fsck_t ctx) +void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx) { e2fsck_get_lock_context(ctx); - pthread_mutex_unlock(&global_ctx->fs_block_map_mutex); + pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock); } + +void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx) +{ + e2fsck_get_lock_context(ctx); + pthread_rwlock_rdlock(&global_ctx->fs_block_map_rwlock); +} + +void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx) +{ + e2fsck_get_lock_context(ctx); + pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock); + } #else void e2fsck_pass1_fix_lock(e2fsck_t ctx) { @@ -613,14 +625,24 @@ void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { } +void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx) +{ -void e2fsck_pass1_block_map_lock(e2fsck_t ctx) +} + +void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx) { } -void e2fsck_pass1_block_map_unlock(e2fsck_t ctx) +void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx) { + +} + +void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx) +{ + } #endif diff --git a/tests/f_itable_collision/expect.1 b/tests/f_itable_collision/expect.1 index 01c85d4d..7e98baa8 100644 --- a/tests/f_itable_collision/expect.1 +++ b/tests/f_itable_collision/expect.1 @@ -1,6 +1,5 @@ Pass 1: Checking inodes, blocks, and sizes Inode 12 block 37 conflicts with critical metadata, skipping block checks. -Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map Inode 12, i_blocks is 48, should be 56. Fix? yes @@ -27,9 +26,7 @@ Clear inode? yes Restarting e2fsck from the beginning... Pass 1: Checking inodes, blocks, and sizes Inode 12 block 37 conflicts with critical metadata, skipping block checks. -Illegal block number passed to ext2fs_test_block_bitmap #4294967294 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #4294967294 for in-use block map -Illegal block number passed to ext2fs_test_block_bitmap #268435455 for in-use block map Illegal block number passed to ext2fs_mark_block_bitmap #268435455 for in-use block map Running additional passes to resolve blocks claimed by more than one inode... From patchwork Mon Nov 7 12:21:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700595 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=pCOARm7k; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VqS28nGz23lx for ; Mon, 7 Nov 2022 23:27:32 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqR5qS8z4xGj for ; Mon, 7 Nov 2022 23:27:31 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VqR5mSFz4xZ3; Mon, 7 Nov 2022 23:27:31 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=pCOARm7k; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqR1rryz4xGj for ; Mon, 7 Nov 2022 23:27:31 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232179AbiKGM1a (ORCPT ); Mon, 7 Nov 2022 07:27:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232258AbiKGM1U (ORCPT ); Mon, 7 Nov 2022 07:27:20 -0500 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 621086247 for ; Mon, 7 Nov 2022 04:27:18 -0800 (PST) Received: by mail-pf1-x42c.google.com with SMTP id q9so10453352pfg.5 for ; Mon, 07 Nov 2022 04:27:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OeEkOAiz0Ab66xS7o/5LER3ddboExhEpNk0u/MGhN+E=; b=pCOARm7kcBgdAFtONwUIM512UpGNsKFeVDjXd5wGoIJ/cXm7qXk3ldqDJLQ+gYGA3y GuKeNxF82/k2lsCYkTmJTtkxjv36o+N3XkeMgxR3Ve0nVrZUKGMG7XjGdxxutvCUcrkf 82Pg7RjFpt0s+IzA0+mYWMYQRxaBsBRcaTmMzqwETU/Wc9EayhL0dCcUJ698isQ/RUS1 4uljm0RdbbfLPydstEE1BOSKWH55rUI8AM6buKW20eVvlB2wDB5JAj+YmZgQ/cSJ4kMq xEC3xDdGOBiH6B/XhogYFnTL1/I/5f/l8P51CPUfI2jjRzfJe0k/aZQI75g6ZbG27cRz nFMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OeEkOAiz0Ab66xS7o/5LER3ddboExhEpNk0u/MGhN+E=; b=JmRBOTZUXTsdEYmxceJtr6Cm0pODM55GDNS8a7iKxmtMKtILGWStvxTVR072HndFGQ /D1zRS/4rPDIeequPUqouWCDThBCct3iYtCdG7zWDT66pTQlXpkZvzYCJ7WWGqAigPS1 Al9DlL5gEQlANvhfUOyO5vqxHPfC/arsubYIVSC6W4+yag404+AjpaHTpCR6No3/QTXd 0nmKBe5KuHOEQGGpuxZuwB78BLKIQrSi0uj26sUgTuz7mNG6oRh0WT1IJu/QGXXLeCGi kQrcLUTqkILMLgtPJEjclMx2d2si2r3JnyB/e5uK+A7tfMsmA5MYR1r5v7G5FZCb6V8S zK9Q== X-Gm-Message-State: ACrzQf1Tttic4lTXERzNITxpptI90lj1hSF936JcmpSDCT24f0fcW+y+ 2upm30U/r5HJ/bVJli6p/Wc= X-Google-Smtp-Source: AMsMyM7sU+LvDdqyVejLXCq1MnQ2Uu5qc77wk+2Pa1jKLGRSI5VItBhQ/6MkEZi3YqJVY7ShByMz6g== X-Received: by 2002:a05:6a00:98d:b0:56c:7c5c:da30 with SMTP id u13-20020a056a00098d00b0056c7c5cda30mr49471658pfg.22.1667824037822; Mon, 07 Nov 2022 04:27:17 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id w69-20020a627b48000000b00545f5046372sm4324904pfc.208.2022.11.07.04.27.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:17 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 48/72] e2fsck: allow admin specify number of threads Date: Mon, 7 Nov 2022 17:51:36 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong -m option is introduced to specify number of threads for pfsck. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 2 + e2fsck/pass1.c | 154 ++++++++++++++++---------- e2fsck/unix.c | 17 ++- tests/f_multithread/script | 2 +- tests/f_multithread_completion/script | 2 +- tests/f_multithread_logfile/script | 2 +- tests/f_multithread_no/script | 2 +- tests/f_multithread_preen/script | 2 +- tests/f_multithread_yes/script | 2 +- 9 files changed, 122 insertions(+), 63 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 23e8d2ed..e3276924 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -487,6 +487,7 @@ struct e2fsck_struct { char *undo_file; #ifdef HAVE_PTHREAD + __u32 fs_num_threads; /* serialize fix operation for multiple threads */ pthread_mutex_t fs_fix_mutex; /* protect block_found_map, block_dup_map */ @@ -727,6 +728,7 @@ void check_resize_inode(e2fsck_t ctx); int check_init_orphan_file(e2fsck_t ctx); /* util.c */ +#define E2FSCK_MAX_THREADS (65536) extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size, const char *description); extern int ask(e2fsck_t ctx, const char * string, int def); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index fdd1f3d6..5bf6980b 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1199,6 +1199,97 @@ static int e2fsck_should_abort(e2fsck_t ctx) return 0; } +static void init_ext2_max_sizes() +{ + int i; + __u64 max_sizes; + + /* + * Init ext2_max_sizes which will be immutable and shared between + * threads + */ +#define EXT2_BPP(bits) (1ULL << ((bits) - 2)) + + for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) { + max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i); + max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i); + max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i); + max_sizes = (max_sizes * (1UL << i)); + ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes; + } +#undef EXT2_BPP +} + +#ifdef HAVE_PTHREAD +/* TODO: tdb needs to be handled properly for multiple threads*/ +static int multiple_threads_supported(e2fsck_t ctx) +{ +#ifdef CONFIG_TDB + unsigned int threshold; + ext2_ino_t num_dirs; + errcode_t retval; + char *tdb_dir; + int enable; + + profile_get_string(ctx->profile, "scratch_files", "directory", 0, 0, + &tdb_dir); + profile_get_uint(ctx->profile, "scratch_files", + "numdirs_threshold", 0, 0, &threshold); + profile_get_boolean(ctx->profile, "scratch_files", + "icount", 0, 1, &enable); + + retval = ext2fs_get_num_dirs(ctx->fs, &num_dirs); + if (retval) + num_dirs = 1024; /* Guess */ + + /* tdb is unsupported now */ + if (enable && tdb_dir && !access(tdb_dir, W_OK) && + (!threshold || num_dirs > threshold)) + return 0; +#endif + return 1; +} + +/** + * Even though we could specify number of threads, + * but it might be more than the whole filesystem + * block groups, correct it here. + */ +static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) +{ + unsigned flexbg_size = 1; + ext2_filsys fs = ctx->fs; + int num_threads = ctx->fs_num_threads; + int max_threads; + + if (num_threads < 1) { + num_threads = 1; + goto out; + } + + if (!multiple_threads_supported(ctx)) { + num_threads = 1; + fprintf(stderr, "Fall through single thread for pass1 " + "because tdb could not handle properly\n"); + goto out; + } + + if (ext2fs_has_feature_flex_bg(fs->super)) + flexbg_size = 1 << fs->super->s_log_groups_per_flex; + max_threads = fs->group_desc_count / flexbg_size; + if (max_threads == 0) + max_threads = 1; + + if (num_threads > max_threads) { + fprintf(stderr, "Use max possible thread num: %d instead\n", + max_threads); + num_threads = max_threads; + } +out: + ctx->fs_num_threads = num_threads; +} +#endif + /* * We need call mark_table_blocks() before multiple * thread start, since all known system blocks should be @@ -1209,6 +1300,11 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) struct problem_context pctx; ext2_filsys fs = ctx->fs; + init_ext2_max_sizes(); +#ifdef HAVE_PTHREAD + e2fsck_pass1_set_thread_num(ctx); +#endif + clear_problem_context(&pctx); if (!(ctx->options & E2F_OPT_PREEN)) fix_problem(ctx, PR_1_PASS_HEADER, &pctx); @@ -2271,27 +2367,6 @@ endit: ctx->invalid_bitmaps++; } -static void init_ext2_max_sizes() -{ - int i; - __u64 max_sizes; - - /* - * Init ext2_max_sizes which will be immutable and shared between - * threads - */ -#define EXT2_BPP(bits) (1ULL << ((bits) - 2)) - - for (i = EXT2_MIN_BLOCK_LOG_SIZE; i <= EXT2_MAX_BLOCK_LOG_SIZE; i++) { - max_sizes = EXT2_NDIR_BLOCKS + EXT2_BPP(i); - max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i); - max_sizes = max_sizes + EXT2_BPP(i) * EXT2_BPP(i) * EXT2_BPP(i); - max_sizes = (max_sizes * (1UL << i)); - ext2_max_sizes[i - EXT2_MIN_BLOCK_LOG_SIZE] = max_sizes; - } -#undef EXT2_BPP -} - #ifdef HAVE_PTHREAD static errcode_t e2fsck_pass1_merge_bitmap(ext2_filsys fs, ext2fs_generic_bitmap *src, ext2fs_generic_bitmap *dest) @@ -3151,7 +3226,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, static void e2fsck_pass1_multithread(e2fsck_t global_ctx) { struct e2fsck_thread_info *infos = NULL; - int num_threads = 1; + int num_threads = global_ctx->fs_num_threads; errcode_t retval; retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); @@ -3174,48 +3249,15 @@ out_abort: } #endif -/* TODO: tdb needs to be handled properly for multiple threads*/ -static int multiple_threads_supported(e2fsck_t ctx) -{ -#ifdef CONFIG_TDB - unsigned int threshold; - ext2_ino_t num_dirs; - errcode_t retval; - char *tdb_dir; - int enable; - - profile_get_string(ctx->profile, "scratch_files", "directory", 0, 0, - &tdb_dir); - profile_get_uint(ctx->profile, "scratch_files", - "numdirs_threshold", 0, 0, &threshold); - profile_get_boolean(ctx->profile, "scratch_files", - "icount", 0, 1, &enable); - - retval = ext2fs_get_num_dirs(ctx->fs, &num_dirs); - if (retval) - num_dirs = 1024; /* Guess */ - - /* tdb is unsupported now */ - if (enable && tdb_dir && !access(tdb_dir, W_OK) && - (!threshold || num_dirs > threshold)) { - fprintf(stderr, "Fall through single thread for pass1 " - "because tdb could not handle properly\n"); - return 0; - } - #endif - return 1; -} - void e2fsck_pass1(e2fsck_t ctx) { errcode_t retval; - init_ext2_max_sizes(); retval = e2fsck_pass1_prepare(ctx); if (retval) return; #ifdef HAVE_PTHREAD - if (ctx->options & E2F_OPT_MULTITHREAD && multiple_threads_supported(ctx)) + if (ctx->options & E2F_OPT_MULTITHREAD || ctx->fs_num_threads > 1) e2fsck_pass1_multithread(ctx); else #endif diff --git a/e2fsck/unix.c b/e2fsck/unix.c index af2457e3..dfa3f897 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -826,6 +826,10 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) int res; /* result of sscanf */ #ifdef CONFIG_JBD_DEBUG char *jbd_debug; +#endif +#ifdef HAVE_PTHREAD + char *pm; + unsigned long thread_num; #endif unsigned long long phys_mem_kb, blk; @@ -859,7 +863,7 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) ctx->readahead_kb = ~0ULL; #ifdef HAVE_PTHREAD - while ((c = getopt(argc, argv, "pamnyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) + while ((c = getopt(argc, argv, "pam:nyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) #else while ((c = getopt(argc, argv, "panyrcC:B:dE:fvtFVM:b:I:j:P:l:L:N:SsDkz:")) != EOF) #endif @@ -905,7 +909,18 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) break; #ifdef HAVE_PTHREAD case 'm': + thread_num = strtoul(optarg, &pm, 0); + if (*pm) + fatal_error(ctx, + _("Invalid multiple thread num.\n")); + if (thread_num > E2FSCK_MAX_THREADS) { + fprintf(stderr, + _("threads %lu too large (max %lu)\n"), + thread_num, E2FSCK_MAX_THREADS); + fatal_error(ctx, 0); + } ctx->options |= E2F_OPT_MULTITHREAD; + ctx->fs_num_threads = thread_num; break; #endif case 'n': diff --git a/tests/f_multithread/script b/tests/f_multithread/script index 0fe96cd0..83cd0f03 100644 --- a/tests/f_multithread/script +++ b/tests/f_multithread/script @@ -1,4 +1,4 @@ -FSCK_OPT="-fy -m" +FSCK_OPT="-fy -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_completion/script b/tests/f_multithread_completion/script index bf23cd61..0ec13816 100644 --- a/tests/f_multithread_completion/script +++ b/tests/f_multithread_completion/script @@ -1,4 +1,4 @@ -FSCK_OPT="-fy -m -C 1" +FSCK_OPT="-fy -m1 -C 1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_logfile/script b/tests/f_multithread_logfile/script index 4f9ca6f8..dbb65319 100644 --- a/tests/f_multithread_logfile/script +++ b/tests/f_multithread_logfile/script @@ -1,5 +1,5 @@ LOG_FNAME="f_multithread_logfile_xxx" -FSCK_OPT="-fy -m -y -E log_filename=$LOG_FNAME" +FSCK_OPT="-fy -m1 -y -E log_filename=$LOG_FNAME" SKIP_VERIFY="true" ONE_PASS_ONLY="true" SKIP_CLEANUP="true" diff --git a/tests/f_multithread_no/script b/tests/f_multithread_no/script index b93deb3a..db791e11 100644 --- a/tests/f_multithread_no/script +++ b/tests/f_multithread_no/script @@ -1,4 +1,4 @@ -FSCK_OPT="-fn -m" +FSCK_OPT="-fn -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_preen/script b/tests/f_multithread_preen/script index ecb79cd6..8965f4a7 100644 --- a/tests/f_multithread_preen/script +++ b/tests/f_multithread_preen/script @@ -1,4 +1,4 @@ -FSCK_OPT="-fp -m" +FSCK_OPT="-fp -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck diff --git a/tests/f_multithread_yes/script b/tests/f_multithread_yes/script index 38891f6a..8b4aa9b8 100644 --- a/tests/f_multithread_yes/script +++ b/tests/f_multithread_yes/script @@ -1,4 +1,4 @@ -FSCK_OPT="-f -m" +FSCK_OPT="-f -m1" SECOND_FSCK_OPT=-yf . $cmd_dir/run_e2fsck From patchwork Mon Nov 7 12:21:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700596 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Iy1pba0/; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VqS16Dnz23lW for ; Mon, 7 Nov 2022 23:27:32 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqS0hycz4xZ3 for ; Mon, 7 Nov 2022 23:27:32 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VqS0fFNz4xwy; Mon, 7 Nov 2022 23:27:32 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Iy1pba0/; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqS0ZnTz4xZ3 for ; Mon, 7 Nov 2022 23:27:32 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232225AbiKGM1b (ORCPT ); Mon, 7 Nov 2022 07:27:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232220AbiKGM1Z (ORCPT ); Mon, 7 Nov 2022 07:27:25 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 275F11AD9C for ; Mon, 7 Nov 2022 04:27:24 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id h14so10410304pjv.4 for ; Mon, 07 Nov 2022 04:27:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GliyG/gfgbrd/RaksM0VE0kE757GWQ+YGnDdx0Zlfkc=; b=Iy1pba0/sdFMGQqzSq9ZTNq5snxoqbwSTVIRi4jvw/hPjnXrd1CLdCelQcgycD/Vk2 /3b1VGkCxOiV5En72P47BOQeJWQxB4Dj0MmnRQoYqPKYJE16AEc1KIbMgaRZ22DxZvJX BkrXtHLpdwUVkWUg9zACrk3qEQZWSYhWMpyyuLE3ahSt8TshxvLeACDccHUxY1MXeX7u ajSDE+NIK3tYZHuAJcR2kd3d2AIbIgqHzRfynUb2RmCSb5ZRmzQ3fIYqKDAGsGLt57U5 068G1p3ZrSnCHMYqRrgbbQh9EyV+z7LGfSzwcbzsLzuIYVEo7Sr/AWbgG2iA5wa4RR8x 6+rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GliyG/gfgbrd/RaksM0VE0kE757GWQ+YGnDdx0Zlfkc=; b=XhTZgwv83hZrXhUSjwt5BtXfOLXso7q6DS5e9TAfKWU4DEi14pi/xTB2APCZRYk28g 2xiWRERHEgFYeoYMqpQFeByGqWt4NzM5urhxEoFfZnytSHZeXVnCfH0GU4DNgvmTC6d4 MgMkO0CVpXxlSaE7nV5fdV3IAP+zKaD8k3GVzUknqlPfnW1REbrwTo0xRmJ4WZiB2Rrh 91P6+xmFele1q7oMWSqxonDzYnRLM+e2d2og7H8y/UR6fYIwTusApKB9fbTtPrn375Dd EahTq03O2KbF7TnFNDPcqTGyV+uP1SEHmU/w8+4o7+uEkjZQH2CMzeq+vex4F4VsfI0e y7pg== X-Gm-Message-State: ACrzQf1jgYEa94I1Ij5CQ6HTvZKxTAxawJFcgcv+TAQbTBZ9/w3dINfm /jbxrd4ZIF9NaYNzUSCxGI4= X-Google-Smtp-Source: AMsMyM5GHwheccaNhiFg4HwuyTs17W3vjuWKGnWEgCjbcG3gjHA6zKqwS7SHvrZyFkNfLgKXpfdPYA== X-Received: by 2002:a17:90a:578c:b0:213:b509:9474 with SMTP id g12-20020a17090a578c00b00213b5099474mr48745598pji.45.1667824043633; Mon, 07 Nov 2022 04:27:23 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id q23-20020a170902bd9700b001784a45511asm4885435pls.79.2022.11.07.04.27.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:22 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 49/72] e2fsck: adjust number of threads Date: Mon, 7 Nov 2022 17:51:37 +0530 Message-Id: <787fea1b97ccfda61736f7a9e07d9eeed17b1bc8.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong number of threads should not exceed flex bg numbers, and output messages if we adjust threads number. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 5bf6980b..1d4f576c 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1287,6 +1287,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) } out: ctx->fs_num_threads = num_threads; + ctx->fs->fs_num_threads = num_threads; } #endif @@ -2481,14 +2482,14 @@ static void e2fsck_pass1_merge_invalid_bitmaps(e2fsck_t global_ctx, } static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thread_ctx, - int thread_index, int num_threads) + int thread_index, int num_threads, + dgrp_t average_group) { errcode_t retval; e2fsck_t thread_context; ext2_filsys thread_fs; ext2_filsys global_fs = global_ctx->fs; struct e2fsck_thread *tinfo; - dgrp_t average_group; assert(global_ctx->inode_used_map == NULL); assert(global_ctx->inode_dir_map == NULL); @@ -2535,16 +2536,9 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre thread_context->thread_info.et_thread_index = thread_index; set_up_logging(thread_context); - /* - * Distribute work to multiple threads: - * Each thread work on fs->group_desc_count / nthread groups. - */ tinfo = &thread_context->thread_info; - average_group = thread_fs->group_desc_count / num_threads; - if (average_group == 0) - average_group = 1; tinfo->et_group_start = average_group * thread_index; - if (thread_index == num_threads - 1) + if (thread_index == global_fs->fs_num_threads - 1) tinfo->et_group_end = thread_fs->group_desc_count; else tinfo->et_group_end = average_group * (thread_index + 1); @@ -3060,12 +3054,13 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) } static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, - int num_threads, e2fsck_t global_ctx) + e2fsck_t global_ctx) { errcode_t rc; errcode_t ret = 0; int i; struct e2fsck_thread_info *pinfo; + int num_threads = global_ctx->fs_num_threads; /* merge invalid bitmaps will recalculate it */ global_ctx->invalid_bitmaps = 0; @@ -3147,7 +3142,7 @@ out: } static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, - int num_threads, e2fsck_t global_ctx) + e2fsck_t global_ctx) { struct e2fsck_thread_info *infos; pthread_attr_t attr; @@ -3156,6 +3151,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, struct e2fsck_thread_info *tmp_pinfo; int i; e2fsck_t thread_ctx; + dgrp_t average_group; + int num_threads = global_ctx->fs_num_threads; #ifdef DEBUG_THREADS struct e2fsck_thread_debug thread_debug = {PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0}; @@ -3179,6 +3176,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, return retval; } + average_group = ext2fs_get_avg_group(global_ctx->fs); for (i = 0; i < num_threads; i++) { tmp_pinfo = &infos[i]; tmp_pinfo->eti_thread_index = i; @@ -3186,7 +3184,8 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, tmp_pinfo->eti_debug = &thread_debug; #endif retval = e2fsck_pass1_thread_prepare(global_ctx, &thread_ctx, - i, num_threads); + i, num_threads, + average_group); if (retval) { com_err(global_ctx->program_name, retval, _("while preparing pass1 thread\n")); @@ -3216,7 +3215,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, } if (retval) { - e2fsck_pass1_threads_join(infos, num_threads, global_ctx); + e2fsck_pass1_threads_join(infos, global_ctx); return retval; } *pinfo = infos; @@ -3226,17 +3225,16 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, static void e2fsck_pass1_multithread(e2fsck_t global_ctx) { struct e2fsck_thread_info *infos = NULL; - int num_threads = global_ctx->fs_num_threads; errcode_t retval; - retval = e2fsck_pass1_threads_start(&infos, num_threads, global_ctx); + retval = e2fsck_pass1_threads_start(&infos, global_ctx); if (retval) { com_err(global_ctx->program_name, retval, _("while starting pass1 threads\n")); goto out_abort; } - retval = e2fsck_pass1_threads_join(infos, num_threads, global_ctx); + retval = e2fsck_pass1_threads_join(infos, global_ctx); if (retval) { com_err(global_ctx->program_name, retval, _("while joining pass1 threads\n")); From patchwork Mon Nov 7 12:21:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700598 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ZY/uTqZr; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VqX6Vb7z23lW for ; Mon, 7 Nov 2022 23:27:36 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqX63zBz4xDK for ; Mon, 7 Nov 2022 23:27:36 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VqX61SLz4xZ3; Mon, 7 Nov 2022 23:27:36 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ZY/uTqZr; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VqX5vsqz4xDK for ; Mon, 7 Nov 2022 23:27:36 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232226AbiKGM1e (ORCPT ); Mon, 7 Nov 2022 07:27:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232224AbiKGM1b (ORCPT ); Mon, 7 Nov 2022 07:27:31 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DAE61AD9C for ; Mon, 7 Nov 2022 04:27:30 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id io19so10902582plb.8 for ; Mon, 07 Nov 2022 04:27:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3vXp8gmCs7RJtvSM1K3okUNxQLRdt0i4KzUMdrYIHMw=; b=ZY/uTqZrOX1BayaeCvfHJBmak+TKfUkEja263WJS+tgEqk0ZGqXUyfyFohSNWW6RSF +IGTjs+Oi/201H79RkHlse85j77kre8kn3NfXA2vOI77pHfHIaekNIdn2Ps1KDYLWIOh w5hP8WGcCMilBloH0m27Al4r0naAHOt2Rid+haClglASWYEy3tVCV2aC2foGUR7cBRGW pG5Ob2wfFFDRIBhY+sFpOC7gd+4eIlC4vwUKODZH+mTYAGqq+NU0Y0hB1an0sNe/Nk3g HtFhfp217opTyykG0wpz6gJtgljqc7i4x12xP8NI2md+GJ/wYniMtsKviZzxGXq64XEe vbGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3vXp8gmCs7RJtvSM1K3okUNxQLRdt0i4KzUMdrYIHMw=; b=dFSWwVc9C/HpSIwmJjCPXN+t2J2wrLSKRVoCwCpzQu9MmA3rzL7kuC8tsWhGNhLZ6J oa/c0puZhx6O+ddmLzK+GAPGR/haYks8OFntRfZJW8X1Pzqsu90xLsqdHHLtPpZlZPjJ eI3WCU+3sQIHbz2GWsDuBQ32Zqn5MLLwwLg2OEAn46dvfqdE0/RFrdsZjsqkbNUwv1Vj qKHKWwIbLR8wVjT+YApHm9K/HVtl5tce7OExmHlB8uB16Tpdw8q0gUPXu7y2G5NkdPvU UDqTYMDLK5vW84kmBzmZcG5g+Twj7crdGLib8helpkjdpyH4Ddd15EaFilBK5Qf3arDh Oj3w== X-Gm-Message-State: ACrzQf1Rckt+4v9r21UI15QVQ2S2c+25uDPaX2ZNaksJzoQHIkfoXlRO 2KltBY0rLjhAE2AGhTn1t1M= X-Google-Smtp-Source: AMsMyM5Ae9XAhgp11eTyPnWSKe22pspMICLnwupLSoNxkwH+9c4sul9qH+BdHyFsr+gy8Q7A/cs6IQ== X-Received: by 2002:a17:902:e8cd:b0:186:9efc:6790 with SMTP id v13-20020a170902e8cd00b001869efc6790mr50590382plg.91.1667824049651; Mon, 07 Nov 2022 04:27:29 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id h20-20020aa796d4000000b00560a25fae1fsm4325343pfq.206.2022.11.07.04.27.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:28 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 50/72] e2fsck: fix readahead for pfsck of pass1 Date: Mon, 7 Nov 2022 17:51:38 +0530 Message-Id: <7fce79fc82c97b0ff5c9b7e81f3c2f343316c706.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Several improvments for this patch: 1) move readahead_kb detection to preparing phase. 2) inode readahead should be aware of thread block group boundary. 3) make readahead_kb aware of multiple threads. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 1d4f576c..5d07daec 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1098,16 +1098,20 @@ out: static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino) { ext2_ino_t inodes_in_group = 0, inodes_per_block, inodes_per_buffer; - dgrp_t start = *group, grp; + dgrp_t start = *group, grp, grp_end = ctx->fs->group_desc_count; blk64_t blocks_to_read = 0; errcode_t err = EXT2_ET_INVALID_ARGUMENT; +#ifdef HAVE_PTHREAD + if (ctx->fs->fs_num_threads > 1) + grp_end = ctx->thread_info.et_group_end; +#endif if (ctx->readahead_kb == 0) goto out; /* Keep iterating groups until we have enough to readahead */ inodes_per_block = EXT2_INODES_PER_BLOCK(ctx->fs->super); - for (grp = start; grp < ctx->fs->group_desc_count; grp++) { + for (grp = start; grp < grp_end; grp++) { if (ext2fs_bg_flags_test(ctx->fs, grp, EXT2_BG_INODE_UNINIT)) continue; inodes_in_group = ctx->fs->super->s_inodes_per_group - @@ -1300,12 +1304,25 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) { struct problem_context pctx; ext2_filsys fs = ctx->fs; + unsigned long long readahead_kb; init_ext2_max_sizes(); -#ifdef HAVE_PTHREAD +#ifdef HAVE_PTHREAD e2fsck_pass1_set_thread_num(ctx); #endif + /* If we can do readahead, figure out how many groups to pull in. */ + if (!e2fsck_can_readahead(ctx->fs)) + ctx->readahead_kb = 0; + else if (ctx->readahead_kb == ~0ULL) + ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs); +#ifdef HAVE_PTHREAD + /* don't use more than 1/10 of memory for threads checking */ + readahead_kb = get_memory_size() / (10 * ctx->fs_num_threads); + /* maybe better disable RA if this is too small? */ + if (ctx->readahead_kb > readahead_kb) + ctx->readahead_kb = readahead_kb; +#endif clear_problem_context(&pctx); if (!(ctx->options & E2F_OPT_PREEN)) fix_problem(ctx, PR_1_PASS_HEADER, &pctx); @@ -1482,13 +1499,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) init_resource_track(&rtrack, ctx->fs->io); clear_problem_context(&pctx); - /* If we can do readahead, figure out how many groups to pull in. */ - if (!e2fsck_can_readahead(ctx->fs)) - ctx->readahead_kb = 0; - else if (ctx->readahead_kb == ~0ULL) - ctx->readahead_kb = e2fsck_guess_readahead(ctx->fs); pass1_readahead(ctx, &ra_group, &ino_threshold); - if (ext2fs_has_feature_dir_index(fs->super) && !(ctx->options & E2F_OPT_NO)) { if (ext2fs_u32_list_create(&ctx->dirs_to_hash, 50)) From patchwork Mon Nov 7 12:21:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700599 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=HNn/5cFg; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vql10NLz23lW for ; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vql0YvYz4xGj for ; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vql0X26z4xZ3; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=HNn/5cFg; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vql0RTLz4xGj for ; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232233AbiKGM1p (ORCPT ); Mon, 7 Nov 2022 07:27:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232224AbiKGM1g (ORCPT ); Mon, 7 Nov 2022 07:27:36 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0C441A817 for ; Mon, 7 Nov 2022 04:27:35 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id m14-20020a17090a3f8e00b00212dab39bcdso14465028pjc.0 for ; Mon, 07 Nov 2022 04:27:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=81vBt7uHuryaeGLS3/KgcqL9AcD3bdCb+FwgpUv3UgU=; b=HNn/5cFgF4ITAXweRUF6fpM0A0ibNMQFy3oAmFHBr0QfPIPbMFLtvmUR0WZF3/NrW9 JObKGaAB7jIsjthgQxWWhsO7P9gfvXU0sCf7ZxZAOfBmG5K/L1BIyXkhFYzyweGiVVrT zqtNMCUWjIA/gDqrWmF0m71VuV4OwRUFJI36lb9RMF01tqm3+TK3lMmodiX2Le0Pgar6 S75WUoiev+kiAGAMaiHzz5RK4qJiBWhpwuUDSNQ5NXIl2PjaH9yfj5rqmli6R2Ec7BvM JctLzxok+4B7qXf484CcL+Afdq3oE5YpDeVJoV2YNys+7YLzg9+7BHURNoZX6pSZ6KAt LfNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=81vBt7uHuryaeGLS3/KgcqL9AcD3bdCb+FwgpUv3UgU=; b=zfyH3vXMTTS6WbSWTLJpE5Bd3lO8N16u5vrmiwb2AWoA2dRB6VKsvbSmmsJsTvJ46E VasGk2LC8lgR5Drr8JLTnH5SmrcwTZujNDyoRKWDJoQvoMdOaIyn2w/RaOmDiBzSBGyb lMbesCMOStVzRLdtC+VuzjcFQOMxhnEBA0tySz3aIEymFlU/KmwncTSGz3019Hoz8k++ rAJ+PPbgQyOlZqEQ32RNZ7uPKdq/ll07lpgQ0oPrsQS+5xAUMkZCkOUf8tDsmTAYZczb +JljRaOTOGiJu+Ew37LJEdJjO1fSi0WYp+mXc9un9wG4TebOWO3s1NXaxZfSvkyuJ7Er PIPw== X-Gm-Message-State: ACrzQf0e9KZ9B8T70nk7yAbeiklQR/anUtbow0hHXF6IfyTO9qSjsbRv eEdyBSTw14BLlWuohRZjEwI= X-Google-Smtp-Source: AMsMyM49Mxhbt01KxqeMHTBK0VTij6Rw7bHGHDTM1zqgN4Dn4bDNPdu7Fzf0+6MTd3mXyjaqhFWhPw== X-Received: by 2002:a17:90a:7061:b0:213:da75:f149 with SMTP id f88-20020a17090a706100b00213da75f149mr43493265pjk.85.1667824055315; Mon, 07 Nov 2022 04:27:35 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id v1-20020a622f01000000b0056be55df0c8sm4376744pfv.116.2022.11.07.04.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:34 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 51/72] e2fsck: merge options after threads finish Date: Mon, 7 Nov 2022 17:51:39 +0530 Message-Id: <162e7927ff2cb9a94f9ed812477765c6ead16754.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong It will be possible that threads might append E2F_OPT_YES, so we need merge options to global, test f_yesall cover this. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 5d07daec..59ff888f 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2866,6 +2866,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; + int options = global_ctx->options; #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2918,7 +2919,8 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->fs_fragmented += fs_fragmented; global_ctx->fs_fragmented_dir += fs_fragmented_dir; global_ctx->large_files += large_files; - + /* threads might enable E2F_OPT_YES */ + global_ctx->options |= options; global_ctx->flags |= flags; global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; @@ -2954,6 +2956,7 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx thread_ctx->qctx); if (retval) return retval; + global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; From patchwork Mon Nov 7 12:21:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700600 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=kwaVVbb6; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vqm0MSMz23lx for ; Mon, 7 Nov 2022 23:27:48 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vql74lKz4xGj for ; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vql72PZz4xZ3; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=kwaVVbb6; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vql6xxYz4xGj for ; Mon, 7 Nov 2022 23:27:47 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232161AbiKGM1q (ORCPT ); Mon, 7 Nov 2022 07:27:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232214AbiKGM1l (ORCPT ); Mon, 7 Nov 2022 07:27:41 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 740911B7B0 for ; Mon, 7 Nov 2022 04:27:41 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id 130so10450790pfu.8 for ; Mon, 07 Nov 2022 04:27:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wxJyL+w+pfAfLkAJLcGqmfLql0/jYkk6WTRWqCNCjIA=; b=kwaVVbb6ud897u5c8jn9pnnK8rc+BYg2iRJGFuxESbXtK/jjXg6Ypf8ybJIUTZj3wO L99T3+ZVuY7DKGnZ7PZE5sAD3wL76cKBejn8YOLPbyR/c0lato4mrdy+X87S/s4Fdzcv cctTzn/U/oozTXzoRUxYS3u7Bhmvjc/VvyzYgSH3uNdq1II1TnaGI9c/qz6GbNxjo8+n wXBuD6he/68uMbQBCQKsU0+d2h0g1UWjPyRYOOXFNfZZQry1E7Qm69vIUdiGEPmGNtat FB1BkZzZ3rF9e9Vra3aAF6XIoTG1NGrOxV+O+YUTqLVIcpXyW0nGHbn0HdLKQY8RzJo9 7stQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wxJyL+w+pfAfLkAJLcGqmfLql0/jYkk6WTRWqCNCjIA=; b=XTuXkPY5LCzIpLebQOZ6J76fnnlzYlCb7wuhw+1KjsTAzFLvTMx+qwnsOg9f69NH6w 882RcEAMkHWKrzc70hK+n4b3Y+moLs6fekWnkPmhLbrbQF9WlwwoNRpjUkPMqY73+ElC aPPDPMtr/no1WwqXTQYGAIRXv0BczFsL2wLufq+ffGgq+lgmCzbCLGcPA/TTewrBJ0ia Mz14vcRRrjw5jLM3KBKPGhM2zBJJ332x2Xl7MbJhZU7+Ejy/Ch/+wao0B0Ozj9ZnXU7h LHzNfMKTKKGpi79/aW+gW+rhw0qcmjOEuEV0I7dFe15ghf9O9VlblKt42a+uuh8/ezkT vUdg== X-Gm-Message-State: ACrzQf2lwo93YK4ODZaLJr9ma+G3qu0erc18rOuLKzqxkyO2OykxZcJI RSQB0ftU/FCjCkod8IK/bP0= X-Google-Smtp-Source: AMsMyM4x81BZuibCfsIAtkIlHmhcYrNRBJEknXCszdH3/lDZF1JoXH72W5Zie7bFpvoyHuLwenj79w== X-Received: by 2002:a05:6a00:1884:b0:56c:636a:d554 with SMTP id x4-20020a056a00188400b0056c636ad554mr50133166pfh.18.1667824061003; Mon, 07 Nov 2022 04:27:41 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id k64-20020a17090a3ec600b00205f013f275sm6058993pjc.22.2022.11.07.04.27.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:40 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 52/72] e2fsck: reset lost_and_found after threads finish Date: Mon, 7 Nov 2022 17:51:40 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong This should not be kept, the reaons is similar to what e2fsck_pass1 has done before. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 59ff888f..1a5fcf66 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2925,6 +2925,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx global_ctx->logf = global_logf; global_ctx->problem_logf = global_problem_logf; global_ctx->global_ctx = NULL; + /* + * The l+f inode may have been cleared, so zap it now and + * later passes will recalculate it if necessary + */ + global_ctx->lost_and_found = 0; global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; From patchwork Mon Nov 7 12:21:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700601 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=k5L+/hzo; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vqr1BqVz23lW for ; Mon, 7 Nov 2022 23:27:52 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vqr0lPgz4xZ3 for ; Mon, 7 Nov 2022 23:27:52 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vqr0hzpz4xwy; Mon, 7 Nov 2022 23:27:52 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=k5L+/hzo; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vqr0cR9z4xZ3 for ; Mon, 7 Nov 2022 23:27:52 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232173AbiKGM1u (ORCPT ); Mon, 7 Nov 2022 07:27:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232229AbiKGM1r (ORCPT ); Mon, 7 Nov 2022 07:27:47 -0500 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33B2B1AD9C for ; Mon, 7 Nov 2022 04:27:47 -0800 (PST) Received: by mail-pj1-x1031.google.com with SMTP id b11so10421483pjp.2 for ; Mon, 07 Nov 2022 04:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lRf7GG49fBpc8qWJATB6Z8IF0kB25ctC/s18amrBQPo=; b=k5L+/hzob9boPoutckNo9yYIyy5p/VzyWQ0QyRYbRxmp1GMvqlpOeUREwBTFGYQYb3 60Rn0ijW/HTLYTPTFxGYBmb0DeLY3pFO6aFfqkedCZotXxp8PTtUa1N4y49Jd4u9unu8 Qtiy/zEM7kj8zdWE+Kwkjf9hG1w7usJzEVcSSnrJ6tqV2KoH7KNLISh0w1zWAPs7IQqB 7dVHRptcKSbci91yTvba0fnr/n7bAR91wSHi7nrLvRNwm7ilcdu3uuTp7iQsVTEAxjV0 Opx1sbFiv2siRw8jQu16scqZ9PC4eOxiS1PnAQRt9WZjadXPYIQBCfAgw1v7it2qTT6O aMOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lRf7GG49fBpc8qWJATB6Z8IF0kB25ctC/s18amrBQPo=; b=e6v+uOD67jJq3HzVfKk38RlTTRb2J+pD50KXgCzGVxSVKq6krxIoij+k/shEz6I46B YU9BMMVf19yLEsyFhIZXjoq+REyUMHsgvH7+6MbT337uB7Z94gRNQYZEI5C2cB1Q/AL3 boMm6tSmdR6INTv15JYYndR02tBeNsKkcXToNGxLDAKNqtomljjgF21o0A3Bj1umFoab rX2tKDWWkwnvjEGKn9D1L2iKQ5WByz1gHllU2VGpItG5qRiMTRwQCEIzvMntIOXNGbd9 hw+35I8tZx6DmKNzjmFHYhgeUef+9fWzQsjq2KmXDUNMDSPsoz2Cej8Icv4rFZ9LeJiC TtPw== X-Gm-Message-State: ACrzQf2EDuabGBUTtAmAKt+ZBhOF6bHbyhVvcHLSqQ5WU+J9C0/+Cs3E 6jhLgR7h5b5IQcUA2JCEZBo= X-Google-Smtp-Source: AMsMyM5XmP2rjf9Ysg8zWeBwdd32hp4Ojv7a5Rb4x/5WT5jYWr64luk7YZRKCtkqAG32af9JYPWFBA== X-Received: by 2002:a17:90b:1c8c:b0:203:89fb:ba79 with SMTP id oo12-20020a17090b1c8c00b0020389fbba79mr68670812pjb.92.1667824066759; Mon, 07 Nov 2022 04:27:46 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id n4-20020a170902e54400b0017f73dc1549sm4859348plf.263.2022.11.07.04.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:46 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 53/72] e2fsck: merge extent depth count after threads finish Date: Mon, 7 Nov 2022 17:51:41 +0530 Message-Id: <6a4a5461ba2ca87b7bf1ac6923663d41e3d11677.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong tests covered by f_extent_htree. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 1a5fcf66..c89c424d 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2866,8 +2866,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; - int options = global_ctx->options; + int options = global_ctx->options, i; + __u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT]; + memcpy(extent_depth_count, global_ctx->extent_depth_count, + sizeof(extent_depth_count)); #ifdef HAVE_SETJMP_H jmp_buf old_jmp; @@ -2930,6 +2933,12 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx * later passes will recalculate it if necessary */ global_ctx->lost_and_found = 0; + memcpy(global_ctx->extent_depth_count, extent_depth_count, + sizeof(extent_depth_count)); + /* merge extent depth count */ + for (i = 0; i < MAX_EXTENT_DEPTH_COUNT; i++) + global_ctx->extent_depth_count[i] += + thread_ctx->extent_depth_count[i]; global_fs->priv_data = global_ctx; global_ctx->fs = global_fs; From patchwork Mon Nov 7 12:21:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700602 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=idB6WIWy; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vr32cW2z23lW for ; Mon, 7 Nov 2022 23:28:03 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vr329QSz4xDK for ; Mon, 7 Nov 2022 23:28:03 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vr326wMz4xZ3; Mon, 7 Nov 2022 23:28:03 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=idB6WIWy; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vr30XLJz4xDK for ; Mon, 7 Nov 2022 23:28:03 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232227AbiKGM2B (ORCPT ); Mon, 7 Nov 2022 07:28:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232229AbiKGM1y (ORCPT ); Mon, 7 Nov 2022 07:27:54 -0500 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 94AED1A817 for ; Mon, 7 Nov 2022 04:27:52 -0800 (PST) Received: by mail-pl1-x62c.google.com with SMTP id d20so9844328plr.10 for ; Mon, 07 Nov 2022 04:27:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZvzSnOk6N2gJ52ngbooAIOmxuIXr9zJ3A/nrYrNKyog=; b=idB6WIWy+aDX1rLNDWYK552R6h52I5AFSe0TVGFPRiyaYOYQpC/ffW1G1Ou2/Ol41L StYdeFkCqZC6VrNSgRYyX7tuWNejrT5jcAplhfC87dtFq9ic3JayQ73FusFAUsoI6S7S PX1cUQepnspum3Z+mmjAdHXdbFZGvBOb7OTcjx3CJrGzkWybD2sRnuj5wD26+78V4q9W Qw2aqAgxS+9D99y4UxLIFyeE5SguGpKGFQhB7kpqSnXe0iT6ryVoa0Krb/JEj+8eUMqn 9i5dgIFH42zU1vughL1JdUGy4g7uRicRkGe0C1qaj2f+x1SeJ3g4We6a23+P42IZp2NZ zScg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZvzSnOk6N2gJ52ngbooAIOmxuIXr9zJ3A/nrYrNKyog=; b=heGso9nZTJdtsMUEUuw1Pe+KZcdpMSSVybHHp3t6P7qoOZVx3Zu1PvFSlHz49KhQHP D5YBZ6MseBKFkfg9DFDexpC28BScjkuMePCc7w0CDmAvDfGUubFahETgxh8fMQVssy0s 84jdgHAXsM0iUrvQ/eKQa3kjMVbKRWUfTk6sCsJDzynuudyE4BktY+dvzTLXwA7XFOXa 8ck9aEITGZcdXz1oQ9QkI18YMiXPt8tFlwOGtVDxjG0QMbOUBTRBcSnWix49X2ah/xoz lfyxJH2vGf29CutXJG9aF4SxttpY1sd49b3HCd5p94uZ1hlAjpNtZGQn3kA5/Do80ZHu x1HA== X-Gm-Message-State: ACrzQf026imAIW8mg/wlTR9qIkue0gNFc0/94oy2OkzJvsF3Zaa6P2C0 9sfn24ryh/K4XyKPlhZxBl8= X-Google-Smtp-Source: AMsMyM4IEkNfO2J75xu59rpwvyN6CDx6t5RLp18NQY+Yi8gDY/903f3J4AWL/LgTwUmz3G+I43SMUw== X-Received: by 2002:a17:903:2446:b0:187:11c6:6a1b with SMTP id l6-20020a170903244600b0018711c66a1bmr43952690pls.39.1667824072400; Mon, 07 Nov 2022 04:27:52 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id x17-20020a17090300d100b0016eef326febsm4910425plc.1.2022.11.07.04.27.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:51 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 54/72] e2fsck: simplify e2fsck context merging codes Date: Mon, 7 Nov 2022 17:51:42 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong We tried to copy thread context to global context directly and then copy back some saved variables before merging. Since we have finished almost all necessary variables in the e2fsck context, we could simplify codes, and this could help us understand what is missing rather than hide problems. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 158 ++++++++++--------------------------------------- 1 file changed, 31 insertions(+), 127 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index c89c424d..8e85f70f 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2813,136 +2813,51 @@ static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx, return retval; } -static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx) +static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx, + e2fsck_t thread_ctx) { - errcode_t retval = 0; - int flags = global_ctx->flags; - FILE *global_logf = global_ctx->logf; - FILE *global_problem_logf = global_ctx->problem_logf; - ext2_filsys thread_fs = thread_ctx->fs; ext2_filsys global_fs = global_ctx->fs; - ext2fs_inode_bitmap inode_bad_map = global_ctx->inode_bad_map; - struct dir_info_db *dir_info = global_ctx->dir_info; - struct dx_dir_info *dx_dir_info = global_ctx->dx_dir_info; - ext2fs_inode_bitmap inode_used_map = global_ctx->inode_used_map; - ext2fs_inode_bitmap inode_dir_map = global_ctx->inode_dir_map; - ext2fs_inode_bitmap inode_bb_map = global_ctx->inode_bb_map; - ext2fs_inode_bitmap inode_imagic_map = global_ctx->inode_imagic_map; - ext2fs_inode_bitmap inode_reg_map = global_ctx->inode_reg_map; - ext2fs_block_bitmap inodes_to_rebuild = global_ctx->inodes_to_rebuild; - ext2_icount_t inode_count = global_ctx->inode_count; - ext2_icount_t inode_link_info = global_ctx->inode_link_info; - __u32 fs_directory_count = global_ctx->fs_directory_count; - __u32 fs_regular_count = global_ctx->fs_regular_count; - __u32 fs_blockdev_count = global_ctx->fs_blockdev_count; - __u32 fs_chardev_count = global_ctx->fs_chardev_count; - __u32 fs_links_count = global_ctx->fs_links_count; - __u32 fs_symlinks_count = global_ctx->fs_symlinks_count; - __u32 fs_fast_symlinks_count = global_ctx->fs_fast_symlinks_count; - __u32 fs_fifo_count = global_ctx->fs_fifo_count; - __u32 fs_total_count = global_ctx->fs_total_count; - __u32 fs_badblocks_count = global_ctx->fs_badblocks_count; - __u32 fs_sockets_count = global_ctx->fs_sockets_count; - __u32 fs_ind_count = global_ctx->fs_ind_count; - __u32 fs_dind_count = global_ctx->fs_dind_count; - __u32 fs_tind_count = global_ctx->fs_tind_count; - __u32 fs_fragmented = global_ctx->fs_fragmented; - __u32 fs_fragmented_dir = global_ctx->fs_fragmented_dir; - __u32 large_files = global_ctx->large_files; - ext2_ino_t dx_dir_info_size = global_ctx->dx_dir_info_size; - ext2_ino_t dx_dir_info_count = global_ctx->dx_dir_info_count; - ext2_u32_list dirs_to_hash = global_ctx->dirs_to_hash; - quota_ctx_t qctx = global_ctx->qctx; - int *invalid_block_bitmap_flag = global_ctx->invalid_block_bitmap_flag; - int *invalid_inode_bitmap_flag = global_ctx->invalid_inode_bitmap_flag; - int *invalid_inode_table_flag = global_ctx->invalid_inode_table_flag; - int invalid_bitmaps = global_ctx->invalid_bitmaps; - ext2_refcount_t refcount = global_ctx->refcount; - ext2_refcount_t refcount_extra = global_ctx->refcount_extra; - ext2_refcount_t refcount_orig = global_ctx->refcount_orig; - ext2_refcount_t ea_block_quota_blocks = global_ctx->ea_block_quota_blocks; - ext2_refcount_t ea_block_quota_inodes = global_ctx->ea_block_quota_inodes; - ext2fs_block_bitmap block_ea_map = global_ctx->block_ea_map; - ext2_refcount_t ea_inode_refs = global_ctx->ea_inode_refs; - ext2fs_block_bitmap block_found_map = global_ctx->block_found_map; - ext2fs_block_bitmap block_dup_map = global_ctx->block_dup_map; - int options = global_ctx->options, i; - __u32 extent_depth_count[MAX_EXTENT_DEPTH_COUNT]; - - memcpy(extent_depth_count, global_ctx->extent_depth_count, - sizeof(extent_depth_count)); -#ifdef HAVE_SETJMP_H - jmp_buf old_jmp; - - memcpy(old_jmp, global_ctx->abort_loc, sizeof(jmp_buf)); -#endif - memcpy(global_ctx, thread_ctx, sizeof(struct e2fsck_struct)); -#ifdef HAVE_SETJMP_H - memcpy(global_ctx->abort_loc, old_jmp, sizeof(jmp_buf)); -#endif + errcode_t retval = 0; + int i; - global_ctx->inode_used_map = inode_used_map; - global_ctx->inode_bad_map = inode_bad_map; - global_ctx->inode_dir_map = inode_dir_map; - global_ctx->inode_bb_map = inode_bb_map; - global_ctx->inode_imagic_map = inode_imagic_map; - global_ctx->inodes_to_rebuild = inodes_to_rebuild; - global_ctx->inode_reg_map = inode_reg_map; - global_ctx->block_dup_map = block_dup_map; - global_ctx->block_found_map = block_found_map; - global_ctx->dir_info = dir_info; - e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); - global_ctx->dx_dir_info = dx_dir_info; - global_ctx->dx_dir_info_count = dx_dir_info_count; - global_ctx->dx_dir_info_size = dx_dir_info_size; - e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx); - global_ctx->inode_count = inode_count; - global_ctx->inode_link_info = inode_link_info; - global_ctx->refcount = refcount; - global_ctx->refcount_extra = refcount_extra; - global_ctx->refcount_orig = refcount_orig; - global_ctx->ea_block_quota_blocks = ea_block_quota_blocks; - global_ctx->ea_block_quota_inodes = ea_block_quota_inodes; - global_ctx->block_ea_map = block_ea_map; - global_ctx->ea_inode_refs = ea_inode_refs; - global_ctx->fs_directory_count += fs_directory_count; - global_ctx->fs_regular_count += fs_regular_count; - global_ctx->fs_blockdev_count += fs_blockdev_count; - global_ctx->fs_chardev_count += fs_chardev_count; - global_ctx->fs_links_count += fs_links_count; - global_ctx->fs_symlinks_count += fs_symlinks_count; - global_ctx->fs_fast_symlinks_count += fs_fast_symlinks_count; - global_ctx->fs_fifo_count += fs_fifo_count; - global_ctx->fs_total_count += fs_total_count; - global_ctx->fs_badblocks_count += fs_badblocks_count; - global_ctx->fs_sockets_count += fs_sockets_count; - global_ctx->fs_ind_count += fs_ind_count; - global_ctx->fs_dind_count += fs_dind_count; - global_ctx->fs_tind_count += fs_tind_count; - global_ctx->fs_fragmented += fs_fragmented; - global_ctx->fs_fragmented_dir += fs_fragmented_dir; - global_ctx->large_files += large_files; + global_ctx->fs_directory_count += thread_ctx->fs_directory_count; + global_ctx->fs_regular_count += thread_ctx->fs_regular_count; + global_ctx->fs_blockdev_count += thread_ctx->fs_blockdev_count; + global_ctx->fs_chardev_count += thread_ctx->fs_chardev_count; + global_ctx->fs_links_count += thread_ctx->fs_links_count; + global_ctx->fs_symlinks_count += thread_ctx->fs_symlinks_count; + global_ctx->fs_fast_symlinks_count += thread_ctx->fs_fast_symlinks_count; + global_ctx->fs_fifo_count += thread_ctx->fs_fifo_count; + global_ctx->fs_total_count += thread_ctx->fs_total_count; + global_ctx->fs_badblocks_count += thread_ctx->fs_badblocks_count; + global_ctx->fs_sockets_count += thread_ctx->fs_sockets_count; + global_ctx->fs_ind_count += thread_ctx->fs_ind_count; + global_ctx->fs_dind_count += thread_ctx->fs_dind_count; + global_ctx->fs_tind_count += thread_ctx->fs_tind_count; + global_ctx->fs_fragmented += thread_ctx->fs_fragmented; + global_ctx->fs_fragmented_dir += thread_ctx->fs_fragmented_dir; + global_ctx->large_files += thread_ctx->large_files; /* threads might enable E2F_OPT_YES */ - global_ctx->options |= options; - global_ctx->flags |= flags; - global_ctx->logf = global_logf; - global_ctx->problem_logf = global_problem_logf; - global_ctx->global_ctx = NULL; + global_ctx->options |= thread_ctx->options; + global_ctx->flags |= thread_ctx->flags; /* * The l+f inode may have been cleared, so zap it now and * later passes will recalculate it if necessary */ global_ctx->lost_and_found = 0; - memcpy(global_ctx->extent_depth_count, extent_depth_count, - sizeof(extent_depth_count)); /* merge extent depth count */ for (i = 0; i < MAX_EXTENT_DEPTH_COUNT; i++) global_ctx->extent_depth_count[i] += thread_ctx->extent_depth_count[i]; - global_fs->priv_data = global_ctx; - global_ctx->fs = global_fs; + e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); + e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx); + retval = ext2fs_merge_fs(&(thread_ctx->fs)); + if (retval) { + com_err(global_ctx->program_name, 0, _("while merging fs\n")); + return retval; + } retval = e2fsck_pass1_merge_icounts(global_ctx, thread_ctx); if (retval) { com_err(global_ctx->program_name, 0, @@ -2950,12 +2865,6 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx return retval; } - retval = ext2fs_merge_fs(&(thread_ctx->fs)); - if (retval) { - com_err(global_ctx->program_name, 0, _("while merging fs\n")); - return retval; - } - global_ctx->dirs_to_hash = dirs_to_hash; retval = e2fsck_pass1_merge_dirs_to_hash(global_ctx, thread_ctx); if (retval) { com_err(global_ctx->program_name, 0, @@ -2965,16 +2874,11 @@ static int e2fsck_pass1_thread_join_one(e2fsck_t global_ctx, e2fsck_t thread_ctx e2fsck_pass1_merge_ea_inode_refs(global_ctx, thread_ctx); e2fsck_pass1_merge_ea_refcount(global_ctx, thread_ctx); - global_ctx->qctx = qctx; retval = quota_merge_and_update_usage(global_ctx->qctx, thread_ctx->qctx); if (retval) return retval; - global_ctx->invalid_block_bitmap_flag = invalid_block_bitmap_flag; - global_ctx->invalid_inode_bitmap_flag = invalid_inode_bitmap_flag; - global_ctx->invalid_inode_table_flag = invalid_inode_table_flag; - global_ctx->invalid_bitmaps = invalid_bitmaps; e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, @@ -3043,7 +2947,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) { errcode_t retval; - retval = e2fsck_pass1_thread_join_one(global_ctx, thread_ctx); + retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx); e2fsck_pass1_free_bitmap(&thread_ctx->inode_used_map); e2fsck_pass1_free_bitmap(&thread_ctx->inode_bad_map); e2fsck_pass1_free_bitmap(&thread_ctx->inode_dir_map); From patchwork Mon Nov 7 12:21:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700603 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=c0gv/aUw; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vr85WW1z23lW for ; Mon, 7 Nov 2022 23:28:08 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vr854BYz4xDK for ; Mon, 7 Nov 2022 23:28:08 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vr851rJz4xGj; Mon, 7 Nov 2022 23:28:08 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=c0gv/aUw; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vr84vtyz4xDK for ; Mon, 7 Nov 2022 23:28:08 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232239AbiKGM2G (ORCPT ); Mon, 7 Nov 2022 07:28:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232235AbiKGM2B (ORCPT ); Mon, 7 Nov 2022 07:28:01 -0500 Received: from mail-pg1-x535.google.com (mail-pg1-x535.google.com [IPv6:2607:f8b0:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1AB281B9C5 for ; Mon, 7 Nov 2022 04:27:58 -0800 (PST) Received: by mail-pg1-x535.google.com with SMTP id q71so10302147pgq.8 for ; Mon, 07 Nov 2022 04:27:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JNdDEX4CpPBN9MdYhG0WPzSQr4IM9bJjX2aLY1pgkWQ=; b=c0gv/aUwWBDsz+rpAhFK1OJM1ZyarjQ6j1OHF/XzR+8JxUx+64PbcMng9V+n31C5xv gbwpCos9lB0Uads3MgUnWl+J1QG0Py6M1Ue5QHGDFtaVYAf3+zVyr/YoNp6Xfh+ljUMk bk04gbJ9zhrGbIDaYYtu1TJqGvhXjFX0KTkhj+QUPbTqAyCVHEnvbXJJ8RewiTEWDlwg hd8NMjVGxuGPAncVGi5REoHMHaDyATKs3ObEPNi5TMmyYtkhkrNRuvQ1ZvYy6qoviyeX /ToxJbtNSz2g07SjTvkqP/mbkQpRtXXbjNYO3u1Kj0FURnVaz5OygElWTcoQNM6ARD8x /f4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JNdDEX4CpPBN9MdYhG0WPzSQr4IM9bJjX2aLY1pgkWQ=; b=4M8Fw+SDr6KR0Zh95LRiljOdtRKlqo/b7RkNv0s7rYERZUCBeLW31Ul+j27aX+fyN8 S0+ENcNNFJtQGP/tydh6lL5ycblxYkCFavIrqA15QCySzoi+nzTj/adBHBr8/eKIhuV0 3GlzxWtTVDV+HEaNDRhC0uY5Zsl5o7pxGJbEO4ai7cmvaAbjR7OGv+C+O2+opwZtvLam dn+ThO8At+hAjHzHZQH5U5dPmLafK41Q1Ak2vO/IkOMypk2QIlPFWSefS5yD+LPsNr2H fPpRIl9E3GTpV7jxM9U73oetHf9m4y9ZPGe5hKL/xndZ5GFKUd1kE1yibp9QoayCdJhp oOJg== X-Gm-Message-State: ACrzQf2Br16T4Nq1D3/DiDfVreIK/g0Qrp/8B3WOoL+QfMOaXgtkH1rQ TdJ3xfmbx+0u5fcqDC+2EYr1ZYb7L0Q= X-Google-Smtp-Source: AMsMyM7F4U2+iAhgn4j5mTtt4MHCIO5DhrXKbev7na2bMej6M5cpzzBqSARbvH901cKJw2bQLrz8ag== X-Received: by 2002:a63:e10:0:b0:46f:9191:7672 with SMTP id d16-20020a630e10000000b0046f91917672mr40135327pgl.552.1667824078182; Mon, 07 Nov 2022 04:27:58 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y13-20020aa78f2d000000b0056babe4fb8asm4348001pfr.49.2022.11.07.04.27.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:27:57 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 55/72] e2fsck: set E2F_FLAG_ALLOC_OK after threads Date: Mon, 7 Nov 2022 17:51:43 +0530 Message-Id: <474758c68a19b0a2fa5d5647e81d047247a93f3e.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Only flag ALLOC OK after all threads finished without problem. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 8e85f70f..a5dc6e44 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1385,9 +1385,12 @@ static void e2fsck_pass1_post(e2fsck_t ctx) { struct problem_context pctx; ext2_filsys fs = ctx->fs; + char *block_buf; - char *block_buf = - (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3, + if (e2fsck_should_abort(ctx)) + return; + + block_buf = (char *)e2fsck_allocate_memory(ctx, ctx->fs->blocksize * 3, "block interate buffer"); reserve_block_for_root_repair(ctx); reserve_block_for_lnf_repair(ctx); @@ -1465,6 +1468,8 @@ static void e2fsck_pass1_post(e2fsck_t ctx) ext2fs_free_mem(&block_buf); ctx->flags &= ~E2F_FLAG_DUP_BLOCK; } + + ctx->flags |= E2F_FLAG_ALLOC_OK; } @@ -2355,6 +2360,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } ctx->flags |= E2F_FLAG_ALLOC_OK; + ext2fs_free_mem(&inodes_to_process); endit: e2fsck_use_inode_shortcuts(ctx, 0); ext2fs_free_mem(&inodes_to_process); From patchwork Mon Nov 7 12:21:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700604 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Pe8RIuGt; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VrK1WrKz23lW for ; Mon, 7 Nov 2022 23:28:17 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VrK14Llz4xGj for ; Mon, 7 Nov 2022 23:28:17 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VrK11N9z4xZ3; Mon, 7 Nov 2022 23:28:17 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Pe8RIuGt; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VrJ6b8Rz4xGj for ; Mon, 7 Nov 2022 23:28:16 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232200AbiKGM2P (ORCPT ); Mon, 7 Nov 2022 07:28:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52874 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232244AbiKGM2I (ORCPT ); Mon, 7 Nov 2022 07:28:08 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03AD01B7AA for ; Mon, 7 Nov 2022 04:28:04 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id 4so10959470pli.0 for ; Mon, 07 Nov 2022 04:28:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7NwtTBgowLxr+7Lmil9mnFdsbJs/rqDOnikb6oPdack=; b=Pe8RIuGtO7V8axRq8Zn1pqb6nORa1NyCinm11i6AZiDWZymmvrkfmbMwBvZnf2cAsw gk/LpfNd+0ChbZWKJqfmSFQvBA/PP9+Ls9oc9/lL5HfwrVIKannpxR8p+8H6TxknzpWe qqJ4MfkW+Dev6tm1VDPjcmzq36kVw4a6Jcj4Aw15pncwTJ9+PnN/wD772cl117haw8Ya Q5u8zt9I9vmPg0Uoqw9v11wDqb4Vp3JjA4kgH1dsUndVmRgT+XBHpKbTQ990z4RUX4nh CGQNTVjBABHFIyLxAaUY4550ePw1div19xkbB1HmpquljpnNm+826cF5x9rJFXrtufVO 31VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7NwtTBgowLxr+7Lmil9mnFdsbJs/rqDOnikb6oPdack=; b=8Fq2a54GGWbVdfFcNA+xsiZeAg+Zq/xDSwhPJxCZZFGXs7rk68KjRqDbcD+ux47FJt WD9v505SOQMa4KUFxFM08iN/2/0VWn1SOOOmSpJjPk6zMQACE8Bjd7AiVKqsPQf2zIw2 nFZ6dRJoveyq0vnkA2NJp+iql6yHFytc+UlX5V7oDRlBhhxbm1mI28yPwmZlEV0QfNa8 UXqFy4gEfKLB9ntNjD6MQ6TGYf6Kwf1P0Jj+um0Vv+Mi3rZAO8mtFt9LIqArIbokoc/y Lxb71sklE5+DBTovn0B8uTPDauUK+txS8LS2Yga77rYhXHPhKp4C6Ni2h/tH4dqPCwcU F66A== X-Gm-Message-State: ACrzQf1TneLqykifjX2mdHm0yVcD9NjdmT74U4L3JdmY6ohDNpSmORUZ d3nShg0Y8b3x0YfcvNTSy3U= X-Google-Smtp-Source: AMsMyM7+sLKcGzEnmz6T+4yhEzGGY24hmdnoNSKmLU5I0xB/h4hRbngeGfwyc6vF78eyzTokcYBJ9g== X-Received: by 2002:a17:90a:748c:b0:213:854f:f78a with SMTP id p12-20020a17090a748c00b00213854ff78amr49999825pjk.41.1667824084003; Mon, 07 Nov 2022 04:28:04 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id q10-20020aa7982a000000b005618189b0ffsm4382758pfl.104.2022.11.07.04.28.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:03 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 56/72] e2fsck: wait fix thread finish before checking Date: Mon, 7 Nov 2022 17:51:44 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Before proceeding next inodes, waitting existed fixing finished. Signed-off-by: Wang Shilong [unlock from Jan's orphan inode path as well] Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.c | 3 +++ e2fsck/e2fsck.h | 5 +++- e2fsck/pass1.c | 70 +++++++++++++++++++++++++++++++++++++++++++------ e2fsck/util.c | 56 ++++++++++++++++++++++++++++++++++++--- 4 files changed, 122 insertions(+), 12 deletions(-) diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c index 1e295e3e..a5150dab 100644 --- a/e2fsck/e2fsck.c +++ b/e2fsck/e2fsck.c @@ -187,6 +187,9 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx) ctx->fs_fragmented_dir = 0; ctx->large_files = 0; ctx->large_dirs = 0; +#ifdef HAVE_PTHREAD + ctx->fs_need_locking = 0; +#endif for (i=0; i < MAX_EXTENT_DEPTH_COUNT; i++) ctx->extent_depth_count[i] = 0; diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index e3276924..01bd9d01 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -488,8 +488,9 @@ struct e2fsck_struct { #ifdef HAVE_PTHREAD __u32 fs_num_threads; + int fs_need_locking; /* serialize fix operation for multiple threads */ - pthread_mutex_t fs_fix_mutex; + pthread_rwlock_t fs_fix_rwlock; /* protect block_found_map, block_dup_map */ pthread_rwlock_t fs_block_map_rwlock; #endif @@ -553,6 +554,8 @@ extern int e2fsck_strnlen(const char * s, int count); extern void e2fsck_pass1(e2fsck_t ctx); extern void e2fsck_pass1_dupblocks(e2fsck_t ctx, char *block_buf); +extern void e2fsck_pass1_check_lock(e2fsck_t ctx); +extern void e2fsck_pass1_check_unlock(e2fsck_t ctx); extern void e2fsck_pass2(e2fsck_t ctx); extern void e2fsck_pass3(e2fsck_t ctx); extern void e2fsck_pass4(e2fsck_t ctx); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index a5dc6e44..29333acf 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -991,8 +991,10 @@ static void finish_processing_inode(e2fsck_t ctx, ext2_ino_t ino, #define FINISH_INODE_LOOP(ctx, ino, pctx, failed_csum) \ do { \ finish_processing_inode((ctx), (ino), (pctx), (failed_csum)); \ - if ((ctx)->flags & E2F_FLAG_ABORT) \ + if ((ctx)->flags & E2F_FLAG_ABORT) { \ + e2fsck_pass1_check_unlock(ctx); \ return; \ + } \ } while (0) static int could_be_block_map(ext2_filsys fs, struct ext2_inode *inode) @@ -1374,8 +1376,12 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) ext2fs_mark_block_bitmap2(ctx->block_found_map, fs->super->s_mmp_block); #ifdef HAVE_PTHREAD - pthread_mutex_init(&ctx->fs_fix_mutex, NULL); + pthread_rwlock_init(&ctx->fs_fix_rwlock, NULL); pthread_rwlock_init(&ctx->fs_block_map_rwlock, NULL); + if (ctx->fs_num_threads > 1) + ctx->fs_need_locking = 1; + else + ctx->fs_need_locking = 0; #endif return 0; @@ -1387,6 +1393,10 @@ static void e2fsck_pass1_post(e2fsck_t ctx) ext2_filsys fs = ctx->fs; char *block_buf; +#ifdef HAVE_PTHREAD + ctx->fs_need_locking = 0; +#endif + if (e2fsck_should_abort(ctx)) return; @@ -1662,6 +1672,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) #endif while (1) { + e2fsck_pass1_check_lock(ctx); if (ino % (fs->super->s_inodes_per_group * 4) == 1) { if (e2fsck_mmp_update(fs)) fatal_error(ctx, 0); @@ -1672,8 +1683,10 @@ void e2fsck_pass1_run(e2fsck_t ctx) if (ino > ino_threshold) pass1_readahead(ctx, &ra_group, &ino_threshold); ehandler_operation(old_op); - if (e2fsck_should_abort(ctx)) + if (e2fsck_should_abort(ctx)) { + e2fsck_pass1_check_unlock(ctx); goto endit; + } if (pctx.errcode == EXT2_ET_BAD_BLOCK_IN_INODE_TABLE) { /* * If badblocks says badblocks is bad, offer to clear @@ -1694,27 +1707,45 @@ void e2fsck_pass1_run(e2fsck_t ctx) fix_problem(ctx, PR_1_ISCAN_ERROR, &pctx); ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); + goto endit; } else ctx->flags |= E2F_FLAG_RESTART; - goto endit; + err = ext2fs_inode_scan_goto_blockgroup(scan, + 0); + if (err) { + fix_problem(ctx, PR_1_ISCAN_ERROR, + &pctx); + ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); + goto endit; + } + e2fsck_pass1_check_unlock(ctx); + continue; } if (!ctx->inode_bb_map) alloc_bb_map(ctx); ext2fs_mark_inode_bitmap2(ctx->inode_bb_map, ino); ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino); + e2fsck_pass1_check_unlock(ctx); continue; } - if (pctx.errcode == EXT2_ET_SCAN_FINISHED) + if (pctx.errcode == EXT2_ET_SCAN_FINISHED) { + e2fsck_pass1_check_unlock(ctx); break; + } if (pctx.errcode && pctx.errcode != EXT2_ET_INODE_CSUM_INVALID && pctx.errcode != EXT2_ET_INODE_IS_GARBAGE) { fix_problem(ctx, PR_1_ISCAN_ERROR, &pctx); ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } - if (!ino) + if (!ino) { + e2fsck_pass1_check_unlock(ctx); break; + } #ifdef HAVE_PTHREAD if (ctx->global_ctx) ctx->thread_info.et_inode_number++; @@ -1767,6 +1798,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) pctx.num = inode->i_links_count; fix_problem(ctx, PR_1_ICOUNT_STORE, &pctx); ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } } else if ((ino >= EXT2_FIRST_INODE(fs->super)) && @@ -1781,6 +1813,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } } FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } @@ -1801,6 +1834,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) &pctx); if (res < 0) { /* skip FINISH_INODE_LOOP */ + e2fsck_pass1_check_unlock(ctx); continue; } } @@ -1822,6 +1856,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } else if (fix_problem(ctx, PR_1_INLINE_DATA_SET, &pctx)) { e2fsck_clear_inode(ctx, ino, inode, 0, "pass1"); /* skip FINISH_INODE_LOOP */ + e2fsck_pass1_check_unlock(ctx); continue; } } @@ -1866,6 +1901,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) if (err) { pctx.errcode = err; ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } inode->i_flags &= ~EXT4_INLINE_DATA_FL; @@ -1880,6 +1916,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) /* Some other kind of non-xattr error? */ pctx.errcode = err; ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } } @@ -1917,6 +1954,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino); /* skip FINISH_INODE_LOOP */ + e2fsck_pass1_check_unlock(ctx); continue; } } @@ -1980,6 +2018,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) pctx.num = 4; fix_problem(ctx, PR_1_ALLOCATE_BBITMAP_ERROR, &pctx); ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } pb.ino = EXT2_BAD_INO; @@ -1997,16 +2036,19 @@ void e2fsck_pass1_run(e2fsck_t ctx) if (pctx.errcode) { fix_problem(ctx, PR_1_BLOCK_ITERATE, &pctx); ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } if (pb.bbcheck) if (!fix_problem(ctx, PR_1_BBINODE_BAD_METABLOCK_PROMPT, &pctx)) { ctx->flags |= E2F_FLAG_ABORT; + e2fsck_pass1_check_unlock(ctx); goto endit; } ext2fs_mark_inode_bitmap2(ctx->inode_used_map, ino); clear_problem_context(&pctx); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } else if (ino == EXT2_ROOT_INO) { /* @@ -2048,6 +2090,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } check_blocks(ctx, &pctx, block_buf, NULL); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } if ((inode->i_links_count || @@ -2075,6 +2118,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } check_blocks(ctx, &pctx, block_buf, NULL); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } if ((inode->i_links_count || @@ -2101,6 +2145,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) } check_blocks(ctx, &pctx, block_buf, NULL); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } if ((inode->i_links_count || @@ -2139,11 +2184,13 @@ void e2fsck_pass1_run(e2fsck_t ctx) } check_blocks(ctx, &pctx, block_buf, NULL); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } if (!inode->i_links_count) { FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } /* @@ -2253,12 +2300,14 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->fs_symlinks_count++; if (inode->i_flags & EXT4_INLINE_DATA_FL) { FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } else if (ext2fs_is_fast_symlink(inode)) { ctx->fs_fast_symlinks_count++; check_blocks(ctx, &pctx, block_buf, &ea_ibody_quota); FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); + e2fsck_pass1_check_unlock(ctx); continue; } } @@ -2306,16 +2355,21 @@ void e2fsck_pass1_run(e2fsck_t ctx) FINISH_INODE_LOOP(ctx, ino, &pctx, failed_csum); - if (ctx->flags & E2F_FLAG_SIGNAL_MASK) + if (e2fsck_should_abort(ctx)) { + e2fsck_pass1_check_unlock(ctx); goto endit; + } if (process_inode_count >= ctx->process_inode_size) { process_inodes(ctx, block_buf, inodes_to_process, &process_inode_count); - if (e2fsck_should_abort(ctx)) + if (e2fsck_should_abort(ctx)) { + e2fsck_pass1_check_unlock(ctx); goto endit; + } } + e2fsck_pass1_check_unlock(ctx); } process_inodes(ctx, block_buf, inodes_to_process, &process_inode_count); diff --git a/e2fsck/util.c b/e2fsck/util.c index 5714576a..b7c1e7a5 100644 --- a/e2fsck/util.c +++ b/e2fsck/util.c @@ -82,7 +82,8 @@ void fatal_error(e2fsck_t ctx, const char *msg) } out: ctx->flags |= E2F_FLAG_ABORT; - if (ctx->flags & E2F_FLAG_SETJMP_OK) + if (!(ctx->options & E2F_OPT_MULTITHREAD) && + ctx->flags & E2F_FLAG_SETJMP_OK) longjmp(ctx->abort_loc, 1); if (ctx->logf) fprintf(ctx->logf, "Exit status: %d\n", exit_value); @@ -580,38 +581,79 @@ void e2fsck_read_inode_full(e2fsck_t ctx, unsigned long ino, if (!global_ctx) \ global_ctx = ctx; \ +/** + * before we hold write lock, read lock should + * has been held. + */ void e2fsck_pass1_fix_lock(e2fsck_t ctx) { + int err; + + if (!ctx->fs_need_locking) + return; + e2fsck_get_lock_context(ctx); - pthread_mutex_lock(&global_ctx->fs_fix_mutex); + err = pthread_rwlock_trywrlock(&global_ctx->fs_fix_rwlock); + assert(err != 0); + pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock); + pthread_rwlock_wrlock(&global_ctx->fs_fix_rwlock); } void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { + if (!ctx->fs_need_locking) + return; e2fsck_get_lock_context(ctx); - pthread_mutex_unlock(&global_ctx->fs_fix_mutex); + /* unlock write lock */ + pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock); + /* get read lock again */ + pthread_rwlock_rdlock(&global_ctx->fs_fix_rwlock); +} + +void e2fsck_pass1_check_lock(e2fsck_t ctx) +{ + if (!ctx->fs_need_locking) + return; + e2fsck_get_lock_context(ctx); + pthread_rwlock_rdlock(&global_ctx->fs_fix_rwlock); +} + +void e2fsck_pass1_check_unlock(e2fsck_t ctx) +{ + if (!ctx->fs_need_locking) + return; + e2fsck_get_lock_context(ctx); + pthread_rwlock_unlock(&global_ctx->fs_fix_rwlock); } void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx) { + if (!ctx->fs_need_locking) + return; e2fsck_get_lock_context(ctx); pthread_rwlock_wrlock(&global_ctx->fs_block_map_rwlock); } void e2fsck_pass1_block_map_w_unlock(e2fsck_t ctx) { + if (!ctx->fs_need_locking) + return; e2fsck_get_lock_context(ctx); pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock); } void e2fsck_pass1_block_map_r_lock(e2fsck_t ctx) { + if (!ctx->fs_need_locking) + return; e2fsck_get_lock_context(ctx); pthread_rwlock_rdlock(&global_ctx->fs_block_map_rwlock); } void e2fsck_pass1_block_map_r_unlock(e2fsck_t ctx) { + if (!ctx->fs_need_locking) + return; e2fsck_get_lock_context(ctx); pthread_rwlock_unlock(&global_ctx->fs_block_map_rwlock); } @@ -624,6 +666,14 @@ void e2fsck_pass1_fix_lock(e2fsck_t ctx) void e2fsck_pass1_fix_unlock(e2fsck_t ctx) { +} +void e2fsck_pass1_check_lock(e2fsck_t ctx) +{ + +} +void e2fsck_pass1_check_unlock(e2fsck_t ctx) +{ + } void e2fsck_pass1_block_map_w_lock(e2fsck_t ctx) { From patchwork Mon Nov 7 12:21:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700605 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=DCab9Ze1; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VrM3tgXz23lW for ; Mon, 7 Nov 2022 23:28:19 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VrM3S7Jz4xGj for ; Mon, 7 Nov 2022 23:28:19 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VrM3PbGz4xZ3; Mon, 7 Nov 2022 23:28:19 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=DCab9Ze1; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VrM3JH5z4xGj for ; Mon, 7 Nov 2022 23:28:19 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232263AbiKGM2S (ORCPT ); Mon, 7 Nov 2022 07:28:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232256AbiKGM2O (ORCPT ); Mon, 7 Nov 2022 07:28:14 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 540521A817 for ; Mon, 7 Nov 2022 04:28:10 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id e7-20020a17090a77c700b00216928a3917so9571991pjs.4 for ; Mon, 07 Nov 2022 04:28:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sj8svmfV5V0dXHS0bF642t6Ke/oNORK33zioqZN2lqI=; b=DCab9Ze1Fawpsj5UwqXhTUzrLqBzRb5CD82N01XKLOeTcJY5DAAq9MpmcCgpjLNRaz eP7vpJGsj690vzoyNsyfQzrZio3LePBbBO1OEK/Ri2Ang76YL1H4pXBLf+ZKsmLtIkCf xFeSbP1IuRcJbUte11CxsBGB5w3Znp/JjXnCXcxoquVXxaONayOH4JntiXmbrC/KDBXu qC15oMEaF0Nc6LwSDRGbeEM8rjLcA6JnfFnkkT0ZzKCYWqX9LEhAiiCpSwfgE+P4s+NT 4gXLVs94/IGXbt5Lq0Xs6HTKGGo/MjL7QEtSEz6WRQmW8Ll3srBvbJ/lPwrDB/HmTNYO pqVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sj8svmfV5V0dXHS0bF642t6Ke/oNORK33zioqZN2lqI=; b=gfKaEKP8OWWQ6348F9Jw5/HzlVDZB2vL0GzA3V02m6t+l5S9YUIpWvLFcYyS5vNwEg BZ9SkmrSiPafXbDwjhs23GcluywoZOaSlnlJk5f2GmX4klNL+AepKakhUnT1i0NH8E4f G/NQ8++onOv/fMeOd+19yYHCyaJiCqHTxCdH+77Z/o8xr0HYmofm+azKieU0QvgAgKAt gFM9j4EJuD/rjGF04+T0HD+ZIHBR+ypG6Pg0UqMIdQMCjW3d0UXGXGJ6e2r/NOadou0W iaG1sVyeftMpoUjXuBBVJTDGhfBSqzQK9/yse7SXeG4XvrP5C4mjHppojwzRq9lIc0R9 jmcQ== X-Gm-Message-State: ANoB5pk8oCRwbOGM3rQa3rGNpuiyhCiHZAXt5KtqUUSYFGTdOHDTdRZ7 p82FvSZA37Q0ZjqBn3xW9CU= X-Google-Smtp-Source: AA0mqf6AL7gV0QwRz7kaI/sos0Vc6HJUiP/p3Q3hk3qUOQ9fxQ6z0XjF2umnwNc+p4Gm2+0ZOPM+PA== X-Received: by 2002:a17:902:bf0a:b0:188:6862:cc3d with SMTP id bi10-20020a170902bf0a00b001886862cc3dmr16798966plb.17.1667824089777; Mon, 07 Nov 2022 04:28:09 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id k64-20020a17090a3ec600b00205f013f275sm6059853pjc.22.2022.11.07.04.28.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:09 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 57/72] e2fsck: cleanup e2fsck_pass1_thread_join() Date: Mon, 7 Nov 2022 17:51:45 +0530 Message-Id: <52955ad12afbc737db46adcbc1963706b2d002ff.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Use e2fsck_reset_context() to free memory to simpify codes. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 33 +++++++-------------------------- 1 file changed, 7 insertions(+), 26 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 29333acf..93cff80e 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -3008,39 +3008,20 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) errcode_t retval; retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_used_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_bad_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_dir_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_bb_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_imagic_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inode_reg_map); - e2fsck_pass1_free_bitmap(&thread_ctx->inodes_to_rebuild); - e2fsck_pass1_free_bitmap(&thread_ctx->block_found_map); - e2fsck_pass1_free_bitmap(&thread_ctx->block_ea_map); - if (thread_ctx->refcount) - ea_refcount_free(thread_ctx->refcount); - if (thread_ctx->refcount_extra) - ea_refcount_free(thread_ctx->refcount_extra); - if (thread_ctx->ea_inode_refs) - ea_refcount_free(thread_ctx->ea_inode_refs); - if (thread_ctx->refcount_orig) - ea_refcount_free(thread_ctx->refcount_orig); - e2fsck_free_dir_info(thread_ctx); - ext2fs_free_icount(thread_ctx->inode_count); - ext2fs_free_icount(thread_ctx->inode_link_info); - if (thread_ctx->dirs_to_hash) - ext2fs_badblocks_list_free(thread_ctx->dirs_to_hash); - quota_release_context(&thread_ctx->qctx); - ext2fs_free_mem(&thread_ctx->invalid_block_bitmap_flag); - ext2fs_free_mem(&thread_ctx->invalid_inode_bitmap_flag); - ext2fs_free_mem(&thread_ctx->invalid_inode_table_flag); + /* + * @block_metadata_map and @block_dup_map are + * shared, so we don't free them. + */ + thread_ctx->block_metadata_map = NULL; + thread_ctx->block_dup_map = NULL; if (thread_ctx->logf) fclose(thread_ctx->logf); if (thread_ctx->problem_logf) { fputs("\n", thread_ctx->problem_logf); fclose(thread_ctx->problem_logf); } + e2fsck_reset_context(thread_ctx); ext2fs_free_mem(&thread_ctx); return retval; } From patchwork Mon Nov 7 12:21:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700606 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Kxz3xD7W; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vrb5rSFz23lW for ; Mon, 7 Nov 2022 23:28:31 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrb5PnWz4xDK for ; Mon, 7 Nov 2022 23:28:31 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vrb5L3Dz4xZ3; Mon, 7 Nov 2022 23:28:31 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Kxz3xD7W; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrb5F5fz4xDK for ; Mon, 7 Nov 2022 23:28:31 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232272AbiKGM2a (ORCPT ); Mon, 7 Nov 2022 07:28:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232244AbiKGM2Q (ORCPT ); Mon, 7 Nov 2022 07:28:16 -0500 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5742140F6 for ; Mon, 7 Nov 2022 04:28:15 -0800 (PST) Received: by mail-pj1-x102f.google.com with SMTP id e7-20020a17090a77c700b00216928a3917so9572266pjs.4 for ; Mon, 07 Nov 2022 04:28:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QtXmD/lEl778OTe8VYWxOkNds5J1K1V5LHDiok1FnYM=; b=Kxz3xD7WwfSPZlUxKcITjoSiSeITQJR1+THK00JSuyp8C0PDmoUtY8lmQZ1UBTKMAP cXk0Zh6wiExpaQRCbtQV5Sr0WPeyQTAvR3D9GZAdJ6wUBBDs73CQvNADbKWMB2r/QlGR e0JzDPuLS5LaCUa1vE8f35HrysYflm/m73DSttJYRSZY1ZVDBf3W4ILlbySGSUlkY50e enebH91FAdIb8XR+UALwXoasbN+BFFDImvecZePXruNO3DI2pc6xgI/cKs2b8FDUHcJY G2sEyDLimbDdO49ClfyWGGIy+Fyyo9GeDCMSzFDWZUAtXAjryHHBE/kAvkEqd9njRHv6 tOCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QtXmD/lEl778OTe8VYWxOkNds5J1K1V5LHDiok1FnYM=; b=i+AsLz2+IWiRRgoJ2kH+DAhBNFs/npUwG4UydxnIHAeA08WtrXPd5VNzw8XVsYOFgG dEmJfOD/j54EVSbSOWnl4up93bL0l9skMRKs8dZcW/9mOqXg1iNK+Abqhb/RWx49dO5U qDiwSirChIUiBhQDveTMTZLa6y8UjRybWvrATevCtGxbtOFJ2W39R2hmS5wb49Ftn5fn kN5ItM3zY9b1TYQThmA8TYuBRqkYbhcNf0WBICZhp2R0YAmOfoL0VRxERhZ5C/H2Lrnk a0kkrBgOg3YccCiQcmFs2a5JXnvjp8AmVGPiOyw588opeBEh59gzI0xlDViaqVZ8aAOE frcQ== X-Gm-Message-State: ACrzQf1OOvbJkvJ6jAbMz6HYrPZTuUuPyPrlyrCK7U7oTCgLQUek79+e Z1ya9zOEEqojFgLG+hhOuX0= X-Google-Smtp-Source: AMsMyM5DqbpwtzYuU40o4T7Y+UmmzJID0dWzNkQz5edvjaAY2XAvxVt/rzzYE8Mgg0HZ3a7lCWuocA== X-Received: by 2002:a17:90b:264e:b0:212:d06f:35ad with SMTP id pa14-20020a17090b264e00b00212d06f35admr52612000pjb.2.1667824095649; Mon, 07 Nov 2022 04:28:15 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id s23-20020a632157000000b0046ec0ef4a7esm4113356pgm.78.2022.11.07.04.28.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:15 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 58/72] e2fsck: make default smallest RA size to 1M Date: Mon, 7 Nov 2022 17:51:46 +0530 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong If we have a smaller inodes per group, default ra size could be very small(etc 128KiB), this hurts performances. Tune above 128K to 1M, i see pass1 time drop down from 677.12 seconds to 246 secons with 32 threads. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/readahead.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/e2fsck/readahead.c b/e2fsck/readahead.c index 38d4ec42..40b73664 100644 --- a/e2fsck/readahead.c +++ b/e2fsck/readahead.c @@ -234,6 +234,8 @@ int e2fsck_can_readahead(ext2_filsys fs) return err != EXT2_ET_OP_NOT_SUPPORTED; } +#define MIN_DEFAULT_RA (1024 * 1024) + unsigned long long e2fsck_guess_readahead(ext2_filsys fs) { unsigned long long guess; @@ -245,6 +247,8 @@ unsigned long long e2fsck_guess_readahead(ext2_filsys fs) * in e2fsck runtime. */ guess = 2ULL * fs->blocksize * fs->inode_blocks_per_group; + if (guess < MIN_DEFAULT_RA) + guess = MIN_DEFAULT_RA; /* Disable RA if it'd use more 1/50th of RAM. */ if (get_memory_size() > (guess * 50)) From patchwork Mon Nov 7 12:21:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700607 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BR1hUebD; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vrk6xq1z23lW for ; Mon, 7 Nov 2022 23:28:38 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrk6XJrz4xGj for ; Mon, 7 Nov 2022 23:28:38 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vrk6Tnqz4xZ3; Mon, 7 Nov 2022 23:28:38 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BR1hUebD; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrk6M7wz4xGj for ; Mon, 7 Nov 2022 23:28:38 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232297AbiKGM2h (ORCPT ); Mon, 7 Nov 2022 07:28:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232256AbiKGM2b (ORCPT ); Mon, 7 Nov 2022 07:28:31 -0500 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E40691B9F6 for ; Mon, 7 Nov 2022 04:28:21 -0800 (PST) Received: by mail-pj1-x102c.google.com with SMTP id m6-20020a17090a5a4600b00212f8dffec9so10169204pji.0 for ; Mon, 07 Nov 2022 04:28:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U3+ch67cgLBbn0L6V94LJXJr3S/uxJxECMuGxF8cj5o=; b=BR1hUebDNKgjB1qq+Nzbo6U/Z67vJ78KT3DXNcHZ04c9ShwQ1NU212ueAoMYIc5tHX 4FYBGCOZscEXARSdBHn75N8CsaVb69tGYj3UmyiEuwjSDdaa6u8hdQwpQWrnwQXef+EW Zqb9h0a1wfbHywGRH9xWVGxh17yHojwDJnXXqd0a8FlwmxBtz+lZHmbHfENcx73xocQ9 sd6ezF4xVRG+AKWapDOvJCpRl5EHpJvYPdkfhj/LHf1z64x64rFdsoVqfeDwjV+0wpQg hzECqBNcTQQnqC265Rw6JH8y5Vm4LIDhv1XL3oUObd8msJ+uJ9VEoI/7GuSKv3+40iuf NGig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U3+ch67cgLBbn0L6V94LJXJr3S/uxJxECMuGxF8cj5o=; b=W7bOw0SNqPGRrKjHipVmPjpsKvC5RxgOXzxuXQNLssoXO9lNFcFI5xEyC5X5c4vZh/ w8q91fqrRsElkkJE85HHjVM/7qjeEjVYrSbRDnS/Fh7kANWGV+vB9W27EE9qEz91Fvap TW/kGYbOX1dei3upuBJBlG3e+d/kVGlZjKl7ewY9FpIrOkzKgh5k/goE3hnQVRyENiPh 6wbnB8SDEEjKCUlEujPSZ9zioFk+UmHfnb0se9muZCnN0QO2tyASe1tDQ4u02IfuR+6w RscWLWQ4t8TOOGEFFTQepLpv32brrzioAHLOY85b1ilveE+dCqFTr6e5ISa/ApTNaUNu R2WA== X-Gm-Message-State: ACrzQf0u1jVHkB6AzWEzWT7Erd9uZa9jYcyMok0nuKUrCEhsGP+QSU0o CQBQSK4DZKzIdejfuUx5n8s= X-Google-Smtp-Source: AMsMyM74V9jJLYodD/jnz6kQ5A0fU8Q9Y9QJR66D7NDw7GqtXncbVRIjPKuWNDUuzd2IKzxeGiXuEA== X-Received: by 2002:a17:90b:19d1:b0:213:7030:6bd9 with SMTP id nm17-20020a17090b19d100b0021370306bd9mr51816168pjb.43.1667824101353; Mon, 07 Nov 2022 04:28:21 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y13-20020a170902b48d00b001869f2120a5sm4854904plr.34.2022.11.07.04.28.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:20 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 59/72] e2fsck: update mmp block in one thread Date: Mon, 7 Nov 2022 17:51:47 +0530 Message-Id: <48d152d713a9af21eefbfa69b26a7dd417f0897c.1667822611.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong For multiple threads, different threads will try to update mmp block at the same time, only allow one thread to update it. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 1 + e2fsck/pass1.c | 34 ++++++++++++++++++++++++++++++++-- 2 files changed, 33 insertions(+), 2 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 01bd9d01..2dd7ba27 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -488,6 +488,7 @@ struct e2fsck_struct { #ifdef HAVE_PTHREAD __u32 fs_num_threads; + __u32 mmp_update_thread; int fs_need_locking; /* serialize fix operation for multiple threads */ pthread_rwlock_t fs_fix_rwlock; diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 93cff80e..ed4275c3 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1509,7 +1509,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) dgrp_t ra_group = 0; struct ea_quota ea_ibody_quota; struct process_inode_block *inodes_to_process; - int process_inode_count; + int process_inode_count, check_mmp; + e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx; init_resource_track(&rtrack, ctx->fs->io); clear_problem_context(&pctx); @@ -1672,8 +1673,30 @@ void e2fsck_pass1_run(e2fsck_t ctx) #endif while (1) { + check_mmp = 0; e2fsck_pass1_check_lock(ctx); - if (ino % (fs->super->s_inodes_per_group * 4) == 1) { +#ifdef HAVE_PTHREAD + if (!global_ctx->mmp_update_thread) { + e2fsck_pass1_block_map_w_lock(ctx); + if (!global_ctx->mmp_update_thread) { + global_ctx->mmp_update_thread = + ctx->thread_info.et_thread_index + 1; + check_mmp = 1; + } + e2fsck_pass1_block_map_w_unlock(ctx); + } + + /* only one active thread could update mmp block. */ + e2fsck_pass1_block_map_r_lock(ctx); + if (global_ctx->mmp_update_thread == + ctx->thread_info.et_thread_index + 1) + check_mmp = 1; + e2fsck_pass1_block_map_r_unlock(ctx); +#else + check_mmp = 1; +#endif + + if (check_mmp && (ino % (fs->super->s_inodes_per_group * 4) == 1)) { if (e2fsck_mmp_update(fs)) fatal_error(ctx, 0); } @@ -2437,6 +2460,13 @@ endit: print_resource_track(ctx, _("Pass 1"), &rtrack, ctx->fs->io); else ctx->invalid_bitmaps++; +#ifdef HAVE_PTHREAD + /* reset update_thread after this thread exit */ + e2fsck_pass1_block_map_w_lock(ctx); + if (check_mmp) + global_ctx->mmp_update_thread = 0; + e2fsck_pass1_block_map_w_unlock(ctx); +#endif } #ifdef HAVE_PTHREAD From patchwork Mon Nov 7 12:21:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700608 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JKu3DYpz; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vrr6J9lz23lW for ; Mon, 7 Nov 2022 23:28:44 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrr5sGGz4xDK for ; Mon, 7 Nov 2022 23:28:44 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vrr5pnTz4xZ3; Mon, 7 Nov 2022 23:28:44 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=JKu3DYpz; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vrr5k1jz4xDK for ; Mon, 7 Nov 2022 23:28:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232282AbiKGM2n (ORCPT ); Mon, 7 Nov 2022 07:28:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232277AbiKGM2g (ORCPT ); Mon, 7 Nov 2022 07:28:36 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3551C12D1B for ; Mon, 7 Nov 2022 04:28:29 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id j12so10911395plj.5 for ; Mon, 07 Nov 2022 04:28:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jFIewdpm8KInBVBgrg0PGTjUi+OYV8V6Z0qzY/EFJPg=; b=JKu3DYpz7MaA651zSPW2aryhNcbW0+bOB4HrOWsqHRaTIoj6R9dZ8bLAZ9gF9toBjj aemJreuoeR632vtT6O7Y23f26EXQMhQ9+Fdl9Nukr0ZUNwUsuPw48+8uYgVrLtfEOalg cU97O6MwN3zdn2beb/5gjETBabwFTrsp6ElVtgvdYofJjc6gHFukAn/JFDaQhnKfWNqS b2TXN2YBDVuAeTadUwSPSnG4MZn/ICHN3TtNzd3ltOd9eSGrtWSpI9Mg6jd6dg+pnyto jRx3/WcFxM/BNFu+ZwrN8HHcSHVHzzMgWFagl09MYFVSGOOm96SDfPfeBLx1T74R+eDC ZvSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jFIewdpm8KInBVBgrg0PGTjUi+OYV8V6Z0qzY/EFJPg=; b=TFcsK2iqtCc2GS5Ii/bnxUeoVN0O9XyMR04M9M0CKdQjoSJsYBBP6lWVqOrYKQ7CYS nzM9TOF7xhy+f4J3IEX085SoUNIdKOSJRls623n0Z9rQGB4zjghf0Z25Umam0bHwd7JV Fwwn0PUxjPTSUtosZzQzCf8wjtjrwZe9Bm264CDzmGYXls4ef0SDh2mCsoX/+1Tqm7bI emweB+BGq5xzitisElNkTvK6t9CHQCEjude45s7KyJRrycIxAGw1FAVWUxJ252NdO2gn Xmef3XXW9Lb2ZoP4Dm5SqztwDDYd2suJnFw0E9b9ZOgaTED6Z+9UaEm1lusqn+Y371pF koSw== X-Gm-Message-State: ACrzQf1VBKOoCpeeJHbVCr3zakWQvb85n4fd8An2Bi5YMU5iwyzUgLyi /XhSZdwlbsRuzeIqEFHB0Rs= X-Google-Smtp-Source: AMsMyM7gc1wNho74ImwF+ZPwBfiqyWsVc9sHBGvveqEuY/bHNQCuf3BH31y3mVdc4Qkq5uuWvf1Ojw== X-Received: by 2002:a17:902:ec01:b0:186:748f:e8b6 with SMTP id l1-20020a170902ec0100b00186748fe8b6mr49858814pld.131.1667824108741; Mon, 07 Nov 2022 04:28:28 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id w3-20020a628203000000b00562784609fbsm4332436pfd.209.2022.11.07.04.28.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:28 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 60/72] e2fsck: reset @inodes_to_rebuild if restart Date: Mon, 7 Nov 2022 17:51:48 +0530 Message-Id: <89bc200265e9aaa5a4d610f0fd18fc7acd934dec.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong Verify multiple thread on a corrupted images hit following bug: pass1.c:2902: e2fsck_pass1_thread_prepare: Assertion `global_ctx->inodes_to_rebuild == NULL' failed. Signal (6) SIGABRT si_code=SI_TKILL ./e2fsck/e2fsck[0x43829e] /lib64/libpthread.so.0(+0x14b20)[0x7f3b45135b20] /lib64/libc.so.6(gsignal+0x145)[0x7f3b44f2c625] /lib64/libc.so.6(abort+0x12b)[0x7f3b44f158d9] /lib64/libc.so.6(+0x257a9)[0x7f3b44f157a9] /lib64/libc.so.6(+0x34a66)[0x7f3b44f24a66] ./e2fsck/e2fsck(e2fsck_pass1+0x1662)[0x423572] ./e2fsck/e2fsck(e2fsck_run+0x5a)[0x41611a] ./e2fsck/e2fsck(main+0x1608)[0x4121b8] /lib64/libc.so.6(__libc_start_main+0xf3)[0x7f3b44f171a3] ./e2fsck/e2fsck(_start+0x2e)[0x413dde] @inodes_to_rebuild could be not NULL after we restart pass1 Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c index a5150dab..53af8905 100644 --- a/e2fsck/e2fsck.c +++ b/e2fsck/e2fsck.c @@ -79,6 +79,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx) ext2fs_free_block_bitmap(ctx->inode_casefold_map); ctx->inode_casefold_map = 0; } + if (ctx->inodes_to_rebuild) { + ext2fs_free_inode_bitmap(ctx->inodes_to_rebuild); + ctx->inodes_to_rebuild = 0; + } if (ctx->inode_link_info) { ext2fs_free_icount(ctx->inode_link_info); ctx->inode_link_info = 0; From patchwork Mon Nov 7 12:21:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700609 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=oUm1/pVB; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vs31XZsz23lW for ; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vs313Xrz4xGj for ; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vs310Lsz4xDK; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=oUm1/pVB; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vs15vqVz4xGj for ; Mon, 7 Nov 2022 23:28:53 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232273AbiKGM2x (ORCPT ); Mon, 7 Nov 2022 07:28:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232276AbiKGM2p (ORCPT ); Mon, 7 Nov 2022 07:28:45 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2932B1C for ; Mon, 7 Nov 2022 04:28:40 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id d59-20020a17090a6f4100b00213202d77e1so14433732pjk.2 for ; Mon, 07 Nov 2022 04:28:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mUzIobAiOAfLrum5dabg5BemWMtExp3+RtX/f3aQJn8=; b=oUm1/pVB6mNnqJsRbFV4MpovPjGDnYyIx10ZmfdURrEAF4tLQ3ikMF3gLqE/u016vh 4dtPMVeL6BINzH7rPHNWTHD6jha163gx1d2LCbDSM9Er6dp4XtjB617EdHuDpHijSZwi bCYHxwE1VQ98lMqGKIisIJP1jsFDBByf30QzZGc1NVfUsXPo4e9exB/gFy9YsQ41iQfd LOzVHriYjBgJKNw8StGKkpbCh1LviFVLHEVjku6+OaGVMXeXnrL3TMiIwr+EqsA/Uo+A rgGBSECpPJSEzjAoKrhQowo11zci9B1lJT99KMWuABpCRTh7ySjZGiUCha1Zc3UHskRE SBMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mUzIobAiOAfLrum5dabg5BemWMtExp3+RtX/f3aQJn8=; b=XCSrgaGqZaC4kzANAQfAssQIlHf+d0oLvfT0v6IY4NJoJ2bW2kCfSqmnvku3YtUxXP rkv6Ks8BMghFi2IqqOLfeUo0pM742cXmNPmmzLu4fIihSVDi2xXpiGqyv1fCBI2dhxIJ tfAaRpYHk6CoDw4Jp1tQStCZ0NJo0xQhMGEQZGDLHq2nkMH25K6cnqpmF6UESko3P4KK jO+YD3BcreYSIXuG8nl5qmXmACl1ZxhEMfP8VO2wEfwYreeTDfwbD7ai5EI5ZqykVa40 Nw4l2OzEPqJw4wPofjBz+rBSWdoujEA6i9n1B2i55cRQxkCc54VLmGTu3WVZazjWnapZ 9UXA== X-Gm-Message-State: ACrzQf1oDFDNlYLD8bjNpczKbBn3iy+5UANZK0QHY0Kc9Po+StaZfIj9 f8BZH5tpQV6Dghl9Wc20ogw= X-Google-Smtp-Source: AMsMyM7Zmjzy6SPGMGZSsWMkFEEr6t9vXzCKML7PuvrCfvKZkovYcs9JbWZAeHfZxzUWvNdCtYym9w== X-Received: by 2002:a17:902:ea95:b0:186:a6b7:4410 with SMTP id x21-20020a170902ea9500b00186a6b74410mr51733863plb.109.1667824117019; Mon, 07 Nov 2022 04:28:37 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id t17-20020a17090a5d9100b00209a12b3879sm4143955pji.37.2022.11.07.04.28.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:35 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 61/72] tests: add pfsck test Date: Mon, 7 Nov 2022 17:51:49 +0530 Message-Id: <308dc2a6ca223852608b0a7342fe915c18be8f2f.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong pfsck run on a clean fs should not return any errors. Generate an image with possible features enabled, especially EA shared blocks etc. Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- tests/f_multithread_ok/expect.1 | 7 +++++++ tests/f_multithread_ok/image.gz | Bin 0 -> 796311 bytes tests/f_multithread_ok/name | 1 + tests/f_multithread_ok/script | 21 +++++++++++++++++++++ 4 files changed, 29 insertions(+) create mode 100644 tests/f_multithread_ok/expect.1 create mode 100644 tests/f_multithread_ok/image.gz create mode 100644 tests/f_multithread_ok/name create mode 100644 tests/f_multithread_ok/script diff --git a/tests/f_multithread_ok/expect.1 b/tests/f_multithread_ok/expect.1 new file mode 100644 index 00000000..4742f408 --- /dev/null +++ b/tests/f_multithread_ok/expect.1 @@ -0,0 +1,7 @@ +Pass 1: Checking inodes, blocks, and sizes +Pass 2: Checking directory structure +Pass 3: Checking directory connectivity +Pass 4: Checking reference counts +Pass 5: Checking group summary information +test_filesys: 10251/32768 files (0.0% non-contiguous), 2218/131072 blocks +Exit status is 0 diff --git a/tests/f_multithread_ok/image.gz b/tests/f_multithread_ok/image.gz new file mode 100644 index 0000000000000000000000000000000000000000..42622c3786a81f348f2935d4f9cc2812bcf539d4 GIT binary patch literal 796311 zcmZsDd0b5U|Np&QLdaH0YS1O5hC*5oq7Vj2v<*@r85O1V*b*usDQ&||l9u77P0Jva z)moi2ay?#B-{A1g-MTuXxlc&!%*nZ`Yr&2R^&X_lC!>_;{TW8E#9C_u2 z?X$mT&bu@3*^C+27RxT$Fk|(}1BVYBuI6ReHoL_PRy-a|8Mbk%tQWKj@bvCM&9##cH`ABn8T#VO2Cw9fWDfZ12lq^AW5)8dn$>?Yc*2zkIZCI;rnxhkQ?M zcfe9Vom}Q)zZ`+mxSgx813C2`7Y5H0+Fgz*D+~7zQ4Z4i8dnqh&!Bv2@R{0QbbMpi zzR2~EDlwakQk(s8RI4KQ(b*l_}%GXG&h2_*3^- z-|%A-4qp?$eadlQe4YocN30c4a6hJ`<3EcPu>-OM?El&HZqHsn(oZf5g%oe_uhul5 zTRFJW|7`H7a?#LoF6J?L*Z>thvrqqB?8JW;+xdSiHvFH(7XJ9pVV_6*SMs0#m3+_W ze#0F1Gf+i_QE$)cEDUYFznWHHP{A$0mP` z{YUaK0s9j=Ecd_WT#84c9si$G1#p=7GJaGGpO~oJM zqXyZ+75(YA{vdA>3SYJl4JLKO=Z86PZc4dIzK99K0s8{2JDo*~G+@=IVXzXQK6BC9G@=5qWEU7w}y-kwd%^F`AI~ z9<1ivjp7V?KkA8(A-GV4^{Xj)F3?2}@$oi#+E#MIo#Fu7686n0=+?W@Pxr(x4itGl z7G1rx#QhfI6|(3mDdq5;*Wc+Ob@b8J-nqwu?bh_gh3(lOClEarr3-d?1kGaXhd-j% zG0(aBt5Slq>*q_JQLohT@UB8tyPqoXMQKez!g;@yJzpREvuO0o#E$+EqLnhOU5hfh zPc5E;SSBvAd(B**hAii7O{nH4nEfzO3p=@PME=AGTi|!H;}+Sk_r|-BBWnwf_K-_^ zNDr6};T$ISSCHhH%<&O{`O(^VzyAAhgTe^W>tKP$zEqq0aQ!W5NSev(^UhY}-aBxy z_dPt__QRPEKXKuUD<6xFu`+qeTXbZm`Mo)`rsyQP#sRH5+SN^y>Z#4DaCWP_1g30>zy-ckJMg#>qn& z7o;JcUuEXY8#DJU$3Hu+cqlfUnwDT{JFQ1a;Nv_?aK`T3Pg6H8{NWC!h2 zBedAd=f1~B9D`a4ju6=|Oqe4RRd;7Q9o;vmm?{oFc7!)Zv%CJwYPaB~ip0O7F7PSi zDZg&TaG?F8Tb5xQ^Uctrie%n)PvZVQ)gZc&^ZEx%8qIrbwUlu7GRJXjVUqXC>OMnm`f+3~{H zGn$a!zLe=Wl3;`Ar1li1La`_pSh#4Ds8i=yHhk6Sv8XuE@bb8A^L6+urK}G8tfv$$ zY&Y>Fy93{Dvj0=ldMAB}|KhuW1n*_->~d0~uE6xc;nF)aPimPA!!YH2?2(DA^*uRi zS_n7P11fX=q}BPk)NJAWx9D%f^FA2h9<86By-@0fR>{#66#H)W9H=v6?t_Z-02_Fx z|FpEFJccQYS80HUdl)sgBPItWD&zl%#ij3fJwI1hx(IF{<2YiHvAX}9WknlnnZy(5 zKe_Z~$JjY@=Mcv&X+mCWJ^y=7sR>keU>(!%{HN+hmQw7s&mzumr~ivDWS3?p6y;&F zp+RXt9eHu|vPRpe?fj71iRA2?CFm%4awcCzk!}Rf*2bmt3}f8+16!fP?BZf2gY|<4 z`Fjri!uo#)E{QLsJ{I;n21ffTKNS=^O=Ll=9NQj}P@I{p6H06`HA|4t{jf8NetQ2^ ze9s_~R)ekPVhYFj{;!%7tL96R!_;hnV^M|t*HuCBPzHRN1Acwyb!zekKH<9W8~OB% z@R5}%OyLS*2crm)0xxE8Q+-3A#PsOuG-N30hC(-a(G!W~Nks-Ye~_v*C0}okL~}^A z05iLvM;;r{Hz7ng?SOg(Cpbqr*a z!p5KvB@F0Es_O-`je-m3PY372ka^uWh}yg@H$6kW0gtgF4d>p9zP)WKrQC=(?gBn0 z&Bi_s^*h4=QFY_96kiW^GY3ppNM1$mf{<_M9aF1yhOUaZCGSB z0*4-vkt^D$w@YGCTpw7ax^kyN-l~J}{79@f&VG2GQ5^iq5+VzFWfLN;wyNo}OH((j zF!CU$d{f^y3ynkI27%?+G~PvObgX3%;cSLn5gI!D{xM!^=s@)&UY3 zs32+cIoF1}X~X}(N2Lb1#rktbCDzkm9uIe6qh3S2IWuM1@$cxC18wKhQ5_&bhPU?g z8mHY*sOiABSmRN07sowSprcOXV@wzJxq-muJnmG&&6yO=d|RE3CS+t_V+S03=8tX9Q&In+V7@^!#*z z_5~iU82d&g9i(OUEIjd^92w_Hfvt&+phfD_v5n0c*F310jPN%b_# z;?s)_zH<=aH0W_Q;4EpYdV`~yBVb~22J5v(3eK;qs1KC*2)iN==N)g)zoAYvHfG=; zhPILJyrVneS_WV(_M!%)^{g?GxX8dCmL~A2m)D0trIWy;t?aAcr+7ktr~-i+xzXVA z%F&pqP)Q;Hm`~))b$4C}>pFS`smE z-MZRV&w`*A*J~8u?iI+=2iDi1pzHZqpxxNrbA}h%*TZKs08Ee%2ze;HJYAw+R5uOz z`Ch9tk&3(gMZjL9G!JK7j){A(Nxt#RIgp&BsMm;lf=ARTeP9%{gFwh%|m*-)33z5AhikinyTd}Um3g@}N? ztk!0?Ef5K7V}cR2oX+?5P?_pL zCtAnoE0;DAWk4N-OzUT462d03!`)%l$4t%Bu-njQC zffwy=Q$HGhy$*jg7bN5W)l`R!a#NyA^1x|t3jJzon_wour+VOf8;0$V>@|a$%mHHF z4zV|`5MrkAtRam8@@BsGVBou$6zEi z0%jt-InPahGfWSNOd8^GsEnsrArQ>{t*8jJ2dx^!RPg>xpBF9exh0MH<#Esl>+e05%a$`~z$`Ri&qW|d^RNIhpLbOp5a zopS8cRL47&b(`qK4q*|Z1Wkv(s<%9XC7-G3feYsyTGts^K=4<~fOn#qN4ZigV#^aF z0Y?^_>8D;+nsoIKJ(M%n@jD!P6$^=K$tZ0?TEh3-K(en*VKRoFESTur-#3c1+S4C+ zsF=b8W&wiu#@+avdkw7?z#6(b4Q+ccbjY!T^$g+#^pXSm)W&R2W@F=Ip5GK9>gUy} zyV;{>VL?Op0p83rZST>dqfo(?AYhquHKImDDo{%y8w8|9YG$d%7^`#zY(S||-WX#< zs%?K*2yF8h1wE`^qpoAd!P3wH0Si~W6SML24e@}c@7^Gw?X9F{$JEvn)Zl=^xFj#BByUI$CjhThHrm_-z4%E4G;)z zXAd)YOwK|0t{xXVGh{y;77_&QLSq@=wNI(R+f$~%W6+O}cqlD6pq0-v*))$}r`kYE z(3=-|H7*DGo5v3>y^vfTsig%wa?}JAyb8T(YP zAn!!A3AOY*Dn4Aem4Z?jZeH#B}w9-j^W~ zUjj^=6gOFTS@?3!96|x;ogDOQI1-yqCirnx58Ow&dHul2nE-c7csDXwPB6A4rc}3T z-cy!@I}8Dr7E(*kEE=?!bP8`bI$rIS;bL4D5KL^sEAYh@S|;p1UC|HqE({xSiO8SKNdQ?(u(yWd5^;$W7oLM}BkkoJ4x{|MGDx^sD1$ z)8_khxZE9qu%iHplGTmbPpn7!`Ks+YR+hlmuVGBhUP5U*zo z686+Qa=UZ>%+=2w5^7hb#Sr)OF)x$a6ma0zpoE?d&=eTG-TCIuO!Z0T;Y4Ex5;rR& zf8(#~M`j2yUpa>)Q9(8fwe`xpWPOgP02|Yg;o(Ql=gUPoo3_&-(RHnkl*{SZJYR7H zCY6v@Kdc}z#knd(e;*MZ2f-e-0U1g7u}K=lE{l%{k7bDbdClh_T|la6LhLo((4sQo zg%G}&+ksT~2tQ^#`2bU%9k@24w9c4w&P6zrVBU*ecdoiUzXbx%_YZb?5eqN>gQGPeEt zZ6aHUnK>ige4CT>(NryoU!b!9?wrvRsdK?u9`1P@SdA6HxqK4Ast+LL#WpcYroG*l5C8`c;f{H z$;we^Gl96L0F>N0+_Ew-h)B>lpyqB(pBlR-#YcUq|f?jeljslWn|2Jp3pZ;f8JkbvBEl(5U-=CskIq0`JOD_1d1Y^($wSe%CIdpJ1Gi5PEQq*_g zW&=xq(Hd$1OMtwlK#SSCW&ICJKf)==KlAa20>;%fNCm{7XiN^ewpU@%(X43@sex8= zqoVef*W;AyP)UK=bhLh~`@Kwq^HWKNr2^9T%*U6@Hykt~SZn6PI_6m6t*00VQM?9! zSCogh1Yaw7UU_T_yslX7HEGI8G)3f7zROaG-)hqMap+?{~24Wd-q4-EAjgj!E;6tEl-l9{0J8{D@W?EB5o z(5r1gumS#T80;I6ucS2a?XQU>f^|5&LZ_x7Jyoq6qA2UN~ zpLrWlL$!skXEIe`eWn3AF=@0k>&Nz;(5{&v*~~a2xcD_rwRpLNCCdbNiBeHh%ayP2Jm=g z)ZYAugP;F~<~h&B61^Q=j~13dWhMgsHr%~q>SPJ2byNl<4SWylFMh6E0Fjn#9nc{y z65{4F`UgC)Z#foJyMXRIwA}?h65lIq@=Qpkx{n`&d-`!P9Zc}>N)yp-Oa%-=TWVsy z+rBjr_oDZ5P`^M=E?1jPfj7ZmiiN&lTh%|V=Y8f91<50e&#n<&tgm}_m55=ayoX#p zlGtIjSsi9R^hORkE-0q?b%{FWj|(N2Y!|uMZM_Ag&AC$vgc+hKP@#Ied({sGOj?Io z=8L7pS4wYs)Ej$O5>NnHa_9M&I4uI(AQj-EC((es z(|;dp{7EBym0kmYvjn-7A z8n|{#xWTAAB0wBQ5{ju2iW@TsM4{R!O1n6cJB=(4Z^b}b9g~T2``ggk*GCyv8*uA3 zXm{o|XRiJv%r9$XA)=EXgxei?s<7WjK-H59$=4n3b{>Q&h^{558en)1KcW)6qbv~O z{e8yHBQM}E3OD!gvr;6i3VQc!$fGtNdL{Ec zs6cMJ5A~AAVe(>wfrP>C{L5{OKVgtl-FOI@?zp>5I4B1R3ucQSX2dZv%`kU{NC^!D z&8J;xyuB5k@s}h!;$qV7vz}?kj_0ubW&?5v6OP-uMXB>Kk=~+F zUu|DNCEI|^a+1c#6?hO#9x0now@Ar5yK9Ve1lkl8lbx4*YwC;@VM<0pG7lWW?3l${ z3H7o$&8T5#Voif_IMLnXzFchg5QRG~OiF_8z`Vd`Jy$6?6Q~D|ivv1eKcy~m$r_nZ z_$!$#Iy(NPX;WEc58;@>pww1APY+J5hGB<;k}s5ESm9;C#KjvR?dah-xv8gKhe2D( zfS#ZxdpN(?i|D>^3K#2)dew41U<#4*@IZj}p2;Z&%+`7PClQ-W0G!)ngXN#^gZJqC zsKQ2scDJG0)x|(8zEjvCK$;yNDNZnN^0DVU+6OCWbUBtBNO$IXU7SHAN4E_E72dkW z2P2*egz7pRMv&2@AK#WReO^r?Hr6l4B3z3zG^8UY>C#+KA8uAPmx_+65g8LTbU;t@ zr#F1dY~;%k;SYu%sw1o#_KYY^My#+Ld-6CbQ+01L#42EDjvjmXd~uVfMTQD29MpSt zRJ6li$!K%_e&RIS%dys~G>`Fk+2{9uGsq_Z;Qyk#a^z=!3>0`pIp)l-I+S|KU~*GH z*}c2+-Nt5c6X6!nGIp}sqzEa(Kcu~OI;(^b$%@rKZoCOWJ zGD3tc419j`qje*k9wV3ZghY90UJ{uy4!}!x=Atyb=w_$)_E1+C!Ql=gCeO6%quz4f zDZ{$2;DG1Jc=XIMG$2Ox)}^7lzTNVbPOXKM1i83}c`x&%rAXk!haja)OVDtnWS6N% zJ#1Q;Qve?mk;y!eauS{uFm|a;150no7^IV+jQ8u%)74L>9=mQksf(2kXmI!{(%v9Z z?;nzM%hUwCO{ryoa``?TNGXACH=ySzy4s;&HADt$%dv`t>A7y5yH>(SCSbggA&)E_ zuPf#)>nrxnT4&YEDD9z)4i2>_vWKBiGVNTtrqSA9sN#z&<0I)}~ESB=P=M1>17QmCazpQ&#*h9^U zhvqh|9(w=pk`C5#7^LiLb!cMQYZ)Q^f6~JvfOn1e>miqptIZNdsR%14h9on8a!J(d z(koSa^dPB31Kfhzd7E9A?Fm9((E;ywE-=i^OkF}8xd9y6i_uuPJb>YI3*s774>XIl z8rHcNIxi6^EE6=(0cpscJR9OFdnOlK)3i!Oi+?)=8t5rF-&sJtTI$3SwpEG{-R`wR z46Bou6Z`6E3P{H0_%rv9y&(kNLjlk9+*;IHed{!I_w`)tSk`FOJSuUezP}tx?7m}L zFDxpg5O4Cz0XmBb4KL+-`a}1T2KCUiimVzSNupvNYNz zSp0YuB4NN_x~9?o=CK7to;C!HKyP}vDg7=461=h=W|-4h`YZZcm<>&UUDulbOsd`v zZrzBiNdhYIa?GEVYB))51J3SCjH5N8`~Wg6=pP>Vj)2C~)!luUU?yGUf#;4E1h*#8 z;o(T1dRW%$#~t29*TdMk4zM|Gns2X_w`dh%{p+A-jTs&4eph`J{tgV#rE5_|tNO1$ zARavyIif?%;l6JIHoR940COyBANo02r#^;1D*%fb=(aVO%0`(%sgTGzYKBcygIRaH zADpNF%A|B>kBy}4w}UZe0xk{>3`TQq{W5{vjL8N;s6%+AF7}~~MfS<@&dS0p%+GdH9>9cwU|ZfXwN0mGx*7 z>#C%OqQC1QS+u!VhS_Jr#9b`D1U(vB#ymj)?i~(**89|GmNPl&_7RM3?i_rml)FV+ z5)&x6DJs`{>{#h$1J%L-yugu+3UA)9HB>Tdy+E;7u!%NwcPjBnQ&5X$sZfg&Z!a2k zOVBP&5Dyb0{$QZ-+YJ0WE58hKn)K2p#8a)Em7Dnu7Daq1xUc@#bTB}^7lLL)jP&}s z)8^K!GbS9n1KdQvZC|sv=A`YP(C_p;@F3m8edl;1@F-Q;eB7Y+*@3d}>{$@NQo%TY zw`l*<3O8avwkR{Wp2Fvml~(1a`D244jT;CC`v(310~s z)WM5ip7FdOfE7;-*#5n7nMbn^;S?0GY0{X<)^ndv5W&wB=b_t~fz$Mrw@V-a5}1Qd z57yvzj~i`}NQIvRWoOCZpWIKug%A#lr|H`B53V>{GzqwwV%kb%wvv^)diPblU$m5|az>T2rT<0r>@*_AY%<5=HV>^n!o{{k& zLI*boJ{YDQo_Nq<9c-+eK(TJ-4F`p2N46a^oeQjG}@cQ3%(3O_+2Pvx2YMFk= z$uMi~1es~i_OUC8(+HL0x*+AqqG5b>-93b%|8_vfV+~}v=c6W{foVFo6V^Yj6(Htt z1dI#@y&KHc40GSCoJYJHUyePJu`plCADlONJYZMbw;u6o{$&I#n3$-W?!PvmW(##D z@wPU=4`r`Q&-6MT{K2eAHnX1?BbF`4q*R_y+i5%s(M>62Q}grMbf z=e{M%DXI>jX-jmCT&v;yAXtiVU0|UazJ>SEHIoIk8zc$eo;bB`{Lmb+mzCm;vqxya zMlZT*q%YNBtD%il0HL03Y2demNE55Ta&9Y}8ogdmA!@OVS7}Z z5+1#T*pPW0bid2N6`TdtLQxVcV(!k%zIr#U7Q$}CA8?PixR<(3WFd>sE(9@Fs6Twf z(EFtZQD$gb0>(PzFyr$F`89o+(N+1e6MWuz2eh`j1klf-6i9U-fTlKlQM0Ts=HWPO zYYc$XYu0K<40aOOI}dPtD5-DsI!O(B1&`y0ok?6UC%nxPgD4dN;Grw#OhnG1Eii)o zK-ca5H2CNC0r(;b45%3X2r<_7Qii=Q@(D;$m#Ju?FPc3KyYcH@(F8mRh!9E31)X>0 z`;(Ty6vYN{ka;a~?y>yWGhvDX!*zp3w2gVMyqCdk=L6Hy4Q0O$fT{|wQ7Itku{bo% zIkQ=28!VVOm{w0Z?v<9x^nmRG3a$c_W4`LAw9+deL@5RtDH8jhYn*o&1~>=k#QXgD z*U42!33sOhV6^f4g|d5pYFoj|O>yP3dcLN7L(=|T2~TVQkepkpdC9aqSVAelQP`qW z-8Vp0g`gLW1cjlIn!-~?CUb)p*p}9NChbS49IOI>w4+oeG>nTz%@l9^9{pp1q8KK8 z?tJcj3j9?E5TPr-j=LxQDSSk_z(aV|IlewstOnTavA~7-65C(5lN039A#SIFaOy0M zy|uI!4w&HL@w!?||JcYj*i)fmX8fD?)8$Shle~iputPDs>lOCnGNB|#Fb;36^zC)4 zYR&4tfY_X{lo7PN31NnLYYs=%D8V8?2a|-dySy~FI8vd?DWE96yfo8l-q9O`uZIHy zB(tQhgwc6>{a@Jlr4GPYhJ&1dWBW@^5L5GFrY`GyrTr6>};W~hqN^XeZB+wFw z3?ruMN}S%FQV)JkJd+3UFxS|9U!!ZfG&JB^aQ$rkrs29>U=FOiOfW^6qncA_-3#vu z=q2FelQO)C!mQ3UN$3vO+e7;5VO zv+sxuER%D9!cRPI=@fl~fh4BsqVk6l$6Z83D~4|Zu@K+nZF^aVvj)DZ4Fqs+>#W)A zX-k36#KU!Vn04+7xAmXaegf#;PmoB`my_k`^=(7wosTOE|i-s}zX7(<1Au9c?{HrAam6^1xq>L@kBsX20}`YgDf8USlWT2jg{n;>U`{|OYV zTp?BCH^90IwZsP3PQha?pZ|TDNMsu^S+`)b%-qM;#7L@)1Nyz|z0qbCQwM(MhKr|) z+Xn7Eh_Zl`FAlthk6n8@lQ%MXCy7uueh8ag@F1T86Bj?K zq7iB1&*hh~A8hv%*3SqO=|qT}ooYkwri>3>keD9b?||+prv0O3|=*_Ncj@c*bMltvIAaF=x@Ap<+>$sx|dZ3iX)PxSM^I-#yME=+Kpwm z?%Ybewrtm?wO(I;R)2}8Kb*eWB>mdM7i)H@-O{O9b}MoF^%tL)-8{Qjc8S}|o6Fh{ zUtjh-@=!ci(BsXsc5XN60U`quPA)oM=l)A z;d-j*pLUwRwh5_Rd;jOr(Hu?eUjp({{*mm-KQ1&1&mZ?J z&o#MzP_n1&VqB?H+mL*e`DhP6ByHjxvI;1^ux$#F!nh>Iuw5?egw&J5yeSQ{L%clk^^1Ix7p%GK zkII58^n?qS{ra}79X&cBCAo;5$?BVLld}mbmQgyY>QlE&)9vW{8It#y+8mxHSzGhm zAyo1nel`fSXwt^YHiu%MsssA@-)&AD?jGv7zxwy;cvlcwo;@}0>s@pol;?Us`SKRO zi>Ga8!lfdEP;!XeK|QQ_!xH!_&?78qzB|BJvKu~$>JOPQ7hlm|V|-YmK(ZF5L;5EC zIvi6*ee%O@Lg?=G*z$t?z6!GWw+N-?2chG&T1Bmt+3{aRoZpJD0Ub@M^HA#zBRi;7 zK&7d27{8>tR~>j?JmF5Yj>4%SAxjnA*H13y2NL8ojPozb)RL1NjeR8t-P0EAB;tD< zev3v=cqi>s6Ot|;!cwP3Nz}uf4_c5o)4it)qWf1vhZO)A9B_W2`%`gYKwADY zbk!)@c$bi2AMmoD-3u2oFK_-6dMN%J+@W2j{p0k(frtPp(7lDM2|aXVGjw;>Oys+*Wf1QZl}lVira^1V@pOEK!iTqfcWxz*1YE=>Ra z`Kc2TP3#BWpy&U};O_5>hsOX3^PAuLJd2;%3}_b|0FS9$s-yE`EhUM@N#&zC^qBEW zeXmF0IcT8L7+hA-XmglM2jXvh4s%aLTa;WR6qI>Vn4`Tb=Duybf~+JcoIp?{@%Azu zY8{~)HGtK8FPw1Lp71yECLY?)Qhu<$%zmpj@r(&Rh}f8C=i^%a`WW>23}6-Oa~Z{r z-!Dt_jWdDuYQNQOaI!imaji@{q1N4e-3~5hl6DX}4I1T*ceV%Xn6*F<0p>0zMleJXf=nF zjd+|Gpr{e{#FhD?0Jhw^0sX=Gvce3LB=R9~qUM37y`@u+{jBMe?G0#WdOy=@!?PA36=+;_kB%kr}qeN z5E;-2076&fX!zy&K;l^exFoqP82{1j_U);}@&TX^YAYX3EYyQpJOuQbFZm9Ul;D40 zvbYbd;;j6sf0m|~HeIVjgz)WVHF z0^j`ttYUc1l&%`$w^ic0At3%<`ni=IWL<*4;sC2$dxS4;d_y93DAvNh*myll$lyPP z{)f)yi>hXXq{MhtQcNVPQ9wRDBglLn(z93Mk_hP9&YXyH_UqSx>xVAJ`}X-a$fefp zC#+J^$S3(7V;Y<`af6Y!2IM+6YBG>`tryxe5fn(Yo!gh2c@6pt_!`xW<*%n2-U+KY zsmvFDT9Sh_rC8l3tV!K~{n=~Ey+fPNUkG2-=VE^aD`Lx*(?0W7K}StQ(P+@ugtTw-hb6wL}f2gE`@p&ArYI z&sM_3F@-Pa;B9Ko)M@3~LJty;6mIhr_y|%VI;68e6}i-pjlRw!0+xBDABn7P+xjj! zS_H-P2bn1|M`^F6LH8d}LGwYk^X=Jn?&Dr~er8K7o-;E3^j9Ej?mrSS<+U)qJDAI- z4(@3v(+)1yf6RAF>P0j7WF=rLE$PYI*+~Hqqnaw^`7H0;$4c^+4+-aU0s&>LU#_t5ksZ)<(-S5Rg^q%>n-lOybF_1f#E3h}l~8#%d@H!?(;$OI5EQ(4}pef;5pU)#{bq^cI$T z_WLHJ!b!L}d%vp&R1p~7&-EV2l%fqm42pwq%BL6E%Q5MpmDpe$l{JmdJm;z`K6+r7z?0p?dkC*yP+ETXQuP zdbAYa(1eTm3;EEGe!ludN~*-iRYf7_SM(je*aN@pAlGT0I~BN|`8r_; zkgsaP&a{{E5TGN#lD$!~(zEuE??^m_1$6qVyYSH4+0avv5)oGLDZempdM#{F#h1YS z%^INzHmmPS-a`O{a=M#6s?lXEaYH(|0`Rso{rlWn6Uh@4)JKiqDl`Kabx(+;I|X&TJp>gA1`eynA2@9b^@Bf>2hFyIL8Djt zWW5A8U3}6C{XLGu1}y}LLpAuKoHGJbsoTgzRc)J(u%HbmtFB-hza zhjFFnfk&^sKw&N24@Cr1lP#9`zxm-y2%Ln^;*gFc8)x4@+{*re_Cv!2A}M-k+mtdQ zKA2#3RQ+eEYE8;0yAXdr!BXHppbXu6psbuC^9UArs-w_Iz#Y0Zw)!UFTH1h}3JOlX zp#3ih56k)D@Q4C)kgpCFW+f`Fg;_-!^eLRSJCx85(6wkF&)lm^I`nZQ5~?i|Tsd1O zw?1|oIK1*9BpZ0^FG-httbF6U!aC9`=!^aj{{-~b!6RQ@`P$7cL2 z=I_K`JUW_1tVZU7vAQFIEQ>o&U?&6qodw^L-gPjz`@nljf8O}^F~m+-Wt3FicQ*Z* zE<6Se_|F&_6B?vVuf5yo&XkiZ&)mYHySG^OHuh|Ss?Z2RQ$EC#1DeNfz#rY<;9WRS zhb)pD+Hxdp7S^<)gSr}j5QLinhxC0}k#b!k6yVlOa$`E9Y95+2U#9!{+`W)FB3F!duy&=$FK7yad(P&t4dQ8{p|RR-n1h=X@DC~ST>|G_ z5dV_RBace_vgMyBJBF4Bc6TaGIim&*td_?Ll$Z2}XjsfJ0h;AshAaKqQWb zQi>24*N`Gg0CfPyk_AX^G=s<;_Ol?-e#@5|i@Vxg_47jCIMM??II<0pERpJ1uuqbxf8BFJCM$ zXE2C6=EuHVe<<-#T%vipdC-DqXSG+JowjYBcl9l$zh1B1<>YVNn|E#5jFZJjP8Pph zwsK|l*{GKbQs@}Fwy6Ub7I{hW^IwZkD;U|RhZ@iDG zB#v!^)}<98WuJRD`3mlek>GD1SjQpB)iA!beNos1A#Yn23;ia?=tVqie%D7pn|c-@ zC;30?XIj05p=xnlsHKtd(yXUi>M-~)U_v9fY7qD=h@GeynrtdU+FO~! zWRkAfXMYRe7{s&~-;d&oai9nOZBj<%s7d~k@5@B}bPW3O5>RiS4255<+f2k^E$D97 zW(v!U3caC2*ZL`MF>^Css7xd*Y2Ba?q54me zYdrR4ktBer;{PkyoBd}jqaLPz0hl+Gl{qbUWSP9IFv41%s)kPwLbs=6MtX-u)z5~J zzmh|J+1uQHA0sft9MCL|k7j)OHIS_D2)-x54$W!DdJFQ(vDle^Pu=jN&p6 zt`kN9g0c53Tk~Q?%2t@A%s|Gtmt)@b-cudA1)yjKf9**6;PL$6dzq}}Oii))uC|8J^X<3;G7?RD_&%4|Km@HE;+yuPP$67cxlBAY#v; z7ltawp*5YsfT8i}%O*|$b zHekp0ldUt{im#k3BqRk?r%4UJYba~rpS#c@?*eQ$b}?c9pXx9p>j7GN^ke1jJOkp1 z78YA&>z-=ZvJ8fRd=aAW;2r+GQ02vs1RrW#;DmN90HT^IdOJ=l{dG-J4|yQ3)!y+i z-SRqB4O&@`gAcX*JRg7F1G*pYeIEWW{Sse?j6igN-|44lMCJvUU};SU`+S=N{ZfS9 znwZ}v8UE3hratCR^umPjY;fOk?E|ya1~-?1?mz?QXr|;ZY4DVS2PPFE8&3MNidCe< z;PJQGK8#u%ZA`JKuJ$&7UgHbm$Gw#olkIhe3|B+IX98xH1N zZ|iM7-qTe=IBEb0<^By*rFoZOjzqU`*lXGtR}Xf0h>5%jg&@}eiN3q>Nw~~(h?qh^ zaw@HJyRhq#=Hz0awRtRKWH6YrG)Q7)67c$Ajhi53J$%HsR_*a3XBQgr#+`bJyg>K3Kd;-(s=s;0L0Sp&0 z*;L7xHK^AdC~++08W^OO>J2DPkcGWC{T&F6BeC=Uq_oe0z(oAt;a2QcS>U!2qA&VK zFdR4BucG#_1Y&R!m`P7b@+Xh3aQR!ZJgx}r8Qb3y8*}p@R2i<$VW0M)A8b)6C7vOV zb5J=s1)Jl0%2z`<7Xcs2($sY|H-Xwn1TD74+48E7xiH$R0N=7r^xA+CyHtFm_1mp< zfm^W!#jEi7+n|C_FhJ|G-Y1%Rw~Md|3ow&=Wb^k6PB(0Ud+r7QF~O&!`(2kW3}hx4 z3PT>IhK*%4$LsSMBe!uDLSH zZ`eO};Y0C7sUx^7klD1V=sdFJPf4an0zmU(rLa&sT9G(0ZVfbtjPJXXi_aw7elUxN zUe4UnTBXg~za8omq4)Deqn52p83Zo?I; +|SE0a$X#z<_t>Fy}b?j3`23SAbH8 zbv3+UtN`5FZz-tdqw-Np4R!juEeSo5Tk`P}_VH_OOULdthMO`U=Aa`@QHM?lya;e0 zCgDO%6Jm9aKWuKBWnN=i3NzFvV?>2LSq;*;xXVW!CLJXU+n#D9h^N|SbKqquTLE+Usu3mZ6yx; zNPq=QOVldJw}Ra<^KO3#d$SlR>L@^v4S(yNqz-t%P#Ok>u}m>d zGj;D>V@Q92xvQ4Kh+{FDUC`dNQ^N3E>aakoj$T1nhg2-Wj0ywTGOL%uIAH?jX-P_S zjAhDfnD!ism|Fv6IyRp5blWTm9aNfwtnB$2-6ZOO8x(^~N53XfM@E=0D1U-3^$vz-WHL2YrtVphOEnj>;H0=7nS|g=_g0u?2hY z+nbFPt=ri-NdBG2r9k)1wK?|4Gg}4zZX0)yQ7%{Jjm3t;{4C0S7ovIRYavgZ-B1|Tl&Jz(6O{BlBh%<_OpK3o-jZDBO?qtf?W_g^eR1R?r^s$ zL}^n2{tBv0Qu8r`Z79A4u;j@W<&QlBkHig)h5tt83xm+E`&Ap#GczCqNdeyq(^sY5 z*6W2A73c$i6*I$~W;N>Vf~U9x9>v^9H`6yJfQooZk8ujW^w@vxq4)es@b7rr#MD*S zqgyd%%iWcr?CU^lM-8m_`idX{bO04CNVg?@Gax~oc>-MfsYmf^QM646%%I3)V3o0! zIadcKGXwJ-V9P#eJ0yFj!?cBp|4D`Uoo@G2#m3P0R)Xbnx;d54<`QiWuE`hDP5ArL z>EV-Y-z*N2ao}o&i|6@sM5y3kUUoE=-T2~^BJ{r`0JVNLsONOILX1NJ_YK|c^La`> z7o`cD=>%?p?A!~>yOU?Ri2!u?od~h0u6c?#7fxP3MYyT z+*15c)DAVA56Ld3U-<6d8o1ZNp zp^Akca7UmGU6%Xoz9u7RPVxUrYI+`%SYZS!3?Q@o14bHUKWEOC6~1m#&Od^6g0g>* z#LY*%+BBeM!3@bUZx@fg#^dm$-ijGPj92n(RY^TNZ;YwRx|Q zx9?4e`hb^;Fwb4FQw@9Dc@Qt)0K$eon7dBlYcxED2ROejcKq&(WSEr{z+yB{mkT}h zJBX~0gDV7G1@u?{(-ZJ#6cFfN1xx4V9~2L${0=4rpnUWDDvv|#<`VpwCKfvl)nD)+ zWg*lC6}Uy)ln`4D%#bLdtSmM{yVo0m_L`XhivKs`fH(?Oa+J3`YA&Zsj*Ea%XyZBf z^3|<$JLoI&fB{VS&~LObQ*=cF;_xVeh?(vbB3RdJ3gH2c0J%tAJ|_!PPvWxp_1`BB z2PUn}d*EuF>osK>bEIS^BMip@dSO)UcJZ);8J>AkxX^8%?y=D@RVV}#w7;qCi7%!E z?1$;r;5mk4yVDo!w?Cq=TfD8@nqYriF$f*b5K#g)CJ}8E9up%rZ%4r9yih?7#vpR zs8qG3`-yX5+(}*|CkQF^9DJkG)1+;baQqf=7Pg}>ZjhDj^()VvI5bn9!#fy%<_@Kf{{sA{q~9i& zS&Vq3Pwc-)ae+2>Dzw|XT4{oJ|$kA5`=SjBwMn+^Fl(= z)O-EBH)q;{Uqr+b`=ZOie^glbKCxMk)(lBX6#OeEH@#Yl^n;G)&|{S>?fO}WhtLZ>Cj2#0VJ->tBT$1>mw~m zYt-m|zC!e)Zz=q>X_I3mhaWFZE78$$MFM%$uyv zOqh$}!M}f4*}wnYZ7+B*k`{JQCMjMO6aJhlu|IP*hyJ)Nb7&Ts12bu!0)FowC+@1Y zb;La5qkc-_mrmPDyao@rdfQ^$cpDF2U`1;&kR(@BTm@~l{QA3=!n#Wp|FiB7xL@n1 zbA*$iA4W-5LoCrDGgU}CSpZzE)S&$Tn7Z;nsJi$6ZIw#eWGUt?NhPU>%5vWpR6?Z) zF_lonBq8g4OQk|nNh-@!vLz(hvfLtDVv?pAz6 zxfL8t9Iv-Uuz8=`-m%3G{%9K?BH)eRQ0voq^2|pI$eoWtI3Ej?i_sp%`YP43I=y*q zZ+0%+5a2(VB*M8Vjj5+5cW`gR+>oEdH|B?x31v^#Vq@dsw+=J>><-Kf3%QEQT@k@ z?-fG;I?by*3QYo(e{OD$F_4w=i-9>gA;}TaDzgut4Do_#6bH1k z8-`*z14Y*$yA=Z9uJfI6@U%mp<)*~Kd&9-#OPy8#kZ!Le4ak5X`F?}PBZ zY%!ga`o73vP(XW|mvOGBEi8Xzd?sR|$W_Me>=Y!P@&^-DFfpcpKK6shmg{29ORnci zMtPb39X6zZFvrb*y$J90(Or{yp*WaixDyas9ew~dqv@yk>=BAtQJk4PQVxC4>~q9USgghx)*$dy3b)Ymip=h1d3(P$9*oUZSd7F=)cCI4J+Si4b^ZBlqj6*X5%>zo1#u2R9sP?{t+ro&UigL0h-!tQ%!N~%t?VVJWA5TCZhfyX!p%h(jqyuO%b8u^pfSBL5j z*v$dr9-d_MT}2);1RU-&zIK{?vnv$pE)_VaV(o4(o;B*M8Mgv+zZmO$>fmCy;*mZM zd3{YyxfpG#j@APR7%=Qnp}4)2n^a32bjV)jnqk;xIJ1gH0o8JLS{Sj+a2Don6FK{D z^Qaog<=mb^N478Pf0NjW`RCxy_Z^mcxw!2{d{YIeONQt@;&`Eaj`2>wVf;z6{-g49 zs{XSaOy-f?n4jGgod!D{)_gA=5? z&&o_hH>2atf^@r74dL89&~k|Xcmh|tNJX;tcI5O)%O(~cpP%P>W^{ahW&#huz~6Jz zyUKla+22-7GK;taieP@W7J3(tvtSBN03%)#LG#?W1?JZHW)6q4c(=tYBc)x42e@b( zW?L9ydREB@@rt+)elBSD&XtW*!8I5Gp$47x{a#4*hYqhCgy8n96Rh8|Nzef`fT6U0gY8rIfl8eU%99{-_)2tL(H+~e`Yr|CAv9@^ zh-NrCMSv+Zopr3uYvf~QK>gGP(_iGvwooPymT1P$q7t2Mb4RnKkXV;z6Xf^dmnOx7 z6o~q9=W+7s$lZl3hiZg#EO2UFw%DW6tLxAlxPU6?;_v3aoq3VfEDeQq4N%{v;j!!+eU#INyrh`&+w_{jHmzx8UJ{nH^bI{{r94szaS zKNp4waiEr1Slmn@&%aqdAq62>@ebDnTVb-h025RuaAHAx%#2 zp${)O7B!{Va^U9{lniHQ4bxj-%#s&GrZc;*&Z4jyMIf)}1IFBwj$flvoz5_bI_u z=6!6&^%2>-4O*IWC-;Tl_69@4zxJktg$1Lm>|--VI~{yLsvcqksnv$_d!LwH=)h!e z!g|7ZY*t@rw(77dYC#g9AEOy6C*YothbqdH!kO#(B!8 zz-Ch=qIb=`4o#j4zMC?7?RI7S@d(5VgC;ii##enuZYw&?p#pN!&vzfbOh!sD9_Y1A zFP<6XvlO;%2@pGl}zyi6245I^#9FM`a{?lLQZ=_oNi7-^k) z)F^31z$E2NJH9+_u`uUn4w`Fcfv9@!8(EUeoQ^;dFV7$Y8GXkE9z%AQ#zyCebpN!D3SwK$DjTjIFC}4!7hQ zxIiW70ZK5W8uBb-HH25zcVHtCn?`@6xuDwE zRltn(g39|*v-l7TXdry?-)TZO8#_4{%2E;3X&aA6H9Ywy12+r4@Iu_sO-bpg9zT#U z;gxd0vohJ(^c5~6ZZNW92I0THlvIDQuY^R6&%tSON_a2N!g(ih9!JPN*4#C?;O7T~ zHOy8&lKp46%d;~RCI2`$$07GJa*?n1CJFve$D{UZ$Im@t-}@*`9o2b!Puto1efW)Q zmk9}g^MTiv{MfJZ2&yu(T_`*oF1{qWo;^MiIx+IG8)zhB8ExqIACKvX3^3fNU?Z37 zqSZY$xx`Sfxva5**@&e z-v(j8K>}}e(XDFMP1mpr2)Y>1&EAJO|Ja*=G@u1nXS559OMeIsg0rW&&5(ID4ou}EHcX!=IS=8o%D^@F~xq76Tlc*kb5YbCBaJg2YAfvyFrMl`%qqstRw^ZKiK|Dm2E=?1^7)bd3#buqA(&Ia z!2NCO{W^KRXp&<)7@fKQX2+())rhEeX)=q#unZaNeN>C6R{}Y7wT)vgZj72CED&+~ ztVz;7$YpMY2y+WGIX>W#v|wLHD49^8iS^<=58n8}V^GAFHvHt*C&QzHL=@vvuMLPt z_2sQp@B8D0C_O4o4jr}ExK!B%>kd?KN5)7+gB0V(&PipUMh-BwMlZEB%Tr(!!2@$5 ztUlwLS!yt3AZE8QA~YOKS}G!zPNEBLSj>t(l3r0|oaqD;UZRv6H=5k&6H~BoQh9N# zn3dzkQ0@)Sg$HD)py>RYx^4Jhrg7$z?^O1ipZG^Yy^WzcP+0ToWF#~i3ui;X?@<*#CGPJt zAngR|aQia={0P6OF>*ISJG&l)@3~gl7_uMD@@bsn8*5auqB`+=2%*BH$ahfz9(z2z zBGI$|0WI6=9;Z(>$wEOEg9bs4-q`({njX;8g>Mpym^vRI^3P9&;)c>VE6>;CT1;p( zG&}6h0M#$!SKrGelI~9-&d@-0LF2W%zSVD3jEl&@|HS2ax3*L_!$p^YDY`ONi`d2? zIi!usz_ST+Mt&#A!8Roeq$WG`dWS{5?#}f))5Z|S@%;hBp1J1xNL#%ZrY4zoe5N7K zhe5`lhYUK=CtwwP7L?sAf>;MmtCAVojqKEcsb+oS7U4?8+bi}qE`ixQ3uG# zN2;bl_Ira#vY#Gn2a5ar;qDiJhE>Uf-Py)^P!T;&^Jlye-?rjt{whRIB#37%LQ6|i z(z-WI`bZ2MVk|JW|KR=`w)0t!28xMtuqr|JT<)eK;3D0Z|$L^BVq~b zHu@rE|bQ^QvO6k`d$>t>V$Rv=i!H-U@fW$;a< znVVsGW+70P_ngnypE!yf>Q8%2<64f6k9d_12D}Z>gBgZ6sPveN2AGr_c?S1|ji2*A zTXvDSrnmqFH+0ZyANH9-qpTSpPlwZU>5`YITCgZbDdO@(BDL~gopqzy4@|e==V1|-UVEQ1K`wKA8070A zMR|5L(-@-7cx~{AyWXXSIdB0{AQIwF&ESThp&Z?S;XGhaybd{y5YS`I6^}C7 zmvSz?PPpY|Lr<^(hefa_}8hOnD!(@}?LZp<#fyx@>UcwWJNm zn7|Wn3o9eusd@z%kNW_dmR=VosE&g=s{&fwZ?}$>JXaD!3PdgC+Dj%pS~B#{s6IqX zP&Va8RDA3I1pO*mpUon-it);FcFuzG)dl`*Z>myY&I$pH^uXs4gw^rQW^xN6VH=Ei z2(G|1CLV9=s*->k0|#;jFXPE2_ms^J{b(B>l%*C31l7G}u5k)W_SyZP9k=KB zqF|Z|va5?lJzif&UF%T;S;d~eWG>xGCF@%NOk$M5J*#i}^^V+K3|T#%>QrZVmN>>X z!?alOJPTvL-l`JcJ0EE_t5qoM`aFQq=F%Vz65w#R9k*pD>SC`wQd%XDr525bC%D{& z3)}>U0o4_nUXVK#AcKxJf!0jeC*vmfAIp%ocmO+zD;a%c?wv3M$LYO6pFOMBy~=nH=8&ka)>I6ZAYb8_$iFV!YJ?a90+S#p!c&d14NQh}sru~|i717C-P)!vgj zC}iRSax8^Z;o_eJ!rtiJa;nm);<;kZbdL>VO(I5&UekV z>OJ5%FX`1cYFz($@yXtDnA+kXS~;Y9gvu==R7@*+__OC{K>i$>+fyN|@z$J&`gY*NNk4$Jk z7@iiF%ZmB^=5rUqT^i^bXn3B>ZrUtF&u~yhF8BEupqy*503jK;na>qGIGl&LpU!VZI=!1P1KIP$LTH| zWP;;<6B+FIVdiBK{UZ-U|DA=Mzkt#H4n8&mL^$cw{xe1aGL8chnA`40bL-eiaBSHa zTqkz^v}Gvw4YHHGAiSaW*Na(_k5Ohq&F6MCip=_0x)po-4u42n<*OwPWRYXb*=!Erpsp z2wZ7}-qyL!pZ|kmnhvhu*y~0nSzVAp5hnp|P;@>LOR!#XkZS-%yyy-U-;FE(U+csG zXw}Zn;%Nq781NbxDk%T;5?igqwf^D)=n<0o;Dd>K&mML+R)X7D(FYEGo|W3fM*P5ny+6^p6>b|6lnm^tZ9C-s@=`M&;VU9 zPw2r6l+|<>j^BDUQP*UFGrzg7pP3k8Xz)-U1j;G;op)|3YQp)4dD5gU`!_q+abOi( z00UmrS0^!$d^>`G0t5FH?5I)O)|De-b+DgKYaYn_0|t5Q+kegyiHbl?2r$ns=gTj+ z&Hx8%C+_1S19-wn`_SYTxiD)Ou8`w*Jh6||b7iIuK9OMZKd1VKv-osx)X`-Rs1FJP zR?5Cr%)j(%IWVKZ`^5#soHk;lmYRdfZimo7tZietxE!EA^hDPmfs)xeGM;m)@^QHK zluYgUiM}4AMX!<=aC>1V_+gQjwW{O^bRosPAiOp2&YhLp4$KFphE>iBM8C2)AbOKleq(qBT$Q3l(4X zMDEC%2BRADV)A`c_esB$jv+W2A7;bXPcEBpaNrUhi4+{BEv`4jqAlnCy~s)vz=Yr^D1tbrUN)~P7eV#i zl#g6*#ZQ#4g5W7wNZc$OSUyAkFy^VY5Uzl`6_f8o$*=+Xp6-5TkNhun{Msiu>^JvW@e#BD_@KkNq2M6vypvK$ zE?ywz{qA$R?!jJU-{2Fh_(sJ&ZrGul=-*}skw327D>R`++&~_Z1ul{_xxhM4U~MRh z#^pg=82LzkapEl0L|_BAFFW1Zx#|)RbpgR03aY<~%gdV4Af1LC-?dnrVm@Mvp`C=E zXX48|iNzY7KlY-CX&Q(&8bVUIxy=xI$G_F;lXIr#H9l3u19bsP1ASu?fiFeDwuU|U z`jcGH?R>4N7Rs_9@;Z57MTp?lI;hbGfQ0n7HaTWo&-sV}rXcg}{_<6e2!jA>2^__- zA6YjR9EEb20rC+a=75Oxj;#wO{RHd!0Oz6Wd7r?ufhgCbm2-`~Y3>*QSjr;DNdx|b zx|OjocO?7_%>Tf#w-v*bghjO3f|8olNDMH%EGOXJH8r-=zLVd`2MRaE@H#ov~vD?CP;W!9!loC`wix zo}nP-wJ~`t^o*JykRsl36`e~TJr)k^SkDz_$!9FV6J6g3It;&Dw%V@|BNn3)76w57 zf!V|M?cNm!kbdO>DHc|lr?JFP^#TK*&dC?`Q&x$YD&qP|kpbT>r^(;$^%FyYKrP0H zBbC#tQw(8DI{uwjh2n~{12qh!hvS!q^;)=wB>t@2*#(D%g+aJ8?`8ITVIfiq$~$fH z<|Xfzx?~BcbRrj+cV0c|M(Ff8CW?34PBN_;|{XxMF!-DKS(or zmlOZ&Xa_gwPh3Pug4@xyyE^S0sS<<#wHsuEMqdi0+Wrcr-t%C{kph?dVfXw zR2>}@-O|ptEw@fFd~yYh&PVzlVRCpS;n$9pet33ZgUMCBGe31iSB8h%NmtEz8slqU zx5hQ?j%YSrcO_?KneUu^Yt(j`FWuD_`^b=+pWMUu>+~VVtn0Vek6PoGq6W2$No_6C zC#t8KH*4LV*D+Q2wshQdLAZ44=S9Ih#TUPVyRj<`*t(3g6^l(PXSNASo-ZtWphy1e zKt^b)Dvf>qpIZOQYkM=)#^}%cuxr|OVbqLWZNkkHzqK?S-_S@d<(J(3cA^Pe^Y{Nm z4)NmLKN@@1v7We)50|d`_T?9;Ff6>>$X&}=qx?AwKQQ<1xwAtQ!(K*9aK}>b)w-7l zgf@q7oHmFir5wnaR*zKst6R^YpEKdvTYPE?5_Ow;)u5Mg?PzDVZe--VV!`CA+Z@Pq z+jpq=H6KvxqlD~QZ7CxtG2wN7xPK;mw=8;9{OO4C$@0-MqVMKxM8))UT$ruRYIU9? z!=dk6c~3c5xFmr^n2QKk2r`zSXCd(%ah*4o2d&8lCZaarFiutvH(kkk;Q#quzy!4t z>9`I<)~2>mM$BjL-=8~KBOEm=xn^?ZcFBL2#V2^TB^^KYaoGd@CJFdotu-w2Wq8(^ z@3-h^!{p;JLF4=4Kvnm!4b$O{NooxTQ|VM$uw(OL_%iDtsAT;38o}y$23MGyj<oOIdAezOyD z1Yc8yp zZH2GWpN%m8+!_$EXYyvvpOm5N4PkqDY*9{f?{&1_oOVCmBr|6bp8Tm)_8rcfBHqOfQC%YV^tO{>D9 z2l*dfXZRrommK8T$JdWNnHF00jy_!G6+Nc%yak(sr?yYIy;{`F93o z!nRI5!!P9a6c^ai)&GNrNii7m&^q|0idH`x$tCO{i{v}merapFeP&W?sQY^v`ZLsL z9Va4Y!rj`|2oo%9T}K8P8keEMZLeZAoGi9uTC||uraT!j?xba$a}r)qm6~LS*gVjK_4&Y#LgJER|R&$b0SGHwUwpK)-g$fo$?ikAC%iE&4Y{AQn@H3qPY5AVT$f zDLhe4ikau*`B2C>fQN={y)vzDJQX1YOKu8XwCjI=>|1){@4x*dSxd)K;zpRIJ9D(L zwB1O4Gpg|YSKamtvvhObPudo7w*fPt$8kS(UxU0{?LhvHc&peeIS8Y`&o(}IuvSR-;7RNLDN@kHaeUa5Gny`rQTpD@9Wzp0>J|AK7KhJ1q zX*Y_)Rjq1-`y4K8I^C5q16mOa7@JMKc!#3MQz$LQE^sMiJ$Z*I?TzT-EEGh`)oqJ# z?Qyc%KN*&1Qj*EP+}!|xWM7deXrg*^6N39<@>VRX$l;XDmvNC_%PxJVn413b4+5@A1Nww>^V-WWadQfa< zFGD)iI47WjITI=zx!R31cjaC@^+pn@dE5E#E??2NylHy+3LzQUn}NK z`nB=9lqs089`ctp!(eV zZ+G+&;rC_fvjFGzt&^#4z#(>c!BEH_T&4`dqW0y0ypFCbDIoO z?0k2LoCq{XtzjYOy5;sqi6IEeiGO`b-kBy(`kPj$BE;TiJlWT4C1)sf5Go-vW`xN& zs^TS4a{PY~3TZXM)a~w@+m9Nlt48 zy@JRdKEE6OnuG)CBXMb+A2*WqZqml_&|eI)E#08br2YvCZnky{a%uxN>VH_UYw}fG zXIR+WV(l5zm;@aI_1__n{1Yw;>wVqjAljsu@Zv7ALp#khp=&2Sx&`cyA<45zb9dyR zuGMfTzx%8!l2WI(yL=DcST|6g`}j5r+Q>D-LMztF(IVNo$q>1T8x5Gv3(0L_7(;xv z@ZI~=(Z7Bx5CCM`%_~Enbr6Sjdl^|bdYlK>Pf0r5i*)9t^$hfIkeTBF4cMbbzf2=> zU8tGF)doy==e0#<;ZH6T!~8sr7o?Q~nfr!lJ@__bA#}2wc!7<_fZoLzCXA+V#xF+K z7fy$({&J+21_>N??uNiCnh|eBp+=K4083OBEHxkQy#+BT0EAuYFSq%88e9dD5Ce|! zRG4K^wZ~F_2ukEbK!I;T(q)(EDrj6d0II&y&y8IP!{{05|BL35-XWVgn4S3^{Uh*= zuRm#S*Wf*!bb~}t16C_<@HY454Y(030o?uE*H_0`5c(flHOPz15me?ZNGh0GArS23 zV1M2jmK_3H47io&vpJE(Ft{ig+g^n!boTrIFsYtgRbWqEB@dBdHNx%IwWm5lCB-MSc#IT^lJ!_C{+R^M zdG2kC`;lEY0YMMc#)w$Zpga_isn_IZJ8j~T<(?&M+{|;#^tSGL5cV8iZ6+gNtr~0G z=9K)o$ZtA~gp?Kv({MV0r$Xs7nS@SPf=Xn$M9|!D zBt?d|^PB{HW+p$@c}f3h`mY+1U>|PgJ+(12#W4@j-+ZHZd=~t%`hUh~g1&E>*G}@F ze$#djc^rFN(cs}X6_{)V)R*n7-Cy67HXEW$&K~m52443xUY+R-C`;vMKnH3;YWfw*<>-%AhdBt44TJ&Z7L3ov+%TR@VM6zi7t2jNq+K zZDAV{XH7~47kP&NsXAa8{?iYlNecKc{@HZ-wDkzse4ml;MezEK$~qH~703$l8PABs z+R7c%QWT@1@vuQwp!c2~+Nkw%29)~s@o3=P#;k8+94KNIz_r~QzqJp~L1D1S9y}-2 z&&^k_1|cHr<}b`uuia~%Nm}<6xB~0pFKWivdF?ob>Z*4XsotZ;e=}47Df@U0IyQxY zUeLqPb`8-9^}6?~dhTjVjrP+N&JKCRjxhaK1$_P?b|!Z+2WIc;WqeHRmduyAjvS&P z@SDFvID+mvJt$JuZeg2w%+9mbv)Blq*!!FD*>M_kezgWew8LGW;m6N=m;7#3Sq?)9 zVpckCr{yyA@bQ$(RZbA3?MOIz`1d8XmdV81Lw+pmw`@zPePPmdL#+Kw4o?o4LrJLD2+u|~Y|DSD zhys|*m=R-^D5c?&j?;WZfA0<9?e|88t3+tRg+ZLjtX&(#9Bt`q=(}ei&D@-h|5&zN zkJ&MP!=h9 zhIA*m0b}W<&k^tSpY8YKVYl+t@dAnm0P-J|%1(Y-Rnvne19)5olr8YOEn7BDN5&+# zha8fB>P$DWoKkl~F!10ILZ4pAa-D%Zs$x3+F&V$_iz}lI>VT{)_07>z0~Ahio{i8A zSaqIkt8=rE_RzXHG+Cqck+WMR(Xu*w$oQRDR@|sN^4YAgx8wnb>SE7Nt{xEQI3S*C z`l2pvQUDQ*MQ^}viaIq*=0ImeVCnen`A%V3Nw1WpU`fTVMworLx^RuFxhCSB$U=dj zk2!cF_nSD{hjF~XB$Zb}FIPhRB-f?mE6CApZ@pGSMs+rWUW`(M$# zdJ+FB2iZ%$uMs|~d8dkh9a{j+m9ncD?>PQv=+WZ;AU_DqjocT(+U#^l8O0b_^awNR zV`N08ts9E2@wMsrg#{kBw71FzLvAx_JzVa1T)xyha+j?810K z^^>eX!(kWX+3}stctc9JOjpxFM1AWX^5JF1Meio%6eYwuc@QM;$gzC)Y8HYM3IM=T z>E<+IKo2c@7vwLsElQ`2ygU$x^$+pvnM+dmS82!}h=UL)`?;J>fF+9INCh`CyQ)-E znQbaQ^{NS*lAHCTm#?+}ZErKsue4Y8=h81DDnv|pB^4zldGAxS-@|wv-;j=v<&Ny* zUs{F`fDRZMmXLQ�!Ewu9c3jQXIM+!F?l%XkA$&uy2iQp+~DD?Z+;Aktcc{>&P#j z4h4Wd|H)Q}~;6Xu@V|v}0UXAWX7|^V4}i zL$MM?ua~g~A2IO8#1OG%z@fZNCkRU==)VI%0pi4csvSE=0MmJLYZdFlsmlyr*udlh z!iwf>z2OMlTWfXju2StX>U>q352rWH&kioOZQ6AtcIDNDyXIuOFB<=w_SHMybEGpD z?pi9d%PwfMyjldar86SVo}E5o&b!R9V5j4bQMnz%A1v6zf?=Km*LXGGzre<8&8J@h zMFN}Tg=M#|ldHQ0A_G=l-%{2G!e{uVVL_%%k+)*+9QLePI>%SulNBQYJ=t#YnbqIa zk9{wcSoN){pRtR}(-9QQ@8B?64#H>qxzr5jUPahY`dF^BEQ^E9HY?QflX_xBmDahxBSmT5)Y$tdUTx-35Nf3sT|KP11Y zuo9Oqtd4esS{ISl3!jbIk89SO9Za^xtYk1-geVzXhU$C1{0P;OE?FtSe&auigbH?Rg#DN`Rt+{d(cz zd|HeOq(Jg8PqDL}I#*@*-Q=R=Dj-j`-GV<%m1{_zYORbDCTeTbxS|{SD6W4QDGAcT zmi+g!4n|Xq6b01;J+6X=;|=&GdDwxRAFutPtwh-mk??ecdG+PW1b>lJz-3KrEvP?p zPS4gFTQ3E)K-d!D<>fldmq~iNtVNV6%^`owClp#Zk<;e+Wc?*R73kIWGHi1%>m*!( zx{oWVHCoMDsVEKqD={9FukxKPaU1U*Yr;DIs!YlnezmNR{!-JI{%F541%zkXdU*kn zuR8q}uVg}X(}5{^dOlAN7lAUZKLkwC`QOj~TfSjZ|56nBA#uL*OXuyl26bUu0c-on zyR~%PkW{EwtbcnM@{Rg&&aOzAv8VG3sg21_G<|7=pJrXzRXYJ{w1XS8AF|ePx3o-MNmvGQ2)n$QOVe!<%YYkYRNr`Uy*9d(;7yL~p~Tq6kLR&6Zxvergh@c{K88X>yVg>&Tzq|6v7 zW>?~pKMP7updSlqz&M49UQNHQAi_3wbJ%km59}92Z(9b{HwTn{1}avEeF;E(GJZB< zTxj*)rP2mHV(so_80}uun2|t7zh(`z>%zmuUoiVEOF71LuJK2ll8*06Hg=KvMhS%C zJKfErDXogrt4J?|=7ED4A;aSO(qv3S0*QGBNJ4#Ae<1rWZqOkokcRxHl^AZxI&1#O zFm=+57$B4`m|g8J$BvqfHnI{dAt85wnI?4|Jv-?@Zu73_k8ho_hD8qKo8*O$>xPQw zL)@Z({zs=|;-ih)&vre9%*6EaNadejKV>cIMf77v0s-^S6LWW0R>f6ltfC-*6Zo2z zj%hnycz~5f+#sn$i{e<$s;*4U%V32`7*lM0?=l<~mR8AHMFZ<$@Ue*Us!Bs%N2ey5i`nDK3 zM8UYz>ocW3!h5YZK+i%s$ijJ@-TUd1n~^eM&qwH1A70c`K0>cbE&{r#(rn5*s+lu) z?j)jn0=VVcxpin%L-vx(TWR&Q-!hqu<46th1KfbyNrZYCIVywr%K*saS7{OVZ2*X+2M#NGN4~&jMMrcMc8r`LDlkZ0qv-nla)#3l;ZDl zAl16ElaBUUT0WhGJ`$AGn_p(qoU+d^hjL~u5{}NWcS>o zi5vaKmSY!6dP;q#Xa!9bE6c?K_275YF+0P;J%0m5abTFS;!S5gH>oUwDnUtL;aMT+ zs{P##NY|B@@ma!sDU!V(eOu;@^J?9_jwA0skMuyzpx*8`mVK+%9%D>JMD)%fQ|#Pze##!7jkN3!&@!vv=MU}qx)N^J z?8yl8^8LbTqDkn-?)J01lFRP?Zo?_pC-scbZlw;lR?GKgEbME1 zMW`rQ@G+`H%b*3D3#nfZiZ#dkE?FemmOPo%GHPgl<~fQ>({N5Qlue2rS7bCRa&Q0I z%TQ-=w*t)LOZbuzn+4F(5wih;{oXGYEQRcqn30Yz4v4sDA<&Y5y0kSx;9A*wWJt&j zfpYc$!Ha)S+^ysx6hITp((zlXj77B4P|(1-*>AkTWQ%g+7J(Wv{&Wz$kY^ryCrMAK zHRwO|7Blvp9Z*KJyMfw`pvauLai2ZHl-#&;QYBf}`csJR54OyrG^EV51Kg4|#<}Ym0Ql~>q?y*inOahT-`l93U;T8nr|}?6 zVKU|UotN5K`})#nAUhrbRi{@%XK~^!loR-DI^$RVx2lu1uihC)MhW(fjw}4g7S3_g zjTO*}NpTSVhPbOZQ|;mQJy!A4Xc_r5?`$c=vA(N#Jx1nJ?lGoy&Pb1#( zsbDYaDq&E0hxJTFo9X{!a59DUUL>iC%6x{lKv$geLitvKj|VnS^Y z6T0DE_j(V?LD=;LkDL_-cjrfuGg0!8U&CR|UQwZPo)%ElIhkN`qJ=`mGkg=~d)nYr z77+5-V^_*cd!_mh6{P zjNOneY(uVht`_x<523k?F_}!WL45v2mAZ{UNI?GEJxVX+NmfbFZ6@9vB!r^<>`HQo zAj?Sx$povtj@+=}<$jw9D_|Q4g!ca8%Pt!!0_0DW5F@Feg6M~DZd{`bUBP(CbH?}egTuDuGS(@m&lo5f?7n`qtXuF5Wb~7XR9ZZTa=`0gk?+On z&|>U?<-44#{drym5eg^&d@>aWQ5cMbYYbH7>lDvi0`{4%!=Kr#3n2leRm)?L4GFA5f5FY$y_@ zzW|034C7u3&ATMNGSLA#BMj8y-p60Nt?Q?a;v?$6Tn%ydv2yUaa7_*oC9ZP2^L2`) zyp3IT#+|aC>MJ=5w}85A-^vd%x^p&jcCF36y6}Sd%Aa!cvNg7ZeY3Mn)62-%T#?@y zA(+;wCD%Ey4ZEZ*mr~M|+1WYT)-xa+5`NmSFGeuK?C%|rq8wg*_+0Fhx$nJO{Cnkg zgsIRVTj0w7&wyJQd*o-^-w%4xTYo=Ecen&gf?0YgGor&8H9W7z|^!f=x?!Gmi<&|}d?Snog? zs?E{<($*)dh$s8rJGL{xDPmE%j=s(RWU8Y#gPS<44#HCLbg{O_hF-=ivA(~I3eRE_ zPPwKf>te-2qb3>30@vy)`}#GP%F+&ZjZV_G8XT={Up4$^%a-oHgxtr~XOPO^NOaR1 zuW76U(UYtqO^dOoXZ8vT_*nNz$SYQ{FWD=2EOw>*cQRd9$NDd`PWSO>hH`4935)i^ zN4-tLuAk&hx|ziKKLdYd#s+7U=d09jN+N@eUqK0l@Pv&+gJM>N%K#H*dYp~lbIYcc z2n0`Gbje8%-a3Rf-5i9E_|f&`meajXpv{0Y+ht#?*A9H`L{j2_IeNLEGrupE2Sfhj zb2Lz*B+jY2p!iYL zF{lS7*(~fAqeU&N5}mFAg2RR5y*v^_yfPDCi|9@idRrcDd5(r6THj{ z*Eanr($t*E2&>uxoD_lPf)(^KM4Bz_PW#qQH-K7n2yl5+(!z_s)DW#9#St>Uq**9P z3D>0^A+r=ZNvI zcj{Gz`J93;!pnb+2nW~AHa0an&6Te^2ziH9^fEpuIQh4EPdzH^WkmNjZg(bXmIeCc z{#8i(Y{B5slR5oqY0x4W?|T_>BDtjDCGTWItu42b@XnNcY3$ZX=NmlWIo&KEizXMW;n@Y6rAZa#beY5jC>!(bv0B zB}j#047VLL+Gt=siqwhuqlL_7XBurT6M-AXA2niO8MwuzH4o9N63!%l+jG}lBXkB@ zVLgW@vLG~Pn~qhzG!ocWXYxpPgF@~R8q$;cY?5?g*KEyvE03UNIWU!9pDT#kZ^=kr zvA`3TxIRG&e8Q@UE=vn|c#`X!XGbX1#aeCbViM zz~i&J^!xH`1T{FapJyG6E z;av9m7F4|CZud8-X~~h z_`2*s8)gHg>x}Yi!7d@)#6R>h{>Xzvw>$$3stF08)+{_7z!l2tC5vN0=!yuAkN!`wtu~ny_lDb;P)A?rwkK z@vSqVff)ng^#1sKPLYa4Jql>p?D){5cQN`k25^iio&7Zsc zOwCL@UrJU=wKfvcfKSWB>#~E+-{9#iG?=&!5I@gT>>asrHE9My4YI74vD`p!M^y4& z#19o`QfvM8^*{Pz?jp@(xbiw{%MNAZpJLIPR~j+4(Sko74tnTS@OjPnqOVoV#Ms#< z5&DM#8yP7)Lmw%F5UB#N>R`dc=kNVVCBNAdp#ljs%snMlBF^*~G!Z3d@}-h&gp7ph zGGv!T6Hj~rV}bj(280Jh;bJ@T_!*sz9W{w3Hh|D$;+wfrq$7(|uWaqxIr-6SNn5^U zZi9{vmwp2JHSif-kE3#9k8Ex-ijj9wspM*Eyj-#Pkek3BF@SY~FL^Of=3afS2DAyX zgqtJQH>c#Q!cq0nwqv{PJq z;N6$O$!H0j_SJv1byuD^JK_fNV&$8JuK6tsAA9*qLCwbXi!tJf@k7U!9LP~Tq!DZX zXmo3oHN0t@1%ENz7N}TH)9Qd1^CoXcrrvJES{8-uew-P*3L6M{7V_7&BeMjm@h7uv z>g7XFKo-^s`T!-Zi=ua-xQzhc15Ww6O@U@=h{%F53fKaAXc?e?wdPX zbUe*Um||bY2(x}(oToJ#p$xG&6Q7&A>Ym@a5$b^IL=1@sG-9?4{Y#|klpfm_vr_6> zx)RuG&yebEY{OqPmr5ryej1jbhXBaDu!u#~wyKa(jykqN5`#DawB`?!wKA ziBn-ku6d242(wTjo@dVON1>$!EpLw&A(}O>z;Y_Yr zk@v*!OZDin$v{4(sF&fmao!T+b5{Q${le_{CUPw6Ul;BFL2PRX!ov#W0$2H^AXH#Q z4j88#S-+lL@lqE_6 zH56?lUv*@)8&T@wAUoLr6*H2R4tKMU-G)JNY?DwlE2kiP@E`;ajkXcy&-TFlIOk^zWzg-ramvDKrHhI^ zrZYKY@oLRLbtPhXUba9rdDtRukuGuynZWq3VD);Rv;db>*ZoNU!v-GLFlR|b*w*Ke zN36SzSjmifUA9jIa;2<*0p{C3C#+htgVFD6ZWcyiB}w95zl`KYVy%XaPTP{WeP_V-IG4w1ySlh)Hmqls7EdkG3YBiFeO(GPp#RnZoFW z(>`JxCmv$p<^1hB+#3!)#`3QFXMfZ*v?%yC$JvwljjsMm2cZ#RZEP8nQza47ftfZi ziTz(D?jG^=(5sK(5YNC#`~laKnlr?6n=V75zz1Z0B*kf_sjyMP^iMIX>sy_bb#^q8 zJ2*ZrUp~Jf^6;DW5~Jc&AJSq}97Ha`nH;X^@toxlx^`y^+CtM6^PG*LC4f(m%=5nZ zwWlRvDRhnjs>1j0$=`-jez%Z@|I@}$RoNP&drTF+NCyFbaVqa#n2y1F==Jvk;7ZIO z{@niQ3xSJLlt=bmC=kw)_J`&TK09pBuj%>sz5jm1GJRE{oNoE^;*OFP@8GXV6=P+i z2OI{r=s;IV0fw>0WWn_su7(68>3tKA;^T*LUk$HB5(Zz=2y4|`FggN0=nCs4S{z|c z*KTpfT(=mwfPItjW=h1eO*FwtDB%ErVr1=`mmgZ-nq)*HWY+P6m~O!o%JqK zly3_>3^h%rjpCZon!xwfH-f1X2B3{YVaVK**abDMD+n*BephXd)ghz+7cGHOhfl2T zU78UJvKVE(l=^Uc_oL>}KuI9@Hx?J`;%+a2N@NeBUut;Jo7>bV1gBW|05gKCRp|6} zJ`xr6JR5(g*?Rok9(nZ4Rf_yAmR>1IXKEtY#Ww=j)zPB7rKyJM?EX_a_%}=fL41fT zX z+@e&-r&K~r%S9zgViKAnRAee!*2$7dV(c@_^8C(}_j`WdKRupj-pe`Xb(Z%z=Y0op zL^Q}+$?@D%EanQdS|x9R>-N<*SZ}zx7{zIYe7)WNix<219m6ZLgnonD9fIm~>SD2I zO=S?e{RO97>PaX+yWYB zPqEF2J=QwV>ykJ@oYHwZx&jaC1xEan;ee2`HCWu>df1`x;rwN9NO$~NU!yo~*N?+3 z-;<3H$TBZAP#0S@O--sigQiO!^C17*TJe%0rYB%yqV@!rX)>^+fwO55-pgz-x?+WP z^&g}rg2Hp~5x+WmSUtq{6%E2c{oKY1p;62K4bcUE%P9>E!4o!GD!!u=eDbR4dm-Kc z1+ck;$qx^{h^FFEqhJvyzHe{;E_L*byad_-jkUU!AxvbhkslhxbsYglS{|jf2&k-D zA=xd@SKa2k&BbU!^&8ok=(dw}lvlE0;#>~o*UoKM(_g!wbiwlVvUBH45^m>)-SXbF zEUI#OSo!RtePOo_tH86Axyws>nLKCGHJ#Jc?AAT>Q^`C1bh;>zHxlgZ?D>*$o-l3U z2Dt1v{829lp969=g)z>1*dNMY#!teB;ZP;~sG@mVV66f=ZMX=z(gqyo%TYK3g=Z}& z%`OgRmn`I=XgHG|Kt8tmESl1!HFhv`uZ1)m66N)cbI=tJ;b=9aIai@wVZ z!^e9B6v6e_p76QjtCRUQoW12!FBQ`49$t5_XRs_ml59CQVBAt>SSvelh)^9@I7oWvWEWi!W!fs6&ZQ)$lQm3movo#d`1G zpoGs6Y~dLCNp)6E;FObSn}}QWR3&eElj(|CXnVEcFy{!*o7XiBZxVHL0l~{DiD++p z#K8x&sq$j~L!Jds^fV$6vB-sRxO?e+)Z->4JP$6E=rs=|dzVGzqJsqTOg(kJd4A-; z6R>_q$3gHzCf5FK&`uausf(yE0Hgh;g(&(r>HS6;=O#yMC9 zz!!nnbqqX2&VQlvKF%ca65*6{iWPwlu*eCJsJ46W@`k%J5nJfla0w$Xoep>3lm&v$a(&?cRgMV z>Cqq))^3%*t0ZveRD2S~v}TjeY7WE4cYC4my<={6m^va0 z7g9T~+i%bgSDl3!9Zpz9OBNUV>~WrfPqvppZReGNMGn(Cr6>N7S`wE1P;FKEZFwx# za+ApRX}Q<4>@}pDIU}2pMB!&aWo5TPU`VH+KtJ7C&)L4gsACUQEgs zrES9n0^~HvRgukz>9JA8qXr!#`MVx=)fnBA#f$KK00r#n9yiE85GOHV^;Pp^SjL(- z*+!Cu5;ix7qJNQ-vH7?GdHR7P8{Ai64Fjp6}n2rpJ(DhAt}V)A@xjkTj>y< zFPYHGd-9@bPG6Js0UxIqMeQ zDaaoTWW8)KJ+MU$50E^O%v!cgr{A8BbAmj#@$=MvHk}*<#fdU|V&Ra!xKQ`T!Yew6 z$7ql!R9EVBHp5q%)3l_8`0ie|%ORJ$7bhYy<_f#rouB=%7ve0IvKeCQD4*=@>1O{Y z-_MKkxg_J8s0K-5#%jJy)RUtL9ocuLA*5bxBXF%5$#EZFfEg-deK-B4f498S9(*`X zi|b`w)H~vu*DXEqw*yT&x=--d#TKEar)4+2!NsJVCan-5>>x`my+vilA*fLREZ60WHc65+(dyjF=ALCfK@=R1NdPJ&tz$iBTwH8{d&K(A64nTMcg?0*#=(d=mUj9!U-%Q^;m~DSUi|e2z^k8YCYKcQ>k+i9h0O#35MM zT$Wi%%T&BB)g1!Y9czs&d0LnoOytELZO44&l#`mKA>9l4cv*Aq1@8K;DE$(a)%8O| zR5J|{YJ>cfHGWmdL;>VJ%d==sdZvMAYyx?DB8nW7qKIuUc33Za^u%G-Gj;=>G5cCQ zm0aHaEuM6>!eWyEisOBZ^Tk8M_*?GQZhA*UK;W0!VwA`NnCll1nDUI-q>5&&1CX0@ zEw=5pVG^DQWYV$D8aOYDF~jWTJ6GuW_;j~Rt}sgpBVGDLg6G*hIH}HC3QjeE!%Swz z2Wy(QqAQ}pO8T%g!%U}zBKnj=G!Jx(u#9+G@J+5e)Uq?a@58&-0JVgfwf{cb zlZ+Wak~#NSaeQdJp86kS=tS5<$Bj7srCGk(2y-Rbbmi}wHC))TVHjW)S}3_c<9xUH z$zvW)oJ5{&@(NXB{DnoIyOAF)ZgxFU?pux5FDcfJ)Hm1Vxct3tEQgYu0#ahk$>OZR zCW9~3G8=bdS`XyjoTHn3U?S>d^iKQ*{Yj$p1J3QqqFltD$?U;mm z+em-{f{0*92|il=<<`7eXnU#9e&oZ8sDX6ZMe#i&sU=Zt?6in*Z_8 zJ`=o0B9NJT@Azgh{eNP$LSC(>eyVG0;#eUaMVEhzsnig(t<5dB$cGZoNU6!f9s^km-17QCC~>oG$PY)9pYlNEc$A`eZIci zLxHh9-Xm>!@eJdN%Bx&v++>XU(_85FD_!G7;UdiUlB%*O^|`5*)hOYb1XJ#)f!_;n zQTcSFeI)5||9&=TZLHabSqmty{N&X6p(oDLADb&QsPSwLVG_y#5>(FH|86KGX*^m!3H3sLweKDl(6gq?<=kHKX!58$_dO%u`|T48EbEpBxmE2C&Cz7G#*hqDjKY|!Q`@s z7AEcY-u`!XHPQl5>|VNmKtFHtFy@gANE7FX%kISsbEe`wvI`^6g~Z-a8r+QwY{_HC z$gO*y&pWYGnz(sy68WaM`|hK)A-l&RaWI5%MOnmT2gC6w4gp={yyxvd(?dq66*z5V6%g1wSJl=L0V&?MH5{e3w9!16H$W0Me zOB|vaDA+(i3b<@w>6@PC1+vIgXaL*sK~+GSf}6CCjlCUuZuPVd z-7zvl;>8T@wF=?Bm@BB}4B>H0!9EERpTDp+3$t8IZ?xBFfJ;br2f`6HI6E5m4u~AT z9mhJGFlwQ25|k5~ePyuD4xC6N9PL_>yKQ#lZ)PPuCyN1=9O~S>ZPp)M(wo)sAKZz_ zgF@=p2O>DiAa4R%sQ?@fdsa?~`F8voL2aS$ z&-PBRyYmUN5&@asRYOf0d(u}a@-Ket0Rv^n$of0hmkdk)NtluKuIBAC zog(`s-dnsU8pCNp$Wn%XL&o6%2 z@1MB{K~=OhTn^A)eEEsM0t33$y_ckWo~NsIcC_?a>Pxc8W6xdRfnmhLEU3~Fi@?*b2~Ebf-AHQan?+)$XVn@Ey^0=Oa|UI?B5v3-vxXW% z>N+X5(BjRF-px&h%J-zeMPh4lDhOtJQS-G*K@5r-N!pq+w+!bjiNi&0l@Nb@IHAXJ z=!G(tBv)v4YLvzHto!=j9B&f~)R}LOILbX3PYw*KWmja5%Q&FG8a1|_niZ&N@z-HB zWXX`mKX?>#tL-5xscHxc{lf^s&|Ddj{r@L#^ieVIj$-(t91rS5#J~ffqfQL%*c31w z(LfKd{P>0~E!CrFYnht>YEDPdP03ZUe$ z{Lz-|JPjc$kx)-|H5~Hu%6X^3kJPak+kVlkksO0`WaX%PS2lq-CwN2cnpc z0DbY-%B}c4E;JH;Ul5bS@x^FBszRqBBUjBP9~>86*g|)9Yg|}rZ#I2? zjimDV(T&PC<(yIZg9HbnA2m5J!=*i8tqw1$6GCMGzWV3 ziNb9|buu>v5Tj(d#S`QrEuY%Vw3l5pS(7y6GdzbXy>lt1!U7|FI4c!!ha%_@= z`64W?(7EfBhuHf>rF$$w4blkkW}FvQX>cNTfY|fS^Tkg8vJT170wpU;-sG`dxOj$m z#=xT~F>Af`u;oVn-9nz&0Q~xN(~{nT`F|_@YUp9FB@C4>%J?rM&ub1m~dm@y(cc>)oSr%B-?$uHOs^k6_GF%TPR+G0oLy-HY1RN=#)uERPMLL>OTZ>&&HRbG7MI!_q# zwfHz%fYCf5Wkpg@ZQlpnP6%pBWpWK)#x@6(Ibl3W08ilF2K+4GsMnrwp?vp^?!e}1C+~I@#RL0Qk#hzjtGj)UevS@Bb6lisn-|$)$CXAy~T_C$0M3lfWvy&)4|5%u|5iBHtgV1 zom`d#H;iIp>I{hodcH2|)Tlu;@w6x)w+1^EYc9pULpQi){wJ%W28uFYvE;h!?0%I` zLdA6J+J{5V;^&Ld04C66ZpW0gH|Y&&NC=n;Ewmd2%@3`mTGVMG!92kJVQt@qp_>hd zbs5KYo5ZHy$J#-+O6;|Azq;e;iAe2154+T|vnsnjEWt|3S5s(a`rfU)^Ylc-+N4Jq zd1jJ{`<9L27^m2v@&>f42dBA7U8x3%ZGO9^s8|&jV6njo&l_-?P#lE<_=H>`bxGjS zZege)wy?NR?7uhtV1pp&usi0TKx!gdw(tfdoVm`T44{JUKI7D z?)D!*7iw@;d!R3d4jFl|PUYV2U^(+(g~`IVPT{1rSOct^ldPA*Dl96{am{L^V{{E8 z5zj-?Ave3kxZS?-ksN|La^DY8Y3}y%yiqqbgwc`?hDl|r|M+NNlaQPa9)_%6+7eqD<~yY`Sh4g;W_AjLt>X@f*d;P0}LVDh}MuXIk)2pd$h!M4eA zYso(DjhHXFK<9jxV-EE&ZUzOtg}oUTX5O^JQUR8yRa!b2bS>mNDO6`^l*6nG4h}DjMMDTzxrtW@>Dj1iTR`I@%Pmy z9kDP#I~c1^i9XK$bFnNy|2@vP(cuS720niy#}_gSOYW=-EOYC|bf$0RTb+L2HNBx- z8rlSxE|t7`*+UmYIDJbug)~phnP&D0xBjSv04O6rrbuQA@(`$ry}YQ&r!VfUQbt=u zDu)pr9lH|CYHz&2o+SmkD1ZGvRFUSh9VZ!VJx19Wul{wiEuPu;TN*sPJx}41Q_`i78q|G`!6<$tjX3KLi9$tXjiyGm z$o3^op;X$Nd*y2Ux4)6GdZLgi>}b)@(IfB?>PQ&T8&PLwZ*~%66u2(}wvT&&L;bZO z9P%I)3fTp=))`M~95MLPKpW0=3ut&@hzCuD5SvD`_JN&iJQ3RyU^KHH4(d#CXqG|B zDPkfyyewjLR1c|4i0Xp-$+#EDytR!bz#&HFO=zIV zxUuZa<8_Ki?@B7ZDcpfC{Fk^tU@SD&X*e2soErFb9R4x;FeWdhb-Nto~Tw1u|O zj{;MV)BeEt_6d9j|Mq?4UK6l}4fM!+LMDWg*DYpUQe{uY@ia)AdT^yjxj$$+{*Wy% zZcrIgv)RqPu8qy|Wns*shUoKtKBi;l2ZUKB7gCS4=iFJCWbovbUU#k6{a+mzeZjY! zv9jE^F=HJLP!txSKl^I!U~7VP8vI3z8a|rHD)oU`AcYX~y<5xvG$wS|ryz zOGg`nBWc#cZtxy_`@axbJdsjsF4f?C+Fkb97&yXsnahGzjfB)YbY2=E@=6YXl}IIv4K8`V9A^VNQCKW8zxWpZ6H33H;tfXDCy?I&X z6eNv#(3gt0X?4m50*i{Tg7}6`-nJ9&{3U;2C)O;Cy!G;sLTg?(_FkwL>!~4*f13Tx zKo%1Tz4n5Z>n=EYzz#LDY%FYrFnMaVH2#zYuWu{ zCmjN8!wD?yVs#pJxgYdI@1X>scte2q&ucy$P3B0g{VZ?h!MyO@iaSU#x zb|E{rpvOvXX8?3#TViu3%w6E)b5J|s0^RK%sU9o{CI3!Peix6e$S zMDnN7pPZff%i;bNp=V7O`_!hp2JS7N7K%ibBI%?1>rGoq@;MU?m zisYEcN0*Xpw307P_vB0@BFs(9PkJR{gM-j$q3HXx-XgqxQ$Cro9hIyuIv8Gv=iJ;f z@$AQ{j}KEihEHxgs+t_Xi1F~jvih60XXm8YOgnosb?3SR3#`{1(>S~S?3{_;RN`|2 z%-C7A1CNICO{KY#s)I0QNT_m)Dnq{n`eoV9qMzt29}@evX!K z5tg7dFZX6xiCGo0RK7$!Hjlq~;@AJpjq%>tzHJ^}|27YDN!qI9+H$*?cW+XElQ5!_%U}*x ziP8jCbJ2^0E>Ap^T=hW1;ziNdO3(QYgG{Kx&aX|KmghWC%ltRFOd3gYrr2dq3GLm| zzJsF*n0Q4x(P(*Cl!DWVo{xjUUk^(RQATF z-^ptNv1`^z?~Thh(YK*5*-#;E=)V1(J^LIQfl6L%(wE@@^BC<(T+%SXpK2fK8i95u!A;ZP@Bq`p6xVHqI3^1gYmi?#)|C z6}8Kt9VVgflXcf+RdcX19+lH5O5J&J!lZMTOaGTS#&f)zW%qns=M;tiG^=G$&MQM`mZ80auv@7gwe!zR~W4c%O?#=_>&gfcZiH< zYlU357~1s<19%;vghSHqI+yR+jDJy$LF@@EHcMlin1uk9c6e0gRNM@u(Ey{_2rB!b zK;?eXkHLLb_|fF8XQ_x+E0sBexciSfTN`c-81a` z-QMCiJBC>oANDN_+?B40H~L%ymBA_zB=f@2+A`h`QuC?+-vxw8sprYG>Vy*Wlk{1;yf>Fzvo3Rd>;Ew^L+ercE-g(u~P_8GEWZo(K=RIGlf)P-&mUUXS+24UJr;TU(5fbFZQ8Z}h^ zmlcIyTh}8tm=l~=OTB@)%m@lu?i-ur$m+@XY-t|s-b5NQNnJo=O<-}4#-e>_%|P*Cgd=i(aRVTvbW;6Yv!_LMFo zs?H(_Fopuk;VtDuF9+u%W)j7T&Bv801RBOoK?G!m5l);?CI5-+8E7#af z<&WsF!Xh;2NJ23dk{ zCr@5qS0(|TU%HcaXjFz%(V%d>S`|@&`o9M1gjHPk!QM9-2r8LStoKmwr{k^zqt56V z;~Ti9Z-O^;<)C>m)hnsryAKOWpAk=^@s0$h5UsZx1l|JAFfUpjRagAanxu)(zcz|MWINCNSua2UQ8t1-mT6eJ3_7@ru37zC4ry5q& zo^M6wDzHvvBn68xhXrJ<+cG|Wh_Q}(TO0-X;Tn|a#vr%-lyJ0*cEK{kHSz_X;##P=q z8$UCur2eR+UZ#Fm8i#(43gy(N8!mo*vI{8%kaMf!-`_o}<5~R=ahP5uvwhf2dwgyB zc}&*QK}t+c`%dBOeOUCx0FkF_I&XV(tdL3|py1beL4cM%`TZ#7>esM_eD2nexXox# zCX~e8xK|-N><%EuHx)Pz5un)3f?57K4PH0U){!!eJSz2ru~Loa0`!cS$RG;YRJv%x zHaw3r(5q%=&stVpC+8iKY2xi*bn^|L{e(~rF_R3y(wFwZ|1}lw8{-H{c{e@5T{k7M ziM}1}8adyCY}ej9Rd`xzIZRI>{Nnb4~T3RC@o|iC49MD|xxk zB_k}DG7;ej73y?lgioLPn!UilO40y;^RLhHI=&LIA#`QoFuv8vdGKE$*rT59zy^^Xu$G(4_E!F3{N=ry;*qkh=KIWEA6X2npBi~wpF_Qpd3QqMyJo&z2M z8eQ1$p@Kr^`0+l!`$%mrDrN z6tPcYcCh+7Y!#iyShva@b2JHspY%uh1COkN5r;5AM_&u^4UZ~+u?~UPL@?-i@5?Ui z@ry+~NUrxF|Iw(JvZrAh;u=D}lG=YHJn$mbA1f&)I=RNxx%3vc*B>slJNz5)~vpEHGkg{@x zW>5i{Yc=k5_+9;NiBq5p$92J!X%Rhtp%Is)kR|HwlU*N?Y18qC2BI`IV)yf|JAa|w z<3Rr&qnDcw)TAO-w}94Qt2cdmDd#zJF8Z;(Da4WL{$d-;Ap2{GumjJzY{NLmCAU5F z@Q)dYikf8?Y1F>_3vD|ARm6Xr>Gha>#ma#E6Kt96to4OYR-+GCL&0RhNwi>6e;cmHp-vmKvyLK5E*$ICHxre#Ct)QmA@#N;GVhkC>GS;v4D_yFHj= zAbkxiudpxrmY7^DcHrw*J;D+MoB`?aX|){Vwu?`InLdB8VUwbWXCP|uv`nMU z3z0=bl2+iie`)`_PG;5XF*}O@Thyo_Up~Z#7>~Gw(hdc&Oxn9Vs&r;EQ;2)|MJj(Q zx&4X8B)}fiS*wy<^Rn)Lcmx*sE7r}7?~bu5NU@MhA%bwHG`Mu?Ezk~w-QS>5;Gs{{ z>z(wahmN^$^|oLein4!H5JdTM@cpClQ8?0at%2(3yTU);X9*ICQV@yb&uMSRQ4v5? z0!gRQQ?v~{M)0UgJ>s@2pH96Iox!|Wnk%qM9{KPzViz_RsjmlF1Kh@Zx9*q^Z;@t| z1n3fN;YDWQVUO<*|wwRJ=q%gEYF z-cHZdX^X8rhvb|Jw$i=eJ^tMH$FUPjn);IOR#qQ-A2%B#$rLa?RFr%9&Pw>vDR5Xk z*a0U#odq*9i8AUqByrErGqZ>9FCX_eJw)~0T~gD7JXPO#IE+cfbUD-`3N;_1edq~c zOv7q>DtKY~UDc22kMMr5pif#=;j8v{A}kykCj_qB7`11WZcX|JqlO}Qjs3?Hat@Wa zp|!I>(|-FeXPkuzCMXR!wem`Lt9O3`_Vk%01;mba=e)@*1`2=>iJ#ZK}m!Sz0 z;FkU=bIPwVUj;oPr}c=s%nvz7`+IxIV1vg9qKP9{YR%RLydsO5a>IFhnZzv>MeqBH zwxb<4?cN`~uCJ+8v{V_I+38Q6zbv(UF8mdK^VWG|)3YCfAD{W3x=cfel_zjhTaVV z-=arv((xnvTv1HUiaCBC{$Vz<7ImYNv0}4)n33DEgD&su)QnS}XJ#C2yuwEcW`nq01OV%C@7kRz94% zCFsR3+K|wHBW*m@(sDF=7M`(oBq1JlqT4kO>fk9ulfMksRF6b{0u? zdL=N`Q1Y>kjmi}AY$PGNYCQKHIk*vRsB#H)D9Yb`%gD|(=#fJSGiY@~PG+`#!wy6X z-aeu4vv)@ll$KA^H2=MKYa@y5?_S-7FN`*zxAe;S+QB^QBcpH6Vs_dvo5(2kT_=k9 zyrb1(o9gJ!kwAU%vIhGc)?T|Q=(jkt$y=4QQxhABY%{#nyOG2ZahZChZaH@~+9u}g zHahJ{fCZbVzcWUEE;QP^CO6q~iLE?-YbAp)56X$LNH5sGvsF;AO#GlXXy(FYrEIerzUU6Ari%uKQ#2A2eacmh@hY?k+0Y zb$uOLJpnDgl#|VgeD{qjuaAiiv+}yVseW~|> zFSzFa#qh}+MLec{B(e8HmEPwdCKd?Zej#OfmqtrN?>79}Q?Nz)sZGOS|5#wzJmf*X zN&4PV%b1VYfm{r7u0S5@=PThAlr0lGES}=4cRlE<~)EqR}3aD&Q6{tLL}W*Kb~Zk*Y4O!`G_+{|=W3Ny+R@3=r;Jbt^~_k^H-^5igg8G#}e>}3BtkZ6s54Y~pr zE8MlE6oer;b%aPk{d3yL=kR+E#@LMp?L?;~4O-q`_BR@nboU^`+<%hKk2>LX`odX3 z>ZrYSlLrq21`Vd`70Z0DmN^xD$^;)s@q_yGl_qNVkrsorXVWN0QeZb(;DfsE{x5WN#O}g-QLb&T9C5;q1Q<0;v|ltE*MGPh8dr`Ok8(* z*Bigu@1ba^0&p3cPMv^3n4zFd_FcqVwXMYiV<1o6Cv-WyLt^5OXb?jlN4p!b&Fn}l`#y4hspBBQn``7dNA z(9Kng3z^X7_=jHHY*C2ZHU0qm!3t|h>wG43k6(2vzP-A=bv;511~i1MZH@^KUKS{i z@N%6p$$uBF#gX+qhxnJB)W>uBxM1$78`^l2WJ-t{RcSE>^nvd{ujhCFV;n=!#i3&aK}}?r{nt zJ{(>)2S3xw|GWt~3#A|--(FNj2AoD*NnQ>l*F=qpFLk(QPDIR^a71E~Fl||A_&I3L2j7Q#aElV;=gFfVTRD zmZ{$uo9fqNB7h5YUSoU2_fIN_q{?8Tm>>2o`FA-`%WOd0+|pk~A%$rnCRoZG2@2(TWa{ zGJd>C$<(@MiRjm+K%nd&SBsDX6wC_ez~k_(Xi&~fnT&sIUI`)o^fd1>^ve zm{n1cPD5F#zbV>J|L}?cPv_E`?ur+VYUnjhFtd95lD^t7e0DbgMVnq&#n(!i}!;HOBgDmRmN<*jmCs527zrf-!niCeJJ5(r99XF zgg$K(vFi8`No*I7OrITO{B_GkCQ)B-xEd*qF*j?EA!)Iv{VAx2XBcq|3*e* zOM%pjB({za+mo-=ok7sUjD?&XkColFZW#Ybb7-d8d0I>NyZSZ$(P^G$c4C3VH*BqY zpJ-cw9V>~IXcD|)S;`bdQ5j{j&3lh)z}Cf4=;w(Gf#j#?4dm=6KTc>NyQWf^tcrv* zw{BG&DWE`YJk~^wp=&N zfGp^w9rUj8+5E~3X)x*)pZeU)yq32|${`y{sCW6=M)GE+c)nQ>?;3hNExI;j<8K>$ zgi6jFB~r<5QYg6&4&2&&wH^Cj4Qv2-&AxPw1gDm7w&<(V(SKk@yE1#3K-(K#lS^vS zkQ*im9R>{gX=OBhIEy%d`JP`pJD@MFS8R!3T9PBtnML0g&Bd-Nq~H9P)89+`5rxz# zWb}oGCY{r-)bht14DRMrhSKC!H4_oQF~JPszA8Lwq@i>dsncx`r+s1frPg~q+ByG7 zqTO-fhQoG0&!Lxez)|_p+G(9JZ%YuBX=!!+mm3@#qMN(nsy5<#?g=h+_DQ}mS0f!O z@BwA=E5BqzX{0W)AxWnaqV%fvhJlWr^~jr%9LXoDMx0A6#-IX0I9A<)j6Y>OmF}Ij zqDLKQk%ZuQi)KZ?ckbAL78DKDKdCik42jw_krr$Cm>Ysh^g*E-Rm#^527n7dNF{*s?)P7l;UP z9=Gd7Mpwv3!HecvQSfko0RN@=favZRf|HOp)M2zleW^bkZz5e^+_uY8C-H_UhB~&( zJ93zr>$yWYBT-MHTaR7&yqixg;)jRqdT?AAm#>c*e9BUBX}|aT0HO(pQe2e@=DbPH zo0+jlK{BD)_?|2~V>wog^UmMV;wj6-I;vbfrJWFifxWGtC%esxSfs*?{@re-!mIh+DiPnVu(i7e5)X!xW&q3T=I^J-k*R&bowW z8?j8hfpzQQ%oE3TG2D5T5RE+B#i_xwE+C|&K`U)5VVWOYuXt!tUueIaFjZ)IQjmHe za&6Kr3Qjh`BkX25!tpmDPNzmY)Cie)wU-nAD zyjk3_prSow%`oSiE6-hn$F<89DHw<~>HA4O5`U%&Pn`9ibNYYEc{HpgBvGlOs>l2v zsa*aoR5WPwZy~6;+}wqA8W(bb)0|)W8`El1Vib2ip)zv&lDC zvwpNwbuS@8k@I-sHqn3Ecb{{?Gc#`#SPewRW~pw+(oS3qBs-Vg^;o@WJpQ#k(2iZS zwxIE4!Vl!CFjs(dIHxPktfUkR#zG-48^xO~AJQ|Qn_`AhFCkl!II8^P#{aPdZYQn)y`OAALHAgPVhTBEhbLPZ*Qf_PWcnvBEk z%}3XwA0zD`;XQ`;kQFM2nKA(cBj{vhyL>ze*L)0?TQlh$Y!(#Nmd|vD?o==Ivu}P6G;@yr_bA&m-ATo;zy_N;-8ZNI=n5Q*pkC;Uo=8!-Sv{9I3uz@b1n%UY1Im{@D!e| zEG(Gqp0B0CxBV~Y~PuJ47Xso2Eq0%D}O;Q0i%MZG&*mdh+W=Mu=fMsD3nG z+_`BfmIV(W#GqRsncjw=l3T&Ipv`WcJ9N+tFJgxW+1S_f-Crb-#ZVczRO|%zv(4u5 zCST)s@&rCmQjpmzxdG}Y9bDC#f$ZX%-NB2-PfBFu=*&15NO}S$8nCMSVrKk5LRjv zd$QT7?8SJkA#lbw5X5-2c(W%Cm2$z&{j)T0giT1DE->!%_xM=p3_nxF-$Ewe(cas6 zuIUAsJfYyjg;*Zh`*qVy1iliU_gG57a<47o;NIZ_q2Va3^ z&t)t-lrlb@oOhRO8_34k#D)x+0q5)EtQpKsDKhqs z_0!9{=~Z1bV%ZdImqKcs+U^zqobWb8VFMT7x(%E=o!Yu`caV;wUWa=fX;Y>Rh!2Z* zV3w_%P4>FBp4T)#hK&y@s9L-Djpn)oyruX(xGKVbDE96B(Yzh48U4yiP*mj7LIoY{ zR4b>~{7@G`1w@hz<_QyTHXEInPPv%xTH@@_smLqC8`YM}CHLm+TmMDigrB*B_}K5* zS(7pklhGRjfnxgo3*!wCd=SyF42n;5evmPaG*Zv>@q%`G?);d+yo^t;_m)r}mw&MC zWVvJgO)urM(-HW(Q7%M?m!tj@y zcHC#ulTEj_v?1`S5 zJmf*qfU?xGXyTMx^-2`U)&q+vZc5vH)3tb)V!)sC91bhH9)aVJ#49Kq6rWnVBc=`S zoFy0%XXLa}eB;6p2QVW8$+)MQB?5Kn@@Mw&GGb+KU7JamoJTY)gF7)}oW1(n4BfP> z`@~}@2a`*f{M?9`d&4|rjuV<~lt8zNYx%4Q98S6vNRCv_U(p_`V2e^8nDGJu<8xeQ z+=f)-q(g9u*y%gCldYtR4#>&5ZIrslHP4Qd?zWf^@A)*16YlXu>}wP;CEoC5wrx&I zT9L1hrt79mM#`-!@h_&uZ%119bR^R^;Eb}Dn|v#p2EBx@o!CTeb3VNImzaz&A<73z zv0sIy);mHmK<_c(zF?v;(O=BzMuU1ne77?@YmcHjiULdW?W(%f>Gqus4j93V48*?8 zy|>g`nah)~HK7Q26>nsc<*kibE*(t9%7Dw<-m{om4N&{SKeGM(qfk61xRK`*xwG~f zD56EF1Cky*vbMfL$itY+g(SM#>W+X!r&#G31_&^*H%XPq!m-x8=c2Z_mE2}#GUg*+ zfU0wa12Q{T8ARjRmVzqr_Y2wHwh}>!0caApy$8xFw&5>%5Zc(4G_i8t;>pIlVt^y%+vvMFaaHS@9Vo~zBH`5d*@vw|?#m`1JxF?kc&Li3 zw?uEmgT^Y8jW_qWjX&=$xPZ4r3GBm@?(X%Gukm~Kv&n>`1vUQZ5ApIz$rXGzwnnf6 zkE&xt-V4VrW^uzywZ|brN(0QFooC*;aU9}uX6vxkW^$T_)o@1YSkT50PKbri=DsML zjk(vfN5GDktGikIQJN72NhSe1n`X-QcW=bXk_`ef$Vrf$<|Oqn;b^&|qCrig?IH>` zNMd2x%~uQfgNG572?!_6kIwn)IQ=}PXlP&bJLSC7m7Q87ZPe&@>KF;Q)l*Z~ae6)> zH6at=q;;?Abv?FES0S*Zf|cbTtG@Xq?YNYW;P$Ofzr*1c4eMtH zjUJylg*1sIzJ7e3_$jyn3kM1E_dKOD>7E=Ht5_$veNU)q=F@}Gf&f*QG?dq0TfJe- zP^LmSsjJe^C!CU-8G*^kiX`6q+WWY2bg`RDp6k;a^?O}9PVR+tQ1BJ{JrA}kn(~$* zP0aNYlybc`o`{M=P|0=`wl!@167%ViG9JLQkxL7&Wp^nAcuKQrWFm<-ZxRAyEZolYMjX1 zkNc@Ta^s;j5qUt8Wrk#7mq?dhghU`410aoD5mq%sN_m`5J~yfSn#6q^WucS3{#=R1Uzt@Ze_1>3-1&3Omgx`wT664y&Z3FgmWG$=KReFM zZdRFih6+%CP3d>?)n*)ttb`{z-A1CSwEFo`Jiy<%S%xr!8P4jwTHGe@tC@ zJXGud|8~3OmZVZii@B{T)fAPi=eD|~nk1BMif|()6v}dLt6NAVNtS7oijpK`nOi~& zGq;d66JnBOFqy&3ob&t4a{~dGAvwimE{Y;k@#L3Sa=gsCh2DQx?rYA9=N(wd= zYFJ9A&rW&u(NnfmlrVCXh$JN&Gx2bW`s4{H((g#|mWbN6uG_GPRLyr@qDy z6R{7O@6_2Gu(9c9V6Zd&-$CAm9DOH0)tl6twg<&w-p4~1d3S}P5w<0ecf0@OH5zN9 zZ^eq(7MAWUQGCX5n!T8Kmq41N{;=qNq~`treXU7WDs7;Dp!W|u@Xolk8+p&;GtYG( z@DBAWsSlyCXXX66?H~O14Ezk-NmExz1^uR%GV-L`(LBBsps4tI?`XQB zsJN5$W{~l*S$ALfwqYYbgELux1#NhIj%DU^mYUq?b)=zGQdKE*N4sgzCKG)6>N!f& z9v!N|0dW+s;>*xbLTNRk5rbA)O7N%to3AKwSVMy$qt_T*kw3ge9gTsDm^Pv7SVi`+ zHnpYHy3+NfDie*4>EGf_8CK+C19vekJTXW&z`-0H*YpH(87pR5!D99Cq_U3L8MkmF zpU+`D)idbYa$Xp>x|Np2nI^zWpMSnsTQLdk#~J}Qr_(}mL?9A1i zJ@aRVj7#!4gPdoaIRfm!7?UgJCI#j{CnLh82BeTEC(p`RzJ$I9CyYfthq3I6o$(+p z-h88N>-8TEr{OegGHPhA26+#q$9*-0!=9QmuENFry#kl`IC@bV6#(Mk;3g*?E4^^j zPrE2wp16+86?qiaM8q5)cEK_@`T}h5V%z%@Q-{M1#l(s-;zn{^t2*6oILH|;cQhn> zdF5=Imj6-okOx$F$o=FSxAI{fS3`R@$h$O|o+1g}^*8#JzBHYY?yxejvLqIbs3O2p z^~Nva%^I$XFD8=82o?LFseO*NQ>e4SB1G(>v*UF-F`Iv=vmxyhNC&gMHRRWL>KKH@ zgFJThg$0d7>Po~~jS1w{G+}!j%XTs%rKSWZ(`{LVP^Cb?FC#b!@sm^C3*#V(21VWA6yS^T9tsW-_vT--$8V!>BOB)GAE zMC|VtPW|_5RL4M*O0hG!pNrQW{c&;PGIVr*a`ByDt^SJ2+B>=yE$@1f5r^tE4tcFi(@?6g>#_}p(L`rQ#O&h1~Japd-F zY6SBmgs3OxX?yIoYqzKk!mfze^TOVb#e`?5N3`-mzsDFy^UW-bp|4 ztuCSg7IX0vhw1G94lF~D{_2*qKWq#(ndf=?O9kuB-cDN4AkTq4{zJuzLByoA_k+AT z{b?o|LpK8{AuG8qHA` zfUdlY#l+h(;>kB}YpbE8M;p))hj8(>sXd~^Nz53u(y4A7vQ8d4j+V8VBCIG^u};?< zX1U9IKWr&9rY)W>P4OlCWT9E#D(9dxtpL`}x^<$hc=1%;u=2+eMeNw2>VHCxWEmme zSjWYK^F7@(ovcrzK__tW8?zs2IXyB)kG6C14RrkW3Ny z@bDGl`|(GfFF+#%cMmdl-a4VJez7X~6oV7#oPm#$$g^$Lt1RACpiNgbBp=MO;=dPc zHXSx}JK;=q?iS*QqaFK^m4LtLjG`T_z>HVVMDv&|A@LIR!;@A$T}rJ1YnHiNwx2tc zNnd@FA{#bXFl`!Yqob!I{YP&r#O3nNEn3DGkOIWZ26?RIznqE+sU8tqQ@FUVw#wax z1SADG0PM5=mSv>8Z#A+Um?eWz%&bM@S~F)_Q9DPT5wUIJc9`5qTd(Lt&Zyxeh}dQp zH!hCr0)noK^?C?l+6;V*0BbQCdqT~q>MGi}ZCreM>!Inhj14ZKXG^*G+Ai;F@~yf^ zcUCn>_~O;Urp~3&Zpg&YFeJBf^?`x2s{-J*ZvPa}RHiw7CEdZueJa8D$+ za5VvD9iBB|Nk-Zjio0}HN!`~Z@%55-x}#yUfJ;<8TC2_IXmajgbOqLVv>OLe8ifO_ zy06||8l_6$B|>?@@%pi4?J$Z0pXRQAgG{PoHnDWbb-yH z`=L=-l*vMxhCC%=f1TeI>m6ZE5dZ^htdyKda7_s!F(Fu@TPh}Qbr**r*+*WfWy>V; ziB6JIM4rO0vXp_Iiy31&(5oq6Vb*6``Nw$9K|CRBmI=!J66e`H-arjXg4b{|+$g2L z`{FD7#fbnLNUqh8U@TeH9XmfW^qD@ACZAyy@4K$_z8tZgS?kC_XZPw$_!_i+KJfpo z7oT_gW8b!+eWwdBj+3x0z->OF05doID{d3EqQKyP3q!y6zq_T#a!VzCL`zj>9?93b zg$R@brj~z`_EuMo7l>D|e+LpgDnUUCHXiqf83-Tn~gwbA(i^Z9-3&} zfCTb2Yhw7o??@_;L2h`nU^M@o;|cu;>Ua230T#ARa$%^sA{!aE8ox!;Iw?$SFOR%NQq;2#)jERSD;t5jXWyqsJM!pZzAU?SN!+K7|&87Ol={CWC1)uye7ZDfUA`@gi zyOmq0ibNgVmyi9HWVS5hKqzHjGeJrG5xtnVLQ17Qok>?NzK9=wL2K zM)L2MFQe(Q^rQvjgJ;e8VLKab?s7LgL3k!0+Ct?vf9{DvK;MH%|cRD8Y3i!^()i zKU$ZV7y7!)M&ic2jObV6MlsUk{-H>mye?vM$7!6Z7Y~R0oJm>&Il&9-^{we0M+s_$ z9?j9z*l5n%@G~&zOycXveFdH$Uk;X|QwANEscUqPJ7gY>_*(#WU`zS+*DM3X2N=-D zyOd~Cp86(AV&dFEhIfs0Rg?w#75R3Mv4pq8wGWq>p#?Z_@tMJxt5fnsBrI@nq}t^* z32xfx5j{tMZJf0|a)Q+88agU|0=d1;XiZ4vVrp|kqg*4%V3WV>dSo7occHm9Tz>qz z+^7ECPKHygVdMqDaQFp3UrT=JdC-0ni!9sZO*w-xQYs=hRKMYoBu`%wEnMA+dgrK3O1Nt%=EVMgr&M<)}P z6A%18TAf0cdN|O({x5A~*}+C()bB`Lh_1XaHW(E9Q@cetCy;@U;|1}o#gv^_u#SB2 zA%J*!C02lJRXUtxe6y+yA~q=CG^LD~KKYR96k~rRgYcMIj#u-Kvq>BK(1{1>OO1I! z&sX{nnNdPQ5tGaOI*=CALfJHmh@8C0EFrt!$pn$b$1-A?VCjl?g9qecX!-9~N=2qd zd5ime!IMztXq8d9uWIte^5f&sBZ$iF2{xRY^OZpxSp>+#FWKXk^t<1D&2chaw;r0` zoH^)`!tBzS$<^-cdRx>0AR|ixL)rJkqGlMfO&bz1eNguR1nD3jlyNha_qxpB3=0I_&Hc+YFyy!M}* zf<`RAF4ypwzBl`wpvMl0n{udYunGD!jirYM<$!-9*GXeH7R*H_1%b6_Pp=-W=dPyI zpkYAG?Nw?S!igvbYyl3>iVHmZ-CEQF6k)&zi3VG|usJA>i(RdykrnE*cqu%zx2*C( z-su2?v!5p-o@aq&lX0_r>!4^Y;xyL#PG0}}+S!)=^U&002(U+=7TnEum_-drPIJQx zyRD!9>i=dt@+kS>-DJ3Dr_FrvEA=~$sQ{yuOYaTjIih_tK0;(ZEdO+d@v(5Y6ttMQ z2w|C!{3`dBAheh1deZq7Itxp*J&=>hKVQrCPvt((*Og2`=6-;i*q|H_vES2ZNAWVT z0z`<@b9rzaw9h3KKlHX$*!z57(~>6sPO)(|)q^F4pr3}*f+eY~G8Prh!@wb3C5@gp zbQWoX0CzF{dgD>^?{{^!QVN(4Y`T!s^_%Pf0TVQIf0x_d!l<( z7@3G(3>UHOo$@rep&8+5hgohom&xuF4~|4Ea2gkX_pDyS{@G&mGbR`RC8_&u>g7|2 zR-;60@z$W&v1<^C;9!SFd=_XCN9afdR)Znh9Y5Q2Q7DZfE^|F;bQ_()y|D(#HvE5? ze0rfEV6yIeqyyAJ;P20n=bl$ByhE8X>%opQe1iA%iYY-xgD3&>1K+(N_&J-JEjg)M zsqy8m#th06Wc4Dak_ zW)}Xd0MlS-Pnhv>7~LXVKHrdx!=$?^(=INSx!d(_R61AYK1Al+1yz2%w>Bb4a|c5G z*W+gh zAMVZcN%U%4JKtH(>k46lZGUHA@&3%wd zMJE*7_W&9k(iOzggGALu@wLUABI7F6t8_bZI-^=MSdI`a71_QU@cl)58J zmcxTKg$GjiSk2mV6lunl5u#(CsdRp##W*|XQ`%46iM`~nfD&-rb0 z43PCp>T~hk*W62jTacp-sd2VdEBloLIX;})trA%n9xfqXX5vc;`y2_goI>+-4>5xnv0sz`}5-*raKX${bK{nE9J ziYYKlI?156PG(Ejmt#7t;gVfv(wB?xY<+)` z3>AAJLS?R!W~MXB#f3AF1m&Enw`^>2zreuv7eXY+#oQbsVgNVsrjOtKW%t1u}~6I1uFzu2u~cJVHLP8y8tX{#f2P2%D<2&jXF<9Vibo6UpU=_ zX5cvqNl`O5vR2L1A~0 z$Md!!-jqP1Z{jlf7xtWFB<51UYxFq0)HLT~Aa$m+{bIVh*6u|GmuYAw>mgHiB37;2 z|G@-Aj2jz-_)e33;=V6Vl%{61%5?mn+lCz~Mh2NWWH{#DZRp+{mX8L7C?K!CATn^Y*y4clokU6ve2i|DOne( zE}9~lVO1<)8nvPf3YJ*Q{@8z{SS9^63h=V`i&ZWrIUVZxGLr4XAaZ}8=v})N&H6TV zUO1%ZrUmG#(Cbc3!5uo!yi*zj`s>Krf;vIpkR4*<6tD$8XKxdXwhbAspUogwXJU&w zj>deBosQ_M~J{nEcZm$~7eUg%^M6xJdBz@uwvm!B4J(;Qi7i}+y4G=GcLP8g5E zt{m`J*j2xb6*L5%-zFPl@QF+nyZiAu;uR9#Elu>%Xi@wFVy5>c!uLqOHnjqdQenOr zxBLPkL0d3^MzcTlwCjYAz6EGL6wDe)oe;GliHbBmnh>TK{H7kE@W`YAk8H5 z4pwjn-BFwNC)FP(#alwj-@7)1@ElK5?xiB?7j8OhyO}l0{-S?g(Wtr@1=ctDz%i9zKusG<#r3E* z+D{TZ*l*J!Pn!>-?Sw=Y&q26ooIZUGO5?CWyVOPR2);%~`c{$KlVrx1baoCST@@4L z5yEfo-F;45%%&32{@>u3!Kld1Ex+c(MnYAQ{aHBWMv&Q~kvUwg6)5Sa{2aIa~r z?mTk0S4W?7E(=XJIQ43gdQJ66h|{l(*yrc_>S^Ohc$b+_E0napv7I;l6Ot@=l88OW zx-NOq!NOrQrEvMeb%6X?zoO^O6g0#Y2s0;Eo3#XwSkJp51o*6L(vZ1b5VCQSntuc9em{ZU09DKoUR{X6}mC|UqLC-15yx;>sFeyoK5paMq27h zsXlXgwdu3k*e3Ll3AvO_CnT?vOewBp#srba^BmRL)!`_jX4}$;n+4U)U%O}w#rhnh zOf1B2H;=;!RiV~U2oqkp*7?xKqNF|fRB0i2Cvk-ZCX5nuq}?Sh;vg*dKiwhj@94B{ zfGU@!1=zeCk8}@RPwLlk?3+gKtEd8`E~~*@?tf|aUd9J4wRJ6hWJQ})}7n@KBT!=#hp?6=1h%|w-;I>y4(ngAjHsea{MB+z1ePJbu8QM zsE3Ca+RU{HSwgi9R;T`}Ieq`qo(I2YmF(O+Gw82LcVE|4?tXX(I=|KkjYXlx?rmj$ zWwKT%WVX^xHR3FWaAafTT+hXweh1XQ{@Q)JCvDmT?AJWxPFQpLb)fq!)0>x>l zMS~wxk40OAm@Qwh{LHX_LwtrdlVg2?T2UL-M@EB*2stw}v>g#AYm3>3wps3b{;~X;|*l1Z%@#!e>jcW z-O)41w5VC!p6=eGM#Iq=83sMbca8O5Z7D^~w(8JHcI6vA39nM823QZh5JT(#B5QmX zi+SI5$)zOVEds)4nV$+H3@YCJH=k2tk0@G;AI;aVcN&Kf=YEqAmPe}7BEpWNn*Y;U z$@Wq$z#_vO}z66r_UU763C(jiw6u>tNIk)?#{(N66rPTOCV^l*w0J;%S|2H)n-LzyNB{v&-0^_2D<$OHecLIf zAyuJYtK4ad=;d1UJ7~b)Uel9ReSf4i3%Yvq~ zIBXF;^3P7xCZ#~aETdm*HL{;HhD~@uE0SRYpCpOur(aZ*jVlUAl<>=y(0E<%E0{CG zs5U7@=a4eC0apeQ{NPwT!1M{vO5sB{WB;TKVTfDg@ob28ldQ)n*QD z4+6`@;o_}%_37T?&F_{DTL=Ttpe~3%o$6x4MJzTkG?&$9PdxE5T# zREoD``>vx@zbqEoj2`gM-A}9|r6TRl~Ekphu z!wn^8NxN<#S*rdIV@@FbB1OA^L?>#^3M92WsW+nlOk`*wza;(JPs|>YqcxBY{iDRr z`1isesHR0&t z0g2;PY_40leiy!Z7R4N#mO_#Ud?Wrghdu?3c~&Z`jH$oy{8kqMa-5P!fn!{q&g*5`ENrA&jaGe&nrsN z*1^3TI&MAPXIhUT0gnUHfhV)ow9r^D0MR}l(2Xn(1m~s^vq|71^Lj)dh^6q3IF&rHuX7t zibNh;#9G)tbaXzo0V)7Q;@)~v;F5}#0Jm^3w*U8Xy(*oCuw?LmdTH6|J-40>j8Pfs z6^4ZSZg*{upfpoz2y!x5PyJZJ5jlGVB{9lZTH5mV06$?45(lL@LQdM$fCW!~tfWpm z3XG9K9rF&U>l_5mfCFC5de^AUoZO+sNW{ZsDEiC#ue%E+h^ZOALaf3_o4xw^VrnM1 zUJI@8an*UL>O>mqxx+6=<>>_s#*YhZ1UtiiC6`;+$)etx&q6p}8qh~-V_e%glt>$n zuQ##Zb6;0_KO%4*>T>;37=PW zl@U3ksBu$#gc#PpZqAWsCe*>kf=N;;DocvUq*7i@I3Up+k77TN3mm3O0oGn$sWmXz z_vXsXx10M=dv}UHaj5)8f~QRgZy%bmZ4Mvn7JqT*oOK87d%7>MJO1Uk&}q%cGia;! zC|14Y@|cb6M?kK-mc;2_Fzt2+}E)X%{CjZ_Y$_XN~f7^L{JT0fMBy% z458$aPHq4dZ09dD&Fv^=BeLc7jhlvl9-y>5e!6VxOKM@r8au{ zJqABv_#zD8y7{%X8qXuCq~cj}H1t{5A1OCTo`{qS4Rp|EcEp>svJoglyw!}f8Q8t$ zalY=Zo^~i&@8Pw7rU@s6e~fGtU%Od zAe49_ut)MtJtjJ>AGG86(UE3dt8OV--SBSv#X7_d3Y2Yc_Cs}g>^gkn@#5f zy^%+7&xqr|^8lN7%h8qtv|~!ot@>NX-EL8ul?3S4hdbZ&^}j<9R0?EAJA)jJeF5$x zXxOu1B6+Vr_kLSBfl{piZKxr8`+2hbAyVfg0GaJ`6}oQU$+19|7frE-yN#+5cEgGC zv-somIC*A}Sl$`A85E=<*4M);9}pz4`fpY@6^$S)2!LBN*=Bzh8cqKBIRPmxmB-3N zp#D|_eeEOfG;RE8<;d80136SG*OZygAblAAf(r!(V_>GKdj|wAvdjyS+^)nCT{);l zcCYbh8z(w`iK^u#=&OjZdIw%h{gK}(4jnUtED5>Yw`J@eN>u`oAMPDEPCe?RKt!Pf za3;0zPjruqnp{ytX?wV-gCX^26-KmrQN{`V)qu2 zbIuI7`XL>E01A|n9GZY_ozXbvil2A>aKD=>czf7qUg}Le&UFl>c1b-rwKClq^*2?K zfE@=QQCYfQ=j=j&Gn@s0sw|i~UV0SWWumx-x9zZQr@>1BNm4?Z@kz~U2noy>PXG^IZg;nQPD-*H*Kl*^=JnKR*M?~ z-si`Y&0CWX&&-RVPM#lCk69UC>oQ(e6^{-O1n#o;jw=ig`;SQZ zHNe=H_HuDYU8I)mBFMXU)R^b_(r2@g60U=bB6CC+0_^@Da36rzX*rs8T8_Zzz#2y z7`?D=6un`A12HB(o?BfkP~q2cd$TRr8whGVHsHkjKUu2Y9&L z!W#)6M^mDNvjnaiEqm0Amm*v-4tNF^zKSAwLHQ$~wrv3W_Pn$@50aMD9W&FxCv-Vl z^~7xlT7f;t_@WzCNdY0dVyGSGVA*(yL;Cb8{CDaRi32xGw*Ktmu^D|0VD-*-bZS2R z5qu2sLb^UNE@46pF`_1?4fl>Wg1d5W{TyxXU5gvtt}?m7u)Q)WVdbHZEK4M1V?Z~5O>_kR8W>M$ku!u=gAd*)~SBalXW)) z7@!F9A6(qIt?IWMdr(;)2Rgz$`Dt-aO_Il=NH#ZtT(GbzEyG<8#o0Bv_~VOdSxv#q z(Z_!0;z=QPd4HBLvrxt+04fN`&vobHUQrnqK9q4p*$%|4*gc+#o^y-^7}53p^`3(7 zD36L=uf;RuaYjky0+g7NK)U8l<%^9QenE-rAZQnVFH}wc5Y|D}Mc{Ndj-)PyxoJxj zN}35FKm2k>zc%Xuf_N~4qN^@T?Az;t6DaNu4bY+LI+M^NOx8pAIh-oYGtF%k{T}!(~1-d`bO954C$;*Lv>OZqvVW7Ala41Zp^k~%<%aL!`Y=sHZ=iZ&(vyqlhs6I{P$1Do zX@eq2>_5%gdV#kK9p@3~i|s%3z4||t<-peou#j*A3(xEqOQ=G)D5(1B+mQM9%S3c0 z=1^aC*hcg^&Ix5|ps2Fensb)X(A$PK+uRS@cuS|RQYk?P90iG*Pu0s!(}T}Z$rkLI zR6aVY@6ju^+cT7Tq`mBnTDI?YmM!;^4fqZ zOMn1h)${D!P{ji{)cj|kGAjFX`7jhxG4TKrXj1ReSVn03A7@FEy?u6F?2-_o ztH5fGi-|p2J;PmGYMnIo+de%5bbb>Qm`tBKr#gxD|Bl!(u0hJanV9JPoPu^PCiX#9 zRVP2U>gENshD@k0aoy?1A!>9XSFI^X$S z#M6>36br%GYdo8m6Ht!|7)qg0|F)w#&&>8T%68KCi!~ES8(z;iQ4mU>GT#)CwZS=p z@`a<&Ou~V@gw?6m8>S=bg(5)~A+SenWnQ2Bv+%nukO$2-&LVuPPE92AXb00-2`W!1AdS?Na(BeVp?BBy=Z>W4eU7hJ`dVY*mAR&wUMv zzq*l*UA0VXZb8f6#s-Zj&pKOx8TN%`oj4ZyKlCcJ7zZC&w(P-|k>1o}WklMV6152{ z;zt)thbx7lP%tG@c=PIsAQjYz3w45wnxgfgj^|MJk@Z#P9?X8(TNHx0+yH8)a|SLo zTN;c-ZG2D{=?9bOA5%r{n95r*$)wqu-qZxXtF@IGtRsSDH)vCi8s7>-DA_={0b3|5Q@RV9A_p~ z=BlERcY>3dQm$orJaE)h;czyW4b33-RYwkX=10$`M1rF3&sZE)mbYX$od|V*aq{w= zVNn@AJ4DZ@Wr%i%dK0@`D~y(lzx|7Uq67k|;zl+%Ta@OPwIDv>#7YU0 zXCI}u-0%1VN|umsI@MW;#`F5ab6rq5C=F_07Hu8~@J99HPzo54YPe!m=8od51M>q1F#_5=0Uu<2RQ$*{wZ^ zcIf~ln1FAYL9N<=aT3{0RNWl~?~8HA@GHB@6&XZl_x9P8o|1nhg}^lJ|hfIEyMh zaH#uwV_z{voD?@w{XR95tW6I~i78w?e5q5o9E#5KXPcUSXJC}LVYkUr{)o|v$?S?l z1)3zFK257|e?5KH8A{jS*J^3}z}B+-s`sdtgbig4Rv-3Fe)Q1|F$gr=@}Br*>J6Er z>5@<#)zF@K`rg`MAJ>^21Kl^3i6Y+veTUHoL4}@SU+~Uv5*M^8I&`Lb$_yT)JVe|^ zzUt%v=#>MRC*h#>GCWQy7yMG(w(y|i8q~t7Xv$8WUZ(4~48i)4pns0VZ{8eojYip5 zB>d{6VVkYDiT(w2q|kDEt}BsNe@B~ogwGVfoqY07y3NJGRNV*(eK0ZVoXEX7s2xI4 z%ZH_1xASIR?P(pU5aJt>?V3+>jz;w% z$qEeual4(jR@T^uBSFB2hSfX^i=}u4D&J9HgV5TRMLIJ7hxW(}mN0XL$SBx%2@;|z zptuY1eVgn?5IHt-@uw4*a2=ii4XRZ}j2GJOo&1B#evd}Qnq{RD@%X6SatH&$b&OK- zi!7z}qi{pV2q2fDNtW|T*UH$57y*ER0+vOBi)sTWy0@W2fQ=gy?X1zZ8tG^pK#-o_ zNZpn$K?(_fCnrRVDBI%Mb;$AnREdUt93xLVd@A90tJwn}tHb3y5Vapec68n`PDRrt z%jLwr50d75UpjLm+LJMvoE%|$&OcFgh&DL@Dqz`?r4@P{X#%oIrn64;;^xJ$Pl!cu zs13XB*s=TE#SJfown2$X92ne?+&AgD+0nk}W9VQ_ooA4>HXa+lUx@5b%qHv4L(9bO z&f*ky{)@T4ubM&41=G*O-(~sF(gvbK`Yn zL;_ek>jyc=7+e(}67EKovE%-3c)=FyW|!2iS_I#Qsbu0ok{^NT(@&!Xn#Rc=of9`q z(Xty$iCn`txztVb^Nm$Yyi^gnH~=AZaJ|&*CL=*lj?Ki6fBfO4UEg=>O9fm2Ef%aAE&sNarP)Jg9vxg;E|XN#N>JiHcbDe^U*5^ge( zL^&EFJt@1Yu<*{&EbGO+^)bSNjf#F(9N;8`6IJ~eNj0hJJhHxw2;X>P;ebDRTn9~B zRz}Q?zPRx9#U#|zC@k*e%zJAioLwZ*e+L2^MpBuMQkeoFn zK(=Y6O3HG-4_~HJOq_z+`|k*26;+V?rbQC0u+jn~eUIM|@AbEQ|@O@~H^ zVii~>u{FZHwcaDvRWo={d-JjG{hE}Xz@apewOl>WaXIR+VM4LjcGm0$;s|#X#gFv2 zos6ZyUv37h0Ji$sn;gpG;twY>ZnVgS>yfml=}E0-5^-NGgH*oq>0qY!Zhd<)&kt## zSQl}rL+`p9!QK1yr~?kAxp4%UiRK}+i%3+E*K28n_-_{ zA}y>*meLf{QlEMh8D&tt$*8`s(OBMys4^YwmYE5HuL*@I3}mL0S4Hf&u$?EiC!9LP z-~o0I6WT~}EU~l8QxS70I;Ka*>}>t~v_pqlXgUbCn7ouNb4CDU(g9knV%n1S>}{xg z&(=jO{kqK8=)V@b@o1sY@VG6>G*Q-=ji{ClKA~rw7Jn&oWO6n+~XC@VRnpA zm7*(x4vx9u*WD*_Idmjk)1hskLM8T>StCWu(B;fvTXPnu^dtIU!ATwb;>y2Vj0D*% zJ*m8UwMAq+DRN!fI9#L32Yxx@@?cDhJ7uvd5D?u<9nYKC4A)A9%b{n=j}IM%@$|>2 z#@eM@#7fe#JxzW_$SnWsCh=`)xz~2zr;iUUj7ndkXbd7R)UvbP1kaxM8l;c34!`P* z`b)}9Kh&?8Om$)5R?wDGMy&b${uqh^+`SV1DD0ah#wxVsUhENA_3V-B#E@E46-!J}q)Ntfa@FD2XSR_2?B^7#6L@fll za%MayOs&BOfi&Re(3sXhn0lQUkp+K1Q&PF3aIJmU@C9PU#7<}mULHvI;kV`?ooNh_ zla@+T_Ip3eeA2LDC`qYZ0l4r#+vtysy8=W4>QFAc|BrrA%C$|Q4Jz}8 zZBR0RJV$;cyc5W&Z@*hUDM?WuULc|PM=hs`^F$G$s}cupUxFz#VCvpZb+D)hZ*Gs0 zr`~)m5eMB{@XjTJ1JH)IT=mS4)g+v!&M!z$>Y|hE$~AhXgC2q7IM2wUrt>x>v4!O! zgqLSw+96M#E9&xw_*}wvZr!p&=tvj)==Z5%)IH@xQEt1o|yJxEsK7bM1#TEGEm?`WX!zCBTD0-ba6M7>C z+TfCix|Gy}simjLjj%p%zRVz$->K}#R6N8@ir9`u`uLj;B`m?atv<^Jz)DTAOZnwN zs8U8KA3>w^p?h825P#wiuKVkX$>AhIH4Q}3lq|5FH8<+|cP_lk5c8~uoF zRxp=xW$r+pU!+s4-5)7cqw;Hj^Yn4I@XWD9%K_ea7E3ujzZrz5j(zNehW=y&g$h!B za0W!h1xsU&9`aZAJOBJ)t^{MKU+pXWtPHX%*Q;G5WCreOzwQuap_GWqQ9y(j=Oxo( zwki2n@hA#_zI<-;{x*^;SXDVa%_Bs`BTua;o9YlHmnwFE4!*L($)}S;THY!3ui{lE zM2t`AyRx=>C`FiJ2RKlqer(6PvuT0K)>7p}#uae}zC`;zFjiMu_{y(AX?)-FK5Kd; zng@XM@lKB_9GBg=i6V)0T)b~+&u_sm8kNqdf)W)?v`d{HUvjuF}_-49U)$xYw=cYZ_Uq&5NHvo|p2>tkzTx#{Vj=LIH|yL0ZtMi-Wi(v?>ZaTzRB z+lM8uA1k9XI;}jne-&M}b5tJ=rY}E54I6Jr=FZq-YS!w!W@%T2werogfM`uxOL9!q zDBi)&H>scKI}rJgv>y>kww^4g;Nh5q1veZ|L#dmRO>l^H#-#myX}rC{98l5{Kslup zf6vbJ9K1ce=eS-GTQ|+B{E0-5uWSZW4hk8Y^XXOF)R}vtsjsoz2TWbuE5|2%P-509 zX$i!C{H)5IPNMwj+jw7oGDITCE5fo~$CXaKDFi7Yy=$~Y^Wh!ppuM@cw&Xzv`&jBk z#7U+T@R0N={mtao`xewvae5KEDerEQ`CDa{LV5Th3FNP1tE2A!G_92LzyOE9sgmb` zqp6XAMC&5v=w~v1r=|g5fcp5R)1Bkg`RH5}QM2+;wW4M-FNo9s6ipB!lc-Nj^SWMG zpag4F5;{c43cF{|2z;r;I8h8r{wBaACbJfjGS}bH;_pl#3rtdG4a)RqCCmMz>GM7&b2RRf^Cy7{YF>}`vO z=|R!S+Me7PdHqO%GHRjxnH6N~g8RKcJza+=vtj~=fAYH9t@?WEq)9DUgU3e`=|G$t z8j}YaR=8F4B-K)3K`Z%EFoXDT{JBap_E6bKr(8gih{htPpB#g$A~}x(2;~jz`JbVl zMI*ew84!9O{Z;$;mJ*p-xpND-_}FaLvAd$yYiLveZ;fl4+=-oVQs$$S!-7Q~ePrFb z(8fG86Xp><*XK*xpSv)_jcT+wj|tsMQKjfZPi~qM+-AL=l9}59@+C&~{O7>T(bI^`6BYOc z(imI?$8o)Uz6K&V2-X>B9*tnoPbp$)iVY`CB~rHg`ouoMKVv?ctS8uO>l&9d$F!p@ zss^BH8X0Pn8-REmhp^M}tpENSTPswCTb0`Z^2cf4tp~U4ch%vQ{(s$9k`2I(Y1 zd+i%dG!-2%QT@MP(N3(5qs$F*N4JzctK!9K#{#6zNOvxNvHaVO{Pb3I_&B&lacVCH z?_xq_uvWQq(~J0`Cfa9b{Bt-(`q`l6IazsR=6AFT&Jm)8_Gy0MKvy?9XW>fO*79w^ zmn2Lvs1(Tb0XR8!air$Cit487nE9d0pvgDF-zGpy9%AS~EjN#DR7Z?V9XK zb-T63t^wYaf1Q-lPzZE$c$4t&kWtjJDq3{6bIi^&iI4G=+#tC<`~i{F%!%`OGj*#Cr`5O=YOxLC2_5#LU{)9hx!9ZwOf`_?wRg+xq^i z>HUoKl;5L+GWVkV1H9=$iZx_Ke}w6}4Ri?mXJ`|(xd+hkGo1&b3)h*Lh{~oS&WzOx z73R5^|H{7)GbscUtv6_;OrQDKq>oE8-hymb+ZlwpYv@d^?}o})qJkta^&;$MsO)}4 z%sXWDaNhL@Jgfqbb_H;>Hd9+7(v#g$ahQ=fuzne$f6obI7*ztc`vQ~Q*xMdUwXF(w{sO{@)<%G)H%* zrE=JFO&op0qaHmysaE$X3ufOrY_%El=)2B^Ru8CWhUds1-O(RHJ5HlN?hCdV;SO20$_!=#hC{Cx+sGR zN(}(g)ZV;&>$bG>)F#D1D#?1uqQHQ@+2~Dc=#gzYv3;8&+&^rlK7{b=>+*hS@>FU} zd=2FH=2uiUP%5azR_~M&rq3=8`U)_4I zijHc|IZQN^yE%zp1_j?bkL2@|=D;y6;!DDk@M!NKq~@z#2CT+cJ2odu-#Am|F_YaS zN%zOMZg2NNGJIkYt=gmIqxrF=l)Z_MKF9HwJbgd*jO^J}iZl%l4j5dNy6r3ZFOWKF z^@9Ti(@RR$Bvy^0NMzP3*@vy=u|kJf74%47R;xogX+PDSNj-{RDKkwM+Fd^JKpTA? zoT~LPQK^lgPfk(K0C$drLK{_Eq(@kgLd|JObsl=IjPfeP5(}c0r+QQU7Ma>3iDth_roFXl%Gk7;%o}J> z2(+tYFF4paTZ!7BI2!(qCi;yJK|B6i$fR}y@c($Hojn5@-#Kh7ma;ulNh=vN8^4tf z)?G?b7XK)vt6THGT$a>ibrr|!Hjg@!nI}n9H-0YLn=Pw1nqlki{W=$nFcuHonvJYITrlbuL zI$ecTX4qS-gVYH0V`AAmy(e(is#0eKu{4qMt9ubTClTR0t|OQ9i`#B)i28|isMt#( zcu!r%@tI2_kOvI$vixjsi$Cuepy}p89Qm%rZjWkDd!+UxUkY)VId|L*o7Mz5qzrlUKk&wjrKNhrX` zpbzPkHgKd`$TD0Bf$(dM*F@ik4kFPIk}G7|>?>TJzd>33r?mJ&p&9?LY}5Hqen&@z zbMaSBjgu?;JN?nCqjQ8L!{nKZ@cLVds}v<_Ijf8}x|w`dUZbR7dwdcUO~?Y0AJ-N} zQ{N-^pTp#@qVj&(Jr-%&Fn!s{eY*ZS!CgonDdKpgPC+I@W}~tWNBKDjn+lBJ znsKOBQJ6C7;IQ^p7D$5QDefSTaB;iJ&(+I&(hDe&q=<-xbPL}f!X{FB5nmu7`HQ?8 zXxdInEKKD9Q;TSmB`;4#v}0XL7!n2HV>g<3bx!LR{Q*n)>4bVMR>y~&mUlM}2Xmgo@#=E`_MY4txW%#kQ(Ks2bL*F~Ul zj}my~f^KQ-gO;c;7VfXab64u|sRGP4HQ6nJ@0Wl^g`y5^*W|Hslc{Lu#!KYyPmR+N zn@L9EVZaQ7%{uRc@PJd)s|xf#H4MwmX>&%iI|OS;I=<9aYO`3ZXoXRP<#^xDrI@P| zf~nJp3h8V7f&#x!M3{6L&PC=qzNko;d8i}qzUzn}aQDGUQD9ANP`x)B`Rf<}U8Fxw z>hcd&R;noZ9)R;g?R~ptp+%_vz06>f|{2TeWIGBXC80x9U;embZy zZ>{#G2j^-gFK*{BQZCqxyy zLTN!Zq?GAj8c9xSPsW?&I@CR{85sC@(JB`mi34k?z`G`R_pEol-&99Z_nJOJlE)No z;?kIN5EKvRdM$ak`Lo8Y&GV?F3;XpT<8x1RnCBjp2xJ1b+pb!xQC^P|?ove`97y`j zO<^RTTaKz=^b{4w2`jXwq57p(sDASpwCcPaOvV41FGI+~f)=)~0glqfF=e4-w7tHD zq;h1$;!Z}c@jyiB*`T;RKUc2frnS*A^rkR(~B z5|U*qqOncZ$vT7?%<}xs49{nN|9W}mc|M=d`JCl_-shbCAt%r~fe8t%X{v2Wrf4>R z2H-3E$Xvw@z?B_7?`JQ`qvJLX*w%_~GXrvT*j!l7!7eom)!8+$jRJJsE69U{s6ITO1 z|2N6)k^YGZW=OA%bNT8NbP6BG!mu|B!fw+1A5Tn!k#t!!B)^`=I%AB!jRBuAr+IX@ zrMxV$w6{VW*LB=_)5CNeP2j4@mX23&q5R{w~Wd}-K4!kABjfT}TfE068 z;Id;M6%nTypbz#QU|DP0H%h@vNwXFlIv&gQG)~{w5VMz%1`0&_{X<4wr9049VBp;p zUG^09A417q0B}g!bXwV%R@3A}`C{=U$nwX(wtD_<8QKi+V^XXQlHK#s#pVM@-8fwn^{H|MF|_?;bnJ_88PeHINEo%yxHZMjk~m4K#+kZ;Oi#U4-I*gX z(%g{rKF@@N=6LFv#ugP~j~E}um<7z69rnsH+8m?-qNdH4$0qy~$$TSx9ejvcqM4VE zx+2oiI9%Snp%K@gv8G5YwetLK!px))cfY$SHGaM z(ueVCV@8pKWhVVHBH6-B@}pt#x{KQ!kQHMCEYaUy=jyax_G#eWK5=0^D#@cDi8KPq zD%0v=Iju*>cNFqM&ouP=1D>LdEbV2#da`HinL$)JVi+(X^HN^+^KH*W{!nrY*^WGQ z@!Gt1%_i}P{|@@iD%NT^Zvx6{F={>G-m3Bf$n%9IX4|3{9pDQEwJ zRYG-#D)Kp84Xy^()|2yeIeO(u@NO1axHndn79#nh!x@O}zdlp2Me>Kk)ebXHnaWl{ zTscV}zN-RDc8@ihE{baaY!UQzAU)bT-b{EoF;eu9CrJs7v@BKWTME82Fr4F2Psr0v675Zvl6U!He~QlON)dSYJ?PvjVr`O z?85^o8Qvmgg8oL2VsX8*Mq^vlBL#>H0?ks#*$T}S`PH)ty$ypBZ(p6_BUt?@9moFN@Kb;t`;Zai5qM4^1-hVSYxB)lTO> z#NE<_qV%(Y-1gh7Gtt%@cciFJKq`Lah{v&{nX{!HzeKDR!6B^4$fgeWTqZm*32lv) z@ylBUXaGF<0s!AQ)hj}HEBa?5#u2_Mldb0LsD|&AW_+Z|HW8Yhn!dRB^Q|Ge(}@u0 zr+s;^*<*PUc{{+*XwR!THB=-GoTX2m29JgDBsnZfn(vW1N|2=Z?b7*$R()G7(37VP zaPQKGW8smRchP0l1~WR^-Z@uvm#`KLKP4f|3L{OICY^6TgGxjkW(SS$nH1Kw3W;fG3F3O6YpzFGMr^&Z^{J1*W-&N3w z@BHT#V-fji^@WfX@~*Z^6Pya4U44uAI49su7^PU8`07q*Oy~z_9(X$5 z#D=%+t#q^=m)T1l_piHmr#~j1C_Z3lWnx1$!lU`yOXSc=Nm$%dtMj%p&&havVJy1X zyC}^-19M0G9q9D8LE8LQ7+(8)qA#{YKtMq@`;CWE`6k3lsX!=v z-Dkp^Gvn_;!k^d+W}N&i39peed4i^8I#h@w=~mqlFV3GdN3vQ8&IpzkRmYRj&SBne zTvxM^;X_0U$96;5Jij;Gt*5jZsbyPW9i5@l|5bM;+;{d}Nc`SdYJQW&G1#3F=)^gxYaG` z;pDBJOXwvA#2f7Q*ZTLaL6?RNm!@LHaj}4Z74hJ>?|Z}YvvgN?aL|Y)aaDzQb=Uz> zkZCfC%b>{fUWK`&P3Oi~qhb)4ay0lucl1qqR1p$uFoo$0lv;7uFUQb9`@z(y*>mRXO8$a| zrASaIk2Tj8IwwdoqEgoYe2-?@`t^qGPDo>bdf3C7cWdbQUuHtrqMmIlB>@pvGPrHv z`1A}vPbNhwjnbD5Zi(zs>}{ZuG((YQ!{j!7!NUoPV_Pvs^c(bCcL7TmW8G#o|7; zGaAt+sqM6g^!sn!Gg%~gtO}cVbKAm@O5s1~$v>(vQ==tH1%}9tkTC89kHgBwp6cBs zMs7h>D95!(ULo{x7z)eD&|Y%~pX}>#5_t#`^zNqfBIQB4tmn2GtX1t&zN62bHh4Xi(CF#j_O@v5Um+k*Dp zojcaL4b`FnHj?y{r2E3xgRsYBpie!<$5*~?CfWefCEbh_d@(+%kf9VV8^2b%PQau_k6irwwjz?O(hBh$T5P3RQ!Xy^GBUv#$)ZIv zZ^_C|p{}(|$;61~bCht$N}+`6VAktK4l5H`OYr`aeljn%m}-D{2t*sD4vHJ_Y;LbF z&8&4oGBXHA&~L22ZlFe*Mw9*HtMHfOVGy|J79@n5849Mvk-}1GE~F0;NRim zuvYX|ZQzX8HU0Z(_AT@RIO8}QBkeYUl;9x!eVEsqqqvUIRjfHg)D2>D`9jWE*I07S zZY1G~V6}bve>JNF`^0%nCq?VD2H~OHXXh7EA+m;%g}PR>DrBM zsU9RESkG5yU6kf;q@ErGv&Zk38~t|Fc01wzI~m|NR%NuNEY*xe6{q+G&&O{~*hiJ+ z4?fr&@#tVk{XwVyo-Y4db5J8A(B?`_|L|q zQRL0U+E3r=4<#oiclV7YI3+lCdV5w?#0c^)z_>DhU3*{G0$~MJhyg5EVY)HBDyfC= zDFFLq-Zf|JjibyUtSqLHNv3kxY#*2UiE@<=EcUJVG;p%%8!AYC!&A^=0sn0D*3`hG z{=PG68iircc(Ch9b|X_2NLuD7*ro%<9$!7v#Y7jCsA0>&ZdKZC{IQLK&a1T<&u6(D za7k*E)yi4v3YZ}x#{(ap??M7_7Ao0j=UwaHFGOnq#OwC8AWu5;wN}@afQ^_QAE*&)cd<&ILOMufV<=ykNMR#Oqq6uG|bUr3AL0%ye6@jpUhIu-^ zY4;AY0y5?Fx4rby`oQv^0jg^f(TjD#q4svP_Db4Id>#p^ z+xPeXBi2uK-~~P!G21)N>zzU9OEw%%s9JKfvLk_El7j@I%-uB05|u7gHHRvO%dd;83zcS>Ec@kb6BY?#nvpHlF9_4iZeb}7b!6SX_ zvR;ykwfA28()RV}D~h2SFZ!g@=0F~Sg<~5y1k}X&6sx`lKPPm% z^!cJp73-dP>(JOlMj(q$TeZRDhd|JSGzJV9!D{U3cNiurqD*dDy0moD&XAfG80MUk{ZHftSC)!11boE8>1^KyoAp zcaWN!QKr5v=FliJ+Bxj27)A;|n(tDvdy$kNC<$=s<-OY93Qt-5=*~qClGh4~%tspH za@M_QX@wrHzEiWRQyX$V}%FIHapj4B0c0GFq3C8^j$9URFU@*CE%LVQU*1t3MW{INl^OZi^}G#6 zQf|8R4E6#CL9<_U|M`&}=#<$&2he_LyrhL9Lw$j0liIIRb@ml17RKS)ioS&Y=84I| z0D$zi?@-z6m$#76O;8^>bh_iDDV|h-v@`QR<@7{0i?OW%IhxXD>O3#H0tyOc> z#ijqu?m(8Ft1Af5dTE&OUmkY{IxGyPE^FpujcqndCYBFAf++K*)6j_^Jz0_hqVazW z)vHYSPO8MUB7-^kTC5Nsn@G$t!hwo4US(kYtyxwRzLYQKh_sTH1uh|@IZ60h!O6m5 z<9G%AKFH5b=ArNlZ1V0V55v(|nHmUt&1N*#4=2@(Y)e8XO{;Jes;3XA(;M9-CoLVw zTO2Ji!6>c{KffL)j#|5_mu;Jiv;KJ?)J1o2_gX=~udCvAKUvFPO}tq$O~bW(3deZy zr7aV(k6mH04fFfPzVVhL2xRv12; zo;hW{TgHWw%9*kk>k5KiJnh`^SSwG;qLT{VE(g4E%g|3Z4_Sf;hGNi;&SuQ_Eks&} zp`VGx1=e-ckmc=>-KGHT_hX)x;Ls?NDF*cC*jp-$vmR=ZRo&qlRamax%HPk#49W(| z|Acy!l`D5%UxaWx7&nCf82!b{xH&y^vW^m%)&_-ECmrJok@00L@tcI*Ah#Ex6 zxZA)LM|<+~RA%61tNiv#E&&5fhLw$-wsC@A2vVK8l3BriCOND0-* zaiM`iUD%8%E6kA~L*1wO_+$1w#mUKSLpVQOq}_)l(D3k9u})7F6;i&em3TJRhxbzi zly8&>7R7%@GT;mlIJdT_6Y)cJ#7$rT-C+5_ zn^f!%RAG0wLL7K_Jo5M9ksUWkU4DZZ^(3gAyx`4jv~e>-d8(Ne;%XBfZCZ908VO_r z+`Vk?WmybV+6FkA?)@Lr89EI_dVvC?!MuCS_U&7QqOx4DEweNi{CDsO(Vtw#qQsPTeilI@f6=36GUQ7AyiL5dW#Xq?bsbShLyTMO^g0ZEpq zP8gRZtFLN(`*)rGOVY_b1M zLUG2riQC*#d`}GOAoBpNt=8KOzH|f_W3Wp)TMmOt;W<-Qz2BhiC8e!Nq#2v(fWKeK zzB$&!v69K;j7-KrTjR|ZwF#!61Rf7uD^TO?`^ge}%VyzN#fWm0ZAPbQRq+sF8CfB| zMbq^Qe-(l{1ZY-aTKS^jIv{hsF!_kaL;l^tE^`Lv&4;K@v9IEHB!e@4EE_SZ&NKl*LJIZe8Jz#X946bBQZfg9w&Q$F{ zB9kHMEuwx??qyYz74oPs!1?@{Gj=i-2ZDYdv;0MBS!sZ{ju$At3+k6c5eOG};eW^5 zuol!7BELeS8~0kAo4R3@M72R>XSLi0)Qw&R#Hl79|r4@oC^TMJdMR) zyeRXM1uzir%;uvEZ4)HsP(sP7e`)XA{VEqd$%Z*;>Cc;+njRvnF9(S`_amYoL+Sc z^}Y|d#+s{uULb+U$hil;3o6l{0G8BE^56d@I5q@D137rFlD*}d-0=+dAQS0w#T zdGZkUeacs~$b3qChy)YTYzYzl+*c?wo@=Tw<9z#@41+E-K& zRX`2hHYlF-T4C?JeNF{!Oy9n8uv)dr{}~VZMVO23rjH)BXGw{0Qcn};X7%yj%L~h7 znyh)mg~K<32N~h5H=bG~YZXQY=H7U})zFH)NC8XcwJjjMnsrn*gH+Ph zj(f!>k(yI|+e~=HJytbEbP+(Ng4$KW??&hvlEC(1$9sCO$g-e-Yu({Ct@!boRZFB^ zC0r@J&{w;Bc$!FY17h{>9>2=GT{2JE9L9tRUY=L3%RqS%~KwcaJ=HuY4cfg(T=D(jn~8>A6jWA-DktW@Wr8H~N3g2B8j$;eL)} zvtP>Ml=x-j1Qt2x(2eI;XoIq}7!HEeZCmuNJR6-S9hy#kpsBp*5h=-XN;CTf%z~av zvwZxV<&c0>LpZyuD|-EfmFQq*0l?kUP$p25a_FULSZ(m*_ZPa=E4mpTK+>QK@UT9Y z+snvDh33HC8@sqUYV^F!`GLNC^EdR_Tyn5gL@&*Z5xvE&>-0LCWtoMTm3CFZ^xR3dANilmJyBt(bN;r?Hyn&dm-foD^2H(Trz_Dptbvh+NA{R=dt<3cjnE+oefIoZ`fJ^C zpMz-qyMO{`me76JOCR+}n+c2PX*O!rAQ2V`DwC5ktmtvMfj$VbTjs#9iaP3dWeXXV z*8F~fcn5Czjm4YC=fF)@=cLOFEx!DHqt(Xv$tU&IOsyIm3b z1-tzf5qwRSOoGSqm8_g)Xs{Wu?$Vd+|Mp-m5}Q>H*yrb)rl|QGRY8UZgBruOs#z7@ zp1%0M*zwuJFwBSklCwFkD zn%Dm|bVio6G0epK4s~~Fu7@RImIPmUwbu^ll&w8*@$bC{ zbkyT+&q`{2lBi^GM@cENv|{c8$2ivqcm6JlndNw>;>LW=^aTqXm$+^%(mA+C_oMNH z0)`)>$JdW<@5FGze%SZe3#|Qf^2zo+xH0yV+#BV~F+-#9*l&By*I48;s4WH}1g62H zOuduiJFMmqt{<-DfccdFcRhc_Rl=8N#Io4;8XjKnkIyEMMNB>u8~ortWu4rd?QgjY z*8eSaOktWi9z&N8j=SI~YsgqoAdg^4LcwepCzFoCZPi7LQLw;OrxXy@%tcN)`It9yC z4ewGxW(O#E=PoH{We$hP(rro6A|?E%ipsZR;{W%@95xai zQo6kKx?}>2G^sP}#$pB<(FntUGcUD$i)Du*^ANac}W84h+YEZjDVNat{XRQOf%cNIYhV|Hk*(W1@Up_bRb-m;=vo$7u1t zzWHZXA7uqEWUwH=uRQa8i632zhN+74fI-4u97{Jv zUDt9N1=?=YikBw{)FRR6nE*=kc5!vSzpVm-D{(6aEGH^$Uy#pznV$7?z-#I{_b`*~ zrHx0VNY5{P0lx1(`_dc(6B&w51!9HAKGq<4BmG{j=!YI+(h?Jit+kI^nR*ZMz*G_dY>w%y4x+pXx6L4>pG`>ne%8@0DMD zwIw<(`gPS?&w>y!jW5DHp`Y+@)N$8b0xE(%$)Buk= z$a&!JJSF6GL!rIDv2YD+fV6L);xV2hp5;`?44A%B$l+KHN&OSWQ#{YrdcwRrznXU< zCOAylWn1g7{mf{+#zm`zZb4HTns){}H6o~$@w@>S;~iv@yX}AM&ocPZzyTXNUN}>2 zoK8XZmhpQg*?au+)#f%sLa>08M{)W3Xxo9I{QbrRdw-}xoGgf4T{U)`xeR$_+hRoC zR;sJ#7;X5K*fH(30!KDC`!A8lQ9?`kQ{ggve!4qgMx%`kep{4@9eGonxNE^Yq*N)V zc#aHJ#_C5dO2o6`)(UYbDQlP6SvOy4v6e(lL3=hlz@HW_q-{mp9d(Qsh$&?myPVfW z&%!+4)lV7Au>PVBbb7l%-bB2c*GZO$6h*_ePcV>miZ=i&-*h3?q*-3If=<-4cT&=?u+}j}SBO)~nPYt^Odg4cq)ptVFg??gDTI(pFv=id?c+VRFR!7yySqaC(mBRS6fRQ^FymJ)A^wGy zKShU7j#iK-vb;PpX5H0;L54O?JpM0R!ikh>gr|2D8!AOi~L8)=` zhF4rKVkh+>zrXJzFRm;E392&K#6o`P^Q-(4ZOhp;s1g9{~HEl>6&XsoLKreyzAq@KVHrp{rvk%mvn2=YKT z%9VRuoXvhN_0; zZ8?q}bcQ@%f!|*}C97l)0#f{9;LMLqVxU$I~Ql6ESUsoypdx5D%2T`hYtnM zr0Z5pt}*AY1%hlgn&95`{Vz*N#ZcrX z>6_{aLS*+=cW$wKoUFJ9nEL7GjoOZO%qAq<6Es`Xhs=|)BJXR6j)ntQu368&LUgRp z8Z8wjFWh|?DO|LWCGEW;bqCZOuy@a%yD3qu5MTtAzs%fLPCyjhROVg)&2-(I1ah7M zvaT%jtAj*}U<1^B%qx%0Uc)Z<;! zZ7WaKOA@#!DRaM(&px-IvTS8I zPnM254i{u^<#d;gGGgDxO!8AVoN}9;97qtK0Nb4P%%?QFE|r49Ngx^OC~_Z#3Z-?l zQtS$vU>2?xyS%AbhXmvy3=iye2{)ou4(1VSAm0^;m{%QhR0AIzm=Ju)P>g*w1s|L$ z3DrY}$~5LQh6CB#D{R8|#+w+;7x4yM6ooTHZ6J5+OVozz>OkE}%Eq*Tm;d?6-~DV` z6Zk@>1;AO15skU`&;MPUgh&Jc4(4Y)I<|d3x=K(;tp05C{qJ+EW)l2${4eMl9;bDu z(G!t!)PYQ**w&@2f|`JyfwcX#`TOGCJ`wXd1<-Eo7#BewP#MW-t0Ts*7 zG#?MEC&T2KGVyZtS@s7|xQ#*Al9^Re6(N0ybcI8oJKM)mGbk;np{Zn4N0EJD>6g$* zl)T1Op!XDibQ&cAQN`&H`DD&b49Cx0Au=)yz#d#jKHeD`OC_e4@gzXs=hStdvxq6U z-xz>gdn%MqTc3`yER1NHxSAt!&CIh~Ep6&1xfs}Vz9@UwHj%MhX%s4vkrWcMKKtLw za7Jz{;CGp6)?>XLOreak58^n7yqc}!tRTGyLLAC#0ZP`lJp94rFbWp1sT{I-p4BX7 z!O_cv3Bv)yyGcp&>dm=|wzmvCt%R|I7b4`*Tdly`9q%(MtIS7E4+GL53Sl3e*`lZO zkX>Wk9iktwT}AC{LeYa0xdwJ2M}wO0bnV z+3E0(gHca!B?-}&SZu`0G%eQ%TpSLgp94~rD2ViAL(5sC`Gkd)V0|_xr?{3cMD8t6 z>C98Me=VZ07NS^*49MGHS$uctW}*%3LV>=HKzr*N?;LOLVgk-#SM*YiG@ot%`94Kf zdI=0W^DkYCJLYljq3VfLP{%it4cy&U5@Q zE%4;loBT9mUaKbbI32Q%oEss=whK_chYS(Mz3=yqH+kqIX02LVWRC>Rzo|c|aj!Z{QqTv(v<QonaHNHSswOd%jg(5IzqY)~RJQW9vS+x3s^Dj(xTO)=B>UY=BA-k*o)U!Eq0 zM7G|4l)Btz_a9k*d8~rjWb6}RW}jyqyz=SJpUZjifAYR%Klv8$ZOfWXPtSFoyulkw z=pi$E++bi#Rgk@IT^ucc?6tjrXO&Pku16OV4E4hT;}^{&JrqjNA>Ulu$E3@p69}GU z1V&qH;qOW;*CdLKYV?o4WxRXJ8Uc$YA>lAeX(f^*B4AT&^d&hy`$ z%ibwSS%?6vAovc~@fw!E%gq@HM^v;(gWHtaW^s%d4i3naxUITUJMFxTWuOU1u3=M` z(tMgsa0M72d*ypCzHspxfWbV~xz3Y>I1pT;4ExWQWh1kIBNt!&!H-%Atpm9Tby3Khdovw> zC0r_8AHH98j&H);>x3PWc=gLp-sM%@QXnLhp(_YHoL0X&tG((mdLs!w!zqqc!S+On z6PyHbPA|(igO99a$+bSwc$Pb4DfMfUIZ-?A4g&%d!vR=Q8LM z#$Nee%G|rZED@bEBq0Z5t2gBMtR)mI1#DABu~mUGorwH8g?S-UcbyV4M9Fho!1j5bY4yz7|;V5oOklMqb1G#d#>Vp zD1%}3P)STa&nebQ)|3HCoRd7Zp<>fl{JSis9g7h;tLpVedi8l(p~JSX5bt2|Jcs<} zBKn=A6W_dtg`xf_D~K(@AkXr2>=Iuiv%^WoK9~(IXmg(`#Sy84nQ%H(|LazPveI}e zpsg9sVFSmbWce68h(%v7I2BfLRu^e#Ht;su-Y0C*R7Q{>H1ShUv`lZGqKYlTRN~r5F!}RNK1m|+bV5cX*Vl`0Djw?}LB^B<&egf%@fDGwvN}Zkr~_VoO~-F{ z-B}daK&i2>$|vnyzfHuJOAyS_>F(pSSw=Nslcdz?kj@p-MO%`tBBT$@iEnBBFIJYm zs3wIEbyylkYX{zlej(H^ZkkE%uFAT+%Sw^(QC+Y zj`tRAYp!esm^rA%@suVf&}(T})*l&dUm%V&PWe}RlZB5jWy zhMZOYNS_9?gLr2;&@76*PYO+YQJ0Z!5E<6F^P5wb5&y&ogYDFO@p5HSSR$dgDUcHR zOe|Xs*UcSv2Ug1Su}<*L-q0KC0#6zYJYJux-55(PlTAth)^mG3~n6 znfCU|p^cP)t3!R)Z5cWKQaPdwh)rjaMlG^G{5$BWj$%NV z#KC+~ertIr6Ukf@xXZC)TWVgtA~uQ&fmTOJcxIIs5f3xw0!7tgPaQK2x#9&eq>c-#(c}JDI{;O8}k60-Kf+||kLs6|G@=7HQ2cehw zUrU-)rET1#h!OPPiE(ng^Lhaa1ECI@ZfxSPU33^3MoAfTqM0!I@bLH^q>lmFO!nrL zHq$FN-!Px(yJQ6?qi0Z(xoIgnEU1E>l_sz_5~ZetEI#1UF+YV#!3Rzfk}sLcK@Y4r zr(hv#699;%Z8_8QwU(RA*$ah6NseC}tqgKylc%6Q4`%aC9b`BI77Q1uwoO)Ar>kA(JgBQBI{+%G(-p9MQg+0IX!5U`Re`J`?Ry z0@QzXGwxL;8lx)$mC@d!b8p`!J zv}xxT|CH67NwPwk&IdL(RH5G&L7HCPDeIfjOPSmCKj3Xv@}A}BP`5xlW_>oMuIqgwVM54oGer4`9!@slrz5|P z3_aIl%q_?K^ilDjq_jCJ#ZtXu2nBKyPGSESr~K^3qv$T+%20%&ZEvsOcMgdIjIJ@LC_88E;FlfbJ5iU27-n{1}L|;%dLchM%0dM~G@CKTrv$<^70Ql$E zIfwewWc?ceCDrpeU-nZ2ogNN%(%I~MAL)YMOya0u#IECaR;rax1t`4udg@^b{eTP| z*GVg=GfCEnICB6pq-l93CrfV0O1qL=+2n>1U)ra;vNT~e7;%GBJASr2Me+uaUk=6J zF+w>;mhOUs2_Gt{swtK6dI8@XvOj+tyUa(Zj3Yqb569B}eZjenOa%@TYR|e}it;Rw z@%w-j>J!|cqGE?YDG8OM#F8DZ*V^HNwl@Z&C|of-I9(GOA<2HBUByH)lrdM^9}nQSk3 zW!%f6=3DXT_n}5BGxzJQAKSK}ZBqnnXeHIv^5Dn0Xzf7;_%3#ABmT|{t(^gvIZ7?e zr->F19zL|W`iN73$p?G>=%;pz!3Gj?DHw1AKYIF?tz@AMmH>h7d8#viie#E#Y9{$r zn~_FYs+NijYlQneSN-7ASW{&Z=$M9}tx2gVOwDrpKwx~vX5&vYVEN+`s$B9dcDreE+!=VUMMb&QjjPatW z^n{QT8Ip0kBAQ({yFf;B{oG?QsRCpaFZZ3;j|#rZCGJB=_# zF3^(gno>5&h1uOlA4X@f1)qmf%IZpxeI*^^C3Wn6-6?pm3hAv)4h$^DzAh?%q(Kjf zWgrY-^Jw#FG22b}5*S#W$c(PJA%;ZUBANany36(&9%1Znjq1OkbyUg&gh59NZtP&y zh3*099S5qgR?nv4XnWGc+Hpk(2D_x2aV^c)8>LHtRu}$rpiWb61NvUzOuKn@rM~%p zU4p`N1rJ}i=f)2UQS%=<@H)L(jNXoQHlsQbsAWB54bPyMJJ>($&Baq&>*cvbwjBzwEIe${f-^_L5)UuMs6_x-_A-CT?IR z9yM`&j;RuE4-Y`r=R~}k0F-3+sC%(+-Kx94&r0K`#cB#4=JuA+O*1)8qDW8pWY9X}_pzc4D#4euEnX!TNV*xb2r-LC$n{OvAfCjUOL{ypRO znIT8|m!5*ZyUOPTH*V?A(a&7tPW3HHVytkm1MN}gTUHs}+_q%xU9 zrFCu9g7~XAy;01Yg2UhoyUQ6=CV)@r1xAa%x zLm^GY&tGESCCJ$*Uff5Iij6x#pY`yI__CytVGI6W1NpQ-k?H9w`_uIX^6cFVl1-1z z#nnFtrB8A$3X*z?`6_q$kLHoc%Ej++oowt{$(`%HBCQCXyd4lSX5#9NTyk}6M1_+3 zv)>|#KQeMenO^3*@YgF>5uY{&OVG(bG+vU6mVh4@&aAT3Qh9;7X%O$k%lqhYJ&D?O z)9)`MrGt9Eid(qg`A7Db2<(;6k1=11EM7fyGrM1Y8m*Je5sOnhr#mWU#~f;JllU+L z*xNq(D}|3GKKe;$>9-rmqMnFyZiLms-2ypDBiO6hFWK0NMh(Rc=QJby@#)Ie`#`z8 z!^tlAU(d9-YYS#XjE;gpiPK+;G9O0mZvUvk(U&Zqsn?0A35p>fgV*DNi;yegd+ zC}?ime~aGLc<+Jq8;_UZ*QifYJOlC;$X;GFDj3nGXe5>|wn8$%T_+ZlQj*)}E!>(Y znj(qn-r*)C+uoNa(B`Ws;lZ=hn|>;oTb#w;NViM--6$=Q^lZs1>aqo6UQ#Iw zS|{#F4|CP_<#9hAncP+FlF=w%y8^TJzP|n=ggC}8!lO^H4!b7EKPI-C5;utLKe_Ac z|2Q+yfzTVsl{c++r(;>Gh~2{HHA3f}C!MU*GNHO1jBS12-o1-Hy8l*~bP0_4Fq>t4 zT8oKksv&yv3&_-k^z9!X=%8hluN9x&QCD|~We=0ICN`9z+eQ(Mmu9uD^H006&D35h zwl5mRV{HvKW7&}h(eg|C=-C|ae@(dCiH&0zXJdALPL2^8u$LCSjd%oOW;S-$Bvr-w zP{s%2;RU6}Fm-g(Tlt6U!=_R9zbM~B{Ut1D_KE%C*++Cdz zk~wbg^e?e(iyOr;J{cL*L&k)JIJpWOx9th9x!T8`&zLfy6KsO9<^%`({s#N?Zzpdt zL#~Zy!m zze?Sbz4s=+4BO&@KU>6l=XBg;RUe(@ja!LB;G!p)l|OU0wLL~i2eo39V^{e3MbWLx zgfvs%jmkA1ty!g)(i_|A`On066_3&{{b;Ng9dOV@SCZr^NE%MAoxbfv^5o~U?=+AT zKATScYIBefaHVX_Q2&fV%g}qdpvli~ZWJr|W`|h({To_iCmg_w5`CXjFtKh@>91LY zzwX%N>~@kmegb`M*(iDUnp!Qdrq)+T7;JUK$D_=%%QyD!KwGQVMjsKp=h<7QHyoe* z5_Vt%*&KJF3kUur4!PliAgLyF&hnt($HaMRXJf{Jmx7Zm1BlIb&Jk57Z%)duWvLLS z#10+gFY(Q?*f}tdIDl+NVfYGWU!1`!%?KXzdTYE?&ls&0?<@BcE<2%Xc#>EH_rvIp zByn3*fyX-HJB)P1IZ+-SgGPo$C4g++)N7!Wzi9xbbUSix2Tu&la9} z^692X!)X@8ersl9-QxnqG|MZ*Pc<#V=fux&H|O2ugb^FWrVpCWHu=(&lcY_2M4p?# z@tIXv^jz~i*|&nF85Nj7L+A&D`eTO=QeV5DWPZxpLYzA3O&{IDZ^{f0cK;3H1!S2*-^nvC>naf1D_NovFwQDvoX^-@@BmU0*Qr^+yorm?6CaJ;Q`Nw1(SySm<#@%B3S8t5JQsXm&2#H z+~Bt5KUYa~PdO}l(eoOqC;1MZIsJ`Ttb+XrB%XX5 zj14%3EGhE2)3CGzp%d z4Y&^`!KLTTlOQ$XyvV@~Wcg`xT*X~gf1|awG>W~`?auU)Q{;#ngKcc1@9*gOls~Tk zombd*o_Tmxz?Po*iRkNY)9}-LtAKQmjoXM-n1hBAzWL8G-~sVc5{OdFTzYTZw||K9 zdia`;s~?`}cGQQjKp229FaY;skDra*B6}rld3I2sPuyyABwe=$`TpPaR}XeZ+((Ni zpA?3(-GBEKI`Y<{pF1s#Pgyb}Y}MzxgaN=emXKd{{Alq!#|b<+d27fwX*kAjY}usZ zL0D!CY~;JTkc|cxekX2-`daZET|w~qjTyfZeSDy8^&3J ztS6INmK@PQ4$%x%vKQ2zAr@-h#tXwePcE^YhNO#L)kh!jzIA{UagR7EQc)jWsgT~J zep@iM+}3NK)B~lw8_k=eyw=(5 z6V-ySA&ff$yi}*EniOoWH=D4G<*zxo)zJjib=vFJIV@#kT4)3(B4r6Z9|)9p-Y|u%S!8hCc0|jfNUzIa@29X@S^3& z#{i_O)Gsm_mv&#@d&VB-E={u{aqh@x>4)Uju65Dz&F76tG@Vib=ovSWU|39?znn&PD<^X zHxmIPCI_E?^ys1-hCI5>@nAp<+!QM2i;fkiy#r$;(Cq5Q}H+FF!DXD0B z*VV_!rm!JgstLC1?W+Bk*hXwsHdbFdzV5StSxf{dY`DDQ?84#x#mJGzz+G9f+V=K5 z%Ui^gxGBVUquTB<;--0ofO9zsIQzgE>)wyvyU~3ifhHfXIXx!uMVgj$N?71OV592M z(X|fAOyFzL(5U!lN=_cywGvpuUd?3`FSG=5#b{|wZ&Y+!T;{JxvCMVBMNJ{{Y( zY~f_!6}C>iRHXfE5BCeq7oH3Z4YRRj^{GE+(E^D-;W{pOgfkp!4k8>I5ExNM;ON@U31h zAG)$VP0%ZcAXtcFZ}=^kvmzgfYEaBT2ZW|#N<)(_@&T}r1~Rs__d4S%@&W3=#j+fU zPw;N=kg4~}gVc`k>Z(T%0+493j|naA`j16e%vL0>Gyb8EL{2*T*|~6R23o^j7krjM zre-#G3Hmp1CzHcHzBRgr8wAejpK6hrlO($p_k?bGsnw*(3lpKlVsj_H6)tmum~g9jlF5w z=z|4NM!Y-Bl@R2s95$GkGLbjAq|+7k>spgUcq;gO6j=;NkmWBc??T3)J) zHlhy!EN*CsqW(ZT4EYLlW*_V^5 z$i{pZ#_`72{$nO6&8#>pMN=FB0qMW6BBeU98a+%0LFp~mRhh;lB5izT3HjLD=HQyL zX6jA$FH-Cc-(7;Y*UoipiXwkR@ek$AC@sN*9Q9vr$7-|=9Mr4msoT`3Y3P73xHy5A zIcol%gv8=b(2+TGo7O2tp|2_-vU7T(MHYP+G(osj#jm}#B_LX1^Xz1u>t=e(&2D(l zIPqlK{cF(yPyX!u^dy=SQ~QVBp@;{EEwa>YRz03^@WGWD-Jn;&#Nb7R+`fi~Z}%bg@(pW{hr5UAol>JXs_6a6N}LkKf< z#n$mpRd1Ym)wUM)PQpu_iPtT{O^ec0c7-n4I+Y~75)e1?_x!lH-pkHl)#wDQYp@9a zc6eJ@s;zJT${D5;nB~3_yt}`x)Hl2G;gv8c^gs_9qPMt?hSBaFE{V}Myc%1LR^ zU7q_fL;txi~p?i+SAEvlDX&S|||b zYMSiZQE}IGMxsbsx_6bK-$u%!)pfPcb_e~{D0Kn($AuQ=es^q~@V4relk2~_2w&l; zDSXv0v`R*wG7}U%{VQY4v~L^wJ+SgOJq^RZAH70s6PVUR+820lW7UE6-6q`V81GbXI9qKD6*yH8?0idy6HD7|;1}pY~;cJ$Yg`6&;0bmd&P7 zrn=YA^5+FJ*tMpNK<6ib5l*ZN+vS2U(en29{8G$5FmO?daEQPc?Wh_1XDBFn@;ew{ zl(EN2+O&D#ri5Fsm87)ZNpQ}adinb`>+WAd=amQ=(WK{H?h6Ibbraj9T{Ox)Q!w&? zx^p(@4NvJS%GZfoOjc3MenflEMV|sDH@NKe@ShHAB8cPf06+80+3oEWT8D^_10CTW zOE$KA>~QaWsl;Ws0-UU6Nz_BnO5qgrTiaZ4JC>(`=#(0IrzQkDWqDTb*3pk{CyrVY z=U?Hc^_J(v6UT5Gj=>(&yLWy*`U)z5V!P^`%DeA8o)lop)dq6YOiKD!pD{hOb^r*J z+}g);#eHb)SWp9brh4;$=q~_VCq??O626X7mmBlEwNx322v{M`ayDb^80UBN!2ye~ zFimBbZ3(j;eGyBC&Ar1_O30ntvKur!(;7FwEtQI4dI_xIZ1xsc|Lv3ak^%15DrP}u z&K?>e=Kw)Cv^m81ySsu`K4io*8>>4eRLf<2XjI&A0||(y65jsVsDIjUI(Sf%0s_{z z;)xHH^*+$^z=UuK>#)2waW*~{g_CgrP;DBI$ftkHg%e6W(SoaHW08g7Ydoq>%Fp+; zOF1%H2jM?2mX+oryh-e_N(meP*4;aurn!G|5fTh$BsAS*=U!$gvBwZjvSxT~Z>Eec zCLS~a8*Ka|=62qK1-=si?l`!CDgL3;#4JKs0XH=tN;23`*RX}ykEPj|k4srsoxwHY zhtvQ<8Y#=7e(#GTu5Jj(+M`ESZ`$>8_D>31>{@ZwEBa5HSBB`8T9jaIpSyAjcni^? zFaQtL%8yjvFcQ^>4m1|-JhQ+%tyvimj>)ZDnT^euU9OyvH;wrEg|%eNJ?(Ml4nmsQcL(Kd zZ$JFr`ESfrqzJNMXBA0`!?sP=(ZjETu{wJ5NJ19JdJ`OyRK0sNigz)3;@y3eiI3yM zRpS`apU_0a)@E~fVdiz4?Ny`v(C=e!Mg`LrY8mIg|GAmi%9dbkbEewW^l90kwS6PwWwSfA0;t%oi+quWRW#KYq2?X%9Rk$+DLZCHCVX3%R> z_URX8Fbd1$BL0u6E02qDefu0s*5ZVuWzMmMkct*9bLc2VIwVnQBo(DaY18r`3Z=0} zsX-DYA!%?e{WL2)*HPeVuiOgdfKQQldu}&y8FRTi zmw(BFI>MN}1OU{s;Gch5;#5>`BcDQn__;=ELjEtu_*=lJ*XpU;k6dL3eNG;e9O}t> zDq|;6<=LL_4v)kHia>4M+Aif$xAB_H;R#&wYIX(|7XFF9nkY}CTh2c5(Nr!Kd1E?g z(pAe7hnQ-BYybWzu`-Jg-w^L=zq|$uN-5N$cNeGRO+N;;qE`I@Gydomn_bmE|Hmm( zAg-;kWf}gLJQ;sdc0PGUEIN5o&$;kC`T*%T>J+D#MeW&w%?O~F^HztH206>&Z>o3; zIGtM6EwuuOIDhXx2@&=spPgzMZqDnFj+-`!bDpM;JiL4!Z>j}&&&cMNm0bPQX!KD+ z13bIcgW2&uL6Y!qc@)|18sm2N>RG58jLm)po6tPW&%5nw243a&XQ}4}qBwE4Yfq2Y zcRTdmAK)27?%pHbA4a;V33V&4{b`#tW0417ye%M!EonUcTGDR%m@Esh-dA{k+n4Obp3*!(6;kx{*2Ok)|4C%yAA z%Dr~T0)4_Ti&&tY8uZZ51C4EZwx~+9squm|#L2%SA~_J6f#_q0M6P(AOkg2QpIj{y zKl~Ve*nP~pyoNK0vaYy27G#_s9CMZcDLM#c6xY|jAY5ede#O9kHAu~rOB#jT#}*U3 zh;rMZ)<8~x5e_NJAe{P^Z2UxbspZ%VNGQjRcU<6dMH=KcP}HW{M#6&&q}H7!TwEOS zgMcSibsJxnN*l$FfA0+QQZHfXYE|_}B*Y;s4NSz3*Re{GRMs3MU!*G}CY{^XPUjyU zk7e^LR2+_&^`B>DBD>5C?`4TfzkJxgTp2ABAm?zIpz>${_x`i|6^Kn_NcxyDOuk_!EAFB$XEYN-Gad9^Z@P$!Ln|~EqSi$D4)gCQ5fSZDh%0G7LG;L6L34qx~?$8kc>Jks$_UC zJ)WkkKgdp_+(Pn8UJD}0y3A&C!)~aM{SF~WX%ABPA*nc8$rI@f8C_3x30ePK;B-g| zZ@(V6Zfh@C0&v+X+4x1rq%nopL`!u&u#yIBs{4W3)!Kr+JyB!VnDpMtSnJ{9GxQjLq;2GhUiwJKn~1k% z^=Ko=i}h6M+8+z@7s(-UARsR9V%gP-Mi!wny_RQrsEdZeH^7hoYU+kJtfL?9*&5k{W$SMJAtgURXb)gbQoQ#^kNcwn@x{iV__re0*rsx$AMlu%m3c_GsY@SYxn3- zsON*kNn5ydecOfwNOFj1fw=a}pGNzGR6o}fz@;rGJHf0P`t0nADaLv_nrpLaq6<*KGn9NUoCLjdGmKR>nCIL;NHVy3T5asS2RHNX3Qz+!Ht9L z(j|pgFQe>dDNmGMv5wt$TD(RY8O_yj?Mu*+BL(}Bonx_^NF5i^@cZl)XrsA4m88nG zgjr7p`U22Eq@YR?RJnC?Q{hcCQ==Xck6A9xu6g2zo+Qf+NN=X8u%t8R{+QBGTMEgm zg1~&w=LFg+;=M>5LuSd!nnnNe4WkUeQv$Jd!3oANb8z26^Z`qd-}tg^v9-cT3zV#p z2ZXW_$>x3S)$zZj9M0Adm+Jb+?r%AVj3oiNa&?YMvYPI-<4DvXI3HSSE?sYkC5;8C zkkUFQ*2g+DdWH{dKX zqqCth&wDbOzFQRWp~FN|u<|QXe1PIC8Zr*;diVS#8ncfM8BMy8cZY5^e!EXa#_MJj z(NM78=eeC8RRO7y9Gmgus)kBer8IU^0LIysS;Xg6s!*}@0xZtax3hfu4E;57=usIq zrP_~|?^Irb(+);VuZ6;z1BKR=uK0BvN1>7Iiq-QCdTN4^+z6w{Xe+sSd+l!G#BLm$ z3n%Io{dA8Ede`xi#KpQBB zjEVimD8e}B_Lc?1!kqa?KM02s>O+qgpWLkq?N6e>iv>7?yCJQk_MsYDq=gW#ehB>f za5J@oJmPiqU}j_|j$9 z#H_s=$F`XTG^NVLA+et_upCVWcGriHB%RkI2-?oVn45`SmS}3!)z(AK(dEt|EQ6=b$>ac#!svnA7U-x=gt`WO48Z^&L}}P0_qv zo!j+yeeE3-a(TWS=2YYtt`6w<7cZKvhS)*%TDk@Q@!wbz0O|X>!zZjZH-8ZO09$qO zglNV33l|1wBQYwKC-2(sVKnA`h#b>g!VTiL>O<@^c|>UVm;s`^8dj<0JF%a*>Y*LY zf%A+9nR_lizir}(7TjWuSYFM<<59ox^jf4;`+E54GOF&T!67f5 z(KSEt>u$`39Z%OF15E^#uoiU=RysZx7EBu3W99^&p!D>ufhyfRLzA%pgZoBAmZhEl z;ZVC2S!fvtq3nvNrdY9y>%`8UF^(!*CH817=(NuvUxyNvA#(*OYO)CU);4(LwcSp-s{H~B34%1tPn;LwJf*za# zTY4vpSIXc?WS+vNCI<^9oM!ek($O49`4Z|azmXI7Q>u{l-oV8M8vPD27Y`EI+Pkn!!7;Fb$Z%dkY#GXZ(#^3;5ZM#VUPa6$_0tw&A z*wRl-yHH7%JRL$e_7`=MF&TS4D&VgvR|A3zO*UckQR+pKe}3lw`+)V5jctV^%)p&< z_REC;$$bDZ{M(hq53|JyI~6j!^mP?88NqB@q|3b^MP7evJ623XYbFa)bm*o3_A&`J zK`!KxWJHn0>N;z{_FYQP5?9E7OUQw%OpD2co&;@+nNKPcUF)wN`=4-QcL}xNduvsb z@{(k1o-Kgpm{C+UH8c*1kp(cqYOlJS(~CwGkJAR=zH*0%pjVr?$@K67I1VW0=oYVhdXZv*LQ#zlz32;7Eg5H^$G9{b~+g zGg{kFWu@cli4BJVG5r;65u=rlT7e;F!@EiNo9O_*{dmg_b2G*}H448y^69sUtKA&lwP(f3;Mb%T$6CYfM+-U{{p9bdMeSxCDRI z6O2}WUz=mSHG*!4xOysQ9__5CPJ%}RH=y5#c~V%G{0JxhKuephdX>4L_gmN>_;p+; z3yAfsHS_cEBu3dHN_Dj92Sdmec*MMHE=49c#NCv86iEnRH)qWYdS+`Tp!jhm)O3`^ zMjfe>YOx|S;g=l(tOgaD`eTI0kxe1A*jz?nt8q(-7^w#^|2P3WN@RvA&UHqKf2U~?8Eb-2Eh^ZAhmVKLL}!IeoSh|n8_ZMD=CuEN3r#n#cN3%;W8BidXmpP zzKG+9N=IShe$HH{8T}Y}9fg_m+X7X;RVQx3dc}c4{@W9_TMT1?1u(WZe|V6G<3j6w z95Mbx2Qq(tb$oVLOFh`Yf_joB%o0Ti0p#Pn=yb}j+?GEEmvabFmWWP1Cy$d0p`JKp zMcnm_)ETgdRx5>6plN?BdHUK~1$jmqupFi(QqGAprXtf#sEe124colkI8>C`GEV_N z#HZ}^b2}9E5)i;7#CKnu_c3BxXee)m@-LHI29lSwg@0!IS7l`P01WJZ+1Lj^5{J}K z1?J1YGHv|ydbFf8kWM>uiv3f*^CV<5EZ|oJIQGF`G*wO`0Wbp!#j3ujyb(n-)HzUD zqX)doYt^$r&shOkL{w(A_2GKnY%GVV@+3cgXy?kQQ&8!F8D3AF{qpLaJknlL%LT4m z$y}> zyEqvz$()E(=N_&E3j-?0@^+h5Kl7!~B+kQ?0&DlT=|LFbWC2yk=7)WPRWjBa284%N zY)a#YZ_nk>f;fAS1B0nOjmrnO)sp1_*v4hBroTJta=g$)?c5R-2?r9I2nTZe!Ne^4)aYGyED zhYBi$N*3X|Dm8wm(_%DFyC`DYhZpO4v4P080DYGGrMAjtV^*Rs*zLdj#oNX5a9CN1 z^3UF=E{QgHl(6(1jx`RyO)^Ac1(@F}r?d_`Mn=JEv>f&&%mbx^$1b`pMS@icgxcHN z3%?2$peroE(oeUcGH=HzG)!fa;aC*dti}R+4j7*Sz9`E)pY-lSX{{ECu8s^w^2W0x}=7h)RMzAAQ)sEIXwFNHm6A*XhvT4d2Vf?WU89t6I7mC+)@5c z1+0*A(B!de1^Mp}kxU)V*_fcK!6hFAr-;f zhAIFwAb))M?034@28F3i$RT(US@ht{TS{1O-C!fE3XFzKwnzYbX2me4$Kcc70gj_B zD8D%^#6Dqh#nPi&-#(H=!ZYDFie4*i%wNszJcHNW1TM06-JpHs9GQG4{!$Bn#)6Xk?H%ALRk4+&%D^>biqQfOi zR1Uu!oHX$EC*2uJl4C*=(Mxa0E7(%v0{HCjDi;Z?amJ0|otL{>RwLu^8frC1iXU%^ zsiB@mfSD%n51+nuQ0AE+*GIzI7KY`ORNvo(#$oM2K1=`7ZI;TmL)Kvn^oGdje_7T1 zAQ_uyX4$Yx!5mgWKs&7w&(l%3#U-;xeW8boKi+$$rufNn?PH2@a<~vW+AB&_o=dpR zScjreg3&8OmHp~gY%O;TWujE5aGsYf=-6w6aAQ&i)WL~G9IGhsLP1gz0Peg40j7f* zcsU?m5nTFteHRUh2N~IGq4fAcR;i_=tU|^GkzcnY3SP}z)rG6J%=al|J=Niw1xvMR z*U*O1HQ!|B*i5_Dt7dpGdu7t1SgL`hultvu-efQVl@ z+I~7lx6=@Bw=kJ`y(&o=>kWO4_^aAZg#fjV)^R8bI0wy3aS@YCw#MOwCzg~@eS2-g z^qsHb@R_N>=4P%7@M?Hx5GEN6RLno#d1>uufgyD!Lnvv_F%au!oA2jVi!UDN`f6(dK20e=M|I1&Z_fRhO@K+Rv+ypf&Bc2exvd)>X@(h8=K6c+y z=$KDXJAam+Hy4kD4%yW9#Oqat)Tm#AJ=&$ux&pmheZgDl-wt4N7zd6ihwqrqdx&fT z^*EG77|PmzjSTO;68ifGSc;gsKvNJeh#54NOm-C{e;%CSTNdk4c;orDn-TUz_7atJ z(#NCwr-a(g%+afASLS-mj975;uPG<*FaQ46_x$i3Rz?quw-+==Qt zXR2SBuV86@$b7?;xn`5ptqz)*rKmqEPu>2R4@b&m)bNiE*}J9t6YZneELq<+5|AE{ zz&Vz=!9H0#LLwri!vaIG!+!^ysd%QHn&OX5k&|6ScGt`ZG_CP$hHY~Hg$v%C$%$i- zn3RV2N!q1|yK55=YR~+e$79&bhHL$(iA915_~`xv#SRKZ-r>lC0X(Y7ah}Lbz0>5$$2oEfca%inD21HHW;{PKbp6)r*GL(% z9E9~MnW6*5)~slWYyq!_BNN=>h$y%gOHt);Y35!n!_4iTXiDjtVtcM%+=q_xKana!gYU&=zREpfjsG85)}o_` zp_{MHhBCIHA~@@8keFb%zqxKzj?yuIXQ5^E=NmgaQ{yip2kHsUX;k6b_i)|55Qev^RrNjgyO{1V^a)Ws*)eD|GzODi&1LhlUC)(X;;Z z$wwx^cb>b`)G;6n*ys+92|wv5^zM#f?kYTS42{N^?NbJ47mOJTe`s$e`B4onCTKl) zkl(kRy5Znd3Sq|_H+9QjCE!!NbWimIHPzvRXy2Ko2IQv<*Q2l8dNZ)^;5iEujrT@Y z1T&$1VJxmAUk`KM+w0_&W@}(S#%oR>zJ55e?bDr%4D2VEU}Ejq_i|1DxRy3RnQ;! zhDf+SC!ow~kUy{Wx$RmC3gU@nY_9IYzztsn67t9!15zWXs3_IZ0Ph3|Tq0GUdCzIj zVo<GEzvP^T4ME($k723 zROqx0d?q+!7Azd4I_P+i!CwTCKMm)iEu{kmhp}V7Z;8qw8o0l3HGv7=;#&eqyZCgy zMy@1_kd-H|pIYwrw6+ZkKJ}77+{-I(8BsLbE8$fB2}zA#Nzz_<%$g>FBcd+~aR0#d zMI*C-B=Uz@XG?)c+NLoTXF$3!Ii{9hvv=DhW>r~8&FC)^4O;-ef^cjaIdy9XUONKr z{4k^-Wa$BmdXz6SF92oaY(~A2tQ=UDu0WT(UwI?| zLVWd()L@nz$_GYSdPHW!>;&HdRf*L2u*g2Xqls}-FJ53Z0JP6L=!L0HM(_p=N*2$> zd4I;S8qbbpK1?V^6r^*f1q7`_Q+0|WKd49gr7x%hL+Emg>+y?Ay&D=%+w)6(u3?f> z8vqh57iUrQd}Q!m&28Y(>0I-lja8D~maDLup^9SPfv-0wBAaXlh1DyAz7>sWc(?>; z$igG6Q{R;^inxv^F6$^5y2rl?(%PHTOYfrr!S?ykHm1)W1=C|)?|`jyQR{qxUZ9sH~U^opz*_?FVR?$>tp z5pgF-W57yNF0}g&eaGZ-bC|->zmGIDE`cd%8~ypK34{p=RS!NYzx%FIGvwWw zz#+S^L&a$FUhzDXMFYt6UMHk4pjCfGH{;_=m0#<|(-8Ap?j@b7P}_{Ql{D-T#Ymmp zmh_sk4Qm&Z(ObDWC?LhXZ$4fmnKj~b{1FFAku!lU$9I&=-%^42s-viwc`2=7Pxos{X=pjfrRtirK2_FU z-2O$J1E7d|Z`BNW)K5dMWiwZ}4ko!IN|>{ww973aYBKTuqCfFyHG%)@pOAM`+_n(U zwGzq-c{i-rD|K!~yBY)}fQAv*n%AwWc=nZWUW}Ueqj3G}7(E(o7=P!Pcdg~?8MIe` zs7?GleA_gW_6VmWOvnSJwL`yrSPzZ;V{RZ4&KJrW&b8RAhUD!KB#>X0gmm2BkK|1a zl*6=1fxds+U69>7cgXGH7y#IM8VJYc4lQhIXpqDI_!(C@WKf99<>U%BchIMt5@FvE zDyiH6zfIp>;5Z`s4^N(ea`Rc9M((i@1Qju%`kvcFb-HRBi(f~Lhe9Vq_+VG3E*hLC zknvcpwoeM|0Xg__jeG5vfrq5sePdO<`DCU1(be@~Pf&JZv z4ZnRVd83}{OXli5YuJfr%7v^Uo%_zrvq1tA-)DhYWX7)~U4?;FjLxeN!efT%5=`(Cdk`Yok5auK;IF| zu}5&guv}7UKi-?(Wyjs{nU%-=os0@;qyhOi83E$|=@a`R_nXw1Ac|{=N^+RXml>vx zb^ue9MdyMcBdlF9Ps}{RCDWNpEL;a2&@K^RO*!Jg2rXtDDzT4JCT9$UEH4mrKugnD zQz*#Z(VnD7MnTBAk^soeJ6aK=Q-zo@qm2%Shu+ZK+`SFa^r%p%3-%O$36n)Z6vvxO z&CAO?&S?urV}KZtV>)G)_YR6pgT`qv`l=R={gaTOiy4e_dA^CK)&1>m~nkn{E5&K!l>ZwYXeaq_CU1by}TcvKkME3`mzPvlIZ~L zO?XPK zgw(+;>t<}E3@3>L;ddC2ef0)xWLNI~`b(-J6l;`$;$798n>8sPX8{BhT+|PYl!Yoy zmuy7oFi&16Y`ZQt#U|Do0->E7lN!!C;sFz2V%)-aO555n#w2VMbPXk{S0wcIUd2;| zv-K?MGFPyp2b0EPR2o3>GV^woZ&kykB*LnMTAE>*sZ1oKN&4j$uvzJKk*=j%-Xjk= z3Lf5LizZC4!K0;tW%iu3i{AM&f{Mm5vxMsSp|PV$l!b8dbU^3sSFc;BGK6vm8d$m1 zpqha={{l%(v<@=I&~zJh+UgW+ftk{5N|1U^O-3TlkZYZU29&M$m9F&w`Tut@j2OqJ zR%C3CPo?u}5I&L)1*g9Hj-xBoQPqzKfhbFUxAtw1naG;Y1foq^@NptS!50Y|aGgGE z;7o|-b>Q%fFe;(0{{F84{jA*>9HzZrSip>WNGV*5IiV#{WJL(Q)1~VRY8_%uWpn+H z1cg0HE=IwLwv*7YCt~ta=NiNvVny{5yFR2lyoiiDf$U*9pkXO;3XfJ~p#mK-i%n!! z-!$IfTYz2##Rl%@&gzohbjD#s9$=m9Woc_+p9BDIsX!w(yk|EWJvkSg$L^YygF7)8<_^I;)3*QljU@n0fL}}$)@z6@ zr_Y=6ZkX5b9h(?5dunM4{u`1$FVsxZc>SrxbJyxC3M+c< z-nxH3J5E);=YFk7+P#9qC(yraLMLcj8Tf6vVu znq>bSx9_pVc`}Tc+6E>oB<%Nb0jvu_;xg*fnkF;4E6!m(q3-uhFLQo)zQ{^w2S^2{@j* z{7sFfZ#PMBn(Mu^3$__e1@A?^SOv8q9<6d8JjSA6?i?{40#aT&G4h&+9M>qmdW?Vk zxY^f1B#2>!^;AqRSb#s6iSY$jY!Grfu9^1=k zxJcW{*V*^8;vej4;vw!K>*=!2GbF@DP3V*j`PATIj%meqFpG}Hh2LNBzawF`gSenA zd0rs%J#yWw^SwMmW683Xrc$t~S?4DGLQ?!WO{AKg+LsJ3vsrj3%27=JOu@_ThK_iV z$y2?Id2?7A$$smQ;V^(mcx1?RZe5S@79=G0z@y6ju)^sB=H7wh(j+#VVMR$Jbp+qz zVi{R;+m2#qlhsQU2!R)p@^08fJZPMp9T&466)ym{wCL50&!6@h1%Gr1s6DrG+P%}@L9?sv z4@XN#)SJB)9$)T9T2v>-W8I>EDUl()lb@;RAq_KXF1`X&$rjN~;g zyu&?;Sr*iDy>wbgl5NeMb!cx)fGtI8Hzu}}VgE)2?vni92Hw()Zb|p+N;dhkav|fC zo%pE~-Z7vbE)V^7`AX?|Y+6Q1gx;CQMy|dSOhmd#av(DB&2rg$wOcYgVEJ_C+Apyk zNIuERZVoEAGI}aD6JV_u9tEBEFBZya z@Hyu`1tYMd8QD5b>dc^IDg2~nHkWx=54cESpU{fBK-t3LW2G?nW;>=Kk1i>LB3$Y5 zN>U4C9uQ_Q<>zkqeuVo6_5eZZp~hdw*tFTphR>$@iBp9x5J9~f9B(KXJvL{on>eC3 z-P`f&3ti3%$+Z}PFnOO~17EC9owdYJJPPvamoRsKZ3$a+uAFiTC58l)Q4O!J*Xiv3 zj<-S$1d170@b`w9eMq2aZ@L*xyoklV+YpE`3J8^y%FI;wPlBtaLx^gg?(o3Ua|RYM zO4%@@No;eMl`^*&=L-aYZj^cdmUMUHJpLDX@UVJ*YmOY=tx>X|$t2^gBkYJ~q-0xQ zBfNQM@-~%%e9oCM|MsX}jAD+I8S=WE^HPiON7SGeP+GYC$M`H`q-JW6ig6>ii{q`~ z;b%W&lYTvXvX`PKPTjs|wv*(`Z-+UrCex*3kGaml(rpz*eDR~Jo2t4*;jEDv+iPJ^ z=n}y=Y(2xg{d@2wc``4}HbnVPFK2!94JTWCm9ujS zScINb04n+}ClKCX&ldY-s+xk#+5H#@14`kS`~EL_Jjq0aKbwmmyXgvGzlKuUuahILD~N~gD@ z5-y>@H9Gfm=Q`S2*etz`^hXhBAs?-k_axV#j8@n6gTJHhzq54a`Tfld2z@%uT^{OO)59vf8lguZ3Qt(; za)WlyuM0=vWGHkOF0G8W(tNrW;olRY2=?3)8EohH>xdr&=Q*nMglgrrhT|ZW3Bp-- zU-z;!A88#G%3!`}?;7&t#Ia#x4vIJvBvvIV8kA+2D4SsvK{sb-87C>G-E@h#e=g^j zUngG}kb~O={4mu6<+vC2y#YCQNAQMS1^j*#VbK9Z;(VU8K`9P>)&k=5k{a7=|J}H# zZ33RYFqiqke;2MVJ3=(LfVAow*?^JoT8 z2;*-nai;FZRHKQ|Rg-Q(E`9dzEVO2f`J^so@=CJ@dlB13#6qKcaMToLq7QA0^_{x5F;x9T|)FeMx(bY({c6^kl^BSwOa>EWy{H^>(D^^Cyi? zJ>+t!#UUj7hkV(7>0|h9Wv~OuFZ&|)m|R0Guncw+@ixmpa5$_N&0G_REp?CQ%BzK- zQ@=>L9sz~=@R`QrzV5l`n>Qf9OIw}({!zVz0^Z-iB|;YDtyud|!t3ADz++jsX0CT? zzhQ+F3N^1vVwI|*bz9l>YV<-T6e#~*@9{d<7>z^?*pI}+k_G8u^cJMwfMIg7^xW=b zJV!HSf?;)WtNAWgf^gi2)0p{-D`5?@u&MP`Pq3CWdjZ*x!o1Yk-?aA`YJ!EcC3uGi z*&{!5)3BE(&qEXY(W3Au=>)7a&T5tH zIP3*jP%y}2c}9o&NH7xUC#YKNy5$j}giIw9qK{IksdnpyIg=0o4ZXhf#PlVtrzOW~ zkWl5Hx4LnGVecv&X_8=!u6=Nq-=dj?Lw*uU9;Jz!vZR)lH{y0yQW*|UwP;ER(71qI ziU2AoT*mI^p^R>{rvyZFu0vN3 zPn0YkRyj)e##ooN>EU&wwlop)@rA1E^9^u_LI+)*x7GYz{C^SH<)mFq(sENQ zaHDl&-5+_+>>Tkxpl;0Hkmr{05PH3Q2E@#FqsMCWFbh=Ju0lzx7k`(gfb*Khn>coq z8~g_DC@e>P_Nl1gY*RXJ-2qsRY0Y^ZGB<_M5Qzd3=3i`9o~Suw`pIN`BlqQSvW#*T zYmq*x#}I!5GwmMh#7kM_F4Tvo1}61kj_(V%m$UHZM6MA#U9w(c;yHx5b-2 z;?eQ?01h9aun@@0m-fpUsg~4Q7;wh5{Z@nPLyO^9$+4QjohYUYKRm+qm>dd(Xi>fN zl>W-wgVjSwz0?4{_nW%B>18omxzX~&OHt_N4nI9CUz&)_b<>)gN&uYsWvHgwnJEW484J1@+rxNN&Q{v|>)^`xZ5pzJu zEqvU4@Xh4~NNb=@LAPjg4BA45B3{Di33aYCgIzgwxWf$2r>pY#NI!Hw7zac^G|`-# zxvih;*JFZGJzyBUkWVPBH7PeULa`{hfX$fW|1Fy*T#5|=@PMs#*0j8OW*%6IyiOSi zm#F7#kjZnr%jD!N!Xr)afOl1lAB}Mmvf%`OYP>aS%OP}2$V;B}jDldZ?LYcuezS^0 z=93qUD;ZIbLWC}Jgr`z`#8zS;ezrbhF$Si(;tG{{3DbM$=LgpdEi6z(2tVVZc7ylw zM}E6fGP0MdYsBIs3foLpMkCOVcCnYnWR~7|sN{(}0|8c0oHX$D#lU+_q`FYvs1mO~ zqV?nZ-W4HZZJ$gvHu-q?ijPC!CbTakgcoI59>%T@j5zq4>f&UDrm~-@>k-F|gu=*y z_7}7ko1Mtf(*WE>%PPT5Og(>qw4-?+V1z0D1@0*`SaZkG@JkAJ~;V zzT3VB=6J7w7L;7`F4ENMm}E=gOu4#nyFT~c^~kS}9)|k!b=d>EdGpE24McodI;w}nA( z5l@Ioh2OSl8cr)^M}9?<_XVJ4Xuvb@^?ID<(&+WnOlSMZ0M6*qt-l2eSc{JJTKNgp ze+@3cHeei-k;jvJWjgwF(X1vzYVhpT&AYQH_;sucy%u)W+;DC~y)Ir~g(HAr=ISEod5wpb>e2r-$tKY?=e<@=aYb5EFr%`Kw^HUFUeOtApPsD z6(nCpoKCb5FPW3z3^d`5N@klhjA7T4pYSsdl^166*~N>HQIi6wz?uF>lPn9_LKc*d z-L}Zxe7-hg92%u6ROA@T+VhrR7#GfbsjwJ2V_<`?xWRmRYONPTK1>+%zRa0y+NVzY z+i#K+(dV+9gtBEaD{pq&T}AJPvjT;x^QJyXu$+&c1CC>z%Un}wdz8yC4d6$v-t!** z*@R7q6u>!$#SP?R|D_++$G9ml&)lWvtVc~wb|9wfc`?6`7(~~yZ znwpty3SWGB*|ek?rYp?u99V5MZBwD3u$~kMYWA@9RAv-%1j?Q4oSNLqLA|n|+@{p3 zW0g6(1!PaYr-k(TcPiRZpRb)4pFJm*qLp7s-8S;<=FxoO!^}mQPKku}CbA~3w#}yS zsWOF2Ia<8g{cAhRE|Vzy$yv?+!Odx3#%Nfx!&YmA_g^ z2~ML@+Ab>?yOtS~3FpKgz0U@6+Nf6|qIoj84tJj)9^fX`QK$0zozk?l)s*mOY%_`6 z(3@(Rrf!B!)b{X;bx%Js>30a5cN6&Zp?5{40;$H|pSFU7c~`f260D$LJaB5Yc+caf zFTN*p+t{KokNFQ)DToBALzQ(WCb6w|H75!U(-R6GjL5*S)y$d-U8>3^{Xlb(+1bsg zqBYrb>OJra$-k<_Z^Z$(+vKMI>=keLy=Gx^MU`{P=UHozLmTm$k+U|DE2dhv+_R2< zkH;tjW2|wUM)Oeoj)xQRV5st&>!$F5k)$)&pe^J{f1c0qGg z|G9gHyJG2R#VEJysNCnOen)$)H~k18Z~b1}izyr`_pNd+$rGLv_=m{67!PTRB~)SycHwKa}0t?xA1r2uT#?MsD-s=tq;~;q4&N0B^BiS zYVo1QjMlp)PJFax$ty-Y73Wx|9EjmBkqqbw%*NN`!HvZAKhbO`$?a4wKf;Wba4}3W zo%A}Y>Z5DQPXlDJ^iVep3d<~-R%ei5M}n_d{N6&YC15iVSKYmDO~xW6uo7@wbG+R` z^=z)unyUXhIlG!i zk^8#CiKw|I8BSsfhw++4%iJXYEiu^pmu^zcGl>Ou4|A73A7Ui(j{jb?d{;AeMXr?2`^7IkT;~L+ zsNvlrPu(HjE*f;*HWKVMb3M{m(iFx`4hEHLs;Kl~wXuL1X8$~AW4dR8WHKwO#SixC zh6eqx%xuR3{WV2Y)&5`b+UIAm>?u}@O=CN4KP|sV=UJrwY4~f}tey<30pf?( z)mg!X8kH&o2EMi3GX1Oc{rwNp#lbp?-w z0Kr?oE1k%0`+7`b3OhmnTari8r1DF!Zm4bVW*q8QI2`;~1?xsqiYVj0QE^A*>)u}R zjY0DXzjrmVkSvjy5AOY;%15o&6tZesu;j78da&o(Sdq=Uw_yM5W>v|2FSJXYWNnYNow+TO zFssPXGubPLwYbwtpz`VI;HlU^$-XAHGapTOw!G%s#~R7rOx{GEd+Szz*EhWkPup|n zfCX{%Kz-x`x%v-SU%9FyE_OQ~G#*dBmYIb1fjr?#KD7Vh!TJ=64TCnw*`r6@3Wrx! zAUPv4TuH^sdWY)U2Hn`f68GzblGfMhfce1OPwpdWeotcE6R<=v$wUu~!RB%4*m>&6`)&FC+ zq^tWGngutqxrZ-cWnevMC$eAN@|fuzZ+aJRtN-$mESZ>&!ju-(Mv0U*o`XZ~xt;dU zL;dl5NFQ)C;nXI_U0Emb*Lff*cMqFdmmcPBK&wkmuNDXTIqvAYd>PM%3(}KSGN7Nm z_&vUD3mAnxX|<<#y9OoW;VmDrp!(gdJsT3n#}+qzj!3Q8r$Dyynj~0Y?i!%{cPE6A zrpxRx$Ff2swlmd^+Oqb&RvGz#ygMtsopo0v6(U#bR)oKP+?5hB5(@y=H`Xqo-&E8P=P>aN80R=(1HtF z@d&TDl5_M8_q#-EV|`~pz}TS{y|plLp=83wnMACwyziRf#?9Cwk#g0QB=7CzPj?z` zZkJew^mF2~vRQW@D%4%VB27)LqiS|G*)EwWSy&&K;K=KsD%xI&1z}isGh|}USLB8+ z#;QF%Mdb6Cb^WuNYA>2H3GPC?EMWn!zerv3;Z2*!r<0hQdvpV9 zeTw8_Lvr#fpt{6=Gqr{(ayo+9O>(i zh0T`iF{?E2yzifT@7HM#?-#$n;pQAH*tJxs;I+=us)u%@+A(Xc5#e8cOBnbuhQZi< zrtt35?ysNkZ2Azn(rmSl!pKM2bk?40H0x~R@gr_cmg{#<3-RA*vvJ-)L9+4ZWHnyT z^)!de8`}kHzV3UZ4TeLT{Bk}-*y(*eTPXBD*1X_sRw8*r>@PE-@%e|P>fqbxrq-8= zMrOB?eS5B7k}CMxGgJ259mX=gXwEUx-DOqCsX~3e>uEYg^-ST4f{&JMA9)5JmKmrN zRH{~SKE`a}K3daqx@aKuT*#2C)&So^#>RfKsMI6Fnq$QCe^hv_@RVQm4>yH#i7M{2 zCreaE@^!8J8#Zu>9V1P`b~&vyoqtFrXYtbd*(01K4;tN7*dN?HcLy127cf5RreFCE zP?IKq(YUJy(SPjh)tM|jo&4%Vz;Rb)!ML8jFEal%vd8^1lO8s1oq_cD&wV;GW^vWO zkLZ`|WtLu7c=PSALY;Q?LY-FluM7Tbf&V(-zwhdW+0RRdTPD(bU;FBp$kmpAoAte> zf4)y=P*l!L{gQ@oWk-)iboHz!rPu3UIlg!AQ2(w4e~xqUn(t#95S6paprm1!vSUvp z+~)@Oorn8s;69G}_vgI%K6(edCi>_di^_?IcU8bWa&S*->GdFb>GimF_3wW0t~~gZ zfriex98>ta1^hh*-cj%PzMHN7eQ0pLk03BAM+Bc451)B4BifvTCo1zDT09i$l}oXs}zr`7j4))SNi4d<~32W(Jum; zmu;OD{W3&Z(9#_0K-oHHsCaSnIQ_pAPf49NTC+!IhUKY6Ypisp$xJQ%=0F$qURc@{ zW^zjO^S;-(dsC$xa=oTFPl;$&Z=RyRTJiLaBK3eGN8yWOJHAL+Ocgj>^BU(oUCJTa zYf8dAX`@`PiQlK0G|M;7*Y8vh-B&I<^wOs&^tzhjan9+m$N%U|RD(BdYM!!kPIS?h zBK6&^a8tp`t}yFUqQ|>+#-&e_YF^hoD}S|O_>!X8yH_Zl);{I<_&t2iW_4HC+@jFk z%N37n!<)=>CVEbdaLDwU*fcM?XhHM*eD>&@RJ+15;U*QuH(UaWY0RndkoD-=(<7dZ^?*7<`m#l#`cYlh(<+++v? zmv*3b&5?ewy7{lJdD1UAMGjvUDV|U}mH60NXEM$CF@>5wG*$3T6Qp z)T!d1XrA9hjcC5_H8B)MuHHOf zkvE^3zHN%&+ug^MZBwPbJ@cAXF(u-gT=N>?3dIu{McZ{$6vO(J*_`T04iw?iu87E@ z@xyjHQwX@}lGpsz(@nme^O_@`6aA7ldXtp{Wv}c|F{@~N_G-n}=3beIzb<>tCu1tu zQ^cvI+uy8E3_A?>S@}6o8sKG$MdRNrS3I@1=5Ye) zXI+^sZM3I($`#q@je*Vccm9Owt(Ydbb?GssXO8qgPR)PqoGrbmTTdQSVy8&mdf+uHc51|}9IyE{Q%r6p zdCjo_3HN|Or(`~sshc9Wo#r*JZmQJn0$sPuNk=kQ4DLl(;;5; z$CN)$PxxT2Go|sg(FY@)8K!TJyeIjC-Z}jK?@SiC?f?JNzBsL+UHe<>|NA#Utj;3i zZ~pH;kI)K|F|DFf(P>dIp(Kqphi#lYEt<&(PjLn{-5o8f}BJ ztgfZ&d$8le#|Z-IH@C%GbaemUKapDMz8mFI?E=xg&lPV?-X-+9ef9W~2j6>oo9NHH z3#*fL{ORu{Ia+*W(3m-1MI~8bX;v?+hlWUQ^%~L0AR!ujZzV1#el{5OWX+N+lbCVe z`v{K%Yn+wR5a92f>)R$4vflovxhm)x;a$RAI~p$?iKzV(bB&}B$>^h z8bS_QXE?W*h>8cJXr#X)cypfp-Xzmwu+#Nz#;Ba-kux2uioz=I;?fP6tZVFO!k)Yvok}Aw_7&WC)Lhlo^|A*Q2ThhjJq}={RiA!wq~T%alZRk zzg%m=d-$AwFQ?G=@smA!yLb3Vtt|i4aV55`T6dk}N)a;*}m_}msVej zn{MM4=pHBiWAH+s_DJ?z#Rh4wXO~mcOihZl7*q0&u2<|@xV!b3jzDkUdEPF)MV>`} zyK6nO5h+(U2_}5|tMtX6>1opT9w)!0dGG-jnw`FoWQ>V*)>YJ`ppJl7(=#=W4V!8fy-DCr)YhKE$*Ni`69PZj&-siT5)@PvDV|<$`41pGtWD$h_^WH zqm!N+yicyUX78lDpW_t!{W_lyecZJBeUF#c<}lavtVb2btP^SSxuKadoyC72p047u?_yX=b?nrmI#SyX zDX;W9(z);99?vk>>a&~nY`z?<>ey!}6WKGK6Y!reUs3JEYw^R} zV3HCZPn?og{nlE$U{Abg^5cTY}sC2QMT@JM_P4Ltd*A^Gug88d$p@yy7b5R-Ad>6&TdP7snsx1h0u$aE8gF8 zr|r`!o5*{Us=r>}^sGrnO5mpRy>{Q!^r|COt87#IFYgr#Ei zZGl0%MT{ztqDj2kqwA+_^V7O1y78>yL^Y9>J25S$xr?w>+S=fo+HpUq`0J{N(WdDq zHqjn#Gc24Ym8Tf(F!yG|YTK-N3PTY_Hap`B$VoQ`%3arav};v21y7Uq(9^kLWaHx4 zz3y>cY08!&-keqGFV%%|H{UO{w9#W%qzq~8{Xe$8JR0gR{69suP!ZWvmSj(|o0R2K zwovw6mh4Pp8#5!4%AO(&LPGXrUnYcv8Qa*mF&N7*m|>VPf1 z$35qs=bqE1O z#y7bUUo|q>p))?2bF*`{{_Rv1_}mj4+Ell_X4?B2O85G!hoJt|wcnlc<(y*haJ{YD zm}hG7M&$!HLzsn5ur*&)T{)by7VO4_t#chCtLz{`&=;Wo>SR0(HW8np(e>r2+ni3D z=))|a76G>~6(&F+XSfUn1|we7c|d{BngQMi5ILpVmsY3tVn&F)E5Ls_`rrbIPbN}Y<@P{Pav642IQ;rSi@yQ7FXPy5 zYvW6xpuHTuY7afV2sYe$rn5Fb6RBaJ73or+>7HC>+HpL;goYUs$ZgE^^&CYuz`mY} zly2=aQ*bS>GpG7QT|Nl-81Gt3C_Gg8Ddw}dbjoIx8{cxj-(viyn4G(#V5lB#b5Ll! z%qk$d2*#jNUK{(gE;9FBZ?Hu! zx3^`8rnyYd`V8BQ<{De zdH6^JCJg%0Vq02V0V{72zesG(7yvuOj$N-Pa{C@6oHXX@oASNeBj|}yt$bu%qVJ1P z2X<_@+TVT$VpmqpKM4oiGDn5Go=wp=g<69Vg>E0jTY;Y5mo>_LqmgV5Zkj$kfeoH= zH)lI1wlBm--Pz=F_Z)-D`(bl~?gkdg*Fq&X6H^1daxn|hmQj^!^!_!}1iW3Gs5!D> zU&>nO)Twb}j)NM`mFpCfz7rj_xaGD&oo+w-n3hW$z$t#Pik+MLBYH!$k-`QfDqdG_ zGOv^^xE;z<384flj)d_n6~vqEOgt#946TAL)d)Yr;`g^4QOxP$V# z`Gb6HfpR_}_REV_rSn*iB#QXnd6ebHt^oW}({f#gi=XOumo{;k`J=8qB=OsNYj8X^ zhGO>Tv8qTgyuSKHk?++KH9*)9LZaL;-Np)+S^cO_X;hlbM^NnBZ+!-z5a)F3z}Dv* z?VY0z2T#zKCD-(}T9tUP{oez$R>OY@6C=JU%Xdv$=|t1rl-6DxUJ5ymZ&k|1c1=F5 zN(;cRg{RV&!*ea=JDVhxgo%~u^kq$)-lB5_A-7I9zfKc0{BDib(MQ^;jQ1qk?h-=oE1}s10=}PkZ#CflRTcNgV+4_ zsOdByF#5TZ{-PKq#pS(r{p3^Lg1YJ0cdE6_R|xU?(08O@Sg+ z9{D#r7^LP!z{{pwZ+cJ$sIJZ9AX`xk+$spBwbKLHJ$gRX8v7E)3!k^ZzfXt|m4D6N zo>EHfobEu17z)MH^%c9PN>Wjs{UUJ-Z|d^Ee_UmSmcw`W`W^>;@cJt1Iu~^)y#mA= z9$mtPGe%xsfDb_R=q%K>!t~*=%j%pHK{s3hINJiuBYV15Cgv|+<*#`n2e2s ze#Hkb-go{3nc59p=d%H;M>0<6aFj${2Qzx!-8OyQ-Y$@_d!Kq<>6Ufqq5R695x^kr ziS2vZlcMf4Y-FQ|aEapLXRIqB)FnVb7%v{2Kaz~qao5N^_HM~hFKz3B^ah;^G4VZ( zJNCFg#Z3CiM`py@s7=SZC`kpuhLMF>+Zv$REoK$Wt*I@Rm2C4Wx1BGldI4S_HuAv> zk&vb(QQZa4uH?Boqxg_~(~f>fi(+g>x(lmU5K5lo5p?auVdcdL(hIo~7!wNumkRiNvQ75*Qe&+yMWa;fKO zfSP)DG~Vw$z4}wdMp+%uoYD$iu2ovuvHP(1G3=nRDufde*b_x7et@rPzmBTVH>yMm zXofAQ)_-rc0?vi41v&q0V_Xa_u|nkrtIVq)k`pjs`QDrW*(Eiqvqi{SK;rMF)h%gf z;&_tq^|0LIRqAk(v=A317|SkV|FJTlqBDF6eDgK+@73exPgg&x!20`JgFj&LtQo6t zRMr&uY1t@Cd(>oGLxerw($W% zxbS5!>j?~f8Jt^`OHUTqn{Pq|tDI^wua}=1an_jeYu5)ZEG;On4^`GfGfwF9^;YEI z%}mPLC%g72H$XX2LUq!$!?wYs504J&xP9&#$wb%|zNzg?KyAr&c z$ho6K;)C0E5;=crBQh!;=|3gYmj9LiuNJYG8uc>1D(KMNWvI1i(q%C3mu4pJ zXp9MO_@#U*MG>&t{9f_AHVb#-k~;Kx9#$1h6ha6Z^#$Xdo62gk`PXf)!!DxSnwWzA zCLDp!+4gt5t>{q}2o{$O;BolyGH?E`(eS`Ky7r;U+{JYdwO0O0HmOGDO6D3e#_Ufs zR}OA3?0kqA>g^O^gf2n9{~)td>{4goISZ{X+Mm>3oAz}*KFvcg|HTo>=K{oFTsKm> zq}wlfO@Hw+cyS-~{z!Ds7@?I?C#EBZ%Svkt*lzv<=k##yMLkmJ?InyxFo5qJU&|9! zk_rC~NY+Dq7ypRIt~>9`zSqux2xaMRZ3KN{k4Oe|^wE>S`J8XP$nC}53tFwBhsDH8 zhtbRDNzpOQ$vDd{h}Kx3NCDBZ=3lE;_j2agOK#G~kM@6w(jgKsuBCw2m2(DLDXx5r zt_11o7_?s^)Kc08a~bjbj8HCY>Uo}O8Nh~WI6|yWXGF2Xl$SDza3eYk=T8PGAobO~ z3QtL-=J&RbUal{=;c`(*;5F$m6KJWKjxseS*+DZ;2pjDk=o*^JR`y;mBnVH7GLji zq=R3T%$sa9TJl@O@aHv7qm__)3#@~jn) zS#5>gR&zrxXo9(R(r4Quua2wRH`j9E)5w&s%1bSwZSkGX8K5~|RuqHZeEnS5=UE%W z{@kEfqRS`kCWzS6_;zAq%MY!w-Gp}^-o1C%|CDxfb85kf$1sM;P?o>O_gw9jn;)(z zA(z%=?eDSQ^YLSBefQ*P5hH81UQcvknoW?buB@0%q0LhsFVVL&huMmQApzKn)xId) zB& z;Ca2Uji%Nb@`boq?aR5}1Fx8@cJq+Fcfs0Vf6PNf+F;%j#P%VJYghbEvu=_+AgJhDfmC-qpX)=?esI%oiE3Z z7$Jb`#T!xibZBm|)63B-U7d-=$z^?=q54Cg^G?{3bDoH%PlDNSyVWGsRngJ0Z&GZA z;JToR#?FB6gXg@-6*VU8Ssc4Eap|^}Ym64zab4*o1?*<1Go6+DC;b2IXz!BlH&&evRR^G4jfm%*K}|4iF$zF(f6(I zQI-DV3FSE(4HK5?B^1GzRh2?1#g!l{t%e&9)qlaFA=d7W8Q22zTb$$c z9|tYL$&EuGhh*!7+kS|K4%Ot$HM>O(`u207d1l{ds$<*7uc;y$-|5gbUw{+8hV$LM(Vg5Lu9WzpYD5Dg+VUi* zvZ`w=v~PSN{GF8cGu&DPpsa+%9u5lKp0zJQ{&2SB1DS85y^7>3QtYeO(AC@rZLGm^{aO z`Om&Go28T{g`zOGiLb`x)t$DW^&iB^(kk6Dt}Vh=Ze-EgCa7a*@L`s2m2Qgx)X56u zOw*`De!O4}EKk_kp}@V-$|TM&!3Q}0qM0W;urTV|LA8~nrH~pDCnHP?!sscEN4jwq z&LP_#MSbLFp0tc)d$?l+dp&EbHeE2N@(J62&A90AwRI!pjuPOI@2x@Q}b?`8Buj~BI&2+{9r0goh5gJUGc)3 zblX41Z=iz7TOUyQgQm-vt$V4LSim2G_PO^Cu@;!CnQ6=z&3E~M``^2qxWpF~@=fo* zvDXuMvmKL9F#M38KYBc9dz7%W|1IcA{t_m$YfxXyA}YK*&FcN;WJ+hFvxZ`~DLSdWn#WxYx zLhawH1`N=SFdMp<3emU3y5*#;Up7;eKukK0VTI*$QCzGJLvki7AA;$C{wqI-lQ|uQ zH7h)Ht`Y-uEj(&p)PBsFqL~GUAF;I0ZA=C^=~g-7jZqCc@C84bJ7!%ZRo!cwmz~;n zp?$(rbS-N@x=SDN;9~B!R8~k2M{gge`@vuHUiZXbw3S_|mCGqiKt_lb*JX>6@RQ2EOF*{Ma2y5|0<|FB+ijp~pmCjBVunEn+7q=lgv=TB&`- zCm(F2Mv&8KmN8t;+M@1u6SXie!g$OyH*^QDZ@mnkp@_M!3Ob=*bWt@~93_TJ`~39Kxpz)( zeCzzli|$ahh1=~Nov2$S^75V{HV7&FAh~CZlg_&ap1<1WTmC}CFkZe(az{5K*HN}k z9U1!ZQu6i2%g7!qs1*}Fzu|Y{ubQu(OS+i_+K_1SyPemNFu&JMdk{s?y1v*7m(Obe zI`;oe*y=7*P#5Zbw)HpFAlXo65<6~SYIqQMD|hT~y%G9&a>;sj;DV$oR;N0Ka1=StP!wxd)3%iAm4MQr=lC7PGCnEo;PuJHs{8JyMk&NSSz6JsNRXdjgSs{&*O$cMUx^J#uHVb4+Ssp`|JE=OrrxKz zuNmc@SK4yrWO0YVjt<)raq;Op82@A^V+H6)UfJx)|B4whEJu}P092gTwkxwArL@fF zM;#w2kCfWd?1y^Gt@Rf*K(*fBa;J1`2jTJv%5LZf;=TkZug%n4`)`v%Ai71Pnfq-hkmVI;HUzD53K2JE9`G{wEJ%) zmY}O`-|xTwX%yGNYm#zLz--J3PxQm>|5g^ zncF{b$jKW@Q>YO&Mj-^Bh`hLN;`m@ucLEmSq!yR=g0`hoGb?b&%$6B0s(rfW5lJ5_ zdGUx+Uv_0ZVPktT{qNg*ja|p`uA(A5oO`aA_53SHPvN_?v5A$VK0TApl&{IDRbNp6 zCsrKOI1edu{K}}Lh2e0S0)M(ZJ6PFjlV!&Fw%>s)W+h{^vD*5n^E*hX^f$VN<*@6U z8W}HhUq!-jdH%#rsjf=)lA%(CFUlM3T$3!!w9=qpAZGvmL0sD#dK67ZNHw$N(Vddv zbkWmTEjO?<*?-62uJnnlYoJCy4+S{#WG8>|>Q4y5}md;)DY;ZVby+5$7 z{wrtH{tK8TvqsqV_eiQBPT^~Qjq#1yMh8t@mEKu^`FJfjTz ztbT2)NO&84pOK%o_I#(^LFz7LJFJQPbJ3$gE9vVgAQ77P#Iv^t%g*pczVN|I=;g-hmhkjrop7v$6Y$w zV(Ua04U9?)ADimB(WE67~|U{24<# zJrVmTVbefX%{rcBK_iIxs?#9*mxUV0YkmeQ8$9zV_Xwe@qj6jS|YBjgH6L-B9E0Tjb1{_~5=_Vm`r5 zXIi!-5CAGZOlwUuFXR$yT}-^uIxfsrGS;u8WX-}%y$Ir-)c29-KfKv@&(Emiz0&wo zQdO6gsHwL8qHO)uEw5VU2NAh*W3R;dZ%6PnpYA-wxRfZeUlw)a{f%-1{78yw@ z*;whQv4iZE$AkWr^W%mB&59$co02Om&stFR^%IRqTl8tZaYubifv9|oYMIW;Y@F_T z1R{RDze3{!tO~u0o~c*CD1DpgMTdQ{@ue1`uizoL!&Q~}i-n-d7RSS#Iv;Sq0O^PQ zcAzz(0#=DBcYd$?v*f|EgyPUtw5x?Ci_fKwU&j`0kZR7JvE*A4mN!cKE%;b`cW=oj$$~Bb@r*K?ZL{-NFOEn}-orT}c)35GyI=PbHmjQ)#)=E!qRh!YO5eJb)lTli%u)x2aZommIZk~* zD_CD@iv)JfT;xuEDMzYP|IB~1-?HxHPLC1c+^Rc$s3tIF`rOWioWzYf3J^|`E7uM~no08A(oDUUnN`FYXUZ+ibCOoOj(&AV01dx5OY9d%v5%BP zi`SofnSMVcC_KpgzW7yrENA7ztb6GgZt<(wd{K~%>7mU`VixgkT0{sWbNx~Jg1*d5 zayTBU=^Cier)&o3Q4UjOGo%`lT-V>3RZYekS(6GD;KqTfPoqL|xPBXP#eY0lMZH+l zGVV1pL+~7idX0QA=jswfJ<0-@Nu?5mr4)gAl3fvOL|iDpQ>0L|nTV-@+2$A~=%SFQ zWS_l)WRWJc=TbA_Epl8FQ=T&)2NaRZ9y?Z!Yu9Aw1U$^72=?zJhbaT|2(Jg+3mq#R z69IvfWd|kw9)VgB%ID2GjCqqbhwhPl;Bj2eB7CMEkB88ZU8%17pofbyFuk$0y+ZvW zgs|LZ$?rWm%omNX09x<5*DJgl?(LRmIhL4Lyv>UNZ8d1?B6k``I5kou z%ZZ|^hU_775i$F+%DX!$gTG6J`$_v3k<94~JAH~50;~A6E4p(@?4OPYyW^P#ySIi$ z`aKmWOeY22o?t0zApsTmk*>|6N>w0h=O zu*pDc_QG(d+@sJ?_s|Zs=B_t3qHJd|PIjX7}d4Gv9C7$!5%PT0NA z%!{BsnQg3pyPTh{@N`~a2?t)>9TdzeLetJF@Xy$S;M`7?_5iaSN)hre%N zTO_&r+v38HlH$|FawMsLR-w6J%APWd&244k#8;lkBMbWT_4F#t5dx>B{B*6o)alnc0{&co8{3i9*3d8IqAYoJW72h=V zYiWHGGDsN0e>_9Mt>%V=bmWHGI3|=qZ$*yeq+Vpjo!ppN&`D}YA=Hsw>DL1n$%XL< zn_m#M*G{ykgdln6_xbq;{NYE(b|bs;b7J^`XNR|H!n)}UBt#BVW@ahVr)wnKtb)tr zYZ?#Hs;kz|2=QPhuR^z7MkS^RpmH8|#l&v_Of)K0VE|d%yE%;c|nZGW&gV znOfS?J-y8vc3gb-19((;!tN+@dJ4f|G9Yeu7&9*Huga@|vdTAuc|0DM!4@k8;|`;Y zo3bzOfOrv~61(TJI^NDDB|ZY!_$c+lA7m5=GsTYtiQK{u9phaKq(-SrVK)bc6TPb0 zG}Gx!M9rV9Sm2<^%H+4?BBI--W`p4Ln_w=Az`hthA}zUa0*fq~aL;pt4K*dw?$Sb< zD!%P-w+vHLRd3V{hm~-31>V9h*S_HX4*b^PK7_`XrAdv+qy@wb+!}xT4?F(0>q!|i z6Y-kg=kd2Ovu`@y565Slt4GhI^6SDX)I~UPEG4Atpqn&)&`qj3K5~ipshO0UJ+y28 zPnwjy2HZ2F=d>$xHT-B9J#u|2c2)0+zR5D90S^cjTUdHs(EO>;kR{3t^)# zN(l7a*G=;_J8S{N_#;e>t6d?@^L z(pyUbt-Rw+A><#-L*yUNt5as3*x{3BHMPWD6xS>}tP|%HQ?$jA?2b`sA_VJ90C@1- zpM&^GDoeZPPApC@dwH~9Hq$$0JF%D-ftGE{;#@q!`26XnepWox_NqE0qGjt@JxF7d$8!Atkd zLM;AOT*WWjg18E~^%3Ut3& zN;_T1GCaENW4?k~Gm{HKRs9z1lKL&6`U2rFbLIVbEGeoM*trA1*p`AoexEe2RqMgz zh1dYkYZAz;ralr;AI7if#LWl@f1qD%7KdCuE>cWAF>gtPr?o!W*LzYO{RjiN>Gdh> zb5yEohL-tEzLvQLFEUO?(BuQcEa<+BxYr6v(1pxb(>nSbjsBKfC%Zq#!*gdIot*B! z&*|Drp53R2Jrgi+s{}{EDF<{4;8uc?4yKkemYpcRfkzmM9k0PF~@NfSpnb zRjFfL;)DfAIQMIZu5_~fO4EnFrar>H;>KIQrxw&5cIBZtDFzTIUq)oNP{V~rE=>nm zEFNw^+b;M`Qo{JUYj*bY96Or4`6o4aL|HOBaUxxn?|SY=KtwZ)oO_><)-;odrS3UW zOkS;*@zsCEx0@nb)LzqpbO#D*L$mZk z34M2#m{NRkYy0Z2)zI7}@0PD}C`TqQqPPz5kUxmdsde~@wS93E+vTY0dNT?{WN%x> z8wW+crj$5eLe5$cUxC?8gN5t=4d>|iL|y{@2UCvdt6>airYTSgH%yP(j{~0qet}1B z$#HzoSl>z0C}5RgDyfm4QACuu1RDHSo`mW3T>}_Yf7ZDsG(I`(JMH-USgmPw({TJ+ z_q|^>p@M2Fp)Bpg%lF#6_Se?oV~k}Ck`N6w)F(=wrF-f8#{2P|@#o{~c-a$|5GkJk z%K3Y}p-S4NXC39f2#&d$@K56tiK35Flt3aPewc{1i8dCw`NwA(F6Q+5KsV~S(*q-3|+FiBm*710JwnZB}`(sCj#>WUo?MfZv zO@LTbKE;fpck~;{Yk>-G7N*r2GY8=%)qa?-d(y6rM|BmQO;-uiU0+(>`_+=;!X3#T zdTd*U@g`=uoN0LH_e&Lq9mI{>yqe<#C>E7=AMx*opyMq7tO1NF9}l{QMy>?}B*}m+ zR&gOR!1S_$+s^d%Q{dTSV>$Ilky#uts=oOt*sN+q;|363MgsnHwh_Hv?QP=7k~G1Q zg_VKi{UfxunEXr_o*`~9j#SGnNBqXC|O;+f2uh{Xp8?d`Aqg3q< z-%)+C%Xg@`_S@WSl|1Dw`bqs7s(EF1SBBg->|OmyePLw+pK}-RyHVlvl>?KqNW0H3 z)Ia!sLG&@i)T8Z&nlCjDV%zIDHKm;m%LzyF6Mk2`@)^>RC#t|Sn-eYXhr%*pzIF>{ znFngP@>$Rw&&Mat_^Cd%Jwgg!{9i4`G|t>GQ7c zd+uQ}E5K}CsHW%gGYtuX%&?n&TZ;@+Co-ub^h_TC3X zD=jaYwc2Tb_(I6evm;CdhY(@T$<6OG>f`m<5#ymkrW4{ptlrE6*54sIPiWeA%Iyb}x_WM(y-`F?B3 z6qmq=i;{-!hYHXz2LdN6{@|vu!`VM4o;txDk;04N?E>48n+etS8Wur8qRZ2sho7f( zaPPb9LpR)_W=5ZN_9OqC%F6NzEn9S&%Q45BUQHK-NC*(ZbC>q1J@FHu*JI`2rJO<- zvRQrc1NpI&6x^xkFbxR&k8kPmTJ>s$kY`bXrG4l80d}}%m3dtvZ?ZsD`O&WTQ2NN1 zW88}0CJ9j_F;LEANEFlVMmMf==ZVBoH}lSwaE5#*g~}i^Lk*)4T>(KJ{-1!~EZo(6 zXSPNnkFC`WDGlDu+ss`apy1nc&j6gT{S$7azw38+e`2Ij&<83;JK$|$*@P+^=S!Wr zXJnrC*7?JuBr0vhy!;lUCL$I187hF+F;wv9igp%Y?v$>V0(y>`4eZd*S8d0T>Y^wt{zoDKoicKhL6Ap?~10e-4Udp zR^+8HkG0NzNyCF9Knatyd{%@bP{rHEbE3XZ zRx$(m@c|QswQ}wQziPM4XqTS#o;Hlwi36jGtLt%>a0B|^Mk0IQLV}f!ccl!R9s6Fi zX{lLH-s4Ag{_=#BqZCesWP-sYG{~$ssUEG8x@Uu){*mM;xVmT;!rPhkVQu0vPSZZi zvR_(FwA@z~`&`>!f|IOs-!9(6^f(`qrDE$OWu?_PaIR6sw|>w}?D?cWKZ>Nx#a~^W zI&w8LP%PR+f-YtH?i%X2w6c7MJ&_16GUXt%XIYBu{{Y7)+Iy5!S+Gi0WFG!7dDR)` z7s+8&D&V9+uFog>$Ly2Y+W)OdRU-zW;=zv1(C1{yUdJChmUpX;bIr0(Oh;M$#`z9a zdS#B!r@Mg=>A~f%Km_->;!T~~*i~JiJ6Z3zQKXuB_fu8kne>$u>SGiVO?%es-hgg@ z-fOu^lghNO!%g6DAIx-*OLzsgdj`w?;8wqnWou_2SD2q&J3qjx*rUVD+3&Qw;Fh*z zFS&o8wPmdHBKV=Jrx6#|M`OCF{u<;p#7Q3F`H@SfHF@I0za z4{A|_sI%8RQ@*2~`FtB{U zR*{eSiKF)FVo^KU>uXY2PA8utLc;oLesO&M<_&s6M^h@fh>~jf7ba`B_GtjTr>K1e zpSOIP`5}Px^eJ<9&)>R!hUUjn2JIIpFy(97Q6gpv4~RyT(=zTq*ch5|PqYfAV0 zo)%(uE%M%J!-cu5D9+|@K{??sez=-1;v!X;O6a>?Jh z3L$UuI1fuk%e#1Ev)m`xN+fyR`!5MXCjtM( z%j}-0HvZ<`LvRa?<`1KAa=V(B(ns27TjQh(w=cvb&7U`E#6x-cCuvXhPZB0r zP13d5#dqe37Jp$MK~-2{>X`pe`i6d|k}NlT4#A3a4O=#VG%X4B7CDC1G2)uei9%b2 z2#`r5w}-j1kq0E@d$<4%wIRuXO9^{kqyBop07uOrZ?5EArq(7+0gz_ultpfwyyMU# zV1U&X)n09hsyffWnnh-zt57QRUY;?=izItVd3RP-rgU|tal^6{@Y}i{#=Gos zr1}K!HP=15{-JB}>#iU|C#!idUu=A_Ue06RT?djV z_hd729Y=E_-C&8`v%1GO)~cj^XC_>fccgCwBCip#t%<4p(imUvL4E+pH+&pmJBS3f>WL#8+pGSV!f=fq4SO#{Wi0zd{!ZC^~dD{`LN+pvj^3< z(9iL)n+tX+FWakk>!O}jP6Iul&1uib&b|G@N9UZ2<}yt{DH(eq9A!I`iG6d~?e=R@ z`@uTT7tB6fg9D4&oz`A7U0QXjKAqrqX+L$2xLCSzN>*_Hc;R1bV8o=>iBCJ0Q<@O? znOv~yNK7F(FPtr`wZilwxMT^stB@BFlTpNrndFgGh+l-L^+I1Jd3ayMYC-A~d3RsL zd|^?Zz&+5)+xq`0eg97#ymh)`C$~VYQmb{s{b~+3P&;n9{$zVTIbFlwAz#^h?5CtK3x^BxEiOj9RS_jpMx!OWC$CW*ES z_KBYt7^DviCj1j9LbwDd$%UP79N_-9=W^3Iy3h#Un?ghLZvWZwXNCzQzqdY<3W4dX z?Qth(WGCpC@!nX`+o#p61WaS#_W|V;G4$C}cn&fLPTa}eDK~`WiJt{tH@3k<@}pM2 zNx~DV_bQgrAe}hno;{N9v+@KERNuGda}k_Y`R%T4lGlWcAPaLhR95D~-bEQ~A>^S9 zKd#FYi>3ttF-`SRyuN%-a_4qVV!R)?1z!|7SbN%0-KObuVWOWblHjpC@-=ed`Dx+^ zzI0Wc@j#HIuw&39C;!!J4^n*UcE=M90Ppr(-P0WIi4D@2;BK1$GCnjnUOGM^M5EFB z@OJwL!cL3Hiye!NbdB2ER{M?V$WMfsHCHV&bXrGrfML|pBhk)IQ3Dg5q_mmVpV;5Q zH&r5(e6xOfnhO%z&} zsDZGHyZLPLCp4O1fl_@bt;4zFzj<;jOuDg?Gk!c&8v_w~3+rgUdKyK$4x+bR zQ#lFOp>tBs*EmY5L}g}J&Z-J;*se!D@n8J1uoQJ=;fQjzCg%wauyA9lH)rYm`txus zuKMZek9I?=T&g22mG>@`#ec&N|B6>;v=7T&Dh;>>KiOVVW9DqjGRyr# z`a0oC?gIljc=2QX-ZR_9;e@{O-GvlY)`rlurEgYjUQXph7?l%Mu$PG6b5QZ#g;H}t z{VRP-dcQg+Y``MFdcFovsNJ|?|9+`$DF|11pCF1S$0m$gOR(X z;hZfOl|@leByvd|&-(8H(9ujdu?>y~KA>w`o8we1~9xDf9|GEg~1j$F+~&NEIj z3msk&Lc=g%(ZJ#q_b-g5z`br&g#zIqL2NvsL*JxHCptR_l<&>pM`4tUo;N;%odv^G z*^Wy)$CO5-2Bnh}Fvva3wBE|(MH38%9^9M?yr~HO8syQZ^ zb+q!r`wFi-B_zl{IOZ5W|1HsK?ox4BkvcPeC;rsB?)MHboF3k_CpY!yUd8*$yWO2T z2A#+`1+_Ow?*?Gly;-OkX7jT~K0RTLFQ%^L3j90vb-r^Cc>$sC+*^|ubnuTsgg~D~jSZZ8_ z8)9DOSM%c0!1daB4U%4B&hpDZN@st(C6ggYV=9l86Y-U+u+_PSUAc3^ztdKsb8Erx zL1o4!Ltr)6WjGoYm(S%Y;Lx5c=g>X@?A(y>X&ToY4=R=jqL#5AIJ{I2hX!H}tgkK_ z^3*$YWspEn9rc|@5KY7M$cT`=b(Rggrg7i2P^M%mL(z*vEb#q zkyMIIH9h0{5Rpg4Haik7(04akxyr=et!1#9`>#q8LF<0s-HCywV)|cIagUs;f^w5Q!qbEp@DjGR1PZ%021k!r6U=WN=vqZIbEiaWERM^=j01Pkjq ze`soNk}gLUltHp1GDFc5Lv2-1I_G-h{9EreqKMqFMRz|AMKWsS^l#gdC%ce{LnlwS zHqAp6!_-;=!Y&=``e@}b`9K!RwG!qOIMUP&I6f24TUW}pKrFX=`QPNd*Krm$KsAT^ z5Ni*p_fy+OuQ_brCyw^CWxnBMzIg zl;;agK0s}@;@~BzOy!iP zsKC%0W@D}9-GR(3$uCovKB;z=n+fYJT!a3{fj~Uom+;8v#-puB*XsCEZ}n{L?T)dF zqm(w?LqzIioFK>Y6S^wGcpR2#DZrECE}0d%sY)WT4ea9B2Re_vqZ;Py*MH8rMMh4W zG5XS&)TH`V-Gtt({Y$xwM>Br}FxFp!Pjs8fXz86+{*n<`JK^4RIUtU=&M@H*y5rBr zj-Mqdo>lmbIsrX7Z;=lQpR(k^VccXlgb25GF@EMJ*F@PF)|p&rYQb?XK7Cq-S~oJW zna>hRxf+>QF4Nf|2wpvboTTn^;Dd?rXEt#p*zCv{KWBDWZ|C@70iApvrxa5JX>2wF z&B$N#o9p#{I?seR!QMOk8p=R5JxE*nl9CGXA{iD4KbJE1T9^Nh2%`wMJUx(x0$P+? zoRVzMe&iQrMqZsQ?p{ziI~LMff>d2~?QowE%~Fn@wzY?1&QpYv*b^Mpm`HIc;i<-dF z+6pD`Eqgo)UcXAYfz7iUi>$1H4YY`%xbLF)Kg&#V_nOE~^7gWec0L<4Td4P-r5!hy zzP9$M2R-gpp`Zk+8=U%^I@2qh=(OxA_kEiLQJeN5^HoZ(8o{*`@Hb>WeedabKeMCD z;`e&BN+kwxjO+l90TcUtb6<^)agWbbLT4 zuW#yy7gjBN0v)t1_*p3Bqi*u)@HCrV{bw^6UgP25sZ}!Jc81OIrQLN>M$UXpmscWZ zM3wJ3u8jvSTk~%TS`RE1V4BC?(dO^CosNL$h+kP3G{cq*(RchRsyvtC`4lAb9gcM? zSH!o%0$9}>BF*+KEVp&H+oxk@prWYRfqKPWO0VPUY{2^s;O-v z2IIsI-}oYxU0qHWfc>L#bG-~%I^gyGw7b%>DK`hL@*Ifrf=#rkZk11yGA7OXi8xDz zuNO5D^1vASGo3L9VPw|B%?n;T4XMhxU%gaLD=VxnUE7W=TN`!O_LDyGXS=nBOR!EW zZQAj4^o;Z&c%s(TMyutAPcFQKikigG6d~gz5 z&9QuQBb1#!2Fab%lq)D{JZA9=eCz&wMfb7HBvllJnD_XJAxnbYPAdQTL$wL*N$VX;}3Evx+@Mb;v(^|R;-c)flEe_Mb znlNK9wx8#a2?r}NQ044PpLQ^5uL5%rR$txj_!qzvWlfiW6ddH@+0jEcNlS z?8Pq2p6=b7{}QGFJ;meM*$0?UCkrH-`+YxQA1?gfxO5Uf`rn~3Mc~jlvTf-_K(WYk zlZ$S2q z9^87nPzMj(gfG107ofg~{vb#zSHK*Q2p!nr_0hu$@0#zt>{M@0L_M|B+xEswQ~!so z?~H2V`=V75QBe>PkQxzzpCV0qjf#MZfQW#!(7W^&NE47IB`N|^BO)MOq=X)NkrL@W zfT0HnJtQH?B&zo1o&t14q=;KdGCJhAS%i>MeWc zYH1);=!h<{qYh0Tm#b;Bnxi4wmg8D!HcOB|P}8((FZe${;=lQO>Qxa6aQz7rAHW3$+@`tsZelrV7BjIz=pzn zO%1&-Gt}XA*E!yDgiL8?sfT3tg92qlA?7>+ zsVsp-(ne zt4w}p`GFHuyoCa4at*Y)jNpb8$3dHsOT*beA7aaI3ySh?* zquJrZ)l9={S(psp`>H_*v3g&nW-=cELJi@Z(9;z!WPUn!W7k4&?2=@8v2Q4XD}C!1 zGbKdLXz*NP?`LcSr0aXBkHy~Bjl?{-FhEi>sK}xq)h|ks%3_5~!R`QK&n#5v}#a;rg820$7!fAgi~kBCl^ASiA1c zjrawbde1I3z9{z1OwBV7&kA|`3q9ixa%O9tkv{$V6Mt>7IIvb|D>LhSfuM7nTXgj9 z$q2U9yC8LFsLuNzx!{QFEzIVGF>JWbo|=DA*K;JGRceq+;4zB|MlN4u((Be_eL{=x zMS(y*WSOS^6TNb;^BIt#F~Y=-z$-|sGj7Tae0G{;0ky^bnV(uKr{cdBjkO6q>oq0W zO`U_Y9a%mD714si6X;U2zk%7aW6~XH?EP~y2H%r{-6VZ*XK9Vw;wnE(a(8W&99oXJ zst!VhEhtq|@J*E@t^C#lhq#H{JkrR{OdRsk;5zyg&^p)J)pW^#alf9j$?{pN$iG(@ zAF=>trEi2@pPbY8S}%y$KX*bNRoUrU(eK^Oq`dNZn)RL~(OIpR@I~=>X*u^ib>|ii zyesIZKE~nQc#3L*C!z%mvtloRnk2}{NI)*cOrxj#M5!f*r!ED@B~WVXLD%;gO??J; z8cnu^G}x9GgwtNM3OC(S&Ir-p{DQkC^u{}}4CK~&H&cxKRnH}HA`e&q?p_15HD#-RPc>S)F5CCv9EDeSSH97~aC0w{d}mUx~Q@ zILZGKy^+d8z4jnP$^)6W^EFBRA+FU1i(=USx6S^Dbknk=naRNk4oj z!Qud9QW#$vpD}8Y>6D0e>QDByIcdNhKj4sq~h(#iw*fwarNXcom%gM_CI0 z33sg-R5EjYJ`H8!Q2|={@fcyo=pni9GCVfV9u@5~iZ*J07VXEXY82I5V;~}P)>{_E zT4Q;;^U{`dhPilF$T%Zw1YT{kWkwzQ|2B-ZiAxMj`G)gJSO1xf{dTs|RYKol)0Tn> zlajK(%|QFepcEHqy7sEs;QHSj-(OXaSiJ-+?mHN$>IJOS*V`zsUK3~Mlc4%K=kwjy~ z9B;Ggi^z!t+A`Tc&}UzP&)Ql_z2;9-r=gUoVXuQfG?O}-y9|FE$YJF*dlpa)1~#Ip zigf6%euz+kW-d=I^|8B@U!YbJqkNt*vlva=-^@85n@K7!I7f^0Y%4odp}&s4>r53EJ1JE{f_6#?{XZNgp@6O0~VNrA->sC0F77!4GDc=C_-X z<=MdNHqWTweFz8>6i57~l9(NykJ;IgLPZ0zn?71xk>lN-)0CQAb@|yBgAMoRahH7| z-Nj z#g=wy&qVH9hh6Nq2P4p=@(6$F#f>Dzoug1OA;|jO)Jo?d0Pp zU(E_#cIS4QDjOEAa&W<@@^JNg0{Zis{#R+_y|=Y-$~@a`p0PZMGqLGB)ddT#gWTDz z9TMwf1>O;&u*e1akuY{xkf4Qq<$S49@D?a7X&MRV@9?7QH@T%iJ%B6vvX_?6dX)eBF;wZ|4k^iW^Pb^=o7U1 z*TK*_CTjM@L)MRsJG0ID{QuGlkS*fUsLNPDS#CF|Ia>%4D(T*d?>b&BEVDhu9hSCP z4#wh0iwUc^=vC}Y+iLOY9rtnmoKHluvQ@}UZ`Se)Gz*3|Ppb=CHSMBWXBu+4(F>|Q z8h<+!JIv0nDhse#X(VV)^|xC}HM6mlp(B}sX@w!>ZBaO%gI6dIv(|;;OE?TH2~EryRqR&idcG#F zukS%h`gzXX!I6X`%fEg4qIT4) zx3Z`xcHezhuPccaL3}fQf_C4ujPrP)%SdkWKu#AGjj%j-ypeoUo+VY}MeU8~02>4q zJ)rkp_P3=*IZD?4p!r;HB+i74$UM$c5vnVeikWne|w%jmn}X$+;ytqb1=khbj)Hsj zt?En(4{fYlHBIW!y^_VR+dpFC>F_0BC7@TxdY?OxxcR9+Zkqaufi!uTJwhn}XP;v8 zT^v27u~`XLRYLxEs&|KbH|N>#FNq&%-o2!hlnS&WG=?a#BD=Rp{#24I-JCZ`3oXG?{5gJ}; z9dLn}%spfamrqrXgv*D*U~2;>KbrEV!Gq%k8z(E8s-^TSRUh(OTXB?qnaTJ=Lt!({ z!U$VPIQ3SC$VYLrI-%+dU8nv$W&UF($}T99xJypP)a=L)8_IQRRkadjN0hGQxBkjH zLglXKSuf^7(H7+Q`o8ioe;|(Vtfv43xNrUwYx{xyJVY}dnot3xd%d9b7=MNT0R=l(yVWRsS2Y7*n11zcQYjRH#ST`rC8-!PR+dwTt30i|A9T{I4 z@tnPKU*;Wb%8pzN9w5R(o~oP;c`CsfW}Vp&vVySGeG2qm;Z(XNAXz?-oc42&E`4jE zRa9hd*kdOI8!}su6Uagq%6YJ@u=vqCZ`H_oaIEk^tq-wHf2IE$ROzxRvlcL99_*B5 zohy7z)c8-So+(-j+tI_XrORP4HCaJ z@w$6HlDj^@VhuU^=6OS}FK#3>_CcP;vN9@cx`u*uaPbw0ZB8EO%zB@xaZqiaU%k$h*Kfh{-vi;tx&! zl|aI{FVl7eBV|9PEA^+~pQ(eXS?tSJzl!sG2JDVMAM${#gY{qAO?r$?4Trl&SJF7muwoy9(kZ;6a9B)imn zA4IX2WF7G=Hcu7-tL_+AUzfda21{?Qph`h)t7nudA}B7J@A$5HF;`!AVh6FPr8IhZ zMPEgrA!xR!+4jHe|BA8Re8|IlU`g_@{rH;@6Y$wX4qz zCUuh%3ltqKT8Ugo!YmnpP6`?Pn_i zWytqDV)s>A++8YDCi{hh;%eDB^|CyJ73*5t_LmTB9_B!SagJhPz?;i0gNduS9+D&N z7m-QeL|_}(KhUvmx+mG7TgxL0t3b)r0Dm3YoKJip^aj7ZF#mkrZaQ%P%2w!!=oWe^ zU7+VtPfqet0lfOf?E>H04~JcK{96@Mtjc}lrPj|=JH-hEol+6<71-NKiHaFi<{MRB z&6)wdcgFC{!!9@7jq9EG#pv$FOx58&!h?*2lhv&*Ux zzB%5e_%~%h0cm;0E1A-9(ZL~uQlp*uc=U3pNqqNUiS<`v#}~rQE?7Y6xcn)8P8lrm z{wh6QZ9$MJIEwod$d0#I<8${F+-NebQ*2|%UhfP1j9(JhJG}6HjMA%eh|GDTCkt{# z=26h*WfcO_^fJBrIuB`0f(Y!bjR5d#8`8Gt?b2RA26S@9dI+#m3xnC@Y^z4~X% ziki-{mgcG<0qKK_9jZ)FXG08 zR`zI=z;H@`MYLM)#5J_TR)k7j9gj{iVp*Zz#+zOib+3}UMdMM~d6MAuedkERFk3W!z7pYB>5nA%?8XZn9 zsr;B1PnxP8ZM-6FB+~L$zp4F~k^@VSyaUUgJY;DPy!V$fGKpQuzt$&jOZ2kyw%qI! zu+Hqt%m8)KWl4AZUCAm7H*D^V3oUu+vz#f;>}+bFsZv|}1vjaF7+YjYNW`Rfw~!>s zX>+|3busfmXM5f0ZrwQkvR`Sm;(^&r6%&avBxB~x#7qQ9myFisKmn}sjj5bDrY*;TZ_G6bk3LkNmi6kodO7k};L7o>kEBP3x`Fp!x zU}^xL@%d;sn?W&tS6NrvitCzJQUCoYzI~cu zvoF=dU2U2Dr?ep2PN;qBV&;kMwgT<7yINlNFU%D;QJ*g;BM;RYqxQS(OPa$L;|lLP zH1eiW-y~ibbUsTE3=k&8t>{ilPoJo&^y3F7d7S+(QtR7Yw;64*B^f?RVa6w75Dcxl zWqh_r)2H$)Y+(^tXHx|{YYKdKvg-UFL3%i+Ma3$X_Wi5MCVMccQe3^}>;y3WJG#Mv zGxXX#izPPg6^4Q1<`2OR=#HbE=gnt~yVzdvQhVyGm}Egq!g&#{mz9@H33pb1&d(}v zq5_4J3F6%*YLzMW@L$Z%>^=!0f{#Bj*oRUHGTns@rubQRSXbs#U_sZbvYDGaR2L8B z?g+Q)YzKpT@*=ER{zC_5=KC3ValM3k$GF<41X1Vs_7Boq%w5(-$b}Es`r9@CK41uG zE@iNUm(zlYJfTU4UG-JEH`zPMEGUbqyMu**m^NqM#uzPa%48v2NY!CN8B;5!?P-)? z?D7_{GQrbS6(NAMn0}JT<13*M>FntHB2PL?{sz9{rkUlmdHe*cm79NtufoCBX$OD} zA9s@zS*|nNT`{4YC|7Y>EoA$I z1RC=?U<{1)H#N|A;@qXV;BO6@77TST9}hR`S;*L7gOd_9UQN57Y~kL##byiAk&RcO zE3T%-_yJlbJcjp*wGsqt)|;DB<^rEQjO4a15_;S`x}MuJ^8DliRAzv%7@T(^Io{jt z`6fd!{~zT?BU2)Z=w4v6A+`%$;yf;i3K?aVpLobAYlbORv^UGBIW2I3UC4(?hF-i{ zw6*|wkE;h#qipA4Eb~TNs>k2?K(^NJni^D`Vfrh-6I*Zt;OM3c6MvUWchN6u6Uir4 zEGSMO7d)_Lrkzn$&vm@NdD8GRm0RkeWGHmOF*71KJhP6nJ9@oxu26!P@hOlqNErWQ z&$tWu&!d*@x~{CE9mWt|d%(HZv6KYH^pnihY!T6fl)(GV4owS>bY)(bCsF?bdsI_T z_?o?6Fdd%|hDs)6EM;(aSbH4Asg-PP)k9dMv=TOIXd`Yj_fuzWK~Cs9Ultk@#Wk}o z9-1`Fk1Z{)5ne^5MG$VSn5akRsGdL6%nucm6~Xi5ptOrfJ2O|Dx$zfOYh!uc%CdDm zBVTvh!+I8VaJv4d5~sA{THV2|UIlSb7mhRdtUd4Ng31dWSX75nYT?lj#hm%1K)|k_yUR+suxWA>{PS$I z+c^hWx7*v0=Sk`0!|L=%BLBeQ8Ir&mAryY29z{&R&4?+1quK=##JaPiCxS-qQb1L*4w zpiQnn{<~KiYQwR0{f@vm_y?=cD+u#wQAP(Q*G}k-ymGV!?X*1N_=alo`xQ@En zXVBzUa3bTbnveCY&lXJkU449&Z(m43VNPZ?m@x78C{JmZ`g{AnUm}HTsbeg8(^oF| zU;25f;$}>-fy>sL$v9aX8elv<(!bWLfLa%nr@8d!w;EG1|A*S_{q`9A`J=PT#_u}y z9p*mvpBeX&^-YgH{-%IFt45Uxsaaw^y4I=C(p$zUoEVn z2h2-gb>#iE(KK3n+0ba>UezZp2@I0}jb-!huWZ=)Jr!G?sF!_z5^qJPG=1@X0RgC# zZR)kSn{v#cm|T+oqsts@jjJ38hI0pLR^tGlNP zLyU`?`ccTNf!2eH_5j;mrH{Y|1EPc8t*HjPunpbmtT+484IhES@!y?)2pUQ~1IC^% ztK0bf4wHkhT)Wa{z+;;UHu1seVPYUpWtlJ{BkJyA=Un(<{Rp5Sa1QWl`SEdJ>l|AN{q8Z? z6w#ZBIk3%w3UtgiPes?!8sAkDytsaKDRdOF?4JbjEd0k56LNsJZ&YFLK?nIvZ_eq&v$8z6vjiQ`@;O` z!vDV7LvDlCzki$=7Sb2gRr;We>Jg{~J=fO-rta!9m^$P=!F-csrFU9FFL$i|2$y-H z!&ir!0)1xw-ac9sIj{xsP?J?bs2_wxq4`p?hv|=K z35k$UcWU$@#-$WB1^1^hv~XEdn&0T8QtirVbTLElxU1x7&V#`)Sn2WvZhChR$B)%p zXMOKO_5yho7Uu#si|qwEpubO8_dnXHgdPuM!hX$PK6<^WJ&Fo<;0qSp-&YMM+d+=< zn+&$t&4Meqr>}PyyXqH1^siOXaFE^wkzsW(@k%;*#kzLGFDDK^?4OAyZ5IN|` zCVxa#QPI)y9sF~U@Q-tQzKP-Ab^bts=M`><4!tk?aL(yk&h4d$(Wm-sXPcRC$jh7u ztlFl3zvP}a`-&s>;@!sy#Xm#1q^{o!x7OY%^W5BB1g=Fxj-;s#6DlrQ)hibMTVz4J zT64Q~!jW)b_(WuO3FAa`bTKuu5MCDzX>3w1MB=rQKoOdHFA zi=%pkxegn!>JxsqMuKoC|AKOEa#ieiK8a_XTv|~S>99Yt#%$}IFRl&0;2p&1EyI_j zoOO=?)5R5-;a?dKv60|Ahqt_#nnp(Y;t>`-Z`NrWvk$0t(Xm~33gNTSPIvgY=Tol>Z6lVLBG} z>TKo%iDN}V$fcWYb;tG{?Yo1A?d!S=HLuXuQ#iDq&HdE9h|Pul&Ha0}(?&MiPGEm- zVCCzfpwD3Ab=Z?>W2nGy%9FCxiPS&ys+bpLS$dTl*sI>7+M_wOV}8<3Fs{bp`^(<# z_LL`+LYZdk1Jymm@zuA1?D6lSXR>~y?3)K_*F!QL)Y;16!DpajL7hprtshBWqVn^) zY#Aoav+usQo~Yn%_4_9L$Pl>ns(&;4ZDRp9hhZhRRi3*jEpHMt-!d zF=v_`W&rHGxdB@W$#8(!chg1WAFKc#AdE95s8{Un@!j4>od$1i3jqw~v>AM~`-}IlJ>@6KO{31B(`cY zEYK{{1`T+)?GTF~|9;xcI$Sbv9PpK`!)4@Fv&(=3R6NB1TAl|9fp-CW>4BMpll5>P zh`*SHkOsGxclzSix|4FI6I-tLC(_ z4-TN$^UThK`fM<vR&$!?f}}69IeG7mZNE^4W{X>CY>0;ISx6o-cfmh3 zv#;M=+{nN34x4dlCjN4X(BK>+#j)5PjUPOxUc-4@G}spw9_ zmFHT20DM5k51}F%8O;B?hBJJ50h?Z`M{jm<`E#{3g~waDP_||YSq_?5Mid0g<7}03H(zq!ow3@apNs`Byz_x#_ z@2)TkajPJ7ZOF4~*dBYfyH^PgwOQX7;(XY=T_BO6WCBF&eLEZ=z9v!uI|loP#as1# zDJ!S;oQj*X!;N{<&D;5&1MDU<7}Y$zKCZDN%-rl>KdVE!H$-r9FZ57zpAd}}u%Act_$&rgzaMJN*F?c2V=4S!^o zgXK{Ox%EtptC)mmC_ld<>O|A;=?hDJFL3d!sh>l6vQ6IjDLp_ zXX75VWRgic)AT)r7go|>0iB3T*YWN*j7!oS(+JJG@GLR!htl82-I(uz@wA~xis0z? zZuUFnTyx%IK!#bVpmR>ml(#6$XH6~5sZvoxe$&fGUI0H=O7Ows;E0_j36J1YAG9P@ zqdhaxl1CC{09|o6{*5|am(hpnhcycw;h#k( zH+uAJDbEjIy6Qf8OXUqd=t1_nk6&FqZ$hxSq)M?p%O_v}Ku;5xXV&A}sZ^I3qN4H! z{qSJ*QP#axj}g1yhIqJ?#qPJ}aG_Q~Jg-l%zzW^o{rsNBD|XbFW55r)D-;ge zI{~Dz_cx9)>IcQvG%i2JSNS)Itp$aymtU~gHEVmO&~C_C%2z4ElQJ`!-dn-eW9gzO z;R)IGiJxPJh$QlIKVQ{7@`HokZKcusYslLCZC%%3S{&G=3$%vKEt-$c`5`jY9i_Wu zl}WBmOkndYG>OH06aXK=2>5Hc_3Q~{f`))=!3_r!{}n~f+JbZeVZUFmBpMq3)cut; z^r)8WnE6NHzsirJpJw&)L{ye!F4G!ee1?C1B@mJeVhWi(a%o^6?UMKo(9ilb18DZV2?pGWbI3dUu*#w_1yWkRZl_`BQZ zLuyu)ucK}5s9K-|oE22yriwi~k<-k0`T8I4Ha2-x=O8`w=Yt4LLfp^d z%S0c;X;DA+JJqp#0J$Pfx@M91Pv48Fxl|dfe2^enAWuG6M&kL?Zy0kMwSlYr1Eo~f{^wA?7alz+jb)eJDx~;# z2XR`pxA$^Ha(uFXR`+kMW1 zot24ENVj;a(_Qk>p7`Dt1+hz(P}?EztF_u0_V$|P#;n>1Ju%QsXc@uHN0n(diPtWX z2$LSPkluDPo5!Qh-E`MY+RNTv52d8gw{rXf#2-R4)xyB?hL1_;vk6UtaR^YTp5JDk zz^Yoj9)vB} zOP!zT4YZPsCUR`jAc0n;xj~nMgp-PcZQA?hGKD9nnLj5fIU_?8HBu(onwoZx7bA_C z#pSNO1*9~>?qhTzM8*@g`J-FjDDLN=xfSvA4ZqkJ9F>fJiemoX5FhB*t(5s>xv4W@92W^AnmySyLUkn19 zp0WYy);_(KYs`4f+bHkpr<(IiT08&k>RwDd0-roTIdk&7zi`1vs>$J?=^$>``}-o& z8*gpTKBbQ-XP@?cI62UF2RB@?xG^WswEkX%FQ2#)oIw$o+u897rR^c$Rv6u51W~)MDr7xE5sSLgL5OvU zEq;&s@QBp|_gSlbBE#SHR6qp8_m$rxG`vvyz@udST%wg?iIEus=&`cXm9A+EZs15D z6k{E*fHCE|(0P({E}uteSmbErboK8p{XFe)`9YH17MD7$rR|0}dR_S@j=$_vweyF$ z-BoPrVj2(SA0UOKqm|ud)V+4}u{yP4!1Y1<%VU#T9%&`og_A$cSU!nvN+_B_{~FOe z8u$d6F61zm(iI~%LOB~K_1bricl4jT2;q21E!T~GqK|H^Y;&^oneV=bJ*RU|`b-hc z+@D{gDDmPtsVz|vba=U_HDuz zqzx4pI8)w38Y4M0QY16~3K>|5m6z{_eb|5T-AfdnW&xW^QPxwxr2jTkKmI;~B>^oI zr5gcEBJcx|^l|SWLX!hYWT!)0b-_Q*i+8wy1vups6Utp@g?E%ipAy&OY6@>ULpe_K zoKsl*Gb5l_&_{7Om-a&NMV9aIV8*5STImY#+-JV|96S;R;Tau=eh?U^3?zgmlK7X(YKVOz}{ z#MN)j{iam7<(4JWSq3rnkj=RGl@mj|he7kBRC!LOK;jcY5vaZhNCmO3SWGoRXg4UkCRS5N2LYmCH50>eogS3AzP+ZIMX+u+`P z_3RHGZJ}d4v4Amz{>2$InS^}iBg#i9Yj0Ldq*(j+)kRE8VT5eB<~YhZC(Z(=53Xt0 z;4eb8U@ZSk%hFUI_ygw6k8Wf`>{up5;l@WcC!pElb%l_`G)@~)9RPB{d~3!?#P(># z+|f%c2xa;6YAj3J*huf_Pr|1ie_*;0nDV?fH01Sh9Wt)afyBSCwVtJ#I33J|+tyNQ z6^`eGZ@1+wKm`Rf=ieI^WJxNMS#~!Mq;q-=6WV41HE(DFqF;&wEZ<`xRIxx}1rHQt z6EAL+$r`N$?B*O)Kf}fzj(-V;sANnBk~Dnkn5EG>a;-REI6K1Pyc8y4KLhqX<-{N^<+l;&nTQ>;xbAOx?5mg9mE450U5+H( z7v{KIg*Tbaq;0~B34^u{Ka;RLmj+it1NPR>w&K*!A7Qu!PDWhU=kB%MZMw`r3-0zS zT^${c&N&@M;hv2s3(Bt;O&>dzY_i)`g`+xHR+1UECq75N&`YIh}_0X|zBugkFU(qhCX){nbvF(MK9CC10tilhW z^5^D9)L9His0?Ch{$;Z2yW| zXXu#DWQYZL6m_08n3}9~gC3H@IH?L_$B2RoT84WgV!bLU!}RqG79jhxu!~xth?;th zr$JWj^c8TJz&25Rh35H#?Sp*X;?ssAu1Q}Zn_&?ho#B3XW;GaA;%0P6+G?)KFEl)C zeiJ$!qQF7t?oQH^jVo+AFeZ4sbQx{(**iHy$S~9w0**1e=B0du7)n2H?km!Nr9>D_ z?HyYUUesIe9-X1~f{(t&bx24Q&g}lVHy_!M-c49_s#G;mh8p2Fx@~-;G!`y|9F#ef z)zuz3TJx*@PPv!c!z->Q;Lg&;dUJKgn7s>*FVjx7FA8&|Hm;hpb~U07x+>iftp@rj1dl#Y zhtnU^fA^j43+b&t?=4pyyF$F`5UF=VDQdUB{2RWfpe3vA$xt>};RzhLmH^hu_lVK$ zfQbfu96eX3^osW$h)KsYbzk!6ey!J3+==_KKkZo0$Z#RVX#Ahyy216be_K8qGXr5@ z`7e-NbY)y8t4NXH^?6^q(}7rF!;VDp;O%DHaU*|z%|idtYh%^@algVTpvp(!$3}u{ zoE$h&_!1}@xRiHD{#Q&8oxYmetWA8l-z#(&;=Xx5)=o5#ixfpTyLzP@-<{%}XI4WL z#&forcj&(|^ecjkYTWI20-yiEb+8~a-wl*TyaCeP&!HY~=D%no8XMl)cIf`ZucY=@ zV|llK#7710yqK|;v1j7v&5{?xn?^0HmWnotw%P8k961y&&sL4Gaeis@El3q{3tuu< zdF(o=<+S}~o(KCB!GygHxlajiOxnK}x11dun$oX}kJ=919#cI@ECe!1utB9{UzTo= z91j~|AN^w!8sM05&%%P^O7}zI&@yG~Y;EnPKmJ|lPEJ{*wyr~Hk~emrfENh2K$dl5 zh07eL`)-Chfsdx0zKck>*B(gux{E+@)zSPjXI)2z-G&R~ibt4v0#z%11yuGoEMW6V z`sr~lcA;kzwCZ)X6V?ibn3j#@{uyE#zt1Hn@J7-HA}Z?47Wr$mU~>L z^TCAu4>xl@t&a6-`RKSS?(dvf_V>L646y@_bY`VWpFnspxrVrSreZ2j>!OzmU&E{} zu<`l#u5B3S#=+B2ytRi*-<+DAmot8ohq+vQt* zhOW2gR4D%bpj!2#8z-P=1Dj^1+2C%H&v|%>up5>-HpA*a)WQ5%V292;fv(Slve%-d zZ*2iQ3a~$FMAnN)^Jw(8aISvCHn`cs>%P2DoBF&NASd-NDA_=GesEsUqgd!M#7yod zt?~fdXC^Ypvv-EJZ;;y_H?kgPhLpird!0*~p?rNNH7m8(J7DLyQ1{H0XfLd6g?S-J zk6;79S&xEJbJ2YjuA4#k_#l+~r^BdYxNS57ttIzh=LK_Ju>F~)UARnW1Lh0)){&SW zamjIG$GT%|>Kv;+$YZBcMRMFgKgR`!)sE)bFQggxRC`}88z1;N$IQX+p%JgH?hp8C z4!>w%kP#6*7Q3+c?eF52j32QB{Hr45(!lK7m|C;CP?=xTJRSsjUT~4pq_Rc?`PR02 zKp;*0biskW?fx?@{WX^?LZ^98N+eZ6>}96F@S9WMjrunpfg06m53>wdAk;C(6>@I5 z`_GXby^SifT+Ne@ivz+5J2kz7^`)P>%6`)j@*C~gU6T6|ED@t5-!`w(O#fkYA-m@Y zV-M8gjuxMHrWObsFlk}|m~MTMd!W8S!myuXr`;4C6P;N2pYsfBIi~Pm#pZELJCA8R z_NY^%_#VX>f%s)^x%(vOBwS9>w50ctgs(BZ-g@Y2L#!tY>bk# zf_+h9$w=pJ-74i0Ki?pFvUK3j&8#iMHMMWQ6=$SEj#vILP3_i?qhG_*J*g;-QooQI zLHS_k<-94Z<%^K|y$^3^@~DJlk|R>>yRrUqy*dUvW?I%in4!5opZs7ZO1I{9Sq~k@ zXaonCz-p}NKGdQ)@;(BiWwO6Xni0bk7WE8zC_FPcWpB{_!SXjYHdh_E0}Ja_u$O)k z%@5I=jh2YHf+GJQP@K(`vQPyz?|~A%BB?)#fY7~U6Lu%6)A$ISO9gZUmig7Fx*@D1 zd7C<*IrWb72dX|R&_tPSZYTFY;z4!K7O zPxdRuj&aKY<1XvXQU1R>*U$fM-jrNgjm7qiE0Q__Ge(gz{%=6+nmm@drt3?hYj=v` zl`N$^-$XbclV2QIVq__Is+)y{ur2>tq}a3snmVLCvdvi7HPB5WI8WmPNuG-r59m=s zEFwvFm-470yEv9GATTQy*cQ+>;zjK|0iy2gdS8OTmW-_)TpH{Nk$>UOU9NGUrDed= z?U6Ss3@%!hTL`NAN?2IYv2y=Bba3&jTCRbFm?oEtPP%l8ZsV2H+5n&T^{SS-L*oC>|59Y`twBt~mN3U1a zM3|m72AoJ@!p~pLf6De9cNrT_6=er?!L89O{@ODy+n=UfS71{knDHknzKaLc2HTSsLhp4M*fd$4eiWhq( zZEQa$?Hb4A>{h!%E>AoW8gQ8%hz9KX?^YDA<_2rsNh0H9_jhIr+Iw+&(%t@c$Tq@{ zhQ-vOtc8fT(!T@e@cql5SbPUMQGXeoO16+{)HL858xZ-`reN#uhKXS#$@u}LG4u=! zM~weEKXv(6lPgnAu=r<)__;}t%{xtL-LS0z_}=QsNKcv>?dO(uh3Uov=f*Et5eP_?qGI(}TB}4Ar$uaktBHrQ+qF z?fB~P=x>XC+!aAbk7XJg&tECm>6-on8QP!IIzAcLKR&U(3WpnYVZD=L$dT&rr_J!I z69sJ??}hsVjZ?Mzr8}~~^nuqK2j&lshS|SMKa55FTMX81<>?t8tzn-H!`vFzDQgq4 zWdhNvSSgrIZNbY62ojqdc9z3&W$Muqj@;>bXn(hw&QnM7l52c7&&0D(yt_pec0Js7 zhf@C$ea`;A=D`b(5|f_MIgwuW9bG(lV<~r|yt(PZ_N^wt=7-$&TlzCS%*w#*5(xSX zriq-ct}YwRi^_>n+@9CwC(S2?@xWt+ywtvj5sZ7$!93xsTTG`Q$3-HnK4l8!Zlje| z8mA-60-`BZL(TvjOGM@kl;S|aJVhc$+?4z`ENYxvs*O*ZvmmihwF)6%BmLbAwTdtc z*x4u&OOm&EVkdh!-~!|nkHbxwEci4;slr(2W*T`sN);EQC)6Ca9Ft;Znl!FoDYtk@ z5K7o-9Pq(UZO*8h5i51T>fNh{2|Hj-LwbS8=OrR=!8dDl^AIglh_*DQ%f*#66q)rh>u*|jxzL-N*`D1{g&#h~N5>>V&61C+8yb9)G%`#;&LE1kudE3v^;#BRGS0ke%2f zp-G)hQ^c!M>~~ivw0sZ`kH0nSMZu>|D-32lQF4U>b1L`kRG%-OkLvzV2gii0CUsh& zrQzsbUkRHTQe~${HSuD8gI$7f4T9r0kbe@JmM>5nwfQaemB63PP^wsSXmzGSlLps|pY`9* zp2}c88UZcHA3p2vGJ7UB9#eMJ!`0~jkagWbO+De3Dkx1vdI^Y%N>hL&B4s*CQJ@RiOezf9k}Qd(ac#P9Zk*iF zly>k>?_yDynz#A;(i#`@uRegDq+!LScNH66UJezhuu;GjZ;jO7Ays2w zEDxV_A8_in;L+Y~E&d;XlS#Q7{H>7atbp`^Rqv>eUojPZJzte8ek5sldfvk6pI#sq zPK#-3$ys0CuVPO)AvX$Rzl^&cN<&Y(;7Cc>u)5;eHvmo8cMJXaaWC8sx*13O zpv6I=LvmGoZoV+X#DPKfe&L-Eb?g0}!X^NE>tkDgWoR7T0h9;EO-)8!E=NsmZ&IFc z#nQl`0dq!>wjQ6ZMqiJMDpwA0#Y#X~j z!~Y4MV$DrKE&jYQ^GJGx5ANH!jOqX0=Ch0=kZhfx$dUrp#4Nv=3jpMn0lHZPtUhHojT@wshjj2}5Yk zB!8EhqWrKG4%o^6nk(GyE(1bBG+cSXw#!9BA$G=>%GpeJy4Nepx_>JmG@S(?5&dpw zwhk*fZs2~|0jNnj)jMH8F(X^+oE|WmzCWpe=TS1aHvUBdo(du~@_W}C1f@=`Xyfl!x7P)gYPYS^Qh~Is zDnsJ?S;5i^$qNIq&~2sq#AO0l4q*8&GZS-KS@NIX9xcmZ`9JU~I&f&WpxN~G`;Q-- zyoTi2v(5a)XE9aSif0RGnl@$hS+omn>0*f?%lV^fdc%wDW6zQUo;J$Dnz1-ap%>g| zf&AvEAOk&Ji|>DyRY!YI#2MOpqFRrP`(3LjgJWFsv^%p(ir-h~(1B-w^kf2+tXMPH zFSSNss-;$D5P18gfAxD-_RPeC^_F#`o;Zqhw6*`v|de8UT4)k}!YK2+JcqYp?=GS8R*^`6f>8-#oU>%!J zRtn=)wU!gyEH7cuakTA%T;|F);`X<9(pj9=gRtGYhre$n>{pn!T`&J;w<ZauLHZUG|pKubT0_Ob=N4e0}Qhf?|8KmBh9qW^(7fmZy zA3O+Tz&X_ek89rr^2bFXlf+z9h2+09rCJ{%IfxD0{|$Y52{>znKNAKz4|!{wDkP7Y zW{F)ryZ`*YIEb)weR4yM9=f^SYO-X1S9`i|=N?5Aesp^I0T@L$ckACL*LQ&C)y-0~ zK$8UM56~TdyC>Zb{pDr=O9kPCkjI>z#DCG&j^7-vO$}x|;R@Rw*pZtF26+*5Iur-? zD!|LM%h?&MnCrjr^7XqM#6s@O$+oI#D1J?V*H15Xj^Tn5%i{7t?1{>q)V=|wsRS9* zqg$SfHm3KD=bBTV+)gkbc5tO#;Jnpj)vrI0ioK>A!XB9?&G}XxQAMu zn&Q{|IJDX`xmyme6?d9TVy??;-&o=QXECsuXs}fN(Q`@j^*8FF|B3Ah!}#fj=SZK< zNpEM(Nq9i;@5`Q|V$%FeV2LC7eL*uAY0uCKZ);B+GtD>XJhM4<_A}B{@{m@tZlm49 zd?P-eXSTXE)k%EbFSdU+PRb(%I)*PB;HTm}Q?(3|3b*FsonI%sr?LHtjfifsA45VPO^4TkaGML(N;=}S-GrjV9A zD~aRf_=V`n@k)Lzvb&lXJrwb~KseG)o4w#zd67S!_zerDeulPAEMmDJ_k>LoizXaY zSZ6dmF1H)Es*6JkO>j5Zw8^P2HDm0w(Rn6QOQKJu+KeN|@T7E4?CrVNWQJq00Q%$Q zUUT1lxL)R^5k5Cv5GxBdTjxCLK%0CdWd*hfnmw#4rpF*ad z^zy&3Hg=V|&xM|kUdI^qte1<5A$E%g|GZEqz*LwOlU~wpcw8El&#}Uk=1E~{e{f*{ z`k7gL+v9WmBLP%`-(`AwkG_j?eln;k;s4b+T7C7)S-^J;n5u>WuMMOsfYaBj5R6dq z!;b@d8Yj%z%qDG-cVQU1IeJ|5PD{X*8Q8vVN@iM+bvXm7l`EHqp1ZWyI-!ef)rt`b zPW>X&LK0Lz-YK{%Y$+TNJle~BD|)7xSx&n=4A5r6xj?`Bv)w!g_^Gh9v`tR4f=!E| zPPmFjFA}UC4$=gvoiJyt>8l;FDib>@BmrmUD#wQutbVK_cHtU{PH=hR89dLiHJeHg z0LUZuZ30KD(7WiLp1{AnEGHd)GdmyAl75ckghEE*%oD1? z<>*<3Z_Z2Q{3wJ;B{TL!za$}fX?s1>CR5}w?C@{t6TU)J;c4T_L_RGJlXzVdzHx|7 z=nNUJr$xWBTT+yBeWE0(NyRjM3ndbN%Nt#hzgOQisG8rgURs)>r-nGy?GkCRRF!*rEQA$@RQc>6E8Ra;If}sU9_S3wPy!hE)ssUL zRE$tvDLl!xGFTapEbpnMW{s(8fdZ<>>6Wb1GRqajrI9YER1R@qyRg+z!PRJ_!B9L+ z^Q6W4`Q_@Te6knCT!=H+oZ#6ao(DdD4;=d)b`IltCd~p*9OkWt@7RJDS)iK!-=B*O z1+Rn!@#=dqoi@x1+?B0M{)E#mqUqPA8iM+2PESNwU&ajSmj}=CA6{!?$qRX!90r`s zW`}#<{~K(o(?VYw4(A`L_yXs$VJzVQ{%H!0RZf1CHHQnGdf6OJkX7Pbe|K;pgQZH3 z-tM*qix|RIsKvY)R?n4Dr#*EQ$9wk=FYO`UbK_u6r*Z>aVmf_>WPb#~?9C%x=kLl4 z_R|*l-Pz;eYKqPw_DmI9vz}+i;BeOrzJZ7hn+n0kEZW6)F`6WeC5N5K7^n8vng148 z{U8+#9F1rdOl4lQSrH-c`oCDb@09@S*v9A^2Vuxp4sGi7dvEuCZ@uWjI&?OV!@q9g zXr;L@i7D>`7Ww8G{+07@$kq{BoqJU*_Gnk3y%2C$2cgTGw78GmGC$E7VBNs)(JjdD^(`u19(hCt zK^QHS`?+OY^}mViM3z2dNGxYr=~$%9P;f z>1!IbweZ#oC4cn*vCnhrBX;6^9k|wQrh&EhJd5v8*P1yqML+KZk*}USRSdJ!yB=J( z>s_CHEtnWB7slxclhFuItJC+fy>pIf68Eb7Y>(Bvl?M*t@6(T|5Z3qNM2lLvmb(k? zuYZ6ZKJ1+uQQVkZtm!ccTaPq=?OU&}<#6fmap1TMe@XV_`!B10r|8n6zc@%hARc;U zo%&~A?fIc^fuEdsvebFsw_$SkI+k5%jAfSDZ0Ni(ySDyT#3dT)O9LnM_7u-awmh3i zysE;51wI71H}|*y6{#XS3q!dG8jTMYh8b zB4&Ul?>RVqg!6(>l~`0JA%NXMk^gLcUgx!kpV-S5nXruJ#Nk~F=|y)8r;3d?{c#eP zy-!?62ks|5ESh(iL_!7C4RSBKS~H3v=DEcX6CCrfw@+KqMy`*_v*s_^45D>u{HV8< z-obT#&aYim`E319RR`9LeR1<2yf#Un?!qiMeA74YI(sW?95?fNoYZpMRo&)5X#%1G zkYX`2W!hR8h=EkLdX$M4P5o~3<5tO>^!D?2)b288g=eeRSoA@%RW*r$=x!|bC10p= zWimxVF2n`dL?N_D`gm6NW6|RULiX^il!iO9L2BTcS7?OY5gm1^Dt=wrvqNU)(II}n z1%F(kfF}@~<9GQ7gGt6sL8R0MKvh5E8u6_C_X#ElJHv)wPcLr8BZN+mr`ZP%*xj3c zvvL8;L#^k89Qr>nc2qgw*`_7%oGSnms{yV|DDu5Y#8?#C1wB0*b)T*GG57AXg7Lcwr-DZqiFU1TlDQ? zj9d9y`C`asRG}sJLQCPt&+dLPwK3+~jyMV|(_DC|A4Ro8UtMK4DG)O@L%w!RMc!_2 zTjsxEo+!LAbyhqccH(3BPBZxjSsV9v+1&74r*HMf?XCH3uaEYhJs2^&y@p@oXoOLg zXNin2sB6JjS6L^MObc$w=Y)$If91li!VC+Q%|O^^mT6u`lToPr+g!tM9Ks%K;tjg3 zjLqdm6!YJ6XNotEq>|;3%pA<-v%^{M;r4J3wrLSwnyoL_U84hR{(DM5q@7gq9UC9eZ!Ao`*8^t9 zRkjv1t}beMfqm1A-~pThAs}`jAF)kYJK~3Y%xvNWzVUutXpiM~+9GK6qnBsV#NsWA z@^D2=C^X8u3wc5HHaT!)gdYFYk&H0X-fMH-8@l8X2khdS$v4t(1z^ zP?@C5jde4u0352rLye73|| zdb|RTjx1BCEj+~*oO<+j0CDkwXYIuoFIRPd$?L%dN16C_;pj{s7ZEkW&i1sG)TFNA zy70YOTyV+Jn~n7YrbADJrUvvRYFKtGjUL1AC!6yNFOIJL#OvO>umF0&T)owqeTT$q`m7gjhkNogR_3Ct$4Ha;L& z4Zt(mCJ?hBupEZyeS=o@D|@4EgaNhabqW2p)P!; zL<`^7k4!mg&w;Tuu~uQu&u6A1t;NCt3sxKgpiT9!9nhL!f;IGTeSbZ-q{?OmKeL|l z{^mfE=8Jr`IQf5W>wNp&CAVQa$0xov`~5w!qAG9t%>k6~vEv4TYz|@HA><1K+bfUUb8$8}^*M}FiuJ(d@oO!D zzQi@jY%bq4<0MZ?%}FO3VpMy-mcti5E6;J9&oMis!gM}B_9EL{)=B-qR#+_lAW@-2 zbaP3XSn3@LVDXvVav0v_b%7@K%R~FQl@q-1bqJrri-lt9+YVzy!>cM{fNTh%DP0&) z?vk|{Wl%Iznjs~Yiy6Iyyh6!>5Ag*4sj#k<%7FFfKg#m`bbss>Y>B1&r@CJ|?ZW_L z7M_NmW6@ltplx?{qF9P%k?B%mIa_J_S-ufvV6guZY)jl^0H~`W~5ZTuQ!EhbE9YP!}sa`RQv+YnD#H&iESr}BgG-kc((he^y{WS7=3bQys;VI_ltGxtpW^&Hx84i8#!;SH{6xf7Yl33Lp>GwEp$zh zw?a`?*%9)#NH$q3MXtc6sL6qL!^2$1))~CPiFW3vx2Iyd+DkJ(w>h%z-HLj z!Bq9b3W5kZX32g>%nBm0ycMVm&fTCJ>%;T+0lCA&ZxQZ$ILjOR=+u#QC~9Uu$osn*2UnHa zXHV$yMIYm+0LL?%Mb&TWAw;K}dwpQiD>C7Hovg*yd1@sm1eIR-!zH7ac@?g0lFHu?B zL1(I)VpUinr7O#dG9<;7X-tjk+vA~8LyK^Fx%| zk~_{Lv9`f}owf1*iA~Xef`3a*#?!)@E?`Nk0YEMLnJf4&OMR_?CqB@6u9ry<14HeC zC|{2qy#A5e@NW2?iUeH?WwI&lXmd%K)F zF=%?q9e*5246jvr>4Z6n*H@6=HM|ED7&hbDwNpn=>xaKNKCp76E~()een`~Q%`*BU z0h}Q1xK5YCR(ssKeNUF(ISHbPy=wM?NgM6cf|kz}1=L0Ir~h&kFk0tZ8{v={k#0Fn znB?hkzt&*%o@k8(773L65^s9&&f=|kXG=4grdDxoEqZ&@XJ*Y7TL_B2(8DuhAT7F&#!4G zKqWhVpS&3GxrT2}jnMQ&+&&&4aphvuO|hxU7N0kcy}n)le6m>Fvv5_l_TJu>Whh&Y z2$@HvHoqTz>&I+@Jc1=w{%k{@Z${6ck!%6)sJJeQdKj?tnfB2L6?hmuaX%kY?hJ#4 zXDrlk<)?o|+V-dVEFc+lued3kMMibON0iqdmlw~ioR zC-{+%CM_nxMd0$e?v~V$#D3b!89SC}VJd>Q>E87=eb2tS6wNDty?H3 zsymryp`kb)51TSyy>!;u17G_te_bp8jayi|ygcD@4IAdB*y~v7>lt!#K62MVum77h zxO5R6xBUQ%+OZ|Uu3AjWF)}=2dY2qr6m)p0xY43+3uMG(9dL!^p7^7yp+z6B@;`rl znSSPB{Ef@p*Y0!Qy>l&oq4kfX`lo$8zV~D7>5&4*gY?FM-dcTd!T|6thG2xTJALl$ zU2rz}O>N!Qgn5;)YhY9=9-!$5LNISKe@lYD)FDyeqI_0gZfdpTq@`IgcKCT^-o*8C zU}fE>gVUWfjMY=Ez^LyQPqi>Rtu>e0r=&uq@7yZS-Vdlz*%!ZN@T;qMZxJ#nZsLb1 z#BwaHFsju2_?U6RZ1`J!IiE=MaTJznp|!vF&p4odP9IxNEH_1Fh-K>?B{0Ty7jxz? z!+MH2-N`e#AAg~3d_O$Q@FFL8KI$qK8%9!Jef&!_oHdH#a-WJ;h*)|KaP(jEC10B` zyqOhgo|)s{wW>04@zCFD`8msQb-sDp+0LSt#|Mt}xwFJ$NZzUO;k#0&ePJAzpASAh z5Oee170LgXmT70cTPT>LB(~I5#LTxQH||2Al}r#cStAIvI(fY&YkLMSk&M1A(_4vr zc9P{95R5SS&4s@ zAv$JT2yl-n=Mi%ocwR(Iu(xDp$R`5%nzmT%{-qS%eISu`dthvZ_KBs2xAk}+&n&}> zhbQgWbmQ@nRV%pm{=Vi>U8`gb?dhx`JZ7{h?WJNtL|yfLcjhdOF!=b^7^W2fASJZA??~c>t76+dO@3M~C4F<6Xbqf{9u6FFp)-0EQ0$JdHD%#+ zGM=K&#l5*-`Z!Y|DfOaBFj6pkb-PUvTRc4QN>+M(X1sbfIJHWiw=jhA^`dd`_uuYT zWvjn1L%tpbQ)#=7ep_>QJK2WiQ(}hm4DC3O;l9*vX#5_uhz-L;;dRfK6`d1Ayuez` zHE?)#d0?kr$9{s+1s-%$na8RXx5$bAB3fN$R{oOS#ZKDpMhHg)aeS8e3~~9O*9u023q|wb@n}-ZenMqwZEN?mOkb)>?h)?WXd` zZX&R4#1i`t8rxBac~QEiW0U>~RWfDSGd1F7USH-4X>I6KlBh*6*7o(ct=@ApJFBRk z!gAb>)y-?03Q?DxS`3KKZcm?2D+rL={YYf+bU++}R(G)!2RDoL(s8~v$w9$TMDP$v zxaVKXzZ%!!p%+j7n_3iz%LO)PvH=VVFH;x>+VINUfkha%&$f76b;P1b(#@eaA4Nr^ zlgi89n>!cpVH-(8D)Ogl8}eU{E=kpqHnRJ%9% zHf!eUVHppgT^OTPf_J|u6r2DlWMr&^b(ju`(9OL9UI40Eio%yqc<^P)KR+2lL=14s zetcv<%C)!KZoa*n(Y(r8QLHC~H9p6YQqjowf&n|LsH@4KI=}_wp#QZs!(l*Hc_`J! zg=uovh5@{A@lNxOF=Or89O7KbUa+{tpR_6<<$Y&oMTUbT;?pn1%BJ zA$<+pO$2k6dpJwMCS_#So!UXIJ3iAJf2YU01Uf4pcP051-LLg(kvQ#EFJA0^#rHNf z3UP4Sf2Veb8K0r794v)?HX;qevEr1$Oyrea4c_!gqYZS7WNWuRn{QWi8{VU8fl0|N zMt1|-MavL##cLKK}zSxe|YE}MpNe90D@HAAST z=K&4>$t&-FAPjp$Nxn6}d*E%HtloH`Ocz>r#Q;(7;9n$+O>u4tWd+-ucv?ofpNs2; zT_)wZY7pN>y3d6bhv>4+?*^Kp**LuF45ei+am%IK5)v+B%NI7YBlwzd@=>yvj=YG< zM^m!7@lA6#T%99*CpyY@IKP&|`i|yvk)cgPntU(Un4NW#9M=p`R z_%u(|hXEwane9dHAvJpo!d>hS&mjyCXw^-5r-}1TJ8(T{bv6D_uB_9^(C(ZovpO(4IR<^G7eEf$gDeM4QW8zzw9`aEZ^mrZa>`VI%iJSo&m+ zekgvKBItT8)EuxU5NY!R6$n0iEJ5l+dD-Q&?RT9)W8X=?BoN}V{8_?o$Qx?jf?cNs`qNgDY&zvt0HuHYUGG_6{D)LV|E z94+1}il15KC%|_P^I5n=^nAeGktIUJX5v^o)g>Njh#*QP* z4p5LxW%1Li`*g<~u~!gwVxoLE=Jnp;dT(tm9Z~c0dgdvL&4Dh70cZco8%16LTq|t* zcI)WgZ~j>QKIm)e$+uhk0LwvJ;TvScEzA|l48>+IU)Jn^KcnS?^I>9>M$^%PBioYo z@7vMeK(Eh%zj>)%S-h01=h#e)GZ*E`4NQoHA(i(As6Fj?;%(t3vDr-d5h6VEQhMiF zQ+W)FQcR}jR9rxw)6S2xG}{G@M{4oj%jR!GagdrDUIYOJhU6_tZkm;J-3>J`%_eey-WIBHN07CygJt zGlu`t4++z~Efp{cfpWV6TW8CaJlo zCDMz!@5;hcFN-~&Y!I;Xud5kM(+gUs=IetRJXE%{WDMb!%4*n*2RDC#Eo}qBG>nzZ zD2-u1F})GoAGt1aK3Df1qF&4=lp6y{CLxc?pt}=G|I;+d^JO4-6g#ft$ zwxXKUa>;BkRJ*X$^r4;CzF0Hz(4aM%&#q)!e+rf(c3r6z9k<#ZGMJG|HsB}~nM(5$ zqGBzTEuij@55sPFOfLOB$g=}D;11YdMp@~wQyiqKI*KFK7$FpWLITAx_$4mR%W-V?+rDvx$>QhMMS|d_ZCA& zwE$;2|4SPsBmCN!$nBw9!N6PDWo6d0YQF*=8SToP%crLPQtGTJ`&E|fY?z42A1}iO zCMgIhI=09L+}d4MQC_)eBRN@Dzpf4~%E*{W#V1u!KTm7LX=Hc@bBUlIVZ1#V4_ql~ z9xi5F8lE!`yMFEldCK`q5Pd+wKJ!9_Z~N!M7It>-dqA)Td#?G{Y5|rNH%vYO+%h88 zY)lR0T-S?i(ge{Gd=Q;LH?(=hpW2T8L8K_D)|ZD=MG5|F?9BNVA<|Io!U$24Ym=zU z(i{4X;&5#M<-El<$Gwoz1?~P0$2HanBe#d%uLeD zuL~m;=JiP1p@Sr`gCmglhNfv&6J;zgbyXs};_-sax!P*LW)y>|G?BvQ6ejjJTkb*X z`V$`@|Lk!^gl`-8pX||7;Jy&@eU?&nw)eJ%d}DbFI$$%m`4?v98P}?rJsIV{;7PQq zJZp>~_pCn{YaTDO#X%#K(+{^a_YSWBNw1r0lDC+mg_v6sY~K*|LExUt9~n#g zN=_IeHp=B{niJQVT5Ow{_Yext6q*6`V;5NWf%PRP8TCfL4WfexlZrhz|@YuX=)zNE?wz#gM2-s&h1&XbQ;FZAqEZo zijH>_PIu?gy)gUzpAd@6L+l?j9|eQK@^U2?PT9 zIh5qn?E>7Km>qbrR!FYs&Zt5BLfM?0T>!4mvuxw`A>xSB(+v##~f(mzc`J zknfD1y1;Ua3;75BBItmWG~3O5%hYWizuRhgVBlzZIj8Y?(@OwzZy@m7;m4qWGt3O} zW6+jtSd|vXQ!UY#&9}6Vio+u;&#^4CKmj`*aSVl-3&QzSw7`91NIUfmF}cGS{C=gb zjuJsPVfKNSd^M*T_`bjRyb!+NWf;Hp6y^7lsaWh@pht=L<`&xfn7O#@?uyOb$J8^Y znq6@c8338P_gIUgK+>P-iRoS99yrAKvGR!h)A%i=K41*Tv#c!3ME-pLY3z3#^OHK@ zEa~oJ@02fG>&(S&!*%PG;uF?L$nz`t{(o)HP#2$3@-4?S+mz=iaP^}X@nqrfDH*PB zT8rR#Jut^3i@xvFTgDPg8Gi1;9o2aKKPvAcEA3o zW>LUN{_(eVfpbt8&U#UEs-oHjUww=T7S<5~TW)!%ekP6#@PlCuWc;mgZQ z>-Pp4ziYThIMuduh^TT1e68kX0e#T?Zld!tvlVx)4UbB*X2<#aT-7?4ME-L-qQi6~ zjcZ^qK#<@puY^98~2E%YloFS^}_^#GdzBjK4%)~uk|8u!c8~9c}B)cHB9GSjZZLxmGety_~lzgRyQD5nF;YqmGyEU_u z);`_C{&;A}`dj`GMi*4cAG>tp=a}HAjK8qN_l|LxdZ?=Im`7X05;O?%w&g->wXcv6 zkLN`)p2Pgjcq5hf#JDoK5i%VhADf&IYJVrN{pg2X7^3ZEm};W=ghmrs*;jdz)m`uR zn?=WS|0j^3w4vG)S46P4Zf2ei$(I;ou$&mBd$L{c`TfWPa*AF)w#@u7~g+92s5)g|EF2U_s zV_E*J4@2u)NbrlER^G}RzJjNZ6=`aL!qb&07g3M2=s~ZrnI29nycLRUNv$msT z4*C0S*;E)X)fP~(Lx-$Wtu;h6Yjk@_-$78TWYp^#KXKG6GP zf8V%1u|+@65>n6?ghg{#Ms$b5BP%R?d%fFXplDRmNG|PbMav~lSItHhdsE)>d)rT+ zbW1=R$ZJ49>~BIWd;9WR=uV$F4L4l?xF0D6U+N(Uc7kn{iY;$q8&*+Dz1IQ16I!U> zp48*N%LLxsf7=f|Rfxj)A3VRC;69y598MYw^HC89x#?7nT|9d$x{A7Fb_^#-~hrh0gZ`wq%M%r{f<~gsv zc5ThW>20gB6md|2%1t2MXu22Gw+BsL&rnq${7tDS8mkGo_D<%Yf!|I@zzI~gZZlDbAB9RCne&%j#JEE?Y+$LD`;A>7T?iY?6b)i zJ5OHOeN*?jyY|bq9}w+7E7agCM$Mp{^xMPOMA;LLjK|s12aZB(4mi>KN0lm4Y31FA797;>tA7hq6;A=xYnCRq!GN zgBq!#M`v{7^7EPU%;JH7sx5!Itr7E^7*_3}nWpVhy7374JttOe3Vbg;kVPClPwPJp z<`D)0JD2yh$6v8baa?++spLNtuf4+d8}UJWD>qC~lFfNsv&WsF0CwH+7b!gQ7$k!fdD<4W1i6j#aC*+ym#+}Z@ zmNUII3D8~5(B6p#w%k;_>^%Yt{CVLTuwsAqi}p$$F5r*SWfDmhogxi_g=H*Cs~1dYVv!kb8rweNTPI@9v&^}W#r(;8tfa^zelfQ8g_?0c6P@jk zwqib)bj9YNF6H6_^Vwz?DOSRQ$t01ZolNm-N)#Wjl0-fI6tAx0Jt4(PVu*90iH{%k zhzjzDHj+{S*Fx?l2SVC1#8)1OX{pMy!VYwA4shva<=ywNzU;A92$H_ko`E{zF$`Jg z-gch{>+}L`L{Y#ZrvkEm~&+aj=mtV4oZVduzFH!Tig{{N#vRk2(v`o zAL{U~_`sPVjpTNDjp8R@8}fA-@SSo9PE!UT`kHzEO~N>Kz*0H9#@-UjW2cEpt^SL*t-#*64s*IHuUPPRTTyN!UWfU zG6#lEK}#CiQVe|YI8s}0@!U-6l`k6qtptvSn4$JmrIn`zaR%jlOp@~VaXY844~*n6r%DFSxJCE=|0pH`a%x=MzVJ|TAuO-BD4g&Pm-o+tdEA? z8INba9w3&?BIE_&0I(Q@`;&%Ye^HNaeBF$4sq1o*IAo5Sl@CdCq#E zTI28Ao6?Pr>x;Xn6WCcFS$}zecJ?ss$|u$E2tJdG6~e&U*^_6i9vp0qE!>|Pf~4wX z*mdlTNO{qEd&<;|-^RrfDQ8g1Y>dQ)K@?&$;qr>ev6iLqr;NexPtNrIpk5Oz^dyRK z~wOD~={oK)*4rWz)e*fI4l ze)7BdEqnQiNTd4S&C+Xq!kmR`o<_^4CX{*LZ;vWf=MSl|qDvXSiai~>n>0>n=)b>q z+-(X58vIN%`Z-9DuX>X*kX)VzaDmJ-i;(IHD&QaPtqU*Md6pWsCM3YM0Tu#&4ybvn zX`=dWH|d1p+C!%%>=YMCOa5tmE#`xU$-3}r?un?8`eR2#M;R|z%bcRKqTyZgFeC29 zuI!el`KMwQ9aY1YuUjUg7mpu}#+;+-Hv4iDv#kt^rnE`l5G;~W$0mJe#kAqbpST+7 z;s|R*^6{7>wL0C(C(mzfO?7Lv_L`Gi-KpBDYK;3|k6F#o$@`@<;~g$iY=e@09J>92 zKCk|1N`D_7VwQY*BJ;O;a4A406%y&(tpws-w;%hZwX6D%rOY-L zB%m2NRPH17&$Yzi8CU7139Nq=+Iu#9$r1W>)-Usb9iWm{&hcU&B0mWmO*J;j#05+1Icgv-KLDK z{hIQM*&Va-ketjPaKkN@CBq$WJ#W*WtOb(qXx5_92us=(|$PJMB zuZo1`J7>cG^qflhajY0r`yW1`4ZugK@cTs@Jy? zl0EyJYwevdP1WgHY?|}?DBO46`LEX&tQ6nx&}g&oZgKWt9yC2PuV%M>tQaOYpi z1NBEAe`I=|${n;H`mIFuf|hE;rB%mizb9E`ONCsY7)nbP^S>nW*BKwFHo6)W+Rkn1 z{`EF&auTaY7&)F}pyUTH)QLP0wn3OUx>%$dH6d4YN2j;gbcGGWLynzFY;UEz0{7zt z=OEI)6cx20Prr++g$H-0c9+dRoA+U~Qm-ULeJhr;fi@oV;7dy9} zT3xw1oSkb2u!*&C*hTjwEeB+3Y_xl17}T}+*{dF;n&=Rd^6ctXw`#3zbIOmQy`QMR z;x?1Q=6;S9>{+9gj;b2LSn~RFsxRL`r?4{nUtO9LT2QYte*5stNaZjW%q#AD*qk#p zPO@q^+5h-IBfe8J{^vdCHZKFO-D7KPK79C$@6?YC1G~WnE7h>e%wiU$_JKVsb4($7 znF-9ASrhNw_=d`BiXq@PoV7is&UdI<+vR!^u|Zy_w-)WZ1&YFl5&9)S-h*#lKZq7||MIX@~{{>#bmS(t8AFKv=#N--`B>&8epuXF%w8 zlA*`2-6H9vQe>lQOt#W(vJ=*t{)86~Q0hT7 zF`Ux6|0JFl-ujfjq(@0E9%;Zpo7Ugr(AX@IA^rG|^vOVF_ z2~(;6o(+9q&YPhjUObD^fA6Ii8NY15Y+`_~DSSe{3Pv2ME)SJD{_C-FhcJ^Ri3%Lj z!>IVV$=m#=sP!JX24g?6qFci-N>kc@rJYN~6QjxB==+EzZtka65HYLQbqnWf@K;|8 zWyq$=k9;<#t;nxd4FgK@dvagc75o~l%AuDfn8`-#`^ufchIu`Ic)MXZbLv^)TDx-5 zf`%r)mdRfAlm`}Y2!(0#!*B6q`>XRKb$UPqPpt6a|4?-0@lbx>|67tmdq}Mm6_CqYF9}%68UvZ+O9%pvC#?!gs9h#d9x?8DKPkF!OJ*dm|re#z0M zux#^O*@%j-;40$WwHm zF|jn-A3Xxha!zA&)-%MUvH|254xWex(U)rZ4~p?75i}FgAld-er4aNU1nJdr<(RQ% zsfW0tTiYk&Z1)S&eh56uB=S$A{sv0~F(#?{GJu~a-&Ed9kQl{%>zPEQ`nm`=32+&B zZpXdPN&(uH;x_V-|3187{h8Q^L`|6uyrS8*;DGmldoD^+M{H!eLec0SU#keLq!n## z!eoZj?oA|l&54wso}=oA%5(lDTNmm)@-cCZ)n9wik8WutDwk~BV~QV zlOAdfRG#f~-6fb|Q46i^uX0V?_8Ml*3L)85AhQ;+k%pF~{CD%Feq&s!L%AdH-u}%c zJHvO{rlF2`=0JGm1@o6)L))@4U5_;TYKk3lVOFT`P6l&^4stkXjZV6k)8PPf=3;!A zX6S^dr$lCviLJQhAR`q=8?;+pfD5vHVp#16kH4Wx>`^=p-_Di#(hRiDX6CNLu(M^| z<&~byMyu<{Ui6h}$b_JgR)Jima@t`LW&YBRYmK96ca_5AA^%MBO>S7%EY)+R`e6+e zeHg16cerk!JFIEaERsCo^a$^7m8_C+e{fqlesgmsPnP+hqdK$d^r=E4mK|u^niSuM z&PqmeNoqXp%#!ft4snT>ZP3VD^Bxp`pp_3QexYUUVUletkDPgoDO+@ycGz=~#BZ;s z*I+Kff8Igz09~czhiATtX&t8cJ`w)q_++@FX~Jv(mRGkcy_CS)>_i)B5PnD>Kjcuh zF~{8$bvN3Pqp=R{aXKGN&p(-;8OI8yB*p2Z4)W|Z^^Fadwg-=U4L+Pz`jnI@!c*-c z2jCqo{OVhF%-iyiAlaL)obY6PTj!^P)DqLy=X%JweHk%eh5FUceZR-q;!#6)4y5?q zoXSHYHt*-rU+5Z!ZD#~3EB&b|pQ)gFAFLI7MaQxO`tHi3!6tIixT$f1vYo+%XT+yz z7g|?CP}U|fx3#~rWq$mhP^r#v*^?wR=Y=rHd*Jwv<51ni{RnDs(z=sfty*8eOMYu>K{cVzGuq%=VhF!^o7j*FTB)dS z)0j@}^5rHU1l$p>gTELKt{B8L+O1i3W^4BHAlCi>T(!s4wAR4IlB9p+SP#odGS%Fi z^hp%eOf|AB;GfwI?ct#YQ%f|8&?{z~a@HPa^|mROW?_gx)--LGBW{_ubX`*((P zV3p_PNA~WAd;ckSvYcdfcH@6P;zMJo9O>@gcdjgN9+_ZaVzz=qOGe&sR_)zq2Bvjl zcyzn_qafN_YD>wxdx?_N&x<@sp$2M2dSR11TPuZ?k)%&9^!4naB6RIZ4Ls~uYgK{& zX*w`fKIxOvk@kt|5)^b?BpDP~`z^}_9j5e8)rBsk7rBmq=c11C4gZ?JPkMOVR_}rq z7z+-4udi0Hq%(;h=!JW080|I?m=)mYS~hu;|E-^cK33$8A%A3m)s<&Cs7lA# zoC?@J3#IH3b3YoPmb~GD*234jhX^DdnLzvZJ{I+FA0hb<)66k0%Q1-gKf^ID=*DTH zLTB-O2RCz$BeZ&PvGmluV1JzzR%Y48klv{c#x1ld4n#ozm#0hU#-%8qPS||iv`^QV zovzwwj}Y}W;UK)Ta+zKC&JQ8P@6zmlm*^p0pEfToich@{HSGHrzSJi~N2OXO9>)4Q zu8XWBB5{pyldx{)GmQ5~wGpz6-ZShadQdpyr5|>vd%R-_?K9Pt(6tizPaQEP1e#K1 zQ4KTfdpCDbeKg*ClGN3_L`-biq4o;V>ETHAokS|5D-jji4Q&fLvh$;yS0lI_7z2aI z9CF}g8)r2~>b>A~x1fV4)Hed&R~HRnlcAnm2L$X;o&zXSM|24FN3s-IVAEACA`(J~ zJ^7JcRd*M>+)6qSSYasy)nOWS`(-%ucD4>cYkfooTKF=c&_Z~JMdS^q>ApVmeK+Bu zE^&UhOZz7qB4C+f2R2q+0@4P909azyML){*$Oe`07H-^fAb68{57Gz9-(o>|&yjm{ z(LONeY|0PG!N_mF*>vY&zshxq+ZH3Edr{mC-(8Eiac#anPfva@2lg2KY#;~b!X;g_ zWrE)%<({1y_%MbW*S{iuNd3XD#HcyY*8}?}*yy5BbgxLFzHg65;1U}W1G9uve?)q( z*%4So5G{S?04z!xiXlJj#t&>*0^q&LWH^1B1#msE+Yo>~Mhy9YVSVn#%||;NI-MPr z3IS3*36K-y0)>2|krs~mjo&W+ACMy12VtZB+G^ty#BxNwo;O2s_tP+nh`LihoC_UjP%N$Jq{H{5E#P`u+6(mvagidQ$AYrFNGLh&V@2qLc zDHfH!X@*fu#4rc6PQp{!azp7dK+4m=KH)qYfHS{P=0bIHehC2BLcze5j zC!xSNdu|?y{)umJICFBFuN0A93!wc=Z@afqsWId-cNH=7s|Wfz~^g`+*tKF?shKBjnU&5qDy;}=X#X;wKVRrkijERc zyf%>U(5mXr8r^p3TX< zg{NP4Ru|qlzJ7VD^J~`wP?&21v8xX2hkod1BAeU^2HQ*CTK19=+cU#k_0%g`=ARs_j$kVUtxd^!b9m5Of zuyrtKq-G>OIm%*7tN+FFS`lzll}CI z>$@zDxpr)7G-N_fqf_h^G}QVnm&S%2%M<-|`&H1*>X9Z%gAN5{m>l%75-q5YB`^9l z?WT)&T0_?1n_AuVb`Q9f59XN#Dt}*l_`l}fiAS=fEITGCCf2(Tkp>+Ge;&SMehjMm z+;GPs<~{dhnZ@d{?M{@ko)f4fH!>iVD1B2rcXKOOd>}y)jt={%5s&N5s%*>R4OAT7 z%m_o*`|Wt{AzCsot@=+V_IV^#H<;5HHSO8&&R~6hlb`2QlMaTHUS2JIJzDI-GMl}B z>~)zObj{TxsSsh@3B}`#_H+I{eHj_=LirG&_>_AECY8kRkyQuEpowd_IHgyRl)5av zKn7uEhxU65+lS_PDjqWlqxqYb83)#~p?Ye8)e85*&1i`gd!ix#nQ9KJOpd0&Vf8l^ z26Qc4+H>zCoFg)EhjXGeQ$lB+Ku&{Cxn6^sh&nXn%Q4c zo4A!6$rMlURbD!c9m#4QzQTBRLMIO+mBxz!zRgRi$%uLKM&>bGWeLzwdpq*!THovZ z+Jn@I!#BY6}%@gb{6oSF7Z*P#wgPoq;APpF4RR0%?+(~i#o1xc+K>Vi> z&y}wyd%tU@3sYf$UA2*b^8X^J&j!2S*c{^1Lv;M7J9zvP!mh~lvFJu|tQbBM#qb;G zZR#C_-`GTy>xR|;2kO^A6)JpUXo2^9Z-75}`!H^Q!UXg%^>2_Y_Jp|Et!7ZOYcFmD zePHfEV$ddRjgEXPw8;IszW8Dj{;ybvb97WtuFYlAVNav=WpKPkD#NIXUtUwA=0XSr zB!9!G%5mx1ovsVWvHI9;BeXMpW<>5|@*Q{ofXm?O@}8lDJEL_QQ~$&!Q!V8jfu9hd zM`N*KlVon-Nz;Xpq!63i3CofqI@Y$Iw4+3#C4~=%{Z7T?WUb z*KZ}<>D$4<=8e!AM_hVRAN^EjPF`Fl9c4z##alrasOCef4o+}TT|BHid%9UVmH}jj zb-euJKZMzQ%E^2|oRy5X%H|fFL0u!y_69yyq~#NTrS{|y-sw(tTnn77Z##F@&|L?7 zaLB<2+E_^-bX9=}5PlcZt)rQqkT2!_8@RCjOlsHcCcz{?KE z>Ff}?>!wI*hxb8yh-=g@p^rHM;z4E!S0+1xG2Zqc+nIJK!sCzHmWE!HSJp%BCaY)P zJ1t9s1z)6EE&%^Ip4F2)I^>Z+Z#2R=rK3jMs^Bi;8QqkL%5x!5Pn&SF6hbvf=u}I* z^9J%9LNSoKeg*rFO01SZXPVpvN=HG{cMsoPB<=U>B$yRLlbKmgAKRna0|QNqqebeT zX8g#uZ})zE5uC2-7w&X6a(D3K;1$ZPhudLh5#BV!3&4(oK4Z@2~HC>J==* zET+$N+1p1nvg90SGh6{Gqff_#ZIG`}t}JN@nJ)9RXflH@1{mD{G)c!;j`^EqTd(uh zPj_Veg6L%89oXqEk!``!`!|lCDZMFJt#(gLFCZ##Xb713UDN{Qjaf zcF&TLY0b{8n)tL)PULQht&@vMf@4}sdE4`|=HF$BwN4WMWS0}e0@fq5OGb?y!DUU6 zYkdWKeWt07@iq#XVy(RPVWqr?u9ne4Yl=uw*`D3RZOeh}57AMPNblCA{`dmJR;}diAW2=Vs$rnlh}ja3pEk zX|CCIW&-?Nr2rSTAG#9ku^BoCD?hw>$u0IX#@u?8-TB2(^zRV6ad-Cc*ZBmruIid! zo`7##Lf>QCu1`yk6A9A5I_EN_E`-Iu)Nzs5|DRNO5ZW!Q57It()7biQ&T(|YMJili zs(j};)iL7+>100tx9dxRWL2hIn$P_S+0f$_tg~8?pSf1S7lB{-k2l`n@|$r|t#8;S zv|o6h3sG!f)44?j!EE15*}v+)gn!jIJ?_)zlxk>2;%5SR=F+ zirWxB)V*y}Ahjm9HMVqpAlmJuRu3)U%u$ADZAKjXXMy@2=0z8y#JmGHwBJoFL0&@D#@kY@LCAZR-KFyUBmx<6PtUDo+(QQkDg@AUzuey-JHq{T_W#Ty~` zAWXij{t2YX0ew)VZQA@&`ZG0b%IxK>_Fb&StiV*v=BFV^X;%gQxIPn8u9D-?(A#td zZf6sN$;$G9Aw%V2UGg{Ij2~={k`U;uao@9nrSEq+k5%$gVUMo1UJ8-g;N=AFspV1O z!!FS!>c*ds*n5I1XOQ7(RKb5dB<FI1hjsiRKE1R5+Xc5H%+5d zc_Gz?&nOmp5-P@^`#bymzl5y6csI?UzrY?rLS5d!sBwk0Jq6)y_XV4_@>S|2rWkVd z0)|@pgaP9#8w&sQGd5BWdT1dND`QUXs-Ql>ef9Mj4`%s>bSK|p(f!cB7j6L!+vN_! z`^W7O5&uTF6(?W*jxagpyW2Tu%ezOtn{Jv96i0-BO4@pFHfoW1a$Tia%ALxY^zQpGmzUE%cJM2p;yxD~%6AHYIzJ z_w^BVk%so++aKHNOu4HqmqZ^lho8Kz*%E5a_fvC`;{NSp?`^2`cWOyFdQtGf$!L)M zf-0w<|9qPJV>-s^QP*DRjgU)+eXA|yU~w>;k-%Z{q9^h4M=a9KP>n8b_#rglP!G&L z2{~XY&N~||*6%TXGRfzw(4$Bf#sE6C5)vz62!A>fZ`yl|dJB9Luyva7_a(o)OPa`u z_Pf~~nKPj&Apk<(un9=>-_;INQxgDHg9w`vhJDu6^2r1Xn;X;LRZncc!eR#pp=mH^OEWU- zOM&Op73ct3KQ$e2%Mci(F)zkS)vE3hm##w9A2)#5!I`?5A~tX9wKry^S;LPQP>@-a z@3WbPVD_Njop{TN>vrMK+cQ2EctdBs*m3jaEZx@2-NK0ximaf&FS;=g{Jm%)i$|lD zQT3W@M;7csvjzR-(*1mFq(>G}5%yhrDy-yw9MIaPSW+ud!6FLfzcEhbEtoM3P%U59 z*50FT@~%TG7nX`X3NyzgbuFV@fOAM$R$Y^O&qLU8BjKjEZBqK45o|Nh0^D5JcX=s- zBFL}i`nyQrmZ3Cy%w!*F7FG0ce5CwGYg=TSF6$O9`e(3pvA6KgBkN*-nGK=xdf#pq z@eV6_oAlV^Hm+gIr1g=};@g=#)oP;JYTA6|FG73!+Sn5(fSF~Gd5kYWgpe06{ChI= z=m{29WJawt03vwLRT%KUalDq#4y{y`yur6cLzvvEDA`q`FA7u>{i)4Gx$N}dh><@} zCjIT+6&A8DYLvZnxgB))70X&pEH?l;6t3uRr$gtSeoHBFxQ0i zr1D>**VJ&^B(eO z=cgyp=awDULqief^zw=)8a@;GZK6bwpx{Nch>mbnX0|I&&ifr+w1^g6oL0PXDfGB1 zFXKGa^{CBmJf$)xBsf%%7X89(tKuinYf44q5@;ccWnyC4y-ahX{HHpqcf9Y?l8T9Xqr1yC0{9jvyj7a-ORUxm54z&niKF)O;JYf4VL3h|{^Jv9PCU};LTZRt@2z$H z8ois6jU$eya#GkzN5iwvF#RmH4g`oGNO+_U3(|_d;CN~fe77s}44`)ODAMgmr0@b9 zbPL=w1$g)8zI;t3`PnV-`JLgB+^R~9_KXHk`m-+$xV8x&{bZ?nOg7IYUUbOx9r&d8zRSz; z7H#3$@xddC-`I@a=qC(fqV-l|wZcH4-7Q+epWlnQM7a*9#q78OyIA$?oN>4^&|JWL zOSCzsDZcY5_T2o^13_zmNnk#Sq0I7UcUpVY(qDbk%CgkKNb7szK#;2Sq6eW>J1Zb| z`a+FLwRaU^Qod5ab^p25%w4i=Mp!n@b17dVG1SzYJ5{OmPsgyk_|)4rYs-%BMVlA; z4?JB8pO^}cbHR$G}}F1G#0qug>JbIdfuorrCojyWHDtv{>k_pSycu#-f1|T z&X7M=-`QwstVTJ9Bpg04vr^U@$T308EnN4_AAAj?bDAWf5%CP(4b{x<41x-n7BlJmIUY0i}wNK$AjWp=GkIi-uT<**wdL6i}2`@N{XJU#7*A zwM?0+Bin!qjll7<6?yq^^=l1ww`BOO++5AU7(8w}_>Zf7=h%Z65i z(O+&f%`7fR?002#emRWK&0Gim9es>KkVQfQpQgRPRO9uv?eNi>5aK*Otn*=n2hFCMkS*~(xSxu; zCC8tq4<;K+%MyxMC%#8*dYGz^zz+$j@E0*DekIV7@c2WCL7S?iO@i&4=Gynjn;RGR zPGlSecaE&%x9R5eIIzx!$zEPs;+w(C9zxh=SD!M$j_v=7s$VNqhGb0}Q$tVqQ$cn` zoRvkg;mYKooV|>lhL0YAnpuNCk>4{;R>L1=8&+G$J2rJZIV{Z;v<}1NHdv%oRusZ? zb~exNd)4i(k72{N{cU*jw%p-m0s>y&iBjfW;?DDj{`+5TgU7O8Xf;&(J*Y0WkL%@L z?RsDnP_^XCFjfI?+!$s&DUC^XtS|$6`WaN+D~yu7()zNT{!YY*bA*% zU45nY@iV7z1l}!ger690t;I%=yV%nnQD*W~YUDt?B+xKbZwCo_Ew!IfdB_LZ^{vVF zVK?`pfp-V$VCtvHczKQ(Br2;zi&xXTcBtrafkL|8EP%bM%lpp^%q9<40WlcUi>UhN zAw?P*rK&OyT$ga-K=VsVoFIk1~XbPycx%aK2CtZiByr zEwSGc<>=N3b+w$igH7z0%)Eo0#)(9{s4`(KyvPw<4-Hnn%L&5ugq3tJ%TPwc6gf2@ z1!fb!9%C|)AxfNE6!|?r2n;yUM6gvsHrJL|VL>%;XpHR3)LkKdV$ zgK$3Mk99H{-GhG8Pu>!q7l;jv?4BxoQ59$or5Rld;I6aS;WkWqiIV}wtbnyo=3P&h z`;@HC6H7gqT^@R&&X1W3P~oJ4=Awb!k1-v`TZ9)?Ga`FPf%9ETtH9Rol;pQzyAjt& z*r`6vPiqBr;j=e{HFoqF@+-tMVyB(;V1R}s@&(8Ufa0F~7P~)X1K@^OrrN+`4b>`x zRlCP9&Y^vrCiKS$4KZQzVj@bLC+Resi!nKU|0s+(`*za3)hA4~)6>NszI0tf1qDu6~ML(}p6|RfQ3!cVwPl{tfL=ifjvaQOD7RP?2iFAyE+y4(cQ1cC+zcR@{F8DFR%; z{*yYs6=Ehr>UgCp-M7nHm%#~aA--AQ zqnJ|W%JN@$x0M=MmkCm*W#!=fx>HMYI(I$PaRecmWQe%s_t5^?x9ifH)6h#0Zq-+~ z_O}#$i#qw3?oq_-uC;mqU)Id z7qDc_R@i0qcW-*xg?0INErJ43V8cH)jLXCXx32;`XHdP72RRqkeV?{oNcwFD2kVMr zfNg<`7-Cwf?$rBYz`J{kimB)g$>kX&xk`fqF)r4lch!0kmY0+m*xMN*0UwUG?%D-pBJpp8`Y*sj)X@fEK3x|4vZ=Eon=DDf3X-b4TUu+tXQ*(N0^ z;@fSEF)51n3}aD|o#E=KpDnencgxQ|@nfXw*S0WTaP(@2PRX)gJ84HKh9NE22IQr+ z31!zwt$Ux}3^Dl$Y>qvD{|5KQHD1ve2Cea+yOc4p?tRnv#^_k!5=m-uJ{noTHk#_+<0$AA zAy;aB$!nA;%&I55{cBNba&I(p|9Xw5%gOutVH{g4?jj>8?4ohf^imo6LmaLX01e_(H# zno;SAL?}KZabZudxOfo$&W_kqy0n@%Lk{;mYbx{Z_i5j}%g@=Y-R7w&%BXh+9-g+f zzSHG31x;Vv?hfQXnDLv74}H7^w} zRP5`J<%A3$s@)z?HnezmHPkq*b6e4>)f(F3Y;uyaL7q&1LnzhQ3B5h+jqbBvdbWMP ztHQ0i3RAP~=mCfdN^PpG8OM0g4U~E_U6l34zx4#Za)@IRPM+m;@8zpknvPr5NN=lo z`Zj7mT~GJ%=GxGoR8j^GGb)aqyuFihOE%#Vjr!}iWcyg>e)|3?EBF|i7Zjr#A8Oge z^<-wMGEUTWR>sMLZ(jKR%s0XNy4he8tfIMy`J6HR8O^!j5b;;%#&AtJqJGIt`!o;~ zy&3k9Z!)l6uH5A`^qZ$q*(|B!+Lp#;H}0wWiO$;<(F~7S`X-zl&&>99;?Wx47!|F= z=;JNAsJJ&PI0Bwoc)i{nbbIIOca=S*OCh|&p4rI%^7`(~JyvL1 zirv?xI_+J{^F!(fj(k-)Z9oLDPEHnBplRQZ7dP)jd5*ix*@^jGajASWHRkOU4plCD zcMoel!F7GH_GBVZj=NMqYX9|knCUgo=jADG;6c53@9uA*V@ck%U#K6-#xgpxqGQ}& znI7~I;14V(oUV8ZkcoIc(o&Vw%cZDYiZRJnAjyB#KYfb)t1(?9TDn%jvLYg)xhS;% zb>Y%PnM=lo)igWcX(p;dI3>+U=Xn2= z5|Z(KJPZwf?<4#=Y0@|2NKt+)?Bk-1c5>eAy>NNz^Zm=~A?r$c={*r_i+lGvmq69= zZi*f*>&^DfFI6jd`YSa=DlF*gPqR1ir-=Nlmk}_LROyJM@`#)DY_{zWx?FK#tJImy zqx?uU$ArJ~^^S%DmdVkFT*F_#7v1ExqsDG>Jk$ELY3HL*(`vT%+h_huNl4gSN7-@R z6Mx5Sc>1OtP;${1mUQ5cZ830G;&+xg^2*^;0Gf9?s~F&Rp#re1aln+$Joxz2B7y)aVflyAQ$bsyM6px zAm~TZ-pSxw`AZ^HTLJ1gKZc6n)LiRtl^jHGEo=@DhKgGe3&_XkoptaNL$B2dnlA#w8IOE#tAR@?$45$;?5bhoCN|LS zC)0)Ue&gn&%x6h~xD}SILD)Qg3HgkxOVo}QrV(Tm;S{bxTKUG$aO7Ha*$S)`CJ$vQ z(44sn7dHnn8|pKg@}7=S)ZG=pylt@s;Uv+9b!4vxxy#qz2N>A4Ia7#g^+U0fq&ffX zzhWhIUC}K&8~U$TwEQfhIwCumm1JQ#0Y8g0sug6iV{0HX;xfet+;WN49~ds4ziqFN zK8_3vU?mMUB*C9tKN76zbU5QIN}c)0PPwEH6X5F~u0N4Fg&EJe+IzAF4Nm>K#D`AZ z2v)Vb4piLT-Hfk5vUt0*;`|ese}QM53E$Ui*|DHx2Ik2bIk`rN;}zJb zrHTLP=_9kE$6GJ(_@c;ei2rj5+PR|_(K2JL<4l{bD~#@cdOM~hXMiQS`je$KQX6sQ zI8H0SR0gKRvVZG|0h=yo&a;bj7wXb)Q;LhBm`nYeIkr2;&88%wg=gxGS9BM!AMG>I zM>eI|Pf7qZY{x(*`dIF$!tNK_=M&k|D^%Csnf%);z@2MtjSUY`!n4j7b^ZqaP0l2m z4R+yWR#?<6rtAzx{piPT5~R@a5lAMnU6fH7&KG1qI#WO{B^3ghrfR#`6p_RJ>wI6}%>)D+(bEj0_90#>`OuldQtlp-)(3q95?Vqyqm9G2f z7kE9~hY#HR^JLoYhThvi`sQscWwt)7X1VnZ1a!At`O<8omwAi z?sR5dM2;T#+D^C}pQ)=!RXYA%>8cX+F8@HJ)(WD#yJ>xC@$0ktDfF-j@?CAL-bBM- zsUV~G_n74qLp_Z5EJ`rcE-dt|3e_~rmb=b3&xTs(a0wt?HuKpF`*-ounW_HoUt58- zv7>F9acVU%o0$k?n;>Jf_r$CQW{1Zu)y1Ne;9;gJ;NC_%o7xq>tjNdKm!Mof0>*=ALUY`Cn9$sZ7`U32sK5Li#gpm&O2 zC}q1|kP>3YE&NsAw|{TFT9Bgdvjm4n`AzKZg{ctX$?|Wiq9W(N5%ezwXl|@N@|;We zIA1h&Pe4!UX@O8s5AW7Bb87SPmlCtVXE?lS0QSqvrONiFRQ190^Z3&;FKDrB)nxlI zO;RZ7#&E09s7rF3QtQhk>SX%Xvz z{I;eK_m1uz`Ed)aW1qabI`hj+ppBmNh;>DTe!=4B+R@^-(7z(*X`l6@cVLBD$!Cxr)7K^hrLpRZyYCj2;#Nr z(gd%{c)@JkwTE4M0SYxd4n+p8`G*#JP9BHWIA=Q_f1jagnWpG_UjH?F47pf@uzQ{( z`}cEH0ha!hIzOIvH}LB>mhZay?`&RS1=vV5k*qlUK&+MIdRHgrIa`0G5Hy9nufR-sh`?1v2cHzyCy`id^kCB z&HJEy6Uo`LLP_0}mU8@!pHwa}s&@p@ZXJ3*NSfIWwh=8qJ&VtVhUc;>^BZbfgah`K zl-cfE$4sjT3vDxNkw#RLc<+dXQ(r1C^m9{qlchzM0+lzYuQF&@O62#Sv%`;I_&#-` zkgMUm8(nqvFS**~_#P*B9X7pZS~bOykT+0s=|6;=0P23fx1ubbBWHM|?zq+toQeD|kAuK?FqS@%z*U5o4v7mmZ3uH_R%uT{{5 z#y775?1S;eHjf7My#756KFYBT0%m?Z=>uCFfi_xze^7!10u|b#mLa<4S}aLx6mxyM zTRjipW1$B|Le1J@C{b$7QOIK^0J&OES>vmOG3<&hgyk=Y|uO8l+0C7=^`A_ zXAsPZpVPal37a-d>S+(aucv-Ly-ZuyAAIR)Mq+=3dM6YG!^Kt0>r=c!+Rde}Y742C8 zlP>8O7qe?F*rjq+vp8&w@vPt^?2w^)b&KY2Wa8?NBp~-ZWJeAXx<`MPFqR0LG0!F| zwWpopd)k*IR*;?IlYS(*4J`afy1}BquI95{{pS7Z{b7_8KTE_UJL2R*LVYcuE0yS2B zl8~+$F3P5(O?YN542Ip{AL&s!)|rs+U2)n4k{pF;Uj(KL)v>vF%Z21<%#FpM-`$Zo zy>CX^`(lB1&50&U&zJL)d&MzfhovKXL}hn|wG?Ot7Blj`VR zTgk7>KQhqXXFsE&xQDh?6X#i7N!VbfT;m3KF4KR%pWHbm4fv0HQPF>T^8LDY(~Mhh zH?Y5e_)eQ-)Au`g73R&tp51F`C%Tk*HM!HIDf2S7h%M^HYi-hQQza%D!ko}qrN5uF zN!Pm_e`;`r;Y095XA3isM+aYMvqxOP-C_uH(<3)JfNQYs>BtCi(NLvu-*9bOLqwln zPbspig%P0*E;6f|zj+;CO&fE)Xs-J+vf#>c_aE&=$ukJe?WN*#6tE6cFz) z51NSi-PDW3o|l5gw|>k>7j5MK4^=`8VV=V>M7D~|KAcwPzxOyBW(qApj2278+3~En-cAj?vJ&mgd=Vi=1tGHKOz%U#_`MAf+rLw{ z`|9Q|xMs78HcfF zE6zMjAGnx(0pq(}0q02#^W+Oqzswj;DG`q&T3NG!^DjVE8;1sdR}}Qc*~AeoshQ4Z z_lsbik%nTl%2Ww%o-z04wFk8~DV~>P93^3TZbyvihs5%%n0F68S~mC@LvB*$7qx;+ zKXMnQPfB0Z-E49#OkcKZIi8Zfh$R1h+j?`|+kr7Hevz@WXP7McQ!g$ABYqKyhYW8y z54`0pqjfH%dHL_BE3?>B9se9=CtjP=cmw}$>$&~BcvJ1m9SQmyAS|?@Jv-nRRjbMS zp1nNI4nYza=4;fha`6)VPr%c#(4dQj>Zz%QKc(@3;pCdW!}LrK{+8)IToagofWhgd zDqPnwCcmgvSalnt1b-vNINhO z9T?vp9Uyr4YJ4!@HhFDsETb=lqZ;#r21i#{T(%(J4J_|}UgegDW^dV-*mjkPzRNeQ zvl2Dn2E=i+#!l$|uP)2&%ACT)!i8-va(^CtOqZpaW4O;|UU`P?(v+eR_Kul|DcshZ zeOu^&ZEm9Y5$r1pVUP7Rq<^|A+Hi^*U#uy`+X4)!mW%o)vr`+SoM$th@M7?n0Kswh zCXV0n<&1zjmDk*5GoSc99+-WyNc-aMVOf|Z$v`0*AcsK5gSS}*G6nNJj6Y2ap5+$h z)v+CJPcTvm5>LRjpZcSsiduk(N5rtz+d!$8Hbv9Oqyj0PWs4B7b_=CSV2s;h*P z8r=ulzIhC7o!B|HRm_9oJ(-<#bdiLD{c+OWn+}e1rL_fDwD=yGnB}=w=^E>p&0R`0 zOxfVe&`Z@Vx95-5+exz$-^PfNXE*yQ_b`NKdHOY;88Zz&R()#C^bF7Ma~2?Vhi31* zx=O+D)?lM9XIxvY%m~kl&C$D_<6!UErF!+zos$4Ry8@f?zQf=c-vDs z{)__#hdAk}$;iTaws{1v)dpAR-V`Z)kxQUy*vPDHCQlxN{fD?0D9Nkev&@4Sg4rohi*DEg6@*%U{f!Rh*(lY3+P7Q1-B`Y zyt7XllxVRx=1sH=Q46_udIb<3+u3JEs?s|~1-HIXDZfWc3SsGU&gO4pEu0(OD*|ms zsH65H=3&1!hu;~8(^i0f8r`h|feasa$~`E}9jS!F`u9x~z-FlL%e4&Q=0YiJoX$$& zySLo2)8!c7cd?^)-w$7PpBw-rVQ^m#nvZ@Iju0!SU;hL7)=bE0E^|7sYVCejB8Be% zoftvmv(x<4yDR5b{3iquYUCKL5hCPjdT@R!zX)Dc1J*^+6Z; zeg%r3ozt>w9T~83HnQgRc%VF*Cq3NIso48t1iOEfWz$-hzdTw(C{mPgmU%ne;44Q^ z|DW}9{MZzkW`&M>YR0}{J*gxleP<&osT)@$3O(iY{+HsHBqwTWtF=8!o zJ5!zr^McH@{W!|g*(&BBS-(Fil_#2PZWO+a8i6x*CHdER8>~kVKO%GZgGN}Xb(kc) zJdsaOb+Y3gL5;+KbI=-A%e3T)18zIWwU3?M@_o+y*zw7^6M1y5iLT$i-YjU)$nd2=C>ihUABRkO7wsevhgrt{AH)2FE0IMdL zN}N|3L7{i2`N#{iJCbc;l>zTpU)fQVmlWiJVnz^l^duQ@^W>!jXV{S+C~St0yqfT{ zx*i=@Nj9(}rK{KIy^Fz2>{TW^r*!Q_wcgMqmCPKwE^Pi7z5QM9r(nak`L z>`T;8J-RQU&&faL|Jb|tsHVDo1Elfql8zLa2 zctb&mihx{qJB%`|BK1lIkt$W>8U-P8O)XVaiV+ZkKnPxtON0mn0tw0fy}|A`{r9)l zS!W&B`p%ivb=Gb|_WRlI^?9Fnk05kG#~~~AS4|UbH$O@)SeZ%w{*p-DC9B~JFXSxw zGxb+mkNU0W{hSbe_J`A-$1BAZI@Vnd>X{+=HyJepdT=Whruydkqc;i^1TdOrG3I(duO{&~+S3-v%w zKzUQkdUHofNVM~rpEd{kHSqSf=8zkE+$t%!k*XfJ{%8zyrA=Vl)YA4BgWmWGXAR81_{)9u2rJxAh8mIL@$rmol~JhO_0w%;0Eeukf|XF>J5FKf@xzF)ldR zcJK##c5aTva7m$ga=;5KQI>h_I74lA!PdeX8aH$E>T&uvS6kf*igw=b>-M?rV7A@- zkAkAvhr2qRu(W72BCIh_B${f~!0|F3$00bZa z0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_0D)Itz}sy6Xk#wG*TVvfrLTNx zPyh%(00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*yz+Anz9m<#Y) zFXKNN91ws21Rwwb2tWV=5P$##AOHafKmY;|fB*y_009U<00Izz00bZa0SLVE0uu&W zJ{!pexXtFhv*DF54hjGP2tWV=5P$##AOHafKmY;|fB*y_009U<00Izz00bZa0SG_< z0uX>eee|f;jmZG-voeP@;D7)GAOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX=z z1Rwwb2teSK7w|S4KiZfJ@b$33V(BYi8WaEm5P$##AOHafKmY;|fB*y_009U<00Izz z00bZa0SG_<0uX=z1Rwx`gZ2yFHs%7n*30;h1_uNn009U<00Izz00bZa0SG_<0uX=z z1Rwwb2tWV=5P$##AOHafKmY=-yugHkmd{3V0dBK-?`(MGi-Q6{00Izz00bZa0SG_< z0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_P#-<&bz?HX`>f1i4LBeG0SG_<0uX=z z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_009U<00IzrRI2J+v-mU5j=5LH_&chsWoJzj-Y1+ow-g z_bxsWa^j42U)Sn|Z;iZO0RMmh1Rwwb2tWV=5P$##AOHafKmY;|_#YRLtda8E7ECqs zR@of7wK{O2Zsl6(+=rX_p7rU6KDDg#TK1n^QOBjO=MK9M?_Rih<(3~R!hgD1db0dj zkuo;1_GU&^bXjif#fOrGE7Sb@hxcWc7Dnz{`z-|9b7U&+J-Hrkz}IH!^GO-R~Nk;=}A~*U$LGEN|HQGgdtQ>Ljf3Y82rj?{AlA%eLXC&So+H6X`lcQfB*y_009U<00Izz00bZa z0SG_<0uX=z1Rwwb2tWV=5P$##AOHaf9JF8X_DF(4FXKNN91ws21Rwwb2tWV=5P$## zAOHafKmY;|fB*y_009U<00Izz00bZa0SLVE0uu&WJ{!pexXtFhv*DF54hjGP2tWV= z5P$##AOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX>eee|f;jmZG-voeP@;D7)G zAOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX=z1Rwwb2teSK7w|S4KiZfJ@b$33 zV(BYi8WaEm5P$##AOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX=z1Rwx`gZ2yF zHs%7n*30;h1_uNn009U<00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY=- zyugHkmd{3V0dBK-?`(MGi-Q6{00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHaf zKmY;|fB*y_P#-<&bz?HX`>f1i4LBeG0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;| zfB*y_009U<00Izrm;v+<*ixd2}e3oMqt z@})rmAOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5IAVR;B8|r zz-zsX|7dVP00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_@X8BJ z7-;!yBp2W|oA=I!SH3tX00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_ z009U<00Q+_mrr}`wYSG?`Ip}3&>L(1+%oH7%l5ZM(gW}h2tWV=5P$##AOHafKmY;| z_&+1?v**IUHNI}VozMKVEct(!-72&HGk1p@fdB*`009U<00Izz00bZa0SG_<0ucB= zFYsf1^r(^e()+B;VGYtHAOHafKmY;|fB*y_009U<00Izz00bZa0SG_<0uX=z1Rwwb z2tWV=5P0PUyv@dsHfAh*JuI+T`pTCE1%LnqAOHafKmY;|fB*y_009U<00Izz00bZa z0SG_<0uX=z1Rwwb2teSV{erhg-d@nl_>Tq$1Rwwb2tWV=5P$##AOHafKmY;|fB*y_ z009U<00Izz00bZa0SG_<0;8D0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;| zfB*y_009U<00Iy=XusfXV=lmJy^Q~8a6kY85P$##AOHafKmY;|fB*y_009U<00Izz z00bZa0SG_<0uX=z1R(Ir3rrYj`D`Q?;5M80&W2aMI4A%FAOHafKmY;|fB*y_009U< z00Izz00bZa0SG_<0uX=z1Rwwb2tWV=_0gkVHzotT&&nLufCB;$fB*y_009U<00Izz z00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOL|^UclRI{Agn?z}Le9i>0r8X;1(NKmY;| zfB*y_009U<00Izz00bZa0SG_<0uX=z1Rwwb2tWV=5P$##4%#nx+n5XRS})^28XORS z00bZa0SG_<0uX=z1Rwwb2tWV=5P$##AOHafKmY;|fB*y_009WR@&XeE3}XkCHySmi zx|p|@{dw(-zkTqx5B~l3inV_}_5J4;|Ni$cwaeH2{ezRs(*FEKhtmi8wa5Sc_5||p zm;G52o>ijx?Ogc1$hrhKA5maHU&L8A*}ef)gRa#^wt221kFV0SjT*@_z&{`W0SG_< z0uX=z1Rwwb2tWV=|JwpB?Cbic?Al8YMis@+uz4!3z8F<7#5YuIHw3Er@o57*wtx1x zBF_!!8!Fe$WIlFD-`~UVX0n>Ok{UKs%^2dEHnP^+oov2mC*NeQ?w*~zThp1jS~QOz zd~1AJcht3-H~I}ecQ@!2hSYacyQ@`o=Y28@G?$xC3 zZF=w-wubzlVom-J`)=-^#WFg@$b0%;NOGNL*B(w=`J*p*VI$$GTQ0y^Mcz zxTn5DmF)41(I>lna_Q+Ad*S8b7t>0%{aq0JY4yw$Hrau7KddP5-t(oO<(+uTBl|1b zB@f&AzFl1u8vbBHJ#?RLZ zukWl}HS^x{%hQH(qK^ek)>ll@lQ@h-b#`@l$3IJ-fAy1x1DoT(emG59T+sb>YvlId z@0?kf!j$>sx7&vOW@%f&eRAe^1{*CaVpIRgej<`~yPkjgId^B0i&c`2yVC+X`tGUp zwEP<#*yDoIqU~~R+M?f!B`L0wY`pn~-a}aXC07KYznT~HS>o%7GnhJogDQE;NWp|# z)z%@s_eR-S8h(ClGRK!yC29NO@Yq?xx@C^sCtW|)ib@`xUK(rXe=$Yle08L#;<;M! zq=zka9eTmkpQf?nZ;ntan%ntp9lU_7`OnC`x|MN?`4KxTxod)|cFFS;-)N_j zz9IiP3D)e{zo)QA9oR#Xh)=IqXvYR|k9c*RKDW$MENbZ8&3#vrTP3;sdg@;~)`#Bd zniIzUZRB}Mj#n{zo=6U5@^^pDJ$S{X8>dwHtwk3~GZ z@lWv)L(yb?ykdRirf9>7LwC97akoxC*IRw1C&jv&e$GQ9wVJ8QoEPXyZzrZoQtQ~* zWb>lSR&|p5V7R+RddpRnJ}{KFh~DCNLG1HAWiK8v`pEbr&HUfVD%$zk!SW&}m#C=g zN1j#ir{8Iph^r*ao`|};qc+js5!4nk_X-aDntsV9NNxCJqy;cGN<)X&(78+TqA4G6 zl~6NVYi8M7XP+yae4e|OLGF9d!?DghGJ(Spd8<|GyH_={!hDPMP+S(YxI9pOw2jGfyORTy<>PiYE~l6>TdkQy zf7Nlmm%t><^Hy8i82;th7`LhKKc*5@7cx8k!R&u{QpXuFqm%}>;PS1p@10BI9&TJt zcd~Hx6n<;X%>7)usjAbj?~dznnJ=y5%2G}`rwvK-y$6CFJL!`-Y`MRkv-Ub&xq8#73-dyE9;Q zE^-;Qw4JXFu@(>9+0OSoPkSL}Qj0trSM9ncUqYK1`!}VYO}Q84d)Huf$yBEGS-Lj$ zht4hwduGvRuK2Z(nPu@peSdyxJJ;%KNGYGCBtP6Q7dX?dxrTiC1~0i|zh*zz?wLbX zOtt#U^%eZ!E3{?`ebpj$-4``sCF+|SXn_>Jwew#nJ(E3kH`;g69*bR>!rsbpO7`zx z@5KF4MN>O}U;IbXO3{@puEGhu<@_KvG`rgn$khVYjWj52Url+^9y-eL8`~w9mdhqj zXnashDtfDwO*hOWxBVfELM^)Fl(uejUJ*L`zJ>N+{%D!p%hdAqn^ksh{;pnhe@tHF5gP;aL9qZTQp^0%ZWE7 z#qV>qz?#&8<)u-3yS(K4X_s-3l%?UiTbB9rRQl76lk|4`7YGN2b6us!X|wguQt_2V zdZ+%vqi1Jwf3tD2-opM$%f-FoBvZYxo}+b+%$8P6;LS{-AM{W#1ue38U1ii#t``#Q zmh+Pr_utiOij-Uha3U2TZhTx--igguE~Z> zm2XZ8BdR&{o9EtbUzn;H8ST2O-+gAYF#OSGS}T}>e!WkmZKYL5wJUc&A#=1+UjJ{y z3F2BA*KS=&XZ3F0#3K>g^by=|GZ%x{-vqfoHa#!rnmps9Gz>j8$Om}c`H0-b-M0oUuL?OQ~d_^hV{&bHLZWAb+NKtQt+iz);MLT{J-fZ z74mjPYxSs!)(ia_l%~4utJGPGlCE6$j$cjtftVn+srFgym*S1JTv=qKEKif-H$*+v z+j7$w#yN#i4jqpF%Q>5sq)PdC{NoMdtW8o^$Sf&stxOOGm}VWKxmua78AFeJ+DX$E5 zbohPFsL4CWy7xXQny|xD#MM6Ge68)!^}b#&m1QP(P3I&NPgP5rE5hn#<+RC}f2`gE>p zv0o_-XEZMymkWwFQOW>0KSSzP5E{Cj)brDs!>;{2Qri+|WSZ++|8vSzGsY|X;_RJ7 zUU~ig8Qddh*N}_a4>J-UEMCq1L9^T??9QsLC&!;exy?*TW?oL~*wspX{9%dVdfm`^ zdOMzVZM^7>O!ax`0!zJ4@^VZs%aZbM9o?KeT-vu^tzutJauxoOhMvchljkSommD?d z1uv%~@6;*Sj5v08Wz&shI@=4+R7kX4GU@yX_Fr84;q5A6uh!HgiyvxzIZ8&HN!Y_9Gc5kyFjk+iLomM} z=Yr{$#vU<5$DGooZ)^F*dON*+K6#Y2zq$!(%^O`ES%dYbJv(mV~L(A1{$A~ z2XhdPzsXV|QtCnV*mbc}ZoQhlTNePY2Xy7)TjpwuweFjnh)WeIzrrF`A0F#==D z$=#-AnzX}PwLevV7&n!+o9XTfwp4v#PM(+c!ZohiCvTJU{g(V=Zf|rIEm%$vJ3PC1 zcR{jyC)c}A&(!(Ef7BYbv3?_0^Z2{u{FveO0bNmKfR$7?M;l!9sidRokP53rOP6!T z?8$zet7Tz8!K3HNV!Cp0_E4}PX_ln=3XIVXfLU3;l+`$-N!S{|6j)C zW~ztA`C?gqmP`IN3)(^R%fbxG??qzEJXc$K>&5-UvRzMRobU3!B{Okch- zbfEK1+Fm8UhOs58fksi+oPBn*Jpbx9lwbZ+qxgE)U+CYwP$kg^Mh5@T_s_}PHGV}Q z^WJdba`{7RI-h!IEYA7~MwCRI_kz}DJ1C|wx=#>jLJzNJ^S{l zde?sDCaJ;E3VGUHuQvM0g=E#AI__cUi(?aLH=I$Y5_pC!ySU0{W|JG&4NLP%Nl`AX zNGep*0jMuiqj0p(t?2bo(mzP1DtXf2ro5hM$`Cs9e=AePFo7cZC281hSL%>J99=6y zGKLI)qR;g_sbo<*S|3>iKj4PB!{p0~&za()T%8w_I&XMfV*Z=?C9Z|WlWAPJ__@8I`Fmee3I=3%j=baWR7#PG~0J1@=`?RZ1;clwaJ+B6YU)0E%eIyyTsBL)9N;! zpmR3WWPQ13j#K|SANy-@zb|ju(tbn5DyQ@iRYX3mj_l8cO!wl8x6+q-DCt6E0$G9& ziOg4(S5|ShJa-9WBMhs2XzfJ@S3X$*)h22DOP-b|(3)(VBkfKvRa~&1-epDa(si=l z^|`t#YTOF3jNW7hS-kDY(3h(U5>5A0Pu6*j`5P}zJKb$fzs@HfeQkB-^;^E@8mC~Z zoG)uy6JB;|9IcP3#K%WIs=-WKkk55cELk{y6dCkcny#pGXX$X@@gs(0vMeu-F?+rw z>k|E4fnN!mqdmJQqj^)PX>rT`QpiLd$XH!t?Py$ez8uG_vvfYn#_Xdb*U)YkbuC{;v{m3d-n5kh!1=j+k zR+^e)!(UEh^rB8StBTQHu^*rHz-O~3?Pes+T7AZsLf)bm#_{&|cmx|#r(*-h^78j~ z4$x{9PFAd5&t41&|FK{tUCpsy$@vP&p1~YzE$u1B)oB#M$eY2|HJ1?%6jGwKwJ)qG$q`uAd%h`~2U% z8Nqe5m9!o)oT^BU*dh{4On#q!RQ##dEVFdz*SrZjQ@foRYp_vzhOEmN6<1Ba?=e~L zcrJ9!lcDLt1z*yix7EfMZt;Io*fpKXWyKogV;Ar5z24kKd&>C^tj*SYOD&Qpbnf--a?Nr<; zWERb|-zqQ2qBnoffsNIVweb(?&!;Oz60nq%{cx!AnPRp$&2&E`WEt^HTG?qIMOXF; z68ZB|g+05zFvwI50!Sqct$a4YH7e*5_tT=?YGIG&?lXN{uO#v)ElYr!C*Jmi+!JVOg>VhOWij z2PL8^iF>ZlL%ehN8hu(MVyW(RlqSo&=Wt(^NUE5=nQekKZHH!Xg^`@Dl2o*RH*wbF zeTqMs3X@;O-0XYHdiR3!XSlz}|G8c9w5!zrqmp%|btiA9+IlEIs_`zb(svqHY~>~? z<3Hrtc*7!`c6ZU4ZIbj@p?I%u6s;&X9N2y3n-8pwX-LjL!TH2&N@mXHuGh6rk1hNs z*KTBaOW5{-Fr7nJ0-eY#k2UB67Ir*NE6Jx*k(kMP+r{ego3-7cbVGm_P{Q^OUvt_K z$6rU+%d)vzYqz?N>P4am=381HMMAZ3(7$$k3jge$H165(e+e`A9X4BZQc+KPKpvSy z7bOL`iMp(4=RJnhvzK@%N`aFd>m}1t^ zaIx;lEPCw-&)U}YE`!7e$lj~u(21Ir9sMRdknNv4?huBZCw-j&kn zvJctD*q$mV`l&LQ!;usOv1+SG&oX73z>2$V53*c}{!q56KgbH{tx~;EMW?OO+;gn&(BoLY)jNNp_vj&If=UXV56_p#Kjtn*@s@nw@Z;XvjoZ4%ggN%f? zRJ-ljpn)WD{Mq2s^x77+;z8@v5AqhBq)|`LSVK}k&bhCPYvVBMMci^-Zi;Vthk~(r9trNKoqT{%we(; z&Z=;jsSM3Fl}KpM+H~iSHE+&Kq8E8zk&DXfV^de2W54EV25+aF@3dlS<{Xyx7yR3W zOljx9{gybh8gAqhlkK+-y`!)}-M^fx%j|rmhRKLn9QI(orK!jNiu94wx-Hj7Dd-fK z?IJN;ADhVQZ9Qa1kB$P%$*{X9-m++5xRH07UJW4~*ecst|JvMRT&?7hjbe?=!nW<6 z;uft3-ed@RE`HLvI?A1^`ifno!5??zO}15;dW(4SbV3a&;}p85?@%;;Qz5fLHCTU= z*B!or0xSIW2GzO4Q!{cZ&(PT+^Ka7Kyki^3a}RurKJvIA_N-PnNLIt9N%=G~Ej^z0 z?xG|nmb-BY*~!#yHVl5}H=fS1G-SQ64|m=j(UuxV$5S0yKQ@HFo9b+Sk?Ri~-4yKL zt@~fvwAs+6Xtqfui3qKJy0Ai!{C8R>dDloU(BJm7kgu+xjAmprrfcZ=yHD~1ex#pt z^iWA2NFLw$&W1U#j2ryot~N8f*7~nVelnF=$6e&_CKRW5E8P@@bo+}(*28mN+)HiP zyq->A;z;*y9^PAX_)tmDNv=*YzqSja8mydce|SK%H~7Bf*;vTz=kDrWOUIGfWJzc} zKEZug{~;e5?mUpfWULdcv>a^woV1Bj$vzV3U{$B@Gyke$3{z85YnuuSX`N=8$Pgb@ z;S(bp+>^lle)7%|wmKrQZH1-nu-orJABIfwAD!!BTqBIj=i;lvBx?v|Gi0HC0ry>|K3k=$D_j zhSSNV4_S%pU91BZtJZK8l0Dn6*VuO3Wh{>ivZhU>gY+({bhnB_N9i6$8;#6P+`B#SzJi)%B@CE`i= zBV6^VxOal4bOzTX1oO4BVohGbsIl{m8>=sO8$^@zj!Ob-ietClHEpIZBpuFSWuf-Y5(n3~x78E6IZ}mi;-5bA%~dOEh0A{^5mY?0VH?(*_n3i_@W`_=ZP`lZX+tLMYcKB$uiQM7eqWLk z#rh2w#wq22r)ZZTAWc9j=)E(TIzYP)0h#*b$YmQgU+a9AV#bouwP^P$iL{%BXB5WL z+-z~S-MxhFGl<9|_bq4U@7Mh)+0-gjM+_-(hW^~jJgq5|s>)LEm$aoxi_Y%9PVa}k zp*7?NSk`6l)K#{bs%CT*Bku~2s`^Pb&5zD53ei#9r%swRm;-MikBIBt5k>0vNLA)_Tdvq|Mc5AMT_7W1(w`QqI{d#+Ce(z%j6goc3f4~Zr znz**9F6rBDzRvCSH8TB-fp7fR_gS7&$0{PTH)Ppn8|Qm&bF}ecP1niDo1?yMJ>J-N zSN&6T#BedzP*LG(y*~TF#ETp@GdYEMV$M6=QBb~<>xTq&Bn(vlXuGV=k1S_Sbv}2W zZ_F@~oF#!E<@$<#yVlAaZVJh~mBPN5%EUL`kD}`mMRmI*wEf7;2E_r|mPo3MV*8f( znek1!n)kRb8#63_c}o!Ra8Ktp?#Cs0RT62@*8GH=>$E8;Zjq#qqK&Oq%+*A$c`54J z6@?K8v&OQy*T`5MKOCl!q%(g?VJ~#{40ko4P}2vdccOpZvRG^XG9@)-y#wtQCN!0j-Toi$I~&eXp-L5zh60i98_x*f8auDMrR zl$q4$@u8_Z6jw22TKA&-O+)Hgv|$_LmHLf-!AX4;+v#{INgzp^Q8xmDtp3o9G-Ma3 zEF^QW_UqQ7JKPenlid35wXJVXSJ1GJAGxE{#E|5D-S3(H;3X12;ctnX;VNzR)1?1{-FcFJ zTSR6!jF~*6$GVbs2BSO+_Eo*ELPe?4|eL;GNe0eUQ*yNvg!DUbg{A^8{5dDf!Gr9Zou#hj(V?6~>bxbRZfu66GkvOoMiIARQ39*#rc^%yyScpbJypD*`|JPQH z52Ub{mMhY03L6%3J%Qv{mE^|4zVYTk)w{TBBxczc7f@n0OObt&YqtCS&Zkui50+Uy-{|1TXFqMqy2a=Zy?pI0qFY}I>!YU>>BegS&pHgJXikW-F5L6g+S#jx!VbO8^wJuwd)3}P{ z`2;BX{e>nOwgWEWCT&~rZ@8>`HFVp?Bkx2gLpdDr3`N#9<-hh-u@~=l8-m2To zI4-AM6mzbMY2TU~=gDZkHT4~NB#vk88Mry|6A6z7Ur7uz#C`A2J%_y#!_IOmK*`T! z1(@m_Cw$S-O6$lnlAO5D-+YxeP{=hv1(_KRG!OlIMNz4OZZ3$8kuBezH%2MP(^yQ` z5ksFx^S;NoYAnb;U*O}7)5jRkM+mm5Md2>dCmixahd!rwuV^Ma1TV~{pDHxCTmGK^ zuqB1LSw2y1Ioz!x#8;=)k6l|;r!-cb9CxzG6bX>MW2)lyb@aeZR& zf!@lwTv&l|CHn(rhJeD|H$LOUnNw9vW!&5Z8}~pu(@^cOXI;)K&#Y$$)96--D!`u2 z6I4EP2@ZX{k~ZHw5_o&VD$&gT=@#xUDvT*Vzq~VZe)2{)ZWtGke#q16Y;9VZ$4q)R z&ME9qjn9(uE~w~CgLO|~S{=ia13_r zw#TJnBx>kYr(z8Y@@t2k2TUt9V;Ivvym?^|yQ!A$cWCm@u?<39V7ZHVDBb6iRFOOa zhqvx!9k^}kFys1%$`3}~Q--+H)FOt&G4~Jk7x%}P(x$8<;c9>9ynq^U5!dAz&;MwT zjc>D_oJX^FRX!ww6fS+se6glyh?~Z-?~x~?Q51Ytov5Le%s|Q#a9#h)N6F7k;j6JD zWm%TkA&~^1r{fN9CwVxkOY?24`srFFnxy>*)hip1jg`^*D0Y`Jp7*bM7Sz;4zQRSAi+XwXjVAz^j5&vbblX;{lWS2GefJCk?wW^S_Q z123{JxD#^CY$RQ_jHHN`Je(*#!(5;vHqV#j+a2O<&S6Q?Wgg)%BR|zZ|j0g{yAHFy_66p`075npHHlTSaof?;W;o zJaFez`Z?0429#Xf8k1;FdW@RDjH4rmFEk{GaD7aCe37#CD4j1bKaqH6CU2H%$&fo; z!jM!;ovX^ec~@oeAG(YsBnHOp4FBSA9A9M$N2&0KfExkdnkz~J$>z%G7+w5>QtzIP0GJYbz*w=8)l#ZBbEY3L9M&Xw<;h%p`YbfWf z?kbJwPn2`*l$}NbHy&(FNzbKkxL;w{rZ9&Sqyqb()I;>z>=gFwlfr@eHInS3PhIWW z+ZY$NhdkUiS@Dj+G-vh8^f6ZAv5b9#T$W`P@94WZk%sijV+|9!r&Q%SCEw-#By)ju zFF(~}dj=^y?78(8`*(6oV{dcbs!MV@_nB^?_Yj>6^}J(IPMgnkV+yU|zS1wUpoeCm zvc@M{ClK&+9{kNT60SdDsPH?YbP13;1*!YuUS{j^x2uH_O`*2xOi2uVTn%A{jQ)WV z**r!!W-=YwrKC}M=KdIR*lMhjPEC@1)x0H{(%8T;Q)y~gs+*KqwYb8hmpV^dnO z8+o<`3bosuJlb_I_ef9V64&(LMXYJdX+Qa#!d;Wpd$Zh@4)?}ARlAMF1^d!;S{n9j zi#6nR{-Zeb#nei=npsKaWb3s9jT=OC&(HcQ*_kX!6U02rr5}5XbUOnFo}F%WRhwo+ z?1g@vq-@cEaPgIV(>%?M{PepT^DEDq%eAHk{g7N0&EnnP()lOT(E6cLBibmRz31NJ zKLiRZ$R{P|7F@nJ@$eW^NW_)wi3AUCz2D9R(C|IL^K*Fh!bMNxrM)r#rV@ zl#A);qauS!8RIZu7eP9Mmz%AOg7nmphURgV0d!E|k?yzFQ{|^fqaPEH1$M}TV#bXX zbuE&uxVskz!$E(cz9#nX&RYGc2h~3uIkH12`4Rd;GQiRi`Y5dH_9x! z7n)AviSnY@kk!doyOL*_vfm_EO4*(+LHR5X^}py?%96u)b$pxbM{RQYcwNca!q)9^ zD~J4y3)T5ib7vW2PE1l26PY(7Ze!Z;ooTc@e#Y|19erd=Xegf{`2+j!eP8ZtlWhv> zO2{3Fb#FXz%<~xhflfCBLUQxlY6g34O}WepvP^ED`fgbBdI24;84@0l%Az8=)+?ve zC=ZX!OCrK6pJcsfPZNM`Ni`5_oX_?>q?ydDBguNp8Tx6!BHsBVwd7CM7r8d>>&@wk zAz94a{s*7z=}pmBKXu4+rrRtQBpFZIG4$)n6NVq?T%Nf~iqrUw^Tu7aU2o9J!Df)1 zBk{@u`*w=xc_G%8j53!iPp+DlPDd1Ds^#!K=b*4o{U=m1M5$B7&y4cD`3p@zW6j8( zo=_NC*y9sRpAI=5{*A|!{@>J<%n}lLmP)PJ3@tf_Bg`y(bY^3tz;xV<&1q)&VUHgS z<&6X$N2V!^g+4yd*EXyBOTSdw$up!=eDd9{Ck;zzT8Y!DdELoTA@&|uWe<&`sCmb! zD#E3H?-5{Y0Fk_PCy}{Pj)sAvg%LyS<(27gN zITu>J=_p9DESav#_ zvPi!dO~7>FKxlEYTw<2rPr>+itt;r|0sTFc^L?go6OOtF$Xtp<=HOXibrLCdi z1Jy3IXq~w3Uzc=@oq+3^T*--Td;N(&DJ?c%qEAsXM=R{!qww>0-&jSP9XYIKCvV#Q z=t8!t^dtJw40FBf;xz$lhl(c7qA|hRBZgI%r0x%tcS`Bxj+|gqU$plU6#ZDrRY6gJ zS~L{Yb#oN|z_TE3C?k26{G8{hhiwh@bm1YJr;Tq2e_&xb<2l{o(Aa20g!d!vQY^}4qxUaF5f8eV#wCf~c zOF_YN3!z4P^&sud`6Ro4M$+)TLjMX{&z6%Y>i3nZYYnOi-0&hGsT8ZjyX8Z%BbyX2 z508_ilo+dcSJOjq7kvR9VvIg1N4nb~SZlk*;zIEeF1&9MK7Vdl9ar9R5} zELBvJ#I%yoyJKB;ScjT+{CM7Ef!QrGJ5d91_9Fx#egzkTiUriwPM|iFIejE%B|sOyeN@TV#G+A-8gxRlkaRTDD{* zSeSPwJwZ0h^Aq|P$+Mnh6Hr_F5XB9^b_9Cs1vekc#X_xLOS>{J4D=zi)n@$e3l5256^59N6UpS5S z@OGpQszv!(^IPAbUI&BN;{H7cABjBa!AKF=;}c0A2J|J+1-&FUh&|aDpqs^%=FvK& z+CtKzn)}3}EAkH0P?^zhVAtAq-}hsy=|y%%3fpoz#$|KJvODAHEMAgW#jKZ!tnveD zOuPRfB*>a*E#GSh($Nlr9IR@)VIQ0xqUp5cq6v%{$!jtUtbUR_g(jXFXZ9DPL$cKk zrXV0Wj}Q?VaWReek9s#cX%P6Slh+T3SJXyzn&#`G+)_45l^QbH>-;&|!%LdUL7N-f zivpTejh^&%AdEGM%ba!zR_^;HrO31Zwm4#lEG#W*On*yCJCYJ|@6UFeO5E&9$3O*1 zTQob_Sh8*Km%CigBq3ROi*5Zh?j-)nb$7OID%QH|DmgOQ&$UrTKxU{xd74bkjk)2b ziAZ9sA+{o_ux*fMXPPcWkvb3_o4zjXmuAz0c#qupm`Y8jSP|~R1&SDkY-Hz(o5Gi; zoDZkr39`#qb6(Nh^@p+6j8u$L$Bmx*x_IHVPA(;r>z_3)7JnFOg(ULc_R)sK-p{Ag zdX`A4MWX+}u0^_BU#`&16>_mKDI>m^{dozk7Hlxt$(|$JAnM=x7VXaY2@O+}Yx#W=2E^B=SeI-ZZS37=nRIu$-wYxaE3UPa!-DRjoIG8RVn zT=uvjPe*&yA~M%{@SXlyQ$IbJ#|wyL^?@N>0?oJd5I-xg(6-AG-FNK4&h|}RPozo`|V{_^dXQarC#GwIAzWD#$;dW7*6tLk7n*|X^EW9GsQ#6 z5$KP$j&Ir@5l$r~{W?K>etJ#EzvNsO&wfn0_+5?B8`YcLY5qB%%#2f>O?@z!y_B1} zu}jI~Xy8zEs&fF%KjsD?Idv0UuOEN;=i?@{PhlO#+xhA}x%&Nw?EZk9R z6E}4xH+m$Kj2@lJ?ovmR={RyK2?e?59?G5WLeJbtNy2RL+U|PyzW3(S`S^O0&|Z2r z==tSZ(+=yr6!yDyvMH`n>*+~;mK?^fI5&0*KT4TaeuqAmwj+ibB@-5LF(IDB(tkhy z*+E{mG}FPx{6LebpI{D=LZ7O1uCLdYI8end1T>#ICgK{mJ2kVZmDN=}Sm;6h<)BCktx0&#{Lc*nYvvotw7@(T;*&K|o1{^ZbmpY*>CxdQ?Owyv8ic{vGo? zn6Iq4T4p@hks2C5xAE*g`rar;PB!VjGq0SO$M(|QJV`zAX>qKy>)%Nimx@odny|s{ z_pAzRR*~<* zURkn~&d4hWY`Us3{lQV^{F7A?Lv4SbtXGhCl%(6{yJ$vP(@3XklY%|)te`D(VQ2_f zJk~f5oY0umcuk)ZNMZL8*w_1{NpjtM`a!aZ-S>lqG$T-D%-j9T%t)`1X}^x#(T6d4 zUcCqIgwi@X%J^VP1LQ5W0kj1lBr$MBtQ}K**K}q+hJ4v0uk^=WQHGR0KpqJiyC@$0 z^34QQ$!A=}HBN`hgpxf^USCC@;0&^RB;EbsQrVsD^qMbui<;Uaar?v)+PCuOYh95`{05rQ&G?ez9rubR zR-OwirIQl&Hs`S=UsReiA>VQnRhFcXE{Y7>m2vnMUEBs4_ZWD!-iC1#Y0VX7 zDOk&z;hgay#j>Szip6Xt2Q~sLj#f5(Gk&Br15TC}l?&+tUL+>5(@1GWYLwd1 zk-Kr0oO6<$4qD`*+uqPhpKUmKwvkIz^Y+J>g7Ri$(Xy%0YQ8L+R&!fY+3GW^PkEX% zdk)glRFM^G_|QG826-r*i?esgX3Ynp{Q>*R?r^n*^&!9VhAill0-r!1?taMr^6-;W z(S>yp+5*$Dy@^T{E7cVEKD}U7M5hn}^12q*eQ#~Au(6|E!M;p)A-R=0M*K)_?q85$ zN=#WrQe$G08X|r3Y>V!gJ}W0%izmFgyt;n)W|3CIynIt4OWtX+Q>N@Z-!>&Bg<&VY z+yZbWci8)860S{=w zNQNa-EtP9C2cLavsaO22$XJl%g)WUG?aq4&HA#y4{S`YB^2UNaS#iP{Y1;po3ic)$ zA*MGsil0?w1<^O`2z}J5`}rM}ebNJ|^avk0fhQT(=IA?viuF5aMIx`UtoqMHU93v> zPpNGlJ=wIWOx)aV3G4oN#0k$ZR)E0 z)ru=l(RS`9!$bdbgeIJ&ldTgrH{`d6I{Kne0eRur(7&w?)h$e?5pqR&yW--iJns~b zs3mkd>PjMOh27EK6SBi-Kv4Fn*0w-*&n~rTGwq;^Adr=SYr**ZC!? z%omr?3=w0d<-W?mGTWKt1TvTKPhytVnzmCRA%kIMd$?~4D%ur+@txC7$0zL; zYN?BxT6I8mn9e=OA}nxOMfj!p^)6g(F&=uG7%(@-y~0*SLs&sa3|*q|prdt4GUz!^-6x25WC~qLS3P#eaE6w*g&ewUsnE?>srp0aCfP5{ znjFX^U$mh&bDn&gxn`ahAa~TzWlPx+L*fjV13v~md?$qt;*T8IwK}=>=L;pPX;`eE z#6|zTmACu;k2JGTM7EhG21M)`emIxj^*iJ>IQx4}|vNGPuI<7c6yn`mn@tesz zAO{DJ)ujfRj$x8`Wm0yDrs|9M?lc;NalX>b6l8R49j7hryG+{`zp7m^_-)RU^0-3% zY`Vc^%(B!DPb_y0%RWX&f8lNtrMgr5?wHb{#sl2F8;^tS9t{5Fkmwv;kQ(1Ul^D~EPQ=flnp(}FDB=e{14gZ(qpwrVm&fOt z%sKP=2R=XS?S1^VkDY1Gb-%82o$I>Jbod)+yy?!? z)+4s0Q+0P(zoht6E|_B))J1zG;YD`~60v+hJrADUMgD(N`gBU8vvtMk$TDa>Vaua@DeXL;BYjIX_ z30^^p3&5r_JMNuK<>Km*zIyk%)b7svAN*=D*)AwD%&Kr+@7er4%3HTVC@by{-P(Ma zw#5^8CQS7l-H`n2$EKLvMy=_#P?KL0F|>_uQjQa5XsA4GVY$L9BKsuPsk&WZx%i%M zh4`)$irwgR2cJzF z$rVrU>*Bf}?Am2jtH!<1Ro!iI50Mp=%MR#lCU7imvISZhCR%)j| z5W00N5{**F!{;x>HdM-=sqfotEm1;KcW63wU*xuB@%vb7TwJ1)z%nfcf+O14^gQiO zySkOKKD(ksX(+sc_m*HEu$+HsNB5=B>DUhA;f9>8?*^my0HXYXFMz#rtKQz?hc>1BN{CgMMMAN+&BF#mGxCC1f6CJG+Rpv^m=KTAY+?ep;6WCv>&w~d&uk{l&DfewhMn?Su ziJp;3WVu6y9?~K87d-3o3-M4TyWj=I1uQw#Jh-@Zk_nAFsQsYUc3N0z;d`|ZfB7O* zWg~C(9M)~bu~ji#)i>x_C>u@;uiS!iM0G!<)6liBPEiPsB}7}{pe*E~qF3eGCatv~ zqH$W!wPU$O*iYX82dty$j2crf#%5swNtI>D@Zr@H1lXjw&^8mcPOO`9YN$=yoY)L` z4Y!|{$vxSJx9Dhf!K6KgRX=UF_8_`?IHorok4!7THv-btP)+FH;FxF}5`dE;)-Jd@ ztf$_gMKlE`vQo2Qdu2xBTeD6c1|d@?%dkZkn;YqdYg^LdG-mnp9G-`?_W%kBS=*H= zzqT0mu7F_e{v7j1ztTuQwP45Uf<(Tyl06nQ<9qJ?uBczXY0y56flIKaFO$}-R@!Sf zLDKoQ2W`7VPNr(iGusP1=wmAF9?hGrcS>i14ks zJkbC2fx&H0PvmGX-?UNdCRiS8Esw*6IUdADW`>H6lQ)XAm%s6VMxli&NAJE~xb}c! zHF*D)_nnRfCQ{8tgNiCXpqdz`vXz8*x~8gPaB5u*lBY@D_rHE9x_~pLiBM5WK^(V&BcrmhvZmh>?g^`hk z*g;ZjqqxZ7k zl23Ct;Q&zxpR5Sp^z?p$)oeT|`S8I|L3ru>Udea(s6rM{RlZ+SIw{87vTW z*jXt{k}y6Yv_6-Mc`nD_EmpFvRntG55@EyX@C?=2d0vK(agdM%#n#IuJ9Ca^ZNl@- zWHZdSj=YW<*Pa)fkdI-00yaPRosd5DC^^#3qcgV;F&#_|GYalphTZb17P~tSbA9nm z9v&_<)&Fu~Q(sEo+XlS;Ff-wgbg+F2w))~~A9D#@yey&Er(pPVq-*9dJZsQ*+}zWY z)P!TRWze=)f9{Iwja0)tyi>p_v|^S@a=T{LFT;oMgs_mj%JSh1J^l=-mP!M4JDzy9 z_5H2hq9jzN?giT$Q`@|?ExrQ#?f<|K^Bo=D?G~L@7u}i$;SRr((0Xk0uDq=$yK^r1 za?>|=4v#c5jDzo=5IoBlH`}7MfcYA}FO@#sb^pA!)2$Omp2&6=>(TR?@ZD`DMUl*T z>*O^7d#7C~Kv52Lql|jnpS`iTwi^!}s>+Mm;Ykm(6wdg|G#oK?3@@;aj@2I2SGQF@ z|E7GPEFWfr{hNFyZ)A{uFhebyLyPa;uxfR{bt;pBNJbVteQt5uaIG&^vJbdvQABRf znOntpf~p%UE9qUbP|o5-_Mr)qefJN(_d)Tj*(2R?lE8$%iSH_*+XtDi%8vgxJ&}tg`uB7q(fXopmdB!k+gY%!| zJdXExt4U*6Q<9SK_<21YoYi zk{-{QUb=(4qg2d??r8yN!!zf+{PD23PJIwhvAoiGq}o~yx^ti?{!f#dwA)+uFr=wE zId0Ocsn~F)!q{uQwzw{ z#wZNRb%1D?#_o-*+^O@z zd9NR|lhvIaW$mi6&__7&PHO>@VR8a1}AynZv43@ z$UH1Y>3Vj-sXbCU}FCNdQ-2 zWk2nbcT5?YF(ai+_xX8E@Ba|!U^5zNu*?mAeY{Gy5w8q1{IR@X8IGEX;zuC)re2+82n=gE8eGPm^Q4vAIXv3$5GJg>p~7 zyg@qjFriO{rV3DUw>xDKE$T0UJ_V^BZcUwhZU6no+UfnoB`d+_r;%O_>wDQ*e={-HMnASY~!QbMl1!W$(aku4p(k4Nk;Czxd>{ zS2-upaA@d7A9#LOys=cJ4TI>ShjgvGU#e-JBmk567|!IHw1i4t85!e5W)!>xrgd(R zTUO(%R2;x{PKJwNEqkv_@(RVvYn1w4*dVW#cHv4iXBl<9GEB-<^|@C1{-4|YKZjSI zjL8uSoVcg7XIjdv8J&4HZVQ+LT8E*oSh@;JF8Zbvo8ZJ1?y@wPFu!g`b5N8CE{wz& zD6_1Fe@r>dW#KK(`M{v zp<~LDIWv=bhx?NZAPMhs&5ww3ZLPqkWLdChQUBXk`RbVQ(4p%=@8}xYBjmwaw7gt; z%kJ0N+Ad-qY!SY7W{$nluPxh*jof_ku#omq&NLHrYA@sA!QtD`zIQHk`5TjGAuJS@~S0gH;aq%v>mMyJW^cg$J*G_s%7wAn)O7oLfT+ zh3aW1??{1h1#YXchlXP-|7RURI&bh-4Io}mZON`XSEqxMC?j|llpXq0)HtI+ZJ8nq z2(=8TCMD$g;1JqgEy?ds5))&jxJ4}mK6l@@Ipq1oSOM0d`jA!y;~|dk^}|W(NUAXC zoutEGj6IhJ+)9ufRW@I)jKO<(bT$+d3yZwc*Wk*9x?V|t!z%N9d#wwbcruhgJyKh8 z@9xspz+S@k~;8R@&&46}4W|-UzC$R~AflGtaxj)+W5YP$68-guC9d<>N$1f3h;CrN6#k z>H8L++%vgs#r&<o})$9o{!$Zo$r}U8Tfy@K3zU z4~M|v^rcC~F85evAP&Pho_&JG0AA(v<$}u9`1{mYxCF&QBqVA>F6ybbODSYTdH$z_st=sV5naXRnJ^G zX25gZI(+#7ynTmxZe=vyncH!FpLUN;aH*0});dKw>%PrFv1U2=uwUNp=~U$RYd6!l z#h^P!c~PiI9_^qVRXvPK;1*X#w*+t?9O=#6(3LE6otjfi|2P+fs914OBTYcFT_kdpZL2MD7aEf$z+S1E4 ziFFd}sw?0<3sSlF>+}0k@wX>H!8H5+#aZs!*A=wFBd*xO)-9O&`-s9q^UfFRPPxc& zUS&Z;4?f^S^ZJz>y2Zcow$uPms}yKI=dSP!F3mL;&4hR&+Y#Q}A)l%=XneXLTw&O# z?X-ofEk*Xp!Q6~tZL3TqOiKM?gNK5lXXRMxc@VanOZ;`WGWi<*{w|;_1d?Gsq5J61c{5x z%R{@TUko~b_fVy3uxNO2RB^3?R=-up8!plCD_)m!o>z)bjEHTWDFTEFQFFSuy2r%HYt z+U~B+8~zIpBQ&FWAUm?@1`c2uLwIUxTb1V(dIMaqZ(v_^`PSVWqwkw(UOW`E9A16ix27kzoQe@NOlgeaK6g&QYz(rpC))&y5R;` z9`rWL`|T2J?kD3*I$01*3@MV0?&iM3wF6e6N~J7sOR%J}an!(q@0RzpZfKCaf0F>+ zAks~3jDxb;MaWE3wI1Im-}0eK{GqUUK}?;T5aIc2+biV3m_40x+9I_tP#(ac@GTDZ za5&@*Q(ae62Cs z(xt-VhU>vHb~M`gH4n~4z}fY~Drt@G$%Pdy2j46@Mh~14tKahD#K;+!hT5I=N_-9h z8ZZ^Y*D+~))8ARz9Q_m|XT2W63l>{%^`cmdDgzJSY9(Gfom>zJ&SBOli-O% zbNzT#hl$8%Z`5AaeY{*3T#&JRotB6FRJ;j&mn!O4-B=mr=DFf`7p*LCCux1}1WuY& zq{J4al~!l?RjT0HVO!4=GxiwoBGab1G8|rCvN$PH1qN9ECNR3~^R zQCxWLqnI#^@m7PBV=XKwp!z5CA3&#syf|K`8jbm9-M zIkYT(a!OI=@?CZhGk;1s-|&^^f7kDyadTt*db==*{9TaLS=Q9*)fF@lW94Pp+14ea_#^C5PMTu!CeZr^^;Xx+8yxkq1a9^|x3_)iYhMb?FFKu@+QjAn9| z7wig|Gk0+F%=Od8F*VJ-98U4@viIpvOlP;SYe&8*-c3`#vj;;4j_yRCaam&I? zia$kzm?V+g
gGgl(@MzeBO9d>^~C5EYLIPFITN&=Ia*?>`s3qWqtTjju>_gID!9L8vZf*g$a@&!C=eNu*|~D{XB=S zZ{@!ix+_eJYoSL2wb zIO=(7WRg+GrVRAA#3~i%(4}RiLLSFjLrC;O!V=tA_CJTVbDHr>{e{%%n~Gx(ugud7 zi~3lmOpm-TIKXX|k6R<-6^!Uj27szWW&VLbf0ZacEoGG*4oL%Q4w(RGX z)a99rxn3_baxidqdn9vvHaEBGNJl9W!KPUm;QykqL>h%qQXLY$VsGO(fl53T*~Y>) zDd)`l(DvSljT!@@_sHyXr<-3|VYF5wl)hSoxZ5d@XE0;%cs0}Zcx~$g>g}Kwh9-1Jzxuqh3r%{8+O9}s_qoke5KZJxb0gXE6NDqb{M7GS_ zw`A()o}E$PG#XhXvs=V|>-aK_l6^B;!;DJmy4TBD$7yX^?5~q3;a}S)yZrO6?-a2M zRi#q9O4ic;Hb1aVOj$-K>KQlk589V(|8*BHRE5VQ7JVOe} zn8M^nJFNL#|CGrQjm%tkh1bTVo4zi2ydH_OD2@tGAD#E6)%9GTXvPQosad3oPIXQi z?-TN1l7=bgNH=fVz`5Ud++M8tCdI8{yzgaYC~VA;Wy^k3jB?a5PF>CxApbyv6FIvh zDevz~N6<+B!)2yzd8Jn_U?Ic)sfu}UD{s5u_QcbgfeLQ6u%oHOPfeDtLIX(dwx2^6 zd#_t(fNk1wtD?o=r^MLvs*>-pfu+S!zhsq3mdz_lT&dB%<90=FjGMo%&Y^!EDceR| z8D&<*{%%L9&2UqFb|CY?J#bRXDZM0PsVQ@IEE?<>7=ENnWPMpQ!fLV`&Ka7{Es=QH zAbb0o%WUb(nVj11G8$Q>Z5;K?uh{riYs;@lfP`+@d^&%<_c}`~K;t&W#E3DQA~(*< z`VtG^7Dui7%O>%?p0RN{9$HE_Zz?pW2MZ0+I9lVV+CHbWuGab1=oeyO{Os=LsVV3; znMGVC{BnV3e9%NRnLGVq_}}E*lKOm)zQ*eo-iyp1F4#bCM8q0B6iUj%HKM-I9ODuF z(QV(qdCQH$b1U$KG+3mX6!9OPTyboYV8}8(F;2HxJJ3*I%dcA$BElBh_2m%6u2(1|v%4;&d~!4adf!DQOPgNFX}b zQCa*ovLDkb_R^+$<5!P$3)gD?()}q+Uo|JBLzHZrAs0K_n5b7Bid`kDc3;yq$mWNf zMrcL*yE*k2^JN9+cNMNHqfU+Zd%rt_UHjisHOZ z4R76me8!PD%FQ};Rw_6A+2jm)wbRIoK0__RQ@cl%ENlGi*Jim1J-@fVHg^qvRvSn2 z!-_Kx33>7}SO?KzOvW{l9j|fuFK8f%@TjeI$GZDX$ccmZ^3XlxnO3LVV~X7A0-2)m zaarX5jy9c4*Q}i7u=>q(>#^k<{DA&eR5!aUB4iw2HwRgtti6@GGrMuoF4-MDq=U~v zf?iQkpk?V_8rhi%Rdlbw)a_8MADZl<*ly8=ylBd;`?}-W_m+IA+-6Aj3f_Y7mz34H zY#;I)x#ff z(iDX>aJ-yaQmp$Vd1IXB3H1@o>;7;5c_j%%I@t%2hokiJ5%$8PXz+FtRmkE?_Y0aP zAX~gqLNl8s8Aa}gIlH1Xu9kJPf?3b}(o|sPh~Vh%P*{%AFOK!yv=4iwm6Ms0Yx@M^ z(X3rDns4f=SMA8*(AT$@A*ZwwqOKENp484GY!pl_sA79Hozsb2J-Q1`^BXX!b8@CH ziO5H>CVPK5Rh#>1Aba7TX!P@{*zFGurhAC;kZrQ}%jL4|wqsh1_g?-6k@%L&JQn`< zvz%o-%@g*qZu_?jGU^I4sZa1EXCLn78Z?*m{{8&&uQ>XlVR4@nksC1Z=R-99boKaW z1-)5O9gBWod>gAQm)mIRB@FsOv!3ueVCOz>V{s_w+rXdDGnnQ9`7K}axMwtYC~(Jy zMW5|0RbDYz6Rv2Qt*2VoETX=XF9=?rZ*ne>SrMzVNgZ6WgrEUwf<)pFj-4%gU$@}t z)}6ekc7B02XiigKxA7O;E)ZAv-mzSZTowQ3AUnnXg2$6TD>ve2_Xc@RWi#4edz&Ns zp!UdwnP>j`qi`%*39?`zODY#V86HAQI{ z#Cc7&r>*!lDC88v;!tpU)UhKDS?JGK4|=d?n?LAkxbrcZVw+*2FA`0N+Z%$6hO&a; zwX<2~${mJUXXtgkT zCE<^?zdq476HQNA*uwVZPv{SMb_`EXTNuP!YaQG3f!x{n;iqv-LnCLZ(%WzzD_pFcs<>L@4xZelPF=%0qf?0{6^ zx1Rsfiz)PEQnRwRKzQ4ZJt^*c&BPG2LrF`S^Nn$9^N$)9^x-h(mXA++t;lOV0!x|6 z6pjfq(>*sWH#me=HfE|s?&!L|RY%5N{u%o63Ayr{m%L)$z)9qj`8SoSg{D4%SD!Q@ zrLfy8nUn{?_ZqlT9!j!|U}1>x_lXl9+&EXMdhsEpT@H)*jn(dLk7Cxok?DSje>apc z3_oc>-uqv+Yd;tY9~9YEOblVF_aH->0KR$>9b%Nz7q0!FwM0(aWm=D}oc4W&i2Wg+ z5Z#Pm&c{jzFYnb&MpSEt^&~Q0@|}r`s)CWODfptNgXmDD)a~y- zG-hMp5y=b|H=RA(Z)PaXq_pn^)-$5g`-z&1S9exfxRr@YLkGNq^{*Qol`oRbG| z&P&}@s=C`BwkyKQkvAyzr>Tw2!-5Boq*@TgFxA^$Z1~KpS)q>&sHs%S3Ch-+KD6fU zS2%;?Lh3%A{YAt;yb8Gv3y^QeO}h!xHk`oUkppY6Y~0S8;6TS}$Cm6uh& zg|!kBW6U8is_aSxKJV@16Fe+~Uly$O(>JMqtcMAG> zBy(<$r}f_KkI_)dOQp;biIr)|5etOc5VVx)x+9}l={pO#dXqNJQg7x$=qL~5iKK0n zl$rbWcMb{8$i+y-l~my+)wIT$Lj`C-d($jpo+X+ZnQi|B{r!)t*j=yhmQIqP8DYRa)!UIADqd!9)NbzID14(1gsF0dZDRdxm!S2K*(X;y#9r%;xDud`))F3A-GPnHrylO`S)tJa zVNN`iC1*AA3ZMQ1}s%Gm5ckxf42u`>g-VNY)Vsue5it`v#E;By$aTnF)MbEIN3U4^-)L5 zsYsWLb6@%Dndoxr zpbn>#HM|JfAC%l2>vCph{900?mm&Aqc$a;bnG3_{nYcRDU5#HXu7G-3HIhg$|zwjL)56 zRyV`B{%@hQwKJkIlwA=fC~SPnYW8aBtSzqT-($jvLoSZg^h%q1O_n_!SD~wm>W?(k zuc$ti@5}zfXi^jdpPa{MuKZ|2BhMjv)yi!45}w&-G7Etfg9mRaxt07wF5)WPEn?!m zFMnGiuDAt0A=TV+gNoV>SN=Y)I`;0|WH2A!ks>0~o<^s3oewD2H7KBoFPGf}in0er08g?zI5ZO=uSLp(xUj%RZYYKcKN0 zk^AUgXsnaw_EU##*%yz$%lA--`^pRE&}}hDMNoV^BhB(YVz{2KAxe+uGQULZ zvTd+QGDoR;4RF8CDLQZudrx(pQFN@t%*V%GBNXLV#g?v$@p)0M>-!UU#*s=sC#{?H zx9oS9;nHBNSmOmMo5KQ;VK6-!+|LKTCJR*8%&e8iw@8Ft8ZSbzB&uDZ{YHXe5prCP zuw!93?or!v9b}m@$Roa}*xRl%fV`r*BDtYruXT7}Fe(m;c;(bFZr45L!bJRoiKTLn znggy6Yb-G`>S#C5^5JjXhP(;okhOiV1}I&cv{V*Zv^Vd>0_6D1?DzdI>`{-GdKvm4 zXM8y|cS!Y)BY9_tN=&v84D`Jg9_l}aTwYFq%zpFr)~gK}_(x@85Fpp|m|XHS7VC3N z8-IC++d)U~X?VP5Qz*JUQU21(YZbP&c@mY=1~1C@ZhQn*u^0Trt5W3`<$CIf{lnNG zzs(B|n$hNsgPxjZ<%WN}cJy_*kHwNU0?IXMW9{`*P8`xKL%Pr5RwWmmgM-Bxcx;Ak zEW;%OYn^ixQTRtYO;y~n9fm#9^1~28x{Q>W9HB($`|!n0*+D!I*6kK{fqZ)R;Dtqr zgM;ckd%|R9iPzm}@vHAF-~UM2+tnyMPOSuo^w*xpUIFG9xB=wEiS|csZMU?+!n;>d z2W<-4N{fpOS0YJ82g<3tdCLr)zB_@88B(TCR(WjSD1h?f2aiXY!sgi8j#X=O#yOz{ zYX*cefjz6%+}wg@s3YXlEK~b~<4v?b86Z(5PxqfYFOpe}eZaiQOiKL4h;bqm0A2bOc7XB<>tM^7Pwyxk=mJ{WX@+Z9a(L zuZ;e?stw90AJ!(_?FvWPl%R!KF6xb?4?e-QRK~ZP95^TNNj=`T^Npk09j%gze|6ja zsm8sEQoDs)f(HGs?#FTHRv3SJUE+{I1)6zb3shOc;4Sk)!y6}|`DqYGRfiem9VypV z1C3;=f=BLou5%xu$|Czzw@l#D8ybG=F|PbU;r~&mhd|`B7_CCId7E_Ef;ZM1%{j<` zptyeTz!cC*lKU}$e^XKDcD3` z^F?MBmpN6sC!i`n@q^!H8mV6D4jYV0XGwqXaf6_7BcJT0RVhq0{I6>aROMbjv*R5V ztBZxsx31IdHvanEtM}E(Ab#ZZ)z&zxHBDS^JUZx7dG$sc5It9~(XI z&HGd12Us^PnMdx_>Gw;vqIDx`3Ulsg!V0d;7zOd{19D~K`&{D#>-@@hyWj%beKhU3twedoi)1qA=c$;x@72CKVc#g1T ztSv-Z-mAki6(Y#{gMw_b+b2%Pex6mMROMV96x)g|=R_lC%2-O|oXC9{MtU4(1s?8Z z$xu?Jqpf(DhgLa?-%Wj0^v^H9d*K3(Zi$qNf9spyrqp%Zh zgAHN*$?O03W}vVKdw%xIO2*-iY_#y@ok^%(!h$8J;;jBQg&h3?b-jf>y=<`k@?5m_ z%Ld;uM*qU(1};w*vHgvfX*>-@lrTM4ba=uO-(_X69&Jv${+w6&uT&tzN}oEpXzfRZlAt zCmyy#wN86*cYp9WS6nL4raqGePV4mQ#Iv(p+|)(J5f8|QRewy!rta|ucy(a{tLVVy zGw1VfbRy}6@W+x9b2@H{H{!U6ols7#zjQu2LT-lr9e<(fkVO`KyfWJW<#XRjRCjzG zEoXEtLo%&WuNsm4_iFe!_F$IDlx>T=6j7EZ@p{9ImMN(31LMCExu~$R=-(CQMS}Hp zS$B?2#Sar;o!;hq|A6Euj({ncweB}OJ9Ldv;f|eBPVLako<51D^)%{Yyde50s@U7G ze>L(60h1|dswZ@q z_yH?R@Vt^K?2FBG;|yVo;NJzgY_fLj$aIeX^W#D^q#49nMC&_5!n6S~~R7dL>$78zCx|TFtvsK7`^K z0c^E!ZAVwzuza8@VmjkZa*w^cqApANY2NbF7{WbXmLNM=*xBEk!M-^3srmO(in^t@h&HxK^~n(W%|K$(3&lZ?&ah z)XYSgQZRGn$}0)}92|%8Yb}{$ZbSWpEWKz{&*FkhRh;Y(b+cw8!$^fJkW<&Bdu21~ z4x#k9%vbZ9U)*ks8$e#Y1wtwxmzdx9htSBvV2e*LfBFW4CFzfp8^qU4I~Y?Kg^Ei}dwnA?Akd|Lvo>T^Ve;;B%_H~Fda>Ab+ZG07kgG}mpm}>hkJ2vlT%<)>v zu4-W?Jd^!A-4X4KFiT;T2y4C}t-<@O;5wc71J>8?>LMN6!|;!OVc9EdKzmN?J5ZID z{PF(vxTG*Ne)~N%Gq?Xo!3CS%8>noYRVt$DtsWbxjvZ{vqa&F^TX=qI$_y(>9iAe{$5 z#8Sk6(^=^MIr0V}5*ddMe`w+>&_ymoN2ZdqEh>`bO)nzRH$^2PD)GC|r*!Xpn}VK= zgTLnK7kBU6gTrWG&L2>3n(#f?-$1zT_%8 zm%C3KgXWhX6wrii>pphffie#ftP>CGTGM*>IwGyDfOzXupBTN=*Q?R^V<2ihqc}YW zj(_0XiKWJ$8(M8q2*n~zHbN~@sj_0MtKq%kIyew1Z+Ko~H*xHn*HCbPftiYWCRprslOTKvP$;D35 z`#-(7dEfNg{@Deq{+R7>a@CL5j|SUcs(mzRz*R;SRV&`qC}jC}8ecRB#EI02vhLL% zk$;2$AwUQa0{?FiP`N4p{A%62;@-er?DFixb81X z7vQ5F%IA}Ry);AsLVyq;1PB2_fDj-A2mwNX5Fi8y0YZQfAOr{jLVyq;1PB2_fDj-A z2mwNX5Fi8y0YZQfm^ASG`;lIN+f3mX8wn3WfDj-A2mwNX5Fi8y0YZQfAOr{jLVyq; z1PB2_fDj-A2mwNX5Fi8y0YZQfAOr{jLVytX*AsXWHR@w^Gr$a{(R%H_UK%0*AwUQa z0)zk|KnM^5ga9Ex2oM5<03kpK5CVh%AwUQa0)zk|KnM^5ga9Ex2oM5<03kpKY}M8I zMBNMUhk|491HnNE5CVh%AwUQa0)zk|KnM^5ga9Ex2oM5<03kpK5CVh%AwUQa0)zk| zKnM^5ga9Ex2oM7QdIHgwOFvil0({g%`F!%Pmxc&H2oM5<03kpK5CVh%AwUQa0)zk| zKnM^5ga9Ex2oM5<03kpK5CVh%AwUQa0)zk|KnM^5lLnrDKhg_un<@NaBjG^^5CVh% zAwUQa0)zk|KnM^5ga9Ex2oM5<03kpK5CVh%AwUQa0)zk|KnM^5ga9Ex2oM7QdIC?P zMt!Vq2AIJ#TCe@rOG5-81PB2_fDj-A2mwNX5Fi8y0YZQfAOr{jLVyq;1PB2_fDj-A z2mwNX5Fi8y0YZQfAOr}3t-3m&sCxnaP;d->AUFsCLVyq;1PB2_fDj-A2mwNX5Fi8y z0YZQfAOr{jLVyq;1PB2_fDj-A2mwNX5Fi8y0YczkPaxWI>F4TRfRB18pHKev(hva% z0YZQfAOr{jLVyq;1PB2_fDj-A2mwNX5Fi8y0YZQfAOr{jLVyq;1PB2_fDj-A2mwMM zap3u1>Q;d7A25x17e`(fp#D$tj}RaP2mwNX5Fi8y0YZQfAOr{jLVyq;1PB2_fDj-A d2mwNX5Fi8y0Yc#a2Lf^bSyjKe8UFB${|}|d@C5(> literal 0 HcmV?d00001 diff --git a/tests/f_multithread_ok/name b/tests/f_multithread_ok/name new file mode 100644 index 00000000..45dd561d --- /dev/null +++ b/tests/f_multithread_ok/name @@ -0,0 +1 @@ +multiple threads checking on clean fs diff --git a/tests/f_multithread_ok/script b/tests/f_multithread_ok/script new file mode 100644 index 00000000..7334cde6 --- /dev/null +++ b/tests/f_multithread_ok/script @@ -0,0 +1,21 @@ +FSCK_OPT="-fym4" +SKIP_VERIFY="true" +ONE_PASS_ONLY="true" +SKIP_CLEANUP="true" + +. $cmd_dir/run_e2fsck + +grep -v Thread $OUT1 > $OUT1.tmp +cmp -s $EXP1 $OUT1.tmp +status1=$? +if [ "$status1" -eq 0 ]; then + echo "$test_name: $test_description: ok" + touch $test_name.ok +else + echo "$test_name: $test_description: failed" + diff $DIFF_OPTS $EXP1 $OUT1.tmp > $test_name.failed +fi + +unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2 +unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD +unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO From patchwork Mon Nov 7 12:21:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700610 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Vb0c3YYy; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vs352qLz23lx for ; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vs34c33z4xDK for ; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vs34Yfdz4xGj; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Vb0c3YYy; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vs34TVlz4xDK for ; Mon, 7 Nov 2022 23:28:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232280AbiKGM2x (ORCPT ); Mon, 7 Nov 2022 07:28:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53892 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232286AbiKGM2q (ORCPT ); Mon, 7 Nov 2022 07:28:46 -0500 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19C11A186 for ; Mon, 7 Nov 2022 04:28:45 -0800 (PST) Received: by mail-pf1-x431.google.com with SMTP id k22so10458988pfd.3 for ; Mon, 07 Nov 2022 04:28:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2EkOIBNA7D1BIBwaIa8nmWutEc3lXF7Ye4dNCdSq3Zo=; b=Vb0c3YYyinSpocRDLeVPm3+j8BuhZZnwThaJg/Z3YlgMVJl7H0v49qd497i8qvoEix 5cpnYrTqYBoqgJZQEgUU1kPg3/JHW9/qR8rudacymuNcnM4NmfZpkGgNM42wPYChKGUw OTmqWqdkQN3WARXBcH8ZIRxK39E3XRQIbCxYGgFi3YWiotuc35kGDeTf/Fw3a6Hd0Kmv 13bcMeGkLltdAqXOY6IVPl3K63HoPAbDUf5rRidvFE+yibPUMreHt1ZH72go9QBElKaJ qvSkAiRoUMab/5UEDl0hITwnldwGTEagAkEnT03OHNUoWI1smQvKPESAmiNlTiLMmCVm vevA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2EkOIBNA7D1BIBwaIa8nmWutEc3lXF7Ye4dNCdSq3Zo=; b=ElN05yQWIRURD9z78IGcWDeO1VogHICA6t32QXM3iyvBXEQ0E4kjz5fcuDlFta9kws vtGzX89pHVgo3fn2w7yMxrx50KQCONsplHTnwNT/yJOqBYvcN7lhmHHHTxZauWQmd/mX KaQG2wk41MDEK8FFdO+GKevvG2C99gGplwhqb5WvxZAJ/1W57HOY4+2dIYenmYjaPmsh xRYPdR/vypTGYx8PGUKaRfoYvPLLPHZuM7VfX8CkfUSCTzp8bcpVIaq1vIdgyfGj+XXL A+hpoFpqSJSl4VdydkY26w/DLlZyoqEzNS1n3btQRflZQumpBZFtCumx33XMawRR9qjE SALA== X-Gm-Message-State: ANoB5pm0NTFFuF4j0u2nwyh5kHmeba7EJCWoHq49WKiW5y9gHOYuBoOO NFie0eF54p4eZ5tN3q+v0ZIhPZVcn8E= X-Google-Smtp-Source: AA0mqf6ySxtcbQxnbU+V1sGDTVtvrnqX1Dnf6LnvSXEtMBHOTEuN3zfEBcSnd6njyJl4cS9zD9EwgQ== X-Received: by 2002:a63:e50d:0:b0:470:60a4:892b with SMTP id r13-20020a63e50d000000b0047060a4892bmr8384190pgh.148.1667824124613; Mon, 07 Nov 2022 04:28:44 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id c18-20020a63ea12000000b00462255f5aeasm4128183pgi.40.2022.11.07.04.28.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:43 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Ritesh Harjani Subject: [RFCv1 62/72] e2fsck: fix memory leaks with pfsck enabled Date: Mon, 7 Nov 2022 17:51:50 +0530 Message-Id: <1a72168b983a877458d5fc20409620df31b6a619.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong valgrind detected two memory leaks: 1) quota context is not released after merging. 2) @refcount_orig should be released Signed-off-by: Wang Shilong Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.c | 4 ++++ e2fsck/pass1.c | 1 + 2 files changed, 5 insertions(+) diff --git a/e2fsck/e2fsck.c b/e2fsck/e2fsck.c index 53af8905..db0a5059 100644 --- a/e2fsck/e2fsck.c +++ b/e2fsck/e2fsck.c @@ -106,6 +106,10 @@ errcode_t e2fsck_reset_context(e2fsck_t ctx) ea_refcount_free(ctx->refcount_extra); ctx->refcount_extra = 0; } + if (ctx->refcount_orig) { + ea_refcount_free(ctx->refcount_orig); + ctx->refcount_orig = 0; + } if (ctx->ea_block_quota_blocks) { ea_refcount_free(ctx->ea_block_quota_blocks); ctx->ea_block_quota_blocks = 0; diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index ed4275c3..d745699d 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -3039,6 +3039,7 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx); + quota_release_context(&thread_ctx->qctx); /* * @block_metadata_map and @block_dup_map are * shared, so we don't free them. From patchwork Mon Nov 7 12:21:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700611 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=fdfk6xNa; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vv32qJDz1yqS for ; Mon, 7 Nov 2022 23:30:39 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vv31xDPz4xG8 for ; Mon, 7 Nov 2022 23:30:39 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vv31vXgz4xGT; Mon, 7 Nov 2022 23:30:39 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=fdfk6xNa; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vv31qJ2z4xG8 for ; Mon, 7 Nov 2022 23:30:39 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232222AbiKGMah (ORCPT ); Mon, 7 Nov 2022 07:30:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232255AbiKGM2v (ORCPT ); Mon, 7 Nov 2022 07:28:51 -0500 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D3BC3A2 for ; Mon, 7 Nov 2022 04:28:51 -0800 (PST) Received: by mail-pl1-x631.google.com with SMTP id y4so10922189plb.2 for ; Mon, 07 Nov 2022 04:28:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RqRhkkJs9Imy/9nsxcxcZAJBsG1Q3o+zeup0dYMht2E=; b=fdfk6xNaeigJgyGy7UPfMxTfsFtyH/VLyd01sjuRzJ7W0WNkLUNFTq7ixU36Uk4Qbs tobR2bf5DIlvDJZgAKRflcGDAa2RrOiFTvSsBhuHPyyz6gPwYlvtRSVp168VdDS7o6ZG tXySCbOiCm2rU+O3UikTeN7fC1/M8LpTDnp4u8u6FozpRVj91Q0G6BDQ/g42vuFCnpYK ZQrgxHM2jjMwacFlQ168x9VXKnBUgzgq91ahnbvEgKYjG3zTNsqmzpcnXoA6RjIqebNR YAea0KykuCs9oIpdAvesvyeP0Jp2ZoQbaMn8qyNUmyNGwKt9ODgOvqRp90vKmbFAv+k9 LrtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RqRhkkJs9Imy/9nsxcxcZAJBsG1Q3o+zeup0dYMht2E=; b=x8GZ/vc6gvk/4MZJ9mXPsb0Bs+tJmm4yahlyAP0Fe2JkecAgYuEAOFiB843ImIr6rE GwYTERBOuAWocuVqVYFpzwqUyf0kUx+zD1JAvBV+r2uB533novz08SsdabycdHAodzsQ f7vd1zIVQNwnA1xWs/hVxO67JlH5oQ0N/DvYs8qpwiurtFG+cuZI5HV75Cogo8l2x+jk UVXJ/aVhLl/IcaUmQVk3SeJrZihbo6GeAv2ynYYW+spsLAyz+vMgvNJcJau8OKDZzTVO i+qn0T1hag+VzFAAb/V/42wk3wLWlqD0HKZhq23RKUQFFzDJ/jrM9yfRTEnD0RK3vdr1 yq2g== X-Gm-Message-State: ACrzQf1fzn9J1xvZIsFMBmxRsSXPqDImXJ7yQXhyNfcBSH+VzsjBS6KI 6ZzP+dIAPpDGzk6zVCvtJmk= X-Google-Smtp-Source: AMsMyM7fJgRpWEiLg63aomqdCeiHOSifjzgOwxsq21ruAg2fc4P0PvjzEMS8/0hZ5NWS8irjdDn4Lg== X-Received: by 2002:a17:903:2289:b0:187:21f6:fdea with SMTP id b9-20020a170903228900b0018721f6fdeamr41706160plh.120.1667824130457; Mon, 07 Nov 2022 04:28:50 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id y8-20020a17090a104800b002137d3da760sm6017506pjd.39.2022.11.07.04.28.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:49 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Andreas Dilger , Ritesh Harjani Subject: [RFCv1 63/72] e2fsck: misc cleanups for pfsck Date: Mon, 7 Nov 2022 17:51:51 +0530 Message-Id: <59a08a01fd7e24b8fe9341d647510f55389f951d.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Andreas Dilger Add -m option description to e2fsck.8 man page. Rename e2fsck_struct fs_num_threads to pfs_num_threads to avoid confusion with the ext2_filsys fs_num_threads field, and move thread_info to be together with the other HAVE_PTHREAD fields. Move ext2_filsys fs_num_threads to fit into the __u16 "pad" field to avoid consuming one of the few remaining __u32 reserved fields. Fix a few print format warnings. Signed-off-by: Andreas Dilger Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.8.in | 8 +++++++- e2fsck/e2fsck.h | 13 +++++-------- e2fsck/pass1.c | 20 +++++++++++--------- e2fsck/unix.c | 4 ++-- 4 files changed, 25 insertions(+), 20 deletions(-) diff --git a/e2fsck/e2fsck.8.in b/e2fsck/e2fsck.8.in index dc6a5856..37dc8936 100644 --- a/e2fsck/e2fsck.8.in +++ b/e2fsck/e2fsck.8.in @@ -8,7 +8,7 @@ e2fsck \- check a Linux ext2/ext3/ext4 file system .SH SYNOPSIS .B e2fsck [ -.B \-pacnyrdfkvtDFV +.B \-pacnyrdfkmvtDFV ] [ .B \-b @@ -333,6 +333,12 @@ Set the bad blocks list to be the list of blocks specified by option, except the bad blocks list is cleared before the blocks listed in the file are added to the bad blocks list.) .TP +.B \-m " threads" +Run e2fsck with up to the specified number of +.IR threads . +The actual number of threads may be lower, if the filesystem does not +have enough block groups to effectively parallelize the workload. +.TP .B \-n Open the file system read-only, and assume an answer of `no' to all questions. Allows diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 2dd7ba27..33866316 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -277,8 +277,8 @@ struct e2fsck_thread { dgrp_t et_group_next; /* Scanned inode number */ ext2_ino_t et_inode_number; - char et_log_buf[2048]; char et_log_length; + char et_log_buf[2048]; }; #endif @@ -367,11 +367,6 @@ struct e2fsck_struct { ext2_ino_t stashed_ino; struct ext2_inode *stashed_inode; - /* if @global_ctx is null, this field is unused */ -#ifdef HAVE_PTHREAD - struct e2fsck_thread thread_info; -#endif - /* * Location of the lost and found directory */ @@ -487,7 +482,9 @@ struct e2fsck_struct { char *undo_file; #ifdef HAVE_PTHREAD - __u32 fs_num_threads; + /* if @global_ctx is null, this field is unused */ + struct e2fsck_thread thread_info; + __u32 pfs_num_threads; __u32 mmp_update_thread; int fs_need_locking; /* serialize fix operation for multiple threads */ @@ -732,7 +729,7 @@ void check_resize_inode(e2fsck_t ctx); int check_init_orphan_file(e2fsck_t ctx); /* util.c */ -#define E2FSCK_MAX_THREADS (65536) +#define E2FSCK_MAX_THREADS (65535) extern void *e2fsck_allocate_memory(e2fsck_t ctx, unsigned long size, const char *description); extern int ask(e2fsck_t ctx, const char * string, int def); diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index d745699d..8a6cdd8f 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1265,7 +1265,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) { unsigned flexbg_size = 1; ext2_filsys fs = ctx->fs; - int num_threads = ctx->fs_num_threads; + int num_threads = ctx->pfs_num_threads; int max_threads; if (num_threads < 1) { @@ -1285,6 +1285,8 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) max_threads = fs->group_desc_count / flexbg_size; if (max_threads == 0) max_threads = 1; + if (max_threads > E2FSCK_MAX_THREADS) + max_threads = E2FSCK_MAX_THREADS; if (num_threads > max_threads) { fprintf(stderr, "Use max possible thread num: %d instead\n", @@ -1292,7 +1294,7 @@ static void e2fsck_pass1_set_thread_num(e2fsck_t ctx) num_threads = max_threads; } out: - ctx->fs_num_threads = num_threads; + ctx->pfs_num_threads = num_threads; ctx->fs->fs_num_threads = num_threads; } #endif @@ -1320,7 +1322,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) #ifdef HAVE_PTHREAD /* don't use more than 1/10 of memory for threads checking */ - readahead_kb = get_memory_size() / (10 * ctx->fs_num_threads); + readahead_kb = get_memory_size() / (10 * ctx->pfs_num_threads); /* maybe better disable RA if this is too small? */ if (ctx->readahead_kb > readahead_kb) ctx->readahead_kb = readahead_kb; @@ -1378,7 +1380,7 @@ static errcode_t e2fsck_pass1_prepare(e2fsck_t ctx) #ifdef HAVE_PTHREAD pthread_rwlock_init(&ctx->fs_fix_rwlock, NULL); pthread_rwlock_init(&ctx->fs_block_map_rwlock, NULL); - if (ctx->fs_num_threads > 1) + if (ctx->pfs_num_threads > 1) ctx->fs_need_locking = 1; else ctx->fs_need_locking = 0; @@ -1659,7 +1661,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) if (ctx->global_ctx) { if (ctx->options & E2F_OPT_DEBUG && ctx->options & E2F_OPT_MULTITHREAD) - fprintf(stderr, "thread %d jumping to group %d\n", + fprintf(stderr, "thread %d jumping to group %u\n", ctx->thread_info.et_thread_index, ctx->thread_info.et_group_start); pctx.errcode = ext2fs_inode_scan_goto_blockgroup(scan, @@ -3064,7 +3066,7 @@ static int e2fsck_pass1_threads_join(struct e2fsck_thread_info *infos, errcode_t ret = 0; int i; struct e2fsck_thread_info *pinfo; - int num_threads = global_ctx->fs_num_threads; + int num_threads = global_ctx->pfs_num_threads; /* merge invalid bitmaps will recalculate it */ global_ctx->invalid_bitmaps = 0; @@ -3130,7 +3132,7 @@ static void *e2fsck_pass1_thread(void *arg) out: if (thread_ctx->options & E2F_OPT_MULTITHREAD) log_out(thread_ctx, - _("Scanned group range [%lu, %lu), inodes %lu\n"), + _("Scanned group range [%u, %u), inodes %u\n"), thread_ctx->thread_info.et_group_start, thread_ctx->thread_info.et_group_end, thread_ctx->thread_info.et_inode_number); @@ -3156,7 +3158,7 @@ static int e2fsck_pass1_threads_start(struct e2fsck_thread_info **pinfo, int i; e2fsck_t thread_ctx; dgrp_t average_group; - int num_threads = global_ctx->fs_num_threads; + int num_threads = global_ctx->pfs_num_threads; #ifdef DEBUG_THREADS struct e2fsck_thread_debug thread_debug = {PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER, 0}; @@ -3259,7 +3261,7 @@ void e2fsck_pass1(e2fsck_t ctx) if (retval) return; #ifdef HAVE_PTHREAD - if (ctx->options & E2F_OPT_MULTITHREAD || ctx->fs_num_threads > 1) + if (ctx->options & E2F_OPT_MULTITHREAD || ctx->pfs_num_threads > 1) e2fsck_pass1_multithread(ctx); else #endif diff --git a/e2fsck/unix.c b/e2fsck/unix.c index dfa3f897..461ab8cb 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -915,12 +915,12 @@ static errcode_t PRS(int argc, char *argv[], e2fsck_t *ret_ctx) _("Invalid multiple thread num.\n")); if (thread_num > E2FSCK_MAX_THREADS) { fprintf(stderr, - _("threads %lu too large (max %lu)\n"), + _("threads %lu too large (max %u)\n"), thread_num, E2FSCK_MAX_THREADS); fatal_error(ctx, 0); } ctx->options |= E2F_OPT_MULTITHREAD; - ctx->fs_num_threads = thread_num; + ctx->pfs_num_threads = thread_num; break; #endif case 'n': From patchwork Mon Nov 7 12:21:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700612 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ApQufOvn; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vv45NRyz23lx for ; Mon, 7 Nov 2022 23:30:40 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vv44ty2z4xG8 for ; Mon, 7 Nov 2022 23:30:40 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vv44rQrz4xGT; Mon, 7 Nov 2022 23:30:40 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ApQufOvn; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vv44m5Pz4xG8 for ; Mon, 7 Nov 2022 23:30:40 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232326AbiKGMaj (ORCPT ); Mon, 7 Nov 2022 07:30:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232217AbiKGM25 (ORCPT ); Mon, 7 Nov 2022 07:28:57 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90FBC232 for ; Mon, 7 Nov 2022 04:28:56 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id j12so10912295plj.5 for ; Mon, 07 Nov 2022 04:28:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Mdcb4BTkVS1l8ipTC8GIkWGgMANpQ3W9U+W9dp2alyU=; b=ApQufOvnqFbcLAPqh6CBr8vpZKFwfA4Ohjc54JH24j5vA/hvmPurDM+acY6x1pPAEv Ok829Ap0iQSSCCzsCgd8gofewa6XTCzNPCZsNxHcziSng0SbNbYMdKcN1y+a4/oEXz4b gXu7173oPwPI8e0uqJo8IOfeImxULnI2BgNisOCARaiX7mbzGVx90J7oid0RXedH3Iep E8q3nLBnXCPSLD4kiX2LuXEJE2ODGymFBJ+tNjljVn6Di5Igx4EiZ2cu17olSP4lBoBk qOA8n/RD0OkjPE/JZLfTkCQVFoOD4csl+Y8DDeS6u0A/i63G4rHe+b0+yo/WiGuTnXqc KW6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mdcb4BTkVS1l8ipTC8GIkWGgMANpQ3W9U+W9dp2alyU=; b=pD5a2YLpPpJQ0nD24HbuxmDGq4PM3mk7t1BU34OretT1LN6Rkx4emvFCDFmQirJaWv OS8OLrEWyMw9DIdRagt93F+PpEyWdYPqIhNilpqi8GyikCgS+myuA0GVxuoTazWKND6N wIuXNKRI6L5XIHU8qP45eGtJrbT6vXfSksqv7hva1d1P9WWPkaHBO6s6zLHNj98l6wOZ xZ6oceh7BOp9yIePvwQQyVwBdV5zyZ+hrKgbAmyDFl2Bi5VYK625yL95GVbW4Fndtqje EKa47Oct1IKKXEMXSjgpP0cZDBXGrpOEJwuprxXecOn/tkU5ZjE7dRTQ0C++dhXVK754 eW5A== X-Gm-Message-State: ACrzQf3fUzKDuajeAzhiY8ulw6OdTztDtrxbzxX9Uef3SCLYpPP2hbu8 H2WNod9xUv52GrZEHskPFEY= X-Google-Smtp-Source: AMsMyM5AL8qEdPleK8TQeL6wPdLDTmqwK6hdbEKsoXUj8Kb8iYaN/soHtAtX5l1/ts4IhuKTWQ7hYg== X-Received: by 2002:a17:90b:1095:b0:213:ee6a:f268 with SMTP id gj21-20020a17090b109500b00213ee6af268mr39138430pjb.213.1667824136085; Mon, 07 Nov 2022 04:28:56 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id f2-20020a623802000000b0056232682a7esm4386947pfa.2.2022.11.07.04.28.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:28:55 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Saranya Muruganandam , Ritesh Harjani Subject: [RFCv1 64/72] e2fsck: propagate number of threads Date: Mon, 7 Nov 2022 17:51:52 +0530 Message-Id: <538c698d8259613f69d46ae29847ac256778e135.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Saranya Muruganandam Sometimes, such as in orphan_inode case, e2fsck_pass1 is called after reading the block bitmaps. This results in reading the block bitmap sequentially and multithreading only gets kicked in later. Fix the thread count earlier while setting up the file system. Signed-off-by: Saranya Muruganandam Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/unix.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/e2fsck/unix.c b/e2fsck/unix.c index 461ab8cb..fb0df85a 100644 --- a/e2fsck/unix.c +++ b/e2fsck/unix.c @@ -1745,6 +1745,9 @@ failure: ctx->fs = fs; fs->now = ctx->now; +#ifdef HAVE_PTHREAD + fs->fs_num_threads = ctx->pfs_num_threads; +#endif sb = fs->super; if (sb->s_rev_level > E2FSCK_CURRENT_REV) { From patchwork Mon Nov 7 12:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700613 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=IhrE6Lsh; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvF5VDHz1yqS for ; Mon, 7 Nov 2022 23:30:49 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvF52F1z4xG8 for ; Mon, 7 Nov 2022 23:30:49 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvF4zj2z4xGT; Mon, 7 Nov 2022 23:30:49 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=IhrE6Lsh; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvF4shgz4xG8 for ; Mon, 7 Nov 2022 23:30:49 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232287AbiKGMas (ORCPT ); Mon, 7 Nov 2022 07:30:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232298AbiKGM3C (ORCPT ); Mon, 7 Nov 2022 07:29:02 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 396F91038 for ; Mon, 7 Nov 2022 04:29:02 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id b185so10452183pfb.9 for ; Mon, 07 Nov 2022 04:29:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZW6y/QZcxmvZCTvnr9AhcCkA/wkdGYiX//7dn8rWZ28=; b=IhrE6LshpY0kCaNyJT9iNbBFJvfo7qiRUhmcjTqlyJjob47horGomCIcY+ztL6GZsB wy6ay6dHHIeQugNCX0oFgok4SAKgGgGHvW7yE+6cVV12ZOCFdoVlMeuSFb5popdt3k0C w3gmACEjlRdYQtYpGUsnkTHgKXQiXuwCV2c7gb6SP4klEhzz+1VFyrKD92j4mDvpd+sj ZovBrvC/yMxwEsL7AdgxXk8Blmzzh4W8ipQR6HdX+Z1pYfDRAWo6+mhJlYJmSwrp6dSV FEBKuiyCUTkmwh1d7OcAaYrfX6WXkQTFC4mYueyN2aaXjX57DUctU+Bm4kfrvhD0ruQh UBIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZW6y/QZcxmvZCTvnr9AhcCkA/wkdGYiX//7dn8rWZ28=; b=2Oigkv22mq59eE2ZOvhvvZkqaG40Q6NPUy14uF815WcYgwd0VNl404Xfh/rto/U5WG q0DS9nZkLOE1onywTaJxhzPMt2O+K2/YJwzW7DLZ9iHfoLIy+dQIYchMMndOcnMW5ias m0uIpoQ93SdbQGXBOq9ctQVtvTooaPKXOin+dMhqYdKSRMl+7kJ9S9OBgGDecHRiwpbm Ls7dra447++8vwSDORgXRSIIFwf67AL4TD5SM0EzSLcae6XVy79NJ/J8Vl54QIgXR/yD ZwlCIPDXRt1f6uUOw/RN/bFxbAbwIIZoE3SEEmZ6aczZPo1A6TUYQDQOIOqp9zC0/e4e +5eQ== X-Gm-Message-State: ACrzQf07LVuxovnaf7BEa0Esp35pHOIv+VxrMiUQxnGVlE+fS5PypnY4 A0jXOEBXklYBz09ls6AbxX7rtB8etTk= X-Google-Smtp-Source: AMsMyM7WcyetuWj+u7pRp/hvOO4wTrjAgGWF+OjmgWEbprPpsBI5WQ7L+euQ3rFmhWOonLBsgg8v0w== X-Received: by 2002:a63:c04b:0:b0:46f:c183:242d with SMTP id z11-20020a63c04b000000b0046fc183242dmr34938582pgi.287.1667824141704; Mon, 07 Nov 2022 04:29:01 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id e18-20020a056a0000d200b0056c5aee2d6esm4380740pfj.213.2022.11.07.04.29.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:01 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Saranya Muruganandam , Ritesh Harjani Subject: [RFCv1 65/72] e2fsck: Annotating fields in e2fsck_struct Date: Mon, 7 Nov 2022 17:51:53 +0530 Message-Id: <17fb23e236be36b22ca578def6d5efeadfba2d0c.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Saranya Muruganandam Adding information on fields in e2fsck_struct on how they are used when running parallel fsck. Signed-off-by: Saranya Muruganandam Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 33866316..1e82b048 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -285,7 +285,7 @@ struct e2fsck_thread { struct e2fsck_struct { /* Global context to get the cancel flag */ e2fsck_t global_ctx; - ext2_filsys fs; + ext2_filsys fs; /* [fs_fix_rwlock] */ const char *program_name; char *filesystem_name; char *device_name; @@ -294,7 +294,9 @@ struct e2fsck_struct { char *log_fn; FILE *problem_logf; char *problem_log_fn; - int flags; /* E2fsck internal flags */ + /* E2fsck internal flags. + * shared by different threads for pass1 [fs_fix_rwlock] */ + int flags; int options; unsigned blocksize; /* blocksize */ blk64_t use_superblock; /* sb requested by user */ @@ -314,6 +316,7 @@ struct e2fsck_struct { int (*progress)(e2fsck_t ctx, int pass, unsigned long cur, unsigned long max); + /* The following inode bitmaps are separately used in thread_ctx Pass1*/ ext2fs_inode_bitmap inode_used_map; /* Inodes which are in use */ ext2fs_inode_bitmap inode_bad_map; /* Inodes which are bad somehow */ ext2fs_inode_bitmap inode_dir_map; /* Inodes which are directories */ @@ -322,12 +325,14 @@ struct e2fsck_struct { ext2fs_inode_bitmap inode_reg_map; /* Inodes which are regular files*/ ext2fs_inode_bitmap inode_casefold_map; /* Inodes which are casefolded */ + /* Following 3 protected by [fs_block_map_rwlock] */ ext2fs_block_bitmap block_found_map; /* Blocks which are in use */ ext2fs_block_bitmap block_dup_map; /* Blks referenced more than once */ ext2fs_block_bitmap block_ea_map; /* Blocks which are used by EA's */ /* - * Inode count arrays + * Inode count arrays. + * Separately used in thread_ctx, pass1 */ ext2_icount_t inode_count; ext2_icount_t inode_link_info; @@ -349,7 +354,8 @@ struct e2fsck_struct { /* * Array of flags indicating whether an inode bitmap, block - * bitmap, or inode table is invalid + * bitmap, or inode table is invalid. + * Separately used in thread_ctx, pass1 */ int *invalid_inode_bitmap_flag; int *invalid_block_bitmap_flag; @@ -362,7 +368,8 @@ struct e2fsck_struct { char *block_buf; /* - * For pass1_check_directory and pass1_get_blocks + * For pass1_check_directory and pass1_get_blocks. + * Separately used in thread_ctx in pass1 */ ext2_ino_t stashed_ino; struct ext2_inode *stashed_inode; @@ -421,6 +428,7 @@ struct e2fsck_struct { /* * How we display the progress update (for unix) + * shared by different threads for pass1 [fs_fix_rwlock] */ int progress_fd; int progress_pos; @@ -429,7 +437,7 @@ struct e2fsck_struct { int interactive; /* Are we connected directly to a tty? */ char start_meta[2], stop_meta[2]; - /* File counts */ + /* File counts. Separately used in thread_ctx, pass1 */ __u32 fs_directory_count; __u32 fs_regular_count; __u32 fs_blockdev_count; From patchwork Mon Nov 7 12:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700614 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ePJ+cziZ; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvH0d6zz1yqS for ; Mon, 7 Nov 2022 23:30:51 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvH07frz4xG8 for ; Mon, 7 Nov 2022 23:30:51 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvH02gXz4xGT; Mon, 7 Nov 2022 23:30:51 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ePJ+cziZ; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvG700Zz4xG8 for ; Mon, 7 Nov 2022 23:30:50 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232284AbiKGMat (ORCPT ); Mon, 7 Nov 2022 07:30:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54132 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232134AbiKGM3I (ORCPT ); Mon, 7 Nov 2022 07:29:08 -0500 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17E93A186 for ; Mon, 7 Nov 2022 04:29:08 -0800 (PST) Received: by mail-pj1-x1034.google.com with SMTP id q1-20020a17090a750100b002139ec1e999so10148326pjk.1 for ; Mon, 07 Nov 2022 04:29:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2nvBMel3mK2Mvtlci+Tq1QH/JT3NGu7C2JwIvGymT7Q=; b=ePJ+cziZfDJxFxKvbhpzfqF8URTyI6EmFSPGx3+aAJrdEAkraliTEn570uM8cuxJEH taNm/9liXM4MxlZViQS77Yf5b03lcsgcUOc1T0Fj4BwUuc+vAUurrouTAnVFIKsgxAXW JCaM+VqtsBTRBDi6DdOZZDMhQLM39gwWb044FEQ8Rm0dSrRXwzrUBfiw5EhsCG1Ia6VP SuT1kiTz2UP20S49rT6JL6N8cKsqZTWzwhQL5uexcFTCX17SKBU0/BR3STu3/iBtDkkN MaMsDl+XuMBSPMu7lJnEGmJdG+ushXK4kv4oie3E9YO4jS7dNdYNjMuQkAVulpppJM86 B1aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2nvBMel3mK2Mvtlci+Tq1QH/JT3NGu7C2JwIvGymT7Q=; b=HvvL2KYae5ynmPpLX6DsafnCAFUuN6UKO/hHsFXH+9jDN9+CNM3tXrdASDKmEAcnGc YivCV+8lx1wou+2Fp3sorgFgxBRIIkBW4wce65eYHP2cxGNYT0oxnL5PZ4SGupFAMmmB 34bzm8c7dogmlinKeOca0ikn88db6j6tBBcJSkqyCXzKd3TQGU3V4NeRCRABwJjnfYdN y2yLvw+mQIbyAIMgIEtlYQV/uWfKkZfX1XT4jqjRDZT8vwfEeAat/qhPi/9P5ZMRPLDb RIUbsPmbwLbNJkeemDmtAKrLTaKdpxAnvxxP8Dh9ogdMMgrA1AAj5JVknm2FNY84xtSW Nbmw== X-Gm-Message-State: ANoB5pm4y5QHqr2+YDQf7U/FdIemAtc4fb8iioIaleSdEGtfp3zuU/1b cx5V5xirNZcTpK++WtM1HWk= X-Google-Smtp-Source: AA0mqf6WIuLEIz+ZU0GRcWdutsJ6t8qFnTs4leRBJjZsnPj4BUdyYFeNVOaSt3rLfhtYP8AmLtxmjA== X-Received: by 2002:a17:903:1209:b0:188:8239:8ee3 with SMTP id l9-20020a170903120900b0018882398ee3mr4423144plh.21.1667824147644; Mon, 07 Nov 2022 04:29:07 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id n7-20020a63f807000000b0046ece12f042sm4047214pgh.15.2022.11.07.04.29.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:06 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Li Dongyang , Maloo , Andreas Dilger , Ritesh Harjani Subject: [RFCv1 66/72] e2fsck: merge casefolded dir lists after thread finish Date: Mon, 7 Nov 2022 17:51:54 +0530 Message-Id: <8f798915e2322756a7c01126865997ee3c77cc78.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Wang Shilong This is missed and should be done after threads finish. Original commit: 18538b27 ("LU-8465 e2fsck: merge encrypted dir lists after thread finish") Upstream is now using e2fsck_struct->casefolded_dirs instead of encrypted_dirs. Only for the correctness of pfsck, we won't use casefolded feature on ldiskfs. Signed-off-by: Wang Shilong Signed-off-by: Li Dongyang Tested-by: Maloo Reviewed-by: Andreas Dilger Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 8a6cdd8f..7345c96d 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2614,6 +2614,7 @@ static errcode_t e2fsck_pass1_thread_prepare(e2fsck_t global_ctx, e2fsck_t *thre } memcpy(thread_context, global_ctx, sizeof(struct e2fsck_struct)); thread_context->block_dup_map = NULL; + thread_context->casefolded_dirs = NULL; retval = e2fsck_allocate_block_bitmap(global_ctx->fs, _("in-use block map"), EXT2FS_BMAP64_RBTREE, @@ -2905,6 +2906,24 @@ static errcode_t e2fsck_pass1_merge_ea_refcount(e2fsck_t global_ctx, return retval; } +static errcode_t e2fsck_pass1_merge_casefolded_dirs(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + errcode_t retval = 0; + + if (!thread_ctx->casefolded_dirs) + return 0; + + if (!global_ctx->casefolded_dirs) + retval = ext2fs_badblocks_copy(thread_ctx->casefolded_dirs, + &global_ctx->casefolded_dirs); + else + retval = ext2fs_badblocks_merge(thread_ctx->casefolded_dirs, + global_ctx->casefolded_dirs); + + return retval; +} + static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx, e2fsck_t thread_ctx) { @@ -2971,6 +2990,13 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx, if (retval) return retval; + retval = e2fsck_pass1_merge_casefolded_dirs(global_ctx, thread_ctx); + if (retval) { + com_err(global_ctx->program_name, 0, + _("while merging casefolded dirs\n")); + return retval; + } + e2fsck_pass1_merge_invalid_bitmaps(global_ctx, thread_ctx); retval = e2fsck_pass1_merge_bitmap(global_fs, From patchwork Mon Nov 7 12:21:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700618 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=a8MyoMNn; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvQ4K2Rz1yqS for ; Mon, 7 Nov 2022 23:30:58 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvQ3qPSz4xGj for ; Mon, 7 Nov 2022 23:30:58 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvQ3mhJz4xx0; Mon, 7 Nov 2022 23:30:58 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=a8MyoMNn; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvQ3gKGz4xGj for ; Mon, 7 Nov 2022 23:30:58 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232299AbiKGMaz (ORCPT ); Mon, 7 Nov 2022 07:30:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232314AbiKGM3P (ORCPT ); Mon, 7 Nov 2022 07:29:15 -0500 Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 01956140F6 for ; Mon, 7 Nov 2022 04:29:14 -0800 (PST) Received: by mail-pf1-x434.google.com with SMTP id k15so10468929pfg.2 for ; Mon, 07 Nov 2022 04:29:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZnDhELuUV+zsI8HDQ2YNpQ64OU5t0mjR3T71P+XOOb8=; b=a8MyoMNnwQNapkVU9vXHl1SynLKNVtf1HlTMYrxAOkJC8GJJeR8lrCFQxXcyvLd50v hUFmLWGdlyoIHG7lcNcWilZrQXmLW+wzSPrm6Czl3o6gLAkTyOYZioenwDk0SlWGryaT VhITN/gahSiEgoAqbbzZh7X0b2ym1bR/sgn11g54ZWblVJjCf8EjAy8MrBbAOd2dWAKB xFoZRrIok1Ix4K2tZ0JsSTdYT9hOuQQYY+nrJuh6AdyGId0AwheOJsfUvs8h3NC1DFIt ofgz3+bwQvmORE+20IUYpMmWRxtocOip84nxHPVeKkXJRa6oNUktXdOVChnE1Uqw6V0d 05HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZnDhELuUV+zsI8HDQ2YNpQ64OU5t0mjR3T71P+XOOb8=; b=LewHbNgmx8XUZ0K8j8oC2RX9/ZUHw8xIrxziKLBVtE9bbJS4KjnYLfYNZnlI4VyNpc UAvDaZr72UuiH84odFutohg8h5j7LUJXJjXrKj+ffJcshM60D3Yxnp7oTpNPvtYdIiTA 1I4e/rUegKx+O8yDG+DEh3laah+lo3oRnfTF/Ysu9nFLRyuQe8ORuglq9CjNBQudkBU8 rT/ZuaB0Qamy52UKf5ebYFtOvk7jmZxYtRGH424rzRTjF7wRbxoA86miu85z/CJe/W7N +lPBON+6l2/3zY2C4zTCTvEbboyDzkIJ3+U/i3U2I5VZG6RmhhD25VBpABve0G1N9IJi HOiw== X-Gm-Message-State: ACrzQf1MaB9SX79dcDwyCoqfFj19/t06brqDvJK6+iytWH7cgQXC6jcr RbjLFolkc7mnn7SAospDDxs= X-Google-Smtp-Source: AMsMyM7cgeXY+NWEal5v1KVDsWJHP0aMxV0ltnds59gS2+yMdyWKN05KmRTw+61XgMyHyFlrxMKe2g== X-Received: by 2002:a05:6a00:ac6:b0:530:3197:48b6 with SMTP id c6-20020a056a000ac600b00530319748b6mr50472097pfl.80.1667824153467; Mon, 07 Nov 2022 04:29:13 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id j7-20020a170902758700b0017f7e0f4a4esm4910492pll.35.2022.11.07.04.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:12 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , Sebastien Buisson , Maloo , Li Dongyang , Andreas Dilger , Ritesh Harjani Subject: [RFCv1 67/72] sec: support encrypted files handling in pfsck mode Date: Mon, 7 Nov 2022 17:51:55 +0530 Message-Id: <77a302b36f3576b9a9f7ef6e42bc1ef939227090.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org From: Sebastien Buisson e2fsck needs to be improved in order to support encrypted files handling in parallel fsck mode. The e2fsck_merge_encrypted_info() function is added to merge encrypted inodes info collected from different threads in pass1, so that it can be used in pass2. Signed-off-by: Sebastien Buisson Tested-by: Maloo Reviewed-by: Li Dongyang Reviewed-by: Andreas Dilger Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/e2fsck.h | 2 + e2fsck/encrypted_files.c | 139 +++++++++++++++++++++++++++++++++++++++ e2fsck/pass1.c | 26 +++++++- 3 files changed, 164 insertions(+), 3 deletions(-) diff --git a/e2fsck/e2fsck.h b/e2fsck/e2fsck.h index 1e82b048..e4fb782a 100644 --- a/e2fsck/e2fsck.h +++ b/e2fsck/e2fsck.h @@ -642,6 +642,8 @@ __u32 find_encryption_policy(e2fsck_t ctx, ext2_ino_t ino); void destroy_encryption_policy_map(e2fsck_t ctx); void destroy_encrypted_file_info(e2fsck_t ctx); +int e2fsck_merge_encrypted_info(e2fsck_t ctx, struct encrypted_file_info *src, + struct encrypted_file_info *dest); /* extents.c */ errcode_t e2fsck_rebuild_extents_later(e2fsck_t ctx, ext2_ino_t ino); diff --git a/e2fsck/encrypted_files.c b/e2fsck/encrypted_files.c index 16be2d6d..53e03a62 100644 --- a/e2fsck/encrypted_files.c +++ b/e2fsck/encrypted_files.c @@ -456,3 +456,142 @@ void destroy_encrypted_file_info(e2fsck_t ctx) ctx->encrypted_files = NULL; } } + +/** + * Search policy matching @policy in @info->policies + * @ctx: e2fsck context + * @info: encrypted_file_info to look into + * @policy: the policy we are looking for + * @parent: (out) last known parent, useful to insert a new leaf + * in @info->policies + * + * Return: id of found policy on success, -1 if no matching policy found. + */ +static inline int search_policy(e2fsck_t ctx, struct encrypted_file_info *info, + union fscrypt_policy policy, + struct rb_node **parent) +{ + struct rb_node *n = info->policies.rb_node; + struct policy_map_entry *entry; + + while (n) { + int res; + + *parent = n; + entry = ext2fs_rb_entry(n, struct policy_map_entry, node); + res = cmp_fscrypt_policies(ctx, &policy, &entry->policy); + if (res < 0) + n = n->rb_left; + else if (res > 0) + n = n->rb_right; + else + return entry->policy_id; + } + return -1; +} + +/* + * Merge @src encrypted info into @dest + */ +int e2fsck_merge_encrypted_info(e2fsck_t ctx, struct encrypted_file_info *src, + struct encrypted_file_info *dest) +{ + struct rb_root *src_policies = &src->policies; + __u32 *policy_trans; + int i, rc = 0; + + if (dest->file_ranges[src->file_ranges_count - 1].last_ino > + src->file_ranges[0].first_ino) { + /* Should never get here */ + fatal_error(ctx, "Encrypted inodes processed out of order"); + } + + rc = ext2fs_get_array(src->next_policy_id, sizeof(__u32), + &policy_trans); + if (rc) + return rc; + + /* First, deal with the encryption policy => ID map. + * Compare encryption policies in src with policies already recorded + * in dest. It can be similar policies, but recorded with a different + * id, so policy_trans array converts policy ids in src to ids in dest. + * This loop examines each policy in src->policies rb tree, updates + * policy_trans, and removes the entry from src, so that src->policies + * rb tree is cleaned up at the end of the loop. + */ + while (!ext2fs_rb_empty_root(src_policies)) { + struct policy_map_entry *entry, *newentry; + struct rb_node *new, *parent = NULL; + int existing_polid; + + entry = ext2fs_rb_entry(src_policies->rb_node, + struct policy_map_entry, node); + existing_polid = search_policy(ctx, dest, + entry->policy, &parent); + if (existing_polid >= 0) { + /* The policy in src is already recorded in dest, + * so just update its id. + */ + policy_trans[entry->policy_id] = existing_polid; + } else { + /* The policy in src is new to dest, so insert it + * with the next available id (its original id could + * be already used in dest). + */ + rc = ext2fs_get_mem(sizeof(*newentry), &newentry); + if (rc) + goto out_merge; + newentry->policy_id = dest->next_policy_id++; + newentry->policy = entry->policy; + ext2fs_rb_link_node(&newentry->node, parent, &new); + ext2fs_rb_insert_color(&newentry->node, + &dest->policies); + policy_trans[entry->policy_id] = newentry->policy_id; + } + ext2fs_rb_erase(&entry->node, src_policies); + ext2fs_free_mem(&entry); + } + + /* Second, deal with the inode number => encryption policy ID map. */ + if (dest->file_ranges_capacity < + dest->file_ranges_count + src->file_ranges_count) { + /* dest->file_ranges is too short, increase its capacity. */ + size_t new_capacity = dest->file_ranges_count + + src->file_ranges_count; + + /* Make sure we at least double the capacity. */ + if (new_capacity < (dest->file_ranges_capacity * 2)) + new_capacity = dest->file_ranges_capacity * 2; + + /* We won't need more than the filesystem's inode count. */ + if (new_capacity > ctx->fs->super->s_inodes_count) + new_capacity = ctx->fs->super->s_inodes_count; + + rc = ext2fs_resize_mem(dest->file_ranges_capacity * + sizeof(struct encrypted_file_range), + new_capacity * + sizeof(struct encrypted_file_range), + &dest->file_ranges); + if (rc) { + fix_problem(ctx, PR_1_ALLOCATE_ENCRYPTED_INODE_LIST, + NULL); + /* Should never get here */ + ctx->flags |= E2F_FLAG_ABORT; + goto out_merge; + } + + dest->file_ranges_capacity = new_capacity; + } + /* Copy file ranges from src to dest. */ + for (i = 0; i < src->file_ranges_count; i++) { + /* Make sure to convert policy ids in src. */ + src->file_ranges[i].policy_id = + policy_trans[src->file_ranges[i].policy_id]; + dest->file_ranges[dest->file_ranges_count++] = + src->file_ranges[i]; + } + +out_merge: + ext2fs_free_mem(&policy_trans); + return rc; +} diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 7345c96d..e7dc017c 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2411,9 +2411,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) ctx->ea_block_quota_inodes = 0; } - /* We don't need the encryption policy => ID map any more */ - destroy_encryption_policy_map(ctx); - if (ctx->flags & E2F_FLAG_RESTART) { /* * Only the master copy of the superblock and block @@ -2703,6 +2700,23 @@ static void e2fsck_pass1_merge_dx_dir(e2fsck_t global_ctx, e2fsck_t thread_ctx) e2fsck_merge_dx_dir(global_ctx, thread_ctx); } +static int e2fsck_pass1_merge_encrypted_info(e2fsck_t global_ctx, + e2fsck_t thread_ctx) +{ + if (thread_ctx->encrypted_files == NULL) + return 0; + + if (global_ctx->encrypted_files == NULL) { + global_ctx->encrypted_files = thread_ctx->encrypted_files; + thread_ctx->encrypted_files = NULL; + return 0; + } + + return e2fsck_merge_encrypted_info(global_ctx, + thread_ctx->encrypted_files, + global_ctx->encrypted_files); +} + static inline errcode_t e2fsck_pass1_merge_icount(ext2_icount_t *dest_icount, ext2_icount_t *src_icount) @@ -2963,6 +2977,12 @@ static errcode_t e2fsck_pass1_merge_context(e2fsck_t global_ctx, e2fsck_pass1_merge_dir_info(global_ctx, thread_ctx); e2fsck_pass1_merge_dx_dir(global_ctx, thread_ctx); + retval = e2fsck_pass1_merge_encrypted_info(global_ctx, thread_ctx); + if (retval) { + com_err(global_ctx->program_name, 0, + _("while merging encrypted info\n")); + return retval; + } retval = ext2fs_merge_fs(&(thread_ctx->fs)); if (retval) { From patchwork Mon Nov 7 12:21:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700616 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=WMVM5cbr; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvN0d8xz1yqS for ; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvN09jVz4xwy for ; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvN079dz4xx0; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=WMVM5cbr; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvN03Jbz4xwy for ; Mon, 7 Nov 2022 23:30:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232301AbiKGMax (ORCPT ); Mon, 7 Nov 2022 07:30:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232252AbiKGM3U (ORCPT ); Mon, 7 Nov 2022 07:29:20 -0500 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4F9314D05 for ; Mon, 7 Nov 2022 04:29:19 -0800 (PST) Received: by mail-pj1-x1029.google.com with SMTP id v4-20020a17090a088400b00212cb0ed97eso10125347pjc.5 for ; Mon, 07 Nov 2022 04:29:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C7sm+0ZjlPeHWK2QGLjZh2+Qy84xOiJKbExlK4PBZVE=; b=WMVM5cbrGLw1MAqYZ6qhcs4gxabPs1mpZzeqsupExnKZ+WxBeYt+LlpiEvebCijQTE QmOPcpF23UNE88IH943l06T/d+Rb9/XAKkewdABPEi4T2QOE5nNKlvqJx5aWclkcYRLO 8vL3JR3j6c8HcD1P2YU8baXleuMxFgYsuAfVcRiyd+d+QNn98hjVZ7nADOkAPYb3P1h9 m1SFnU62e4Qg4aP3OuiJqoMrUWdqxcqfdUi+9Zo4Jte4a3A89yQbVp956nFBPRlqmzMI t35Y54MQc0M3gBrewf27NCs/bXR42ogR3TKFHU+y50a8D8W3tM/qHQPO4MjS28M/gvW4 PIfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C7sm+0ZjlPeHWK2QGLjZh2+Qy84xOiJKbExlK4PBZVE=; b=WiK9mcvFrzaUdwMbq2akeznQp6s1EBjmJHM+npijbNA8Tdm1bI+zDsjvUsFZOt1YMu YdJGIcVbc0QlnU3/+3gGWWJg/gYq5V4rnnSodQUdTO9FRoJ/9uyfIcZt4o7htznqZi71 1DD3hus3yYa/b3vl8qKpiNGmdzTO5TqEyyNnze/rUCFJglnG9hK3HqMlUFPo09BTGCeB D41PZQkBkJSsBuBtHhs4LxEnXIdKGigA001WuAs61aIKXbDTDL6oPgbRkNom3kdPZciy sPxr08095cQfC1nM0z1YMImTbXtbQC+dqFmcG2aVZ6nMp0qhFAVrhionQmWGLn18YeCK BqxA== X-Gm-Message-State: ACrzQf3EP2J2bGhfCOXI3SiNaoSGC1CsY+Xa6i1hCXrkMY102g940+Yg /ShHkBMR2C3Lw/nFeRfNURA= X-Google-Smtp-Source: AMsMyM7pRlAP46P6LBusLV4ML6Usf71ylzCdVMFEWbJlfbJLacmnSOAKFJR04jtelaRTT2HUD2pX8g== X-Received: by 2002:a17:902:e5cc:b0:187:2b02:969d with SMTP id u12-20020a170902e5cc00b001872b02969dmr37862751plf.9.1667824159419; Mon, 07 Nov 2022 04:29:19 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id q4-20020a17090311c400b00186a1b243basm1526027plh.226.2022.11.07.04.29.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:18 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 68/72] e2fsck: Fix io->align assert check Date: Mon, 7 Nov 2022 17:51:56 +0530 Message-Id: <3d088e682f655c3886e0c4c7f278a43b01a9b3f7.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org dest->io->align might still get set later in case of non-aligned read done by any thread in raw_read_blk(). Hence remove this assert check. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 1 - 1 file changed, 1 deletion(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index e7dc017c..c934b021 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2514,7 +2514,6 @@ static errcode_t e2fsck_open_channel_fs(ext2_filsys dest, e2fsck_t dest_context, assert(dest->io->flags == src->io->flags); assert(dest->io->app_data == dest); assert(src->io->app_data == src); - assert(dest->io->align == src->io->align); dest->priv_data = dest_context; dest_context->fs = dest; From patchwork Mon Nov 7 12:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700615 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ulyyhmy6; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvM17Dkz1yqS for ; Mon, 7 Nov 2022 23:30:55 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvM0hPhz4xGT for ; Mon, 7 Nov 2022 23:30:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvM0fb1z4xGj; Mon, 7 Nov 2022 23:30:55 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ulyyhmy6; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvM0ZQGz4xGT for ; Mon, 7 Nov 2022 23:30:55 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231831AbiKGMau (ORCPT ); Mon, 7 Nov 2022 07:30:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232208AbiKGM31 (ORCPT ); Mon, 7 Nov 2022 07:29:27 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B740C1B7B9 for ; Mon, 7 Nov 2022 04:29:25 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id q9so10457934pfg.5 for ; Mon, 07 Nov 2022 04:29:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vm7hA39O7IoRiEPnNzVpo9eYZrFR9MuwdVKpH1WkXyw=; b=Ulyyhmy61IqcJV0xQeuXsHBBbmB5rT0zFmpEj5neoUwo1QrP1JSfZSKScdyVdqX2rO MER6aSikvY8emKy+JgdvgbSVN5G1iiV1lpuByw2mGJT/HhyZ+hB/tu0VmM9qW2tnoBXp FZ/0qiqIvFq7vkALaq7o+NJqmLbPHECzPUun9KrLC0FgB6lYSskm8RmFhfOSqNfx5V25 m0cpWReR3H+ISjSav9aeFXJPsfMc2dSsRw/jVuJ7YERM4HrIMh/GvUveGmzjNJWoOKrM TOKNeUIUZTUHHs+QGBUyTSkvvKti/kWtFRbDrfZrmS0RCSjwxlYuhcaJkvUvBf0Wo/zb Rm8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vm7hA39O7IoRiEPnNzVpo9eYZrFR9MuwdVKpH1WkXyw=; b=iv21G46GL4ilwM1VrZgvRHwV1bfuq5xw1BEhP5hpsG+zZsugToMTuZ8BY2TSruQZ3d gVHnzsTJAwog+u8ZPC+uyVRFmSsZfq3dc0jLb9n9SAx2/cz4cZkW5KwXYMw5NB6J6Brj EZRuT9ANfAMpqF7+otuH7iYznD1KS/3BUyTcWJkJLetSciZNX4TwcwCk3EQBVWwJRH3m NnQNbIPT5wxAM2iHFL11/eZVRTXdPwHl9RURfE3A4zNf3QnT4XVR8LULq6iKdvj0w9X4 kAxWoAm51rFqBOVebj+Rqup5SSlfeH5sXkuYkXE5+UvovTf3/oKlvLejcjthmvUN0jT2 TSYA== X-Gm-Message-State: ACrzQf1nLCjKXXTcRsGE2IZWI/z61bgO5IBu9WBnnBvPgTEa07wBsADr 32Jr7CoNzdd30Y1oESApdP8= X-Google-Smtp-Source: AMsMyM7kWCiHOcXqct8qiDs9SRPSdYiTmnELvWrpxl52VJnuYlIvZiD8tvZZqks03BFHEhilieWYKQ== X-Received: by 2002:a63:581d:0:b0:42b:399:f15a with SMTP id m29-20020a63581d000000b0042b0399f15amr43135266pgb.337.1667824165272; Mon, 07 Nov 2022 04:29:25 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id m2-20020a170902768200b0017f72a430adsm4858722pll.71.2022.11.07.04.29.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:24 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 69/72] e2fsck: Fix double free of inodes_to_process Date: Mon, 7 Nov 2022 17:51:57 +0530 Message-Id: <0de5025b40eb8f462d2f01321432ac0535345e3d.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Found during code review, this fixes the double free of inodes_to_process in e2fsck_pass1_run. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 1 - 1 file changed, 1 deletion(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index c934b021..4168a45d 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -2436,7 +2436,6 @@ void e2fsck_pass1_run(e2fsck_t ctx) } ctx->flags |= E2F_FLAG_ALLOC_OK; - ext2fs_free_mem(&inodes_to_process); endit: e2fsck_use_inode_shortcuts(ctx, 0); ext2fs_free_mem(&inodes_to_process); From patchwork Mon Nov 7 12:21:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700617 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Z5a0TsZ6; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5VvN3kBmz23lx for ; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvN3K9Dz4xwy for ; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5VvN3GN1z4xx0; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Z5a0TsZ6; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5VvN3BPyz4xwy for ; Mon, 7 Nov 2022 23:30:56 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232235AbiKGMay (ORCPT ); Mon, 7 Nov 2022 07:30:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53632 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231935AbiKGMag (ORCPT ); Mon, 7 Nov 2022 07:30:36 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 661371B9D2 for ; Mon, 7 Nov 2022 04:29:31 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id v4-20020a17090a088400b00212cb0ed97eso10125795pjc.5 for ; Mon, 07 Nov 2022 04:29:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7mCTw1xNENoPySQjpxJulxZ+cgE3/DtDrhwZ00hou64=; b=Z5a0TsZ6akgVS5InsU+efI070jXFge2KncXT0iqgNNdAvRawfnCumcq52lAe793gWR 81MiIJctZ7WyoIBRzQxsaCdoL+Ct93pXL/qDoBST60CfwPyrPn0zvIWmD87L6Z7ZyUKY em/0x2yaDxmq/1BHUC9ZiMUEjqvi8IPJDisjpE2xBBLdi6HilMcnvSRjNEX07Yi/Wp3p y7KNkG/NOMB+F52TDZFVRoMGz78xEGAnIGdLTs3E5jd0ODvjiqakS0VixeyVrmVrc9fN vvC677Hh1WfYe9tjN4XCUQnoo3Gqjszui7FuVzHwQ43awyTLLDfTuaIure+i+yZ/YVoO 4iTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7mCTw1xNENoPySQjpxJulxZ+cgE3/DtDrhwZ00hou64=; b=7FDKHtWnqYRvDqR0oGPQI9mlGSAdiz3Mpo3G6UPPjb4VCsY083u2DUIgTgBcEbZoe6 pqp1arGLBj3nKS/FvTPgdW5Qnb4Gb3ZstmmXVXu/GVBtG2xxyAXpIYhA4vrnKZ/Q+40H ik9IJ6FGf52nrMpNetl9xcO5t36rdM6yT1xMj9bRf6N+dbEczVnu2v1DPcQZ/flBke5E ZEd9NrVwqKHN1SJAONMrnobPqstIAF++LMDDMYKup3FWAqNWpr8fDrMHEWHWF9D7+Au9 OkMf0QEA3f2k0QjEEWJNG2cJxBMHDvnU1+KGPfT9X7Fec3aSIGZ7hr4DeHnPjwLdiG8u OG+g== X-Gm-Message-State: ACrzQf0yBbg+drtCgrY+LH2jMTVmxP+FyDY7MWTh3tayFwsMSGfzXLuM us+Y346gjbrr08Hh+683mEz6N7f2ZtM= X-Google-Smtp-Source: AMsMyM7vVzHUZVeTbOMwSivZ8sak45p3GRBoovd8VcmGTzUmu068MhJGZ/6A/PSkhP8l0eVa23dUhw== X-Received: by 2002:a17:90a:a415:b0:20a:f813:83a3 with SMTP id y21-20020a17090aa41500b0020af81383a3mr51751288pjp.238.1667824170932; Mon, 07 Nov 2022 04:29:30 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id o22-20020a170902779600b0018668bee7cdsm4877440pll.77.2022.11.07.04.29.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:30 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 70/72] e2fsck: Fix and simplify update_mmp in case of pfsck Date: Mon, 7 Nov 2022 17:51:58 +0530 Message-Id: <846558b5f837ed2d32246a072d3b97f584573c7d.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This adds pass1_update_mmp_enter() & pass1_update_mmp_exit() routines to update mmp block. This also fixes a data race reported by threadsan because of reading and writing to mmp_update_thread variable. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 97 ++++++++++++++++++++++++++++++++------------------ 1 file changed, 63 insertions(+), 34 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 4168a45d..2ff83fcb 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -1097,6 +1097,66 @@ out: return 0; } +static void pass1_update_mmp_enter(e2fsck_t ctx, ext2_ino_t ino) +{ + ext2_filsys fs = ctx->fs; + e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx; + int check_mmp = 0; + int set_mmp = 0; + +#ifdef HAVE_PTHREAD + /* only one active thread could update mmp block. */ + e2fsck_pass1_block_map_r_lock(ctx); + if (!global_ctx->mmp_update_thread) + set_mmp = 1; + if (global_ctx->mmp_update_thread == ctx->thread_info.et_thread_index + 1) + check_mmp = 1; + e2fsck_pass1_block_map_r_unlock(ctx); + + if (!check_mmp && !set_mmp) + return; + + if (set_mmp) { + e2fsck_pass1_block_map_w_lock(ctx); + if (!global_ctx->mmp_update_thread) { + global_ctx->mmp_update_thread = + ctx->thread_info.et_thread_index + 1; + check_mmp = 1; + } + e2fsck_pass1_block_map_w_unlock(ctx); + } +#else + check_mmp = 1; +#endif + + if (check_mmp && (ino % (fs->super->s_inodes_per_group * 4) == 1)) { + if (e2fsck_mmp_update(fs)) + fatal_error(ctx, 0); + } +} + +static void pass1_update_mmp_exit(e2fsck_t ctx) +{ + ext2_filsys fs = ctx->fs; + e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx; + int set_mmp = 0; + +#ifdef HAVE_PTHREAD + e2fsck_pass1_block_map_r_lock(ctx); + if (global_ctx->mmp_update_thread == ctx->thread_info.et_thread_index + 1) + set_mmp = 1; + e2fsck_pass1_block_map_r_unlock(ctx); + + if (!set_mmp) + return; + + /* reset update_thread after this thread exit */ + e2fsck_pass1_block_map_w_lock(ctx); + global_ctx->mmp_update_thread = 0; + e2fsck_pass1_block_map_w_unlock(ctx); +#endif +} + static void pass1_readahead(e2fsck_t ctx, dgrp_t *group, ext2_ino_t *next_ino) { ext2_ino_t inodes_in_group = 0, inodes_per_block, inodes_per_buffer; @@ -1511,7 +1571,7 @@ void e2fsck_pass1_run(e2fsck_t ctx) dgrp_t ra_group = 0; struct ea_quota ea_ibody_quota; struct process_inode_block *inodes_to_process; - int process_inode_count, check_mmp; + int process_inode_count; e2fsck_t global_ctx = ctx->global_ctx ? ctx->global_ctx : ctx; init_resource_track(&rtrack, ctx->fs->io); @@ -1675,33 +1735,8 @@ void e2fsck_pass1_run(e2fsck_t ctx) #endif while (1) { - check_mmp = 0; e2fsck_pass1_check_lock(ctx); -#ifdef HAVE_PTHREAD - if (!global_ctx->mmp_update_thread) { - e2fsck_pass1_block_map_w_lock(ctx); - if (!global_ctx->mmp_update_thread) { - global_ctx->mmp_update_thread = - ctx->thread_info.et_thread_index + 1; - check_mmp = 1; - } - e2fsck_pass1_block_map_w_unlock(ctx); - } - - /* only one active thread could update mmp block. */ - e2fsck_pass1_block_map_r_lock(ctx); - if (global_ctx->mmp_update_thread == - ctx->thread_info.et_thread_index + 1) - check_mmp = 1; - e2fsck_pass1_block_map_r_unlock(ctx); -#else - check_mmp = 1; -#endif - - if (check_mmp && (ino % (fs->super->s_inodes_per_group * 4) == 1)) { - if (e2fsck_mmp_update(fs)) - fatal_error(ctx, 0); - } + pass1_update_mmp_enter(ctx, ino); old_op = ehandler_operation(eop_next_inode); pctx.errcode = ext2fs_get_next_inode_full(scan, &ino, inode, inode_size); @@ -2458,13 +2493,7 @@ endit: print_resource_track(ctx, _("Pass 1"), &rtrack, ctx->fs->io); else ctx->invalid_bitmaps++; -#ifdef HAVE_PTHREAD - /* reset update_thread after this thread exit */ - e2fsck_pass1_block_map_w_lock(ctx); - if (check_mmp) - global_ctx->mmp_update_thread = 0; - e2fsck_pass1_block_map_w_unlock(ctx); -#endif + pass1_update_mmp_exit(ctx); } #ifdef HAVE_PTHREAD From patchwork Mon Nov 7 12:21:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700619 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=AKQtLeQj; dkim-atps=neutral Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vvc6V7rz1yqS for ; Mon, 7 Nov 2022 23:31:08 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vvc64Q5z4xGT for ; Mon, 7 Nov 2022 23:31:08 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vvc627Gz4xGj; Mon, 7 Nov 2022 23:31:08 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=AKQtLeQj; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vvc5vnMz4xGT for ; Mon, 7 Nov 2022 23:31:08 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232334AbiKGMbH (ORCPT ); Mon, 7 Nov 2022 07:31:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232153AbiKGMah (ORCPT ); Mon, 7 Nov 2022 07:30:37 -0500 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EDA91B7AA for ; Mon, 7 Nov 2022 04:29:37 -0800 (PST) Received: by mail-pj1-x1030.google.com with SMTP id m6-20020a17090a5a4600b00212f8dffec9so10172195pji.0 for ; Mon, 07 Nov 2022 04:29:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=brABaUCcI2WmtiC9btp4x4LzbwAmsKEEIExja+PubBU=; b=AKQtLeQj2gU9Cpx6Cgl1I3ItI9fra2tqkX4jvbMRiJfShMzSZNUbaAUatvx26N1kHt jRPTu1tsFWmPGn15HGxgUnS07OiQdDYrBDTnYZtCyRa8FvcstyQl563CaKdNHdzmbkxS fWD7WoqCFQRfPlFiGXyH0WyK9HAaW04+1prO6YGirCtBfFIuiDHfKNZCkO5FLfNbJQI+ 7W8/PkwGcMhYsYx6sFhbe9u7gDtaLZ7C/b5Ero3mAV3aHcQX16W/xnos3NQMyCSpfZAT lCcj1XSa2Fbm4Hg94v0dvtugcOhfSBm07JsOby5VxLrF8i9LIU2AB/Tc2RggZFVy2oHr j/PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=brABaUCcI2WmtiC9btp4x4LzbwAmsKEEIExja+PubBU=; b=6EtvtfQgc+b+qs+yowvQL0MdFZBJTQl3U3MgFPo96SRC2vrYlDdDB3UxmgVth6IBnA z7mm7CvaiJDzAcyqdUyKlGg50OsOCTcMoawYwGx38MlSNEGHFRY9fXJjvXeDWx0OWZSM e9v3aNtaYdya1KfXu0bFT1h7nmlEtq0yRDPt6me/tmprxuWP8hS09b9cLy2q3AEF6OrR 9G+3miE/jVK6Xti5207aPbcvRYwsKmfjYcDwRNFJFpfAwg3w9Uxrq0RktDVR8oLqcd4V d52X5qxLLHpxhd6PxuRE5K5lu/AygKCpnmOVnhFH9ioRgo05WQeHpOTycPRID7bFioBi Am9Q== X-Gm-Message-State: ACrzQf3dqmGNkfJz0KuwilKUER+GXDeIfpuGg//MMdkvn3VSwKnDxKip Kp59VLPdnpTgcGxYYqtSUY0= X-Google-Smtp-Source: AMsMyM5xFatw0U4M8F8g/S4R3nOCTABfOUjCvzOEt0WFP4eF4xORPqgPDZBSdcGT4fRi2+JWVuaRGg== X-Received: by 2002:a17:90a:20c:b0:213:1179:1fff with SMTP id c12-20020a17090a020c00b0021311791fffmr52845952pjc.23.1667824177086; Mon, 07 Nov 2022 04:29:37 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id fy14-20020a17090b020e00b002036006d65bsm4162096pjb.39.2022.11.07.04.29.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:36 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 71/72] e2fsck: Make threads call log_out after pthread_join Date: Mon, 7 Nov 2022 17:51:59 +0530 Message-Id: <6a1aa669f380236663941522f84acf840919aac0.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org All fsck threads will call for log_out prints after the pass1 their respective pass1 scanning is completed. This patch moves the log_out print from to after the pthread_join operation. This makes the threads always print the info in order. Signed-off-by: Ritesh Harjani (IBM) --- e2fsck/pass1.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/e2fsck/pass1.c b/e2fsck/pass1.c index 2ff83fcb..90adc419 100644 --- a/e2fsck/pass1.c +++ b/e2fsck/pass1.c @@ -3112,6 +3112,12 @@ static int e2fsck_pass1_thread_join(e2fsck_t global_ctx, e2fsck_t thread_ctx) { errcode_t retval; + log_out(thread_ctx, + _("Scanned group range [%u, %u), inodes %u\n"), + thread_ctx->thread_info.et_group_start, + thread_ctx->thread_info.et_group_end, + thread_ctx->thread_info.et_inode_number); + retval = e2fsck_pass1_merge_context(global_ctx, thread_ctx); quota_release_context(&thread_ctx->qctx); @@ -3203,13 +3209,6 @@ static void *e2fsck_pass1_thread(void *arg) e2fsck_pass1_run(thread_ctx); out: - if (thread_ctx->options & E2F_OPT_MULTITHREAD) - log_out(thread_ctx, - _("Scanned group range [%u, %u), inodes %u\n"), - thread_ctx->thread_info.et_group_start, - thread_ctx->thread_info.et_group_end, - thread_ctx->thread_info.et_inode_number); - #ifdef DEBUG_THREADS pthread_mutex_lock(&thread_debug->etd_mutex); thread_debug->etd_finished_threads++; From patchwork Mon Nov 7 12:22:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 1700620 X-Patchwork-Delegate: tytso@mit.edu Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=gandalf.ozlabs.org; envelope-from=srs0=bncz=3h=vger.kernel.org=linux-ext4-owner@ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=YLXVudDB; dkim-atps=neutral Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N5Vvl0CbPz1yqS for ; Mon, 7 Nov 2022 23:31:15 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vvk6rDGz4xG8 for ; Mon, 7 Nov 2022 23:31:14 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4N5Vvk6p3Fz4xGT; Mon, 7 Nov 2022 23:31:14 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=YLXVudDB; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4N5Vvk6jnvz4xG8 for ; Mon, 7 Nov 2022 23:31:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232244AbiKGMbO (ORCPT ); Mon, 7 Nov 2022 07:31:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232218AbiKGMah (ORCPT ); Mon, 7 Nov 2022 07:30:37 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B8311B9E0 for ; Mon, 7 Nov 2022 04:29:43 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id 130so10455260pfu.8 for ; Mon, 07 Nov 2022 04:29:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8MjcZCvN/lfLd6qf7J9BhPaXmm8DKAs/HisJf7aYy1o=; b=YLXVudDBSoBEPWhTG+NXiPJs7QotfnYm3TMmhav3GiTIerEbwDAKpG/YLMa6Zfvvsr oMirVA4IcmMgHp0PRlEPgxSdiQwlI+H7L/9kvDIBX8VV5A4sMFCzAGPJHpygfh2uMGy3 EenN49UzDOD0ZHu+foVAXm61YpDQKuqMKQPHlS+i9oyvQ/Kn8AHPjeu1tQgezY+WNrk6 SHcUQPf6c0G+CFN8HcvwWC9lAnaSNLVOJweyXR+8F7blSzgcF705zU84vDHcn/x6pwPz S8YexEkePZiBGBS4Qx7QI/deqYsF4D4GCfXJkmOF1XwW4HRQZKtpj0m9hbcaQx0U4upr 7iZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8MjcZCvN/lfLd6qf7J9BhPaXmm8DKAs/HisJf7aYy1o=; b=khLt7CEkVaYQo09MogPoMcudcAn7MQhpqteaaIMh38rh72/AOWyukZxK6ro+UZh4wS /ivlDIL6LWY47VcgXjNrm/b4kuH3ASUUJmH580F3pAY2odcsfhtaL+j5AknNSWTgaL7S znVotDK/rqyzec65R2X/fn73zhqxoatJv+cSasKazmnTih5jkHTU0vVli2Tkb7Iuc6XH eDka60UQY/n549Ls4LbdwgHmSVewNGYyJje52Bq6jC6iDpQroY36oIp5jSBFkT6eAy83 RnSZUxPuF0sDBV3thN4WUV/zBrCbzXHYS7DXXszx9YxrIzdJ+VOijMU4sljFAhDxKprF jCXA== X-Gm-Message-State: ACrzQf0eFS9E/Z2vDChp+I00wp0PanSH5QAUFyv+yEvpL8tF9Tz/HVQV t/VVPQohaTcow3bjxouHz10= X-Google-Smtp-Source: AMsMyM5rckrwUV2UL3jhtaAchM2AD/H5Zj7Tl92BOVw4QWNP4IqqmrsRZYV6nux4NJzkt4mGBmbLWg== X-Received: by 2002:a63:e045:0:b0:46f:e244:3136 with SMTP id n5-20020a63e045000000b0046fe2443136mr31109160pgj.95.1667824183080; Mon, 07 Nov 2022 04:29:43 -0800 (PST) Received: from localhost ([2406:7400:63:f20b:312d:45b2:85c1:c486]) by smtp.gmail.com with ESMTPSA id a3-20020a170902710300b0017534ffd491sm4925875pll.163.2022.11.07.04.29.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Nov 2022 04:29:42 -0800 (PST) From: "Ritesh Harjani (IBM)" To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Harshad Shirwadkar , Wang Shilong , Andreas Dilger , Li Xi , "Ritesh Harjani (IBM)" Subject: [RFCv1 72/72] tests/f_multithread: Fix f_multithread related tests Date: Mon, 7 Nov 2022 17:52:00 +0530 Message-Id: <9757f60b44fa61367923b7094d91abfad40dbd4d.1667822612.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org With log_out() function now changed to print the message after the pthread join operation, it is safe to also add "Scan group range" related messages in expect files to compare against the pfsck output. Signed-off-by: Ritesh Harjani (IBM) --- tests/f_multithread_ok/expect.1 | 8 ++++++++ tests/f_multithread_ok/script | 17 ----------------- 2 files changed, 8 insertions(+), 17 deletions(-) diff --git a/tests/f_multithread_ok/expect.1 b/tests/f_multithread_ok/expect.1 index 4742f408..cecc11db 100644 --- a/tests/f_multithread_ok/expect.1 +++ b/tests/f_multithread_ok/expect.1 @@ -1,4 +1,12 @@ Pass 1: Checking inodes, blocks, and sizes +[Thread 0] Scan group range [0, 1) +[Thread 1] Scan group range [1, 2) +[Thread 2] Scan group range [2, 3) +[Thread 3] Scan group range [3, 4) +[Thread 0] Scanned group range [0, 1), inodes 8192 +[Thread 1] Scanned group range [1, 2), inodes 8192 +[Thread 2] Scanned group range [2, 3), inodes 8192 +[Thread 3] Scanned group range [3, 4), inodes 8192 Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts diff --git a/tests/f_multithread_ok/script b/tests/f_multithread_ok/script index 7334cde6..f14034cf 100644 --- a/tests/f_multithread_ok/script +++ b/tests/f_multithread_ok/script @@ -1,21 +1,4 @@ FSCK_OPT="-fym4" -SKIP_VERIFY="true" ONE_PASS_ONLY="true" -SKIP_CLEANUP="true" . $cmd_dir/run_e2fsck - -grep -v Thread $OUT1 > $OUT1.tmp -cmp -s $EXP1 $OUT1.tmp -status1=$? -if [ "$status1" -eq 0 ]; then - echo "$test_name: $test_description: ok" - touch $test_name.ok -else - echo "$test_name: $test_description: failed" - diff $DIFF_OPTS $EXP1 $OUT1.tmp > $test_name.failed -fi - -unset IMAGE FSCK_OPT SECOND_FSCK_OPT OUT1 OUT2 EXP1 EXP2 -unset SKIP_VERIFY SKIP_CLEANUP SKIP_GUNZIP ONE_PASS_ONLY PREP_CMD -unset DESCRIPTION SKIP_UNLINK AFTER_CMD PASS_ZERO