From patchwork Thu May 6 04:04:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474769 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKjS1S0Wz9sV5; Thu, 6 May 2021 14:04:59 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVFw-00053N-Mc; Thu, 06 May 2021 04:04:56 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVFv-000533-4X for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:04:55 +0000 Received: from mail-pg1-f198.google.com ([209.85.215.198]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVFu-0005OV-Sj for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:04:54 +0000 Received: by mail-pg1-f198.google.com with SMTP id z19-20020a63e1130000b02901fcdcf0c5a3so2504073pgh.19 for ; Wed, 05 May 2021 21:04:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T5q23CIUjr6+9H0ew4bphhIE3c1YhyF6EH4kP29UHkU=; b=Raxuh5Gv9ifinY5ZPrcWqhKFJHTIIYpOwSoiYBfrJY+y6vGU1BHU/h6/q/RtJiD4q/ GcN/Tbfnf7fn8vPukTj4lDhK8pgL2JqbJI3H7q6LLmlGTxFr9VieDv8WpoaH1440Mg9G LaQ05ZBnQd9bZ8Zy4P9d4SGvEjaxYqNOMQSuoPm9sDH6OZebx1nlzJrap4xOSHiX7VHO KeIuk6kwwmZfjY7l3BE2HPmiuO9kP8c0vxB8R9ROvr6gVRhT4bwqEo4RhL+PleaNC3U8 jJUm/QNv3Ww40ogUSZx0hCmN2OSc3YJSA64eHV7gKBC2G+SNb89IifLbs61CUQUnYXEZ uCJw== X-Gm-Message-State: AOAM5300E8NXBzyYIFUw10ZAxh0YYFPSQ8TOrTws4pVSgYsajRDlgaS5 zsWHdNQ7tYrOmNh1HLtLgsLekYXxtpDCFT4kaRt4s55Ln2+2Mkw7Qg1OfkPr1ApBynCjijDJLhU fZRKeVTYK6jZsP4Q/zosgl+1qyTyQetUKaF/ViAit8w== X-Received: by 2002:aa7:842a:0:b029:28d:f82e:a4ba with SMTP id q10-20020aa7842a0000b029028df82ea4bamr2417509pfn.33.1620273893626; Wed, 05 May 2021 21:04:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxYlkZdPPoNw3Ae13ngIH0NM0+yLmmUtFqoS9R/1uRaxHaCscfRshn0hYapELEgQUqf+tdl2A== X-Received: by 2002:aa7:842a:0:b029:28d:f82e:a4ba with SMTP id q10-20020aa7842a0000b029028df82ea4bamr2417479pfn.33.1620273893318; Wed, 05 May 2021 21:04:53 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.04.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:04:52 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 1/6] md: add md_submit_discard_bio() for submitting discard bio Date: Thu, 6 May 2021 16:04:32 +1200 Message-Id: <20210506040442.10877-2-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Xiao Ni BugLink: https://bugs.launchpad.net/bugs/1896578 Move these logic from raid0.c to md.c, so that we can also use it in raid10.c. Reviewed-by: Coly Li Reviewed-by: Guoqing Jiang Tested-by: Adrian Huang Signed-off-by: Xiao Ni Signed-off-by: Song Liu (cherry picked from commit cf78408f937a67f59f5e90ee8e6cadeed7c128a8) Signed-off-by: Matthew Ruffell --- drivers/md/md.c | 20 ++++++++++++++++++++ drivers/md/md.h | 2 ++ drivers/md/raid0.c | 14 ++------------ 3 files changed, 24 insertions(+), 12 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 04384452a7ab..476c22067798 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -8590,6 +8590,26 @@ void md_write_end(struct mddev *mddev) EXPORT_SYMBOL(md_write_end); +/* This is used by raid0 and raid10 */ +void md_submit_discard_bio(struct mddev *mddev, struct md_rdev *rdev, + struct bio *bio, sector_t start, sector_t size) +{ + struct bio *discard_bio = NULL; + + if (__blkdev_issue_discard(rdev->bdev, start, size, GFP_NOIO, 0, + &discard_bio) || !discard_bio) + return; + + bio_chain(discard_bio, bio); + bio_clone_blkg_association(discard_bio, bio); + if (mddev->gendisk) + trace_block_bio_remap(discard_bio, + disk_devt(mddev->gendisk), + bio->bi_iter.bi_sector); + submit_bio_noacct(discard_bio); +} +EXPORT_SYMBOL_GPL(md_submit_discard_bio); + /* md_allow_write(mddev) * Calling this ensures that the array is marked 'active' so that writes * may proceed without blocking. It is important to call this before diff --git a/drivers/md/md.h b/drivers/md/md.h index 34070ab30a8a..af1f78c5eca0 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -713,6 +713,8 @@ extern void md_write_end(struct mddev *mddev); extern void md_done_sync(struct mddev *mddev, int blocks, int ok); extern void md_error(struct mddev *mddev, struct md_rdev *rdev); extern void md_finish_reshape(struct mddev *mddev); +void md_submit_discard_bio(struct mddev *mddev, struct md_rdev *rdev, + struct bio *bio, sector_t start, sector_t size); extern bool __must_check md_flush_request(struct mddev *mddev, struct bio *bio); extern void md_super_write(struct mddev *mddev, struct md_rdev *rdev, diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c index fdada0bf2217..cbc3fbe7b54a 100644 --- a/drivers/md/raid0.c +++ b/drivers/md/raid0.c @@ -478,7 +478,6 @@ static void raid0_handle_discard(struct mddev *mddev, struct bio *bio) for (disk = 0; disk < zone->nb_dev; disk++) { sector_t dev_start, dev_end; - struct bio *discard_bio = NULL; struct md_rdev *rdev; if (disk < start_disk_index) @@ -501,18 +500,9 @@ static void raid0_handle_discard(struct mddev *mddev, struct bio *bio) rdev = conf->devlist[(zone - conf->strip_zone) * conf->strip_zone[0].nb_dev + disk]; - if (__blkdev_issue_discard(rdev->bdev, + md_submit_discard_bio(mddev, rdev, bio, dev_start + zone->dev_start + rdev->data_offset, - dev_end - dev_start, GFP_NOIO, 0, &discard_bio) || - !discard_bio) - continue; - bio_chain(discard_bio, bio); - bio_clone_blkg_association(discard_bio, bio); - if (mddev->gendisk) - trace_block_bio_remap(discard_bio, - disk_devt(mddev->gendisk), - bio->bi_iter.bi_sector); - submit_bio_noacct(discard_bio); + dev_end - dev_start); } bio_endio(bio); } From patchwork Thu May 6 04:04:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474773 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKjc5fgcz9sV5; Thu, 6 May 2021 14:05:08 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVG5-00058i-4e; Thu, 06 May 2021 04:05:05 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVG0-00055D-EH for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:00 +0000 Received: from mail-pf1-f199.google.com ([209.85.210.199]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVG0-0005P0-5n for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:00 +0000 Received: by mail-pf1-f199.google.com with SMTP id a4-20020a056a000c84b029025d788a548fso2738959pfv.8 for ; Wed, 05 May 2021 21:05:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=do+sFb69NLOf1Gq4yJz4LBF6uq3/seRT5Qas0YrU5/c=; b=Z5gt4SFduLaAirjlBkA9pvw1hYgCx38UEoluGKV55OlLWNqaOSzsz2w2uLHGQG61dz P48Uqh24piVOP5kQTZJR1Fas/NviGtnmf7OK6LTe76i2BkPhFOeqCDS3dk70+F/QXADi 6BIX1pKjl2Yipp2vwkHpN48URKdZlderuoJuN6/cqwyeBGSrZ84ckRvhw5eKLupdol7/ xH6yRdZAcsCLYZ/sSaUIHshe9DqwQ+osnBm3JPksRWDyu4hwtTD1kCajg97fdtoWqLp6 3aBMMPtV+rXXAcgv0ca8orC4H6M7oxXp4cNy2I0ajUqH6Q0JzmWbROtDf/6Rsn9o/yT8 JlWw== X-Gm-Message-State: AOAM533V1v0N5RZHKOrdN/eozIX9sltsMJvk1etLmoru1TXVT8+oTICu todjTX1jvww0lH8xMBIz3aeEW5yZzf7O8T1+iBZrhc9GELx/gRZz/LUwma/U8q3yr85YjZqcY1c 05ay2Viq2IJKh2iUXNKdsmGgov1sFBcMshUcoOYlsTQ== X-Received: by 2002:a63:e514:: with SMTP id r20mr2195685pgh.423.1620273898888; Wed, 05 May 2021 21:04:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7uHqaUMqxPSIbSfLFx1ZoRIZycrsT1A76w5Cp7ENTys7JDI218jPxIQiE3M84XK8kI3iddQ== X-Received: by 2002:a63:e514:: with SMTP id r20mr2195672pgh.423.1620273898667; Wed, 05 May 2021 21:04:58 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.04.57 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:04:58 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][B][F][G][H][PATCH 2/6] md/raid10: extend r10bio devs to raid disks Date: Thu, 6 May 2021 16:04:35 +1200 Message-Id: <20210506040442.10877-5-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Xiao Ni BugLink: https://bugs.launchpad.net/bugs/1896578 Now it allocs r10bio->devs[conf->copies]. Discard bio needs to submit to all member disks and it needs to use r10bio. So extend to r10bio->devs[geo.raid_disks]. Reviewed-by: Coly Li Tested-by: Adrian Huang Signed-off-by: Xiao Ni Signed-off-by: Song Liu (cherry picked from commit c2968285925adb97b9aa4ede94c1f1ab61ce0925) Signed-off-by: Matthew Ruffell --- drivers/md/raid10.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index c5d88ef6a45c..8b132ae8279c 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -91,7 +91,7 @@ static inline struct r10bio *get_resync_r10bio(struct bio *bio) static void * r10bio_pool_alloc(gfp_t gfp_flags, void *data) { struct r10conf *conf = data; - int size = offsetof(struct r10bio, devs[conf->copies]); + int size = offsetof(struct r10bio, devs[conf->geo.raid_disks]); /* allocate a r10bio with room for raid_disks entries in the * bios array */ @@ -238,7 +238,7 @@ static void put_all_bios(struct r10conf *conf, struct r10bio *r10_bio) { int i; - for (i = 0; i < conf->copies; i++) { + for (i = 0; i < conf->geo.raid_disks; i++) { struct bio **bio = & r10_bio->devs[i].bio; if (!BIO_SPECIAL(*bio)) bio_put(*bio); @@ -327,7 +327,7 @@ static int find_bio_disk(struct r10conf *conf, struct r10bio *r10_bio, int slot; int repl = 0; - for (slot = 0; slot < conf->copies; slot++) { + for (slot = 0; slot < conf->geo.raid_disks; slot++) { if (r10_bio->devs[slot].bio == bio) break; if (r10_bio->devs[slot].repl_bio == bio) { @@ -336,7 +336,6 @@ static int find_bio_disk(struct r10conf *conf, struct r10bio *r10_bio, } } - BUG_ON(slot == conf->copies); update_head_pos(slot, r10_bio); if (slotp) @@ -1492,7 +1491,8 @@ static void __make_request(struct mddev *mddev, struct bio *bio, int sectors) r10_bio->sector = bio->bi_iter.bi_sector; r10_bio->state = 0; r10_bio->read_slot = -1; - memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * conf->copies); + memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * + conf->geo.raid_disks); if (bio_data_dir(bio) == READ) raid10_read_request(mddev, bio, r10_bio); From patchwork Thu May 6 04:04:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474775 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKjm4y3Gz9sTD; Thu, 6 May 2021 14:05:16 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVGB-0005Df-NQ; Thu, 06 May 2021 04:05:11 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVG3-000578-Mq for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:03 +0000 Received: from mail-pl1-f199.google.com ([209.85.214.199]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVG1-0005PB-T2 for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:01 +0000 Received: by mail-pl1-f199.google.com with SMTP id t12-20020a170902dcccb02900ed4648d0f9so1730089pll.2 for ; Wed, 05 May 2021 21:05:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=utpFvmNa3ERUOOdbji85xyTxxTqOqKigItYTtiV2cp4=; b=O8qffi+iC2ngS8nj+90mGdqI22JNcyvkgw0Dk+cc3Q8/WScvJNHaaIpkgqYXsRx/VT sf7l4vCmRHeCSg0swtTVu8KyDiV/2Wc/Vt0b9IIihOXdISNUcdvNpa60kYjSgJVpe7bT ikbonC8LAkDZOcj8+2uqtrJl3xKGpIUYAjTYLmUecsarmvK+PLo6AktCd4jpB5QMllqq WwJjzuNCBOiGDWqAsiOIvp7LRbKgMIohhTgNrhnchT+t9Nm684WfFQFs348G56NT5EnP g3uK82PevHofymtP1i64tdAU3nByKyEV2L+dw0SheEYxZSXoyyHXgAVHpJjBS+ikH/vn tmOw== X-Gm-Message-State: AOAM530GZUw9YbzHzGqgoVmEHHQ++WDr3c7RrXbvERz9BUgC2OZzAOJS PEci0FJ/JSp6Oyt2+zcZbUD3iEz2x7TFAwIif8dbjCWRz0KsmiNXHAGfzkLyMDrzHKAY5L+mQBE wxMhQ2KIsEE1pBigSxzhnFVtZmLp4q74j+cRVXCLrRQ== X-Received: by 2002:a62:5209:0:b029:278:648f:99b6 with SMTP id g9-20020a6252090000b0290278648f99b6mr2339633pfb.9.1620273900621; Wed, 05 May 2021 21:05:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx3RY9pfYzq1Ph/rBjiZB8bOAQAU8FITD6E5mNW8GAzI6fpnmmWDaUYmzzLLIs0CI27HuT69A== X-Received: by 2002:a62:5209:0:b029:278:648f:99b6 with SMTP id g9-20020a6252090000b0290278648f99b6mr2339623pfb.9.1620273900400; Wed, 05 May 2021 21:05:00 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.04.59 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:05:00 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][B][F][G][H][PATCH 3/6] md/raid10: pull the code that wait for blocked dev into one function Date: Thu, 6 May 2021 16:04:36 +1200 Message-Id: <20210506040442.10877-6-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Xiao Ni BugLink: https://bugs.launchpad.net/bugs/1896578 The following patch will reuse these logics, so pull the same codes into one function. Tested-by: Adrian Huang Signed-off-by: Xiao Ni Signed-off-by: Song Liu (cherry picked from commit f2e7e269a7525317752d472bb48a549780e87d22) Signed-off-by: Matthew Ruffell --- drivers/md/raid10.c | 120 +++++++++++++++++++++++++------------------- 1 file changed, 69 insertions(+), 51 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 8b132ae8279c..789f575503de 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -1273,12 +1273,77 @@ static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio, } } +static void wait_blocked_dev(struct mddev *mddev, struct r10bio *r10_bio) +{ + int i; + struct r10conf *conf = mddev->private; + struct md_rdev *blocked_rdev; + +retry_wait: + blocked_rdev = NULL; + rcu_read_lock(); + for (i = 0; i < conf->copies; i++) { + struct md_rdev *rdev = rcu_dereference(conf->mirrors[i].rdev); + struct md_rdev *rrdev = rcu_dereference( + conf->mirrors[i].replacement); + if (rdev == rrdev) + rrdev = NULL; + if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { + atomic_inc(&rdev->nr_pending); + blocked_rdev = rdev; + break; + } + if (rrdev && unlikely(test_bit(Blocked, &rrdev->flags))) { + atomic_inc(&rrdev->nr_pending); + blocked_rdev = rrdev; + break; + } + + if (rdev && test_bit(WriteErrorSeen, &rdev->flags)) { + sector_t first_bad; + sector_t dev_sector = r10_bio->devs[i].addr; + int bad_sectors; + int is_bad; + + /* + * Discard request doesn't care the write result + * so it doesn't need to wait blocked disk here. + */ + if (!r10_bio->sectors) + continue; + + is_bad = is_badblock(rdev, dev_sector, r10_bio->sectors, + &first_bad, &bad_sectors); + if (is_bad < 0) { + /* + * Mustn't write here until the bad block + * is acknowledged + */ + atomic_inc(&rdev->nr_pending); + set_bit(BlockedBadBlocks, &rdev->flags); + blocked_rdev = rdev; + break; + } + } + } + rcu_read_unlock(); + + if (unlikely(blocked_rdev)) { + /* Have to wait for this device to get unblocked, then retry */ + allow_barrier(conf); + raid10_log(conf->mddev, "%s wait rdev %d blocked", + __func__, blocked_rdev->raid_disk); + md_wait_for_blocked_rdev(blocked_rdev, mddev); + wait_barrier(conf); + goto retry_wait; + } +} + static void raid10_write_request(struct mddev *mddev, struct bio *bio, struct r10bio *r10_bio) { struct r10conf *conf = mddev->private; int i; - struct md_rdev *blocked_rdev; sector_t sectors; int max_sectors; @@ -1336,8 +1401,9 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, r10_bio->read_slot = -1; /* make sure repl_bio gets freed */ raid10_find_phys(conf, r10_bio); -retry_write: - blocked_rdev = NULL; + + wait_blocked_dev(mddev, r10_bio); + rcu_read_lock(); max_sectors = r10_bio->sectors; @@ -1348,16 +1414,6 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, conf->mirrors[d].replacement); if (rdev == rrdev) rrdev = NULL; - if (rdev && unlikely(test_bit(Blocked, &rdev->flags))) { - atomic_inc(&rdev->nr_pending); - blocked_rdev = rdev; - break; - } - if (rrdev && unlikely(test_bit(Blocked, &rrdev->flags))) { - atomic_inc(&rrdev->nr_pending); - blocked_rdev = rrdev; - break; - } if (rdev && (test_bit(Faulty, &rdev->flags))) rdev = NULL; if (rrdev && (test_bit(Faulty, &rrdev->flags))) @@ -1378,15 +1434,6 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, is_bad = is_badblock(rdev, dev_sector, max_sectors, &first_bad, &bad_sectors); - if (is_bad < 0) { - /* Mustn't write here until the bad block - * is acknowledged - */ - atomic_inc(&rdev->nr_pending); - set_bit(BlockedBadBlocks, &rdev->flags); - blocked_rdev = rdev; - break; - } if (is_bad && first_bad <= dev_sector) { /* Cannot write here at all */ bad_sectors -= (dev_sector - first_bad); @@ -1422,35 +1469,6 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, } rcu_read_unlock(); - if (unlikely(blocked_rdev)) { - /* Have to wait for this device to get unblocked, then retry */ - int j; - int d; - - for (j = 0; j < i; j++) { - if (r10_bio->devs[j].bio) { - d = r10_bio->devs[j].devnum; - rdev_dec_pending(conf->mirrors[d].rdev, mddev); - } - if (r10_bio->devs[j].repl_bio) { - struct md_rdev *rdev; - d = r10_bio->devs[j].devnum; - rdev = conf->mirrors[d].replacement; - if (!rdev) { - /* Race with remove_disk */ - smp_mb(); - rdev = conf->mirrors[d].rdev; - } - rdev_dec_pending(rdev, mddev); - } - } - allow_barrier(conf); - raid10_log(conf->mddev, "wait rdev %d blocked", blocked_rdev->raid_disk); - md_wait_for_blocked_rdev(blocked_rdev, mddev); - wait_barrier(conf); - goto retry_write; - } - if (max_sectors < r10_bio->sectors) r10_bio->sectors = max_sectors; From patchwork Thu May 6 04:04:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474774 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKjk5fMGz9sV5; Thu, 6 May 2021 14:05:14 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVG9-0005C7-Jq; Thu, 06 May 2021 04:05:09 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVG4-000582-Rk for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:04 +0000 Received: from mail-pj1-f71.google.com ([209.85.216.71]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVG3-0005QT-W8 for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:04 +0000 Received: by mail-pj1-f71.google.com with SMTP id j91-20020a17090a1464b0290155d0a238deso2217201pja.1 for ; Wed, 05 May 2021 21:05:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uzbmiJQF8p0+5a5rBmgCzQg9QcVuP/qk6uzkM4V0Srs=; b=L1QBfI4ti5j/xqR6YAf2/La+SSr9gz2yvUlBnhinBujcv//NOM54AlbTPJ0jz6UkU/ 1sspHd9GV2gvVtmrQ57L1yJC3O1HWSAbktTZbEa26KaUsk2kTgO0r4MlfEyVjmi4U8S0 MV9xLVgkOwomu1qqHu78QPZz2EdTZqP7EOrDUQUxIJ+8jiJ9Mses/QjFClzvVu416ajT Y4mZ5idfJkmxWW+RL99aG8sHtGoncdM2Z4pofP0Tle6giFQ8b5oateMlZ66j90ilqLHm hy6Zn4E3kZbqkdWkTG2mcDCJzLKjQ8MsXx9KFvEY/JnyudaTcMCpLjxulFaqO+3iKell N5VQ== X-Gm-Message-State: AOAM530SId2065L9WO0eABoH+XVlsYKAmV/uS59j73FP5Y3jm6eRyf6m weRBJZtuxO0qgOi+QSm+ffLmGePUIw+cqTeKdt050dsSUgrOhU/MrwuQ2E1jryzTcEzZ6w+v7hW oub9C/9WjnjH7zDcTAusoqJnerTU5h25DwFycM7zrkQ== X-Received: by 2002:a62:f202:0:b029:28e:96e7:a084 with SMTP id m2-20020a62f2020000b029028e96e7a084mr2391943pfh.59.1620273902599; Wed, 05 May 2021 21:05:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJaAc+k/I0Y3OAxUfMno10miD5xy2gP46XjUTIXL/F2Oc8Jpus11xJSQLSk+eCVhiuZ92+tg== X-Received: by 2002:a62:f202:0:b029:28e:96e7:a084 with SMTP id m2-20020a62f2020000b029028e96e7a084mr2391909pfh.59.1620273902207; Wed, 05 May 2021 21:05:02 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.05.00 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:05:01 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][H][PATCH 4/6] md/raid10: improve raid10 discard request Date: Thu, 6 May 2021 16:04:37 +1200 Message-Id: <20210506040442.10877-7-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Xiao Ni BugLink: https://bugs.launchpad.net/bugs/1896578 Now the discard request is split by chunk size. So it takes a long time to finish mkfs on disks which support discard function. This patch improve handling raid10 discard request. It uses the similar way with patch 29efc390b (md/md0: optimize raid0 discard handling). But it's a little complex than raid0. Because raid10 has different layout. If raid10 is offset layout and the discard request is smaller than stripe size. There are some holes when we submit discard bio to underlayer disks. For example: five disks (disk1 - disk5) D01 D02 D03 D04 D05 D05 D01 D02 D03 D04 D06 D07 D08 D09 D10 D10 D06 D07 D08 D09 The discard bio just wants to discard from D03 to D10. For disk3, there is a hole between D03 and D08. For disk4, there is a hole between D04 and D09. D03 is a chunk, raid10_write_request can handle one chunk perfectly. So the part that is not aligned with stripe size is still handled by raid10_write_request. If reshape is running when discard bio comes and the discard bio spans the reshape position, raid10_write_request is responsible to handle this discard bio. I did a test with this patch set. Without patch: time mkfs.xfs /dev/md0 real4m39.775s user0m0.000s sys0m0.298s With patch: time mkfs.xfs /dev/md0 real0m0.105s user0m0.000s sys0m0.007s nvme3n1 259:1 0 477G 0 disk └─nvme3n1p1 259:10 0 50G 0 part nvme4n1 259:2 0 477G 0 disk └─nvme4n1p1 259:11 0 50G 0 part nvme5n1 259:6 0 477G 0 disk └─nvme5n1p1 259:12 0 50G 0 part nvme2n1 259:9 0 477G 0 disk └─nvme2n1p1 259:15 0 50G 0 part nvme0n1 259:13 0 477G 0 disk └─nvme0n1p1 259:14 0 50G 0 part Reviewed-by: Coly Li Reviewed-by: Guoqing Jiang Tested-by: Adrian Huang Signed-off-by: Xiao Ni Signed-off-by: Song Liu (cherry picked from commit d30588b2731fb01e1616cf16c3fe79a1443e29aa) Signed-off-by: Matthew Ruffell --- drivers/md/raid10.c | 263 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 262 insertions(+), 1 deletion(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 789f575503de..346d7e8d554e 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -1518,6 +1518,263 @@ static void __make_request(struct mddev *mddev, struct bio *bio, int sectors) raid10_write_request(mddev, bio, r10_bio); } +static void raid10_end_discard_request(struct bio *bio) +{ + struct r10bio *r10_bio = bio->bi_private; + struct r10conf *conf = r10_bio->mddev->private; + struct md_rdev *rdev = NULL; + int dev; + int slot, repl; + + /* + * We don't care the return value of discard bio + */ + if (!test_bit(R10BIO_Uptodate, &r10_bio->state)) + set_bit(R10BIO_Uptodate, &r10_bio->state); + + dev = find_bio_disk(conf, r10_bio, bio, &slot, &repl); + if (repl) + rdev = conf->mirrors[dev].replacement; + if (!rdev) { + /* + * raid10_remove_disk uses smp_mb to make sure rdev is set to + * replacement before setting replacement to NULL. It can read + * rdev first without barrier protect even replacment is NULL + */ + smp_rmb(); + rdev = conf->mirrors[dev].rdev; + } + + if (atomic_dec_and_test(&r10_bio->remaining)) { + md_write_end(r10_bio->mddev); + raid_end_bio_io(r10_bio); + } + + rdev_dec_pending(rdev, conf->mddev); +} + +/* + * There are some limitations to handle discard bio + * 1st, the discard size is bigger than stripe_size*2. + * 2st, if the discard bio spans reshape progress, we use the old way to + * handle discard bio + */ +static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) +{ + struct r10conf *conf = mddev->private; + struct geom *geo = &conf->geo; + struct r10bio *r10_bio; + struct bio *split; + int disk; + sector_t chunk; + unsigned int stripe_size; + unsigned int stripe_data_disks; + sector_t split_size; + sector_t bio_start, bio_end; + sector_t first_stripe_index, last_stripe_index; + sector_t start_disk_offset; + unsigned int start_disk_index; + sector_t end_disk_offset; + unsigned int end_disk_index; + unsigned int remainder; + + if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery)) + return -EAGAIN; + + wait_barrier(conf); + + /* + * Check reshape again to avoid reshape happens after checking + * MD_RECOVERY_RESHAPE and before wait_barrier + */ + if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery)) + goto out; + + if (geo->near_copies) + stripe_data_disks = geo->raid_disks / geo->near_copies + + geo->raid_disks % geo->near_copies; + else + stripe_data_disks = geo->raid_disks; + + stripe_size = stripe_data_disks << geo->chunk_shift; + + bio_start = bio->bi_iter.bi_sector; + bio_end = bio_end_sector(bio); + + /* + * Maybe one discard bio is smaller than strip size or across one + * stripe and discard region is larger than one stripe size. For far + * offset layout, if the discard region is not aligned with stripe + * size, there is hole when we submit discard bio to member disk. + * For simplicity, we only handle discard bio which discard region + * is bigger than stripe_size * 2 + */ + if (bio_sectors(bio) < stripe_size*2) + goto out; + + /* + * Keep bio aligned with strip size. + */ + div_u64_rem(bio_start, stripe_size, &remainder); + if (remainder) { + split_size = stripe_size - remainder; + split = bio_split(bio, split_size, GFP_NOIO, &conf->bio_split); + bio_chain(split, bio); + allow_barrier(conf); + /* Resend the fist split part */ + submit_bio_noacct(split); + wait_barrier(conf); + } + div_u64_rem(bio_end, stripe_size, &remainder); + if (remainder) { + split_size = bio_sectors(bio) - remainder; + split = bio_split(bio, split_size, GFP_NOIO, &conf->bio_split); + bio_chain(split, bio); + allow_barrier(conf); + /* Resend the second split part */ + submit_bio_noacct(bio); + bio = split; + wait_barrier(conf); + } + + r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO); + r10_bio->mddev = mddev; + r10_bio->state = 0; + r10_bio->sectors = 0; + memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks); + + wait_blocked_dev(mddev, r10_bio); + + r10_bio->master_bio = bio; + + bio_start = bio->bi_iter.bi_sector; + bio_end = bio_end_sector(bio); + + /* + * Raid10 uses chunk as the unit to store data. It's similar like raid0. + * One stripe contains the chunks from all member disk (one chunk from + * one disk at the same HBA address). For layout detail, see 'man md 4' + */ + chunk = bio_start >> geo->chunk_shift; + chunk *= geo->near_copies; + first_stripe_index = chunk; + start_disk_index = sector_div(first_stripe_index, geo->raid_disks); + if (geo->far_offset) + first_stripe_index *= geo->far_copies; + start_disk_offset = (bio_start & geo->chunk_mask) + + (first_stripe_index << geo->chunk_shift); + + chunk = bio_end >> geo->chunk_shift; + chunk *= geo->near_copies; + last_stripe_index = chunk; + end_disk_index = sector_div(last_stripe_index, geo->raid_disks); + if (geo->far_offset) + last_stripe_index *= geo->far_copies; + end_disk_offset = (bio_end & geo->chunk_mask) + + (last_stripe_index << geo->chunk_shift); + + rcu_read_lock(); + for (disk = 0; disk < geo->raid_disks; disk++) { + struct md_rdev *rdev = rcu_dereference(conf->mirrors[disk].rdev); + struct md_rdev *rrdev = rcu_dereference( + conf->mirrors[disk].replacement); + + r10_bio->devs[disk].bio = NULL; + r10_bio->devs[disk].repl_bio = NULL; + + if (rdev && (test_bit(Faulty, &rdev->flags))) + rdev = NULL; + if (rrdev && (test_bit(Faulty, &rrdev->flags))) + rrdev = NULL; + if (!rdev && !rrdev) + continue; + + if (rdev) { + r10_bio->devs[disk].bio = bio; + atomic_inc(&rdev->nr_pending); + } + if (rrdev) { + r10_bio->devs[disk].repl_bio = bio; + atomic_inc(&rrdev->nr_pending); + } + } + rcu_read_unlock(); + + atomic_set(&r10_bio->remaining, 1); + for (disk = 0; disk < geo->raid_disks; disk++) { + sector_t dev_start, dev_end; + struct bio *mbio, *rbio = NULL; + struct md_rdev *rdev = rcu_dereference(conf->mirrors[disk].rdev); + struct md_rdev *rrdev = rcu_dereference( + conf->mirrors[disk].replacement); + + /* + * Now start to calculate the start and end address for each disk. + * The space between dev_start and dev_end is the discard region. + * + * For dev_start, it needs to consider three conditions: + * 1st, the disk is before start_disk, you can imagine the disk in + * the next stripe. So the dev_start is the start address of next + * stripe. + * 2st, the disk is after start_disk, it means the disk is at the + * same stripe of first disk + * 3st, the first disk itself, we can use start_disk_offset directly + */ + if (disk < start_disk_index) + dev_start = (first_stripe_index + 1) * mddev->chunk_sectors; + else if (disk > start_disk_index) + dev_start = first_stripe_index * mddev->chunk_sectors; + else + dev_start = start_disk_offset; + + if (disk < end_disk_index) + dev_end = (last_stripe_index + 1) * mddev->chunk_sectors; + else if (disk > end_disk_index) + dev_end = last_stripe_index * mddev->chunk_sectors; + else + dev_end = end_disk_offset; + + /* + * It only handles discard bio which size is >= stripe size, so + * dev_end > dev_start all the time + */ + if (r10_bio->devs[disk].bio) { + mbio = bio_clone_fast(bio, GFP_NOIO, &mddev->bio_set); + mbio->bi_end_io = raid10_end_discard_request; + mbio->bi_private = r10_bio; + r10_bio->devs[disk].bio = mbio; + r10_bio->devs[disk].devnum = disk; + atomic_inc(&r10_bio->remaining); + md_submit_discard_bio(mddev, rdev, mbio, + dev_start + choose_data_offset(r10_bio, rdev), + dev_end - dev_start); + bio_endio(mbio); + } + if (r10_bio->devs[disk].repl_bio) { + rbio = bio_clone_fast(bio, GFP_NOIO, &mddev->bio_set); + rbio->bi_end_io = raid10_end_discard_request; + rbio->bi_private = r10_bio; + r10_bio->devs[disk].repl_bio = rbio; + r10_bio->devs[disk].devnum = disk; + atomic_inc(&r10_bio->remaining); + md_submit_discard_bio(mddev, rrdev, rbio, + dev_start + choose_data_offset(r10_bio, rrdev), + dev_end - dev_start); + bio_endio(rbio); + } + } + + if (atomic_dec_and_test(&r10_bio->remaining)) { + md_write_end(r10_bio->mddev); + raid_end_bio_io(r10_bio); + } + + return 0; +out: + allow_barrier(conf); + return -EAGAIN; +} + static bool raid10_make_request(struct mddev *mddev, struct bio *bio) { struct r10conf *conf = mddev->private; @@ -1532,6 +1789,10 @@ static bool raid10_make_request(struct mddev *mddev, struct bio *bio) if (!md_write_start(mddev, bio)) return false; + if (unlikely(bio_op(bio) == REQ_OP_DISCARD)) + if (!raid10_handle_discard(mddev, bio)) + return true; + /* * If this request crosses a chunk boundary, we need to split * it. @@ -3771,7 +4032,7 @@ static int raid10_run(struct mddev *mddev) if (mddev->queue) { blk_queue_max_discard_sectors(mddev->queue, - mddev->chunk_sectors); + UINT_MAX); blk_queue_max_write_same_sectors(mddev->queue, 0); blk_queue_max_write_zeroes_sectors(mddev->queue, 0); blk_queue_io_min(mddev->queue, mddev->chunk_sectors << 9); From patchwork Thu May 6 04:04:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474778 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKjx0pCvz9sTD; Thu, 6 May 2021 14:05:25 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVGK-0005KN-CR; Thu, 06 May 2021 04:05:20 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVGA-0005CG-Fh for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:10 +0000 Received: from mail-pf1-f197.google.com ([209.85.210.197]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVG9-0005R5-DI for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:09 +0000 Received: by mail-pf1-f197.google.com with SMTP id k7-20020aa788c70000b029028ead4f0f50so2733565pff.10 for ; Wed, 05 May 2021 21:05:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=513yznwsJ/ggaYdHeVdovQzK4OKj0imcTFBZJVDqv28=; b=foZgbgE7TzyFa6YwmExnzG5QIOicZBp+lqIzU9URMVzhId877ifCQVwUpxo+CThZ24 W6Pem++e3zzi0LOk7sUTX4aczn21MxnwUNaW+TsPuMX4cKbz1t2T/9sAkg4hTTLghb4v SKq9L/3FW53dxh66MX/48Jsx9HUXGx7MucODRHUXL2tba5Qalk4E00eAO+wbrgm0dWqj RJBU14cvg6FoetPQ9CFPp3tou/vsl5fhJg5TsQJVmQWn6kUQjAE/PY3CyDyFWDXa9nzM 7u8jKc98Foa25HIHBwZ/Oio0iTALhTlpVcVpIv7gphYayJYJHMUIbb6/gp57yllXvxnd 0KrA== X-Gm-Message-State: AOAM531xJ4z8/dPF1Qvg+Zn8cn7Zs1PGjekI3//w4GvSHRVkP3hVI3kQ pzP9Hxboo5QqnVSXGifQyf6XkijpBZ6a266DtK9n4ux+e9q0OyGsJ2gkTnVrxzav05mPvrdGvDo jOR/KoJ8jGwvyyr6J/a8QT5FEIOESKhlaDiJpGiEpUg== X-Received: by 2002:a63:a511:: with SMTP id n17mr2278830pgf.9.1620273908101; Wed, 05 May 2021 21:05:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzehryWfVnV30ckCL5FxVZGl90cyhhqSxyTircep7ZfkoKFikhEmHXx2XzR5vzhusS8Tpui6Q== X-Received: by 2002:a63:a511:: with SMTP id n17mr2278808pgf.9.1620273907832; Wed, 05 May 2021 21:05:07 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.05.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:05:07 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][F][G][H][PATCH 5/6] md/raid10: improve discard request for far layout Date: Thu, 6 May 2021 16:04:40 +1200 Message-Id: <20210506040442.10877-10-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Xiao Ni BugLink: https://bugs.launchpad.net/bugs/1896578 For far layout, the discard region is not continuous on disks. So it needs far copies r10bio to cover all regions. It needs a way to know all r10bios have finish or not. Similar with raid10_sync_request, only the first r10bio master_bio records the discard bio. Other r10bios master_bio record the first r10bio. The first r10bio can finish after other r10bios finish and then return the discard bio. Tested-by: Adrian Huang Signed-off-by: Xiao Ni Signed-off-by: Song Liu (cherry picked from commit 254c271da0712ea8914f187588e0f81f7678ee2f) Signed-off-by: Matthew Ruffell --- drivers/md/raid10.c | 79 ++++++++++++++++++++++++++++++++++----------- drivers/md/raid10.h | 1 + 2 files changed, 61 insertions(+), 19 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 346d7e8d554e..30a7f0e3e68c 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -1518,6 +1518,28 @@ static void __make_request(struct mddev *mddev, struct bio *bio, int sectors) raid10_write_request(mddev, bio, r10_bio); } +static void raid_end_discard_bio(struct r10bio *r10bio) +{ + struct r10conf *conf = r10bio->mddev->private; + struct r10bio *first_r10bio; + + while (atomic_dec_and_test(&r10bio->remaining)) { + + allow_barrier(conf); + + if (!test_bit(R10BIO_Discard, &r10bio->state)) { + first_r10bio = (struct r10bio *)r10bio->master_bio; + free_r10bio(r10bio); + r10bio = first_r10bio; + } else { + md_write_end(r10bio->mddev); + bio_endio(r10bio->master_bio); + free_r10bio(r10bio); + break; + } + } +} + static void raid10_end_discard_request(struct bio *bio) { struct r10bio *r10_bio = bio->bi_private; @@ -1545,11 +1567,7 @@ static void raid10_end_discard_request(struct bio *bio) rdev = conf->mirrors[dev].rdev; } - if (atomic_dec_and_test(&r10_bio->remaining)) { - md_write_end(r10_bio->mddev); - raid_end_bio_io(r10_bio); - } - + raid_end_discard_bio(r10_bio); rdev_dec_pending(rdev, conf->mddev); } @@ -1563,7 +1581,9 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) { struct r10conf *conf = mddev->private; struct geom *geo = &conf->geo; - struct r10bio *r10_bio; + int far_copies = geo->far_copies; + bool first_copy = true; + struct r10bio *r10_bio, *first_r10bio; struct bio *split; int disk; sector_t chunk; @@ -1637,16 +1657,6 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) wait_barrier(conf); } - r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO); - r10_bio->mddev = mddev; - r10_bio->state = 0; - r10_bio->sectors = 0; - memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks); - - wait_blocked_dev(mddev, r10_bio); - - r10_bio->master_bio = bio; - bio_start = bio->bi_iter.bi_sector; bio_end = bio_end_sector(bio); @@ -1673,6 +1683,29 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) end_disk_offset = (bio_end & geo->chunk_mask) + (last_stripe_index << geo->chunk_shift); +retry_discard: + r10_bio = mempool_alloc(&conf->r10bio_pool, GFP_NOIO); + r10_bio->mddev = mddev; + r10_bio->state = 0; + r10_bio->sectors = 0; + memset(r10_bio->devs, 0, sizeof(r10_bio->devs[0]) * geo->raid_disks); + wait_blocked_dev(mddev, r10_bio); + + /* + * For far layout it needs more than one r10bio to cover all regions. + * Inspired by raid10_sync_request, we can use the first r10bio->master_bio + * to record the discard bio. Other r10bio->master_bio record the first + * r10bio. The first r10bio only release after all other r10bios finish. + * The discard bio returns only first r10bio finishes + */ + if (first_copy) { + r10_bio->master_bio = bio; + set_bit(R10BIO_Discard, &r10_bio->state); + first_copy = false; + first_r10bio = r10_bio; + } else + r10_bio->master_bio = (struct bio *)first_r10bio; + rcu_read_lock(); for (disk = 0; disk < geo->raid_disks; disk++) { struct md_rdev *rdev = rcu_dereference(conf->mirrors[disk].rdev); @@ -1764,11 +1797,19 @@ static int raid10_handle_discard(struct mddev *mddev, struct bio *bio) } } - if (atomic_dec_and_test(&r10_bio->remaining)) { - md_write_end(r10_bio->mddev); - raid_end_bio_io(r10_bio); + if (!geo->far_offset && --far_copies) { + first_stripe_index += geo->stride >> geo->chunk_shift; + start_disk_offset += geo->stride; + last_stripe_index += geo->stride >> geo->chunk_shift; + end_disk_offset += geo->stride; + atomic_inc(&first_r10bio->remaining); + raid_end_discard_bio(r10_bio); + wait_barrier(conf); + goto retry_discard; } + raid_end_discard_bio(r10_bio); + return 0; out: allow_barrier(conf); diff --git a/drivers/md/raid10.h b/drivers/md/raid10.h index 79cd2b7d3128..1461fd55311b 100644 --- a/drivers/md/raid10.h +++ b/drivers/md/raid10.h @@ -179,5 +179,6 @@ enum r10bio_state { R10BIO_Previous, /* failfast devices did receive failfast requests. */ R10BIO_FailFast, + R10BIO_Discard, }; #endif From patchwork Thu May 6 04:04:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1474780 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FbKk12kB6z9sV5; Thu, 6 May 2021 14:05:29 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1leVGO-0005PD-Fg; Thu, 06 May 2021 04:05:24 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1leVGD-0005FI-Nf for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:13 +0000 Received: from mail-pj1-f69.google.com ([209.85.216.69]) by youngberry.canonical.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from ) id 1leVGC-0005RR-Px for kernel-team@lists.ubuntu.com; Thu, 06 May 2021 04:05:12 +0000 Received: by mail-pj1-f69.google.com with SMTP id u13-20020a17090a3fcdb0290155c6507e67so2213244pjm.6 for ; Wed, 05 May 2021 21:05:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yRLv2k72csbb+cxx6dFGJgBYMCF5EBDh0mBmJgmNVwI=; b=iF/bvBZEkbiLtaUDJbic6MgYI5Tg7D2tPdiL99uRmSRpWUx+r5/lELKqIxNmVPAfpA vlNQ8Vi2DMo/QRBt29tk/WDa7/aslJseLAct4Gi8ZltV5TMixh8LvUTh0isEs+oq/T/s E89E/7sY3Oi/O/VHHUtrPl2ubp4MELUAt9tN3Xf6wAaYHgty+4ktjQncwFNHJELl2pYu 4gOKeMye8Ha4TT/CEXR/96bxv2cYkeiT+U4tj6FnaOsMYQhJ627JYc26QRpAtwoQ9CSf boJsNNOdeP5fjtFczXbNSjYRCfuPBfYQZnM/RlEEzUc6r7KbQ2ZDvNWT7lgkpzSxVECS dkdg== X-Gm-Message-State: AOAM530bglNUYYnEeSTmByZTlSin53hIjFk7mXUr5/hesfzDrq60U9FT KZzLhkBCpILaqgtFAoGBkYIz7Vam7IFHxeQSk7rn8C3kSKmW3q6VFuP3fCb6qrSJ1MtdGWbEqsr We8x21TKrQMSySI9yyYli9bjlq8JAvLO/Xrt8DeFpHQ== X-Received: by 2002:aa7:82c3:0:b029:276:1d63:cd0e with SMTP id f3-20020aa782c30000b02902761d63cd0emr2336759pfn.13.1620273911536; Wed, 05 May 2021 21:05:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwqhdYlyK+UBBS1XgLfQLzexFS/8mquT/Wkg8NErf8WZPZ3Wq1hJseuAF2CoaBlsi+tG3Le8g== X-Received: by 2002:aa7:82c3:0:b029:276:1d63:cd0e with SMTP id f3-20020aa782c30000b02902761d63cd0emr2336741pfn.13.1620273911295; Wed, 05 May 2021 21:05:11 -0700 (PDT) Received: from desktop.. (122-58-78-211-adsl.sparkbb.co.nz. [122.58.78.211]) by smtp.gmail.com with ESMTPSA id ge4sm675315pjb.4.2021.05.05.21.05.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 May 2021 21:05:10 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][F][G][H][PATCH 6/6] dm raid: remove unnecessary discard limits for raid0 and raid10 Date: Thu, 6 May 2021 16:04:42 +1200 Message-Id: <20210506040442.10877-12-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210506040442.10877-1-matthew.ruffell@canonical.com> References: <20210506040442.10877-1-matthew.ruffell@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Mike Snitzer BugLink: https://bugs.launchpad.net/bugs/1896578 Commit 29efc390b946 ("md/md0: optimize raid0 discard handling") and commit d30588b2731f ("md/raid10: improve raid10 discard request") remove MD raid0's and raid10's inability to properly handle large discards. So eliminate associated constraints from dm-raid's support. Depends-on: 29efc390b946 ("md/md0: optimize raid0 discard handling") Depends-on: d30588b2731f ("md/raid10: improve raid10 discard request") Reported-by: Matthew Ruffell Signed-off-by: Mike Snitzer (cherry picked from commit ca4a4e9a55beeb138bb06e3867f5e486da896d44) Signed-off-by: Matthew Ruffell --- drivers/md/dm-raid.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index cab12b2251ba..294f34d2d61b 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -3727,15 +3727,6 @@ static void raid_io_hints(struct dm_target *ti, struct queue_limits *limits) blk_limits_io_min(limits, chunk_size_bytes); blk_limits_io_opt(limits, chunk_size_bytes * mddev_data_stripes(rs)); - - /* - * RAID0 and RAID10 personalities require bio splitting, - * RAID1/4/5/6 don't and process large discard bios properly. - */ - if (rs_is_raid0(rs) || rs_is_raid10(rs)) { - limits->discard_granularity = chunk_size_bytes; - limits->max_discard_sectors = rs->md.chunk_sectors; - } } static void raid_postsuspend(struct dm_target *ti)