From patchwork Mon Jul 8 00:50:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mauricio Faria de Oliveira X-Patchwork-Id: 1128787 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45hn2c4mlpz9sNf; Mon, 8 Jul 2019 10:51:35 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1hkHsR-0002Ts-F9; Mon, 08 Jul 2019 00:51:31 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1hkHsO-0002So-Vj for kernel-team@lists.ubuntu.com; Mon, 08 Jul 2019 00:51:28 +0000 Received: from mail-qk1-f199.google.com ([209.85.222.199]) by youngberry.canonical.com with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1hkHsO-0004ci-Gj for kernel-team@lists.ubuntu.com; Mon, 08 Jul 2019 00:51:28 +0000 Received: by mail-qk1-f199.google.com with SMTP id v4so15096119qkj.10 for ; Sun, 07 Jul 2019 17:51:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=rckZ7fabQhIrcJBh+qAjL+5ScDiip6TpIJ6N3Ndtfw0=; b=WH9eLD7y4l63qD6MzrdxJmLstujSK949cNs/bvdSBbjJpE5yzrGqXxPZipn0oTYysN 5P7i5cceRWG2NRGnj9KnmZlfzATgYgYmxIcFhhw/7yMEkMoPluZNwdUhFy/56/z10pj3 oxBIM2NuPgWwYVwjcm154nuho7v/5NMfXWIiq2M9ce/fBCOro9NHUlGTqaa5a2KQ0iiR lzX4xeKmgQ3jP4CKk543GMyRA6qr/sS7Lf1dOv5ce0JJuh7XiC01go1GB3QM468mAS2T zPUfIzR2TYshTLHP9rOTBpXDGSKtJJ4w2dYSNq5MRd5O771nK9Jf2T+J1L8Z5qjKVNb1 Rk0A== X-Gm-Message-State: APjAAAU+BV8mbF+gdu3Mzhk1a2tTb770xHnxl6A9Ni5MHn3m3rK8bdyd tTnfwk+ZpEPz0PaL1Fm/B7+ZD6q6Jbt4Aj74/x3gpq7hqCcijw++XpTnTjlrpanxluWvafAztNk b81/ApvJMyfIKPM2+xQv4lljJSS3KwWIo/0IAhOBppg== X-Received: by 2002:a37:490c:: with SMTP id w12mr12069307qka.327.1562547087621; Sun, 07 Jul 2019 17:51:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqyOuR05OMtIuBQ775YxuOw3JfrNNvo7fNglQd/RFRHWkIvugdRANpOYQPreh47okcfVZQJLqQ== X-Received: by 2002:a37:490c:: with SMTP id w12mr12069303qka.327.1562547087427; Sun, 07 Jul 2019 17:51:27 -0700 (PDT) Received: from localhost.localdomain ([2804:14c:4e7:1017:3da7:3d04:ea25:3a0]) by smtp.gmail.com with ESMTPSA id t197sm6697527qke.2.2019.07.07.17.51.26 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sun, 07 Jul 2019 17:51:27 -0700 (PDT) From: Mauricio Faria de Oliveira To: kernel-team@lists.ubuntu.com Subject: [B][PATCH 03/11] bcache: fix inaccurate io state for detached bcache devices Date: Sun, 7 Jul 2019 21:50:30 -0300 Message-Id: <20190708005038.13184-4-mfo@canonical.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190708005038.13184-1-mfo@canonical.com> References: <20190708005038.13184-1-mfo@canonical.com> X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Tang Junhui BugLink: https://bugs.launchpad.net/bugs/1829563 When we run IO in a detached device, and run iostat to shows IO status, normally it will show like bellow (Omitted some fields): Device: ... avgrq-sz avgqu-sz await r_await w_await svctm %util sdd ... 15.89 0.53 1.82 0.20 2.23 1.81 52.30 bcache0 ... 15.89 115.42 0.00 0.00 0.00 2.40 69.60 but after IO stopped, there are still very big avgqu-sz and %util values as bellow: Device: ... avgrq-sz avgqu-sz await r_await w_await svctm %util bcache0 ... 0 5326.32 0.00 0.00 0.00 0.00 100.10 The reason for this issue is that, only generic_start_io_acct() called and no generic_end_io_acct() called for detached device in cached_dev_make_request(). See the code: //start generic_start_io_acct() generic_start_io_acct(q, rw, bio_sectors(bio), &d->disk->part0); if (cached_dev_get(dc)) { //will callback generic_end_io_acct() } else { //will not call generic_end_io_acct() } This patch calls generic_end_io_acct() in the end of IO for detached devices, so we can show IO state correctly. (Modified to use GFP_NOIO in kzalloc() by Coly Li) Changelog: v2: fix typo. v1: the initial version. Signed-off-by: Tang Junhui Reviewed-by: Coly Li Reviewed-by: Hannes Reinecke Reviewed-by: Michael Lyle Signed-off-by: Jens Axboe (cherry picked from commit bc082a55d25c837341709accaf11311c3a9af727) Signed-off-by: Mauricio Faria de Oliveira --- drivers/md/bcache/request.c | 58 ++++++++++++++++++++++++++++++++----- 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c index b00a0f298531..da16e43adfe7 100644 --- a/drivers/md/bcache/request.c +++ b/drivers/md/bcache/request.c @@ -986,6 +986,55 @@ static void cached_dev_nodata(struct closure *cl) continue_at(cl, cached_dev_bio_complete, NULL); } +struct detached_dev_io_private { + struct bcache_device *d; + unsigned long start_time; + bio_end_io_t *bi_end_io; + void *bi_private; +}; + +static void detached_dev_end_io(struct bio *bio) +{ + struct detached_dev_io_private *ddip; + + ddip = bio->bi_private; + bio->bi_end_io = ddip->bi_end_io; + bio->bi_private = ddip->bi_private; + + generic_end_io_acct(ddip->d->disk->queue, + bio_data_dir(bio), + &ddip->d->disk->part0, ddip->start_time); + + kfree(ddip); + + bio->bi_end_io(bio); +} + +static void detached_dev_do_request(struct bcache_device *d, struct bio *bio) +{ + struct detached_dev_io_private *ddip; + struct cached_dev *dc = container_of(d, struct cached_dev, disk); + + /* + * no need to call closure_get(&dc->disk.cl), + * because upper layer had already opened bcache device, + * which would call closure_get(&dc->disk.cl) + */ + ddip = kzalloc(sizeof(struct detached_dev_io_private), GFP_NOIO); + ddip->d = d; + ddip->start_time = jiffies; + ddip->bi_end_io = bio->bi_end_io; + ddip->bi_private = bio->bi_private; + bio->bi_end_io = detached_dev_end_io; + bio->bi_private = ddip; + + if ((bio_op(bio) == REQ_OP_DISCARD) && + !blk_queue_discard(bdev_get_queue(dc->bdev))) + bio->bi_end_io(bio); + else + generic_make_request(bio); +} + /* Cached devices - read & write stuff */ static blk_qc_t cached_dev_make_request(struct request_queue *q, @@ -1027,13 +1076,8 @@ static blk_qc_t cached_dev_make_request(struct request_queue *q, else cached_dev_read(dc, s); } - } else { - if ((bio_op(bio) == REQ_OP_DISCARD) && - !blk_queue_discard(bdev_get_queue(dc->bdev))) - bio_endio(bio); - else - generic_make_request(bio); - } + } else + detached_dev_do_request(d, bio); return BLK_QC_T_NONE; }