From patchwork Sun Sep 8 23:18:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159561 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="MMhE+g1E"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS1K6yKNz9sPD for ; Mon, 9 Sep 2019 09:19:33 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731252AbfIHXTD (ORCPT ); Sun, 8 Sep 2019 19:19:03 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:42605 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731201AbfIHXTD (ORCPT ); Sun, 8 Sep 2019 19:19:03 -0400 Received: by mail-pl1-f195.google.com with SMTP id x20so93493plm.9 for ; Sun, 08 Sep 2019 16:18:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=6uMksyYoQE2pF7FGUiDsdEdyfVo+lbVfSO3ifjA8fXY=; b=MMhE+g1E1klM3GkMUOXoVuIrg5F35jG9Fp0fFFgF8T1WvGR8mKBuORKcSbOf6JZ9fi 5JZVDzs3CXlPPTVKejo301V6FkbOyxCwekmAT8TvcqFY8+i3VG7SkpIKoaBFii2tsaNf WEbIxgMIN5+jxpBGfhh+qHECzLIzU52RO36ePGtC/lBR9XsuwJj2ED5cHuDa8FkqY9aB AAZ/HK4ruGNY73iWuBQLk7dKBdjpTl1EmhJ8cw3fd6F/yetcOjbdm2VKzD6wtsnxDvzI hV3TVDBBE+3KB2r5hV/nLsmCKt0etczoBW2z1KZ1MZOcLErOS0QEbRzSsaVXCRlwNlbu 2KUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=6uMksyYoQE2pF7FGUiDsdEdyfVo+lbVfSO3ifjA8fXY=; b=ubqO5lJFcpSo5bQEMQWEflXYqFYqUhXkuTdD6Y6tmpViML0tvLQvWxlVNHY8jkKG0M foE9MJHcwCLOVxkChIPqrTBav+3M/sFp9M4Whh7S7IglUGob/kYjt+FlydEK52mpHwFY yfodD+bWivDRuw+lAnp0XFVDPEU2T5wkpygWv7NYRmeOm88CQ4Ug+wlE1m29hwm/m6X2 FV/JPNagbogRgI/71sMzwKgWSBPZX7OXxFEdaIdfz9V9oWxX2GfJyaUnC0KTK5msStog fD5wHbl+sXwL2GNCvxUgF1+VqTxrKMupLIr2ydo8N08ATVyU1jfj1FsernnsfsHTsZwl lfoQ== X-Gm-Message-State: APjAAAWpmLRDsFRcaCEdhEPJ3L33ZLPYqBJ91j+L8g7VftikznnLaiRw oJP5+0OpGAwBx+o0tks3IgAG X-Google-Smtp-Source: APXvYqx8az6HHPPG8qGquQzVzZroFH+w7Yb0fkUIJ0hdikaAfDhkQYunKjRl6y+shJ06O8LY/q9x/w== X-Received: by 2002:a17:902:8e82:: with SMTP id bg2mr21041767plb.323.1567984737618; Sun, 08 Sep 2019 16:18:57 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id g8sm10832669pgk.1.2019.09.08.16.18.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:18:57 -0700 (PDT) Date: Mon, 9 Sep 2019 09:18:51 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 1/6] ext4: introduce direct IO read path using iomap infrastructure Message-ID: <75a6ead09a10e362526a849af482510a0090f82a.1567978633.git.mbobrowski@mbobrowski.org> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This patch introduces a new direct IO read path that makes use of the iomap infrastructure. The new function ext4_dio_read_iter() is responsible for calling into the iomap infrastructure via iomap_dio_rw(). If the inode in question does not pass preliminary checks in ext4_dio_checks(), then we simply fallback to buffered IO and take that path to fulfil the request. It's imperative that we drop the IOCB_DIRECT flag from iocb->ki_flags in order to prevent generic_file_read_iter() from trying to take the direct IO code path again. Signed-off-by: Matthew Bobrowski Reviewed-by: Ritesh Harjani --- fs/ext4/file.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 70b0438dbc94..e52e3928dc25 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -34,6 +34,53 @@ #include "xattr.h" #include "acl.h" +static bool ext4_dio_checks(struct inode *inode) +{ +#if IS_ENABLED(CONFIG_FS_ENCRYPTION) + if (IS_ENCRYPTED(inode)) + return false; +#endif + if (ext4_should_journal_data(inode)) + return false; + if (ext4_has_inline_data(inode)) + return false; + return true; +} + +static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) +{ + ssize_t ret; + struct inode *inode = file_inode(iocb->ki_filp); + + /* + * Get exclusion from truncate and other inode operations. + */ + if (!inode_trylock_shared(inode)) { + if (iocb->ki_flags & IOCB_NOWAIT) + return -EAGAIN; + inode_lock_shared(inode); + } + + if (!ext4_dio_checks(inode)) { + inode_unlock_shared(inode); + /* + * Fallback to buffered IO if the operation being + * performed on the inode is not supported by direct + * IO. The IOCB_DIRECT flag flags needs to be cleared + * here to ensure that the direct IO code path within + * generic_file_read_iter() is not taken again. + */ + iocb->ki_flags &= ~IOCB_DIRECT; + return generic_file_read_iter(iocb, to); + } + + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL); + inode_unlock_shared(inode); + + file_accessed(iocb->ki_filp); + return ret; +} + #ifdef CONFIG_FS_DAX static ssize_t ext4_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) { @@ -64,16 +111,19 @@ static ssize_t ext4_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) static ssize_t ext4_file_read_iter(struct kiocb *iocb, struct iov_iter *to) { - if (unlikely(ext4_forced_shutdown(EXT4_SB(file_inode(iocb->ki_filp)->i_sb)))) + struct inode *inode = file_inode(iocb->ki_filp); + + if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb)))) return -EIO; if (!iov_iter_count(to)) return 0; /* skip atime */ -#ifdef CONFIG_FS_DAX - if (IS_DAX(file_inode(iocb->ki_filp))) + if (IS_DAX(inode)) return ext4_dax_read_iter(iocb, to); -#endif + + if (iocb->ki_flags & IOCB_DIRECT) + return ext4_dio_read_iter(iocb, to); return generic_file_read_iter(iocb, to); } From patchwork Sun Sep 8 23:19:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159562 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="Hc5ap/DF"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS1L3YnFz9sPF for ; Mon, 9 Sep 2019 09:19:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731256AbfIHXTX (ORCPT ); Sun, 8 Sep 2019 19:19:23 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:34129 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731253AbfIHXTX (ORCPT ); Sun, 8 Sep 2019 19:19:23 -0400 Received: by mail-pg1-f195.google.com with SMTP id n9so6718134pgc.1 for ; Sun, 08 Sep 2019 16:19:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=NSL01mkUl8fEtBlCwOw2cj9yasSdvgdfrSqSeiKCTmQ=; b=Hc5ap/DFW2n2quEwo6i8ElkD/wvYVSJ8INTCfC7QW5Bf44rLm2V5h+dYg5hL4mGxGb Mi8nkEoY63QYWPZF6RXIKvcH+VzMRCj9yw7lEwFyecgDn783uehOx2sJpsbRsxAuHrLZ hq2h1gbcvQyrEPLXiBplmLGJFQv3Bw6OxmRaWdym4nXkz0E7DJO0EfWibAEJYKkyhHO+ lR8Uysl1P0WxbiC5jRxHCZNNAqsdjlcPPsj2Zu/PIq/M/X2P4PlIyh/eVIK/KJfnrVXS XYuDfH9f4GZHfYydFN2oUORE+E+FHYqFqCK7LReN/XQ6UcU4zavtevFQmVkgUNs6ajuy nG4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=NSL01mkUl8fEtBlCwOw2cj9yasSdvgdfrSqSeiKCTmQ=; b=BJhhoHROJKaALBBKZxEwWfOATjI7+HDAK1ZowAmBObzHYdwKMP7Et1i5hPM/L5eYNZ oVzp0GvGxb0/OsPBU9oCvG8j+NvkK5oneqKToVO/B//FMmGGxGCXU1c5Ooi64B+NzU8C 6+AlUMR223Lfxgp92SMmUszYmrYPfxw0KS+BV3dyv8qTJtG098sawlvLoLhx0uxLui3E zKJdFMEjSI3rg8PESVrRuAU7/rVH3w+ecgdUlzYVfU1ESbzB47go2qnxPWsoji5IpTIH i5VEeRQ+vos86kOedvR5J0WEcAXbYD/5NH2krw4t7ovcPAZxGEApsd2wUePwDcnf45Sb XB+A== X-Gm-Message-State: APjAAAWKeOahBqYL5roL2Qy87gzIjLbdJAq+iNpRM/Z9gDdviQlC1hLL xeSMvPigWRMS1XRxSgZBCMFY X-Google-Smtp-Source: APXvYqx+Wqr5mMuBQ66NOiGFoN5UlNdtQAbjv38+YJ/rfLZIyIz7GlmCQW9jjzaJL0D+zx0sFf3WpA== X-Received: by 2002:a63:f907:: with SMTP id h7mr18832410pgi.418.1567984762022; Sun, 08 Sep 2019 16:19:22 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id u7sm10507007pgr.94.2019.09.08.16.19.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:19:21 -0700 (PDT) Date: Mon, 9 Sep 2019 09:19:15 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 2/6] ext4: move inode extension/truncate code out from ext4_iomap_end() Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org In preparation for implementing the iomap direct IO write path modifications, the inode extension/truncate code needs to be moved out from ext4_iomap_end(). For direct IO, if the current code remained within ext4_iomap_end() it would behave incorrectly. If we update the inode size prior to converting unwritten extents we run the risk of allowing a racing direct IO read operation to find unwritten extents before they are converted. The inode extension/truncate code has been moved out into a new helper ext4_handle_inode_extension(). This helper has been designed so that it can be used by both DAX and direct IO paths in the instance that the result of the write is extending the inode. Signed-off-by: Matthew Bobrowski --- fs/ext4/file.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++++- fs/ext4/inode.c | 48 +------------------------ 2 files changed, 93 insertions(+), 48 deletions(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index e52e3928dc25..8e586198f6e6 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -33,6 +33,7 @@ #include "ext4_jbd2.h" #include "xattr.h" #include "acl.h" +#include "truncate.h" static bool ext4_dio_checks(struct inode *inode) { @@ -233,12 +234,91 @@ static ssize_t ext4_write_checks(struct kiocb *iocb, struct iov_iter *from) return iov_iter_count(from); } +static int ext4_handle_inode_extension(struct inode *inode, loff_t offset, + ssize_t len, size_t count) +{ + handle_t *handle; + bool truncate = false; + ext4_lblk_t written_blk, end_blk; + int ret = 0, blkbits = inode->i_blkbits; + + handle = ext4_journal_start(inode, EXT4_HT_INODE, 2); + if (IS_ERR(handle)) { + ret = PTR_ERR(handle); + goto orphan_del; + } + + if (ext4_update_inode_size(inode, offset + len)) + ext4_mark_inode_dirty(handle, inode); + + /* + * We may need truncate allocated but not written blocks + * beyond EOF. + */ + written_blk = ALIGN(offset + len, 1 << blkbits); + end_blk = ALIGN(offset + len + count, 1 << blkbits); + if (written_blk < end_blk && ext4_can_truncate(inode)) + truncate = true; + + /* + * Remove the inode from the orphan list if it has been + * extended and everything went OK. + */ + if (!truncate && inode->i_nlink) + ext4_orphan_del(handle, inode); + ext4_journal_stop(handle); + + if (truncate) { + ext4_truncate_failed_write(inode); +orphan_del: + /* + * If the truncate operation failed early the inode + * may still be on the orphan list. In that case, we + * need try remove the inode from the linked list in + * memory. + */ + if (inode->i_nlink) + ext4_orphan_del(NULL, inode); + } + return ret; +} + +/* + * The inode may have been placed onto the orphan list or has had + * blocks allocated beyond EOF as a result of an extension. We need to + * ensure that any necessary cleanup routines are performed if the + * error path has been taken for a write. + */ +static int ext4_handle_failed_inode_extension(struct inode *inode, loff_t size) +{ + int ret = 0; + handle_t *handle; + + if (size > i_size_read(inode)) + ext4_truncate_failed_write(inode); + + if (!list_empty(&EXT4_I(inode)->i_orphan)) { + handle = ext4_journal_start(inode, EXT4_HT_INODE, 2); + if (IS_ERR(handle)) { + if (inode->i_nlink) + ext4_orphan_del(NULL, inode); + return PTR_ERR(handle); + } + if (inode->i_nlink) + ext4_orphan_del(handle, inode); + ext4_journal_stop(handle); + } + return ret; +} + #ifdef CONFIG_FS_DAX static ssize_t ext4_dax_write_iter(struct kiocb *iocb, struct iov_iter *from) { - struct inode *inode = file_inode(iocb->ki_filp); ssize_t ret; + int error = 0; + loff_t offset; + struct inode *inode = file_inode(iocb->ki_filp); if (!inode_trylock(inode)) { if (iocb->ki_flags & IOCB_NOWAIT) @@ -255,7 +335,18 @@ ext4_dax_write_iter(struct kiocb *iocb, struct iov_iter *from) if (ret) goto out; + offset = iocb->ki_pos; ret = dax_iomap_rw(iocb, from, &ext4_iomap_ops); + if (ret > 0 && iocb->ki_pos > i_size_read(inode)) + error = ext4_handle_inode_extension(inode, offset, ret, + iov_iter_count(from)); + + if (ret < 0) + error = ext4_handle_failed_inode_extension(inode, + iocb->ki_pos); + + if (error) + ret = error; out: inode_unlock(inode); if (ret > 0) diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 420fe3deed39..761ce6286b05 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3601,53 +3601,7 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, static int ext4_iomap_end(struct inode *inode, loff_t offset, loff_t length, ssize_t written, unsigned flags, struct iomap *iomap) { - int ret = 0; - handle_t *handle; - int blkbits = inode->i_blkbits; - bool truncate = false; - - if (!(flags & IOMAP_WRITE) || (flags & IOMAP_FAULT)) - return 0; - - handle = ext4_journal_start(inode, EXT4_HT_INODE, 2); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto orphan_del; - } - if (ext4_update_inode_size(inode, offset + written)) - ext4_mark_inode_dirty(handle, inode); - /* - * We may need to truncate allocated but not written blocks beyond EOF. - */ - if (iomap->offset + iomap->length > - ALIGN(inode->i_size, 1 << blkbits)) { - ext4_lblk_t written_blk, end_blk; - - written_blk = (offset + written) >> blkbits; - end_blk = (offset + length) >> blkbits; - if (written_blk < end_blk && ext4_can_truncate(inode)) - truncate = true; - } - /* - * Remove inode from orphan list if we were extending a inode and - * everything went fine. - */ - if (!truncate && inode->i_nlink && - !list_empty(&EXT4_I(inode)->i_orphan)) - ext4_orphan_del(handle, inode); - ext4_journal_stop(handle); - if (truncate) { - ext4_truncate_failed_write(inode); -orphan_del: - /* - * If truncate failed early the inode might still be on the - * orphan list; we need to make sure the inode is removed from - * the orphan list in that case. - */ - if (inode->i_nlink) - ext4_orphan_del(NULL, inode); - } - return ret; + return 0; } const struct iomap_ops ext4_iomap_ops = { From patchwork Sun Sep 8 23:19:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159563 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="F01PsGlp"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS1S17BDz9sCJ for ; Mon, 9 Sep 2019 09:19:40 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731280AbfIHXTi (ORCPT ); Sun, 8 Sep 2019 19:19:38 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:37304 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731203AbfIHXTi (ORCPT ); Sun, 8 Sep 2019 19:19:38 -0400 Received: by mail-pf1-f196.google.com with SMTP id y5so5238454pfo.4 for ; Sun, 08 Sep 2019 16:19:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=gx5XXdzXQF7G+1VrbudI1A9EwDDJ+tyVBcppt9K0Szs=; b=F01PsGlpkKUdf7ARLGoLElKXYgoOD/rlV0S5n7k8NS7EP45gvPuJjctcl+F0L4k9bt pDQXERm5AzCRpD3dA1v0uoQIDzBWCX1AGcdXTMQ2swDhFSRUaF8LIthjXc+fZIig+zLQ s/7z5bNVbrZwqKM7RAAQG1VijKSr88f27jHgnsbY0tUZfcsG5/SBvOqpzcNzXQsn8XB/ J0MXTuCwsEmPEBxptasCFIhg26bDPdRLBq4eFmmB8B0TL4o7ap7flYLw3uwnqls7GuxD RgtfGkyD0IuiBJ/B8ZSqxJlUlH9WpzpR5SxzDVfsxeA6+rArw29HoA72JuYPlezWpxCG +EkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=gx5XXdzXQF7G+1VrbudI1A9EwDDJ+tyVBcppt9K0Szs=; b=UdExyZLEqJvgCz/bHBV544NK7D6QYd8ySezOAQEAwgWNsWWhdSJOljFj4pjM05Fxv9 6wHnk24lCaVdDoykBhBs8Q2XmtNCDEtLB1ZCeDrvvWVhLH7LTgIHFcdaFSLas9gGKu4J NrPOT0Xa3HC4jtAWV73nQLwGsfdEbGRPOJm9V8UeXl27yi/WrMGkBp+vfi66bGqVFxk1 W6/IRrMTiRRgpzOSIW9Z/trfpZJ51roeilXVPtk0zCVxuiSFgBSpxzCKti3MJ3QhuX81 kyfKGzoDJlIFVXStzdWIblvxF6ySUNbI+7Ikz/SuvZ/d5YE6MnFv/QpM7LWX9SoTJhBs R84w== X-Gm-Message-State: APjAAAXQHxKd3NU8/H1K/jsd4O05vQe4j5qNan80Ottjc0E1MWqBtPGw zVAX3aNM7eEshWYZHlslWgmk X-Google-Smtp-Source: APXvYqz7aWiON36Pum31Iaiaxq6K6qf3NBcQmcHZejX3zQ68ZdEeugxZEoHLn+LUsyAMIyFjFw0/qg== X-Received: by 2002:a65:5cca:: with SMTP id b10mr19318386pgt.365.1567984777787; Sun, 08 Sep 2019 16:19:37 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id u5sm12572075pfl.25.2019.09.08.16.19.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:19:37 -0700 (PDT) Date: Mon, 9 Sep 2019 09:19:31 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 3/6] iomap: modify ->end_io() calling convention Message-ID: <8368d2ea5f2e80fed7fbba3685b0d3c1e5ff742a.1567978633.git.mbobrowski@mbobrowski.org> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This patch modifies the calling convention for the iomap ->end_io() callback. Rather than passing either dio->error or dio->size as the 'size' argument, we instead pass both dio->error and dio->size values separately. In the instance that an error occurred during a write, we currently cannot determine whether any blocks have been allocated beyond the current EOF and data has subsequently been written to these blocks within the ->end_io() callback. As a result, we cannot judge whether we should take the truncate failed write path. Having both dio->error and dio->size will allow us to perform such checks within this callback. Signed-off-by: Matthew Bobrowski --- fs/iomap/direct-io.c | 9 +++------ fs/xfs/xfs_file.c | 8 +++++--- include/linux/iomap.h | 4 ++-- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 10517cea9682..2ccf1c6460d4 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -77,13 +77,10 @@ static ssize_t iomap_dio_complete(struct iomap_dio *dio) loff_t offset = iocb->ki_pos; ssize_t ret; - if (dio->end_io) { - ret = dio->end_io(iocb, - dio->error ? dio->error : dio->size, - dio->flags); - } else { + if (dio->end_io) + ret = dio->end_io(iocb, dio->size, dio->error, dio->flags); + else ret = dio->error; - } if (likely(!ret)) { ret = dio->size; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 28101bbc0b78..d49759008c54 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -369,21 +369,23 @@ static int xfs_dio_write_end_io( struct kiocb *iocb, ssize_t size, + int error, unsigned flags) { struct inode *inode = file_inode(iocb->ki_filp); struct xfs_inode *ip = XFS_I(inode); loff_t offset = iocb->ki_pos; unsigned int nofs_flag; - int error = 0; trace_xfs_end_io_direct_write(ip, offset, size); if (XFS_FORCED_SHUTDOWN(ip->i_mount)) return -EIO; - if (size <= 0) - return size; + if (error) + return error; + if (!size) + return 0; /* * Capture amount written on completion as we can't reliably account diff --git a/include/linux/iomap.h b/include/linux/iomap.h index bc499ceae392..d983cdcf2e72 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -188,8 +188,8 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno, */ #define IOMAP_DIO_UNWRITTEN (1 << 0) /* covers unwritten extent(s) */ #define IOMAP_DIO_COW (1 << 1) /* covers COW extent(s) */ -typedef int (iomap_dio_end_io_t)(struct kiocb *iocb, ssize_t ret, - unsigned flags); +typedef int (iomap_dio_end_io_t)(struct kiocb *iocb, ssize_t size, + int error, unsigned int flags); ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, iomap_dio_end_io_t end_io); int iomap_dio_iopoll(struct kiocb *kiocb, bool spin); From patchwork Sun Sep 8 23:19:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159564 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="VX1ufSX+"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS1h4ffXz9sNx for ; Mon, 9 Sep 2019 09:19:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731290AbfIHXTv (ORCPT ); Sun, 8 Sep 2019 19:19:51 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:36811 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731282AbfIHXTv (ORCPT ); Sun, 8 Sep 2019 19:19:51 -0400 Received: by mail-pg1-f194.google.com with SMTP id l21so6707077pgm.3 for ; Sun, 08 Sep 2019 16:19:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=BEDKpXPdUTV7XE4Q14Y66bFncWGnnigD83Lkh90WyW8=; b=VX1ufSX+EkshMc6s83N6qaf+btBEf9yhfbW6461tj53vckr/OoZpr8badf1c6bVRW2 Xy4o50K2g/HgazBt17HSZx2wlmvSA+iDTiixxTMb88DQgpdWJ1zGSpY8ovSwe2fKg2mM 8g0CwKKXWpzS+GGk7KjRMb/HnGsU2At8MlqcSRPE6bxllUHdnQiAf7ZGiAaTsIx0xsGj JMK9/lFKyEjE9wfOqSxxyTu3W7n9yxZGa5FD1c6rQaqgQaQDQY+Vn+o7XNj9r2+XpBAL gWsfOhupO5gXSKP4xgGU1dCQ9tjrtZT+Oh2KfPucABgzCcf6zXbuAq0Ys/h6fENTWdVm Nhkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=BEDKpXPdUTV7XE4Q14Y66bFncWGnnigD83Lkh90WyW8=; b=G1oGrgKrl8wMF1ECD6t6ocTSVlKnBAx0wfvqWvMJL50Ez21WimOAo46mxGGsxx4A8R dB5ugQA+rNzTEAUIMA8ELABzfd4SuRi60BA+IBPy8gxHWUAlHoMj3kGVlnMOZY0LzMwQ i63r0gGsZyrCzDuUSXy3lyNxRzbZJYzE+Qcd8J+M4/vagzsq2Xl/qsWUFFCslWzqCAI1 lzmwKuC6neW5IWgBVaYAdrPV0gXBtvj9B8js0OC6dj2qMVKgeg2qRDkbLdoGugrfyyUU b5tq8YXeo9GDvKyNYA/aI54NVs0HCHt+nceAsee1U/V0OUpS87bgX8paNS3tGLA3gZa/ sTNw== X-Gm-Message-State: APjAAAU2cOz60885iZ+hWHXWXfcfbXIN4a4knjSguTTZsPeHpABHr2oC kQRugqYAAwROrCCM0ihwn+1G X-Google-Smtp-Source: APXvYqy3mQm3z49ZC/NdLW7xH3ra53grqxkIsg20iZa9ZuVw7JEBHIfynKTyoqo6TLwR4fEcaiuMBg== X-Received: by 2002:a17:90a:8c16:: with SMTP id a22mr13087982pjo.111.1567984790208; Sun, 08 Sep 2019 16:19:50 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id v18sm10904747pgl.87.2019.09.08.16.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:19:49 -0700 (PDT) Date: Mon, 9 Sep 2019 09:19:44 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 4/6] ext4: reorder map.m_flags checks in ext4_iomap_begin() Message-ID: <38c2c1dd6f62f82e485b1a767ddeb49606439d67.1567978633.git.mbobrowski@mbobrowski.org> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org For iomap direct IO write code path changes, we need to accommodate for the case where the block mapping flags passed to ext4_map_blocks() will result in m_flags having both EXT4_MAP_MAPPED and EXT4_MAP_UNWRITTEN bits set. In order for the allocated unwritten extents to be converted properly in the end_io handler, iomap->type must be set to IOMAP_UNWRITTEN, so we need to reshuffle the conditional statement in order to achieve this. This change is a no-op for DAX code path as the block mapping flag passed to ext4_map_blocks() when IS_DAX(inode) never results in EXT4_MAP_MAPPED and EXT4_MAP_UNWRITTEN being set at once. Signed-off-by: Matthew Bobrowski Reviewed-by: Ritesh Harjani --- fs/ext4/inode.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 761ce6286b05..efb184928e51 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3581,10 +3581,21 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, iomap->type = delalloc ? IOMAP_DELALLOC : IOMAP_HOLE; iomap->addr = IOMAP_NULL_ADDR; } else { - if (map.m_flags & EXT4_MAP_MAPPED) { - iomap->type = IOMAP_MAPPED; - } else if (map.m_flags & EXT4_MAP_UNWRITTEN) { + /* + * Flags passed to ext4_map_blocks() for direct IO + * writes can result in m_flags having both + * EXT4_MAP_MAPPED and EXT4_MAP_UNWRITTEN bits set. In + * order for allocated unwritten extents to be + * converted to written extents in the end_io handler + * correctly, we need to ensure that the iomap->type + * is also set appropriately in that case. Thus, we + * need to check whether EXT4_MAP_UNWRITTEN is set + * first. + */ + if (map.m_flags & EXT4_MAP_UNWRITTEN) { iomap->type = IOMAP_UNWRITTEN; + } else if (map.m_flags & EXT4_MAP_MAPPED) { + iomap->type = IOMAP_MAPPED; } else { WARN_ON_ONCE(1); return -EIO; From patchwork Sun Sep 8 23:19:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159565 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="aRUJnTyT"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS1x1wv7z9sCJ for ; Mon, 9 Sep 2019 09:20:05 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731296AbfIHXUE (ORCPT ); Sun, 8 Sep 2019 19:20:04 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:40244 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731252AbfIHXUD (ORCPT ); Sun, 8 Sep 2019 19:20:03 -0400 Received: by mail-pg1-f196.google.com with SMTP id w10so6707261pgj.7 for ; Sun, 08 Sep 2019 16:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=ZXe8GCRxPXTndFWuYeSWB+6dNeh/wQOQbmrtOTcAGj8=; b=aRUJnTyTOdGpYdf5q51USGqiRac27Am5gV26rMbNCYSAIEofNGYw+ppJ7JgtfVSmVQ ztBImzq2/yGgedfUuxMY6gq6ESOQmvgamcUnKbyJ2sCri3i2XyF0NL/wIbsIQ95veTzn foGGYLXsXaBb4LHv93dPscw5wG8ODcNXEEbRSycesEpQTM9uMGsz+QFnDjRt4hSR9HEf jyNl2lxUUS7iW/uwSBj3xczdXn2WVrkgL7s+B+wsdV55XSJ2NM9LS2RyD7Fzn4mf6u2O RcfZYE+aU2iEu+d2fB2fJDlEUy4DPQWAD+rasfTCggBB3wCtW3j2JUlXBNiuESTIOwIh xPpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=ZXe8GCRxPXTndFWuYeSWB+6dNeh/wQOQbmrtOTcAGj8=; b=NM7hWixK0C47ucuxBPeDJDxWC6K70PMledBwKSKtHBhjVL71LHAHiCiv7MVEWkFZtk q8CepMuOjuvnqALxd08/7FoYlaKocOfmmkc7ypaWeiH0DbUD8nfMtBfe3fauYuWMV7NS tcmCH0tVZlqLcoELMz4/nQvklPZH4RMbWFYs6qZrPLuejWAtNBZd4+eALGmHxKOzOV7l ZLF8QpQRGPRadT04r8b3qqHVUJvf8HELxh3q2OrGb5U18ztW6W83lZUNb4zKSucn/pB7 tWOTNTKSw1xcqiZfaHS2pfgP6toBziJJ6syioFybx4VnkjCLEaVknaQtrPsVqtS/+UBr rfPQ== X-Gm-Message-State: APjAAAWRqh5dxe2UcH+pH68KoTDm/CcLingJHrAxC7jbOxMfm1M+haRB FmjTcFzA9TzEU7M2K4SYqZ7z X-Google-Smtp-Source: APXvYqzmi1IdmMHCJgeLG02tJ2axSk8U5rWDzebNs1HQFVl16EVwRIpz++mi8+H0nuTzSFvxO2E52A== X-Received: by 2002:a65:534c:: with SMTP id w12mr10249267pgr.51.1567984801761; Sun, 08 Sep 2019 16:20:01 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id z189sm13760624pfb.137.2019.09.08.16.19.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:20:01 -0700 (PDT) Date: Mon, 9 Sep 2019 09:19:55 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 5/6] ext4: introduce direct IO write path using iomap infrastructure Message-ID: <7c2f0ee02b2659d5a45f3e30dbee66b443b5ea0a.1567978633.git.mbobrowski@mbobrowski.org> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org This patch introduces a new direct IO write code path implementation that makes use of the iomap infrastructure. All direct IO write operations are now passed from the ->write_iter() callback to the new function ext4_dio_write_iter(). This function is responsible for calling into iomap infrastructure via iomap_dio_rw(). Snippets of the direct IO code from within ext4_file_write_iter(), such as checking whether the IO request is unaligned asynchronous IO, or whether it will ber overwriting allocated and initialized blocks has been moved out and into ext4_dio_write_iter(). The block mapping flags that are passed to ext4_map_blocks() from within ext4_dio_get_block() and friends have effectively been taken out and introduced within the ext4_iomap_begin(). If ext4_map_blocks() happens to have instantiated blocks beyond the i_size, then we attempt to place the inode onto the orphan list. Despite being able to perform i_size extension checking earlier on in the direct IO code path, it makes most sense to perform this bit post successful block allocation. The ->end_io() callback ext4_dio_write_end_io() is responsible for removing the inode from the orphan list and determining if we should truncate a failed write in the case of an error. We also convert a range of unwritten extents to written if IOMAP_DIO_UNWRITTEN is set and perform the necessary i_size/i_disksize extension if the iocb->ki_pos + dio->size > i_size_read(inode). In the instance of a short write, we fallback to buffered IO and complete whatever is left the 'iter'. Any blocks that may have been allocated in preparation for direct IO will be reused by buffered IO, so there's no issue with leaving allocated blocks beyond EOF. Signed-off-by: Matthew Bobrowski Reviewed-by: Ritesh Harjani Reviewed-by: Ritesh Harjani --- fs/ext4/file.c | 219 +++++++++++++++++++++++++++++++++--------------- fs/ext4/inode.c | 57 ++++++++++--- 2 files changed, 198 insertions(+), 78 deletions(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 8e586198f6e6..bf22425a6a6f 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -29,6 +29,7 @@ #include #include #include +#include #include "ext4.h" #include "ext4_jbd2.h" #include "xattr.h" @@ -217,6 +218,14 @@ static ssize_t ext4_write_checks(struct kiocb *iocb, struct iov_iter *from) if (ret <= 0) return ret; + ret = file_remove_privs(iocb->ki_filp); + if (ret) + return 0; + + ret = file_update_time(iocb->ki_filp); + if (ret) + return 0; + if (unlikely(IS_IMMUTABLE(inode))) return -EPERM; @@ -234,6 +243,34 @@ static ssize_t ext4_write_checks(struct kiocb *iocb, struct iov_iter *from) return iov_iter_count(from); } +static ssize_t ext4_buffered_write_iter(struct kiocb *iocb, + struct iov_iter *from) +{ + ssize_t ret; + struct inode *inode = file_inode(iocb->ki_filp); + + if (iocb->ki_flags & IOCB_NOWAIT) + return -EOPNOTSUPP; + + if (!inode_trylock(inode)) + inode_lock(inode); + + ret = ext4_write_checks(iocb, from); + if (ret <= 0) + goto out; + + current->backing_dev_info = inode_to_bdi(inode); + ret = generic_perform_write(iocb->ki_filp, from, iocb->ki_pos); + current->backing_dev_info = NULL; +out: + inode_unlock(inode); + if (likely(ret > 0)) { + iocb->ki_pos += ret; + ret = generic_write_sync(iocb, ret); + } + return ret; +} + static int ext4_handle_inode_extension(struct inode *inode, loff_t offset, ssize_t len, size_t count) { @@ -311,6 +348,118 @@ static int ext4_handle_failed_inode_extension(struct inode *inode, loff_t size) return ret; } +/* + * For a write that extends the inode size, ext4_dio_write_iter() will + * wait for the write to complete. Consequently, operations performed + * within this function are still covered by the inode_lock(). + */ +static int ext4_dio_write_end_io(struct kiocb *iocb, ssize_t size, int error, + unsigned int flags) +{ + int ret = 0; + loff_t offset = iocb->ki_pos; + struct inode *inode = file_inode(iocb->ki_filp); + + if (error) { + ret = ext4_handle_failed_inode_extension(inode, offset + size); + return ret ? ret : error; + } + + if (flags & IOMAP_DIO_UNWRITTEN) { + ret = ext4_convert_unwritten_extents(NULL, inode, + offset, size); + if (ret) + return ret; + } + + if (offset + size > i_size_read(inode)) { + ret = ext4_handle_inode_extension(inode, offset, size, 0); + if (ret) + return ret; + } + return ret; +} + +static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) +{ + ssize_t ret; + loff_t offset = iocb->ki_pos; + size_t count = iov_iter_count(from); + struct inode *inode = file_inode(iocb->ki_filp); + bool extend = false, overwrite = false, unaligned_aio = false; + + if (!inode_trylock(inode)) { + if (iocb->ki_flags & IOCB_NOWAIT) + return -EAGAIN; + inode_lock(inode); + } + + if (!ext4_dio_checks(inode)) { + inode_unlock(inode); + /* + * Fallback to buffered IO if the operation on the + * inode is not supported by direct IO. + */ + return ext4_buffered_write_iter(iocb, from); + } + + ret = ext4_write_checks(iocb, from); + if (ret <= 0) { + inode_unlock(inode); + return ret; + } + + /* + * Unaligned direct AIO must be serialized among each other as + * the zeroing of partial blocks of two competing unaligned + * AIOs can result in data corruption. + */ + if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) && + !is_sync_kiocb(iocb) && ext4_unaligned_aio(inode, from, offset)) { + unaligned_aio = true; + inode_dio_wait(inode); + } + + /* + * Determine whether the IO operation will overwrite allocated + * and initialized blocks. If so, check to see whether it is + * possible to take the dioread_nolock path. + */ + if (!unaligned_aio && ext4_overwrite_io(inode, offset, count) && + ext4_should_dioread_nolock(inode)) { + overwrite = true; + downgrade_write(&inode->i_rwsem); + } + + if (offset + count > i_size_read(inode) || + offset + count > EXT4_I(inode)->i_disksize) { + ext4_update_i_disksize(inode, inode->i_size); + extend = true; + } + + ret = iomap_dio_rw(iocb, from, &ext4_iomap_ops, ext4_dio_write_end_io); + + /* + * Unaligned direct AIO must be the only IO in flight or else + * any overlapping aligned IO after unaligned IO might result + * in data corruption. We also need to wait here in the case + * where the inode is being extended so that inode extension + * routines in ext4_dio_write_end_io() are covered by the + * inode_lock(). + */ + if (ret == -EIOCBQUEUED && (unaligned_aio || extend)) + inode_dio_wait(inode); + + if (overwrite) + inode_unlock_shared(inode); + else + inode_unlock(inode); + + if (ret >= 0 && iov_iter_count(from)) + return ext4_buffered_write_iter(iocb, from); + return ret; +} + #ifdef CONFIG_FS_DAX static ssize_t ext4_dax_write_iter(struct kiocb *iocb, struct iov_iter *from) @@ -325,15 +474,10 @@ ext4_dax_write_iter(struct kiocb *iocb, struct iov_iter *from) return -EAGAIN; inode_lock(inode); } + ret = ext4_write_checks(iocb, from); if (ret <= 0) goto out; - ret = file_remove_privs(iocb->ki_filp); - if (ret) - goto out; - ret = file_update_time(iocb->ki_filp); - if (ret) - goto out; offset = iocb->ki_pos; ret = dax_iomap_rw(iocb, from, &ext4_iomap_ops); @@ -359,73 +503,16 @@ static ssize_t ext4_file_write_iter(struct kiocb *iocb, struct iov_iter *from) { struct inode *inode = file_inode(iocb->ki_filp); - int o_direct = iocb->ki_flags & IOCB_DIRECT; - int unaligned_aio = 0; - int overwrite = 0; - ssize_t ret; if (unlikely(ext4_forced_shutdown(EXT4_SB(inode->i_sb)))) return -EIO; -#ifdef CONFIG_FS_DAX if (IS_DAX(inode)) return ext4_dax_write_iter(iocb, from); -#endif - if (!o_direct && (iocb->ki_flags & IOCB_NOWAIT)) - return -EOPNOTSUPP; - if (!inode_trylock(inode)) { - if (iocb->ki_flags & IOCB_NOWAIT) - return -EAGAIN; - inode_lock(inode); - } - - ret = ext4_write_checks(iocb, from); - if (ret <= 0) - goto out; - - /* - * Unaligned direct AIO must be serialized among each other as zeroing - * of partial blocks of two competing unaligned AIOs can result in data - * corruption. - */ - if (o_direct && ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) && - !is_sync_kiocb(iocb) && - ext4_unaligned_aio(inode, from, iocb->ki_pos)) { - unaligned_aio = 1; - ext4_unwritten_wait(inode); - } - - iocb->private = &overwrite; - /* Check whether we do a DIO overwrite or not */ - if (o_direct && !unaligned_aio) { - if (ext4_overwrite_io(inode, iocb->ki_pos, iov_iter_count(from))) { - if (ext4_should_dioread_nolock(inode)) - overwrite = 1; - } else if (iocb->ki_flags & IOCB_NOWAIT) { - ret = -EAGAIN; - goto out; - } - } - - ret = __generic_file_write_iter(iocb, from); - /* - * Unaligned direct AIO must be the only IO in flight. Otherwise - * overlapping aligned IO after unaligned might result in data - * corruption. - */ - if (ret == -EIOCBQUEUED && unaligned_aio) - ext4_unwritten_wait(inode); - inode_unlock(inode); - - if (ret > 0) - ret = generic_write_sync(iocb, ret); - - return ret; - -out: - inode_unlock(inode); - return ret; + if (iocb->ki_flags & IOCB_DIRECT) + return ext4_dio_write_iter(iocb, from); + return ext4_buffered_write_iter(iocb, from); } #ifdef CONFIG_FS_DAX diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index efb184928e51..f52ad3065236 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3513,11 +3513,13 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, } } } else if (flags & IOMAP_WRITE) { - int dio_credits; handle_t *handle; - int retries = 0; + int dio_credits, retries = 0, m_flags = 0; - /* Trim mapping request to maximum we can map at once for DIO */ + /* + * Trim mapping request to maximum we can map at once + * for DIO. + */ if (map.m_len > DIO_MAX_BLOCKS) map.m_len = DIO_MAX_BLOCKS; dio_credits = ext4_chunk_trans_blocks(inode, map.m_len); @@ -3533,8 +3535,30 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, if (IS_ERR(handle)) return PTR_ERR(handle); - ret = ext4_map_blocks(handle, inode, &map, - EXT4_GET_BLOCKS_CREATE_ZERO); + /* + * DAX and direct IO are the only two operations that + * are currently supported with IOMAP_WRITE. + */ + WARN_ON(!IS_DAX(inode) && !(flags & IOMAP_DIRECT)); + if (IS_DAX(inode)) + m_flags = EXT4_GET_BLOCKS_CREATE_ZERO; + else if (round_down(offset, i_blocksize(inode)) >= + i_size_read(inode)) + m_flags = EXT4_GET_BLOCKS_CREATE; + else if (ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS)) + m_flags = EXT4_GET_BLOCKS_IO_CREATE_EXT; + + ret = ext4_map_blocks(handle, inode, &map, m_flags); + + /* + * We cannot fill holes in indirect tree based inodes + * as that could expose stale data in the case of a + * crash. Use the magic error code to fallback to + * buffered IO. + */ + if (!m_flags && !ret) + ret = -ENOTBLK; + if (ret < 0) { ext4_journal_stop(handle); if (ret == -ENOSPC && @@ -3544,13 +3568,14 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, } /* - * If we added blocks beyond i_size, we need to make sure they - * will get truncated if we crash before updating i_size in - * ext4_iomap_end(). For faults we don't need to do that (and - * even cannot because for orphan list operations inode_lock is - * required) - if we happen to instantiate block beyond i_size, - * it is because we race with truncate which has already added - * the inode to the orphan list. + * If we added blocks beyond i_size, we need to make + * sure they will get truncated if we crash before + * updating the i_size. For faults we don't need to do + * that (and even cannot because for orphan list + * operations inode_lock is required) - if we happen + * to instantiate block beyond i_size, it is because + * we race with truncate which has already added the + * inode to the orphan list. */ if (!(flags & IOMAP_FAULT) && first_block + map.m_len > (i_size_read(inode) + (1 << blkbits) - 1) >> blkbits) { @@ -3612,6 +3637,14 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, static int ext4_iomap_end(struct inode *inode, loff_t offset, loff_t length, ssize_t written, unsigned flags, struct iomap *iomap) { + /* + * Check to see whether an error occurred while writing data + * out to allocated blocks. If so, return the magic error code + * so that we fallback to buffered IO and reuse the blocks + * that were allocated in preparation for the direct IO write. + */ + if (flags & IOMAP_DIRECT && written == 0) + return -ENOTBLK; return 0; } From patchwork Sun Sep 8 23:20:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Bobrowski X-Patchwork-Id: 1159566 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=linux-ext4-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=mbobrowski.org Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=mbobrowski-org.20150623.gappssmtp.com header.i=@mbobrowski-org.20150623.gappssmtp.com header.b="ffKY96pY"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 46RS274JdLz9sNx for ; Mon, 9 Sep 2019 09:20:15 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731300AbfIHXUO (ORCPT ); Sun, 8 Sep 2019 19:20:14 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:43478 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731256AbfIHXUO (ORCPT ); Sun, 8 Sep 2019 19:20:14 -0400 Received: by mail-pg1-f194.google.com with SMTP id u72so6690289pgb.10 for ; Sun, 08 Sep 2019 16:20:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mbobrowski-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Tr2zH7micHSXUeeeM7DiKZNyewx1wfbg0fdC6H8DW7M=; b=ffKY96pYazV0mdsKjw5Pvj1amVHUmcHb+AIKzJif0Uy6PLpoD3lP0YS5LAvNmo2J/2 4Q/t/sDGiAzFf5LMuFgFcESUKUM6lSaJxrVPlfCzUNwtZBMK12j46LJ6eMAVs3/i+atV DYkaYt3lAX8kW+t8gMBlXq1i3TxxDoD1P/CO7spqVd+aOoOQsCTPhSTzcnPMhUPgUSZ0 zkZ3ga174C+g+FGaQ/yxWX9AiUkVWhN7LIW9JptLtFu4lJLgfgdgICWGYWzG7XB14EUM tTm3zRNBKsSQh9RsNaC7CpW00pCThzH+s/HTBj8K9weM1aL7Y+P6dKl+aDeLWS/BcaUc ct9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Tr2zH7micHSXUeeeM7DiKZNyewx1wfbg0fdC6H8DW7M=; b=J4GmDPei9Hok63k9lPuZgU8zIwBC9tE6OFolwVuHwl7KxKTzk89ZjCig98gEt+5sgg qFE9FnyOGBwbqj7Z8P78SZv+R53ZPsAns7N/49PAtpFP8bf5/uolfwI6NRM2PVhFsYNe ggpyHeigDvP2vyD+KNI6v8vRhmyAhTYZ89svaF4+eIxQtlCpEeEA5UK5nhABVQmi9WPD 0eBzSanYxXMiPGMSCOdyzDLqH5TUt0tZHEZLml/xbpSbLXIv9dwdAeP+s9nQtvb2dqdb vpcHPh0ANagNpDOM5CQ66jKvn+5vORJNYxY5B9uy/huqPCjiSPY+iP5iozmvP6PavLGv 8AkA== X-Gm-Message-State: APjAAAWGcRaNPZdSzU71g73cYmctb0FBoVxqIM2cqNGJRchM6eMVZqUc ZgvqOLTFuBTePwXXzLyNyY56 X-Google-Smtp-Source: APXvYqxQ6oNep5VHFsKmVgQiaWDI0Ef3/kXMOzS/oR8u9ScuSWOi9AA4aagTlWHFEvSVokg66W0f1Q== X-Received: by 2002:a17:90a:aa95:: with SMTP id l21mr1941500pjq.69.1567984813165; Sun, 08 Sep 2019 16:20:13 -0700 (PDT) Received: from bobrowski ([110.232.114.101]) by smtp.gmail.com with ESMTPSA id g24sm12828119pfo.178.2019.09.08.16.20.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Sep 2019 16:20:12 -0700 (PDT) Date: Mon, 9 Sep 2019 09:20:07 +1000 From: Matthew Bobrowski To: tytso@mit.edu, jack@suse.cz, adilger.kernel@dilger.ca Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, david@fromorbit.com, hch@infradead.org, darrick.wong@oracle.com Subject: [PATCH v2 6/6] ext4: cleanup legacy buffer_head direct IO code Message-ID: <8c22105d832117969690af05d13782e69fb7619a.1567978633.git.mbobrowski@mbobrowski.org> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Remove buffer_head direct IO code that is now redundant as a result of porting across the read/write paths to use iomap infrastructure. Signed-off-by: Matthew Bobrowski Reviewed-by: Ritesh Harjani --- fs/ext4/ext4.h | 3 - fs/ext4/extents.c | 11 +- fs/ext4/file.c | 7 - fs/ext4/inode.c | 398 +--------------------------------------------- 4 files changed, 5 insertions(+), 414 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index bf660aa7a9e0..2ab91815f52d 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -1555,7 +1555,6 @@ enum { EXT4_STATE_NO_EXPAND, /* No space for expansion */ EXT4_STATE_DA_ALLOC_CLOSE, /* Alloc DA blks on close */ EXT4_STATE_EXT_MIGRATE, /* Inode is migrating */ - EXT4_STATE_DIO_UNWRITTEN, /* need convert on dio done*/ EXT4_STATE_NEWENTRY, /* File just added to dir */ EXT4_STATE_MAY_INLINE_DATA, /* may have in-inode data */ EXT4_STATE_EXT_PRECACHED, /* extents have been precached */ @@ -2522,8 +2521,6 @@ int ext4_get_block_unwritten(struct inode *inode, sector_t iblock, struct buffer_head *bh_result, int create); int ext4_get_block(struct inode *inode, sector_t iblock, struct buffer_head *bh_result, int create); -int ext4_dio_get_block(struct inode *inode, sector_t iblock, - struct buffer_head *bh_result, int create); int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, struct buffer_head *bh, int create); int ext4_walk_page_buffers(handle_t *handle, diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index 92266a2da7d6..a869e206bd81 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -1753,16 +1753,9 @@ ext4_can_extents_be_merged(struct inode *inode, struct ext4_extent *ex1, */ if (ext1_ee_len + ext2_ee_len > EXT_INIT_MAX_LEN) return 0; - /* - * The check for IO to unwritten extent is somewhat racy as we - * increment i_unwritten / set EXT4_STATE_DIO_UNWRITTEN only after - * dropping i_data_sem. But reserved blocks should save us in that - * case. - */ + if (ext4_ext_is_unwritten(ex1) && - (ext4_test_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN) || - atomic_read(&EXT4_I(inode)->i_unwritten) || - (ext1_ee_len + ext2_ee_len > EXT_UNWRITTEN_MAX_LEN))) + (ext1_ee_len + ext2_ee_len > EXT_UNWRITTEN_MAX_LEN)) return 0; #ifdef AGGRESSIVE_TEST if (ext1_ee_len >= 4) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index bf22425a6a6f..d4b9a82aed6c 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -155,13 +155,6 @@ static int ext4_release_file(struct inode *inode, struct file *filp) return 0; } -static void ext4_unwritten_wait(struct inode *inode) -{ - wait_queue_head_t *wq = ext4_ioend_wq(inode); - - wait_event(*wq, (atomic_read(&EXT4_I(inode)->i_unwritten) == 0)); -} - /* * This tests whether the IO in question is block-aligned or not. * Ext4 utilizes unwritten extents when hole-filling during direct IO, and they diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index f52ad3065236..a4f0749527c7 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -826,136 +826,6 @@ int ext4_get_block_unwritten(struct inode *inode, sector_t iblock, /* Maximum number of blocks we map for direct IO at once. */ #define DIO_MAX_BLOCKS 4096 -/* - * Get blocks function for the cases that need to start a transaction - - * generally difference cases of direct IO and DAX IO. It also handles retries - * in case of ENOSPC. - */ -static int ext4_get_block_trans(struct inode *inode, sector_t iblock, - struct buffer_head *bh_result, int flags) -{ - int dio_credits; - handle_t *handle; - int retries = 0; - int ret; - - /* Trim mapping request to maximum we can map at once for DIO */ - if (bh_result->b_size >> inode->i_blkbits > DIO_MAX_BLOCKS) - bh_result->b_size = DIO_MAX_BLOCKS << inode->i_blkbits; - dio_credits = ext4_chunk_trans_blocks(inode, - bh_result->b_size >> inode->i_blkbits); -retry: - handle = ext4_journal_start(inode, EXT4_HT_MAP_BLOCKS, dio_credits); - if (IS_ERR(handle)) - return PTR_ERR(handle); - - ret = _ext4_get_block(inode, iblock, bh_result, flags); - ext4_journal_stop(handle); - - if (ret == -ENOSPC && ext4_should_retry_alloc(inode->i_sb, &retries)) - goto retry; - return ret; -} - -/* Get block function for DIO reads and writes to inodes without extents */ -int ext4_dio_get_block(struct inode *inode, sector_t iblock, - struct buffer_head *bh, int create) -{ - /* We don't expect handle for direct IO */ - WARN_ON_ONCE(ext4_journal_current_handle()); - - if (!create) - return _ext4_get_block(inode, iblock, bh, 0); - return ext4_get_block_trans(inode, iblock, bh, EXT4_GET_BLOCKS_CREATE); -} - -/* - * Get block function for AIO DIO writes when we create unwritten extent if - * blocks are not allocated yet. The extent will be converted to written - * after IO is complete. - */ -static int ext4_dio_get_block_unwritten_async(struct inode *inode, - sector_t iblock, struct buffer_head *bh_result, int create) -{ - int ret; - - /* We don't expect handle for direct IO */ - WARN_ON_ONCE(ext4_journal_current_handle()); - - ret = ext4_get_block_trans(inode, iblock, bh_result, - EXT4_GET_BLOCKS_IO_CREATE_EXT); - - /* - * When doing DIO using unwritten extents, we need io_end to convert - * unwritten extents to written on IO completion. We allocate io_end - * once we spot unwritten extent and store it in b_private. Generic - * DIO code keeps b_private set and furthermore passes the value to - * our completion callback in 'private' argument. - */ - if (!ret && buffer_unwritten(bh_result)) { - if (!bh_result->b_private) { - ext4_io_end_t *io_end; - - io_end = ext4_init_io_end(inode, GFP_KERNEL); - if (!io_end) - return -ENOMEM; - bh_result->b_private = io_end; - ext4_set_io_unwritten_flag(inode, io_end); - } - set_buffer_defer_completion(bh_result); - } - - return ret; -} - -/* - * Get block function for non-AIO DIO writes when we create unwritten extent if - * blocks are not allocated yet. The extent will be converted to written - * after IO is complete by ext4_direct_IO_write(). - */ -static int ext4_dio_get_block_unwritten_sync(struct inode *inode, - sector_t iblock, struct buffer_head *bh_result, int create) -{ - int ret; - - /* We don't expect handle for direct IO */ - WARN_ON_ONCE(ext4_journal_current_handle()); - - ret = ext4_get_block_trans(inode, iblock, bh_result, - EXT4_GET_BLOCKS_IO_CREATE_EXT); - - /* - * Mark inode as having pending DIO writes to unwritten extents. - * ext4_direct_IO_write() checks this flag and converts extents to - * written. - */ - if (!ret && buffer_unwritten(bh_result)) - ext4_set_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN); - - return ret; -} - -static int ext4_dio_get_block_overwrite(struct inode *inode, sector_t iblock, - struct buffer_head *bh_result, int create) -{ - int ret; - - ext4_debug("ext4_dio_get_block_overwrite: inode %lu, create flag %d\n", - inode->i_ino, create); - /* We don't expect handle for direct IO */ - WARN_ON_ONCE(ext4_journal_current_handle()); - - ret = _ext4_get_block(inode, iblock, bh_result, 0); - /* - * Blocks should have been preallocated! ext4_file_write_iter() checks - * that. - */ - WARN_ON_ONCE(!buffer_mapped(bh_result) || buffer_unwritten(bh_result)); - - return ret; -} - - /* * `handle' can be NULL if create is zero */ @@ -3653,268 +3523,6 @@ const struct iomap_ops ext4_iomap_ops = { .iomap_end = ext4_iomap_end, }; -static int ext4_end_io_dio(struct kiocb *iocb, loff_t offset, - ssize_t size, void *private) -{ - ext4_io_end_t *io_end = private; - - /* if not async direct IO just return */ - if (!io_end) - return 0; - - ext_debug("ext4_end_io_dio(): io_end 0x%p " - "for inode %lu, iocb 0x%p, offset %llu, size %zd\n", - io_end, io_end->inode->i_ino, iocb, offset, size); - - /* - * Error during AIO DIO. We cannot convert unwritten extents as the - * data was not written. Just clear the unwritten flag and drop io_end. - */ - if (size <= 0) { - ext4_clear_io_unwritten_flag(io_end); - size = 0; - } - io_end->offset = offset; - io_end->size = size; - ext4_put_io_end(io_end); - - return 0; -} - -/* - * Handling of direct IO writes. - * - * For ext4 extent files, ext4 will do direct-io write even to holes, - * preallocated extents, and those write extend the file, no need to - * fall back to buffered IO. - * - * For holes, we fallocate those blocks, mark them as unwritten - * If those blocks were preallocated, we mark sure they are split, but - * still keep the range to write as unwritten. - * - * The unwritten extents will be converted to written when DIO is completed. - * For async direct IO, since the IO may still pending when return, we - * set up an end_io call back function, which will do the conversion - * when async direct IO completed. - * - * If the O_DIRECT write will extend the file then add this inode to the - * orphan list. So recovery will truncate it back to the original size - * if the machine crashes during the write. - * - */ -static ssize_t ext4_direct_IO_write(struct kiocb *iocb, struct iov_iter *iter) -{ - struct file *file = iocb->ki_filp; - struct inode *inode = file->f_mapping->host; - struct ext4_inode_info *ei = EXT4_I(inode); - ssize_t ret; - loff_t offset = iocb->ki_pos; - size_t count = iov_iter_count(iter); - int overwrite = 0; - get_block_t *get_block_func = NULL; - int dio_flags = 0; - loff_t final_size = offset + count; - int orphan = 0; - handle_t *handle; - - if (final_size > inode->i_size || final_size > ei->i_disksize) { - /* Credits for sb + inode write */ - handle = ext4_journal_start(inode, EXT4_HT_INODE, 2); - if (IS_ERR(handle)) { - ret = PTR_ERR(handle); - goto out; - } - ret = ext4_orphan_add(handle, inode); - if (ret) { - ext4_journal_stop(handle); - goto out; - } - orphan = 1; - ext4_update_i_disksize(inode, inode->i_size); - ext4_journal_stop(handle); - } - - BUG_ON(iocb->private == NULL); - - /* - * Make all waiters for direct IO properly wait also for extent - * conversion. This also disallows race between truncate() and - * overwrite DIO as i_dio_count needs to be incremented under i_mutex. - */ - inode_dio_begin(inode); - - /* If we do a overwrite dio, i_mutex locking can be released */ - overwrite = *((int *)iocb->private); - - if (overwrite) - inode_unlock(inode); - - /* - * For extent mapped files we could direct write to holes and fallocate. - * - * Allocated blocks to fill the hole are marked as unwritten to prevent - * parallel buffered read to expose the stale data before DIO complete - * the data IO. - * - * As to previously fallocated extents, ext4 get_block will just simply - * mark the buffer mapped but still keep the extents unwritten. - * - * For non AIO case, we will convert those unwritten extents to written - * after return back from blockdev_direct_IO. That way we save us from - * allocating io_end structure and also the overhead of offloading - * the extent convertion to a workqueue. - * - * For async DIO, the conversion needs to be deferred when the - * IO is completed. The ext4 end_io callback function will be - * called to take care of the conversion work. Here for async - * case, we allocate an io_end structure to hook to the iocb. - */ - iocb->private = NULL; - if (overwrite) - get_block_func = ext4_dio_get_block_overwrite; - else if (!ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS) || - round_down(offset, i_blocksize(inode)) >= inode->i_size) { - get_block_func = ext4_dio_get_block; - dio_flags = DIO_LOCKING | DIO_SKIP_HOLES; - } else if (is_sync_kiocb(iocb)) { - get_block_func = ext4_dio_get_block_unwritten_sync; - dio_flags = DIO_LOCKING; - } else { - get_block_func = ext4_dio_get_block_unwritten_async; - dio_flags = DIO_LOCKING; - } - ret = __blockdev_direct_IO(iocb, inode, inode->i_sb->s_bdev, iter, - get_block_func, ext4_end_io_dio, NULL, - dio_flags); - - if (ret > 0 && !overwrite && ext4_test_inode_state(inode, - EXT4_STATE_DIO_UNWRITTEN)) { - int err; - /* - * for non AIO case, since the IO is already - * completed, we could do the conversion right here - */ - err = ext4_convert_unwritten_extents(NULL, inode, - offset, ret); - if (err < 0) - ret = err; - ext4_clear_inode_state(inode, EXT4_STATE_DIO_UNWRITTEN); - } - - inode_dio_end(inode); - /* take i_mutex locking again if we do a ovewrite dio */ - if (overwrite) - inode_lock(inode); - - if (ret < 0 && final_size > inode->i_size) - ext4_truncate_failed_write(inode); - - /* Handle extending of i_size after direct IO write */ - if (orphan) { - int err; - - /* Credits for sb + inode write */ - handle = ext4_journal_start(inode, EXT4_HT_INODE, 2); - if (IS_ERR(handle)) { - /* - * We wrote the data but cannot extend - * i_size. Bail out. In async io case, we do - * not return error here because we have - * already submmitted the corresponding - * bio. Returning error here makes the caller - * think that this IO is done and failed - * resulting in race with bio's completion - * handler. - */ - if (!ret) - ret = PTR_ERR(handle); - if (inode->i_nlink) - ext4_orphan_del(NULL, inode); - - goto out; - } - if (inode->i_nlink) - ext4_orphan_del(handle, inode); - if (ret > 0) { - loff_t end = offset + ret; - if (end > inode->i_size || end > ei->i_disksize) { - ext4_update_i_disksize(inode, end); - if (end > inode->i_size) - i_size_write(inode, end); - /* - * We're going to return a positive `ret' - * here due to non-zero-length I/O, so there's - * no way of reporting error returns from - * ext4_mark_inode_dirty() to userspace. So - * ignore it. - */ - ext4_mark_inode_dirty(handle, inode); - } - } - err = ext4_journal_stop(handle); - if (ret == 0) - ret = err; - } -out: - return ret; -} - -static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter) -{ - struct address_space *mapping = iocb->ki_filp->f_mapping; - struct inode *inode = mapping->host; - size_t count = iov_iter_count(iter); - ssize_t ret; - - /* - * Shared inode_lock is enough for us - it protects against concurrent - * writes & truncates and since we take care of writing back page cache, - * we are protected against page writeback as well. - */ - inode_lock_shared(inode); - ret = filemap_write_and_wait_range(mapping, iocb->ki_pos, - iocb->ki_pos + count - 1); - if (ret) - goto out_unlock; - ret = __blockdev_direct_IO(iocb, inode, inode->i_sb->s_bdev, - iter, ext4_dio_get_block, NULL, NULL, 0); -out_unlock: - inode_unlock_shared(inode); - return ret; -} - -static ssize_t ext4_direct_IO(struct kiocb *iocb, struct iov_iter *iter) -{ - struct file *file = iocb->ki_filp; - struct inode *inode = file->f_mapping->host; - size_t count = iov_iter_count(iter); - loff_t offset = iocb->ki_pos; - ssize_t ret; - -#ifdef CONFIG_FS_ENCRYPTION - if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode)) - return 0; -#endif - - /* - * If we are doing data journalling we don't support O_DIRECT - */ - if (ext4_should_journal_data(inode)) - return 0; - - /* Let buffer I/O handle the inline data case. */ - if (ext4_has_inline_data(inode)) - return 0; - - trace_ext4_direct_IO_enter(inode, offset, count, iov_iter_rw(iter)); - if (iov_iter_rw(iter) == READ) - ret = ext4_direct_IO_read(iocb, iter); - else - ret = ext4_direct_IO_write(iocb, iter); - trace_ext4_direct_IO_exit(inode, offset, count, iov_iter_rw(iter), ret); - return ret; -} - /* * Pages can be marked dirty completely asynchronously from ext4's journalling * activity. By filemap_sync_pte(), try_to_unmap_one(), etc. We cannot do @@ -3952,7 +3560,7 @@ static const struct address_space_operations ext4_aops = { .bmap = ext4_bmap, .invalidatepage = ext4_invalidatepage, .releasepage = ext4_releasepage, - .direct_IO = ext4_direct_IO, + .direct_IO = noop_direct_IO, .migratepage = buffer_migrate_page, .is_partially_uptodate = block_is_partially_uptodate, .error_remove_page = generic_error_remove_page, @@ -3969,7 +3577,7 @@ static const struct address_space_operations ext4_journalled_aops = { .bmap = ext4_bmap, .invalidatepage = ext4_journalled_invalidatepage, .releasepage = ext4_releasepage, - .direct_IO = ext4_direct_IO, + .direct_IO = noop_direct_IO, .is_partially_uptodate = block_is_partially_uptodate, .error_remove_page = generic_error_remove_page, }; @@ -3985,7 +3593,7 @@ static const struct address_space_operations ext4_da_aops = { .bmap = ext4_bmap, .invalidatepage = ext4_da_invalidatepage, .releasepage = ext4_releasepage, - .direct_IO = ext4_direct_IO, + .direct_IO = noop_direct_IO, .migratepage = buffer_migrate_page, .is_partially_uptodate = block_is_partially_uptodate, .error_remove_page = generic_error_remove_page,