diff mbox

[v3,for-2.2,2/3] raw-posix: SEEK_HOLE suffices, get rid of FIEMAP

Message ID 87k32uf5zs.fsf@blackfin.pond.sub.org
State New
Headers show

Commit Message

Markus Armbruster Nov. 17, 2014, 10:58 a.m. UTC
Max Reitz <mreitz@redhat.com> writes:

> On 2014-11-17 at 11:18, Markus Armbruster wrote:
>> Commit 5500316 (May 2012) implemented raw_co_is_allocated() as
>> follows:
>>
>> 1. If defined(CONFIG_FIEMAP), use the FS_IOC_FIEMAP ioctl
>>
>> 2. Else if defined(SEEK_HOLE) && defined(SEEK_DATA), use lseek()
>>
>> 3. Else pretend there are no holes
>>
>> Later on, raw_co_is_allocated() was generalized to
>> raw_co_get_block_status().
>>
>> Commit 4f11aa8 (May 2014) changed it to try the three methods in order
>> until success, because "there may be implementations which support
>> [SEEK_HOLE/SEEK_DATA] but not [FIEMAP] (e.g., NFSv4.2) as well as vice
>> versa."
>>
>> Unfortunately, we used FIEMAP incorrectly: we lacked FIEMAP_FLAG_SYNC.
>> Commit 38c4d0a (Sep 2014) added it.  Because that's a significant
>> speed hit, the next commit 7c159037 put SEEK_HOLE/SEEK_DATA first.
>>
>> As you see, the obvious use of FIEMAP is wrong, and the correct use is
>> slow.  I guess this puts it somewhere between -7 "The obvious use is
>> wrong" and -10 "It's impossible to get right" on Rusty Russel's Hard
>> to Misuse scale[*].
>>
>> "Fortunately", the FIEMAP code is used only when
>>
>> * SEEK_HOLE/SEEK_DATA aren't defined, but CONFIG_FIEMAP is
>>
>>    Uncommon.  SEEK_HOLE had no XFS implementation between 2011 (when it
>>    was introduced for ext4 and btrfs) and 2012.
>>
>> * SEEK_HOLE/SEEK_DATA and CONFIG_FIEMAP are defined, but lseek() fails
>>
>>    Unlikely.
>>
>> Thus, the FIEMAP code executes rarely.  Makes it a nice hidey-hole for
>> bugs.  Worse, bugs hiding there can theoretically bite even on a host
>> that has SEEK_HOLE/SEEK_DATA.
>>
>> I don't want to worry about this crap, not even theoretically.  Get
>> rid of it.
>>
>> [*] http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html
>>
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>> ---
>>   block/raw-posix.c | 60 ++++---------------------------------------------------
>>   1 file changed, 4 insertions(+), 56 deletions(-)
>
> I only just now realized that you're not getting rid of FIEMAP
> completely; there's still the skip_fiemap flag in BDRVRawState. I
> don't care for now, though, thus my R-b stands.

You're right!  The appended patch should be squashed in, either on
commit, or in a respin.

Comments

Max Reitz Nov. 17, 2014, 10:59 a.m. UTC | #1
On 2014-11-17 at 11:58, Markus Armbruster wrote:
> Max Reitz <mreitz@redhat.com> writes:
>
>> On 2014-11-17 at 11:18, Markus Armbruster wrote:
>>> Commit 5500316 (May 2012) implemented raw_co_is_allocated() as
>>> follows:
>>>
>>> 1. If defined(CONFIG_FIEMAP), use the FS_IOC_FIEMAP ioctl
>>>
>>> 2. Else if defined(SEEK_HOLE) && defined(SEEK_DATA), use lseek()
>>>
>>> 3. Else pretend there are no holes
>>>
>>> Later on, raw_co_is_allocated() was generalized to
>>> raw_co_get_block_status().
>>>
>>> Commit 4f11aa8 (May 2014) changed it to try the three methods in order
>>> until success, because "there may be implementations which support
>>> [SEEK_HOLE/SEEK_DATA] but not [FIEMAP] (e.g., NFSv4.2) as well as vice
>>> versa."
>>>
>>> Unfortunately, we used FIEMAP incorrectly: we lacked FIEMAP_FLAG_SYNC.
>>> Commit 38c4d0a (Sep 2014) added it.  Because that's a significant
>>> speed hit, the next commit 7c159037 put SEEK_HOLE/SEEK_DATA first.
>>>
>>> As you see, the obvious use of FIEMAP is wrong, and the correct use is
>>> slow.  I guess this puts it somewhere between -7 "The obvious use is
>>> wrong" and -10 "It's impossible to get right" on Rusty Russel's Hard
>>> to Misuse scale[*].
>>>
>>> "Fortunately", the FIEMAP code is used only when
>>>
>>> * SEEK_HOLE/SEEK_DATA aren't defined, but CONFIG_FIEMAP is
>>>
>>>     Uncommon.  SEEK_HOLE had no XFS implementation between 2011 (when it
>>>     was introduced for ext4 and btrfs) and 2012.
>>>
>>> * SEEK_HOLE/SEEK_DATA and CONFIG_FIEMAP are defined, but lseek() fails
>>>
>>>     Unlikely.
>>>
>>> Thus, the FIEMAP code executes rarely.  Makes it a nice hidey-hole for
>>> bugs.  Worse, bugs hiding there can theoretically bite even on a host
>>> that has SEEK_HOLE/SEEK_DATA.
>>>
>>> I don't want to worry about this crap, not even theoretically.  Get
>>> rid of it.
>>>
>>> [*] http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html
>>>
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>> ---
>>>    block/raw-posix.c | 60 ++++---------------------------------------------------
>>>    1 file changed, 4 insertions(+), 56 deletions(-)
>> I only just now realized that you're not getting rid of FIEMAP
>> completely; there's still the skip_fiemap flag in BDRVRawState. I
>> don't care for now, though, thus my R-b stands.
> You're right!  The appended patch should be squashed in, either on
> commit, or in a respin.
>
>
> diff --git a/block/raw-posix.c b/block/raw-posix.c
> index 0b5d5a8..414e6d1 100644
> --- a/block/raw-posix.c
> +++ b/block/raw-posix.c
> @@ -148,9 +148,6 @@ typedef struct BDRVRawState {
>       bool has_write_zeroes:1;
>       bool discard_zeroes:1;
>       bool needs_alignment;
> -#ifdef CONFIG_FIEMAP
> -    bool skip_fiemap;
> -#endif
>   } BDRVRawState;
>   
>   typedef struct BDRVRawReopenState {

With or without thank hunk (better with):

Reviewed-by: Max Reitz <mreitz@redhat.com>
Eric Blake Nov. 17, 2014, 4:31 p.m. UTC | #2
On 11/17/2014 03:58 AM, Markus Armbruster wrote:
> Max Reitz <mreitz@redhat.com> writes:
> 
>> On 2014-11-17 at 11:18, Markus Armbruster wrote:
>>> Commit 5500316 (May 2012) implemented raw_co_is_allocated() as
>>> follows:
>>>

>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>> Reviewed-by: Eric Blake <eblake@redhat.com>
>>> ---
>>>   block/raw-posix.c | 60 ++++---------------------------------------------------
>>>   1 file changed, 4 insertions(+), 56 deletions(-)
>>
>> I only just now realized that you're not getting rid of FIEMAP
>> completely; there's still the skip_fiemap flag in BDRVRawState. I
>> don't care for now, though, thus my R-b stands.
> 
> You're right!  The appended patch should be squashed in, either on
> commit, or in a respin.

My R-b stands whether or not you squash this in; best situation would be
squashing this in during commit.

> 
> 
> diff --git a/block/raw-posix.c b/block/raw-posix.c
> index 0b5d5a8..414e6d1 100644
> --- a/block/raw-posix.c
> +++ b/block/raw-posix.c
> @@ -148,9 +148,6 @@ typedef struct BDRVRawState {
>      bool has_write_zeroes:1;
>      bool discard_zeroes:1;
>      bool needs_alignment;
> -#ifdef CONFIG_FIEMAP
> -    bool skip_fiemap;
> -#endif
>  } BDRVRawState;
>  
>  typedef struct BDRVRawReopenState {
> 
> 
>
diff mbox

Patch

diff --git a/block/raw-posix.c b/block/raw-posix.c
index 0b5d5a8..414e6d1 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -148,9 +148,6 @@  typedef struct BDRVRawState {
     bool has_write_zeroes:1;
     bool discard_zeroes:1;
     bool needs_alignment;
-#ifdef CONFIG_FIEMAP
-    bool skip_fiemap;
-#endif
 } BDRVRawState;
 
 typedef struct BDRVRawReopenState {