[v2] xfstests: add a new test case for ext4 indirect-based file

From: Zheng Liu <wenqing.lz@taobao.com>

After applied this commit (864688d3), xfstests #255 will not test a
file system that cannot support fallocate(2), such as a indirect-based
file in ext4.  So we need to add a new generic test case to test it.

The difference between #255 and this test case is only to use pwrite to
allocate blocks.  Other filesystems should survive in this test case.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
---
v2 <- v1:
 * rebase against the latest master branch of xfstests

Here [1] is the original patch set.  It has been splitted into two parts.
In ohter parts, it will tries to improve seek data/hole test case.

1. http://www.spinics.net/lists/xfs/msg16234.html

 314     |  68 ++++++++++++++
 314.out | 307 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 group   |   1 +
 3 files changed, 376 insertions(+)
 create mode 100755 314
 create mode 100644 314.out

On 3/19/13 3:53 AM, Zheng Liu wrote:
> From: Zheng Liu <wenqing.lz@taobao.com>
> 
> After applied this commit (864688d3), xfstests #255 will not test a
> file system that cannot support fallocate(2), such as a indirect-based
> file in ext4.  So we need to add a new generic test case to test it.
> 
> The difference between #255 and this test case is only to use pwrite to
> allocate blocks.  Other filesystems should survive in this test case.
> 
> Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
> ---
> v2 <- v1:
>  * rebase against the latest master branch of xfstests
> 
> Here [1] is the original patch set.  It has been splitted into two parts.
> In ohter parts, it will tries to improve seek data/hole test case.
> 
> 1. http://www.spinics.net/lists/xfs/msg16234.html
> 
>  314     |  68 ++++++++++++++
>  314.out | 307 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  group   |   1 +
>  3 files changed, 376 insertions(+)
>  create mode 100755 314
>  create mode 100644 314.out
> 
> diff --git a/314 b/314
> new file mode 100755
> index 0000000..9a75857
> --- /dev/null
> +++ b/314
> @@ -0,0 +1,68 @@
> +#! /bin/bash
> +# FS QA Test No. 314
> +#
> +# Test Generic fallocate hole punching w/o unwritten extent
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2013 Alibaba Group.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=wenqing.lz@taobao.com
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +    cd /
> +    rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +. ./common.punch
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os IRIX Linux

This is not supported on IRIX, I think.

> +
> +_require_xfs_io_falloc_punch
> +_require_xfs_io_fiemap
> +
> +testfile=$TEST_DIR/314.$$

$seq.$$

(looks like this problem snuck in on test 255, first)

> +
> +# Standard punch hole tests
> +_test_generic_punch pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> +
> +# Delayed allocation punch hole tests
> +_test_generic_punch -d pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> +
> +# Multi hole punch tests
> +_test_generic_punch -k pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> +
> +# Delayed allocation multi punch hole tests
> +_test_generic_punch -d -k pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/314.out b/314.out
> new file mode 100644
> index 0000000..2eb1f27
> --- /dev/null
> +++ b/314.out
> @@ -0,0 +1,307 @@
> +QA output created by 314
> +	1. into a hole
> +daa100df6e6711906b61c9ab5aa16032
> +	2. into allocated space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	3. into unwritten space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	4. hole -> data
> +0: [0..23]: hole
> +1: [24..31]: extent
> +2: [32..39]: hole
> +cc63069677939f69a6e8f68cae6a6dac
> +	5. hole -> unwritten
> +0: [0..23]: hole
> +1: [24..31]: extent
> +2: [32..39]: hole
> +cc63069677939f69a6e8f68cae6a6dac
> +	6. data -> hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +1b3779878366498b28c702ef88c4a773
> +	7. data -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..31]: extent
> +3: [32..39]: hole
> +5309cc2f1fc5a7f94cf52915caef35bd
> +	8. unwritten -> hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +1b3779878366498b28c702ef88c4a773
> +	9. unwritten -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..31]: extent
> +3: [32..39]: hole
> +5309cc2f1fc5a7f94cf52915caef35bd
> +	10. hole -> data -> hole
> +daa100df6e6711906b61c9ab5aa16032
> +	11. data -> hole -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	12. unwritten -> data -> unwritten

It's a little odd that the output contains "unwritten" when this test
is explicitly for testing *without* unwritten extents.  Should this be
cleaned up a little in common.punch, maybe?

> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	13. data -> unwritten -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	14. data -> hole @ EOF
> +0: [0..23]: extent
> +1: [24..39]: hole
> +e1f024eedd27ea6b1c3e9b841c850404
> +	15. data -> hole @ 0
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	16. data -> cache cold ->hole
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	17. data -> hole in single block file
> +0: [0..7]: extent
> +13535fd4d496bf0b74bb2335aa4d1b31
> +	1. into a hole
> +daa100df6e6711906b61c9ab5aa16032
> +	2. into allocated space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	3. into unwritten space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	4. hole -> data
> +0: [0..23]: hole
> +1: [24..31]: extent
> +2: [32..39]: hole
> +cc63069677939f69a6e8f68cae6a6dac
> +	5. hole -> unwritten
> +0: [0..23]: hole
> +1: [24..31]: extent
> +2: [32..39]: hole
> +cc63069677939f69a6e8f68cae6a6dac
> +	6. data -> hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +1b3779878366498b28c702ef88c4a773
> +	7. data -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..31]: extent
> +3: [32..39]: hole
> +5309cc2f1fc5a7f94cf52915caef35bd
> +	8. unwritten -> hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +1b3779878366498b28c702ef88c4a773
> +	9. unwritten -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..31]: extent
> +3: [32..39]: hole
> +5309cc2f1fc5a7f94cf52915caef35bd
> +	10. hole -> data -> hole
> +daa100df6e6711906b61c9ab5aa16032
> +	11. data -> hole -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	12. unwritten -> data -> unwritten
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	13. data -> unwritten -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	14. data -> hole @ EOF
> +0: [0..23]: extent
> +1: [24..39]: hole
> +e1f024eedd27ea6b1c3e9b841c850404
> +	15. data -> hole @ 0
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	16. data -> cache cold ->hole
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	17. data -> hole in single block file
> +0: [0..7]: extent
> +13535fd4d496bf0b74bb2335aa4d1b31
> +	1. into a hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +5a58e46082be047d0f13bee7974015b9
> +	2. into allocated space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	3. into unwritten space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	4. hole -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	5. hole -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	6. data -> hole
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	7. data -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	8. unwritten -> hole
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	9. unwritten -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	10. hole -> data -> hole
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	11. data -> hole -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	12. unwritten -> data -> unwritten
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	13. data -> unwritten -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	14. data -> hole @ EOF
> +0: [0..23]: extent
> +1: [24..39]: hole
> +e1f024eedd27ea6b1c3e9b841c850404
> +	15. data -> hole @ 0
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	16. data -> cache cold ->hole
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	17. data -> hole in single block file
> +0: [0..7]: extent
> +13535fd4d496bf0b74bb2335aa4d1b31
> +	1. into a hole
> +0: [0..7]: extent
> +1: [8..39]: hole
> +5a58e46082be047d0f13bee7974015b9
> +	2. into allocated space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	3. into unwritten space
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	4. hole -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	5. hole -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	6. data -> hole
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	7. data -> unwritten
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	8. unwritten -> hole
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	9. unwritten -> data
> +0: [0..7]: extent
> +1: [8..23]: hole
> +2: [24..39]: extent
> +cc58a7417c2d7763adc45b6fcd3fa024
> +	10. hole -> data -> hole
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	11. data -> hole -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	12. unwritten -> data -> unwritten
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	13. data -> unwritten -> data
> +0: [0..7]: extent
> +1: [8..31]: hole
> +2: [32..39]: extent
> +f6aeca13ec49e5b266cd1c913cd726e3
> +	14. data -> hole @ EOF
> +0: [0..23]: extent
> +1: [24..39]: hole
> +e1f024eedd27ea6b1c3e9b841c850404
> +	15. data -> hole @ 0
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	16. data -> cache cold ->hole
> +0: [0..15]: hole
> +1: [16..39]: extent
> +eecb7aa303d121835de05028751d301c
> +	17. data -> hole in single block file
> +0: [0..7]: extent
> +13535fd4d496bf0b74bb2335aa4d1b31
> diff --git a/group b/group
> index a11e832..36f63ab 100644
> --- a/group
> +++ b/group
> @@ -431,3 +431,4 @@ stress
>  305 aio dangerous enospc rw stress
>  306 auto quick rw
>  313 auto quick
> +314 auto quick prealloc

I wonder if we should create a "punch" group ...

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 3/20/13 12:45 AM, Zheng Liu wrote:
>>> +f6aeca13ec49e5b266cd1c913cd726e3
>>> > > +	12. unwritten -> data -> unwritten
>> > 
>> > It's a little odd that the output contains "unwritten" when this test
>> > is explicitly for testing *without* unwritten extents.  Should this be
>> > cleaned up a little in common.punch, maybe?
> I will try to define a new function called _test_indirect_punch() to
> test punching hole without unwritten extent.

It's just the helper which prints "unwritten" regardless of what
is passed as "$alloc_cmd" to _test_generic_punch, right... so there's
nothing wrong with the test, really - it's just odd output.

I'm not sure it's worth a big copy & paste just to change
the output text, but if you can think of something simple to clean
it up, it might be worth it.

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Tue, Mar 19, 2013 at 11:29:27AM -0500, Eric Sandeen wrote:
[snip]
> > +# real QA test starts here
> > +_supported_fs generic
> > +_supported_os IRIX Linux
> 
> This is not supported on IRIX, I think.

Thanks, It will be fixed.

> 
> > +
> > +_require_xfs_io_falloc_punch
> > +_require_xfs_io_fiemap
> > +
> > +testfile=$TEST_DIR/314.$$
> 
> $seq.$$
> 
> (looks like this problem snuck in on test 255, first)

Yes, I will fix it.

> 
> > +
> > +# Standard punch hole tests
> > +_test_generic_punch pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> > +
> > +# Delayed allocation punch hole tests
> > +_test_generic_punch -d pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> > +
> > +# Multi hole punch tests
> > +_test_generic_punch -k pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> > +
> > +# Delayed allocation multi punch hole tests
> > +_test_generic_punch -d -k pwrite fpunch fpunch fiemap _filter_hole_fiemap $testfile -F
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/314.out b/314.out
> > new file mode 100644
> > index 0000000..2eb1f27
> > --- /dev/null
> > +++ b/314.out
> > @@ -0,0 +1,307 @@
> > +QA output created by 314
> > +	1. into a hole
> > +daa100df6e6711906b61c9ab5aa16032
> > +	2. into allocated space
> > +0: [0..7]: extent
> > +1: [8..23]: hole
> > +2: [24..39]: extent
> > +cc58a7417c2d7763adc45b6fcd3fa024
> > +	3. into unwritten space
> > +0: [0..7]: extent
> > +1: [8..23]: hole
> > +2: [24..39]: extent
> > +cc58a7417c2d7763adc45b6fcd3fa024
> > +	4. hole -> data
> > +0: [0..23]: hole
> > +1: [24..31]: extent
> > +2: [32..39]: hole
> > +cc63069677939f69a6e8f68cae6a6dac
> > +	5. hole -> unwritten
> > +0: [0..23]: hole
> > +1: [24..31]: extent
> > +2: [32..39]: hole
> > +cc63069677939f69a6e8f68cae6a6dac
> > +	6. data -> hole
> > +0: [0..7]: extent
> > +1: [8..39]: hole
> > +1b3779878366498b28c702ef88c4a773
> > +	7. data -> unwritten
> > +0: [0..7]: extent
> > +1: [8..23]: hole
> > +2: [24..31]: extent
> > +3: [32..39]: hole
> > +5309cc2f1fc5a7f94cf52915caef35bd
> > +	8. unwritten -> hole
> > +0: [0..7]: extent
> > +1: [8..39]: hole
> > +1b3779878366498b28c702ef88c4a773
> > +	9. unwritten -> data
> > +0: [0..7]: extent
> > +1: [8..23]: hole
> > +2: [24..31]: extent
> > +3: [32..39]: hole
> > +5309cc2f1fc5a7f94cf52915caef35bd
> > +	10. hole -> data -> hole
> > +daa100df6e6711906b61c9ab5aa16032
> > +	11. data -> hole -> data
> > +0: [0..7]: extent
> > +1: [8..31]: hole
> > +2: [32..39]: extent
> > +f6aeca13ec49e5b266cd1c913cd726e3
> > +	12. unwritten -> data -> unwritten
> 
> It's a little odd that the output contains "unwritten" when this test
> is explicitly for testing *without* unwritten extents.  Should this be
> cleaned up a little in common.punch, maybe?

I will try to define a new function called _test_indirect_punch() to
test punching hole without unwritten extent.

[snip]
> > diff --git a/group b/group
> > index a11e832..36f63ab 100644
> > --- a/group
> > +++ b/group
> > @@ -431,3 +431,4 @@ stress
> >  305 aio dangerous enospc rw stress
> >  306 auto quick rw
> >  313 auto quick
> > +314 auto quick prealloc
> 
> I wonder if we should create a "punch" group ...
> 

Thanks for the suggestions.

Regards,
                                                - Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Wed, Mar 20, 2013 at 12:37:42AM -0500, Eric Sandeen wrote:
> On 3/20/13 12:45 AM, Zheng Liu wrote:
> >>> +f6aeca13ec49e5b266cd1c913cd726e3
> >>> > > +	12. unwritten -> data -> unwritten
> >> > 
> >> > It's a little odd that the output contains "unwritten" when this test
> >> > is explicitly for testing *without* unwritten extents.  Should this be
> >> > cleaned up a little in common.punch, maybe?
> > I will try to define a new function called _test_indirect_punch() to
> > test punching hole without unwritten extent.
> 
> It's just the helper which prints "unwritten" regardless of what
> is passed as "$alloc_cmd" to _test_generic_punch, right... so there's
> nothing wrong with the test, really - it's just odd output.
> 
> I'm not sure it's worth a big copy & paste just to change
> the output text, but if you can think of something simple to clean
> it up, it might be worth it.

No, definitely not worth duplicating code. In fact, seeing as it's
just informational output, I'd be inclined to ignore it as it's
never going to cause the test to fail.

This is one of the reasons I tend not to like these "pretty output"
lines in the actual test output. If you need more verbose
information for debugging, that's what $seq.full is for....

Cheers,

Dave.