Patchwork [2/4] large-fs: improve space diversification for ext4

login
register
mail settings
Submitter Dmitri Monakho
Date July 9, 2013, 11:05 a.m.
Message ID <1373367918-7516-2-git-send-email-dmonakhov@openvz.org>
Download mbox | patch
Permalink /patch/257699/
State Not Applicable
Headers show

Comments

Dmitri Monakho - July 9, 2013, 11:05 a.m.
Currently we allocated several giant files one by one until limit,
so empty space is located as one chunk which limit code-path coverage.
This patch consume all space with NUM_SPACE_FILES files (by default 1024)
each has same size, and when truncate each one by required delta.
As result we have $NUM_SPACE_FILES chunks of free blocks distributed
across whole filesystem.
This should help us to avoid regressions similar to e7c9e3e99adf6c49

Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
---
 common/rc |   40 ++++++++++++++++------------------------
 1 files changed, 16 insertions(+), 24 deletions(-)
Dave Chinner - July 11, 2013, 1:10 a.m.
On Tue, Jul 09, 2013 at 03:05:16PM +0400, Dmitry Monakhov wrote:
> Currently we allocated several giant files one by one until limit,
> so empty space is located as one chunk which limit code-path coverage.
> This patch consume all space with NUM_SPACE_FILES files (by default 1024)
> each has same size, and when truncate each one by required delta.
> As result we have $NUM_SPACE_FILES chunks of free blocks distributed
> across whole filesystem.
> This should help us to avoid regressions similar to e7c9e3e99adf6c49

Sounds like a good idea - distributing free space around the
filesystem - but why limit this to ext4?  If you turn this into a
generic "largefs fill space" function, it will work just as well
with XFS as it does for ext4, and with any other filesystem that we
want to support --largefs testing on....

I'd also add a CLI option to check to set NUM_SPACE_FILES like we
do for LARGE_SCRATCH_DEV and SCRATCH_DEV_EMPTY_SPACE. I'd probably
also call it SCRATCH_DEV_EMPTY_SPACE_FILES....

> Signed-off-by: Dmitry Monakhov <dmonakhov@openvz.org>
> ---
>  common/rc |   40 ++++++++++++++++------------------------
>  1 files changed, 16 insertions(+), 24 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index c44acea..902fc19 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -440,12 +440,17 @@ _setup_large_ext4_fs()
>  	fs_empty_space=$((50*1024*1024*1024))
>  
>  	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
> +	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
>  	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
>  	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
>  	[ $fs_empty_space -ge $fs_size ] && return 0
>  
>  	# calculate the size of the file we need to allocate.
> +
>  	space_to_consume=$(($fs_size - $fs_empty_space))
> +	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
> +	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))

spaces around "/"

> +
>  	# mount the filesystem and create 16TB - 4KB files until we consume
>  	# all the necessary space.
>  	_scratch_mount 2>&1 >$tmp_dir/mnt.err
> @@ -457,33 +462,20 @@ _setup_large_ext4_fs()
>  		return $status
>  	fi
>  	rm -f $tmp_dir/mnt.err
> -
> -	file_size=$((16*1024*1024*1024*1024 - 4096))
> -	nfiles=0
> -	while [ $space_to_consume -gt $file_size ]; do
> -
> +	mkdir $SCRATCH_MNT/.use_space
> +	# Consume all space on filesytem
> +	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 

Is that bashism supported on older versions of bash? i.e. like the
versions found on RHEL5, SLES10, etc? If not, then a simple:

	for nfiles in `seq 0 1 $nfiles_total`; do

will work just as well....

>  		xfs_io -F -f \

change that to XFS_IO_PROG and we can drop the -F there.

> -			-c "truncate $file_size" \
> -			-c "falloc -k 0 $file_size" \
> -			$SCRATCH_MNT/.use_space.$nfiles 2>&1
> -		status=$?
> -		if [ $status -ne 0 ]; then
> -			break;
> -		fi
> -
> -		space_to_consume=$(( $space_to_consume - $file_size ))
> -		nfiles=$(($nfiles + 1))
> +			-c "truncate $file_size_falloc" \
> +			-c "falloc -k 0 $file_size_falloc" \
> +			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1

Is there any need for the truncate + falloc -k? I can't remember why
I did that in the first place. Just a "falloc 0 $file_size_falloc"
shoul dbe sufficient, right?

>  	done
> -
> -	# consume the remaining space.
> -	if [ $space_to_consume -gt 0 ]; then
> +	# Truncate files to smaller size, will free chunks of space
> +	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
>  		xfs_io -F -f \

Same again for XFS_IO_PROG.

Cheers,

Dave.

Patch

diff --git a/common/rc b/common/rc
index c44acea..902fc19 100644
--- a/common/rc
+++ b/common/rc
@@ -440,12 +440,17 @@  _setup_large_ext4_fs()
 	fs_empty_space=$((50*1024*1024*1024))
 
 	[ "$LARGE_SCRATCH_DEV" != yes ] && return 0
+	[ -z "$NUM_SPACE_FILES" ] && export NUM_SPACE_FILES=1024
 	[ -z "$SCRATCH_DEV_EMPTY_SPACE" ] && SCRATCH_DEV_EMPTY_SPACE=0	
 	fs_empty_space=$((fs_empty_space + $SCRATCH_DEV_EMPTY_SPACE))
 	[ $fs_empty_space -ge $fs_size ] && return 0
 
 	# calculate the size of the file we need to allocate.
+
 	space_to_consume=$(($fs_size - $fs_empty_space))
+	file_size_falloc=$(($fs_size/$NUM_SPACE_FILES))
+	file_size_final=$(($space_to_consume/$NUM_SPACE_FILES))
+
 	# mount the filesystem and create 16TB - 4KB files until we consume
 	# all the necessary space.
 	_scratch_mount 2>&1 >$tmp_dir/mnt.err
@@ -457,33 +462,20 @@  _setup_large_ext4_fs()
 		return $status
 	fi
 	rm -f $tmp_dir/mnt.err
-
-	file_size=$((16*1024*1024*1024*1024 - 4096))
-	nfiles=0
-	while [ $space_to_consume -gt $file_size ]; do
-
+	mkdir $SCRATCH_MNT/.use_space
+	# Consume all space on filesytem
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $file_size" \
-			-c "falloc -k 0 $file_size" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-		if [ $status -ne 0 ]; then
-			break;
-		fi
-
-		space_to_consume=$(( $space_to_consume - $file_size ))
-		nfiles=$(($nfiles + 1))
+			-c "truncate $file_size_falloc" \
+			-c "falloc -k 0 $file_size_falloc" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
 	done
-
-	# consume the remaining space.
-	if [ $space_to_consume -gt 0 ]; then
+	# Truncate files to smaller size, will free chunks of space
+	for ((nfiles = 0; nfiles < nfiles_total; nfiles++)); do 
 		xfs_io -F -f \
-			-c "truncate $space_to_consume" \
-			-c "falloc -k 0 $space_to_consume" \
-			$SCRATCH_MNT/.use_space.$nfiles 2>&1
-		status=$?
-	fi
-	export NUM_SPACE_FILES=$nfiles
+			-c "truncate $file_size_final" \
+			$SCRATCH_MNT/.use_space/use_space.$nfiles 2>&1
+	done
 
 	umount $SCRATCH_MNT
 	if [ $status -ne 0 ]; then