From patchwork Fri Feb 25 22:38:06 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chunqiang Tang X-Patchwork-Id: 84586 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [199.232.76.165]) by ozlabs.org (Postfix) with ESMTP id A535DB70ED for ; Sat, 26 Feb 2011 10:09:25 +1100 (EST) Received: from localhost ([127.0.0.1]:34707 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pt6g7-0004ow-AA for incoming@patchwork.ozlabs.org; Fri, 25 Feb 2011 18:02:27 -0500 Received: from [140.186.70.92] (port=49145 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Pt6SG-00065K-No for qemu-devel@nongnu.org; Fri, 25 Feb 2011 17:48:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Pt6S6-0004kb-39 for qemu-devel@nongnu.org; Fri, 25 Feb 2011 17:48:08 -0500 Received: from e8.ny.us.ibm.com ([32.97.182.138]:53371) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Pt6S6-0004kB-0B for qemu-devel@nongnu.org; Fri, 25 Feb 2011 17:47:58 -0500 Received: from d01dlp01.pok.ibm.com (d01dlp01.pok.ibm.com [9.56.224.56]) by e8.ny.us.ibm.com (8.14.4/8.13.1) with ESMTP id p1PITLPv000343 for ; Fri, 25 Feb 2011 13:29:21 -0500 Received: from d01relay01.pok.ibm.com (d01relay01.pok.ibm.com [9.56.227.233]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id E394A38C803B for ; Fri, 25 Feb 2011 17:47:56 -0500 (EST) Received: from d01av03.pok.ibm.com (d01av03.pok.ibm.com [9.56.224.217]) by d01relay01.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p1PMlupI401000 for ; Fri, 25 Feb 2011 17:47:56 -0500 Received: from d01av03.pok.ibm.com (loopback [127.0.0.1]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p1PMluQ1021046 for ; Fri, 25 Feb 2011 19:47:56 -0300 Received: from localhost.localdomain ([9.59.229.24]) by d01av03.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id p1PMlp9O020693; Fri, 25 Feb 2011 19:47:56 -0300 From: Chunqiang Tang To: qemu-devel@nongnu.org Date: Fri, 25 Feb 2011 17:38:06 -0500 Message-Id: <1298673486-3573-26-git-send-email-ctang@us.ibm.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1298673486-3573-1-git-send-email-ctang@us.ibm.com> References: <1298673486-3573-1-git-send-email-ctang@us.ibm.com> X-Content-Scanned: Fidelis XPS MAILER X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Received-From: 32.97.182.138 Cc: Chunqiang Tang Subject: [Qemu-devel] [PATCH 26/26] FVD: add fully automated test-fvd.sh X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch is part of the Fast Virtual Disk (FVD) proposal. See http://wiki.qemu.org/Features/FVD. test-fvd.sh drives 'qemu-io --auto' to perform fully automated testing for FVD. Signed-off-by: Chunqiang Tang --- test-fvd.sh | 161 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 161 insertions(+), 0 deletions(-) create mode 100755 test-fvd.sh diff --git a/test-fvd.sh b/test-fvd.sh new file mode 100755 index 0000000..3d67c3f --- /dev/null +++ b/test-fvd.sh @@ -0,0 +1,161 @@ +#!/bin/bash + +# Drive 'qemu-io --auto' to test the FVD image format. +# +# Copyright IBM, Corp. 2010 +# +# Authors: +# Chunqiang Tang +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING.LIB file in the top-level directory. + +if [ $USER != "root" ]; then + echo "This command must be run by root in order to mount tmpfs." + exit 1 +fi + +QEMU_DIR=. +QEMU_IMG=$QEMU_DIR/qemu-img +QEMU_IO=$QEMU_DIR/qemu-io + +if [ ! -e $QEMU_IMG ]; then + echo "$QEMU_IMG does not exist." + exit 1; +fi + +if [ ! -e $QEMU_IO ]; then + echo "$QEMU_IO does not exist." + exit 1; +fi + +DATA_DIR=/var/ramdisk +TRUTH_IMG=$DATA_DIR/truth.raw +TEST_IMG=$DATA_DIR/test.fvd +TEST_BASE=$DATA_DIR/zero-500M.raw +TEST_IMG_DATA=$DATA_DIR/test.dat +CMD_LOG=./test-fvd.log + +G1=1073741824 +MAX_MEM=536870912 +MAX_ROUND=1000000 +MAX_IO_SIZE=100000000 +fail_prob=0.1 +cancel_prob=0.1 +flush_prob_base=0.05 +aio_flush_prob_base=0.1 +seed=$RANDOM$RANDOM +count=0 + +function invoke() { + echo "$*" >> $CMD_LOG + sync + $* + ret=$? + if [ $ret -ne 0 ]; then + echo "$Exit with error code $ret: $*" + exit $ret + fi +} + +mount | grep $DATA_DIR > /dev/null +if [ $? -ne 0 ]; then + echo "Create tmpfs at $DATA_DIR to store testing images." + if [ ! -e $DATA_DIR ]; then mkdir -p $DATA_DIR ; fi + invoke "mount -t tmpfs none $DATA_DIR -o size=4G" + if [ $? -ne 0 ]; then exit 1; fi +fi + +/bin/rm -f $CMD_LOG $DATA_DIR/* +touch $CMD_LOG + +while [ -t ]; do + for block_size in 7680 512 1024 15872 65536 65024 1048576 1048064; do + for chunk_mult in 5 1 2 3 7 9 12 16 33 99 ; do + for cache in writeback writethrough ; do + #for compact_image in on off ; do + for compact_image in on ; do + for prefetch_delay in 1 0; do + for copy_on_read in on off; do + for base_img in "-b $TEST_BASE" "" ; do + chunk_size=$[$block_size * $chunk_mult] + large_io_size=$[$chunk_size * 5] + if [ $large_io_size -gt $MAX_IO_SIZE ]; then large_io_size=$MAX_IO_SIZE; fi + for io_size in $large_io_size 1048576 ; do + for use_data_file in "" "data_file=$TEST_IMG_DATA," ; do + + if [ cache == "writethrough" ]; then + JOURNAL_BUF_SIZE=0 + JOURNAL_CLEAN_BUF_PERIOD=0 + else + JOURNAL_BUF_SIZE="512 1024 65536" + JOURNAL_CLEAN_BUF_PERIOD="5000 1000 60000" + fi + + for journal_buf_size in $JOURNAL_BUF_SIZE ; do + for journal_clean_buf_period in $JOURNAL_CLEAN_BUF_PERIOD ; do + /bin/rm -rf /tmp/fvd.log* + + # FVD image is about 1G + img_size=$[(1073741824 + ($RANDOM$RANDOM$RANDOM % 104857600)) / 512 * 512] + + # base image is about 500MB + base_size=$[(536870912 + ($RANDOM$RANDOM$RANDOM % 104857600)) / 512 * 512] + + count=$[$count + 1] + echo "Round $count" >> $CMD_LOG + + invoke "/bin/rm -rf $TRUTH_IMG $TEST_IMG $TEST_BASE $TEST_IMG_DATA" + + if [ -z "$base_img" ]; then + # Use zero-filled empty images. + invoke "dd if=/dev/zero of=$TRUTH_IMG count=0 bs=1 seek=$img_size" + else + # Use images with random contents. + invoke "$QEMU_IO --auto --create=$TEST_BASE --seed=$seed --block_size=$block_size --empty_block_prob=0.2 --empty_block_chain=10 --file_size=$base_size" + invoke "cp --sparse=always $TEST_BASE $TRUTH_IMG" + invoke "dd if=/dev/zero of=$TRUTH_IMG count=0 bs=1 seek=$img_size" + fi + + if [ ! -z $use_data_file ]; then invoke "touch $TEST_IMG_DATA"; fi + + # Ensure the journal is large enough to hold at least one write. + mixed_records_per_journal_sector=119 + if [ cache == "writethrough" ]; then + journal_size_factor=1000 + else + journal_size_factor=100 + fi + journal_size=$[(((($io_size / $chunk_size ) + 1 ) / $mixed_records_per_journal_sector ) + 1) * 512 * (1 + $RANDOM$RANDOM % $journal_size_factor) ] + + invoke "$QEMU_IMG create -f fvd $base_img -ojournal_buf_size=$journal_buf_size,journal_clean_buf_period=$journal_clean_buf_period,${use_data_file}data_file_fmt=blksim,backing_fmt=blksim,compact_image=$compact_image,copy_on_read=$copy_on_read,block_size=$block_size,chunk_size=$chunk_size,journal_size=$journal_size,prefetch_start_delay=$prefetch_delay $TEST_IMG $img_size" + invoke "$QEMU_IMG update -oinit_data_region=on $TEST_IMG" + if [ $prefetch_delay -eq 1 ]; then invoke "$QEMU_IMG update -f fvd -oprefetch_over_threshold_throttle_time=0 $TEST_IMG" ; fi + + # Use no more 1GB memory. + mem=$[$io_size * 1000] + if [ $mem -gt $MAX_MEM ]; then + parallel=$[$MAX_MEM / $io_size] + else + parallel=200 + fi + parallel=$[${RANDOM}${RANDOM} % $parallel] + + flush_prob=`echo $flush_prob_base / $parallel | bc -l` + aio_flush_prob=`echo $aio_flush_prob_base / $parallel | bc -l` + + round=$[$G1 * 10 / $io_size] + if [ $round -gt $MAX_ROUND ]; then round=$MAX_ROUND; fi + + b3=$[$round * 2 / 3] + [ $b3 -eq 0 ] && b3=1 + for rep in 0 1 2 3 4 5 6 7 8 ; do + if [ $rep -eq 0 ]; then + compare_before=false + else + compare_before=true + fi + r=$[${RANDOM}${RANDOM} % $b3] + seed=$[$seed + 1] + invoke "$QEMU_IO --auto --cache=$cache --truth=$TRUTH_IMG --format=fvd --test="blksim:$TEST_IMG" --verify_write=true --parallel=$parallel --io_size=$io_size --fail_prob=$fail_prob --cancel_prob=$cancel_prob --aio_flush_prob=$aio_flush_prob --flush_prob=$flush_prob --compare_after=true --round=$r --compare_before=$compare_before --instant_qemubh=false --seed=$seed" +done; done; done; done; done; done; done; done; done; done; done; done; done