From patchwork Wed Sep 26 15:56:50 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 187116 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 8A76A2C007B for ; Thu, 27 Sep 2012 02:08:02 +1000 (EST) Received: from localhost ([::1]:56964 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGu9Y-00021S-F8 for incoming@patchwork.ozlabs.org; Wed, 26 Sep 2012 12:08:00 -0400 Received: from eggs.gnu.org ([208.118.235.92]:59588) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGu9J-0001zf-Qw for qemu-devel@nongnu.org; Wed, 26 Sep 2012 12:07:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TGu9B-0000jv-7n for qemu-devel@nongnu.org; Wed, 26 Sep 2012 12:07:45 -0400 Received: from mail-pa0-f45.google.com ([209.85.220.45]:56550) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TGu9A-0000it-TY for qemu-devel@nongnu.org; Wed, 26 Sep 2012 12:07:37 -0400 Received: by padfb10 with SMTP id fb10so570293pad.4 for ; Wed, 26 Sep 2012 09:07:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=N02P046YhVDUYgw3FCgnkL2JgI1HNkgZfk0Cx9HUYYI=; b=FL6n/a6QYEhoZ4PSQ62HiYAwVdoRvK53CKxOCFNpVaCtmNuqde5y64F2TqSb6Gj801 ZkGVpH+N2/ZeL7aTKff00iBpym4xpicmdkfIiQAp1fYZAFE+i6rCKiDzfchRExnKLCNO vP+MvobgWIgUJzV+t/MUnBW/e2TQCNLakRwJ4W5JqtgcgC+zxODewe1JhYEVePkhBgEN iMf1wZ5fAqp3CfsHbIshVWC341QmxHSmakzpzTYM2Dz8tL0M9FqqgStkeUw+QdD0LmaL VtHnYYR/XhCCTqiwWu+KezT4prqRHpCjvdeuN0z+0Pfv5SyYX7f5ab4TMbye46zZOHqp hxBg== Received: by 10.66.74.100 with SMTP id s4mr2358753pav.27.1348675213891; Wed, 26 Sep 2012 09:00:13 -0700 (PDT) Received: from yakj.usersys.redhat.com (93-34-169-1.ip50.fastwebnet.it. [93.34.169.1]) by mx.google.com with ESMTPS id i2sm2114149pav.13.2012.09.26.09.00.10 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 26 Sep 2012 09:00:12 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Wed, 26 Sep 2012 17:56:50 +0200 Message-Id: <1348675011-8794-45-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.7.12 In-Reply-To: <1348675011-8794-1-git-send-email-pbonzini@redhat.com> References: <1348675011-8794-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 209.85.220.45 Cc: kwolf@redhat.com, jcody@redhat.com Subject: [Qemu-devel] [PATCH v2 44/45] mirror: support more than one in-flight AIO operation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org With AIO support in place, we can start copying more than one chunk in parallel. This patch introduces the required infrastructure for this: the buffer is split into multiple granularity-sized chunks, and there is a free list to access them. Because of copy-on-write, a single operation may already require multiple chunks to be available on the free list. In addition, two different iterations on the HBitmap may want to copy the same cluster. We avoid this by keeping a bitmap of in-flight I/O operations, and blocking until the previous iteration completes. This should be a pretty rare occurrence, though; as long as there is no overlap the next iteration can start before the previous one finishes. Signed-off-by: Paolo Bonzini --- block/mirror.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++------- trace-events | 4 ++- 2 file modificati, 100 inserzioni(+), 13 rimozioni(-) diff --git a/block/mirror.c b/block/mirror.c index e6426bb..9545f90 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -17,7 +17,15 @@ #include "qemu/ratelimit.h" #include "bitmap.h" -#define SLICE_TIME 100000000ULL /* ns */ +#define SLICE_TIME 100000000ULL /* ns */ +#define MAX_IN_FLIGHT 16 + +/* The mirroring buffer is a list of granularity-sized chunks. + * Free chunks are organized in a list. + */ +typedef struct MirrorBuffer { + QSIMPLEQ_ENTRY(MirrorBuffer) next; +} MirrorBuffer; typedef struct MirrorBlockJob { BlockJob common; @@ -33,7 +41,10 @@ typedef struct MirrorBlockJob { unsigned long *cow_bitmap; HBitmapIter hbi; uint8_t *buf; + QSIMPLEQ_HEAD(, MirrorBuffer) buf_free; + int buf_free_count; + unsigned long *in_flight_bitmap; int in_flight; int ret; } MirrorBlockJob; @@ -41,7 +52,6 @@ typedef struct MirrorBlockJob { typedef struct MirrorOp { MirrorBlockJob *s; QEMUIOVector qiov; - struct iovec iov; int64_t sector_num; int nb_sectors; } MirrorOp; @@ -62,8 +72,22 @@ static BlockErrorAction mirror_error_action(MirrorBlockJob *s, bool read, static void mirror_iteration_done(MirrorOp *op) { MirrorBlockJob *s = op->s; + struct iovec *iov; + int64_t cluster_num; + int i, nb_chunks; s->in_flight--; + iov = op->qiov.iov; + for (i = 0; i < op->qiov.niov; i++) { + MirrorBuffer *buf = (MirrorBuffer *) iov[i].iov_base; + QSIMPLEQ_INSERT_TAIL(&s->buf_free, buf, next); + s->buf_free_count++; + } + + cluster_num = op->sector_num / s->granularity; + nb_chunks = op->nb_sectors / s->granularity; + bitmap_clear(s->in_flight_bitmap, cluster_num, nb_chunks); + trace_mirror_iteration_done(s, op->sector_num, op->nb_sectors); g_slice_free(MirrorOp, op); qemu_coroutine_enter(s->common.co, NULL); @@ -110,8 +134,8 @@ static void mirror_read_complete(void *opaque, int ret) static void coroutine_fn mirror_iteration(MirrorBlockJob *s) { BlockDriverState *source = s->common.bs; - int nb_sectors, nb_sectors_chunk; - int64_t end, sector_num, cluster_num; + int nb_sectors, nb_sectors_chunk, nb_chunks; + int64_t end, sector_num, cluster_num, next_sector, hbitmap_next_sector; MirrorOp *op; s->sector_num = hbitmap_iter_next(&s->hbi); @@ -122,6 +146,8 @@ static void coroutine_fn mirror_iteration(MirrorBlockJob *s) assert(s->sector_num >= 0); } + hbitmap_next_sector = s->sector_num; + /* If we have no backing file yet in the destination, and the cluster size * is very large, we need to do COW ourselves. The first time a cluster is * copied, copy it entirely. @@ -137,21 +163,58 @@ static void coroutine_fn mirror_iteration(MirrorBlockJob *s) bdrv_round_to_clusters(s->target, sector_num, nb_sectors_chunk, §or_num, &nb_sectors); - bitmap_set(s->cow_bitmap, sector_num / nb_sectors_chunk, - nb_sectors / nb_sectors_chunk); + + /* The rounding may make us copy sectors before the + * first dirty one. + */ + cluster_num = sector_num / nb_sectors_chunk; + } + + /* Wait for I/O to this cluster (from a previous iteration) to be done. */ + while (test_bit(cluster_num, s->in_flight_bitmap)) { + trace_mirror_yield_in_flight(s, sector_num, s->in_flight); + qemu_coroutine_yield(); } end = s->common.len >> BDRV_SECTOR_BITS; nb_sectors = MIN(nb_sectors, end - sector_num); + nb_chunks = (nb_sectors + nb_sectors_chunk - 1) / nb_sectors_chunk; + while (s->buf_free_count < nb_chunks) { + trace_mirror_yield_buf_busy(s, nb_chunks, s->in_flight); + qemu_coroutine_yield(); + } + + /* We have enough free space to copy these sectors. */ + if (s->cow_bitmap) { + bitmap_set(s->cow_bitmap, cluster_num, nb_chunks); + } /* Allocate a MirrorOp that is used as an AIO callback. */ op = g_slice_new(MirrorOp); op->s = s; - op->iov.iov_base = s->buf; - op->iov.iov_len = nb_sectors * 512; op->sector_num = sector_num; op->nb_sectors = nb_sectors; - qemu_iovec_init_external(&op->qiov, &op->iov, 1); + + /* Now make a QEMUIOVector taking enough granularity-sized chunks + * from s->buf_free. + */ + qemu_iovec_init(&op->qiov, nb_chunks); + next_sector = sector_num; + while (nb_chunks-- > 0) { + MirrorBuffer *buf = QSIMPLEQ_FIRST(&s->buf_free); + QSIMPLEQ_REMOVE_HEAD(&s->buf_free, next); + s->buf_free_count--; + qemu_iovec_add(&op->qiov, buf, s->granularity); + + /* Advance the HBitmapIter in parallel, so that we do not examine + * the same sector twice. + */ + if (next_sector > hbitmap_next_sector && bdrv_get_dirty(source, next_sector)) { + hbitmap_next_sector = hbitmap_iter_next(&s->hbi); + } + + next_sector += nb_sectors_chunk; + } bdrv_reset_dirty(source, sector_num, nb_sectors); @@ -162,6 +225,23 @@ static void coroutine_fn mirror_iteration(MirrorBlockJob *s) mirror_read_complete, op); } +static void mirror_free_init(MirrorBlockJob *s) +{ + int granularity = s->granularity; + size_t buf_size = s->buf_size; + uint8_t *buf = s->buf; + + assert(s->buf_free_count == 0); + QSIMPLEQ_INIT(&s->buf_free); + while (buf_size != 0) { + MirrorBuffer *cur = (MirrorBuffer *)buf; + QSIMPLEQ_INSERT_TAIL(&s->buf_free, cur, next); + s->buf_free_count++; + buf_size -= granularity; + buf += granularity; + } +} + static void mirror_drain(MirrorBlockJob *s) { while (s->in_flight > 0) { @@ -190,6 +270,9 @@ static void coroutine_fn mirror_run(void *opaque) return; } + length = (bdrv_getlength(bs) + s->granularity - 1) / s->granularity; + s->in_flight_bitmap = bitmap_new(length); + /* If we have no backing file yet in the destination, we cannot let * the destination do COW. Instead, we copy sectors around the * dirty data if needed. We need a bitmap to do that. @@ -200,7 +283,6 @@ static void coroutine_fn mirror_run(void *opaque) bdrv_get_info(s->target, &bdi); if (s->buf_size < bdi.cluster_size) { s->buf_size = bdi.cluster_size; - length = (bdrv_getlength(bs) + s->granularity - 1) / s->granularity; s->cow_bitmap = bitmap_new(length); } } @@ -208,6 +290,7 @@ static void coroutine_fn mirror_run(void *opaque) end = s->common.len >> BDRV_SECTOR_BITS; s->buf = qemu_blockalign(bs, s->buf_size); nb_sectors_chunk = s->granularity >> BDRV_SECTOR_BITS; + mirror_free_init(s); if (s->mode != MIRROR_SYNC_MODE_NONE) { /* First part, loop on the sectors and initialize the dirty bitmap. */ @@ -253,8 +336,9 @@ static void coroutine_fn mirror_run(void *opaque) */ if (qemu_get_clock_ns(rt_clock) - last_pause_ns < SLICE_TIME && s->common.iostatus == BLOCK_DEVICE_IO_STATUS_OK) { - if (s->in_flight > 0) { - trace_mirror_yield(s, s->in_flight, cnt); + if (s->in_flight == MAX_IN_FLIGHT || s->buf_free_count == 0 || + (cnt == 0 && s->in_flight > 0)) { + trace_mirror_yield(s, s->in_flight, s->buf_free_count, cnt); qemu_coroutine_yield(); continue; } else if (cnt != 0) { @@ -345,6 +429,7 @@ immediate_exit: assert(s->in_flight == 0); g_free(s->buf); g_free(s->cow_bitmap); + g_free(s->in_flight_bitmap); bdrv_set_dirty_tracking(bs, 0); bdrv_iostatus_disable(s->target); if (s->complete && ret == 0) { diff --git a/trace-events b/trace-events index ed65538..6521504 100644 --- a/trace-events +++ b/trace-events @@ -84,7 +84,9 @@ mirror_before_sleep(void *s, int64_t cnt, int synced) "s %p dirty count %"PRId64 mirror_one_iteration(void *s, int64_t sector_num, int nb_sectors) "s %p sector_num %"PRId64" nb_sectors %d" mirror_cow(void *s, int64_t sector_num) "s %p sector_num %"PRId64 mirror_iteration_done(void *s, int64_t sector_num, int nb_sectors) "s %p sector_num %"PRId64" nb_sectors %d" -mirror_yield(void *s, int64_t cnt, int in_flight) "s %p dirty count %"PRId64" in_flight %d" +mirror_yield(void *s, int64_t cnt, int buf_free_count, int in_flight) "s %p dirty count %"PRId64" free buffers %d in_flight %d" +mirror_yield_in_flight(void *s, int64_t sector_num, int in_flight) "s %p sector_num %"PRId64" in_flight %d" +mirror_yield_buf_busy(void *s, int nb_chunks, int in_flight) "s %p requested chunks %d in_flight %d" # blockdev.c qmp_block_job_cancel(void *job) "job %p"