From patchwork Fri Feb 15 17:46:54 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 220799 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 373F22C007C for ; Sat, 16 Feb 2013 05:31:42 +1100 (EST) Received: from localhost ([::1]:54368 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U6POy-0007Ip-Sy for incoming@patchwork.ozlabs.org; Fri, 15 Feb 2013 12:48:48 -0500 Received: from eggs.gnu.org ([208.118.235.92]:51269) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U6POd-0006mq-E6 for qemu-devel@nongnu.org; Fri, 15 Feb 2013 12:48:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1U6POV-0004OR-Bm for qemu-devel@nongnu.org; Fri, 15 Feb 2013 12:48:27 -0500 Received: from mail-ve0-f170.google.com ([209.85.128.170]:34746) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U6POV-0004MP-3r for qemu-devel@nongnu.org; Fri, 15 Feb 2013 12:48:19 -0500 Received: by mail-ve0-f170.google.com with SMTP id 14so3298060vea.15 for ; Fri, 15 Feb 2013 09:48:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:from:to:cc:subject:date:message-id:x-mailer :in-reply-to:references; bh=K0t3Ap56F8SS3vhgI0SdAiTRa9P6f6y0LiytnJLnhck=; b=t0TOZId5mUBMYTF2rswOFdFjn8l2nuA/IvAVQd4pJM7DnKD2zy4GAZAVDdwJRDHaBc KA1qi/goNyKr7IIYyKFlLfIcOilFjKEQhRINzYO3ez6aCU8L7pzwlhCmLwc3aqsx2wly nTA6lPmj0aIJnZEeJM8MUKNdXY8cqDYyCmgmkSPPZNqhyWDT4dPTPQVYRi/dm09o+kJt hOfm4xSRBJq4Si5v3V/n6Sq4i2BOfm7wIejbVjNK3qIwG+2wmOS0YqSt3nM8ve1g0Q8Q 00GOB/i/PNPcT+0V0r3X+ZKk6mfULba3gIcCKwlLlssCZRN7/Te1oPmpvOIAv3uAu+BW quhQ== X-Received: by 10.52.94.71 with SMTP id da7mr3872275vdb.13.1360950490985; Fri, 15 Feb 2013 09:48:10 -0800 (PST) Received: from yakj.usersys.redhat.com (93-34-179-137.ip50.fastwebnet.it. [93.34.179.137]) by mx.google.com with ESMTPS id yu12sm65402142vec.6.2013.02.15.09.48.08 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 15 Feb 2013 09:48:10 -0800 (PST) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Fri, 15 Feb 2013 18:46:54 +0100 Message-Id: <1360950433-17106-23-git-send-email-pbonzini@redhat.com> X-Mailer: git-send-email 1.8.1.2 In-Reply-To: <1360950433-17106-1-git-send-email-pbonzini@redhat.com> References: <1360950433-17106-1-git-send-email-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 209.85.128.170 Cc: owasserm@redhat.com, chegu_vinod@hp.com, quintela@redhat.com Subject: [Qemu-devel] [PATCH 22/41] migration: yay, buffering is gone X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Buffering was needed because blocking writes could take a long time and starve other threads seeking to grab the big QEMU mutex. Now that all writes (except within _complete callbacks) are done outside the big QEMU mutex, we do not need buffering at all. Signed-off-by: Paolo Bonzini Reviewed-by: Orit Wasserman Reviewed-by: Juan Quintela --- include/migration/migration.h | 3 -- migration.c | 78 ++++++++++------------------------------ savevm.c | 1 + 3 files changed, 21 insertions(+), 61 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index d78bbbb..172ef95 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -34,9 +34,6 @@ struct MigrationState int64_t bandwidth_limit; size_t bytes_xfer; size_t xfer_limit; - uint8_t *buffer; - size_t buffer_size; - size_t buffer_capacity; QemuThread thread; QEMUBH *cleanup_bh; diff --git a/migration.c b/migration.c index d6a7dff..1f6fbdc 100644 --- a/migration.c +++ b/migration.c @@ -503,73 +503,41 @@ int64_t migrate_xbzrle_cache_size(void) /* migration thread support */ - -static void buffered_flush(MigrationState *s) -{ - size_t offset = 0; - ssize_t ret = 0; - - DPRINTF("flushing %zu byte(s) of data\n", s->buffer_size); - - if (qemu_file_get_error(s->file)) { - s->buffer_size = 0; - return; - } - qemu_fflush(s->file); - - while (s->bytes_xfer < s->xfer_limit && offset < s->buffer_size) { - size_t to_send = MIN(s->buffer_size - offset, s->xfer_limit - s->bytes_xfer); - ret = migrate_fd_put_buffer(s, s->buffer + offset, to_send); - if (ret <= 0) { - DPRINTF("error flushing data, %zd\n", ret); - break; - } else { - DPRINTF("flushed %zd byte(s)\n", ret); - offset += ret; - s->bytes_xfer += ret; - } - } - - DPRINTF("flushed %zu of %zu byte(s)\n", offset, s->buffer_size); - memmove(s->buffer, s->buffer + offset, s->buffer_size - offset); - s->buffer_size -= offset; - - if (ret < 0) { - qemu_file_set_error(s->file, ret); - } -} - static int buffered_put_buffer(void *opaque, const uint8_t *buf, int64_t pos, int size) { MigrationState *s = opaque; - ssize_t error; + ssize_t ret; + size_t sent; DPRINTF("putting %d bytes at %" PRId64 "\n", size, pos); - error = qemu_file_get_error(s->file); - if (error) { + ret = qemu_file_get_error(s->file); + if (ret) { DPRINTF("flush when error, bailing: %s\n", strerror(-error)); - return error; + return ret; } if (size <= 0) { return size; } - if (size > (s->buffer_capacity - s->buffer_size)) { - DPRINTF("increasing buffer capacity from %zu by %zu\n", - s->buffer_capacity, size + 1024); - - s->buffer_capacity += size + 1024; - - s->buffer = g_realloc(s->buffer, s->buffer_capacity); + sent = 0; + while (size) { + ret = migrate_fd_put_buffer(s, buf, size); + if (ret <= 0) { + DPRINTF("error flushing data, %zd\n", ret); + return ret; + } else { + DPRINTF("flushed %zd byte(s)\n", ret); + sent += ret; + buf += ret; + size -= ret; + s->bytes_xfer += ret; + } } - memcpy(s->buffer + s->buffer_size, buf, size); - s->buffer_size += size; - - return size; + return sent; } static int buffered_close(void *opaque) @@ -691,10 +659,9 @@ static void *buffered_file_thread(void *opaque) /* usleep expects microseconds */ g_usleep((initial_time + BUFFER_DELAY - current_time)*1000); } - buffered_flush(s); if (qemu_file_get_error(s->file)) { __sync_val_compare_and_swap(&s->state, MIG_STATE_ACTIVE, MIG_STATE_ERROR); - } else if (last_round && s->buffer_size == 0) { + } else if (last_round) { __sync_val_compare_and_swap(&s->state, MIG_STATE_ACTIVE, MIG_STATE_COMPLETED); } } @@ -714,7 +681,6 @@ static void *buffered_file_thread(void *opaque) qemu_bh_schedule(s->cleanup_bh); qemu_mutex_unlock_iothread(); - g_free(s->buffer); return NULL; } @@ -731,10 +697,6 @@ void migrate_fd_connect(MigrationState *s) { s->state = MIG_STATE_ACTIVE; s->bytes_xfer = 0; - s->buffer = NULL; - s->buffer_size = 0; - s->buffer_capacity = 0; - s->xfer_limit = s->bandwidth_limit / XFER_LIMIT_RATIO; s->cleanup_bh = qemu_bh_new(migrate_fd_cleanup, s); diff --git a/savevm.c b/savevm.c index 7c7774e..ce10295 100644 --- a/savevm.c +++ b/savevm.c @@ -1724,6 +1724,7 @@ void qemu_savevm_state_complete(QEMUFile *f) } qemu_put_byte(f, QEMU_VM_EOF); + qemu_fflush(f); } uint64_t qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size)