From patchwork Thu Nov 25 06:06:57 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yoshiaki Tamura X-Patchwork-Id: 72996 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [199.232.76.165]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 1739EB70AA for ; Thu, 25 Nov 2010 18:09:43 +1100 (EST) Received: from localhost ([127.0.0.1]:54034 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PLVxc-0005VI-8E for incoming@patchwork.ozlabs.org; Thu, 25 Nov 2010 02:09:40 -0500 Received: from [140.186.70.92] (port=32805 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PLVh8-0007aA-FW for qemu-devel@nongnu.org; Thu, 25 Nov 2010 01:52:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PLVh0-0007uq-NK for qemu-devel@nongnu.org; Thu, 25 Nov 2010 01:52:38 -0500 Received: from sh.osrg.net ([192.16.179.4]:33026) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PLVh0-0007sc-37 for qemu-devel@nongnu.org; Thu, 25 Nov 2010 01:52:30 -0500 Received: from fs.osrg.net (postfix@fs.osrg.net [10.0.0.12]) by sh.osrg.net (8.14.3/8.14.3/OSRG-NET) with ESMTP id oAP6q2SB003253; Thu, 25 Nov 2010 15:52:03 +0900 Received: from localhost (hype-nh0.osrg.net [10.72.1.48]) by fs.osrg.net (Postfix) with ESMTP id 8D1473E0435; Thu, 25 Nov 2010 15:51:57 +0900 (JST) From: Yoshiaki Tamura To: kvm@vger.kernel.org, qemu-devel@nongnu.org Date: Thu, 25 Nov 2010 15:06:57 +0900 Message-Id: <1290665220-26478-19-git-send-email-tamura.yoshiaki@lab.ntt.co.jp> X-Mailer: git-send-email 1.7.1.2 In-Reply-To: <1290665220-26478-1-git-send-email-tamura.yoshiaki@lab.ntt.co.jp> References: <1290665220-26478-1-git-send-email-tamura.yoshiaki@lab.ntt.co.jp> X-Dispatcher: imput version 20070423(IM149) Lines: 340 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (sh.osrg.net [192.16.179.4]); Thu, 25 Nov 2010 15:52:04 +0900 (JST) X-Virus-Scanned: clamav-milter 0.96.4 at sh X-Virus-Status: Clean X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) Cc: aliguori@us.ibm.com, mtosatti@redhat.com, ananth@in.ibm.com, ohmura.kei@lab.ntt.co.jp, dlaor@redhat.com, vatsa@linux.vnet.ibm.com, Yoshiaki Tamura , avi@redhat.com, psuriset@linux.vnet.ibm.com, stefanha@linux.vnet.ibm.com Subject: [Qemu-devel] [PATCH 18/21] migration: introduce migrate_ft_trans_{put, get}_ready(), and modify migrate_fd_put_ready() when ft_mode is on. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Introduce migrate_ft_trans_put_ready() which kicks the FT transaction cycle. When ft_mode is on, migrate_fd_put_ready() would open ft_trans_file and turn on event_tap. To end or cancel FT transaction, ft_mode and event_tap is turned off. migrate_ft_trans_get_ready() is called to receive ack from the receiver. Signed-off-by: Yoshiaki Tamura --- migration.c | 256 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- migration.h | 2 +- 2 files changed, 249 insertions(+), 9 deletions(-) diff --git a/migration.c b/migration.c index 40e4945..c03d660 100644 --- a/migration.c +++ b/migration.c @@ -21,6 +21,7 @@ #include "qemu_socket.h" #include "block-migration.h" #include "qemu-objects.h" +#include "event-tap.h" //#define DEBUG_MIGRATION @@ -307,6 +308,20 @@ void migrate_fd_put_notify(void *opaque) qemu_file_put_notify(s->file); } +static void migrate_fd_get_notify(void *opaque) +{ + FdMigrationState *s = opaque; + + qemu_set_fd_handler2(s->fd, NULL, NULL, NULL, NULL); + qemu_file_get_notify(s->file); + if (qemu_file_has_error(s->file)) { + ft_mode = FT_ERROR; + qemu_savevm_state_cancel(s->mon, s->file); + migrate_fd_error(s); + event_tap_unregister(); + } +} + ssize_t migrate_fd_put_buffer(void *opaque, const void *data, size_t size) { FdMigrationState *s = opaque; @@ -331,15 +346,20 @@ ssize_t migrate_fd_put_buffer(void *opaque, const void *data, size_t size) return ret; } -int migrate_fd_get_buffer(void *opaque, uint8_t *data, int64_t pos, int size) +int migrate_fd_get_buffer(void *opaque, uint8_t *data, int64_t pos, size_t size) { FdMigrationState *s = opaque; - ssize_t ret; + int ret; ret = s->read(s, data, size); - if (ret == -1) + if (ret == -1) { ret = -(s->get_error(s)); - + } + + if (ret == -EAGAIN) { + qemu_set_fd_handler2(s->fd, NULL, migrate_fd_get_notify, NULL, s); + } + return ret; } @@ -366,6 +386,195 @@ void migrate_fd_connect(FdMigrationState *s) migrate_fd_put_ready(s); } +static int migrate_ft_trans_commit(void *opaque) +{ + FdMigrationState *s = opaque; + int ret = -1; + + if (ft_mode != FT_TRANSACTION_COMMIT && ft_mode != FT_TRANSACTION_ATOMIC) { + fprintf(stderr, + "migrate_ft_trans_commit: invalid ft_mode %d\n", ft_mode); + goto out; + } + + do { + if (ft_mode == FT_TRANSACTION_ATOMIC) { + if (qemu_ft_trans_begin(s->file) < 0) { + fprintf(stderr, "qemu_ft_trans_begin failed\n"); + goto out; + } + + if ((ret = qemu_savevm_trans_begin(s->mon, s->file, 0)) < 0) { + fprintf(stderr, "qemu_savevm_trans_begin failed\n"); + goto out; + } + + ft_mode = FT_TRANSACTION_COMMIT; + if (ret) { + /* don't proceed until if fd isn't ready */ + goto out; + } + } + + /* make the VM state consistent by flushing outstanding events */ + vm_stop(0); + qemu_aio_flush(); + bdrv_flush_all(); + + if ((ret = qemu_savevm_trans_complete(s->mon, s->file)) < 0) { + fprintf(stderr, "qemu_savevm_trans_complete failed\n"); + goto out; + } + + if (ret) { + /* don't proceed until if fd isn't ready */ + ret = 1; + goto out; + } + + if ((ret = qemu_ft_trans_commit(s->file)) < 0) { + fprintf(stderr, "qemu_ft_trans_commit failed\n"); + goto out; + } + + if (ret) { + ft_mode = FT_TRANSACTION_RECV; + ret = 1; + goto out; + } + + /* flush and check if events are remaining */ + if ((ret = event_tap_flush_one()) < 0) { + fprintf(stderr, "event_tap_flush_one failed\n"); + goto out; + } + + ft_mode = ret ? FT_TRANSACTION_BEGIN : FT_TRANSACTION_ATOMIC; + } while (ft_mode != FT_TRANSACTION_BEGIN); + + vm_start(); + ret = 0; + +out: + return ret; +} + +static int migrate_ft_trans_get_ready(void *opaque) +{ + FdMigrationState *s = opaque; + int ret = -1; + + if (ft_mode != FT_TRANSACTION_RECV) { + fprintf(stderr, + "migrate_ft_trans_get_ready: invalid ft_mode %d\n", ft_mode); + goto error_out; + } + + /* flush and check if events are remaining */ + if ((ret = event_tap_flush_one()) < 0) { + fprintf(stderr, "event_tap_flush_one failed\n"); + goto error_out; + } + + if (ret) { + ft_mode = FT_TRANSACTION_BEGIN; + } else { + ft_mode = FT_TRANSACTION_ATOMIC; + if ((ret = migrate_ft_trans_commit(s)) < 0) { + goto error_out; + } + if (ret) { + goto out; + } + } + + vm_start(); + ret = 0; + goto out; + +error_out: + ft_mode = FT_ERROR; + qemu_savevm_state_cancel(s->mon, s->file); + migrate_fd_error(s); + event_tap_unregister(); + +out: + return ret; +} + +static int migrate_ft_trans_put_ready(void) +{ + FdMigrationState *s = migrate_to_fms(current_migration); + int ret = -1, init = 0, timeout; + static int64_t start, now; + + switch (ft_mode) { + case FT_INIT: + init = 1; + case FT_TRANSACTION_BEGIN: + now = start = qemu_get_clock(vm_clock); + if (qemu_ft_trans_begin(s->file) < 0) { + fprintf(stderr, "qemu_transaction_begin failed\n"); + goto error_out; + } + + if ((ret = qemu_savevm_trans_begin(s->mon, s->file, init)) < 0) { + fprintf(stderr, "qemu_savevm_trans_begin\n"); + goto error_out; + } + + if (ret) { + ft_mode = FT_TRANSACTION_ITER; + } else { + ft_mode = FT_TRANSACTION_COMMIT; + if (migrate_ft_trans_commit(s) < 0) { + goto error_out; + } + } + break; + + case FT_TRANSACTION_ITER: + now = qemu_get_clock(vm_clock); + timeout = ((now - start) >= max_downtime); + if (timeout || qemu_savevm_state_iterate(s->mon, s->file) == 1) { + DPRINTF("ft trans iter timeout %d\n", timeout); + + ft_mode = FT_TRANSACTION_COMMIT; + if (migrate_ft_trans_commit(s) < 0) { + goto error_out; + } + return 1; + } + + ft_mode = FT_TRANSACTION_ITER; + break; + + case FT_TRANSACTION_ATOMIC: + case FT_TRANSACTION_COMMIT: + if (migrate_ft_trans_commit(s) < 0) { + goto error_out; + } + break; + + default: + fprintf(stderr, + "migrate_ft_trans_put_ready: invalid ft_mode %d", ft_mode); + goto error_out; + } + + ret = 0; + goto out; + +error_out: + ft_mode = FT_ERROR; + qemu_savevm_state_cancel(s->mon, s->file); + migrate_fd_error(s); + event_tap_unregister(); + +out: + return ret; +} + void migrate_fd_put_ready(void *opaque) { FdMigrationState *s = opaque; @@ -393,13 +602,38 @@ void migrate_fd_put_ready(void *opaque) } else { state = MIG_STATE_COMPLETED; } - if (migrate_fd_cleanup(s) < 0) { + + if (ft_mode && state == MIG_STATE_COMPLETED) { + /* close buffered_file and open ft_trans_file + * NB: fd won't get closed, and reused by ft_trans_file + */ + qemu_fclose(s->file); + + s->file = qemu_fopen_ops_ft_trans(s, + migrate_fd_put_buffer, + migrate_fd_get_buffer, + migrate_ft_trans_put_ready, + migrate_ft_trans_get_ready, + migrate_fd_wait_for_unfreeze, + migrate_fd_close, + 1); + socket_set_nodelay(s->fd); + + /* events are tapped from now */ + event_tap_register(migrate_ft_trans_put_ready); + if (old_vm_running) { vm_start(); } - state = MIG_STATE_ERROR; + } else { + if (migrate_fd_cleanup(s) < 0) { + if (old_vm_running) { + vm_start(); + } + state = MIG_STATE_ERROR; + } + s->state = state; } - s->state = state; } } @@ -419,8 +653,14 @@ void migrate_fd_cancel(MigrationState *mig_state) DPRINTF("cancelling migration\n"); s->state = MIG_STATE_CANCELLED; - qemu_savevm_state_cancel(s->mon, s->file); + if (ft_mode) { + qemu_ft_trans_cancel(s->file); + ft_mode = FT_OFF; + event_tap_unregister(); + } + + qemu_savevm_state_cancel(s->mon, s->file); migrate_fd_cleanup(s); } diff --git a/migration.h b/migration.h index f033262..7bf6747 100644 --- a/migration.h +++ b/migration.h @@ -116,7 +116,7 @@ void migrate_fd_put_notify(void *opaque); ssize_t migrate_fd_put_buffer(void *opaque, const void *data, size_t size); -int migrate_fd_get_buffer(void *opaque, uint8_t *data, int64_t pos, int size); +int migrate_fd_get_buffer(void *opaque, uint8_t *data, int64_t pos, size_t size); void migrate_fd_connect(FdMigrationState *s);