From patchwork Thu Feb 12 03:16:51 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 439023 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 5A9C2140083 for ; Thu, 12 Feb 2015 14:21:40 +1100 (AEDT) Received: from localhost ([::1]:48117 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLkLS-0008A1-Fk for incoming@patchwork.ozlabs.org; Wed, 11 Feb 2015 22:21:38 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLkIN-0002h1-Hh for qemu-devel@nongnu.org; Wed, 11 Feb 2015 22:18:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YLkID-0005dq-Fk for qemu-devel@nongnu.org; Wed, 11 Feb 2015 22:18:27 -0500 Received: from szxga01-in.huawei.com ([119.145.14.64]:43763) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLkIC-0005d2-MR for qemu-devel@nongnu.org; Wed, 11 Feb 2015 22:18:17 -0500 Received: from 172.24.2.119 (EHLO szxeml432-hub.china.huawei.com) ([172.24.2.119]) by szxrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id CJK65056; Thu, 12 Feb 2015 11:18:03 +0800 (CST) Received: from localhost (10.177.22.69) by szxeml432-hub.china.huawei.com (10.82.67.209) with Microsoft SMTP Server id 14.3.158.1; Thu, 12 Feb 2015 11:17:52 +0800 From: zhanghailiang To: Date: Thu, 12 Feb 2015 11:16:51 +0800 Message-ID: <1423711034-5340-5-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 1.9.2.msysgit.0 In-Reply-To: <1423711034-5340-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1423711034-5340-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.22.69] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.64 Cc: zhanghailiang , yunhong.jiang@intel.com, eddie.dong@intel.com, dgilbert@redhat.com, peter.huangpeng@huawei.com, Gonglei , stefanha@redhat.com, pbonzini@redhat.com, Lai Jiangshan Subject: [Qemu-devel] [PATCH RFC v3 04/27] migration: Integrate COLO checkpoint process into migration X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a migrate state: MIG_STATE_COLO, enter this migration state after the first live migration successfully finished. Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Lai Jiangshan --- include/migration/migration-colo.h | 2 ++ include/migration/migration.h | 13 +++++++ migration/Makefile.objs | 1 + migration/colo.c | 72 ++++++++++++++++++++++++++++++++++++++ migration/migration.c | 38 +++++++++++--------- stubs/Makefile.objs | 1 + stubs/migration-colo.c | 17 +++++++++ 7 files changed, 128 insertions(+), 16 deletions(-) create mode 100644 migration/colo.c create mode 100644 stubs/migration-colo.c diff --git a/include/migration/migration-colo.h b/include/migration/migration-colo.h index d52ebd0..b72662c 100644 --- a/include/migration/migration-colo.h +++ b/include/migration/migration-colo.h @@ -18,4 +18,6 @@ void colo_info_mig_init(void); +void colo_init_checkpointer(MigrationState *s); + #endif diff --git a/include/migration/migration.h b/include/migration/migration.h index 3f5c705..c4c98d2 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -65,6 +65,19 @@ struct MigrationState int64_t dirty_sync_count; }; +enum { + MIG_STATE_ERROR = -1, + MIG_STATE_NONE, + MIG_STATE_SETUP, + MIG_STATE_CANCELLING, + MIG_STATE_CANCELLED, + MIG_STATE_ACTIVE, + MIG_STATE_COLO, + MIG_STATE_COMPLETED, +}; + +void migrate_set_state(MigrationState *s, int old_state, int new_state); + void process_incoming_migration(QEMUFile *f); void qemu_start_incoming_migration(const char *uri, Error **errp); diff --git a/migration/Makefile.objs b/migration/Makefile.objs index 97b72ad..895583e 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -1,5 +1,6 @@ common-obj-y += migration.o tcp.o common-obj-y += colo-comm.o +common-obj-$(CONFIG_COLO) += colo.o common-obj-y += vmstate.o common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o common-obj-y += xbzrle.o diff --git a/migration/colo.c b/migration/colo.c new file mode 100644 index 0000000..f40b0d8 --- /dev/null +++ b/migration/colo.c @@ -0,0 +1,72 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2015 HUAWEI TECHNOLOGIES CO.,LTD. + * Copyright (c) 2015 FUJITSU LIMITED + * Copyright (c) 2015 Intel Corporation + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "sysemu/sysemu.h" +#include "migration/migration-colo.h" +#include "qemu/error-report.h" + +/* #define DEBUG_COLO */ + +#ifdef DEBUG_COLO +#define DPRINTF(fmt, ...) \ +do { fprintf(stdout, "colo: " fmt , ## __VA_ARGS__); } while (0) +#else +#define DPRINTF(fmt, ...) do {} while (0) +#endif + +static QEMUBH *colo_bh; + +static void *colo_thread(void *opaque) +{ + MigrationState *s = opaque; + + qemu_mutex_lock_iothread(); + vm_start(); + qemu_mutex_unlock_iothread(); + DPRINTF("vm resume to run\n"); + + + /*TODO: COLO checkpoint savevm loop*/ + + migrate_set_state(s, MIG_STATE_COLO, MIG_STATE_COMPLETED); + + qemu_mutex_lock_iothread(); + qemu_bh_schedule(s->cleanup_bh); + qemu_mutex_unlock_iothread(); + + return NULL; +} + +static void colo_start_checkpointer(void *opaque) +{ + MigrationState *s = opaque; + + if (colo_bh) { + qemu_bh_delete(colo_bh); + colo_bh = NULL; + } + + qemu_mutex_unlock_iothread(); + qemu_thread_join(&s->thread); + qemu_mutex_lock_iothread(); + + migrate_set_state(s, MIG_STATE_ACTIVE, MIG_STATE_COLO); + + qemu_thread_create(&s->thread, "colo", colo_thread, s, + QEMU_THREAD_JOINABLE); +} + +void colo_init_checkpointer(MigrationState *s) +{ + colo_bh = qemu_bh_new(colo_start_checkpointer, s); + qemu_bh_schedule(colo_bh); +} diff --git a/migration/migration.c b/migration/migration.c index 8403c8a..536ba01e 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -25,16 +25,7 @@ #include "qemu/thread.h" #include "qmp-commands.h" #include "trace.h" - -enum { - MIG_STATE_ERROR = -1, - MIG_STATE_NONE, - MIG_STATE_SETUP, - MIG_STATE_CANCELLING, - MIG_STATE_CANCELLED, - MIG_STATE_ACTIVE, - MIG_STATE_COMPLETED, -}; +#include "migration/migration-colo.h" #define MAX_THROTTLE (32 << 20) /* Migration speed throttling */ @@ -227,6 +218,11 @@ MigrationInfo *qmp_query_migrate(Error **errp) get_xbzrle_cache_stats(info); break; + case MIG_STATE_COLO: + info->has_status = true; + info->status = g_strdup("colo"); + /* TODO: display COLO specific informations(checkpoint info etc.),*/ + break; case MIG_STATE_COMPLETED: get_xbzrle_cache_stats(info); @@ -270,7 +266,8 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, MigrationState *s = migrate_get_current(); MigrationCapabilityStatusList *cap; - if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP) { + if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP || + s->state == MIG_STATE_COLO) { error_set(errp, QERR_MIGRATION_ACTIVE); return; } @@ -291,7 +288,7 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, /* shared migration helpers */ -static void migrate_set_state(MigrationState *s, int old_state, int new_state) +void migrate_set_state(MigrationState *s, int old_state, int new_state) { if (atomic_cmpxchg(&s->state, old_state, new_state) == new_state) { trace_migrate_set_state(new_state); @@ -437,7 +434,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, params.shared = has_inc && inc; if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP || - s->state == MIG_STATE_CANCELLING) { + s->state == MIG_STATE_CANCELLING || s->state == MIG_STATE_COLO) { error_set(errp, QERR_MIGRATION_ACTIVE); return; } @@ -611,6 +608,7 @@ static void *migration_thread(void *opaque) int64_t max_size = 0; int64_t start_time = initial_time; bool old_vm_running = false; + bool enable_colo = migrate_enable_colo(); qemu_savevm_state_begin(s->file, &s->params); @@ -647,7 +645,10 @@ static void *migration_thread(void *opaque) } if (!qemu_file_get_error(s->file)) { - migrate_set_state(s, MIG_STATE_ACTIVE, MIG_STATE_COMPLETED); + if (!enable_colo) { + migrate_set_state(s, MIG_STATE_ACTIVE, + MIG_STATE_COMPLETED); + } break; } } @@ -697,11 +698,16 @@ static void *migration_thread(void *opaque) } runstate_set(RUN_STATE_POSTMIGRATE); } else { - if (old_vm_running) { + if (s->state == MIG_STATE_ACTIVE && enable_colo) { + colo_init_checkpointer(s); + } else if (old_vm_running) { vm_start(); } } - qemu_bh_schedule(s->cleanup_bh); + + if (!enable_colo) { + qemu_bh_schedule(s->cleanup_bh); + } qemu_mutex_unlock_iothread(); return NULL; diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs index 5e347d0..9fe6b4c 100644 --- a/stubs/Makefile.objs +++ b/stubs/Makefile.objs @@ -40,3 +40,4 @@ stub-obj-$(CONFIG_WIN32) += fd-register.o stub-obj-y += cpus.o stub-obj-y += kvm.o stub-obj-y += qmp_pc_dimm_device_list.o +stub-obj-y += migration-colo.o diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c new file mode 100644 index 0000000..b2cff9c --- /dev/null +++ b/stubs/migration-colo.c @@ -0,0 +1,17 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2015 HUAWEI TECHNOLOGIES CO.,LTD. + * Copyright (c) 2015 FUJITSU LIMITED + * Copyright (c) 2015 Intel Corporation + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "migration/migration-colo.h" + +void colo_init_checkpointer(MigrationState *s) +{ +}