From patchwork Wed Jul 29 08:45:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 501586 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 363EF14030D for ; Wed, 29 Jul 2015 18:52:32 +1000 (AEST) Received: from localhost ([::1]:34029 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZKN6E-0003Nj-Er for incoming@patchwork.ozlabs.org; Wed, 29 Jul 2015 04:52:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58650) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZKN0b-0001fW-8J for qemu-devel@nongnu.org; Wed, 29 Jul 2015 04:46:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZKN0Z-00064o-3u for qemu-devel@nongnu.org; Wed, 29 Jul 2015 04:46:40 -0400 Received: from szxga02-in.huawei.com ([119.145.14.65]:22307) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZKN0X-00062y-CD for qemu-devel@nongnu.org; Wed, 29 Jul 2015 04:46:39 -0400 Received: from 172.24.1.47 (EHLO szxeml430-hub.china.huawei.com) ([172.24.1.47]) by szxrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id CPL67439; Wed, 29 Jul 2015 16:46:23 +0800 (CST) Received: from localhost (10.177.25.63) by szxeml430-hub.china.huawei.com (10.82.67.185) with Microsoft SMTP Server id 14.3.158.1; Wed, 29 Jul 2015 16:46:14 +0800 From: zhanghailiang To: Date: Wed, 29 Jul 2015 16:45:17 +0800 Message-ID: <1438159544-6224-8-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 1.9.5.msysgit.1 In-Reply-To: <1438159544-6224-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1438159544-6224-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.25.63] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.65 Cc: lizhijian@cn.fujitsu.com, quintela@redhat.com, yunhong.jiang@intel.com, eddie.dong@intel.com, peter.huangpeng@huawei.com, dgilbert@redhat.com, arei.gonglei@huawei.com, amit.shah@redhat.com, Yang Hongyang , zhanghailiang Subject: [Qemu-devel] [PATCH COLO-Frame v8 07/34] migration: Integrate COLO checkpoint process into loadvm X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Switch from normal migration loadvm process into COLO checkpoint process if COLO mode is enabled. We add three new members to struct MigrationIncomingState, 'have_colo_incoming_thread' and 'colo_incoming_thread' record the colo related threads for secondary VM, 'migration_incoming_co' records the original migration incoming coroutine. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Yang Hongyang --- include/migration/colo.h | 8 ++++++++ include/migration/migration.h | 7 +++++++ migration/colo-comm.c | 10 ++++++++++ migration/colo.c | 22 ++++++++++++++++++++++ migration/migration.c | 40 ++++++++++++++++++++++++++++++---------- stubs/migration-colo.c | 10 ++++++++++ trace-events | 1 + 7 files changed, 88 insertions(+), 10 deletions(-) diff --git a/include/migration/colo.h b/include/migration/colo.h index dface19..2559f90 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -15,6 +15,8 @@ #include "qemu-common.h" #include "migration/migration.h" +#include "block/coroutine.h" +#include "qemu/thread.h" bool colo_supported(void); void colo_info_mig_init(void); @@ -22,4 +24,10 @@ void colo_info_mig_init(void); void colo_init_checkpointer(MigrationState *s); bool migration_in_colo_state(void); +/* loadvm */ +extern Coroutine *migration_incoming_co; +bool migration_incoming_enable_colo(void); +void migration_incoming_exit_colo(void); +void *colo_process_incoming_checkpoints(void *opaque); +bool migration_incoming_in_colo_state(void); #endif diff --git a/include/migration/migration.h b/include/migration/migration.h index 6fde8df..0548371 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -22,6 +22,7 @@ #include "migration/vmstate.h" #include "qapi-types.h" #include "exec/cpu-common.h" +#include "block/coroutine.h" #define QEMU_VM_FILE_MAGIC 0x5145564d #define QEMU_VM_FILE_VERSION_COMPAT 0x00000002 @@ -51,6 +52,12 @@ struct MigrationIncomingState { QEMUFile *file; int state; + + bool have_colo_incoming_thread; + QemuThread colo_incoming_thread; + /* The coroutine we should enter (back) after failover */ + Coroutine *migration_incoming_co; + /* See savevm.c */ LoadStateEntry_Head loadvm_handlers; }; diff --git a/migration/colo-comm.c b/migration/colo-comm.c index 4330bd8..0808d6c 100644 --- a/migration/colo-comm.c +++ b/migration/colo-comm.c @@ -52,3 +52,13 @@ void colo_info_mig_init(void) { vmstate_register(NULL, 0, &colo_state, &colo_info); } + +bool migration_incoming_enable_colo(void) +{ + return colo_info.colo_requested; +} + +void migration_incoming_exit_colo(void) +{ + colo_info.colo_requested = 0; +} diff --git a/migration/colo.c b/migration/colo.c index 97e64a3..364e0dd 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -13,6 +13,7 @@ #include "sysemu/sysemu.h" #include "migration/colo.h" #include "trace.h" +#include "qemu/error-report.h" static QEMUBH *colo_bh; @@ -28,6 +29,13 @@ bool migration_in_colo_state(void) return (s->state == MIGRATION_STATUS_COLO); } +bool migration_incoming_in_colo_state(void) +{ + MigrationIncomingState *mis = migration_incoming_get_current(); + + return (mis && (mis->state == MIGRATION_STATUS_COLO)); +} + static void *colo_thread(void *opaque) { MigrationState *s = opaque; @@ -74,3 +82,17 @@ void colo_init_checkpointer(MigrationState *s) colo_bh = qemu_bh_new(colo_start_checkpointer, s); qemu_bh_schedule(colo_bh); } + +void *colo_process_incoming_checkpoints(void *opaque) +{ + MigrationIncomingState *mis = opaque; + + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_COLO); + + /* TODO: COLO checkpoint restore loop */ + + migration_incoming_exit_colo(); + + return NULL; +} diff --git a/migration/migration.c b/migration/migration.c index a4a77b4..876c323 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -273,7 +273,29 @@ static void process_incoming_migration_co(void *opaque) MIGRATION_STATUS_ACTIVE); ret = qemu_loadvm_state(f); - qemu_fclose(f); + if (!ret) { + /* Make sure all file formats flush their mutable metadata */ + bdrv_invalidate_cache_all(&local_err); + if (local_err) { + error_report_err(local_err); + migrate_decompress_threads_join(); + exit(EXIT_FAILURE); + } + } + /* we get colo info, and know if we are in colo mode */ + if (!ret && migration_incoming_enable_colo()) { + mis->migration_incoming_co = qemu_coroutine_self(); + qemu_thread_create(&mis->colo_incoming_thread, "colo incoming", + colo_process_incoming_checkpoints, mis, QEMU_THREAD_JOINABLE); + mis->have_colo_incoming_thread = true; + qemu_coroutine_yield(); +#if 0 + /* FIXME wait checkpoint incoming thread exit, and free resource */ + qemu_thread_join(&mis->thread); +#endif + } else { + qemu_fclose(f); + } free_xbzrle_decoded_buf(); migration_incoming_state_destroy(); @@ -288,18 +310,9 @@ static void process_incoming_migration_co(void *opaque) MIGRATION_STATUS_COMPLETED); qemu_announce_self(); - /* Make sure all file formats flush their mutable metadata */ - bdrv_invalidate_cache_all(&local_err); - if (local_err) { - error_report_err(local_err); - migrate_decompress_threads_join(); - exit(EXIT_FAILURE); - } - /* If global state section was not received or we are in running state, we need to obey autostart. Any other state is set with runstate_set. */ - if (!global_state_received() || global_state_get_runstate() == RUN_STATE_RUNNING) { if (autostart) { @@ -733,6 +746,13 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk, error_setg(errp, QERR_MIGRATION_ACTIVE); return; } + + if (migration_incoming_in_colo_state()) { + error_setg(errp, "Secondary VM is not allowed to do migration while" + "in COLO status"); + return; + } + if (runstate_check(RUN_STATE_INMIGRATE)) { error_setg(errp, "Guest is waiting for an incoming migration"); return; diff --git a/stubs/migration-colo.c b/stubs/migration-colo.c index 51b8f66..a105049 100644 --- a/stubs/migration-colo.c +++ b/stubs/migration-colo.c @@ -22,6 +22,16 @@ bool migration_in_colo_state(void) return false; } +bool migration_incoming_in_colo_state(void) +{ + return false; +} + void colo_init_checkpointer(MigrationState *s) { } + +void *colo_process_incoming_checkpoints(void *opaque) +{ + return NULL; +} diff --git a/trace-events b/trace-events index 3ea19ff..025d71c 100644 --- a/trace-events +++ b/trace-events @@ -1473,6 +1473,7 @@ rdma_start_outgoing_migration_after_rdma_source_init(void) "" # migration/colo.c colo_vm_state_change(const char *old, const char *new) "Change '%s' => '%s'" +colo_receive_message(const char *msg) "Receive '%s'" # kvm-all.c kvm_ioctl(int type, void *arg) "type 0x%x, arg %p"