From patchwork Thu May 21 08:13:21 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhanghailiang X-Patchwork-Id: 474877 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id CA9DF14027F for ; Thu, 21 May 2015 18:28:40 +1000 (AEST) Received: from localhost ([::1]:55926 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YvLqH-0007ah-LG for incoming@patchwork.ozlabs.org; Thu, 21 May 2015 04:28:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52634) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YvLd4-0002o3-6x for qemu-devel@nongnu.org; Thu, 21 May 2015 04:15:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YvLd2-0002pn-S2 for qemu-devel@nongnu.org; Thu, 21 May 2015 04:14:58 -0400 Received: from szxga03-in.huawei.com ([119.145.14.66]:24575) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YvLd1-0002p0-VO for qemu-devel@nongnu.org; Thu, 21 May 2015 04:14:56 -0400 Received: from 172.24.2.119 (EHLO szxeml428-hub.china.huawei.com) ([172.24.2.119]) by szxrg03-dlp.huawei.com (MOS 4.4.3-GA FastPath queued) with ESMTP id BGI98582; Thu, 21 May 2015 16:14:53 +0800 (CST) Received: from localhost (10.177.22.69) by szxeml428-hub.china.huawei.com (10.82.67.183) with Microsoft SMTP Server id 14.3.158.1; Thu, 21 May 2015 16:14:42 +0800 From: zhanghailiang To: Date: Thu, 21 May 2015 16:13:21 +0800 Message-ID: <1432196001-10352-30-git-send-email-zhang.zhanghailiang@huawei.com> X-Mailer: git-send-email 1.9.2.msysgit.0 In-Reply-To: <1432196001-10352-1-git-send-email-zhang.zhanghailiang@huawei.com> References: <1432196001-10352-1-git-send-email-zhang.zhanghailiang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.177.22.69] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020203.555D93FD.00E0, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0, ip=0.0.0.0, so=2013-05-26 15:14:31, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 14658b4d1736aaf39ad5180e05c6bc3a X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 119.145.14.66 Cc: lizhijian@cn.fujitsu.com, quintela@redhat.com, yunhong.jiang@intel.com, eddie.dong@intel.com, peter.huangpeng@huawei.com, dgilbert@redhat.com, zhanghailiang , arei.gonglei@huawei.com, amit.shah@redhat.com, Yang Hongyang , david@gibson.dropbear.id.au Subject: [Qemu-devel] [PATCH COLO-Frame v5 29/29] COLO: Add block replication into colo process X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Wen Congyang Make sure master start block replication after slave's block replication started Signed-off-by: zhanghailiang Signed-off-by: Wen Congyang Signed-off-by: Yang Hongyang Signed-off-by: Li Zhijian --- migration/colo.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- trace-events | 2 + 2 files changed, 138 insertions(+), 2 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 111062f..5b600b1 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -19,6 +19,8 @@ #include "migration/migration-failover.h" #include "net/colo-nic.h" #include "qmp-commands.h" +#include "block/block.h" +#include "sysemu/block-backend.h" /* * We should not do checkpoint one after another without any time interval, @@ -102,6 +104,76 @@ static bool colo_runstate_is_stopped(void) return runstate_check(RUN_STATE_COLO) || !runstate_is_running(); } +static void blk_start_replication(bool primary, Error **errp) +{ + ReplicationMode mode = primary ? REPLICATION_MODE_PRIMARY : + REPLICATION_MODE_SECONDARY; + BlockBackend *blk, *temp; + Error *local_err = NULL; + + for (blk = blk_next(NULL); blk; blk = blk_next(blk)) { + if (blk_is_read_only(blk) || !blk_is_inserted(blk)) { + continue; + } + + bdrv_start_replication(blk_bs(blk), mode, &local_err); + if (local_err) { + error_propagate(errp, local_err); + goto fail; + } + } + + return; + +fail: + for (temp = blk_next(NULL); temp != blk; temp = blk_next(temp)) { + bdrv_stop_replication(blk_bs(temp), false, NULL); + } +} + +static void blk_do_checkpoint(Error **errp) +{ + BlockBackend *blk; + Error *local_err = NULL; + + for (blk = blk_next(NULL); blk; blk = blk_next(blk)) { + if (blk_is_read_only(blk) || !blk_is_inserted(blk)) { + continue; + } + + bdrv_do_checkpoint(blk_bs(blk), &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } +} + +static void blk_stop_replication(bool failover, Error **errp) +{ + BlockBackend *blk; + Error *local_err = NULL; + + for (blk = blk_next(NULL); blk; blk = blk_next(blk)) { + if (blk_is_read_only(blk) || !blk_is_inserted(blk)) { + continue; + } + + bdrv_stop_replication(blk_bs(blk), failover, &local_err); + if (!errp) { + /* + * The caller doesn't care the result, they just + * want to stop all block's replication. + */ + continue; + } + if (local_err) { + error_propagate(errp, local_err); + return; + } + } +} + /* * there are two way to entry this function * 1. From colo checkpoint incoming thread, in this case @@ -112,6 +184,8 @@ static bool colo_runstate_is_stopped(void) */ static void slave_do_failover(void) { + Error *local_err = NULL; + /* Wait for incoming thread loading vmstate */ while (vmstate_loading) { ; @@ -121,6 +195,11 @@ static void slave_do_failover(void) error_report("colo proxy failed to do failover"); } colo_proxy_destroy(COLO_SECONDARY_MODE); + blk_stop_replication(true, &local_err); + if (local_err) { + error_report_err(local_err); + } + trace_colo_stop_block_replication("failover"); colo = NULL; @@ -139,6 +218,7 @@ static void slave_do_failover(void) static void master_do_failover(void) { MigrationState *s = migrate_get_current(); + Error *local_err = NULL; if (!colo_runstate_is_stopped()) { vm_stop_force_state(RUN_STATE_COLO); @@ -150,6 +230,12 @@ static void master_do_failover(void) migrate_set_state(s, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED); } + blk_stop_replication(true, &local_err); + if (local_err) { + error_report_err(local_err); + } + trace_colo_stop_block_replication("failover"); + vm_start(); } @@ -223,6 +309,7 @@ static int colo_do_checkpoint_transaction(MigrationState *s, QEMUFile *control) int colo_shutdown, ret; size_t size; QEMUFile *trans = NULL; + Error *local_err = NULL; ret = colo_ctl_put(s->file, COLO_CHECKPOINT_NEW); if (ret < 0) { @@ -275,6 +362,16 @@ static int colo_do_checkpoint_transaction(MigrationState *s, QEMUFile *control) goto out; } + /* we call this api although this may do nothing on primary side */ + qemu_mutex_lock_iothread(); + blk_do_checkpoint(&local_err); + qemu_mutex_unlock_iothread(); + if (local_err) { + error_report_err(local_err); + ret = -1; + goto out; + } + ret = colo_ctl_put(s->file, COLO_CHECKPOINT_SEND); if (ret < 0) { goto out; @@ -305,6 +402,10 @@ static int colo_do_checkpoint_transaction(MigrationState *s, QEMUFile *control) trace_colo_receive_message("COLO_CHECKPOINT_LOADED"); if (colo_shutdown) { + qemu_mutex_lock_iothread(); + blk_stop_replication(false, NULL); + trace_colo_stop_block_replication("shutdown"); + qemu_mutex_unlock_iothread(); colo_ctl_put(s->file, COLO_GUEST_SHUTDOWN); qemu_fflush(s->file); colo_shutdown_requested = 0; @@ -336,6 +437,7 @@ static void *colo_thread(void *opaque) QEMUFile *colo_control = NULL; int64_t current_time, checkpoint_time = qemu_clock_get_ms(QEMU_CLOCK_HOST); int ret; + Error *local_err = NULL; if (colo_proxy_init(COLO_PRIMARY_MODE) != 0) { error_report("Init colo proxy error"); @@ -367,6 +469,12 @@ static void *colo_thread(void *opaque) } qemu_mutex_lock_iothread(); + /* start block replication */ + blk_start_replication(true, &local_err); + if (local_err) { + goto out; + } + trace_colo_start_block_replication(); vm_start(); qemu_mutex_unlock_iothread(); trace_colo_vm_state_change("stop", "run"); @@ -417,7 +525,11 @@ do_checkpoint: } out: - error_report("colo: some error happens in colo_thread"); + if (local_err) { + error_report_err(local_err); + } else { + error_report("colo: some error happens in colo_thread"); + } qemu_mutex_lock_iothread(); if (!failover_request_is_set()) { error_report("master takeover from checkpoint channel"); @@ -498,6 +610,8 @@ static int colo_wait_handle_cmd(QEMUFile *f, int *checkpoint_request) case COLO_GUEST_SHUTDOWN: qemu_mutex_lock_iothread(); vm_stop_force_state(RUN_STATE_COLO); + blk_stop_replication(false, NULL); + trace_colo_stop_block_replication("shutdown"); qemu_system_shutdown_request_core(); qemu_mutex_unlock_iothread(); trace_colo_receive_message("COLO_GUEST_SHUTDOWN"); @@ -521,6 +635,7 @@ void *colo_process_incoming_checkpoints(void *opaque) QEMUFile *ctl = NULL, *fb = NULL; int ret; uint64_t total_size; + Error *local_err = NULL; qdev_hotplug = 0; @@ -550,6 +665,15 @@ void *colo_process_incoming_checkpoints(void *opaque) goto out; } + qemu_mutex_lock_iothread(); + /* start block replication */ + blk_start_replication(false, &local_err); + if (local_err) { + goto out; + } + qemu_mutex_unlock_iothread(); + trace_colo_start_block_replication(); + ret = colo_ctl_put(ctl, COLO_CHECPOINT_READY); if (ret < 0) { goto out; @@ -636,7 +760,13 @@ void *colo_process_incoming_checkpoints(void *opaque) } vmstate_loading = false; + + /* discard colo disk buffer */ + blk_do_checkpoint(&local_err); qemu_mutex_unlock_iothread(); + if (local_err) { + goto out; + } ret = colo_ctl_put(ctl, COLO_CHECKPOINT_LOADED); if (ret < 0) { @@ -654,7 +784,11 @@ void *colo_process_incoming_checkpoints(void *opaque) } out: - error_report("Detect some error or get a failover request"); + if (local_err) { + error_report_err(local_err); + } else { + error_report("Detect some error or get a failover request"); + } /* determine whether we need to failover */ if (!failover_request_is_set()) { /* diff --git a/trace-events b/trace-events index b1c263a..d0ffade 100644 --- a/trace-events +++ b/trace-events @@ -1451,6 +1451,8 @@ colo_vm_state_change(const char *old, const char *new) "Change '%s' => '%s'" colo_receive_message(const char *msg) "Receive '%s'" colo_do_failover(void) "" colo_rcv_pkt(int result) "Result of net packets comparing is different: %d" +colo_start_block_replication(void) "Block replication is started" +colo_stop_block_replication(const char *reason) "Block replication is stopped(reason: '%s')" # kvm-all.c kvm_ioctl(int type, void *arg) "type 0x%x, arg %p"