From patchwork Wed Jun 27 20:41:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935699 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="IFTmQV4z"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFGV5BMKz9s19 for ; Thu, 28 Jun 2018 06:42:42 +1000 (AEST) Received: from localhost ([::1]:33095 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGx-0003Vz-8g for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:42:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGF-0003U0-0t for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:41:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGD-0005FY-Gw for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:41:55 -0400 Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:40287) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGD-0005Dj-9D for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:41:53 -0400 Received: by mail-pg0-x241.google.com with SMTP id w8-v6so1424907pgp.7 for ; Wed, 27 Jun 2018 13:41:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Po6Cl8sT4RA/Q9026TaxM6RUqB4BpHpP0mZycx0od4o=; b=IFTmQV4zGENRnRN+9cLHm6tMbHGNwJ0jZUgLiRw7Q3nGa4Zwj7zUwvIAN+tBvIIXrU 0NVl5qIYZTBEoAHHJAwRgyaE4+73yRsiTeXXZy1gR0ETeEibJeD8WMPtq/ileT5q8jZA xYOZf2fEv/hBlJapJsKbR8ackNsT/ZawrrpqAKHK/G0vHx5Hg7CO+94DLOvf7AT3UAmt iMKptKE+GrYRmmxQ2BldlyKWnOEqGYc35ahVsqDBmzi2zoQbW3g48zd5lpw9BAe3hNsW 05HNp+7k4GlxJPhSnXEFTrS1FIqX9wwTPLiZlPc+8gFXwB2utut7H/JjWcQXZpUEdxZ/ +Izw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Po6Cl8sT4RA/Q9026TaxM6RUqB4BpHpP0mZycx0od4o=; b=FMRyDcun09DzTGOyc1zKG+HzueoNB0soYcczyamOWXZYPtXbY4ZHyL5BAhUm03wxFW p/jZBDIpZrNVnwrYvCF8eAkQYKpvkFL7upGevHJlVMwTItKW4KWQAguJW2Pm8KaHJPvA HeWZKFxom/AQFZpUGUbbHHtnAZQ0ly9O+WSvupoPK9bRc0cevXcM4P/WhyttG5uuf7ig FCG8YfUGuhX3+WJxIcs1+rbjZ3kxQ2T5TaZT62MdllJENY4tCheFr5Fmh2IZ6KHMQf09 Xs8gFQsaa+EPmCDO1y4cudP9Qw0C6JBSWoUEZlASOa1OoXyXOpkdiMIqwwfodR8Qaj6G ksMQ== X-Gm-Message-State: APt69E2CDCtdZmMNL6GnXitDgM7ot1r89Xoqy47gJLAaN/yCMiahv68T 9e7dsGedlH8NLTsPL+2o+SoPWQ== X-Google-Smtp-Source: ADUXVKJVDA4KGp6+mocykSyBSKw4t9Ra6xdWgJfawzanKkdwTdLltViGDDiAPUKIYFfsIitJR9rtFg== X-Received: by 2002:a63:8c10:: with SMTP id m16-v6mr6584642pgd.120.1530132111783; Wed, 27 Jun 2018 13:41:51 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.41.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:41:50 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:17 +0800 Message-Id: <20180627204136.4177-2-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::241 Subject: [Qemu-devel] [PATCH V9 01/20] filter-rewriter: Add TCP state machine and fix memory leak in connection_track_table X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We add almost full TCP state machine in filter-rewriter, except TCPS_LISTEN and some simplify in VM active close FIN states. After a net connection is closed, we didn't clear its releated resources in connection_track_table, which will lead to memory leak. Let't track the state of net connection, if it is closed, its related resources will be cleared up. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- net/colo.c | 2 +- net/colo.h | 9 ++-- net/filter-rewriter.c | 105 ++++++++++++++++++++++++++++++++++++++---- 3 files changed, 100 insertions(+), 16 deletions(-) diff --git a/net/colo.c b/net/colo.c index 6dda4ed66e..97c8fc928f 100644 --- a/net/colo.c +++ b/net/colo.c @@ -137,7 +137,7 @@ Connection *connection_new(ConnectionKey *key) conn->ip_proto = key->ip_proto; conn->processing = false; conn->offset = 0; - conn->syn_flag = 0; + conn->tcp_state = TCPS_CLOSED; conn->pack = 0; conn->sack = 0; g_queue_init(&conn->primary_list); diff --git a/net/colo.h b/net/colo.h index da6c36dcf7..0277e0e9ba 100644 --- a/net/colo.h +++ b/net/colo.h @@ -18,6 +18,7 @@ #include "slirp/slirp.h" #include "qemu/jhash.h" #include "qemu/timer.h" +#include "slirp/tcp.h" #define HASHTABLE_MAX_SIZE 16384 @@ -81,11 +82,9 @@ typedef struct Connection { uint32_t sack; /* offset = secondary_seq - primary_seq */ tcp_seq offset; - /* - * we use this flag update offset func - * run once in independent tcp connection - */ - int syn_flag; + + int tcp_state; /* TCP FSM state */ + tcp_seq fin_ack_seq; /* the seq of 'fin=1,ack=1' */ } Connection; uint32_t connection_key_hash(const void *opaque); diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c index f584e4eba4..f18a71bf2e 100644 --- a/net/filter-rewriter.c +++ b/net/filter-rewriter.c @@ -59,9 +59,9 @@ static int is_tcp_packet(Packet *pkt) } /* handle tcp packet from primary guest */ -static int handle_primary_tcp_pkt(NetFilterState *nf, +static int handle_primary_tcp_pkt(RewriterState *rf, Connection *conn, - Packet *pkt) + Packet *pkt, ConnectionKey *key) { struct tcphdr *tcp_pkt; @@ -74,23 +74,28 @@ static int handle_primary_tcp_pkt(NetFilterState *nf, trace_colo_filter_rewriter_conn_offset(conn->offset); } + if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == (TH_ACK | TH_SYN)) && + conn->tcp_state == TCPS_SYN_SENT) { + conn->tcp_state = TCPS_ESTABLISHED; + } + if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_SYN)) { /* * we use this flag update offset func * run once in independent tcp connection */ - conn->syn_flag = 1; + conn->tcp_state = TCPS_SYN_RECEIVED; } if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_ACK)) { - if (conn->syn_flag) { + if (conn->tcp_state == TCPS_SYN_RECEIVED) { /* * offset = secondary_seq - primary seq * ack packet sent by guest from primary node, * so we use th_ack - 1 get primary_seq */ conn->offset -= (ntohl(tcp_pkt->th_ack) - 1); - conn->syn_flag = 0; + conn->tcp_state = TCPS_ESTABLISHED; } if (conn->offset) { /* handle packets to the secondary from the primary */ @@ -99,15 +104,63 @@ static int handle_primary_tcp_pkt(NetFilterState *nf, net_checksum_calculate((uint8_t *)pkt->data + pkt->vnet_hdr_len, pkt->size - pkt->vnet_hdr_len); } + /* + * Case 1: + * Step 3: + * The *server* side of this connect is VM, *client* tries to close + * the connection. + * + * We got 'ack=1' packets from client side, it acks 'fin=1, ack=1' + * packet from server side. From this point, we can ensure that there + * will be no packets in the connection, except that, some errors + * happen between the path of 'filter object' and vNIC, if this rare + * case really happen, we can still create a new connection, + * So it is safe to remove the connection from connection_track_table. + * + */ + if ((conn->tcp_state == TCPS_LAST_ACK) && + (ntohl(tcp_pkt->th_ack) == (conn->fin_ack_seq + 1))) { + conn->tcp_state = TCPS_CLOSED; + g_hash_table_remove(rf->connection_track_table, key); + } + } + + if ((tcp_pkt->th_flags & TH_FIN) == TH_FIN) { + /* + * Case 1: + * Step 1: + * The *server* side of this connect is VM, *client* tries to close + * the connection. We will into CLOSE_WAIT status. + */ + if (conn->tcp_state == TCPS_ESTABLISHED) { + conn->tcp_state = TCPS_CLOSE_WAIT; + } + + /* + * Case 2: + * Step 2: + * The *server* side of this connect is VM, *server* tries to close + * the connection. We will into CLOSE_WAIT status. + * We simplify the TCPS_FIN_WAIT_2, TCPS_TIME_WAIT and CLOSING status. + */ + if (conn->tcp_state == TCPS_FIN_WAIT_1) { + conn->tcp_state = TCPS_TIME_WAIT; + /* + * For simplify implementation, we needn't wait 2MSL time + * in filter rewriter. + */ + conn->tcp_state = TCPS_CLOSED; + g_hash_table_remove(rf->connection_track_table, key); + } } return 0; } /* handle tcp packet from secondary guest */ -static int handle_secondary_tcp_pkt(NetFilterState *nf, +static int handle_secondary_tcp_pkt(RewriterState *rf, Connection *conn, - Packet *pkt) + Packet *pkt, ConnectionKey *key) { struct tcphdr *tcp_pkt; @@ -121,7 +174,8 @@ static int handle_secondary_tcp_pkt(NetFilterState *nf, trace_colo_filter_rewriter_conn_offset(conn->offset); } - if (((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == (TH_ACK | TH_SYN))) { + if (conn->tcp_state == TCPS_SYN_RECEIVED && + ((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == (TH_ACK | TH_SYN))) { /* * save offset = secondary_seq and then * in handle_primary_tcp_pkt make offset @@ -130,6 +184,12 @@ static int handle_secondary_tcp_pkt(NetFilterState *nf, conn->offset = ntohl(tcp_pkt->th_seq); } + /* VM active connect */ + if (conn->tcp_state == TCPS_CLOSED && + ((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_SYN)) { + conn->tcp_state = TCPS_SYN_SENT; + } + if ((tcp_pkt->th_flags & (TH_ACK | TH_SYN)) == TH_ACK) { /* Only need to adjust seq while offset is Non-zero */ if (conn->offset) { @@ -140,6 +200,31 @@ static int handle_secondary_tcp_pkt(NetFilterState *nf, pkt->size - pkt->vnet_hdr_len); } } + /* + * Case 1: + * Step 2: + * The *server* side of this connect is VM, *client* tries to close + * the connection. In this step we will into LAST_ACK status. + * + * We got 'fin=1, ack=1' packet from server side, we need to + * record the seq of 'fin=1, ack=1' packet. + */ + if (conn->tcp_state == TCPS_CLOSE_WAIT && + (tcp_pkt->th_flags & (TH_ACK | TH_FIN)) == (TH_ACK | TH_FIN)) { + conn->fin_ack_seq = ntohl(tcp_pkt->th_seq); + conn->tcp_state = TCPS_LAST_ACK; + } + + /* + * Case 2: + * Step 1: + * The *server* side of this connect is VM, *server* tries to close + * the connection. + */ + if (conn->tcp_state == TCPS_ESTABLISHED && + (tcp_pkt->th_flags & (TH_ACK | TH_FIN)) == TH_FIN) { + conn->tcp_state = TCPS_FIN_WAIT_1; + } return 0; } @@ -190,7 +275,7 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf, if (sender == nf->netdev) { /* NET_FILTER_DIRECTION_TX */ - if (!handle_primary_tcp_pkt(nf, conn, pkt)) { + if (!handle_primary_tcp_pkt(s, conn, pkt, &key)) { qemu_net_queue_send(s->incoming_queue, sender, 0, (const uint8_t *)pkt->data, pkt->size, NULL); packet_destroy(pkt, NULL); @@ -203,7 +288,7 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf, } } else { /* NET_FILTER_DIRECTION_RX */ - if (!handle_secondary_tcp_pkt(nf, conn, pkt)) { + if (!handle_secondary_tcp_pkt(s, conn, pkt, &key)) { qemu_net_queue_send(s->incoming_queue, sender, 0, (const uint8_t *)pkt->data, pkt->size, NULL); packet_destroy(pkt, NULL); From patchwork Wed Jun 27 20:41:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935702 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kEh5WfI1"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFL20lRZz9s19 for ; Thu, 28 Jun 2018 06:45:46 +1000 (AEST) Received: from localhost ([::1]:33106 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHJv-0005xI-LB for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:45:43 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49178) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGN-0003cc-W2 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGI-0005O9-Qc for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:04 -0400 Received: from mail-pf0-x229.google.com ([2607:f8b0:400e:c00::229]:44674) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGI-0005MZ-Io for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:41:58 -0400 Received: by mail-pf0-x229.google.com with SMTP id j3-v6so1498050pfh.11 for ; Wed, 27 Jun 2018 13:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=gPq5tX24Vp8K/wmWJzlQy9l4dfz/NBDtr6KZm2Ectbw=; b=kEh5WfI1MPuT57S3ahRSUzV5bhFEl8bXsbJKGodDWfe5vNtaFx+6XkgOl5XeE/oRC8 0AS8bLhfQyOUh5JMy3anqJoYk/sfy4jch2x2uEqXU4CLNYgEl5WcOlPhH436jKR25hc/ f18LDmZe+l4YlBnZSng51ZqSq0b+rplnMOF7JO0yniZEhs0c+hAcV0HuJJ1AihF6WjQX UhZLtDCh1a/uhzvPd4YrmBfsy9MuQKwjrtVEiU4TKteyTkgl4oKaLXfIt+bb9Ob05//l 4IYgZLv49ymKuNTTnCD7+NuPIMoFOVZFb1RW0wIJpTEa10ZnetDKM/lqW5jfch+4Ettm cPDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gPq5tX24Vp8K/wmWJzlQy9l4dfz/NBDtr6KZm2Ectbw=; b=AklhFul4E0rqYpN97vgzx9DdyumQQ5dcgEUeJrTn2vfq7Gs9JehFTrMXfnX9P8Gy8v 2TeQuFMLcFg7FrLkax0IeVmyUQ8PJbZkvnzLEu9fFgYZiuJrEjjS5d2BLFT+r45E1EVT JMrvoAnlEy1l/fD05P8z7MEbSGuymiqB8y+cXFNu3zbChjF7s1eDJDXjDy+zb6in6hWa z9W1i0H4074WrRUueqUXerSMhEzO0Zk0yByjbad2ifNwA9znxzNoLyMWrAY7gLdobqTy EV7YzD2Fp4pe6vfyGGWEBovGMcxbQl1haHx3g3bydog7XZ+P6IyYDLXyRo+eBAWGgr+a Z15A== X-Gm-Message-State: APt69E3IeD+hWICwboQ8JhyhTQZR0j2t4QUQfXpqSoGalxF7nIBjgrb0 GQ6plAs/fn9iywf/esadBX86RA== X-Google-Smtp-Source: ADUXVKJvzhKCR3qOfN9mtpNTat6YwJ65p0T1Bf0PmCy6xjm/I3mN4GJfydF5O+8T4jGrbie44HGtwQ== X-Received: by 2002:a63:3dcc:: with SMTP id k195-v6mr6501729pga.254.1530132117246; Wed, 27 Jun 2018 13:41:57 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.41.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:41:56 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:18 +0800 Message-Id: <20180627204136.4177-3-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::229 Subject: [Qemu-devel] [PATCH V9 02/20] colo-compare: implement the process of checkpoint X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" While do checkpoint, we need to flush all the unhandled packets, By using the filter notifier mechanism, we can easily to notify every compare object to do this process, which runs inside of compare threads as a coroutine. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- include/migration/colo.h | 6 ++++ net/colo-compare.c | 78 ++++++++++++++++++++++++++++++++++++++++ net/colo-compare.h | 22 ++++++++++++ 3 files changed, 106 insertions(+) create mode 100644 net/colo-compare.h diff --git a/include/migration/colo.h b/include/migration/colo.h index 2fe48ad353..fefb2fcf4c 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -16,6 +16,12 @@ #include "qemu-common.h" #include "qapi/qapi-types-migration.h" +enum colo_event { + COLO_EVENT_NONE, + COLO_EVENT_CHECKPOINT, + COLO_EVENT_FAILOVER, +}; + void colo_info_init(void); void migrate_start_colo_process(MigrationState *s); diff --git a/net/colo-compare.c b/net/colo-compare.c index dd745a491b..80e6532e8b 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -27,11 +27,16 @@ #include "qemu/sockets.h" #include "colo.h" #include "sysemu/iothread.h" +#include "net/colo-compare.h" +#include "migration/colo.h" #define TYPE_COLO_COMPARE "colo-compare" #define COLO_COMPARE(obj) \ OBJECT_CHECK(CompareState, (obj), TYPE_COLO_COMPARE) +static QTAILQ_HEAD(, CompareState) net_compares = + QTAILQ_HEAD_INITIALIZER(net_compares); + #define COMPARE_READ_LEN_MAX NET_BUFSIZE #define MAX_QUEUE_SIZE 1024 @@ -41,6 +46,10 @@ /* TODO: Should be configurable */ #define REGULAR_PACKET_CHECK_MS 3000 +static QemuMutex event_mtx; +static QemuCond event_complete_cond; +static int event_unhandled_count; + /* * + CompareState ++ * | | @@ -87,6 +96,11 @@ typedef struct CompareState { IOThread *iothread; GMainContext *worker_context; QEMUTimer *packet_check_timer; + + QEMUBH *event_bh; + enum colo_event event; + + QTAILQ_ENTRY(CompareState) next; } CompareState; typedef struct CompareClass { @@ -736,6 +750,25 @@ static void check_old_packet_regular(void *opaque) REGULAR_PACKET_CHECK_MS); } +/* Public API, Used for COLO frame to notify compare event */ +void colo_notify_compares_event(void *opaque, int event, Error **errp) +{ + CompareState *s; + + qemu_mutex_lock(&event_mtx); + QTAILQ_FOREACH(s, &net_compares, next) { + s->event = event; + qemu_bh_schedule(s->event_bh); + event_unhandled_count++; + } + /* Wait all compare threads to finish handling this event */ + while (event_unhandled_count > 0) { + qemu_cond_wait(&event_complete_cond, &event_mtx); + } + + qemu_mutex_unlock(&event_mtx); +} + static void colo_compare_timer_init(CompareState *s) { AioContext *ctx = iothread_get_aio_context(s->iothread); @@ -756,6 +789,30 @@ static void colo_compare_timer_del(CompareState *s) } } +static void colo_flush_packets(void *opaque, void *user_data); + +static void colo_compare_handle_event(void *opaque) +{ + CompareState *s = opaque; + + switch (s->event) { + case COLO_EVENT_CHECKPOINT: + g_queue_foreach(&s->conn_list, colo_flush_packets, s); + break; + case COLO_EVENT_FAILOVER: + break; + default: + break; + } + + assert(event_unhandled_count > 0); + + qemu_mutex_lock(&event_mtx); + event_unhandled_count--; + qemu_cond_broadcast(&event_complete_cond); + qemu_mutex_unlock(&event_mtx); +} + static void colo_compare_iothread(CompareState *s) { object_ref(OBJECT(s->iothread)); @@ -769,6 +826,7 @@ static void colo_compare_iothread(CompareState *s) s, s->worker_context, true); colo_compare_timer_init(s); + s->event_bh = qemu_bh_new(colo_compare_handle_event, s); } static char *compare_get_pri_indev(Object *obj, Error **errp) @@ -926,8 +984,13 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp) net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr); net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr); + QTAILQ_INSERT_TAIL(&net_compares, s, next); + g_queue_init(&s->conn_list); + qemu_mutex_init(&event_mtx); + qemu_cond_init(&event_complete_cond); + s->connection_track_table = g_hash_table_new_full(connection_key_hash, connection_key_equal, g_free, @@ -990,6 +1053,7 @@ static void colo_compare_init(Object *obj) static void colo_compare_finalize(Object *obj) { CompareState *s = COLO_COMPARE(obj); + CompareState *tmp = NULL; qemu_chr_fe_deinit(&s->chr_pri_in, false); qemu_chr_fe_deinit(&s->chr_sec_in, false); @@ -997,6 +1061,16 @@ static void colo_compare_finalize(Object *obj) if (s->iothread) { colo_compare_timer_del(s); } + + qemu_bh_delete(s->event_bh); + + QTAILQ_FOREACH(tmp, &net_compares, next) { + if (tmp == s) { + QTAILQ_REMOVE(&net_compares, s, next); + break; + } + } + /* Release all unhandled packets after compare thead exited */ g_queue_foreach(&s->conn_list, colo_flush_packets, s); @@ -1009,6 +1083,10 @@ static void colo_compare_finalize(Object *obj) if (s->iothread) { object_unref(OBJECT(s->iothread)); } + + qemu_mutex_destroy(&event_mtx); + qemu_cond_destroy(&event_complete_cond); + g_free(s->pri_indev); g_free(s->sec_indev); g_free(s->outdev); diff --git a/net/colo-compare.h b/net/colo-compare.h new file mode 100644 index 0000000000..1b1ce76aea --- /dev/null +++ b/net/colo-compare.h @@ -0,0 +1,22 @@ +/* + * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) + * (a.k.a. Fault Tolerance or Continuous Replication) + * + * Copyright (c) 2017 HUAWEI TECHNOLOGIES CO., LTD. + * Copyright (c) 2017 FUJITSU LIMITED + * Copyright (c) 2017 Intel Corporation + * + * Authors: + * zhanghailiang + * Zhang Chen + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_COLO_COMPARE_H +#define QEMU_COLO_COMPARE_H + +void colo_notify_compares_event(void *opaque, int event, Error **errp); + +#endif /* QEMU_COLO_COMPARE_H */ From patchwork Wed Jun 27 20:41:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935700 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="YWDaTZFi"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFGl5nFvz9s19 for ; Thu, 28 Jun 2018 06:42:55 +1000 (AEST) Received: from localhost ([::1]:33096 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHB-0003f8-9n for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:42:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49198) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGO-0003dH-VH for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGN-0005UV-T9 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:05 -0400 Received: from mail-pg0-x231.google.com ([2607:f8b0:400e:c05::231]:38617) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGN-0005TX-N4 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:03 -0400 Received: by mail-pg0-x231.google.com with SMTP id c9-v6so1428357pgf.5 for ; Wed, 27 Jun 2018 13:42:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IowdkrtjK78dChGKyNkmOTIIjUjqLeYtoZz7kVwiBzA=; b=YWDaTZFioZl6Bh99CdlRCBRdgSqC279OqSBhA2q5Eyz2ao1mnza97r2/kKYUeTKM/5 /WpLj4cknvFYo2NGKmb+90bsmlcEl1N1TuicWg3jPwVEFbv4hzFpIEdVOzgbYA/kIJjn 4bJKn0fbVOSnDmXp7i3ERbpT1CnS+Q/zLjvvahk8andKLpc6Gq9B89EZASuUwHLFbEmJ zRmLJtm6agdwADf1O6oCe8vI6XwHRgBW9GWOciSbwFpuSXEG8uTEl9wLMuNYRCWYcw/G pyueRjT217nsDTytAJlP8In62efkEOfy5z0mjIzw1T2XDIsNduNTC3FG8fjkEq3J5b9z iqow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IowdkrtjK78dChGKyNkmOTIIjUjqLeYtoZz7kVwiBzA=; b=QepWYH+BexaotOmhOQQKqLgN1ZYnGq3VzdS1WFpc8TwhrXl6IAeVsevGBL72m/U002 CrDtzZBldUSbBzIBaGsx9mlOoxU90//mbB2oD3FCywVWrPTO+H/JzN48YzEA2Gsn4S7N VvJnWqJyTd/GRN1r3DS2nj/O7g/Q37vPY1hLn5soAkLGLPwVATH5zgqIO2swJ4Zl5xBQ xX0P1poKUG9yWATL9SnNNiL9T8Ncb16i91zHiOYHWASWrnVYtY/D0QnvjsIUlXMvHfvh K11rTxGpeuNbJUixXYTFgxm+2QwbSz4TazjBg71rkVSN5IeDUk7RcWKNFpx8D9RCdvte MHhA== X-Gm-Message-State: APt69E3GyJ5kAdpagpBd4d1m5ay6CnSIOA9AL5N1L/+fVyQeMT6gLxjZ SWKCSEU0rJ2RwA/P1YvIDofL/g== X-Google-Smtp-Source: ADUXVKJBCddlTF+Sof2dMnbZ8cnpNWiYoxsgFVvKfjalCa8rHf8mn16UVvc3t+AFdRkCNUsEHOsTDw== X-Received: by 2002:a63:7558:: with SMTP id f24-v6mr6599251pgn.314.1530132122497; Wed, 27 Jun 2018 13:42:02 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.41.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:01 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:19 +0800 Message-Id: <20180627204136.4177-4-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::231 Subject: [Qemu-devel] [PATCH V9 03/20] colo-compare: use notifier to notify packets comparing result X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" It's a good idea to use notifier to notify COLO frame of inconsistent packets comparing. Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang --- net/colo-compare.c | 37 ++++++++++++++++++++++++++----------- net/colo-compare.h | 2 ++ 2 files changed, 28 insertions(+), 11 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index 80e6532e8b..426eab5973 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -29,6 +29,7 @@ #include "sysemu/iothread.h" #include "net/colo-compare.h" #include "migration/colo.h" +#include "migration/migration.h" #define TYPE_COLO_COMPARE "colo-compare" #define COLO_COMPARE(obj) \ @@ -37,6 +38,9 @@ static QTAILQ_HEAD(, CompareState) net_compares = QTAILQ_HEAD_INITIALIZER(net_compares); +static NotifierList colo_compare_notifiers = + NOTIFIER_LIST_INITIALIZER(colo_compare_notifiers); + #define COMPARE_READ_LEN_MAX NET_BUFSIZE #define MAX_QUEUE_SIZE 1024 @@ -427,10 +431,7 @@ sec: qemu_hexdump((char *)spkt->data, stderr, "colo-compare spkt", spkt->size); - /* - * colo_compare_inconsistent_notify(); - * TODO: notice to checkpoint(); - */ + colo_compare_inconsistency_notify(); } } @@ -561,8 +562,24 @@ static int colo_old_packet_check_one(Packet *pkt, int64_t *check_time) } } +static void colo_compare_inconsistency_notify(void) +{ + notifier_list_notify(&colo_compare_notifiers, + migrate_get_current()); +} + +void colo_compare_register_notifier(Notifier *notify) +{ + notifier_list_add(&colo_compare_notifiers, notify); +} + +void colo_compare_unregister_notifier(Notifier *notify) +{ + notifier_remove(notify); +} + static int colo_old_packet_check_one_conn(Connection *conn, - void *user_data) + void *user_data) { GList *result = NULL; int64_t check_time = REGULAR_PACKET_CHECK_MS; @@ -573,10 +590,7 @@ static int colo_old_packet_check_one_conn(Connection *conn, if (result) { /* Do checkpoint will flush old packet */ - /* - * TODO: Notify colo frame to do checkpoint. - * colo_compare_inconsistent_notify(); - */ + colo_compare_inconsistency_notify(); return 0; } @@ -620,11 +634,12 @@ static void colo_compare_packet(CompareState *s, Connection *conn, /* * If one packet arrive late, the secondary_list or * primary_list will be empty, so we can't compare it - * until next comparison. + * until next comparison. If the packets in the list are + * timeout, it will trigger a checkpoint request. */ trace_colo_compare_main("packet different"); g_queue_push_head(&conn->primary_list, pkt); - /* TODO: colo_notify_checkpoint();*/ + colo_compare_inconsistency_notify(); break; } } diff --git a/net/colo-compare.h b/net/colo-compare.h index 1b1ce76aea..22ddd512e2 100644 --- a/net/colo-compare.h +++ b/net/colo-compare.h @@ -18,5 +18,7 @@ #define QEMU_COLO_COMPARE_H void colo_notify_compares_event(void *opaque, int event, Error **errp); +void colo_compare_register_notifier(Notifier *notify); +void colo_compare_unregister_notifier(Notifier *notify); #endif /* QEMU_COLO_COMPARE_H */ From patchwork Wed Jun 27 20:41:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935706 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="OjBoC2C7"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFQ41Q4Tz9s1R for ; Thu, 28 Jun 2018 06:49:16 +1000 (AEST) Received: from localhost ([::1]:33126 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHNJ-0008Sr-Nw for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:49:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49311) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGY-0003iy-EY for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGT-0005eF-Gc for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:14 -0400 Received: from mail-pf0-x242.google.com ([2607:f8b0:400e:c00::242]:43229) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGT-0005cZ-70 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:09 -0400 Received: by mail-pf0-x242.google.com with SMTP id y8-v6so1496393pfm.10 for ; Wed, 27 Jun 2018 13:42:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cknl3XOHHisLUPzcd0U51KwZXV/0FIPdV8vi8HSSPng=; b=OjBoC2C7anM2HWXgIrxEyPMKxWKSZBy9rWDa4IABQ1Dyd1MXfoSYL0pSyvL9GcJvkD oRkf0ij/JaJk7A3jXGjH/jQyGGs63NhvX2yDXWFMU5o9FzmEE3S0C52G5eZOSxpZhFvv VHR23VyrOHsa4w7hB8uKnY3jcRR8Ff0G270ni83dQVS7XhDJs+tAM6sYgSHvTGi8Gfn9 mvnZ72l3/6Zl1yxw9kqV2NuAxYLTBQ6p6WXncDAzAW9niOuz0lnOK/eJ7wCs7YG/hFmL 3VfzecEP3qbVuPiA1D/WKauwmmftGqOiudO4JLfyagZOfXJKdEPX2tIqwLnnh8Nx4l25 h19Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cknl3XOHHisLUPzcd0U51KwZXV/0FIPdV8vi8HSSPng=; b=VGzVZgSou0WOdAiv3EQBc2FOXgU+DJPwygZ37JUdqvgz9gHz9d7WFyPhdQJagMUha2 84qqi5jUuJmLMa0VZss2NLVD3XOOcmbpKI2xE49BN+XFSaop9evXmPgxq4pwL8TJpvl5 /XRoIVOnn7EdV3n3ummzgOaEZlEBzLrZY4FE5EufjRV04b9WtXggAF1PjeR+ymIUXgAo KcDY+85YXLmRBeSHorwXpF0vNVH/h9gXLAFg/KR0Rc9R2syaMrbSuvvRBl/4smozdPia YopW1Ro3CPqI9pgGk4ps+L3u1AagTUhaIqZdTG3LfPspiiT8hGKId6MXkJFoCVzDYfDW dFug== X-Gm-Message-State: APt69E1w3mWohRVipybtXCZ1+ZvUmSyythUgdtS8mvwcfDidLG5G6x6t jRp8PtBoVyu4o1x9OaAha5eb9A== X-Google-Smtp-Source: AAOMgpe/3WAiEMk4BipvbvQXaHCogg5jO4Nd/5qyQvfDGg4PiPPjjKv3zod8mt9Y8qKeYSuZHU9VZg== X-Received: by 2002:a62:93:: with SMTP id 141-v6mr4087847pfa.14.1530132127899; Wed, 27 Jun 2018 13:42:07 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:06 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:20 +0800 Message-Id: <20180627204136.4177-5-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::242 Subject: [Qemu-devel] [PATCH V9 04/20] COLO: integrate colo compare with colo frame X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For COLO FT, both the PVM and SVM run at the same time, only sync the state while it needs. So here, let SVM runs while not doing checkpoint, change DEFAULT_MIGRATE_X_CHECKPOINT_DELAY to 200*100. Besides, we forgot to release colo_checkpoint_semd and colo_delay_timer, fix them here. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 42 ++++++++++++++++++++++++++++++++++++++++-- migration/migration.c | 6 ++---- 2 files changed, 42 insertions(+), 6 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index 4381067ed4..081df1835f 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -25,8 +25,11 @@ #include "qemu/error-report.h" #include "migration/failover.h" #include "replication.h" +#include "net/colo-compare.h" +#include "net/colo.h" static bool vmstate_loading; +static Notifier packets_compare_notifier; #define COLO_BUFFER_BASE_SIZE (4 * 1024 * 1024) @@ -343,6 +346,11 @@ static int colo_do_checkpoint_transaction(MigrationState *s, goto out; } + colo_notify_compares_event(NULL, COLO_EVENT_CHECKPOINT, &local_err); + if (local_err) { + goto out; + } + /* Disable block migration */ migrate_set_block_enabled(false, &local_err); qemu_savevm_state_header(fb); @@ -400,6 +408,11 @@ out: return ret; } +static void colo_compare_notify_checkpoint(Notifier *notifier, void *data) +{ + colo_checkpoint_notify(data); +} + static void colo_process_checkpoint(MigrationState *s) { QIOChannelBuffer *bioc; @@ -416,6 +429,9 @@ static void colo_process_checkpoint(MigrationState *s) goto out; } + packets_compare_notifier.notify = colo_compare_notify_checkpoint; + colo_compare_register_notifier(&packets_compare_notifier); + /* * Wait for Secondary finish loading VM states and enter COLO * restore. @@ -461,11 +477,21 @@ out: qemu_fclose(fb); } - timer_del(s->colo_delay_timer); - /* Hope this not to be too long to wait here */ qemu_sem_wait(&s->colo_exit_sem); qemu_sem_destroy(&s->colo_exit_sem); + + /* + * It is safe to unregister notifier after failover finished. + * Besides, colo_delay_timer and colo_checkpoint_sem can't be + * released befor unregister notifier, or there will be use-after-free + * error. + */ + colo_compare_unregister_notifier(&packets_compare_notifier); + timer_del(s->colo_delay_timer); + timer_free(s->colo_delay_timer); + qemu_sem_destroy(&s->colo_checkpoint_sem); + /* * Must be called after failover BH is completed, * Or the failover BH may shutdown the wrong fd that @@ -558,6 +584,11 @@ void *colo_process_incoming_thread(void *opaque) fb = qemu_fopen_channel_input(QIO_CHANNEL(bioc)); object_unref(OBJECT(bioc)); + qemu_mutex_lock_iothread(); + vm_start(); + trace_colo_vm_state_change("stop", "run"); + qemu_mutex_unlock_iothread(); + colo_send_message(mis->to_src_file, COLO_MESSAGE_CHECKPOINT_READY, &local_err); if (local_err) { @@ -577,6 +608,11 @@ void *colo_process_incoming_thread(void *opaque) goto out; } + qemu_mutex_lock_iothread(); + vm_stop_force_state(RUN_STATE_COLO); + trace_colo_vm_state_change("run", "stop"); + qemu_mutex_unlock_iothread(); + /* FIXME: This is unnecessary for periodic checkpoint mode */ colo_send_message(mis->to_src_file, COLO_MESSAGE_CHECKPOINT_REPLY, &local_err); @@ -630,6 +666,8 @@ void *colo_process_incoming_thread(void *opaque) } vmstate_loading = false; + vm_start(); + trace_colo_vm_state_change("stop", "run"); qemu_mutex_unlock_iothread(); if (failover_get_state() == FAILOVER_STATUS_RELAUNCH) { diff --git a/migration/migration.c b/migration/migration.c index e1eaa97df4..a4b478718f 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -75,10 +75,8 @@ /* Migration XBZRLE default cache size */ #define DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE (64 * 1024 * 1024) -/* The delay time (in ms) between two COLO checkpoints - * Note: Please change this default value to 10000 when we support hybrid mode. - */ -#define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY 200 +/* The delay time (in ms) between two COLO checkpoints */ +#define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY (200 * 100) #define DEFAULT_MIGRATE_MULTIFD_CHANNELS 2 #define DEFAULT_MIGRATE_MULTIFD_PAGE_COUNT 16 From patchwork Wed Jun 27 20:41:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935701 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="JNy8RyIN"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFJK2Bk2z9s1R for ; Thu, 28 Jun 2018 06:44:17 +1000 (AEST) Received: from localhost ([::1]:33101 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHIU-0004l5-VW for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:44:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49336) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGa-0003kZ-25 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGY-0005oQ-Pm for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:16 -0400 Received: from mail-pl0-x235.google.com ([2607:f8b0:400e:c01::235]:40008) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGY-0005n2-I2 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:14 -0400 Received: by mail-pl0-x235.google.com with SMTP id t6-v6so1593006plo.7 for ; Wed, 27 Jun 2018 13:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TPawtPgLqdrFV1QK3KMsmmn98rew7bufpMNjFbIbc60=; b=JNy8RyINWQEt3+uLqSbV1O1Ii5RAj3ly3vrAlMy/0/dTeoM3JDXXL2iZMwQ7Io8eVE H91i8oCgekQHiiahXUQt9tSbLbL/XcbL5qHt9YF4nW11ehSHg6Iti4a4Jj+qmyU9BCU4 HbT5cT9ON9Gio5w8qIcI7ruV1mF1YYWo4fA7mYQi+Q8MZXOwK9eI7MvAAY5L9XIWY2N3 hrG6tsmjItQbEF6AiTdRiZCAJ9mZNuF1bs9NnN6I3UpkmHlhaOF8SOMfe0cX5Ll/PZ+p Kk0jgg7S4COqL3t5eupUvJCufoyNAog1fc2aGYdXx58RWalquPhQ/I8A93odcFb3YD6W MZmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TPawtPgLqdrFV1QK3KMsmmn98rew7bufpMNjFbIbc60=; b=ZnWdXsGomjePWIJyzj7IrVBfb+tsUnkEJLRG3M228p/Mq3fJ/5FPYjAYjIVwzfmvc9 TgHpzyTKfRHx62TZw5U75qRiDZCGseE5xOR1X52zB05fToQPou4k+TzierV2i6WdjKwa nFlBIsTwTkkPl+LfcVlHKDHWql6uadG/KJDugRbkZGMCmcrFNEM77y1/DOIdlqtuCRIf J9FuLAhwR7qrDY00IAMOW0d8jmSJwY6k7sIwUMDH1L3ky4nLQnIezAFUy2Mzex/RBQ79 iWLwjrsyBdv2jv3d5QVMNRMqpSARzzqfji739y+V1XWnvRqvdU0FkCOj0Zrrzn00/Hh3 uPfg== X-Gm-Message-State: APt69E0PohFIw9OGQ7HRQRjWvOma37k5+vcvhvx8AZwlEb9ria8P02Pv Y3t+ZciSZT1NedlFFFcaEgND9Q== X-Google-Smtp-Source: ADUXVKLdRCbgf7Os8hHCb2ghDYun/JrYWhHe/qRx/inGDKeb0uDfJR7NDdO3s2y1XD+WrKFzaKYoGg== X-Received: by 2002:a17:902:b48f:: with SMTP id y15-v6mr7671645plr.261.1530132133237; Wed, 27 Jun 2018 13:42:13 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:12 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:21 +0800 Message-Id: <20180627204136.4177-6-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::235 Subject: [Qemu-devel] [PATCH V9 05/20] COLO: Add block replication into colo process X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Make sure master start block replication after slave's block replication started. Besides, we need to activate VM's blocks before goes into COLO state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- migration/colo.c | 43 +++++++++++++++++++++++++++++++++++++++++++ migration/migration.c | 9 +++++++++ 2 files changed, 52 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 081df1835f..e06640c3d6 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -27,6 +27,7 @@ #include "replication.h" #include "net/colo-compare.h" #include "net/colo.h" +#include "block/block.h" static bool vmstate_loading; static Notifier packets_compare_notifier; @@ -56,6 +57,7 @@ static void secondary_vm_do_failover(void) { int old_state; MigrationIncomingState *mis = migration_incoming_get_current(); + Error *local_err = NULL; /* Can not do failover during the process of VM's loading VMstate, Or * it will break the secondary VM. @@ -73,6 +75,11 @@ static void secondary_vm_do_failover(void) migrate_set_state(&mis->state, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED); + replication_stop_all(true, &local_err); + if (local_err) { + error_report_err(local_err); + } + if (!autostart) { error_report("\"-S\" qemu option will be ignored in secondary side"); /* recover runstate to normal migration finish state */ @@ -110,6 +117,7 @@ static void primary_vm_do_failover(void) { MigrationState *s = migrate_get_current(); int old_state; + Error *local_err = NULL; migrate_set_state(&s->state, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED); @@ -133,6 +141,13 @@ static void primary_vm_do_failover(void) FailoverStatus_str(old_state)); return; } + + replication_stop_all(true, &local_err); + if (local_err) { + error_report_err(local_err); + local_err = NULL; + } + /* Notify COLO thread that failover work is finished */ qemu_sem_post(&s->colo_exit_sem); } @@ -356,6 +371,11 @@ static int colo_do_checkpoint_transaction(MigrationState *s, qemu_savevm_state_header(fb); qemu_savevm_state_setup(fb); qemu_mutex_lock_iothread(); + replication_do_checkpoint_all(&local_err); + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } qemu_savevm_state_complete_precopy(fb, false, false); qemu_mutex_unlock_iothread(); @@ -446,6 +466,12 @@ static void colo_process_checkpoint(MigrationState *s) object_unref(OBJECT(bioc)); qemu_mutex_lock_iothread(); + replication_start_all(REPLICATION_MODE_PRIMARY, &local_err); + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } + vm_start(); qemu_mutex_unlock_iothread(); trace_colo_vm_state_change("stop", "run"); @@ -585,6 +611,11 @@ void *colo_process_incoming_thread(void *opaque) object_unref(OBJECT(bioc)); qemu_mutex_lock_iothread(); + replication_start_all(REPLICATION_MODE_SECONDARY, &local_err); + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } vm_start(); trace_colo_vm_state_change("stop", "run"); qemu_mutex_unlock_iothread(); @@ -665,6 +696,18 @@ void *colo_process_incoming_thread(void *opaque) goto out; } + replication_get_error_all(&local_err); + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } + /* discard colo disk buffer */ + replication_do_checkpoint_all(&local_err); + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } + vmstate_loading = false; vm_start(); trace_colo_vm_state_change("stop", "run"); diff --git a/migration/migration.c b/migration/migration.c index a4b478718f..e665a1cab0 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -385,6 +385,7 @@ static void process_incoming_migration_co(void *opaque) MigrationIncomingState *mis = migration_incoming_get_current(); PostcopyState ps; int ret; + Error *local_err = NULL; assert(mis->from_src_file); mis->largest_page_size = qemu_ram_pagesize_largest(); @@ -416,6 +417,14 @@ static void process_incoming_migration_co(void *opaque) /* we get COLO info, and know if we are in COLO mode */ if (!ret && migration_incoming_enable_colo()) { + /* Make sure all file formats flush their mutable metadata */ + bdrv_invalidate_cache_all(&local_err); + if (local_err) { + migrate_set_state(&mis->state, MIGRATION_STATUS_ACTIVE, + MIGRATION_STATUS_FAILED); + error_report_err(local_err); + exit(EXIT_FAILURE); + } mis->migration_incoming_co = qemu_coroutine_self(); qemu_thread_create(&mis->colo_incoming_thread, "COLO incoming", colo_process_incoming_thread, mis, QEMU_THREAD_JOINABLE); From patchwork Wed Jun 27 20:41:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935712 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="f+wtBTl/"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFTp5mY9z9s19 for ; Thu, 28 Jun 2018 06:52:30 +1000 (AEST) Received: from localhost ([::1]:33144 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHQS-0002hV-Ep for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:52:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49441) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGj-0003rM-E0 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGe-0005ws-FU for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:25 -0400 Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:37958) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGe-0005vU-6l for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:20 -0400 Received: by mail-pg0-x241.google.com with SMTP id c9-v6so1428623pgf.5 for ; Wed, 27 Jun 2018 13:42:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dVXunUg1A+/09BDAejzJRJ/DFyKLavNnf0zGCL9mN1Y=; b=f+wtBTl/84ZZEyBvrqUOM8SLJPYUTRN3ZauqRxuqwicwp9L3yt684HiLaSJn5pUyOc DAvR6vhZycl3pgTpCF7RvfDFXFuMBVF829saiz9MJ41xY+hpvWrJt4S8f79rwe6FjFef WXBz04OWJKxPe+uKxfUsD5/6nDXGPu/MTISb5Vum2Cmt36IYolE+8QTrTsBvojeU53UY gzbuG8jzO+iZOuNuZa/aaqNbqO/bhV5LuQ6U6ORAHfATKRc5JXENth53yQSKPUMuPxs+ fNxFzu0tAgVrr34RPHr6RxIfmKkLkakVKYvF8D6asey6ZN6ZVQsuE4xIJBg7/6MPItBu RfOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dVXunUg1A+/09BDAejzJRJ/DFyKLavNnf0zGCL9mN1Y=; b=HU1ndFh/Kdwb8Ubdl09kExLwoyCmkkT26Wt0+Vs6PNutS2gxDFW3lu0QjR6SIvo0Sx LWc/ft89mWMJ+LGL6Y4691oDLDUJ4tkOIXSUnU5Wxi1r+oWM7sCUT3EgDFi7E64BPU9+ kZI81h9CzVKifiLRRFAl2CmHKHZtrKXiFgM01/FHPG+qVBInUfFfSBBJe6AA8Jvo2uIV RC/Ncwl+qvdpHzZ+AS4wbHAivbB57rJ1g0NJLOiWR60/NDIMG5Zudj5JhcnM14lG9NPi gf98ooGzl+Mq+jBuuzHFMpiVmoXcQ1gC+3C05Jwe3jREfyCjhwPfGgy0FVXXnTRYeSHx XazQ== X-Gm-Message-State: APt69E26yZVCBXUDdRyE6Cu+RxaxW4/z5cxJOFBpkOdLLmFRYAFTIJJZ zmAMX5NMScOHdj8Qnqq2MTY+tQ== X-Google-Smtp-Source: AAOMgpfBL4tK44lyQGRb6qPTFRY60y73mh/F75iniifxReievi9NbAknxc+hVBoOIlfcS1w2GPzW4w== X-Received: by 2002:aa7:8298:: with SMTP id s24-v6mr7309706pfm.136.1530132138760; Wed, 27 Jun 2018 13:42:18 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:17 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:22 +0800 Message-Id: <20180627204136.4177-7-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::241 Subject: [Qemu-devel] [PATCH V9 06/20] COLO: Remove colo_state migration struct X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We need to know if migration is going into COLO state for incoming side before start normal migration. Instead by using the VMStateDescription to send colo_state from source side to destination side, we use MIG_CMD_ENABLE_COLO to indicate whether COLO is enabled or not. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- include/migration/colo.h | 5 +-- migration/Makefile.objs | 2 +- migration/colo-comm.c | 76 ---------------------------------------- migration/colo.c | 13 ++++++- migration/migration.c | 23 +++++++++++- migration/savevm.c | 17 +++++++++ migration/savevm.h | 1 + migration/trace-events | 1 + vl.c | 2 -- 9 files changed, 57 insertions(+), 83 deletions(-) delete mode 100644 migration/colo-comm.c diff --git a/include/migration/colo.h b/include/migration/colo.h index fefb2fcf4c..99ce17aca7 100644 --- a/include/migration/colo.h +++ b/include/migration/colo.h @@ -28,8 +28,9 @@ void migrate_start_colo_process(MigrationState *s); bool migration_in_colo_state(void); /* loadvm */ -bool migration_incoming_enable_colo(void); -void migration_incoming_exit_colo(void); +void migration_incoming_enable_colo(void); +void migration_incoming_disable_colo(void); +bool migration_incoming_colo_enabled(void); void *colo_process_incoming_thread(void *opaque); bool migration_incoming_in_colo_state(void); diff --git a/migration/Makefile.objs b/migration/Makefile.objs index c83ec47ba8..a4f3bafd86 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -1,6 +1,6 @@ common-obj-y += migration.o socket.o fd.o exec.o common-obj-y += tls.o channel.o savevm.o -common-obj-y += colo-comm.o colo.o colo-failover.o +common-obj-y += colo.o colo-failover.o common-obj-y += vmstate.o vmstate-types.o page_cache.o common-obj-y += qemu-file.o global_state.o common-obj-y += qemu-file-channel.o diff --git a/migration/colo-comm.c b/migration/colo-comm.c deleted file mode 100644 index df26e4dfe7..0000000000 --- a/migration/colo-comm.c +++ /dev/null @@ -1,76 +0,0 @@ -/* - * COarse-grain LOck-stepping Virtual Machines for Non-stop Service (COLO) - * (a.k.a. Fault Tolerance or Continuous Replication) - * - * Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD. - * Copyright (c) 2016 FUJITSU LIMITED - * Copyright (c) 2016 Intel Corporation - * - * This work is licensed under the terms of the GNU GPL, version 2 or - * later. See the COPYING file in the top-level directory. - * - */ - -#include "qemu/osdep.h" -#include "migration.h" -#include "migration/colo.h" -#include "migration/vmstate.h" -#include "trace.h" - -typedef struct { - bool colo_requested; -} COLOInfo; - -static COLOInfo colo_info; - -COLOMode get_colo_mode(void) -{ - if (migration_in_colo_state()) { - return COLO_MODE_PRIMARY; - } else if (migration_incoming_in_colo_state()) { - return COLO_MODE_SECONDARY; - } else { - return COLO_MODE_UNKNOWN; - } -} - -static int colo_info_pre_save(void *opaque) -{ - COLOInfo *s = opaque; - - s->colo_requested = migrate_colo_enabled(); - - return 0; -} - -static bool colo_info_need(void *opaque) -{ - return migrate_colo_enabled(); -} - -static const VMStateDescription colo_state = { - .name = "COLOState", - .version_id = 1, - .minimum_version_id = 1, - .pre_save = colo_info_pre_save, - .needed = colo_info_need, - .fields = (VMStateField[]) { - VMSTATE_BOOL(colo_requested, COLOInfo), - VMSTATE_END_OF_LIST() - }, -}; - -void colo_info_init(void) -{ - vmstate_register(NULL, 0, &colo_state, &colo_info); -} - -bool migration_incoming_enable_colo(void) -{ - return colo_info.colo_requested; -} - -void migration_incoming_exit_colo(void) -{ - colo_info.colo_requested = false; -} diff --git a/migration/colo.c b/migration/colo.c index e06640c3d6..c083d3696f 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -152,6 +152,17 @@ static void primary_vm_do_failover(void) qemu_sem_post(&s->colo_exit_sem); } +COLOMode get_colo_mode(void) +{ + if (migration_in_colo_state()) { + return COLO_MODE_PRIMARY; + } else if (migration_incoming_in_colo_state()) { + return COLO_MODE_SECONDARY; + } else { + return COLO_MODE_UNKNOWN; + } +} + void colo_do_failover(MigrationState *s) { /* Make sure VM stopped while failover happened. */ @@ -745,7 +756,7 @@ out: if (mis->to_src_file) { qemu_fclose(mis->to_src_file); } - migration_incoming_exit_colo(); + migration_incoming_disable_colo(); return NULL; } diff --git a/migration/migration.c b/migration/migration.c index e665a1cab0..d57a23dcfd 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -295,6 +295,22 @@ int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char *rbname, return migrate_send_rp_message(mis, msg_type, msglen, bufc); } +static bool migration_colo_enabled; +bool migration_incoming_colo_enabled(void) +{ + return migration_colo_enabled; +} + +void migration_incoming_disable_colo(void) +{ + migration_colo_enabled = false; +} + +void migration_incoming_enable_colo(void) +{ + migration_colo_enabled = true; +} + void qemu_start_incoming_migration(const char *uri, Error **errp) { const char *p; @@ -416,7 +432,7 @@ static void process_incoming_migration_co(void *opaque) } /* we get COLO info, and know if we are in COLO mode */ - if (!ret && migration_incoming_enable_colo()) { + if (!ret && migration_incoming_colo_enabled()) { /* Make sure all file formats flush their mutable metadata */ bdrv_invalidate_cache_all(&local_err); if (local_err) { @@ -2906,6 +2922,11 @@ static void *migration_thread(void *opaque) qemu_savevm_send_postcopy_advise(s->to_dst_file); } + if (migrate_colo_enabled()) { + /* Notify migration destination that we enable COLO */ + qemu_savevm_send_colo_enable(s->to_dst_file); + } + qemu_savevm_state_setup(s->to_dst_file); s->setup_time = qemu_clock_get_ms(QEMU_CLOCK_HOST) - setup_start; diff --git a/migration/savevm.c b/migration/savevm.c index c2f34ffc7c..3e5b85636f 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -56,6 +56,7 @@ #include "io/channel-file.h" #include "sysemu/replay.h" #include "qjson.h" +#include "migration/colo.h" #ifndef ETH_P_RARP #define ETH_P_RARP 0x8035 @@ -83,6 +84,7 @@ enum qemu_vm_cmd { precopy but are dirty. */ MIG_CMD_POSTCOPY_RESUME, /* resume postcopy on dest */ MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream */ + MIG_CMD_ENABLE_COLO, /* Enable COLO */ MIG_CMD_RECV_BITMAP, /* Request for recved bitmap on dst */ MIG_CMD_MAX }; @@ -841,6 +843,12 @@ static void qemu_savevm_command_send(QEMUFile *f, qemu_fflush(f); } +void qemu_savevm_send_colo_enable(QEMUFile *f) +{ + trace_savevm_send_colo_enable(); + qemu_savevm_command_send(f, MIG_CMD_ENABLE_COLO, 0, NULL); +} + void qemu_savevm_send_ping(QEMUFile *f, uint32_t value) { uint32_t buf; @@ -1918,6 +1926,12 @@ static int loadvm_handle_recv_bitmap(MigrationIncomingState *mis, return 0; } +static int loadvm_process_enable_colo(MigrationIncomingState *mis) +{ + migration_incoming_enable_colo(); + return 0; +} + /* * Process an incoming 'QEMU_VM_COMMAND' * 0 just a normal return @@ -1997,6 +2011,9 @@ static int loadvm_process_command(QEMUFile *f) case MIG_CMD_RECV_BITMAP: return loadvm_handle_recv_bitmap(mis, len); + + case MIG_CMD_ENABLE_COLO: + return loadvm_process_enable_colo(mis); } return 0; diff --git a/migration/savevm.h b/migration/savevm.h index a5e65b8ae3..8373c2f6bd 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -55,6 +55,7 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name, uint16_t len, uint64_t *start_list, uint64_t *length_list); +void qemu_savevm_send_colo_enable(QEMUFile *f); int qemu_loadvm_state(QEMUFile *f); void qemu_loadvm_state_cleanup(void); diff --git a/migration/trace-events b/migration/trace-events index 3f67758893..86459d0580 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -37,6 +37,7 @@ savevm_send_ping(uint32_t val) "0x%x" savevm_send_postcopy_listen(void) "" savevm_send_postcopy_run(void) "" savevm_send_postcopy_resume(void) "" +savevm_send_colo_enable(void) "" savevm_send_recv_bitmap(char *name) "%s" savevm_state_setup(void) "" savevm_state_resume_prepare(void) "" diff --git a/vl.c b/vl.c index d26f19b06d..33b723d075 100644 --- a/vl.c +++ b/vl.c @@ -4360,8 +4360,6 @@ int main(int argc, char **argv, char **envp) #endif } - colo_info_init(); - if (net_init_clients(&err) < 0) { error_report_err(err); exit(1); From patchwork Wed Jun 27 20:41:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935716 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="pvnNQg8P"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFXq0k8cz9s19 for ; Thu, 28 Jun 2018 06:55:07 +1000 (AEST) Received: from localhost ([::1]:33162 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHSy-0004f8-Mh for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:55:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49463) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGl-0003sj-1G for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGj-00063R-JL for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:27 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:44821) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGj-00062V-BJ for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:25 -0400 Received: by mail-pf0-x244.google.com with SMTP id j3-v6so1498489pfh.11 for ; Wed, 27 Jun 2018 13:42:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=30eKUXvxzwjFqWsxHDMb6nxNjHROjvAZ4MbIO2v76Dg=; b=pvnNQg8PXmPyI/kJypFGXWgl/ZlduDTPpni5jGRj2/A/L/+bXQ6phxq0SiYn7GO653 wHfmifKuR4h/Fh7Lj/T7G50DkbNgp4BKMAap8taKLR1+vdgtJmFAFJspa97V20sxnit2 SA42un5M0hlIHFh7ZztNBec+t0A4X7cvZvk7yzxiZAL+PrLFOu5cLfjQSCcf4ysi3RwN gHbMgZzy8xnj+8r7c3t8gGOUNzQu1mczinAj37KQcL7/LByd0NrZ0ZM7fbivLL85v0wc 0TsKtdGg8xwrGkKrN7lXrrvwdIFXvqlCBkas1ygVaRV5zGlGHRbm+zxaNjoJYCi1UORU onFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=30eKUXvxzwjFqWsxHDMb6nxNjHROjvAZ4MbIO2v76Dg=; b=RX/yReIyCUt36oERFzCa63gLoWVIhoyMoxx+RGo70XEStSEIpE19sTrsipE/aLcoEY u50YbugYfhmIAXEaQ0PhTsJXFLNx3Yy+3Fv68N7H1ZRpAplGDxRPQzMIkVHN+w+vYpxI jZWARtlEUAIcegRqxsyl0uTaL4s3Til84nXzSTaCS69zpyk3Q5pMWt9cJR5bxwJirzqj lOcGz+KSYAZIW4PZ0sZjAbMoxZCO/6YLZ+iHz5yBTXeYgngay7tAgB0Qu1wf8PbIEjxF JxO6yTKWfPmd7pOslZd8R7EoqJCR3RwVbLldmBCaYKVGg/9xmGmYhkD1KjRuTyNAW5rd XTKw== X-Gm-Message-State: APt69E1ZDJpog4Ejs7Grbuc0fIFqvvuXt1vguGI8i6G3YSEt9PvIG2De 08z1cJKFqoRdpjD4IKYgZrrd2A== X-Google-Smtp-Source: ADUXVKL7C3rn22kdAs29Uau63xAps3tVLlo1iVNQvACLFCU1OmNUt0DPNUFXfWai8lErA0gGzH1gtg== X-Received: by 2002:a63:b213:: with SMTP id x19-v6mr6425133pge.393.1530132144023; Wed, 27 Jun 2018 13:42:24 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:23 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:23 +0800 Message-Id: <20180627204136.4177-8-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PATCH V9 07/20] COLO: Load dirty pages into SVM's RAM cache firstly X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We should not load PVM's state directly into SVM, because there maybe some errors happen when SVM is receving data, which will break SVM. We need to ensure receving all data before load the state into SVM. We use an extra memory to cache these data (PVM's ram). The ram cache in secondary side is initially the same as SVM/PVM's memory. And in the process of checkpoint, we cache the dirty pages of PVM into this ram cache firstly, so this ram cache always the same as PVM's memory at every checkpoint, then we flush this cached ram to SVM after we receive all PVM's state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen --- include/exec/ram_addr.h | 1 + migration/migration.c | 6 +++ migration/ram.c | 83 ++++++++++++++++++++++++++++++++++++++++- migration/ram.h | 4 ++ migration/savevm.c | 2 +- 5 files changed, 93 insertions(+), 3 deletions(-) diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h index cf2446a176..51ec153a57 100644 --- a/include/exec/ram_addr.h +++ b/include/exec/ram_addr.h @@ -27,6 +27,7 @@ struct RAMBlock { struct rcu_head rcu; struct MemoryRegion *mr; uint8_t *host; + uint8_t *colo_cache; /* For colo, VM's ram cache */ ram_addr_t offset; ram_addr_t used_length; ram_addr_t max_length; diff --git a/migration/migration.c b/migration/migration.c index d57a23dcfd..a17f81a245 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -441,6 +441,10 @@ static void process_incoming_migration_co(void *opaque) error_report_err(local_err); exit(EXIT_FAILURE); } + if (colo_init_ram_cache() < 0) { + error_report("Init ram cache failed"); + exit(EXIT_FAILURE); + } mis->migration_incoming_co = qemu_coroutine_self(); qemu_thread_create(&mis->colo_incoming_thread, "COLO incoming", colo_process_incoming_thread, mis, QEMU_THREAD_JOINABLE); @@ -449,6 +453,8 @@ static void process_incoming_migration_co(void *opaque) /* Wait checkpoint incoming thread exit before free resource */ qemu_thread_join(&mis->colo_incoming_thread); + /* We hold the global iothread lock, so it is safe here */ + colo_release_ram_cache(); } if (ret < 0) { diff --git a/migration/ram.c b/migration/ram.c index cd5f55117d..30cc83996f 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2858,6 +2858,20 @@ static inline void *host_from_ram_block_offset(RAMBlock *block, return block->host + offset; } +static inline void *colo_cache_from_block_offset(RAMBlock *block, + ram_addr_t offset) +{ + if (!offset_in_ramblock(block, offset)) { + return NULL; + } + if (!block->colo_cache) { + error_report("%s: colo_cache is NULL in block :%s", + __func__, block->idstr); + return NULL; + } + return block->colo_cache + offset; +} + /** * ram_handle_compressed: handle the zero page case * @@ -3062,6 +3076,58 @@ static void decompress_data_with_multi_threads(QEMUFile *f, qemu_mutex_unlock(&decomp_done_lock); } +/* + * colo cache: this is for secondary VM, we cache the whole + * memory of the secondary VM, it is need to hold the global lock + * to call this helper. + */ +int colo_init_ram_cache(void) +{ + RAMBlock *block; + + rcu_read_lock(); + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + block->colo_cache = qemu_anon_ram_alloc(block->used_length, + NULL, + false); + if (!block->colo_cache) { + error_report("%s: Can't alloc memory for COLO cache of block %s," + "size 0x" RAM_ADDR_FMT, __func__, block->idstr, + block->used_length); + goto out_locked; + } + memcpy(block->colo_cache, block->host, block->used_length); + } + rcu_read_unlock(); + return 0; + +out_locked: + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + if (block->colo_cache) { + qemu_anon_ram_free(block->colo_cache, block->used_length); + block->colo_cache = NULL; + } + } + + rcu_read_unlock(); + return -errno; +} + +/* It is need to hold the global lock to call this helper */ +void colo_release_ram_cache(void) +{ + RAMBlock *block; + + rcu_read_lock(); + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + if (block->colo_cache) { + qemu_anon_ram_free(block->colo_cache, block->used_length); + block->colo_cache = NULL; + } + } + rcu_read_unlock(); +} + /** * ram_load_setup: Setup RAM for migration incoming side * @@ -3078,6 +3144,7 @@ static int ram_load_setup(QEMUFile *f, void *opaque) xbzrle_load_setup(); ramblock_recv_map_init(); + return 0; } @@ -3091,6 +3158,7 @@ static int ram_load_cleanup(void *opaque) g_free(rb->receivedmap); rb->receivedmap = NULL; } + return 0; } @@ -3324,13 +3392,24 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) RAM_SAVE_FLAG_COMPRESS_PAGE | RAM_SAVE_FLAG_XBZRLE)) { RAMBlock *block = ram_block_from_stream(f, flags); - host = host_from_ram_block_offset(block, addr); + /* + * After going into COLO, we should load the Page into colo_cache. + */ + if (migration_incoming_in_colo_state()) { + host = colo_cache_from_block_offset(block, addr); + } else { + host = host_from_ram_block_offset(block, addr); + } if (!host) { error_report("Illegal RAM offset " RAM_ADDR_FMT, addr); ret = -EINVAL; break; } - ramblock_recv_bitmap_set(block, host); + + if (!migration_incoming_in_colo_state()) { + ramblock_recv_bitmap_set(block, host); + } + trace_ram_load_loop(block->idstr, (uint64_t)addr, flags, host); } diff --git a/migration/ram.h b/migration/ram.h index d386f4d641..d5e81d4d48 100644 --- a/migration/ram.h +++ b/migration/ram.h @@ -70,4 +70,8 @@ int64_t ramblock_recv_bitmap_send(QEMUFile *file, const char *block_name); int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *rb); +/* ram cache */ +int colo_init_ram_cache(void); +void colo_release_ram_cache(void); + #endif diff --git a/migration/savevm.c b/migration/savevm.c index 3e5b85636f..fe9ca86a78 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1929,7 +1929,7 @@ static int loadvm_handle_recv_bitmap(MigrationIncomingState *mis, static int loadvm_process_enable_colo(MigrationIncomingState *mis) { migration_incoming_enable_colo(); - return 0; + return colo_init_ram_cache(); } /* From patchwork Wed Jun 27 20:41:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935704 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HMDFTf1+"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFLR4LmVz9s19 for ; Thu, 28 Jun 2018 06:46:07 +1000 (AEST) Received: from localhost ([::1]:33117 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHKG-0006Jj-8Z for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:46:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49514) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGp-0003vc-IE for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGo-0006Az-IO for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:31 -0400 Received: from mail-pl0-x244.google.com ([2607:f8b0:400e:c01::244]:41162) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGo-0006A2-Bb for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:30 -0400 Received: by mail-pl0-x244.google.com with SMTP id w8-v6so1594783ply.8 for ; Wed, 27 Jun 2018 13:42:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BlmXYOMD7aS9b5gmx01wGSp+AEPIzdhdvfz8COXFZ6I=; b=HMDFTf1+hoU/JboktfQ+Ub5PDmGu+X1C2ifTCmjCErvWIC4E+0OpW8xaY8H0CRSPM2 ze9t3lh48u1z8KZasfuW6LUCRwapHEv4rM3FyPuEpKYirkMZj87/EN4Z4SNNoTsIsDA7 pMOy9Pq4ZyrxQ0fvhA2PThAFLcxN3+LCdzFD7oe78+FQ0ufBnFo8ucoayyRM0RIORhL7 Su43qDDTOpk+lM1r5qDt7MprVh0CtEmFK3nZBizT20FhIcjBqIoKPt/wsaSAxzJxHJZA u70en4sUw2e5ntNCXfVoiWW7KYjBlyfszJqyZTZjbbrztSVerQmnioCvw1QnXEMGYKVT OKmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BlmXYOMD7aS9b5gmx01wGSp+AEPIzdhdvfz8COXFZ6I=; b=PeijHQVzmrVjF/aJHsVk28UnCJF4v5cHFS4dBtfJw+mQXta6yEydzgTa3iEhCgtqQS oQqKDzHkUjMXTsG069lVBp/EvybgsHgK2VcZWOdMtUxIhh4sQ3sB7QFSFolbOl2DCIhl ssiwi9qaNjhWXW/3J8SQje2yTi7fxkLu8WI9KmfFGjUKga4RxRn5phoUdwH/3cqa62sc A6b74CVbRWM032/2P7EQ6WChwrl1SlJ1k8Bz9LznaNCarZzN/gVVkCQHWlKSynzidGVU vMc9C0olozlxGqPlgeXEgsMxRDwgTc11ZB64PXPVRYzNNrUoXF2KNtMRrdeAjxOzDSsi 2RTA== X-Gm-Message-State: APt69E2FzH3gOwIcMBGneShDAtlVmPLE5bsbG0iJbENq6B6nyfZMfNEV 3qb1jjSK9cMEP9X498ukhgiU6A== X-Google-Smtp-Source: ADUXVKJJCsMfl6mlout0KSUc/L8nqpH6kUAjD6h5OmK4F7+VWVHFPThNkhqD3GrY7LN3EaABZ6k+gg== X-Received: by 2002:a17:902:8f94:: with SMTP id z20-v6mr7527052plo.337.1530132149095; Wed, 27 Jun 2018 13:42:29 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:28 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:24 +0800 Message-Id: <20180627204136.4177-9-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::244 Subject: [Qemu-devel] [PATCH V9 08/20] ram/COLO: Record the dirty pages that SVM received X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We record the address of the dirty pages that received, it will help flushing pages that cached into SVM. Here, it is a trick, we record dirty pages by re-using migration dirty bitmap. In the later patch, we will start the dirty log for SVM, just like migration, in this way, we can record both the dirty pages caused by PVM and SVM, we only flush those dirty pages from RAM cache while do checkpoint. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 30cc83996f..ecc5f46068 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -2869,6 +2869,15 @@ static inline void *colo_cache_from_block_offset(RAMBlock *block, __func__, block->idstr); return NULL; } + + /* + * During colo checkpoint, we need bitmap of these migrated pages. + * It help us to decide which pages in ram cache should be flushed + * into VM's RAM later. + */ + if (!test_and_set_bit(offset >> TARGET_PAGE_BITS, block->bmap)) { + ram_state->migration_dirty_pages++; + } return block->colo_cache + offset; } @@ -3099,6 +3108,24 @@ int colo_init_ram_cache(void) memcpy(block->colo_cache, block->host, block->used_length); } rcu_read_unlock(); + /* + * Record the dirty pages that sent by PVM, we use this dirty bitmap together + * with to decide which page in cache should be flushed into SVM's RAM. Here + * we use the same name 'ram_bitmap' as for migration. + */ + if (ram_bytes_total()) { + RAMBlock *block; + + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + unsigned long pages = block->max_length >> TARGET_PAGE_BITS; + + block->bmap = bitmap_new(pages); + bitmap_set(block->bmap, 0, pages); + } + } + ram_state = g_new0(RAMState, 1); + ram_state->migration_dirty_pages = 0; + return 0; out_locked: @@ -3118,6 +3145,10 @@ void colo_release_ram_cache(void) { RAMBlock *block; + QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { + g_free(block->bmap); + block->bmap = NULL; + } rcu_read_lock(); QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { if (block->colo_cache) { @@ -3126,6 +3157,8 @@ void colo_release_ram_cache(void) } } rcu_read_unlock(); + g_free(ram_state); + ram_state = NULL; } /** From patchwork Wed Jun 27 20:41:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935705 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HXfpA5Ok"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFNC2mgqz9s19 for ; Thu, 28 Jun 2018 06:47:37 +1000 (AEST) Received: from localhost ([::1]:33120 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHLh-0007Fy-PP for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:47:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49578) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHGu-0003zd-MZ for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGt-0006In-OQ for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:36 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:32983) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGt-0006Hk-Hn for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:35 -0400 Received: by mail-pf0-x241.google.com with SMTP id b17-v6so1506051pfi.0 for ; Wed, 27 Jun 2018 13:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=H+yNpWwmw2fVFWlK+oQ5FKI0hQO0uaKHnCD0IBWFQmY=; b=HXfpA5OkXtaw+ms3wgnJUkvdbJLBAQaftjhHgULqrZsGTSXRakyt1GLY60WNJIq9WB MzZ1kmb0fpXtR+6RDaZlTClgPAPZga+nwNSLy4N6Ko6zGLRRQTDvUFstfzVGS0CmkJVT gdma2v586TbHcHk2qqsBykjmR9HJRGXruWcUOcQpQRqOBupYuj1cLVTDZNaudkkg+ugs fqjLITEUQeD8dANB9dC+BTy8eJpkt75tVgTk0Myocy2+UBqmrsuA67JALEzauAKa4uci 6JXqKRIloKPnTXVUv3wW5JZ5pm0dT5hpBm6v0416q5EU2I0ej4Qq4eR1k0f2kcxe+5xG 68kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=H+yNpWwmw2fVFWlK+oQ5FKI0hQO0uaKHnCD0IBWFQmY=; b=ify2y/dPl1F//sLFN4joNGxfDQJKH4EJe8HKzmWOQBDxZrDZUSkJLggyT5KiYt22mI 8mYD91OjwGUcnVdXgJw/ny6DXBXd/GzSCdlCS/Whp+mT3NoTDu0HaibKfscDdXK3RYpr XZlaOSF07W9dVYw2AFhuHXo94sCMu6+qB5Z3PtFYIldPv5Lj5iTxkAhhQLzURLza2oM9 o4dTFB1s04fJuMKa5YbqcnNgYMlwuDBMpZYaHXuSYK1QbEApT2peVlRDmC8CdepI8dvg WqQBttrHeP3ZUD6/mI0/izP4HenLL0MQ7reg+0FucfMiXoU1rPDMbfFXXjnC+QHm+GBO Gz2A== X-Gm-Message-State: APt69E28j35Rk5EfuScrhwyhXdVG6HsDCkUNSoys27zvlPu/h6u04f3u aY33CtU4pHevHH8KDKn4QsSFIg== X-Google-Smtp-Source: ADUXVKLKfhxmG5uHF85sZQ8+BGw/nnzuzdyza2vkrq5cE68ni8/tbnk7p+oztUrLdWC/4dxIIaNL+A== X-Received: by 2002:a63:8c10:: with SMTP id m16-v6mr6585955pgd.120.1530132154181; Wed, 27 Jun 2018 13:42:34 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:33 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:25 +0800 Message-Id: <20180627204136.4177-10-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PATCH V9 09/20] COLO: Flush memory data from ram cache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" During the time of VM's running, PVM may dirty some pages, we will transfer PVM's dirty pages to SVM and store them into SVM's RAM cache at next checkpoint time. So, the content of SVM's RAM cache will always be same with PVM's memory after checkpoint. Instead of flushing all content of PVM's RAM cache into SVM's MEMORY, we do this in a more efficient way: Only flush any page that dirtied by PVM since last checkpoint. In this way, we can ensure SVM's memory same with PVM's. Besides, we must ensure flush RAM cache before load device state. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 37 +++++++++++++++++++++++++++++++++++++ migration/trace-events | 2 ++ 2 files changed, 39 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index ecc5f46068..4199d64ebd 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3370,6 +3370,39 @@ static bool postcopy_is_running(void) return ps >= POSTCOPY_INCOMING_LISTENING && ps < POSTCOPY_INCOMING_END; } +/* + * Flush content of RAM cache into SVM's memory. + * Only flush the pages that be dirtied by PVM or SVM or both. + */ +static void colo_flush_ram_cache(void) +{ + RAMBlock *block = NULL; + void *dst_host; + void *src_host; + unsigned long offset = 0; + + trace_colo_flush_ram_cache_begin(ram_state->migration_dirty_pages); + rcu_read_lock(); + block = QLIST_FIRST_RCU(&ram_list.blocks); + + while (block) { + offset = migration_bitmap_find_dirty(ram_state, block, offset); + + if (offset << TARGET_PAGE_BITS >= block->used_length) { + offset = 0; + block = QLIST_NEXT_RCU(block, next); + } else { + migration_bitmap_clear_dirty(ram_state, block, offset); + dst_host = block->host + (offset << TARGET_PAGE_BITS); + src_host = block->colo_cache + (offset << TARGET_PAGE_BITS); + memcpy(dst_host, src_host, TARGET_PAGE_SIZE); + } + } + + rcu_read_unlock(); + trace_colo_flush_ram_cache_end(); +} + static int ram_load(QEMUFile *f, void *opaque, int version_id) { int flags = 0, ret = 0, invalid_flags = 0; @@ -3545,6 +3578,10 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) ret |= wait_for_decompress_done(); rcu_read_unlock(); trace_ram_load_complete(ret, seq_iter); + + if (!ret && migration_incoming_in_colo_state()) { + colo_flush_ram_cache(); + } return ret; } diff --git a/migration/trace-events b/migration/trace-events index 86459d0580..6e1b58a6a3 100644 --- a/migration/trace-events +++ b/migration/trace-events @@ -90,6 +90,8 @@ ram_dirty_bitmap_sync_start(void) "" ram_dirty_bitmap_sync_wait(void) "" ram_dirty_bitmap_sync_complete(void) "" ram_state_resume_prepare(uint64_t v) "%" PRId64 +colo_flush_ram_cache_begin(uint64_t dirty_pages) "dirty_pages %" PRIu64 +colo_flush_ram_cache_end(void) "" # migration/migration.c await_return_path_close_on_source_close(void) "" From patchwork Wed Jun 27 20:41:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935708 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="IbXkErN6"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFQP51Bwz9s19 for ; Thu, 28 Jun 2018 06:49:33 +1000 (AEST) Received: from localhost ([::1]:33130 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHNb-0000F3-3f for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:49:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49646) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHH0-00044z-1V for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHGy-0006No-Tj for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:42 -0400 Received: from mail-pl0-x22d.google.com ([2607:f8b0:400e:c01::22d]:39687) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHGy-0006NC-O0 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:40 -0400 Received: by mail-pl0-x22d.google.com with SMTP id s24-v6so1590249plq.6 for ; Wed, 27 Jun 2018 13:42:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RdboUqsFCsnS69Tet/dfnkPc7+x0C1nplx9y+BKOcOs=; b=IbXkErN6CeFb41ekzCW0uFqOhGF+DOiT5aEe0ksqKh2RwxdnEn9f5CcArP7arDbZZw Tvtzm4scqDYzy6tZsW6FAwi9CytZPaMW8IPxulKT8uEU6k6H/LulNDz2ZAUKSitqFq9r VNEjhtzNZNcX+EbDxg9IrWZN8YsxWQD6wtXMuzmrQPAcXtFvvt0RFome46ZVBFrkP7dj 0TrSgWgom9EBNIQQ5R4Z04axF/xAlqRVo2aRsu7ZH77VUIa9LvWOBMcyMeEq4NHJKqi/ d8wkbJ9CMphdflZT1ufil5brIMCkxmaG4fS4oX/X1yurnqBs5Gu/iGHyUoAXk4NfJSAW GKkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RdboUqsFCsnS69Tet/dfnkPc7+x0C1nplx9y+BKOcOs=; b=AT5OS8HNZa7W3f5dYLAeIHsustq7vpazO7K3JiTg5C0YKZkDC6i3RxjN5/v9/4uraY CXnDrGEJmOxUXzX2ZwbnKNys9YLu7hHmI2a6PY+layJxnKQFV6rplmkOjADRq96JpJO5 0cszpLF/oLvBG0XiQprOxsflBcp+ySQa+c36Zv4aKmVdmOmBYlUCW6vrz8QnSOall5jX vmqs/oJkCuaJ4su5wzHWkpD2tkcU4P76mGSDwGP2WeG+jaoGvJLv93t0bi1t5htjRH9E hOhYpEPDVV/3eyN01CxF+jXavrAIkivnLYw7jR1S76JkzrzGrCZjfvb68D93Ki/yrwgT MEyw== X-Gm-Message-State: APt69E1N2Ay1UzJguYokFdSmjGfa0HYDCFjjtWbbcG/rpJ3DBbbWiYec tFL+CG7g8eiHj99V9g3vv0TWrg== X-Google-Smtp-Source: ADUXVKLmlV7R0AId9ZdWClSVkrw3omvWisUDOFvSFS3uii5TOnmvTj8FTyJYZWWp69+a7DbLSvGSoQ== X-Received: by 2002:a17:902:e187:: with SMTP id cd7-v6mr7703301plb.166.1530132159457; Wed, 27 Jun 2018 13:42:39 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:38 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:26 +0800 Message-Id: <20180627204136.4177-11-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22d Subject: [Qemu-devel] [PATCH V9 10/20] qmp event: Add COLO_EXIT event to notify users while exited COLO X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: zhanghailiang If some errors happen during VM's COLO FT stage, it's important to notify the users of this event. Together with 'x-colo-lost-heartbeat', Users can intervene in COLO's failover work immediately. If users don't want to get involved in COLO's failover verdict, it is still necessary to notify users that we exited COLO mode. Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo.c | 31 +++++++++++++++++++++++++++++++ qapi/migration.json | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index c083d3696f..ab484ad754 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -28,6 +28,7 @@ #include "net/colo-compare.h" #include "net/colo.h" #include "block/block.h" +#include "qapi/qapi-events-migration.h" static bool vmstate_loading; static Notifier packets_compare_notifier; @@ -514,6 +515,23 @@ out: qemu_fclose(fb); } + /* + * There are only two reasons we can get here, some error happened + * or the user triggered failover. + */ + switch (failover_get_state()) { + case FAILOVER_STATUS_NONE: + qapi_event_send_colo_exit(COLO_MODE_PRIMARY, + COLO_EXIT_REASON_ERROR, NULL); + break; + case FAILOVER_STATUS_REQUIRE: + qapi_event_send_colo_exit(COLO_MODE_PRIMARY, + COLO_EXIT_REASON_REQUEST, NULL); + break; + default: + abort(); + } + /* Hope this not to be too long to wait here */ qemu_sem_wait(&s->colo_exit_sem); qemu_sem_destroy(&s->colo_exit_sem); @@ -745,6 +763,19 @@ out: error_report_err(local_err); } + switch (failover_get_state()) { + case FAILOVER_STATUS_NONE: + qapi_event_send_colo_exit(COLO_MODE_SECONDARY, + COLO_EXIT_REASON_ERROR, NULL); + break; + case FAILOVER_STATUS_REQUIRE: + qapi_event_send_colo_exit(COLO_MODE_SECONDARY, + COLO_EXIT_REASON_REQUEST, NULL); + break; + default: + abort(); + } + if (fb) { qemu_fclose(fb); } diff --git a/qapi/migration.json b/qapi/migration.json index 1b4c1db670..d8c3b2e443 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -897,6 +897,44 @@ { 'enum': 'FailoverStatus', 'data': [ 'none', 'require', 'active', 'completed', 'relaunch' ] } +## +# @COLO_EXIT: +# +# Emitted when VM finishes COLO mode due to some errors happening or +# at the request of users. +# +# @mode: report COLO mode when COLO exited. +# +# @reason: describes the reason for the COLO exit. +# +# Since: 3.0 +# +# Example: +# +# <- { "timestamp": {"seconds": 2032141960, "microseconds": 417172}, +# "event": "COLO_EXIT", "data": {"mode": "primary", "reason": "request" } } +# +## +{ 'event': 'COLO_EXIT', + 'data': {'mode': 'COLOMode', 'reason': 'COLOExitReason' } } + +## +# @COLOExitReason: +# +# The reason for a COLO exit +# +# @none: no failover has ever happened. This can't occur in the +# COLO_EXIT event, only in the result of query-colo-status. +# +# @request: COLO exit is due to an external request +# +# @error: COLO exit is due to an internal error +# +# Since: 3.0 +## +{ 'enum': 'COLOExitReason', + 'data': [ 'none', 'request', 'error' ] } + ## # @x-colo-lost-heartbeat: # From patchwork Wed Jun 27 20:41:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935710 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="MakkX+cU"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFS00Mx6z9s19 for ; Thu, 28 Jun 2018 06:50:56 +1000 (AEST) Received: from localhost ([::1]:33134 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHOv-0001SI-Hg for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:50:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49725) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHH5-0004AB-Dz for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHH4-0006V1-AV for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:47 -0400 Received: from mail-pg0-x244.google.com ([2607:f8b0:400e:c05::244]:36365) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHH4-0006UE-3e for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:46 -0400 Received: by mail-pg0-x244.google.com with SMTP id m5-v6so1430318pgd.3 for ; Wed, 27 Jun 2018 13:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lV3MMVg+nxJIUYS6pgww3wyyUK7k1YWd2JYqafC9seY=; b=MakkX+cUIcwpk9fs9ug0dPbtK8gboV5N2g59DJ7NrxbZ5QHrRb+4VJM3Pyvc6a7G5l DPLk8L21XeZ3QnvUDPefXlOwFbBQmaHFw/NMazvGC8S+uBfRvoB/e11UhuCbiD7d036g HmEIlm3mPNLLw0u94KkuQev5IxWhSq9zOwdhBYzqOidrXe6oQAw2IDUoWa/yV62hknnC Jg8Yl4pHHnykB79b3sUQ3NmyYzdG/rjQ5NSGPIz2/zjk2H6wNQziiXix65UipYKJcwaO XcZ3dvFz7bhoep8FUEmXUc8cD/nLRZ2LV2M7+FSLg1EMI9lVgE2wFRU3mf8dURRwUbul JTPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lV3MMVg+nxJIUYS6pgww3wyyUK7k1YWd2JYqafC9seY=; b=Bbtktye3929jyufqwlEz67z8Lmf4M0X+hdpJ0Fwr2IZiFAWa6MNdzdNfmQzm2eIZ04 04KRvsAwxOIm9RTg11bAtPrV+IpDYNDysbXPs6yES8DfJxlDGDIA0G2rtB3vl+CChamg yvxzbmjGbOgzFCZdU6Jw2GgYO5bXXTzJzggCPuZjajQ8CWnYPQiZzsUrXkanYbsr3VEC Ye8XTS0C5/B2zGu84mK2SpGZ2x3+BncootRLmpbNMQwVk/zbF+hkaBz1w1pu4aeQu9CK 5Q26UkklFGlMjrBFuJ0QHZ4EHr4E9ZddFoRKq54AS7jfdqW4E2qcssL+muycPnEBfUYi Q0zw== X-Gm-Message-State: APt69E18YCXYnwF0RpYVzdaNjD07xu1MOeDyLmwA8GsOsIKhj/DlsITK 2pi63uLdUIv+7QDUHen6IpmReQ== X-Google-Smtp-Source: AAOMgpe/axD+ZuvS/ka4hYvFQSorK7DChxZZHWKwgaYzZf1gjoiXsiKcMmnmKwSvelvaKzduFR/WOQ== X-Received: by 2002:a62:170c:: with SMTP id 12-v6mr7311338pfx.139.1530132164883; Wed, 27 Jun 2018 13:42:44 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:43 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:27 +0800 Message-Id: <20180627204136.4177-12-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::244 Subject: [Qemu-devel] [PATCH V9 11/20] qapi/migration.json: Rename COLO unknown mode to none mode. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Zhang Chen Suggested by Markus Armbruster rename COLO unknown mode to none mode. Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Reviewed-by: Markus Armbruster --- migration/colo-failover.c | 2 +- migration/colo.c | 2 +- qapi/migration.json | 10 +++++----- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/migration/colo-failover.c b/migration/colo-failover.c index 0ae0c41221..4854a96c92 100644 --- a/migration/colo-failover.c +++ b/migration/colo-failover.c @@ -77,7 +77,7 @@ FailoverStatus failover_get_state(void) void qmp_x_colo_lost_heartbeat(Error **errp) { - if (get_colo_mode() == COLO_MODE_UNKNOWN) { + if (get_colo_mode() == COLO_MODE_NONE) { error_setg(errp, QERR_FEATURE_DISABLED, "colo"); return; } diff --git a/migration/colo.c b/migration/colo.c index ab484ad754..8fdb79ac73 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -160,7 +160,7 @@ COLOMode get_colo_mode(void) } else if (migration_incoming_in_colo_state()) { return COLO_MODE_SECONDARY; } else { - return COLO_MODE_UNKNOWN; + return COLO_MODE_NONE; } } diff --git a/qapi/migration.json b/qapi/migration.json index d8c3b2e443..c24f114104 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -864,18 +864,18 @@ ## # @COLOMode: # -# The colo mode +# The COLO current mode. # -# @unknown: unknown mode +# @none: None mode. # -# @primary: master side +# @primary: COLO node in primary side. # -# @secondary: slave side +# @secondary: COLO node in slave side. # # Since: 2.8 ## { 'enum': 'COLOMode', - 'data': [ 'unknown', 'primary', 'secondary'] } + 'data': [ 'none', 'primary', 'secondary'] } ## # @FailoverStatus: From patchwork Wed Jun 27 20:41:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935719 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="Ufo4yhIB"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFcG0cGsz9s19 for ; Thu, 28 Jun 2018 06:58:06 +1000 (AEST) Received: from localhost ([::1]:33184 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHVr-00072B-Lv for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:58:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49802) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHA-0004Ck-IK for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHH9-0006ay-DL for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:52 -0400 Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:42551) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHH9-0006ab-6k for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:51 -0400 Received: by mail-pg0-x241.google.com with SMTP id c10-v6so1423936pgu.9 for ; Wed, 27 Jun 2018 13:42:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CUHDpejl1Yfbe3KWR9UH3WKXVdZ/Cg9dmENYGdQT85I=; b=Ufo4yhIB2cnCCYyRP3UGWpFg6OgpEhgwC7/yeGDjWPdVLClH5nIbN9kAToeTvfhU6w YEEcuFBW96ejZlKAmjhCnNLHSMfZxagswVUMG3dPfkcUNCWD64twoxv4zLtR7KawIF5F kno4s+AyzmGvaY8ZrxmBVeB/JrnEDBgDBBriXcBRo9atOCBYWt7NjrKbYjs4/xtR+UTG yFxP8cnhWo8yX1N1AS6vlQRNt299li9gHdYQR1gG4Uwaaw30jnmGHni9e19eF/14Q0Bm 5ARpjS/2yJiTf4vHb9zU8RM23R3B/ihXqBWEhu6/SieflKEldsqfCcBrEXyZ0TfFzPfa LcKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CUHDpejl1Yfbe3KWR9UH3WKXVdZ/Cg9dmENYGdQT85I=; b=cIqKg8qMiSHs+Q8fPR0/beA5I5bBXFmMsOfVxRkC4VG3bsKGTwB/3Bd7H74pU6fxvW uPW5iuzneeBKujXzslDIRnalegpe8/O7MpijHbE4/6Q4W5YgX3kZsppYvQwoSqGqCKpE wWYUZpPCknMaAEvt83yP3fd0Z6V67LFZrClLcO94tXcOoSJELY15V1YIi6bPzAXx3Pk6 g3eI6MmtMF+YfrJe4conSJx6R2T/xEQZ99Jg5fYip3KyAdQKE5i6S9N0bFNjVZTV8dZB Oq6SPR/XbKM3vPD1JhD2+I3GvZBBagX7lPj97LSWv8j8RDm/+wjDjsbY01HYppm13YBs /rFA== X-Gm-Message-State: APt69E1l10S6G0yM+hmZoViCqkTQJyg13jni+7iAIT113iWYLjqfT7Vh d6B6FzwCEtxm7scY6viVcV9vCQ== X-Google-Smtp-Source: AAOMgpc65cG28Rnd6bGT0QIpzKoCUw3NITz1mqlB0hef/vWROf3uz1VFhpROaPSztmtj6J82NK/RwA== X-Received: by 2002:a65:4c87:: with SMTP id m7-v6mr6385065pgt.98.1530132170029; Wed, 27 Jun 2018 13:42:50 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:49 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:28 +0800 Message-Id: <20180627204136.4177-13-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::241 Subject: [Qemu-devel] [PATCH V9 12/20] qapi: Add new command to query colo status X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Libvirt or other high level software can use this command query colo status. You can test this command like that: {'execute':'query-colo-status'} Signed-off-by: Zhang Chen --- migration/colo.c | 39 +++++++++++++++++++++++++++++++++++++++ qapi/migration.json | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 8fdb79ac73..eb21978bff 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -29,6 +29,7 @@ #include "net/colo.h" #include "block/block.h" #include "qapi/qapi-events-migration.h" +#include "qapi/qmp/qerror.h" static bool vmstate_loading; static Notifier packets_compare_notifier; @@ -237,6 +238,44 @@ void qmp_xen_colo_do_checkpoint(Error **errp) #endif } +COLOStatus *qmp_query_colo_status(Error **errp) +{ + int state; + COLOStatus *s = g_new0(COLOStatus, 1); + + s->mode = get_colo_mode(); + + switch (s->mode) { + case COLO_MODE_NONE: + error_setg(errp, "COLO is disabled"); + state = MIGRATION_STATUS_NONE; + break; + case COLO_MODE_PRIMARY: + state = migrate_get_current()->state; + break; + case COLO_MODE_SECONDARY: + state = migration_incoming_get_current()->state; + break; + default: + abort(); + } + + s->active = state == MIGRATION_STATUS_COLO; + + switch (failover_get_state()) { + case FAILOVER_STATUS_NONE: + s->reason = COLO_EXIT_REASON_NONE; + break; + case FAILOVER_STATUS_REQUIRE: + s->reason = COLO_EXIT_REASON_REQUEST; + break; + default: + s->reason = COLO_EXIT_REASON_ERROR; + } + + return s; +} + static void colo_send_message(QEMUFile *f, COLOMessage msg, Error **errp) { diff --git a/qapi/migration.json b/qapi/migration.json index c24f114104..73c64686ec 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -1248,6 +1248,40 @@ ## { 'command': 'xen-colo-do-checkpoint' } +## +# @COLOStatus: +# +# The result format for 'query-colo-status'. +# +# @mode: COLO running mode. If COLO is running, this field will return +# 'primary' or 'secondary'. +# +# @active: true if COLO is active. +# +# @reason: describes the reason for the COLO exit. +# +# Since: 3.0 +## +{ 'struct': 'COLOStatus', + 'data': { 'mode': 'COLOMode', 'active': 'bool', 'reason': 'COLOExitReason' } } + +## +# @query-colo-status: +# +# Query COLO status while the vm is running. +# +# Returns: A @COLOStatus object showing the status. +# +# Example: +# +# -> { "execute": "query-colo-status" } +# <- { "return": { "mode": "primary", "active": true, "reason": "request" } } +# +# Since: 3.0 +## +{ 'command': 'query-colo-status', + 'returns': 'COLOStatus' } + ## # @migrate-recover: # From patchwork Wed Jun 27 20:41:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935703 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="ccsXl0YI"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFL82cYRz9s19 for ; Thu, 28 Jun 2018 06:45:52 +1000 (AEST) Received: from localhost ([::1]:33110 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHK1-000662-VD for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:45:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49866) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHJ-0004KL-Ir for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHF-0006fz-4w for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:01 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:37902) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHE-0006fW-Te for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:42:57 -0400 Received: by mail-pf0-x243.google.com with SMTP id j17-v6so1503981pfn.5 for ; Wed, 27 Jun 2018 13:42:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=221mPVv99/yEy8TW3gh2G4e0ACYIWwRzVFIcq6ERD90=; b=ccsXl0YIOPGfQliWzsSraZKiH9L1X5DMhD9MaogSt4ekZqmeWRPiPIcxaQWhNIMVBa bJK0+z+9hEzcrch3AZphlC48rjLjxrKguC6WfiJlNEIPuz1COfxxtmaGScHN+MQIsNFx hBW3yy572/LJJiBoDETJVd7Q8sHBnevqhUaf/9GXI6axEBMY3A6DuEwWcc9q0n7+1rTG JEqMDGHx6n1d6vnPnwDkNtU09/FSsAB9/woo3MDPNhRyAqR6IThzUWdg5FPrsfu0odpF HAyhXu4IDT5j9OMlE78IsfkReoX3oSOW66kiSfJ7zGx68KNM4A8HDl4mnItYSI3QIxQ9 x2Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=221mPVv99/yEy8TW3gh2G4e0ACYIWwRzVFIcq6ERD90=; b=EJwka0mu7l/0X1ii8xOA+oob2MD2xmP7p4W+5oZZqNLAUnz2lAwKqu4iTpkINuy9xd 4u715VBEMCAScBFqn9+bnlaEhOQv2vENKI79oi1t4qQy4+9TiWxstUvUu6m2PU7kepR1 /vcJCB2OcbbSUXYpRlNvjpvsGvrfv6PozPqyXb250USoHxiDoj1qQozuriCABr3lqtg+ eb/s1+wD13AYkKhIa6p4iL9g/hL6E7LC1Tu/aIsSNFJNNn5vE/+Zsij+pvsrJ42MzDJK 48j3R6XEoE4s388/rLngxSgRIdLlt/QjAVPOuyQH7W8tm0OGcBvVoWomkRSB9p8Xl++f sgfQ== X-Gm-Message-State: APt69E2zBZRr28TeBYAxLNNC/uDFBLvXp+juzuM4AXRxRWsmCV8G/9p4 jTbVrNpydBD+zXkptsDBoxAYAQ== X-Google-Smtp-Source: ADUXVKL0AMhhJkt/gLCeXj3vRM6t8jiabdVNz4pn204WajszKeunDBnaXGp2oHM4E+GCjZPQazjXZA== X-Received: by 2002:a65:61d1:: with SMTP id j17-v6mr6609940pgv.447.1530132175567; Wed, 27 Jun 2018 13:42:55 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:54 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:29 +0800 Message-Id: <20180627204136.4177-14-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::243 Subject: [Qemu-devel] [PATCH V9 13/20] savevm: split the process of different stages for loadvm/savevm X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" There are several stages during loadvm/savevm process. In different stage, migration incoming processes different types of sections. We want to control these stages more accuracy, it will benefit COLO performance, we don't have to save type of QEMU_VM_SECTION_START sections everytime while do checkpoint, besides, we want to separate the process of saving/loading memory and devices state. So we add three new helper functions: qemu_load_device_state() and qemu_savevm_live_state() to achieve different process during migration. Besides, we make qemu_loadvm_state_main() and qemu_save_device_state() public, and simplify the codes of qemu_save_device_state() by calling the wrapper qemu_savevm_state_header(). Signed-off-by: zhanghailiang Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 41 ++++++++++++++++++++++++++++++++--------- migration/savevm.c | 36 +++++++++++++++++++++++++++++------- migration/savevm.h | 4 ++++ 3 files changed, 65 insertions(+), 16 deletions(-) diff --git a/migration/colo.c b/migration/colo.c index eb21978bff..a32c8e2244 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -30,6 +30,7 @@ #include "block/block.h" #include "qapi/qapi-events-migration.h" #include "qapi/qmp/qerror.h" +#include "sysemu/cpus.h" static bool vmstate_loading; static Notifier packets_compare_notifier; @@ -419,23 +420,34 @@ static int colo_do_checkpoint_transaction(MigrationState *s, /* Disable block migration */ migrate_set_block_enabled(false, &local_err); - qemu_savevm_state_header(fb); - qemu_savevm_state_setup(fb); qemu_mutex_lock_iothread(); replication_do_checkpoint_all(&local_err); if (local_err) { qemu_mutex_unlock_iothread(); goto out; } - qemu_savevm_state_complete_precopy(fb, false, false); - qemu_mutex_unlock_iothread(); - - qemu_fflush(fb); colo_send_message(s->to_dst_file, COLO_MESSAGE_VMSTATE_SEND, &local_err); if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } + /* Note: device state is saved into buffer */ + ret = qemu_save_device_state(fb); + + qemu_mutex_unlock_iothread(); + if (ret < 0) { goto out; } + /* + * Only save VM's live state, which not including device state. + * TODO: We may need a timeout mechanism to prevent COLO process + * to be blocked here. + */ + qemu_savevm_live_state(s->to_dst_file); + + qemu_fflush(fb); + /* * We need the size of the VMstate data in Secondary side, * With which we can decide how much data should be read. @@ -653,6 +665,7 @@ void *colo_process_incoming_thread(void *opaque) uint64_t total_size; uint64_t value; Error *local_err = NULL; + int ret; qemu_sem_init(&mis->colo_incoming_sem, 0); @@ -725,6 +738,16 @@ void *colo_process_incoming_thread(void *opaque) goto out; } + qemu_mutex_lock_iothread(); + cpu_synchronize_all_pre_loadvm(); + ret = qemu_loadvm_state_main(mis->from_src_file, mis); + qemu_mutex_unlock_iothread(); + + if (ret < 0) { + error_report("Load VM's live state (ram) error"); + goto out; + } + value = colo_receive_message_value(mis->from_src_file, COLO_MESSAGE_VMSTATE_SIZE, &local_err); if (local_err) { @@ -756,10 +779,10 @@ void *colo_process_incoming_thread(void *opaque) } qemu_mutex_lock_iothread(); - qemu_system_reset(SHUTDOWN_CAUSE_NONE); vmstate_loading = true; - if (qemu_loadvm_state(fb) < 0) { - error_report("COLO: loadvm failed"); + ret = qemu_load_device_state(fb); + if (ret < 0) { + error_report("COLO: load device state failed"); qemu_mutex_unlock_iothread(); goto out; } diff --git a/migration/savevm.c b/migration/savevm.c index fe9ca86a78..036d030197 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -1378,13 +1378,21 @@ done: return ret; } -static int qemu_save_device_state(QEMUFile *f) +void qemu_savevm_live_state(QEMUFile *f) { - SaveStateEntry *se; + /* save QEMU_VM_SECTION_END section */ + qemu_savevm_state_complete_precopy(f, true, false); + qemu_put_byte(f, QEMU_VM_EOF); +} - qemu_put_be32(f, QEMU_VM_FILE_MAGIC); - qemu_put_be32(f, QEMU_VM_FILE_VERSION); +int qemu_save_device_state(QEMUFile *f) +{ + SaveStateEntry *se; + if (!migration_in_colo_state()) { + qemu_put_be32(f, QEMU_VM_FILE_MAGIC); + qemu_put_be32(f, QEMU_VM_FILE_VERSION); + } cpu_synchronize_all_states(); QTAILQ_FOREACH(se, &savevm_state.handlers, entry) { @@ -1440,8 +1448,6 @@ enum LoadVMExitCodes { LOADVM_QUIT = 1, }; -static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis); - /* ------ incoming postcopy messages ------ */ /* 'advise' arrives before any transfers just to tell us that a postcopy * *might* happen - it might be skipped if precopy transferred everything @@ -2241,7 +2247,7 @@ static bool postcopy_pause_incoming(MigrationIncomingState *mis) return true; } -static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) +int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis) { uint8_t section_type; int ret = 0; @@ -2414,6 +2420,22 @@ int qemu_loadvm_state(QEMUFile *f) return ret; } +int qemu_load_device_state(QEMUFile *f) +{ + MigrationIncomingState *mis = migration_incoming_get_current(); + int ret; + + /* Load QEMU_VM_SECTION_FULL section */ + ret = qemu_loadvm_state_main(f, mis); + if (ret < 0) { + error_report("Failed to load device state: %d", ret); + return ret; + } + + cpu_synchronize_all_post_init(); + return 0; +} + int save_snapshot(const char *name, Error **errp) { BlockDriverState *bs, *bs1; diff --git a/migration/savevm.h b/migration/savevm.h index 8373c2f6bd..51a4b9caa8 100644 --- a/migration/savevm.h +++ b/migration/savevm.h @@ -56,8 +56,12 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, const char *name, uint64_t *start_list, uint64_t *length_list); void qemu_savevm_send_colo_enable(QEMUFile *f); +void qemu_savevm_live_state(QEMUFile *f); +int qemu_save_device_state(QEMUFile *f); int qemu_loadvm_state(QEMUFile *f); void qemu_loadvm_state_cleanup(void); +int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis); +int qemu_load_device_state(QEMUFile *f); #endif From patchwork Wed Jun 27 20:41:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935721 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="my/xJkMO"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFg03HKhz9s19 for ; Thu, 28 Jun 2018 07:00:28 +1000 (AEST) Received: from localhost ([::1]:33195 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHYA-0000PE-39 for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 17:00:26 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49896) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHK-0004Kc-S7 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHK-0006km-1q for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:02 -0400 Received: from mail-pg0-x241.google.com ([2607:f8b0:400e:c05::241]:37538) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHJ-0006k3-Sh for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:02 -0400 Received: by mail-pg0-x241.google.com with SMTP id o11-v6so1427815pgv.4 for ; Wed, 27 Jun 2018 13:43:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WaxGjCXY9Ei05zOa9jv5htcGx2JnN5cSYbe3SQMjPPo=; b=my/xJkMOGmo2O1t792h4/qayC+ikzg/BqkVza1Vx9wY27HbXazdCwBOkR4yrLb+9Sd Tl1aBANqxjAU//UGAonCgxm/oobk2au0A12GZGJIxnUw9ofR3HB5YfO/au7a+RQJ8LpY hW3BHRdfBQDGW+8nrcJnI7lZxEPlCoI6iHMc5SDyzayt3VwrrmxzLHOKqdauUT4wrP0W CNuLKaX7Mg/tgkEa9hU43BTcm+z78HAjD5wGWgOyevpF/uZGIi21cRmDC0tjsH63uxUs UqpxBkcqhY4LCFDGk/gq8KozDOx+1mWgBXhuj5BhoZB3d2mkrRonuhlWokaG2u0FkGbx DM6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WaxGjCXY9Ei05zOa9jv5htcGx2JnN5cSYbe3SQMjPPo=; b=JfmkxcLqH1ksGReKazTXm/fTT5BZIfTzO0U2vK6sKQ5vevanDgyfkyJHOhoX/chqSM v79lcgS5PWC9KE3a7zt5+/t7K6vvC3dHN7YjN6nhqJtuGtEVbVsXrZSWtbXY4jtrZwv1 yYDxFWZf4P3ucu5fEPeS8Zs7oI5QOE91obuPdqkejkuS1nM/M7fYrghUErmVnlCM1a9S q3SpDiNf7kIM/jBM/POlK6Q90rLTK96lEU/RjwnhDXSdLtlQAaJZORc7pOzWVmlJh9gu pXivWI1H72uF6PgfIW4CWnfVucdi6xHSb9V6ypTK8JfO6euBH5W5LW8hcF0OLApwL6+h mCoQ== X-Gm-Message-State: APt69E0LZZR8PT1Wi9LtUk90BLAGGq+9mNmMfrlpRaa01Nv24ny83K2d HFTjgN9WwguXIgLkZtdcy+c+gg== X-Google-Smtp-Source: ADUXVKK0ru75GQUxhnHh2MbEKE21i195mgzBaihBLDVmwyPtJoFhVCrUZVdu0weIaWBJYBpTDXUZmQ== X-Received: by 2002:a65:60d2:: with SMTP id r18-v6mr6508371pgv.306.1530132180698; Wed, 27 Jun 2018 13:43:00 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.42.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:42:59 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:30 +0800 Message-Id: <20180627204136.4177-15-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::241 Subject: [Qemu-devel] [PATCH V9 14/20] COLO: flush host dirty ram from cache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: zhanghailiang Don't need to flush all VM's ram from cache, only flush the dirty pages since last checkpoint Signed-off-by: Li Zhijian Signed-off-by: Zhang Chen Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/ram.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index 4199d64ebd..fc0151cbfc 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -3125,6 +3125,7 @@ int colo_init_ram_cache(void) } ram_state = g_new0(RAMState, 1); ram_state->migration_dirty_pages = 0; + memory_global_dirty_log_start(); return 0; @@ -3145,10 +3146,12 @@ void colo_release_ram_cache(void) { RAMBlock *block; + memory_global_dirty_log_stop(); QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { g_free(block->bmap); block->bmap = NULL; } + rcu_read_lock(); QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { if (block->colo_cache) { @@ -3381,6 +3384,13 @@ static void colo_flush_ram_cache(void) void *src_host; unsigned long offset = 0; + memory_global_dirty_log_sync(); + rcu_read_lock(); + RAMBLOCK_FOREACH_MIGRATABLE(block) { + migration_bitmap_sync_range(ram_state, block, 0, block->used_length); + } + rcu_read_unlock(); + trace_colo_flush_ram_cache_begin(ram_state->migration_dirty_pages); rcu_read_lock(); block = QLIST_FIRST_RCU(&ram_list.blocks); From patchwork Wed Jun 27 20:41:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935722 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="HHat3ZKG"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFjt2kDpz9s19 for ; Thu, 28 Jun 2018 07:02:57 +1000 (AEST) Received: from localhost ([::1]:33214 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHaY-0002CR-1r for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 17:02:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49952) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHQ-0004Qq-EE for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHP-0006rm-Ik for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:08 -0400 Received: from mail-pl0-x243.google.com ([2607:f8b0:400e:c01::243]:39601) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHP-0006qu-Bv for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:07 -0400 Received: by mail-pl0-x243.google.com with SMTP id s24-v6so1590772plq.6 for ; Wed, 27 Jun 2018 13:43:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Abb4T//uHh4P68qhN6MzsusgRCJnnTDffWntB+KBgsQ=; b=HHat3ZKGrxrs5+IXfrntd/CjHX6ECrYyXGTHJrj2IiRGaXiVPCki+24ZCccamG5IX+ ImerzWSO1vd8Mv+O9S/OsUEA66JZp3LPNfEVvLvA2ECwGjUVYKeR90uGYYamfZcppBq+ dzGkn/X6a6dwG5mk3ELpGwHqLsTwhzZnAprHAP8dAOGn7Lrpi2vnMAQEdI/miOXfjKop FPb4TGjUJ1AK9mOvKTKtsyXajJfPrRClDFclf1Cr/QDVrtPncB73l798kKYqWMc9OdBx z6ZWfu8QYUV85G1f7pLH3xFIawUv7Ork4yuAXBGvbu/21kierFf2zxBi27AYJ8x9xRZG UHzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Abb4T//uHh4P68qhN6MzsusgRCJnnTDffWntB+KBgsQ=; b=QXcZQalFh8dAJaEnEiRBCVxpEd8zSSOnVfW/mNvAmdPwt9gXtx7HbI8Eq7QTr1ERNy nKC27zfKsQjm20tMhIpZ2pLlWewYSu20ZmbestZDCkqapgH+Sx4hud46A3JKXLjduQhF yw3Q9nvDtDroiHfSwN1d9jG/crhLcbUuvCNRLUuehI5FX1jj7r1WPmplkcYL3dPok+mB RSnLIPjP5V+KLc/ZK3stC5CSTbrAkb6jtCzSGSiFedHIKoDZVwhbbfczlQqosw4yhnwt p1ww+cWc027qpJ1J0InQUjCLXC4I7C+qea2mXDAOn47AUneyuLqvTRZDr29FxS3QxCVA OEBg== X-Gm-Message-State: APt69E15GzXX7i5l3FfULeA+zJey/luntly6uDyxg2W9IpL5QLOv3xaA vEg0cTcuI2h1qKptkyEPEgeSFw== X-Google-Smtp-Source: ADUXVKKM/jAxdk35YOL8QjcaoGhqdu0DYEoD/yd0gsvtUAB141DBOMIW1If11rCbv3OFjG0Vhh5uMg== X-Received: by 2002:a17:902:6ac7:: with SMTP id i7-v6mr7808933plt.288.1530132186073; Wed, 27 Jun 2018 13:43:06 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:05 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:31 +0800 Message-Id: <20180627204136.4177-16-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::243 Subject: [Qemu-devel] [PATCH V9 15/20] net/net.c: Add net client type check function for COLO X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Zhang Chen We add is_colo_support_client_type() to check the net client type for COLO-compare. Currently we just support TAP. Suggested by Jason. Signed-off-by: Zhang Chen --- include/net/net.h | 1 + net/colo-compare.c | 5 +++++ net/net.c | 14 ++++++++++++++ 3 files changed, 20 insertions(+) diff --git a/include/net/net.h b/include/net/net.h index 1425960f76..dcbc7ba9c0 100644 --- a/include/net/net.h +++ b/include/net/net.h @@ -212,6 +212,7 @@ void hmp_host_net_add(Monitor *mon, const QDict *qdict); void hmp_host_net_remove(Monitor *mon, const QDict *qdict); void netdev_add(QemuOpts *opts, Error **errp); void qmp_netdev_add(QDict *qdict, QObject **ret, Error **errp); +bool is_colo_support_client_type(void); int net_hub_id_for_client(NetClientState *nc, int *id); NetClientState *net_hub_port_find(int hub_id); diff --git a/net/colo-compare.c b/net/colo-compare.c index 426eab5973..b8c0240725 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -996,6 +996,11 @@ static void colo_compare_complete(UserCreatable *uc, Error **errp) return; } + if (!is_colo_support_client_type()) { + error_setg(errp, "COLO-compare: Net client type is not supported"); + return; + } + net_socket_rs_init(&s->pri_rs, compare_pri_rs_finalize, s->vnet_hdr); net_socket_rs_init(&s->sec_rs, compare_sec_rs_finalize, s->vnet_hdr); diff --git a/net/net.c b/net/net.c index 2a3133990c..a77ea88fff 100644 --- a/net/net.c +++ b/net/net.c @@ -1733,3 +1733,17 @@ int net_fill_rstate(SocketReadState *rs, const uint8_t *buf, int size) assert(size == 0); return 0; } + +/* Currently, COLO just support TAP */ +bool is_colo_support_client_type(void) +{ + NetClientState *nc; + + QTAILQ_FOREACH(nc, &net_clients, next) { + if (nc->info->type != NET_CLIENT_DRIVER_TAP) { + return false; + } + } + + return true; +} From patchwork Wed Jun 27 20:41:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935713 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="LHU9cWIu"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFV42brrz9s19 for ; Thu, 28 Jun 2018 06:52:42 +1000 (AEST) Received: from localhost ([::1]:33147 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHQd-0002sy-IL for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:52:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50031) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHV-0004YZ-Me for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHU-0006xr-Lf for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:13 -0400 Received: from mail-pf0-x22a.google.com ([2607:f8b0:400e:c00::22a]:35755) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHU-0006wl-Ef for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:12 -0400 Received: by mail-pf0-x22a.google.com with SMTP id c22-v6so1504959pfi.2 for ; Wed, 27 Jun 2018 13:43:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ktX0J9bUdPM6g6Qi3dOomt5iQ9JyknqMDbDqhS9ZkG4=; b=LHU9cWIuCGP1YP/cWXBnLlheTQAOncXqJCQW3swsDGwJnpgbUgqTN7qHDhP2l1j1YT hEi/fvmHX0Utu6skTNafoqq+VHKQPwK+iGHd3RfzP8luKRdL7fC0YgbxF9EFn0DT0D94 gKECsEow3rzcnCv/WQ8vmQbkeOXFbmt/3QDUX1mmt4NP0k6AvppOGTwFRqYY0J0C0R0z 5kfy6UBbdaAVnJsGJApFVfbxbQcQlv7nOQdlv4WZkCBIAqaF/PYgFrnbf/d3a36OHm9m arCQfOUzXcDTPkwQPMgBqmo3FF36kuQcyV0EkS/ISXOe6sUGBK5XnQbz/NkaCpC1BJth cipg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ktX0J9bUdPM6g6Qi3dOomt5iQ9JyknqMDbDqhS9ZkG4=; b=HwVemI8HGS+QeJ/v8W2Fk5zQea1vmTCRGoh4zyGM4pGEdd7iRjUynW2ccAeb1WPd/0 nGPRvP7cDwXDb01gqpEk/6EvtSzVaahT0gYIdI9dsjt1nvgFNscGUqFIQ2tLr28awfo6 U2WP6C/xiCJAaKT5UKtvGNrUiLL8v3r/sz7P/GllIdMgFXu7Lkl3IbqP7LbNHsiikKSx zP9IWIaatRFjOBIZNCYTYT3aR2kdZmf99F8d6wD5tcVzzg1OqYR27s5JNChAh3WTlmKO hsGa12b5frsGPjbzJKEnA2UCijOjrhRQsw3Xy/epYX1GqDrm5lF0hhl0uAGlzwsucr7T THZA== X-Gm-Message-State: APt69E0SSSGFlimym3bd5p/HXSJWBNDkOnMbPp5dx3UAyVIZLvrFvmry B4g5DdhPNaYUy5CJPnM1xdKZrA== X-Google-Smtp-Source: AAOMgpf0oYA2jApftsEntNcOEqSp+x3LEcpmpMcN+MNYCH00bvhldizNhHl3opS2KSZc9FRY7FwFwQ== X-Received: by 2002:a62:494f:: with SMTP id w76-v6mr7319374pfa.152.1530132191159; Wed, 27 Jun 2018 13:43:11 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:10 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:32 +0800 Message-Id: <20180627204136.4177-17-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::22a Subject: [Qemu-devel] [PATCH V9 16/20] filter: Add handle_event method for NetFilterClass X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Filter needs to process the event of checkpoint/failover or other event passed by COLO frame. Signed-off-by: zhanghailiang --- include/net/filter.h | 5 +++++ net/filter.c | 17 +++++++++++++++++ net/net.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) diff --git a/include/net/filter.h b/include/net/filter.h index 435acd6f82..49da666ac0 100644 --- a/include/net/filter.h +++ b/include/net/filter.h @@ -38,6 +38,8 @@ typedef ssize_t (FilterReceiveIOV)(NetFilterState *nc, typedef void (FilterStatusChanged) (NetFilterState *nf, Error **errp); +typedef void (FilterHandleEvent) (NetFilterState *nf, int event, Error **errp); + typedef struct NetFilterClass { ObjectClass parent_class; @@ -45,6 +47,7 @@ typedef struct NetFilterClass { FilterSetup *setup; FilterCleanup *cleanup; FilterStatusChanged *status_changed; + FilterHandleEvent *handle_event; /* mandatory */ FilterReceiveIOV *receive_iov; } NetFilterClass; @@ -77,4 +80,6 @@ ssize_t qemu_netfilter_pass_to_next(NetClientState *sender, int iovcnt, void *opaque); +void colo_notify_filters_event(int event, Error **errp); + #endif /* QEMU_NET_FILTER_H */ diff --git a/net/filter.c b/net/filter.c index 2fd7d7d663..0f17eba143 100644 --- a/net/filter.c +++ b/net/filter.c @@ -17,6 +17,8 @@ #include "net/vhost_net.h" #include "qom/object_interfaces.h" #include "qemu/iov.h" +#include "net/colo.h" +#include "migration/colo.h" static inline bool qemu_can_skip_netfilter(NetFilterState *nf) { @@ -245,11 +247,26 @@ static void netfilter_finalize(Object *obj) g_free(nf->netdev_id); } +static void dummy_handle_event(NetFilterState *nf, int event, Error **errp) +{ + switch (event) { + case COLO_EVENT_CHECKPOINT: + break; + case COLO_EVENT_FAILOVER: + object_property_set_str(OBJECT(nf), "off", "status", errp); + break; + default: + break; + } +} + static void netfilter_class_init(ObjectClass *oc, void *data) { UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc); + NetFilterClass *nfc = NETFILTER_CLASS(oc); ucc->complete = netfilter_complete; + nfc->handle_event = dummy_handle_event; } static const TypeInfo netfilter_info = { diff --git a/net/net.c b/net/net.c index a77ea88fff..b4f6a2efb2 100644 --- a/net/net.c +++ b/net/net.c @@ -1331,6 +1331,34 @@ void hmp_info_network(Monitor *mon, const QDict *qdict) } } +void colo_notify_filters_event(int event, Error **errp) +{ + NetClientState *nc, *peer; + NetClientDriver type; + NetFilterState *nf; + NetFilterClass *nfc = NULL; + Error *local_err = NULL; + + QTAILQ_FOREACH(nc, &net_clients, next) { + peer = nc->peer; + type = nc->info->type; + if (!peer || type != NET_CLIENT_DRIVER_TAP) { + continue; + } + QTAILQ_FOREACH(nf, &nc->filters, next) { + nfc = NETFILTER_GET_CLASS(OBJECT(nf)); + if (!nfc->handle_event) { + continue; + } + nfc->handle_event(nf, event, &local_err); + if (local_err) { + error_propagate(errp, local_err); + return; + } + } + } +} + void qmp_set_link(const char *name, bool up, Error **errp) { NetClientState *ncs[MAX_QUEUE_NUM]; From patchwork Wed Jun 27 20:41:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="g/ypvNvf"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFWC4wk2z9s19 for ; Thu, 28 Jun 2018 06:53:43 +1000 (AEST) Received: from localhost ([::1]:33153 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHRd-0003j9-Aq for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:53:41 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50106) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHb-0004g1-8P for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHa-00074O-0B for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:19 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:36464) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHZ-00073P-MQ for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:17 -0400 Received: by mail-pf0-x244.google.com with SMTP id u16-v6so1505382pfh.3 for ; Wed, 27 Jun 2018 13:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xOiP6weBN/x09k0IWsAa49sSKZnMmtRRxbyT4cOFDDQ=; b=g/ypvNvf2IViCP+XEy3zVY028M4XJDyjbvDj0S1leLNWmz5UJXnM5DRtIEMX6aaeSx sTTwuLbGYL4K5Eo3f7im42f7ddYuM1IapW/P8jRfiN5LAbCU41LZEl721AGTakB31nNw rOVy1MdHxdiBi3UmvDs9gxOE8POiaOCqLIqqnxVoKUalaPeHmnOstVG92s/Yv7jXZQdf LqTq5dROVMWQd5ykSBVkNYGeVeXgCShS15QP0flrCbvp+12RFPNwBtmQ/WEAN1NfJOke LBgfos+DM3OWuBd1DiBUh1P5BMwEr7ZKaOM4bVJjNAB5CNCDpzE/dJ0lfEaITeu4YbQt toeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xOiP6weBN/x09k0IWsAa49sSKZnMmtRRxbyT4cOFDDQ=; b=Vgvhv+NJokVJ9rQ4toBp0oR8H8KUFbxWcv5TIRJ2GBSODjv7UOmlqS1hxL4/w7B/oD JsNlSfjvUc6WfThp7S0RnRLMjG+5WSBpma6sMRvcJ3W5ZiDlYiLtyX5Nw81CHiv4VPDb tvsh2H1sJ0p6mKUEhZEaCnbB8WdteUnWnsHRmhP8kRwNkPLgmzfUHiDF4LTySiFiDe2u fSUrJuIrltWukgohvdCZ7uo8lWbRO3V8JmWim7mx5a8ds/7c96nVIN6P3C94fWvIu+mG n6XxB+ESm3PKvJE+ErNWPv90RcHWiv8rLD3kzuqGvT7QAvVTdk5QMU3NLhD2S3/0xBva KKzQ== X-Gm-Message-State: APt69E1BYOxUE9F5LVlfx6aaJKGzSNWTn7OZq1V7f1JOZ7QdNjihzC/N fSDlpQaMsYLua6xjZu7W1ikJEw== X-Google-Smtp-Source: AAOMgpfRcbtZuJP//RTlBZfSD8neoF29vYJdZ3jMLK8B3uq3wVd9zaYIf8EK3u2SFicXkNNPfq0ivQ== X-Received: by 2002:a62:d34a:: with SMTP id q71-v6mr2183637pfg.17.1530132196472; Wed, 27 Jun 2018 13:43:16 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:15 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:33 +0800 Message-Id: <20180627204136.4177-18-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PATCH V9 17/20] filter-rewriter: handle checkpoint and failover event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" After one round of checkpoint, the states between PVM and SVM become consistent, so it is unnecessary to adjust the sequence of net packets for old connections, besides, while failover happens, filter-rewriter will into failover mode that needn't handle the new TCP connection. Signed-off-by: zhanghailiang Signed-off-by: Zhang Chen --- net/colo-compare.c | 12 ++++----- net/colo.c | 8 ++++++ net/colo.h | 2 ++ net/filter-rewriter.c | 57 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 73 insertions(+), 6 deletions(-) diff --git a/net/colo-compare.c b/net/colo-compare.c index b8c0240725..462e822be6 100644 --- a/net/colo-compare.c +++ b/net/colo-compare.c @@ -116,6 +116,12 @@ enum { SECONDARY_IN, }; +static void colo_compare_inconsistency_notify(void) +{ + notifier_list_notify(&colo_compare_notifiers, + migrate_get_current()); +} + static int compare_chr_send(CompareState *s, const uint8_t *buf, uint32_t size, @@ -562,12 +568,6 @@ static int colo_old_packet_check_one(Packet *pkt, int64_t *check_time) } } -static void colo_compare_inconsistency_notify(void) -{ - notifier_list_notify(&colo_compare_notifiers, - migrate_get_current()); -} - void colo_compare_register_notifier(Notifier *notify) { notifier_list_add(&colo_compare_notifiers, notify); diff --git a/net/colo.c b/net/colo.c index 97c8fc928f..49176bf07b 100644 --- a/net/colo.c +++ b/net/colo.c @@ -221,3 +221,11 @@ Connection *connection_get(GHashTable *connection_track_table, return conn; } + +bool connection_has_tracked(GHashTable *connection_track_table, + ConnectionKey *key) +{ + Connection *conn = g_hash_table_lookup(connection_track_table, key); + + return conn ? true : false; +} diff --git a/net/colo.h b/net/colo.h index 0277e0e9ba..11c5226488 100644 --- a/net/colo.h +++ b/net/colo.h @@ -98,6 +98,8 @@ void connection_destroy(void *opaque); Connection *connection_get(GHashTable *connection_track_table, ConnectionKey *key, GQueue *conn_list); +bool connection_has_tracked(GHashTable *connection_track_table, + ConnectionKey *key); void connection_hashtable_reset(GHashTable *connection_track_table); Packet *packet_new(const void *data, int size, int vnet_hdr_len); void packet_destroy(void *opaque, void *user_data); diff --git a/net/filter-rewriter.c b/net/filter-rewriter.c index f18a71bf2e..c463a0c1d0 100644 --- a/net/filter-rewriter.c +++ b/net/filter-rewriter.c @@ -20,11 +20,15 @@ #include "qemu/main-loop.h" #include "qemu/iov.h" #include "net/checksum.h" +#include "net/colo.h" +#include "migration/colo.h" #define FILTER_COLO_REWRITER(obj) \ OBJECT_CHECK(RewriterState, (obj), TYPE_FILTER_REWRITER) #define TYPE_FILTER_REWRITER "filter-rewriter" +#define FAILOVER_MODE_ON true +#define FAILOVER_MODE_OFF false typedef struct RewriterState { NetFilterState parent_obj; @@ -32,8 +36,14 @@ typedef struct RewriterState { /* hashtable to save connection */ GHashTable *connection_track_table; bool vnet_hdr; + bool failover_mode; } RewriterState; +static void filter_rewriter_failover_mode(RewriterState *s) +{ + s->failover_mode = FAILOVER_MODE_ON; +} + static void filter_rewriter_flush(NetFilterState *nf) { RewriterState *s = FILTER_COLO_REWRITER(nf); @@ -269,6 +279,13 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf, */ reverse_connection_key(&key); } + + /* After failover we needn't change new TCP packet */ + if (s->failover_mode && + connection_has_tracked(s->connection_track_table, &key)) { + goto out; + } + conn = connection_get(s->connection_track_table, &key, NULL); @@ -302,11 +319,49 @@ static ssize_t colo_rewriter_receive_iov(NetFilterState *nf, } } +out: packet_destroy(pkt, NULL); pkt = NULL; return 0; } +static void reset_seq_offset(gpointer key, gpointer value, gpointer user_data) +{ + Connection *conn = (Connection *)value; + + conn->offset = 0; +} + +static gboolean offset_is_nonzero(gpointer key, + gpointer value, + gpointer user_data) +{ + Connection *conn = (Connection *)value; + + return conn->offset ? true : false; +} + +static void colo_rewriter_handle_event(NetFilterState *nf, int event, + Error **errp) +{ + RewriterState *rs = FILTER_COLO_REWRITER(nf); + + switch (event) { + case COLO_EVENT_CHECKPOINT: + g_hash_table_foreach(rs->connection_track_table, + reset_seq_offset, NULL); + break; + case COLO_EVENT_FAILOVER: + if (!g_hash_table_find(rs->connection_track_table, + offset_is_nonzero, NULL)) { + filter_rewriter_failover_mode(rs); + } + break; + default: + break; + } +} + static void colo_rewriter_cleanup(NetFilterState *nf) { RewriterState *s = FILTER_COLO_REWRITER(nf); @@ -350,6 +405,7 @@ static void filter_rewriter_init(Object *obj) RewriterState *s = FILTER_COLO_REWRITER(obj); s->vnet_hdr = false; + s->failover_mode = FAILOVER_MODE_OFF; object_property_add_bool(obj, "vnet_hdr_support", filter_rewriter_get_vnet_hdr, filter_rewriter_set_vnet_hdr, NULL); @@ -362,6 +418,7 @@ static void colo_rewriter_class_init(ObjectClass *oc, void *data) nfc->setup = colo_rewriter_setup; nfc->cleanup = colo_rewriter_cleanup; nfc->receive_iov = colo_rewriter_receive_iov; + nfc->handle_event = colo_rewriter_handle_event; } static const TypeInfo colo_rewriter_info = { From patchwork Wed Jun 27 20:41:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935707 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="U7VZVAoa"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFQ822hTz9s19 for ; Thu, 28 Jun 2018 06:49:20 +1000 (AEST) Received: from localhost ([::1]:33128 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHNN-00004z-VK for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:49:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50200) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHg-0004iy-Cu for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHf-00079N-0Y for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:24 -0400 Received: from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]:36366) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHe-00078i-Py for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:22 -0400 Received: by mail-pg0-x243.google.com with SMTP id m5-v6so1430889pgd.3 for ; Wed, 27 Jun 2018 13:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z+1kNOSm5VHVyAAnQmFmRuVBcI2IKRsDYoqjEPHEH9I=; b=U7VZVAoaLaLy/2qRmi369pAIDbhPeTkfFs3y9MZb7vCvtppTkIWXau5lBf+COXC8g0 DM/syjbaH2wHmiJ5DkQ8EG1U2cPSog1K/TvoqTh834Untt1Z3bNk7BUwuCA0FUKmGvRo MrA6ft+m/MTtqko4GXvKmTNLNnj1iwUr3F5gbVV1xIhTLXe47gZB9vBpZz/j9++EyPhS ykc/MIMPK94JkyBZz/UiZWxeS2nlt7nTEvbTqtvTEwVqJAKQycc1GaSenFb2yde87SAq e7KFcxcl4rcRbUjPPLWtxuOKNoY9IVk4fFZLUJg8SuvskibnbDqxk0RyNeUt8f6YdrN8 d3ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z+1kNOSm5VHVyAAnQmFmRuVBcI2IKRsDYoqjEPHEH9I=; b=cDxOYwn9u9F5mPTl0AQwA+VFgoIgywWmRmn4+g3WuS3xjxb8m5ZhMwkdY7MzrS569r wG8Z/UdNBPkf0PkjLkp5zNkY7v5Qgr3ZfEzUr08FhNkc/aSjl4axZl8CTdkwSH3WPKzc sgD9V0ZZHBl3h3tzugD4NeLYBpvz8lTm+nKbdoYPBGa3GWFXo5JTgE3aE4942sL4jLvw Uo2VHBEcX80WtIJ34jFejaH2Q/zdtdAEMhqYKY9udgG13P1NJbz+FdZ8oZxjfhPexdMw HeENWwjfrh8Fg5WvcP2WF0StIDdUffQT5wPqfH3pBtgwEo0WNVEz68mgomc2j7tjQrSr MtKw== X-Gm-Message-State: APt69E0l6odn4JoFSabflrj3GMJu0joRCUwBsLrVFV9WXefASWZtdUZC l19DmrstRdcluk3OBUUuFgtPlQ== X-Google-Smtp-Source: ADUXVKJBaum8LJ1c5JDa7OSkPh8n6pGTWWm19IodQejRC/GTiMv+xCNw5JXpYWh1xeWr1amoAJptEQ== X-Received: by 2002:a63:89c7:: with SMTP id v190-v6mr6333637pgd.194.1530132201573; Wed, 27 Jun 2018 13:43:21 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:20 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:34 +0800 Message-Id: <20180627204136.4177-19-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::243 Subject: [Qemu-devel] [PATCH V9 18/20] COLO: notify net filters about checkpoint/failover event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: zhanghailiang Notify all net filters about the checkpoint and failover event. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index a32c8e2244..4d0ee7d8aa 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -31,6 +31,7 @@ #include "qapi/qapi-events-migration.h" #include "qapi/qmp/qerror.h" #include "sysemu/cpus.h" +#include "net/filter.h" static bool vmstate_loading; static Notifier packets_compare_notifier; @@ -83,6 +84,12 @@ static void secondary_vm_do_failover(void) error_report_err(local_err); } + /* Notify all filters of all NIC to do checkpoint */ + colo_notify_filters_event(COLO_EVENT_FAILOVER, &local_err); + if (local_err) { + error_report_err(local_err); + } + if (!autostart) { error_report("\"-S\" qemu option will be ignored in secondary side"); /* recover runstate to normal migration finish state */ @@ -799,6 +806,14 @@ void *colo_process_incoming_thread(void *opaque) goto out; } + /* Notify all filters of all NIC to do checkpoint */ + colo_notify_filters_event(COLO_EVENT_CHECKPOINT, &local_err); + + if (local_err) { + qemu_mutex_unlock_iothread(); + goto out; + } + vmstate_loading = false; vm_start(); trace_colo_vm_state_change("stop", "run"); From patchwork Wed Jun 27 20:41:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="myi9J2n2"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFZX1Xp7z9s19 for ; Thu, 28 Jun 2018 06:56:36 +1000 (AEST) Received: from localhost ([::1]:33177 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHUP-0005pQ-PC for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 16:56:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50270) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHk-0004nB-QE for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHk-0007FW-0v for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:28 -0400 Received: from mail-pl0-x243.google.com ([2607:f8b0:400e:c01::243]:39602) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHj-0007EM-RL for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:27 -0400 Received: by mail-pl0-x243.google.com with SMTP id s24-v6so1591157plq.6 for ; Wed, 27 Jun 2018 13:43:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ragI1+v4Vn2KDxHmu3KSKIVydBgNG/mgE2wgjdn2G64=; b=myi9J2n2znbQxhQAsWw2SlCC2LxJAi2L78JEFsrSm3AQjUIlzknEi6yH8j7DnBaSTE cCVqQXBc93gBdtdqAVkHZrFv6cENmDE5OPMF2DsRBLyBaqO0XHYSwITa7vK7dYMZQ2BN v/f0bio73oP3wrnV0pUOHIFH2pZ0A+G0P9IykeHjLiOw4D1f5I2dK4EtRPY0P6pmaAEf 4khAOEP34+7RFQ4c+Q9uIj5LFidHv1a6HSvX45HOW/FlHoBh7bYGZAYWnwnDw2vlSMqi VM9B/tC0xyC9MYstlYI4pDVWppA6zKNfEfXwiOfFL0GQhxncL7dknLYvRMlNygA+o8Da mx6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ragI1+v4Vn2KDxHmu3KSKIVydBgNG/mgE2wgjdn2G64=; b=nWD2T66nGbxKJdpd+Q9t6OzAjbGiBWXVjZkMGC8Qh0zd2J0pDHIZ/X+i7EF14zBDQW XRE7nRXKA6Q2MZzGqHuV4+48MgzENlpa8oXci3kXk7MsW6th53K1P3UkgFuY8KFFbZm6 9RLBfmXoAm1f2VfTzOvu3G3Sq59QhXcIQCE5d7ywXn6Bzsf3k6W4eH2+3pR7AyNVqVg5 lGJzhFes7gs8bk4YDWyrlb+csik3syntaO2OIlUkFQIrH2gF1xe5w5z6qFak+aiX+J/i th5CmaNjTdBHc5kM+KOlyL9HH94H+l3y7/rawCmwaBZ2A1ulAyFNYpXEGAUzbJRBlk6a 1QMA== X-Gm-Message-State: APt69E0hkqmRS+8SAvUl/WLVdIMVCrWU/wLCUSCzcg7NsTmR+yTRnYMM ge5zMySM5bSSc6PO3C1ATpVU2A== X-Google-Smtp-Source: AAOMgpdzbbhtW/goItwUcUSolG2y+1U1RfHniDXQJNicFgTnAM+UwhFJJnsbKi+KDQNZVw6j42hZow== X-Received: by 2002:a17:902:6047:: with SMTP id a7-v6mr95134plt.188.1530132206702; Wed, 27 Jun 2018 13:43:26 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:25 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:35 +0800 Message-Id: <20180627204136.4177-20-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::243 Subject: [Qemu-devel] [PATCH V9 19/20] COLO: quick failover process by kick COLO thread X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: zhanghailiang COLO thread may sleep at qemu_sem_wait(&s->colo_checkpoint_sem), while failover works begin, It's better to wakeup it to quick the process. Signed-off-by: zhanghailiang Reviewed-by: Dr. David Alan Gilbert --- migration/colo.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/migration/colo.c b/migration/colo.c index 4d0ee7d8aa..553b28adbb 100644 --- a/migration/colo.c +++ b/migration/colo.c @@ -131,6 +131,11 @@ static void primary_vm_do_failover(void) migrate_set_state(&s->state, MIGRATION_STATUS_COLO, MIGRATION_STATUS_COMPLETED); + /* + * kick COLO thread which might wait at + * qemu_sem_wait(&s->colo_checkpoint_sem). + */ + colo_checkpoint_notify(migrate_get_current()); /* * Wake up COLO thread which may blocked in recv() or send(), @@ -557,6 +562,9 @@ static void colo_process_checkpoint(MigrationState *s) qemu_sem_wait(&s->colo_checkpoint_sem); + if (s->state != MIGRATION_STATUS_COLO) { + goto out; + } ret = colo_do_checkpoint_transaction(s, bioc, fb); if (ret < 0) { goto out; From patchwork Wed Jun 27 20:41:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 935723 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QiZFtgYn"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GFmS3kwMz9s19 for ; Thu, 28 Jun 2018 07:05:12 +1000 (AEST) Received: from localhost ([::1]:33225 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHck-0003Sw-7w for incoming@patchwork.ozlabs.org; Wed, 27 Jun 2018 17:05:10 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50350) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYHHu-0004tp-F6 for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYHHp-0007MJ-IP for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:38 -0400 Received: from mail-pg0-x235.google.com ([2607:f8b0:400e:c05::235]:32843) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fYHHp-0007LC-Bm for qemu-devel@nongnu.org; Wed, 27 Jun 2018 16:43:33 -0400 Received: by mail-pg0-x235.google.com with SMTP id e11-v6so1433594pgq.0 for ; Wed, 27 Jun 2018 13:43:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZfdwIt/xMuOXsRNG76y6SsgDcUVgDt+VWcXS0m2Nuog=; b=QiZFtgYnku5GOysvUMqTNiaWI60+p96n6UP+hZPqInWyumx5qRnvfRuMNwQ36g1c7R 50l8IzvY4Id2MOQjwy9qH8RIu5zY6D43XzmNvbAWD017GPFOiOXVyPjKmNZOja3ZWhI3 PWKJEZe7UL4o6R6ig/Drb+k1xQXcnocn64ILMLHc58kQBKuq6QkYzMrcsFmz/sqsqzWh sGmDzVre6C9UmiiCKgTRp+w3gNe/szzZsO9WzQo/VYxG0GQG+i1Egf0EiRP1H5asFpON VLrzWwpEzWgsc6+Jw3D1eV0jg9fTu/AarETOJegWhOOPZ5VxOUNqRF+QTmyS/2gEqgs3 F/CQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZfdwIt/xMuOXsRNG76y6SsgDcUVgDt+VWcXS0m2Nuog=; b=sXDKbQ5S/mGM3BYsdfRArKc0XW10qOgDkKuwb+EjGwIhzZoJ/qsQF64nSxU4leorXI RFVHcZvJ58e2lvXl+QXxv/WE0xWTg6XrMJoaOpdqI1zpPYcPgMo1PIre7yVkfl6by1Tw KmK5kwLslxmgxR1gLIaZSGca70A9HuA+ZsVmCF1lPfc2cWo28dthX0+TnaEHlnf0mnZ2 C5LfIBz553b1clIEdBMYrXGpWIOdXAJ933D+f0HIZ8xn1YetZF4AgJxvK3tE/OMqmUxO fSuH6F5mlEnOW8XGpcppohvp6Tj8I5YG1hCFr90/569ttjV58/2h5SfUsgbesaMs32bv 4CoA== X-Gm-Message-State: APt69E05JUdpncC1LKmK/qLcY4m91/oVnxDjXfFznoOnTZB8jMkpoUEl /kUx3ZBPqNMuvHo0YFUm+svYtA== X-Google-Smtp-Source: AAOMgpcBEkWmFJWOG2xLRr7H+FEkaWGY1kF5UT2aFbb4dCiQimuyfJEwBgfIHjlPuTSjiHwA6gpZAg== X-Received: by 2002:a62:104e:: with SMTP id y75-v6mr7335583pfi.109.1530132212093; Wed, 27 Jun 2018 13:43:32 -0700 (PDT) Received: from localhost.localdomain (155.150.229.35.bc.googleusercontent.com. [35.229.150.155]) by smtp.gmail.com with ESMTPSA id c67-v6sm9690428pfj.173.2018.06.27.13.43.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 13:43:31 -0700 (PDT) From: Zhang Chen To: qemu-devel@nongnu.org, Paolo Bonzini , Juan Quintela , "Dr . David Alan Gilbert" , Jason Wang , Eric Blake , Markus Armbruster Date: Thu, 28 Jun 2018 04:41:36 +0800 Message-Id: <20180627204136.4177-21-zhangckid@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180627204136.4177-1-zhangckid@gmail.com> References: <20180627204136.4177-1-zhangckid@gmail.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::235 Subject: [Qemu-devel] [PATCH V9 20/20] docs: Add COLO status diagram to COLO-FT.txt X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zhang Chen , zhanghailiang , Li Zhijian , Zhang Chen Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Zhang Chen This diagram make user better understand COLO. Suggested by Markus Armbruster. Signed-off-by: Zhang Chen --- docs/COLO-FT.txt | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt index e289be2f41..caa68f0a9a 100644 --- a/docs/COLO-FT.txt +++ b/docs/COLO-FT.txt @@ -110,6 +110,40 @@ Note: HeartBeat has not been implemented yet, so you need to trigger failover process by using 'x-colo-lost-heartbeat' command. +== COLO operation status == + ++-----------------+ +| | +| Start COLO | +| | ++--------+--------+ + | + | Main qmp command: + | migrate-set-capabilities with x-colo + | migrate + | + v ++--------+--------+ +| | +| COLO running | +| | ++--------+--------+ + | + | Main qmp command: + | x-colo-lost-heartbeat + | or + | some error happened + v ++--------+--------+ +| | send qmp event: +| COLO failover | COLO_EXIT +| | ++-----------------+ + +COLO use the qmp command switching and report operation status. +The diagram just write the main qmp command, you can get the detail +in test procedure. + == Test procedure == 1. Startup qemu Primary: