From patchwork Tue Dec 22 10:42:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhang Chen X-Patchwork-Id: 560013 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id D5587140BC5 for ; Tue, 22 Dec 2015 23:43:41 +1100 (AEDT) Received: from localhost ([::1]:50059 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMHz-00051k-P7 for incoming@patchwork.ozlabs.org; Tue, 22 Dec 2015 07:43:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41547) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMDl-0005Hx-KX for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:18 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aBMDk-00008f-4u for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:17 -0500 Received: from [59.151.112.132] (port=13328 helo=heian.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aBMDj-00005d-PD for qemu-devel@nongnu.org; Tue, 22 Dec 2015 07:39:16 -0500 X-IronPort-AV: E=Sophos;i="5.20,346,1444665600"; d="scan'208";a="1856287" Received: from bogon (HELO cn.fujitsu.com) ([10.167.33.5]) by heian.cn.fujitsu.com with ESMTP; 22 Dec 2015 18:43:11 +0800 Received: from G08CNEXCHPEKD02.g08.fujitsu.local (unknown [10.167.33.83]) by cn.fujitsu.com (Postfix) with ESMTP id 864C741887DB; Tue, 22 Dec 2015 18:42:54 +0800 (CST) Received: from G08FNSTD140215.g08.fujitsu.local (10.167.226.56) by G08CNEXCHPEKD02.g08.fujitsu.local (10.167.33.89) with Microsoft SMTP Server (TLS) id 14.3.181.6; Tue, 22 Dec 2015 18:42:54 +0800 From: Zhang Chen To: qemu devel , Jason Wang , Stefan Hajnoczi Date: Tue, 22 Dec 2015 18:42:58 +0800 Message-ID: <1450780978-19123-11-git-send-email-zhangchen.fnst@cn.fujitsu.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1450780978-19123-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> References: <1450780978-19123-1-git-send-email-zhangchen.fnst@cn.fujitsu.com> MIME-Version: 1.0 X-Originating-IP: [10.167.226.56] X-yoursite-MailScanner-ID: 864C741887DB.ADA56 X-yoursite-MailScanner: Found to be clean X-yoursite-MailScanner-From: zhangchen.fnst@cn.fujitsu.com X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 59.151.112.132 Cc: Li Zhijian , Gui jianfeng , "eddie.dong" , "Dr. David Alan Gilbert" , Huang peng , Gong lei , jan.kiszka@siemens.com, Zhang Chen , Yang Hongyang , zhanghailiang Subject: [Qemu-devel] [RFC PATCH v2 10/10] net/colo-proxy: Colo-proxy do checkpoint and clear X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: zhangchen Do checkpoint and flush Signed-off-by: zhangchen Signed-off-by: zhanghailiang --- net/colo-proxy.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/net/colo-proxy.c b/net/colo-proxy.c index abb289f..79b1b1b 100644 --- a/net/colo-proxy.c +++ b/net/colo-proxy.c @@ -144,6 +144,8 @@ static inline void colo_proxy_dump_packet(Packet *pkt) printf("\n"); } +static void packet_destroy(void *opaque, void *user_data); + static uint32_t connection_key_hash(const void *opaque) { const ConnectionKey *key = opaque; @@ -190,6 +192,28 @@ static Connection *connection_new(ConnectionKey *key) return conn; } +static void colo_send_primary_packet(void *opaque, void *user_data) +{ + Packet *pkt = opaque; + qemu_net_queue_send(pkt->s->incoming_queue, pkt->sender, 0, + (const uint8_t *)pkt->data, pkt->size, NULL); +} + +static void colo_flush_connection(void *opaque, void *user_data) +{ + Connection *conn = opaque; + Packet *pkt = NULL; + + while (!g_queue_is_empty(&conn->primary_list)) { + pkt = g_queue_pop_head(&conn->primary_list); + colo_send_primary_packet(pkt, NULL); + } + while (!g_queue_is_empty(&conn->secondary_list)) { + pkt = g_queue_pop_head(&conn->secondary_list); + packet_destroy(pkt, NULL); + } +} + /* * Clear hashtable, stop this hash growing really huge */ @@ -205,6 +229,52 @@ bool colo_proxy_query_checkpoint(void) return colo_do_checkpoint; } +static int colo_proxy_primary_checkpoint(COLOProxyState *s) +{ + g_queue_foreach(&s->conn_list, colo_flush_connection, NULL); + return 0; +} + +static int colo_proxy_secondary_checkpoint(COLOProxyState *s) +{ + return 0; +} + +static void colo_proxy_checkpoint_one(NetFilterState *nf, + void *opaque, Error **errp) +{ + COLOProxyState *s; + int mode; + + if (strcmp(object_get_typename(OBJECT(nf)), TYPE_FILTER_COLO_PROXY)) { + return; + } + + s = FILTER_COLO_PROXY(nf); + mode = *(int *)opaque; + assert(s->colo_mode == mode); + + if (s->colo_mode == COLO_MODE_PRIMARY) { + colo_proxy_primary_checkpoint(s); + } else { + /* secondary do checkpoint */ + colo_proxy_secondary_checkpoint(s); + } +} + +int colo_proxy_do_checkpoint(int mode) +{ + Error *err = NULL; + qemu_foreach_netfilter(colo_proxy_checkpoint_one, &mode, &err); + if (err) { + error_report("colo proxy do checkpoint failed"); + return -1; + } + + colo_do_checkpoint = false; + return 0; +} + /* Return 0 on success, or return -1 if the pkt is corrupted */ static int parse_packet_early(Packet *pkt, ConnectionKey *key) { @@ -294,6 +364,7 @@ static Connection *colo_proxy_get_conn(COLOProxyState *s, if (s->hashtable_size > hashtable_max_size) { trace_colo_proxy("colo proxy connection hashtable full, clear it"); clear_connection_hashtable(s); + /* TODO:clear conn_list */ } else { g_hash_table_insert(colo_conn_hash, new_key, conn); } @@ -751,6 +822,8 @@ void colo_proxy_stop(int mode) static void colo_proxy_setup(NetFilterState *nf, Error **errp) { COLOProxyState *s = FILTER_COLO_PROXY(nf); + ssize_t factor = 8; + struct sysinfo si; if (!s->addr) { error_setg(errp, "filter colo_proxy needs 'addr' property set!"); @@ -768,6 +841,21 @@ static void colo_proxy_setup(NetFilterState *nf, Error **errp) colo_do_checkpoint = false; qemu_event_init(&s->need_compare_ev, false); + /* + * Idea from kernel tcp.c: use 1/16384 of memory. On i386: 32MB + * machine has 512 buckets. >= 1GB machines have 16384 buckets. + * default factor = 8 + */ + sysinfo(&si); + hashtable_max_size = 16384; + if (si.totalram > (1024 * 1024 * 1024)) { + hashtable_max_size = 16384; + } + if (hashtable_max_size < 32) { + hashtable_max_size = 32; + } + + hashtable_max_size = hashtable_max_size * factor; s->incoming_queue = qemu_new_net_queue(qemu_netfilter_pass_to_next, nf); colo_conn_hash = g_hash_table_new_full(connection_key_hash, connection_key_equal,