From patchwork Mon Feb 11 22:49:56 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 219692 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id C77E82C02C7 for ; Tue, 12 Feb 2013 09:51:37 +1100 (EST) Received: from localhost ([::1]:39836 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U52Do-00080T-1z for incoming@patchwork.ozlabs.org; Mon, 11 Feb 2013 17:51:36 -0500 Received: from eggs.gnu.org ([208.118.235.92]:40364) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U52DA-0006a6-TM for qemu-devel@nongnu.org; Mon, 11 Feb 2013 17:51:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1U52D4-0007sZ-0T for qemu-devel@nongnu.org; Mon, 11 Feb 2013 17:50:56 -0500 Received: from e34.co.us.ibm.com ([32.97.110.152]:41515) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1U52D3-0007s3-Oa for qemu-devel@nongnu.org; Mon, 11 Feb 2013 17:50:49 -0500 Received: from /spool/local by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 11 Feb 2013 15:50:48 -0700 Received: from d03dlp02.boulder.ibm.com (9.17.202.178) by e34.co.us.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Mon, 11 Feb 2013 15:50:47 -0700 Received: from d03relay04.boulder.ibm.com (d03relay04.boulder.ibm.com [9.17.195.106]) by d03dlp02.boulder.ibm.com (Postfix) with ESMTP id 2415E3E40045 for ; Mon, 11 Feb 2013 15:50:38 -0700 (MST) Received: from d03av04.boulder.ibm.com (d03av04.boulder.ibm.com [9.17.195.170]) by d03relay04.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r1BMoXMi081334 for ; Mon, 11 Feb 2013 15:50:33 -0700 Received: from d03av04.boulder.ibm.com (loopback [127.0.0.1]) by d03av04.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r1BMoXxw010233 for ; Mon, 11 Feb 2013 15:50:33 -0700 Received: from mrhinesdev.klabtestbed.com (klinux.watson.ibm.com [9.2.208.21]) by d03av04.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r1BMoUWL010052; Mon, 11 Feb 2013 15:50:32 -0700 From: "Michael R. Hines" To: qemu-devel@nongnu.org Date: Mon, 11 Feb 2013 17:49:56 -0500 Message-Id: <1360622997-26904-5-git-send-email-mrhines@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1360622997-26904-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1360622997-26904-1-git-send-email-mrhines@linux.vnet.ibm.com> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13021122-2876-0000-0000-0000052CEB4A X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.110.152 Cc: aliguori@us.ibm.com, abali@us.ibm.com, "Michael R. Hines" , gokul@us.ibm.com Subject: [Qemu-devel] [RFC PATCH RDMA support v2: 5/6] connection-setup code between client/server X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Michael R. Hines" Signed-off-by: Michael R. Hines --- migration-tcp.c | 19 +++++++++++++++++++ migration.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 71 insertions(+), 1 deletion(-) diff --git a/migration-tcp.c b/migration-tcp.c index e78a296..113576e 100644 --- a/migration-tcp.c +++ b/migration-tcp.c @@ -14,6 +14,7 @@ */ #include "qemu-common.h" +#include "qemu/rdma.h" #include "qemu/sockets.h" #include "migration/migration.h" #include "migration/qemu-file.h" @@ -55,6 +56,9 @@ static void tcp_wait_for_connect(int fd, void *opaque) if (fd < 0) { DPRINTF("migrate connect error\n"); + if (migration_use_rdma()) { + rdma_cleanup(&rdma_mdata); + } s->fd = -1; migrate_fd_error(s); } else { @@ -62,6 +66,10 @@ static void tcp_wait_for_connect(int fd, void *opaque) s->fd = fd; socket_set_block(s->fd); migrate_fd_connect(s); + + if (migration_use_rdma() && rdma_wait_for_connect(fd, opaque)) { + migrate_fd_error(s); + } } } @@ -101,6 +109,12 @@ static void tcp_accept_incoming_migration(void *opaque) goto out; } + if (migration_use_rdma() && + rdma_accept_incoming_migration(&rdma_mdata)) { + close(s); + return; + } + process_incoming_migration(f); return; @@ -117,6 +131,11 @@ void tcp_start_incoming_migration(const char *host_port, Error **errp) return; } + if (migration_use_rdma() && rdma_start_incoming_migration(s)) { + close(s); + return; + } + qemu_set_fd_handler2(s, NULL, tcp_accept_incoming_migration, NULL, (void *)(intptr_t)s); } diff --git a/migration.c b/migration.c index 77c1971..5ef923d 100644 --- a/migration.c +++ b/migration.c @@ -22,6 +22,7 @@ #include "qemu/sockets.h" #include "migration/block.h" #include "qemu/thread.h" +#include "qemu/rdma.h" #include "qmp-commands.h" //#define DEBUG_MIGRATION @@ -262,6 +263,16 @@ void qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params, } for (cap = params; cap; cap = cap->next) { + if (cap->value->capability == MIGRATION_CAPABILITY_RDMA) { +#ifndef CONFIG_RDMA + error_set(errp, QERR_MIGRATION_RDMA_NOT_ENABLED); + continue; +#endif + if (!migration_use_rdma()) { + error_set(errp, QERR_MIGRATION_RDMA_NOT_CONFIGURED, rdma_mdata.host, rdma_mdata.port); + continue; + } + } s->enabled_capabilities[cap->value->capability] = cap->value->state; } } @@ -279,6 +290,11 @@ static int migrate_fd_cleanup(MigrationState *s) } assert(s->fd == -1); + + if (migrate_rdma_enabled()) { + rdma_cleanup(&rdma_mdata); + } + return ret; } @@ -386,6 +402,9 @@ static MigrationState *migrate_init(const MigrationParams *params) s->params = *params; memcpy(s->enabled_capabilities, enabled_capabilities, sizeof(enabled_capabilities)); + + rdma_update_capability(s); + s->xbzrle_cache_size = xbzrle_cache_size; s->bandwidth_limit = bandwidth_limit; @@ -481,6 +500,29 @@ int64_t qmp_query_migrate_cache_size(Error **errp) return migrate_xbzrle_cache_size(); } +void qmp_migrate_set_rdma_port(int64_t port, Error **errp) +{ + MigrationState *s = migrate_get_current(); + if (s && (s->state == MIG_STATE_ACTIVE)) { + return; + } + DPRINTF("rdma migration port: %" PRId64 "\n", port); + rdma_mdata.port = port; + rdma_update_capability(s); +} + +void qmp_migrate_set_rdma_host(const char *host, Error **errp) +{ + MigrationState *s = migrate_get_current(); + if (s && (s->state == MIG_STATE_ACTIVE)) { + return; + } + DPRINTF("rdma migration host name: %s\n", host); + strncpy(rdma_mdata.host, host, 64); + rdma_mdata.host[63] = '\0'; + rdma_update_capability(s); +} + void qmp_migrate_set_speed(int64_t value, Error **errp) { MigrationState *s; @@ -505,6 +547,15 @@ int migrate_use_xbzrle(void) { MigrationState *s; + /* + * RFC RDMA: time(run-length encoding) + + * time(communication) is too big. RDMA throughput tanks + * when this feature is enabled. But there's no need + * to change the code since the feature is optional. + */ + if (migrate_rdma_enabled()) + return 0; + s = migrate_get_current(); return s->enabled_capabilities[MIGRATION_CAPABILITY_XBZRLE]; @@ -571,7 +622,7 @@ static int buffered_put_buffer(void *opaque, const uint8_t *buf, } if (size > (s->buffer_capacity - s->buffer_size)) { - DPRINTF("increasing buffer capacity from %zu by %zu\n", + DPRINTF("increasing buffer capacity from %zu by %d\n", s->buffer_capacity, size + 1024); s->buffer_capacity += size + 1024;