From patchwork Tue Jan 28 09:05:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230227 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h5T2Y5zK; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LPC0KNhz9sP6 for ; Tue, 28 Jan 2020 20:07:59 +1100 (AEDT) Received: from localhost ([::1]:55680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMqj-0008Uk-2Y for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:07:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56635) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMos-0005lv-U0 for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMor-0002qG-Ts for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:02 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:58151 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMor-0002pX-QO for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PbBVwTwxgB7nPVUXFlKEK3auQkBLFXi2IJU563asPOQ=; b=h5T2Y5zKZadDtaT2JRGoIEViwkWR/rmucSi18H/0A+XiHt8NwElMsURjIs10zt8T0JIAXA MwJh/zwFDkKVgdG7amAgJ3jl4L+ci/y4dooHAAB/poAPFlb5iWvdN6j/BwaYqUjQ06liF/ Cn9+FbHg1ymvEJDQHRnUAEzxTPf1vo0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-417-MbwWItdsNcqCebVC92WGiQ-1; Tue, 28 Jan 2020 04:05:57 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A50C9800D5E for ; Tue, 28 Jan 2020 09:05:56 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E89210027A1; Tue, 28 Jan 2020 09:05:54 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 1/7] migration: Add support for modules Date: Tue, 28 Jan 2020 10:05:44 +0100 Message-Id: <20200128090550.5112-2-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: MbwWItdsNcqCebVC92WGiQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" So we don't have to compile everything in, or have ifdefs Signed-off-by: Juan Quintela --- include/qemu/module.h | 2 ++ vl.c | 1 + 2 files changed, 3 insertions(+) diff --git a/include/qemu/module.h b/include/qemu/module.h index 65ba596e46..907cb5c0a5 100644 --- a/include/qemu/module.h +++ b/include/qemu/module.h @@ -40,6 +40,7 @@ static void __attribute__((constructor)) do_qemu_init_ ## function(void) \ #endif typedef enum { + MODULE_INIT_MIGRATION, MODULE_INIT_BLOCK, MODULE_INIT_OPTS, MODULE_INIT_QOM, @@ -56,6 +57,7 @@ typedef enum { #define xen_backend_init(function) module_init(function, \ MODULE_INIT_XEN_BACKEND) #define libqos_init(function) module_init(function, MODULE_INIT_LIBQOS) +#define migration_init(function) module_init(function, MODULE_INIT_MIGRATION) #define block_module_load_one(lib) module_load_one("block-", lib) #define ui_module_load_one(lib) module_load_one("ui-", lib) diff --git a/vl.c b/vl.c index b0f52c4d6e..9f8577955a 100644 --- a/vl.c +++ b/vl.c @@ -2890,6 +2890,7 @@ int main(int argc, char **argv, char **envp) qemu_init_exec_dir(argv[0]); module_call_init(MODULE_INIT_QOM); + module_call_init(MODULE_INIT_MIGRATION); qemu_add_opts(&qemu_drive_opts); qemu_add_drive_opts(&qemu_legacy_drive_opts); From patchwork Tue Jan 28 09:05:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230222 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=f0TJ00RZ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LMj0vBBz9sP6 for ; Tue, 28 Jan 2020 20:06:41 +1100 (AEDT) Received: from localhost ([::1]:55656 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMpS-0005ra-Vl for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:06:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56645) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMou-0005me-B5 for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMos-0002r4-D9 for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:04 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:45065 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMos-0002qI-8L for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:02 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202361; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vLBg1TwogXqTj4/1felBiP4HmJK2MgVOjC1jH/jJUXA=; b=f0TJ00RZPd7/A+XV/p+iT4SIqMgYw6oGKHMZ9DB4uvr4R4wvP+jEmUCyetsNYeDwkYLAtl rKCko0AOXPHvTc3O5QqUKqvmekyDR9q8AHl3JSml4X+cUmW5m64r5H8Ti9ZXqL/UOMWPXl DhbsLLPUbCOWPJ2hp/Bxi+ovL2c/zIo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-208-sJc5KdTKMsu9cecjgvOCgA-1; Tue, 28 Jan 2020 04:06:00 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 49C9E801E6D for ; Tue, 28 Jan 2020 09:05:59 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 06ADF10027A1; Tue, 28 Jan 2020 09:05:56 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 2/7] multifd: Make no compression operations into its own structure Date: Tue, 28 Jan 2020 10:05:45 +0100 Message-Id: <20200128090550.5112-3-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: sJc5KdTKMsu9cecjgvOCgA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" It will be used later. Signed-off-by: Juan Quintela --- Move setup of ->ops helper to proper place (wei) Rename s/none/nocomp/ (dave) Introduce MULTIFD_FLAG_NOCOMP right order of arguments for print Introduce MULTIFD_FLAG_METHOD_MASK (now it is a 4 bit value) --- migration/migration.c | 9 ++ migration/migration.h | 1 + migration/multifd.c | 185 ++++++++++++++++++++++++++++++++++++++++-- migration/multifd.h | 25 ++++++ migration/ram.c | 1 + 5 files changed, 213 insertions(+), 8 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index cd72bb6e9a..06f6c2d529 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2245,6 +2245,15 @@ int migrate_multifd_channels(void) return s->parameters.multifd_channels; } +MultiFDMethod migrate_multifd_method(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.multifd_method; +} + int migrate_use_xbzrle(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index 8473ddfc88..3d23a0852e 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -300,6 +300,7 @@ bool migrate_auto_converge(void); bool migrate_use_multifd(void); bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); +MultiFDMethod migrate_multifd_method(void); int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/migration/multifd.c b/migration/multifd.c index b3e8ae9bcc..1c49c2a665 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -38,6 +38,134 @@ typedef struct { uint64_t unused2[4]; /* Reserved for future use */ } __attribute__((packed)) MultiFDInit_t; +/* Multifd without compression */ + +/** + * nocomp_send_setup: setup send side + * + * For no compression this function does nothing. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_send_setup(MultiFDSendParams *p, Error **errp) +{ + return 0; +} + +/** + * nocomp_send_cleanup: cleanup send side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + */ +static void nocomp_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + return; +} + +/** + * nocomp_send_prepare: prepare date to be able to send + * + * For no compression we just have to calculate the size of the + * packet. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int nocomp_send_prepare(MultiFDSendParams *p, uint32_t used, + Error **errp) +{ + p->next_packet_size = used * qemu_target_page_size(); + p->flags |= MULTIFD_FLAG_NOCOMP; + return 0; +} + +/** + * nocomp_send_write: do the actual write of the data + * + * For no compression we just have to write the data. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int nocomp_send_write(MultiFDSendParams *p, uint32_t used, Error **errp) +{ + return qio_channel_writev_all(p->c, p->pages->iov, used, errp); +} + +/** + * nocomp_recv_setup: setup receive side + * + * For no compression this function does nothing. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int nocomp_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + return 0; +} + +/** + * nocomp_recv_cleanup: setup receive side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + */ +static void nocomp_recv_cleanup(MultiFDRecvParams *p) +{ +} + +/** + * nocomp_recv_pages: read the data from the channel into actual pages + * + * For no compression we just need to read things into the correct place. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int nocomp_recv_pages(MultiFDRecvParams *p, uint32_t used, Error **errp) +{ + uint32_t flags = p->flags & MULTIFD_FLAG_METHOD_MASK; + + if (flags != MULTIFD_FLAG_NOCOMP) { + error_setg(errp, "multifd %d: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_NOCOMP); + return -1; + } + return qio_channel_readv_all(p->c, p->pages->iov, used, errp); +} + +static MultiFDMethods multifd_nocomp_ops = { + .send_setup = nocomp_send_setup, + .send_cleanup = nocomp_send_cleanup, + .send_prepare = nocomp_send_prepare, + .send_write = nocomp_send_write, + .recv_setup = nocomp_recv_setup, + .recv_cleanup = nocomp_recv_cleanup, + .recv_pages = nocomp_recv_pages +}; + +static MultiFDMethods *multifd_ops[MULTIFD_METHOD__MAX] = { + [MULTIFD_METHOD_NONE] = &multifd_nocomp_ops, +}; + static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) { MultiFDInit_t msg = {}; @@ -246,6 +374,8 @@ struct { * We will use atomic operations. Only valid values are 0 and 1. */ int exiting; + /* multifd ops */ + MultiFDMethods *ops; } *multifd_send_state; /* @@ -397,6 +527,7 @@ void multifd_save_cleanup(void) } for (i = 0; i < migrate_multifd_channels(); i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; + Error *local_err = NULL; socket_send_channel_destroy(p->c); p->c = NULL; @@ -410,6 +541,10 @@ void multifd_save_cleanup(void) p->packet_len = 0; g_free(p->packet); p->packet = NULL; + multifd_send_state->ops->send_cleanup(p, &local_err); + if (local_err) { + migrate_set_error(migrate_get_current(), local_err); + } } qemu_sem_destroy(&multifd_send_state->channels_ready); g_free(multifd_send_state->params); @@ -494,7 +629,14 @@ static void *multifd_send_thread(void *opaque) uint64_t packet_num = p->packet_num; flags = p->flags; - p->next_packet_size = used * qemu_target_page_size(); + if (used) { + ret = multifd_send_state->ops->send_prepare(p, used, + &local_err); + if (ret != 0) { + qemu_mutex_unlock(&p->mutex); + break; + } + } multifd_send_fill_packet(p); p->flags = 0; p->num_packets++; @@ -513,8 +655,7 @@ static void *multifd_send_thread(void *opaque) } if (used) { - ret = qio_channel_writev_all(p->c, p->pages->iov, - used, &local_err); + ret = multifd_send_state->ops->send_write(p, used, &local_err); if (ret != 0) { break; } @@ -604,6 +745,7 @@ int multifd_save_setup(Error **errp) multifd_send_state->pages = multifd_pages_init(page_count); qemu_sem_init(&multifd_send_state->channels_ready, 0); atomic_set(&multifd_send_state->exiting, 0); + multifd_send_state->ops = multifd_ops[migrate_multifd_method()]; for (i = 0; i < thread_count; i++) { MultiFDSendParams *p = &multifd_send_state->params[i]; @@ -623,6 +765,18 @@ int multifd_save_setup(Error **errp) p->name = g_strdup_printf("multifdsend_%d", i); socket_send_channel_create(multifd_new_send_channel_async, p); } + + for (i = 0; i < thread_count; i++) { + MultiFDSendParams *p = &multifd_send_state->params[i]; + Error *local_err = NULL; + int ret; + + ret = multifd_send_state->ops->send_setup(p, &local_err); + if (ret) { + error_propagate(errp, local_err); + return ret; + } + } return 0; } @@ -634,6 +788,8 @@ struct { QemuSemaphore sem_sync; /* global number of generated multifd packets */ uint64_t packet_num; + /* multifd ops */ + MultiFDMethods *ops; } *multifd_recv_state; static void multifd_recv_terminate_threads(Error *err) @@ -673,7 +829,6 @@ static void multifd_recv_terminate_threads(Error *err) int multifd_load_cleanup(Error **errp) { int i; - int ret = 0; if (!migrate_use_multifd()) { return 0; @@ -706,6 +861,7 @@ int multifd_load_cleanup(Error **errp) p->packet_len = 0; g_free(p->packet); p->packet = NULL; + multifd_recv_state->ops->recv_cleanup(p); } qemu_sem_destroy(&multifd_recv_state->sem_sync); g_free(multifd_recv_state->params); @@ -713,7 +869,7 @@ int multifd_load_cleanup(Error **errp) g_free(multifd_recv_state); multifd_recv_state = NULL; - return ret; + return 0; } void multifd_recv_sync_main(void) @@ -778,6 +934,8 @@ static void *multifd_recv_thread(void *opaque) used = p->pages->used; flags = p->flags; + /* recv methods don't know how to handle the SYNC flag */ + p->flags &= ~MULTIFD_FLAG_SYNC; trace_multifd_recv(p->id, p->packet_num, used, flags, p->next_packet_size); p->num_packets++; @@ -785,8 +943,7 @@ static void *multifd_recv_thread(void *opaque) qemu_mutex_unlock(&p->mutex); if (used) { - ret = qio_channel_readv_all(p->c, p->pages->iov, - used, &local_err); + ret = multifd_recv_state->ops->recv_pages(p, used, &local_err); if (ret != 0) { break; } @@ -825,6 +982,7 @@ int multifd_load_setup(Error **errp) multifd_recv_state->params = g_new0(MultiFDRecvParams, thread_count); atomic_set(&multifd_recv_state->count, 0); qemu_sem_init(&multifd_recv_state->sem_sync, 0); + multifd_recv_state->ops = multifd_ops[migrate_multifd_method()]; for (i = 0; i < thread_count; i++) { MultiFDRecvParams *p = &multifd_recv_state->params[i]; @@ -839,6 +997,18 @@ int multifd_load_setup(Error **errp) p->packet = g_malloc0(p->packet_len); p->name = g_strdup_printf("multifdrecv_%d", i); } + + for (i = 0; i < thread_count; i++) { + MultiFDRecvParams *p = &multifd_recv_state->params[i]; + Error *local_err = NULL; + int ret; + + ret = multifd_recv_state->ops->recv_setup(p, &local_err); + if (ret) { + error_propagate(errp, local_err); + return ret; + } + } return 0; } @@ -896,4 +1066,3 @@ bool multifd_recv_new_channel(QIOChannel *ioc, Error **errp) return atomic_read(&multifd_recv_state->count) == migrate_multifd_channels(); } - diff --git a/migration/multifd.h b/migration/multifd.h index d8b0205977..c7fea4914c 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -25,6 +25,10 @@ int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); #define MULTIFD_FLAG_SYNC (1 << 0) +/* We reserve 3 bits for METHODS */ +#define MULTIFD_FLAG_METHOD_MASK (7 << 1) +#define MULTIFD_FLAG_NOCOMP (1 << 1) + /* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024) @@ -96,6 +100,8 @@ typedef struct { uint64_t num_pages; /* syncs main thread and channels */ QemuSemaphore sem_sync; + /* used for compression methods */ + void *data; } MultiFDSendParams; typedef struct { @@ -133,7 +139,26 @@ typedef struct { uint64_t num_pages; /* syncs main thread and channels */ QemuSemaphore sem_sync; + /* used for de-compression methods */ + void *data; } MultiFDRecvParams; +typedef struct { + /* Setup for sending side */ + int (*send_setup)(MultiFDSendParams *p, Error **errp); + /* Cleanup for sending side */ + void (*send_cleanup)(MultiFDSendParams *p, Error **errp); + /* Prepare the send packet */ + int (*send_prepare)(MultiFDSendParams *p, uint32_t used, Error **errp); + /* Write the send packet */ + int (*send_write)(MultiFDSendParams *p, uint32_t used, Error **errp); + /* Setup for receiving side */ + int (*recv_setup)(MultiFDRecvParams *p, Error **errp); + /* Cleanup for receiving side */ + void (*recv_cleanup)(MultiFDRecvParams *p); + /* Read all pages */ + int (*recv_pages)(MultiFDRecvParams *p, uint32_t used, Error **errp); +} MultiFDMethods; + #endif diff --git a/migration/ram.c b/migration/ram.c index ed23ed1c7c..73a141bb60 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -43,6 +43,7 @@ #include "page_cache.h" #include "qemu/error-report.h" #include "qapi/error.h" +#include "qapi/qapi-types-migration.h" #include "qapi/qapi-events-migration.h" #include "qapi/qmp/qerror.h" #include "trace.h" From patchwork Tue Jan 28 09:05:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230229 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CaTTdeQy; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LQN5ZwHz9sR1 for ; Tue, 28 Jan 2020 20:09:00 +1100 (AEDT) Received: from localhost ([::1]:55694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMri-00022V-RS for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:08:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56664) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMow-0005pC-Bn for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMou-0002tR-Sf for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:06 -0500 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:24911 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMou-0002tL-P9 for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202364; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bh/FIRwBnOtySDetKn7Tt0sK/mF/BVwJ/xp7qo9LFwE=; b=CaTTdeQyAZyC/zl3tkpaZoKpI5kZjkVQ/t5fv7Vf6CKkrjWEApqE3fPEH7DVgzhav7GPPN AHwLvg1OLs+dySftWMdDrctjzLQqT5YyPKONy+Dm1od10xgHl005u2rab+1i5C4B3IkOpl cafRKF5ozb1AjP5yt4j7uNTLAk3JwPU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-119-kuSJhIyrM4K-sVjNvEZBHA-1; Tue, 28 Jan 2020 04:06:02 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E71CC189F762 for ; Tue, 28 Jan 2020 09:06:01 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9E72F1001B39; Tue, 28 Jan 2020 09:05:59 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 3/7] multifd: Add multifd-zlib-level parameter Date: Tue, 28 Jan 2020 10:05:46 +0100 Message-Id: <20200128090550.5112-4-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: kuSJhIyrM4K-sVjNvEZBHA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" It will indicate which level use for compression. Signed-off-by: Juan Quintela --- migration/migration.c | 15 +++++++++++++++ monitor/hmp-cmds.c | 4 ++++ qapi/migration.json | 30 +++++++++++++++++++++++++++--- 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 06f6c2d529..4f88f8e958 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -89,6 +89,8 @@ #define DEFAULT_MIGRATE_X_CHECKPOINT_DELAY (200 * 100) #define DEFAULT_MIGRATE_MULTIFD_CHANNELS 2 #define DEFAULT_MIGRATE_MULTIFD_METHOD MULTIFD_METHOD_NONE +/*0: means nocompress, 1: best speed, ... 9: best compress ratio */ +#define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1 /* Background transfer rate for postcopy, 0 means unlimited, note * that page requests can still exceed this limit. @@ -801,6 +803,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->multifd_channels = s->parameters.multifd_channels; params->has_multifd_method = true; params->multifd_method = s->parameters.multifd_method; + params->has_multifd_zlib_level = true; + params->multifd_zlib_level = s->parameters.multifd_zlib_level; params->has_xbzrle_cache_size = true; params->xbzrle_cache_size = s->parameters.xbzrle_cache_size; params->has_max_postcopy_bandwidth = true; @@ -1208,6 +1212,13 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp) return false; } + if (params->has_multifd_zlib_level && + (params->multifd_zlib_level > 9)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zlib_level", + "is invalid, it should be in the range of 0 to 9"); + return false; + } + if (params->has_xbzrle_cache_size && (params->xbzrle_cache_size < qemu_target_page_size() || !is_power_of_2(params->xbzrle_cache_size))) { @@ -3536,6 +3547,9 @@ static Property migration_properties[] = { DEFINE_PROP_MULTIFD_METHOD("multifd-method", MigrationState, parameters.multifd_method, DEFAULT_MIGRATE_MULTIFD_METHOD), + DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState, + parameters.multifd_zlib_level, + DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL), DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState, parameters.xbzrle_cache_size, DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE), @@ -3627,6 +3641,7 @@ static void migration_instance_init(Object *obj) params->has_block_incremental = true; params->has_multifd_channels = true; params->has_multifd_method = true; + params->has_multifd_zlib_level = true; params->has_xbzrle_cache_size = true; params->has_max_postcopy_bandwidth = true; params->has_max_cpu_throttle = true; diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 16f01d4244..7f11866446 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -1836,6 +1836,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) } p->multifd_method = compress_type; break; + case MIGRATION_PARAMETER_MULTIFD_ZLIB_LEVEL: + p->has_multifd_zlib_level = true; + visit_type_int(v, param, &p->multifd_zlib_level, &err); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size = true; visit_type_size(v, param, &cache_size, &err); diff --git a/qapi/migration.json b/qapi/migration.json index 96a126751c..289dce0da7 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -602,6 +602,13 @@ # @multifd-method: Which compression method to use. # Defaults to none. (Since 5.0) # +# @multifd-zlib-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 9, where 0 means no compression, 1 means the best +# compression speed, and 9 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## { 'enum': 'MigrationParameter', @@ -614,7 +621,8 @@ 'downtime-limit', 'x-checkpoint-delay', 'block-incremental', 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', - 'max-cpu-throttle', 'multifd-method' ] } + 'max-cpu-throttle', 'multifd-method', + 'multifd-zlib-level' ] } ## # @MigrateSetParameters: @@ -707,6 +715,13 @@ # @multifd-method: Which compression method to use. # Defaults to none. (Since 5.0) # +# @multifd-zlib-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 9, where 0 means no compression, 1 means the best +# compression speed, and 9 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## # TODO either fuse back into MigrationParameters, or make @@ -733,7 +748,8 @@ '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', '*max-cpu-throttle': 'int', - '*multifd-method': 'MultiFDMethod' } } + '*multifd-method': 'MultiFDMethod', + '*multifd-zlib-level': 'int' } } ## # @migrate-set-parameters: @@ -846,6 +862,13 @@ # @multifd-method: Which compression method to use. # Defaults to none. (Since 5.0) # +# @multifd-zlib-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 9, where 0 means no compression, 1 means the best +# compression speed, and 9 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## { 'struct': 'MigrationParameters', @@ -870,7 +893,8 @@ '*xbzrle-cache-size': 'size', '*max-postcopy-bandwidth': 'size', '*max-cpu-throttle': 'uint8', - '*multifd-method': 'MultiFDMethod' } } + '*multifd-method': 'MultiFDMethod', + '*multifd-zlib-level': 'uint8' } } ## # @query-migrate-parameters: From patchwork Tue Jan 28 09:05:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230228 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=B1XsABDg; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LPK08Pbz9sPn for ; Tue, 28 Jan 2020 20:08:05 +1100 (AEDT) Received: from localhost ([::1]:55684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMqp-0000J9-2M for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:08:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56680) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMp0-0005sN-DO for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMox-0002zw-VR for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:10 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:39874 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMox-0002z7-Pf for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I9bcYUVBS7YlvljqOSTUMtgB4UJko9QBdoOwL2ixfs0=; b=B1XsABDgo1wP6/RTs8/G0o9dlFbw9r7kvHBnaJq+li5zsZvmIGNG24XlQCG+Wtr2kBLQys e1wmfmsVjRGcVCxT44R7cRE2/7RMSgUfuhroMSUrnPNkKHin1EBM0a141lR5P2jux9INIH 0L+RKTG38HtNxjmLsxZq1raFM+mLXDo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-184-pNoKkH48P1m8xL1bci60Dw-1; Tue, 28 Jan 2020 04:06:05 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8A304A0CBF for ; Tue, 28 Jan 2020 09:06:04 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 472D210027A1; Tue, 28 Jan 2020 09:06:02 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 4/7] multifd: Add zlib compression multifd support Date: Tue, 28 Jan 2020 10:05:47 +0100 Message-Id: <20200128090550.5112-5-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: pNoKkH48P1m8xL1bci60Dw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Juan Quintela --- hw/core/qdev-properties.c | 2 +- migration/Makefile.objs | 1 + migration/migration.c | 9 ++ migration/migration.h | 1 + migration/multifd-zlib.c | 293 +++++++++++++++++++++++++++++++++++ migration/multifd.c | 6 + migration/multifd.h | 4 + qapi/migration.json | 3 +- tests/qtest/migration-test.c | 6 + 9 files changed, 323 insertions(+), 2 deletions(-) create mode 100644 migration/multifd-zlib.c diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c index 4442844d37..bf88a50cdf 100644 --- a/hw/core/qdev-properties.c +++ b/hw/core/qdev-properties.c @@ -645,7 +645,7 @@ const PropertyInfo qdev_prop_fdc_drive_type = { const PropertyInfo qdev_prop_multifd_method = { .name = "MultiFDMethod", .description = "multifd_method values, " - "none", + "none/zlib", .enum_table = &MultiFDMethod_lookup, .get = get_enum, .set = set_enum, diff --git a/migration/Makefile.objs b/migration/Makefile.objs index d3623d5f9b..0308caa5c5 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -8,6 +8,7 @@ common-obj-y += xbzrle.o postcopy-ram.o common-obj-y += qjson.o common-obj-y += block-dirty-bitmap.o common-obj-y += multifd.o +common-obj-y += multifd-zlib.o common-obj-$(CONFIG_RDMA) += rdma.o diff --git a/migration/migration.c b/migration/migration.c index 4f88f8e958..3b081e8147 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2265,6 +2265,15 @@ MultiFDMethod migrate_multifd_method(void) return s->parameters.multifd_method; } +int migrate_multifd_zlib_level(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.multifd_zlib_level; +} + int migrate_use_xbzrle(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index 3d23a0852e..95e9c196ff 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -301,6 +301,7 @@ bool migrate_use_multifd(void); bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); MultiFDMethod migrate_multifd_method(void); +int migrate_multifd_zlib_level(void); int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/migration/multifd-zlib.c b/migration/multifd-zlib.c new file mode 100644 index 0000000000..1589ea78b2 --- /dev/null +++ b/migration/multifd-zlib.c @@ -0,0 +1,293 @@ +/* + * Multifd zlib compression implementation + * + * Copyright (c) 2020 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include +#include "qemu/rcu.h" +#include "exec/target_page.h" +#include "qapi/error.h" +#include "migration.h" +#include "trace.h" +#include "multifd.h" + +struct zlib_data { + /* stream for compression */ + z_stream zs; + /* compressed buffer */ + uint8_t *zbuff; + /* size of compressed buffer */ + uint32_t zbuff_len; +}; + +/* Multifd zlib compression */ + +/** + * zlib_send_setup: setup send side + * + * Setup each channel with zlib compression. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int zlib_send_setup(MultiFDSendParams *p, Error **errp) +{ + uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + struct zlib_data *z = g_malloc0(sizeof(struct zlib_data)); + z_stream *zs = &z->zs; + + zs->zalloc = Z_NULL; + zs->zfree = Z_NULL; + zs->opaque = Z_NULL; + if (deflateInit(zs, migrate_multifd_zlib_level()) != Z_OK) { + g_free(z); + error_setg(errp, "multifd %d: deflate init failed", p->id); + return -1; + } + /* We will never have more than page_count pages */ + z->zbuff_len = page_count * qemu_target_page_size(); + z->zbuff_len *= 2; + z->zbuff = g_try_malloc(z->zbuff_len); + if (!z->zbuff) { + deflateEnd(&z->zs); + g_free(z); + error_setg(errp, "multifd %d: out of memory for zbuff", p->id); + return -1; + } + p->data = z; + return 0; +} + +/** + * zlib_send_cleanup: cleanup send side + * + * Close the channel and return memory. + * + * @p: Params for the channel that we are using + */ +static void zlib_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + struct zlib_data *z = p->data; + + deflateEnd(&z->zs); + g_free(z->zbuff); + z->zbuff = NULL; + g_free(p->data); + p->data = NULL; +} + +/** + * zlib_send_prepare: prepare date to be able to send + * + * Create a compressed buffer with all the pages that we are going to + * send. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + */ +static int zlib_send_prepare(MultiFDSendParams *p, uint32_t used, Error **errp) +{ + struct iovec *iov = p->pages->iov; + struct zlib_data *z = p->data; + z_stream *zs = &z->zs; + uint32_t out_size = 0; + int ret; + uint32_t i; + + for (i = 0; i < used; i++) { + uint32_t available = z->zbuff_len - out_size; + int flush = Z_NO_FLUSH; + + if (i == used - 1) { + flush = Z_SYNC_FLUSH; + } + + zs->avail_in = iov[i].iov_len; + zs->next_in = iov[i].iov_base; + + zs->avail_out = available; + zs->next_out = z->zbuff + out_size; + + ret = deflate(zs, flush); + if (ret != Z_OK) { + error_setg(errp, "multifd %d: deflate returned %d instead of Z_OK", + p->id, ret); + return -1; + } + out_size += available - zs->avail_out; + } + p->next_packet_size = out_size; + p->flags |= MULTIFD_FLAG_ZLIB; + + return 0; +} + +/** + * zlib_send_write: do the actual write of the data + * + * Do the actual write of the comprresed buffer. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int zlib_send_write(MultiFDSendParams *p, uint32_t used, Error **errp) +{ + struct zlib_data *z = p->data; + + return qio_channel_write_all(p->c, (void *)z->zbuff, p->next_packet_size, + errp); +} + +/** + * zlib_recv_setup: setup receive side + * + * Create the compressed channel and buffer. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int zlib_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + struct zlib_data *z = g_malloc0(sizeof(struct zlib_data)); + z_stream *zs = &z->zs; + + p->data = z; + zs->zalloc = Z_NULL; + zs->zfree = Z_NULL; + zs->opaque = Z_NULL; + zs->avail_in = 0; + zs->next_in = Z_NULL; + if (inflateInit(zs) != Z_OK) { + error_setg(errp, "multifd %d: inflate init failed", p->id); + return -1; + } + /* We will never have more than page_count pages */ + z->zbuff_len = page_count * qemu_target_page_size(); + /* We know compression "could" use more space */ + z->zbuff_len *= 2; + z->zbuff = g_try_malloc(z->zbuff_len); + if (!z->zbuff) { + inflateEnd(zs); + error_setg(errp, "multifd %d: out of memory for zbuff", p->id); + return -1; + } + return 0; +} + +/** + * zlib_recv_cleanup: setup receive side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + */ +static void zlib_recv_cleanup(MultiFDRecvParams *p) +{ + struct zlib_data *z = p->data; + + inflateEnd(&z->zs); + g_free(z->zbuff); + z->zbuff = NULL; + g_free(p->data); + p->data = NULL; +} + +/** + * zlib_recv_pages: read the data from the channel into actual pages + * + * Read the compressed buffer, and uncompress it into the actual + * pages. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int zlib_recv_pages(MultiFDRecvParams *p, uint32_t used, Error **errp) +{ + struct zlib_data *z = p->data; + z_stream *zs = &z->zs; + uint32_t in_size = p->next_packet_size; + /* we measure the change of total_out */ + uint32_t out_size = zs->total_out; + uint32_t expected_size = used * qemu_target_page_size(); + uint32_t flags = p->flags & MULTIFD_FLAG_METHOD_MASK; + int ret; + int i; + + if (flags != MULTIFD_FLAG_ZLIB) { + error_setg(errp, "multifd %d: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_ZLIB); + return -1; + } + ret = qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); + + if (ret != 0) { + return ret; + } + + zs->avail_in = in_size; + zs->next_in = z->zbuff; + + for (i = 0; i < used; i++) { + struct iovec *iov = &p->pages->iov[i]; + int flush = Z_NO_FLUSH; + + if (i == used - 1) { + flush = Z_SYNC_FLUSH; + } + + zs->avail_out = iov->iov_len; + zs->next_out = iov->iov_base; + + ret = inflate(zs, flush); + if (ret != Z_OK) { + error_setg(errp, "multifd %d: inflate returned %d instead of Z_OK", + p->id, ret); + return ret; + } + } + out_size = zs->total_out - out_size; + if (out_size != expected_size) { + error_setg(errp, "multifd %d: packet size received %d size expected %d", + p->id, out_size, expected_size); + return -1; + } + return 0; +} + +static MultiFDMethods multifd_zlib_ops = { + .send_setup = zlib_send_setup, + .send_cleanup = zlib_send_cleanup, + .send_prepare = zlib_send_prepare, + .send_write = zlib_send_write, + .recv_setup = zlib_recv_setup, + .recv_cleanup = zlib_recv_cleanup, + .recv_pages = zlib_recv_pages +}; + +static void multifd_zlib_register(void) +{ + multifd_register_ops(MULTIFD_METHOD_ZLIB, &multifd_zlib_ops); +} + +migration_init(multifd_zlib_register); diff --git a/migration/multifd.c b/migration/multifd.c index 1c49c2a665..7bb9c3582f 100644 --- a/migration/multifd.c +++ b/migration/multifd.c @@ -166,6 +166,12 @@ static MultiFDMethods *multifd_ops[MULTIFD_METHOD__MAX] = { [MULTIFD_METHOD_NONE] = &multifd_nocomp_ops, }; +void multifd_register_ops(int method, MultiFDMethods *ops) +{ + assert(0 < method && method < MULTIFD_METHOD__MAX); + multifd_ops[method] = ops; +} + static int multifd_send_initial_packet(MultiFDSendParams *p, Error **errp) { MultiFDInit_t msg = {}; diff --git a/migration/multifd.h b/migration/multifd.h index c7fea4914c..3fa5132f1d 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -23,11 +23,13 @@ void multifd_recv_sync_main(void); void multifd_send_sync_main(QEMUFile *f); int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); +/* Multifd Compression flags */ #define MULTIFD_FLAG_SYNC (1 << 0) /* We reserve 3 bits for METHODS */ #define MULTIFD_FLAG_METHOD_MASK (7 << 1) #define MULTIFD_FLAG_NOCOMP (1 << 1) +#define MULTIFD_FLAG_ZLIB (2 << 1) /* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024) @@ -160,5 +162,7 @@ typedef struct { int (*recv_pages)(MultiFDRecvParams *p, uint32_t used, Error **errp); } MultiFDMethods; +void multifd_register_ops(int method, MultiFDMethods *ops); + #endif diff --git a/qapi/migration.json b/qapi/migration.json index 289dce0da7..032ee7d3e6 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -494,12 +494,13 @@ # An enumeration of multifd compression. # # @none: no compression. +# @zlib: use zlib compression method. # # Since: 5.0 # ## { 'enum': 'MultiFDMethod', - 'data': [ 'none' ] } + 'data': [ 'none', 'zlib' ] } ## # @MigrationParameter: diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index d2f9ef38f5..8effed205d 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1313,6 +1313,11 @@ static void test_multifd_tcp_none(void) test_multifd_tcp("none"); } +static void test_multifd_tcp_zlib(void) +{ + test_multifd_tcp("zlib"); +} + /* * This test does: * source target @@ -1475,6 +1480,7 @@ int main(int argc, char **argv) qtest_add_func("/migration/auto_converge", test_migrate_auto_converge); qtest_add_func("/migration/multifd/tcp/none", test_multifd_tcp_none); qtest_add_func("/migration/multifd/tcp/cancel", test_multifd_tcp_cancel); + qtest_add_func("/migration/multifd/tcp/zlib", test_multifd_tcp_zlib); ret = g_test_run(); From patchwork Tue Jan 28 09:05:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230231 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=PCaejXQz; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LRh5sSlz9sP6 for ; Tue, 28 Jan 2020 20:10:07 +1100 (AEDT) Received: from localhost ([::1]:55708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMsl-0004EW-Vm for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:10:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56690) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMp2-0005vr-5d for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMp0-00032G-Bm for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:11 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:55846 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMoz-00031h-Rw for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202369; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kjg8R4UdaoSoMSC6zqKoCCwIvu14OyCAXYMpRWt8jgk=; b=PCaejXQz6H1scoNQTmgDkyU32BaT0ZbadFGeO+7Auv3aBbQNdrUnTCXTaSrYDXuEu/TPoE 1hporPVxv2IFBhVRZMpIibVP5mxunnGjpzCFMEkRyDzs7XfZawhzS1tvPPpi7Z31xfJtLt 1c3EM+8Hp6wYUt2F2JzmxGc3d9JIDfQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-289-xA0AK4OtMASpYUAgijdklw-1; Tue, 28 Jan 2020 04:06:08 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32F07A0CDD for ; Tue, 28 Jan 2020 09:06:07 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD15C1001B39; Tue, 28 Jan 2020 09:06:04 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 5/7] configure: Enable test and libs for zstd Date: Tue, 28 Jan 2020 10:05:48 +0100 Message-Id: <20200128090550.5112-6-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: xA0AK4OtMASpYUAgijdklw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert --- configure | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/configure b/configure index a72a5def57..7c1dca326f 100755 --- a/configure +++ b/configure @@ -448,6 +448,7 @@ lzo="" snappy="" bzip2="" lzfse="" +zstd="" guest_agent="" guest_agent_with_vss="no" guest_agent_ntddscsi="no" @@ -1343,6 +1344,10 @@ for opt do ;; --disable-lzfse) lzfse="no" ;; + --disable-zstd) zstd="no" + ;; + --enable-zstd) zstd="yes" + ;; --enable-guest-agent) guest_agent="yes" ;; --disable-guest-agent) guest_agent="no" @@ -1795,6 +1800,8 @@ disabled with --disable-FEATURE, default is enabled if available: (for reading bzip2-compressed dmg images) lzfse support of lzfse compression library (for reading lzfse-compressed dmg images) + zstd support for zstd compression library + (for migration compression) seccomp seccomp support coroutine-pool coroutine freelist (better performance) glusterfs GlusterFS backend @@ -2409,6 +2416,24 @@ EOF fi fi +########################################## +# zstd check + +if test "$zstd" != "no" ; then + if $pkg_config --exist libzstd ; then + zstd_cflags="$($pkg_config --cflags libzstd)" + zstd_libs="$($pkg_config --libs libzstd)" + LIBS="$zstd_libs $LIBS" + QEMU_CFLAGS="$QEMU_CFLAGS $zstd_cflags" + zstd="yes" + else + if test "$zstd" = "yes" ; then + feature_not_found "libzstd" "Install libzstd devel" + fi + zstd="no" + fi +fi + ########################################## # libseccomp check @@ -6578,6 +6603,7 @@ echo "lzo support $lzo" echo "snappy support $snappy" echo "bzip2 support $bzip2" echo "lzfse support $lzfse" +echo "zstd support $zstd" echo "NUMA host support $numa" echo "libxml2 $libxml2" echo "tcmalloc support $tcmalloc" @@ -7143,6 +7169,10 @@ if test "$lzfse" = "yes" ; then echo "LZFSE_LIBS=-llzfse" >> $config_host_mak fi +if test "$zstd" = "yes" ; then + echo "CONFIG_ZSTD=y" >> $config_host_mak +fi + if test "$libiscsi" = "yes" ; then echo "CONFIG_LIBISCSI=m" >> $config_host_mak echo "LIBISCSI_CFLAGS=$libiscsi_cflags" >> $config_host_mak From patchwork Tue Jan 28 09:05:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230230 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KceTiJTS; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LQX3k81z9sP6 for ; Tue, 28 Jan 2020 20:09:08 +1100 (AEDT) Received: from localhost ([::1]:55698 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMrq-0002Dh-Hl for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:09:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56706) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMp5-0005zO-Up for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMp4-00037d-3g for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:15 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:26174 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMp3-00034x-D0 for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202372; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Yx6NVrkH5zxu4je3L/Hh3hmQyfdqMN/BdJ5eDYZYxNM=; b=KceTiJTS1fTYSJxJFijGSgSBT2Eyv2aG7+Tw2UkwHwqhR99kahfJvgpmO11mDfSag0dzhb HtuuaPdsk4vlcbf/TlVs+IdEQY3vAppAS09Kt/hQiewHzbnQrC62BE229+egVDi0wy7/vf NK35DeUopCbsBAe83SJlWWitcOvjoP8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-34-YXbgzbLcOWOf8fTq__w1Ew-1; Tue, 28 Jan 2020 04:06:10 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C6576800EBB for ; Tue, 28 Jan 2020 09:06:09 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 82A731084192; Tue, 28 Jan 2020 09:06:07 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 6/7] multifd: Add multifd-zstd-level parameter Date: Tue, 28 Jan 2020 10:05:49 +0100 Message-Id: <20200128090550.5112-7-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: YXbgzbLcOWOf8fTq__w1Ew-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Juan Quintela --- migration/migration.c | 15 +++++++++++++++ monitor/hmp-cmds.c | 4 ++++ qapi/migration.json | 29 ++++++++++++++++++++++++++--- 3 files changed, 45 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 3b081e8147..b690500545 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -91,6 +91,8 @@ #define DEFAULT_MIGRATE_MULTIFD_METHOD MULTIFD_METHOD_NONE /*0: means nocompress, 1: best speed, ... 9: best compress ratio */ #define DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL 1 +/* 0: means nocompress, 1: best speed, ... 20: best compress ratio */ +#define DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL 1 /* Background transfer rate for postcopy, 0 means unlimited, note * that page requests can still exceed this limit. @@ -805,6 +807,8 @@ MigrationParameters *qmp_query_migrate_parameters(Error **errp) params->multifd_method = s->parameters.multifd_method; params->has_multifd_zlib_level = true; params->multifd_zlib_level = s->parameters.multifd_zlib_level; + params->has_multifd_zstd_level = true; + params->multifd_zstd_level = s->parameters.multifd_zstd_level; params->has_xbzrle_cache_size = true; params->xbzrle_cache_size = s->parameters.xbzrle_cache_size; params->has_max_postcopy_bandwidth = true; @@ -1219,6 +1223,13 @@ static bool migrate_params_check(MigrationParameters *params, Error **errp) return false; } + if (params->has_multifd_zstd_level && + (params->multifd_zstd_level > 20)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "multifd_zstd_level", + "is invalid, it should be in the range of 0 to 20"); + return false; + } + if (params->has_xbzrle_cache_size && (params->xbzrle_cache_size < qemu_target_page_size() || !is_power_of_2(params->xbzrle_cache_size))) { @@ -3559,6 +3570,9 @@ static Property migration_properties[] = { DEFINE_PROP_UINT8("multifd-zlib-level", MigrationState, parameters.multifd_zlib_level, DEFAULT_MIGRATE_MULTIFD_ZLIB_LEVEL), + DEFINE_PROP_UINT8("multifd-zstd-level", MigrationState, + parameters.multifd_zstd_level, + DEFAULT_MIGRATE_MULTIFD_ZSTD_LEVEL), DEFINE_PROP_SIZE("xbzrle-cache-size", MigrationState, parameters.xbzrle_cache_size, DEFAULT_MIGRATE_XBZRLE_CACHE_SIZE), @@ -3651,6 +3665,7 @@ static void migration_instance_init(Object *obj) params->has_multifd_channels = true; params->has_multifd_method = true; params->has_multifd_zlib_level = true; + params->has_multifd_zstd_level = true; params->has_xbzrle_cache_size = true; params->has_max_postcopy_bandwidth = true; params->has_max_cpu_throttle = true; diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c index 7f11866446..87db07694b 100644 --- a/monitor/hmp-cmds.c +++ b/monitor/hmp-cmds.c @@ -1840,6 +1840,10 @@ void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict) p->has_multifd_zlib_level = true; visit_type_int(v, param, &p->multifd_zlib_level, &err); break; + case MIGRATION_PARAMETER_MULTIFD_ZSTD_LEVEL: + p->has_multifd_zstd_level = true; + visit_type_int(v, param, &p->multifd_zstd_level, &err); + break; case MIGRATION_PARAMETER_XBZRLE_CACHE_SIZE: p->has_xbzrle_cache_size = true; visit_type_size(v, param, &cache_size, &err); diff --git a/qapi/migration.json b/qapi/migration.json index 032ee7d3e6..bb5cb6b4f4 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -610,6 +610,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @multifd-zstd-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 20, where 0 means no compression, 1 means the best +# compression speed, and 20 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## { 'enum': 'MigrationParameter', @@ -623,7 +630,7 @@ 'multifd-channels', 'xbzrle-cache-size', 'max-postcopy-bandwidth', 'max-cpu-throttle', 'multifd-method', - 'multifd-zlib-level' ] } + 'multifd-zlib-level' ,'multifd-zstd-level' ] } ## # @MigrateSetParameters: @@ -723,6 +730,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @multifd-zstd-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 20, where 0 means no compression, 1 means the best +# compression speed, and 20 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## # TODO either fuse back into MigrationParameters, or make @@ -750,7 +764,8 @@ '*max-postcopy-bandwidth': 'size', '*max-cpu-throttle': 'int', '*multifd-method': 'MultiFDMethod', - '*multifd-zlib-level': 'int' } } + '*multifd-zlib-level': 'int', + '*multifd-zstd-level': 'int' } } ## # @migrate-set-parameters: @@ -870,6 +885,13 @@ # will consume more CPU. # Defaults to 1. (Since 5.0) # +# @multifd-zstd-level: Set the compression level to be used in live +# migration, the compression level is an integer between 0 +# and 20, where 0 means no compression, 1 means the best +# compression speed, and 20 means best compression ratio which +# will consume more CPU. +# Defaults to 1. (Since 5.0) +# # Since: 2.4 ## { 'struct': 'MigrationParameters', @@ -895,7 +917,8 @@ '*max-postcopy-bandwidth': 'size', '*max-cpu-throttle': 'uint8', '*multifd-method': 'MultiFDMethod', - '*multifd-zlib-level': 'uint8' } } + '*multifd-zlib-level': 'uint8', + '*multifd-zstd-level': 'uint8' } } ## # @query-migrate-parameters: From patchwork Tue Jan 28 09:05:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 1230223 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RF5KvqT6; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 486LMw58gjz9sP6 for ; Tue, 28 Jan 2020 20:06:52 +1100 (AEDT) Received: from localhost ([::1]:55670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMpe-0006Ay-MG for incoming@patchwork.ozlabs.org; Tue, 28 Jan 2020 04:06:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56722) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iwMp8-00063v-3N for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iwMp6-0003AX-2D for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:17 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:34966 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iwMp5-00039e-TE for qemu-devel@nongnu.org; Tue, 28 Jan 2020 04:06:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1580202375; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EEPuHB5jija28oBZo50X+P8kYkIas0oWsil7KuDKjjU=; b=RF5KvqT6PA/dQlCGfn2zVbVTK7x3OvDAEywe0g/5LPZD13eHRk0pc4RfVOaTSyYMR5RhRf q2xbcKFGTVWipOfoMPxpwK3EALZzZdJ4Xu/Gv6x38WHNpDS1ysvfAXG0mUL8ZmB9GctzwE MoSJTSflmyGMLyxHCtuZP9wIMgk4TTw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-5SIoVx8fNCylbzZLPMqEnA-1; Tue, 28 Jan 2020 04:06:13 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 66D98800D4C for ; Tue, 28 Jan 2020 09:06:12 +0000 (UTC) Received: from secure.mitica (ovpn-116-64.ams2.redhat.com [10.36.116.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id 25D2A1001B3F; Tue, 28 Jan 2020 09:06:09 +0000 (UTC) From: Juan Quintela To: qemu-devel@nongnu.org Subject: [PATCH v4 7/7] multifd: Add zstd compression multifd support Date: Tue, 28 Jan 2020 10:05:50 +0100 Message-Id: <20200128090550.5112-8-quintela@redhat.com> In-Reply-To: <20200128090550.5112-1-quintela@redhat.com> References: <20200128090550.5112-1-quintela@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 5SIoVx8fNCylbzZLPMqEnA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Laurent Vivier , Thomas Huth , =?utf-8?q?Daniel_P=2E_Berrang=C3=A9?= , Eduardo Habkost , Juan Quintela , "Dr. David Alan Gilbert" , Markus Armbruster , Paolo Bonzini Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Juan Quintela --- Remove CONFIG_ZSTD gross test for enable zstd option. Thanhs Markus!!! --- hw/core/qdev-properties.c | 2 +- migration/Makefile.objs | 1 + migration/migration.c | 9 ++ migration/migration.h | 1 + migration/multifd-zstd.c | 305 +++++++++++++++++++++++++++++++++++ migration/multifd.h | 2 +- migration/ram.c | 1 - qapi/migration.json | 4 +- tests/qtest/migration-test.c | 10 ++ 9 files changed, 331 insertions(+), 4 deletions(-) create mode 100644 migration/multifd-zstd.c diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c index bf88a50cdf..9440ca78c3 100644 --- a/hw/core/qdev-properties.c +++ b/hw/core/qdev-properties.c @@ -645,7 +645,7 @@ const PropertyInfo qdev_prop_fdc_drive_type = { const PropertyInfo qdev_prop_multifd_method = { .name = "MultiFDMethod", .description = "multifd_method values, " - "none/zlib", + "none/zlib/zstd", .enum_table = &MultiFDMethod_lookup, .get = get_enum, .set = set_enum, diff --git a/migration/Makefile.objs b/migration/Makefile.objs index 0308caa5c5..0fc619e380 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -9,6 +9,7 @@ common-obj-y += qjson.o common-obj-y += block-dirty-bitmap.o common-obj-y += multifd.o common-obj-y += multifd-zlib.o +common-obj-$(CONFIG_ZSTD) += multifd-zstd.o common-obj-$(CONFIG_RDMA) += rdma.o diff --git a/migration/migration.c b/migration/migration.c index b690500545..aff081128c 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -2285,6 +2285,15 @@ int migrate_multifd_zlib_level(void) return s->parameters.multifd_zlib_level; } +int migrate_multifd_zstd_level(void) +{ + MigrationState *s; + + s = migrate_get_current(); + + return s->parameters.multifd_zstd_level; +} + int migrate_use_xbzrle(void) { MigrationState *s; diff --git a/migration/migration.h b/migration/migration.h index 95e9c196ff..2eb72aee0a 100644 --- a/migration/migration.h +++ b/migration/migration.h @@ -302,6 +302,7 @@ bool migrate_pause_before_switchover(void); int migrate_multifd_channels(void); MultiFDMethod migrate_multifd_method(void); int migrate_multifd_zlib_level(void); +int migrate_multifd_zstd_level(void); int migrate_use_xbzrle(void); int64_t migrate_xbzrle_cache_size(void); diff --git a/migration/multifd-zstd.c b/migration/multifd-zstd.c new file mode 100644 index 0000000000..a594c1af30 --- /dev/null +++ b/migration/multifd-zstd.c @@ -0,0 +1,305 @@ +/* + * Multifd zlib compression implementation + * + * Copyright (c) 2020 Red Hat Inc + * + * Authors: + * Juan Quintela + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include +#include "qemu/rcu.h" +#include "exec/target_page.h" +#include "qapi/error.h" +#include "migration.h" +#include "trace.h" +#include "multifd.h" + +struct zstd_data { + /* stream for compression */ + ZSTD_CStream *zcs; + /* stream for decompression */ + ZSTD_DStream *zds; + /* buffers */ + ZSTD_inBuffer in; + ZSTD_outBuffer out; + /* compressed buffer */ + uint8_t *zbuff; + /* size of compressed buffer */ + uint32_t zbuff_len; +}; + +/* Multifd zstd compression */ + +/** + * zstd_send_setup: setup send side + * + * Setup each channel with zstd compression. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int zstd_send_setup(MultiFDSendParams *p, Error **errp) +{ + uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + struct zstd_data *z = g_new0(struct zstd_data, 1); + int res; + + p->data = z; + z->zcs = ZSTD_createCStream(); + if (!z->zcs) { + g_free(z); + error_setg(errp, "multifd %d: zstd createCStream failed", p->id); + return -1; + } + + res = ZSTD_initCStream(z->zcs, migrate_multifd_zstd_level()); + if (ZSTD_isError(res)) { + ZSTD_freeCStream(z->zcs); + g_free(z); + error_setg(errp, "multifd %d: initCStream failed", p->id); + return -1; + } + /* We will never have more than page_count pages */ + z->zbuff_len = page_count * qemu_target_page_size(); + z->zbuff_len *= 2; + z->zbuff = g_try_malloc(z->zbuff_len); + if (!z->zbuff) { + ZSTD_freeCStream(z->zcs); + g_free(z); + error_setg(errp, "multifd %d: out of memory for zbuff", p->id); + return -1; + } + return 0; +} + +/** + * zstd_send_cleanup: cleanup send side + * + * Close the channel and return memory. + * + * @p: Params for the channel that we are using + */ +static void zstd_send_cleanup(MultiFDSendParams *p, Error **errp) +{ + struct zstd_data *z = p->data; + + ZSTD_freeCStream(z->zcs); + z->zcs = NULL; + g_free(z->zbuff); + z->zbuff = NULL; + g_free(p->data); + p->data = NULL; +} + +/** + * zstd_send_prepare: prepare date to be able to send + * + * Create a compressed buffer with all the pages that we are going to + * send. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + */ +static int zstd_send_prepare(MultiFDSendParams *p, uint32_t used, Error **errp) +{ + struct iovec *iov = p->pages->iov; + struct zstd_data *z = p->data; + int ret; + uint32_t i; + + z->out.dst = z->zbuff; + z->out.size = z->zbuff_len; + z->out.pos = 0; + + for (i = 0; i < used; i++) { + ZSTD_EndDirective flush = ZSTD_e_continue; + + if (i == used - 1) { + flush = ZSTD_e_flush; + } + z->in.src = iov[i].iov_base; + z->in.size = iov[i].iov_len; + z->in.pos = 0; + + ret = ZSTD_compressStream2(z->zcs, &z->out, &z->in, flush); + if (ZSTD_isError(ret)) { + error_setg(errp, "multifd %d: compressStream error %s", + p->id, ZSTD_getErrorName(ret)); + return -1; + } + } + p->next_packet_size = z->out.pos; + p->flags |= MULTIFD_FLAG_ZSTD; + + return 0; +} + +/** + * zstd_send_write: do the actual write of the data + * + * Do the actual write of the comprresed buffer. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int zstd_send_write(MultiFDSendParams *p, uint32_t used, Error **errp) +{ + struct zstd_data *z = p->data; + + return qio_channel_write_all(p->c, (void *)z->zbuff, p->next_packet_size, + errp); +} + +/** + * zstd_recv_setup: setup receive side + * + * Create the compressed channel and buffer. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @errp: pointer to an error + */ +static int zstd_recv_setup(MultiFDRecvParams *p, Error **errp) +{ + uint32_t page_count = MULTIFD_PACKET_SIZE / qemu_target_page_size(); + struct zstd_data *z = g_new0(struct zstd_data, 1); + int res; + + p->data = z; + z->zds = ZSTD_createDStream(); + if (!z->zds) { + g_free(z); + error_setg(errp, "multifd %d: zstd createDStream failed", p->id); + return -1; + } + + res = ZSTD_initDStream(z->zds); + if (ZSTD_isError(res)) { + ZSTD_freeDStream(z->zds); + g_free(z); + error_setg(errp, "multifd %d: initDStream failed", p->id); + return -1; + } + + /* We will never have more than page_count pages */ + z->zbuff_len = page_count * qemu_target_page_size(); + /* We know compression "could" use more space */ + z->zbuff_len *= 2; + z->zbuff = g_try_malloc(z->zbuff_len); + if (!z->zbuff) { + ZSTD_freeDStream(z->zds); + g_free(z); + error_setg(errp, "multifd %d: out of memory for zbuff", p->id); + return -1; + } + return 0; +} + +/** + * zstd_recv_cleanup: setup receive side + * + * For no compression this function does nothing. + * + * @p: Params for the channel that we are using + */ +static void zstd_recv_cleanup(MultiFDRecvParams *p) +{ + struct zstd_data *z = p->data; + + ZSTD_freeDStream(z->zds); + z->zds = NULL; + g_free(z->zbuff); + z->zbuff = NULL; + g_free(p->data); + p->data = NULL; +} + +/** + * zstd_recv_pages: read the data from the channel into actual pages + * + * Read the compressed buffer, and uncompress it into the actual + * pages. + * + * Returns 0 for success or -1 for error + * + * @p: Params for the channel that we are using + * @used: number of pages used + * @errp: pointer to an error + */ +static int zstd_recv_pages(MultiFDRecvParams *p, uint32_t used, Error **errp) +{ + uint32_t in_size = p->next_packet_size; + uint32_t out_size = 0; + uint32_t expected_size = used * qemu_target_page_size(); + uint32_t flags = p->flags & MULTIFD_FLAG_METHOD_MASK; + struct zstd_data *z = p->data; + int ret; + int i; + + if (flags != MULTIFD_FLAG_ZSTD) { + error_setg(errp, "multifd %d: flags received %x flags expected %x", + p->id, flags, MULTIFD_FLAG_ZSTD); + return -1; + } + ret = qio_channel_read_all(p->c, (void *)z->zbuff, in_size, errp); + + if (ret != 0) { + return ret; + } + + z->in.src = z->zbuff; + z->in.size = in_size; + z->in.pos = 0; + + for (i = 0; i < used; i++) { + struct iovec *iov = &p->pages->iov[i]; + + z->out.dst = iov->iov_base; + z->out.size = iov->iov_len; + z->out.pos = 0; + + ret = ZSTD_decompressStream(z->zds, &z->out, &z->in); + if (ZSTD_isError(ret)) { + error_setg(errp, "multifd %d: decompressStream returned %s", + p->id, ZSTD_getErrorName(ret)); + return ret; + } + out_size += iov->iov_len; + } + if (out_size != expected_size) { + error_setg(errp, "multifd %d: packet size received %d size expected %d", + p->id, out_size, expected_size); + return -1; + } + return 0; +} + +static MultiFDMethods multifd_zstd_ops = { + .send_setup = zstd_send_setup, + .send_cleanup = zstd_send_cleanup, + .send_prepare = zstd_send_prepare, + .send_write = zstd_send_write, + .recv_setup = zstd_recv_setup, + .recv_cleanup = zstd_recv_cleanup, + .recv_pages = zstd_recv_pages +}; + +static void multifd_zstd_register(void) +{ + multifd_register_ops(MULTIFD_METHOD_ZSTD, &multifd_zstd_ops); +} + +migration_init(multifd_zstd_register); diff --git a/migration/multifd.h b/migration/multifd.h index 3fa5132f1d..621db316c1 100644 --- a/migration/multifd.h +++ b/migration/multifd.h @@ -30,6 +30,7 @@ int multifd_queue_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset); #define MULTIFD_FLAG_METHOD_MASK (7 << 1) #define MULTIFD_FLAG_NOCOMP (1 << 1) #define MULTIFD_FLAG_ZLIB (2 << 1) +#define MULTIFD_FLAG_ZSTD (3 << 1) /* This value needs to be a multiple of qemu_target_page_size() */ #define MULTIFD_PACKET_SIZE (512 * 1024) @@ -163,6 +164,5 @@ typedef struct { } MultiFDMethods; void multifd_register_ops(int method, MultiFDMethods *ops); - #endif diff --git a/migration/ram.c b/migration/ram.c index 73a141bb60..0ef68798d2 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -28,7 +28,6 @@ #include "qemu/osdep.h" #include "cpu.h" -#include #include "qemu/cutils.h" #include "qemu/bitops.h" #include "qemu/bitmap.h" diff --git a/qapi/migration.json b/qapi/migration.json index bb5cb6b4f4..8ccec7fa79 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -495,12 +495,14 @@ # # @none: no compression. # @zlib: use zlib compression method. +# @zstd: use zstd compression method. # # Since: 5.0 # ## { 'enum': 'MultiFDMethod', - 'data': [ 'none', 'zlib' ] } + 'data': [ 'none', 'zlib', + { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] } ## # @MigrationParameter: diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 8effed205d..ec9be28bc9 100644 --- a/tests/qtest/migration-test.c +++ b/tests/qtest/migration-test.c @@ -1318,6 +1318,13 @@ static void test_multifd_tcp_zlib(void) test_multifd_tcp("zlib"); } +#ifdef CONFIG_ZSTD +static void test_multifd_tcp_zstd(void) +{ + test_multifd_tcp("zstd"); +} +#endif + /* * This test does: * source target @@ -1481,6 +1488,9 @@ int main(int argc, char **argv) qtest_add_func("/migration/multifd/tcp/none", test_multifd_tcp_none); qtest_add_func("/migration/multifd/tcp/cancel", test_multifd_tcp_cancel); qtest_add_func("/migration/multifd/tcp/zlib", test_multifd_tcp_zlib); +#ifdef CONFIG_ZSTD + qtest_add_func("/migration/multifd/tcp/zstd", test_multifd_tcp_zstd); +#endif ret = g_test_run();