[{"id":1767802,"web_url":"http://patchwork.ozlabs.org/comment/1767802/","msgid":"<20170913113534.GC3067@redhat.com>","list_archive_url":null,"date":"2017-09-13T11:35:34","subject":"Re: [Qemu-devel] [PATCH v8 12/20] migration: Start of multiple fd\n\twork","submitter":{"id":2694,"url":"http://patchwork.ozlabs.org/api/people/2694/","name":"Daniel P. Berrangé","email":"berrange@redhat.com"},"content":"On Wed, Sep 13, 2017 at 12:59:45PM +0200, Juan Quintela wrote:\n> We create new channels for each new thread created. We send through\n> them a string containing multifd so we are\n> sure that we connect the right channels in both sides.\n> \n> Signed-off-by: Juan Quintela \n> \n> --\n> Split SocketArgs into incoming and outgoing args\n> \n> Use UUID's on the initial message, so we are sure we are connecting to\n> the right channel.\n> \n> Remove init semaphore. Now that we use uuids on the init message, we\n> know that this is our channel.\n> \n> Fix recv socket destwroy, we were destroying send channels.\n> This was very interesting, because we were using an unreferred object\n> without problems.\n> \n> Move to struct of pointers\n> init channel sooner.\n> split recv thread creation.\n> listen on main thread\n> We count the number of created threads to know when we need to stop listening\n> Use g_strdup_printf\n> report channel id on errors\n> Add name parameter\n> Use local_err\n> Add Error * parameter to socket_send_channel_create()\n> Use qio_channel_*_all\n> Use asynchronous connect\n> ---\n> migration/migration.c | 5 ++\n> migration/ram.c | 138 +++++++++++++++++++++++++++++++++++++++++++-------\n> migration/ram.h | 3 ++\n> migration/socket.c | 34 ++++++++++++-\n> migration/socket.h | 10 ++++\n> 5 files changed, 172 insertions(+), 18 deletions(-)\n> \n> diff --git a/migration/migration.c b/migration/migration.c\n> index 1401841997..679be8e8d4 100644\n> --- a/migration/migration.c\n> +++ b/migration/migration.c\n> @@ -419,6 +419,11 @@ void migration_ioc_process_incoming(QIOChannel *ioc)\n> */\n> bool migration_has_all_channels(void)\n> {\n> + if (migrate_use_multifd()) {\n> + int thread_count = migrate_multifd_channels();\n> +\n> + return thread_count == multifd_created_channels();\n> + }\n> return true;\n> }\n> \n> diff --git a/migration/ram.c b/migration/ram.c\n> index a3e2abb2a5..8577eeb032 100644\n> --- a/migration/ram.c\n> +++ b/migration/ram.c\n> @@ -36,6 +36,7 @@\n> #include \"xbzrle.h\"\n> #include \"ram.h\"\n> #include \"migration.h\"\n> +#include \"socket.h\"\n> #include \"migration/register.h\"\n> #include \"migration/misc.h\"\n> #include \"qemu-file.h\"\n> @@ -46,6 +47,8 @@\n> #include \"exec/ram_addr.h\"\n> #include \"qemu/rcu_queue.h\"\n> #include \"migration/colo.h\"\n> +#include \"sysemu/sysemu.h\"\n> +#include \"qemu/uuid.h\"\n> \n> /***********************************************************/\n> /* ram save/restore */\n> @@ -362,6 +365,7 @@ struct MultiFDSendParams {\n> uint8_t id;\n> char *name;\n> QemuThread thread;\n> + QIOChannel *c;\n> QemuSemaphore sem;\n> QemuMutex mutex;\n> bool quit;\n> @@ -378,6 +382,12 @@ static void terminate_multifd_send_threads(Error *errp)\n> {\n> int i;\n> \n> + if (errp) {\n> + MigrationState *s = migrate_get_current();\n> + migrate_set_error(s, errp);\n> + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE,\n> + MIGRATION_STATUS_FAILED);\n> + }\n> for (i = 0; i < multifd_send_state->count; i++) {\n> MultiFDSendParams *p = &multifd_send_state->params[i];\n> \n> @@ -403,6 +413,7 @@ int multifd_save_cleanup(Error **errp)\n> qemu_thread_join(&p->thread);\n> qemu_mutex_destroy(&p->mutex);\n> qemu_sem_destroy(&p->sem);\n> + socket_send_channel_destroy(p->c);\n> g_free(p->name);\n> p->name = NULL;\n> }\n> @@ -413,9 +424,32 @@ int multifd_save_cleanup(Error **errp)\n> return ret;\n> }\n> \n> +/* Default uuid for multifd when qemu is not started with uuid */\n> +static char multifd_uuid[] = \"5c49fd7e-af88-4a07-b6e8-091fd696ad40\";\n\nSame comment as last time - this is pointless. You should just\nunconditionally send 'qemu_uuid'. If the user hasn't set it\nvia '--uuid', it'll be a fixed UUID of all-zeros which is not\na semantic problem for the usage you have here.\n\n> +/* strlen(multifd) + '-' + + '-' + UUID_FMT + '\\0' */\n> +#define MULTIFD_UUID_MSG (7 + 1 + 3 + 1 + UUID_FMT_LEN + 1)\n\n> static void *multifd_send_thread(void *opaque)\n> {\n> MultiFDSendParams *p = opaque;\n> + Error *local_err = NULL;\n> + char *string;\n> + char *string_uuid;\n> + size_t ret;\n> +\n> + if (qemu_uuid_set) {\n> + string_uuid = qemu_uuid_unparse_strdup(&qemu_uuid);\n> + } else {\n> + string_uuid = g_strdup(multifd_uuid);\n> + }\n> + string = g_strdup_printf(\"%s multifd %03d\", string_uuid, p->id);\n\nAs before, if we need to send some structured data to the other\nend, we should define a struct with fields for that, not send\nplain text with printf+scanf.\n\nEven better if you put a version in the struct so that you have\nsomething to detect if you need to add more fields at a later\ndate. eg\n\n struct MigrateMultiFDInit {\n uint32_t version;\n char uuid[32];\n uint32_t id;\n };\n\n\n> + g_free(string_uuid);\n> + ret = qio_channel_write_all(p->c, string, MULTIFD_UUID_MSG, &local_err);\n> + g_free(string);\n> + if (ret != 0) {\n> + terminate_multifd_send_threads(local_err);\n> + return NULL;\n> + }\n> \n> while (true) {\n> qemu_mutex_lock(&p->mutex);\n> @@ -430,6 +464,27 @@ static void *multifd_send_thread(void *opaque)\n> return NULL;\n> }\n> \n> +static void multifd_new_channel_async(QIOTask *task, gpointer opaque)\n> +{\n> + MultiFDSendParams *p = opaque;\n> + QIOChannel *sioc = QIO_CHANNEL(qio_task_get_source(task));\n> + Error *local_err;\n> +\n> + if (qio_task_propagate_error(task, &local_err)) {\n> + if (multifd_save_cleanup(&local_err) != 0) {\n> + migrate_set_error(migrate_get_current(), local_err);\n> + }\n> + } else {\n> + p->c = QIO_CHANNEL(sioc);\n> + qio_channel_set_delay(p->c, false);\n> +\n> + qemu_thread_create(&p->thread, p->name, multifd_send_thread, p,\n> + QEMU_THREAD_JOINABLE);\n> +\n> + multifd_send_state->count++;\n> + }\n> +}\n> +\n> int multifd_save_setup(void)\n> {\n> int thread_count;\n> @@ -450,10 +505,7 @@ int multifd_save_setup(void)\n> p->quit = false;\n> p->id = i;\n> p->name = g_strdup_printf(\"multifdsend_%d\", i);\n> - qemu_thread_create(&p->thread, p->name, multifd_send_thread, p,\n> - QEMU_THREAD_JOINABLE);\n> -\n> - multifd_send_state->count++;\n> + socket_send_channel_create(multifd_new_channel_async, p);\n> }\n> return 0;\n> }\n> @@ -462,6 +514,7 @@ struct MultiFDRecvParams {\n> uint8_t id;\n> char *name;\n> QemuThread thread;\n> + QIOChannel *c;\n> QemuSemaphore sem;\n> QemuMutex mutex;\n> bool quit;\n> @@ -472,12 +525,22 @@ struct {\n> MultiFDRecvParams *params;\n> /* number of created threads */\n> int count;\n> + /* Should we finish */\n> + bool quit;\n> } *multifd_recv_state;\n> \n> static void terminate_multifd_recv_threads(Error *errp)\n> {\n> int i;\n> \n> + if (errp) {\n> + MigrationState *s = migrate_get_current();\n> + migrate_set_error(s, errp);\n> + migrate_set_state(&s->state, MIGRATION_STATUS_ACTIVE,\n> + MIGRATION_STATUS_FAILED);\n> + }\n> + multifd_recv_state->quit = true;\n> +\n> for (i = 0; i < multifd_recv_state->count; i++) {\n> MultiFDRecvParams *p = &multifd_recv_state->params[i];\n> \n> @@ -503,6 +566,7 @@ int multifd_load_cleanup(Error **errp)\n> qemu_thread_join(&p->thread);\n> qemu_mutex_destroy(&p->mutex);\n> qemu_sem_destroy(&p->sem);\n> + socket_recv_channel_destroy(p->c);\n> g_free(p->name);\n> p->name = NULL;\n> }\n> @@ -531,10 +595,56 @@ static void *multifd_recv_thread(void *opaque)\n> return NULL;\n> }\n> \n> +void multifd_new_channel(QIOChannel *ioc)\n> +{\n> + MultiFDRecvParams *p;\n> + char string[MULTIFD_UUID_MSG];\n> + char string_uuid[UUID_FMT_LEN];\n> + Error *local_err = NULL;\n> + char *uuid;\n> + size_t ret;\n> + int id;\n> +\n> + ret = qio_channel_read_all(ioc, string, sizeof(string), &local_err);\n> + if (ret != 0) {\n> + terminate_multifd_recv_threads(local_err);\n> + return;\n> + }\n\nNow here initially read sizeof(struct MigrateMultiFDInit).\n\nIf we need to add extra fields in a v2 struct for example\n\n struct MigrateMultiFDInitV2 {\n struct MigrateMultiFDInit v1;\n uint32_t extra;\n };\n\nThis side now just has to check of version == 2, and then it can\nread (sizeof(MigrateMultiFDInitV2) - sizeof(MigrateMultiFDInit))\nto get the extra bytes for the new fields from v2.\n\nThis is vastly better than non-extensible printf()+scanf()\n\n\n> + sscanf(string, \"%s multifd %03d\", string_uuid, &id);\n> +\n> + if (qemu_uuid_set) {\n> + uuid = qemu_uuid_unparse_strdup(&qemu_uuid);\n> + } else {\n> + uuid = g_strdup(multifd_uuid);\n> + }\n> + if (strcmp(string_uuid, uuid)) {\n> + error_setg(&local_err, \"multifd: received uuid '%s' and expected \"\n> + \"uuid '%s' for channel %d\", string_uuid, uuid, id);\n> + terminate_multifd_recv_threads(local_err);\n> + return;\n> + }\n> + g_free(uuid);\n\nRegards,\nDaniel","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=berrange@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xsfkj5kZSz9s9Y\n\tfor ;\n\tWed, 13 Sep 2017 21:36:28 +1000 (AEST)","from localhost ([::1]:41703 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1ds5xx-00074B-PP\n\tfor incoming@patchwork.ozlabs.org; Wed, 13 Sep 2017 07:36:25 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:34238)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1ds5xM-00072K-Ao\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:35:50 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1ds5xI-00019Q-S4\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:35:48 -0400","from mx1.redhat.com ([209.132.183.28]:42340)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1ds5xI-00018p-H5\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:35:44 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 5C2B8C04B945\n\tfor ; Wed, 13 Sep 2017 11:35:43 +0000 (UTC)","from redhat.com (unknown [10.33.36.24])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id DFE9F18AC4;\n\tWed, 13 Sep 2017 11:35:36 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 5C2B8C04B945","Date":"Wed, 13 Sep 2017 12:35:34 +0100","From":"\"Daniel P. Berrange\" ","To":"Juan Quintela ","Message-ID":"<20170913113534.GC3067@redhat.com>","References":"<20170913105953.13760-1-quintela@redhat.com>\n\t<20170913105953.13760-13-quintela@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20170913105953.13760-13-quintela@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tWed, 13 Sep 2017 11:35:43 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v8 12/20] migration: Start of multiple fd\n\twork","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"\"Daniel P. Berrange\" ","Cc":"lvivier@redhat.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\tdgilbert@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1767807,"web_url":"http://patchwork.ozlabs.org/comment/1767807/","msgid":"<20170913114410.GE3067@redhat.com>","list_archive_url":null,"date":"2017-09-13T11:44:10","subject":"Re: [Qemu-devel] [PATCH v8 12/20] migration: Start of multiple fd\n\twork","submitter":{"id":2694,"url":"http://patchwork.ozlabs.org/api/people/2694/","name":"Daniel P. Berrangé","email":"berrange@redhat.com"},"content":"On Wed, Sep 13, 2017 at 12:59:45PM +0200, Juan Quintela wrote:\n> We create new channels for each new thread created. We send through\n> them a string containing multifd so we are\n> sure that we connect the right channels in both sides.\n> \n> Signed-off-by: Juan Quintela \n> \n> --\n> Split SocketArgs into incoming and outgoing args\n> \n> Use UUID's on the initial message, so we are sure we are connecting to\n> the right channel.\n> \n> Remove init semaphore. Now that we use uuids on the init message, we\n> know that this is our channel.\n> \n> Fix recv socket destwroy, we were destroying send channels.\n> This was very interesting, because we were using an unreferred object\n> without problems.\n> \n> Move to struct of pointers\n> init channel sooner.\n> split recv thread creation.\n> listen on main thread\n> We count the number of created threads to know when we need to stop listening\n> Use g_strdup_printf\n> report channel id on errors\n> Add name parameter\n> Use local_err\n> Add Error * parameter to socket_send_channel_create()\n> Use qio_channel_*_all\n> Use asynchronous connect\n> ---\n> migration/migration.c | 5 ++\n> migration/ram.c | 138 +++++++++++++++++++++++++++++++++++++++++++-------\n> migration/ram.h | 3 ++\n> migration/socket.c | 34 ++++++++++++-\n> migration/socket.h | 10 ++++\n> 5 files changed, 172 insertions(+), 18 deletions(-)\n> \n\n> @@ -531,10 +595,56 @@ static void *multifd_recv_thread(void *opaque)\n> return NULL;\n> }\n> \n> +void multifd_new_channel(QIOChannel *ioc)\n> +{\n> + MultiFDRecvParams *p;\n> + char string[MULTIFD_UUID_MSG];\n> + char string_uuid[UUID_FMT_LEN];\n> + Error *local_err = NULL;\n> + char *uuid;\n> + size_t ret;\n> + int id;\n> +\n> + ret = qio_channel_read_all(ioc, string, sizeof(string), &local_err);\n> + if (ret != 0) {\n> + terminate_multifd_recv_threads(local_err);\n> + return;\n> + }\n> + sscanf(string, \"%s multifd %03d\", string_uuid, &id);\n> +\n> + if (qemu_uuid_set) {\n> + uuid = qemu_uuid_unparse_strdup(&qemu_uuid);\n> + } else {\n> + uuid = g_strdup(multifd_uuid);\n> + }\n> + if (strcmp(string_uuid, uuid)) {\n> + error_setg(&local_err, \"multifd: received uuid '%s' and expected \"\n> + \"uuid '%s' for channel %d\", string_uuid, uuid, id);\n> + terminate_multifd_recv_threads(local_err);\n> + return;\n> + }\n> + g_free(uuid);\n> +\n> + p = &multifd_recv_state->params[id];\n> + if (p->id != 0) {\n> + error_setg(&local_err, \"multifd: received id '%d' already setup'\", id);\n> + terminate_multifd_recv_threads(local_err);\n> + return;\n> + }\n> + qemu_mutex_init(&p->mutex);\n> + qemu_sem_init(&p->sem, 0);\n> + p->quit = false;\n> + p->id = id;\n> + p->c = ioc;\n\nYou must acquire a reference on 'ioc', so that the thread you are\nabout to spawn will own a reference on it.\n\nThe function that you later call multifd_new_channel() from\nexpects it to acquire its own reference.\n\n> + multifd_recv_state->count++;\n> + p->name = g_strdup_printf(\"multifdrecv_%d\", id);\n> + qemu_thread_create(&p->thread, p->name, multifd_recv_thread, p,\n> + QEMU_THREAD_JOINABLE);\n> +}\n\nRegards,\nDaniel","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=berrange@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xsfwS5vrgz9sNw\n\tfor ;\n\tWed, 13 Sep 2017 21:44:56 +1000 (AEST)","from localhost ([::1]:41732 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1ds66B-0003uD-0o\n\tfor incoming@patchwork.ozlabs.org; Wed, 13 Sep 2017 07:44:55 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:39365)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1ds65e-0003rM-8p\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:44:23 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1ds65a-0000AN-6k\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:44:22 -0400","from mx1.redhat.com ([209.132.183.28]:49254)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1ds65Z-00009o-Tq\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 07:44:18 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id DA4965F73C\n\tfor ; Wed, 13 Sep 2017 11:44:16 +0000 (UTC)","from redhat.com (unknown [10.33.36.24])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 617D35D6AE;\n\tWed, 13 Sep 2017 11:44:13 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com DA4965F73C","Date":"Wed, 13 Sep 2017 12:44:10 +0100","From":"\"Daniel P. Berrange\" ","To":"Juan Quintela ","Message-ID":"<20170913114410.GE3067@redhat.com>","References":"<20170913105953.13760-1-quintela@redhat.com>\n\t<20170913105953.13760-13-quintela@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20170913105953.13760-13-quintela@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.39]);\n\tWed, 13 Sep 2017 11:44:17 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH v8 12/20] migration: Start of multiple fd\n\twork","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"\"Daniel P. Berrange\" ","Cc":"lvivier@redhat.com, qemu-devel@nongnu.org, peterx@redhat.com,\n\tdgilbert@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]