{"id":810555,"url":"http://patchwork.ozlabs.org/api/patches/810555/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170906115143.27451-22-quintela@redhat.com/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20170906115143.27451-22-quintela@redhat.com>","list_archive_url":null,"date":"2017-09-06T11:51:42","name":"[v7,21/22] migration: Transfer pages over new channels","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"1c4533e9ece6a4ac4aeffb2c6e2807985100592d","submitter":{"id":2643,"url":"http://patchwork.ozlabs.org/api/people/2643/?format=json","name":"Juan Quintela","email":"quintela@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170906115143.27451-22-quintela@redhat.com/mbox/","series":[{"id":1773,"url":"http://patchwork.ozlabs.org/api/series/1773/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=1773","date":"2017-09-06T11:51:21","name":"Multifd","version":7,"mbox":"http://patchwork.ozlabs.org/series/1773/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/810555/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/810555/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=quintela@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xnMnZ2FRvz9sBZ\n\tfor ;\n\tWed, 6 Sep 2017 22:09:06 +1000 (AEST)","from localhost ([::1]:35797 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dpZ8i-0003vr-EK\n\tfor incoming@patchwork.ozlabs.org; Wed, 06 Sep 2017 08:09:04 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:60564)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dpYss-0007qz-88\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 07:52:43 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dpYsr-0008G2-6y\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 07:52:42 -0400","from mx1.redhat.com ([209.132.183.28]:9839)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from ) id 1dpYsq-0008Fb-Up\n\tfor qemu-devel@nongnu.org; Wed, 06 Sep 2017 07:52:41 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 01177C049D7F\n\tfor ; Wed, 6 Sep 2017 11:52:40 +0000 (UTC)","from secure.mitica (ovpn-117-188.ams2.redhat.com [10.36.117.188])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id A030A19C94;\n\tWed, 6 Sep 2017 11:52:38 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 01177C049D7F","From":"Juan Quintela ","To":"qemu-devel@nongnu.org","Date":"Wed, 6 Sep 2017 13:51:42 +0200","Message-Id":"<20170906115143.27451-22-quintela@redhat.com>","In-Reply-To":"<20170906115143.27451-1-quintela@redhat.com>","References":"<20170906115143.27451-1-quintela@redhat.com>","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tWed, 06 Sep 2017 11:52:40 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"[Qemu-devel] [PATCH v7 21/22] migration: Transfer pages over new\n\tchannels","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"lvivier@redhat.com, dgilbert@redhat.com, peterx@redhat.com","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"},"content":"We switch for sending the page number to send real pages.\n\nSigned-off-by: Juan Quintela \n\n--\n\nRemove the HACK bit, now we have the function that calculates the size\nof a page exported.\n---\n migration/migration.c | 7 ++++++-\n migration/ram.c | 38 +++++++++++---------------------------\n 2 files changed, 17 insertions(+), 28 deletions(-)","diff":"diff --git a/migration/migration.c b/migration/migration.c\nindex 62c410aac3..1f6efc3207 100644\n--- a/migration/migration.c\n+++ b/migration/migration.c\n@@ -2071,6 +2071,7 @@ static void *migration_thread(void *opaque)\n */\n int64_t threshold_size = 0;\n int64_t qemu_file_bytes = 0;\n+ int64_t multifd_pages = 0;\n int64_t start_time = initial_time;\n int64_t end_time;\n bool old_vm_running = false;\n@@ -2159,8 +2160,11 @@ static void *migration_thread(void *opaque)\n current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);\n if (current_time >= initial_time + BUFFER_DELAY) {\n uint64_t qemu_file_bytes_now = qemu_ftell(s->to_dst_file);\n+ uint64_t multifd_pages_now = ram_counters.multifd;\n uint64_t transferred_bytes =\n- qemu_file_bytes_now - qemu_file_bytes;\n+ (qemu_file_bytes_now - qemu_file_bytes) +\n+ (multifd_pages_now - multifd_pages) *\n+ qemu_target_page_size();\n uint64_t time_spent = current_time - initial_time;\n double bandwidth = (double)transferred_bytes / time_spent;\n threshold_size = bandwidth * s->parameters.downtime_limit;\n@@ -2180,6 +2184,7 @@ static void *migration_thread(void *opaque)\n qemu_file_reset_rate_limit(s->to_dst_file);\n initial_time = current_time;\n qemu_file_bytes = qemu_file_bytes_now;\n+ multifd_pages = multifd_pages_now;\n }\n if (qemu_file_rate_limit(s->to_dst_file)) {\n /* usleep expects microseconds */\ndiff --git a/migration/ram.c b/migration/ram.c\nindex 348ce1141a..0c2782c452 100644\n--- a/migration/ram.c\n+++ b/migration/ram.c\n@@ -498,21 +498,16 @@ static void *multifd_send_thread(void *opaque)\n if (p->pages.num) {\n Error *local_err = NULL;\n size_t ret;\n- int i;\n int num;\n \n num = p->pages.num;\n p->pages.num = 0;\n qemu_mutex_unlock(&p->mutex);\n \n- for (i = 0; i < num; i++) {\n- ret = qio_channel_write(p->c,\n- (const char *)&p->pages.iov[i].iov_base,\n- sizeof(uint8_t *), &local_err);\n- if (ret != sizeof(uint8_t *)) {\n- terminate_multifd_send_threads(local_err);\n- return NULL;\n- }\n+ ret = qio_channel_writev_all(p->c, p->pages.iov, num, &local_err);\n+ if (ret != num * TARGET_PAGE_SIZE) {\n+ terminate_multifd_send_threads(local_err);\n+ return NULL;\n }\n qemu_mutex_lock(&multifd_send_state->mutex);\n p->done = true;\n@@ -685,7 +680,6 @@ int multifd_load_cleanup(Error **errp)\n static void *multifd_recv_thread(void *opaque)\n {\n MultiFDRecvParams *p = opaque;\n- uint8_t *recv_address;\n \n qemu_sem_post(&p->ready);\n while (true) {\n@@ -697,27 +691,16 @@ static void *multifd_recv_thread(void *opaque)\n if (p->pages.num) {\n Error *local_err = NULL;\n size_t ret;\n- int i;\n int num;\n \n num = p->pages.num;\n p->pages.num = 0;\n \n- for (i = 0; i < num; i++) {\n- ret = qio_channel_read(p->c, (char *)&recv_address,\n- sizeof(uint8_t *), &local_err);\n- if (ret != sizeof(uint8_t *)) {\n- terminate_multifd_recv_threads(local_err);\n- return NULL;\n- }\n- if (recv_address != p->pages.iov[i].iov_base) {\n- error_setg(&local_err, \"received %p and expecting %p (%d)\\n\",\n- recv_address, p->pages.iov[i].iov_base, i);\n- terminate_multifd_recv_threads(local_err);\n- return NULL;\n- }\n+ ret = qio_channel_readv_all(p->c, p->pages.iov, num, &local_err);\n+ if (ret != num * TARGET_PAGE_SIZE) {\n+ terminate_multifd_recv_threads(local_err);\n+ return NULL;\n }\n-\n p->done = true;\n qemu_mutex_unlock(&p->mutex);\n qemu_sem_post(&p->ready);\n@@ -1279,8 +1262,10 @@ static int ram_multifd_page(RAMState *rs, PageSearchStatus *pss,\n offset | RAM_SAVE_FLAG_MULTIFD_PAGE);\n fd_num = multifd_send_page(p, rs->migration_dirty_pages == 1);\n qemu_put_be16(rs->f, fd_num);\n+ if (fd_num != MULTIFD_CONTINUE) {\n+ qemu_fflush(rs->f);\n+ }\n ram_counters.transferred += 2; /* size of fd_num */\n- qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);\n ram_counters.transferred += TARGET_PAGE_SIZE;\n pages = 1;\n ram_counters.normal++;\n@@ -3143,7 +3128,6 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)\n case RAM_SAVE_FLAG_MULTIFD_PAGE:\n fd_num = qemu_get_be16(f);\n multifd_recv_page(host, fd_num);\n- qemu_get_buffer(f, host, TARGET_PAGE_SIZE);\n break;\n \n case RAM_SAVE_FLAG_EOS:\n","prefixes":["v7","21/22"]}