From patchwork Wed Apr 17 04:20:16 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mrhines@linux.vnet.ibm.com X-Patchwork-Id: 237139 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 7A0D92C014A for ; Wed, 17 Apr 2013 14:21:27 +1000 (EST) Received: from localhost ([::1]:51217 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1USJs5-0000bf-Aw for incoming@patchwork.ozlabs.org; Wed, 17 Apr 2013 00:21:25 -0400 Received: from eggs.gnu.org ([208.118.235.92]:43750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1USJrX-0000bN-Uf for qemu-devel@nongnu.org; Wed, 17 Apr 2013 00:21:01 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1USJrU-0001vt-1X for qemu-devel@nongnu.org; Wed, 17 Apr 2013 00:20:51 -0400 Received: from e7.ny.us.ibm.com ([32.97.182.137]:50189) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1USJrT-0001vW-NB for qemu-devel@nongnu.org; Wed, 17 Apr 2013 00:20:47 -0400 Received: from /spool/local by e7.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 17 Apr 2013 00:20:47 -0400 Received: from d01dlp01.pok.ibm.com (9.56.250.166) by e7.ny.us.ibm.com (192.168.1.107) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 17 Apr 2013 00:20:46 -0400 Received: from d01relay06.pok.ibm.com (d01relay06.pok.ibm.com [9.56.227.116]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 9088438C801A for ; Wed, 17 Apr 2013 00:20:45 -0400 (EDT) Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by d01relay06.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id r3H4KjTx32571498 for ; Wed, 17 Apr 2013 00:20:45 -0400 Received: from d01av01.pok.ibm.com (loopback [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id r3H4KjLB003581 for ; Wed, 17 Apr 2013 00:20:45 -0400 Received: from mrhinesdev.klabtestbed.com (klinux.watson.ibm.com [9.2.208.21]) by d01av01.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id r3H4Kh3U003460; Wed, 17 Apr 2013 00:20:44 -0400 From: mrhines@linux.vnet.ibm.com To: qemu-devel@nongnu.org Date: Wed, 17 Apr 2013 00:20:16 -0400 Message-Id: <1366172418-8729-6-git-send-email-mrhines@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1366172418-8729-1-git-send-email-mrhines@linux.vnet.ibm.com> References: <1366172418-8729-1-git-send-email-mrhines@linux.vnet.ibm.com> X-TM-AS-MML: No X-Content-Scanned: Fidelis XPS MAILER x-cbid: 13041704-5806-0000-0000-000020C241EC X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.4.x-2.6.x [generic] X-Received-From: 32.97.182.137 Cc: aliguori@us.ibm.com, mst@redhat.com, quintela@redhat.com, owasserm@redhat.com, abali@us.ibm.com, mrhines@us.ibm.com, gokul@us.ibm.com, pbonzini@redhat.com Subject: [Qemu-devel] [PULL v3 5/7] rdma: send pc.ram X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Michael R. Hines" This takes advantages of the previous patches: 1. use the new QEMUFileOps hook 'save_page' 2. call out to the right accessor methods to invoke the iteration hooks defined in QEMUFileOps 3. Expose a common way to zap zero pages Signed-off-by: Michael R. Hines --- arch_init.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 49 insertions(+), 10 deletions(-) diff --git a/arch_init.c b/arch_init.c index 92de1bd..2628ad3 100644 --- a/arch_init.c +++ b/arch_init.c @@ -115,6 +115,7 @@ const uint32_t arch_type = QEMU_ARCH; #define RAM_SAVE_FLAG_EOS 0x10 #define RAM_SAVE_FLAG_CONTINUE 0x20 #define RAM_SAVE_FLAG_XBZRLE 0x40 +/* 0x80 is reserved in migration.h start with 0x100 next */ static struct defconfig_file { @@ -454,8 +455,15 @@ static int ram_save_block(QEMUFile *f, bool last_stage) p = memory_region_get_ram_ptr(mr) + offset; /* In doubt sent page as normal */ - bytes_sent = -1; - if (is_zero_page(p)) { + bytes_sent = ram_control_save_page(f, block->offset, + offset, TARGET_PAGE_SIZE); + if (bytes_sent != -1) { + if (bytes_sent > 0) { + acct_info.norm_pages++; + } else if(bytes_sent == 0) { + acct_info.dup_pages++; + } + } else if (is_zero_page(p)) { acct_info.dup_pages++; if (!ram_bulk_stage) { bytes_sent = save_block_hdr(f, block, offset, cont, @@ -598,6 +606,15 @@ static int ram_save_setup(QEMUFile *f, void *opaque) } qemu_mutex_unlock_ramlist(); + + /* + * Please leave in place. These calls generate reserved messages in + * the RDMA protocol in order to pre-register RDMA memory in the + * future to before the bulk round begins. + */ + ram_control_before_iterate(f, RAM_CONTROL_SETUP); + ram_control_after_iterate(f, RAM_CONTROL_SETUP); + qemu_put_be64(f, RAM_SAVE_FLAG_EOS); return 0; @@ -616,6 +633,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) reset_ram_globals(); } + ram_control_before_iterate(f, RAM_CONTROL_ROUND); + t0 = qemu_get_clock_ns(rt_clock); i = 0; while ((ret = qemu_file_rate_limit(f)) == 0) { @@ -646,6 +665,12 @@ static int ram_save_iterate(QEMUFile *f, void *opaque) qemu_mutex_unlock_ramlist(); + /* + * Must occur before EOS (or any QEMUFile operation) + * because of RDMA protocol. + */ + ram_control_after_iterate(f, RAM_CONTROL_ROUND); + if (ret < 0) { bytes_transferred += total_sent; return ret; @@ -663,6 +688,8 @@ static int ram_save_complete(QEMUFile *f, void *opaque) qemu_mutex_lock_ramlist(); migration_bitmap_sync(); + ram_control_before_iterate(f, RAM_CONTROL_FINISH); + /* try transferring iterative blocks of memory */ /* flush all remaining blocks regardless of rate limiting */ @@ -676,6 +703,8 @@ static int ram_save_complete(QEMUFile *f, void *opaque) } bytes_transferred += bytes_sent; } + + ram_control_after_iterate(f, RAM_CONTROL_FINISH); migration_end(); qemu_mutex_unlock_ramlist(); @@ -770,6 +799,21 @@ static inline void *host_from_stream_offset(QEMUFile *f, return NULL; } +/* + * If a page (or a whole RDMA chunk) has been + * determined to be zero, then zap it. + */ +void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) +{ + memset(host, ch, TARGET_PAGE_SIZE); +#ifndef _WIN32 + if (ch == 0 && (!kvm_enabled() || kvm_has_sync_mmu()) && + getpagesize() <= TARGET_PAGE_SIZE) { + qemu_madvise(host, size, QEMU_MADV_DONTNEED); + } +#endif +} + static int ram_load(QEMUFile *f, void *opaque, int version_id) { ram_addr_t addr; @@ -837,14 +881,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) } ch = qemu_get_byte(f); - memset(host, ch, TARGET_PAGE_SIZE); -#ifndef _WIN32 - if (ch == 0 && - (!kvm_enabled() || kvm_has_sync_mmu()) && - getpagesize() <= TARGET_PAGE_SIZE) { - qemu_madvise(host, TARGET_PAGE_SIZE, QEMU_MADV_DONTNEED); - } -#endif + ram_handle_compressed(host, ch, TARGET_PAGE_SIZE); } else if (flags & RAM_SAVE_FLAG_PAGE) { void *host; @@ -864,6 +901,8 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) ret = -EINVAL; goto done; } + } else if (flags & RAM_SAVE_FLAG_HOOK) { + ram_control_load_hook(f, flags); } error = qemu_file_get_error(f); if (error) {