From patchwork Tue Sep 29 08:38:18 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 523876 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 1B1BF1402B0 for ; Wed, 30 Sep 2015 00:24:02 +1000 (AEST) Received: from localhost ([::1]:52670 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Zgvp2-00075C-3m for incoming@patchwork.ozlabs.org; Tue, 29 Sep 2015 10:24:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55432) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZgqSS-0006Rn-TF for qemu-devel@nongnu.org; Tue, 29 Sep 2015 04:40:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZgqSL-0002qj-T0 for qemu-devel@nongnu.org; Tue, 29 Sep 2015 04:40:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36091) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZgqSL-0002qM-Lu for qemu-devel@nongnu.org; Tue, 29 Sep 2015 04:40:13 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 5A58B8E3E9; Tue, 29 Sep 2015 08:40:13 +0000 (UTC) Received: from dgilbert-t530.redhat.com (ovpn-116-104.ams2.redhat.com [10.36.116.104]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t8T8cJZZ015509; Tue, 29 Sep 2015 04:40:11 -0400 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org, quintela@redhat.com, amit.shah@redhat.com Date: Tue, 29 Sep 2015 09:38:18 +0100 Message-Id: <1443515898-3594-55-git-send-email-dgilbert@redhat.com> In-Reply-To: <1443515898-3594-1-git-send-email-dgilbert@redhat.com> References: <1443515898-3594-1-git-send-email-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: aarcange@redhat.com, pbonzini@redhat.com, liang.z.li@intel.com, luis@cs.umu.se, bharata@linux.vnet.ibm.com Subject: [Qemu-devel] [PATCH v8 54/54] Inhibit ballooning during postcopy X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Dr. David Alan Gilbert" Postcopy detects accesses to pages that haven't been transferred yet using userfaultfd, and it causes exceptions on pages that are 'not present'. Ballooning also causes pages to be marked as 'not present' when the guest inflates the balloon. Potentially a balloon could be inflated to discard pages that are currently inflight during postcopy and that may be arriving at about the same time. To avoid this confusion, disable ballooning during postcopy. When disabled we drop balloon requests from the guest. Since ballooning is generally initiated by the host, the management system should avoid initiating any balloon instructions to the guest during migration, although it's not possible to know how long it would take a guest to process a request made prior to the start of migration. Guest initiated ballooning will not know if it's really freed a page of host memory or not. Queueing the requests until after migration would be nice, but is non-trivial, since the set of inflate/deflate requests have to be compared with the state of the page to know what the final outcome is allowed to be. Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Reviewed-by: Amit Shah --- balloon.c | 11 +++++++++++ hw/virtio/virtio-balloon.c | 4 +++- include/sysemu/balloon.h | 2 ++ migration/postcopy-ram.c | 9 +++++++++ 4 files changed, 25 insertions(+), 1 deletion(-) diff --git a/balloon.c b/balloon.c index 5d69e8a..0f45d1b 100644 --- a/balloon.c +++ b/balloon.c @@ -36,6 +36,17 @@ static QEMUBalloonEvent *balloon_event_fn; static QEMUBalloonStatus *balloon_stat_fn; static void *balloon_opaque; +static bool balloon_inhibited; + +bool qemu_balloon_is_inhibited(void) +{ + return balloon_inhibited; +} + +void qemu_balloon_inhibit(bool state) +{ + balloon_inhibited = state; +} static bool have_balloon(Error **errp) { diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c index c419b17..9671635 100644 --- a/hw/virtio/virtio-balloon.c +++ b/hw/virtio/virtio-balloon.c @@ -37,9 +37,11 @@ static void balloon_page(void *addr, int deflate) { #if defined(__linux__) - if (!kvm_enabled() || kvm_has_sync_mmu()) + if (!qemu_balloon_is_inhibited() && (!kvm_enabled() || + kvm_has_sync_mmu())) { qemu_madvise(addr, TARGET_PAGE_SIZE, deflate ? QEMU_MADV_WILLNEED : QEMU_MADV_DONTNEED); + } #endif } diff --git a/include/sysemu/balloon.h b/include/sysemu/balloon.h index 17fe300..3f976b4 100644 --- a/include/sysemu/balloon.h +++ b/include/sysemu/balloon.h @@ -22,5 +22,7 @@ typedef void (QEMUBalloonStatus)(void *opaque, BalloonInfo *info); int qemu_add_balloon_handler(QEMUBalloonEvent *event_func, QEMUBalloonStatus *stat_func, void *opaque); void qemu_remove_balloon_handler(void *opaque); +bool qemu_balloon_is_inhibited(void); +void qemu_balloon_inhibit(bool state); #endif diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index ec649e1..a6fdefe 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -24,6 +24,7 @@ #include "migration/migration.h" #include "migration/postcopy-ram.h" #include "sysemu/sysemu.h" +#include "sysemu/balloon.h" #include "qemu/error-report.h" #include "trace.h" @@ -308,6 +309,8 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) mis->have_fault_thread = false; } + qemu_balloon_inhibit(false); + if (enable_mlock) { if (os_mlock() < 0) { error_report("mlock: %s", strerror(errno)); @@ -533,6 +536,12 @@ int postcopy_ram_enable_notify(MigrationIncomingState *mis) return -1; } + /* + * Ballooning can mark pages as absent while we're postcopying + * that would cause false userfaults. + */ + qemu_balloon_inhibit(true); + trace_postcopy_ram_enable_notify(); return 0;