From patchwork Fri Jul 4 17:41:38 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 367169 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 0AD1814009C for ; Sat, 5 Jul 2014 03:52:40 +1000 (EST) Received: from localhost ([::1]:37447 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X37f4-0002w9-9X for incoming@patchwork.ozlabs.org; Fri, 04 Jul 2014 13:52:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32794) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X37Ve-0007ty-I1 for qemu-devel@nongnu.org; Fri, 04 Jul 2014 13:43:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X37VY-0000Ko-Al for qemu-devel@nongnu.org; Fri, 04 Jul 2014 13:42:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:21813) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X37VY-0000KA-1y for qemu-devel@nongnu.org; Fri, 04 Jul 2014 13:42:48 -0400 Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s64HgjAQ001809 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Jul 2014 13:42:45 -0400 Received: from dgilbert-t530.home.treblig.org (vpn1-7-141.ams2.redhat.com [10.36.7.141]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s64HfvTu030576; Fri, 4 Jul 2014 13:42:43 -0400 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Fri, 4 Jul 2014 18:41:38 +0100 Message-Id: <1404495717-4239-28-git-send-email-dgilbert@redhat.com> In-Reply-To: <1404495717-4239-1-git-send-email-dgilbert@redhat.com> References: <1404495717-4239-1-git-send-email-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, lilei@linux.vnet.ibm.com, quintela@redhat.com Subject: [Qemu-devel] [PATCH 27/46] postcopy: Add incoming_init/cleanup functions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Dr. David Alan Gilbert" Provide functions to be called before the start of a postcopy enabled migration (even if it's not eventually used) and at the end. During the init we must disable huge pages in the RAM that we will receive postcopy data into, since if they start off as hugepage and get a 4k page written to them, the rest of the hugepage won't get userfault'd and won't work as a destination for remap. Signed-off-by: Dr. David Alan Gilbert --- include/migration/postcopy-ram.h | 12 +++++ postcopy-ram.c | 100 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) diff --git a/include/migration/postcopy-ram.h b/include/migration/postcopy-ram.h index 75ca0fd..b5f0cb5 100644 --- a/include/migration/postcopy-ram.h +++ b/include/migration/postcopy-ram.h @@ -30,6 +30,18 @@ int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, /* + * Initialise postcopy-ram, setting the RAM to a state where we can go into + * postcopy later; must be called prior to any precopy. + * called from arch_init's similarly named ram_postcopy_incoming_init + */ +int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_pages); + +/* + * At the end of a migration where postcopy_ram_incoming_init was called. + */ +int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis); + +/* * Called back from arch_init's ram_postcopy_each_ram_discard to handle * discarding one RAMBlock's pre-postcopy dirty pages */ diff --git a/postcopy-ram.c b/postcopy-ram.c index f92f516..2159c60 100644 --- a/postcopy-ram.c +++ b/postcopy-ram.c @@ -250,6 +250,93 @@ int postcopy_ram_discard_range(MigrationIncomingState *mis, uint8_t *start, return 0; } +/* + * Setup an area of RAM so that it *can* be used for postcopy later; this + * must be done right at the start prior to pre-copy. + * opaque should be the MIS. + */ +static int init_area(const char *block_name, void *host_addr, + ram_addr_t offset, ram_addr_t length, void *opaque) +{ + MigrationIncomingState *mis = opaque; + + DPRINTF("init_area: %s: %p offset=%zx length=%zd(%zx)", + block_name, host_addr, offset, length, length); + /* + * We need the whole of RAM to be truly empty for postcopy, so things + * like ROMs and any data tables built during init must be zero'd + * - we're going to get the copy from the source anyway. + */ + if (postcopy_ram_discard_range(mis, host_addr, (host_addr + length - 1))) { + return -1; + } + + /* + * We also need the area to be normal 4k pages, not huge pages + * (otherwise we can't be sure we can use remap_anon_pages to put + * a 4k page in later). THP might come along and map a 2MB page + * and when it's partially accessed in precopy it might not break + * it down, but leave a 2MB zero'd page. + */ + if (madvise(host_addr, length, MADV_NOHUGEPAGE)) { + perror("init_area: NOHUGEPAGE"); + return -1; + } + + return 0; +} + +/* + * At the end of migration, undo the effects of init_area + * opaque should be the MIS. + */ +static int cleanup_area(const char *block_name, void *host_addr, + ram_addr_t offset, ram_addr_t length, void *opaque) +{ + /* Turn off userfault here as well? */ + + DPRINTF("cleanup_area: %s: %p offset=%zx length=%zd(%zx)", + block_name, host_addr, offset, length, length); + /* + * We turned off hugepage for the precopy stage with postcopy enabled + * we can turn it back on now. + */ + if (madvise(host_addr, length, MADV_HUGEPAGE)) { + perror("init_area: HUGEPAGE"); + return -1; + } + + return 0; +} + +/* + * Initialise postcopy-ram, setting the RAM to a state where we can go into + * postcopy later; must be called prior to any precopy. + * called from arch_init's similarly named ram_postcopy_incoming_init + */ +int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_pages) +{ + postcopy_pmi_init(mis, ram_pages); + + if (qemu_ram_foreach_block(init_area, mis)) { + return -1; + } + + return 0; +} + +/* + * At the end of a migration where postcopy_ram_incoming_init was called. + */ +int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) +{ + if (qemu_ram_foreach_block(cleanup_area, mis)) { + return -1; + } + + return 0; +} + #else /* No target OS support, stubs just fail */ @@ -265,6 +352,19 @@ int postcopy_ram_discard_range(MigrationIncomingState *mis, void *start, error_report("postcopy_ram_discard_range: No OS support"); return -1; } + +int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_pages) +{ + error_report("postcopy_ram_incoming_init: No OS support\n"); + return -1; +} + +int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) +{ + error_report("postcopy_ram_incoming_cleanup: No OS support\n"); + return -1; +} + #endif /* ------------------------------------------------------------------------- */