From patchwork Wed Feb 25 16:52:04 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Dr. David Alan Gilbert" X-Patchwork-Id: 443561 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 6A0BE140083 for ; Thu, 26 Feb 2015 04:25:43 +1100 (AEDT) Received: from localhost ([::1]:55440 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YQfiP-0006k3-Gv for incoming@patchwork.ozlabs.org; Wed, 25 Feb 2015 12:25:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35111) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YQfDl-0002Kx-Ur for qemu-devel@nongnu.org; Wed, 25 Feb 2015 11:54:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YQfDe-0001Tf-Qg for qemu-devel@nongnu.org; Wed, 25 Feb 2015 11:54:01 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55274) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YQfDe-0001TP-K8 for qemu-devel@nongnu.org; Wed, 25 Feb 2015 11:53:54 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1PGrngl009927 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 25 Feb 2015 11:53:49 -0500 Received: from dgilbert-t530.redhat.com (ovpn-116-97.ams2.redhat.com [10.36.116.97]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1PGq8lY032146; Wed, 25 Feb 2015 11:53:47 -0500 From: "Dr. David Alan Gilbert (git)" To: qemu-devel@nongnu.org Date: Wed, 25 Feb 2015 16:52:04 +0000 Message-Id: <1424883128-9841-42-git-send-email-dgilbert@redhat.com> In-Reply-To: <1424883128-9841-1-git-send-email-dgilbert@redhat.com> References: <1424883128-9841-1-git-send-email-dgilbert@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: aarcange@redhat.com, yamahata@private.email.ne.jp, quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, yanghy@cn.fujitsu.com, david@gibson.dropbear.id.au Subject: [Qemu-devel] [PATCH v5 41/45] Start up a postcopy/listener thread ready for incoming page data X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: "Dr. David Alan Gilbert" The loading of a device state (during postcopy) may access guest memory that's still on the source machine and thus might need a page fill; split off a separate thread that handles the incoming page data so that the original incoming migration code can finish off the device data. Signed-off-by: Dr. David Alan Gilbert --- include/migration/migration.h | 4 +++ migration/migration.c | 6 ++++ savevm.c | 71 +++++++++++++++++++++++++++++++++++++++++-- trace-events | 2 ++ 4 files changed, 81 insertions(+), 2 deletions(-) diff --git a/include/migration/migration.h b/include/migration/migration.h index cec064f..c2af2ef 100644 --- a/include/migration/migration.h +++ b/include/migration/migration.h @@ -90,6 +90,10 @@ struct MigrationIncomingState { QemuThread fault_thread; QemuSemaphore fault_thread_sem; + bool have_listen_thread; + QemuThread listen_thread; + QemuSemaphore listen_thread_sem; + /* For the kernel to send us notifications */ int userfault_fd; /* To tell the fault_thread to quit */ diff --git a/migration/migration.c b/migration/migration.c index 939f426..c108851 100644 --- a/migration/migration.c +++ b/migration/migration.c @@ -1067,6 +1067,12 @@ static int postcopy_start(MigrationState *ms, bool *old_vm_running) goto fail; } + /* + * Make sure the receiver can get incoming pages before we send the rest + * of the state + */ + qemu_savevm_send_postcopy_listen(fb); + qemu_savevm_state_complete(fb); qemu_savevm_send_ping(fb, 3); diff --git a/savevm.c b/savevm.c index 014ba08..eb22410 100644 --- a/savevm.c +++ b/savevm.c @@ -1307,6 +1307,51 @@ static int loadvm_postcopy_ram_handle_discard(MigrationIncomingState *mis, return 0; } +/* + * Triggered by a postcopy_listen command; this thread takes over reading + * the input stream, leaving the main thread free to carry on loading the rest + * of the device state (from RAM). + * (TODO:This could do with being in a postcopy file - but there again it's + * just another input loop, not that postcopy specific) + */ +static void *postcopy_ram_listen_thread(void *opaque) +{ + QEMUFile *f = opaque; + MigrationIncomingState *mis = migration_incoming_get_current(); + int load_res; + + qemu_sem_post(&mis->listen_thread_sem); + trace_postcopy_ram_listen_thread_start(); + + load_res = qemu_loadvm_state_main(f, mis); + + trace_postcopy_ram_listen_thread_exit(); + if (load_res < 0) { + error_report("%s: loadvm failed: %d", __func__, load_res); + qemu_file_set_error(f, load_res); + } + postcopy_ram_incoming_cleanup(mis); + /* + * If everything has worked fine, then the main thread has waited + * for us to start, and we're the last use of the mis. + * (If something broke then qemu will have to exit anyway since it's + * got a bad migration state). + */ + migration_incoming_state_destroy(); + + if (load_res < 0) { + /* + * If something went wrong then we have a bad state so exit; + * depending how far we got it might be possible at this point + * to leave the guest running and fire MCEs for pages that never + * arrived as a desperate recovery step. + */ + exit(EXIT_FAILURE); + } + + return NULL; +} + /* After this message we must be able to immediately receive postcopy data */ static int loadvm_postcopy_handle_listen(MigrationIncomingState *mis) { @@ -1326,8 +1371,25 @@ static int loadvm_postcopy_handle_listen(MigrationIncomingState *mis) return -1; } - /* TODO start up the postcopy listening thread */ - return 0; + if (mis->have_listen_thread) { + error_report("CMD_POSTCOPY_RAM_LISTEN already has a listen thread"); + return -1; + } + + mis->have_listen_thread = true; + /* Start up the listening thread and wait for it to signal ready */ + qemu_sem_init(&mis->listen_thread_sem, 0); + qemu_thread_create(&mis->listen_thread, "postcopy/listen", + postcopy_ram_listen_thread, mis->file, + QEMU_THREAD_JOINABLE); + qemu_sem_wait(&mis->listen_thread_sem); + qemu_sem_destroy(&mis->listen_thread_sem); + + /* + * all good - cause the loop that handled this command to exit because + * the new thread is taking over + */ + return LOADVM_QUIT_PARENT; } /* After all discards we can start running and asking for pages */ @@ -1670,6 +1732,11 @@ int qemu_loadvm_state(QEMUFile *f) ret = qemu_loadvm_state_main(f, mis); trace_qemu_loadvm_state_post_main(ret); + if (mis->have_listen_thread) { + /* Listen thread still going, can't clean up yet */ + return ret; + } + if (ret == 0) { cpu_synchronize_all_post_init(); ret = qemu_file_get_error(f); diff --git a/trace-events b/trace-events index d955a28..82b3631 100644 --- a/trace-events +++ b/trace-events @@ -1183,6 +1183,8 @@ loadvm_postcopy_ram_handle_discard(void) "" loadvm_postcopy_ram_handle_discard_end(void) "" loadvm_process_command(uint16_t com, uint16_t len) "com=0x%x len=%d" loadvm_process_command_ping(uint32_t val) "%x" +postcopy_ram_listen_thread_exit(void) "" +postcopy_ram_listen_thread_start(void) "" qemu_savevm_send_postcopy_advise(void) "" qemu_savevm_send_postcopy_ram_discard(void) "" qemu_savevm_state_complete_skip_for_postcopy(const char *section) "skipping: %s"