From patchwork Wed Dec 19 12:33:23 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Juan Quintela X-Patchwork-Id: 207329 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 1BC812C008E for ; Wed, 19 Dec 2012 23:34:28 +1100 (EST) Received: from localhost ([::1]:58004 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TlIqv-0003f5-OQ for incoming@patchwork.ozlabs.org; Wed, 19 Dec 2012 07:34:25 -0500 Received: from eggs.gnu.org ([208.118.235.92]:44038) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TlIqb-0003dS-Lq for qemu-devel@nongnu.org; Wed, 19 Dec 2012 07:34:10 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TlIqX-0004A5-E2 for qemu-devel@nongnu.org; Wed, 19 Dec 2012 07:34:05 -0500 Received: from mx1.redhat.com ([209.132.183.28]:1581) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TlIqX-00049w-6D for qemu-devel@nongnu.org; Wed, 19 Dec 2012 07:34:01 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qBJCXxWB030056 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 19 Dec 2012 07:34:00 -0500 Received: from trasno.mitica (ovpn-113-59.phx2.redhat.com [10.3.113.59]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qBJCXwuO027690; Wed, 19 Dec 2012 07:33:58 -0500 From: Juan Quintela To: qemu-devel@nongnu.org Date: Wed, 19 Dec 2012 13:33:23 +0100 Message-Id: <1355920437-29882-1-git-send-email-quintela@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 00/34] migration thread and queue X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Hi This is my queue for migration-thread and patches associated. This integrates review comments & code for Paolo. This is the subset from both approachs that we agreed with. rest of patches need more review and are not here. Migrating and idle guest with upstwream: (qemu) info migrate capabilities: xbzrle: off Migration status: completed total time: 34251 milliseconds downtime: 492 milliseconds transferred ram: 762458 kbytes remaining ram: 0 kbytes total ram: 14688768 kbytes duplicate: 3492606 pages normal: 189762 pages normal bytes: 759048 kbytes with this series of patches. (qemu) info migrate capabilities: xbzrle: off Migration status: completed total time: 30712 milliseconds downtime: 29 milliseconds transferred ram: 738857 kbytes remaining ram: 0 kbytes total ram: 14688768 kbytes duplicate: 3503423 pages normal: 176671 pages normal bytes: 706684 kbytes Notice the big difference in downtime. And that is also seen inside the guest a program that just do an idle loop seeing how "long" it takes to wait for 10ms. with upstream: [root@d1 ~]# ./timer delay of 452 ms delay of 114 ms delay of 136 ms delay of 135 ms delay of 136 ms delay of 131 ms delay of 134 ms with this series of patches, wait never takes 100ms, nothing is printed. Please review. Thanks, Juan. The following changes since commit 914606d26e654d4c01bd5186f4d05e3fd445e219: Merge remote-tracking branch 'stefanha/trivial-patches' into staging (2012-12-18 15:41:43 -0600) are available in the git repository at: git://repo.or.cz/qemu/quintela.git thread.next for you to fetch changes up to 065bebbacf54dcad48aad42112417dbb44451499: migration: merge QEMUFileBuffered into MigrationState (2012-12-19 13:21:29 +0100) ---------------------------------------------------------------- Juan Quintela (25): migration: include qemu-file.h migration-fd: remove duplicate include buffered_file: Move from using a timer to use a thread migration: make qemu_fopen_ops_buffered() return void migration: move migration thread init code to migrate_fd_put_ready migration: make writes blocking migration: remove unfreeze logic migration: just lock migrate_fd_put_ready buffered_file: Unfold the trick to restart generating migration data buffered_file: don't flush on put buffer buffered_file: unfold buffered_append in buffered_put_buffer savevm: New save live migration method: pending migration: move buffered_file.c code into migration.c migration: add XFER_LIMIT_RATIO migration: move migration_fd_put_ready() migration: Inline qemu_fopen_ops_buffered into migrate_fd_connect migration: move migration notifier ram: rename last_block to last_seen_block ram: Add last_sent_block memory: introduce memory_region_test_and_clear_dirty ram: Use memory_region_test_and_clear_dirty ram: optimize migration bitmap walking ram: account the amount of transferred ram better ram: refactor ram_save_block() return value migration: merge QEMUFileBuffered into MigrationState Paolo Bonzini (7): migration: fix migration_bitmap leak buffered_file: do not send more than s->bytes_xfer bytes per tick migration: remove double call to migrate_fd_close exec: change ramlist from MRU order to a 1-item cache exec: change RAM list to a TAILQ exec: sort the memory from biggest to smallest migration: fix qemu_get_fd for BufferedFile Umesh Deshpande (2): add a version number to ram_list protect the ramlist with a separate mutex Makefile.objs | 2 +- arch_init.c | 245 +++++++++++++++--------------- block-migration.c | 49 ++---- buffered_file.c | 269 --------------------------------- buffered_file.h | 22 --- cpu-all.h | 15 +- dump.c | 8 +- exec.c | 128 +++++++++++----- memory.c | 16 ++ memory.h | 16 ++ memory_mapping.c | 4 +- migration-exec.c | 3 +- migration-fd.c | 4 +- migration-tcp.c | 3 +- migration-unix.c | 3 +- migration.c | 390 ++++++++++++++++++++++++++++++++++++------------ migration.h | 13 +- qemu-file.h | 5 - savevm.c | 24 ++- sysemu.h | 1 + target-i386/arch_dump.c | 2 +- vmstate.h | 1 + 22 files changed, 599 insertions(+), 624 deletions(-) delete mode 100644 buffered_file.c delete mode 100644 buffered_file.h Acked-by: Paolo Bonzini