From patchwork Mon Oct 3 15:00:48 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony Liguori X-Patchwork-Id: 117454 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [140.186.70.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 9F41DB6F7C for ; Tue, 4 Oct 2011 02:01:12 +1100 (EST) Received: from localhost ([::1]:45633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RAk0z-0004g5-E1 for incoming@patchwork.ozlabs.org; Mon, 03 Oct 2011 11:01:09 -0400 Received: from eggs.gnu.org ([140.186.70.92]:56779) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RAk0r-0004fd-6G for qemu-devel@nongnu.org; Mon, 03 Oct 2011 11:01:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RAk0j-0001lt-69 for qemu-devel@nongnu.org; Mon, 03 Oct 2011 11:01:01 -0400 Received: from mail-iy0-f173.google.com ([209.85.210.173]:57327) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RAk0j-0001li-1U for qemu-devel@nongnu.org; Mon, 03 Oct 2011 11:00:53 -0400 Received: by iagf6 with SMTP id f6so6365818iag.4 for ; Mon, 03 Oct 2011 08:00:52 -0700 (PDT) Received: by 10.42.135.134 with SMTP id p6mr30521ict.136.1317654051925; Mon, 03 Oct 2011 08:00:51 -0700 (PDT) Received: from [192.168.0.103] (cpe-70-123-132-139.austin.res.rr.com. [70.123.132.139]) by mx.google.com with ESMTPS id ee29sm30074306ibb.9.2011.10.03.08.00.49 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 03 Oct 2011 08:00:50 -0700 (PDT) Message-ID: <4E89CE20.6050706@codemonkey.ws> Date: Mon, 03 Oct 2011 10:00:48 -0500 From: Anthony Liguori User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.21) Gecko/20110831 Lightning/1.0b2 Thunderbird/3.1.13 MIME-Version: 1.0 To: "Michael S. Tsirkin" References: <1316443309-23843-1-git-send-email-mdroth@linux.vnet.ibm.com> <4E88C7DB.9090105@linux.vnet.ibm.com> <20111002210802.GC8072@redhat.com> <4E89B0D4.3090203@us.ibm.com> <20111003133802.GD18920@redhat.com> <4E89BDCE.2010502@codemonkey.ws> <20111003144109.GE19689@redhat.com> In-Reply-To: <20111003144109.GE19689@redhat.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 209.85.210.173 Cc: Michael Roth , aliguori@linux.vnet.ibm.com, Anthony Liguori , qemu-devel@nongnu.org, Stefan Berger Subject: Re: [Qemu-devel] [RFC] New Migration Protocol using Visitor Interface X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org On 10/03/2011 09:41 AM, Michael S. Tsirkin wrote: > On Mon, Oct 03, 2011 at 08:51:10AM -0500, Anthony Liguori wrote: >> On 10/03/2011 08:38 AM, Michael S. Tsirkin wrote: >>> On Mon, Oct 03, 2011 at 07:55:48AM -0500, Anthony Liguori wrote: >>>> On 10/02/2011 04:08 PM, Michael S. Tsirkin wrote: >>>>> On Sun, Oct 02, 2011 at 04:21:47PM -0400, Stefan Berger wrote: >>>>>> >>>>>>> 4) Implement the BERVisitor and make this the default migration protocol. >>>>>>> >>>>>>> Most of the work will be in 1), though with the implementation in this series we should be able to do it incrementally. I'm not sure if the best approach is doing the mechanical phase 1 conversion, then doing phase 2 sometime after 4), doing phase 1 + 2 as part of 1), or just doing VMState conversions which gives basically the same capabilities as phase 1 + 2. >>>>>>> >>>>>>> Thoughts? >>>>>> Is anyone working on this? If not I may give it a shot (tomorrow++) >>>>>> for at least some of the primitives... for enabling vNVRAM metadata >>>>>> of course. Indefinite length encoding of constructed data types I >>>>>> suppose won't be used otherwise the visitor interface seems wrong >>>>>> for parsing and skipping of extra data towards the end of a >>>>>> structure if version n wrote the stream and appended some of its >>>>>> version n data and now version m< n is trying to read the struct >>>>>> and needs to skip the version [m+1, n ] data fields ... in that case >>>>>> the de-serialization of the stream should probably be stream-driven >>>>>> rather than structure-driven. >>>>>> >>>>>> Stefan >>>>> >>>>> Yes I've been struggling with that exactly. >>>>> Anthony, any thoughts? >>>> >>>> It just depends on how you write your visitor. If you used >>>> sequences, you'd probably do something like this: >>>> >>>> start_struct -> >>>> check for sequence tag, push starting offset and size onto stack >>>> increment offset to next tag >>>> >>>> type_int (et al) -> >>>> check for explicit type, parse data >>>> increment offset to next tag >>>> >>>> end_struct -> >>>> pop starting offset and size to temp variables >>>> set offset to starting offset + size >>>> >>>> This is roughly how the QMP input marshaller works FWIW. >>>> >>>> Regards, >>>> >>>> Anthony Liguori >>> >>> One thing I worry about is enabling zero copy for >>> large string types (e.g. memory migration). >> >> Memory shouldn't be done through Visitors. It should be handled as a special case. > > OK, that's fine then. > >>> So we need to be able to see a tag for memory page + address, >>> read that from socket directly at the correct virtual address. >>> >>> Probably, we can avoid using visitors for memory, and hope >>> everything else can stand an extra copy since it's small. >>> >>> But then, why do we worry about the size of >>> encoded device state as Anthony seems to do? >> >> There's a significant difference between the cost of something on >> the wire and the cost of doing a memcpy. The cost of the data on >> the wire is directly proportional to downtime. So if we increase >> the size of the device state by a factor of 10, we increase the >> minimum downtime by a factor of 10. >> >> Of course, *if* the size of device state is already negligible with >> respect to the minimum downtime, then it doesn't matter. This is >> easy to quantify though. For a normal migration session today, >> what's the total size of the device state in relation to the >> calculated bandwidth of the minimum downtime? >> >> If it's very small, then we can add names and not worry about it. >> >> Regards, >> >> Anthony Liguori > > Yes, it's easy to quantify. I think the following gives us > the offset before and after, so the difference is the size > we seek, right? Yeah, you'll also want: You'll want to compare the size to max bwidth. BTW, putting this info properly into migration stats would probably be pretty useful. Regards, Anthony Liguori diff --git a/arch_init.c b/arch_init.c index a6c69c7..0d64200 100644 --- a/arch_init.c +++ b/arch_init.c @@ -334,6 +334,10 @@ int ram_save_live(Monitor *mon, QEMUFile *f, int stage, voi expected_time = ram_save_remaining() * TARGET_PAGE_SIZE / bwidth; + if (stage == 2 && expected_time <= migrate_max_downtime()) { + fprintf(stderr, "max bwidth: %lld\n", (long)(expected_time * bwidth)); + } + return (stage == 2) && (expected_time <= migrate_max_downtime()); }