[{"id":1768691,"web_url":"http://patchwork.ozlabs.org/comment/1768691/","msgid":"<20170914154221.GD7370@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-14T15:42:21","subject":"Re: [Qemu-devel] [PATCH 00/18] block/mirror: Add active-sync\n\tmirroring","submitter":{"id":17227,"url":"http://patchwork.ozlabs.org/api/people/17227/","name":"Stefan Hajnoczi","email":"stefanha@redhat.com"},"content":"On Wed, Sep 13, 2017 at 08:18:52PM +0200, Max Reitz wrote:\n> There may be a couple of things to do on top of this series:\n> - Allow switching between active and passive mode at runtime: This\n> should not be too difficult to implement, the main question is how to\n> expose it to the user.\n> (I seem to recall we wanted some form of block-job-set-option\n> command...?)\n> \n> - Implement an asynchronous active mode: May be detrimental when it\n> comes to convergence, but it might be nice to have anyway. May or may\n> not be complicated to implement.\n\nIdeally the user doesn't have to know about async vs sync. It's an\nimplementation detail.\n\nAsync makes sense during the bulk copy phase (e.g. sync=full) because\nguest read/write latencies are mostly unaffected. Once the entire\ndevice has been copied there are probably still dirty blocks left\nbecause the guest touched them while the mirror job was running. At\nthat point it definitely makes sense to switch to synchronous mirroring\nin order to converge.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx03.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=stefanha@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtN8t0nMYz9sRW\n\tfor ;\n\tFri, 15 Sep 2017 01:43:08 +1000 (AEST)","from localhost ([::1]:48516 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsWIE-0004Xn-E4\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 11:43:06 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54730)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsWHi-0004W2-Ls\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 11:42:35 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsWHh-00010Z-Qz\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 11:42:34 -0400","from mx1.redhat.com ([209.132.183.28]:50590)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsWHd-0000x3-Ew; Thu, 14 Sep 2017 11:42:29 -0400","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 2D7297E427;\n\tThu, 14 Sep 2017 15:42:28 +0000 (UTC)","from localhost (ovpn-117-58.ams2.redhat.com [10.36.117.58])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 13A10183C0;\n\tThu, 14 Sep 2017 15:42:22 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 2D7297E427","Date":"Thu, 14 Sep 2017 16:42:21 +0100","From":"Stefan Hajnoczi ","To":"Max Reitz ","Message-ID":"<20170914154221.GD7370@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170913181910.29688-1-mreitz@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.13","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.27]);\n\tThu, 14 Sep 2017 15:42:28 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH 00/18] block/mirror: Add active-sync\n\tmirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , John Snow ,\n\tFam Zheng , qemu-devel@nongnu.org, qemu-block@nongnu.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769607,"web_url":"http://patchwork.ozlabs.org/comment/1769607/","msgid":"","list_archive_url":null,"date":"2017-09-16T14:02:45","subject":"Re: [Qemu-devel] [PATCH 00/18] block/mirror: Add active-sync\n\tmirroring","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-14 17:42, Stefan Hajnoczi wrote:\n> On Wed, Sep 13, 2017 at 08:18:52PM +0200, Max Reitz wrote:\n>> There may be a couple of things to do on top of this series:\n>> - Allow switching between active and passive mode at runtime: This\n>> should not be too difficult to implement, the main question is how to\n>> expose it to the user.\n>> (I seem to recall we wanted some form of block-job-set-option\n>> command...?)\n>>\n>> - Implement an asynchronous active mode: May be detrimental when it\n>> comes to convergence, but it might be nice to have anyway. May or may\n>> not be complicated to implement.\n> \n> Ideally the user doesn't have to know about async vs sync. It's an\n> implementation detail.\n> \n> Async makes sense during the bulk copy phase (e.g. sync=full) because\n> guest read/write latencies are mostly unaffected. Once the entire\n> device has been copied there are probably still dirty blocks left\n> because the guest touched them while the mirror job was running. At\n> that point it definitely makes sense to switch to synchronous mirroring\n> in order to converge.\n\nMakes sense, but I'm not sure whether it really is just an\nimplementation detail. If you're in the bulk copy phase in active/async\nmode and you have enough write requests with the target being slow\nenough, I suspect you might still not get convergence then (because the\nwrites to the target yield for a long time while ever more write\nrequests pile up) -- so then you'd just shift the dirty tracking from\nthe bitmap to a list of requests in progress.\n\nAnd I think we do want the bulk copy phase to guarantee convergence,\ntoo, usually (when active/foreground/synchronous mode is selected). If\nwe don't, then that's a policy decision and would be up to libvirt, as I\nsee it.\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xvYsD0Gr3z9t2M\n\tfor ;\n\tSun, 17 Sep 2017 00:03:44 +1000 (AEST)","from localhost ([::1]:57419 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtDh8-0005It-6t\n\tfor incoming@patchwork.ozlabs.org; Sat, 16 Sep 2017 10:03:42 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:40852)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtDge-0005Fy-Ku\n\tfor qemu-devel@nongnu.org; Sat, 16 Sep 2017 10:03:13 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtDgd-0000ar-LU\n\tfor qemu-devel@nongnu.org; Sat, 16 Sep 2017 10:03:12 -0400","from mx1.redhat.com ([209.132.183.28]:54222)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtDgY-0000TS-6W; Sat, 16 Sep 2017 10:03:06 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 26B7CC0587C7;\n\tSat, 16 Sep 2017 14:03:05 +0000 (UTC)","from dresden.str.redhat.com (ovpn-204-52.brq.redhat.com\n\t[10.40.204.52])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id BE9215C552;\n\tSat, 16 Sep 2017 14:02:57 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 26B7CC0587C7","To":"Stefan Hajnoczi ","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170914154221.GD7370@stefanha-x1.localdomain>","From":"Max Reitz ","Message-ID":"","Date":"Sat, 16 Sep 2017 16:02:45 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170914154221.GD7370@stefanha-x1.localdomain>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"UIFLOJWKsSLTLNsTKqXVTGf47TMWAsK7a\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tSat, 16 Sep 2017 14:03:05 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH 00/18] block/mirror: Add active-sync\n\tmirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , John Snow ,\n\tFam Zheng , qemu-devel@nongnu.org, qemu-block@nongnu.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770046,"web_url":"http://patchwork.ozlabs.org/comment/1770046/","msgid":"<20170918100226.GD31063@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-18T10:02:26","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 00/18] block/mirror: Add\n\tactive-sync mirroring","submitter":{"id":2747,"url":"http://patchwork.ozlabs.org/api/people/2747/","name":"Stefan Hajnoczi","email":"stefanha@gmail.com"},"content":"On Sat, Sep 16, 2017 at 04:02:45PM +0200, Max Reitz wrote:\n> On 2017-09-14 17:42, Stefan Hajnoczi wrote:\n> > On Wed, Sep 13, 2017 at 08:18:52PM +0200, Max Reitz wrote:\n> >> There may be a couple of things to do on top of this series:\n> >> - Allow switching between active and passive mode at runtime: This\n> >> should not be too difficult to implement, the main question is how to\n> >> expose it to the user.\n> >> (I seem to recall we wanted some form of block-job-set-option\n> >> command...?)\n> >>\n> >> - Implement an asynchronous active mode: May be detrimental when it\n> >> comes to convergence, but it might be nice to have anyway. May or may\n> >> not be complicated to implement.\n> > \n> > Ideally the user doesn't have to know about async vs sync. It's an\n> > implementation detail.\n> > \n> > Async makes sense during the bulk copy phase (e.g. sync=full) because\n> > guest read/write latencies are mostly unaffected. Once the entire\n> > device has been copied there are probably still dirty blocks left\n> > because the guest touched them while the mirror job was running. At\n> > that point it definitely makes sense to switch to synchronous mirroring\n> > in order to converge.\n> \n> Makes sense, but I'm not sure whether it really is just an\n> implementation detail. If you're in the bulk copy phase in active/async\n> mode and you have enough write requests with the target being slow\n> enough, I suspect you might still not get convergence then (because the\n> writes to the target yield for a long time while ever more write\n> requests pile up) -- so then you'd just shift the dirty tracking from\n> the bitmap to a list of requests in progress.\n> \n> And I think we do want the bulk copy phase to guarantee convergence,\n> too, usually (when active/foreground/synchronous mode is selected). If\n> we don't, then that's a policy decision and would be up to libvirt, as I\n> see it.\n\nThis is a good point. Bulk copy should converge too.\n\nCan we measure the target write rate and guest write rate? A heuristic\ncan choose between async vs sync based on the write rates.\n\nFor example, if the guest write rate has been larger than the target\nwrite rate for the past 10 seconds during the bulk phase, switch to\nsynchronous mirroring.\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"CBrnuvCw\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwhQg4kDrz9s7c\n\tfor ;\n\tMon, 18 Sep 2017 20:03:07 +1000 (AEST)","from localhost ([::1]:35529 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtstN-0007DP-Ra\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 06:03:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:37573)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtssu-0007Cy-9D\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 06:02:42 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtsso-00050o-Cb\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 06:02:36 -0400","from mail-wr0-x233.google.com ([2a00:1450:400c:c0c::233]:53290)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtsso-00050B-5O; Mon, 18 Sep 2017 06:02:30 -0400","by mail-wr0-x233.google.com with SMTP id l22so5833401wrc.10;\n\tMon, 18 Sep 2017 03:02:29 -0700 (PDT)","from localhost ([51.15.41.238]) by smtp.gmail.com with ESMTPSA id\n\ts9sm6405111wra.73.2017.09.18.03.02.28\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tMon, 18 Sep 2017 03:02:28 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=ZTSYVkYJ2RZJ1lWVG5KABTAwofpbTFIwkbnL/vlnyqM=;\n\tb=CBrnuvCww0ebTsu7h3AqSlCogDsS3R77f3ztAAZkqy5a9FvjHFRSgSPfME3OXQpSVf\n\tjJ2dXsqgqZB50VNfmmtMlRzopfBElQb4of/sUPjslaW3v4v66LEhsCcmnYTCcyLbjlH/\n\tkNgtmfzIiXbPhR+jJKZgbGmWJLmXwHc/P+ba3FWG1DoPDWK6uwgKo7bkHj0HwG7gPCRP\n\tquWuz6QvtaT0G8jbhxoPeAJCiinXx2Gba/immyDPOQYQzeWWlyc8YZ6YT1IvJ+7UQ22F\n\tFNy7Ne/c0UaCBzpw3SvFtqluuPef2dEyPHvY5tiVOb83BAWPn5qdG3WrKzkOzei3U7Ub\n\tkJPg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=ZTSYVkYJ2RZJ1lWVG5KABTAwofpbTFIwkbnL/vlnyqM=;\n\tb=QAfS2dGA+NwkXLaXTLc+aSLkumePk0pmsNiz7p3+v1SHe9yBN3fdW1vudT2bxl54aP\n\ttL96hEGAe4YdfLuHhBZsT5Va9jrW53czEhseke3ih1K3GlrsdH4i7YnMQy4gHiOwJcOu\n\t7SaEO5GZyZoOa6AdnlUfUZfGFwlNfmqwrN8mcEoOxDZ7WS85R6PkCZHtK2M1lgrG00ot\n\tpKeIl5vt69/Fyyv3gCJ8X9g0hX704gPdyy9T/N068HWFugdx0JawkTW5OwOcUfyEitdc\n\tB9BetLMeH6LoqEjqGM7+a7/xEfCehxuoAmwzd2WjPWOMMozLCSsRbCaSjTflKvjoHsLw\n\txfCQ==","X-Gm-Message-State":"AHPjjUhypLkbzaqQ8Yd5++71UaygAv56DZ4EFkk+hUCdaQzHaPLSs2Gc\n\tG5nvBpj8uL4tL2ds6bE=","X-Google-Smtp-Source":"ADKCNb6xxm+TlPUJova5YrH9nYT8rBVJ+pgKY/WzlyoACqZYR1RiB5LpW767FbknJDpLRhObP8anag==","X-Received":"by 10.223.129.133 with SMTP id 5mr28732362wra.231.1505728948983; \n\tMon, 18 Sep 2017 03:02:28 -0700 (PDT)","Date":"Mon, 18 Sep 2017 11:02:26 +0100","From":"Stefan Hajnoczi ","To":"Max Reitz ","Message-ID":"<20170918100226.GD31063@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170914154221.GD7370@stefanha-x1.localdomain>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::233","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 00/18] block/mirror: Add\n\tactive-sync mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770261,"web_url":"http://patchwork.ozlabs.org/comment/1770261/","msgid":"<38bf2d1a-ac37-84b0-7d36-689ba4697476@redhat.com>","list_archive_url":null,"date":"2017-09-18T15:42:40","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 00/18] block/mirror: Add\n\tactive-sync mirroring","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-18 12:02, Stefan Hajnoczi wrote:\n> On Sat, Sep 16, 2017 at 04:02:45PM +0200, Max Reitz wrote:\n>> On 2017-09-14 17:42, Stefan Hajnoczi wrote:\n>>> On Wed, Sep 13, 2017 at 08:18:52PM +0200, Max Reitz wrote:\n>>>> There may be a couple of things to do on top of this series:\n>>>> - Allow switching between active and passive mode at runtime: This\n>>>> should not be too difficult to implement, the main question is how to\n>>>> expose it to the user.\n>>>> (I seem to recall we wanted some form of block-job-set-option\n>>>> command...?)\n>>>>\n>>>> - Implement an asynchronous active mode: May be detrimental when it\n>>>> comes to convergence, but it might be nice to have anyway. May or may\n>>>> not be complicated to implement.\n>>>\n>>> Ideally the user doesn't have to know about async vs sync. It's an\n>>> implementation detail.\n>>>\n>>> Async makes sense during the bulk copy phase (e.g. sync=full) because\n>>> guest read/write latencies are mostly unaffected. Once the entire\n>>> device has been copied there are probably still dirty blocks left\n>>> because the guest touched them while the mirror job was running. At\n>>> that point it definitely makes sense to switch to synchronous mirroring\n>>> in order to converge.\n>>\n>> Makes sense, but I'm not sure whether it really is just an\n>> implementation detail. If you're in the bulk copy phase in active/async\n>> mode and you have enough write requests with the target being slow\n>> enough, I suspect you might still not get convergence then (because the\n>> writes to the target yield for a long time while ever more write\n>> requests pile up) -- so then you'd just shift the dirty tracking from\n>> the bitmap to a list of requests in progress.\n>>\n>> And I think we do want the bulk copy phase to guarantee convergence,\n>> too, usually (when active/foreground/synchronous mode is selected). If\n>> we don't, then that's a policy decision and would be up to libvirt, as I\n>> see it.\n> \n> This is a good point. Bulk copy should converge too.\n> \n> Can we measure the target write rate and guest write rate? A heuristic\n> can choose between async vs sync based on the write rates.\n> \n> For example, if the guest write rate has been larger than the target\n> write rate for the past 10 seconds during the bulk phase, switch to\n> synchronous mirroring.\n\nI guess we can just count how many unfinished target write requests are\npiling up.\n\n...or libvirt can simply see that the block job is not progressing and\nswitch the mode. :-)\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwqzF4CMGz9s06\n\tfor ;\n\tTue, 19 Sep 2017 01:43:21 +1000 (AEST)","from localhost ([::1]:37382 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtyCd-0006xv-OV\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 11:43:19 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:56361)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtyCG-0006vD-QC\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 11:43:01 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtyCF-0005fL-Nl\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 11:42:56 -0400","from mx1.redhat.com ([209.132.183.28]:51778)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtyCA-0005az-Tc; Mon, 18 Sep 2017 11:42:51 -0400","from smtp.corp.redhat.com\n\t(int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 3AF9A81227;\n\tMon, 18 Sep 2017 15:42:49 +0000 (UTC)","from dresden.str.redhat.com (unknown [10.40.205.73])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 0D10E600C2;\n\tMon, 18 Sep 2017 15:42:42 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 3AF9A81227","To":"Stefan Hajnoczi ","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170914154221.GD7370@stefanha-x1.localdomain>\n\t\n\t<20170918100226.GD31063@stefanha-x1.localdomain>","From":"Max Reitz ","Message-ID":"<38bf2d1a-ac37-84b0-7d36-689ba4697476@redhat.com>","Date":"Mon, 18 Sep 2017 17:42:40 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170918100226.GD31063@stefanha-x1.localdomain>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"nOeiSfdPTVsNilGBEwiSsH8lk9oAwWKSk\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.11","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tMon, 18 Sep 2017 15:42:49 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 00/18] block/mirror: Add\n\tactive-sync mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]