[{"id":1768703,"web_url":"http://patchwork.ozlabs.org/comment/1768703/","msgid":"<20170914155738.GE7370@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-14T15:57:38","subject":"Re: [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring","submitter":{"id":17227,"url":"http://patchwork.ozlabs.org/api/people/17227/","name":"Stefan Hajnoczi","email":"stefanha@redhat.com"},"content":"On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> This patch implements active synchronous mirroring. In active mode, the\n> passive mechanism will still be in place and is used to copy all\n> initially dirty clusters off the source disk; but every write request\n> will write data both to the source and the target disk, so the source\n> cannot be dirtied faster than data is mirrored to the target. Also,\n> once the block job has converged (BLOCK_JOB_READY sent), source and\n> target are guaranteed to stay in sync (unless an error occurs).\n> \n> Optionally, dirty data can be copied to the target disk on read\n> operations, too.\n> \n> Active mode is completely optional and currently disabled at runtime. A\n> later patch will add a way for users to enable it.\n> \n> Signed-off-by: Max Reitz \n> ---\n> qapi/block-core.json | 23 +++++++\n> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> 2 files changed, 205 insertions(+), 5 deletions(-)\n> \n> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> index bb11815608..e072cfa67c 100644\n> --- a/qapi/block-core.json\n> +++ b/qapi/block-core.json\n> @@ -938,6 +938,29 @@\n> 'data': ['top', 'full', 'none', 'incremental'] }\n> \n> ##\n> +# @MirrorCopyMode:\n> +#\n> +# An enumeration whose values tell the mirror block job when to\n> +# trigger writes to the target.\n> +#\n> +# @passive: copy data in background only.\n> +#\n> +# @active-write: when data is written to the source, write it\n> +# (synchronously) to the target as well. In addition,\n> +# data is copied in background just like in @passive\n> +# mode.\n> +#\n> +# @active-read-write: write data to the target (synchronously) both\n> +# when it is read from and written to the source.\n> +# In addition, data is copied in background just\n> +# like in @passive mode.\n\nI'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\nmeans committing the top-most BDS while the guest is accessing it. The\n\"passive\" mirror block still works on the top-most BDS while the guest\nis accessing it.\n\nCalling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\nthe terminology used in disk replication (e.g. DRBD).\n\nIdeally the user wouldn't have to worry about async vs sync because QEMU\nwould switch modes as appropriate in order to converge. That way\nlibvirt also doesn't have to worry about this.\n\n> static int coroutine_fn bdrv_mirror_top_preadv(BlockDriverState *bs,\n> uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags)\n> {\n> - return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);\n> + MirrorOp *op = NULL;\n> + MirrorBDSOpaque *s = bs->opaque;\n> + int ret = 0;\n> + bool copy_to_target;\n> +\n> + copy_to_target = s->job->ret >= 0 &&\n> + s->job->copy_mode == MIRROR_COPY_MODE_ACTIVE_READ_WRITE;\n> +\n> + if (copy_to_target) {\n> + op = active_write_prepare(s->job, offset, bytes);\n> + }\n> +\n> + ret = bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);\n> + if (ret < 0) {\n> + goto out;\n> + }\n> +\n> + if (copy_to_target) {\n> + do_sync_target_write(s->job, offset, bytes, qiov, 0);\n> + }\n\nThis mode is dangerous. See bdrv_co_do_copy_on_readv():\n\n /* Perform I/O through a temporary buffer so that users who scribble over\n * their read buffer while the operation is in progress do not end up\n * modifying the image file. This is critical for zero-copy guest I/O\n * where anything might happen inside guest memory.\n */\n void *bounce_buffer;\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx07.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=stefanha@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xtNVZ0QM1z9s9Y\n\tfor ;\n\tFri, 15 Sep 2017 01:58:30 +1000 (AEST)","from localhost ([::1]:48588 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsWX6-0003xZ-5S\n\tfor incoming@patchwork.ozlabs.org; Thu, 14 Sep 2017 11:58:28 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35084)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsWWX-0003vz-6B\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 11:57:54 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsWWW-0008Ow-2r\n\tfor qemu-devel@nongnu.org; Thu, 14 Sep 2017 11:57:53 -0400","from mx1.redhat.com ([209.132.183.28]:55180)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsWWP-0008KD-GZ; Thu, 14 Sep 2017 11:57:45 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 72397C047B70;\n\tThu, 14 Sep 2017 15:57:44 +0000 (UTC)","from localhost (ovpn-117-58.ams2.redhat.com [10.36.117.58])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id CE8536FE4E;\n\tThu, 14 Sep 2017 15:57:39 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 72397C047B70","Date":"Thu, 14 Sep 2017 16:57:38 +0100","From":"Stefan Hajnoczi ","To":"Max Reitz ","Message-ID":"<20170914155738.GE7370@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170913181910.29688-16-mreitz@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.31]);\n\tThu, 14 Sep 2017 15:57:44 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , John Snow ,\n\tFam Zheng , qemu-devel@nongnu.org, qemu-block@nongnu.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1769606,"web_url":"http://patchwork.ozlabs.org/comment/1769606/","msgid":"<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>","list_archive_url":null,"date":"2017-09-16T13:58:01","subject":"Re: [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n>> This patch implements active synchronous mirroring. In active mode, the\n>> passive mechanism will still be in place and is used to copy all\n>> initially dirty clusters off the source disk; but every write request\n>> will write data both to the source and the target disk, so the source\n>> cannot be dirtied faster than data is mirrored to the target. Also,\n>> once the block job has converged (BLOCK_JOB_READY sent), source and\n>> target are guaranteed to stay in sync (unless an error occurs).\n>>\n>> Optionally, dirty data can be copied to the target disk on read\n>> operations, too.\n>>\n>> Active mode is completely optional and currently disabled at runtime. A\n>> later patch will add a way for users to enable it.\n>>\n>> Signed-off-by: Max Reitz \n>> ---\n>> qapi/block-core.json | 23 +++++++\n>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n>> 2 files changed, 205 insertions(+), 5 deletions(-)\n>>\n>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n>> index bb11815608..e072cfa67c 100644\n>> --- a/qapi/block-core.json\n>> +++ b/qapi/block-core.json\n>> @@ -938,6 +938,29 @@\n>> 'data': ['top', 'full', 'none', 'incremental'] }\n>> \n>> ##\n>> +# @MirrorCopyMode:\n>> +#\n>> +# An enumeration whose values tell the mirror block job when to\n>> +# trigger writes to the target.\n>> +#\n>> +# @passive: copy data in background only.\n>> +#\n>> +# @active-write: when data is written to the source, write it\n>> +# (synchronously) to the target as well. In addition,\n>> +# data is copied in background just like in @passive\n>> +# mode.\n>> +#\n>> +# @active-read-write: write data to the target (synchronously) both\n>> +# when it is read from and written to the source.\n>> +# In addition, data is copied in background just\n>> +# like in @passive mode.\n> \n> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> means committing the top-most BDS while the guest is accessing it. The\n> \"passive\" mirror block still works on the top-most BDS while the guest\n> is accessing it.\n> \n> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> the terminology used in disk replication (e.g. DRBD).\n\nI'd be OK with that, too, but I think I remember that in the past at\nleast Kevin made a clear distinction between active/passive and\nsync/async when it comes to mirroring.\n\n> Ideally the user wouldn't have to worry about async vs sync because QEMU\n> would switch modes as appropriate in order to converge. That way\n> libvirt also doesn't have to worry about this.\n\nSo here you mean async/sync in the way I meant it, i.e., whether the\nmirror operations themselves are async/sync?\n\nMaybe we could call the the passive mode \"background\" and the active\nmode \"synchronous\" (or maybe even \"foreground\")? Because sync then\nmeans three things:\n(1) The block job sync mode\n(2) Synchronous write operations to the target\n(3) Active mirroring mode\n\n(1) is relatively easy to distinguish because it's just \"the job sync\nmode\". If we call the passive mode \"background\" we can distinguish (2)\nand (3) at least based on their antonyms: \"sync (as in sync/async)\" vs.\n\"sync (as in sync/background)\".\n\n>> static int coroutine_fn bdrv_mirror_top_preadv(BlockDriverState *bs,\n>> uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags)\n>> {\n>> - return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);\n>> + MirrorOp *op = NULL;\n>> + MirrorBDSOpaque *s = bs->opaque;\n>> + int ret = 0;\n>> + bool copy_to_target;\n>> +\n>> + copy_to_target = s->job->ret >= 0 &&\n>> + s->job->copy_mode == MIRROR_COPY_MODE_ACTIVE_READ_WRITE;\n>> +\n>> + if (copy_to_target) {\n>> + op = active_write_prepare(s->job, offset, bytes);\n>> + }\n>> +\n>> + ret = bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);\n>> + if (ret < 0) {\n>> + goto out;\n>> + }\n>> +\n>> + if (copy_to_target) {\n>> + do_sync_target_write(s->job, offset, bytes, qiov, 0);\n>> + }\n> \n> This mode is dangerous. See bdrv_co_do_copy_on_readv():\n> \n> /* Perform I/O through a temporary buffer so that users who scribble over\n> * their read buffer while the operation is in progress do not end up\n> * modifying the image file. This is critical for zero-copy guest I/O\n> * where anything might happen inside guest memory.\n> */\n> void *bounce_buffer;\n\nOoh, yes, right. I'll just drop it for now, then.\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx10.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xvYlx3dHLz9t2M\n\tfor ;\n\tSat, 16 Sep 2017 23:59:07 +1000 (AEST)","from localhost ([::1]:57401 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtDcf-0003q1-El\n\tfor incoming@patchwork.ozlabs.org; Sat, 16 Sep 2017 09:59:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:39646)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtDcI-0003oV-Fl\n\tfor qemu-devel@nongnu.org; Sat, 16 Sep 2017 09:58:43 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtDcH-0006Mm-Dk\n\tfor qemu-devel@nongnu.org; Sat, 16 Sep 2017 09:58:42 -0400","from mx1.redhat.com ([209.132.183.28]:60334)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtDcB-0006KF-Lb; Sat, 16 Sep 2017 09:58:35 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 543B45D697;\n\tSat, 16 Sep 2017 13:58:33 +0000 (UTC)","from dresden.str.redhat.com (ovpn-204-52.brq.redhat.com\n\t[10.40.204.52])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 23E6439C8;\n\tSat, 16 Sep 2017 13:58:24 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 543B45D697","To":"Stefan Hajnoczi ","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>","From":"Max Reitz ","Message-ID":"<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>","Date":"Sat, 16 Sep 2017 15:58:01 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170914155738.GE7370@stefanha-x1.localdomain>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"FvXdb8bLvUKh8nObPwr0qtcxgdBpqAhpN\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.39]);\n\tSat, 16 Sep 2017 13:58:33 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , John Snow ,\n\tFam Zheng , qemu-devel@nongnu.org, qemu-block@nongnu.org","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770049,"web_url":"http://patchwork.ozlabs.org/comment/1770049/","msgid":"<20170918100626.GE31063@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-18T10:06:26","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":2747,"url":"http://patchwork.ozlabs.org/api/people/2747/","name":"Stefan Hajnoczi","email":"stefanha@gmail.com"},"content":"On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> > On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> >> This patch implements active synchronous mirroring. In active mode, the\n> >> passive mechanism will still be in place and is used to copy all\n> >> initially dirty clusters off the source disk; but every write request\n> >> will write data both to the source and the target disk, so the source\n> >> cannot be dirtied faster than data is mirrored to the target. Also,\n> >> once the block job has converged (BLOCK_JOB_READY sent), source and\n> >> target are guaranteed to stay in sync (unless an error occurs).\n> >>\n> >> Optionally, dirty data can be copied to the target disk on read\n> >> operations, too.\n> >>\n> >> Active mode is completely optional and currently disabled at runtime. A\n> >> later patch will add a way for users to enable it.\n> >>\n> >> Signed-off-by: Max Reitz \n> >> ---\n> >> qapi/block-core.json | 23 +++++++\n> >> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> >> 2 files changed, 205 insertions(+), 5 deletions(-)\n> >>\n> >> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> >> index bb11815608..e072cfa67c 100644\n> >> --- a/qapi/block-core.json\n> >> +++ b/qapi/block-core.json\n> >> @@ -938,6 +938,29 @@\n> >> 'data': ['top', 'full', 'none', 'incremental'] }\n> >> \n> >> ##\n> >> +# @MirrorCopyMode:\n> >> +#\n> >> +# An enumeration whose values tell the mirror block job when to\n> >> +# trigger writes to the target.\n> >> +#\n> >> +# @passive: copy data in background only.\n> >> +#\n> >> +# @active-write: when data is written to the source, write it\n> >> +# (synchronously) to the target as well. In addition,\n> >> +# data is copied in background just like in @passive\n> >> +# mode.\n> >> +#\n> >> +# @active-read-write: write data to the target (synchronously) both\n> >> +# when it is read from and written to the source.\n> >> +# In addition, data is copied in background just\n> >> +# like in @passive mode.\n> > \n> > I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> > means committing the top-most BDS while the guest is accessing it. The\n> > \"passive\" mirror block still works on the top-most BDS while the guest\n> > is accessing it.\n> > \n> > Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> > the terminology used in disk replication (e.g. DRBD).\n> \n> I'd be OK with that, too, but I think I remember that in the past at\n> least Kevin made a clear distinction between active/passive and\n> sync/async when it comes to mirroring.\n> \n> > Ideally the user wouldn't have to worry about async vs sync because QEMU\n> > would switch modes as appropriate in order to converge. That way\n> > libvirt also doesn't have to worry about this.\n> \n> So here you mean async/sync in the way I meant it, i.e., whether the\n> mirror operations themselves are async/sync?\n\nThe meaning I had in mind is:\n\nSync mirroring means a guest write waits until the target write\ncompletes.\n\nAsync mirroring means guest writes completes independently of target\nwrites.","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"rSHK8o69\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwhWQ450Vz9s7c\n\tfor ;\n\tMon, 18 Sep 2017 20:07:14 +1000 (AEST)","from localhost ([::1]:35540 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtsxM-0008UO-MR\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 06:07:12 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:38603)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtswj-0008Q9-0U\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 06:06:38 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtswf-0007YN-PW\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 06:06:32 -0400","from mail-wr0-x22c.google.com ([2a00:1450:400c:c0c::22c]:49545)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtswf-0007Xp-JJ; Mon, 18 Sep 2017 06:06:29 -0400","by mail-wr0-x22c.google.com with SMTP id u96so1643wrb.6;\n\tMon, 18 Sep 2017 03:06:29 -0700 (PDT)","from localhost ([51.15.41.238]) by smtp.gmail.com with ESMTPSA id\n\tx5sm6255910wre.18.2017.09.18.03.06.27\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tMon, 18 Sep 2017 03:06:27 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=Cc4dvbpjoUSs/jFp22Xgyuya9o+dsDopaa7eJ06ojMk=;\n\tb=rSHK8o69QQhLZhEop+9wWhAn1qSGFxiPjA4cKXM395gXCDURmvlz5SN6W5waHWM4O8\n\tJYN7AzSNCYKIgCs0Y8vugRQngrSysL+5BcV5OthLc+IKfaN/9+AUThQyRjnbCa9GChXg\n\t1qHNiGysq5XkMECa4FVVQOcYsypO9fRc6l2C3MpK/ToJ8AEcogElOvklyIYgDPZ6XMM+\n\t5vYvUtTiwm2pG3rM7RuiwXCxPaEyxGdIbvdjUL0qB4cNQQTQTP3H8dI0PJERJ7vyA7n7\n\tqQKMMlsBzDDFujjfVnGrJ+e70a+bOl7MoH516vMFrWj7JZ8InFG2quMRCa2kRai+lyHf\n\tQ7Mg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=Cc4dvbpjoUSs/jFp22Xgyuya9o+dsDopaa7eJ06ojMk=;\n\tb=JiyUPvY4B/cV32dNi9V5NzmY7PZVKRzSpzKx+asGYkkCFP6wbcuy2NT4GgfZbVYWzO\n\tDdctbQVreAgvW2/TwbGE7MeOCn8/Z33NB5KKomGRD6Fbw8bXiUAQXNVr8bDlN9T7OVf5\n\t/jmJEUlMUIZQYGG7C1a30qsLTIBStbmCvboeajG7HKFt8M4XOv9KMwnKOwwjH+z47DXl\n\tYzgKI7E4pyDx37NOiL+QyoZyoaRpvkxXN7T8vHmSTEa4gY9T+L1vhQ2K+JbdyD+WCDWp\n\tjQ0mf1BucBi8cq6k+eCCtCJlsEvWbUfhvRbYWb52zQt/UV9GVa3P5McCMFV9hBFTh8Pq\n\tOtcA==","X-Gm-Message-State":"AHPjjUhZ9lcOU8PDX4BMZankTna+FWsI8cOG0OFZV8MbMYUln9A9VIYA\n\t0NUUIQHCtSAejA==","X-Google-Smtp-Source":"ADKCNb4tB8GqPTuoC0FM2LiM1MTQnqIyvZRjyWEV02t5OVMhGPV4yp3pYBOIGhS4wn0pOVW54Bn4Bw==","X-Received":"by 10.223.176.213 with SMTP id j21mr26577734wra.56.1505729188557;\n\tMon, 18 Sep 2017 03:06:28 -0700 (PDT)","Date":"Mon, 18 Sep 2017 11:06:26 +0100","From":"Stefan Hajnoczi ","To":"Max Reitz ","Message-ID":"<20170918100626.GE31063@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c0c::22c","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770294,"web_url":"http://patchwork.ozlabs.org/comment/1770294/","msgid":"","list_archive_url":null,"date":"2017-09-18T16:26:51","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n> On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n>> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n>>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n>>>> This patch implements active synchronous mirroring. In active mode, the\n>>>> passive mechanism will still be in place and is used to copy all\n>>>> initially dirty clusters off the source disk; but every write request\n>>>> will write data both to the source and the target disk, so the source\n>>>> cannot be dirtied faster than data is mirrored to the target. Also,\n>>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n>>>> target are guaranteed to stay in sync (unless an error occurs).\n>>>>\n>>>> Optionally, dirty data can be copied to the target disk on read\n>>>> operations, too.\n>>>>\n>>>> Active mode is completely optional and currently disabled at runtime. A\n>>>> later patch will add a way for users to enable it.\n>>>>\n>>>> Signed-off-by: Max Reitz \n>>>> ---\n>>>> qapi/block-core.json | 23 +++++++\n>>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n>>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n>>>>\n>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n>>>> index bb11815608..e072cfa67c 100644\n>>>> --- a/qapi/block-core.json\n>>>> +++ b/qapi/block-core.json\n>>>> @@ -938,6 +938,29 @@\n>>>> 'data': ['top', 'full', 'none', 'incremental'] }\n>>>> \n>>>> ##\n>>>> +# @MirrorCopyMode:\n>>>> +#\n>>>> +# An enumeration whose values tell the mirror block job when to\n>>>> +# trigger writes to the target.\n>>>> +#\n>>>> +# @passive: copy data in background only.\n>>>> +#\n>>>> +# @active-write: when data is written to the source, write it\n>>>> +# (synchronously) to the target as well. In addition,\n>>>> +# data is copied in background just like in @passive\n>>>> +# mode.\n>>>> +#\n>>>> +# @active-read-write: write data to the target (synchronously) both\n>>>> +# when it is read from and written to the source.\n>>>> +# In addition, data is copied in background just\n>>>> +# like in @passive mode.\n>>>\n>>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n>>> means committing the top-most BDS while the guest is accessing it. The\n>>> \"passive\" mirror block still works on the top-most BDS while the guest\n>>> is accessing it.\n>>>\n>>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n>>> the terminology used in disk replication (e.g. DRBD).\n>>\n>> I'd be OK with that, too, but I think I remember that in the past at\n>> least Kevin made a clear distinction between active/passive and\n>> sync/async when it comes to mirroring.\n>>\n>>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n>>> would switch modes as appropriate in order to converge. That way\n>>> libvirt also doesn't have to worry about this.\n>>\n>> So here you mean async/sync in the way I meant it, i.e., whether the\n>> mirror operations themselves are async/sync?\n> \n> The meaning I had in mind is:\n> \n> Sync mirroring means a guest write waits until the target write\n> completes.\n\nI.e. active-sync, ...\n\n> Async mirroring means guest writes completes independently of target\n> writes.\n\n... i.e. passive or active-async in the future.\n\nSo you really want qemu to decide whether to use active or passive mode\ndepending on what's enough to let the block job converge and not\nintroduce any switch for the user?\n\nI'm not sure whether I like this too much, mostly because \"libvirt\ndoesn't have to worry\" doesn't feel quite right to me. If we don't make\nlibvirt worry about this, then qemu has to worry. I'm not sure whether\nthat's any better.\n\nI think this really does get into policy territory. Just switching to\nactive mode the instant target writes are slower than source writes may\nnot be what the user wants: Maybe it's OK for a short duration because\nthey don't care about hard convergence too much. Maybe they want to\nswitch to active mode already when \"only\" twice as much is written to\nthe target as to the source.\n\nI think this is a decision the management layer (or the user) has to make.\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xwryv5CGDz9s78\n\tfor ;\n\tTue, 19 Sep 2017 02:28:07 +1000 (AEST)","from localhost ([::1]:37717 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dtytx-0005Po-9m\n\tfor incoming@patchwork.ozlabs.org; Mon, 18 Sep 2017 12:28:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:48115)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dtyt7-0005Fo-PE\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 12:27:15 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dtyt6-0003HW-Iw\n\tfor qemu-devel@nongnu.org; Mon, 18 Sep 2017 12:27:13 -0400","from mx1.redhat.com ([209.132.183.28]:52020)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dtysz-000377-Sm; Mon, 18 Sep 2017 12:27:06 -0400","from smtp.corp.redhat.com\n\t(int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id D74A8806AA;\n\tMon, 18 Sep 2017 16:27:04 +0000 (UTC)","from dresden.str.redhat.com (unknown [10.40.205.73])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id D564160610;\n\tMon, 18 Sep 2017 16:26:52 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com D74A8806AA","To":"Stefan Hajnoczi ","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>","From":"Max Reitz ","Message-ID":"","Date":"Mon, 18 Sep 2017 18:26:51 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20170918100626.GE31063@stefanha-x1.localdomain>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"baU0SGru35xgHo0t7o6fjImp82anbGQHP\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.13","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.26]);\n\tMon, 18 Sep 2017 16:27:05 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770796,"web_url":"http://patchwork.ozlabs.org/comment/1770796/","msgid":"<20170919094416.GC22797@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-19T09:44:16","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":2747,"url":"http://patchwork.ozlabs.org/api/people/2747/","name":"Stefan Hajnoczi","email":"stefanha@gmail.com"},"content":"On Mon, Sep 18, 2017 at 06:26:51PM +0200, Max Reitz wrote:\n> On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n> > On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n> >> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> >>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> >>>> This patch implements active synchronous mirroring. In active mode, the\n> >>>> passive mechanism will still be in place and is used to copy all\n> >>>> initially dirty clusters off the source disk; but every write request\n> >>>> will write data both to the source and the target disk, so the source\n> >>>> cannot be dirtied faster than data is mirrored to the target. Also,\n> >>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n> >>>> target are guaranteed to stay in sync (unless an error occurs).\n> >>>>\n> >>>> Optionally, dirty data can be copied to the target disk on read\n> >>>> operations, too.\n> >>>>\n> >>>> Active mode is completely optional and currently disabled at runtime. A\n> >>>> later patch will add a way for users to enable it.\n> >>>>\n> >>>> Signed-off-by: Max Reitz \n> >>>> ---\n> >>>> qapi/block-core.json | 23 +++++++\n> >>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> >>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n> >>>>\n> >>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> >>>> index bb11815608..e072cfa67c 100644\n> >>>> --- a/qapi/block-core.json\n> >>>> +++ b/qapi/block-core.json\n> >>>> @@ -938,6 +938,29 @@\n> >>>> 'data': ['top', 'full', 'none', 'incremental'] }\n> >>>> \n> >>>> ##\n> >>>> +# @MirrorCopyMode:\n> >>>> +#\n> >>>> +# An enumeration whose values tell the mirror block job when to\n> >>>> +# trigger writes to the target.\n> >>>> +#\n> >>>> +# @passive: copy data in background only.\n> >>>> +#\n> >>>> +# @active-write: when data is written to the source, write it\n> >>>> +# (synchronously) to the target as well. In addition,\n> >>>> +# data is copied in background just like in @passive\n> >>>> +# mode.\n> >>>> +#\n> >>>> +# @active-read-write: write data to the target (synchronously) both\n> >>>> +# when it is read from and written to the source.\n> >>>> +# In addition, data is copied in background just\n> >>>> +# like in @passive mode.\n> >>>\n> >>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> >>> means committing the top-most BDS while the guest is accessing it. The\n> >>> \"passive\" mirror block still works on the top-most BDS while the guest\n> >>> is accessing it.\n> >>>\n> >>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> >>> the terminology used in disk replication (e.g. DRBD).\n> >>\n> >> I'd be OK with that, too, but I think I remember that in the past at\n> >> least Kevin made a clear distinction between active/passive and\n> >> sync/async when it comes to mirroring.\n> >>\n> >>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n> >>> would switch modes as appropriate in order to converge. That way\n> >>> libvirt also doesn't have to worry about this.\n> >>\n> >> So here you mean async/sync in the way I meant it, i.e., whether the\n> >> mirror operations themselves are async/sync?\n> > \n> > The meaning I had in mind is:\n> > \n> > Sync mirroring means a guest write waits until the target write\n> > completes.\n> \n> I.e. active-sync, ...\n> \n> > Async mirroring means guest writes completes independently of target\n> > writes.\n> \n> ... i.e. passive or active-async in the future.\n> \n> So you really want qemu to decide whether to use active or passive mode\n> depending on what's enough to let the block job converge and not\n> introduce any switch for the user?\n> \n> I'm not sure whether I like this too much, mostly because \"libvirt\n> doesn't have to worry\" doesn't feel quite right to me. If we don't make\n> libvirt worry about this, then qemu has to worry. I'm not sure whether\n> that's any better.\n> \n> I think this really does get into policy territory. Just switching to\n> active mode the instant target writes are slower than source writes may\n> not be what the user wants: Maybe it's OK for a short duration because\n> they don't care about hard convergence too much. Maybe they want to\n> switch to active mode already when \"only\" twice as much is written to\n> the target as to the source.\n> \n> I think this is a decision the management layer (or the user) has to make.\n\nEric: Does libvirt want to be involved with converging the mirror job\n(i.e. if the guest is writing to disk faster than QEMU can copy data to\nthe target)?\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=gmail.com header.i=@gmail.com\n\theader.b=\"PH1nZpGu\"; dkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxHzH4xZjz9sPm\n\tfor ;\n\tTue, 19 Sep 2017 19:44:59 +1000 (AEST)","from localhost ([::1]:41022 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duF5M-00058Y-CW\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 05:44:56 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:57672)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duF4q-000578-Ou\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 05:44:25 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duF4m-0000WU-TN\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 05:44:24 -0400","from mail-wm0-x22e.google.com ([2a00:1450:400c:c09::22e]:47271)\n\tby eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)\n\t(Exim 4.71) (envelope-from )\n\tid 1duF4m-0000Te-IY; Tue, 19 Sep 2017 05:44:20 -0400","by mail-wm0-x22e.google.com with SMTP id 13so3534564wmq.2;\n\tTue, 19 Sep 2017 02:44:19 -0700 (PDT)","from localhost (33.59.7.51.dyn.plus.net. [51.7.59.33])\n\tby smtp.gmail.com with ESMTPSA id\n\th128sm995060wmf.9.2017.09.19.02.44.16\n\t(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);\n\tTue, 19 Sep 2017 02:44:17 -0700 (PDT)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;\n\th=date:from:to:cc:subject:message-id:references:mime-version\n\t:content-disposition:in-reply-to:user-agent;\n\tbh=jMeBe2Vs0XKCZW49Ca/GShrNWt9h4uPGH9hMlJtPgkY=;\n\tb=PH1nZpGufXFRnjVEWinAG79hUVToKamMsiWP60/l1kwnQlzCmOUkCeb0rHmNfbo7Y4\n\t+zmuobjrgC7hGJSQaqQu1RCynZR/uZwD0iagU4AE0KF4ipjxNgxIjXSRBuhTzHztSHbc\n\t+0LV9eg30mS4pjB33lI41WDWdSvNyYOQJLogvgSnHlnetbsw28I/tUVS3X2eeID0P5pY\n\tjWmd26wrf4c5eppYRdWgUV0oSjES1Z0sPJpezOhyG/mbf3YWdB0mfUkiCOdyNFuePr4I\n\thmL1PyuOYpx4arudGH1dH55JF9fT3GctpzIiAW0Bqn8fwu+0Bc1uEzPo2gzM+0fWwB2a\n\tl6hg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:date:from:to:cc:subject:message-id:references\n\t:mime-version:content-disposition:in-reply-to:user-agent;\n\tbh=jMeBe2Vs0XKCZW49Ca/GShrNWt9h4uPGH9hMlJtPgkY=;\n\tb=C8DAPGEnDGmrh9E7vPqB/K/YBWln9dcBlH0+Xsc2DG6CQwmAI59T+DaY0JU9609ERz\n\tCtcf1zRMrrV/k4xHi0KyrWjHreg+bhYzAoOYdCsHQ7MQWAOV/H7m69+GQwfV0qPjwn7+\n\tHEL96B6PTJYM7Il6Dokd1PJG18BoYZauTxQxW4J+nfTtc/ZgFmYSA0i/T++aH9l6Vauv\n\tQXZBjjtLFHxTTDR67W4elyZavT9MUPTpnJndY7C3H5HO2E/UiqWhrXnMpR4v8/9L0VO0\n\t/526A9LVgS5awbg0tVyfk1IWwird5gn+IJ5Q/0dMpkezb3hjwJB8UO4cxyIJfMRmW2W0\n\tkngg==","X-Gm-Message-State":"AHPjjUiJ0DodybV+cSXx/2AKqOqW3EjcRoSMtBTa2J8ZQtZtA8/b7kH+\n\tkrh+o6O/aTitD0oFZAwDLY8=","X-Google-Smtp-Source":"AOwi7QAH3tVfnCRJtVnZhSVnessdlIVvMpwWlRyFftYLULk4jbh1CBS317o9EMSX3WfJhykGA7y7xA==","X-Received":"by 10.28.152.23 with SMTP id a23mr505058wme.45.1505814258093;\n\tTue, 19 Sep 2017 02:44:18 -0700 (PDT)","Date":"Tue, 19 Sep 2017 10:44:16 +0100","From":"Stefan Hajnoczi ","To":"Eric Blake ","Message-ID":"<20170919094416.GC22797@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-detected-operating-system":"by eggs.gnu.org: Genre and OS details not\n\trecognized.","X-Received-From":"2a00:1450:400c:c09::22e","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , Fam Zheng ,\n\tqemu-block@nongnu.org, qemu-devel@nongnu.org,\n\tMax Reitz , Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1770806,"web_url":"http://patchwork.ozlabs.org/comment/1770806/","msgid":"<20170919095750.GG9536@redhat.com>","list_archive_url":null,"date":"2017-09-19T09:57:50","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":2694,"url":"http://patchwork.ozlabs.org/api/people/2694/","name":"Daniel P. Berrangé","email":"berrange@redhat.com"},"content":"On Tue, Sep 19, 2017 at 10:44:16AM +0100, Stefan Hajnoczi wrote:\n> On Mon, Sep 18, 2017 at 06:26:51PM +0200, Max Reitz wrote:\n> > On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n> > > On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n> > >> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> > >>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> > >>>> This patch implements active synchronous mirroring. In active mode, the\n> > >>>> passive mechanism will still be in place and is used to copy all\n> > >>>> initially dirty clusters off the source disk; but every write request\n> > >>>> will write data both to the source and the target disk, so the source\n> > >>>> cannot be dirtied faster than data is mirrored to the target. Also,\n> > >>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n> > >>>> target are guaranteed to stay in sync (unless an error occurs).\n> > >>>>\n> > >>>> Optionally, dirty data can be copied to the target disk on read\n> > >>>> operations, too.\n> > >>>>\n> > >>>> Active mode is completely optional and currently disabled at runtime. A\n> > >>>> later patch will add a way for users to enable it.\n> > >>>>\n> > >>>> Signed-off-by: Max Reitz \n> > >>>> ---\n> > >>>> qapi/block-core.json | 23 +++++++\n> > >>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> > >>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n> > >>>>\n> > >>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> > >>>> index bb11815608..e072cfa67c 100644\n> > >>>> --- a/qapi/block-core.json\n> > >>>> +++ b/qapi/block-core.json\n> > >>>> @@ -938,6 +938,29 @@\n> > >>>> 'data': ['top', 'full', 'none', 'incremental'] }\n> > >>>> \n> > >>>> ##\n> > >>>> +# @MirrorCopyMode:\n> > >>>> +#\n> > >>>> +# An enumeration whose values tell the mirror block job when to\n> > >>>> +# trigger writes to the target.\n> > >>>> +#\n> > >>>> +# @passive: copy data in background only.\n> > >>>> +#\n> > >>>> +# @active-write: when data is written to the source, write it\n> > >>>> +# (synchronously) to the target as well. In addition,\n> > >>>> +# data is copied in background just like in @passive\n> > >>>> +# mode.\n> > >>>> +#\n> > >>>> +# @active-read-write: write data to the target (synchronously) both\n> > >>>> +# when it is read from and written to the source.\n> > >>>> +# In addition, data is copied in background just\n> > >>>> +# like in @passive mode.\n> > >>>\n> > >>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> > >>> means committing the top-most BDS while the guest is accessing it. The\n> > >>> \"passive\" mirror block still works on the top-most BDS while the guest\n> > >>> is accessing it.\n> > >>>\n> > >>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> > >>> the terminology used in disk replication (e.g. DRBD).\n> > >>\n> > >> I'd be OK with that, too, but I think I remember that in the past at\n> > >> least Kevin made a clear distinction between active/passive and\n> > >> sync/async when it comes to mirroring.\n> > >>\n> > >>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n> > >>> would switch modes as appropriate in order to converge. That way\n> > >>> libvirt also doesn't have to worry about this.\n> > >>\n> > >> So here you mean async/sync in the way I meant it, i.e., whether the\n> > >> mirror operations themselves are async/sync?\n> > > \n> > > The meaning I had in mind is:\n> > > \n> > > Sync mirroring means a guest write waits until the target write\n> > > completes.\n> > \n> > I.e. active-sync, ...\n> > \n> > > Async mirroring means guest writes completes independently of target\n> > > writes.\n> > \n> > ... i.e. passive or active-async in the future.\n> > \n> > So you really want qemu to decide whether to use active or passive mode\n> > depending on what's enough to let the block job converge and not\n> > introduce any switch for the user?\n> > \n> > I'm not sure whether I like this too much, mostly because \"libvirt\n> > doesn't have to worry\" doesn't feel quite right to me. If we don't make\n> > libvirt worry about this, then qemu has to worry. I'm not sure whether\n> > that's any better.\n> > \n> > I think this really does get into policy territory. Just switching to\n> > active mode the instant target writes are slower than source writes may\n> > not be what the user wants: Maybe it's OK for a short duration because\n> > they don't care about hard convergence too much. Maybe they want to\n> > switch to active mode already when \"only\" twice as much is written to\n> > the target as to the source.\n> > \n> > I think this is a decision the management layer (or the user) has to make.\n> \n> Eric: Does libvirt want to be involved with converging the mirror job\n> (i.e. if the guest is writing to disk faster than QEMU can copy data to\n> the target)?\n\nLibvirt doesn't really want to set the policy - it will just need to expose\nthe mechansim & information to make such decisions upto the application.\n\nI agree with Max that we don't want QEMU making this policy decision on\nits own. Simply switching from passive to active mode is not an approach\nthat will be satisfactory to all scenarios. It might be preferrable to\napply CPU throttling of the guest, or I/O rate throttling, or temporarily\npause the guest to allow catch or any number of other policies. Neither\nlibvirt or QEMU knows which is best for the scenario at hand.\n\n\nRegards,\nDaniel","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=berrange@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xxJH10G3Jz9s7B\n\tfor ;\n\tTue, 19 Sep 2017 19:58:36 +1000 (AEST)","from localhost ([::1]:41103 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duFIZ-0003mm-49\n\tfor incoming@patchwork.ozlabs.org; Tue, 19 Sep 2017 05:58:35 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:35018)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1duFID-0003jT-SH\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 05:58:16 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1duFIC-0008Qy-Dk\n\tfor qemu-devel@nongnu.org; Tue, 19 Sep 2017 05:58:13 -0400","from mx1.redhat.com ([209.132.183.28]:44800)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1duFI7-0008OD-DF; Tue, 19 Sep 2017 05:58:07 -0400","from smtp.corp.redhat.com\n\t(int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id A89C97AEBE;\n\tTue, 19 Sep 2017 09:58:05 +0000 (UTC)","from redhat.com (unknown [10.42.22.189])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id C933960BEC;\n\tTue, 19 Sep 2017 09:57:52 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com A89C97AEBE","Date":"Tue, 19 Sep 2017 10:57:50 +0100","From":"\"Daniel P. Berrange\" ","To":"Stefan Hajnoczi ","Message-ID":"<20170919095750.GG9536@redhat.com>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>\n\t\n\t<20170919094416.GC22797@stefanha-x1.localdomain>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20170919094416.GC22797@stefanha-x1.localdomain>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.12","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tTue, 19 Sep 2017 09:58:05 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"\"Daniel P. Berrange\" ","Cc":"Kevin Wolf , Fam Zheng ,\n\tqemu-block@nongnu.org, qemu-devel@nongnu.org,\n\tMax Reitz , Stefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1771997,"web_url":"http://patchwork.ozlabs.org/comment/1771997/","msgid":"<20170920145648.GH9409@stefanha-x1.localdomain>","list_archive_url":null,"date":"2017-09-20T14:56:48","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":17227,"url":"http://patchwork.ozlabs.org/api/people/17227/","name":"Stefan Hajnoczi","email":"stefanha@redhat.com"},"content":"On Tue, Sep 19, 2017 at 10:57:50AM +0100, Daniel P. Berrange wrote:\n> On Tue, Sep 19, 2017 at 10:44:16AM +0100, Stefan Hajnoczi wrote:\n> > On Mon, Sep 18, 2017 at 06:26:51PM +0200, Max Reitz wrote:\n> > > On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n> > > > On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n> > > >> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> > > >>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> > > >>>> This patch implements active synchronous mirroring. In active mode, the\n> > > >>>> passive mechanism will still be in place and is used to copy all\n> > > >>>> initially dirty clusters off the source disk; but every write request\n> > > >>>> will write data both to the source and the target disk, so the source\n> > > >>>> cannot be dirtied faster than data is mirrored to the target. Also,\n> > > >>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n> > > >>>> target are guaranteed to stay in sync (unless an error occurs).\n> > > >>>>\n> > > >>>> Optionally, dirty data can be copied to the target disk on read\n> > > >>>> operations, too.\n> > > >>>>\n> > > >>>> Active mode is completely optional and currently disabled at runtime. A\n> > > >>>> later patch will add a way for users to enable it.\n> > > >>>>\n> > > >>>> Signed-off-by: Max Reitz \n> > > >>>> ---\n> > > >>>> qapi/block-core.json | 23 +++++++\n> > > >>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> > > >>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n> > > >>>>\n> > > >>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> > > >>>> index bb11815608..e072cfa67c 100644\n> > > >>>> --- a/qapi/block-core.json\n> > > >>>> +++ b/qapi/block-core.json\n> > > >>>> @@ -938,6 +938,29 @@\n> > > >>>> 'data': ['top', 'full', 'none', 'incremental'] }\n> > > >>>> \n> > > >>>> ##\n> > > >>>> +# @MirrorCopyMode:\n> > > >>>> +#\n> > > >>>> +# An enumeration whose values tell the mirror block job when to\n> > > >>>> +# trigger writes to the target.\n> > > >>>> +#\n> > > >>>> +# @passive: copy data in background only.\n> > > >>>> +#\n> > > >>>> +# @active-write: when data is written to the source, write it\n> > > >>>> +# (synchronously) to the target as well. In addition,\n> > > >>>> +# data is copied in background just like in @passive\n> > > >>>> +# mode.\n> > > >>>> +#\n> > > >>>> +# @active-read-write: write data to the target (synchronously) both\n> > > >>>> +# when it is read from and written to the source.\n> > > >>>> +# In addition, data is copied in background just\n> > > >>>> +# like in @passive mode.\n> > > >>>\n> > > >>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> > > >>> means committing the top-most BDS while the guest is accessing it. The\n> > > >>> \"passive\" mirror block still works on the top-most BDS while the guest\n> > > >>> is accessing it.\n> > > >>>\n> > > >>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> > > >>> the terminology used in disk replication (e.g. DRBD).\n> > > >>\n> > > >> I'd be OK with that, too, but I think I remember that in the past at\n> > > >> least Kevin made a clear distinction between active/passive and\n> > > >> sync/async when it comes to mirroring.\n> > > >>\n> > > >>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n> > > >>> would switch modes as appropriate in order to converge. That way\n> > > >>> libvirt also doesn't have to worry about this.\n> > > >>\n> > > >> So here you mean async/sync in the way I meant it, i.e., whether the\n> > > >> mirror operations themselves are async/sync?\n> > > > \n> > > > The meaning I had in mind is:\n> > > > \n> > > > Sync mirroring means a guest write waits until the target write\n> > > > completes.\n> > > \n> > > I.e. active-sync, ...\n> > > \n> > > > Async mirroring means guest writes completes independently of target\n> > > > writes.\n> > > \n> > > ... i.e. passive or active-async in the future.\n> > > \n> > > So you really want qemu to decide whether to use active or passive mode\n> > > depending on what's enough to let the block job converge and not\n> > > introduce any switch for the user?\n> > > \n> > > I'm not sure whether I like this too much, mostly because \"libvirt\n> > > doesn't have to worry\" doesn't feel quite right to me. If we don't make\n> > > libvirt worry about this, then qemu has to worry. I'm not sure whether\n> > > that's any better.\n> > > \n> > > I think this really does get into policy territory. Just switching to\n> > > active mode the instant target writes are slower than source writes may\n> > > not be what the user wants: Maybe it's OK for a short duration because\n> > > they don't care about hard convergence too much. Maybe they want to\n> > > switch to active mode already when \"only\" twice as much is written to\n> > > the target as to the source.\n> > > \n> > > I think this is a decision the management layer (or the user) has to make.\n> > \n> > Eric: Does libvirt want to be involved with converging the mirror job\n> > (i.e. if the guest is writing to disk faster than QEMU can copy data to\n> > the target)?\n> \n> Libvirt doesn't really want to set the policy - it will just need to expose\n> the mechansim & information to make such decisions upto the application.\n> \n> I agree with Max that we don't want QEMU making this policy decision on\n> its own. Simply switching from passive to active mode is not an approach\n> that will be satisfactory to all scenarios. It might be preferrable to\n> apply CPU throttling of the guest, or I/O rate throttling, or temporarily\n> pause the guest to allow catch or any number of other policies. Neither\n> libvirt or QEMU knows which is best for the scenario at hand.\n\nOkay.\n\nStefan","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx04.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=stefanha@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xy3rV42Jnz9sPt\n\tfor ;\n\tThu, 21 Sep 2017 01:41:46 +1000 (AEST)","from localhost ([::1]:48989 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1duh8C-0000E6-Jn\n\tfor incoming@patchwork.ozlabs.org; Wed, 20 Sep 2017 11:41:44 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:38470)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dugRB-0002mX-8Z\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 10:57:18 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dugR6-0004zl-Cx\n\tfor qemu-devel@nongnu.org; Wed, 20 Sep 2017 10:57:17 -0400","from mx1.redhat.com ([209.132.183.28]:39194)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dugQu-0004wo-Tt; Wed, 20 Sep 2017 10:57:01 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id D8D327EAA6;\n\tWed, 20 Sep 2017 14:56:59 +0000 (UTC)","from localhost (ovpn-117-27.ams2.redhat.com [10.36.117.27])\n\tby smtp.corp.redhat.com (Postfix) with ESMTP id 0B79A5EDE4;\n\tWed, 20 Sep 2017 14:56:49 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com D8D327EAA6","Date":"Wed, 20 Sep 2017 15:56:48 +0100","From":"Stefan Hajnoczi ","To":"\"Daniel P. Berrange\" ","Message-ID":"<20170920145648.GH9409@stefanha-x1.localdomain>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>\n\t\n\t<20170919094416.GC22797@stefanha-x1.localdomain>\n\t<20170919095750.GG9536@redhat.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=us-ascii","Content-Disposition":"inline","In-Reply-To":"<20170919095750.GG9536@redhat.com>","User-Agent":"Mutt/1.8.3 (2017-05-23)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.28]);\n\tWed, 20 Sep 2017 14:57:00 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , Fam Zheng ,\n\tqemu-block@nongnu.org, Stefan Hajnoczi ,\n\tqemu-devel@nongnu.org, Max Reitz ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1783585,"web_url":"http://patchwork.ozlabs.org/comment/1783585/","msgid":"<20171010101622.GH4177@dhcp-200-186.str.redhat.com>","list_archive_url":null,"date":"2017-10-10T10:16:22","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":2714,"url":"http://patchwork.ozlabs.org/api/people/2714/","name":"Kevin Wolf","email":"kwolf@redhat.com"},"content":"Am 18.09.2017 um 18:26 hat Max Reitz geschrieben:\n> On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n> > On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n> >> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n> >>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n> >>>> This patch implements active synchronous mirroring. In active mode, the\n> >>>> passive mechanism will still be in place and is used to copy all\n> >>>> initially dirty clusters off the source disk; but every write request\n> >>>> will write data both to the source and the target disk, so the source\n> >>>> cannot be dirtied faster than data is mirrored to the target. Also,\n> >>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n> >>>> target are guaranteed to stay in sync (unless an error occurs).\n> >>>>\n> >>>> Optionally, dirty data can be copied to the target disk on read\n> >>>> operations, too.\n> >>>>\n> >>>> Active mode is completely optional and currently disabled at runtime. A\n> >>>> later patch will add a way for users to enable it.\n> >>>>\n> >>>> Signed-off-by: Max Reitz \n> >>>> ---\n> >>>> qapi/block-core.json | 23 +++++++\n> >>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n> >>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n> >>>>\n> >>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n> >>>> index bb11815608..e072cfa67c 100644\n> >>>> --- a/qapi/block-core.json\n> >>>> +++ b/qapi/block-core.json\n> >>>> @@ -938,6 +938,29 @@\n> >>>> 'data': ['top', 'full', 'none', 'incremental'] }\n> >>>> \n> >>>> ##\n> >>>> +# @MirrorCopyMode:\n> >>>> +#\n> >>>> +# An enumeration whose values tell the mirror block job when to\n> >>>> +# trigger writes to the target.\n> >>>> +#\n> >>>> +# @passive: copy data in background only.\n> >>>> +#\n> >>>> +# @active-write: when data is written to the source, write it\n> >>>> +# (synchronously) to the target as well. In addition,\n> >>>> +# data is copied in background just like in @passive\n> >>>> +# mode.\n> >>>> +#\n> >>>> +# @active-read-write: write data to the target (synchronously) both\n> >>>> +# when it is read from and written to the source.\n> >>>> +# In addition, data is copied in background just\n> >>>> +# like in @passive mode.\n> >>>\n> >>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n> >>> means committing the top-most BDS while the guest is accessing it. The\n> >>> \"passive\" mirror block still works on the top-most BDS while the guest\n> >>> is accessing it.\n> >>>\n> >>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n> >>> the terminology used in disk replication (e.g. DRBD).\n> >>\n> >> I'd be OK with that, too, but I think I remember that in the past at\n> >> least Kevin made a clear distinction between active/passive and\n> >> sync/async when it comes to mirroring.\n> >>\n> >>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n> >>> would switch modes as appropriate in order to converge. That way\n> >>> libvirt also doesn't have to worry about this.\n> >>\n> >> So here you mean async/sync in the way I meant it, i.e., whether the\n> >> mirror operations themselves are async/sync?\n> > \n> > The meaning I had in mind is:\n> > \n> > Sync mirroring means a guest write waits until the target write\n> > completes.\n> \n> I.e. active-sync, ...\n> \n> > Async mirroring means guest writes completes independently of target\n> > writes.\n> \n> ... i.e. passive or active-async in the future.\n\nSo we already have at least three different modes, sync/async doesn't\nquite cut it anyway. There's a reason why we have been talking about\nboth active/passive and sync/async.\n\nWhen I was looking at the code, it actually occurred to me that there\nare more possible different modes than I thought there were: This patch\nwaits for successful completion on the source before it even attempts to\nwrite to the destination.\n\nWouldn't it be generally (i.e. in the success case) more useful if we\nstart both requests at the same time and only wait for both to complete,\navoiding to double the latency? If the source write fails, we're out of\nsync, obviously, so we'd have to mark the block dirty again.\n\nBy the way, what happens when the guest modifies the RAM during the\nrequest? Is it acceptable even for writes if source and target differ\nafter a successful write operation? Don't we need a bounce buffer\nanyway?\n\nKevin","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx08.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=kwolf@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBChf67Lhz9tY3\n\tfor ;\n\tTue, 10 Oct 2017 21:17:06 +1100 (AEDT)","from localhost ([::1]:33959 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e1raz-00089m-0d\n\tfor incoming@patchwork.ozlabs.org; Tue, 10 Oct 2017 06:17:05 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:54922)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e1rac-00087W-HI\n\tfor qemu-devel@nongnu.org; Tue, 10 Oct 2017 06:16:47 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e1raY-0003vP-61\n\tfor qemu-devel@nongnu.org; Tue, 10 Oct 2017 06:16:42 -0400","from mx1.redhat.com ([209.132.183.28]:36472)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1e1raQ-0003ox-OZ; Tue, 10 Oct 2017 06:16:30 -0400","from smtp.corp.redhat.com\n\t(int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id B80D2C0587C9;\n\tTue, 10 Oct 2017 10:16:29 +0000 (UTC)","from dhcp-200-186.str.redhat.com (dhcp-200-186.str.redhat.com\n\t[10.33.200.186])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id F1C3E6248E;\n\tTue, 10 Oct 2017 10:16:23 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com B80D2C0587C9","Date":"Tue, 10 Oct 2017 12:16:22 +0200","From":"Kevin Wolf ","To":"Max Reitz ","Message-ID":"<20171010101622.GH4177@dhcp-200-186.str.redhat.com>","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>\n\t","MIME-Version":"1.0","Content-Type":"multipart/signed; micalg=pgp-sha1;\n\tprotocol=\"application/pgp-signature\"; boundary=\"wac7ysb48OaltWcw\"","Content-Disposition":"inline","In-Reply-To":"","User-Agent":"Mutt/1.9.1 (2017-09-22)","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.15","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.32]);\n\tTue, 10 Oct 2017 10:16:29 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Stefan Hajnoczi , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}},{"id":1784575,"web_url":"http://patchwork.ozlabs.org/comment/1784575/","msgid":"<2def1cd1-e1ca-772f-b026-235bae9bfd9d@redhat.com>","list_archive_url":null,"date":"2017-10-11T12:33:45","subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/","name":"Max Reitz","email":"mreitz@redhat.com"},"content":"On 2017-10-10 12:16, Kevin Wolf wrote:\n> Am 18.09.2017 um 18:26 hat Max Reitz geschrieben:\n>> On 2017-09-18 12:06, Stefan Hajnoczi wrote:\n>>> On Sat, Sep 16, 2017 at 03:58:01PM +0200, Max Reitz wrote:\n>>>> On 2017-09-14 17:57, Stefan Hajnoczi wrote:\n>>>>> On Wed, Sep 13, 2017 at 08:19:07PM +0200, Max Reitz wrote:\n>>>>>> This patch implements active synchronous mirroring. In active mode, the\n>>>>>> passive mechanism will still be in place and is used to copy all\n>>>>>> initially dirty clusters off the source disk; but every write request\n>>>>>> will write data both to the source and the target disk, so the source\n>>>>>> cannot be dirtied faster than data is mirrored to the target. Also,\n>>>>>> once the block job has converged (BLOCK_JOB_READY sent), source and\n>>>>>> target are guaranteed to stay in sync (unless an error occurs).\n>>>>>>\n>>>>>> Optionally, dirty data can be copied to the target disk on read\n>>>>>> operations, too.\n>>>>>>\n>>>>>> Active mode is completely optional and currently disabled at runtime. A\n>>>>>> later patch will add a way for users to enable it.\n>>>>>>\n>>>>>> Signed-off-by: Max Reitz \n>>>>>> ---\n>>>>>> qapi/block-core.json | 23 +++++++\n>>>>>> block/mirror.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++++--\n>>>>>> 2 files changed, 205 insertions(+), 5 deletions(-)\n>>>>>>\n>>>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json\n>>>>>> index bb11815608..e072cfa67c 100644\n>>>>>> --- a/qapi/block-core.json\n>>>>>> +++ b/qapi/block-core.json\n>>>>>> @@ -938,6 +938,29 @@\n>>>>>> 'data': ['top', 'full', 'none', 'incremental'] }\n>>>>>> \n>>>>>> ##\n>>>>>> +# @MirrorCopyMode:\n>>>>>> +#\n>>>>>> +# An enumeration whose values tell the mirror block job when to\n>>>>>> +# trigger writes to the target.\n>>>>>> +#\n>>>>>> +# @passive: copy data in background only.\n>>>>>> +#\n>>>>>> +# @active-write: when data is written to the source, write it\n>>>>>> +# (synchronously) to the target as well. In addition,\n>>>>>> +# data is copied in background just like in @passive\n>>>>>> +# mode.\n>>>>>> +#\n>>>>>> +# @active-read-write: write data to the target (synchronously) both\n>>>>>> +# when it is read from and written to the source.\n>>>>>> +# In addition, data is copied in background just\n>>>>>> +# like in @passive mode.\n>>>>>\n>>>>> I'm not sure the terms \"active\"/\"passive\" are helpful. \"Active commit\"\n>>>>> means committing the top-most BDS while the guest is accessing it. The\n>>>>> \"passive\" mirror block still works on the top-most BDS while the guest\n>>>>> is accessing it.\n>>>>>\n>>>>> Calling it \"asynchronous\" and \"synchronous\" is clearer to me. It's also\n>>>>> the terminology used in disk replication (e.g. DRBD).\n>>>>\n>>>> I'd be OK with that, too, but I think I remember that in the past at\n>>>> least Kevin made a clear distinction between active/passive and\n>>>> sync/async when it comes to mirroring.\n>>>>\n>>>>> Ideally the user wouldn't have to worry about async vs sync because QEMU\n>>>>> would switch modes as appropriate in order to converge. That way\n>>>>> libvirt also doesn't have to worry about this.\n>>>>\n>>>> So here you mean async/sync in the way I meant it, i.e., whether the\n>>>> mirror operations themselves are async/sync?\n>>>\n>>> The meaning I had in mind is:\n>>>\n>>> Sync mirroring means a guest write waits until the target write\n>>> completes.\n>>\n>> I.e. active-sync, ...\n>>\n>>> Async mirroring means guest writes completes independently of target\n>>> writes.\n>>\n>> ... i.e. passive or active-async in the future.\n> \n> So we already have at least three different modes, sync/async doesn't\n> quite cut it anyway. There's a reason why we have been talking about\n> both active/passive and sync/async.\n> \n> When I was looking at the code, it actually occurred to me that there\n> are more possible different modes than I thought there were: This patch\n> waits for successful completion on the source before it even attempts to\n> write to the destination.\n> \n> Wouldn't it be generally (i.e. in the success case) more useful if we\n> start both requests at the same time and only wait for both to complete,\n> avoiding to double the latency? If the source write fails, we're out of\n> sync, obviously, so we'd have to mark the block dirty again.\n\nI've thought about it, but my issues were:\n\n(1) What to do when something fails\nand\n(2) I didn't really want to start coroutines from coroutines...\n\nAs for (1)... My notes actually say I've come to a conclusion: If the\ntarget write fails, that's pretty much OK, because then the source is\nnewer than the target, which is normal for mirroring. If the source\nwrite fails, we can just consider the target outdated, too (as you've\nsaid). Also, we'll give an error to the guest, so it's clear that\nsomething has gone wrong.\n\nSo (2) was the reason I didn't do it in this series. I think it's OK to\nadd this later on and let future me worry about how to coordinate both\nrequests.\n\nI guess I'd start e.g. the target operation as a new coroutine, then\ncontinue the source operation in the original one, and finally yield\nuntil the target operation has finished?\n\n> By the way, what happens when the guest modifies the RAM during the\n> request? Is it acceptable even for writes if source and target differ\n> after a successful write operation? Don't we need a bounce buffer\n> anyway?\n\nSometimes I think that maybe I shouldn't keep my thoughts to myself\nafter I've come to the conclusion \"...naah, it's all bad anyway\". :-)\n\nWhen Stefan mentioned this for reads, I thought about the write\nsituation, yes. My conclusion was that the guest would be required (by\nprotocol) to keep the write buffer constant while the operation is\nrunning, because otherwise the guest has no idea what is going to be on\ndisk. So it would be stupid for the guest to modify the write buffer then.\n\nBut (1) depending on the emulated hardware, maybe the guest does have an\nidea (e.g. some register that tells the guest which offset is currently\nwritten) -- but with the structure of the block layer, I doubt that's\npossible in qemu,\n\nand (2) maybe the guest wants to be stupid. Even if the guest doesn't\nknow what will end up on disk, we have to make sure that it's the same\non both source and target.\n\nSo, yeah, a bounce buffer would be good in all cases.\n\nMax","headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx01.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3yBthw5Zvkz9s7p\n\tfor ;\n\tWed, 11 Oct 2017 23:34:39 +1100 (AEDT)","from localhost ([::1]:40837 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1e2GDc-0002Vb-FQ\n\tfor incoming@patchwork.ozlabs.org; Wed, 11 Oct 2017 08:34:36 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:42123)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1e2GDD-0002VJ-HB\n\tfor qemu-devel@nongnu.org; Wed, 11 Oct 2017 08:34:17 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1e2GD8-00016d-IK\n\tfor qemu-devel@nongnu.org; Wed, 11 Oct 2017 08:34:11 -0400","from mx1.redhat.com ([209.132.183.28]:46640)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1e2GCw-00010r-M3; Wed, 11 Oct 2017 08:33:54 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id A89A881241;\n\tWed, 11 Oct 2017 12:33:53 +0000 (UTC)","from dresden.str.redhat.com (unknown [10.40.205.86])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 281705C8AF;\n\tWed, 11 Oct 2017 12:33:47 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com A89A881241","To":"Kevin Wolf ","References":"<20170913181910.29688-1-mreitz@redhat.com>\n\t<20170913181910.29688-16-mreitz@redhat.com>\n\t<20170914155738.GE7370@stefanha-x1.localdomain>\n\t<2f1c5239-0cde-d164-b803-ebf807e684f2@redhat.com>\n\t<20170918100626.GE31063@stefanha-x1.localdomain>\n\t\n\t<20171010101622.GH4177@dhcp-200-186.str.redhat.com>","From":"Max Reitz ","Message-ID":"<2def1cd1-e1ca-772f-b026-235bae9bfd9d@redhat.com>","Date":"Wed, 11 Oct 2017 14:33:45 +0200","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101\n\tThunderbird/52.3.0","MIME-Version":"1.0","In-Reply-To":"<20171010101622.GH4177@dhcp-200-186.str.redhat.com>","Content-Type":"multipart/signed; micalg=pgp-sha256;\n\tprotocol=\"application/pgp-signature\";\n\tboundary=\"hoL324xwiURJS57NhtpIFqjsVPrWHWAO9\"","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.25]);\n\tWed, 11 Oct 2017 12:33:53 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","X-Content-Filtered-By":"Mailman/MimeDel 2.1.21","Subject":"Re: [Qemu-devel] [Qemu-block] [PATCH 15/18] block/mirror: Add\n\tactive mirroring","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Stefan Hajnoczi , qemu-block@nongnu.org,\n\tFam Zheng , qemu-devel@nongnu.org,\n\tStefan Hajnoczi ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"}}]