{"id":813559,"url":"http://patchwork.ozlabs.org/api/patches/813559/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170913181910.29688-8-mreitz@redhat.com/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<20170913181910.29688-8-mreitz@redhat.com>","list_archive_url":null,"date":"2017-09-13T18:18:59","name":"[07/18] block/mirror: Wait for in-flight op conflicts","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"f4ceba9ac4e9a73fdef818243ce337d3f2c5764f","submitter":{"id":36836,"url":"http://patchwork.ozlabs.org/api/people/36836/?format=json","name":"Max Reitz","email":"mreitz@redhat.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/20170913181910.29688-8-mreitz@redhat.com/mbox/","series":[{"id":2960,"url":"http://patchwork.ozlabs.org/api/series/2960/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=2960","date":"2017-09-13T18:18:52","name":"block/mirror: Add active-sync mirroring","version":1,"mbox":"http://patchwork.ozlabs.org/series/2960/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/813559/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/813559/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n\tspf=pass (mailfrom) smtp.mailfrom=nongnu.org\n\t(client-ip=2001:4830:134:3::11; helo=lists.gnu.org;\n\tenvelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n\treceiver=)","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tdmarc=none (p=none dis=none) header.from=redhat.com","ext-mx02.extmail.prod.ext.phx2.redhat.com;\n\tspf=fail smtp.mailfrom=mreitz@redhat.com"],"Received":["from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11])\n\t(using TLSv1 with cipher AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 3xsqmx5yFrz9s7g\n\tfor ;\n\tThu, 14 Sep 2017 04:24:01 +1000 (AEST)","from localhost ([::1]:44007 helo=lists.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.71) (envelope-from\n\t)\n\tid 1dsCKN-00027m-QJ\n\tfor incoming@patchwork.ozlabs.org; Wed, 13 Sep 2017 14:23:59 -0400","from eggs.gnu.org ([2001:4830:134:3::10]:36893)\n\tby lists.gnu.org with esmtp (Exim 4.71)\n\t(envelope-from ) id 1dsCH7-0008VD-87\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:38 -0400","from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)\n\t(envelope-from ) id 1dsCH5-0005Ma-EV\n\tfor qemu-devel@nongnu.org; Wed, 13 Sep 2017 14:20:37 -0400","from mx1.redhat.com ([209.132.183.28]:46920)\n\tby eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)\n\t(Exim 4.71) (envelope-from )\n\tid 1dsCGy-0005Gc-JH; Wed, 13 Sep 2017 14:20:28 -0400","from smtp.corp.redhat.com\n\t(int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16])\n\t(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))\n\t(No client certificate requested)\n\tby mx1.redhat.com (Postfix) with ESMTPS id 9D6E5D556B;\n\tWed, 13 Sep 2017 18:20:27 +0000 (UTC)","from localhost (ovpn-204-23.brq.redhat.com [10.40.204.23])\n\tby smtp.corp.redhat.com (Postfix) with ESMTPS id 26B854D9E6;\n\tWed, 13 Sep 2017 18:20:19 +0000 (UTC)"],"DMARC-Filter":"OpenDMARC Filter v1.3.2 mx1.redhat.com 9D6E5D556B","From":"Max Reitz ","To":"qemu-block@nongnu.org","Date":"Wed, 13 Sep 2017 20:18:59 +0200","Message-Id":"<20170913181910.29688-8-mreitz@redhat.com>","In-Reply-To":"<20170913181910.29688-1-mreitz@redhat.com>","References":"<20170913181910.29688-1-mreitz@redhat.com>","X-Scanned-By":"MIMEDefang 2.79 on 10.5.11.16","X-Greylist":"Sender IP whitelisted, not delayed by milter-greylist-4.5.16\n\t(mx1.redhat.com [10.5.110.26]);\n\tWed, 13 Sep 2017 18:20:27 +0000 (UTC)","X-detected-operating-system":"by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]\n\t[fuzzy]","X-Received-From":"209.132.183.28","Subject":"[Qemu-devel] [PATCH 07/18] block/mirror: Wait for in-flight op\n\tconflicts","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.21","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"Kevin Wolf , Fam Zheng ,\n\tqemu-devel@nongnu.org, Max Reitz ,\n\tStefan Hajnoczi , John Snow ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n\t"},"content":"This patch makes the mirror code differentiate between simply waiting\nfor any operation to complete (mirror_wait_for_free_in_flight_slot())\nand specifically waiting for all operations touching a certain range of\nthe virtual disk to complete (mirror_wait_on_conflicts()).\n\nSigned-off-by: Max Reitz \n---\n block/mirror.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++-----------\n 1 file changed, 70 insertions(+), 15 deletions(-)","diff":"diff --git a/block/mirror.c b/block/mirror.c\nindex 81253fbad1..2ece38094d 100644\n--- a/block/mirror.c\n+++ b/block/mirror.c\n@@ -14,6 +14,7 @@\n #include \"qemu/osdep.h\"\n #include \"qemu/cutils.h\"\n #include \"qemu/coroutine.h\"\n+#include \"qemu/range.h\"\n #include \"trace.h\"\n #include \"block/blockjob_int.h\"\n #include \"block/block_int.h\"\n@@ -111,6 +112,41 @@ static BlockErrorAction mirror_error_action(MirrorBlockJob *s, bool read,\n }\n }\n \n+static void coroutine_fn mirror_wait_on_conflicts(MirrorOp *self,\n+ MirrorBlockJob *s,\n+ uint64_t offset,\n+ uint64_t bytes)\n+{\n+ uint64_t self_start_chunk = offset / s->granularity;\n+ uint64_t self_end_chunk = DIV_ROUND_UP(offset + bytes, s->granularity);\n+ uint64_t self_nb_chunks = self_end_chunk - self_start_chunk;\n+\n+ while (find_next_bit(s->in_flight_bitmap, self_end_chunk,\n+ self_start_chunk) < self_end_chunk &&\n+ s->ret >= 0)\n+ {\n+ MirrorOp *op;\n+\n+ QTAILQ_FOREACH(op, &s->ops_in_flight, next) {\n+ uint64_t op_start_chunk = op->offset / s->granularity;\n+ uint64_t op_nb_chunks = DIV_ROUND_UP(op->offset + op->bytes,\n+ s->granularity) -\n+ op_start_chunk;\n+\n+ if (op == self) {\n+ continue;\n+ }\n+\n+ if (ranges_overlap(self_start_chunk, self_nb_chunks,\n+ op_start_chunk, op_nb_chunks))\n+ {\n+ qemu_co_queue_wait(&op->waiting_requests, NULL);\n+ break;\n+ }\n+ }\n+ }\n+}\n+\n static void coroutine_fn mirror_iteration_done(MirrorOp *op, int ret)\n {\n MirrorBlockJob *s = op->s;\n@@ -238,7 +274,7 @@ static int mirror_cow_align(MirrorBlockJob *s, int64_t *offset,\n return ret;\n }\n \n-static inline void mirror_wait_for_io(MirrorBlockJob *s)\n+static inline void mirror_wait_for_free_in_flight_slot(MirrorBlockJob *s)\n {\n MirrorOp *op;\n \n@@ -287,7 +323,7 @@ static void coroutine_fn mirror_co_read(void *opaque)\n \n while (s->buf_free_count < nb_chunks) {\n trace_mirror_yield_in_flight(s, op->offset, s->in_flight);\n- mirror_wait_for_io(s);\n+ mirror_wait_for_free_in_flight_slot(s);\n }\n \n /* Now make a QEMUIOVector taking enough granularity-sized chunks\n@@ -381,8 +417,9 @@ static unsigned mirror_perform(MirrorBlockJob *s, int64_t offset,\n static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)\n {\n BlockDriverState *source = s->source;\n- int64_t offset, first_chunk;\n- uint64_t delay_ns = 0;\n+ MirrorOp *pseudo_op;\n+ int64_t offset;\n+ uint64_t delay_ns = 0, ret = 0;\n /* At least the first dirty chunk is mirrored in one iteration. */\n int nb_chunks = 1;\n int sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS;\n@@ -400,11 +437,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)\n }\n bdrv_dirty_bitmap_unlock(s->dirty_bitmap);\n \n- first_chunk = offset / s->granularity;\n- while (test_bit(first_chunk, s->in_flight_bitmap)) {\n- trace_mirror_yield_in_flight(s, offset, s->in_flight);\n- mirror_wait_for_io(s);\n- }\n+ mirror_wait_on_conflicts(NULL, s, offset, 1);\n \n block_job_pause_point(&s->common);\n \n@@ -442,6 +475,20 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)\n nb_chunks * sectors_per_chunk);\n bdrv_dirty_bitmap_unlock(s->dirty_bitmap);\n \n+ /* Before claiming an area in the in-flight bitmap, we have to\n+ * create a MirrorOp for it so that conflicting requests can wait\n+ * for it. mirror_perform() will create the real MirrorOps later,\n+ * for now we just create a pseudo operation that will wake up all\n+ * conflicting requests once all real operations have been\n+ * launched. */\n+ pseudo_op = g_new(MirrorOp, 1);\n+ *pseudo_op = (MirrorOp){\n+ .offset = offset,\n+ .bytes = nb_chunks * s->granularity,\n+ };\n+ qemu_co_queue_init(&pseudo_op->waiting_requests);\n+ QTAILQ_INSERT_TAIL(&s->ops_in_flight, pseudo_op, next);\n+\n bitmap_set(s->in_flight_bitmap, offset / s->granularity, nb_chunks);\n while (nb_chunks > 0 && offset < s->bdev_length) {\n int64_t ret;\n@@ -481,11 +528,12 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)\n \n while (s->in_flight >= MAX_IN_FLIGHT) {\n trace_mirror_yield_in_flight(s, offset, s->in_flight);\n- mirror_wait_for_io(s);\n+ mirror_wait_for_free_in_flight_slot(s);\n }\n \n if (s->ret < 0) {\n- return 0;\n+ ret = 0;\n+ goto fail;\n }\n \n io_bytes = mirror_clip_bytes(s, offset, io_bytes);\n@@ -502,7 +550,14 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)\n delay_ns = ratelimit_calculate_delay(&s->limit, io_bytes_acct);\n }\n }\n- return delay_ns;\n+\n+ ret = delay_ns;\n+fail:\n+ QTAILQ_REMOVE(&s->ops_in_flight, pseudo_op, next);\n+ qemu_co_queue_restart_all(&pseudo_op->waiting_requests);\n+ g_free(pseudo_op);\n+\n+ return ret;\n }\n \n static void mirror_free_init(MirrorBlockJob *s)\n@@ -529,7 +584,7 @@ static void mirror_free_init(MirrorBlockJob *s)\n static void mirror_wait_for_all_io(MirrorBlockJob *s)\n {\n while (s->in_flight > 0) {\n- mirror_wait_for_io(s);\n+ mirror_wait_for_free_in_flight_slot(s);\n }\n }\n \n@@ -685,7 +740,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)\n if (s->in_flight >= MAX_IN_FLIGHT) {\n trace_mirror_yield(s, UINT64_MAX, s->buf_free_count,\n s->in_flight);\n- mirror_wait_for_io(s);\n+ mirror_wait_for_free_in_flight_slot(s);\n continue;\n }\n \n@@ -868,7 +923,7 @@ static void coroutine_fn mirror_run(void *opaque)\n (cnt == 0 && s->in_flight > 0)) {\n trace_mirror_yield(s, cnt * BDRV_SECTOR_SIZE,\n s->buf_free_count, s->in_flight);\n- mirror_wait_for_io(s);\n+ mirror_wait_for_free_in_flight_slot(s);\n continue;\n } else if (cnt != 0) {\n delay_ns = mirror_iteration(s);\n","prefixes":["07/18"]}