{"id":1307008,"url":"http://patchwork.ozlabs.org/api/patches/1307008/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/patch/2a4890c1157da363c419a8c90475dbe192cd9933.1591801197.git.berto@igalia.com/","project":{"id":14,"url":"http://patchwork.ozlabs.org/api/projects/14/?format=json","name":"QEMU Development","link_name":"qemu-devel","list_id":"qemu-devel.nongnu.org","list_email":"qemu-devel@nongnu.org","web_url":"","scm_url":"","webscm_url":"","list_archive_url":"","list_archive_url_format":"","commit_url_format":""},"msgid":"<2a4890c1157da363c419a8c90475dbe192cd9933.1591801197.git.berto@igalia.com>","list_archive_url":null,"date":"2020-06-10T15:02:39","name":"[v8,01/34] qcow2: Make Qcow2AioTask store the full host offset","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"103be3a4d0627dade759ee248208eef9d4946c98","submitter":{"id":65704,"url":"http://patchwork.ozlabs.org/api/people/65704/?format=json","name":"Alberto Garcia","email":"berto@igalia.com"},"delegate":null,"mbox":"http://patchwork.ozlabs.org/project/qemu-devel/patch/2a4890c1157da363c419a8c90475dbe192cd9933.1591801197.git.berto@igalia.com/mbox/","series":[{"id":182563,"url":"http://patchwork.ozlabs.org/api/series/182563/?format=json","web_url":"http://patchwork.ozlabs.org/project/qemu-devel/list/?series=182563","date":"2020-06-10T15:02:49","name":"Add subcluster allocation to qcow2","version":8,"mbox":"http://patchwork.ozlabs.org/series/182563/mbox/"}],"comments":"http://patchwork.ozlabs.org/api/patches/1307008/comments/","check":"pending","checks":"http://patchwork.ozlabs.org/api/patches/1307008/checks/","tags":{},"related":[],"headers":{"Return-Path":"","X-Original-To":"incoming@patchwork.ozlabs.org","Delivered-To":"patchwork-incoming@bilbo.ozlabs.org","Authentication-Results":["ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org\n (client-ip=209.51.188.17; helo=lists.gnu.org;\n envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org;\n receiver=)","ozlabs.org;\n dmarc=none (p=none dis=none) header.from=igalia.com","ozlabs.org;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n unprotected) header.d=igalia.com header.i=@igalia.com header.a=rsa-sha256\n header.s=20170329 header.b=VU0f95uC;\n\tdkim-atps=neutral"],"Received":["from lists.gnu.org (lists.gnu.org [209.51.188.17])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 49hr4y1Rpjz9sQx\n\tfor ; Thu, 11 Jun 2020 01:09:54 +1000 (AEST)","from localhost ([::1]:37716 helo=lists1p.gnu.org)\n\tby lists.gnu.org with esmtp (Exim 4.90_1)\n\t(envelope-from )\n\tid 1jj2MR-0001o9-QW\n\tfor incoming@patchwork.ozlabs.org; Wed, 10 Jun 2020 11:09:51 -0400","from eggs.gnu.org ([2001:470:142:3::10]:35964)\n by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.90_1) (envelope-from )\n id 1jj2Gd-0002K5-7P; Wed, 10 Jun 2020 11:03:51 -0400","from fanzine.igalia.com ([178.60.130.6]:58020)\n by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)\n (Exim 4.90_1) (envelope-from )\n id 1jj2GX-0006vq-IN; Wed, 10 Jun 2020 11:03:50 -0400","from [81.0.38.199] (helo=perseus.local)\n by fanzine.igalia.com with esmtpsa\n (Cipher TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim)\n id 1jj2GQ-0007gc-Lp; Wed, 10 Jun 2020 17:03:38 +0200","from berto by perseus.local with local (Exim 4.92)\n (envelope-from )\n id 1jj2GB-0007MU-LX; Wed, 10 Jun 2020 17:03:23 +0200"],"DKIM-Signature":"v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com;\n s=20170329;\n h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From;\n bh=KVge/m/XS0/yC+nhx3y/fBN7slj1cup5sV6LZnFJwfc=;\n b=VU0f95uCYpFcv+d4NWTjOyyFsWXBeYVpUxH7TpQXGRQEkp6Ytt096ozZbr1CDVl+uNMO9hhPaCh4WSGh5WwmQbQDEycllq8ER6JQyNRH+0MZazmO0ylIIh6kqJBuDchOZZAewOkjDJqOIKe3Y6TbPTlSChfQxdTvuYAE+Hqqh8SkiNXavxC70F2aRQx88FxgFU1qWsTlw7X1A+mIMkqI9NGHobU4BZa6oZFMQwGDOFXlG3UXhjFvwe1bbaCaD5SFxiE6/ZMPxRyEGY9OAmotSeMExUOhDu/yc+jv7UiUCkNROLC9eUl98pxLCOF66yTwqiDvMPgNUPfNsC8PizQd8A==;","From":"Alberto Garcia ","To":"qemu-devel@nongnu.org","Subject":"[PATCH v8 01/34] qcow2: Make Qcow2AioTask store the full host offset","Date":"Wed, 10 Jun 2020 17:02:39 +0200","Message-Id":"\n <2a4890c1157da363c419a8c90475dbe192cd9933.1591801197.git.berto@igalia.com>","X-Mailer":"git-send-email 2.20.1","In-Reply-To":"","References":"","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Received-SPF":"pass client-ip=178.60.130.6; envelope-from=berto@igalia.com;\n helo=fanzine.igalia.com","X-detected-operating-system":"by eggs.gnu.org: First seen = 2020/06/10 11:03:39","X-ACL-Warn":"Detected OS = Linux 2.2.x-3.x (no timestamps) [generic] [fuzzy]","X-Spam_score_int":"-20","X-Spam_score":"-2.1","X-Spam_bar":"--","X-Spam_report":"(-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,\n DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001,\n SPF_PASS=-0.001 autolearn=_AUTOLEARN","X-Spam_action":"no action","X-BeenThere":"qemu-devel@nongnu.org","X-Mailman-Version":"2.1.23","Precedence":"list","List-Id":"","List-Unsubscribe":",\n ","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n ","Cc":"Kevin Wolf ,\n Vladimir Sementsov-Ogievskiy ,\n Alberto Garcia , qemu-block@nongnu.org,\n Derek Su , Max Reitz ","Errors-To":"qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org","Sender":"\"Qemu-devel\"\n "},"content":"The file_cluster_offset field of Qcow2AioTask stores a cluster-aligned\nhost offset. In practice this is not very useful because all users(*)\nof this structure need the final host offset into the cluster, which\nthey calculate using\n\n host_offset = file_cluster_offset + offset_into_cluster(s, offset)\n\nThere is no reason why Qcow2AioTask cannot store host_offset directly\nand that is what this patch does.\n\n(*) compressed clusters are the exception: in this case what\n file_cluster_offset was storing was the full compressed cluster\n descriptor (offset + size). This does not change with this patch\n but it is documented now.\n\nSigned-off-by: Alberto Garcia \nReviewed-by: Eric Blake \nReviewed-by: Max Reitz \nReviewed-by: Vladimir Sementsov-Ogievskiy \n---\n block/qcow2.c | 69 ++++++++++++++++++++++------------------------\n block/trace-events | 2 +-\n 2 files changed, 34 insertions(+), 37 deletions(-)","diff":"diff --git a/block/qcow2.c b/block/qcow2.c\nindex e20590c3b7..d792137af6 100644\n--- a/block/qcow2.c\n+++ b/block/qcow2.c\n@@ -74,7 +74,7 @@ typedef struct {\n \n static int coroutine_fn\n qcow2_co_preadv_compressed(BlockDriverState *bs,\n- uint64_t file_cluster_offset,\n+ uint64_t cluster_descriptor,\n uint64_t offset,\n uint64_t bytes,\n QEMUIOVector *qiov,\n@@ -2103,7 +2103,7 @@ out:\n \n static coroutine_fn int\n qcow2_co_preadv_encrypted(BlockDriverState *bs,\n- uint64_t file_cluster_offset,\n+ uint64_t host_offset,\n uint64_t offset,\n uint64_t bytes,\n QEMUIOVector *qiov,\n@@ -2130,16 +2130,12 @@ qcow2_co_preadv_encrypted(BlockDriverState *bs,\n }\n \n BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO);\n- ret = bdrv_co_pread(s->data_file,\n- file_cluster_offset + offset_into_cluster(s, offset),\n- bytes, buf, 0);\n+ ret = bdrv_co_pread(s->data_file, host_offset, bytes, buf, 0);\n if (ret < 0) {\n goto fail;\n }\n \n- if (qcow2_co_decrypt(bs,\n- file_cluster_offset + offset_into_cluster(s, offset),\n- offset, buf, bytes) < 0)\n+ if (qcow2_co_decrypt(bs, host_offset, offset, buf, bytes) < 0)\n {\n ret = -EIO;\n goto fail;\n@@ -2157,7 +2153,7 @@ typedef struct Qcow2AioTask {\n \n BlockDriverState *bs;\n QCow2ClusterType cluster_type; /* only for read */\n- uint64_t file_cluster_offset;\n+ uint64_t host_offset; /* or full descriptor in compressed clusters */\n uint64_t offset;\n uint64_t bytes;\n QEMUIOVector *qiov;\n@@ -2170,7 +2166,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,\n AioTaskPool *pool,\n AioTaskFunc func,\n QCow2ClusterType cluster_type,\n- uint64_t file_cluster_offset,\n+ uint64_t host_offset,\n uint64_t offset,\n uint64_t bytes,\n QEMUIOVector *qiov,\n@@ -2185,7 +2181,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,\n .bs = bs,\n .cluster_type = cluster_type,\n .qiov = qiov,\n- .file_cluster_offset = file_cluster_offset,\n+ .host_offset = host_offset,\n .offset = offset,\n .bytes = bytes,\n .qiov_offset = qiov_offset,\n@@ -2194,7 +2190,7 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,\n \n trace_qcow2_add_task(qemu_coroutine_self(), bs, pool,\n func == qcow2_co_preadv_task_entry ? \"read\" : \"write\",\n- cluster_type, file_cluster_offset, offset, bytes,\n+ cluster_type, host_offset, offset, bytes,\n qiov, qiov_offset);\n \n if (!pool) {\n@@ -2208,13 +2204,12 @@ static coroutine_fn int qcow2_add_task(BlockDriverState *bs,\n \n static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,\n QCow2ClusterType cluster_type,\n- uint64_t file_cluster_offset,\n+ uint64_t host_offset,\n uint64_t offset, uint64_t bytes,\n QEMUIOVector *qiov,\n size_t qiov_offset)\n {\n BDRVQcow2State *s = bs->opaque;\n- int offset_in_cluster = offset_into_cluster(s, offset);\n \n switch (cluster_type) {\n case QCOW2_CLUSTER_ZERO_PLAIN:\n@@ -2230,19 +2225,17 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs,\n qiov, qiov_offset, 0);\n \n case QCOW2_CLUSTER_COMPRESSED:\n- return qcow2_co_preadv_compressed(bs, file_cluster_offset,\n+ return qcow2_co_preadv_compressed(bs, host_offset,\n offset, bytes, qiov, qiov_offset);\n \n case QCOW2_CLUSTER_NORMAL:\n- assert(offset_into_cluster(s, file_cluster_offset) == 0);\n if (bs->encrypted) {\n- return qcow2_co_preadv_encrypted(bs, file_cluster_offset,\n+ return qcow2_co_preadv_encrypted(bs, host_offset,\n offset, bytes, qiov, qiov_offset);\n }\n \n BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO);\n- return bdrv_co_preadv_part(s->data_file,\n- file_cluster_offset + offset_in_cluster,\n+ return bdrv_co_preadv_part(s->data_file, host_offset,\n bytes, qiov, qiov_offset, 0);\n \n default:\n@@ -2258,7 +2251,7 @@ static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task)\n \n assert(!t->l2meta);\n \n- return qcow2_co_preadv_task(t->bs, t->cluster_type, t->file_cluster_offset,\n+ return qcow2_co_preadv_task(t->bs, t->cluster_type, t->host_offset,\n t->offset, t->bytes, t->qiov, t->qiov_offset);\n }\n \n@@ -2294,11 +2287,20 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs,\n {\n qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes);\n } else {\n+ /*\n+ * For compressed clusters the variable cluster_offset\n+ * does not actually store the offset but the full\n+ * descriptor. We need to leave it unchanged because\n+ * that's what qcow2_co_preadv_compressed() expects.\n+ */\n+ uint64_t host_offset = (ret == QCOW2_CLUSTER_COMPRESSED) ?\n+ cluster_offset :\n+ cluster_offset + offset_into_cluster(s, offset);\n if (!aio && cur_bytes != bytes) {\n aio = aio_task_pool_new(QCOW2_MAX_WORKERS);\n }\n ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret,\n- cluster_offset, offset, cur_bytes,\n+ host_offset, offset, cur_bytes,\n qiov, qiov_offset, NULL);\n if (ret < 0) {\n goto out;\n@@ -2449,7 +2451,7 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)\n * not use it somehow after qcow2_co_pwritev_task() call\n */\n static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,\n- uint64_t file_cluster_offset,\n+ uint64_t host_offset,\n uint64_t offset, uint64_t bytes,\n QEMUIOVector *qiov,\n uint64_t qiov_offset,\n@@ -2458,7 +2460,6 @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,\n int ret;\n BDRVQcow2State *s = bs->opaque;\n void *crypt_buf = NULL;\n- int offset_in_cluster = offset_into_cluster(s, offset);\n QEMUIOVector encrypted_qiov;\n \n if (bs->encrypted) {\n@@ -2471,9 +2472,7 @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,\n }\n qemu_iovec_to_buf(qiov, qiov_offset, crypt_buf, bytes);\n \n- if (qcow2_co_encrypt(bs, file_cluster_offset + offset_in_cluster,\n- offset, crypt_buf, bytes) < 0)\n- {\n+ if (qcow2_co_encrypt(bs, host_offset, offset, crypt_buf, bytes) < 0) {\n ret = -EIO;\n goto out_unlocked;\n }\n@@ -2497,10 +2496,8 @@ static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs,\n */\n if (!merge_cow(offset, bytes, qiov, qiov_offset, l2meta)) {\n BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO);\n- trace_qcow2_writev_data(qemu_coroutine_self(),\n- file_cluster_offset + offset_in_cluster);\n- ret = bdrv_co_pwritev_part(s->data_file,\n- file_cluster_offset + offset_in_cluster,\n+ trace_qcow2_writev_data(qemu_coroutine_self(), host_offset);\n+ ret = bdrv_co_pwritev_part(s->data_file, host_offset,\n bytes, qiov, qiov_offset, 0);\n if (ret < 0) {\n goto out_unlocked;\n@@ -2530,7 +2527,7 @@ static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task)\n \n assert(!t->cluster_type);\n \n- return qcow2_co_pwritev_task(t->bs, t->file_cluster_offset,\n+ return qcow2_co_pwritev_task(t->bs, t->host_offset,\n t->offset, t->bytes, t->qiov, t->qiov_offset,\n t->l2meta);\n }\n@@ -2585,8 +2582,8 @@ static coroutine_fn int qcow2_co_pwritev_part(\n aio = aio_task_pool_new(QCOW2_MAX_WORKERS);\n }\n ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_task_entry, 0,\n- cluster_offset, offset, cur_bytes,\n- qiov, qiov_offset, l2meta);\n+ cluster_offset + offset_in_cluster, offset,\n+ cur_bytes, qiov, qiov_offset, l2meta);\n l2meta = NULL; /* l2meta is consumed by qcow2_co_pwritev_task() */\n if (ret < 0) {\n goto fail_nometa;\n@@ -4569,7 +4566,7 @@ qcow2_co_pwritev_compressed_part(BlockDriverState *bs,\n \n static int coroutine_fn\n qcow2_co_preadv_compressed(BlockDriverState *bs,\n- uint64_t file_cluster_offset,\n+ uint64_t cluster_descriptor,\n uint64_t offset,\n uint64_t bytes,\n QEMUIOVector *qiov,\n@@ -4581,8 +4578,8 @@ qcow2_co_preadv_compressed(BlockDriverState *bs,\n uint8_t *buf, *out_buf;\n int offset_in_cluster = offset_into_cluster(s, offset);\n \n- coffset = file_cluster_offset & s->cluster_offset_mask;\n- nb_csectors = ((file_cluster_offset >> s->csize_shift) & s->csize_mask) + 1;\n+ coffset = cluster_descriptor & s->cluster_offset_mask;\n+ nb_csectors = ((cluster_descriptor >> s->csize_shift) & s->csize_mask) + 1;\n csize = nb_csectors * QCOW2_COMPRESSED_SECTOR_SIZE -\n (coffset & ~QCOW2_COMPRESSED_SECTOR_MASK);\n \ndiff --git a/block/trace-events b/block/trace-events\nindex 29dff8881c..5c9b0769dc 100644\n--- a/block/trace-events\n+++ b/block/trace-events\n@@ -77,7 +77,7 @@ luring_io_uring_submit(void *s, int ret) \"LuringState %p ret %d\"\n luring_resubmit_short_read(void *s, void *luringcb, int nread) \"LuringState %p luringcb %p nread %d\"\n \n # qcow2.c\n-qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) \"co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %\" PRIu64 \" offset %\" PRIu64 \" bytes %\" PRIu64 \" qiov %p qiov_offset %zu\"\n+qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t host_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) \"co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %\" PRIu64 \" offset %\" PRIu64 \" bytes %\" PRIu64 \" qiov %p qiov_offset %zu\"\n qcow2_writev_start_req(void *co, int64_t offset, int bytes) \"co %p offset 0x%\" PRIx64 \" bytes %d\"\n qcow2_writev_done_req(void *co, int ret) \"co %p ret %d\"\n qcow2_writev_start_part(void *co) \"co %p\"\n","prefixes":["v8","01/34"]}