From patchwork Tue Aug 7 17:43:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954609 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMMT5smhz9s0R for ; Wed, 8 Aug 2018 03:44:04 +1000 (AEST) Received: from localhost ([::1]:40213 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn61Z-0007WF-1Q for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:44:01 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42935) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Tg-5B for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60q-000216-Vq for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:45002) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60q-0001xW-Ms; Tue, 07 Aug 2018 13:43:16 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60m-0003lt-74; Tue, 07 Aug 2018 20:43:12 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:05 +0300 Message-Id: <20180807174311.32454-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 1/7] qcow2: move qemu_co_mutex_lock below decryption procedure X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: "Denis V. Lunev" We are not working with a shared state data in the decruption code and thus this operation is safe. On the other hand this significantly reduces the scope of the lock. Signed-off-by: Denis V. Lunev --- block/qcow2.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index ec9e6238a0..41def07c67 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1880,9 +1880,11 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, break; case QCOW2_CLUSTER_NORMAL: + qemu_co_mutex_unlock(&s->lock); + if ((cluster_offset & 511) != 0) { ret = -EIO; - goto fail; + goto fail_nolock; } if (bs->encrypted) { @@ -1899,7 +1901,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, * s->cluster_size); if (cluster_data == NULL) { ret = -ENOMEM; - goto fail; + goto fail_nolock; } } @@ -1909,13 +1911,11 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, } BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - qemu_co_mutex_unlock(&s->lock); ret = bdrv_co_preadv(bs->file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { - goto fail; + goto fail_nolock; } if (bs->encrypted) { assert(s->crypto); @@ -1929,10 +1929,11 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, cur_bytes, NULL) < 0) { ret = -EIO; - goto fail; + goto fail_nolock; } qemu_iovec_from_buf(qiov, bytes_done, cluster_data, cur_bytes); } + qemu_co_mutex_lock(&s->lock); break; default: @@ -1950,6 +1951,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, fail: qemu_co_mutex_unlock(&s->lock); +fail_nolock: qemu_iovec_destroy(&hd_qiov); qemu_vfree(cluster_data); From patchwork Tue Aug 7 17:43:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954610 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMMX5z11z9s0R for ; Wed, 8 Aug 2018 03:44:08 +1000 (AEST) Received: from localhost ([::1]:40214 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn61e-0007f1-Ee for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:44:06 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42937) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Ti-5u for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00021d-A5 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:44998) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60r-0001xS-0K; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60m-0003lt-CU; Tue, 07 Aug 2018 20:43:12 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:06 +0300 Message-Id: <20180807174311.32454-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 2/7] qcow2: bdrv_co_pwritev: move encryption code out of lock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" We don't need locking for encryption code. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 41def07c67..65e3ca00e2 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2039,6 +2039,15 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, assert((cluster_offset & 511) == 0); + ret = qcow2_pre_write_overlap_check(bs, 0, + cluster_offset + offset_in_cluster, + cur_bytes); + if (ret < 0) { + goto fail; + } + + qemu_co_mutex_unlock(&s->lock); + qemu_iovec_reset(&hd_qiov); qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); @@ -2072,30 +2081,25 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, qemu_iovec_add(&hd_qiov, cluster_data, cur_bytes); } - ret = qcow2_pre_write_overlap_check(bs, 0, - cluster_offset + offset_in_cluster, cur_bytes); - if (ret < 0) { - goto fail; - } - /* If we need to do COW, check if it's possible to merge the * writing of the guest data together with that of the COW regions. * If it's not possible (or not necessary) then write the * guest data now. */ if (!merge_cow(offset, cur_bytes, &hd_qiov, l2meta)) { - qemu_co_mutex_unlock(&s->lock); BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); trace_qcow2_writev_data(qemu_coroutine_self(), cluster_offset + offset_in_cluster); ret = bdrv_co_pwritev(bs->file, cluster_offset + offset_in_cluster, cur_bytes, &hd_qiov, 0); - qemu_co_mutex_lock(&s->lock); if (ret < 0) { + qemu_co_mutex_lock(&s->lock); goto fail; } } + qemu_co_mutex_lock(&s->lock); + ret = qcow2_handle_l2meta(bs, &l2meta, true); if (ret) { goto fail; From patchwork Tue Aug 7 17:43:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954612 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMQC6CMhz9s0R for ; Wed, 8 Aug 2018 03:46:27 +1000 (AEST) Received: from localhost ([::1]:40230 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn63t-0001hs-Fz for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:46:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42942) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Tm-7G for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00021K-3s for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:44992) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60q-0001xR-QU; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60m-0003lt-Hl; Tue, 07 Aug 2018 20:43:12 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:07 +0300 Message-Id: <20180807174311.32454-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 3/7] qcow2: split out reading normal clusters from qcow2_co_preadv X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Memory allocation may become less efficient for encrypted case. It's a payment for further asynchronous scheme. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 114 ++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 64 insertions(+), 50 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 65e3ca00e2..5e7f2ee318 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1808,6 +1808,67 @@ out: return ret; } +static coroutine_fn int qcow2_co_preadv_normal(BlockDriverState *bs, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + uint64_t qiov_offset) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + void *crypt_buf = NULL; + QEMUIOVector hd_qiov; + int offset_in_cluster = offset_into_cluster(s, offset); + + if ((file_cluster_offset & 511) != 0) { + return -EIO; + } + + qemu_iovec_init(&hd_qiov, qiov->niov); + + if (bs->encrypted) { + assert(s->crypto); + assert(bytes <= QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + + crypt_buf = qemu_try_blockalign(bs->file->bs, bytes); + qemu_iovec_add(&hd_qiov, crypt_buf, bytes); + } else { + qemu_iovec_concat(&hd_qiov, qiov, qiov_offset, bytes); + } + + BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); + ret = bdrv_co_preadv(bs->file, + file_cluster_offset + offset_in_cluster, + bytes, &hd_qiov, 0); + if (ret < 0) { + goto out; + } + + if (bs->encrypted) { + assert(s->crypto); + assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0); + assert((bytes & (BDRV_SECTOR_SIZE - 1)) == 0); + if (qcrypto_block_decrypt(s->crypto, + (s->crypt_physical_offset ? + file_cluster_offset + offset_in_cluster : + offset), + crypt_buf, + bytes, + NULL) < 0) { + ret = -EIO; + goto out; + } + qemu_iovec_from_buf(qiov, qiov_offset, crypt_buf, bytes); + } + +out: + qemu_vfree(crypt_buf); + qemu_iovec_destroy(&hd_qiov); + + return ret; +} + static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -1819,7 +1880,6 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t cluster_offset = 0; uint64_t bytes_done = 0; QEMUIOVector hd_qiov; - uint8_t *cluster_data = NULL; qemu_iovec_init(&hd_qiov, qiov->niov); @@ -1882,57 +1942,12 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, case QCOW2_CLUSTER_NORMAL: qemu_co_mutex_unlock(&s->lock); - if ((cluster_offset & 511) != 0) { - ret = -EIO; - goto fail_nolock; - } - - if (bs->encrypted) { - assert(s->crypto); - - /* - * For encrypted images, read everything into a temporary - * contiguous buffer on which the AES functions can work. - */ - if (!cluster_data) { - cluster_data = - qemu_try_blockalign(bs->file->bs, - QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); - if (cluster_data == NULL) { - ret = -ENOMEM; - goto fail_nolock; - } - } - - assert(cur_bytes <= QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); - qemu_iovec_reset(&hd_qiov); - qemu_iovec_add(&hd_qiov, cluster_data, cur_bytes); - } - - BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); - ret = bdrv_co_preadv(bs->file, - cluster_offset + offset_in_cluster, - cur_bytes, &hd_qiov, 0); + ret = qcow2_co_preadv_normal(bs, cluster_offset, + offset, cur_bytes, qiov, bytes_done); if (ret < 0) { goto fail_nolock; } - if (bs->encrypted) { - assert(s->crypto); - assert((offset & (BDRV_SECTOR_SIZE - 1)) == 0); - assert((cur_bytes & (BDRV_SECTOR_SIZE - 1)) == 0); - if (qcrypto_block_decrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_cluster : - offset), - cluster_data, - cur_bytes, - NULL) < 0) { - ret = -EIO; - goto fail_nolock; - } - qemu_iovec_from_buf(qiov, bytes_done, cluster_data, cur_bytes); - } + qemu_co_mutex_lock(&s->lock); break; @@ -1953,7 +1968,6 @@ fail: fail_nolock: qemu_iovec_destroy(&hd_qiov); - qemu_vfree(cluster_data); return ret; } From patchwork Tue Aug 7 17:43:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954613 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMSb4h2Qz9ryt for ; Wed, 8 Aug 2018 03:48:31 +1000 (AEST) Received: from localhost ([::1]:40243 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn65t-0003BU-CZ for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:48:29 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42939) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Tk-6Q for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00021P-50 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:45000) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60q-0001xV-Ml; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60m-0003lt-PI; Tue, 07 Aug 2018 20:43:12 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:08 +0300 Message-Id: <20180807174311.32454-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 4/7] qcow2: async scheme for qcow2_co_preadv X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Start several async requests instead of read chunk by chunk. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 208 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 204 insertions(+), 4 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 5e7f2ee318..a0df8d4e50 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1869,6 +1869,197 @@ out: return ret; } +typedef struct Qcow2WorkerTask { + uint64_t file_cluster_offset; + uint64_t offset; + uint64_t bytes; + uint64_t bytes_done; +} Qcow2WorkerTask; + +typedef int (*Qcow2DoWorkFunc)(BlockDriverState *bs, QEMUIOVector *qiov, + Qcow2WorkerTask *task); + +typedef struct Qcow2RWState { + BlockDriverState *bs; + QEMUIOVector *qiov; + uint64_t bytes; + int ret; + bool waiting_one; + bool waiting_all; + bool finalize; + Coroutine *co; + QSIMPLEQ_HEAD(, Qcow2Worker) free_workers; + QSIMPLEQ_HEAD(, Qcow2Worker) busy_workers; + int online_workers; + Qcow2DoWorkFunc do_work_func; +} Qcow2RWState; + +typedef struct Qcow2Worker { + Qcow2RWState *rws; + Coroutine *co; + Qcow2WorkerTask task; + bool busy; + QSIMPLEQ_ENTRY(Qcow2Worker) entry; +} Qcow2Worker; +#define QCOW2_MAX_WORKERS 64 + +static coroutine_fn void qcow2_rw_worker(void *opaque); +static Qcow2Worker *qcow2_new_worker(Qcow2RWState *rws) +{ + Qcow2Worker *w = g_new0(Qcow2Worker, 1); + w->rws = rws; + w->co = qemu_coroutine_create(qcow2_rw_worker, w); + + return w; +} + +static void qcow2_free_worker(Qcow2Worker *w) +{ + g_free(w); +} + +static coroutine_fn void qcow2_rw_worker(void *opaque) +{ + Qcow2Worker *w = opaque; + Qcow2RWState *rws = w->rws; + + rws->online_workers++; + + while (!rws->finalize) { + int ret = rws->do_work_func(rws->bs, rws->qiov, &w->task); + if (ret < 0 && rws->ret == 0) { + rws->ret = ret; + } + + if (rws->waiting_all || rws->ret < 0) { + break; + } + + w->busy = false; + QSIMPLEQ_REMOVE(&rws->busy_workers, w, Qcow2Worker, entry); + QSIMPLEQ_INSERT_TAIL(&rws->free_workers, w, entry); + if (rws->waiting_one) { + rws->waiting_one = false; + /* we must unset it here, to prevent queuing rws->co in several + * workers (it may happen if other worker already waits us on mutex, + * so it will be entered after our yield and before rws->co enter) + * + * TODO: rethink this comment, as here (and in other places in the + * file) we moved from qemu_coroutine_add_next to aio_co_wake. + */ + aio_co_wake(rws->co); + } + + qemu_coroutine_yield(); + } + + if (w->busy) { + w->busy = false; + QSIMPLEQ_REMOVE(&rws->busy_workers, w, Qcow2Worker, entry); + } + qcow2_free_worker(w); + rws->online_workers--; + + if (rws->waiting_all && rws->online_workers == 0) { + aio_co_wake(rws->co); + } +} + +static coroutine_fn void qcow2_rws_add_task(Qcow2RWState *rws, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + uint64_t bytes_done) +{ + Qcow2Worker *w; + + assert(rws->co == qemu_coroutine_self()); + + if (bytes_done == 0 && bytes == rws->bytes) { + Qcow2WorkerTask task = { + .file_cluster_offset = file_cluster_offset, + .offset = offset, + .bytes = bytes, + .bytes_done = bytes_done + }; + rws->ret = rws->do_work_func(rws->bs, rws->qiov, &task); + return; + } + + if (!QSIMPLEQ_EMPTY(&rws->free_workers)) { + w = QSIMPLEQ_FIRST(&rws->free_workers); + QSIMPLEQ_REMOVE_HEAD(&rws->free_workers, entry); + } else if (rws->online_workers < QCOW2_MAX_WORKERS) { + w = qcow2_new_worker(rws); + } else { + rws->waiting_one = true; + qemu_coroutine_yield(); + assert(!rws->waiting_one); /* already unset by worker */ + + w = QSIMPLEQ_FIRST(&rws->free_workers); + QSIMPLEQ_REMOVE_HEAD(&rws->free_workers, entry); + } + w->busy = true; + QSIMPLEQ_INSERT_TAIL(&rws->busy_workers, w, entry); + + w->task.file_cluster_offset = file_cluster_offset; + w->task.offset = offset; + w->task.bytes = bytes; + w->task.bytes_done = bytes_done; + + qemu_coroutine_enter(w->co); +} + +static void qcow2_init_rws(Qcow2RWState *rws, BlockDriverState *bs, + QEMUIOVector *qiov, uint64_t bytes, + Qcow2DoWorkFunc do_work_func) +{ + memset(rws, 0, sizeof(*rws)); + rws->bs = bs; + rws->qiov = qiov; + rws->bytes = bytes; + rws->co = qemu_coroutine_self(); + rws->do_work_func = do_work_func; + QSIMPLEQ_INIT(&rws->free_workers); + QSIMPLEQ_INIT(&rws->busy_workers); +} + +static void qcow2_finalize_rws(Qcow2RWState *rws) +{ + assert(rws->co == qemu_coroutine_self()); + + /* kill waiting workers */ + rws->finalize = true; + while (!QSIMPLEQ_EMPTY(&rws->free_workers)) { + Qcow2Worker *w = QSIMPLEQ_FIRST(&rws->free_workers); + QSIMPLEQ_REMOVE_HEAD(&rws->free_workers, entry); + qemu_coroutine_enter(w->co); + } + + /* wait others */ + if (rws->online_workers > 0) { + rws->waiting_all = true; + qemu_coroutine_yield(); + rws->waiting_all = false; + } + + assert(rws->online_workers == 0); + assert(QSIMPLEQ_EMPTY(&rws->free_workers)); + assert(QSIMPLEQ_EMPTY(&rws->busy_workers)); +} + +static coroutine_fn int qcow2_co_preadv_normal_task(BlockDriverState *bs, + QEMUIOVector *qiov, + Qcow2WorkerTask *task) +{ + return qcow2_co_preadv_normal(bs, + task->file_cluster_offset, + task->offset, + task->bytes, + qiov, + task->bytes_done); +} + static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -1880,12 +2071,15 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t cluster_offset = 0; uint64_t bytes_done = 0; QEMUIOVector hd_qiov; + Qcow2RWState rws = {0}; + + qcow2_init_rws(&rws, bs, qiov, bytes, qcow2_co_preadv_normal_task); qemu_iovec_init(&hd_qiov, qiov->niov); qemu_co_mutex_lock(&s->lock); - while (bytes != 0) { + while (bytes != 0 && rws.ret == 0) { /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); @@ -1942,9 +2136,10 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, case QCOW2_CLUSTER_NORMAL: qemu_co_mutex_unlock(&s->lock); - ret = qcow2_co_preadv_normal(bs, cluster_offset, - offset, cur_bytes, qiov, bytes_done); - if (ret < 0) { + qcow2_rws_add_task(&rws, cluster_offset, offset, cur_bytes, + bytes_done); + if (rws.ret < 0) { + ret = rws.ret; goto fail_nolock; } @@ -1967,6 +2162,11 @@ fail: qemu_co_mutex_unlock(&s->lock); fail_nolock: + qcow2_finalize_rws(&rws); + if (ret == 0) { + ret = rws.ret; + } + qemu_iovec_destroy(&hd_qiov); return ret; From patchwork Tue Aug 7 17:43:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954611 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMQC14zgz9ryt for ; Wed, 8 Aug 2018 03:46:27 +1000 (AEST) Received: from localhost ([::1]:40229 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn63s-0001h4-Qp for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:46:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42946) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007To-8J for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00021q-DE for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:45006) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60r-0001xU-2R; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60m-0003lt-Uh; Tue, 07 Aug 2018 20:43:13 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:09 +0300 Message-Id: <20180807174311.32454-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 5/7] qcow2: refactor qcow2_co_pwritev: split out qcow2_co_do_pwritev X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Split out block which will be reused in async scheme. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 138 ++++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 86 insertions(+), 52 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index a0df8d4e50..4d669432d1 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2210,6 +2210,85 @@ static bool merge_cow(uint64_t offset, unsigned bytes, return false; } +/* qcow2_co_do_pwritev + * Called without s->lock unlocked + * hd_qiov - temp qiov for any use. It is initialized so it is empty and + * support adding up to qiov->niov + 2 elements + * l2meta - if not NULL, qcow2_co_do_pwritev() will consume it. Caller must not + * use it somehow after qcow2_co_do_pwritev() call + */ +static coroutine_fn int qcow2_co_do_pwritev(BlockDriverState *bs, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + uint64_t qiov_offset, + QCowL2Meta *l2meta) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + void *crypt_buf = NULL; + QEMUIOVector hd_qiov; + int offset_in_cluster = offset_into_cluster(s, offset); + + qemu_iovec_reset(&hd_qiov); + qemu_iovec_init(&hd_qiov, qiov->niov); + + if (bs->encrypted) { + assert(s->crypto); + assert(bytes <= QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + crypt_buf = qemu_try_blockalign(bs->file->bs, bytes); + qemu_iovec_to_buf(qiov, qiov_offset, crypt_buf, bytes); + + if (qcrypto_block_encrypt(s->crypto, + (s->crypt_physical_offset ? + file_cluster_offset + offset_in_cluster : + offset), + crypt_buf, + bytes, NULL) < 0) { + ret = -EIO; + goto fail; + } + + qemu_iovec_add(&hd_qiov, crypt_buf, bytes); + } else { + qemu_iovec_concat(&hd_qiov, qiov, qiov_offset, bytes); + } + + /* If we need to do COW, check if it's possible to merge the + * writing of the guest data together with that of the COW regions. + * If it's not possible (or not necessary) then write the + * guest data now. */ + if (!merge_cow(offset, bytes, &hd_qiov, l2meta)) { + BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); + trace_qcow2_writev_data(qemu_coroutine_self(), + file_cluster_offset + offset_in_cluster); + ret = bdrv_co_pwritev(bs->file, + file_cluster_offset + offset_in_cluster, + bytes, &hd_qiov, 0); + if (ret < 0) { + qemu_co_mutex_lock(&s->lock); + goto fail; + } + } + + qemu_co_mutex_lock(&s->lock); + + ret = qcow2_handle_l2meta(bs, &l2meta, true); + if (ret) { + goto fail; + } + +fail: + qcow2_handle_l2meta(bs, &l2meta, false); + qemu_co_mutex_unlock(&s->lock); + + qemu_vfree(crypt_buf); + qemu_iovec_destroy(&hd_qiov); + + return ret; +} + static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -2262,63 +2341,16 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, qemu_co_mutex_unlock(&s->lock); - qemu_iovec_reset(&hd_qiov); - qemu_iovec_concat(&hd_qiov, qiov, bytes_done, cur_bytes); - if (bs->encrypted) { - assert(s->crypto); - if (!cluster_data) { - cluster_data = qemu_try_blockalign(bs->file->bs, - QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); - if (cluster_data == NULL) { - ret = -ENOMEM; - goto fail; - } - } - - assert(hd_qiov.size <= - QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); - qemu_iovec_to_buf(&hd_qiov, 0, cluster_data, hd_qiov.size); - - if (qcrypto_block_encrypt(s->crypto, - (s->crypt_physical_offset ? - cluster_offset + offset_in_cluster : - offset), - cluster_data, - cur_bytes, NULL) < 0) { - ret = -EIO; - goto fail; - } - - qemu_iovec_reset(&hd_qiov); - qemu_iovec_add(&hd_qiov, cluster_data, cur_bytes); - } - - /* If we need to do COW, check if it's possible to merge the - * writing of the guest data together with that of the COW regions. - * If it's not possible (or not necessary) then write the - * guest data now. */ - if (!merge_cow(offset, cur_bytes, &hd_qiov, l2meta)) { - BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); - trace_qcow2_writev_data(qemu_coroutine_self(), - cluster_offset + offset_in_cluster); - ret = bdrv_co_pwritev(bs->file, - cluster_offset + offset_in_cluster, - cur_bytes, &hd_qiov, 0); - if (ret < 0) { - qemu_co_mutex_lock(&s->lock); - goto fail; - } + ret = qcow2_co_do_pwritev(bs, cluster_offset, offset, cur_bytes, + qiov, bytes_done, l2meta); + l2meta = NULL; /* l2meta is consumed by qcow2_co_do_pwritev() */ + if (ret < 0) { + goto fail_nometa; } qemu_co_mutex_lock(&s->lock); - ret = qcow2_handle_l2meta(bs, &l2meta, true); - if (ret) { - goto fail; - } - bytes -= cur_bytes; offset += cur_bytes; bytes_done += cur_bytes; @@ -2331,6 +2363,8 @@ fail: qemu_co_mutex_unlock(&s->lock); +fail_nometa: + qemu_iovec_destroy(&hd_qiov); qemu_vfree(cluster_data); trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); From patchwork Tue Aug 7 17:43:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMMT5t6rz9s0n for ; Wed, 8 Aug 2018 03:44:04 +1000 (AEST) Received: from localhost ([::1]:40212 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn61Y-0007WD-HC for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:44:00 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42944) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Tn-88 for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00022I-JI for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:44988) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60r-0001xT-6u; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60n-0003lt-5y; Tue, 07 Aug 2018 20:43:13 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:10 +0300 Message-Id: <20180807174311.32454-7-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 6/7] qcow2: refactor qcow2_co_pwritev locals scope X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Move local variables related to individual loop iteration into while block. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 4d669432d1..88b3fb8080 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2294,10 +2294,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, int flags) { BDRVQcow2State *s = bs->opaque; - int offset_in_cluster; int ret; - unsigned int cur_bytes; /* number of sectors in current iteration */ - uint64_t cluster_offset; QEMUIOVector hd_qiov; uint64_t bytes_done = 0; uint8_t *cluster_data = NULL; @@ -2312,12 +2309,14 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, qemu_co_mutex_lock(&s->lock); while (bytes != 0) { + int offset_in_cluster = offset_into_cluster(s, offset); + unsigned int cur_bytes = MIN(bytes, INT_MAX); /* number of sectors in + current iteration */ + uint64_t cluster_offset; l2meta = NULL; trace_qcow2_writev_start_part(qemu_coroutine_self()); - offset_in_cluster = offset_into_cluster(s, offset); - cur_bytes = MIN(bytes, INT_MAX); if (bs->encrypted) { cur_bytes = MIN(cur_bytes, QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size From patchwork Tue Aug 7 17:43:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 954620 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41lMWQ1C8Zz9ryt for ; Wed, 8 Aug 2018 03:50:58 +1000 (AEST) Received: from localhost ([::1]:40254 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn68F-0004i0-RI for incoming@patchwork.ozlabs.org; Tue, 07 Aug 2018 13:50:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42936) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fn60u-0007Th-5h for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fn60r-00021w-DP for qemu-devel@nongnu.org; Tue, 07 Aug 2018 13:43:20 -0400 Received: from relay.sw.ru ([185.231.240.75]:45014) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fn60q-0001xY-Uk; Tue, 07 Aug 2018 13:43:17 -0400 Received: from vz-out.virtuozzo.com ([185.231.240.5] helo=kvm.sw.ru) by relay.sw.ru with esmtp (Exim 4.90_1) (envelope-from ) id 1fn60n-0003lt-D1; Tue, 07 Aug 2018 20:43:13 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-devel@nongnu.org, qemu-block@nongnu.org Date: Tue, 7 Aug 2018 20:43:11 +0300 Message-Id: <20180807174311.32454-8-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.11.1 In-Reply-To: <20180807174311.32454-1-vsementsov@virtuozzo.com> References: <20180807174311.32454-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH 7/7] qcow2: async scheme for qcow2_co_pwritev X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, den@openvz.org, vsementsov@virtuozzo.com, mreitz@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Start several async requests instead of read chunk by chunk. Iotest 026 output is changed, as because of async io error path has changed. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 47 ++++++++++++++++++++++++++++++-------- tests/qemu-iotests/026.out | 18 ++++++++------- tests/qemu-iotests/026.out.nocache | 20 ++++++++-------- 3 files changed, 59 insertions(+), 26 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 88b3fb8080..c7501adfc6 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1874,6 +1874,7 @@ typedef struct Qcow2WorkerTask { uint64_t offset; uint64_t bytes; uint64_t bytes_done; + QCowL2Meta *l2meta; } Qcow2WorkerTask; typedef int (*Qcow2DoWorkFunc)(BlockDriverState *bs, QEMUIOVector *qiov, @@ -1965,11 +1966,16 @@ static coroutine_fn void qcow2_rw_worker(void *opaque) } } +/* qcow2_rws_add_task +* l2meta - opaque pointer for do_work_func, so behavior depends on +* do_work_func, specified in qcow2_init_rws +*/ static coroutine_fn void qcow2_rws_add_task(Qcow2RWState *rws, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, - uint64_t bytes_done) + uint64_t bytes_done, + QCowL2Meta *l2meta) { Qcow2Worker *w; @@ -1980,7 +1986,8 @@ static coroutine_fn void qcow2_rws_add_task(Qcow2RWState *rws, .file_cluster_offset = file_cluster_offset, .offset = offset, .bytes = bytes, - .bytes_done = bytes_done + .bytes_done = bytes_done, + .l2meta = l2meta }; rws->ret = rws->do_work_func(rws->bs, rws->qiov, &task); return; @@ -2006,6 +2013,7 @@ static coroutine_fn void qcow2_rws_add_task(Qcow2RWState *rws, w->task.offset = offset; w->task.bytes = bytes; w->task.bytes_done = bytes_done; + w->task.l2meta = l2meta; qemu_coroutine_enter(w->co); } @@ -2137,7 +2145,7 @@ static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, qemu_co_mutex_unlock(&s->lock); qcow2_rws_add_task(&rws, cluster_offset, offset, cur_bytes, - bytes_done); + bytes_done, NULL); if (rws.ret < 0) { ret = rws.ret; goto fail_nolock; @@ -2289,6 +2297,19 @@ fail: return ret; } +static coroutine_fn int qcow2_co_do_pwritev_task(BlockDriverState *bs, + QEMUIOVector *qiov, + Qcow2WorkerTask *task) +{ + return qcow2_co_do_pwritev(bs, + task->file_cluster_offset, + task->offset, + task->bytes, + qiov, + task->bytes_done, + task->l2meta); +} + static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) @@ -2299,16 +2320,19 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes_done = 0; uint8_t *cluster_data = NULL; QCowL2Meta *l2meta = NULL; + Qcow2RWState rws = {0}; trace_qcow2_writev_start_req(qemu_coroutine_self(), offset, bytes); + qcow2_init_rws(&rws, bs, qiov, bytes, qcow2_co_do_pwritev_task); + qemu_iovec_init(&hd_qiov, qiov->niov); s->cluster_cache_offset = -1; /* disable compressed cache */ qemu_co_mutex_lock(&s->lock); - while (bytes != 0) { + while (bytes != 0 && rws.ret == 0) { int offset_in_cluster = offset_into_cluster(s, offset); unsigned int cur_bytes = MIN(bytes, INT_MAX); /* number of sectors in current iteration */ @@ -2340,11 +2364,11 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, qemu_co_mutex_unlock(&s->lock); - - ret = qcow2_co_do_pwritev(bs, cluster_offset, offset, cur_bytes, - qiov, bytes_done, l2meta); - l2meta = NULL; /* l2meta is consumed by qcow2_co_do_pwritev() */ - if (ret < 0) { + qcow2_rws_add_task(&rws, cluster_offset, offset, cur_bytes, bytes_done, + l2meta); + l2meta = NULL; /* l2meta is consumed */ + if (rws.ret < 0) { + ret = rws.ret; goto fail_nometa; } @@ -2362,6 +2386,11 @@ fail: qemu_co_mutex_unlock(&s->lock); + qcow2_finalize_rws(&rws); + if (ret == 0) { + ret = rws.ret; + } + fail_nometa: qemu_iovec_destroy(&hd_qiov); diff --git a/tests/qemu-iotests/026.out b/tests/qemu-iotests/026.out index dd10a82b51..8dd19f1b53 100644 --- a/tests/qemu-iotests/026.out +++ b/tests/qemu-iotests/026.out @@ -481,7 +481,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -55 leaked clusters were found on the image. +119 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -508,7 +508,9 @@ Event: refblock_alloc_write; errno: 28; imm: off; once: off; write Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -No errors were found on the image. + +64 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b @@ -533,7 +535,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -542,7 +544,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -561,7 +563,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -570,7 +572,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -589,7 +591,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -598,7 +600,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. === L1 growth tests === diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache index 1ca6cda15c..10218c1aa6 100644 --- a/tests/qemu-iotests/026.out.nocache +++ b/tests/qemu-iotests/026.out.nocache @@ -489,7 +489,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -55 leaked clusters were found on the image. +119 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -516,8 +516,10 @@ Event: refblock_alloc_write; errno: 28; imm: off; once: off; write Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -No errors were found on the image. -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 + +64 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b Failed to flush the L2 table cache: No space left on device @@ -541,7 +543,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -550,7 +552,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -569,7 +571,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -578,7 +580,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -597,7 +599,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -606,7 +608,7 @@ Failed to flush the L2 table cache: No space left on device Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. === L1 growth tests ===