From patchwork Fri Aug 16 15:30:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1148297 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4696p671NJz9s00 for ; Sat, 17 Aug 2019 01:35:10 +1000 (AEST) Received: from localhost ([::1]:57802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeFw-0004O0-UF for incoming@patchwork.ozlabs.org; Fri, 16 Aug 2019 11:35:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58099) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeBR-0007s8-Kt for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:32 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyeBO-0001dQ-7m for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:29 -0400 Received: from relay.sw.ru ([185.231.240.75]:60434) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hyeBH-0001Qc-0v; Fri, 16 Aug 2019 11:30:19 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hyeBD-0007x1-CO; Fri, 16 Aug 2019 18:30:15 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 16 Aug 2019 18:30:11 +0300 Message-Id: <20190816153015.447957-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190816153015.447957-1-vsementsov@virtuozzo.com> References: <20190816153015.447957-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 1/5] tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" qemu-io now prefixes its error and warnings with "qemu-io:". 36b9986b08787019e fixed a lot of iotests output but forget about 026.out.nocache. Fix it too. Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()") Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- tests/qemu-iotests/026.out.nocache | 168 ++++++++++++++--------------- 1 file changed, 84 insertions(+), 84 deletions(-) diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache index 1ca6cda15c..6dda95dfb4 100644 --- a/tests/qemu-iotests/026.out.nocache +++ b/tests/qemu-iotests/026.out.nocache @@ -14,8 +14,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_update; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error 1 leaked clusters were found on the image. @@ -23,8 +23,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_update; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error 1 leaked clusters were found on the image. @@ -42,8 +42,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_update; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 1 leaked clusters were found on the image. @@ -51,8 +51,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_update; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 1 leaked clusters were found on the image. @@ -136,8 +136,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_update; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -146,8 +146,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_update; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -168,8 +168,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_update; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -178,8 +178,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_update; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device wrote 131072/131072 bytes at offset 0 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) @@ -198,15 +198,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_alloc_write; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_alloc_write; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error 1 leaked clusters were found on the image. @@ -224,15 +224,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_alloc_write; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l2_alloc_write; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 1 leaked clusters were found on the image. @@ -250,15 +250,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: write_aio; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: write_aio; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -274,15 +274,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: write_aio; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: write_aio; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -298,15 +298,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_load; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_load; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -322,15 +322,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_load; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_load; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -346,15 +346,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_update_part; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_update_part; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -370,15 +370,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_update_part; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_update_part; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -394,15 +394,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc; errno: 5; imm: off; once: off; write -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc; errno: 5; imm: off; once: off; write -b -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -418,15 +418,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -485,8 +485,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 55 leaked clusters were found on the image. @@ -494,8 +494,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_hookup; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 251 leaked clusters were found on the image. @@ -513,15 +513,15 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -537,8 +537,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 10 leaked clusters were found on the image. @@ -546,8 +546,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write_blocks; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 23 leaked clusters were found on the image. @@ -565,8 +565,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 10 leaked clusters were found on the image. @@ -574,8 +574,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write_table; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 23 leaked clusters were found on the image. @@ -593,8 +593,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 10 leaked clusters were found on the image. @@ -602,8 +602,8 @@ This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_switch_table; errno: 28; imm: off; once: off; write -b -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 23 leaked clusters were found on the image. @@ -639,8 +639,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_grow_write_table; errno: 5; imm: off; once: off -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -651,8 +651,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_grow_write_table; errno: 28; imm: off; once: off -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -663,8 +663,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_grow_activate_table; errno: 5; imm: off; once: off -Failed to flush the L2 table cache: Input/output error -Failed to flush the refcount block cache: Input/output error +qemu-io: Failed to flush the L2 table cache: Input/output error +qemu-io: Failed to flush the refcount block cache: Input/output error write failed: Input/output error 96 leaked clusters were found on the image. @@ -677,8 +677,8 @@ No errors were found on the image. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: l1_grow_activate_table; errno: 28; imm: off; once: off -Failed to flush the L2 table cache: No space left on device -Failed to flush the refcount block cache: No space left on device +qemu-io: Failed to flush the L2 table cache: No space left on device +qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device 96 leaked clusters were found on the image. From patchwork Fri Aug 16 15:30:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1148292 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4696k60Xfhz9sN1 for ; Sat, 17 Aug 2019 01:31:40 +1000 (AEST) Received: from localhost ([::1]:57756 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeCW-00083z-Nl for incoming@patchwork.ozlabs.org; Fri, 16 Aug 2019 11:31:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58044) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeBM-0007kg-9W for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyeBK-0001Y2-DO for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:24 -0400 Received: from relay.sw.ru ([185.231.240.75]:60424) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hyeBG-0001QY-VF; Fri, 16 Aug 2019 11:30:19 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hyeBD-0007x1-Im; Fri, 16 Aug 2019 18:30:15 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 16 Aug 2019 18:30:12 +0300 Message-Id: <20190816153015.447957-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190816153015.447957-1-vsementsov@virtuozzo.com> References: <20190816153015.447957-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 2/5] block: introduce aio task pool X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/block/aio_task.h | 54 +++++++++++++++++ block/aio_task.c | 124 +++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 + 3 files changed, 180 insertions(+) create mode 100644 include/block/aio_task.h create mode 100644 block/aio_task.c diff --git a/include/block/aio_task.h b/include/block/aio_task.h new file mode 100644 index 0000000000..50bc1e1817 --- /dev/null +++ b/include/block/aio_task.h @@ -0,0 +1,54 @@ +/* + * Aio tasks loops + * + * Copyright (c) 2019 Virtuozzo International GmbH. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef BLOCK_AIO_TASK_H +#define BLOCK_AIO_TASK_H + +#include "qemu/coroutine.h" + +typedef struct AioTaskPool AioTaskPool; +typedef struct AioTask AioTask; +typedef int coroutine_fn (*AioTaskFunc)(AioTask *task); +struct AioTask { + AioTaskPool *pool; + AioTaskFunc func; + int ret; +}; + +AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_tasks); +void aio_task_pool_free(AioTaskPool *); + +/* error code of failed task or 0 if all is OK */ +int aio_task_pool_status(AioTaskPool *pool); + +bool aio_task_pool_empty(AioTaskPool *pool); + +/* User provides filled @task, however task->pool will be set automatically */ +void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *task); + +void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool); + +#endif /* BLOCK_AIO_TASK_H */ diff --git a/block/aio_task.c b/block/aio_task.c new file mode 100644 index 0000000000..88989fa248 --- /dev/null +++ b/block/aio_task.c @@ -0,0 +1,124 @@ +/* + * Aio tasks loops + * + * Copyright (c) 2019 Virtuozzo International GmbH. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#include "qemu/osdep.h" +#include "block/aio.h" +#include "block/aio_task.h" + +struct AioTaskPool { + Coroutine *main_co; + int status; + int max_busy_tasks; + int busy_tasks; + bool waiting; +}; + +static void coroutine_fn aio_task_co(void *opaque) +{ + AioTask *task = opaque; + AioTaskPool *pool = task->pool; + + assert(pool->busy_tasks < pool->max_busy_tasks); + pool->busy_tasks++; + + task->ret = task->func(task); + + pool->busy_tasks--; + + if (task->ret < 0 && pool->status == 0) { + pool->status = task->ret; + } + + g_free(task); + + if (pool->waiting) { + pool->waiting = false; + aio_co_wake(pool->main_co); + } +} + +void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool) +{ + assert(pool->busy_tasks > 0); + assert(qemu_coroutine_self() == pool->main_co); + + pool->waiting = true; + qemu_coroutine_yield(); + + assert(!pool->waiting); + assert(pool->busy_tasks < pool->max_busy_tasks); +} + +void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool) +{ + if (pool->busy_tasks < pool->max_busy_tasks) { + return; + } + + aio_task_pool_wait_one(pool); +} + +void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool) +{ + while (pool->busy_tasks > 0) { + aio_task_pool_wait_one(pool); + } +} + +void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *task) +{ + aio_task_pool_wait_slot(pool); + + task->pool = pool; + qemu_coroutine_enter(qemu_coroutine_create(aio_task_co, task)); +} + +AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_tasks) +{ + AioTaskPool *pool = g_new0(AioTaskPool, 1); + + pool->main_co = qemu_coroutine_self(); + pool->max_busy_tasks = max_busy_tasks; + + return pool; +} + +void aio_task_pool_free(AioTaskPool *pool) +{ + g_free(pool); +} + +int aio_task_pool_status(AioTaskPool *pool) +{ + if (!pool) { + return 0; /* Sugar for lazy allocation of aio pool */ + } + + return pool->status; +} + +bool aio_task_pool_empty(AioTaskPool *pool) +{ + return pool->busy_tasks == 0; +} diff --git a/block/Makefile.objs b/block/Makefile.objs index 35f3bca4d9..c2eb8c8769 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -40,6 +40,8 @@ block-obj-y += throttle.o copy-on-read.o block-obj-y += crypto.o +block-obj-y += aio_task.o + common-obj-y += stream.o nfs.o-libs := $(LIBNFS_LIBS) From patchwork Fri Aug 16 15:30:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1148294 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4696k94klkz9s00 for ; Sat, 17 Aug 2019 01:31:45 +1000 (AEST) Received: from localhost ([::1]:57758 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeCd-0008Ft-Ao for incoming@patchwork.ozlabs.org; Fri, 16 Aug 2019 11:31:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58029) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeBL-0007kE-S1 for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyeBK-0001Xq-B9 for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:23 -0400 Received: from relay.sw.ru ([185.231.240.75]:60438) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hyeBH-0001QZ-1y; Fri, 16 Aug 2019 11:30:19 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hyeBD-0007x1-SM; Fri, 16 Aug 2019 18:30:15 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 16 Aug 2019 18:30:14 +0300 Message-Id: <20190816153015.447957-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190816153015.447957-1-vsementsov@virtuozzo.com> References: <20190816153015.447957-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 4/5] block/qcow2: refactor qcow2_co_pwritev_part X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Similarly to previous commit, prepare for parallelizing write-loop iterations. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 153 +++++++++++++++++++++++++++++--------------------- 1 file changed, 89 insertions(+), 64 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 89afb4272e..3aaa180e2b 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -2234,6 +2234,87 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) return 0; } +/* + * qcow2_co_pwritev_task + * Called with s->lock unlocked + * l2meta - if not NULL, qcow2_co_do_pwritev() will consume it. Caller must not + * use it somehow after qcow2_co_pwritev_task() call + */ +static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs, + uint64_t file_cluster_offset, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, + uint64_t qiov_offset, + QCowL2Meta *l2meta) +{ + int ret; + BDRVQcow2State *s = bs->opaque; + void *crypt_buf = NULL; + int offset_in_cluster = offset_into_cluster(s, offset); + QEMUIOVector encrypted_qiov; + + if (bs->encrypted) { + assert(s->crypto); + assert(bytes <= QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + crypt_buf = qemu_try_blockalign(bs->file->bs, bytes); + if (crypt_buf == NULL) { + ret = -ENOMEM; + goto out_unlocked; + } + qemu_iovec_to_buf(qiov, qiov_offset, crypt_buf, bytes); + + if (qcow2_co_encrypt(bs, file_cluster_offset, offset, + crypt_buf, bytes) < 0) { + ret = -EIO; + goto out_unlocked; + } + + qemu_iovec_init_buf(&encrypted_qiov, crypt_buf, bytes); + qiov = &encrypted_qiov; + qiov_offset = 0; + } + + /* Try to efficiently initialize the physical space with zeroes */ + ret = handle_alloc_space(bs, l2meta); + if (ret < 0) { + goto out_unlocked; + } + + /* + * If we need to do COW, check if it's possible to merge the + * writing of the guest data together with that of the COW regions. + * If it's not possible (or not necessary) then write the + * guest data now. + */ + if (!merge_cow(offset, bytes, qiov, qiov_offset, l2meta)) { + BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); + trace_qcow2_writev_data(qemu_coroutine_self(), + file_cluster_offset + offset_in_cluster); + ret = bdrv_co_pwritev_part(s->data_file, + file_cluster_offset + offset_in_cluster, + bytes, qiov, qiov_offset, 0); + if (ret < 0) { + goto out_unlocked; + } + } + + qemu_co_mutex_lock(&s->lock); + + ret = qcow2_handle_l2meta(bs, &l2meta, true); + goto out_locked; + +out_unlocked: + qemu_co_mutex_lock(&s->lock); + +out_locked: + qcow2_handle_l2meta(bs, &l2meta, false); + qemu_co_mutex_unlock(&s->lock); + + qemu_vfree(crypt_buf); + + return ret; +} + static coroutine_fn int qcow2_co_pwritev_part( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, int flags) @@ -2243,15 +2324,10 @@ static coroutine_fn int qcow2_co_pwritev_part( int ret; unsigned int cur_bytes; /* number of sectors in current iteration */ uint64_t cluster_offset; - QEMUIOVector encrypted_qiov; - uint64_t bytes_done = 0; - uint8_t *cluster_data = NULL; QCowL2Meta *l2meta = NULL; trace_qcow2_writev_start_req(qemu_coroutine_self(), offset, bytes); - qemu_co_mutex_lock(&s->lock); - while (bytes != 0) { l2meta = NULL; @@ -2265,6 +2341,8 @@ static coroutine_fn int qcow2_co_pwritev_part( - offset_in_cluster); } + qemu_co_mutex_lock(&s->lock); + ret = qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, &cluster_offset, &l2meta); if (ret < 0) { @@ -2282,73 +2360,20 @@ static coroutine_fn int qcow2_co_pwritev_part( qemu_co_mutex_unlock(&s->lock); - if (bs->encrypted) { - assert(s->crypto); - if (!cluster_data) { - cluster_data = qemu_try_blockalign(bs->file->bs, - QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); - if (cluster_data == NULL) { - ret = -ENOMEM; - goto out_unlocked; - } - } - - assert(cur_bytes <= QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); - qemu_iovec_to_buf(qiov, qiov_offset + bytes_done, - cluster_data, cur_bytes); - - if (qcow2_co_encrypt(bs, cluster_offset, offset, - cluster_data, cur_bytes) < 0) { - ret = -EIO; - goto out_unlocked; - } - - qemu_iovec_init_buf(&encrypted_qiov, cluster_data, cur_bytes); - } - - /* Try to efficiently initialize the physical space with zeroes */ - ret = handle_alloc_space(bs, l2meta); + ret = qcow2_co_pwritev_task(bs, cluster_offset, offset, cur_bytes, + qiov, qiov_offset, l2meta); + l2meta = NULL; /* l2meta is consumed by qcow2_co_pwritev_task() */ if (ret < 0) { - goto out_unlocked; - } - - /* If we need to do COW, check if it's possible to merge the - * writing of the guest data together with that of the COW regions. - * If it's not possible (or not necessary) then write the - * guest data now. */ - if (!merge_cow(offset, cur_bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ? 0 : qiov_offset + bytes_done, l2meta)) - { - BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); - trace_qcow2_writev_data(qemu_coroutine_self(), - cluster_offset + offset_in_cluster); - ret = bdrv_co_pwritev_part( - s->data_file, cluster_offset + offset_in_cluster, cur_bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ? 0 : qiov_offset + bytes_done, 0); - if (ret < 0) { - goto out_unlocked; - } - } - - qemu_co_mutex_lock(&s->lock); - - ret = qcow2_handle_l2meta(bs, &l2meta, true); - if (ret) { - goto out_locked; + goto fail_nometa; } bytes -= cur_bytes; offset += cur_bytes; - bytes_done += cur_bytes; + qiov_offset += cur_bytes; trace_qcow2_writev_done_part(qemu_coroutine_self(), cur_bytes); } ret = 0; - goto out_locked; -out_unlocked: qemu_co_mutex_lock(&s->lock); out_locked: @@ -2356,7 +2381,7 @@ out_locked: qemu_co_mutex_unlock(&s->lock); - qemu_vfree(cluster_data); +fail_nometa: trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); return ret; From patchwork Fri Aug 16 15:30:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1148295 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=virtuozzo.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4696mk5Sk3z9s00 for ; Sat, 17 Aug 2019 01:33:58 +1000 (AEST) Received: from localhost ([::1]:57784 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeEm-0002Us-NQ for incoming@patchwork.ozlabs.org; Fri, 16 Aug 2019 11:33:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58081) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyeBO-0007oW-Vg for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyeBM-0001b0-Et for qemu-devel@nongnu.org; Fri, 16 Aug 2019 11:30:26 -0400 Received: from relay.sw.ru ([185.231.240.75]:60444) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hyeBH-0001Qb-0u; Fri, 16 Aug 2019 11:30:19 -0400 Received: from [10.94.3.0] (helo=kvm.qa.sw.ru) by relay.sw.ru with esmtp (Exim 4.92) (envelope-from ) id 1hyeBD-0007x1-Vn; Fri, 16 Aug 2019 18:30:16 +0300 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Date: Fri, 16 Aug 2019 18:30:15 +0300 Message-Id: <20190816153015.447957-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20190816153015.447957-1-vsementsov@virtuozzo.com> References: <20190816153015.447957-1-vsementsov@virtuozzo.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 185.231.240.75 Subject: [Qemu-devel] [PATCH v4 5/5] block/qcow2: introduce parallel subrequest handling in read and write X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kwolf@redhat.com, vsementsov@virtuozzo.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" It improves performance for fragmented qcow2 images. It also affect 026 iotest, increasing leaked clusters number, which is not surprising when we run several sub-requests of qcow2 request in parallel. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 + block/qcow2.c | 125 ++++++++++++++++++++++++++--- block/trace-events | 1 + tests/qemu-iotests/026.out | 18 +++-- tests/qemu-iotests/026.out.nocache | 20 ++--- 5 files changed, 138 insertions(+), 29 deletions(-) diff --git a/block/qcow2.h b/block/qcow2.h index 998bcdaef1..fdfa9c31cd 100644 --- a/block/qcow2.h +++ b/block/qcow2.h @@ -65,6 +65,9 @@ #define QCOW2_MAX_BITMAPS 65535 #define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) +/* Maximum of parallel sub-request per guest request */ +#define QCOW2_MAX_WORKERS 8 + /* indicate that the refcount of the referenced cluster is exactly one. */ #define QCOW_OFLAG_COPIED (1ULL << 63) /* indicate that the cluster is compressed (they never have the copied flag) */ diff --git a/block/qcow2.c b/block/qcow2.c index 3aaa180e2b..36b41e8536 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -40,6 +40,7 @@ #include "qapi/qobject-input-visitor.h" #include "qapi/qapi-visit-block-core.h" #include "crypto.h" +#include "block/aio_task.h" /* Differences with QCOW: @@ -2017,6 +2018,60 @@ fail: return ret; } +typedef struct Qcow2AioTask { + AioTask task; + + BlockDriverState *bs; + QCow2ClusterType cluster_type; /* only for read */ + uint64_t file_cluster_offset; + uint64_t offset; + uint64_t bytes; + QEMUIOVector *qiov; + uint64_t qiov_offset; + QCowL2Meta *l2meta; /* only for write */ +} Qcow2AioTask; + +static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task); +static coroutine_fn int qcow2_add_task(BlockDriverState *bs, + AioTaskPool *pool, + AioTaskFunc func, + QCow2ClusterType cluster_type, + uint64_t file_cluster_offset, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, + QCowL2Meta *l2meta) +{ + Qcow2AioTask local_task; + Qcow2AioTask *task = pool ? g_new(Qcow2AioTask, 1) : &local_task; + + *task = (Qcow2AioTask) { + .task.func = func, + .bs = bs, + .cluster_type = cluster_type, + .qiov = qiov, + .file_cluster_offset = file_cluster_offset, + .offset = offset, + .bytes = bytes, + .qiov_offset = qiov_offset, + .l2meta = l2meta, + }; + + trace_qcow2_add_task(qemu_coroutine_self(), bs, pool, + func == qcow2_co_preadv_task_entry ? "read" : "write", + cluster_type, file_cluster_offset, offset, bytes, + qiov, qiov_offset); + + if (!pool) { + return func(&task->task); + } + + aio_task_pool_start_task(pool, &task->task); + + return 0; +} + static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, QCow2ClusterType cluster_type, uint64_t file_cluster_offset, @@ -2066,18 +2121,28 @@ static coroutine_fn int qcow2_co_preadv_task(BlockDriverState *bs, g_assert_not_reached(); } +static coroutine_fn int qcow2_co_preadv_task_entry(AioTask *task) +{ + Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); + + assert(!t->l2meta); + + return qcow2_co_preadv_task(t->bs, t->cluster_type, t->file_cluster_offset, + t->offset, t->bytes, t->qiov, t->qiov_offset); +} + static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, int flags) { BDRVQcow2State *s = bs->opaque; - int ret; + int ret = 0; unsigned int cur_bytes; /* number of bytes in current iteration */ uint64_t cluster_offset = 0; + AioTaskPool *aio = NULL; - while (bytes != 0) { - + while (bytes != 0 && aio_task_pool_status(aio) == 0) { /* prepare next request */ cur_bytes = MIN(bytes, INT_MAX); if (s->crypto) { @@ -2089,7 +2154,7 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, ret = qcow2_get_cluster_offset(bs, offset, &cur_bytes, &cluster_offset); qemu_co_mutex_unlock(&s->lock); if (ret < 0) { - return ret; + goto out; } if (ret == QCOW2_CLUSTER_ZERO_PLAIN || @@ -2098,11 +2163,14 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, { qemu_iovec_memset(qiov, qiov_offset, 0, cur_bytes); } else { - ret = qcow2_co_preadv_task(bs, ret, - cluster_offset, offset, cur_bytes, - qiov, qiov_offset); + if (!aio && cur_bytes != bytes) { + aio = aio_task_pool_new(QCOW2_MAX_WORKERS); + } + ret = qcow2_add_task(bs, aio, qcow2_co_preadv_task_entry, ret, + cluster_offset, offset, cur_bytes, + qiov, qiov_offset, NULL); if (ret < 0) { - return ret; + goto out; } } @@ -2111,7 +2179,16 @@ static coroutine_fn int qcow2_co_preadv_part(BlockDriverState *bs, qiov_offset += cur_bytes; } - return 0; +out: + if (aio) { + aio_task_pool_wait_all(aio); + if (ret == 0) { + ret = aio_task_pool_status(aio); + } + g_free(aio); + } + + return ret; } /* Check if it's possible to merge a write request with the writing of @@ -2315,6 +2392,17 @@ out_locked: return ret; } +static coroutine_fn int qcow2_co_pwritev_task_entry(AioTask *task) +{ + Qcow2AioTask *t = container_of(task, Qcow2AioTask, task); + + assert(!t->cluster_type); + + return qcow2_co_pwritev_task(t->bs, t->file_cluster_offset, + t->offset, t->bytes, t->qiov, t->qiov_offset, + t->l2meta); +} + static coroutine_fn int qcow2_co_pwritev_part( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, size_t qiov_offset, int flags) @@ -2325,10 +2413,11 @@ static coroutine_fn int qcow2_co_pwritev_part( unsigned int cur_bytes; /* number of sectors in current iteration */ uint64_t cluster_offset; QCowL2Meta *l2meta = NULL; + AioTaskPool *aio = NULL; trace_qcow2_writev_start_req(qemu_coroutine_self(), offset, bytes); - while (bytes != 0) { + while (bytes != 0 && aio_task_pool_status(aio) == 0) { l2meta = NULL; @@ -2360,8 +2449,12 @@ static coroutine_fn int qcow2_co_pwritev_part( qemu_co_mutex_unlock(&s->lock); - ret = qcow2_co_pwritev_task(bs, cluster_offset, offset, cur_bytes, - qiov, qiov_offset, l2meta); + if (!aio && cur_bytes != bytes) { + aio = aio_task_pool_new(QCOW2_MAX_WORKERS); + } + ret = qcow2_add_task(bs, aio, qcow2_co_pwritev_task_entry, 0, + cluster_offset, offset, cur_bytes, + qiov, qiov_offset, l2meta); l2meta = NULL; /* l2meta is consumed by qcow2_co_pwritev_task() */ if (ret < 0) { goto fail_nometa; @@ -2382,6 +2475,14 @@ out_locked: qemu_co_mutex_unlock(&s->lock); fail_nometa: + if (aio) { + aio_task_pool_wait_all(aio); + if (ret == 0) { + ret = aio_task_pool_status(aio); + } + g_free(aio); + } + trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); return ret; diff --git a/block/trace-events b/block/trace-events index d724df0117..7f51550ba3 100644 --- a/block/trace-events +++ b/block/trace-events @@ -61,6 +61,7 @@ file_paio_submit(void *acb, void *opaque, int64_t offset, int count, int type) " file_copy_file_range(void *bs, int src, int64_t src_off, int dst, int64_t dst_off, int64_t bytes, int flags, int64_t ret) "bs %p src_fd %d offset %"PRIu64" dst_fd %d offset %"PRIu64" bytes %"PRIu64" flags %d ret %"PRId64 # qcow2.c +qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_offset %zu" qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset 0x%" PRIx64 " bytes %d" qcow2_writev_done_req(void *co, int ret) "co %p ret %d" qcow2_writev_start_part(void *co) "co %p" diff --git a/tests/qemu-iotests/026.out b/tests/qemu-iotests/026.out index fb89b8480c..4849c9c90a 100644 --- a/tests/qemu-iotests/026.out +++ b/tests/qemu-iotests/026.out @@ -481,7 +481,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -55 leaked clusters were found on the image. +119 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -508,7 +508,9 @@ Event: refblock_alloc_write; errno: 28; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -No errors were found on the image. + +64 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b @@ -533,7 +535,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -542,7 +544,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -561,7 +563,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -570,7 +572,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -589,7 +591,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -598,7 +600,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. === L1 growth tests === diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache index 6dda95dfb4..6b56df7788 100644 --- a/tests/qemu-iotests/026.out.nocache +++ b/tests/qemu-iotests/026.out.nocache @@ -489,7 +489,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -55 leaked clusters were found on the image. +119 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -516,8 +516,10 @@ Event: refblock_alloc_write; errno: 28; imm: off; once: off; write qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -No errors were found on the image. -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 + +64 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Event: refblock_alloc_write; errno: 28; imm: off; once: off; write -b qemu-io: Failed to flush the L2 table cache: No space left on device @@ -541,7 +543,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -550,7 +552,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -569,7 +571,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -578,7 +580,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -597,7 +599,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -10 leaked clusters were found on the image. +74 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 @@ -606,7 +608,7 @@ qemu-io: Failed to flush the L2 table cache: No space left on device qemu-io: Failed to flush the refcount block cache: No space left on device write failed: No space left on device -23 leaked clusters were found on the image. +87 leaked clusters were found on the image. This means waste of disk space, but no harm to data. === L1 growth tests ===