From patchwork Tue Aug 24 10:15:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1520218 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=ZEtNOjB5; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Gv4rj3kzNz9sW5 for ; Tue, 24 Aug 2021 20:21:09 +1000 (AEST) Received: from localhost ([::1]:59394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mITYI-0000FA-SB for incoming@patchwork.ozlabs.org; Tue, 24 Aug 2021 06:21:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITT5-0002jl-Ed; Tue, 24 Aug 2021 06:15:45 -0400 Received: from mail-am6eur05on2091.outbound.protection.outlook.com ([40.107.22.91]:59744 helo=EUR05-AM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITT1-00055n-41; Tue, 24 Aug 2021 06:15:40 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FJoojckulCsLVXNtBxrIXYj9Bz1K3KuG0bGyiPkVH8s+349Ox3w51Y0zlsP4qwj53+6Z4I4K5Ild/ywDzAg5Wd6d+3Bd8Hcawnbn79xxmUusflkby24ncDiEw92gk8oPMOWR0Nqx+QO4Qv+Mq6xPAAr78MCwlum1nMHrQrcgpJUbEFHqv7YOZRGvTjZsrSu+wIuNxYY7uGtpCzfbrVSTWOxjlzsL4F2yeDDkanJSGT1JRTh7mO9XUFWzXB5JhAvdXTcEg43UAj4W6XGxmAaTDpTAy12CCSDdRmC2ssWkkmJl4oyPR3ZdFWpo7q3xfHrc585vsbKEuVRapIB6ykCjVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z7Zn/9Xwrld53q2hmwMxOEqfB6plys68/XPyHTn1vIg=; b=oXQx+V86xiVSyxuYOZw0tQSgjsHOeqKSqC6BPvYAvYLC34yeK86sQ2K1QR//CtnH9NmJNltjqkHoX8se99NU9XjqyR/Q4pIlB8Z07HPW6IspqXztsG2UZK8F7vc6g2QMxl+j4pW/rX7kb5f1OvNK59LnzzVNGEKR/Z2YIfniU0YE/OXWWwYCTPCqmj1ctz/zif/yOEHRTOEDLs1tXwsxQKIEl+7u+W9Hw/HGhQqD1ZjIne4FfJ23ilQUqiZQ9nP3RxIzqohTzmjKqZ2p5rNXAL5s36XwbR+oUk0ciLf9VllsFXD/jm19VI2j0MT7w+ouCjp0Hb7lsK1PJpTxUFi3hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z7Zn/9Xwrld53q2hmwMxOEqfB6plys68/XPyHTn1vIg=; b=ZEtNOjB5ZBlpr1hgiihVEyyCAnBIaJtR/s2pQWUA3sfCKRjHY5QyEaEXH6Zulsp08pgED7COm36mV3gYlZWsAN8VNsVGN8cLNuZ1tbTD2zn8K4bxuuRNg+Mk41HgVuGKRyxAfCeIwKvjF63TuofuLYc4LnbP8rRlM+oC50YHd9M= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM5PR0801MB2099.eurprd08.prod.outlook.com (2603:10a6:203:4d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.22; Tue, 24 Aug 2021 10:15:31 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423%2]) with mapi id 15.20.4436.025; Tue, 24 Aug 2021 10:15:31 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com, hreitz@redhat.com, kwolf@redhat.com, den@openvz.org, eblake@redhat.com Subject: [PATCH v2 1/3] simplebench: add img_bench_templater.py Date: Tue, 24 Aug 2021 13:15:15 +0300 Message-Id: <20210824101517.59802-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210824101517.59802-1-vsementsov@virtuozzo.com> References: <20210824101517.59802-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.249) by HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19 via Frontend Transport; Tue, 24 Aug 2021 10:15:30 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 946c4298-1d2b-4fcb-97fe-08d966e81a90 X-MS-TrafficTypeDiagnostic: AM5PR0801MB2099: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1850; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fFOGJqyHZlNg47QfBH5njBQi4IWYLJxYKdTIvDJKXUltCCgWQwUBXoobCpztRqORzzrTHr6by/UH87fJqjh6VgO6kXz11DVzPmAzrFltoyKiO4DwPoWzT+hA9k4HObT3BIUlKC6iUckOQESXztdV9iBUJ6fVUOI3VHQLl1YjA2+fy2Qnsrs2Ar2I/Zyzbejg+OAkOqNKedtHeiGlaWMt0z4iQwNwektjvSmMysQ+pfkDKSpbr01ysNJ4y5r8K21h3mIvpomBfr8c7yu0i2PsDYxL8x/+hyfjpwnlD+Jq4RFYiIrg7wUVxuBGIg4LwFGVGq34Tpy/7lSr34lkKBtSjX27bvX2tXRtcg9kDOUyg/5g+1ILc55W0gv57ZvDoovMY/hyX/Gx3NbDgGztUFgKl8wQxZZ2nyD3VI0ExaPXTiG9iCZUkJM/fq7m1sfS4BMM1rjsvBOaDbrg/8L3jMPFYZs8B02JzohVjEm0/aG/2rio4Ux0liV7JRAYNGuVZM9kSIZjd/Y12mjnpzr82ilA/BEHRUnKMdCQJgr+aQ2Xvfcft+JrYeD5IXxwlXe7acxxgDdRw6qhLaTPT0gZnb03XE6B/NNIOJS5NF4di9Qe/B5d7M/mRwlPHk+Wohb/L27vjH6jo9ytGcRjKcyCCKwcvqWPW40bbfM0tdgmxcJ9/zUwVqAi1ZucEAAidd5+pZ7P1Bde8zSFMc2yIbSuLhQbdzeCFsW15HAuBFNMhgp8k25QTzkLOjLhNdbpuJ5HGjxCu1JZ24E0TBWEuNkD1bKXFD5A5rKbSnb3shPYFBZIa09DpeJDt/XXyzAYYE8ojjxa X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(136003)(39840400004)(366004)(956004)(26005)(5660300002)(8676002)(2906002)(186003)(4326008)(6916009)(1076003)(2616005)(478600001)(6666004)(316002)(6506007)(38350700002)(38100700002)(6512007)(86362001)(6486002)(66946007)(52116002)(66476007)(66556008)(8936002)(36756003)(2004002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Edg0/UJ5qkpnuJpgINZ7kOgF4CpY?= =?utf-8?q?dqmCns2veEWj1DnG7APJwOICa/GAR129zIVo+P9+MsuGrebH+2EUElA1t2GB3NBza?= =?utf-8?q?/lM0U61SoFEkJq4QcIp4TuudTccbLcvG8tKdjNBxOGonty0Ya1kChS2UIFJ9wLAoQ?= =?utf-8?q?ZLsVDKKfn1NdQ7XbpG+kJNGQr7gh7g1WmCSV1AdT6uhhSSIKzjIM3vHP0UB0q6MNo?= =?utf-8?q?tddlC51wN8faionIcVM1s5AWazbOZV9i74WMcyzK5OCcwLsINXOJIz1DWt9ekAcJF?= =?utf-8?q?Sj4kh59S+SUwfGZmhnEp2Gmq/WdyAkC9twjCl9+3sXCovHvkH9ewMUzOi87NFtwXa?= =?utf-8?q?6nmknNcYTBQ1nHTNE9hJtEG2omkLkq7YyauNOAHY49dCv9kDFe0nyfuVvHSgHLyzl?= =?utf-8?q?havw6tCr7zMbvW16Z34QPl00yjkwgl5eYh6BqbQs6qRGtffu0ye7K5L8MuHZtgI1c?= =?utf-8?q?ZmQEE0THrX3U5Ov1CrrasDyzfMU2Aq6v2I4mhs0wIV1UboSf5LLjKWxRhLIgFiXz3?= =?utf-8?q?+EBmQZT19QqwegO2At5eaGzk3VVh12WFc0FfLM2ObtgEwqobfOf139+Iuons0qzho?= =?utf-8?q?48HilAw1nNgzQYELxZx9avfsrCHwjgvzl8MIPYRxycSAPPVoi+Jz8CaICazHqNuQT?= =?utf-8?q?uEiQTGCHwgNRiY1ARx5cVJJt0oh3XDznDgHkdY9Vxc+MJMR3Xz/xcpy6EOvOI67YQ?= =?utf-8?q?eWzy454XvJgzeXWQEi6tEgAQgeuH5DGcIXq/Ky4tz3sb3b31LJPMgrzmQLECrgar8?= =?utf-8?q?iBDfsZj+jSG003hOUvYTnOFQLU0wi/XZPrCxxWE/4OIeANZHVoFIR0OntjH/rhb1Y?= =?utf-8?q?ojRs4T+BZxlyHuK8hg4OaG1dGUCwI6MKKit/qaTlfqPOXglS++Zq+n1e9vqWxp2P2?= =?utf-8?q?UBZoEN+YkdtTSW0WJuyzIxtgel1MoDph38N011MKrn5nYaYEE5ws6HQVnPRVCo1Z5?= =?utf-8?q?+qm9mO14cS/HB+96pZgSJ4pI0mKOQ7c8RUK+ab8LRuRIhf/mONoKNmPs/2I5J0Ppz?= =?utf-8?q?xdEJzNH1g7+p5ePDWhMTzbJsFLQyIpnu8sxgn3Ho8Vrg4qGODkS+kPIlrtEVdwF2p?= =?utf-8?q?30IGCfeXd0cAUI3TcxElfMnXab+7AFWuaiBi5E5LOLm9dCs9FWf5QEKjfi62WOMCX?= =?utf-8?q?n6QGrUX/bYka4yjJc4//ipdh8DKhYFD2Km36dl24pkpshX0qygZiXpSaHAXNxjFol?= =?utf-8?q?FHtOrYVWuhzkpvX5uXgrO0CJVMPk1diuhZmGLwmrqOA81qfAN28wz2aFTjeRIwOA0?= =?utf-8?q?iQBwwPbyJdkzpRi7?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 946c4298-1d2b-4fcb-97fe-08d966e81a90 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2021 10:15:31.5418 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7UyntHOmDB0j6WTfEntoB5uudxhc5BnZ5shvA6sLm4WIltXyw3aihjfdlwSb43ujHtR5RweVfAqrjSAfvDvVhlQxPW03AiZHYBov8nQe4oI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB2099 Received-SPF: pass client-ip=40.107.22.91; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-AM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add simple grammar-parsing template benchmark. New tool consume test template written in bash with some special grammar injections and produces multiple tests, run them and finally print a performance comparison table of different tests produced from one template. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Hanna Reitz --- scripts/simplebench/img_bench_templater.py | 95 ++++++++++++++++++++++ scripts/simplebench/table_templater.py | 62 ++++++++++++++ 2 files changed, 157 insertions(+) create mode 100755 scripts/simplebench/img_bench_templater.py create mode 100644 scripts/simplebench/table_templater.py diff --git a/scripts/simplebench/img_bench_templater.py b/scripts/simplebench/img_bench_templater.py new file mode 100755 index 0000000000..f8e1540ada --- /dev/null +++ b/scripts/simplebench/img_bench_templater.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python3 +# +# Process img-bench test templates +# +# Copyright (c) 2021 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + + +import sys +import subprocess +import re +import json + +import simplebench +from results_to_text import results_to_text +from table_templater import Templater + + +def bench_func(env, case): + test = templater.gen(env['data'], case['data']) + + p = subprocess.run(test, shell=True, stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, universal_newlines=True) + + if p.returncode == 0: + try: + m = re.search(r'Run completed in (\d+.\d+) seconds.', p.stdout) + return {'seconds': float(m.group(1))} + except Exception: + return {'error': f'failed to parse qemu-img output: {p.stdout}'} + else: + return {'error': f'qemu-img failed: {p.returncode}: {p.stdout}'} + + +if __name__ == '__main__': + if len(sys.argv) > 1: + print(""" +Usage: img_bench_templater.py < path/to/test-template.sh + +This script generates performance tests from a test template (example below), +runs them, and displays the results in a table. The template is read from +stdin. It must be written in bash and end with a `qemu-img bench` invocation +(whose result is parsed to get the test instance’s result). + +Use the following syntax in the template to create the various different test +instances: + + column templating: {var1|var2|...} - test will use different values in + different columns. You may use several {} constructions in the test, in this + case product of all choice-sets will be used. + + row templating: [var1|var2|...] - similar thing to define rows (test-cases) + +Test template example: + +Assume you want to compare two qemu-img binaries, called qemu-img-old and +qemu-img-new in your build directory in two test-cases with 4K writes and 64K +writes. The template may look like this: + +qemu_img=/path/to/qemu/build/qemu-img-{old|new} +$qemu_img create -f qcow2 /ssd/x.qcow2 1G +$qemu_img bench -c 100 -d 8 [-s 4K|-s 64K] -w -t none -n /ssd/x.qcow2 + +When passing this to stdin of img_bench_templater.py, the resulting comparison +table will contain two columns (for two binaries) and two rows (for two +test-cases). + +In addition to displaying the results, script also stores results in JSON +format into results.json file in current directory. +""") + sys.exit() + + templater = Templater(sys.stdin.read()) + + envs = [{'id': ' / '.join(x), 'data': x} for x in templater.columns] + cases = [{'id': ' / '.join(x), 'data': x} for x in templater.rows] + + result = simplebench.bench(bench_func, envs, cases, count=5, + initial_run=False) + print(results_to_text(result)) + with open('results.json', 'w') as f: + json.dump(result, f, indent=4) diff --git a/scripts/simplebench/table_templater.py b/scripts/simplebench/table_templater.py new file mode 100644 index 0000000000..950f3b3024 --- /dev/null +++ b/scripts/simplebench/table_templater.py @@ -0,0 +1,62 @@ +# Parser for test templates +# +# Copyright (c) 2021 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import itertools +from lark import Lark + +grammar = """ +start: ( text | column_switch | row_switch )+ + +column_switch: "{" text ["|" text]+ "}" +row_switch: "[" text ["|" text]+ "]" +text: /[^|{}\[\]]+/ +""" + +parser = Lark(grammar) + +class Templater: + def __init__(self, template): + self.tree = parser.parse(template) + + c_switches = [] + r_switches = [] + for x in self.tree.children: + if x.data == 'column_switch': + c_switches.append([el.children[0].value for el in x.children]) + elif x.data == 'row_switch': + r_switches.append([el.children[0].value for el in x.children]) + + self.columns = list(itertools.product(*c_switches)) + self.rows = list(itertools.product(*r_switches)) + + def gen(self, column, row): + i = 0 + j = 0 + result = [] + + for x in self.tree.children: + if x.data == 'text': + result.append(x.children[0].value) + elif x.data == 'column_switch': + result.append(column[i]) + i += 1 + elif x.data == 'row_switch': + result.append(row[j]) + j += 1 + + return ''.join(result) From patchwork Tue Aug 24 10:15:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1520214 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=ZTxAnRy1; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Gv4pQ5pbQz9sW5 for ; Tue, 24 Aug 2021 20:19:10 +1000 (AEST) Received: from localhost ([::1]:49428 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mITWO-00029B-ID for incoming@patchwork.ozlabs.org; Tue, 24 Aug 2021 06:19:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52184) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITT7-0002ld-O9; Tue, 24 Aug 2021 06:15:46 -0400 Received: from mail-am6eur05on2091.outbound.protection.outlook.com ([40.107.22.91]:59744 helo=EUR05-AM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITT6-00055n-7v; Tue, 24 Aug 2021 06:15:45 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Sp6U0CZqa7bI4Cd2bJakaKU+AitgobEtZlEWQi7uLvOlSpFa0TjE/PwjmSpA0OdhAhR3Odh8Zc6+Ok6ek4Epqw5sGok3h7T+0BlI4V2LnUTj6WMK5sbdwV9gkJ1jB4vAbyCwoytm6bIdvkhE9rhcYZ1fxpa77X7gok7jp7h2dvQbcg+rn1Jp5eePBINM4A724Gaf+LhFw0AAtzhO+umIyXLrh7+agPWJvTUHCudBpsR4IiE9RXJaq9z6G2dzu0NJHDtU3OGCaKiQOfRUicfKMv/TA9CDhqCnk8knT/Vhw6Qjsm/BsNDm8K69mnPvtbUmP+SUvoqEmXAb60FqnxXcTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hw2EpU0uol72ORxoZAKrL4Rp7CPVgaUDPLaqQSFbaG0=; b=lUZo/4PeZDPbkLyaI50PLIka/t9Xm4jpx895a8xTqNCR8cBk/tI7Fp7uqaTow5CxCcHif03geo4C7SQ40y56Rnsd4ojZy17r9RpvcK4X6M3GJwcztcEG5Qx83Y1jInT9tSpluxIv7JCPNajUWIwxVcxG8yHSuJxX0uvdb9q2DS9gBj64EVZ3gYtSymTHy8Y+iXliilOxm7tt8Br9SF8pQhQkra5WVYhaiwoT/BmXumurqFixNypXvz0BaVHUNd+qVfC3D+cAJJb5Q+1xj9oNCtUnKxCpi3fBwU+M9gi9RL3RihiJP1kCBSZvJ+I+dKJ4mn7rNsl6KVJh16e3P0BEAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hw2EpU0uol72ORxoZAKrL4Rp7CPVgaUDPLaqQSFbaG0=; b=ZTxAnRy1IccHS93Xpt9sKRAjENi8XJrxeQxjNI8o/qf7ZKrQU81jDt1Lwko2SNTl7HMPD7eIsvlyMt3ye4Vq/NzsWgpTs+VkhGgskZ9YRrdgdw8F/mvNb3oxlDQMft1EXifrx3tpmxVjszZTmC7axbTQMBmFiOTVMJ5Yq1gp3ZU= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM5PR0801MB2099.eurprd08.prod.outlook.com (2603:10a6:203:4d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.22; Tue, 24 Aug 2021 10:15:33 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423%2]) with mapi id 15.20.4436.025; Tue, 24 Aug 2021 10:15:32 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com, hreitz@redhat.com, kwolf@redhat.com, den@openvz.org, eblake@redhat.com Subject: [PATCH v2 2/3] qcow2: refactor handle_dependencies() loop body Date: Tue, 24 Aug 2021 13:15:16 +0300 Message-Id: <20210824101517.59802-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210824101517.59802-1-vsementsov@virtuozzo.com> References: <20210824101517.59802-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.249) by HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19 via Frontend Transport; Tue, 24 Aug 2021 10:15:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5c68e32e-9fa0-438a-61f5-08d966e81b5d X-MS-TrafficTypeDiagnostic: AM5PR0801MB2099: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:134; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zp8Rohcfnn5cSeFrFkLK+WxBHKQri/gex3kC22wUTQkvAJlir8MEd0BVWqIFd3ueaMylMRSY7bgcYmn4nmATbOCKpNWmk9kHawgkRROYCCWPqKcLqF9kxvSkXggJr8E0xPGJ7NvsiUYifRxE+OkxicrZsiogY2TWNJAegER2qICjX86QS8fRGfekkDleZJ/ebcteHdwGibxulLL3OSMLULK3f7JWRi/vQaCDJTWBdxF4ykM0p19OwdQy29K9glvgrwoTeqtiHk4jesoHP4VYh1YjTewWb6Tcnvxuoi5duZ/QhO2qdC6m2FkrA8s82C1FdiMJfXq+F2WkaBjhM8oZ0pXY4VLNbqz9X1Da5GUO6UAaD2nmA1q3nKKHPJDCFet/ylLGseu2PEAj6v0uR/nrhd0sCqN5Lo8os3HP2zXwpvN716FtTT10Cli5TXCmmP12wz11hCfmCI06fp7g0Xk98qG//NgtR9W7L+pyt7tKZgnozfy4dN5DknGS8SwoK6zapdFZYk3+jldtcVHgEhXPsHqzNXncK4iBGi+bcMQEtPQCJ1KwzyRXYYLSjr00hMrtsrhE9C7LaNljCpSayn8qbJThbYzGtXvJwXr5nM07UkIdKj6q6afDtBbnt0neSWCdpZxalCyN3KRvmdCV3UCQKMH7YiXNawii8dOmIyrGdAFex8ZOKbDADAtaSgjY7TtgJn756IKuL7s4X8NkoHp67g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(136003)(39840400004)(366004)(956004)(26005)(5660300002)(8676002)(2906002)(186003)(4326008)(6916009)(1076003)(83380400001)(2616005)(478600001)(6666004)(316002)(6506007)(38350700002)(38100700002)(6512007)(86362001)(6486002)(66946007)(52116002)(66476007)(66556008)(8936002)(36756003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Rs0XLTSPOL8nsiK1qwrcMmwRBim50sHIo7I67Hp9ZGOWzzDNWOmudEVclk9vW+dcla3b0yFJp/txwvmsikg5A2csp258CzIhOr2pXkyQOKOHypCz2VqiN099xVMdhuucqVEucFaHPCtK0HGGvBsHO/n9N4yO+9/rYsiSh4izasvupOuTvoTGMTWEXypVU1t7xx9DRZSiNIzF7dYbiZB4WdH0g1HPy4ToRvOiGrLuwI/nDnOPAQzVnkipVZYcM8APi5Bb1eGva9XXnaLOxz3EBFFGpSXxyfBVm+Gb9h9ZaU3/LbJURlw5RSKAOlv3AVGPzeDLIolYC3AZWfpWSiWpzRFJtP7sL0z4Exeuuq+FZ4zD/kbU08Z1e/+NpMH3MGO2e3Od1XvcdfN6l6UDyD1y25dvSQxk+0uEF0TgBzFu7S4PPk8LuIIbDwGxCyKxX7wfVyDU6Q2k3Y3IW1zClGPqQEFQCDtb3VQjW9nHgpNyvzNi4+WktmjhLL4aiuChdwyPqMtx/HkZQt0FYro+skgqnOJ82qACkgRAD4RJ07dt5zanymLABY3c4G9cHNC3zhmD/D9yKxtsz9yO2LAryrzwTE6cWAeVyGa0cP7/rf8Uh5/gbWo7UapUVjb2G5/aEL0rByrYlSuxXhd3zTCBy3aZwn4zrt1XC92YmK3Ihr3ddj6amkUBat9baFZHqTKNdL2NoKog14HFm0wJaYPw9k+FgCpkNEYLygVr8FW1Y1nbMxpmotTH8xLF8iF+KFWqQC3YWsrv3aaEfUykZJDwo9YMgQ5RyEewuQwMfaV2EYiimxgL/JZYnEWo6hh2ocJk1iQ+zgIGQIH+eZeJap4qkWEjwg2yzpdIghjJNc1dvHzxu7SbFiEslHKyVmvZ3PN1VHArxvzv/Lfov9vDLyB1EV6etJp7dd8uFLkYqQZ0Wi6mbRTRn0dAlq6FdeYuoOTXiMjRnZQpIZZekHNyFtDSG7tx74a70UjIx0lPDHXgFhENyf7nbV8YMHfUaodM0ZQFD7KQZy1+s2aDIT2Y/BNF4oIr+yNssommutub+VCRl9dB0hyLxZjHnlo4GI8omIQJR6YnbGN6TMBZsdjAIQsxXm9kq2iAHYRiRJs/BdEJR3oAUW3gxXSMAxyPBDVR/1Jayp2Zrjdoc074C3bWi2QR9MRF/UzdqPeZfXZqwq6hn+LyK7XWxzKmXE6e9CK0Wut3LP6oS8Qcytvew2FJ6bLySsTohMjy6D+YOWTvlBFCrTK4khTl/OK+AIiRPqu5BlCFw1QRqUcR1qiKAV47aJwaM85Jx1Efj0kYd9X02tzvLfubrc1YWowpQHURSu2SUv8/ZSzv X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5c68e32e-9fa0-438a-61f5-08d966e81b5d X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2021 10:15:32.8650 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: R1e6XKzPvhIhvj+vKe4FdZNKyA6pvXvs9FKE3OGdBxH3idkyTPj4ehseQCHO3D/O4RDG51RFvfWumV2wF++hrq5b51ofpl0V5YfU+samR1s= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB2099 Received-SPF: pass client-ip=40.107.22.91; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-AM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" No logic change, just prepare for the following commit. While being here do also small grammar fix in a comment. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- block/qcow2-cluster.c | 49 ++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index bd0597842f..9917e5c28c 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1400,29 +1400,36 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, if (end <= old_start || start >= old_end) { /* No intersection */ - } else { - if (start < old_start) { - /* Stop at the start of a running allocation */ - bytes = old_start - start; - } else { - bytes = 0; - } + continue; + } - /* Stop if already an l2meta exists. After yielding, it wouldn't - * be valid any more, so we'd have to clean up the old L2Metas - * and deal with requests depending on them before starting to - * gather new ones. Not worth the trouble. */ - if (bytes == 0 && *m) { - *cur_bytes = 0; - return 0; - } + /* Conflict */ - if (bytes == 0) { - /* Wait for the dependency to complete. We need to recheck - * the free/allocated clusters when we continue. */ - qemu_co_queue_wait(&old_alloc->dependent_requests, &s->lock); - return -EAGAIN; - } + if (start < old_start) { + /* Stop at the start of a running allocation */ + bytes = old_start - start; + } else { + bytes = 0; + } + + /* + * Stop if an l2meta already exists. After yielding, it wouldn't + * be valid any more, so we'd have to clean up the old L2Metas + * and deal with requests depending on them before starting to + * gather new ones. Not worth the trouble. + */ + if (bytes == 0 && *m) { + *cur_bytes = 0; + return 0; + } + + if (bytes == 0) { + /* + * Wait for the dependency to complete. We need to recheck + * the free/allocated clusters when we continue. + */ + qemu_co_queue_wait(&old_alloc->dependent_requests, &s->lock); + return -EAGAIN; } } From patchwork Tue Aug 24 10:15:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1520220 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=P+zIZki5; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Gv4td33MMz9sW5 for ; Tue, 24 Aug 2021 20:22:48 +1000 (AEST) Received: from localhost ([::1]:35900 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mITZu-0003WG-Nr for incoming@patchwork.ozlabs.org; Tue, 24 Aug 2021 06:22:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52198) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITTA-0002oD-PQ; Tue, 24 Aug 2021 06:15:48 -0400 Received: from mail-am6eur05on2091.outbound.protection.outlook.com ([40.107.22.91]:59744 helo=EUR05-AM6-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mITT8-00055n-Fw; Tue, 24 Aug 2021 06:15:47 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i/CPy2xE1HHCtYUk3FWrJNjexIUH/gXN9aalZO6kvlf8cbBkibPwTLx0I9fLm8rmuZuklCAcpGbeqguLNM1KV5YuoZG5HVnHRk3IXqmL0ZVqfAt5zb3PwSWiunlqKezJTHnXvWrnpVmRiQUSYXBLT3qOxcHC6OJRWREBil6ET3hHpNTRuXeHwO2iiuQpuEPglWkHtVXyi3ZdiqceWHsFmL42PodRaHy4pGVcylyf4/W+Xfdkn//7zp06qpkCJZST8tEMcQK66wOTBO+O/hS6NC+PBTo3bEwjUmGYSr2bUIgU6Xp0kUd7pGu+UW+CRBRt8ZEliaeGI9OiipsVlR+iaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KXcNwm+eMUS1pPN8JsAiHf+F+pcIAIM+ARCw8vz4nOk=; b=O2v9iNMEveZ+cei80n0TGTemEvhZQ/bcYf5Yq00mKBW/aswFAy/vB9SFq8P43/sDqSl6Ui0ckZNXNSaocZ3YnSJBk1z8OMctufzqbVMyzMG89MtQmtHiBH6Y7DNY6IYu4dUogginfi0w1z5IvPK9e2GHFn4u/MOSMsV8JjcINottOI2x9nFEbClPrc07dwOjwNhHRP5oQdEmyAqKUMmLaCvyw3e7w8DJC2BL0LqYbKt71rcWthQEQoN+36j2UMYuyCGd5QzSpL74x7tp7F0Y/BoVYY2vlKyXvYzTX/VtApifLtruYP3qXM7tR5vUnz0qEo0brt/jVWA0tX5teGci8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KXcNwm+eMUS1pPN8JsAiHf+F+pcIAIM+ARCw8vz4nOk=; b=P+zIZki5tsoxA7h0aWVDu2961pG8DTLoDA8f3c002Ia86qN0ZlDAudTxShHzw9MtjsRlGZwHdZVN3IZiyub0818Gu+pjVlzqf+DiJ9mtI6ei0hae3IPoZPokWcSeAKGIokaBUZziU6c38t2It52PnWtiZYLU5RfVNnTtDp/kM2Q= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM5PR0801MB2099.eurprd08.prod.outlook.com (2603:10a6:203:4d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.22; Tue, 24 Aug 2021 10:15:36 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::3c5e:7829:362d:1423%2]) with mapi id 15.20.4436.025; Tue, 24 Aug 2021 10:15:36 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com, hreitz@redhat.com, kwolf@redhat.com, den@openvz.org, eblake@redhat.com Subject: [PATCH v2 3/3] qcow2: handle_dependencies(): relax conflict detection Date: Tue, 24 Aug 2021 13:15:17 +0300 Message-Id: <20210824101517.59802-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210824101517.59802-1-vsementsov@virtuozzo.com> References: <20210824101517.59802-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.249) by HE1PR0102CA0058.eurprd01.prod.exchangelabs.com (2603:10a6:7:7d::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4436.19 via Frontend Transport; Tue, 24 Aug 2021 10:15:33 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ff8a656e-a7e5-4a98-32cd-08d966e81c34 X-MS-TrafficTypeDiagnostic: AM5PR0801MB2099: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2887; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6cZ2+WyaNdL8gwjSfAVrrSa3qRCWIOmh5xnonFfM5ssiHeV00r2M4XI8owF5f3KLg6hMqr3MAlob7iI32uVGEdS8FFPZBo7i6tcre2cwg2cj2oSQOEWKajIKWqYCari8Ttjd5LzzC0mDC4GDU9grdomrBe4hRJWKcdo3BtdBtIyT/IEdcsnHYA6dHV6Xh9gpj4o1HumdryoKk/8raii185nu3l7vyjjMfCPktuno4Z0JlXmptc8fQc5pbO5HRawz0ZonOl6jnwpbdVmM5wbFwbAjnP7IUK6Q4IduCN+9Xi88SV6Eh5kjDNU9ucbPSO5ySZnjxTeXVWdifTgxMpLQNW+NknQEFjJ+GtV0l2ePjsIxqgBao9ludGAXwn+U6PT9lJ9b5l9MZTvXVVZ403JhZI4cdOz2F9xBBj1xIWGKALgp1aOUHWaJ6Njh2XWTLAXX9z7LRC0uZZKazvJg2QQc0zTwnDjs2nFcN9Olii1KKw8/rje+X/ULXl5Vc5cUdZgPf7FiQLEJBjG4O8W6+FZIxnB+6LF9HWccMqihCLUNRCW7wUrzNDvE6V7kLOqudpbr/1qX7MMlRrlxCCeIL1RAtv9aiSrMXPwna2hKAD2ajJicdCnaG80R8LemS+RDjIr8w+ssh+iLzhenqkoFSyqa1KktN4vQFmkNGfkUcm+Nk4LHvyMkpeREKshrZa50/XhCD5mgi/hNc/hTj1mi8+ZVODFPbIKQRvG7AAzVfm5x0wA= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(136003)(39840400004)(366004)(956004)(26005)(5660300002)(8676002)(2906002)(186003)(4326008)(6916009)(1076003)(83380400001)(2616005)(478600001)(6666004)(316002)(6506007)(38350700002)(38100700002)(6512007)(86362001)(6486002)(66946007)(52116002)(66476007)(66556008)(8936002)(36756003)(21314003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?hraFhsY/HUeyfe4tGD6ITdSvo1yp?= =?utf-8?q?sIAe3knBcyIL48J6i3PZhX6h7zZ1Di6r8+OIq7i+VzJO/ceP23EqJ68v6qJpDjKmP?= =?utf-8?q?E5kwJB6wzK0BFGwoZAf8bQcruNWNxb+O7OgdBAwbhL/6d7qmo/Jhn/e1ReNtSF9b/?= =?utf-8?q?hx6dj/qItUE2+xPQIb+BWSCPAt2YgXYilx1kNakgyjYbGU5EsxqvenWD3FPviEvTy?= =?utf-8?q?IUhylXkqwzBB1VN9CqjVvgTKF8eynSu895CAEDupMpVsfkFqi5hKCLtor/yG1yZP+?= =?utf-8?q?bmzK36Xbx5YBywHRcjT1oOJ3F0ZSO0TP04bd0kKGgLQD4/67NrQZyqsA4RV5g9bGD?= =?utf-8?q?23JLlJUdKtlpeSVP9I4Y80GvNXOBvBPKugn9U5dtawOWTiMSmyZG1NC8+konMLfrJ?= =?utf-8?q?gU+qEscmvw2/IeKeWIP31SNztZFhtBUIIeKzmFJL+6iba0WIwDs9tfn9UR7HBrYd+?= =?utf-8?q?p2IlmbO9YQMhAproXfjWE2TywZ84rGwkyYXQrhfvI2XLbNfXPgGlvQZQH+Jo+0OE8?= =?utf-8?q?1kbBVqRaBssozvYsQkhpMGDkD1ZmseAT0xkVqs5yT9cV4X63UafmIRkEPY1m0QEfR?= =?utf-8?q?jw04yJd5xLETWFXtzCzFJygY4+iX3vlqyIsEHwNp8TE4zOcqPd5R4CVPg/UqwCRJI?= =?utf-8?q?izvvmY0tJgMnTuxvo0/VUsE1d6m8qDCwDaJy/dfaL5HVsk1XdbAsCqk5nQZCO/lYJ?= =?utf-8?q?xNGkbPSJQtwfvQvE0wAKh7kaQ2Y3EhrZMYB+pfXLebZMWhEvmIF78QQKjLCzn6xbl?= =?utf-8?q?Th09Ubj6/fB6WKbIXWyOGOop11qe7YYFlWVxBTRagJ/gqsflzzUj06GKPQH3ojJLd?= =?utf-8?q?EQNTfzdgZEFzZWgdJINVxU9jusrLVpHkwkm2hLeH+GKxKtjYTk6YQkKS/KcYA/mqM?= =?utf-8?q?LnPgEPUZ2zRzckmstbp9EVEpv3WPUQM6N40U68Vq/z+uXIrGbrr5+bJ78o0BDihi0?= =?utf-8?q?tte9ipxquMKXwO1IZ1kHSfeoqwKOb5woTkdRpYQYmssdCze7c5+KBy87g2FF1aO5v?= =?utf-8?q?OwE3qkLdaNAIu8cxwbz4EQltqg0FMAFUOn4m1y/QWBtqImRBCvlUFq0T3C92Zjdq4?= =?utf-8?q?CR7qYsO9DIlf+PZeDYvFz64WjkhRQc5RvevYmxOqcIVXMIvSfBlxqvS8j34KwZtSk?= =?utf-8?q?odFQ1CwAdJWlD7/4N0zOcE04HWTYHRpk5gxcE1bUsZLz66ndfxWxe7lNzomNlZHGq?= =?utf-8?q?KQMh29OjDjEP3KJb437gL9FZ+IWBPgXf4HZkt1RMPr+NHS03qNJn2L7/GXZSpsrms?= =?utf-8?q?XX465UyWbPRendfd?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: ff8a656e-a7e5-4a98-32cd-08d966e81c34 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Aug 2021 10:15:34.2290 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 67Mmq1cAE6MTgnGgWYlZVhAJ3BDFfXN+bd4S0xfqYDBu5nZaD6dT0a9KLHeMnne9SxEfpGxbL9LxMGzgyd3PlJBo9nGQWWUXCE+gIB3nveI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB2099 Received-SPF: pass client-ip=40.107.22.91; envelope-from=vsementsov@virtuozzo.com; helo=EUR05-AM6-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" There is no conflict and no dependency if we have parallel writes to different subclusters of one cluster when the cluster itself is already allocated. So, relax extra dependency. Measure performance: First, prepare build/qemu-img-old and build/qemu-img-new images. cd scripts/simplebench ./img_bench_templater.py Paste the following to stdin of running script: qemu_img=../../build/qemu-img-{old|new} $qemu_img create -f qcow2 -o extended_l2=on /ssd/x.qcow2 1G $qemu_img bench -c 100000 -d 8 [-s 2K|-s 2K -o 512|-s $((1024*2+512))] \ -w -t none -n /ssd/x.qcow2 The result: All results are in seconds ------------------ --------- --------- old new -s 2K 6.7 ± 15% 6.2 ± 12% -7% -s 2K -o 512 13 ± 3% 11 ± 5% -16% -s $((1024*2+512)) 9.5 ± 4% 8.4 -12% ------------------ --------- --------- So small writes are more independent now and that helps to keep deeper io queue which improves performance. 271 iotest output becomes racy for three allocation in one cluster. Second and third writes may finish in different order. Second and third requests don't depend on each other any more. Still they both depend on first request anyway. Filter out second and third write offsets to cover both possible outputs. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Reviewed-by: Hanna Reitz --- block/qcow2-cluster.c | 11 +++++++++++ tests/qemu-iotests/271 | 5 ++++- tests/qemu-iotests/271.out | 4 ++-- 3 files changed, 17 insertions(+), 3 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 9917e5c28c..c1c43a891b 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -1403,6 +1403,17 @@ static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset, continue; } + if (old_alloc->keep_old_clusters && + (end <= l2meta_cow_start(old_alloc) || + start >= l2meta_cow_end(old_alloc))) + { + /* + * Clusters intersect but COW areas don't. And cluster itself is + * already allocated. So, there is no actual conflict. + */ + continue; + } + /* Conflict */ if (start < old_start) { diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 index 599b849cc6..d9d391955e 100755 --- a/tests/qemu-iotests/271 +++ b/tests/qemu-iotests/271 @@ -893,7 +893,10 @@ EOF } _make_test_img -o extended_l2=on 1M -_concurrent_io | $QEMU_IO | _filter_qemu_io +# Second an third writes in _concurrent_io() are independent and may finish in +# different order. So, filter offset out to match both possible variants. +_concurrent_io | $QEMU_IO | _filter_qemu_io | \ + $SED -e 's/\(20480\|40960\)/OFFSET/' _concurrent_verify | $QEMU_IO | _filter_qemu_io # success, all done diff --git a/tests/qemu-iotests/271.out b/tests/qemu-iotests/271.out index 81043ba4d7..5be780de76 100644 --- a/tests/qemu-iotests/271.out +++ b/tests/qemu-iotests/271.out @@ -719,8 +719,8 @@ blkdebug: Suspended request 'A' blkdebug: Resuming request 'A' wrote 2048/2048 bytes at offset 30720 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -wrote 2048/2048 bytes at offset 20480 +wrote 2048/2048 bytes at offset OFFSET 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -wrote 2048/2048 bytes at offset 40960 +wrote 2048/2048 bytes at offset OFFSET 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done