From patchwork Sat Jun 20 14:36:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1313642 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=n+oFodgk; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49pyv64Z4tz9sSF for ; Sun, 21 Jun 2020 00:37:37 +1000 (AEST) Received: from localhost ([::1]:40366 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jmecf-0002Jp-47 for incoming@patchwork.ozlabs.org; Sat, 20 Jun 2020 10:37:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56758) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecF-0002Gd-5Z; Sat, 20 Jun 2020 10:37:07 -0400 Received: from mail-eopbgr50102.outbound.protection.outlook.com ([40.107.5.102]:35810 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecD-0004UK-G2; Sat, 20 Jun 2020 10:37:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oWq+q7/k1hJ3d1MxNAjeQNAIA5tvgQlsrxGaRJiH503csC96C2sfRcVXgHSdCGB6qKhQxB7WnvxsLQJC7Gsb60ybxjhrJd928Z3RWiGTZ1XsRjaz5h7FywmtU61YDdXzDhdNpylQrnVk8cTNfd9M1WwqpNHyN8/W655vfW3OMZgcIF4etxMTXMGmUB39rlUc7Lv9izbU4H7D2Blrgps9zuHh9/We2hfad5dkIGF1Z/WXelA5uD1w9qvPq46/0/wDDQGpba57N4k9qAOWSSDnSLQts+KSrwmZf/+V4IUAALcGqtzi8F+PptdJikFhpD6Z2G66dU7QHIeO6A/OwetlbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPcty3K0u8kAjPxsNmusUaw2qSIiqRdWRRt7a+bfmt8=; b=FYM1qOtEcTqmBs5dHigF9rBtr2u4AbKKW9nQwxGBiz2/sWyzHGwktxen7YxieEnH2w42FA+4B6uTnTqzyeB3egCdEGdhNGy1Pkl4EdtNIGIoTXysmAuystxWLmHZc1+y8ni+3dUPU3LAM13dDEdHICXTbzFlq7+ltFmcMNhmy5VOtAVnsdKH1NmiJcBaSeG4lhRXnIBq2SdevlJWh3Yys8VZoKGZDCupRA00QQ624+fHco+3CZZjz3SGXuu9S908JTE3fJlZauFT1KacBoM/PI5nfcO4PENxvxAwk8I4+YAGCNmmaa5vKdXxHpLckcC0tr+/mKwOrcrnpMbmud7z8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPcty3K0u8kAjPxsNmusUaw2qSIiqRdWRRt7a+bfmt8=; b=n+oFodgk8CU2Aw//VmuVcfAeMk/KXAsofT9eC6p/V0w7zUXpaFT3Fn3SXDn4yWv34zobg1Gmn9iU2IMNmCIj8PUuGhpZmcFQ8OpbLwm35N+X5QiQvDSvt9rTVZPoB+7AReQZKbcYHtYJNGnEPuHr87CEXsM9eGqyy1wPO3KKA3U= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sat, 20 Jun 2020 14:36:58 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%3]) with mapi id 15.20.3109.021; Sat, 20 Jun 2020 14:36:58 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 1/5] block/io: introduce bdrv_try_mark_request_serialising Date: Sat, 20 Jun 2020 17:36:45 +0300 Message-Id: <20200620143649.225852-2-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200620143649.225852-1-vsementsov@virtuozzo.com> References: <20200620143649.225852-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.122) by AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sat, 20 Jun 2020 14:36:57 +0000 X-Mailer: git-send-email 2.18.0 X-Originating-IP: [185.215.60.122] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0e6ad624-68c9-49c9-d4ff-08d8152762ff X-MS-TrafficTypeDiagnostic: AM7PR08MB5430: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:478; X-Forefront-PRVS: 0440AC9990 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uPIY2ZLB3gbQHRd9A/zuLuGmvTDmqByLCWLNxTxQxi+o+BCmQND3w6Ij8bN8hT28Li5yoikfddUU3QlOeIsKZCZ7UBNgheX053QOBSxifBFTfVyx9sgY6FPhc1c/YFZyQeYP2cSQwbCW7FMXTkbOX7/zdu0hIGLnBId/gjBF2SZjYk8tymIeDoAd83H3oN5CjMnqVWr/ZjKMYI6rMwEYOiOkKjVl6o47hyLz8R8QBMMp8tLpzuoEJOzxC1AjD+aDLoUAxEDBdSSl6x5qYnCF+8ucfXKxGtJ5KkOfSFwioXOd0hjHUBT8AUi0/p2P3z279syzBeTCgeePdKm08Q/VpA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(396003)(136003)(39840400004)(6666004)(26005)(186003)(16526019)(86362001)(2906002)(6916009)(6512007)(4326008)(83380400001)(52116002)(2616005)(6506007)(478600001)(956004)(36756003)(5660300002)(6486002)(8676002)(316002)(66946007)(8936002)(66476007)(1076003)(66556008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: z1vjSwixaE3yGb3HAY157LaSq9/R2+wO9zJdcd3eSlU+hSw/FpSgYnksg78xu4Q/Hk8lQNo9ZcZzp7N5ywt8ZcJRBpPfHW1tBjURFLUS3y7qsVbON2I8y6uml4zRCEuIDIIZ+tEnkprPygF+AF/v4kBM1aPtO04KzuBnImZjcif6afUYeFlsXnq1Ayzv3G+iC8Q73F8zhrp6v05OXMnOIpb2kzk3EJ/UUGr5prnwiRWrEKJxTFUw2J1pWDNGfbjYh52PoH2LX/p6v9kKEfzkh5n10HPkmHh2O1N6CLX4prbwJotgkbzsJp4pwnz1DEu8lIHKVgSKrb1MgeBSG9vmoOYtkJIRidM/8lf7Q3CbeShPf9U+fExOjlWKU2tiK20e7YzcfK+8GqFZDC4MHUbGTaM+bMxyHDFcqV1eJ6efI9qjifaUQ9b4MsRnn2RRQRhdFNJfvTXub5hHtu/GY63DhgGVVvyTWjNsN9kgxso5BwQ= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e6ad624-68c9-49c9-d4ff-08d8152762ff X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2020 14:36:58.3439 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pSJ3rB2IfNx+DpYIpR0W9P9GocT+kZdTTxzLVZO7erCY+lX/AJJ6OTRIMmMgAW54i9R8k3PtlFEWVHcnqG+xRqNwiVnyssmq7hp7DPpw9Sg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Received-SPF: pass client-ip=40.107.5.102; envelope-from=vsementsov@virtuozzo.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/20 10:36:59 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, Anton.Nefedov@acronis.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Introduce a function to mark the request serialising only if there are no conflicting request to wait for. The function is static, so mark it unused. The attribute is to be dropped in the next commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- block/io.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/block/io.c b/block/io.c index df8f2a98d4..60cee1d759 100644 --- a/block/io.c +++ b/block/io.c @@ -727,10 +727,11 @@ static bool tracked_request_overlaps(BdrvTrackedRequest *req, return true; } -static bool coroutine_fn -bdrv_wait_serialising_requests_locked(BlockDriverState *bs, - BdrvTrackedRequest *self) +/* Called with self->bs->reqs_lock locked */ +static bool coroutine_fn wait_or_find_conflicts(BdrvTrackedRequest *self, + bool wait) { + BlockDriverState *bs = self->bs; BdrvTrackedRequest *req; bool retry; bool waited = false; @@ -754,6 +755,9 @@ bdrv_wait_serialising_requests_locked(BlockDriverState *bs, * will wait for us as soon as it wakes up, then just go on * (instead of producing a deadlock in the former case). */ if (!req->waiting_for) { + if (!wait) { + return true; + } self->waiting_for = req; qemu_co_queue_wait(&req->wait_queue, &bs->reqs_lock); self->waiting_for = NULL; @@ -767,13 +771,18 @@ bdrv_wait_serialising_requests_locked(BlockDriverState *bs, return waited; } -bool bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) +/* Return true on success, false on fail. Always success with blocking = true */ +static bool bdrv_do_mark_request_serialising(BdrvTrackedRequest *req, + uint64_t align, bool blocking, bool *waited) { BlockDriverState *bs = req->bs; int64_t overlap_offset = req->offset & ~(align - 1); uint64_t overlap_bytes = ROUND_UP(req->offset + req->bytes, align) - overlap_offset; - bool waited; + int64_t old_overlap_offset = req->overlap_offset; + uint64_t old_overlap_bytes = req->overlap_bytes; + bool old_serializing = req->serialising; + bool found; qemu_co_mutex_lock(&bs->reqs_lock); if (!req->serialising) { @@ -783,11 +792,46 @@ bool bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) req->overlap_offset = MIN(req->overlap_offset, overlap_offset); req->overlap_bytes = MAX(req->overlap_bytes, overlap_bytes); - waited = bdrv_wait_serialising_requests_locked(bs, req); + found = wait_or_find_conflicts(req, blocking); + + if (!blocking && found) { + req->overlap_offset = old_overlap_offset; + req->overlap_bytes = old_overlap_bytes; + if (!old_serializing) { + atomic_dec(&req->bs->serialising_in_flight); + req->serialising = false; + } + } + + *waited = found && blocking; + qemu_co_mutex_unlock(&bs->reqs_lock); + + return blocking || !found; +} + +/* Return true if had to wait for conflicts */ +bool bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) +{ + bool waited; + bool success = bdrv_do_mark_request_serialising(req, align, true, &waited); + + assert(success); return waited; } +/* Return true on success, false if there are some conflicts */ +__attribute__ ((unused)) +static bool bdrv_try_mark_request_serialising(BdrvTrackedRequest *req, + uint64_t align) +{ + bool waited; + bool success = bdrv_do_mark_request_serialising(req, align, false, &waited); + + assert(!waited); + return success; +} + /** * Return the tracked request on @bs for the current coroutine, or * NULL if there is none. @@ -865,7 +909,7 @@ static bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self } qemu_co_mutex_lock(&bs->reqs_lock); - waited = bdrv_wait_serialising_requests_locked(bs, self); + waited = wait_or_find_conflicts(self, true); qemu_co_mutex_unlock(&bs->reqs_lock); return waited; From patchwork Sat Jun 20 14:36:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1313640 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=m+gx6qaD; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49pyv674fZz9sT2 for ; Sun, 21 Jun 2020 00:37:38 +1000 (AEST) Received: from localhost ([::1]:40594 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jmeci-0002UF-NE for incoming@patchwork.ozlabs.org; Sat, 20 Jun 2020 10:37:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56772) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecI-0002OO-0C; Sat, 20 Jun 2020 10:37:10 -0400 Received: from mail-eopbgr50102.outbound.protection.outlook.com ([40.107.5.102]:35810 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecF-0004UK-PL; Sat, 20 Jun 2020 10:37:09 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y55lQ0UgSGpO3wTgs26fcy+hfbmq7Yk/rMODwKCxsGMxx6+8tty1ehQoj0pOycS3PQBluc8FhUpaFpk091NYQ2Ym62B3vjtyD4vQENn4ZYa3DCPklxZMKSFkEoAXHbKAHjhOY0neMyXswto3iMRZcwoZ1QxYSSH/+l/phC/EvLKQMYyKiKowq2hEQaK7UTJY18yblJuG0dgWR23uR2fQrZD8da9S2vQc4dp5fIwivrdyXA4rGDWKI0RfrC8L9hPAtrGT19i/B9kCPVs9A721Twn6qV9Wxf1nbRisUwIAeZHVBuAODqrAhBODB3Hp2W+lo4mvp77m6W+HFi4sL/UO1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lR8zsYe66WSgb6b82kT6YRXSMo6yiJ8OZisrxVnh7sE=; b=YVy6FDhzEZKXzQwtdWYxBmVAUjfS80PMzMuvHIvzhe1M/dSzkqfd+uzQ/gPJ8cs0SdKrIvj+prwtnzLrI04M02NbgzTFlRpFyWUHy+ZJ8rUKrp5PNqWWP743UWjMcEgIOnWWISG5q2vUWa8wD0lCRMvM39NEkv03Qp0eNlguuw5Ngde1aTJD9VuKcdlRRlUkang+bJaozhWC9a5TI1VVVrTTKlcDWYQlu3t20IL1zQUOBeBKzWLrjLktS0LhmWz0CFoWvOp/AVIMt6aF2AOTRQ5x/NDGZdPq6/Xj6rdhAMQFi+Ak+/X1leK4ZqZGfdtpAAxKuH/tcJY+QX1H8sa4YQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lR8zsYe66WSgb6b82kT6YRXSMo6yiJ8OZisrxVnh7sE=; b=m+gx6qaDDFZqbGLS8dTnTgevJb0FtGBQ2x2nUaScFreVPFbpoltihMLfo2FQN2NVbus1EupZ1+YRvLUNiV/V41zM5/270HkaAva57ySgim25M45VjNtIKhDIHPSlYmr0TES8iwOGu6p2YnxJwhrjqQnkUqeDvvbmBYady3sEEk0= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sat, 20 Jun 2020 14:36:59 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%3]) with mapi id 15.20.3109.021; Sat, 20 Jun 2020 14:36:59 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 2/5] block/io: introduce bdrv_co_range_try_lock Date: Sat, 20 Jun 2020 17:36:46 +0300 Message-Id: <20200620143649.225852-3-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200620143649.225852-1-vsementsov@virtuozzo.com> References: <20200620143649.225852-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.122) by AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sat, 20 Jun 2020 14:36:58 +0000 X-Mailer: git-send-email 2.18.0 X-Originating-IP: [185.215.60.122] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d9a29a0f-02a8-4290-c40d-08d815276390 X-MS-TrafficTypeDiagnostic: AM7PR08MB5430: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-Forefront-PRVS: 0440AC9990 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZmjHVwDi3L+d2bQPp1Uc2oiykDaf8eRV0YiPgsErEF/4dFdsttizLii5FJu1cbYOwLQO0Bk/Kb9qlrAKICHHm1nHLIHmLgRiPgWayuwkeNWUdg/O38wOxphjpSm4rzJk+iRJzL04GTvoUUS1snADlDSIFydX61KbJWRm+gcRT/ptKrs/76Qee52uMsIrZ88y7Z2Yy0cW5cAnQ3OcrEw8J0Fak9sU1mCg/9Vht89LVnwkcnLUfX6v0oidtMMUMH5ny7ca8+Y1gfWMOrd+fVDf1NTyDjRrXp4pWiwzNZ1R9Q1e0+4I9OJVysxWvkexha0o X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(396003)(136003)(39840400004)(6666004)(26005)(186003)(16526019)(86362001)(2906002)(6916009)(6512007)(4326008)(83380400001)(52116002)(2616005)(6506007)(478600001)(956004)(36756003)(5660300002)(30864003)(6486002)(8676002)(316002)(66946007)(8936002)(66476007)(1076003)(66556008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: QztczFfhUyaerOWpIWwyNmo3NFQ5IJnTRpbaNARDCMij+YMzB9e4O5mp3ju9htTuGCLT6Y5PcfVxqYcGDavz8bMHZ/52CIayNrHH3bjnu9mWsijmtHb/7O2ax0rqdZ3jOlHqDfk5+cpg3MGpeo1wFhrfayXSX13aUD5HKMN/sPzB1RU/J85WaMS03/gRhXobMTzkNXqxrkDSVH+lMisteg7dbquJ3tTPv8dFc9hbENB9wAwlCKQfa9YfvcPaOnycF7gDyv7dcPEpIhlmO1L2/FggyXKq/Bi5O8MkKYBOnkKVT3H62kZPWY/Z9hu3oujCpYuCb+ptxZjoSwUIzlFaVKrpcIlwsBajI791QC2bdCQx/GfsIKpYhVJhDjr0XzdTc5GrXT5OyVDP9FXWhXNVK16pjVBDzJOZDtisI1AA+OSxLJM23mVCtxotv76cPdiElo1DpX+KyJwu+xkST/Y0MeOdHWCtszqYkwd+c0dvFtY= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: d9a29a0f-02a8-4290-c40d-08d815276390 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2020 14:36:59.2947 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m7BrrGZm8wc58+UkJjAiAyTSjLHSiWVfTKLXZDXE8VBBFPazwvREnvWU5ppZtOU3VpZrGMkvVBGMByc74hQwaovRzCusi6c/N7rbCIjrgAs= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Received-SPF: pass client-ip=40.107.5.102; envelope-from=vsementsov@virtuozzo.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/20 10:36:59 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, Anton.Nefedov@acronis.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Introduce an interface to make a "critical section" on top of serialising requests feature. This is needed to avoid intersecting with other requests during a set of operations. Will be used in a next commit to implement preallocate filter. To keep assertions during intersecting requests handling, we need to add _locked versions of read/write requests, to be used inside locked section. The following commit will need only locked version of write-zeroes, so add only it. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 9 ++++ include/block/block_int.h | 16 ++++++++ include/qemu/typedefs.h | 1 + block/io.c | 86 +++++++++++++++++++++++++++++++-------- 4 files changed, 95 insertions(+), 17 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 25e299605e..c4b2a493b4 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -391,6 +391,15 @@ int bdrv_pwrite_sync(BdrvChild *child, int64_t offset, */ int coroutine_fn bdrv_co_pwrite_zeroes(BdrvChild *child, int64_t offset, int bytes, BdrvRequestFlags flags); + +/* + * Version of bdrv_co_pwrite_zeroes to be used inside bdrv_co_range_try_lock() + * .. bdrv_co_range_unlock() pair. + */ +int coroutine_fn bdrv_co_pwrite_zeroes_locked(BdrvChild *child, + int64_t offset, int bytes, BdrvRequestFlags flags, + BdrvTrackedRequest *lock); + BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs, const char *backing_file); void bdrv_refresh_filename(BlockDriverState *bs); diff --git a/include/block/block_int.h b/include/block/block_int.h index 791de6a59c..9ec56f1825 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -66,6 +66,7 @@ enum BdrvTrackedRequestType { BDRV_TRACKED_WRITE, BDRV_TRACKED_DISCARD, BDRV_TRACKED_TRUNCATE, + BDRV_TRACKED_LOCK, }; typedef struct BdrvTrackedRequest { @@ -83,6 +84,12 @@ typedef struct BdrvTrackedRequest { CoQueue wait_queue; /* coroutines blocked on this request */ struct BdrvTrackedRequest *waiting_for; + + /* + * If non-zero, the request is under lock, so it's allowed to intersect + * (actully it must be inside) the @lock request. + */ + struct BdrvTrackedRequest *lock; } BdrvTrackedRequest; struct BlockDriver { @@ -994,6 +1001,15 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, int64_t offset, unsigned int bytes, QEMUIOVector *qiov, size_t qiov_offset, BdrvRequestFlags flags); +/* + * Creates serializing BDRV_TRACKED_LOCK request if no conflicts, on success + * return pointer to it, to be used in bdrv_co_range_unlock(). + */ +BdrvTrackedRequest *coroutine_fn bdrv_co_range_try_lock(BlockDriverState *bs, + int64_t offset, + int64_t bytes); +void coroutine_fn bdrv_co_range_unlock(BdrvTrackedRequest *req); + static inline int coroutine_fn bdrv_co_pread(BdrvChild *child, int64_t offset, unsigned int bytes, void *buf, BdrvRequestFlags flags) { diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h index ecf3cde26c..e2bbe3af96 100644 --- a/include/qemu/typedefs.h +++ b/include/qemu/typedefs.h @@ -28,6 +28,7 @@ typedef struct Aml Aml; typedef struct AnnounceTimer AnnounceTimer; typedef struct BdrvDirtyBitmap BdrvDirtyBitmap; typedef struct BdrvDirtyBitmapIter BdrvDirtyBitmapIter; +typedef struct BdrvTrackedRequest BdrvTrackedRequest; typedef struct BlockBackend BlockBackend; typedef struct BlockBackendRootState BlockBackendRootState; typedef struct BlockDriverState BlockDriverState; diff --git a/block/io.c b/block/io.c index 60cee1d759..0de72cdb4c 100644 --- a/block/io.c +++ b/block/io.c @@ -691,7 +691,8 @@ static void tracked_request_begin(BdrvTrackedRequest *req, BlockDriverState *bs, int64_t offset, uint64_t bytes, - enum BdrvTrackedRequestType type) + enum BdrvTrackedRequestType type, + BdrvTrackedRequest *lock) { assert(bytes <= INT64_MAX && offset <= INT64_MAX - bytes); @@ -704,6 +705,7 @@ static void tracked_request_begin(BdrvTrackedRequest *req, .serialising = false, .overlap_offset = offset, .overlap_bytes = bytes, + .lock = lock, }; qemu_co_queue_init(&req->wait_queue); @@ -745,15 +747,26 @@ static bool coroutine_fn wait_or_find_conflicts(BdrvTrackedRequest *self, if (tracked_request_overlaps(req, self->overlap_offset, self->overlap_bytes)) { - /* Hitting this means there was a reentrant request, for - * example, a block driver issuing nested requests. This must - * never happen since it means deadlock. + if (self->lock == req) { + /* This is atomic request under range_lock */ + assert(req->type == BDRV_TRACKED_LOCK); + assert(self->offset >= req->offset); + assert(self->bytes <= req->bytes); + continue; + } + /* + * Hitting this means there was a reentrant request (except for + * range_lock, handled above), for example, a block driver + * issuing nested requests. This must never happen since it + * means deadlock. */ assert(qemu_coroutine_self() != req->co); - /* If the request is already (indirectly) waiting for us, or + /* + * If the request is already (indirectly) waiting for us, or * will wait for us as soon as it wakes up, then just go on - * (instead of producing a deadlock in the former case). */ + * (instead of producing a deadlock in the former case). + */ if (!req->waiting_for) { if (!wait) { return true; @@ -821,7 +834,6 @@ bool bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) } /* Return true on success, false if there are some conflicts */ -__attribute__ ((unused)) static bool bdrv_try_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) { @@ -1796,7 +1808,7 @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child, bdrv_pad_request(bs, &qiov, &qiov_offset, &offset, &bytes, &pad); - tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_READ); + tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_READ, NULL); ret = bdrv_aligned_preadv(child, &req, offset, bytes, bs->bl.request_alignment, qiov, qiov_offset, flags); @@ -2169,9 +2181,9 @@ int coroutine_fn bdrv_co_pwritev(BdrvChild *child, return bdrv_co_pwritev_part(child, offset, bytes, qiov, 0, flags); } -int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, +static int coroutine_fn bdrv_co_pwritev_part_locked(BdrvChild *child, int64_t offset, unsigned int bytes, QEMUIOVector *qiov, size_t qiov_offset, - BdrvRequestFlags flags) + BdrvRequestFlags flags, BdrvTrackedRequest *lock) { BlockDriverState *bs = child->bs; BdrvTrackedRequest req; @@ -2215,7 +2227,7 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, * Pad qiov with the read parts and be sure to have a tracked request not * only for bdrv_aligned_pwritev, but also for the reads of the RMW cycle. */ - tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_WRITE); + tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_WRITE, lock); if (flags & BDRV_REQ_ZERO_WRITE) { ret = bdrv_co_do_zero_pwritev(child, offset, bytes, flags, &req); @@ -2239,8 +2251,23 @@ out: return ret; } +int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child, + int64_t offset, unsigned int bytes, QEMUIOVector *qiov, size_t qiov_offset, + BdrvRequestFlags flags) +{ + return bdrv_co_pwritev_part_locked(child, offset, bytes, qiov, qiov_offset, + flags, NULL); +} + int coroutine_fn bdrv_co_pwrite_zeroes(BdrvChild *child, int64_t offset, int bytes, BdrvRequestFlags flags) +{ + return bdrv_co_pwrite_zeroes_locked(child, offset, bytes, flags, NULL); +} + +int coroutine_fn bdrv_co_pwrite_zeroes_locked(BdrvChild *child, int64_t offset, + int bytes, BdrvRequestFlags flags, + BdrvTrackedRequest *lock) { trace_bdrv_co_pwrite_zeroes(child->bs, offset, bytes, flags); @@ -2248,8 +2275,8 @@ int coroutine_fn bdrv_co_pwrite_zeroes(BdrvChild *child, int64_t offset, flags &= ~BDRV_REQ_MAY_UNMAP; } - return bdrv_co_pwritev(child, offset, bytes, NULL, - BDRV_REQ_ZERO_WRITE | flags); + return bdrv_co_pwritev_part_locked(child, offset, bytes, NULL, 0, + BDRV_REQ_ZERO_WRITE | flags, lock); } /* @@ -2980,7 +3007,7 @@ int coroutine_fn bdrv_co_pdiscard(BdrvChild *child, int64_t offset, tail = (offset + bytes) % align; bdrv_inc_in_flight(bs); - tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_DISCARD); + tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_DISCARD, NULL); ret = bdrv_co_write_req_prepare(child, offset, bytes, &req, 0); if (ret < 0) { @@ -3252,7 +3279,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( if (recurse_src) { bdrv_inc_in_flight(src->bs); tracked_request_begin(&req, src->bs, src_offset, bytes, - BDRV_TRACKED_READ); + BDRV_TRACKED_READ, NULL); /* BDRV_REQ_SERIALISING is only for write operation */ assert(!(read_flags & BDRV_REQ_SERIALISING)); @@ -3269,7 +3296,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( } else { bdrv_inc_in_flight(dst->bs); tracked_request_begin(&req, dst->bs, dst_offset, bytes, - BDRV_TRACKED_WRITE); + BDRV_TRACKED_WRITE, NULL); ret = bdrv_co_write_req_prepare(dst, dst_offset, bytes, &req, write_flags); if (!ret) { @@ -3381,7 +3408,7 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, bool exact, bdrv_inc_in_flight(bs); tracked_request_begin(&req, bs, offset - new_bytes, new_bytes, - BDRV_TRACKED_TRUNCATE); + BDRV_TRACKED_TRUNCATE, NULL); /* If we are growing the image and potentially using preallocation for the * new area, we need to make sure that no write requests are made to it @@ -3494,3 +3521,28 @@ int bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, return bdrv_run_co(child->bs, bdrv_truncate_co_entry, &tco); } + +BdrvTrackedRequest *coroutine_fn bdrv_co_range_try_lock(BlockDriverState *bs, + int64_t offset, + int64_t bytes) +{ + BdrvTrackedRequest *req = g_new(BdrvTrackedRequest, 1); + bool success; + + tracked_request_begin(req, bs, offset, bytes, BDRV_TRACKED_LOCK, NULL); + success = bdrv_try_mark_request_serialising(req, bdrv_get_cluster_size(bs)); + if (!success) { + g_free(req); + req = NULL; + } + + return req; +} + +void coroutine_fn bdrv_co_range_unlock(BdrvTrackedRequest *req) +{ + assert(req->type == BDRV_TRACKED_LOCK); + + tracked_request_end(req); + g_free(req); +} From patchwork Sat Jun 20 14:36:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1313645 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=OjhDml/d; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49pywh3DK9z9sSF for ; Sun, 21 Jun 2020 00:39:00 +1000 (AEST) Received: from localhost ([::1]:47694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jmee2-0006r4-79 for incoming@patchwork.ozlabs.org; Sat, 20 Jun 2020 10:38:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56784) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecL-0002aS-U3; Sat, 20 Jun 2020 10:37:13 -0400 Received: from mail-eopbgr50102.outbound.protection.outlook.com ([40.107.5.102]:35810 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecI-0004UK-JJ; Sat, 20 Jun 2020 10:37:13 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DY1Tle0A2dpAFngjfitFf4+MWwiUk9YEbCHHNYnhwHbs7sJk1XBx9rZOTd3j2KaxrUmKkgY26GgHtVc+/m/ZaJDSeKFetNh0kmwQgTjQkyM4wYFQ6I3bcV5uBFZH2si2uhQReNBaw+/MAzY+TQsVl6Dx3ov8uL+6Y4s3wmsZJ6Mcimz7igakXzEXZ7Xdf/2nSPj5Q44H1Adhh9IDwXcJTiBdlyfsnpLJ3VPXo8fhRKyWGLQoda8U9uEMXlNyHxyjAbmAZFuOFVbxZcycl2aRZdsWAb7xr2meyxH8b4PbT79wNkEE4L/A4on/nELju+xbPp7I/ZkulgsgnOXqiC/olA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AHpqh/nGoe+Lav09ieBFabav9limNehGi2yb1lXU75g=; b=TlEnX1xRCiBusny6yHQ0YPfr1PswQ2NBhmBmP746mcCldoTtGb2yGHbwQ/8lnaWqZpoSu1w/rQinpFQPDy/1fDBFzkFl9FcaNA4eRyD+zLrbO5umz37W8ydjX+LjQ+WrtChPCMGwpwdR9VJ8Uq9HXnCoq28HcXIaoX4Hs9Pt482Y6EYjH99rKDDufNAMZMypYDuC0ZxV+qzSPRwwYJmSwJ5TT3UqXxb2++0M7FvxeNsXj+nBN+FIXvlmrtaNznw+GY33prTv0I+m1DMAALBT2FZauvqCN2qxBmjtIlOYNSflHs454pzziKLn0nC9iwL3eDejo5lAaMgW+0aES1NDGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AHpqh/nGoe+Lav09ieBFabav9limNehGi2yb1lXU75g=; b=OjhDml/doWXojak+m9sXjhVK5HBU3qdEu12RnHkwCAvOUri3RWX1l7Rn02y0aDC4RhGGmMa2K+S6LWZpKmI2BzM4ZQWnWGxtnZvwPjdZfU5/OfkJcFnYjYb2dLuyl3OcbL/p2Yj5OJNHuwblilB52lpYgAVrhGbpfTDhHG8/Uws= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sat, 20 Jun 2020 14:37:00 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%3]) with mapi id 15.20.3109.021; Sat, 20 Jun 2020 14:37:00 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 3/5] block: introduce preallocate filter Date: Sat, 20 Jun 2020 17:36:47 +0300 Message-Id: <20200620143649.225852-4-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200620143649.225852-1-vsementsov@virtuozzo.com> References: <20200620143649.225852-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.122) by AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sat, 20 Jun 2020 14:36:59 +0000 X-Mailer: git-send-email 2.18.0 X-Originating-IP: [185.215.60.122] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8d24ba8b-da83-43b6-ea30-08d815276422 X-MS-TrafficTypeDiagnostic: AM7PR08MB5430: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2449; X-Forefront-PRVS: 0440AC9990 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AK/o4FhNZwNWhzisjXSzNQQ6JXMe/xcNXseBhXdY7HKVxs0ZPutgG+H57jMYYTmCXtmyaLphW8ezjYGdJfZCigbgBUkDTof/LB/BU1RL4UwWJSepIRw4waSvGVOoEym8ge7ID34YkJNgsHmM5uroakYGSuFU3bhukPqz9yphsfNnBXYYstsXvYJdhCXc5hgjT9aIoSTgK7q32T6/wj5bMrvzodzwfOm9eH61uBALCnPWh8oMp2yLn8lXdsjwJmr/KsmZzbYmm3V25rPOykfgEKkxAVF+gqWoX/iHtxvMlEHrZM9J+qv3OtdoMuB8wf6LNFTTtVp7IZ+NnzNeEwuNRZdU4ragwONi/+iYiHkaFC1GJ1ukvgySowXzLQUrwN1W60rp+HhZ7rMMaZ4U56+bINen4BsAwjhs/Ii+NtKHEuRJjvXActgLocf533peqIzrs2oPOLPKB6Ch2k8RC89odw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(396003)(136003)(39840400004)(6666004)(26005)(186003)(16526019)(86362001)(2906002)(6916009)(6512007)(4326008)(83380400001)(52116002)(2616005)(6506007)(478600001)(956004)(36756003)(5660300002)(6486002)(8676002)(316002)(66946007)(8936002)(66476007)(1076003)(66556008)(2004002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: HwQZ25d9VOEg7ljk63r1xmnST2VHiTDWObBQbxng8G2r2DOq2Cg+LqvdQ82aqgme39nowbBnPybvVdKlqCCeqvePG2jsGNVrg2MWSpG2m+byhB3l9aPxt6AGCqqeB4erMWAMAnv0ZeYX4xf1oPLjPiUd/hexONpFKluJmujleUC3/o195vKyHjyC/wCPKGmVWfk/F+IZQFPEnwqFP71dIw7sNNg1yiwqv8wwC+296nZqbe5ZPgEgfrQI3FwId4CF2hR6MWKTnJzAz6EO6lPuQ6Wo+m+LXDbKktMkPwAVvdQnLUG9uduo+EC6zRcmBE9pRnp2Vj3UXUiOn4HsKmmG3vC04vXZl9LW2UQBqdrPphDZ/N3i3mFMTF0SKHoIMNAJ3i5+EoUeTGxgbHMmqi+l4v/vzAFDw1lxsqNMjBcVb7j/TGTM9VlDoWkigQGDRTlUhNqV/zSzy6rzY3Oi8ZaJ27SwNcLsMJ8FOPaW4Q2sIoo= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8d24ba8b-da83-43b6-ea30-08d815276422 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2020 14:37:00.2216 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F7vxczEWZOuzeLOBG5KXFX7rz96wVSF0Wc6NVjv3NkoReArtfb2sMHUlOl0J5rDFOA3CoWDj3W5iBegy6n83N5rCDmG7gfpumhIWR3siCTU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Received-SPF: pass client-ip=40.107.5.102; envelope-from=vsementsov@virtuozzo.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/20 10:36:59 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, Anton.Nefedov@acronis.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" It may be used for file-systems with slow allocation. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 3 +- block/preallocate.c | 255 +++++++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 1 + 3 files changed, 258 insertions(+), 1 deletion(-) create mode 100644 block/preallocate.c diff --git a/qapi/block-core.json b/qapi/block-core.json index 0e1c6a59f2..a0bda399d6 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -2805,7 +2805,7 @@ 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', - 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', + 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' }, 'sheepdog', 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs' ] } @@ -3995,6 +3995,7 @@ 'null-co': 'BlockdevOptionsNull', 'nvme': 'BlockdevOptionsNVMe', 'parallels': 'BlockdevOptionsGenericFormat', + 'preallocate':'BlockdevOptionsGenericFormat', 'qcow2': 'BlockdevOptionsQcow2', 'qcow': 'BlockdevOptionsQcow', 'qed': 'BlockdevOptionsGenericCOWFormat', diff --git a/block/preallocate.c b/block/preallocate.c new file mode 100644 index 0000000000..c272a6e41d --- /dev/null +++ b/block/preallocate.c @@ -0,0 +1,255 @@ +/* + * preallocate filter driver + * + * The driver performs preallocate operation: it is injected above + * some node, and before each write over EOF it does additional preallocating + * write-zeroes request. + * + * Copyright (c) 2020 Virtuozzo International GmbH. + * + * Author: + * Sementsov-Ogievskiy Vladimir + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include "qemu/osdep.h" + +#include "qemu/module.h" +#include "qemu/units.h" +#include "block/block_int.h" + + +typedef struct BDRVPreallocateState { + int64_t prealloc_size; + int64_t prealloc_align; + + /* + * Track real data end, to crop preallocation on close data_end may be + * negative, which means that actual status is unknown (nothing cropped in + * this case) + */ + int64_t data_end; +} BDRVPreallocateState; + + +static int preallocate_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) +{ + BDRVPreallocateState *s = bs->opaque; + + /* + * Parameters are hardcoded now. May need to add corresponding options in + * future. + */ + s->prealloc_align = 1 * MiB; + s->prealloc_size = 128 * MiB; + + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + false, errp); + if (!bs->file) { + return -EINVAL; + } + + s->data_end = bdrv_getlength(bs->file->bs); + if (s->data_end < 0) { + return s->data_end; + } + + bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + + bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & + bs->file->bs->supported_zero_flags); + + return 0; +} + +static void preallocate_close(BlockDriverState *bs) +{ + BDRVPreallocateState *s = bs->opaque; + + if (s->data_end >= 0 && bdrv_getlength(bs->file->bs) > s->data_end) { + bdrv_truncate(bs->file, s->data_end, true, PREALLOC_MODE_OFF, 0, NULL); + } +} + +static void preallocate_child_perm(BlockDriverState *bs, BdrvChild *c, + BdrvChildRole role, + BlockReopenQueue *reopen_queue, + uint64_t perm, uint64_t shared, + uint64_t *nperm, uint64_t *nshared) +{ + bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared); + + /* Force RESIZE permission, to be able to crop file on close() */ + *nperm |= BLK_PERM_RESIZE; +} + +static coroutine_fn int preallocate_co_preadv_part( + BlockDriverState *bs, uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, size_t qiov_offset, int flags) +{ + return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset, + flags); +} + +static int coroutine_fn preallocate_co_pdiscard(BlockDriverState *bs, + int64_t offset, int bytes) +{ + return bdrv_co_pdiscard(bs->file, offset, bytes); +} + +static bool coroutine_fn do_preallocate(BlockDriverState *bs, int64_t offset, + int64_t bytes, bool write_zero) +{ + BDRVPreallocateState *s = bs->opaque; + int64_t len, start, end; + BdrvTrackedRequest *lock; + int ret; + + if (s->data_end >= 0) { + s->data_end = MAX(s->data_end, + QEMU_ALIGN_UP(offset + bytes, BDRV_SECTOR_SIZE)); + } + + len = bdrv_getlength(bs->file->bs); + if (len < 0) { + return false; + } + + if (s->data_end < 0) { + s->data_end = MAX(len, + QEMU_ALIGN_UP(offset + bytes, BDRV_SECTOR_SIZE)); + } + + if (offset + bytes <= len) { + return false; + } + + lock = bdrv_co_range_try_lock(bs->file->bs, len, INT64_MAX - len); + if (!lock) { + /* There are already preallocating requests in-fligth */ + return false; + } + + /* Length should not have changed */ + assert(len == bdrv_getlength(bs->file->bs)); + + start = write_zero ? MIN(offset, len) : len; + end = QEMU_ALIGN_UP(offset + bytes + s->prealloc_size, s->prealloc_align); + + ret = bdrv_co_pwrite_zeroes_locked(bs->file, start, end - start, + BDRV_REQ_NO_FALLBACK, lock); + + bdrv_co_range_unlock(lock); + + return !ret; +} + +static int coroutine_fn preallocate_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, BdrvRequestFlags flags) +{ + if (do_preallocate(bs, offset, bytes, true)) { + return 0; + } + + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); +} + +static coroutine_fn int preallocate_co_pwritev_part(BlockDriverState *bs, + uint64_t offset, + uint64_t bytes, + QEMUIOVector *qiov, + size_t qiov_offset, + int flags) +{ + do_preallocate(bs, offset, bytes, false); + + return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset, + flags); +} + +static int coroutine_fn +preallocate_co_truncate(BlockDriverState *bs, int64_t offset, + bool exact, PreallocMode prealloc, + BdrvRequestFlags flags, Error **errp) +{ + BDRVPreallocateState *s = bs->opaque; + int ret = bdrv_co_truncate(bs->file, offset, exact, prealloc, flags, errp); + + /* s->data_end may become negative here, which means unknown data end */ + s->data_end = bdrv_getlength(bs->file->bs); + + return ret; +} + +static int coroutine_fn preallocate_co_flush(BlockDriverState *bs) +{ + if (!bs->file) { + return 0; + } + + return bdrv_co_flush(bs->file->bs); +} + +static int64_t preallocate_getlength(BlockDriverState *bs) +{ + /* + * We probably can return s->data_end here, but seems safer to return real + * file length, not trying to hide the preallocation. + * + * Still, don't miss the chance to restore s->data_end if it is broken. + */ + BDRVPreallocateState *s = bs->opaque; + int64_t ret = bdrv_getlength(bs->file->bs); + + if (s->data_end < 0) { + s->data_end = ret; + } + + return ret; +} + +BlockDriver bdrv_preallocate_filter = { + .format_name = "preallocate", + .instance_size = sizeof(BDRVPreallocateState), + + .bdrv_getlength = preallocate_getlength, + .bdrv_open = preallocate_open, + .bdrv_close = preallocate_close, + + .bdrv_co_preadv_part = preallocate_co_preadv_part, + .bdrv_co_pwritev_part = preallocate_co_pwritev_part, + .bdrv_co_pwrite_zeroes = preallocate_co_pwrite_zeroes, + .bdrv_co_pdiscard = preallocate_co_pdiscard, + .bdrv_co_flush = preallocate_co_flush, + .bdrv_co_truncate = preallocate_co_truncate, + + .bdrv_co_block_status = bdrv_co_block_status_from_file, + + .bdrv_child_perm = preallocate_child_perm, + + .has_variable_length = true, + .is_filter = true, +}; + +static void bdrv_preallocate_init(void) +{ + bdrv_register(&bdrv_preallocate_filter); +} + +block_init(bdrv_preallocate_init); diff --git a/block/Makefile.objs b/block/Makefile.objs index 3635b6b4c1..f46a353a35 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -45,6 +45,7 @@ block-obj-y += crypto.o block-obj-y += aio_task.o block-obj-y += backup-top.o block-obj-y += filter-compress.o +block-obj-y += preallocate.o common-obj-y += monitor/ block-obj-y += stream.o From patchwork Sat Jun 20 14:36:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1313643 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=ouh7xgO0; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49pywY1Yvfz9sT2 for ; Sun, 21 Jun 2020 00:38:52 +1000 (AEST) Received: from localhost ([::1]:47074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jmedu-0006SH-M5 for incoming@patchwork.ozlabs.org; Sat, 20 Jun 2020 10:38:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56802) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecO-0002k5-AR; Sat, 20 Jun 2020 10:37:16 -0400 Received: from mail-eopbgr50102.outbound.protection.outlook.com ([40.107.5.102]:35810 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecM-0004UK-GC; Sat, 20 Jun 2020 10:37:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kJCwagPcqxchiaRSwyQwZy618IVFBF1B/u/o/3qdPHWqOHmpH1Faza6DExddBscSahXQ24wpi+MV29JJZNIGDBQruB58paTdeSrt7AgnXIAp/j7wfHdqNJ7PfCG96lwJPaNAMyYdF2oTa9YD58/8YxXau+Mm4wnQ+tNXJCvJDQt4UurwiOOQMUiU0nj3yHCGH7mAxnS38FNwRoOwKVZC4WgxdEQYc92XPQpI8kPpcQh2qKwFCGR8T8f1Zxy6flrT7lW2BNnK88U1Ofg+5loRq5huy6VfTqaO63MKaxKqLgld9fB4tuG9u+rtPdaqRNrILCih4ByNk7LvXzMmr4qWsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=16gT/kaWnO+B9k41bACSIvfnDldsTZvYxZ0E8WNy47c=; b=mJiZwsMUxspfD/TlxHbL6Ey31akCTIuiLeT68ta5c5i12wAGvAmCWRLjb/qPE71xzemPoGREd6uae68V50i0MPW5PXW78SHOySJAs34JJ0PmCVQKZ4P6JPasU76KO2BvvShhXiVsPm4W8uKkruAHKrlWVevCddKEJNR3fi0Gyz5w2/4KtzdGoU8Sf9h1La1Dz25mYWdQdDg5A170Cm30BVH/7FymWFdTO6DL7UGyYISFiCd5q+pM8Q2TrIDDieCrU7RT8Gnyc22dNmpkENz4ni8kmbqhgVtIPw67VbxjyCfEkrTn4bL2KzoZ39l2DajY6j0ApASRiTx0WAeynI34IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=16gT/kaWnO+B9k41bACSIvfnDldsTZvYxZ0E8WNy47c=; b=ouh7xgO0/8w3hlYAC6eIwEGH64MQ3VHV/I/wM+SuWEJ+861wYrZHNfRBwn1vdpUK0NhMVwiGoVDz9EYzamfoPXG5z3MImgRsSLhUzVJ2eYXEPEUcNh0/gX79o4iHmVH7/HY9ZJ6gcJTaFbC5ZuwzL5mF5s7d3s7AD6jrKKGANlo= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sat, 20 Jun 2020 14:37:01 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%3]) with mapi id 15.20.3109.021; Sat, 20 Jun 2020 14:37:01 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 4/5] iotests: QemuIoInteractive: use qemu_io_args_no_fmt Date: Sat, 20 Jun 2020 17:36:48 +0300 Message-Id: <20200620143649.225852-5-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200620143649.225852-1-vsementsov@virtuozzo.com> References: <20200620143649.225852-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.122) by AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sat, 20 Jun 2020 14:37:00 +0000 X-Mailer: git-send-email 2.18.0 X-Originating-IP: [185.215.60.122] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c15f3b53-697e-4f48-19b6-08d8152764ac X-MS-TrafficTypeDiagnostic: AM7PR08MB5430: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-Forefront-PRVS: 0440AC9990 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ixFpgO1SVCvzt0qImOhnGXLDhYBme6LDEomD7fXwE0Xe6jzMtcOxUTDtUAnp/aG5ZgWW1/S/UjSzrBW/W+c1UTckh1jtg2PryQ/GD7qPVtbgLjJOt1Y4ZI43S7WTYl22Qz8RgHRZ7YIS+gJ7d/Gmu3Yn2OJcnvp8vXEIz1cIl78flLnm6EUagJFpofGTqAa4jP/XSHRf4G0B0Zj71RxiH3cjrWRiwegEYExPIVzR+/jjViS+dceP18ePSon2JpuNB0ETyqIUgjfNF/vich5I/5haZB8SPuW01gLC3UaSVFpks+HToeHn5k/pDYJSsFz6qTwcu2e6F0DMj2zxZI4Mdw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(6029001)(4636009)(366004)(376002)(346002)(396003)(136003)(39840400004)(6666004)(26005)(186003)(16526019)(86362001)(2906002)(6916009)(6512007)(4326008)(83380400001)(52116002)(2616005)(6506007)(478600001)(956004)(4744005)(36756003)(5660300002)(6486002)(8676002)(316002)(66946007)(8936002)(66476007)(1076003)(66556008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: 4tv24ca1pB4NfOOcQJVd+hgjWPW9kl3NLY7w5qJN/YIATnrYkl2fmy3Y2Ih8KlqHFL8YWcvUPf9B65RUBk90XthHdojMYmcTzOVB5JeLdnc4HotBUy/I4SKtGn4TL3/E9TpiYfU06iLLXL1q6irKL4+dyntPyqVsTEIf3tupdNJ8Ks9jm78hNoQVF31f+kmjFU7+lNW5Md0BM0mZMil3vhc+J4PlEnw9knVY5BtjxgZTsQJIRYa6MEMBdZRHoz87R2WS9YhNLORaLiVi0ITqQGB6NI0F7HNG+Xw1PGdbzUkZzsv9Z/c5/av6QTpzDqtvkhMOAg/u773mc8gkLy3KbNGmqU3B7Tz7WfiPWhtfbtWW55kFvp/TfgA5/aZfrmtX3QOrLb2qPvX8Y/JdcnybSqTdFFEkks0iL8Q+LCZ+3QrcPm2I+837KRM7JzU5mewzDzPIG0J+ITpC8YALyDDNxsN+sSWsz8dmUhmN9vw2lGk= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: c15f3b53-697e-4f48-19b6-08d8152764ac X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2020 14:37:01.1107 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nW/2xHb6kpzAvCqJb8+yGXtyyfJetwpt1pt4OKlNpkgpFuxfAoOghbs2/PQuAAoEjgixmXr87zH/5PnbLlSNccvvbCsI4kpcVGrFr5vsSwM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Received-SPF: pass client-ip=40.107.5.102; envelope-from=vsementsov@virtuozzo.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/20 10:36:59 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, Anton.Nefedov@acronis.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" All users of QemuIoInteractive provides -f argument, so it's incorrect to use qemu_io_args, which contains -f too. Let's use qemu_io_args_no_fmt, which also makes possible to use --image-opts with QemuIoInteractive in the following patch. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index f20d90f969..a8f71df4eb 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -211,7 +211,7 @@ def get_virtio_scsi_device(): class QemuIoInteractive: def __init__(self, *args): - self.args = qemu_io_args + list(args) + self.args = qemu_io_args_no_fmt + list(args) self._p = subprocess.Popen(self.args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, From patchwork Sat Jun 20 14:36:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladimir Sementsov-Ogievskiy X-Patchwork-Id: 1313644 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=virtuozzo.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=virtuozzo.com header.i=@virtuozzo.com header.a=rsa-sha256 header.s=selector2 header.b=aX9uBkdy; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49pywc6Q9Hz9sTG for ; Sun, 21 Jun 2020 00:38:56 +1000 (AEST) Received: from localhost ([::1]:47450 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jmedy-0006hD-M6 for incoming@patchwork.ozlabs.org; Sat, 20 Jun 2020 10:38:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecQ-0002sn-Pz; Sat, 20 Jun 2020 10:37:18 -0400 Received: from mail-eopbgr50102.outbound.protection.outlook.com ([40.107.5.102]:35810 helo=EUR03-VE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jmecO-0004UK-SP; Sat, 20 Jun 2020 10:37:18 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W4F79wJMwT09FptiTx+Fw7fqc1RcIJG1vKNLfHaWvrqmXiM0ulRw6KN42jitVsP2bLMHtqdsv/+bD5PTVHNV1GFngWFce+xrp6RB9DZXzGNlpbqqtgvrTcQADxQfkhDxwve2y5hFA8hEcMfYN5nBQm5TA7GF0lh5quYDpixDct9WU4R0WV2lWIu7OLhkAuM6gOcWcOs8+DrVG9ntEyBsgDW9IVMwbaItxQg5NCanU7es6kCsgKYxWXqP8nsTkFQvZS9NLu1+yRUlql2nRMQO3o987tupDhqhVYoTYUgaXmJNtRdMDkDfhjkvypVmHENrumJklHIlqMxNB+gBVkHKnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cxq4by7r7/43OxvqEV2GmA4STFQGX0fX6D3AxzMKc0Q=; b=nLrXfrgD6yTMocoyT24AY+ZaCBhJbGA6eGHo4yvMYq21HGm1oWv0fDFnu3eCLzXhTa79Sxj/2G2/7WKVFPvgEmm2p0qpO3trMZl/+Juvvw4KQrN2ZELPrgYASpNRiknRsOoaDQfLYFiLXrW5G84fUHU/B+DV0owOC4zibumOrH7/xacskNgTcABeA2RdSsH7TERoI5kmhz/pRG8kBdw9Xzm5TN+EmkQh6ub5k4HzGK5B+oeOmv2vuqPCIcQQhbh94xZqNz3HB5FrW6W985nsrIU43K1U8w14p4Np/jCAS7ZiFRJdWelCJPou6Z/srrH7IcSOTZmiDKelkwQRGavWqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cxq4by7r7/43OxvqEV2GmA4STFQGX0fX6D3AxzMKc0Q=; b=aX9uBkdyLax3qLupvJpogZUE3Y1a0Xl4uvfX2XkvbhA++e+eMPKyopHwte6ARQZXFg1Zzbh+qwnjPuUOIE1Sy7w1qI7sjxXa+LPtF1FvFTnp0UQsSV0hOHWKdMHcYDaFcvZ6ksop1y9Zi/CbLpaRQ+uTZYcPBymS7mjQ88CwYq8= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sat, 20 Jun 2020 14:37:02 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::a408:2f0f:bc6c:d312%3]) with mapi id 15.20.3109.021; Sat, 20 Jun 2020 14:37:02 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Subject: [PATCH 5/5] iotests: add 298 to test new preallocate filter driver Date: Sat, 20 Jun 2020 17:36:49 +0300 Message-Id: <20200620143649.225852-6-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200620143649.225852-1-vsementsov@virtuozzo.com> References: <20200620143649.225852-1-vsementsov@virtuozzo.com> X-ClientProxiedBy: AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from kvm.sw.ru (185.215.60.122) by AM3PR03CA0055.eurprd03.prod.outlook.com (2603:10a6:207:5::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sat, 20 Jun 2020 14:37:01 +0000 X-Mailer: git-send-email 2.18.0 X-Originating-IP: [185.215.60.122] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dfcaa7bf-7512-47c8-3602-08d81527653b X-MS-TrafficTypeDiagnostic: AM7PR08MB5430: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-Forefront-PRVS: 0440AC9990 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iwCK/FRPyHjD1woIPWJj8WjkJY5O4LyE1YFv9O5o78gDfDewjazc8yXkPno+qm0c8s8ZnjqcWRMYZnMsm2kByqz95a43KpV/XIcVyl4Ujw9R/pFoRy688haAmKANuChn0M72XWK5MpK6GbznvaM4RF3x8uWN1CSkPJCygMtWm5mkkNkpcIfsM8uhM+DmxIx4zHuOUCnv2UiS/YbOhLj6o8UE41XrpHkmBr36DHoa7Q982VyddmtNuWFEJsKxEiAI7zKsnqQVP+R64TCbIln0bthbojZvINWplT8rdkfdYefE78Gwwsr7Vg3PpdwcUyzrTzqr+vo+V88BRJSEhNDjvIx7VF5xrOf1Uuz0sTlVNAyS4r3e8Vur2kYCY3WhkVPkRd7iVUkTkyGkDAnGR0Yx8AQqUH3WkyxFWCBwr8gkTnMAqY1aa9owPOSSzC6NNl0J6SmoE4B9NpmmDhZxrGBvMMyw01fvYiFoOFgpEkHJMK4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(346002)(396003)(136003)(39840400004)(6666004)(26005)(186003)(16526019)(86362001)(2906002)(6916009)(6512007)(4326008)(83380400001)(52116002)(2616005)(6506007)(478600001)(956004)(36756003)(5660300002)(6486002)(8676002)(316002)(66946007)(8936002)(66476007)(1076003)(66556008)(21314003)(2004002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: PUnYz3KfnkF3thYESDf1n2n20j5laY+zoIi/IXiqw5e6V5BVGemgYjS/7SJ27cTsgD5BbpuyiPqaXA65AKtf2E7y83vOpC/OusRCzGecHTmG2nrpNxapVbZ+dzgQ/3k+2DDJ2C+WeT0nCbREKNXAmnwT0q+FVQYXcXgH3wvb980BIee6FN3l/SS2YCx7PUbIeeoYfSxx1SNckIsbT89A1cGxvU5TdjKEn6WgUbgu9nRwLkQpTLmMNG+thqk2Oj4iBOvKyFYVW707srSLAeXdoVK58z8Xl1foJpNaQn/v0Czimz/ng+k0kRyGao2mvA7julPJMIoOEnKvAETo1t2ijzIRkUMM30w1QsB/Ne+XXH6yCjxaP6brFI69Qabrd74ndyQUPkReL2I2f/U8WX84WLWVVLjdL+DBjPIjFJlMVBSIgpL94Y6LoqnCnG+p45DabWPOfx+qRtag+hvkict5zLDOvSLk3dZdltk5MPSATwY= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: dfcaa7bf-7512-47c8-3602-08d81527653b X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jun 2020 14:37:02.0596 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: igr2+sA/HLFIR+R/TOsdZBIt9A0j+vF6dNg1FQCu9h2pQkL9q8W9SpBpa0e5gCSdMcl5vo7ueO8ylUN8qkVYqJEP2RRcvl1xPWShTgDtqDM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Received-SPF: pass client-ip=40.107.5.102; envelope-from=vsementsov@virtuozzo.com; helo=EUR03-VE1-obe.outbound.protection.outlook.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/20 10:36:59 X-ACL-Warn: Detected OS = Windows NT kernel [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, kwolf@redhat.com, vsementsov@virtuozzo.com, Anton.Nefedov@acronis.com, armbru@redhat.com, qemu-devel@nongnu.org, mreitz@redhat.com, stefanha@redhat.com, den@openvz.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/298 | 45 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/298.out | 5 +++++ tests/qemu-iotests/group | 1 + 3 files changed, 51 insertions(+) create mode 100644 tests/qemu-iotests/298 create mode 100644 tests/qemu-iotests/298.out diff --git a/tests/qemu-iotests/298 b/tests/qemu-iotests/298 new file mode 100644 index 0000000000..a6af14fc55 --- /dev/null +++ b/tests/qemu-iotests/298 @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +# +# Test for preallocate filter +# +# Copyright (c) 2020 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +import os +import iotests +from iotests import log + +iotests.script_initialize(supported_fmts=['qcow2']) + +size = 10 * 1024 * 1024 +disk = iotests.file_path('disk') + +iotests.qemu_img_create('-f', iotests.imgfmt, disk, str(size)) + +opts = f'driver={iotests.imgfmt},' \ + f'file.driver=preallocate,file.file.filename={disk}' +p = iotests.QemuIoInteractive('--image-opts', '-t', 'none', opts) + +log(p.cmd('write 0 1M'), filters=[iotests.filter_qemu_io]) +p.cmd('flush') + +if os.path.getsize(disk) > 100 * 1024 * 1024: + log('file in progress is big, preallocation works') + +p.close() + +if os.path.getsize(disk) < 10 * 1024 * 1024: + log('file is small after close') diff --git a/tests/qemu-iotests/298.out b/tests/qemu-iotests/298.out new file mode 100644 index 0000000000..6c34d172a4 --- /dev/null +++ b/tests/qemu-iotests/298.out @@ -0,0 +1,5 @@ +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +file in progress is big, preallocation works +file is small after close diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index d886fa0cb3..1bfb864841 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -302,3 +302,4 @@ 291 rw quick 292 rw auto quick 297 meta +298 meta