From patchwork Tue Mar 1 14:21:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 1599425 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KROnTc8V; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4K7KMb0KHJz9sFy for ; Wed, 2 Mar 2022 01:27:23 +1100 (AEDT) Received: from localhost ([::1]:54094 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nP3TE-0008Bx-QE for incoming@patchwork.ozlabs.org; Tue, 01 Mar 2022 09:27:20 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3Na-0000a8-N4 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:28026) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NZ-00076M-69 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:30 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646144488; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2GFPYF3rd7vEl2S21B2p7zNKVat/lv/1EGml81GNS7s=; b=KROnTc8VDxLWYMfDGlHG7gAkql5qUxe5m2RKj4eo/X7DNfoyVxOka52RxLvwsNwTwK2QJy 0lkkkv2W4H85+by730rYyPL4GcM+JKrTATqIUj9bOzbJeGfgBs7q1uPbYPQUzjWiv+mcyH ZyWKvN8aGgsIGSycJQrmE/Fq7WgFMhI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-110-WplU_KMKMBmIZm42mfKnoA-1; Tue, 01 Mar 2022 09:21:20 -0500 X-MC-Unique: WplU_KMKMBmIZm42mfKnoA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 73A2A520F; Tue, 1 Mar 2022 14:21:19 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id ABA5A8497D; Tue, 1 Mar 2022 14:21:17 +0000 (UTC) From: Emanuele Giuseppe Esposito To: qemu-block@nongnu.org Subject: [RFC PATCH 1/5] aio-wait.h: introduce AIO_WAIT_WHILE_UNLOCKED Date: Tue, 1 Mar 2022 09:21:09 -0500 Message-Id: <20220301142113.163174-2-eesposit@redhat.com> In-Reply-To: <20220301142113.163174-1-eesposit@redhat.com> References: <20220301142113.163174-1-eesposit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eesposit@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=eesposit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , Vladimir Sementsov-Ogievskiy , Emanuele Giuseppe Esposito , qemu-devel@nongnu.org, Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Same as AIO_WAIT_WHILE macro, but if we are in the Main loop do not release and then acquire ctx_ 's aiocontext. Once all Aiocontext locks go away, this macro will replace AIO_WAIT_WHILE. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/aio-wait.h | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index b39eefb38d..ff27fe4eab 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -59,10 +59,11 @@ typedef struct { extern AioWait global_aio_wait; /** - * AIO_WAIT_WHILE: + * _AIO_WAIT_WHILE: * @ctx: the aio context, or NULL if multiple aio contexts (for which the * caller does not hold a lock) are involved in the polling condition. * @cond: wait while this conditional expression is true + * @unlock: whether to unlock and then lock again @ctx * * Wait while a condition is true. Use this to implement synchronous * operations that require event loop activity. @@ -75,7 +76,7 @@ extern AioWait global_aio_wait; * wait on conditions between two IOThreads since that could lead to deadlock, * go via the main loop instead. */ -#define AIO_WAIT_WHILE(ctx, cond) ({ \ +#define _AIO_WAIT_WHILE(ctx, cond, unlock) ({ \ bool waited_ = false; \ AioWait *wait_ = &global_aio_wait; \ AioContext *ctx_ = (ctx); \ @@ -90,11 +91,11 @@ extern AioWait global_aio_wait; assert(qemu_get_current_aio_context() == \ qemu_get_aio_context()); \ while ((cond)) { \ - if (ctx_) { \ + if (unlock && ctx_) { \ aio_context_release(ctx_); \ } \ aio_poll(qemu_get_aio_context(), true); \ - if (ctx_) { \ + if (unlock && ctx_) { \ aio_context_acquire(ctx_); \ } \ waited_ = true; \ @@ -103,6 +104,12 @@ extern AioWait global_aio_wait; qatomic_dec(&wait_->num_waiters); \ waited_; }) +#define AIO_WAIT_WHILE(ctx, cond) \ + _AIO_WAIT_WHILE(ctx, cond, true) + +#define AIO_WAIT_WHILE_UNLOCKED(ctx, cond) \ + _AIO_WAIT_WHILE(ctx, cond, false) + /** * aio_wait_kick: * Wake up the main thread if it is waiting on AIO_WAIT_WHILE(). During From patchwork Tue Mar 1 14:21:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 1599415 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eqkdxKfU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4K7KFm07RPz9sFy for ; Wed, 2 Mar 2022 01:22:20 +1100 (AEDT) Received: from localhost ([::1]:43198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nP3OK-0000cM-PN for incoming@patchwork.ozlabs.org; Tue, 01 Mar 2022 09:22:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40208) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NY-0000SC-22 for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:31141) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NV-000733-6k for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646144484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2ornUYVqO1+q7KCQvgPzm6a778AlxlhjrSuXhmQ9+Vg=; b=eqkdxKfUiN2hkBTi6SXQfekPyivgDJntJW/d4OhXlP7IDH5cGAg/7xJU5z8XcVa2Xrj2Rw 8zQp/COtEjedtrKmTRSUI+kLRmuyKAaPKiNGcUukSgNEp1OrMWvLIauRaruoi1rJPLvTZI sQHzi3eqcM/wSkxqSiwTkNtf94W+s/4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-400-w15McBwsPV-nuSyloAEwVA-1; Tue, 01 Mar 2022 09:21:21 -0500 X-MC-Unique: w15McBwsPV-nuSyloAEwVA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4764E1854E2A; Tue, 1 Mar 2022 14:21:20 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 839BB8497D; Tue, 1 Mar 2022 14:21:19 +0000 (UTC) From: Emanuele Giuseppe Esposito To: qemu-block@nongnu.org Subject: [RFC PATCH 2/5] introduce BDRV_POLL_WHILE_UNLOCKED Date: Tue, 1 Mar 2022 09:21:10 -0500 Message-Id: <20220301142113.163174-3-eesposit@redhat.com> In-Reply-To: <20220301142113.163174-1-eesposit@redhat.com> References: <20220301142113.163174-1-eesposit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eesposit@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=eesposit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , Vladimir Sementsov-Ogievskiy , Emanuele Giuseppe Esposito , qemu-devel@nongnu.org, Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Same as BDRV_POLL_WHILE, but uses AIO_WAIT_WHILE_UNLOCKED. See doc comment for more info. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- include/block/block.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/block/block.h b/include/block/block.h index e1713ee306..5a7a850c16 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -512,6 +512,11 @@ void bdrv_drain_all(void); AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) +#define BDRV_POLL_WHILE_UNLOCKED(bs, cond) ({ \ + BlockDriverState *bs_ = (bs); \ + AIO_WAIT_WHILE_UNLOCKED(bdrv_get_aio_context(bs_), \ + cond); }) + int generated_co_wrapper bdrv_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); int bdrv_co_pdiscard(BdrvChild *child, int64_t offset, int64_t bytes); From patchwork Tue Mar 1 14:21:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 1599417 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=OqEXVBQX; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4K7KHD5XS7z9sFy for ; Wed, 2 Mar 2022 01:23:36 +1100 (AEDT) Received: from localhost ([::1]:45384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nP3Pa-00024S-L9 for incoming@patchwork.ozlabs.org; Tue, 01 Mar 2022 09:23:34 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40294) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NY-0000TM-Kk for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:42446) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NW-00074t-LG for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646144486; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39iZ/EeaX+T3PXuTf7NyVpcfIMoY2ddSVBBEETIorFo=; b=OqEXVBQXRWKBkj58zgIME6fIr140V+jkwiFmutDdOzpUUwFUxxuoD4lro+l/bPJbZYCBNo rKTUm9YetPGvBJaslceBREOHY5edFovaRfOXe43OyvWAf9hdQaFz3HHmPhYHODHZfd1L4Y QY7n5RV27AvE6LzzHHng6iAeQPTDD4g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-135-cy-Y0ALPNgK9bxtmH7jCng-1; Tue, 01 Mar 2022 09:21:22 -0500 X-MC-Unique: cy-Y0ALPNgK9bxtmH7jCng-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 279B7800422; Tue, 1 Mar 2022 14:21:21 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 62B108497D; Tue, 1 Mar 2022 14:21:20 +0000 (UTC) From: Emanuele Giuseppe Esposito To: qemu-block@nongnu.org Subject: [RFC PATCH 3/5] block/io.c: introduce bdrv_subtree_drained_{begin/end}_unlocked Date: Tue, 1 Mar 2022 09:21:11 -0500 Message-Id: <20220301142113.163174-4-eesposit@redhat.com> In-Reply-To: <20220301142113.163174-1-eesposit@redhat.com> References: <20220301142113.163174-1-eesposit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eesposit@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=eesposit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , Vladimir Sementsov-Ogievskiy , Emanuele Giuseppe Esposito , qemu-devel@nongnu.org, Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Same as the locked version, but use BDRV_POLL_UNLOCKED. We are going to add drains to all graph modifications, and they are generally performed without the AioContext lock taken. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Stefan Hajnoczi --- block/io.c | 48 ++++++++++++++++++++++++++++++++----------- include/block/block.h | 2 ++ 2 files changed, 38 insertions(+), 12 deletions(-) diff --git a/block/io.c b/block/io.c index 4e4cb556c5..d474449d2d 100644 --- a/block/io.c +++ b/block/io.c @@ -242,6 +242,7 @@ typedef struct { bool begin; bool recursive; bool poll; + bool unlocked; BdrvChild *parent; bool ignore_bds_parents; int *drained_end_counter; @@ -332,7 +333,7 @@ static bool bdrv_drain_poll_top_level(BlockDriverState *bs, bool recursive, static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, BdrvChild *parent, bool ignore_bds_parents, - bool poll); + bool poll, bool unlocked); static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, BdrvChild *parent, bool ignore_bds_parents, int *drained_end_counter); @@ -350,7 +351,8 @@ static void bdrv_co_drain_bh_cb(void *opaque) if (data->begin) { assert(!data->drained_end_counter); bdrv_do_drained_begin(bs, data->recursive, data->parent, - data->ignore_bds_parents, data->poll); + data->ignore_bds_parents, data->poll, + data->unlocked); } else { assert(!data->poll); bdrv_do_drained_end(bs, data->recursive, data->parent, @@ -372,6 +374,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, BdrvChild *parent, bool ignore_bds_parents, bool poll, + bool unlocked, int *drained_end_counter) { BdrvCoDrainData data; @@ -392,6 +395,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, .parent = parent, .ignore_bds_parents = ignore_bds_parents, .poll = poll, + .unlocked = unlocked, .drained_end_counter = drained_end_counter, }; @@ -439,13 +443,13 @@ void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, BdrvChild *parent, bool ignore_bds_parents, - bool poll) + bool poll, bool unlocked) { BdrvChild *child, *next; if (qemu_in_coroutine()) { bdrv_co_yield_to_drain(bs, true, recursive, parent, ignore_bds_parents, - poll, NULL); + poll, unlocked, NULL); return; } @@ -456,7 +460,7 @@ static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, bs->recursive_quiesce_counter++; QLIST_FOREACH_SAFE(child, &bs->children, next, next) { bdrv_do_drained_begin(child->bs, true, child, ignore_bds_parents, - false); + false, false); } } @@ -471,18 +475,30 @@ static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, */ if (poll) { assert(!ignore_bds_parents); - BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, recursive, parent)); + if (unlocked) { + BDRV_POLL_WHILE_UNLOCKED(bs, + bdrv_drain_poll_top_level(bs, recursive, + parent)); + } else { + BDRV_POLL_WHILE(bs, + bdrv_drain_poll_top_level(bs, recursive, parent)); + } } } void bdrv_drained_begin(BlockDriverState *bs) { - bdrv_do_drained_begin(bs, false, NULL, false, true); + bdrv_do_drained_begin(bs, false, NULL, false, true, false); } void bdrv_subtree_drained_begin(BlockDriverState *bs) { - bdrv_do_drained_begin(bs, true, NULL, false, true); + bdrv_do_drained_begin(bs, true, NULL, false, true, false); +} + +void bdrv_subtree_drained_begin_unlocked(BlockDriverState *bs) +{ + bdrv_do_drained_begin(bs, true, NULL, false, true, true); } /** @@ -510,7 +526,7 @@ static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, if (qemu_in_coroutine()) { bdrv_co_yield_to_drain(bs, false, recursive, parent, ignore_bds_parents, - false, drained_end_counter); + false, false, drained_end_counter); return; } assert(bs->quiesce_counter > 0); @@ -554,12 +570,19 @@ void bdrv_subtree_drained_end(BlockDriverState *bs) BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0); } +void bdrv_subtree_drained_end_unlocked(BlockDriverState *bs) +{ + int drained_end_counter = 0; + bdrv_do_drained_end(bs, true, NULL, false, &drained_end_counter); + BDRV_POLL_WHILE_UNLOCKED(bs, qatomic_read(&drained_end_counter) > 0); +} + void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent) { int i; for (i = 0; i < new_parent->recursive_quiesce_counter; i++) { - bdrv_do_drained_begin(child->bs, true, child, false, true); + bdrv_do_drained_begin(child->bs, true, child, false, true, false); } } @@ -642,7 +665,8 @@ void bdrv_drain_all_begin(void) BlockDriverState *bs = NULL; if (qemu_in_coroutine()) { - bdrv_co_yield_to_drain(NULL, true, false, NULL, true, true, NULL); + bdrv_co_yield_to_drain(NULL, true, false, NULL, true, true, false, + NULL); return; } @@ -667,7 +691,7 @@ void bdrv_drain_all_begin(void) AioContext *aio_context = bdrv_get_aio_context(bs); aio_context_acquire(aio_context); - bdrv_do_drained_begin(bs, false, NULL, true, false); + bdrv_do_drained_begin(bs, false, NULL, true, false, false); aio_context_release(aio_context); } diff --git a/include/block/block.h b/include/block/block.h index 5a7a850c16..072bde370f 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -780,6 +780,7 @@ void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, * exclusive access to all child nodes as well. */ void bdrv_subtree_drained_begin(BlockDriverState *bs); +void bdrv_subtree_drained_begin_unlocked(BlockDriverState *bs); /** * bdrv_drained_end: @@ -812,6 +813,7 @@ void bdrv_drained_end_no_poll(BlockDriverState *bs, int *drained_end_counter); * End a quiescent section started by bdrv_subtree_drained_begin(). */ void bdrv_subtree_drained_end(BlockDriverState *bs); +void bdrv_subtree_drained_end_unlocked(BlockDriverState *bs); void bdrv_add_child(BlockDriverState *parent, BlockDriverState *child, Error **errp); From patchwork Tue Mar 1 14:21:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 1599419 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NGGn/aBk; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4K7KHM3nqbz9sFy for ; Wed, 2 Mar 2022 01:23:43 +1100 (AEDT) Received: from localhost ([::1]:45690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nP3Ph-0002Iw-8K for incoming@patchwork.ozlabs.org; Tue, 01 Mar 2022 09:23:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40272) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NY-0000SM-1F for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:28 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:58036) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NW-00074e-DI for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646144485; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g5f+rQLg9LhPIHUAod7Xj2CQqVzU5134kG9U/8ZVgRw=; b=NGGn/aBkDx92aWnvVjRl81rFvSbTE11bOxyTISXFpMRkpsTSuiqu8yY9eDqvq97IET4R0+ BsmYguA9QLCAbCeG4z48MD28s0+QUldssIDE5As1230K4h3UMmk5MotSrsRaiYJJ1c2PsN fKpKTMHfRWAccwfC9rjTrWZkY8zFf9c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-526-Sg7mSfJyOrW48NWaxYCc-A-1; Tue, 01 Mar 2022 09:21:23 -0500 X-MC-Unique: Sg7mSfJyOrW48NWaxYCc-A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0751F801DDB; Tue, 1 Mar 2022 14:21:22 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 42DED866CD; Tue, 1 Mar 2022 14:21:21 +0000 (UTC) From: Emanuele Giuseppe Esposito To: qemu-block@nongnu.org Subject: [RFC PATCH 4/5] child_job_drained_poll: override polling condition only when in home thread Date: Tue, 1 Mar 2022 09:21:12 -0500 Message-Id: <20220301142113.163174-5-eesposit@redhat.com> In-Reply-To: <20220301142113.163174-1-eesposit@redhat.com> References: <20220301142113.163174-1-eesposit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eesposit@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=eesposit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , Vladimir Sementsov-Ogievskiy , Emanuele Giuseppe Esposito , qemu-devel@nongnu.org, Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" drv->drained_poll() is only implemented in mirror, and allows it to drain from the coroutine. The mirror implementation uses in_drain flag to recognize when it is draining from coroutine, and consequently avoid deadlocking (wait the poll condition in child_job_drained_poll to wait for itself). The problem is that this flag is dangerous, because it breaks bdrv_drained_begin() invariants: once drained_begin ends, all jobs, in_flight requests, and anything running in the iothread are blocked. This can be broken in such way: iothread(mirror): s->in_drain = true; // mirror.c:1112 main loop: bdrv_drained_begin(mirror_bs); /* * drained_begin wait for bdrv_drain_poll_top_level() condition, * that translates in child_job_drained_poll() for jobs, but * mirror implements drv->drained_poll() so it returns * !!in_flight_requests, which his 0 (assertion in mirror.c:1105). */ main loop: thinks iothread is stopped and is modifying the graph... iothread(mirror): *continues*, as nothing is stopping it iothread(mirror): bdrv_drained_begin(bs); /* draining reads the graph while it is modified!! */ main loop: done modifying the graph... In order to fix this, we can simply allow drv->drained_poll() to be called only by the iothread, and not the main loop. We distinguish it by using in_aio_context_home_thread(), that returns false if @ctx is not the same as the thread that runs it. Co-Developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index 10815a89fe..e132d9587e 100644 --- a/blockjob.c +++ b/blockjob.c @@ -107,6 +107,7 @@ static bool child_job_drained_poll(BdrvChild *c) BlockJob *bjob = c->opaque; Job *job = &bjob->job; const BlockJobDriver *drv = block_job_driver(bjob); + AioContext *ctx = block_job_get_aio_context(bjob); /* An inactive or completed job doesn't have any pending requests. Jobs * with !job->busy are either already paused or have a pause point after @@ -117,7 +118,7 @@ static bool child_job_drained_poll(BdrvChild *c) /* Otherwise, assume that it isn't fully stopped yet, but allow the job to * override this assumption. */ - if (drv->drained_poll) { + if (in_aio_context_home_thread(ctx) && drv->drained_poll) { return drv->drained_poll(bjob); } else { return true; From patchwork Tue Mar 1 14:21:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emanuele Giuseppe Esposito X-Patchwork-Id: 1599426 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Mxu12pka; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4K7KMn6jmSz9sFy for ; Wed, 2 Mar 2022 01:27:33 +1100 (AEDT) Received: from localhost ([::1]:54384 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nP3TP-0008Sk-Fh for incoming@patchwork.ozlabs.org; Tue, 01 Mar 2022 09:27:31 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3Na-0000W9-Fp for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53914) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nP3NX-00075O-Hq for qemu-devel@nongnu.org; Tue, 01 Mar 2022 09:21:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646144487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9QgVm+oCHZA9oU7a3UvJSa+DGlQIWb6nrdHnhsz1KTM=; b=Mxu12pkaRNAnZLbxIZ7pqdx/XsSKErKYF5SFuKoia+LLV8gonSv3LxVV1uysCxfALIdb4D g3vJedM07GDmSBeNdk4lsMz9yn+Ly/9nznEMZKgjN4BWRl7ijGAw5VVC0Q2WeICw1fS523 InkPd9S8fKLB4nzE7Ks+O8jSxcQZrRw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-619-RvqFfhtcMeqJUfrk6f5a8Q-1; Tue, 01 Mar 2022 09:21:24 -0500 X-MC-Unique: RvqFfhtcMeqJUfrk6f5a8Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DB43A1854E27; Tue, 1 Mar 2022 14:21:22 +0000 (UTC) Received: from virtlab701.virt.lab.eng.bos.redhat.com (virtlab701.virt.lab.eng.bos.redhat.com [10.19.152.228]) by smtp.corp.redhat.com (Postfix) with ESMTP id 224FD866CC; Tue, 1 Mar 2022 14:21:22 +0000 (UTC) From: Emanuele Giuseppe Esposito To: qemu-block@nongnu.org Subject: [RFC PATCH 5/5] test-bdrv-drain: ensure draining from main loop stops iothreads Date: Tue, 1 Mar 2022 09:21:13 -0500 Message-Id: <20220301142113.163174-6-eesposit@redhat.com> In-Reply-To: <20220301142113.163174-1-eesposit@redhat.com> References: <20220301142113.163174-1-eesposit@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eesposit@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.129.124; envelope-from=eesposit@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Kevin Wolf , Vladimir Sementsov-Ogievskiy , Emanuele Giuseppe Esposito , qemu-devel@nongnu.org, Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , John Snow Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Add 2 tests: test_main_and_then_iothread_drain ensures that if the main thread drains, the iothread cannot drain (and thus read the graph). test_main_and_iothread_drain instead lets main loop and iothread to drain together, and makes sure that no drain happens in parallel. Note that we are using bdrv_subtree_drained_{begin/end}_unlocked to try avoid using the aiocontext lock as much as possible, since it will eventually go away. Signed-off-by: Emanuele Giuseppe Esposito --- tests/unit/test-bdrv-drain.c | 218 +++++++++++++++++++++++++++++++++++ 1 file changed, 218 insertions(+) diff --git a/tests/unit/test-bdrv-drain.c b/tests/unit/test-bdrv-drain.c index 36be84ae55..bf7fdcb568 100644 --- a/tests/unit/test-bdrv-drain.c +++ b/tests/unit/test-bdrv-drain.c @@ -1534,6 +1534,219 @@ static void test_set_aio_context(void) iothread_join(b); } +typedef struct ParallelDrainJob { + BlockJob common; + BlockBackend *blk; + BlockDriverState *bs; + IOThread *a; + bool should_fail; + bool allowed_to_run; + bool conclude_test; + bool job_started; +} ParallelDrainJob; + +typedef struct BDRVParallelTestState { + ParallelDrainJob *job; +} BDRVParallelTestState; + +static void coroutine_fn bdrv_test_par_co_drain(BlockDriverState *bs) +{ + BDRVParallelTestState *s = bs->opaque; + ParallelDrainJob *job = s->job; + assert(!qatomic_read(&job->should_fail)); +} + +static int parallel_run_test(Job *job, Error **errp) +{ + ParallelDrainJob *s = container_of(job, ParallelDrainJob, common.job); + s->job_started = true; + + while (!s->conclude_test) { + if (s->allowed_to_run) { + bdrv_subtree_drained_begin_unlocked(s->bs); + bdrv_subtree_drained_end_unlocked(s->bs); + } + job_pause_point(&s->common.job); + } + s->job_started = false; + + return 0; +} + +static BlockDriver bdrv_test_parallel = { + .format_name = "test", + .instance_size = sizeof(BDRVParallelTestState), + .supports_backing = true, + + .bdrv_co_drain_begin = bdrv_test_par_co_drain, + .bdrv_co_drain_end = bdrv_test_par_co_drain, + + .bdrv_child_perm = bdrv_default_perms, +}; + +static bool parallel_drained_poll(BlockJob *job) +{ + /* need to return false otherwise a drain in coroutine deadlocks */ + return false; +} + +static const BlockJobDriver test_block_job_driver_parallel = { + .job_driver = { + .instance_size = sizeof(ParallelDrainJob), + .run = parallel_run_test, + .user_resume = block_job_user_resume, + .free = block_job_free, + }, + .drained_poll = parallel_drained_poll, +}; + +static ParallelDrainJob *setup_job_test(void) +{ + BlockBackend *blk; + BlockDriverState *bs; + Error *err = NULL; + IOThread *a = iothread_new(); + AioContext *ctx_a = iothread_get_aio_context(a); + ParallelDrainJob *s; + BDRVParallelTestState *p; + int ret; + + blk = blk_new(qemu_get_aio_context(), + BLK_PERM_CONSISTENT_READ, BLK_PERM_ALL); + blk_set_allow_aio_context_change(blk, true); + + bs = bdrv_new_open_driver(&bdrv_test_parallel, "parent", 0, + &error_abort); + p = bs->opaque; + + ret = blk_insert_bs(blk, bs, &error_abort); + assert(ret == 0); + + s = block_job_create("job1", &test_block_job_driver_parallel, + NULL, blk_bs(blk), 0, BLK_PERM_ALL, 0, JOB_DEFAULT, + NULL, NULL, &err); + s->bs = bs; + s->a = a; + s->blk = blk; + p->job = s; + + ret = bdrv_try_set_aio_context(bs, ctx_a, &error_abort); + assert(ret == 0); + assert(blk_get_aio_context(blk) == ctx_a); + assert(s->common.job.aio_context == ctx_a); + + return s; +} + +static void stop_and_tear_down_test(ParallelDrainJob *s) +{ + AioContext *ctx = iothread_get_aio_context(s->a); + + /* stop iothread */ + s->conclude_test = true; + + /* wait that it's stopped */ + while (s->job_started) { + aio_poll(qemu_get_current_aio_context(), false); + } + + aio_context_acquire(ctx); + bdrv_unref(s->bs); + blk_unref(s->blk); + aio_context_release(ctx); + iothread_join(s->a); +} + +/** + * test_main_and_then_iothread_drain: test that if the main + * loop drains, iothread cannot possibly drain. + * + * In this test, make sure that the main loop has firstly + * drained, and then allow the iothread to try and drain. + * + * Therefore, if the main loop drains, there is no way that + * the graph can be read or written by the iothread. + */ +static void test_main_and_then_iothread_drain(void) +{ + ParallelDrainJob *s = setup_job_test(); + + s->allowed_to_run = false; + job_start(&s->common.job); + + /* + * Wait that the iothread starts, even though it just + * loops without doing anything (allowed_to_run is false) + */ + while (!s->job_started) { + aio_poll(qemu_get_current_aio_context(), false); + } + + /* + * Drain in the main loop and ensure that no drain + * is performed by the iothread. + */ + bdrv_subtree_drained_begin_unlocked(s->bs); + /* iothread should be put */ + qatomic_set(&s->should_fail, true); + /* let the iothread do drains */ + s->allowed_to_run = true; + + /* [perform graph modifications here], as iothread is stopped */ + sleep(3); + + /* done with modifications, let the iothread drain once it resumes */ + qatomic_set(&s->should_fail, false); + + /* drained_end should resume the iothread */ + bdrv_subtree_drained_end_unlocked(s->bs); + + stop_and_tear_down_test(s); +} + +/** + * test_main_and_iothread_drain: test that if the main + * loop drains, iothread cannot possibly drain. + * + * In this test, simply let iothread and main loop concurrenly + * loop and drain together, making sure that iothread never + * reads the graph while main loop is draining. + * + * Therefore, if the main loop drains, there is no way that + * the graph can be read or written by the iothread. + */ +static void test_main_and_iothread_drain(void) +{ + ParallelDrainJob *s = setup_job_test(); + int i; + + /* let the iothread do drains from beginning */ + s->allowed_to_run = true; + job_start(&s->common.job); + + /* wait that the iothread starts and begins with drains. */ + while (!s->job_started) { + aio_poll(qemu_get_current_aio_context(), false); + } + + /* at the same time, also main loop performs drains */ + for (i = 0; i < 1000; i++) { + bdrv_subtree_drained_begin_unlocked(s->bs); + /* iothread should be put, regardless of when it drained */ + qatomic_set(&s->should_fail, true); + + /* [perform graph modifications here] */ + + /* done with modifications, let the iothread drain once it resumes */ + qatomic_set(&s->should_fail, false); + + /* drained_end should resume the iothread */ + bdrv_subtree_drained_end_unlocked(s->bs); + } + + stop_and_tear_down_test(s); +} + typedef struct TestDropBackingBlockJob { BlockJob common; @@ -2185,6 +2398,11 @@ int main(int argc, char **argv) g_test_add_func("/bdrv-drain/iothread/drain_subtree", test_iothread_drain_subtree); + g_test_add_func("/bdrv-drain/iothread/drain_main_and_then_iothread", + test_main_and_then_iothread_drain); + g_test_add_func("/bdrv-drain/iothread/drain_parallel", + test_main_and_iothread_drain); + g_test_add_func("/bdrv-drain/blockjob/drain_all", test_blockjob_drain_all); g_test_add_func("/bdrv-drain/blockjob/drain", test_blockjob_drain); g_test_add_func("/bdrv-drain/blockjob/drain_subtree",