From patchwork Wed May 7 10:27:20 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 346518 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 7C5801401B0 for ; Wed, 7 May 2014 20:32:39 +1000 (EST) Received: from localhost ([::1]:40219 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Whz9R-000294-Bq for incoming@patchwork.ozlabs.org; Wed, 07 May 2014 06:32:37 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49310) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Whz59-0002hG-4c for qemu-devel@nongnu.org; Wed, 07 May 2014 06:28:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Whz4x-0005ep-5E for qemu-devel@nongnu.org; Wed, 07 May 2014 06:28:11 -0400 Received: from mx1.redhat.com ([209.132.183.28]:43022) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Whz4w-0005ei-TT for qemu-devel@nongnu.org; Wed, 07 May 2014 06:27:59 -0400 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s47ARwnv030759 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 7 May 2014 06:27:58 -0400 Received: from localhost (dhcp-64-106.muc.redhat.com [10.32.64.106]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s47ARvWO030104; Wed, 7 May 2014 06:27:57 -0400 From: Stefan Hajnoczi To: Date: Wed, 7 May 2014 12:27:20 +0200 Message-Id: <1399458461-3997-5-git-send-email-stefanha@redhat.com> In-Reply-To: <1399458461-3997-1-git-send-email-stefanha@redhat.com> References: <1399458461-3997-1-git-send-email-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: Kevin Wolf , Paolo Bonzini , Stefan Hajnoczi Subject: [Qemu-devel] [PATCH v2 04/25] block: add bdrv_set_aio_context() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Up until now all BlockDriverState instances have used the QEMU main loop for fd handlers, timers, and BHs. This is not scalable on SMP guests and hosts so we need to move to a model with multiple event loops on different host CPUs. bdrv_set_aio_context() assigns the AioContext event loop to use for a particular BlockDriverState. It first detaches the entire BlockDriverState graph from the current AioContext and then attaches to the new AioContext. This function will be used by virtio-blk data-plane to assign a BlockDriverState to its IOThread AioContext. Make bdrv_aio_set_context() public since data-plane should not include block_int.h. Signed-off-by: Stefan Hajnoczi --- block.c | 57 +++++++++++++++++++++++++++++++++++++++++++++-- include/block/block.h | 11 +++++++++ include/block/block_int.h | 36 ++++++++++++++++++++++++++++++ 3 files changed, 102 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index 8cb84b3..ea3446a 100644 --- a/block.c +++ b/block.c @@ -359,6 +359,7 @@ BlockDriverState *bdrv_new(const char *device_name, Error **errp) qemu_co_queue_init(&bs->throttled_reqs[0]); qemu_co_queue_init(&bs->throttled_reqs[1]); bs->refcnt = 1; + bs->aio_context = qemu_get_aio_context(); return bs; } @@ -5518,8 +5519,60 @@ out: AioContext *bdrv_get_aio_context(BlockDriverState *bs) { - /* Currently BlockDriverState always uses the main loop AioContext */ - return qemu_get_aio_context(); + return bs->aio_context; +} + +void bdrv_detach_aio_context(BlockDriverState *bs) +{ + if (!bs->drv) { + return; + } + + if (bs->drv->bdrv_detach_aio_context) { + bs->drv->bdrv_detach_aio_context(bs); + } + if (bs->file) { + bdrv_detach_aio_context(bs->file); + } + if (bs->backing_hd) { + bdrv_detach_aio_context(bs->backing_hd); + } + + bs->aio_context = NULL; +} + +void bdrv_attach_aio_context(BlockDriverState *bs, + AioContext *new_context) +{ + if (!bs->drv) { + return; + } + + bs->aio_context = new_context; + + if (bs->backing_hd) { + bdrv_attach_aio_context(bs->backing_hd, new_context); + } + if (bs->file) { + bdrv_attach_aio_context(bs->file, new_context); + } + if (bs->drv->bdrv_attach_aio_context) { + bs->drv->bdrv_attach_aio_context(bs, new_context); + } +} + +void bdrv_set_aio_context(BlockDriverState *bs, AioContext *new_context) +{ + bdrv_drain_all(); /* ensure there are no in-flight requests */ + + bdrv_detach_aio_context(bs); + + /* This function executes in the old AioContext so acquire the new one in + * case it runs in a different thread. + */ + aio_context_acquire(new_context); + bdrv_attach_aio_context(bs, new_context); + aio_context_release(new_context); } void bdrv_add_before_write_notifier(BlockDriverState *bs, diff --git a/include/block/block.h b/include/block/block.h index 467fb2b..c126c7c 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -542,4 +542,15 @@ int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag); int bdrv_debug_resume(BlockDriverState *bs, const char *tag); bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag); +/** + * bdrv_set_aio_context: + * + * Changes the #AioContext used for fd handlers, timers, and BHs by this + * BlockDriverState and all its children. + * + * This function must be called from the old #AioContext or with a lock held so + * the old #AioContext is not executing. + */ +void bdrv_set_aio_context(BlockDriverState *bs, AioContext *new_context); + #endif diff --git a/include/block/block_int.h b/include/block/block_int.h index 9ffcb69..47c79b3 100644 --- a/include/block/block_int.h +++ b/include/block/block_int.h @@ -247,6 +247,19 @@ struct BlockDriver { */ int (*bdrv_has_zero_init)(BlockDriverState *bs); + /* Remove fd handlers, timers, and other event loop callbacks so the event + * loop is no longer in use. Called with no in-flight requests and in + * depth-first traversal order with parents before child nodes. + */ + void (*bdrv_detach_aio_context)(BlockDriverState *bs); + + /* Add fd handlers, timers, and other event loop callbacks so I/O requests + * can be processed again. Called with no in-flight requests and in + * depth-first traversal order with child nodes before parent nodes. + */ + void (*bdrv_attach_aio_context)(BlockDriverState *bs, + AioContext *new_context); + QLIST_ENTRY(BlockDriver) list; }; @@ -295,6 +308,8 @@ struct BlockDriverState { const BlockDevOps *dev_ops; void *dev_opaque; + AioContext *aio_context; /* event loop used for fd handlers, timers, etc */ + char filename[1024]; char backing_file[1024]; /* if non zero, the image is a diff of this file image */ @@ -388,6 +403,27 @@ void bdrv_add_before_write_notifier(BlockDriverState *bs, */ AioContext *bdrv_get_aio_context(BlockDriverState *bs); +/** + * bdrv_detach_aio_context: + * + * May be called from .bdrv_detach_aio_context() to detach children from the + * current #AioContext. This is only needed by block drivers that manage their + * own children. Both ->file and ->backing_hd are automatically handled and + * block drivers should not call this function on them explicitly. + */ +void bdrv_detach_aio_context(BlockDriverState *bs); + +/** + * bdrv_attach_aio_context: + * + * May be called from .bdrv_attach_aio_context() to attach children to the new + * #AioContext. This is only needed by block drivers that manage their own + * children. Both ->file and ->backing_hd are automatically handled and block + * drivers should not call this function on them explicitly. + */ +void bdrv_attach_aio_context(BlockDriverState *bs, + AioContext *new_context); + #ifdef _WIN32 int is_windows_drive(const char *filename); #endif