From patchwork Fri Mar 29 20:03:23 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Durgin X-Patchwork-Id: 232489 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 900F92C00BC for ; Sat, 30 Mar 2013 07:04:14 +1100 (EST) Received: from localhost ([::1]:35419 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULfX2-0001Zq-Ip for incoming@patchwork.ozlabs.org; Fri, 29 Mar 2013 16:04:12 -0400 Received: from eggs.gnu.org ([208.118.235.92]:48469) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULfWb-0001S4-Ls for qemu-devel@nongnu.org; Fri, 29 Mar 2013 16:03:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ULfWY-00054P-4d for qemu-devel@nongnu.org; Fri, 29 Mar 2013 16:03:45 -0400 Received: from mail.hq.newdream.net ([66.33.206.127]:45181) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULfWX-00054J-Sw for qemu-devel@nongnu.org; Fri, 29 Mar 2013 16:03:42 -0400 Received: from mail.hq.newdream.net (localhost [127.0.0.1]) by mail.hq.newdream.net (Postfix) with ESMTP id D1B6125E16; Fri, 29 Mar 2013 13:08:47 -0700 (PDT) Received: from plana32.front.sepia.ceph.com (ip-64-90-32-34.dreamhost.com [64.90.32.34]) by mail.hq.newdream.net (Postfix) with ESMTPSA id C521525DA0; Fri, 29 Mar 2013 13:08:47 -0700 (PDT) From: Josh Durgin To: qemu-devel@nongnu.org Date: Fri, 29 Mar 2013 13:03:23 -0700 Message-Id: <1364587403-30689-1-git-send-email-josh.durgin@inktank.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1364543983-8180-1-git-send-email-josh.durgin@inktank.com> References: <1364543983-8180-1-git-send-email-josh.durgin@inktank.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 66.33.206.127 Cc: Kevin Wolf , Stefan Hajnoczi Subject: [Qemu-devel] [PATCH v2] rbd: add an asynchronous flush X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The existing bdrv_co_flush_to_disk implementation uses rbd_flush(), which is sychronous and causes the main qemu thread to block until it is complete. This results in unresponsiveness and extra latency for the guest. Fix this by using an asynchronous version of flush. This was added to librbd with a special #define to indicate its presence, since it will be backported to stable versions. Thus, there is no need to check the version of librbd. Implement this as bdrv_aio_flush, since it matches other aio functions in the rbd block driver, and leave out bdrv_co_flush_to_disk when the asynchronous version is available. Reported-by: Oliver Francke Signed-off-by: Josh Durgin --- v2: * include hunk treating write, discard, and flush completions the same, since they have no result data block/rbd.c | 37 +++++++++++++++++++++++++++++++++---- 1 file changed, 33 insertions(+), 4 deletions(-) diff --git a/block/rbd.c b/block/rbd.c index 1a8ea6d..141b488 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -63,7 +63,8 @@ typedef enum { RBD_AIO_READ, RBD_AIO_WRITE, - RBD_AIO_DISCARD + RBD_AIO_DISCARD, + RBD_AIO_FLUSH } RBDAIOCmd; typedef struct RBDAIOCB { @@ -379,8 +380,7 @@ static void qemu_rbd_complete_aio(RADOSCB *rcb) r = rcb->ret; - if (acb->cmd == RBD_AIO_WRITE || - acb->cmd == RBD_AIO_DISCARD) { + if (acb->cmd != RBD_AIO_READ) { if (r < 0) { acb->ret = r; acb->error = 1; @@ -659,6 +659,16 @@ static int rbd_aio_discard_wrapper(rbd_image_t image, #endif } +static int rbd_aio_flush_wrapper(rbd_image_t image, + rbd_completion_t comp) +{ +#ifdef LIBRBD_SUPPORTS_AIO_FLUSH + return rbd_aio_flush(image, comp); +#else + return -ENOTSUP; +#endif +} + static BlockDriverAIOCB *rbd_start_aio(BlockDriverState *bs, int64_t sector_num, QEMUIOVector *qiov, @@ -679,7 +689,7 @@ static BlockDriverAIOCB *rbd_start_aio(BlockDriverState *bs, acb = qemu_aio_get(&rbd_aiocb_info, bs, cb, opaque); acb->cmd = cmd; acb->qiov = qiov; - if (cmd == RBD_AIO_DISCARD) { + if (cmd == RBD_AIO_DISCARD || cmd == RBD_AIO_FLUSH) { acb->bounce = NULL; } else { acb->bounce = qemu_blockalign(bs, qiov->size); @@ -723,6 +733,9 @@ static BlockDriverAIOCB *rbd_start_aio(BlockDriverState *bs, case RBD_AIO_DISCARD: r = rbd_aio_discard_wrapper(s->image, off, size, c); break; + case RBD_AIO_FLUSH: + r = rbd_aio_flush_wrapper(s->image, c); + break; default: r = -EINVAL; } @@ -762,6 +775,16 @@ static BlockDriverAIOCB *qemu_rbd_aio_writev(BlockDriverState *bs, RBD_AIO_WRITE); } +#ifdef LIBRBD_SUPPORTS_AIO_FLUSH +static BlockDriverAIOCB *qemu_rbd_aio_flush(BlockDriverState *bs, + BlockDriverCompletionFunc *cb, + void *opaque) +{ + return rbd_start_aio(bs, 0, NULL, 0, cb, opaque, RBD_AIO_FLUSH); +} + +#else + static int qemu_rbd_co_flush(BlockDriverState *bs) { #if LIBRBD_VERSION_CODE >= LIBRBD_VERSION(0, 1, 1) @@ -772,6 +795,7 @@ static int qemu_rbd_co_flush(BlockDriverState *bs) return 0; #endif } +#endif static int qemu_rbd_getinfo(BlockDriverState *bs, BlockDriverInfo *bdi) { @@ -949,7 +973,12 @@ static BlockDriver bdrv_rbd = { .bdrv_aio_readv = qemu_rbd_aio_readv, .bdrv_aio_writev = qemu_rbd_aio_writev, + +#ifdef LIBRBD_SUPPORTS_AIO_FLUSH + .bdrv_aio_flush = qemu_rbd_aio_flush, +#else .bdrv_co_flush_to_disk = qemu_rbd_co_flush, +#endif #ifdef LIBRBD_SUPPORTS_DISCARD .bdrv_aio_discard = qemu_rbd_aio_discard,