From patchwork Mon Dec 1 09:04:15 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 416331 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id D5E3214014D for ; Mon, 1 Dec 2014 20:06:45 +1100 (AEDT) Received: from localhost ([::1]:53333 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XvMwO-0003Bb-0z for incoming@patchwork.ozlabs.org; Mon, 01 Dec 2014 04:06:44 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41215) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XvMuM-0008Qo-8a for qemu-devel@nongnu.org; Mon, 01 Dec 2014 04:04:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XvMuE-0002Cc-U0 for qemu-devel@nongnu.org; Mon, 01 Dec 2014 04:04:38 -0500 Received: from mail-pd0-f179.google.com ([209.85.192.179]:42166) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XvMuE-0002CY-No for qemu-devel@nongnu.org; Mon, 01 Dec 2014 04:04:30 -0500 Received: by mail-pd0-f179.google.com with SMTP id w10so10556371pde.10 for ; Mon, 01 Dec 2014 01:04:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=F4ITIKpZHwVJeuCKUfbZm7El7QvLadHwbOSZglYALkM=; b=IaWMP3aMzdMZjcE8HPhgaib75F3YGPjRymUkj3WUbKOg0jH8dq61BzbMb1fpYxcQBk sJX3tyDiBGngZ9FfsiaFV8FUBL9ffnFClfv37qO4XGuMTqvY/GNuhJ9RpNZbGy5qA1wT OCA4+SLFDKNS3vcczeVKzMnmFeAE/YxZg+rASNZD7VGH63RUNWEJdlGu0YF2BF293U6f TWD16f3wHzCM8pCKqwX4yn6NTJaASOz42c10U898ZD8h/THxfUwpu0f2d9fkA4ywWxGM Po6APNCdCkxkelTL9mB3iwKN/ZPLQ6BKqCl+/LEarW10VISW9NS7v8BJZLnPMSj4Z/AX 16pA== X-Received: by 10.66.246.196 with SMTP id xy4mr100558424pac.29.1417424670115; Mon, 01 Dec 2014 01:04:30 -0800 (PST) Received: from localhost ([183.13.122.212]) by mx.google.com with ESMTPSA id qm3sm16974113pdb.51.2014.12.01.01.04.28 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 01 Dec 2014 01:04:29 -0800 (PST) From: Ming Lei To: qemu-devel@nongnu.org, Paolo Bonzini , Stefan Hajnoczi , Kevin Wolf Date: Mon, 1 Dec 2014 17:04:15 +0800 Message-Id: <1417424656-29714-3-git-send-email-ming.lei@canonical.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1417424656-29714-1-git-send-email-ming.lei@canonical.com> References: <1417424656-29714-1-git-send-email-ming.lei@canonical.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.192.179 Cc: Ming Lei Subject: [Qemu-devel] [PATCH v7 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Previously -EAGAIN is simply ignored for !s->io_q.plugged case, and sometimes it is easy to cause -EIO to VM, such as NVME device. This patch handles -EAGAIN by io queue for !s->io_q.plugged case, and it will be retried in following aio completion cb. Most of times, -EAGAIN only happens if there is pending I/O, but from linux kernel AIO implementation io_submit() might return it when kmem_cache_alloc(GFP_KERNEL) returns NULL too. So 'pending' in 'struct qemu_laio_state' is introduced for tracking active IO, and -EAGAIN is handled when there is pending I/O. Reviewed-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini Suggested-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 53c5616..9403b17 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -56,6 +56,7 @@ typedef struct { } LaioQueue; struct qemu_laio_state { + unsigned long pending; io_context_t ctx; EventNotifier e; @@ -98,6 +99,7 @@ static void qemu_laio_process_completion(struct qemu_laio_state *s, } } } + s->pending--; laiocb->common.cb(laiocb->common.opaque, ret); qemu_aio_unref(laiocb); @@ -179,6 +181,7 @@ static void laio_cancel(BlockAIOCB *blockacb) return; } + laiocb->ctx->pending--; laiocb->common.cb(laiocb->common.opaque, laiocb->ret); } @@ -280,8 +283,13 @@ static int ioq_enqueue(struct qemu_laio_state *s, struct iocb *iocb) s->io_q.iocbs[idx++] = iocb; s->io_q.idx = idx; - /* submit immediately if queue depth is above 2/3 */ - if (idx > s->io_q.size * 2 / 3) { + /* + * This is reached in two cases: queue not plugged but io_submit + * returned -EAGAIN, or queue plugged. In the latter case, start + * submitting some I/O if the queue is getting too full. In the + * former case, instead, wait until an I/O operation is completed. + */ + if (s->io_q.plugged && unlikely(idx > s->io_q.size * 2 / 3)) { ioq_submit(s); } @@ -346,15 +354,23 @@ BlockAIOCB *laio_submit(BlockDriverState *bs, void *aio_ctx, int fd, } io_set_eventfd(&laiocb->iocb, event_notifier_get_fd(&s->e)); - if (!s->io_q.plugged) { - if (io_submit(s->ctx, 1, &iocbs) < 0) { - goto out_free_aiocb; - } - } else { - if (ioq_enqueue(s, iocbs) < 0) { + /* Switch to queue mode until -EAGAIN is handled */ + if (!s->io_q.plugged && !s->io_q.idx) { + int ret = io_submit(s->ctx, 1, &iocbs); + if (ret >= 0) { + return &laiocb->common; + } else if (ret != -EAGAIN || (ret == -EAGAIN && !s->pending)) { goto out_free_aiocb; } + /* + * In case of -EAGAIN, only queue the req if there is pending + * I/O and it is resubmitted in completion of pending I/O + */ + } + if (ioq_enqueue(s, iocbs) < 0) { + goto out_free_aiocb; } + s->pending++; return &laiocb->common; out_free_aiocb: