From patchwork Thu Nov 22 15:16:52 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Hajnoczi X-Patchwork-Id: 201073 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 82CE02C008C for ; Fri, 23 Nov 2012 02:33:19 +1100 (EST) Received: from localhost ([::1]:54016 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TbYmD-00019X-Ix for incoming@patchwork.ozlabs.org; Thu, 22 Nov 2012 10:33:17 -0500 Received: from eggs.gnu.org ([208.118.235.92]:48632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TbYX5-0007Ti-7w for qemu-devel@nongnu.org; Thu, 22 Nov 2012 10:17:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TbYWy-0007Mj-4B for qemu-devel@nongnu.org; Thu, 22 Nov 2012 10:17:39 -0500 Received: from mx1.redhat.com ([209.132.183.28]:28550) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TbYWx-0007Lh-T2 for qemu-devel@nongnu.org; Thu, 22 Nov 2012 10:17:32 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id qAMFHOU3019814 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 22 Nov 2012 10:17:24 -0500 Received: from localhost (ovpn-112-32.ams2.redhat.com [10.36.112.32]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id qAMFHNS3021519; Thu, 22 Nov 2012 10:17:24 -0500 From: Stefan Hajnoczi To: Date: Thu, 22 Nov 2012 16:16:52 +0100 Message-Id: <1353597412-12232-12-git-send-email-stefanha@redhat.com> In-Reply-To: <1353597412-12232-1-git-send-email-stefanha@redhat.com> References: <1353597412-12232-1-git-send-email-stefanha@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: Kevin Wolf , Anthony Liguori , "Michael S. Tsirkin" , Blue Swirl , khoa@us.ibm.com, Stefan Hajnoczi , Paolo Bonzini , Asias He Subject: [Qemu-devel] [PATCH v4 11/11] virtio-blk: add x-data-plane=on|off performance feature X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The virtio-blk-data-plane feature is easy to integrate into hw/virtio-blk.c. The data plane can be started and stopped similar to vhost-net. Users can take advantage of the virtio-blk-data-plane feature using the new -device virtio-blk-pci,x-data-plane=on property. The x-data-plane name was chosen because at this stage the feature is experimental and likely to see changes in the future. If the VM configuration does not support virtio-blk-data-plane an error message is printed. Although we could fall back to regular virtio-blk, I prefer the explicit approach since it prompts the user to fix their configuration if they want the performance benefit of virtio-blk-data-plane. Limitations: * Only format=raw is supported * Live migration is not supported * Block jobs, hot unplug, and other operations fail with -EBUSY * I/O throttling limits are ignored * Only Linux hosts are supported due to Linux AIO usage Signed-off-by: Stefan Hajnoczi --- hw/virtio-blk.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- hw/virtio-blk.h | 1 + hw/virtio-pci.c | 3 +++ 3 files changed, 62 insertions(+), 1 deletion(-) diff --git a/hw/virtio-blk.c b/hw/virtio-blk.c index e25cc96..7f6004e 100644 --- a/hw/virtio-blk.c +++ b/hw/virtio-blk.c @@ -17,6 +17,8 @@ #include "hw/block-common.h" #include "blockdev.h" #include "virtio-blk.h" +#include "hw/dataplane/virtio-blk.h" +#include "migration.h" #include "scsi-defs.h" #ifdef __linux__ # include @@ -33,6 +35,8 @@ typedef struct VirtIOBlock VirtIOBlkConf *blk; unsigned short sector_mask; DeviceState *qdev; + VirtIOBlockDataPlane *dataplane; + Error *migration_blocker; } VirtIOBlock; static VirtIOBlock *to_virtio_blk(VirtIODevice *vdev) @@ -407,6 +411,14 @@ static void virtio_blk_handle_output(VirtIODevice *vdev, VirtQueue *vq) .num_writes = 0, }; + /* Some guests kick before setting VIRTIO_CONFIG_S_DRIVER_OK so start + * dataplane here instead of waiting for .set_status(). + */ + if (s->dataplane) { + virtio_blk_data_plane_start(s->dataplane); + return; + } + while ((req = virtio_blk_get_request(s))) { virtio_blk_handle_request(req, &mrb); } @@ -446,8 +458,13 @@ static void virtio_blk_dma_restart_cb(void *opaque, int running, { VirtIOBlock *s = opaque; - if (!running) + if (!running) { + /* qemu_drain_all() doesn't know about data plane, quiesce here */ + if (s->dataplane) { + virtio_blk_data_plane_drain(s->dataplane); + } return; + } if (!s->bh) { s->bh = qemu_bh_new(virtio_blk_dma_restart_bh, s); @@ -538,6 +555,10 @@ static void virtio_blk_set_status(VirtIODevice *vdev, uint8_t status) VirtIOBlock *s = to_virtio_blk(vdev); uint32_t features; + if (s->dataplane && !(status & VIRTIO_CONFIG_S_DRIVER)) { + virtio_blk_data_plane_stop(s->dataplane); + } + if (!(status & VIRTIO_CONFIG_S_DRIVER_OK)) { return; } @@ -604,6 +625,7 @@ VirtIODevice *virtio_blk_init(DeviceState *dev, VirtIOBlkConf *blk) { VirtIOBlock *s; static int virtio_blk_id; + int fd = -1; if (!blk->conf.bs) { error_report("drive property not set"); @@ -619,6 +641,21 @@ VirtIODevice *virtio_blk_init(DeviceState *dev, VirtIOBlkConf *blk) return NULL; } + if (blk->data_plane) { + if (blk->scsi) { + error_report("device is incompatible with x-data-plane, " + "use scsi=off"); + return NULL; + } + + fd = raw_get_aio_fd(blk->conf.bs); + if (fd < 0) { + error_report("drive is incompatible with x-data-plane, " + "use format=raw,cache=none,aio=native"); + return NULL; + } + } + s = (VirtIOBlock *)virtio_common_init("virtio-blk", VIRTIO_ID_BLOCK, sizeof(struct virtio_blk_config), sizeof(VirtIOBlock)); @@ -636,6 +673,17 @@ VirtIODevice *virtio_blk_init(DeviceState *dev, VirtIOBlkConf *blk) s->vq = virtio_add_queue(&s->vdev, 128, virtio_blk_handle_output); + if (fd >= 0) { + s->dataplane = virtio_blk_data_plane_create(&s->vdev, fd); + + /* Prevent block operations that conflict with data plane thread */ + bdrv_set_in_use(s->bs, 1); + + error_setg(&s->migration_blocker, + "x-data-plane does not support migration"); + migrate_add_blocker(s->migration_blocker); + } + qemu_add_vm_change_state_handler(virtio_blk_dma_restart_cb, s); s->qdev = dev; register_savevm(dev, "virtio-blk", virtio_blk_id++, 2, @@ -652,6 +700,15 @@ VirtIODevice *virtio_blk_init(DeviceState *dev, VirtIOBlkConf *blk) void virtio_blk_exit(VirtIODevice *vdev) { VirtIOBlock *s = to_virtio_blk(vdev); + + if (s->dataplane) { + migrate_del_blocker(s->migration_blocker); + error_free(s->migration_blocker); + bdrv_set_in_use(s->bs, 0); + virtio_blk_data_plane_destroy(s->dataplane); + s->dataplane = NULL; + } + unregister_savevm(s->qdev, "virtio-blk", s); blockdev_mark_auto_del(s->bs); virtio_cleanup(vdev); diff --git a/hw/virtio-blk.h b/hw/virtio-blk.h index f0740d0..53d7971 100644 --- a/hw/virtio-blk.h +++ b/hw/virtio-blk.h @@ -105,6 +105,7 @@ struct VirtIOBlkConf char *serial; uint32_t scsi; uint32_t config_wce; + uint32_t data_plane; }; #define DEFINE_VIRTIO_BLK_FEATURES(_state, _field) \ diff --git a/hw/virtio-pci.c b/hw/virtio-pci.c index 71f4fb5..32cc910 100644 --- a/hw/virtio-pci.c +++ b/hw/virtio-pci.c @@ -897,6 +897,9 @@ static Property virtio_blk_properties[] = { #endif DEFINE_PROP_BIT("config-wce", VirtIOPCIProxy, blk.config_wce, 0, true), DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), +#ifdef CONFIG_VIRTIO_BLK_DATA_PLANE + DEFINE_PROP_BIT("x-data-plane", VirtIOPCIProxy, blk.data_plane, 0, false), +#endif DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, 2), DEFINE_VIRTIO_BLK_FEATURES(VirtIOPCIProxy, host_features), DEFINE_PROP_END_OF_LIST(),