From patchwork Thu Jun 9 17:21:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641418 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JNXOAz+r; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJvWN36vTz9s5V for ; Fri, 10 Jun 2022 05:37:35 +1000 (AEST) Received: from localhost ([::1]:55814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNyF-0007Jn-Db for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:37:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrs-00089P-Kx for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22670) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrn-0003fE-HJ for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6ISxYIAalEGC/YEQrtDquurBJSikWysnYRP8QL415B0=; b=JNXOAz+rpyz3PMkt6CeqJ35d2LW3br4PuBZzosukX4gYddo44wMLO55nvPQbT91JjhpbbS IPe3PNUcR51nAqhbyfAjEczbSUs1xssiKr+wxfdNN5mnHzfb8jBYQNJ39+3xFP2JnPiovb v+lEWOZVxZ1AGl7FzCsyIEdchqmoXwo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-649-zc3IguqANAWxFSeDKpD4pw-1; Thu, 09 Jun 2022 13:22:13 -0400 X-MC-Unique: zc3IguqANAWxFSeDKpD4pw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9D8D2294EDE5; Thu, 9 Jun 2022 17:22:13 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 093324010E32; Thu, 9 Jun 2022 17:22:12 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 01/18] block: drop unused bdrv_co_drain() API Date: Thu, 9 Jun 2022 19:21:32 +0200 Message-Id: <20220609172149.293877-2-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Stefan Hajnoczi bdrv_co_drain() has not been used since commit 9a0cec664eef ("mirror: use bdrv_drained_begin/bdrv_drained_end") in 2016. Remove it so there are fewer drain scenarios to worry about. Use bdrv_drained_begin()/bdrv_drained_end() instead. They are "mixed" functions that can be called from coroutine context. Unlike bdrv_co_drain(), these functions provide control of the length of the drained section, which is usually the right thing. Signed-off-by: Stefan Hajnoczi Message-Id: <20220521122714.3837731-1-stefanha@redhat.com> Reviewed-by: Emanuele Giuseppe Esposito Reviewed-by: Alberto Faria Signed-off-by: Kevin Wolf --- include/block/block-io.h | 1 - block/io.c | 15 --------------- 2 files changed, 16 deletions(-) diff --git a/include/block/block-io.h b/include/block/block-io.h index 62c84f0519..053a27141a 100644 --- a/include/block/block-io.h +++ b/include/block/block-io.h @@ -270,7 +270,6 @@ void bdrv_drained_end_no_poll(BlockDriverState *bs, int *drained_end_counter); cond); }) void bdrv_drain(BlockDriverState *bs); -void coroutine_fn bdrv_co_drain(BlockDriverState *bs); int generated_co_wrapper bdrv_truncate(BdrvChild *child, int64_t offset, bool exact, diff --git a/block/io.c b/block/io.c index 789e6373d5..1e9bf09a49 100644 --- a/block/io.c +++ b/block/io.c @@ -588,21 +588,6 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_parent) BDRV_POLL_WHILE(child->bs, qatomic_read(&drained_end_counter) > 0); } -/* - * Wait for pending requests to complete on a single BlockDriverState subtree, - * and suspend block driver's internal I/O until next request arrives. - * - * Note that unlike bdrv_drain_all(), the caller must hold the BlockDriverState - * AioContext. - */ -void coroutine_fn bdrv_co_drain(BlockDriverState *bs) -{ - IO_OR_GS_CODE(); - assert(qemu_in_coroutine()); - bdrv_drained_begin(bs); - bdrv_drained_end(bs); -} - void bdrv_drain(BlockDriverState *bs) { IO_OR_GS_CODE(); From patchwork Thu Jun 9 17:21:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641420 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=X4K+iFlQ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJvYY6QYsz9s5V for ; Fri, 10 Jun 2022 05:39:29 +1000 (AEST) Received: from localhost ([::1]:60644 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzO07-0002OL-Vy for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:39:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrv-0008GD-Gr for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:39663) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrn-0003f8-NL for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+M2H+HUKutfKSfadgSOaZOxjRFUqS3xlmYZJOdOi6s=; b=X4K+iFlQjgyI1cUNZKgLWOIdu68grbEsjar2Wxi8MwErd07mbJthr8TyQFotQiHCNEYELF JOr4YKoEYmZ00PUoMLVBPtvIQxmVosQeqc01zunXOpSe4xheUmGpBBbjzvA0bJJDsm+NTf dCNt8d3RwuYTPZ1nDlsbCTjxQHduhdo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-9MnkghdlN62Wqg_WQAxSEA-1; Thu, 09 Jun 2022 13:22:14 -0400 X-MC-Unique: 9MnkghdlN62Wqg_WQAxSEA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 77099294EDE9; Thu, 9 Jun 2022 17:22:14 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id D781540D2827; Thu, 9 Jun 2022 17:22:13 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 02/18] block: get rid of blk->guest_block_size Date: Thu, 9 Jun 2022 19:21:33 +0200 Message-Id: <20220609172149.293877-3-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Stefan Hajnoczi Commit 1b7fd729559c ("block: rename buffer_alignment to guest_block_size") noted: At this point, the field is set by the device emulation, but completely ignored by the block layer. The last time the value of buffer_alignment/guest_block_size was actually used was before commit 339064d50639 ("block: Don't use guest sector size for qemu_blockalign()"). This value has not been used since 2013. Get rid of it. Cc: Xie Yongji Signed-off-by: Stefan Hajnoczi Message-Id: <20220518130945.2657905-1-stefanha@redhat.com> Reviewed-by: Paul Durrant Reviewed-by: Eric Blake Reviewed-by: Alberto Faria Signed-off-by: Kevin Wolf --- include/sysemu/block-backend-io.h | 1 - block/block-backend.c | 10 ---------- block/export/vhost-user-blk-server.c | 1 - hw/block/virtio-blk.c | 1 - hw/block/xen-block.c | 1 - hw/ide/core.c | 1 - hw/scsi/scsi-disk.c | 1 - hw/scsi/scsi-generic.c | 1 - 8 files changed, 17 deletions(-) diff --git a/include/sysemu/block-backend-io.h b/include/sysemu/block-backend-io.h index 6517c39295..ccef514023 100644 --- a/include/sysemu/block-backend-io.h +++ b/include/sysemu/block-backend-io.h @@ -72,7 +72,6 @@ void blk_error_action(BlockBackend *blk, BlockErrorAction action, void blk_iostatus_set_err(BlockBackend *blk, int error); int blk_get_max_iov(BlockBackend *blk); int blk_get_max_hw_iov(BlockBackend *blk); -void blk_set_guest_block_size(BlockBackend *blk, int align); void blk_io_plug(BlockBackend *blk); void blk_io_unplug(BlockBackend *blk); diff --git a/block/block-backend.c b/block/block-backend.c index e0e1aff4b1..d4abdf8faa 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -56,9 +56,6 @@ struct BlockBackend { const BlockDevOps *dev_ops; void *dev_opaque; - /* the block size for which the guest device expects atomicity */ - int guest_block_size; - /* If the BDS tree is removed, some of its options are stored here (which * can be used to restore those options in the new BDS on insert) */ BlockBackendRootState root_state; @@ -998,7 +995,6 @@ void blk_detach_dev(BlockBackend *blk, DeviceState *dev) blk->dev = NULL; blk->dev_ops = NULL; blk->dev_opaque = NULL; - blk->guest_block_size = 512; blk_set_perm(blk, 0, BLK_PERM_ALL, &error_abort); blk_unref(blk); } @@ -2100,12 +2096,6 @@ int blk_get_max_iov(BlockBackend *blk) return blk->root->bs->bl.max_iov; } -void blk_set_guest_block_size(BlockBackend *blk, int align) -{ - IO_CODE(); - blk->guest_block_size = align; -} - void *blk_try_blockalign(BlockBackend *blk, size_t size) { IO_CODE(); diff --git a/block/export/vhost-user-blk-server.c b/block/export/vhost-user-blk-server.c index a129204c44..b2e458ade3 100644 --- a/block/export/vhost-user-blk-server.c +++ b/block/export/vhost-user-blk-server.c @@ -495,7 +495,6 @@ static int vu_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, return -EINVAL; } vexp->blk_size = logical_block_size; - blk_set_guest_block_size(exp->blk, logical_block_size); if (vu_opts->has_num_queues) { num_queues = vu_opts->num_queues; diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index cd804795c6..e9ba752f6b 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1228,7 +1228,6 @@ static void virtio_blk_device_realize(DeviceState *dev, Error **errp) s->change = qemu_add_vm_change_state_handler(virtio_blk_dma_restart_cb, s); blk_set_dev_ops(s->blk, &virtio_block_ops, s); - blk_set_guest_block_size(s->blk, s->conf.conf.logical_block_size); blk_iostatus_enable(s->blk); diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c index 674953f1ad..345b284d70 100644 --- a/hw/block/xen-block.c +++ b/hw/block/xen-block.c @@ -243,7 +243,6 @@ static void xen_block_realize(XenDevice *xendev, Error **errp) } blk_set_dev_ops(blk, &xen_block_dev_ops, blockdev); - blk_set_guest_block_size(blk, conf->logical_block_size); if (conf->discard_granularity == -1) { conf->discard_granularity = conf->physical_block_size; diff --git a/hw/ide/core.c b/hw/ide/core.c index c2caa54285..7cbc0a54a7 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -2548,7 +2548,6 @@ int ide_init_drive(IDEState *s, BlockBackend *blk, IDEDriveKind kind, s->smart_selftest_count = 0; if (kind == IDE_CD) { blk_set_dev_ops(blk, &ide_cd_block_ops, s); - blk_set_guest_block_size(blk, 2048); } else { if (!blk_is_inserted(s->blk)) { error_setg(errp, "Device needs media, but drive is empty"); diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index 072686ed58..91acb5c0ce 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/scsi-disk.c @@ -2419,7 +2419,6 @@ static void scsi_realize(SCSIDevice *dev, Error **errp) } else { blk_set_dev_ops(s->qdev.conf.blk, &scsi_disk_block_ops, s); } - blk_set_guest_block_size(s->qdev.conf.blk, s->qdev.blocksize); blk_iostatus_enable(s->qdev.conf.blk); diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c index 0ab00ef85c..ada24d7486 100644 --- a/hw/scsi/scsi-generic.c +++ b/hw/scsi/scsi-generic.c @@ -321,7 +321,6 @@ static void scsi_read_complete(void * opaque, int ret) s->blocksize = ldl_be_p(&r->buf[8]); s->max_lba = ldq_be_p(&r->buf[0]); } - blk_set_guest_block_size(s->conf.blk, s->blocksize); /* * Patch MODE SENSE device specific parameters if the BDS is opened From patchwork Thu Jun 9 17:21:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641401 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=I45eTENZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJspS0FT0z9s2R for ; Fri, 10 Jun 2022 04:20:31 +1000 (AEST) Received: from localhost ([::1]:58182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMlg-0001EF-7w for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:20:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56308) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007sw-PO for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58165) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003fY-Ui for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795338; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XRlkrCDWZieRoXwiYZRDe+1slboZRIh59YJ+KmkrXKg=; b=I45eTENZOLstYgbllBTv/3TqFBSFedcYJbTEiqDkK3t2UFDHy67G6GRrZ7Kbi+hg9krZMU E50ie5/+uPJFx7pfVQQR4BbF4joS7XwdQp+JTsJmZZYyKENMwCbdnc0R+CiWfmHKDLWih4 bXySG2cgP2yJCtLHzQV5NWUwGpvSeXU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-549-ZlNB_J-pMlO54p1_N6ClGQ-1; Thu, 09 Jun 2022 13:22:15 -0400 X-MC-Unique: ZlNB_J-pMlO54p1_N6ClGQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 51CF880159B; Thu, 9 Jun 2022 17:22:15 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id B14FD40D2827; Thu, 9 Jun 2022 17:22:14 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 03/18] block: block_dirty_bitmap_merge(): fix error path Date: Thu, 9 Jun 2022 19:21:34 +0200 Message-Id: <20220609172149.293877-4-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy At the end we ignore failure of bdrv_merge_dirty_bitmap() and report success. And still set errp. That's wrong. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Nikita Lapshin Reviewed-by: Kevin Wolf Message-Id: <20220517111206.23585-2-v.sementsov-og@mail.ru> Signed-off-by: Kevin Wolf --- block/monitor/bitmap-qmp-cmds.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmds.c index 2b677c4a2f..bd10468596 100644 --- a/block/monitor/bitmap-qmp-cmds.c +++ b/block/monitor/bitmap-qmp-cmds.c @@ -309,7 +309,10 @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target, } /* Merge into dst; dst is unchanged on failure. */ - bdrv_merge_dirty_bitmap(dst, anon, backup, errp); + if (!bdrv_merge_dirty_bitmap(dst, anon, backup, errp)) { + dst = NULL; + goto out; + } out: bdrv_release_dirty_bitmap(anon); From patchwork Thu Jun 9 17:21:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641415 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=IB1Al5I4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJvF90X0Qz9s5V for ; Fri, 10 Jun 2022 05:25:17 +1000 (AEST) Received: from localhost ([::1]:40870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNmM-0004eQ-Qm for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:25:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007t2-QL for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:49539) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003fO-Vg for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qKURiMB5mXsc/snLL+3hfk33KA6vKCmW/7tv4A9nLyQ=; b=IB1Al5I4Bisp/JMFjy7siJZ21tKt3SRxORBjcQlXym5HYX/zlbpdshgxgOrb2DkjZZspTg pIBDRY6pPnn1kqLyArtW7QquPshF1BpzFGMZLZkVtiDROMbRPsXv5Na+czs/YAnAIWv7kQ wGNoFYAb/41/phOwsT1bCSWsbl8I0MM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-645-4cbieJziO-6qrUMaibPWaw-1; Thu, 09 Jun 2022 13:22:16 -0400 X-MC-Unique: 4cbieJziO-6qrUMaibPWaw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 29C1A80A0AD; Thu, 9 Jun 2022 17:22:16 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A3A840D2827; Thu, 9 Jun 2022 17:22:15 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 04/18] block: improve block_dirty_bitmap_merge(): don't allocate extra bitmap Date: Thu, 9 Jun 2022 19:21:35 +0200 Message-Id: <20220609172149.293877-5-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We don't need extra bitmap. All we need is to backup the original bitmap when we do first merge. So, drop extra temporary bitmap and work directly with target and backup. Still to keep old semantics, that on failure target is unchanged and user don't need to restore, we need a local_backup variable and do restore ourselves on failure path. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20220517111206.23585-3-v.sementsov-og@mail.ru> Reviewed-by: Eric Blake Signed-off-by: Kevin Wolf --- block/monitor/bitmap-qmp-cmds.c | 41 +++++++++++++++++---------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/block/monitor/bitmap-qmp-cmds.c b/block/monitor/bitmap-qmp-cmds.c index bd10468596..282363606f 100644 --- a/block/monitor/bitmap-qmp-cmds.c +++ b/block/monitor/bitmap-qmp-cmds.c @@ -261,8 +261,9 @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target, HBitmap **backup, Error **errp) { BlockDriverState *bs; - BdrvDirtyBitmap *dst, *src, *anon; + BdrvDirtyBitmap *dst, *src; BlockDirtyBitmapOrStrList *lst; + HBitmap *local_backup = NULL; GLOBAL_STATE_CODE(); @@ -271,12 +272,6 @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target, return NULL; } - anon = bdrv_create_dirty_bitmap(bs, bdrv_dirty_bitmap_granularity(dst), - NULL, errp); - if (!anon) { - return NULL; - } - for (lst = bms; lst; lst = lst->next) { switch (lst->value->type) { const char *name, *node; @@ -285,8 +280,7 @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target, src = bdrv_find_dirty_bitmap(bs, name); if (!src) { error_setg(errp, "Dirty bitmap '%s' not found", name); - dst = NULL; - goto out; + goto fail; } break; case QTYPE_QDICT: @@ -294,29 +288,36 @@ BdrvDirtyBitmap *block_dirty_bitmap_merge(const char *node, const char *target, name = lst->value->u.external.name; src = block_dirty_bitmap_lookup(node, name, NULL, errp); if (!src) { - dst = NULL; - goto out; + goto fail; } break; default: abort(); } - if (!bdrv_merge_dirty_bitmap(anon, src, NULL, errp)) { - dst = NULL; - goto out; + /* We do backup only for first merge operation */ + if (!bdrv_merge_dirty_bitmap(dst, src, + local_backup ? NULL : &local_backup, + errp)) + { + goto fail; } } - /* Merge into dst; dst is unchanged on failure. */ - if (!bdrv_merge_dirty_bitmap(dst, anon, backup, errp)) { - dst = NULL; - goto out; + if (backup) { + *backup = local_backup; + } else { + hbitmap_free(local_backup); } - out: - bdrv_release_dirty_bitmap(anon); return dst; + +fail: + if (local_backup) { + bdrv_restore_dirty_bitmap(dst, local_backup); + } + + return NULL; } void qmp_block_dirty_bitmap_merge(const char *node, const char *target, From patchwork Thu Jun 9 17:21:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641417 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K6AzXhrr; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJvPw0Ld2z9s5V for ; Fri, 10 Jun 2022 05:32:52 +1000 (AEST) Received: from localhost ([::1]:50720 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNth-0003gj-L6 for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:32:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56306) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007st-Qe for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:52778) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003fS-T3 for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795338; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DE5UCP5pNfGXGkh5ZThekVOIeguC4S8UclRTHWHGJYs=; b=K6AzXhrrBXrghi3Tk9UQrM/jiHHx3829VoT/eP+1eTsQd1IVSbP39hnDto9ILsimOju+xx JdbHnxM660K4AxEXkj2ZYbGAMWHc9FOfjKJK78jCmPd1043tAtSy3Jy8yG5U1abje9a1WR w8PbuSErNvyozPBRuQEP3yYRKHLKFg4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-319-2e1m6ZdIMdaULcJcwt10yw-1; Thu, 09 Jun 2022 13:22:17 -0400 X-MC-Unique: 2e1m6ZdIMdaULcJcwt10yw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 062601E7DCCA; Thu, 9 Jun 2022 17:22:17 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 628E4404E4B2; Thu, 9 Jun 2022 17:22:16 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 05/18] block: simplify handling of try to merge different sized bitmaps Date: Thu, 9 Jun 2022 19:21:36 +0200 Message-Id: <20220609172149.293877-6-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Vladimir Sementsov-Ogievskiy We have too much logic to simply check that bitmaps are of the same size. Let's just define that hbitmap_merge() and bdrv_dirty_bitmap_merge_internal() require their argument bitmaps be of same size, this simplifies things. Let's look through the callers: For backup_init_bcs_bitmap() we already assert that merge can't fail. In bdrv_reclaim_dirty_bitmap_locked() we gracefully handle the error that can't happen: successor always has same size as its parent, drop this logic. In bdrv_merge_dirty_bitmap() we already has assertion and separate check. Make the check explicit and improve error message. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Nikita Lapshin Reviewed-by: Kevin Wolf Message-Id: <20220517111206.23585-4-v.sementsov-og@mail.ru> Signed-off-by: Kevin Wolf --- include/block/block_int-io.h | 2 +- include/qemu/hbitmap.h | 15 ++------------- block/backup.c | 6 ++---- block/dirty-bitmap.c | 26 +++++++++++--------------- util/hbitmap.c | 25 +++++++------------------ 5 files changed, 23 insertions(+), 51 deletions(-) diff --git a/include/block/block_int-io.h b/include/block/block_int-io.h index bb454200e5..ded29e7494 100644 --- a/include/block/block_int-io.h +++ b/include/block/block_int-io.h @@ -102,7 +102,7 @@ bool blk_dev_is_tray_open(BlockBackend *blk); void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, int64_t bytes); void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out); -bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, +void bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, HBitmap **backup, bool lock); diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 5bd986aa44..af4e4ab746 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -76,20 +76,9 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size); * * Store result of merging @a and @b into @result. * @result is allowed to be equal to @a or @b. - * - * Return true if the merge was successful, - * false if it was not attempted. - */ -bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result); - -/** - * hbitmap_can_merge: - * - * hbitmap_can_merge(a, b) && hbitmap_can_merge(a, result) is sufficient and - * necessary for hbitmap_merge will not fail. - * + * All bitmaps must have same size. */ -bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b); +void hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result); /** * hbitmap_empty: diff --git a/block/backup.c b/block/backup.c index 5cfd0b999c..b2b649e305 100644 --- a/block/backup.c +++ b/block/backup.c @@ -228,15 +228,13 @@ out: static void backup_init_bcs_bitmap(BackupBlockJob *job) { - bool ret; uint64_t estimate; BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs); if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) { bdrv_clear_dirty_bitmap(bcs_bitmap, NULL); - ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap, - NULL, true); - assert(ret); + bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap, NULL, + true); } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { /* * We can't hog the coroutine to initialize this thoroughly. diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index da1b91166f..bf3dc0512a 100644 --- a/block/dirty-bitmap.c +++ b/block/dirty-bitmap.c @@ -309,10 +309,7 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *parent, return NULL; } - if (!hbitmap_merge(parent->bitmap, successor->bitmap, parent->bitmap)) { - error_setg(errp, "Merging of parent and successor bitmap failed"); - return NULL; - } + hbitmap_merge(parent->bitmap, successor->bitmap, parent->bitmap); parent->disabled = successor->disabled; parent->busy = false; @@ -912,13 +909,15 @@ bool bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, goto out; } - if (!hbitmap_can_merge(dest->bitmap, src->bitmap)) { - error_setg(errp, "Bitmaps are incompatible and can't be merged"); + if (bdrv_dirty_bitmap_size(src) != bdrv_dirty_bitmap_size(dest)) { + error_setg(errp, "Bitmaps are of different sizes (destination size is %" + PRId64 ", source size is %" PRId64 ") and can't be merged", + bdrv_dirty_bitmap_size(dest), bdrv_dirty_bitmap_size(src)); goto out; } - ret = bdrv_dirty_bitmap_merge_internal(dest, src, backup, false); - assert(ret); + bdrv_dirty_bitmap_merge_internal(dest, src, backup, false); + ret = true; out: bdrv_dirty_bitmaps_unlock(dest->bs); @@ -932,17 +931,16 @@ out: /** * bdrv_dirty_bitmap_merge_internal: merge src into dest. * Does NOT check bitmap permissions; not suitable for use as public API. + * @dest, @src and @backup (if not NULL) must have same size. * * @backup: If provided, make a copy of dest here prior to merge. * @lock: If true, lock and unlock bitmaps on the way in/out. - * returns true if the merge succeeded; false if unattempted. */ -bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, +void bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, HBitmap **backup, bool lock) { - bool ret; IO_CODE(); assert(!bdrv_dirty_bitmap_readonly(dest)); @@ -959,9 +957,9 @@ bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, if (backup) { *backup = dest->bitmap; dest->bitmap = hbitmap_alloc(dest->size, hbitmap_granularity(*backup)); - ret = hbitmap_merge(*backup, src->bitmap, dest->bitmap); + hbitmap_merge(*backup, src->bitmap, dest->bitmap); } else { - ret = hbitmap_merge(dest->bitmap, src->bitmap, dest->bitmap); + hbitmap_merge(dest->bitmap, src->bitmap, dest->bitmap); } if (lock) { @@ -970,6 +968,4 @@ bool bdrv_dirty_bitmap_merge_internal(BdrvDirtyBitmap *dest, bdrv_dirty_bitmaps_unlock(src->bs); } } - - return ret; } diff --git a/util/hbitmap.c b/util/hbitmap.c index ea989e1f0e..297db35fb1 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -873,11 +873,6 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) } } -bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b) -{ - return (a->orig_size == b->orig_size); -} - /** * hbitmap_sparse_merge: performs dst = dst | src * works with differing granularities. @@ -901,28 +896,24 @@ static void hbitmap_sparse_merge(HBitmap *dst, const HBitmap *src) * Given HBitmaps A and B, let R := A (BITOR) B. * Bitmaps A and B will not be modified, * except when bitmap R is an alias of A or B. - * - * @return true if the merge was successful, - * false if it was not attempted. + * Bitmaps must have same size. */ -bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) +void hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) { int i; uint64_t j; - if (!hbitmap_can_merge(a, b) || !hbitmap_can_merge(a, result)) { - return false; - } - assert(hbitmap_can_merge(b, result)); + assert(a->orig_size == result->orig_size); + assert(b->orig_size == result->orig_size); if ((!hbitmap_count(a) && result == b) || (!hbitmap_count(b) && result == a)) { - return true; + return; } if (!hbitmap_count(a) && !hbitmap_count(b)) { hbitmap_reset_all(result); - return true; + return; } if (a->granularity != b->granularity) { @@ -935,7 +926,7 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) if (b != result) { hbitmap_sparse_merge(result, b); } - return true; + return; } /* This merge is O(size), as BITS_PER_LONG and HBITMAP_LEVELS are constant. @@ -951,8 +942,6 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) /* Recompute the dirty count */ result->count = hb_count_between(result, 0, result->size - 1); - - return true; } char *hbitmap_sha256(const HBitmap *bitmap, Error **errp) From patchwork Thu Jun 9 17:21:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641405 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Qaew6YPd; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJt5x5Dnvz9sFk for ; Fri, 10 Jun 2022 04:33:57 +1000 (AEST) Received: from localhost ([::1]:49602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMyh-0006df-Ge for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:33:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56334) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007uA-Qj for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:48743) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLre-0003g7-6A for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2CriZ0ti06dDP+qppflQjeyfKuVpeOf5E4VoqyKBJYc=; b=Qaew6YPdOEbX5a70oBUpSsD0L6gzoiYjtyitMt5LjSnl6Dr25LtAP8KKIjjNV922SVGbAs Y8DFODBebzYk+Ue3+8krjtahwvsfmLZmxEsXsYsScVpicmamHPkQ9AY09AUz0qINXmyMz9 5bsdL6wtr2LFA19m9rRP1yRJNNhgGk0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-164-LoLQf9QxM3uJmlvffxlLAA-1; Thu, 09 Jun 2022 13:22:18 -0400 X-MC-Unique: LoLQf9QxM3uJmlvffxlLAA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E287C811E75; Thu, 9 Jun 2022 17:22:17 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3DA30404E4B5; Thu, 9 Jun 2022 17:22:17 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 06/18] block: Support passing NULL ops to blk_set_dev_ops() Date: Thu, 9 Jun 2022 19:21:37 +0200 Message-Id: <20220609172149.293877-7-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji This supports passing NULL ops to blk_set_dev_ops() so that we can remove stale ops in some cases. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-2-xieyongji@bytedance.com> Signed-off-by: Kevin Wolf --- block/block-backend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index d4abdf8faa..f425b00793 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -1058,7 +1058,7 @@ void blk_set_dev_ops(BlockBackend *blk, const BlockDevOps *ops, blk->dev_opaque = opaque; /* Are we currently quiesced? Should we enforce this right now? */ - if (blk->quiesce_counter && ops->drained_begin) { + if (blk->quiesce_counter && ops && ops->drained_begin) { ops->drained_begin(opaque); } } From patchwork Thu Jun 9 17:21:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641416 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=IzbFCXqZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJvLg0lpCz9s5V for ; Fri, 10 Jun 2022 05:30:01 +1000 (AEST) Received: from localhost ([::1]:46168 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNqt-0000P2-9h for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:29:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56290) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007sH-QK for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51374) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003ft-ON for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795340; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O3ikKUwNOaWf07Mgq0iXgXgq0du0UqZuzpiPPuVZ5LA=; b=IzbFCXqZUE4kJ1gG8liVdDYQqdCuENEYMg57MJNMciAXbgmtBxZjHYq/uLP2RmRLcnC1wS +O7mqkHuraC0b4ijbce/z58jSfrOjicpFYIJwmDFN6E/sJ3WnuB7PCUDxqQrMxa2lX0keQ BjOU9i19rNja5x1UCl+CBNzqvX2Wwsw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-108-v7WETsGjOvOfqmfZdTyhxw-1; Thu, 09 Jun 2022 13:22:19 -0400 X-MC-Unique: v7WETsGjOvOfqmfZdTyhxw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BEEA93C02196; Thu, 9 Jun 2022 17:22:18 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 27C2940D2827; Thu, 9 Jun 2022 17:22:18 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 07/18] block/export: Fix incorrect length passed to vu_queue_push() Date: Thu, 9 Jun 2022 19:21:38 +0200 Message-Id: <20220609172149.293877-8-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji Now the req->size is set to the correct value only when handling VIRTIO_BLK_T_GET_ID request. This patch fixes it. Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-3-xieyongji@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- block/export/vhost-user-blk-server.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/block/export/vhost-user-blk-server.c b/block/export/vhost-user-blk-server.c index b2e458ade3..19c6ee51d3 100644 --- a/block/export/vhost-user-blk-server.c +++ b/block/export/vhost-user-blk-server.c @@ -60,8 +60,7 @@ static void vu_blk_req_complete(VuBlkReq *req) { VuDev *vu_dev = &req->server->vu_dev; - /* IO size with 1 extra status byte */ - vu_queue_push(vu_dev, req->vq, &req->elem, req->size + 1); + vu_queue_push(vu_dev, req->vq, &req->elem, req->size); vu_queue_notify(vu_dev, req->vq); free(req); @@ -207,6 +206,7 @@ static void coroutine_fn vu_blk_virtio_process_req(void *opaque) goto err; } + req->size = iov_size(in_iov, in_num); /* We always touch the last byte, so just see how big in_iov is. */ req->in = (void *)in_iov[in_num - 1].iov_base + in_iov[in_num - 1].iov_len @@ -267,7 +267,6 @@ static void coroutine_fn vu_blk_virtio_process_req(void *opaque) VIRTIO_BLK_ID_BYTES); snprintf(elem->in_sg[0].iov_base, size, "%s", "vhost_user_blk"); req->in->status = VIRTIO_BLK_S_OK; - req->size = elem->in_sg[0].iov_len; break; } case VIRTIO_BLK_T_DISCARD: From patchwork Thu Jun 9 17:21:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641393 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DwxDV9Pa; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsX93FN6z9s2R for ; Fri, 10 Jun 2022 04:08:09 +1000 (AEST) Received: from localhost ([::1]:38314 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMZi-000480-LA for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:08:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56320) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007t7-Qx for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:42867) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003g3-OG for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795341; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z2sEueBGe3ySqwZK07WR8VHIPFvz27D7/GKoH4EZVsk=; b=DwxDV9PaU3VDSHEAqSA79Z9MeJwcb1NQgPIbtEHXzyyhkJSDQYwoaWQE2/N/FDLJOtsy8Q +fAaePXY5x6g2UDYRvz2DzNV5/Cvf7ZFWLg5uNB+Ft7AOvTJR4PdtKu4dSttVShTmoMOPA MuhaNsnHH0Rv/PkMOvcIfL/u/otCKXo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-26-px4zIj1DOgaxZiFy_tXInw-1; Thu, 09 Jun 2022 13:22:20 -0400 X-MC-Unique: px4zIj1DOgaxZiFy_tXInw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B9C85294EDDD; Thu, 9 Jun 2022 17:22:19 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 04D9340D2827; Thu, 9 Jun 2022 17:22:18 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 08/18] block/export: Abstract out the logic of virtio-blk I/O process Date: Thu, 9 Jun 2022 19:21:39 +0200 Message-Id: <20220609172149.293877-9-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji Abstract the common logic of virtio-blk I/O process to a function named virtio_blk_process_req(). It's needed for the following commit. Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-4-xieyongji@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- block/export/virtio-blk-handler.h | 37 ++++ block/export/vhost-user-blk-server.c | 259 +++------------------------ block/export/virtio-blk-handler.c | 240 +++++++++++++++++++++++++ MAINTAINERS | 2 + block/export/meson.build | 2 +- 5 files changed, 301 insertions(+), 239 deletions(-) create mode 100644 block/export/virtio-blk-handler.h create mode 100644 block/export/virtio-blk-handler.c diff --git a/block/export/virtio-blk-handler.h b/block/export/virtio-blk-handler.h new file mode 100644 index 0000000000..1c7a5e32ad --- /dev/null +++ b/block/export/virtio-blk-handler.h @@ -0,0 +1,37 @@ +/* + * Handler for virtio-blk I/O + * + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * + * Author: + * Xie Yongji + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#ifndef VIRTIO_BLK_HANDLER_H +#define VIRTIO_BLK_HANDLER_H + +#include "sysemu/block-backend.h" + +#define VIRTIO_BLK_SECTOR_BITS 9 +#define VIRTIO_BLK_SECTOR_SIZE (1ULL << VIRTIO_BLK_SECTOR_BITS) + +#define VIRTIO_BLK_MAX_DISCARD_SECTORS 32768 +#define VIRTIO_BLK_MAX_WRITE_ZEROES_SECTORS 32768 + +typedef struct { + BlockBackend *blk; + const char *serial; + uint32_t logical_block_size; + bool writable; +} VirtioBlkHandler; + +int coroutine_fn virtio_blk_process_req(VirtioBlkHandler *handler, + struct iovec *in_iov, + struct iovec *out_iov, + unsigned int in_num, + unsigned int out_num); + +#endif /* VIRTIO_BLK_HANDLER_H */ diff --git a/block/export/vhost-user-blk-server.c b/block/export/vhost-user-blk-server.c index 19c6ee51d3..c9c290cc4c 100644 --- a/block/export/vhost-user-blk-server.c +++ b/block/export/vhost-user-blk-server.c @@ -17,31 +17,15 @@ #include "vhost-user-blk-server.h" #include "qapi/error.h" #include "qom/object_interfaces.h" -#include "sysemu/block-backend.h" #include "util/block-helpers.h" - -/* - * Sector units are 512 bytes regardless of the - * virtio_blk_config->blk_size value. - */ -#define VIRTIO_BLK_SECTOR_BITS 9 -#define VIRTIO_BLK_SECTOR_SIZE (1ull << VIRTIO_BLK_SECTOR_BITS) +#include "virtio-blk-handler.h" enum { VHOST_USER_BLK_NUM_QUEUES_DEFAULT = 1, - VHOST_USER_BLK_MAX_DISCARD_SECTORS = 32768, - VHOST_USER_BLK_MAX_WRITE_ZEROES_SECTORS = 32768, -}; -struct virtio_blk_inhdr { - unsigned char status; }; typedef struct VuBlkReq { VuVirtqElement elem; - int64_t sector_num; - size_t size; - struct virtio_blk_inhdr *in; - struct virtio_blk_outhdr out; VuServer *server; struct VuVirtq *vq; } VuBlkReq; @@ -50,247 +34,44 @@ typedef struct VuBlkReq { typedef struct { BlockExport export; VuServer vu_server; - uint32_t blk_size; + VirtioBlkHandler handler; QIOChannelSocket *sioc; struct virtio_blk_config blkcfg; - bool writable; } VuBlkExport; -static void vu_blk_req_complete(VuBlkReq *req) +static void vu_blk_req_complete(VuBlkReq *req, size_t in_len) { VuDev *vu_dev = &req->server->vu_dev; - vu_queue_push(vu_dev, req->vq, &req->elem, req->size); + vu_queue_push(vu_dev, req->vq, &req->elem, in_len); vu_queue_notify(vu_dev, req->vq); free(req); } -static bool vu_blk_sect_range_ok(VuBlkExport *vexp, uint64_t sector, - size_t size) -{ - uint64_t nb_sectors; - uint64_t total_sectors; - - if (size % VIRTIO_BLK_SECTOR_SIZE) { - return false; - } - - nb_sectors = size >> VIRTIO_BLK_SECTOR_BITS; - - QEMU_BUILD_BUG_ON(BDRV_SECTOR_SIZE != VIRTIO_BLK_SECTOR_SIZE); - if (nb_sectors > BDRV_REQUEST_MAX_SECTORS) { - return false; - } - if ((sector << VIRTIO_BLK_SECTOR_BITS) % vexp->blk_size) { - return false; - } - blk_get_geometry(vexp->export.blk, &total_sectors); - if (sector > total_sectors || nb_sectors > total_sectors - sector) { - return false; - } - return true; -} - -static int coroutine_fn -vu_blk_discard_write_zeroes(VuBlkExport *vexp, struct iovec *iov, - uint32_t iovcnt, uint32_t type) -{ - BlockBackend *blk = vexp->export.blk; - struct virtio_blk_discard_write_zeroes desc; - ssize_t size; - uint64_t sector; - uint32_t num_sectors; - uint32_t max_sectors; - uint32_t flags; - int bytes; - - /* Only one desc is currently supported */ - if (unlikely(iov_size(iov, iovcnt) > sizeof(desc))) { - return VIRTIO_BLK_S_UNSUPP; - } - - size = iov_to_buf(iov, iovcnt, 0, &desc, sizeof(desc)); - if (unlikely(size != sizeof(desc))) { - error_report("Invalid size %zd, expected %zu", size, sizeof(desc)); - return VIRTIO_BLK_S_IOERR; - } - - sector = le64_to_cpu(desc.sector); - num_sectors = le32_to_cpu(desc.num_sectors); - flags = le32_to_cpu(desc.flags); - max_sectors = (type == VIRTIO_BLK_T_WRITE_ZEROES) ? - VHOST_USER_BLK_MAX_WRITE_ZEROES_SECTORS : - VHOST_USER_BLK_MAX_DISCARD_SECTORS; - - /* This check ensures that 'bytes' fits in an int */ - if (unlikely(num_sectors > max_sectors)) { - return VIRTIO_BLK_S_IOERR; - } - - bytes = num_sectors << VIRTIO_BLK_SECTOR_BITS; - - if (unlikely(!vu_blk_sect_range_ok(vexp, sector, bytes))) { - return VIRTIO_BLK_S_IOERR; - } - - /* - * The device MUST set the status byte to VIRTIO_BLK_S_UNSUPP for discard - * and write zeroes commands if any unknown flag is set. - */ - if (unlikely(flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP)) { - return VIRTIO_BLK_S_UNSUPP; - } - - if (type == VIRTIO_BLK_T_WRITE_ZEROES) { - int blk_flags = 0; - - if (flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) { - blk_flags |= BDRV_REQ_MAY_UNMAP; - } - - if (blk_co_pwrite_zeroes(blk, sector << VIRTIO_BLK_SECTOR_BITS, - bytes, blk_flags) == 0) { - return VIRTIO_BLK_S_OK; - } - } else if (type == VIRTIO_BLK_T_DISCARD) { - /* - * The device MUST set the status byte to VIRTIO_BLK_S_UNSUPP for - * discard commands if the unmap flag is set. - */ - if (unlikely(flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP)) { - return VIRTIO_BLK_S_UNSUPP; - } - - if (blk_co_pdiscard(blk, sector << VIRTIO_BLK_SECTOR_BITS, - bytes) == 0) { - return VIRTIO_BLK_S_OK; - } - } - - return VIRTIO_BLK_S_IOERR; -} - /* Called with server refcount increased, must decrease before returning */ static void coroutine_fn vu_blk_virtio_process_req(void *opaque) { VuBlkReq *req = opaque; VuServer *server = req->server; VuVirtqElement *elem = &req->elem; - uint32_t type; - VuBlkExport *vexp = container_of(server, VuBlkExport, vu_server); - BlockBackend *blk = vexp->export.blk; - + VirtioBlkHandler *handler = &vexp->handler; struct iovec *in_iov = elem->in_sg; struct iovec *out_iov = elem->out_sg; unsigned in_num = elem->in_num; unsigned out_num = elem->out_num; - - /* refer to hw/block/virtio_blk.c */ - if (elem->out_num < 1 || elem->in_num < 1) { - error_report("virtio-blk request missing headers"); - goto err; - } - - if (unlikely(iov_to_buf(out_iov, out_num, 0, &req->out, - sizeof(req->out)) != sizeof(req->out))) { - error_report("virtio-blk request outhdr too short"); - goto err; - } - - iov_discard_front(&out_iov, &out_num, sizeof(req->out)); - - if (in_iov[in_num - 1].iov_len < sizeof(struct virtio_blk_inhdr)) { - error_report("virtio-blk request inhdr too short"); - goto err; - } - - req->size = iov_size(in_iov, in_num); - /* We always touch the last byte, so just see how big in_iov is. */ - req->in = (void *)in_iov[in_num - 1].iov_base - + in_iov[in_num - 1].iov_len - - sizeof(struct virtio_blk_inhdr); - iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr)); - - type = le32_to_cpu(req->out.type); - switch (type & ~VIRTIO_BLK_T_BARRIER) { - case VIRTIO_BLK_T_IN: - case VIRTIO_BLK_T_OUT: { - QEMUIOVector qiov; - int64_t offset; - ssize_t ret = 0; - bool is_write = type & VIRTIO_BLK_T_OUT; - req->sector_num = le64_to_cpu(req->out.sector); - - if (is_write && !vexp->writable) { - req->in->status = VIRTIO_BLK_S_IOERR; - break; - } - - if (is_write) { - qemu_iovec_init_external(&qiov, out_iov, out_num); - } else { - qemu_iovec_init_external(&qiov, in_iov, in_num); - } - - if (unlikely(!vu_blk_sect_range_ok(vexp, - req->sector_num, - qiov.size))) { - req->in->status = VIRTIO_BLK_S_IOERR; - break; - } - - offset = req->sector_num << VIRTIO_BLK_SECTOR_BITS; - - if (is_write) { - ret = blk_co_pwritev(blk, offset, qiov.size, &qiov, 0); - } else { - ret = blk_co_preadv(blk, offset, qiov.size, &qiov, 0); - } - if (ret >= 0) { - req->in->status = VIRTIO_BLK_S_OK; - } else { - req->in->status = VIRTIO_BLK_S_IOERR; - } - break; - } - case VIRTIO_BLK_T_FLUSH: - if (blk_co_flush(blk) == 0) { - req->in->status = VIRTIO_BLK_S_OK; - } else { - req->in->status = VIRTIO_BLK_S_IOERR; - } - break; - case VIRTIO_BLK_T_GET_ID: { - size_t size = MIN(iov_size(&elem->in_sg[0], in_num), - VIRTIO_BLK_ID_BYTES); - snprintf(elem->in_sg[0].iov_base, size, "%s", "vhost_user_blk"); - req->in->status = VIRTIO_BLK_S_OK; - break; + int in_len; + + in_len = virtio_blk_process_req(handler, in_iov, out_iov, + in_num, out_num); + if (in_len < 0) { + free(req); + vhost_user_server_unref(server); + return; } - case VIRTIO_BLK_T_DISCARD: - case VIRTIO_BLK_T_WRITE_ZEROES: { - if (!vexp->writable) { - req->in->status = VIRTIO_BLK_S_IOERR; - break; - } - - req->in->status = vu_blk_discard_write_zeroes(vexp, out_iov, out_num, - type); - break; - } - default: - req->in->status = VIRTIO_BLK_S_UNSUPP; - break; - } - - vu_blk_req_complete(req); - vhost_user_server_unref(server); - return; -err: - free(req); + vu_blk_req_complete(req, in_len); vhost_user_server_unref(server); } @@ -347,7 +128,7 @@ static uint64_t vu_blk_get_features(VuDev *dev) 1ull << VIRTIO_RING_F_EVENT_IDX | 1ull << VHOST_USER_F_PROTOCOL_FEATURES; - if (!vexp->writable) { + if (!vexp->handler.writable) { features |= 1ull << VIRTIO_BLK_F_RO; } @@ -454,12 +235,12 @@ vu_blk_initialize_config(BlockDriverState *bs, config->opt_io_size = cpu_to_le32(1); config->num_queues = cpu_to_le16(num_queues); config->max_discard_sectors = - cpu_to_le32(VHOST_USER_BLK_MAX_DISCARD_SECTORS); + cpu_to_le32(VIRTIO_BLK_MAX_DISCARD_SECTORS); config->max_discard_seg = cpu_to_le32(1); config->discard_sector_alignment = cpu_to_le32(blk_size >> VIRTIO_BLK_SECTOR_BITS); config->max_write_zeroes_sectors - = cpu_to_le32(VHOST_USER_BLK_MAX_WRITE_ZEROES_SECTORS); + = cpu_to_le32(VIRTIO_BLK_MAX_WRITE_ZEROES_SECTORS); config->max_write_zeroes_seg = cpu_to_le32(1); } @@ -479,7 +260,6 @@ static int vu_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, uint64_t logical_block_size; uint16_t num_queues = VHOST_USER_BLK_NUM_QUEUES_DEFAULT; - vexp->writable = opts->writable; vexp->blkcfg.wce = 0; if (vu_opts->has_logical_block_size) { @@ -493,7 +273,6 @@ static int vu_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, error_propagate(errp, local_err); return -EINVAL; } - vexp->blk_size = logical_block_size; if (vu_opts->has_num_queues) { num_queues = vu_opts->num_queues; @@ -502,6 +281,10 @@ static int vu_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, error_setg(errp, "num-queues must be greater than 0"); return -EINVAL; } + vexp->handler.blk = exp->blk; + vexp->handler.serial = "vhost_user_blk"; + vexp->handler.logical_block_size = logical_block_size; + vexp->handler.writable = opts->writable; vu_blk_initialize_config(blk_bs(exp->blk), &vexp->blkcfg, logical_block_size, num_queues); diff --git a/block/export/virtio-blk-handler.c b/block/export/virtio-blk-handler.c new file mode 100644 index 0000000000..313666e8ab --- /dev/null +++ b/block/export/virtio-blk-handler.c @@ -0,0 +1,240 @@ +/* + * Handler for virtio-blk I/O + * + * Copyright (c) 2020 Red Hat, Inc. + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * + * Author: + * Coiby Xu + * Xie Yongji + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include "qemu/error-report.h" +#include "virtio-blk-handler.h" + +#include "standard-headers/linux/virtio_blk.h" + +struct virtio_blk_inhdr { + unsigned char status; +}; + +static bool virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_size, + uint64_t sector, size_t size) +{ + uint64_t nb_sectors; + uint64_t total_sectors; + + if (size % VIRTIO_BLK_SECTOR_SIZE) { + return false; + } + + nb_sectors = size >> VIRTIO_BLK_SECTOR_BITS; + + QEMU_BUILD_BUG_ON(BDRV_SECTOR_SIZE != VIRTIO_BLK_SECTOR_SIZE); + if (nb_sectors > BDRV_REQUEST_MAX_SECTORS) { + return false; + } + if ((sector << VIRTIO_BLK_SECTOR_BITS) % block_size) { + return false; + } + blk_get_geometry(blk, &total_sectors); + if (sector > total_sectors || nb_sectors > total_sectors - sector) { + return false; + } + return true; +} + +static int coroutine_fn +virtio_blk_discard_write_zeroes(VirtioBlkHandler *handler, struct iovec *iov, + uint32_t iovcnt, uint32_t type) +{ + BlockBackend *blk = handler->blk; + struct virtio_blk_discard_write_zeroes desc; + ssize_t size; + uint64_t sector; + uint32_t num_sectors; + uint32_t max_sectors; + uint32_t flags; + int bytes; + + /* Only one desc is currently supported */ + if (unlikely(iov_size(iov, iovcnt) > sizeof(desc))) { + return VIRTIO_BLK_S_UNSUPP; + } + + size = iov_to_buf(iov, iovcnt, 0, &desc, sizeof(desc)); + if (unlikely(size != sizeof(desc))) { + error_report("Invalid size %zd, expected %zu", size, sizeof(desc)); + return VIRTIO_BLK_S_IOERR; + } + + sector = le64_to_cpu(desc.sector); + num_sectors = le32_to_cpu(desc.num_sectors); + flags = le32_to_cpu(desc.flags); + max_sectors = (type == VIRTIO_BLK_T_WRITE_ZEROES) ? + VIRTIO_BLK_MAX_WRITE_ZEROES_SECTORS : + VIRTIO_BLK_MAX_DISCARD_SECTORS; + + /* This check ensures that 'bytes' fits in an int */ + if (unlikely(num_sectors > max_sectors)) { + return VIRTIO_BLK_S_IOERR; + } + + bytes = num_sectors << VIRTIO_BLK_SECTOR_BITS; + + if (unlikely(!virtio_blk_sect_range_ok(blk, handler->logical_block_size, + sector, bytes))) { + return VIRTIO_BLK_S_IOERR; + } + + /* + * The device MUST set the status byte to VIRTIO_BLK_S_UNSUPP for discard + * and write zeroes commands if any unknown flag is set. + */ + if (unlikely(flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP)) { + return VIRTIO_BLK_S_UNSUPP; + } + + if (type == VIRTIO_BLK_T_WRITE_ZEROES) { + int blk_flags = 0; + + if (flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) { + blk_flags |= BDRV_REQ_MAY_UNMAP; + } + + if (blk_co_pwrite_zeroes(blk, sector << VIRTIO_BLK_SECTOR_BITS, + bytes, blk_flags) == 0) { + return VIRTIO_BLK_S_OK; + } + } else if (type == VIRTIO_BLK_T_DISCARD) { + /* + * The device MUST set the status byte to VIRTIO_BLK_S_UNSUPP for + * discard commands if the unmap flag is set. + */ + if (unlikely(flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP)) { + return VIRTIO_BLK_S_UNSUPP; + } + + if (blk_co_pdiscard(blk, sector << VIRTIO_BLK_SECTOR_BITS, + bytes) == 0) { + return VIRTIO_BLK_S_OK; + } + } + + return VIRTIO_BLK_S_IOERR; +} + +int coroutine_fn virtio_blk_process_req(VirtioBlkHandler *handler, + struct iovec *in_iov, + struct iovec *out_iov, + unsigned int in_num, + unsigned int out_num) +{ + BlockBackend *blk = handler->blk; + struct virtio_blk_inhdr *in; + struct virtio_blk_outhdr out; + uint32_t type; + int in_len; + + if (out_num < 1 || in_num < 1) { + error_report("virtio-blk request missing headers"); + return -EINVAL; + } + + if (unlikely(iov_to_buf(out_iov, out_num, 0, &out, + sizeof(out)) != sizeof(out))) { + error_report("virtio-blk request outhdr too short"); + return -EINVAL; + } + + iov_discard_front(&out_iov, &out_num, sizeof(out)); + + if (in_iov[in_num - 1].iov_len < sizeof(struct virtio_blk_inhdr)) { + error_report("virtio-blk request inhdr too short"); + return -EINVAL; + } + + /* We always touch the last byte, so just see how big in_iov is. */ + in_len = iov_size(in_iov, in_num); + in = (void *)in_iov[in_num - 1].iov_base + + in_iov[in_num - 1].iov_len + - sizeof(struct virtio_blk_inhdr); + iov_discard_back(in_iov, &in_num, sizeof(struct virtio_blk_inhdr)); + + type = le32_to_cpu(out.type); + switch (type & ~VIRTIO_BLK_T_BARRIER) { + case VIRTIO_BLK_T_IN: + case VIRTIO_BLK_T_OUT: { + QEMUIOVector qiov; + int64_t offset; + ssize_t ret = 0; + bool is_write = type & VIRTIO_BLK_T_OUT; + int64_t sector_num = le64_to_cpu(out.sector); + + if (is_write && !handler->writable) { + in->status = VIRTIO_BLK_S_IOERR; + break; + } + + if (is_write) { + qemu_iovec_init_external(&qiov, out_iov, out_num); + } else { + qemu_iovec_init_external(&qiov, in_iov, in_num); + } + + if (unlikely(!virtio_blk_sect_range_ok(blk, + handler->logical_block_size, + sector_num, qiov.size))) { + in->status = VIRTIO_BLK_S_IOERR; + break; + } + + offset = sector_num << VIRTIO_BLK_SECTOR_BITS; + + if (is_write) { + ret = blk_co_pwritev(blk, offset, qiov.size, &qiov, 0); + } else { + ret = blk_co_preadv(blk, offset, qiov.size, &qiov, 0); + } + if (ret >= 0) { + in->status = VIRTIO_BLK_S_OK; + } else { + in->status = VIRTIO_BLK_S_IOERR; + } + break; + } + case VIRTIO_BLK_T_FLUSH: + if (blk_co_flush(blk) == 0) { + in->status = VIRTIO_BLK_S_OK; + } else { + in->status = VIRTIO_BLK_S_IOERR; + } + break; + case VIRTIO_BLK_T_GET_ID: { + size_t size = MIN(strlen(handler->serial) + 1, + MIN(iov_size(in_iov, in_num), + VIRTIO_BLK_ID_BYTES)); + iov_from_buf(in_iov, in_num, 0, handler->serial, size); + in->status = VIRTIO_BLK_S_OK; + break; + } + case VIRTIO_BLK_T_DISCARD: + case VIRTIO_BLK_T_WRITE_ZEROES: + if (!handler->writable) { + in->status = VIRTIO_BLK_S_IOERR; + break; + } + in->status = virtio_blk_discard_write_zeroes(handler, out_iov, + out_num, type); + break; + default: + in->status = VIRTIO_BLK_S_UNSUPP; + break; + } + + return in_len; +} diff --git a/MAINTAINERS b/MAINTAINERS index 5580a36b68..379879fb42 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3568,6 +3568,8 @@ M: Coiby Xu S: Maintained F: block/export/vhost-user-blk-server.c F: block/export/vhost-user-blk-server.h +F: block/export/virtio-blk-handler.c +F: block/export/virtio-blk-handler.h F: include/qemu/vhost-user-server.h F: tests/qtest/libqos/vhost-user-blk.c F: tests/qtest/libqos/vhost-user-blk.h diff --git a/block/export/meson.build b/block/export/meson.build index 0a08e384c7..431e47ca51 100644 --- a/block/export/meson.build +++ b/block/export/meson.build @@ -1,7 +1,7 @@ blockdev_ss.add(files('export.c')) if have_vhost_user_blk_server - blockdev_ss.add(files('vhost-user-blk-server.c')) + blockdev_ss.add(files('vhost-user-blk-server.c', 'virtio-blk-handler.c')) endif blockdev_ss.add(when: fuse, if_true: files('fuse.c')) From patchwork Thu Jun 9 17:21:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641388 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=bmFmWld0; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJs6p0hWvz9s1l for ; Fri, 10 Jun 2022 03:49:38 +1000 (AEST) Received: from localhost ([::1]:39882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMHm-00017W-0a for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 13:49:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56322) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007tB-Qp for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:50103) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003gF-Tq for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795342; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Mdn27kF0AqyI8lZgXYxEJKqvbz4a75NxLPB4q58SUeo=; b=bmFmWld0T1ajTeDZX86TFtLElWoyaNqdpts6W7rHe2Cp/gXqv+YABhK/FYOCW1ylXOOpWV fz1LjMCcWZHI1B4r4K3ZhZQ+D40lARuXcoe94ZyZVxExjYj6uE5KaJ2uM1zDFUHyttktvm jmhnMt/APew/zOL3boIMQX+AhEl8UzA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-107-adLBLg8SOm60g2VKLXY-7Q-1; Thu, 09 Jun 2022 13:22:20 -0400 X-MC-Unique: adLBLg8SOm60g2VKLXY-7Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 99B913C021A0; Thu, 9 Jun 2022 17:22:20 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3E83404E4B5; Thu, 9 Jun 2022 17:22:19 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 09/18] linux-headers: Add vduse.h Date: Thu, 9 Jun 2022 19:21:40 +0200 Message-Id: <20220609172149.293877-10-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji This adds vduse header to linux headers so that the relevant VDUSE API can be used in subsequent patches. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-5-xieyongji@bytedance.com> Signed-off-by: Kevin Wolf --- linux-headers/linux/vduse.h | 306 ++++++++++++++++++++++++++++++++ scripts/update-linux-headers.sh | 2 +- 2 files changed, 307 insertions(+), 1 deletion(-) create mode 100644 linux-headers/linux/vduse.h diff --git a/linux-headers/linux/vduse.h b/linux-headers/linux/vduse.h new file mode 100644 index 0000000000..d47b004ce6 --- /dev/null +++ b/linux-headers/linux/vduse.h @@ -0,0 +1,306 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +#ifndef _VDUSE_H_ +#define _VDUSE_H_ + +#include + +#define VDUSE_BASE 0x81 + +/* The ioctls for control device (/dev/vduse/control) */ + +#define VDUSE_API_VERSION 0 + +/* + * Get the version of VDUSE API that kernel supported (VDUSE_API_VERSION). + * This is used for future extension. + */ +#define VDUSE_GET_API_VERSION _IOR(VDUSE_BASE, 0x00, __u64) + +/* Set the version of VDUSE API that userspace supported. */ +#define VDUSE_SET_API_VERSION _IOW(VDUSE_BASE, 0x01, __u64) + +/** + * struct vduse_dev_config - basic configuration of a VDUSE device + * @name: VDUSE device name, needs to be NUL terminated + * @vendor_id: virtio vendor id + * @device_id: virtio device id + * @features: virtio features + * @vq_num: the number of virtqueues + * @vq_align: the allocation alignment of virtqueue's metadata + * @reserved: for future use, needs to be initialized to zero + * @config_size: the size of the configuration space + * @config: the buffer of the configuration space + * + * Structure used by VDUSE_CREATE_DEV ioctl to create VDUSE device. + */ +struct vduse_dev_config { +#define VDUSE_NAME_MAX 256 + char name[VDUSE_NAME_MAX]; + __u32 vendor_id; + __u32 device_id; + __u64 features; + __u32 vq_num; + __u32 vq_align; + __u32 reserved[13]; + __u32 config_size; + __u8 config[]; +}; + +/* Create a VDUSE device which is represented by a char device (/dev/vduse/$NAME) */ +#define VDUSE_CREATE_DEV _IOW(VDUSE_BASE, 0x02, struct vduse_dev_config) + +/* + * Destroy a VDUSE device. Make sure there are no more references + * to the char device (/dev/vduse/$NAME). + */ +#define VDUSE_DESTROY_DEV _IOW(VDUSE_BASE, 0x03, char[VDUSE_NAME_MAX]) + +/* The ioctls for VDUSE device (/dev/vduse/$NAME) */ + +/** + * struct vduse_iotlb_entry - entry of IOTLB to describe one IOVA region [start, last] + * @offset: the mmap offset on returned file descriptor + * @start: start of the IOVA region + * @last: last of the IOVA region + * @perm: access permission of the IOVA region + * + * Structure used by VDUSE_IOTLB_GET_FD ioctl to find an overlapped IOVA region. + */ +struct vduse_iotlb_entry { + __u64 offset; + __u64 start; + __u64 last; +#define VDUSE_ACCESS_RO 0x1 +#define VDUSE_ACCESS_WO 0x2 +#define VDUSE_ACCESS_RW 0x3 + __u8 perm; +}; + +/* + * Find the first IOVA region that overlaps with the range [start, last] + * and return the corresponding file descriptor. Return -EINVAL means the + * IOVA region doesn't exist. Caller should set start and last fields. + */ +#define VDUSE_IOTLB_GET_FD _IOWR(VDUSE_BASE, 0x10, struct vduse_iotlb_entry) + +/* + * Get the negotiated virtio features. It's a subset of the features in + * struct vduse_dev_config which can be accepted by virtio driver. It's + * only valid after FEATURES_OK status bit is set. + */ +#define VDUSE_DEV_GET_FEATURES _IOR(VDUSE_BASE, 0x11, __u64) + +/** + * struct vduse_config_data - data used to update configuration space + * @offset: the offset from the beginning of configuration space + * @length: the length to write to configuration space + * @buffer: the buffer used to write from + * + * Structure used by VDUSE_DEV_SET_CONFIG ioctl to update device + * configuration space. + */ +struct vduse_config_data { + __u32 offset; + __u32 length; + __u8 buffer[]; +}; + +/* Set device configuration space */ +#define VDUSE_DEV_SET_CONFIG _IOW(VDUSE_BASE, 0x12, struct vduse_config_data) + +/* + * Inject a config interrupt. It's usually used to notify virtio driver + * that device configuration space has changed. + */ +#define VDUSE_DEV_INJECT_CONFIG_IRQ _IO(VDUSE_BASE, 0x13) + +/** + * struct vduse_vq_config - basic configuration of a virtqueue + * @index: virtqueue index + * @max_size: the max size of virtqueue + * @reserved: for future use, needs to be initialized to zero + * + * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue. + */ +struct vduse_vq_config { + __u32 index; + __u16 max_size; + __u16 reserved[13]; +}; + +/* + * Setup the specified virtqueue. Make sure all virtqueues have been + * configured before the device is attached to vDPA bus. + */ +#define VDUSE_VQ_SETUP _IOW(VDUSE_BASE, 0x14, struct vduse_vq_config) + +/** + * struct vduse_vq_state_split - split virtqueue state + * @avail_index: available index + */ +struct vduse_vq_state_split { + __u16 avail_index; +}; + +/** + * struct vduse_vq_state_packed - packed virtqueue state + * @last_avail_counter: last driver ring wrap counter observed by device + * @last_avail_idx: device available index + * @last_used_counter: device ring wrap counter + * @last_used_idx: used index + */ +struct vduse_vq_state_packed { + __u16 last_avail_counter; + __u16 last_avail_idx; + __u16 last_used_counter; + __u16 last_used_idx; +}; + +/** + * struct vduse_vq_info - information of a virtqueue + * @index: virtqueue index + * @num: the size of virtqueue + * @desc_addr: address of desc area + * @driver_addr: address of driver area + * @device_addr: address of device area + * @split: split virtqueue state + * @packed: packed virtqueue state + * @ready: ready status of virtqueue + * + * Structure used by VDUSE_VQ_GET_INFO ioctl to get virtqueue's information. + */ +struct vduse_vq_info { + __u32 index; + __u32 num; + __u64 desc_addr; + __u64 driver_addr; + __u64 device_addr; + union { + struct vduse_vq_state_split split; + struct vduse_vq_state_packed packed; + }; + __u8 ready; +}; + +/* Get the specified virtqueue's information. Caller should set index field. */ +#define VDUSE_VQ_GET_INFO _IOWR(VDUSE_BASE, 0x15, struct vduse_vq_info) + +/** + * struct vduse_vq_eventfd - eventfd configuration for a virtqueue + * @index: virtqueue index + * @fd: eventfd, -1 means de-assigning the eventfd + * + * Structure used by VDUSE_VQ_SETUP_KICKFD ioctl to setup kick eventfd. + */ +struct vduse_vq_eventfd { + __u32 index; +#define VDUSE_EVENTFD_DEASSIGN -1 + int fd; +}; + +/* + * Setup kick eventfd for specified virtqueue. The kick eventfd is used + * by VDUSE kernel module to notify userspace to consume the avail vring. + */ +#define VDUSE_VQ_SETUP_KICKFD _IOW(VDUSE_BASE, 0x16, struct vduse_vq_eventfd) + +/* + * Inject an interrupt for specific virtqueue. It's used to notify virtio driver + * to consume the used vring. + */ +#define VDUSE_VQ_INJECT_IRQ _IOW(VDUSE_BASE, 0x17, __u32) + +/* The control messages definition for read(2)/write(2) on /dev/vduse/$NAME */ + +/** + * enum vduse_req_type - request type + * @VDUSE_GET_VQ_STATE: get the state for specified virtqueue from userspace + * @VDUSE_SET_STATUS: set the device status + * @VDUSE_UPDATE_IOTLB: Notify userspace to update the memory mapping for + * specified IOVA range via VDUSE_IOTLB_GET_FD ioctl + */ +enum vduse_req_type { + VDUSE_GET_VQ_STATE, + VDUSE_SET_STATUS, + VDUSE_UPDATE_IOTLB, +}; + +/** + * struct vduse_vq_state - virtqueue state + * @index: virtqueue index + * @split: split virtqueue state + * @packed: packed virtqueue state + */ +struct vduse_vq_state { + __u32 index; + union { + struct vduse_vq_state_split split; + struct vduse_vq_state_packed packed; + }; +}; + +/** + * struct vduse_dev_status - device status + * @status: device status + */ +struct vduse_dev_status { + __u8 status; +}; + +/** + * struct vduse_iova_range - IOVA range [start, last] + * @start: start of the IOVA range + * @last: last of the IOVA range + */ +struct vduse_iova_range { + __u64 start; + __u64 last; +}; + +/** + * struct vduse_dev_request - control request + * @type: request type + * @request_id: request id + * @reserved: for future use + * @vq_state: virtqueue state, only index field is available + * @s: device status + * @iova: IOVA range for updating + * @padding: padding + * + * Structure used by read(2) on /dev/vduse/$NAME. + */ +struct vduse_dev_request { + __u32 type; + __u32 request_id; + __u32 reserved[4]; + union { + struct vduse_vq_state vq_state; + struct vduse_dev_status s; + struct vduse_iova_range iova; + __u32 padding[32]; + }; +}; + +/** + * struct vduse_dev_response - response to control request + * @request_id: corresponding request id + * @result: the result of request + * @reserved: for future use, needs to be initialized to zero + * @vq_state: virtqueue state + * @padding: padding + * + * Structure used by write(2) on /dev/vduse/$NAME. + */ +struct vduse_dev_response { + __u32 request_id; +#define VDUSE_REQ_RESULT_OK 0x00 +#define VDUSE_REQ_RESULT_FAILED 0x01 + __u32 result; + __u32 reserved[4]; + union { + struct vduse_vq_state vq_state; + __u32 padding[32]; + }; +}; + +#endif /* _VDUSE_H_ */ diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index 839a5ec614..b1ad99cba8 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -161,7 +161,7 @@ done rm -rf "$output/linux-headers/linux" mkdir -p "$output/linux-headers/linux" for header in kvm.h vfio.h vfio_ccw.h vfio_zdev.h vhost.h \ - psci.h psp-sev.h userfaultfd.h mman.h; do + psci.h psp-sev.h userfaultfd.h mman.h vduse.h; do cp "$tmpdir/include/linux/$header" "$output/linux-headers/linux" done From patchwork Thu Jun 9 17:21:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641411 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AFwE96+T; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJtvY0ZB3z9s5V for ; Fri, 10 Jun 2022 05:09:59 +1000 (AEST) Received: from localhost ([::1]:46742 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNXX-0005Jc-1n for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:09:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56434) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrs-00089Y-MY for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:33154) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrn-0003iI-6v for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KulIdqnCXOwmrFKFOjNkQBZYtj8LF+MH5GfD619kfaA=; b=AFwE96+TXYJ1FII8QTIppiNJ3WdX8Y2WgL5S5Sn1yVGkT0pWl26gJ6YmNpdpAtY71bRunF zB9NeX2S2v4gH6EpTicMkb4FuYGpHQ0tkmDBvv2d6NLbblcYEC0P6XJH8Pi0XL184QDlMu bu2jkTRGsn4lmCOgXxm1YyyMlD78amw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-147-ZTGBUXcRPAOeRP8bA_tcGw-1; Thu, 09 Jun 2022 13:22:32 -0400 X-MC-Unique: ZTGBUXcRPAOeRP8bA_tcGw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 92792811E75; Thu, 9 Jun 2022 17:22:21 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id D423C4010E32; Thu, 9 Jun 2022 17:22:20 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 10/18] libvduse: Add VDUSE (vDPA Device in Userspace) library Date: Thu, 9 Jun 2022 19:21:41 +0200 Message-Id: <20220609172149.293877-11-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji VDUSE [1] is a linux framework that makes it possible to implement software-emulated vDPA devices in userspace. This adds a library as a subproject to help implementing VDUSE backends in QEMU. [1] https://www.kernel.org/doc/html/latest/userspace-api/vduse.html Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-6-xieyongji@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- meson_options.txt | 2 + subprojects/libvduse/include/atomic.h | 1 + subprojects/libvduse/include/compiler.h | 1 + subprojects/libvduse/libvduse.h | 235 ++++ subprojects/libvduse/libvduse.c | 1167 +++++++++++++++++++ MAINTAINERS | 5 + meson.build | 15 + scripts/meson-buildoptions.sh | 3 + subprojects/libvduse/linux-headers/linux | 1 + subprojects/libvduse/meson.build | 10 + subprojects/libvduse/standard-headers/linux | 1 + 11 files changed, 1441 insertions(+) create mode 120000 subprojects/libvduse/include/atomic.h create mode 120000 subprojects/libvduse/include/compiler.h create mode 100644 subprojects/libvduse/libvduse.h create mode 100644 subprojects/libvduse/libvduse.c create mode 120000 subprojects/libvduse/linux-headers/linux create mode 100644 subprojects/libvduse/meson.build create mode 120000 subprojects/libvduse/standard-headers/linux diff --git a/meson_options.txt b/meson_options.txt index 2de94af037..50da8dea94 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -253,6 +253,8 @@ option('virtfs', type: 'feature', value: 'auto', description: 'virtio-9p support') option('virtiofsd', type: 'feature', value: 'auto', description: 'build virtiofs daemon (virtiofsd)') +option('libvduse', type: 'feature', value: 'auto', + description: 'build VDUSE Library') option('capstone', type: 'feature', value: 'auto', description: 'Whether and how to find the capstone library') diff --git a/subprojects/libvduse/include/atomic.h b/subprojects/libvduse/include/atomic.h new file mode 120000 index 0000000000..8c2be64f7b --- /dev/null +++ b/subprojects/libvduse/include/atomic.h @@ -0,0 +1 @@ +../../../include/qemu/atomic.h \ No newline at end of file diff --git a/subprojects/libvduse/include/compiler.h b/subprojects/libvduse/include/compiler.h new file mode 120000 index 0000000000..de7b70697c --- /dev/null +++ b/subprojects/libvduse/include/compiler.h @@ -0,0 +1 @@ +../../../include/qemu/compiler.h \ No newline at end of file diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h new file mode 100644 index 0000000000..6c2fe98213 --- /dev/null +++ b/subprojects/libvduse/libvduse.h @@ -0,0 +1,235 @@ +/* + * VDUSE (vDPA Device in Userspace) library + * + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * + * Author: + * Xie Yongji + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#ifndef LIBVDUSE_H +#define LIBVDUSE_H + +#include +#include + +#define VIRTQUEUE_MAX_SIZE 1024 + +/* VDUSE device structure */ +typedef struct VduseDev VduseDev; + +/* Virtqueue structure */ +typedef struct VduseVirtq VduseVirtq; + +/* Some operation of VDUSE backend */ +typedef struct VduseOps { + /* Called when virtqueue can be processed */ + void (*enable_queue)(VduseDev *dev, VduseVirtq *vq); + /* Called when virtqueue processing should be stopped */ + void (*disable_queue)(VduseDev *dev, VduseVirtq *vq); +} VduseOps; + +/* Describing elements of the I/O buffer */ +typedef struct VduseVirtqElement { + /* Descriptor table index */ + unsigned int index; + /* Number of physically-contiguous device-readable descriptors */ + unsigned int out_num; + /* Number of physically-contiguous device-writable descriptors */ + unsigned int in_num; + /* Array to store physically-contiguous device-writable descriptors */ + struct iovec *in_sg; + /* Array to store physically-contiguous device-readable descriptors */ + struct iovec *out_sg; +} VduseVirtqElement; + + +/** + * vduse_get_virtio_features: + * + * Get supported virtio features + * + * Returns: supported feature bits + */ +uint64_t vduse_get_virtio_features(void); + +/** + * vduse_queue_get_dev: + * @vq: specified virtqueue + * + * Get corresponding VDUSE device from the virtqueue. + * + * Returns: a pointer to VDUSE device on success, NULL on failure. + */ +VduseDev *vduse_queue_get_dev(VduseVirtq *vq); + +/** + * vduse_queue_get_fd: + * @vq: specified virtqueue + * + * Get the kick fd for the virtqueue. + * + * Returns: file descriptor on success, -1 on failure. + */ +int vduse_queue_get_fd(VduseVirtq *vq); + +/** + * vduse_queue_pop: + * @vq: specified virtqueue + * @sz: the size of struct to return (must be >= VduseVirtqElement) + * + * Pop an element from virtqueue available ring. + * + * Returns: a pointer to a structure containing VduseVirtqElement on success, + * NULL on failure. + */ +void *vduse_queue_pop(VduseVirtq *vq, size_t sz); + +/** + * vduse_queue_push: + * @vq: specified virtqueue + * @elem: pointer to VduseVirtqElement returned by vduse_queue_pop() + * @len: length in bytes to write + * + * Push an element to virtqueue used ring. + */ +void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem, + unsigned int len); +/** + * vduse_queue_notify: + * @vq: specified virtqueue + * + * Request to notify the queue. + */ +void vduse_queue_notify(VduseVirtq *vq); + +/** + * vduse_dev_get_priv: + * @dev: VDUSE device + * + * Get the private pointer passed to vduse_dev_create(). + * + * Returns: private pointer on success, NULL on failure. + */ +void *vduse_dev_get_priv(VduseDev *dev); + +/** + * vduse_dev_get_queue: + * @dev: VDUSE device + * @index: virtqueue index + * + * Get the specified virtqueue. + * + * Returns: a pointer to the virtqueue on success, NULL on failure. + */ +VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index); + +/** + * vduse_dev_get_fd: + * @dev: VDUSE device + * + * Get the control message fd for the VDUSE device. + * + * Returns: file descriptor on success, -1 on failure. + */ +int vduse_dev_get_fd(VduseDev *dev); + +/** + * vduse_dev_handler: + * @dev: VDUSE device + * + * Used to process the control message. + * + * Returns: file descriptor on success, -errno on failure. + */ +int vduse_dev_handler(VduseDev *dev); + +/** + * vduse_dev_update_config: + * @dev: VDUSE device + * @size: the size to write to configuration space + * @offset: the offset from the beginning of configuration space + * @buffer: the buffer used to write from + * + * Update device configuration space and inject a config interrupt. + * + * Returns: 0 on success, -errno on failure. + */ +int vduse_dev_update_config(VduseDev *dev, uint32_t size, + uint32_t offset, char *buffer); + +/** + * vduse_dev_setup_queue: + * @dev: VDUSE device + * @index: virtqueue index + * @max_size: the max size of virtqueue + * + * Setup the specified virtqueue. + * + * Returns: 0 on success, -errno on failure. + */ +int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size); + +/** + * vduse_dev_create_by_fd: + * @fd: passed file descriptor + * @num_queues: the number of virtqueues + * @ops: the operation of VDUSE backend + * @priv: private pointer + * + * Create VDUSE device from a passed file descriptor. + * + * Returns: pointer to VDUSE device on success, NULL on failure. + */ +VduseDev *vduse_dev_create_by_fd(int fd, uint16_t num_queues, + const VduseOps *ops, void *priv); + +/** + * vduse_dev_create_by_name: + * @name: VDUSE device name + * @num_queues: the number of virtqueues + * @ops: the operation of VDUSE backend + * @priv: private pointer + * + * Create VDUSE device on /dev/vduse/$NAME. + * + * Returns: pointer to VDUSE device on success, NULL on failure. + */ +VduseDev *vduse_dev_create_by_name(const char *name, uint16_t num_queues, + const VduseOps *ops, void *priv); + +/** + * vduse_dev_create: + * @name: VDUSE device name + * @device_id: virtio device id + * @vendor_id: virtio vendor id + * @features: virtio features + * @num_queues: the number of virtqueues + * @config_size: the size of the configuration space + * @config: the buffer of the configuration space + * @ops: the operation of VDUSE backend + * @priv: private pointer + * + * Create VDUSE device. + * + * Returns: pointer to VDUSE device on success, NULL on failure. + */ +VduseDev *vduse_dev_create(const char *name, uint32_t device_id, + uint32_t vendor_id, uint64_t features, + uint16_t num_queues, uint32_t config_size, + char *config, const VduseOps *ops, void *priv); + +/** + * vduse_dev_destroy: + * @dev: VDUSE device + * + * Destroy the VDUSE device. + * + * Returns: 0 on success, -errno on failure. + */ +int vduse_dev_destroy(VduseDev *dev); + +#endif diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c new file mode 100644 index 0000000000..fa4822b9a9 --- /dev/null +++ b/subprojects/libvduse/libvduse.c @@ -0,0 +1,1167 @@ +/* + * VDUSE (vDPA Device in Userspace) library + * + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * Portions of codes and concepts borrowed from libvhost-user.c, so: + * Copyright IBM, Corp. 2007 + * Copyright (c) 2016 Red Hat, Inc. + * + * Author: + * Xie Yongji + * Anthony Liguori + * Marc-André Lureau + * Victor Kaplansky + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "include/atomic.h" +#include "linux-headers/linux/virtio_ring.h" +#include "linux-headers/linux/virtio_config.h" +#include "linux-headers/linux/vduse.h" +#include "libvduse.h" + +#define VDUSE_VQ_ALIGN 4096 +#define MAX_IOVA_REGIONS 256 + +/* Round number down to multiple */ +#define ALIGN_DOWN(n, m) ((n) / (m) * (m)) + +/* Round number up to multiple */ +#define ALIGN_UP(n, m) ALIGN_DOWN((n) + (m) - 1, (m)) + +#ifndef unlikely +#define unlikely(x) __builtin_expect(!!(x), 0) +#endif + +typedef struct VduseRing { + unsigned int num; + uint64_t desc_addr; + uint64_t avail_addr; + uint64_t used_addr; + struct vring_desc *desc; + struct vring_avail *avail; + struct vring_used *used; +} VduseRing; + +struct VduseVirtq { + VduseRing vring; + uint16_t last_avail_idx; + uint16_t shadow_avail_idx; + uint16_t used_idx; + uint16_t signalled_used; + bool signalled_used_valid; + int index; + int inuse; + bool ready; + int fd; + VduseDev *dev; +}; + +typedef struct VduseIovaRegion { + uint64_t iova; + uint64_t size; + uint64_t mmap_offset; + uint64_t mmap_addr; +} VduseIovaRegion; + +struct VduseDev { + VduseVirtq *vqs; + VduseIovaRegion regions[MAX_IOVA_REGIONS]; + int num_regions; + char *name; + uint32_t device_id; + uint32_t vendor_id; + uint16_t num_queues; + uint16_t queue_size; + uint64_t features; + const VduseOps *ops; + int fd; + int ctrl_fd; + void *priv; +}; + +static inline bool has_feature(uint64_t features, unsigned int fbit) +{ + assert(fbit < 64); + return !!(features & (1ULL << fbit)); +} + +static inline bool vduse_dev_has_feature(VduseDev *dev, unsigned int fbit) +{ + return has_feature(dev->features, fbit); +} + +uint64_t vduse_get_virtio_features(void) +{ + return (1ULL << VIRTIO_F_IOMMU_PLATFORM) | + (1ULL << VIRTIO_F_VERSION_1) | + (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) | + (1ULL << VIRTIO_RING_F_EVENT_IDX) | + (1ULL << VIRTIO_RING_F_INDIRECT_DESC); +} + +VduseDev *vduse_queue_get_dev(VduseVirtq *vq) +{ + return vq->dev; +} + +int vduse_queue_get_fd(VduseVirtq *vq) +{ + return vq->fd; +} + +void *vduse_dev_get_priv(VduseDev *dev) +{ + return dev->priv; +} + +VduseVirtq *vduse_dev_get_queue(VduseDev *dev, int index) +{ + return &dev->vqs[index]; +} + +int vduse_dev_get_fd(VduseDev *dev) +{ + return dev->fd; +} + +static int vduse_inject_irq(VduseDev *dev, int index) +{ + return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index); +} + +static void vduse_iova_remove_region(VduseDev *dev, uint64_t start, + uint64_t last) +{ + int i; + + if (last == start) { + return; + } + + for (i = 0; i < MAX_IOVA_REGIONS; i++) { + if (!dev->regions[i].mmap_addr) { + continue; + } + + if (start <= dev->regions[i].iova && + last >= (dev->regions[i].iova + dev->regions[i].size - 1)) { + munmap((void *)dev->regions[i].mmap_addr, + dev->regions[i].mmap_offset + dev->regions[i].size); + dev->regions[i].mmap_addr = 0; + dev->num_regions--; + } + } +} + +static int vduse_iova_add_region(VduseDev *dev, int fd, + uint64_t offset, uint64_t start, + uint64_t last, int prot) +{ + int i; + uint64_t size = last - start + 1; + void *mmap_addr = mmap(0, size + offset, prot, MAP_SHARED, fd, 0); + + if (mmap_addr == MAP_FAILED) { + close(fd); + return -EINVAL; + } + + for (i = 0; i < MAX_IOVA_REGIONS; i++) { + if (!dev->regions[i].mmap_addr) { + dev->regions[i].mmap_addr = (uint64_t)(uintptr_t)mmap_addr; + dev->regions[i].mmap_offset = offset; + dev->regions[i].iova = start; + dev->regions[i].size = size; + dev->num_regions++; + break; + } + } + assert(i < MAX_IOVA_REGIONS); + close(fd); + + return 0; +} + +static int perm_to_prot(uint8_t perm) +{ + int prot = 0; + + switch (perm) { + case VDUSE_ACCESS_WO: + prot |= PROT_WRITE; + break; + case VDUSE_ACCESS_RO: + prot |= PROT_READ; + break; + case VDUSE_ACCESS_RW: + prot |= PROT_READ | PROT_WRITE; + break; + default: + break; + } + + return prot; +} + +static inline void *iova_to_va(VduseDev *dev, uint64_t *plen, uint64_t iova) +{ + int i, ret; + struct vduse_iotlb_entry entry; + + for (i = 0; i < MAX_IOVA_REGIONS; i++) { + VduseIovaRegion *r = &dev->regions[i]; + + if (!r->mmap_addr) { + continue; + } + + if ((iova >= r->iova) && (iova < (r->iova + r->size))) { + if ((iova + *plen) > (r->iova + r->size)) { + *plen = r->iova + r->size - iova; + } + return (void *)(uintptr_t)(iova - r->iova + + r->mmap_addr + r->mmap_offset); + } + } + + entry.start = iova; + entry.last = iova + 1; + ret = ioctl(dev->fd, VDUSE_IOTLB_GET_FD, &entry); + if (ret < 0) { + return NULL; + } + + if (!vduse_iova_add_region(dev, ret, entry.offset, entry.start, + entry.last, perm_to_prot(entry.perm))) { + return iova_to_va(dev, plen, iova); + } + + return NULL; +} + +static inline uint16_t vring_avail_flags(VduseVirtq *vq) +{ + return le16toh(vq->vring.avail->flags); +} + +static inline uint16_t vring_avail_idx(VduseVirtq *vq) +{ + vq->shadow_avail_idx = le16toh(vq->vring.avail->idx); + + return vq->shadow_avail_idx; +} + +static inline uint16_t vring_avail_ring(VduseVirtq *vq, int i) +{ + return le16toh(vq->vring.avail->ring[i]); +} + +static inline uint16_t vring_get_used_event(VduseVirtq *vq) +{ + return vring_avail_ring(vq, vq->vring.num); +} + +static bool vduse_queue_get_head(VduseVirtq *vq, unsigned int idx, + unsigned int *head) +{ + /* + * Grab the next descriptor number they're advertising, and increment + * the index we've seen. + */ + *head = vring_avail_ring(vq, idx % vq->vring.num); + + /* If their number is silly, that's a fatal mistake. */ + if (*head >= vq->vring.num) { + fprintf(stderr, "Guest says index %u is available\n", *head); + return false; + } + + return true; +} + +static int +vduse_queue_read_indirect_desc(VduseDev *dev, struct vring_desc *desc, + uint64_t addr, size_t len) +{ + struct vring_desc *ori_desc; + uint64_t read_len; + + if (len > (VIRTQUEUE_MAX_SIZE * sizeof(struct vring_desc))) { + return -1; + } + + if (len == 0) { + return -1; + } + + while (len) { + read_len = len; + ori_desc = iova_to_va(dev, &read_len, addr); + if (!ori_desc) { + return -1; + } + + memcpy(desc, ori_desc, read_len); + len -= read_len; + addr += read_len; + desc += read_len; + } + + return 0; +} + +enum { + VIRTQUEUE_READ_DESC_ERROR = -1, + VIRTQUEUE_READ_DESC_DONE = 0, /* end of chain */ + VIRTQUEUE_READ_DESC_MORE = 1, /* more buffers in chain */ +}; + +static int vduse_queue_read_next_desc(struct vring_desc *desc, int i, + unsigned int max, unsigned int *next) +{ + /* If this descriptor says it doesn't chain, we're done. */ + if (!(le16toh(desc[i].flags) & VRING_DESC_F_NEXT)) { + return VIRTQUEUE_READ_DESC_DONE; + } + + /* Check they're not leading us off end of descriptors. */ + *next = desc[i].next; + /* Make sure compiler knows to grab that: we don't want it changing! */ + smp_wmb(); + + if (*next >= max) { + fprintf(stderr, "Desc next is %u\n", *next); + return VIRTQUEUE_READ_DESC_ERROR; + } + + return VIRTQUEUE_READ_DESC_MORE; +} + +/* + * Fetch avail_idx from VQ memory only when we really need to know if + * guest has added some buffers. + */ +static bool vduse_queue_empty(VduseVirtq *vq) +{ + if (unlikely(!vq->vring.avail)) { + return true; + } + + if (vq->shadow_avail_idx != vq->last_avail_idx) { + return false; + } + + return vring_avail_idx(vq) == vq->last_avail_idx; +} + +static bool vduse_queue_should_notify(VduseVirtq *vq) +{ + VduseDev *dev = vq->dev; + uint16_t old, new; + bool v; + + /* We need to expose used array entries before checking used event. */ + smp_mb(); + + /* Always notify when queue is empty (when feature acknowledge) */ + if (vduse_dev_has_feature(dev, VIRTIO_F_NOTIFY_ON_EMPTY) && + !vq->inuse && vduse_queue_empty(vq)) { + return true; + } + + if (!vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) { + return !(vring_avail_flags(vq) & VRING_AVAIL_F_NO_INTERRUPT); + } + + v = vq->signalled_used_valid; + vq->signalled_used_valid = true; + old = vq->signalled_used; + new = vq->signalled_used = vq->used_idx; + return !v || vring_need_event(vring_get_used_event(vq), new, old); +} + +void vduse_queue_notify(VduseVirtq *vq) +{ + VduseDev *dev = vq->dev; + + if (unlikely(!vq->vring.avail)) { + return; + } + + if (!vduse_queue_should_notify(vq)) { + return; + } + + if (vduse_inject_irq(dev, vq->index) < 0) { + fprintf(stderr, "Error inject irq for vq %d: %s\n", + vq->index, strerror(errno)); + } +} + +static inline void vring_used_flags_set_bit(VduseVirtq *vq, int mask) +{ + uint16_t *flags; + + flags = (uint16_t *)((char*)vq->vring.used + + offsetof(struct vring_used, flags)); + *flags = htole16(le16toh(*flags) | mask); +} + +static inline void vring_used_flags_unset_bit(VduseVirtq *vq, int mask) +{ + uint16_t *flags; + + flags = (uint16_t *)((char*)vq->vring.used + + offsetof(struct vring_used, flags)); + *flags = htole16(le16toh(*flags) & ~mask); +} + +static inline void vring_set_avail_event(VduseVirtq *vq, uint16_t val) +{ + *((uint16_t *)&vq->vring.used->ring[vq->vring.num]) = htole16(val); +} + +static bool vduse_queue_map_single_desc(VduseVirtq *vq, unsigned int *p_num_sg, + struct iovec *iov, unsigned int max_num_sg, + bool is_write, uint64_t pa, size_t sz) +{ + unsigned num_sg = *p_num_sg; + VduseDev *dev = vq->dev; + + assert(num_sg <= max_num_sg); + + if (!sz) { + fprintf(stderr, "virtio: zero sized buffers are not allowed\n"); + return false; + } + + while (sz) { + uint64_t len = sz; + + if (num_sg == max_num_sg) { + fprintf(stderr, + "virtio: too many descriptors in indirect table\n"); + return false; + } + + iov[num_sg].iov_base = iova_to_va(dev, &len, pa); + if (iov[num_sg].iov_base == NULL) { + fprintf(stderr, "virtio: invalid address for buffers\n"); + return false; + } + iov[num_sg++].iov_len = len; + sz -= len; + pa += len; + } + + *p_num_sg = num_sg; + return true; +} + +static void *vduse_queue_alloc_element(size_t sz, unsigned out_num, + unsigned in_num) +{ + VduseVirtqElement *elem; + size_t in_sg_ofs = ALIGN_UP(sz, __alignof__(elem->in_sg[0])); + size_t out_sg_ofs = in_sg_ofs + in_num * sizeof(elem->in_sg[0]); + size_t out_sg_end = out_sg_ofs + out_num * sizeof(elem->out_sg[0]); + + assert(sz >= sizeof(VduseVirtqElement)); + elem = malloc(out_sg_end); + if (!elem) { + return NULL; + } + elem->out_num = out_num; + elem->in_num = in_num; + elem->in_sg = (void *)elem + in_sg_ofs; + elem->out_sg = (void *)elem + out_sg_ofs; + return elem; +} + +static void *vduse_queue_map_desc(VduseVirtq *vq, unsigned int idx, size_t sz) +{ + struct vring_desc *desc = vq->vring.desc; + VduseDev *dev = vq->dev; + uint64_t desc_addr, read_len; + unsigned int desc_len; + unsigned int max = vq->vring.num; + unsigned int i = idx; + VduseVirtqElement *elem; + struct iovec iov[VIRTQUEUE_MAX_SIZE]; + struct vring_desc desc_buf[VIRTQUEUE_MAX_SIZE]; + unsigned int out_num = 0, in_num = 0; + int rc; + + if (le16toh(desc[i].flags) & VRING_DESC_F_INDIRECT) { + if (le32toh(desc[i].len) % sizeof(struct vring_desc)) { + fprintf(stderr, "Invalid size for indirect buffer table\n"); + return NULL; + } + + /* loop over the indirect descriptor table */ + desc_addr = le64toh(desc[i].addr); + desc_len = le32toh(desc[i].len); + max = desc_len / sizeof(struct vring_desc); + read_len = desc_len; + desc = iova_to_va(dev, &read_len, desc_addr); + if (unlikely(desc && read_len != desc_len)) { + /* Failed to use zero copy */ + desc = NULL; + if (!vduse_queue_read_indirect_desc(dev, desc_buf, + desc_addr, + desc_len)) { + desc = desc_buf; + } + } + if (!desc) { + fprintf(stderr, "Invalid indirect buffer table\n"); + return NULL; + } + i = 0; + } + + /* Collect all the descriptors */ + do { + if (le16toh(desc[i].flags) & VRING_DESC_F_WRITE) { + if (!vduse_queue_map_single_desc(vq, &in_num, iov + out_num, + VIRTQUEUE_MAX_SIZE - out_num, + true, le64toh(desc[i].addr), + le32toh(desc[i].len))) { + return NULL; + } + } else { + if (in_num) { + fprintf(stderr, "Incorrect order for descriptors\n"); + return NULL; + } + if (!vduse_queue_map_single_desc(vq, &out_num, iov, + VIRTQUEUE_MAX_SIZE, false, + le64toh(desc[i].addr), + le32toh(desc[i].len))) { + return NULL; + } + } + + /* If we've got too many, that implies a descriptor loop. */ + if ((in_num + out_num) > max) { + fprintf(stderr, "Looped descriptor\n"); + return NULL; + } + rc = vduse_queue_read_next_desc(desc, i, max, &i); + } while (rc == VIRTQUEUE_READ_DESC_MORE); + + if (rc == VIRTQUEUE_READ_DESC_ERROR) { + fprintf(stderr, "read descriptor error\n"); + return NULL; + } + + /* Now copy what we have collected and mapped */ + elem = vduse_queue_alloc_element(sz, out_num, in_num); + if (!elem) { + fprintf(stderr, "read descriptor error\n"); + return NULL; + } + elem->index = idx; + for (i = 0; i < out_num; i++) { + elem->out_sg[i] = iov[i]; + } + for (i = 0; i < in_num; i++) { + elem->in_sg[i] = iov[out_num + i]; + } + + return elem; +} + +void *vduse_queue_pop(VduseVirtq *vq, size_t sz) +{ + unsigned int head; + VduseVirtqElement *elem; + VduseDev *dev = vq->dev; + + if (unlikely(!vq->vring.avail)) { + return NULL; + } + + if (vduse_queue_empty(vq)) { + return NULL; + } + /* Needed after virtio_queue_empty() */ + smp_rmb(); + + if (vq->inuse >= vq->vring.num) { + fprintf(stderr, "Virtqueue size exceeded: %d\n", vq->inuse); + return NULL; + } + + if (!vduse_queue_get_head(vq, vq->last_avail_idx++, &head)) { + return NULL; + } + + if (vduse_dev_has_feature(dev, VIRTIO_RING_F_EVENT_IDX)) { + vring_set_avail_event(vq, vq->last_avail_idx); + } + + elem = vduse_queue_map_desc(vq, head, sz); + + if (!elem) { + return NULL; + } + + vq->inuse++; + + return elem; +} + +static inline void vring_used_write(VduseVirtq *vq, + struct vring_used_elem *uelem, int i) +{ + struct vring_used *used = vq->vring.used; + + used->ring[i] = *uelem; +} + +static void vduse_queue_fill(VduseVirtq *vq, const VduseVirtqElement *elem, + unsigned int len, unsigned int idx) +{ + struct vring_used_elem uelem; + + if (unlikely(!vq->vring.used)) { + return; + } + + idx = (idx + vq->used_idx) % vq->vring.num; + + uelem.id = htole32(elem->index); + uelem.len = htole32(len); + vring_used_write(vq, &uelem, idx); +} + +static inline void vring_used_idx_set(VduseVirtq *vq, uint16_t val) +{ + vq->vring.used->idx = htole16(val); + vq->used_idx = val; +} + +static void vduse_queue_flush(VduseVirtq *vq, unsigned int count) +{ + uint16_t old, new; + + if (unlikely(!vq->vring.used)) { + return; + } + + /* Make sure buffer is written before we update index. */ + smp_wmb(); + + old = vq->used_idx; + new = old + count; + vring_used_idx_set(vq, new); + vq->inuse -= count; + if (unlikely((int16_t)(new - vq->signalled_used) < (uint16_t)(new - old))) { + vq->signalled_used_valid = false; + } +} + +void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem, + unsigned int len) +{ + vduse_queue_fill(vq, elem, len, 0); + vduse_queue_flush(vq, 1); +} + +static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr, + uint64_t avail_addr, uint64_t used_addr) +{ + struct VduseDev *dev = vq->dev; + uint64_t len; + + len = sizeof(struct vring_desc); + vq->vring.desc = iova_to_va(dev, &len, desc_addr); + if (len != sizeof(struct vring_desc)) { + return -EINVAL; + } + + len = sizeof(struct vring_avail); + vq->vring.avail = iova_to_va(dev, &len, avail_addr); + if (len != sizeof(struct vring_avail)) { + return -EINVAL; + } + + len = sizeof(struct vring_used); + vq->vring.used = iova_to_va(dev, &len, used_addr); + if (len != sizeof(struct vring_used)) { + return -EINVAL; + } + + if (!vq->vring.desc || !vq->vring.avail || !vq->vring.used) { + fprintf(stderr, "Failed to get vq[%d] iova mapping\n", vq->index); + return -EINVAL; + } + + return 0; +} + +static void vduse_queue_enable(VduseVirtq *vq) +{ + struct VduseDev *dev = vq->dev; + struct vduse_vq_info vq_info; + struct vduse_vq_eventfd vq_eventfd; + int fd; + + vq_info.index = vq->index; + if (ioctl(dev->fd, VDUSE_VQ_GET_INFO, &vq_info)) { + fprintf(stderr, "Failed to get vq[%d] info: %s\n", + vq->index, strerror(errno)); + return; + } + + if (!vq_info.ready) { + return; + } + + vq->vring.num = vq_info.num; + vq->vring.desc_addr = vq_info.desc_addr; + vq->vring.avail_addr = vq_info.driver_addr; + vq->vring.used_addr = vq_info.device_addr; + + if (vduse_queue_update_vring(vq, vq_info.desc_addr, + vq_info.driver_addr, vq_info.device_addr)) { + fprintf(stderr, "Failed to update vring for vq[%d]\n", vq->index); + return; + } + + fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (fd < 0) { + fprintf(stderr, "Failed to init eventfd for vq[%d]\n", vq->index); + return; + } + + vq_eventfd.index = vq->index; + vq_eventfd.fd = fd; + if (ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &vq_eventfd)) { + fprintf(stderr, "Failed to setup kick fd for vq[%d]\n", vq->index); + close(fd); + return; + } + + vq->fd = fd; + vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index; + vq->inuse = 0; + vq->used_idx = 0; + vq->signalled_used_valid = false; + vq->ready = true; + + dev->ops->enable_queue(dev, vq); +} + +static void vduse_queue_disable(VduseVirtq *vq) +{ + struct VduseDev *dev = vq->dev; + struct vduse_vq_eventfd eventfd; + + if (!vq->ready) { + return; + } + + dev->ops->disable_queue(dev, vq); + + eventfd.index = vq->index; + eventfd.fd = VDUSE_EVENTFD_DEASSIGN; + ioctl(dev->fd, VDUSE_VQ_SETUP_KICKFD, &eventfd); + close(vq->fd); + + assert(vq->inuse == 0); + + vq->vring.num = 0; + vq->vring.desc_addr = 0; + vq->vring.avail_addr = 0; + vq->vring.used_addr = 0; + vq->vring.desc = 0; + vq->vring.avail = 0; + vq->vring.used = 0; + vq->ready = false; + vq->fd = -1; +} + +static void vduse_dev_start_dataplane(VduseDev *dev) +{ + int i; + + if (ioctl(dev->fd, VDUSE_DEV_GET_FEATURES, &dev->features)) { + fprintf(stderr, "Failed to get features: %s\n", strerror(errno)); + return; + } + assert(vduse_dev_has_feature(dev, VIRTIO_F_VERSION_1)); + + for (i = 0; i < dev->num_queues; i++) { + vduse_queue_enable(&dev->vqs[i]); + } +} + +static void vduse_dev_stop_dataplane(VduseDev *dev) +{ + int i; + + for (i = 0; i < dev->num_queues; i++) { + vduse_queue_disable(&dev->vqs[i]); + } + dev->features = 0; + vduse_iova_remove_region(dev, 0, ULONG_MAX); +} + +int vduse_dev_handler(VduseDev *dev) +{ + struct vduse_dev_request req; + struct vduse_dev_response resp = { 0 }; + VduseVirtq *vq; + int i, ret; + + ret = read(dev->fd, &req, sizeof(req)); + if (ret != sizeof(req)) { + fprintf(stderr, "Read request error [%d]: %s\n", + ret, strerror(errno)); + return -errno; + } + resp.request_id = req.request_id; + + switch (req.type) { + case VDUSE_GET_VQ_STATE: + vq = &dev->vqs[req.vq_state.index]; + resp.vq_state.split.avail_index = vq->last_avail_idx; + resp.result = VDUSE_REQ_RESULT_OK; + break; + case VDUSE_SET_STATUS: + if (req.s.status & VIRTIO_CONFIG_S_DRIVER_OK) { + vduse_dev_start_dataplane(dev); + } else if (req.s.status == 0) { + vduse_dev_stop_dataplane(dev); + } + resp.result = VDUSE_REQ_RESULT_OK; + break; + case VDUSE_UPDATE_IOTLB: + /* The iova will be updated by iova_to_va() later, so just remove it */ + vduse_iova_remove_region(dev, req.iova.start, req.iova.last); + for (i = 0; i < dev->num_queues; i++) { + VduseVirtq *vq = &dev->vqs[i]; + if (vq->ready) { + if (vduse_queue_update_vring(vq, vq->vring.desc_addr, + vq->vring.avail_addr, + vq->vring.used_addr)) { + fprintf(stderr, "Failed to update vring for vq[%d]\n", + vq->index); + } + } + } + resp.result = VDUSE_REQ_RESULT_OK; + break; + default: + resp.result = VDUSE_REQ_RESULT_FAILED; + break; + } + + ret = write(dev->fd, &resp, sizeof(resp)); + if (ret != sizeof(resp)) { + fprintf(stderr, "Write request %d error [%d]: %s\n", + req.type, ret, strerror(errno)); + return -errno; + } + return 0; +} + +int vduse_dev_update_config(VduseDev *dev, uint32_t size, + uint32_t offset, char *buffer) +{ + int ret; + struct vduse_config_data *data; + + data = malloc(offsetof(struct vduse_config_data, buffer) + size); + if (!data) { + return -ENOMEM; + } + + data->offset = offset; + data->length = size; + memcpy(data->buffer, buffer, size); + + ret = ioctl(dev->fd, VDUSE_DEV_SET_CONFIG, data); + free(data); + + if (ret) { + return -errno; + } + + if (ioctl(dev->fd, VDUSE_DEV_INJECT_CONFIG_IRQ)) { + return -errno; + } + + return 0; +} + +int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size) +{ + VduseVirtq *vq = &dev->vqs[index]; + struct vduse_vq_config vq_config = { 0 }; + + if (max_size > VIRTQUEUE_MAX_SIZE) { + return -EINVAL; + } + + vq_config.index = vq->index; + vq_config.max_size = max_size; + + if (ioctl(dev->fd, VDUSE_VQ_SETUP, &vq_config)) { + return -errno; + } + + return 0; +} + +static int vduse_dev_init_vqs(VduseDev *dev, uint16_t num_queues) +{ + VduseVirtq *vqs; + int i; + + vqs = calloc(sizeof(VduseVirtq), num_queues); + if (!vqs) { + return -ENOMEM; + } + + for (i = 0; i < num_queues; i++) { + vqs[i].index = i; + vqs[i].dev = dev; + vqs[i].fd = -1; + } + dev->vqs = vqs; + + return 0; +} + +static int vduse_dev_init(VduseDev *dev, const char *name, + uint16_t num_queues, const VduseOps *ops, + void *priv) +{ + char *dev_path, *dev_name; + int ret, fd; + + dev_path = malloc(strlen(name) + strlen("/dev/vduse/") + 1); + if (!dev_path) { + return -ENOMEM; + } + sprintf(dev_path, "/dev/vduse/%s", name); + + fd = open(dev_path, O_RDWR); + free(dev_path); + if (fd < 0) { + fprintf(stderr, "Failed to open vduse dev %s: %s\n", + name, strerror(errno)); + return -errno; + } + + dev_name = strdup(name); + if (!dev_name) { + close(fd); + return -ENOMEM; + } + + ret = vduse_dev_init_vqs(dev, num_queues); + if (ret) { + free(dev_name); + close(fd); + return ret; + } + + dev->name = dev_name; + dev->num_queues = num_queues; + dev->fd = fd; + dev->ops = ops; + dev->priv = priv; + + return 0; +} + +static inline bool vduse_name_is_valid(const char *name) +{ + return strlen(name) >= VDUSE_NAME_MAX || strstr(name, ".."); +} + +VduseDev *vduse_dev_create_by_fd(int fd, uint16_t num_queues, + const VduseOps *ops, void *priv) +{ + VduseDev *dev; + int ret; + + if (!ops || !ops->enable_queue || !ops->disable_queue) { + fprintf(stderr, "Invalid parameter for vduse\n"); + return NULL; + } + + dev = calloc(sizeof(VduseDev), 1); + if (!dev) { + fprintf(stderr, "Failed to allocate vduse device\n"); + return NULL; + } + + ret = vduse_dev_init_vqs(dev, num_queues); + if (ret) { + fprintf(stderr, "Failed to init vqs\n"); + free(dev); + return NULL; + } + + dev->num_queues = num_queues; + dev->fd = fd; + dev->ops = ops; + dev->priv = priv; + + return dev; +} + +VduseDev *vduse_dev_create_by_name(const char *name, uint16_t num_queues, + const VduseOps *ops, void *priv) +{ + VduseDev *dev; + int ret; + + if (!name || vduse_name_is_valid(name) || !ops || + !ops->enable_queue || !ops->disable_queue) { + fprintf(stderr, "Invalid parameter for vduse\n"); + return NULL; + } + + dev = calloc(sizeof(VduseDev), 1); + if (!dev) { + fprintf(stderr, "Failed to allocate vduse device\n"); + return NULL; + } + + ret = vduse_dev_init(dev, name, num_queues, ops, priv); + if (ret < 0) { + fprintf(stderr, "Failed to init vduse device %s: %s\n", + name, strerror(ret)); + free(dev); + return NULL; + } + + return dev; +} + +VduseDev *vduse_dev_create(const char *name, uint32_t device_id, + uint32_t vendor_id, uint64_t features, + uint16_t num_queues, uint32_t config_size, + char *config, const VduseOps *ops, void *priv) +{ + VduseDev *dev; + int ret, ctrl_fd; + uint64_t version; + struct vduse_dev_config *dev_config; + size_t size = offsetof(struct vduse_dev_config, config); + + if (!name || vduse_name_is_valid(name) || + !has_feature(features, VIRTIO_F_VERSION_1) || !config || + !config_size || !ops || !ops->enable_queue || !ops->disable_queue) { + fprintf(stderr, "Invalid parameter for vduse\n"); + return NULL; + } + + dev = calloc(sizeof(VduseDev), 1); + if (!dev) { + fprintf(stderr, "Failed to allocate vduse device\n"); + return NULL; + } + + ctrl_fd = open("/dev/vduse/control", O_RDWR); + if (ctrl_fd < 0) { + fprintf(stderr, "Failed to open /dev/vduse/control: %s\n", + strerror(errno)); + goto err_ctrl; + } + + version = VDUSE_API_VERSION; + if (ioctl(ctrl_fd, VDUSE_SET_API_VERSION, &version)) { + fprintf(stderr, "Failed to set api version %lu: %s\n", + version, strerror(errno)); + goto err_dev; + } + + dev_config = calloc(size + config_size, 1); + if (!dev_config) { + fprintf(stderr, "Failed to allocate config space\n"); + goto err_dev; + } + + strcpy(dev_config->name, name); + dev_config->device_id = device_id; + dev_config->vendor_id = vendor_id; + dev_config->features = features; + dev_config->vq_num = num_queues; + dev_config->vq_align = VDUSE_VQ_ALIGN; + dev_config->config_size = config_size; + memcpy(dev_config->config, config, config_size); + + ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config); + free(dev_config); + if (ret < 0) { + fprintf(stderr, "Failed to create vduse device %s: %s\n", + name, strerror(errno)); + goto err_dev; + } + dev->ctrl_fd = ctrl_fd; + + ret = vduse_dev_init(dev, name, num_queues, ops, priv); + if (ret < 0) { + fprintf(stderr, "Failed to init vduse device %s: %s\n", + name, strerror(ret)); + goto err; + } + + return dev; +err: + ioctl(ctrl_fd, VDUSE_DESTROY_DEV, name); +err_dev: + close(ctrl_fd); +err_ctrl: + free(dev); + + return NULL; +} + +int vduse_dev_destroy(VduseDev *dev) +{ + int ret = 0; + + free(dev->vqs); + if (dev->fd > 0) { + close(dev->fd); + dev->fd = -1; + } + if (dev->ctrl_fd > 0) { + if (ioctl(dev->ctrl_fd, VDUSE_DESTROY_DEV, dev->name)) { + ret = -errno; + } + close(dev->ctrl_fd); + dev->ctrl_fd = -1; + } + free(dev->name); + free(dev); + + return ret; +} diff --git a/MAINTAINERS b/MAINTAINERS index 379879fb42..880d61d327 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3582,6 +3582,11 @@ L: qemu-block@nongnu.org S: Supported F: block/export/fuse.c +VDUSE library +M: Xie Yongji +S: Maintained +F: subprojects/libvduse/ + Replication M: Wen Congyang M: Xie Changlong diff --git a/meson.build b/meson.build index 21cd949082..2b11b2c689 100644 --- a/meson.build +++ b/meson.build @@ -1527,6 +1527,21 @@ if get_option('fuse_lseek').allowed() endif endif +have_libvduse = (targetos == 'linux') +if get_option('libvduse').enabled() + if targetos != 'linux' + error('libvduse requires linux') + endif +elif get_option('libvduse').disabled() + have_libvduse = false +endif + +libvduse = not_found +if have_libvduse + libvduse_proj = subproject('libvduse') + libvduse = libvduse_proj.get_variable('libvduse_dep') +endif + # libbpf libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config') if libbpf.found() and not cc.links(''' diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 00ea4d8cd1..3fab6c5810 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -109,6 +109,7 @@ meson_options_help() { printf "%s\n" ' libssh ssh block device support' printf "%s\n" ' libudev Use libudev to enumerate host devices' printf "%s\n" ' libusb libusb support for USB passthrough' + printf "%s\n" ' libvduse build VDUSE Library' printf "%s\n" ' linux-aio Linux AIO support' printf "%s\n" ' linux-io-uring Linux io_uring support' printf "%s\n" ' live-block-migration' @@ -302,6 +303,8 @@ _meson_option_parse() { --disable-libudev) printf "%s" -Dlibudev=disabled ;; --enable-libusb) printf "%s" -Dlibusb=enabled ;; --disable-libusb) printf "%s" -Dlibusb=disabled ;; + --enable-libvduse) printf "%s" -Dlibvduse=enabled ;; + --disable-libvduse) printf "%s" -Dlibvduse=disabled ;; --enable-linux-aio) printf "%s" -Dlinux_aio=enabled ;; --disable-linux-aio) printf "%s" -Dlinux_aio=disabled ;; --enable-linux-io-uring) printf "%s" -Dlinux_io_uring=enabled ;; diff --git a/subprojects/libvduse/linux-headers/linux b/subprojects/libvduse/linux-headers/linux new file mode 120000 index 0000000000..04f3304f79 --- /dev/null +++ b/subprojects/libvduse/linux-headers/linux @@ -0,0 +1 @@ +../../../linux-headers/linux/ \ No newline at end of file diff --git a/subprojects/libvduse/meson.build b/subprojects/libvduse/meson.build new file mode 100644 index 0000000000..ba08f5ee1a --- /dev/null +++ b/subprojects/libvduse/meson.build @@ -0,0 +1,10 @@ +project('libvduse', 'c', + license: 'GPL-2.0-or-later', + default_options: ['c_std=gnu99']) + +libvduse = static_library('vduse', + files('libvduse.c'), + c_args: '-D_GNU_SOURCE') + +libvduse_dep = declare_dependency(link_with: libvduse, + include_directories: include_directories('.')) diff --git a/subprojects/libvduse/standard-headers/linux b/subprojects/libvduse/standard-headers/linux new file mode 120000 index 0000000000..c416f068ac --- /dev/null +++ b/subprojects/libvduse/standard-headers/linux @@ -0,0 +1 @@ +../../../include/standard-headers/linux/ \ No newline at end of file From patchwork Thu Jun 9 17:21:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641403 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=JMTpk0Ji; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsws29nTz9s2R for ; Fri, 10 Jun 2022 04:26:05 +1000 (AEST) Received: from localhost ([::1]:39560 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMr5-0007rv-AC for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:26:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56292) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007sN-QW for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59386) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003gQ-W3 for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795343; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FvOUjGBl9ywbUb/wU30bZRDjGeiSDRv5KT7eWbXS+s0=; b=JMTpk0JiCgGcogrRLXauMgZI/GYdENjxcm+HE5to3yuSmY+7szDHnuhxqpsaBX8amaIzZ7 6FULky2VVVsJAe9Zy4mEo2qCe6YP467osxlZdRbHuwvgQgy8XHtfe9WaBCMHSEGeu2pax8 mpa2XjakB9eGWGC2ksQ0jRgON0Gqjsc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-394-PZYQDk0SPDenyHQtN_zdqA-1; Thu, 09 Jun 2022 13:22:22 -0400 X-MC-Unique: PZYQDk0SPDenyHQtN_zdqA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6C3C2800124; Thu, 9 Jun 2022 17:22:22 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB3C5404E4B5; Thu, 9 Jun 2022 17:22:21 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 11/18] vduse-blk: Implement vduse-blk export Date: Thu, 9 Jun 2022 19:21:42 +0200 Message-Id: <20220609172149.293877-12-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji This implements a VDUSE block backends based on the libvduse library. We can use it to export the BDSs for both VM and container (host) usage. The new command-line syntax is: $ qemu-storage-daemon \ --blockdev file,node-name=drive0,filename=test.img \ --export vduse-blk,node-name=drive0,id=vduse-export0,writable=on After the qemu-storage-daemon started, we need to use the "vdpa" command to attach the device to vDPA bus: $ vdpa dev add name vduse-export0 mgmtdev vduse Also the device must be removed via the "vdpa" command before we stop the qemu-storage-daemon. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-7-xieyongji@bytedance.com> Signed-off-by: Kevin Wolf --- qapi/block-export.json | 28 +++- meson_options.txt | 2 + block/export/vduse-blk.h | 20 +++ block/export/export.c | 6 + block/export/vduse-blk.c | 307 ++++++++++++++++++++++++++++++++++ MAINTAINERS | 4 +- block/export/meson.build | 5 + meson.build | 13 ++ scripts/meson-buildoptions.sh | 4 + 9 files changed, 385 insertions(+), 4 deletions(-) create mode 100644 block/export/vduse-blk.h create mode 100644 block/export/vduse-blk.c diff --git a/qapi/block-export.json b/qapi/block-export.json index 0685cb8b9a..e4bd4de363 100644 --- a/qapi/block-export.json +++ b/qapi/block-export.json @@ -177,6 +177,23 @@ '*allow-other': 'FuseExportAllowOther' }, 'if': 'CONFIG_FUSE' } +## +# @BlockExportOptionsVduseBlk: +# +# A vduse-blk block export. +# +# @num-queues: the number of virtqueues. Defaults to 1. +# @queue-size: the size of virtqueue. Defaults to 256. +# @logical-block-size: Logical block size in bytes. Range [512, PAGE_SIZE] +# and must be power of 2. Defaults to 512 bytes. +# +# Since: 7.1 +## +{ 'struct': 'BlockExportOptionsVduseBlk', + 'data': { '*num-queues': 'uint16', + '*queue-size': 'uint16', + '*logical-block-size': 'size'} } + ## # @NbdServerAddOptions: # @@ -280,6 +297,7 @@ # @nbd: NBD export # @vhost-user-blk: vhost-user-blk export (since 5.2) # @fuse: FUSE export (since: 6.0) +# @vduse-blk: vduse-blk export (since 7.1) # # Since: 4.2 ## @@ -287,7 +305,8 @@ 'data': [ 'nbd', { 'name': 'vhost-user-blk', 'if': 'CONFIG_VHOST_USER_BLK_SERVER' }, - { 'name': 'fuse', 'if': 'CONFIG_FUSE' } ] } + { 'name': 'fuse', 'if': 'CONFIG_FUSE' }, + { 'name': 'vduse-blk', 'if': 'CONFIG_VDUSE_BLK_EXPORT' } ] } ## # @BlockExportOptions: @@ -295,7 +314,8 @@ # Describes a block export, i.e. how single node should be exported on an # external interface. # -# @id: A unique identifier for the block export (across all export types) +# @id: A unique identifier for the block export (across the host for vduse-blk +# export type or across all export types for other types) # # @node-name: The node name of the block node to be exported (since: 5.2) # @@ -331,7 +351,9 @@ 'vhost-user-blk': { 'type': 'BlockExportOptionsVhostUserBlk', 'if': 'CONFIG_VHOST_USER_BLK_SERVER' }, 'fuse': { 'type': 'BlockExportOptionsFuse', - 'if': 'CONFIG_FUSE' } + 'if': 'CONFIG_FUSE' }, + 'vduse-blk': { 'type': 'BlockExportOptionsVduseBlk', + 'if': 'CONFIG_VDUSE_BLK_EXPORT' } } } ## diff --git a/meson_options.txt b/meson_options.txt index 50da8dea94..dee5671386 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -255,6 +255,8 @@ option('virtiofsd', type: 'feature', value: 'auto', description: 'build virtiofs daemon (virtiofsd)') option('libvduse', type: 'feature', value: 'auto', description: 'build VDUSE Library') +option('vduse_blk_export', type: 'feature', value: 'auto', + description: 'VDUSE block export support') option('capstone', type: 'feature', value: 'auto', description: 'Whether and how to find the capstone library') diff --git a/block/export/vduse-blk.h b/block/export/vduse-blk.h new file mode 100644 index 0000000000..c4eeb1b70e --- /dev/null +++ b/block/export/vduse-blk.h @@ -0,0 +1,20 @@ +/* + * Export QEMU block device via VDUSE + * + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * + * Author: + * Xie Yongji + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#ifndef VDUSE_BLK_H +#define VDUSE_BLK_H + +#include "block/export.h" + +extern const BlockExportDriver blk_exp_vduse_blk; + +#endif /* VDUSE_BLK_H */ diff --git a/block/export/export.c b/block/export/export.c index 7253af3bc3..4744862915 100644 --- a/block/export/export.c +++ b/block/export/export.c @@ -26,6 +26,9 @@ #ifdef CONFIG_VHOST_USER_BLK_SERVER #include "vhost-user-blk-server.h" #endif +#ifdef CONFIG_VDUSE_BLK_EXPORT +#include "vduse-blk.h" +#endif static const BlockExportDriver *blk_exp_drivers[] = { &blk_exp_nbd, @@ -35,6 +38,9 @@ static const BlockExportDriver *blk_exp_drivers[] = { #ifdef CONFIG_FUSE &blk_exp_fuse, #endif +#ifdef CONFIG_VDUSE_BLK_EXPORT + &blk_exp_vduse_blk, +#endif }; /* Only accessed from the main thread */ diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c new file mode 100644 index 0000000000..143d58a3f2 --- /dev/null +++ b/block/export/vduse-blk.c @@ -0,0 +1,307 @@ +/* + * Export QEMU block device via VDUSE + * + * Copyright (C) 2022 Bytedance Inc. and/or its affiliates. All rights reserved. + * + * Author: + * Xie Yongji + * + * This work is licensed under the terms of the GNU GPL, version 2 or + * later. See the COPYING file in the top-level directory. + */ + +#include + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "block/export.h" +#include "qemu/error-report.h" +#include "util/block-helpers.h" +#include "subprojects/libvduse/libvduse.h" +#include "virtio-blk-handler.h" + +#include "standard-headers/linux/virtio_blk.h" + +#define VDUSE_DEFAULT_NUM_QUEUE 1 +#define VDUSE_DEFAULT_QUEUE_SIZE 256 + +typedef struct VduseBlkExport { + BlockExport export; + VirtioBlkHandler handler; + VduseDev *dev; + uint16_t num_queues; +} VduseBlkExport; + +typedef struct VduseBlkReq { + VduseVirtqElement elem; + VduseVirtq *vq; +} VduseBlkReq; + +static void vduse_blk_req_complete(VduseBlkReq *req, size_t in_len) +{ + vduse_queue_push(req->vq, &req->elem, in_len); + vduse_queue_notify(req->vq); + + free(req); +} + +static void coroutine_fn vduse_blk_virtio_process_req(void *opaque) +{ + VduseBlkReq *req = opaque; + VduseVirtq *vq = req->vq; + VduseDev *dev = vduse_queue_get_dev(vq); + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev); + VirtioBlkHandler *handler = &vblk_exp->handler; + VduseVirtqElement *elem = &req->elem; + struct iovec *in_iov = elem->in_sg; + struct iovec *out_iov = elem->out_sg; + unsigned in_num = elem->in_num; + unsigned out_num = elem->out_num; + int in_len; + + in_len = virtio_blk_process_req(handler, in_iov, + out_iov, in_num, out_num); + if (in_len < 0) { + free(req); + return; + } + + vduse_blk_req_complete(req, in_len); +} + +static void vduse_blk_vq_handler(VduseDev *dev, VduseVirtq *vq) +{ + while (1) { + VduseBlkReq *req; + + req = vduse_queue_pop(vq, sizeof(VduseBlkReq)); + if (!req) { + break; + } + req->vq = vq; + + Coroutine *co = + qemu_coroutine_create(vduse_blk_virtio_process_req, req); + qemu_coroutine_enter(co); + } +} + +static void on_vduse_vq_kick(void *opaque) +{ + VduseVirtq *vq = opaque; + VduseDev *dev = vduse_queue_get_dev(vq); + int fd = vduse_queue_get_fd(vq); + eventfd_t kick_data; + + if (eventfd_read(fd, &kick_data) == -1) { + error_report("failed to read data from eventfd"); + return; + } + + vduse_blk_vq_handler(dev, vq); +} + +static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq) +{ + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev); + + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq), + true, on_vduse_vq_kick, NULL, NULL, NULL, vq); +} + +static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq) +{ + VduseBlkExport *vblk_exp = vduse_dev_get_priv(dev); + + aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq), + true, NULL, NULL, NULL, NULL, NULL); +} + +static const VduseOps vduse_blk_ops = { + .enable_queue = vduse_blk_enable_queue, + .disable_queue = vduse_blk_disable_queue, +}; + +static void on_vduse_dev_kick(void *opaque) +{ + VduseDev *dev = opaque; + + vduse_dev_handler(dev); +} + +static void vduse_blk_attach_ctx(VduseBlkExport *vblk_exp, AioContext *ctx) +{ + int i; + + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev), + true, on_vduse_dev_kick, NULL, NULL, NULL, + vblk_exp->dev); + + for (i = 0; i < vblk_exp->num_queues; i++) { + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i); + int fd = vduse_queue_get_fd(vq); + + if (fd < 0) { + continue; + } + aio_set_fd_handler(vblk_exp->export.ctx, fd, true, + on_vduse_vq_kick, NULL, NULL, NULL, vq); + } +} + +static void vduse_blk_detach_ctx(VduseBlkExport *vblk_exp) +{ + int i; + + for (i = 0; i < vblk_exp->num_queues; i++) { + VduseVirtq *vq = vduse_dev_get_queue(vblk_exp->dev, i); + int fd = vduse_queue_get_fd(vq); + + if (fd < 0) { + continue; + } + aio_set_fd_handler(vblk_exp->export.ctx, fd, + true, NULL, NULL, NULL, NULL, NULL); + } + aio_set_fd_handler(vblk_exp->export.ctx, vduse_dev_get_fd(vblk_exp->dev), + true, NULL, NULL, NULL, NULL, NULL); +} + + +static void blk_aio_attached(AioContext *ctx, void *opaque) +{ + VduseBlkExport *vblk_exp = opaque; + + vblk_exp->export.ctx = ctx; + vduse_blk_attach_ctx(vblk_exp, ctx); +} + +static void blk_aio_detach(void *opaque) +{ + VduseBlkExport *vblk_exp = opaque; + + vduse_blk_detach_ctx(vblk_exp); + vblk_exp->export.ctx = NULL; +} + +static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, + Error **errp) +{ + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export); + BlockExportOptionsVduseBlk *vblk_opts = &opts->u.vduse_blk; + uint64_t logical_block_size = VIRTIO_BLK_SECTOR_SIZE; + uint16_t num_queues = VDUSE_DEFAULT_NUM_QUEUE; + uint16_t queue_size = VDUSE_DEFAULT_QUEUE_SIZE; + Error *local_err = NULL; + struct virtio_blk_config config = { 0 }; + uint64_t features; + int i; + + if (vblk_opts->has_num_queues) { + num_queues = vblk_opts->num_queues; + if (num_queues == 0) { + error_setg(errp, "num-queues must be greater than 0"); + return -EINVAL; + } + } + + if (vblk_opts->has_queue_size) { + queue_size = vblk_opts->queue_size; + if (queue_size <= 2 || !is_power_of_2(queue_size) || + queue_size > VIRTQUEUE_MAX_SIZE) { + error_setg(errp, "queue-size is invalid"); + return -EINVAL; + } + } + + if (vblk_opts->has_logical_block_size) { + logical_block_size = vblk_opts->logical_block_size; + check_block_size(exp->id, "logical-block-size", logical_block_size, + &local_err); + if (local_err) { + error_propagate(errp, local_err); + return -EINVAL; + } + } + vblk_exp->num_queues = num_queues; + vblk_exp->handler.blk = exp->blk; + vblk_exp->handler.serial = exp->id; + vblk_exp->handler.logical_block_size = logical_block_size; + vblk_exp->handler.writable = opts->writable; + + config.capacity = + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS); + config.seg_max = cpu_to_le32(queue_size - 2); + config.min_io_size = cpu_to_le16(1); + config.opt_io_size = cpu_to_le32(1); + config.num_queues = cpu_to_le16(num_queues); + config.blk_size = cpu_to_le32(logical_block_size); + config.max_discard_sectors = cpu_to_le32(VIRTIO_BLK_MAX_DISCARD_SECTORS); + config.max_discard_seg = cpu_to_le32(1); + config.discard_sector_alignment = + cpu_to_le32(logical_block_size >> VIRTIO_BLK_SECTOR_BITS); + config.max_write_zeroes_sectors = + cpu_to_le32(VIRTIO_BLK_MAX_WRITE_ZEROES_SECTORS); + config.max_write_zeroes_seg = cpu_to_le32(1); + + features = vduse_get_virtio_features() | + (1ULL << VIRTIO_BLK_F_SEG_MAX) | + (1ULL << VIRTIO_BLK_F_TOPOLOGY) | + (1ULL << VIRTIO_BLK_F_BLK_SIZE) | + (1ULL << VIRTIO_BLK_F_FLUSH) | + (1ULL << VIRTIO_BLK_F_DISCARD) | + (1ULL << VIRTIO_BLK_F_WRITE_ZEROES); + + if (num_queues > 1) { + features |= 1ULL << VIRTIO_BLK_F_MQ; + } + if (!opts->writable) { + features |= 1ULL << VIRTIO_BLK_F_RO; + } + + vblk_exp->dev = vduse_dev_create(exp->id, VIRTIO_ID_BLOCK, 0, + features, num_queues, + sizeof(struct virtio_blk_config), + (char *)&config, &vduse_blk_ops, + vblk_exp); + if (!vblk_exp->dev) { + error_setg(errp, "failed to create vduse device"); + return -ENOMEM; + } + + for (i = 0; i < num_queues; i++) { + vduse_dev_setup_queue(vblk_exp->dev, i, queue_size); + } + + aio_set_fd_handler(exp->ctx, vduse_dev_get_fd(vblk_exp->dev), true, + on_vduse_dev_kick, NULL, NULL, NULL, vblk_exp->dev); + + blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach, + vblk_exp); + + return 0; +} + +static void vduse_blk_exp_delete(BlockExport *exp) +{ + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export); + + blk_remove_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach, + vblk_exp); + vduse_dev_destroy(vblk_exp->dev); +} + +static void vduse_blk_exp_request_shutdown(BlockExport *exp) +{ + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export); + + vduse_blk_detach_ctx(vblk_exp); +} + +const BlockExportDriver blk_exp_vduse_blk = { + .type = BLOCK_EXPORT_TYPE_VDUSE_BLK, + .instance_size = sizeof(VduseBlkExport), + .create = vduse_blk_exp_create, + .delete = vduse_blk_exp_delete, + .request_shutdown = vduse_blk_exp_request_shutdown, +}; diff --git a/MAINTAINERS b/MAINTAINERS index 880d61d327..096ded4b8a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3582,10 +3582,12 @@ L: qemu-block@nongnu.org S: Supported F: block/export/fuse.c -VDUSE library +VDUSE library and block device exports M: Xie Yongji S: Maintained F: subprojects/libvduse/ +F: block/export/vduse-blk.c +F: block/export/vduse-blk.h Replication M: Wen Congyang diff --git a/block/export/meson.build b/block/export/meson.build index 431e47ca51..c60116f455 100644 --- a/block/export/meson.build +++ b/block/export/meson.build @@ -5,3 +5,8 @@ if have_vhost_user_blk_server endif blockdev_ss.add(when: fuse, if_true: files('fuse.c')) + +if have_vduse_blk_export + blockdev_ss.add(files('vduse-blk.c', 'virtio-blk-handler.c')) + blockdev_ss.add(libvduse) +endif diff --git a/meson.build b/meson.build index 2b11b2c689..2bb5bef65d 100644 --- a/meson.build +++ b/meson.build @@ -1542,6 +1542,17 @@ if have_libvduse libvduse = libvduse_proj.get_variable('libvduse_dep') endif +have_vduse_blk_export = (have_libvduse and targetos == 'linux') +if get_option('vduse_blk_export').enabled() + if targetos != 'linux' + error('vduse_blk_export requires linux') + elif not have_libvduse + error('vduse_blk_export requires libvduse support') + endif +elif get_option('vduse_blk_export').disabled() + have_vduse_blk_export = false +endif + # libbpf libbpf = dependency('libbpf', required: get_option('bpf'), method: 'pkg-config') if libbpf.found() and not cc.links(''' @@ -1783,6 +1794,7 @@ config_host_data.set('CONFIG_VHOST_CRYPTO', have_vhost_user_crypto) config_host_data.set('CONFIG_VHOST_VDPA', have_vhost_vdpa) config_host_data.set('CONFIG_VMNET', vmnet.found()) config_host_data.set('CONFIG_VHOST_USER_BLK_SERVER', have_vhost_user_blk_server) +config_host_data.set('CONFIG_VDUSE_BLK_EXPORT', have_vduse_blk_export) config_host_data.set('CONFIG_PNG', png.found()) config_host_data.set('CONFIG_VNC', vnc.found()) config_host_data.set('CONFIG_VNC_JPEG', jpeg.found()) @@ -3822,6 +3834,7 @@ if have_block summary_info += {'qed support': get_option('qed').allowed()} summary_info += {'parallels support': get_option('parallels').allowed()} summary_info += {'FUSE exports': fuse} + summary_info += {'VDUSE block exports': have_vduse_blk_export} endif summary(summary_info, bool_yn: true, section: 'Block layer support') diff --git a/scripts/meson-buildoptions.sh b/scripts/meson-buildoptions.sh index 3fab6c5810..d18d449eb7 100644 --- a/scripts/meson-buildoptions.sh +++ b/scripts/meson-buildoptions.sh @@ -159,6 +159,8 @@ meson_options_help() { printf "%s\n" ' vhost-user vhost-user backend support' printf "%s\n" ' vhost-user-blk-server' printf "%s\n" ' build vhost-user-blk server' + printf "%s\n" ' vduse-blk-export' + printf "%s\n" ' VDUSE block export support' printf "%s\n" ' vhost-vdpa vhost-vdpa kernel backend support' printf "%s\n" ' virglrenderer virgl rendering support' printf "%s\n" ' virtfs virtio-9p support' @@ -425,6 +427,8 @@ _meson_option_parse() { --disable-vhost-user) printf "%s" -Dvhost_user=disabled ;; --enable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=enabled ;; --disable-vhost-user-blk-server) printf "%s" -Dvhost_user_blk_server=disabled ;; + --enable-vduse-blk-export) printf "%s" -Dvduse_blk_export=enabled ;; + --disable-vduse-blk-export) printf "%s" -Dvduse_blk_export=disabled ;; --enable-vhost-vdpa) printf "%s" -Dvhost_vdpa=enabled ;; --disable-vhost-vdpa) printf "%s" -Dvhost_vdpa=disabled ;; --enable-virglrenderer) printf "%s" -Dvirglrenderer=enabled ;; From patchwork Thu Jun 9 17:21:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641406 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=DagO1N5X; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJtNn4zmGz9s75 for ; Fri, 10 Jun 2022 04:46:49 +1000 (AEST) Received: from localhost ([::1]:39780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNB9-00032i-2U for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:46:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007rB-QJ for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:45791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003gY-St for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795345; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AVFXcIblIZ7wyt78pu+fozyB70XxhKCqNq97rujT4xs=; b=DagO1N5XL28XyetO2I3dXWAqDBGtiEfYyam6KTMvuIueTpQwp5GJPmti9CLyBwFOgrQgnw nzx18VYvLeXYuAgz1Ks92mau/AHy7UuGXeB8rllNgZlzh+ceLsbI1vo1UE+muvqbEUrkjZ yeBvKm8y2ES0BtjO184JW/K0xWjh8qE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-460-A-xKZGsnNjmZ_nH-UkSsjQ-1; Thu, 09 Jun 2022 13:22:23 -0400 X-MC-Unique: A-xKZGsnNjmZ_nH-UkSsjQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 45975802804; Thu, 9 Jun 2022 17:22:23 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id A555F4010E32; Thu, 9 Jun 2022 17:22:22 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 12/18] vduse-blk: Add vduse-blk resize support Date: Thu, 9 Jun 2022 19:21:43 +0200 Message-Id: <20220609172149.293877-13-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji To support block resize, this uses vduse_dev_update_config() to update the capacity field in configuration space and inject config interrupt on the block resize callback. Signed-off-by: Xie Yongji Reviewed-by: Stefan Hajnoczi Message-Id: <20220523084611.91-8-xieyongji@bytedance.com> Signed-off-by: Kevin Wolf --- block/export/vduse-blk.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c index 143d58a3f2..1040130f52 100644 --- a/block/export/vduse-blk.c +++ b/block/export/vduse-blk.c @@ -184,6 +184,23 @@ static void blk_aio_detach(void *opaque) vblk_exp->export.ctx = NULL; } +static void vduse_blk_resize(void *opaque) +{ + BlockExport *exp = opaque; + VduseBlkExport *vblk_exp = container_of(exp, VduseBlkExport, export); + struct virtio_blk_config config; + + config.capacity = + cpu_to_le64(blk_getlength(exp->blk) >> VIRTIO_BLK_SECTOR_BITS); + vduse_dev_update_config(vblk_exp->dev, sizeof(config.capacity), + offsetof(struct virtio_blk_config, capacity), + (char *)&config.capacity); +} + +static const BlockDevOps vduse_block_ops = { + .resize_cb = vduse_blk_resize, +}; + static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, Error **errp) { @@ -279,6 +296,8 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach, vblk_exp); + blk_set_dev_ops(exp->blk, &vduse_block_ops, exp); + return 0; } @@ -288,6 +307,7 @@ static void vduse_blk_exp_delete(BlockExport *exp) blk_remove_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach, vblk_exp); + blk_set_dev_ops(exp->blk, NULL, NULL); vduse_dev_destroy(vblk_exp->dev); } From patchwork Thu Jun 9 17:21:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641409 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=HAZyDvEw; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJtg12PPQz9s5V for ; Fri, 10 Jun 2022 04:59:09 +1000 (AEST) Received: from localhost ([::1]:57682 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNN5-0000KT-BL for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:59:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007so-Qx for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:53852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003gd-Of for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795345; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/CeaDfYrFqx+It4fXbX5GS/IN84tNpZQFMgryQabGV0=; b=HAZyDvEwKj8nzYn/sii94tjy6f04NPQ8k0Ernu5BwghkIPV+G1X3I9uilw4f98c/A/SeXc J2Ub2jmuyXNDEnXnU0XXpXai2u+c7I5Ee6kIZ0Q0rtqEZsxf+HgPDDCeeKutQd3MLutX8E 6+MqANweNy8aP4tObW7sHfA9djHIAXQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-443-f63c-ntFM-2HQOeHJ5RAEQ-1; Thu, 09 Jun 2022 13:22:24 -0400 X-MC-Unique: f63c-ntFM-2HQOeHJ5RAEQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1EE5A85A580; Thu, 9 Jun 2022 17:22:24 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7E812404E4BD; Thu, 9 Jun 2022 17:22:23 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 13/18] libvduse: Add support for reconnecting Date: Thu, 9 Jun 2022 19:21:44 +0200 Message-Id: <20220609172149.293877-14-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xie Yongji To support reconnecting after restart or crash, VDUSE backend might need to resubmit inflight I/Os. This stores the metadata such as the index of inflight I/O's descriptors to a shm file so that VDUSE backend can restore them during reconnecting. Signed-off-by: Xie Yongji Message-Id: <20220523084611.91-9-xieyongji@bytedance.com> Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- subprojects/libvduse/libvduse.h | 12 ++ block/export/vduse-blk.c | 14 ++ subprojects/libvduse/libvduse.c | 235 +++++++++++++++++++++++++++++++- 3 files changed, 256 insertions(+), 5 deletions(-) diff --git a/subprojects/libvduse/libvduse.h b/subprojects/libvduse/libvduse.h index 6c2fe98213..32f19e7b48 100644 --- a/subprojects/libvduse/libvduse.h +++ b/subprojects/libvduse/libvduse.h @@ -173,6 +173,18 @@ int vduse_dev_update_config(VduseDev *dev, uint32_t size, */ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size); +/** + * vduse_set_reconnect_log_file: + * @dev: VDUSE device + * @file: filename of reconnect log + * + * Specify the file to store log for reconnecting. It should + * be called before vduse_dev_setup_queue(). + * + * Returns: 0 on success, -errno on failure. + */ +int vduse_set_reconnect_log_file(VduseDev *dev, const char *filename); + /** * vduse_dev_create_by_fd: * @fd: passed file descriptor diff --git a/block/export/vduse-blk.c b/block/export/vduse-blk.c index 1040130f52..3b10349173 100644 --- a/block/export/vduse-blk.c +++ b/block/export/vduse-blk.c @@ -30,6 +30,7 @@ typedef struct VduseBlkExport { VirtioBlkHandler handler; VduseDev *dev; uint16_t num_queues; + char *recon_file; } VduseBlkExport; typedef struct VduseBlkReq { @@ -107,6 +108,8 @@ static void vduse_blk_enable_queue(VduseDev *dev, VduseVirtq *vq) aio_set_fd_handler(vblk_exp->export.ctx, vduse_queue_get_fd(vq), true, on_vduse_vq_kick, NULL, NULL, NULL, vq); + /* Make sure we don't miss any kick afer reconnecting */ + eventfd_write(vduse_queue_get_fd(vq), 1); } static void vduse_blk_disable_queue(VduseDev *dev, VduseVirtq *vq) @@ -286,6 +289,15 @@ static int vduse_blk_exp_create(BlockExport *exp, BlockExportOptions *opts, return -ENOMEM; } + vblk_exp->recon_file = g_strdup_printf("%s/vduse-blk-%s", + g_get_tmp_dir(), exp->id); + if (vduse_set_reconnect_log_file(vblk_exp->dev, vblk_exp->recon_file)) { + error_setg(errp, "failed to set reconnect log file"); + vduse_dev_destroy(vblk_exp->dev); + g_free(vblk_exp->recon_file); + return -EINVAL; + } + for (i = 0; i < num_queues; i++) { vduse_dev_setup_queue(vblk_exp->dev, i, queue_size); } @@ -309,6 +321,8 @@ static void vduse_blk_exp_delete(BlockExport *exp) vblk_exp); blk_set_dev_ops(exp->blk, NULL, NULL); vduse_dev_destroy(vblk_exp->dev); + unlink(vblk_exp->recon_file); + g_free(vblk_exp->recon_file); } static void vduse_blk_exp_request_shutdown(BlockExport *exp) diff --git a/subprojects/libvduse/libvduse.c b/subprojects/libvduse/libvduse.c index fa4822b9a9..78bb777402 100644 --- a/subprojects/libvduse/libvduse.c +++ b/subprojects/libvduse/libvduse.c @@ -41,6 +41,8 @@ #define VDUSE_VQ_ALIGN 4096 #define MAX_IOVA_REGIONS 256 +#define LOG_ALIGNMENT 64 + /* Round number down to multiple */ #define ALIGN_DOWN(n, m) ((n) / (m) * (m)) @@ -51,6 +53,31 @@ #define unlikely(x) __builtin_expect(!!(x), 0) #endif +typedef struct VduseDescStateSplit { + uint8_t inflight; + uint8_t padding[5]; + uint16_t next; + uint64_t counter; +} VduseDescStateSplit; + +typedef struct VduseVirtqLogInflight { + uint64_t features; + uint16_t version; + uint16_t desc_num; + uint16_t last_batch_head; + uint16_t used_idx; + VduseDescStateSplit desc[]; +} VduseVirtqLogInflight; + +typedef struct VduseVirtqLog { + VduseVirtqLogInflight inflight; +} VduseVirtqLog; + +typedef struct VduseVirtqInflightDesc { + uint16_t index; + uint64_t counter; +} VduseVirtqInflightDesc; + typedef struct VduseRing { unsigned int num; uint64_t desc_addr; @@ -73,6 +100,10 @@ struct VduseVirtq { bool ready; int fd; VduseDev *dev; + VduseVirtqInflightDesc *resubmit_list; + uint16_t resubmit_num; + uint64_t counter; + VduseVirtqLog *log; }; typedef struct VduseIovaRegion { @@ -96,8 +127,36 @@ struct VduseDev { int fd; int ctrl_fd; void *priv; + void *log; }; +static inline size_t vduse_vq_log_size(uint16_t queue_size) +{ + return ALIGN_UP(sizeof(VduseDescStateSplit) * queue_size + + sizeof(VduseVirtqLogInflight), LOG_ALIGNMENT); +} + +static void *vduse_log_get(const char *filename, size_t size) +{ + void *ptr = MAP_FAILED; + int fd; + + fd = open(filename, O_RDWR | O_CREAT, 0600); + if (fd == -1) { + return MAP_FAILED; + } + + if (ftruncate(fd, size) == -1) { + goto out; + } + + ptr = mmap(0, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + +out: + close(fd); + return ptr; +} + static inline bool has_feature(uint64_t features, unsigned int fbit) { assert(fbit < 64); @@ -148,6 +207,105 @@ static int vduse_inject_irq(VduseDev *dev, int index) return ioctl(dev->fd, VDUSE_VQ_INJECT_IRQ, &index); } +static int inflight_desc_compare(const void *a, const void *b) +{ + VduseVirtqInflightDesc *desc0 = (VduseVirtqInflightDesc *)a, + *desc1 = (VduseVirtqInflightDesc *)b; + + if (desc1->counter > desc0->counter && + (desc1->counter - desc0->counter) < VIRTQUEUE_MAX_SIZE * 2) { + return 1; + } + + return -1; +} + +static int vduse_queue_check_inflights(VduseVirtq *vq) +{ + int i = 0; + VduseDev *dev = vq->dev; + + vq->used_idx = le16toh(vq->vring.used->idx); + vq->resubmit_num = 0; + vq->resubmit_list = NULL; + vq->counter = 0; + + if (unlikely(vq->log->inflight.used_idx != vq->used_idx)) { + if (vq->log->inflight.last_batch_head > VIRTQUEUE_MAX_SIZE) { + return -1; + } + + vq->log->inflight.desc[vq->log->inflight.last_batch_head].inflight = 0; + + barrier(); + + vq->log->inflight.used_idx = vq->used_idx; + } + + for (i = 0; i < vq->log->inflight.desc_num; i++) { + if (vq->log->inflight.desc[i].inflight == 1) { + vq->inuse++; + } + } + + vq->shadow_avail_idx = vq->last_avail_idx = vq->inuse + vq->used_idx; + + if (vq->inuse) { + vq->resubmit_list = calloc(vq->inuse, sizeof(VduseVirtqInflightDesc)); + if (!vq->resubmit_list) { + return -1; + } + + for (i = 0; i < vq->log->inflight.desc_num; i++) { + if (vq->log->inflight.desc[i].inflight) { + vq->resubmit_list[vq->resubmit_num].index = i; + vq->resubmit_list[vq->resubmit_num].counter = + vq->log->inflight.desc[i].counter; + vq->resubmit_num++; + } + } + + if (vq->resubmit_num > 1) { + qsort(vq->resubmit_list, vq->resubmit_num, + sizeof(VduseVirtqInflightDesc), inflight_desc_compare); + } + vq->counter = vq->resubmit_list[0].counter + 1; + } + + vduse_inject_irq(dev, vq->index); + + return 0; +} + +static int vduse_queue_inflight_get(VduseVirtq *vq, int desc_idx) +{ + vq->log->inflight.desc[desc_idx].counter = vq->counter++; + + barrier(); + + vq->log->inflight.desc[desc_idx].inflight = 1; + + return 0; +} + +static int vduse_queue_inflight_pre_put(VduseVirtq *vq, int desc_idx) +{ + vq->log->inflight.last_batch_head = desc_idx; + + return 0; +} + +static int vduse_queue_inflight_post_put(VduseVirtq *vq, int desc_idx) +{ + vq->log->inflight.desc[desc_idx].inflight = 0; + + barrier(); + + vq->log->inflight.used_idx = vq->used_idx; + + return 0; +} + static void vduse_iova_remove_region(VduseDev *dev, uint64_t start, uint64_t last) { @@ -596,11 +754,24 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz) unsigned int head; VduseVirtqElement *elem; VduseDev *dev = vq->dev; + int i; if (unlikely(!vq->vring.avail)) { return NULL; } + if (unlikely(vq->resubmit_list && vq->resubmit_num > 0)) { + i = (--vq->resubmit_num); + elem = vduse_queue_map_desc(vq, vq->resubmit_list[i].index, sz); + + if (!vq->resubmit_num) { + free(vq->resubmit_list); + vq->resubmit_list = NULL; + } + + return elem; + } + if (vduse_queue_empty(vq)) { return NULL; } @@ -628,6 +799,8 @@ void *vduse_queue_pop(VduseVirtq *vq, size_t sz) vq->inuse++; + vduse_queue_inflight_get(vq, head); + return elem; } @@ -685,7 +858,9 @@ void vduse_queue_push(VduseVirtq *vq, const VduseVirtqElement *elem, unsigned int len) { vduse_queue_fill(vq, elem, len, 0); + vduse_queue_inflight_pre_put(vq, elem->index); vduse_queue_flush(vq, 1); + vduse_queue_inflight_post_put(vq, elem->index); } static int vduse_queue_update_vring(VduseVirtq *vq, uint64_t desc_addr, @@ -764,12 +939,15 @@ static void vduse_queue_enable(VduseVirtq *vq) } vq->fd = fd; - vq->shadow_avail_idx = vq->last_avail_idx = vq_info.split.avail_index; - vq->inuse = 0; - vq->used_idx = 0; vq->signalled_used_valid = false; vq->ready = true; + if (vduse_queue_check_inflights(vq)) { + fprintf(stderr, "Failed to check inflights for vq[%d]\n", vq->index); + close(fd); + return; + } + dev->ops->enable_queue(dev, vq); } @@ -819,11 +997,15 @@ static void vduse_dev_start_dataplane(VduseDev *dev) static void vduse_dev_stop_dataplane(VduseDev *dev) { + size_t log_size = dev->num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE); int i; for (i = 0; i < dev->num_queues; i++) { vduse_queue_disable(&dev->vqs[i]); } + if (dev->log) { + memset(dev->log, 0, log_size); + } dev->features = 0; vduse_iova_remove_region(dev, 0, ULONG_MAX); } @@ -932,6 +1114,30 @@ int vduse_dev_setup_queue(VduseDev *dev, int index, int max_size) return -errno; } + vduse_queue_enable(vq); + + return 0; +} + +int vduse_set_reconnect_log_file(VduseDev *dev, const char *filename) +{ + + size_t log_size = dev->num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE); + void *log; + int i; + + dev->log = log = vduse_log_get(filename, log_size); + if (log == MAP_FAILED) { + fprintf(stderr, "Failed to get vduse log\n"); + return -EINVAL; + } + + for (i = 0; i < dev->num_queues; i++) { + dev->vqs[i].log = log; + dev->vqs[i].log->inflight.desc_num = VIRTQUEUE_MAX_SIZE; + log = (void *)((char *)log + vduse_vq_log_size(VIRTQUEUE_MAX_SIZE)); + } + return 0; } @@ -976,6 +1182,12 @@ static int vduse_dev_init(VduseDev *dev, const char *name, return -errno; } + if (ioctl(fd, VDUSE_DEV_GET_FEATURES, &dev->features)) { + fprintf(stderr, "Failed to get features: %s\n", strerror(errno)); + close(fd); + return -errno; + } + dev_name = strdup(name); if (!dev_name) { close(fd); @@ -1020,6 +1232,12 @@ VduseDev *vduse_dev_create_by_fd(int fd, uint16_t num_queues, return NULL; } + if (ioctl(fd, VDUSE_DEV_GET_FEATURES, &dev->features)) { + fprintf(stderr, "Failed to get features: %s\n", strerror(errno)); + free(dev); + return NULL; + } + ret = vduse_dev_init_vqs(dev, num_queues); if (ret) { fprintf(stderr, "Failed to init vqs\n"); @@ -1119,7 +1337,7 @@ VduseDev *vduse_dev_create(const char *name, uint32_t device_id, ret = ioctl(ctrl_fd, VDUSE_CREATE_DEV, dev_config); free(dev_config); - if (ret < 0) { + if (ret && errno != EEXIST) { fprintf(stderr, "Failed to create vduse device %s: %s\n", name, strerror(errno)); goto err_dev; @@ -1146,8 +1364,15 @@ err_ctrl: int vduse_dev_destroy(VduseDev *dev) { - int ret = 0; + size_t log_size = dev->num_queues * vduse_vq_log_size(VIRTQUEUE_MAX_SIZE); + int i, ret = 0; + if (dev->log) { + munmap(dev->log, log_size); + } + for (i = 0; i < dev->num_queues; i++) { + free(dev->vqs[i].resubmit_list); + } free(dev->vqs); if (dev->fd > 0) { close(dev->fd); From patchwork Thu Jun 9 17:21:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641389 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=XtzxBJVm; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsCs29dsz9s1l for ; Fri, 10 Jun 2022 03:54:01 +1000 (AEST) Received: from localhost ([::1]:49324 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMM3-00084Z-6t for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 13:53:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007sy-Qj for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38064) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003hE-T4 for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795348; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RmSk+tUEqXiBalaaOmEjXLxaJWQNJUFlikSGSiwR69g=; b=XtzxBJVmX5hpU8Mucz/z5FRVgWf4iDIMhqu5Gdzu/0Hx/AFyPMg/8By2Wsc+V0iXvqaRih JbdeS0iuPXaqrbi+npmyo0DaNOzk5e1+nOWr1Pz8IT9Gnk7m/XgqbW3cZrZOT+mEI1uuDu bhz/7VrPQ85JNWsorqGkzNjut01sNMA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-230-GVsLWVGBPuu0z39dViz82g-1; Thu, 09 Jun 2022 13:22:25 -0400 X-MC-Unique: GVsLWVGBPuu0z39dViz82g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F01BB80159B; Thu, 9 Jun 2022 17:22:24 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 58210404E4B5; Thu, 9 Jun 2022 17:22:24 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 14/18] qsd: document vduse-blk exports Date: Thu, 9 Jun 2022 19:21:45 +0200 Message-Id: <20220609172149.293877-15-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Stefan Hajnoczi Document vduse-blk exports in qemu-storage-daemon --help and the qemu-storage-daemon(1) man page. Based-on: <20220523084611.91-1-xieyongji@bytedance.com> Cc: Xie Yongji Signed-off-by: Stefan Hajnoczi Message-Id: <20220525121947.859820-1-stefanha@redhat.com> Signed-off-by: Kevin Wolf --- docs/tools/qemu-storage-daemon.rst | 21 +++++++++++++++++++++ storage-daemon/qemu-storage-daemon.c | 9 +++++++++ 2 files changed, 30 insertions(+) diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index 8b97592663..fbeaf76954 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -77,6 +77,7 @@ Standard options: --export [type=]vhost-user-blk,id=,node-name=,addr.type=unix,addr.path=[,writable=on|off][,logical-block-size=][,num-queues=] --export [type=]vhost-user-blk,id=,node-name=,addr.type=fd,addr.str=[,writable=on|off][,logical-block-size=][,num-queues=] --export [type=]fuse,id=,node-name=,mountpoint=[,growable=on|off][,writable=on|off][,allow-other=on|off|auto] + --export [type=]vduse-blk,id=,node-name=[,writable=on|off][,num-queues=][,queue-size=][,logical-block-size=] is a block export definition. ``node-name`` is the block node that should be exported. ``writable`` determines whether or not the export allows write @@ -110,6 +111,26 @@ Standard options: ``allow-other`` to auto (the default) will try enabling this option, and on error fall back to disabling it. + The ``vduse-blk`` export type uses the ``id`` as the VDUSE device name. + ``num-queues`` sets the number of virtqueues (the default is 1). + ``queue-size`` sets the virtqueue descriptor table size (the default is 256). + + The instantiated VDUSE device must then be added to the vDPA bus using the + vdpa(8) command from the iproute2 project:: + + # vdpa dev add name mgmtdev vduse + + The device can be removed from the vDPA bus later as follows:: + + # vdpa dev del + + For more information about attaching vDPA devices to the host with + virtio_vdpa.ko or attaching them to guests with vhost_vdpa.ko, see + https://vdpa-dev.gitlab.io/. + + For more information about VDUSE, see + https://docs.kernel.org/userspace-api/vduse.html. + .. option:: --monitor MONITORDEF is a QMP monitor definition. See the :manpage:`qemu(1)` manual page for diff --git a/storage-daemon/qemu-storage-daemon.c b/storage-daemon/qemu-storage-daemon.c index c104817cdd..17fd3f2f5f 100644 --- a/storage-daemon/qemu-storage-daemon.c +++ b/storage-daemon/qemu-storage-daemon.c @@ -121,6 +121,15 @@ static void help(void) " vhost-user-blk device over file descriptor\n" "\n" #endif /* CONFIG_VHOST_USER_BLK_SERVER */ +#ifdef CONFIG_VDUSE_BLK_EXPORT +" --export [type=]vduse-blk,id=,node-name=\n" +" [,writable=on|off][,num-queues=]\n" +" [,queue-size=]\n" +" [,logical-block-size=]\n" +" export the specified block node as a vduse-blk\n" +" device using the id as the VDUSE device name\n" +"\n" +#endif /* CONFIG_VDUSE_BLK_EXPORT */ " --monitor [chardev=]name[,mode=control][,pretty[=on|off]]\n" " configure a QMP monitor\n" "\n" From patchwork Thu Jun 9 17:21:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641391 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L82svzi9; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsPv2rlpz9s2R for ; Fri, 10 Jun 2022 04:02:43 +1000 (AEST) Received: from localhost ([::1]:57864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMUS-00060q-Qf for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:02:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007rd-QV for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:41163) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003hM-Qq for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Of0K4aoxBrEtOf7Oa78quO5qB1obymCESGSL5fDz2+0=; b=L82svzi93tsVNd21i4xEHD3h/A+CRu4eVZ9RR30Jd2yFxt5FX8joLyzVmJj/Vyp6rtvSot tcaLKKNXS20UHaxFe6+TzCrTtR0MbWGC0c/lqPkh7tNio4UbxQKrwKtLqDIso+HqhqBux2 O1XRVvFJKLYCjihFiTQn1WnO7lfrMTs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-523-e0b5bAoUOKqEcfZsN5YuWQ-1; Thu, 09 Jun 2022 13:22:26 -0400 X-MC-Unique: e0b5bAoUOKqEcfZsN5YuWQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CB33838173CD; Thu, 9 Jun 2022 17:22:25 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3649E4010E32; Thu, 9 Jun 2022 17:22:25 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 15/18] block/rbd: report a better error when namespace does not exist Date: Thu, 9 Jun 2022 19:21:46 +0200 Message-Id: <20220609172149.293877-16-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Stefano Garzarella If the namespace does not exist, rbd_create() fails with -ENOENT and QEMU reports a generic "error rbd create: No such file or directory": $ qemu-img create rbd:rbd/namespace/image 1M Formatting 'rbd:rbd/namespace/image', fmt=raw size=1048576 qemu-img: rbd:rbd/namespace/image: error rbd create: No such file or directory Unfortunately rados_ioctx_set_namespace() does not fail if the namespace does not exist, so let's use rbd_namespace_exists() in qemu_rbd_connect() to check if the namespace exists, reporting a more understandable error: $ qemu-img create rbd:rbd/namespace/image 1M Formatting 'rbd:rbd/namespace/image', fmt=raw size=1048576 qemu-img: rbd:rbd/namespace/image: namespace 'namespace' does not exist Reported-by: Tingting Mao Reviewed-by: Ilya Dryomov Signed-off-by: Stefano Garzarella Message-Id: <20220517071012.6120-1-sgarzare@redhat.com> Signed-off-by: Kevin Wolf --- block/rbd.c | 24 ++++++++++++++++++++++++ meson.build | 6 ++++++ 2 files changed, 30 insertions(+) diff --git a/block/rbd.c b/block/rbd.c index 6caf35cbba..f826410f40 100644 --- a/block/rbd.c +++ b/block/rbd.c @@ -831,6 +831,26 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, error_setg_errno(errp, -r, "error opening pool %s", opts->pool); goto failed_shutdown; } + +#ifdef HAVE_RBD_NAMESPACE_EXISTS + if (opts->has_q_namespace && strlen(opts->q_namespace) > 0) { + bool exists; + + r = rbd_namespace_exists(*io_ctx, opts->q_namespace, &exists); + if (r < 0) { + error_setg_errno(errp, -r, "error checking namespace"); + goto failed_ioctx_destroy; + } + + if (!exists) { + error_setg(errp, "namespace '%s' does not exist", + opts->q_namespace); + r = -ENOENT; + goto failed_ioctx_destroy; + } + } +#endif + /* * Set the namespace after opening the io context on the pool, * if nspace == NULL or if nspace == "", it is just as we did nothing @@ -840,6 +860,10 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, r = 0; goto out; +#ifdef HAVE_RBD_NAMESPACE_EXISTS +failed_ioctx_destroy: + rados_ioctx_destroy(*io_ctx); +#endif failed_shutdown: rados_shutdown(*cluster); out: diff --git a/meson.build b/meson.build index 2bb5bef65d..d76e16f87d 100644 --- a/meson.build +++ b/meson.build @@ -1894,6 +1894,12 @@ config_host_data.set('HAVE_GETIFADDRS', cc.has_function('getifaddrs')) config_host_data.set('HAVE_OPENPTY', cc.has_function('openpty', dependencies: util)) config_host_data.set('HAVE_STRCHRNUL', cc.has_function('strchrnul')) config_host_data.set('HAVE_SYSTEM_FUNCTION', cc.has_function('system', prefix: '#include ')) +if rbd.found() + config_host_data.set('HAVE_RBD_NAMESPACE_EXISTS', + cc.has_function('rbd_namespace_exists', + dependencies: rbd, + prefix: '#include ')) +endif if rdma.found() config_host_data.set('HAVE_IBV_ADVISE_MR', cc.has_function('ibv_advise_mr', From patchwork Thu Jun 9 17:21:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641410 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ub193l/4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJtn661wTz9s5V for ; Fri, 10 Jun 2022 05:04:25 +1000 (AEST) Received: from localhost ([::1]:38152 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzNS9-0007g6-W9 for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 15:04:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56378) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrq-0007zf-4F for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:51007) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrn-0003hX-7z for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sd8VhQGWEpTDiSLevFE7jFgiFqnrOHTPUJ08CRqL42Q=; b=Ub193l/4NaA/9m/8ACHKiOK01Z34ZxWNIkg1kil4CARQULGAljokScy4DPz3FyGeIhD9HO +bG8HP2slwYkdAXFYCVdR9Qgieyj4k5TsUOu7XJE423EO22kipjl51+tHqAPpZWDeNN9QL UYnsQKG1mCE/fJadnk9z9Xa21tOqQhw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-352-Bs3B0cnBPb6X5aP38UjpEA-1; Thu, 09 Jun 2022 13:22:27 -0400 X-MC-Unique: Bs3B0cnBPb6X5aP38UjpEA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A203E3C02181; Thu, 9 Jun 2022 17:22:26 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1134B40D2827; Thu, 9 Jun 2022 17:22:25 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 16/18] block/gluster: correctly set max_pdiscard Date: Thu, 9 Jun 2022 19:21:47 +0200 Message-Id: <20220609172149.293877-17-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Fabian Ebner On 64-bit platforms, assigning SIZE_MAX to the int64_t max_pdiscard results in a negative value, and the following assertion would trigger down the line (it's not the same max_pdiscard, but computed from the other one): qemu-system-x86_64: ../block/io.c:3166: bdrv_co_pdiscard: Assertion `max_pdiscard >= bs->bl.request_alignment' failed. On 32-bit platforms, it's fine to keep using SIZE_MAX. The assertion in qemu_gluster_co_pdiscard() is checking that the value of 'bytes' can safely be passed to glfs_discard_async(), which takes a size_t for the argument in question, so it is kept as is. And since max_pdiscard is still <= SIZE_MAX, relying on max_pdiscard is still fine. Fixes: 0c8022876f ("block: use int64_t instead of int in driver discard handlers") Cc: qemu-stable@nongnu.org Signed-off-by: Fabian Ebner Message-Id: <20220520075922.43972-1-f.ebner@proxmox.com> Reviewed-by: Eric Blake Reviewed-by: Stefano Garzarella Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- block/gluster.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/gluster.c b/block/gluster.c index 398976bc66..b60213ab80 100644 --- a/block/gluster.c +++ b/block/gluster.c @@ -891,7 +891,7 @@ out: static void qemu_gluster_refresh_limits(BlockDriverState *bs, Error **errp) { bs->bl.max_transfer = GLUSTER_MAX_TRANSFER; - bs->bl.max_pdiscard = SIZE_MAX; + bs->bl.max_pdiscard = MIN(SIZE_MAX, INT64_MAX); } static int qemu_gluster_reopen_prepare(BDRVReopenState *state, From patchwork Thu Jun 9 17:21:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641402 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BgO6unLZ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsvb49kVz9s2R for ; Fri, 10 Jun 2022 04:24:59 +1000 (AEST) Received: from localhost ([::1]:38822 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMq1-0007LW-KD for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:24:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56304) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrp-0007sv-7H for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:34401) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003hc-UF for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795350; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lFIvPH+fdjrHK1JSwlSp/ERGBTgIqaOOG2KfwhXX78A=; b=BgO6unLZDe3Xx1F7p9NFTFsEt4CJ7Hu7J10xUXDVmduh2me7jAcgHoxYjM/L3Pb+vlY4Se zOZOmOHoQuxJ+R5kaSiALW68tJZfezlCZnOtuvcVp+yxm8lz/PMQkXEuRa0AQKXf7bnWgO eOERBReBZOAybST2F8b/k2gH44bIcXo= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-372-QQGjBQFTOPSaJVWaU4WFlQ-1; Thu, 09 Jun 2022 13:22:27 -0400 X-MC-Unique: QQGjBQFTOPSaJVWaU4WFlQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 78FAC294EDDB; Thu, 9 Jun 2022 17:22:27 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id D981F40D2827; Thu, 9 Jun 2022 17:22:26 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 17/18] aio_wait_kick: add missing memory barrier Date: Thu, 9 Jun 2022 19:21:48 +0200 Message-Id: <20220609172149.293877-18-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Emanuele Giuseppe Esposito It seems that aio_wait_kick always required a memory barrier or atomic operation in the caller, but nobody actually took care of doing it. Let's put the barrier in the function instead, and pair it with another one in AIO_WAIT_WHILE. Read aio_wait_kick() comment for further explanation. Suggested-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito Message-Id: <20220524173054.12651-1-eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi Signed-off-by: Kevin Wolf --- include/block/aio-wait.h | 2 ++ util/aio-wait.c | 16 +++++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h index b39eefb38d..54840f8622 100644 --- a/include/block/aio-wait.h +++ b/include/block/aio-wait.h @@ -81,6 +81,8 @@ extern AioWait global_aio_wait; AioContext *ctx_ = (ctx); \ /* Increment wait_->num_waiters before evaluating cond. */ \ qatomic_inc(&wait_->num_waiters); \ + /* Paired with smp_mb in aio_wait_kick(). */ \ + smp_mb(); \ if (ctx_ && in_aio_context_home_thread(ctx_)) { \ while ((cond)) { \ aio_poll(ctx_, true); \ diff --git a/util/aio-wait.c b/util/aio-wait.c index bdb3d3af22..98c5accd29 100644 --- a/util/aio-wait.c +++ b/util/aio-wait.c @@ -35,7 +35,21 @@ static void dummy_bh_cb(void *opaque) void aio_wait_kick(void) { - /* The barrier (or an atomic op) is in the caller. */ + /* + * Paired with smp_mb in AIO_WAIT_WHILE. Here we have: + * write(condition); + * aio_wait_kick() { + * smp_mb(); + * read(num_waiters); + * } + * + * And in AIO_WAIT_WHILE: + * write(num_waiters); + * smp_mb(); + * read(condition); + */ + smp_mb(); + if (qatomic_read(&global_aio_wait.num_waiters)) { aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL); } From patchwork Thu Jun 9 17:21:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Wolf X-Patchwork-Id: 1641395 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Ko+AWFbE; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LJsdt5Md9z9s2R for ; Fri, 10 Jun 2022 04:13:06 +1000 (AEST) Received: from localhost ([::1]:47208 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzMeW-0001vZ-8V for incoming@patchwork.ozlabs.org; Thu, 09 Jun 2022 14:13:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56314) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLro-0007t1-QW for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:25196) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzLrd-0003hi-Or for qemu-devel@nongnu.org; Thu, 09 Jun 2022 13:22:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654795351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BU9K3IOvrk1T3eT2STc7Mv7/W0/S/Y+dlLuFQAJhhnQ=; b=Ko+AWFbEgWht4bNR4l+lF2Cakyo4jmuiXNkXJHbPWTQ6MmC0G6S3ZmHQ3uNeoGMeDqI1Zp cD7fsQnEZ9dCnMVFC1ioHBsoFBhnGN9kmlqF5+zktW3N4uDvXTVhg2E4uQEBQGoUHOYuy0 hmv8i3fHT9fLKlSkHCqXXsWLPq8IxG4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-391-u4rQuK4oM6irz-JrIKy7tg-1; Thu, 09 Jun 2022 13:22:28 -0400 X-MC-Unique: u4rQuK4oM6irz-JrIKy7tg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4C89D38173C7; Thu, 9 Jun 2022 17:22:28 +0000 (UTC) Received: from merkur.redhat.com (unknown [10.39.195.10]) by smtp.corp.redhat.com (Postfix) with ESMTP id AFFF2404E4B5; Thu, 9 Jun 2022 17:22:27 +0000 (UTC) From: Kevin Wolf To: qemu-block@nongnu.org Cc: kwolf@redhat.com, qemu-devel@nongnu.org Subject: [PULL 18/18] nbd: Drop dead code spotted by Coverity Date: Thu, 9 Jun 2022 19:21:49 +0200 Message-Id: <20220609172149.293877-19-kwolf@redhat.com> In-Reply-To: <20220609172149.293877-1-kwolf@redhat.com> References: <20220609172149.293877-1-kwolf@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=kwolf@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Eric Blake CID 1488362 points out that the second 'rc >= 0' check is now dead code. Reported-by: Peter Maydell Fixes: 172f5f1a40(nbd: remove peppering of nbd_client_connected) Signed-off-by: Eric Blake Message-Id: <20220516210519.76135-1-eblake@redhat.com> Reviewed-by: Peter Maydell Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- block/nbd.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/block/nbd.c b/block/nbd.c index 6085ab1d2c..7f5f50ec46 100644 --- a/block/nbd.c +++ b/block/nbd.c @@ -521,12 +521,8 @@ static int coroutine_fn nbd_co_send_request(BlockDriverState *bs, if (qiov) { qio_channel_set_cork(s->ioc, true); rc = nbd_send_request(s->ioc, request); - if (rc >= 0) { - if (qio_channel_writev_all(s->ioc, qiov->iov, qiov->niov, - NULL) < 0) { - rc = -EIO; - } - } else if (rc >= 0) { + if (rc >= 0 && qio_channel_writev_all(s->ioc, qiov->iov, qiov->niov, + NULL) < 0) { rc = -EIO; } qio_channel_set_cork(s->ioc, false);