From patchwork Fri Apr 1 11:06:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Qiu X-Patchwork-Id: 1612165 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KVHXr6m9pz9sG5 for ; Fri, 1 Apr 2022 22:11:07 +1100 (AEDT) Received: from localhost ([::1]:42830 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1naFBI-0007xA-Cc for incoming@patchwork.ozlabs.org; Fri, 01 Apr 2022 07:11:04 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFAr-0007wj-8b for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:10:37 -0400 Received: from smtpbg139.qq.com ([175.27.65.136]:26771 helo=smtpbg.qq.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFAm-00007n-K6 for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:10:36 -0400 X-QQ-mid: bizesmtp66t1648811186ta0ihv8q Received: from localhost.localdomain.localdoma ( [116.228.45.98]) by bizesmtp.qq.com (ESMTP) with id ; Fri, 01 Apr 2022 19:06:22 +0800 (CST) X-QQ-SSF: 01000000002000803000C00A0000000 X-QQ-FEAT: 3uawQE1sH+0knKo9XmHNVOZ0whvo5fUQuYRAfzvHZCXSqg9nG8Rwk3lWffKee p3KQ1cRjEfIPeUx0ptPNXXpORWljKdlabdSjosa18sp8kEpsqLJFs95mQGms2F70pWjPmq0 k4YytYEJ9Potptpj7yNz8j0i0Q/M6X5jk1TrvHBJnpNOmRQbYrtTtqQ1jzSm7UbwKBGH10k WkUaO5i6cY4jP6eYmwJjfb8UCriw3S/5kE7BdCaT303+7aH1/goBJqhZt+YryL6HboPOHfT gAU9yXAnJOrQ3VwC1SVj04Bb1nIePcaaWrfNgL8AKqAVzLXQ58Bp70vI5qs3mN7F7vDXLRy jaYcsHabBwwkOZEldxQJYDvYy0MdQ== X-QQ-GoodBg: 0 From: Michael Qiu To: jasowang@redhat.com, mst@redhat.com, si-wei.liu@oracle.com Subject: [PATCH 1/3] vhost: Refactor vhost_reset_device() in VhostOps Date: Fri, 1 Apr 2022 07:06:11 -0400 Message-Id: <1648811173-15293-2-git-send-email-qiudayu@archeros.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648811173-15293-1-git-send-email-qiudayu@archeros.com> References: <1648776683-23739-1-git-send-email-qiudayu@archeros.com> <1648811173-15293-1-git-send-email-qiudayu@archeros.com> X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:archeros.com:qybgspam:qybgspam6 Received-SPF: none client-ip=175.27.65.136; envelope-from=qiudayu@archeros.com; helo=smtpbg.qq.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, KHOP_HELO_FCRDNS=0.187, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Qiu , eperezma@redhat.com, lingshan.zhu@intel.com, qemu-devel@nongnu.org, lulu@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Currently in vhost framwork, vhost_reset_device() is misnamed. Actually, it should be vhost_reset_owner(). In vhost user, it make compatible with reset device ops, but vhost kernel does not compatible with it, for vhost vdpa, it only implement reset device action. So we need seperate the function into vhost_reset_owner() and vhost_reset_device(). So that different backend could use the correct function. Signde-off-by: Michael Qiu --- hw/scsi/vhost-user-scsi.c | 6 +++++- hw/virtio/vhost-backend.c | 4 ++-- hw/virtio/vhost-user.c | 22 ++++++++++++++++++---- include/hw/virtio/vhost-backend.h | 2 ++ 4 files changed, 27 insertions(+), 7 deletions(-) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 1b2f7ee..f179626 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -80,8 +80,12 @@ static void vhost_user_scsi_reset(VirtIODevice *vdev) return; } - if (dev->vhost_ops->vhost_reset_device) { + if (virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_RESET_DEVICE) && + dev->vhost_ops->vhost_reset_device) { dev->vhost_ops->vhost_reset_device(dev); + } else if (dev->vhost_ops->vhost_reset_owner) { + dev->vhost_ops->vhost_reset_owner(dev); } } diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c index e409a86..abbaa8b 100644 --- a/hw/virtio/vhost-backend.c +++ b/hw/virtio/vhost-backend.c @@ -191,7 +191,7 @@ static int vhost_kernel_set_owner(struct vhost_dev *dev) return vhost_kernel_call(dev, VHOST_SET_OWNER, NULL); } -static int vhost_kernel_reset_device(struct vhost_dev *dev) +static int vhost_kernel_reset_owner(struct vhost_dev *dev) { return vhost_kernel_call(dev, VHOST_RESET_OWNER, NULL); } @@ -317,7 +317,7 @@ const VhostOps kernel_ops = { .vhost_get_features = vhost_kernel_get_features, .vhost_set_backend_cap = vhost_kernel_set_backend_cap, .vhost_set_owner = vhost_kernel_set_owner, - .vhost_reset_device = vhost_kernel_reset_device, + .vhost_reset_owner = vhost_kernel_reset_owner, .vhost_get_vq_index = vhost_kernel_get_vq_index, #ifdef CONFIG_VHOST_VSOCK .vhost_vsock_set_guest_cid = vhost_kernel_vsock_set_guest_cid, diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 6abbc9d..4412008 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -1475,16 +1475,29 @@ static int vhost_user_get_max_memslots(struct vhost_dev *dev, return 0; } +static int vhost_user_reset_owner(struct vhost_dev *dev) +{ + VhostUserMsg msg = { + .hdr.request = VHOST_USER_RESET_OWNER, + .hdr.flags = VHOST_USER_VERSION, + }; + + return vhost_user_write(dev, &msg, NULL, 0); +} + static int vhost_user_reset_device(struct vhost_dev *dev) { VhostUserMsg msg = { + .hdr.request = VHOST_USER_RESET_DEVICE, .hdr.flags = VHOST_USER_VERSION, }; - msg.hdr.request = virtio_has_feature(dev->protocol_features, - VHOST_USER_PROTOCOL_F_RESET_DEVICE) - ? VHOST_USER_RESET_DEVICE - : VHOST_USER_RESET_OWNER; + /* Caller must ensure the backend has VHOST_USER_PROTOCOL_F_RESET_DEVICE + * support */ + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_RESET_DEVICE)) { + return -EPERM; + } return vhost_user_write(dev, &msg, NULL, 0); } @@ -2548,6 +2561,7 @@ const VhostOps user_ops = { .vhost_set_features = vhost_user_set_features, .vhost_get_features = vhost_user_get_features, .vhost_set_owner = vhost_user_set_owner, + .vhost_reset_owner = vhost_user_reset_owner, .vhost_reset_device = vhost_user_reset_device, .vhost_get_vq_index = vhost_user_get_vq_index, .vhost_set_vring_enable = vhost_user_set_vring_enable, diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h index 81bf310..affeeb0 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -77,6 +77,7 @@ typedef int (*vhost_get_features_op)(struct vhost_dev *dev, uint64_t *features); typedef int (*vhost_set_backend_cap_op)(struct vhost_dev *dev); typedef int (*vhost_set_owner_op)(struct vhost_dev *dev); +typedef int (*vhost_reset_owner_op)(struct vhost_dev *dev); typedef int (*vhost_reset_device_op)(struct vhost_dev *dev); typedef int (*vhost_get_vq_index_op)(struct vhost_dev *dev, int idx); typedef int (*vhost_set_vring_enable_op)(struct vhost_dev *dev, @@ -150,6 +151,7 @@ typedef struct VhostOps { vhost_get_features_op vhost_get_features; vhost_set_backend_cap_op vhost_set_backend_cap; vhost_set_owner_op vhost_set_owner; + vhost_reset_owner_op vhost_reset_owner; vhost_reset_device_op vhost_reset_device; vhost_get_vq_index_op vhost_get_vq_index; vhost_set_vring_enable_op vhost_set_vring_enable; From patchwork Fri Apr 1 11:06:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Qiu X-Patchwork-Id: 1612166 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KVHbY4LT9z9sG5 for ; Fri, 1 Apr 2022 22:13:29 +1100 (AEDT) Received: from localhost ([::1]:45932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1naFDb-0002Mj-DW for incoming@patchwork.ozlabs.org; Fri, 01 Apr 2022 07:13:27 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50656) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFCl-0001th-KO for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:12:35 -0400 Received: from smtpbg123.qq.com ([175.27.65.52]:52650 helo=smtpbg.qq.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFCj-0000hR-08 for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:12:35 -0400 X-QQ-mid: bizesmtp66t1648811189trhoxor6 Received: from localhost.localdomain.localdoma ( [116.228.45.98]) by bizesmtp.qq.com (ESMTP) with id ; Fri, 01 Apr 2022 19:06:28 +0800 (CST) X-QQ-SSF: 01000000002000803000C00A0000000 X-QQ-FEAT: PdU/eI8FBMCoG2KWKMNq1DAve3+NzB7za+6QfTMVl6MPBQ7RK2TIZxsjX61TH kKIeDXoPn93ItCOnqZkXU0PUKtJPlu0wHL5JyuacElYfLR50/vgGm/zUTWatGuQLqtXp4fm LLZR3urV3oevQOd6WUH5P4zNCSudZ363N3kcsBJsDBU4Os85yQGkjwH/W7yyrGXo+dbADuE PEShZRaCWaRPho3eG/rNTVKMKRHwMYK9FNDo1Ixv8eywxdXdttBqoKqEq6JXuOPWwzGZPpA PnAnpBYoz0q72UqaNQa91y2PWgzhYcOA58ST5K1JQS6FtjVX2OOkApZVhN99XuDi6mXJY1t j9CQKtHFsvvqzgZRB9eUpQ4dRMolQ== X-QQ-GoodBg: 0 From: Michael Qiu To: jasowang@redhat.com, mst@redhat.com, si-wei.liu@oracle.com Subject: [PATCH 2/3] vhost: add vhost_dev_reset() Date: Fri, 1 Apr 2022 07:06:12 -0400 Message-Id: <1648811173-15293-3-git-send-email-qiudayu@archeros.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648811173-15293-1-git-send-email-qiudayu@archeros.com> References: <1648776683-23739-1-git-send-email-qiudayu@archeros.com> <1648811173-15293-1-git-send-email-qiudayu@archeros.com> X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:archeros.com:qybgspam:qybgspam8 Received-SPF: none client-ip=175.27.65.52; envelope-from=qiudayu@archeros.com; helo=smtpbg.qq.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, KHOP_HELO_FCRDNS=0.187, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Qiu , eperezma@redhat.com, lingshan.zhu@intel.com, qemu-devel@nongnu.org, lulu@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Not all vhost-user backends support ops->vhost_reset_device(). Instead of adding backend check and call backend ops directly, it's better to implement a function in vhost framework, so that it could hide vhost_ops details. SIgned-off-by: Michael Qiu --- hw/virtio/vhost.c | 14 ++++++++++++++ include/hw/virtio/vhost.h | 1 + 2 files changed, 15 insertions(+) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index b643f42..26667ae 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1854,3 +1854,17 @@ int vhost_net_set_backend(struct vhost_dev *hdev, return -ENOSYS; } + +int vhost_dev_reset(struct vhost_dev *hdev) +{ + int ret = 0; + + /* should only be called after backend is connected */ + assert(hdev->vhost_ops); + + if (hdev->vhost_ops->vhost_reset_device) { + ret = hdev->vhost_ops->vhost_reset_device(hdev); + } + + return ret; +} diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 58a73e7..b8b7c20 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -114,6 +114,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque, void vhost_dev_cleanup(struct vhost_dev *hdev); int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev); void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev); +int vhost_dev_reset(struct vhost_dev *hdev); int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev); void vhost_dev_disable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev); From patchwork Fri Apr 1 11:06:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Qiu X-Patchwork-Id: 1612167 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KVHgM1Z8bz9sG5 for ; Fri, 1 Apr 2022 22:16:45 +1100 (AEDT) Received: from localhost ([::1]:48164 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1naFGk-0004B8-HP for incoming@patchwork.ozlabs.org; Fri, 01 Apr 2022 07:16:42 -0400 Received: from eggs.gnu.org ([209.51.188.92]:51172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFFz-0004Aq-9V for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:15:55 -0400 Received: from smtpbg137.qq.com ([106.55.202.113]:25453 helo=smtpbg.qq.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1naFFw-0001PZ-Lx for qemu-devel@nongnu.org; Fri, 01 Apr 2022 07:15:54 -0400 X-QQ-mid: bizesmtp66t1648811193tlidghw2 Received: from localhost.localdomain.localdoma ( [116.228.45.98]) by bizesmtp.qq.com (ESMTP) with id ; Fri, 01 Apr 2022 19:06:31 +0800 (CST) X-QQ-SSF: 01000000002000803000C00A0000000 X-QQ-FEAT: SIq+DGERJtzzFDgZO+cJW86DuahPtpRbbS1bSpmPzG1D4HP24R9yiij22pfiu s9afFfScbps5niCoD6BmdUjcxa1udQPQ8b6GMKt+pQgJJVDxjYRCOfjJWVvi9tugbr0MgBR sOhk6z+anJze1JhmD3Zc9qvd050UzEVZDJvmuXBxtPuzeOWQ2KqMzRngXj2VSWGtT17DEiZ yEQqzbaOgUHaIaAsGeiB/XVNn+Xl2fjBYa4kjHKhVPSk55UZF6IwHXVwdFPdqloRsKLSjLN U8reP0+HecdwiuO/5ZLyiPPI2tOrAwCyygMIsu9x5RzOnEmNJ6tcqxLJKniayiTMcsHQgs5 OCIfWscnoPJlAOoVSNHsCJOJC3i7qm+BlXauZVu X-QQ-GoodBg: 0 From: Michael Qiu To: jasowang@redhat.com, mst@redhat.com, si-wei.liu@oracle.com Subject: [PATCH 3/3 v5] vdpa: reset the backend device in the end of vhost_net_stop() Date: Fri, 1 Apr 2022 07:06:13 -0400 Message-Id: <1648811173-15293-4-git-send-email-qiudayu@archeros.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1648811173-15293-1-git-send-email-qiudayu@archeros.com> References: <1648776683-23739-1-git-send-email-qiudayu@archeros.com> <1648811173-15293-1-git-send-email-qiudayu@archeros.com> X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:archeros.com:qybgspam:qybgspam9 Received-SPF: none client-ip=106.55.202.113; envelope-from=qiudayu@archeros.com; helo=smtpbg.qq.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, KHOP_HELO_FCRDNS=0.187, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michael Qiu , eperezma@redhat.com, lingshan.zhu@intel.com, qemu-devel@nongnu.org, lulu@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Currently, when VM poweroff, it will trigger vdpa device(such as mlx bluefield2 VF) reset many times(with 1 datapath queue pair and one control queue, triggered 3 times), this leads to below issue: vhost VQ 2 ring restore failed: -22: Invalid argument (22) This because in vhost_net_stop(), it will stop all vhost device bind to this virtio device, and in vhost_dev_stop(), qemu tries to stop the device , then stop the queue: vhost_virtqueue_stop(). In vhost_dev_stop(), it resets the device, which clear some flags in low level driver, and in next loop(stop other vhost backends), qemu try to stop the queue corresponding to the vhost backend, the driver finds that the VQ is invalied, this is the root cause. To solve the issue, vdpa should set vring unready, and remove reset ops in device stop: vhost_dev_start(hdev, false). and implement a new function vhost_dev_reset, only reset backend device after all vhost(per-queue) stopped. Signed-off-by: Michael Qiu Acked-by: Jason Wang --- v5 --> v4: move vhost_dev_reset() call after set_guest_notifiers remove implementation of vhost_dev_reset() remove backend check for VHOST_BACKEND_TYPE_VDPA v4 --> v3: Nothing changed, becasue of issue with mimecast, when the From: tag is different from the sender, the some mail client will take the patch as an attachment, RESEND v3 does not work, So resend the patch as v4 v3 --> v2: Call vhost_dev_reset() at the end of vhost_net_stop(). Since the vDPA device need re-add the status bit VIRTIO_CONFIG_S_ACKNOWLEDGE and VIRTIO_CONFIG_S_DRIVER, simply, add them inside vhost_vdpa_reset_device, and the only way calling vhost_vdpa_reset_device is in vhost_net_stop(), so it keeps the same behavior as before. v2 --> v1: Implement a new function vhost_dev_reset, reset the backend kernel device at last. --- hw/net/vhost_net.c | 22 +++++++++++++++++++--- hw/virtio/vhost-vdpa.c | 15 +++++++++------ 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index 30379d2..30c76ca 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -325,7 +325,7 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, int total_notifiers = data_queue_pairs * 2 + cvq; VirtIONet *n = VIRTIO_NET(dev); int nvhosts = data_queue_pairs + cvq; - struct vhost_net *net; + struct vhost_net *net = NULL; int r, e, i, index_end = data_queue_pairs * 2; NetClientState *peer; @@ -391,13 +391,21 @@ int vhost_net_start(VirtIODevice *dev, NetClientState *ncs, err_start: while (--i >= 0) { peer = qemu_get_peer(ncs , i); - vhost_net_stop_one(get_vhost_net(peer), dev); + + net = get_vhost_net(peer); + + vhost_net_stop_one(net, dev); } + e = k->set_guest_notifiers(qbus->parent, total_notifiers, false); if (e < 0) { fprintf(stderr, "vhost guest notifier cleanup failed: %d\n", e); fflush(stderr); } + + if (net) { + vhost_dev_reset(&net->dev); + } err: return r; } @@ -410,6 +418,7 @@ void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs, VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(vbus); VirtIONet *n = VIRTIO_NET(dev); NetClientState *peer; + struct vhost_net *net = NULL; int total_notifiers = data_queue_pairs * 2 + cvq; int nvhosts = data_queue_pairs + cvq; int i, r; @@ -420,7 +429,10 @@ void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs, } else { peer = qemu_get_peer(ncs, n->max_queue_pairs); } - vhost_net_stop_one(get_vhost_net(peer), dev); + + net = get_vhost_net(peer); + + vhost_net_stop_one(net, dev); } r = k->set_guest_notifiers(qbus->parent, total_notifiers, false); @@ -429,6 +441,10 @@ void vhost_net_stop(VirtIODevice *dev, NetClientState *ncs, fflush(stderr); } assert(r >= 0); + + if (net) { + vhost_dev_reset(&net->dev); + } } void vhost_net_cleanup(struct vhost_net *net) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index c5ed7a3..3ef0199 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -708,6 +708,11 @@ static int vhost_vdpa_reset_device(struct vhost_dev *dev) ret = vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &status); trace_vhost_vdpa_reset_device(dev, status); + + /* Add back this status, so that the device could work next time*/ + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | + VIRTIO_CONFIG_S_DRIVER); + return ret; } @@ -719,14 +724,14 @@ static int vhost_vdpa_get_vq_index(struct vhost_dev *dev, int idx) return idx; } -static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev) +static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev, unsigned int ready) { int i; trace_vhost_vdpa_set_vring_ready(dev); for (i = 0; i < dev->nvqs; ++i) { struct vhost_vring_state state = { .index = dev->vq_index + i, - .num = 1, + .num = ready, }; vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); } @@ -1088,8 +1093,9 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) if (unlikely(!ok)) { return -1; } - vhost_vdpa_set_vring_ready(dev); + vhost_vdpa_set_vring_ready(dev, 1); } else { + vhost_vdpa_set_vring_ready(dev, 0); ok = vhost_vdpa_svqs_stop(dev); if (unlikely(!ok)) { return -1; @@ -1105,9 +1111,6 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) memory_listener_register(&v->listener, &address_space_memory); return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); } else { - vhost_vdpa_reset_device(dev); - vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | - VIRTIO_CONFIG_S_DRIVER); memory_listener_unregister(&v->listener); return 0;