From patchwork Fri Oct 13 08:09:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848054 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=Fsny6DIZ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6KLG0JbNz1yqj for ; Fri, 13 Oct 2023 19:24:38 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFL-0006sH-Cl; Fri, 13 Oct 2023 04:10:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDF8-0006eG-J1 for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:00 -0400 Received: from mail-oo1-xc32.google.com ([2607:f8b0:4864:20::c32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDF6-0001XT-MW for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:09:58 -0400 Received: by mail-oo1-xc32.google.com with SMTP id 006d021491bc7-57bab4e9e1aso1054060eaf.3 for ; Fri, 13 Oct 2023 01:09:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184595; x=1697789395; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hBBVQtIzyB6lcv+H22SmqBtB6k05lp7dOPID3f7Tk/k=; b=Fsny6DIZ36fsEtFfhN55jjoSl7DdGePmvyG7fuYqEPTY2LXq4Tk43acKdVOfdqb7x+ mCnru5alPvxupswgoMo8SF4peTj+33HRW032+pG0aLWGyPoK+jymPWGpweHHYN4m5zND L6pFjBxzoU6FUutN8iyLvqChwnmYnHNJCTotl18CruepkVh2QFSGa9GCgbFkIiddRQSf 3++fxml41Tcc4y3b3Uamq1jk/+1xnbHay8qvGbuQa0piEYHYE/2VV/EeL7OqRb4KCnf5 Th6d42xSR3Siq84SBRIuWLdjMtLqb71URMA20UZkU2AKvZ7rraEpT9VfzH4z2FukZAGK Gv3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184595; x=1697789395; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hBBVQtIzyB6lcv+H22SmqBtB6k05lp7dOPID3f7Tk/k=; b=UGU1uU9GyOEOBbZc0vNmLjz2Y9Ozy8rp+aNrymxzlrdULC+Og/Qh1sK0O5OdVK8cUl WzIXYyCDqlkLdpO9A86hVWthiV3RzRPf8aheuVO+xwi5ZiBR+XLUYacPk5QZC7Evy6do 4N8tfrZ0liMLYY9I9VG6//qYXZ5PVmdeHZ3dNfVRX0n7cKavd0U8q2VINuSCaCryX/VT DMpMygC0NdrRFjCdqhyXkjrjeKSiPH7nAE4qaSmwE80QO2NR2O6eW78/EBUU3hdH807S jFXtkH7JInlYZN/XepVf6jh1x+8UpkqZ1jEUEpumsCmdwzY+Y05ew3SLWrSfts4Nq0n5 +dvA== X-Gm-Message-State: AOJu0Yw9VhzlNBv+hOAdaSI0Bglf12qH+1ZG60Li9k8Kin/VcH7Jic+S Dym4P66hn1lTQwRilw4kNn4= X-Google-Smtp-Source: AGHT+IG6ixcgG6RKO2/ggEK5TwsP5FG/x0AGoQAlXktZW27BohtHpFyo69DP2vThLPL8dmxIv+HJIw== X-Received: by 2002:a05:6358:7e49:b0:134:e603:116e with SMTP id p9-20020a0563587e4900b00134e603116emr28356192rwm.6.1697184594504; Fri, 13 Oct 2023 01:09:54 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id r19-20020aa78453000000b00688965c5227sm13020573pfn.120.2023.10.13.01.09.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:09:54 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 1/7] vdpa: Use iovec for vhost_vdpa_net_cvq_add() Date: Fri, 13 Oct 2023 16:09:36 +0800 Message-Id: <8a328f146fb043f34edb75ba6d043d2d6de88f99.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::c32; envelope-from=yin31149@gmail.com; helo=mail-oo1-xc32.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Next patches in this series will no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. Consequently, there will be multiple pending buffers in the shadow VirtQueue, making it a must for every control command to have its own buffer. To achieve this, this patch refactor vhost_vdpa_net_cvq_add() to accept `struct iovec`, which eliminates the coupling of control commands to `s->cvq_cmd_out_buffer` and `s->status`, allowing them to use their own buffer. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- v5: - no change v4: https://lore.kernel.org/all/5e090c2af922192f5897ba7072df4d9e4754e1e0.1693287885.git.yin31149@gmail.com/ - split `in` to `vdpa_in` and `model_in` instead of reusing `in` in vhost_vdpa_net_handle_ctrl_avail() suggested by Eugenio v3: https://lore.kernel.org/all/b1d473772ec4bcb254ab0d12430c9b1efe758606.1689748694.git.yin31149@gmail.com/ net/vhost-vdpa.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 939c984d5b..618758596a 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -618,22 +618,14 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) vhost_vdpa_net_client_stop(nc); } -static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, - size_t in_len) +static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, + const struct iovec *out_sg, size_t out_num, + const struct iovec *in_sg, size_t in_num) { - /* Buffers for the device */ - const struct iovec out = { - .iov_base = s->cvq_cmd_out_buffer, - .iov_len = out_len, - }; - const struct iovec in = { - .iov_base = s->status, - .iov_len = sizeof(virtio_net_ctrl_ack), - }; VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); int r; - r = vhost_svq_add(svq, &out, 1, &in, 1, NULL); + r = vhost_svq_add(svq, out_sg, out_num, in_sg, in_num, NULL); if (unlikely(r != 0)) { if (unlikely(r == -ENOSPC)) { qemu_log_mask(LOG_GUEST_ERROR, "%s: No space on device queue\n", @@ -659,6 +651,15 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, .cmd = cmd, }; size_t data_size = iov_size(data_sg, data_num); + /* Buffers for the device */ + const struct iovec out = { + .iov_base = s->cvq_cmd_out_buffer, + .iov_len = sizeof(ctrl) + data_size, + }; + const struct iovec in = { + .iov_base = s->status, + .iov_len = sizeof(*s->status), + }; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); @@ -669,8 +670,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, iov_to_buf(data_sg, data_num, 0, s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); - return vhost_vdpa_net_cvq_add(s, data_size + sizeof(ctrl), - sizeof(virtio_net_ctrl_ack)); + return vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) @@ -1248,10 +1248,15 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, .iov_base = s->cvq_cmd_out_buffer, }; /* in buffer used for device model */ - const struct iovec in = { + const struct iovec model_in = { .iov_base = &status, .iov_len = sizeof(status), }; + /* in buffer used for vdpa device */ + const struct iovec vdpa_in = { + .iov_base = s->status, + .iov_len = sizeof(*s->status), + }; ssize_t dev_written = -EINVAL; out.iov_len = iov_to_buf(elem->out_sg, elem->out_num, 0, @@ -1285,7 +1290,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, goto out; } } else { - dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status)); + dev_written = vhost_vdpa_net_cvq_add(s, &out, 1, &vdpa_in, 1); if (unlikely(dev_written < 0)) { goto out; } @@ -1301,7 +1306,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, } status = VIRTIO_NET_ERR; - virtio_net_handle_ctrl_iov(svq->vdev, &in, 1, &out, 1); + virtio_net_handle_ctrl_iov(svq->vdev, &model_in, 1, &out, 1); if (status != VIRTIO_NET_OK) { error_report("Bad CVQ processing in model"); } From patchwork Fri Oct 13 08:09:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848034 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=BvRakcRh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6KFm0S08z1yqj for ; Fri, 13 Oct 2023 19:20:44 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFO-00075d-Eg; Fri, 13 Oct 2023 04:10:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFE-0006qO-Ii for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:06 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFB-0001Y3-VN for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:03 -0400 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1c8a1541233so14438375ad.1 for ; Fri, 13 Oct 2023 01:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184600; x=1697789400; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fM7/Invm03p5i3YtKczc/E5ZspKqc8+FVx1FNwc5f2Q=; b=BvRakcRhjOrcPNYtpmOWQ6ziEGxqK4Z0s9B3MpZg4CGRHnGYMavy4UKKaZrWzXtHXD QR+XcNoQD6Lt8G9m9xL2JArFjKPpHWU923/G1azossRF0VSzPVrOf3YpPjioNT7BT9t1 uLYYOR7OCci2s4UktoqyX43uSlHa+fUDcsJoJPC7CM+fppWWh8sTmRLG/KZ55O64efNq B60m9zmaXfhSmNVDpPIBUHeMQd0eyn6pAqKqIUB0hXMFnnKE/vgKNVDQ955KXLz74df0 AR2tahZ7PLsD5ftRm4l7VJBqss7zuXNoYnREUJDpKQgIz96DL+n00WNGeF2hMjhqgVab Mmbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184600; x=1697789400; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fM7/Invm03p5i3YtKczc/E5ZspKqc8+FVx1FNwc5f2Q=; b=sJX8nXsUfEpHQWhV705OuQ2VT/7lxTjHeTK65QTGeAxfmRqW+hFFBIO3g6ujJowzVP NMcLtvEm10Uq2K3x8n1Z+ee6wyk//4b2rlIJpBiUIUATfEmBNquORzsUHIjw26APaEcK b7YcAPUr92IxV6Qnsu5gz6OGjcTlJBFwL1wsuCVEjlHvMsGKCqrm4erl4+LRYSA31EaO iEwPKnlxNrUr/UZqOxN/Bc2feFZzxFrdVe5qvhoerLOLLRzU1k9fIoHX905YFOVTgGqp GHdlUG8IkrKJVoEHm2TLeOZsRT5YUFeTp+k+g9UpcsEo7og8t09opCtO7npeNrvMJstf /DbQ== X-Gm-Message-State: AOJu0YwZYfxtgJDg/Sx7BTLa09AV3nuZwE82WOj0v9sxgaLFjVEyQ/mr OKmOIn/A6Tn+WTPUsBeuBH0= X-Google-Smtp-Source: AGHT+IHKZ/yQ3CM1GEHeBPbgCTrHgsxzgNwBzPPLefc38ti515n2bFrUrEjxktxskhnTYdR3xsOsxg== X-Received: by 2002:a17:902:e550:b0:1b9:e937:9763 with SMTP id n16-20020a170902e55000b001b9e9379763mr27955703plf.12.1697184600029; Fri, 13 Oct 2023 01:10:00 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id x19-20020a170902ea9300b001c0a414695bsm3268886plb.43.2023.10.13.01.09.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:09:59 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 2/7] vdpa: Avoid using vhost_vdpa_net_load_*() outside vhost_vdpa_net_load() Date: Fri, 13 Oct 2023 16:09:37 +0800 Message-Id: <254f0618efde7af7229ba4fdada667bb9d318991.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=yin31149@gmail.com; helo=mail-pl1-x635.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Next patches in this series will refactor vhost_vdpa_net_load_cmd() to iterate through the control commands shadow buffers, allowing QEMU to send CVQ state load commands in parallel at device startup. Considering that QEMU always forwards the CVQ command serialized outside of vhost_vdpa_net_load(), it is more elegant to send the CVQ commands directly without invoking vhost_vdpa_net_load_*() helpers. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- v5: - remove redundant initialization statement suggested by Eugenio - remove assertion suggested by Eugenio v4: https://lore.kernel.org/all/a56d91c3cc2ab46f9be1770074c920f5375ddb5e.1693287885.git.yin31149@gmail.com/ - pack CVQ command by iov_from_buf() instead of accessing `out` directly suggested by Eugenio v3: https://lore.kernel.org/all/428a8fac2a29b37757fa15ca747be93c0226cb1f.1689748694.git.yin31149@gmail.com/ net/vhost-vdpa.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 618758596a..86b8d31244 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -1114,12 +1114,14 @@ static NetClientInfo net_vhost_vdpa_cvq_info = { */ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, VirtQueueElement *elem, - struct iovec *out) + struct iovec *out, + const struct iovec *in) { struct virtio_net_ctrl_mac mac_data, *mac_ptr; struct virtio_net_ctrl_hdr *hdr_ptr; uint32_t cursor; ssize_t r; + uint8_t on = 1; /* parse the non-multicast MAC address entries from CVQ command */ cursor = sizeof(*hdr_ptr); @@ -1167,7 +1169,13 @@ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, * filter table to the vdpa device, it should send the * VIRTIO_NET_CTRL_RX_PROMISC CVQ command to enable promiscuous mode */ - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 1); + hdr_ptr = out->iov_base; + out->iov_len = sizeof(*hdr_ptr) + sizeof(on); + + hdr_ptr->class = VIRTIO_NET_CTRL_RX; + hdr_ptr->cmd = VIRTIO_NET_CTRL_RX_PROMISC; + iov_from_buf(out, 1, sizeof(*hdr_ptr), &on, sizeof(on)); + r = vhost_vdpa_net_cvq_add(s, out, 1, in, 1); if (unlikely(r < 0)) { return r; } @@ -1285,7 +1293,7 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, * the CVQ command directly. */ dev_written = vhost_vdpa_net_excessive_mac_filter_cvq_add(s, elem, - &out); + &out, &vdpa_in); if (unlikely(dev_written < 0)) { goto out; } From patchwork Fri Oct 13 08:09:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848007 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=SPH839rj; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6K9h6gmQz1yyb for ; Fri, 13 Oct 2023 19:17:12 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFN-000719-DI; Fri, 13 Oct 2023 04:10:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFI-0006sG-OB for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:11 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFG-0001kn-RF for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:08 -0400 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1c894e4573bso14207285ad.0 for ; Fri, 13 Oct 2023 01:10:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184605; x=1697789405; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TORX/2+rZIf+XwtZl4Awh7Er7v8DktbAlILwpC/XKfA=; b=SPH839rjsvWrQZ/QQL3C+LT7Y1EVcrkUlo7mWosW4kTkem2AY55lQT4uPgxo4QaOBh PUwDTjQxMAGMl784RwsRiEWB1IkX0cNolVU+YtCJDvfx1iNUmcfrV5X9WYWdtCuNi8O9 rpzUX+OP5iaO1mMgGgnd7t8JgO2nPC9OblpalJ21ri9xV7vFDIOLmZTRkGPZYbunEZr7 yIDeK7u9KK3CMJ6R5E7Ok6uXNnmPWUxEWKMScODxh2Ve47RF+p2NqSc49KTzHLMBAgZH dCaEUKH1rDAEH1ZjefVda4QSEg8xT1CLf1j/5YpPM3ydL4hielt/s4uWgxWz53cqT5Bz Jb7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184605; x=1697789405; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TORX/2+rZIf+XwtZl4Awh7Er7v8DktbAlILwpC/XKfA=; b=FJVvh0V5zSGA11oogt4YYIfrmYpm5XR1D0N7T4/u8Gfqjv95iLiI0RMtGCUbpNnX0l edKMv0xRNKGY0VMPBIEu6C/m2eWntzVswJ9dwh0ZyMobIYJ3sIq3cHrP2ylPJ8oc1wfo Lub09yYyRioDElCo0PMXL0XGykoO70rxhbK8sBxwjy7E2TjXIkRpjo7xKPLHQwSaDfrd SMag3DpheC/ekL/MNuvd+SHPJihd1l39CzwhEsJzFNFK85teOSzvn4a3FdOTorDPD+El 6RjbRT2quEe16mMJvD8jFO2P478CUiVRzX27dVBdSJvS5L9V4aDe0LZLuzikoNIROPjO p86A== X-Gm-Message-State: AOJu0YynfmeihW9NBK2KH0ua+kDjO0ZjsgnpmSKUjovf0f8Hj4WyjWcO KlByT8aM/R7yfeAL2kKwoEk= X-Google-Smtp-Source: AGHT+IFrfLSuTsz9JfAKcdjCJMlkPNOJMqe1HD/zTqHDs75KVjR0FcUpR6ml4i3iTuurQdmBApoUgg== X-Received: by 2002:a17:902:bf0c:b0:1c5:b4a1:ff6 with SMTP id bi12-20020a170902bf0c00b001c5b4a10ff6mr21604955plb.45.1697184605151; Fri, 13 Oct 2023 01:10:05 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id e4-20020a17090301c400b001aaf2e8b1eesm3252091plh.248.2023.10.13.01.10.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:10:04 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 3/7] vdpa: Check device ack in vhost_vdpa_net_load_rx_mode() Date: Fri, 13 Oct 2023 16:09:38 +0800 Message-Id: <68811d52f96ae12d68f0d67d996ac1642a623943.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=yin31149@gmail.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Considering that vhost_vdpa_net_load_rx_mode() is only called within vhost_vdpa_net_load_rx() now, this patch refactors vhost_vdpa_net_load_rx_mode() to include a check for the device's ack, simplifying the code and improving its maintainability. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- v5: - no change v4: https://lore.kernel.org/all/be0e39e2c76e1ef39a76839b4a4ce90c8e54a98e.1693287885.git.yin31149@gmail.com/ net/vhost-vdpa.c | 76 ++++++++++++++++++++---------------------------- 1 file changed, 31 insertions(+), 45 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 86b8d31244..36a4e57c0d 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -827,14 +827,24 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, .iov_base = &on, .iov_len = sizeof(on), }; - return vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, - cmd, &data, 1); + ssize_t dev_written; + + dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, + cmd, &data, 1); + if (unlikely(dev_written < 0)) { + return dev_written; + } + if (*s->status != VIRTIO_NET_OK) { + return -EIO; + } + + return 0; } static int vhost_vdpa_net_load_rx(VhostVDPAState *s, const VirtIONet *n) { - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_RX)) { return 0; @@ -859,13 +869,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (!n->mac_table.uni_overflow && !n->promisc) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_PROMISC, 0); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 0); + if (unlikely(r < 0)) { + return r; } } @@ -887,13 +893,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->mac_table.multi_overflow || n->allmulti) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_ALLMULTI, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLMULTI, 1); + if (unlikely(r < 0)) { + return r; } } @@ -912,13 +914,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->alluni) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_ALLUNI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLUNI, 1); + if (r < 0) { + return r; } } @@ -933,13 +931,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nomulti) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOMULTI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOMULTI, 1); + if (r < 0) { + return r; } } @@ -954,13 +948,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nouni) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOUNI, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOUNI, 1); + if (r < 0) { + return r; } } @@ -975,13 +965,9 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nobcast) { - dev_written = vhost_vdpa_net_load_rx_mode(s, - VIRTIO_NET_CTRL_RX_NOBCAST, 1); - if (dev_written < 0) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOBCAST, 1); + if (r < 0) { + return r; } } From patchwork Fri Oct 13 08:09:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848030 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=MPOhNsWq; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6KDY3DWRz1yqj for ; Fri, 13 Oct 2023 19:19:41 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFS-0007ak-OQ; Fri, 13 Oct 2023 04:10:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFQ-0007Eo-AU for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:16 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFO-0001mS-DA for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:16 -0400 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1c9a1762b43so15194895ad.1 for ; Fri, 13 Oct 2023 01:10:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184611; x=1697789411; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7IDKITTbgc/cUnDaHPYmgtL0QhMPD6TOubiMOIIaJts=; b=MPOhNsWqZ8/pv3vH1PyzhJV5MDU/EK9MxVXgYBqa7votb45PMSvuJeGa7YAXblW2+f by++X+7bAu/xvSTj2HzyhCumWVOjcCOuhoRsCoYWI+fKvT7ebmTnHdLZDh+LpHu5oU6R 3rFJdvCVuE5sWs0w7/vjMJd3uplwlJ32VkDKAgcaur/S/LVJlFSjIdIAbWZoytx9hMLh Y2t3UmPZT66OmTcIkXA3WI0FzPNkqBqixCBEENrkov/gQNIykUwGTa3HqXOFyetyLXqj o+kgo5j3d1PV4Pk+z1bLkF00Qmih4baQG+8yoTghIC88cxA8Ftnk//5bZ86KMrsPsmMG ichw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184611; x=1697789411; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7IDKITTbgc/cUnDaHPYmgtL0QhMPD6TOubiMOIIaJts=; b=LiIhdsOshylXJ0BAPDDYNfXQv+0fvgc58KLh7POY0W8Vods2vOCaiXHFHXYeAj/hxc qZdiprNe+Rhp98vUYqALd6ptiPr1VvKq7KVmn3XgZ+LqwiJMQqoZCUzBfbUQU8vLtGXn BjxPwlOo+M5C1aDY5eQjB5cJn22m1oElDLL6nCm4/+420ciYbTNFSW/nEyBanr1MX1jS HunTDPTKDauSeFmlZRV82QEfDTbAnnKeu3PR24q0soz45fd+01t6R4zXg5Iy/cGgsdmG +IVCc80T19NjdixwaY+p3fMm5CWR/rBMBjKPjpoCD94AwpRTXuyGgf3X+OYN9g06O7JJ c4vQ== X-Gm-Message-State: AOJu0YzQjlZ9wnrmP+QG1gfMWfPWNVr7cYvZ88evxZCsMbxi1SR9UCOE gVtxtlpJn6PRywUV0U0iJnM= X-Google-Smtp-Source: AGHT+IEJAy+PPMpy66/fAxe2EOJYkRJHfBU4Q9NMI68/G80auw3V0JKrQ4m4A7qcHnNDf8QsCuTNJg== X-Received: by 2002:a17:902:e749:b0:1c5:e00f:2 with SMTP id p9-20020a170902e74900b001c5e00f0002mr32026268plf.28.1697184610672; Fri, 13 Oct 2023 01:10:10 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id p11-20020a170902c70b00b001b0358848b0sm3301933plp.161.2023.10.13.01.10.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:10:10 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 4/7] vdpa: Move vhost_svq_poll() to the caller of vhost_vdpa_net_cvq_add() Date: Fri, 13 Oct 2023 16:09:39 +0800 Message-Id: <196cadb55175a75275660c6634a538289f027ae3.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=yin31149@gmail.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch moves vhost_svq_poll() to the caller of vhost_vdpa_net_cvq_add() and introduces a helper funtion. By making this change, next patches in this series is able to refactor vhost_vdpa_net_load_x() only to delay the polling and checking process until either the SVQ is full or control commands shadow buffers are full. Signed-off-by: Hawkins Jiawei --- v5: - no change v4: https://lore.kernel.org/all/496c542c22ae1b4222175d5576c949621c7c2fc0.1693287885.git.yin31149@gmail.com/ - always check the return value of vhost_vdpa_net_svq_poll() suggested Eugenio v3: https://lore.kernel.org/all/152177c4e7082236fba9d31d535e40f8c2984349.1689748694.git.yin31149@gmail.com/ net/vhost-vdpa.c | 53 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 43 insertions(+), 10 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 36a4e57c0d..ea73e3c410 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -631,15 +631,21 @@ static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, qemu_log_mask(LOG_GUEST_ERROR, "%s: No space on device queue\n", __func__); } - return r; } - /* - * We can poll here since we've had BQL from the time we sent the - * descriptor. Also, we need to take the answer before SVQ pulls by itself, - * when BQL is released - */ - return vhost_svq_poll(svq, 1); + return r; +} + +/* + * Convenience wrapper to poll SVQ for multiple control commands. + * + * Caller should hold the BQL when invoking this function, and should take + * the answer before SVQ pulls by itself when BQL is released. + */ +static ssize_t vhost_vdpa_net_svq_poll(VhostVDPAState *s, size_t cmds_in_flight) +{ + VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); + return vhost_svq_poll(svq, cmds_in_flight); } static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, @@ -660,6 +666,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, .iov_base = s->status, .iov_len = sizeof(*s->status), }; + ssize_t r; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); @@ -670,7 +677,16 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, iov_to_buf(data_sg, data_num, 0, s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); - return vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); + r = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); + if (unlikely(r < 0)) { + return r; + } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + return vhost_vdpa_net_svq_poll(s, 1); } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) @@ -1165,6 +1181,15 @@ static int vhost_vdpa_net_excessive_mac_filter_cvq_add(VhostVDPAState *s, if (unlikely(r < 0)) { return r; } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + r = vhost_vdpa_net_svq_poll(s, 1); + if (unlikely(r < sizeof(*s->status))) { + return r; + } if (*s->status != VIRTIO_NET_OK) { return sizeof(*s->status); } @@ -1284,10 +1309,18 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, goto out; } } else { - dev_written = vhost_vdpa_net_cvq_add(s, &out, 1, &vdpa_in, 1); - if (unlikely(dev_written < 0)) { + ssize_t r; + r = vhost_vdpa_net_cvq_add(s, &out, 1, &vdpa_in, 1); + if (unlikely(r < 0)) { + dev_written = r; goto out; } + + /* + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + dev_written = vhost_vdpa_net_svq_poll(s, 1); } if (unlikely(dev_written < sizeof(status))) { From patchwork Fri Oct 13 08:09:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848011 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=C8agmf7+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6KB33WM0z1yqj for ; Fri, 13 Oct 2023 19:17:31 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFk-0000FA-NK; Fri, 13 Oct 2023 04:10:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFU-0007us-Vm for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:26 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFS-0001nM-De for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:20 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1c9e95aa02dso7772785ad.0 for ; Fri, 13 Oct 2023 01:10:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184617; x=1697789417; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lHFGtUPbr7a0APIT598kvWUHkjHi6m86qE8RHgeWeJs=; b=C8agmf7+VTNs+a8Wut9GfV/XbBTlfHoMQIGvHKhYMYDjP3P1ch6AS6pxTUp6xOe/7a 5O7GDLdGI8b3RiYKaDD5PmGpbxad8DhBd3l5qrnClAarwhfRmrtFyfZq3/rG2+z4R8ZH doySP2bNrYjLV5AQbz4PP7NtwH1Hs5Jgj4Vkg/MIiCTd+ozoDVZdHMowJk9/rcp857PN s9tdD1zhgKD8zVHsJtwfWpJ/1ldTV1vuJVd7oXb93/LVRQTsEq4gDQUI1Nf9NNzjIOJc GiG86ETlRWptiVwe9HoC5JULiFmehGhmWRr7Dmyu8OOUZsXA5+Uv8T1V/6vhRL+j4ey/ 3mxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184617; x=1697789417; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lHFGtUPbr7a0APIT598kvWUHkjHi6m86qE8RHgeWeJs=; b=G+3KsxUH21V9iCL5hQMZAwqkGWnY9Dvcun3UqD0HaTToT1VU4Ju29g1cUAHhKqTDkK 5MtElhH62YhPCdwP2SW6HBgF/IEpL7ZIryjPcKPDMs6MJNOMb3ZTgeLdsBCdbAp5rbF5 xMgBRb/tNd5omIfDhaM2LfDLLCxtt/7jeahHZulxvUOYBohoT7NRF4P/coAJq3kiqMJv rhLn7J+CNhAtDZ3AKYtHtikCOMCPdlH1i3dMBh8U7tjzIrB12lKy1df2UONZdg9XFC1Z zKGt9OPd1LyvWyl5hLV0A4ehQmT+nC7FhRb2Ko3Ra5IoRRnCmsXP+NP9Er+tyN07QR2/ jfKQ== X-Gm-Message-State: AOJu0YzMVGQbFzpxm+x6K0GALWq0Eu4+JB76uiVGY507FhjpljgDv2MJ mPDB9dswwbZGhvXLCYOX+Q0= X-Google-Smtp-Source: AGHT+IE2cosHc56Rq5qjLQlpaDwuQcc4Pkr4Co/tsYCCHw1uXbBcpnh2VCh4P3bbhbw6xbdlgcIHXQ== X-Received: by 2002:a17:902:e5d1:b0:1c6:362:3553 with SMTP id u17-20020a170902e5d100b001c603623553mr33933078plf.31.1697184616716; Fri, 13 Oct 2023 01:10:16 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id iw11-20020a170903044b00b001b03f208323sm3260596plb.64.2023.10.13.01.10.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:10:16 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 5/7] vdpa: Introduce cursors to vhost_vdpa_net_loadx() Date: Fri, 13 Oct 2023 16:09:40 +0800 Message-Id: <1c6516e233a14cc222f0884e148e4e1adceda78d.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=yin31149@gmail.com; helo=mail-pl1-x632.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch introduces two new arugments, `out_cursor` and `in_cursor`, to vhost_vdpa_net_loadx(). Addtionally, it includes a helper function vhost_vdpa_net_load_cursor_reset() for resetting these cursors. Furthermore, this patch refactors vhost_vdpa_net_load_cmd() so that vhost_vdpa_net_load_cmd() prepares buffers for the device using the cursors arguments, instead of directly accesses `s->cvq_cmd_out_buffer` and `s->status` fields. By making these change, next patches in this series can refactor vhost_vdpa_net_load_cmd() directly to iterate through the control commands shadow buffers, allowing QEMU to send CVQ state load commands in parallel at device startup. Signed-off-by: Hawkins Jiawei --- v5: - move iov_copy() call before vhost_vdpa_net_cvq_add() and add comments for iov_copy() to improve readability - fix conflicts with master branch v4: https://lore.kernel.org/all/0e2af3ed5695a8044877911df791417fe0ba87af.1693287885.git.yin31149@gmail.com/ - use `struct iovec` instead of `void **` as cursor suggested by Eugenio - add vhost_vdpa_net_load_cursor_reset() helper function to reset the cursors - refactor vhost_vdpa_net_load_cmd() to prepare buffers by iov_copy() instead of accessing `in` and `out` directly suggested by Eugenio v3: https://lore.kernel.org/all/bf390934673f2b613359ea9d7ac6c89199c31384.1689748694.git.yin31149@gmail.com/ net/vhost-vdpa.c | 111 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 75 insertions(+), 36 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index ea73e3c410..ef4d242811 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -648,7 +648,22 @@ static ssize_t vhost_vdpa_net_svq_poll(VhostVDPAState *s, size_t cmds_in_flight) return vhost_svq_poll(svq, cmds_in_flight); } -static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, +static void vhost_vdpa_net_load_cursor_reset(VhostVDPAState *s, + struct iovec *out_cursor, + struct iovec *in_cursor) +{ + /* reset the cursor of the output buffer for the device */ + out_cursor->iov_base = s->cvq_cmd_out_buffer; + out_cursor->iov_len = vhost_vdpa_net_cvq_cmd_page_len(); + + /* reset the cursor of the in buffer for the device */ + in_cursor->iov_base = s->status; + in_cursor->iov_len = vhost_vdpa_net_cvq_cmd_page_len(); +} + +static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, + struct iovec *out_cursor, + struct iovec *in_cursor, uint8_t class, uint8_t cmd, const struct iovec *data_sg, size_t data_num) { @@ -657,25 +672,21 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, .cmd = cmd, }; size_t data_size = iov_size(data_sg, data_num); - /* Buffers for the device */ - const struct iovec out = { - .iov_base = s->cvq_cmd_out_buffer, - .iov_len = sizeof(ctrl) + data_size, - }; - const struct iovec in = { - .iov_base = s->status, - .iov_len = sizeof(*s->status), - }; + struct iovec out, in; ssize_t r; assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); /* pack the CVQ command header */ - memcpy(s->cvq_cmd_out_buffer, &ctrl, sizeof(ctrl)); - + iov_from_buf(out_cursor, 1, 0, &ctrl, sizeof(ctrl)); /* pack the CVQ command command-specific-data */ iov_to_buf(data_sg, data_num, 0, - s->cvq_cmd_out_buffer + sizeof(ctrl), data_size); + out_cursor->iov_base + sizeof(ctrl), data_size); + + /* extract the required buffer from the cursor for output */ + iov_copy(&out, 1, out_cursor, 1, 0, sizeof(ctrl) + data_size); + /* extract the required buffer from the cursor for input */ + iov_copy(&in, 1, in_cursor, 1, 0, sizeof(*s->status)); r = vhost_vdpa_net_cvq_add(s, &out, 1, &in, 1); if (unlikely(r < 0)) { @@ -689,14 +700,17 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, return vhost_vdpa_net_svq_poll(s, 1); } -static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) +static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor) { if (virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_MAC_ADDR)) { const struct iovec data = { .iov_base = (void *)n->mac, .iov_len = sizeof(n->mac), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MAC, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MAC, VIRTIO_NET_CTRL_MAC_ADDR_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -748,7 +762,7 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) .iov_len = mul_macs_size, }, }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, VIRTIO_NET_CTRL_MAC, VIRTIO_NET_CTRL_MAC_TABLE_SET, data, ARRAY_SIZE(data)); @@ -763,7 +777,9 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) } static int vhost_vdpa_net_load_mq(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor) { struct virtio_net_ctrl_mq mq; ssize_t dev_written; @@ -777,7 +793,8 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, .iov_base = &mq, .iov_len = sizeof(mq), }; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MQ, + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MQ, VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -791,7 +808,9 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, } static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor) { uint64_t offloads; ssize_t dev_written; @@ -822,7 +841,8 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, .iov_base = &offloads, .iov_len = sizeof(offloads), }; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_GUEST_OFFLOADS, + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_GUEST_OFFLOADS, VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, &data, 1); if (unlikely(dev_written < 0)) { @@ -836,6 +856,8 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, } static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, + struct iovec *out_cursor, + struct iovec *in_cursor, uint8_t cmd, uint8_t on) { @@ -845,7 +867,8 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, }; ssize_t dev_written; - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_RX, + dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX, cmd, &data, 1); if (unlikely(dev_written < 0)) { return dev_written; @@ -858,7 +881,9 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, } static int vhost_vdpa_net_load_rx(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor) { ssize_t r; @@ -885,7 +910,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (!n->mac_table.uni_overflow && !n->promisc) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_PROMISC, 0); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_PROMISC, 0); if (unlikely(r < 0)) { return r; } @@ -909,7 +935,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->mac_table.multi_overflow || n->allmulti) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLMULTI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_ALLMULTI, 1); if (unlikely(r < 0)) { return r; } @@ -930,7 +957,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->alluni) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_ALLUNI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_ALLUNI, 1); if (r < 0) { return r; } @@ -947,7 +975,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nomulti) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOMULTI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOMULTI, 1); if (r < 0) { return r; } @@ -964,7 +993,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nouni) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOUNI, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOUNI, 1); if (r < 0) { return r; } @@ -981,7 +1011,8 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, * configuration only at live migration. */ if (n->nobcast) { - r = vhost_vdpa_net_load_rx_mode(s, VIRTIO_NET_CTRL_RX_NOBCAST, 1); + r = vhost_vdpa_net_load_rx_mode(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX_NOBCAST, 1); if (r < 0) { return r; } @@ -992,13 +1023,16 @@ static int vhost_vdpa_net_load_rx(VhostVDPAState *s, static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor, uint16_t vid) { const struct iovec data = { .iov_base = &vid, .iov_len = sizeof(vid), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_VLAN, + ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_VLAN, VIRTIO_NET_CTRL_VLAN_ADD, &data, 1); if (unlikely(dev_written < 0)) { @@ -1012,7 +1046,9 @@ static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, } static int vhost_vdpa_net_load_vlan(VhostVDPAState *s, - const VirtIONet *n) + const VirtIONet *n, + struct iovec *out_cursor, + struct iovec *in_cursor) { int r; @@ -1023,7 +1059,8 @@ static int vhost_vdpa_net_load_vlan(VhostVDPAState *s, for (int i = 0; i < MAX_VLAN >> 5; i++) { for (int j = 0; n->vlans[i] && j <= 0x1f; j++) { if (n->vlans[i] & (1U << j)) { - r = vhost_vdpa_net_load_single_vlan(s, n, (i << 5) + j); + r = vhost_vdpa_net_load_single_vlan(s, n, out_cursor, + in_cursor, (i << 5) + j); if (unlikely(r != 0)) { return r; } @@ -1040,6 +1077,7 @@ static int vhost_vdpa_net_cvq_load(NetClientState *nc) struct vhost_vdpa *v = &s->vhost_vdpa; const VirtIONet *n; int r; + struct iovec out_cursor, in_cursor; assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA); @@ -1047,23 +1085,24 @@ static int vhost_vdpa_net_cvq_load(NetClientState *nc) if (v->shadow_vqs_enabled) { n = VIRTIO_NET(v->dev->vdev); - r = vhost_vdpa_net_load_mac(s, n); + vhost_vdpa_net_load_cursor_reset(s, &out_cursor, &in_cursor); + r = vhost_vdpa_net_load_mac(s, n, &out_cursor, &in_cursor); if (unlikely(r < 0)) { return r; } - r = vhost_vdpa_net_load_mq(s, n); + r = vhost_vdpa_net_load_mq(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_offloads(s, n); + r = vhost_vdpa_net_load_offloads(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_rx(s, n); + r = vhost_vdpa_net_load_rx(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } - r = vhost_vdpa_net_load_vlan(s, n); + r = vhost_vdpa_net_load_vlan(s, n, &out_cursor, &in_cursor); if (unlikely(r)) { return r; } From patchwork Fri Oct 13 08:09:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1848031 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=ZKi8gfiF; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6KDb2cfDz1yqj for ; Fri, 13 Oct 2023 19:19:43 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFl-0000H8-Br; Fri, 13 Oct 2023 04:10:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFb-00007B-Fu for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:30 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFZ-0001rE-A1 for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:27 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-690f7d73a3aso1657916b3a.0 for ; Fri, 13 Oct 2023 01:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184623; x=1697789423; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tI+lfuziwEyYFXJ3qwkvyQxuBIsy3m5tpBIVpjJo/cg=; b=ZKi8gfiFhbwZExOCDsN8srxrcoT4J4VrUXkVzF5smkjNGR8Vzkwt+Bflx3Irsyh+wY RiiPRS4WhDRZeKqP/LvjZdWqA41Yn9kTCSKy3J7+BVG2eOjszocse/4k0IAsDXEBpaDk dICuqsnmx4SD9c3MiwkRoCGO7dRycebcZnzVlMB09qxbsoDqiNrAWLqrKnxbbTznNY/2 hr1Yi5jH3igLF3eEnaD2BP9dlkf4tLbK71eJ3UrXtYgLvhTqb/tNL41At9nMjby0s+rE lNbgvTNxw30nldXdbhv3got4yEgg+hS2GOaAlIvWJtg6q4t/L1j8L1wDBik+XwYbmAMu 2/jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184623; x=1697789423; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tI+lfuziwEyYFXJ3qwkvyQxuBIsy3m5tpBIVpjJo/cg=; b=hY6EcltXJ8bpudqlwi+O6BciYq7pBfKLMrJfZx8EJldfGapEAT1q+UkBE/gIWFFSpo JXEAlD2rZPmwpeB9WrfgRjKCbZtUaHwerfQiqp4m5y4oEl5eNzppGyPh72Gepni+wtP4 FdWMEdvV9HHpS4Xf2sAN56saDS/xDlJBeR7xhGinXHfdr3uegh4c6tMaRjpgxn+LTKuI CyAviJz2WoNGG2NESPZLzXBZhSVxVauPd/s9l3NEYDseW+1HwTkPNYGwltQF72ftqsuH 7Xj/kAZNVRsNXfW9VqqHvczgNa6NcNaMJQAYjBRb/tSadh6pPu7N+qH6IkAeY59NBWlj ASYA== X-Gm-Message-State: AOJu0YxNk14RRFZUK3TK5GeKDWnzcEKuiedLF78OdrJAvRxouTKiO31H dRsnrnnyn18O6aRB1i9TDs8= X-Google-Smtp-Source: AGHT+IG0nBr72JaEbCJmWX+Yorqog/m5Nc6Fh6Mv3+ggkSQ0pxrB0y3GFSSrG/TsyDxOsOC9TxGjxw== X-Received: by 2002:a05:6a00:1912:b0:690:d2b0:1b37 with SMTP id y18-20020a056a00191200b00690d2b01b37mr33179132pfi.10.1697184622786; Fri, 13 Oct 2023 01:10:22 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id k64-20020a632443000000b0057c44503835sm640219pgk.65.2023.10.13.01.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:10:22 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 6/7] vhost: Expose vhost_svq_available_slots() Date: Fri, 13 Oct 2023 16:09:41 +0800 Message-Id: <25938079f0bd8185fd664c64e205e629f7a966be.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=yin31149@gmail.com; helo=mail-pf1-x429.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Next patches in this series will delay the polling and checking of buffers until either the SVQ is full or control commands shadow buffers are full, no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. To achieve this, this patch exposes vhost_svq_available_slots(), allowing QEMU to know whether the SVQ is full. Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- v5: - inline the vhost_svq_available_slots() in the caller and remove the helper function from this patch suggested by Eugenio v4: https://lore.kernel.org/all/13b3a36cc33c443a47525957ea38e80594d90595.1693287885.git.yin31149@gmail.com/ hw/virtio/vhost-shadow-virtqueue.c | 2 +- hw/virtio/vhost-shadow-virtqueue.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-virtqueue.c index e731b1d2ea..fc5f408f77 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -66,7 +66,7 @@ bool vhost_svq_valid_features(uint64_t features, Error **errp) * * @svq: The svq */ -static uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) +uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq) { return svq->num_free; } diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index 5bce67837b..19c842a15b 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -114,6 +114,7 @@ typedef struct VhostShadowVirtqueue { bool vhost_svq_valid_features(uint64_t features, Error **errp); +uint16_t vhost_svq_available_slots(const VhostShadowVirtqueue *svq); void vhost_svq_push_elem(VhostShadowVirtqueue *svq, const VirtQueueElement *elem, uint32_t len); int vhost_svq_add(VhostShadowVirtqueue *svq, const struct iovec *out_sg, From patchwork Fri Oct 13 08:09:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1847987 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=JGrrtMAi; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4S6K3P40xlz1yyb for ; Fri, 13 Oct 2023 19:11:45 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrDFo-0000Nh-W9; Fri, 13 Oct 2023 04:10:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrDFn-0000LM-8R for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:39 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrDFd-0001uD-Id for qemu-devel@nongnu.org; Fri, 13 Oct 2023 04:10:38 -0400 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1c9b95943beso15818935ad.1 for ; Fri, 13 Oct 2023 01:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1697184628; x=1697789428; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OaRHVh+YUDmOgYXrkkTq1bhMOwcUQ/jKkeQOpSeohxY=; b=JGrrtMAibKeA+FVIryUj9iPQjmDmtk/kiRgo6LJjiMTO/yHGgcMY6RKOymbsOM48WW k4kfWpczIamNOQ7ird5cKlnzqYsCO3P6itQ7XZ6U1wE5/0gUy4ypriOi/m5lSCciZXXZ Urwp9koRuw6XGsQ2O61K29RJGFZJgm8eGTHFqv2djEDVWrcd6nQY9f13YH49tnbdEBSP okehHqjM3v8bFdPOfEy1bWfK0S699aT80JawE8DnzylECDOISPf7SogiidHRrSk3cYOQ 6HyupR3pRUC1HPpLSkA1iPzvgdi7c9W7UOVg0zCxT0tvgYgk2frpiY57WVNKCtxi+wBe U2Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697184628; x=1697789428; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OaRHVh+YUDmOgYXrkkTq1bhMOwcUQ/jKkeQOpSeohxY=; b=S8AN7NMn1hv3cg9kVcTryAkvUef36CqG7jvbanCQ2J4nmjTJeQwxqdX+RmxUfXsIzb RsXgBpbGpgI9IUcxwGqqMM1rH/1T1C7R4GgeAgRsbMvQE0lnIXFrZWJnidIbv/gT6IPt BSOCk9tQQ0HFleQ5HcbIrYpkfLbJR9HyQhA9DHpNVnJoSbYnqRZwpyUpsb1auL+1YSF6 Uy05ITPtCUDh2E0M7VYDx5UFZgiYw3N4KZT7VQ6m4AztpczlI0LakfeQBgcC3NUOPju6 jFhKr8BMhe8DNKjoQCw52Q6sJSLNejF7qf8bxXcUWhdDvt5gwkmfrgx7oT4pRiwwvcwq Qjow== X-Gm-Message-State: AOJu0YwDmP9Y7O+31BIHjUVqDafOufYgr3BSg7cNgssFhHW/CZJHnse4 mEeTzatqW70hVjQuobceHUmk97UhpcUWUi4s X-Google-Smtp-Source: AGHT+IEP3nQtWc4GiDAvPXFwFQu5Kk+DUI0tp7utbHO8zYckbebo+shVHzz9rLw82r7zqRCNaQYorw== X-Received: by 2002:a17:903:1c8:b0:1c8:a63a:2087 with SMTP id e8-20020a17090301c800b001c8a63a2087mr16816701plh.65.1697184628062; Fri, 13 Oct 2023 01:10:28 -0700 (PDT) Received: from localhost ([183.242.254.166]) by smtp.gmail.com with ESMTPSA id u9-20020a17090341c900b001c631236505sm3255497ple.228.2023.10.13.01.10.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 01:10:27 -0700 (PDT) From: Hawkins Jiawei To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com Cc: qemu-devel@nongnu.org, yin31149@gmail.com, 18801353760@163.com Subject: [PATCH v5 7/7] vdpa: Send cvq state load commands in parallel Date: Fri, 13 Oct 2023 16:09:42 +0800 Message-Id: <9350f32278e39f7bce297b8f2d82dac27c6f8c9a.1697165821.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=yin31149@gmail.com; helo=mail-pl1-x630.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch enables sending CVQ state load commands in parallel at device startup by following steps: * Refactor vhost_vdpa_net_load_cmd() to iterate through the control commands shadow buffers. This allows different CVQ state load commands to use their own unique buffers. * Delay the polling and checking of buffers until either the SVQ is full or control commands shadow buffers are full. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1578 Signed-off-by: Hawkins Jiawei Acked-by: Eugenio Pérez --- v5: - remove the assertion suggested by Eugenio - inline the vhost_svq_available_slots() in vhost_vdpa_net_load_cmd() suggested by Eugenio - fix conflicts with master branch v4: https://lore.kernel.org/all/f25fea0b0aed78bad2dd5744a4cc5538243672e6.1693287885.git.yin31149@gmail.com/ - refactor argument `cmds_in_flight` to `len` for vhost_vdpa_net_svq_full() - check the return value of vhost_vdpa_net_svq_poll() in vhost_vdpa_net_svq_flush() suggested by Eugenio - use iov_size(), vhost_vdpa_net_load_cursor_reset() and iov_discard_front() to update the cursors instead of accessing it directly according to Eugenio net/vhost-vdpa.c | 165 +++++++++++++++++++++++++++++------------------ 1 file changed, 102 insertions(+), 63 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index ef4d242811..4b7c3b81b8 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -661,6 +661,31 @@ static void vhost_vdpa_net_load_cursor_reset(VhostVDPAState *s, in_cursor->iov_len = vhost_vdpa_net_cvq_cmd_page_len(); } +/* + * Poll SVQ for multiple pending control commands and check the device's ack. + * + * Caller should hold the BQL when invoking this function. + * + * @s: The VhostVDPAState + * @len: The length of the pending status shadow buffer + */ +static ssize_t vhost_vdpa_net_svq_flush(VhostVDPAState *s, size_t len) +{ + /* device uses a one-byte length ack for each control command */ + ssize_t dev_written = vhost_vdpa_net_svq_poll(s, len); + if (unlikely(dev_written != len)) { + return -EIO; + } + + /* check the device's ack */ + for (int i = 0; i < len; ++i) { + if (s->status[i] != VIRTIO_NET_OK) { + return -EIO; + } + } + return 0; +} + static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, struct iovec *out_cursor, struct iovec *in_cursor, uint8_t class, @@ -671,11 +696,31 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, .class = class, .cmd = cmd, }; - size_t data_size = iov_size(data_sg, data_num); + size_t data_size = iov_size(data_sg, data_num), cmd_size; struct iovec out, in; ssize_t r; + unsigned dummy_cursor_iov_cnt; + VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); + cmd_size = sizeof(ctrl) + data_size; + if (vhost_svq_available_slots(svq) < 2 || + iov_size(out_cursor, 1) < cmd_size) { + /* + * It is time to flush all pending control commands if SVQ is full + * or control commands shadow buffers are full. + * + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + r = vhost_vdpa_net_svq_flush(s, in_cursor->iov_base - + (void *)s->status); + if (unlikely(r < 0)) { + return r; + } + + vhost_vdpa_net_load_cursor_reset(s, out_cursor, in_cursor); + } /* pack the CVQ command header */ iov_from_buf(out_cursor, 1, 0, &ctrl, sizeof(ctrl)); @@ -684,7 +729,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, out_cursor->iov_base + sizeof(ctrl), data_size); /* extract the required buffer from the cursor for output */ - iov_copy(&out, 1, out_cursor, 1, 0, sizeof(ctrl) + data_size); + iov_copy(&out, 1, out_cursor, 1, 0, cmd_size); /* extract the required buffer from the cursor for input */ iov_copy(&in, 1, in_cursor, 1, 0, sizeof(*s->status)); @@ -693,11 +738,13 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, return r; } - /* - * We can poll here since we've had BQL from the time - * we sent the descriptor. - */ - return vhost_vdpa_net_svq_poll(s, 1); + /* iterate the cursors */ + dummy_cursor_iov_cnt = 1; + iov_discard_front(&out_cursor, &dummy_cursor_iov_cnt, cmd_size); + dummy_cursor_iov_cnt = 1; + iov_discard_front(&in_cursor, &dummy_cursor_iov_cnt, sizeof(*s->status)); + + return 0; } static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, @@ -709,15 +756,12 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, .iov_base = (void *)n->mac, .iov_len = sizeof(n->mac), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_MAC, - VIRTIO_NET_CTRL_MAC_ADDR_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MAC, + VIRTIO_NET_CTRL_MAC_ADDR_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } } @@ -762,15 +806,12 @@ static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, .iov_len = mul_macs_size, }, }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_MAC, - VIRTIO_NET_CTRL_MAC_TABLE_SET, - data, ARRAY_SIZE(data)); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MAC, + VIRTIO_NET_CTRL_MAC_TABLE_SET, + data, ARRAY_SIZE(data)); + if (unlikely(r < 0)) { + return r; } return 0; @@ -782,7 +823,7 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, struct iovec *in_cursor) { struct virtio_net_ctrl_mq mq; - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_MQ)) { return 0; @@ -793,15 +834,12 @@ static int vhost_vdpa_net_load_mq(VhostVDPAState *s, .iov_base = &mq, .iov_len = sizeof(mq), }; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_MQ, - VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_MQ, + VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -813,7 +851,7 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, struct iovec *in_cursor) { uint64_t offloads; - ssize_t dev_written; + ssize_t r; if (!virtio_vdev_has_feature(&n->parent_obj, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)) { @@ -841,15 +879,12 @@ static int vhost_vdpa_net_load_offloads(VhostVDPAState *s, .iov_base = &offloads, .iov_len = sizeof(offloads), }; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_GUEST_OFFLOADS, - VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_GUEST_OFFLOADS, + VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -865,16 +900,12 @@ static int vhost_vdpa_net_load_rx_mode(VhostVDPAState *s, .iov_base = &on, .iov_len = sizeof(on), }; - ssize_t dev_written; + ssize_t r; - dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_RX, - cmd, &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (*s->status != VIRTIO_NET_OK) { - return -EIO; + r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_RX, cmd, &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -1031,15 +1062,12 @@ static int vhost_vdpa_net_load_single_vlan(VhostVDPAState *s, .iov_base = &vid, .iov_len = sizeof(vid), }; - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, - VIRTIO_NET_CTRL_VLAN, - VIRTIO_NET_CTRL_VLAN_ADD, - &data, 1); - if (unlikely(dev_written < 0)) { - return dev_written; - } - if (unlikely(*s->status != VIRTIO_NET_OK)) { - return -EIO; + ssize_t r = vhost_vdpa_net_load_cmd(s, out_cursor, in_cursor, + VIRTIO_NET_CTRL_VLAN, + VIRTIO_NET_CTRL_VLAN_ADD, + &data, 1); + if (unlikely(r < 0)) { + return r; } return 0; @@ -1106,6 +1134,17 @@ static int vhost_vdpa_net_cvq_load(NetClientState *nc) if (unlikely(r)) { return r; } + + /* + * We need to poll and check all pending device's used buffers. + * + * We can poll here since we've had BQL from the time + * we sent the descriptor. + */ + r = vhost_vdpa_net_svq_flush(s, in_cursor.iov_base - (void *)s->status); + if (unlikely(r)) { + return r; + } } for (int i = 0; i < v->dev->vq_index; ++i) {