From patchwork Wed Jan 5 08:19:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 1575534 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MtaUq4c6; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JTMqF5HGJz9sRR for ; Wed, 5 Jan 2022 19:20:09 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 22AAA82E4C; Wed, 5 Jan 2022 08:20:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sFs7IaoSFGE1; Wed, 5 Jan 2022 08:20:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 83B2082CCB; Wed, 5 Jan 2022 08:20:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 52BF0C0038; Wed, 5 Jan 2022 08:20:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0B301C001E for ; Wed, 5 Jan 2022 08:20:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1B3A882C21 for ; Wed, 5 Jan 2022 08:20:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PgNzhE1fCsID for ; Wed, 5 Jan 2022 08:20:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 083B6813A2 for ; Wed, 5 Jan 2022 08:19:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641370793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fuH8EL+GlHpSJQ+iH5jJZEMrHaoNSR67/AIkwRQA/Yc=; b=MtaUq4c69YYlIskDuBrLcAVEI1kyKuLHFImtSghswsDxjLO/7DDfBBV+SjIOSuwQUSuL2o NC/1ev2bBB4z1qPqAk6g83oiS4/d+/myo3uGFFUw7EEuCFuxoQtOydgfx1H3xw180h8Jki zapgjld/5JBRjxxheXkdusn9G2pn/Ec= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-4epCvgl7PbqeMwzTgEnCxQ-1; Wed, 05 Jan 2022 03:19:48 -0500 X-MC-Unique: 4epCvgl7PbqeMwzTgEnCxQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F35E3835E42; Wed, 5 Jan 2022 08:19:46 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9748B16A51; Wed, 5 Jan 2022 08:19:44 +0000 (UTC) From: Maxime Coquelin To: dev@openvswitch.org Date: Wed, 5 Jan 2022 09:19:22 +0100 Message-Id: <20220105081926.613684-2-maxime.coquelin@redhat.com> In-Reply-To: <20220105081926.613684-1-maxime.coquelin@redhat.com> References: <20220105081926.613684-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, Maxime Coquelin , i.maximets@ovn.org, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v5 1/5] netdev-dpdk: Introduce per rxq/txq Vhost-user statistics. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Hash-based Tx steering feature will enable steering Tx packets on transmit queues based on their hashes. In order to test the feature, it is needed to be able to get the per-queue statistics for Vhost-user ports. This patch introduces "bytes", "packets" and "error" per-queue custom statistics for Vhost-user ports. Suggested-by David Marchand Signed-off-by: Maxime Coquelin Reviewed-by: David Marchand --- lib/netdev-dpdk.c | 147 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 138 insertions(+), 9 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 6782d3e8f..6d301cd2e 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -192,6 +192,13 @@ static const struct rte_vhost_device_ops virtio_net_device_ops = .guest_notified = vhost_guest_notified, }; +/* Custom software per-queue stats for vhost ports */ +struct netdev_dpdk_vhost_q_stats { + uint64_t bytes; + uint64_t packets; + uint64_t errors; +}; + /* Custom software stats for dpdk ports */ struct netdev_dpdk_sw_stats { /* No. of retries when unable to transmit. */ @@ -479,9 +486,10 @@ struct netdev_dpdk { PADDED_MEMBERS(CACHE_LINE_SIZE, struct netdev_stats stats; struct netdev_dpdk_sw_stats *sw_stats; + struct netdev_dpdk_vhost_q_stats *vhost_txq_stats; + struct netdev_dpdk_vhost_q_stats *vhost_rxq_stats; /* Protects stats */ rte_spinlock_t stats_lock; - /* 36 pad bytes here. */ ); PADDED_MEMBERS(CACHE_LINE_SIZE, @@ -1276,6 +1284,13 @@ common_construct(struct netdev *netdev, dpdk_port_t port_no, dev->sw_stats = xzalloc(sizeof *dev->sw_stats); dev->sw_stats->tx_retries = (dev->type == DPDK_DEV_VHOST) ? 0 : UINT64_MAX; + if (dev->type == DPDK_DEV_VHOST) { + dev->vhost_txq_stats = xcalloc(netdev->n_txq, + sizeof *dev->vhost_txq_stats); + dev->vhost_rxq_stats = xcalloc(netdev->n_rxq, + sizeof *dev->vhost_rxq_stats); + } + return 0; } @@ -2354,17 +2369,21 @@ netdev_dpdk_vhost_update_rx_size_counters(struct netdev_stats *stats, } static inline void -netdev_dpdk_vhost_update_rx_counters(struct netdev_dpdk *dev, +netdev_dpdk_vhost_update_rx_counters(struct netdev_dpdk *dev, int qid, struct dp_packet **packets, int count, int qos_drops) { + struct netdev_dpdk_vhost_q_stats *q_stats = &dev->vhost_rxq_stats[qid]; struct netdev_stats *stats = &dev->stats; struct dp_packet *packet; unsigned int packet_size; int i; stats->rx_packets += count; + q_stats->packets += count; stats->rx_dropped += qos_drops; + q_stats->errors += qos_drops; + for (i = 0; i < count; i++) { packet = packets[i]; packet_size = dp_packet_size(packet); @@ -2375,6 +2394,7 @@ netdev_dpdk_vhost_update_rx_counters(struct netdev_dpdk *dev, * further processing. */ stats->rx_errors++; stats->rx_length_errors++; + q_stats->errors++; continue; } @@ -2386,6 +2406,7 @@ netdev_dpdk_vhost_update_rx_counters(struct netdev_dpdk *dev, } stats->rx_bytes += packet_size; + q_stats->bytes += packet_size; } if (OVS_UNLIKELY(qos_drops)) { @@ -2438,7 +2459,7 @@ netdev_dpdk_vhost_rxq_recv(struct netdev_rxq *rxq, } rte_spinlock_lock(&dev->stats_lock); - netdev_dpdk_vhost_update_rx_counters(dev, batch->packets, + netdev_dpdk_vhost_update_rx_counters(dev, rxq->queue_id, batch->packets, nb_rx, qos_drops); rte_spinlock_unlock(&dev->stats_lock); @@ -2552,11 +2573,12 @@ netdev_dpdk_filter_packet_len(struct netdev_dpdk *dev, struct rte_mbuf **pkts, } static inline void -netdev_dpdk_vhost_update_tx_counters(struct netdev_dpdk *dev, +netdev_dpdk_vhost_update_tx_counters(struct netdev_dpdk *dev, int qid, struct dp_packet **packets, int attempted, struct netdev_dpdk_sw_stats *sw_stats_add) { + struct netdev_dpdk_vhost_q_stats *q_stats = &dev->vhost_txq_stats[qid]; int dropped = sw_stats_add->tx_mtu_exceeded_drops + sw_stats_add->tx_qos_drops + sw_stats_add->tx_failure_drops + @@ -2566,10 +2588,15 @@ netdev_dpdk_vhost_update_tx_counters(struct netdev_dpdk *dev, int i; stats->tx_packets += sent; + q_stats->packets += sent; stats->tx_dropped += dropped; + q_stats->errors += dropped; for (i = 0; i < sent; i++) { - stats->tx_bytes += dp_packet_size(packets[i]); + uint64_t bytes = dp_packet_size(packets[i]); + + stats->tx_bytes += bytes; + q_stats->bytes += bytes; } if (OVS_UNLIKELY(dropped || sw_stats_add->tx_retries)) { @@ -2657,7 +2684,7 @@ __netdev_dpdk_vhost_send(struct netdev *netdev, int qid, sw_stats_add.tx_retries = MIN(retries, max_retries); rte_spinlock_lock(&dev->stats_lock); - netdev_dpdk_vhost_update_tx_counters(dev, pkts, total_packets, + netdev_dpdk_vhost_update_tx_counters(dev, qid, pkts, total_packets, &sw_stats_add); rte_spinlock_unlock(&dev->stats_lock); @@ -3287,6 +3314,76 @@ netdev_dpdk_get_sw_custom_stats(const struct netdev *netdev, return 0; } +static int +netdev_dpdk_vhost_get_custom_stats(const struct netdev *netdev, + struct netdev_custom_stats *custom_stats) +{ + struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); + int sw_stats_size, i, j; + + netdev_dpdk_get_sw_custom_stats(netdev, custom_stats); + + ovs_mutex_lock(&dev->mutex); + +#define VHOST_Q_STATS \ + VHOST_Q_STAT(bytes) \ + VHOST_Q_STAT(packets) \ + VHOST_Q_STAT(errors) + + sw_stats_size = custom_stats->size; +#define VHOST_Q_STAT(NAME) + netdev->n_rxq + custom_stats->size += VHOST_Q_STATS; +#undef VHOST_Q_STAT +#define VHOST_Q_STAT(NAME) + netdev->n_txq + custom_stats->size += VHOST_Q_STATS; +#undef VHOST_Q_STAT + custom_stats->counters = xrealloc(custom_stats->counters, + custom_stats->size * + sizeof *custom_stats->counters); + + j = 0; + for (i = 0; i < netdev->n_rxq; i++) { +#define VHOST_Q_STAT(NAME) \ + snprintf(custom_stats->counters[sw_stats_size + j++].name, \ + NETDEV_CUSTOM_STATS_NAME_SIZE, "rx_q%d_"#NAME, i); + VHOST_Q_STATS +#undef VHOST_Q_STAT + } + + for (i = 0; i < netdev->n_txq; i++) { +#define VHOST_Q_STAT(NAME) \ + snprintf(custom_stats->counters[sw_stats_size + j++].name, \ + NETDEV_CUSTOM_STATS_NAME_SIZE, "tx_q%d_"#NAME, i); + VHOST_Q_STATS +#undef VHOST_Q_STAT + } + + rte_spinlock_lock(&dev->stats_lock); + + j = 0; + for (i = 0; i < netdev->n_rxq; i++) { +#define VHOST_Q_STAT(NAME) \ + custom_stats->counters[sw_stats_size + j++].value = \ + dev->vhost_rxq_stats[i].NAME; + VHOST_Q_STATS +#undef VHOST_Q_STAT + } + + for (i = 0; i < netdev->n_txq; i++) { +#define VHOST_Q_STAT(NAME) \ + custom_stats->counters[sw_stats_size + j++].value = \ + dev->vhost_txq_stats[i].NAME; + VHOST_Q_STATS +#undef VHOST_Q_STAT + } + + rte_spinlock_unlock(&dev->stats_lock); + + ovs_mutex_unlock(&dev->mutex); + + return 0; +} + static int netdev_dpdk_get_features(const struct netdev *netdev, enum netdev_features *current, @@ -3556,6 +3653,11 @@ netdev_dpdk_update_flags__(struct netdev_dpdk *dev, if (NETDEV_UP & on) { rte_spinlock_lock(&dev->stats_lock); memset(&dev->stats, 0, sizeof dev->stats); + memset(dev->sw_stats, 0, sizeof *dev->sw_stats); + memset(dev->vhost_rxq_stats, 0, + dev->up.n_rxq * sizeof *dev->vhost_rxq_stats); + memset(dev->vhost_txq_stats, 0, + dev->up.n_txq * sizeof *dev->vhost_txq_stats); rte_spinlock_unlock(&dev->stats_lock); } } @@ -5048,9 +5150,12 @@ static int dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) OVS_REQUIRES(dev->mutex) { + int old_n_txq = dev->up.n_txq; + int old_n_rxq = dev->up.n_rxq; + int err; + dev->up.n_txq = dev->requested_n_txq; dev->up.n_rxq = dev->requested_n_rxq; - int err; /* Always keep RX queue 0 enabled for implementations that won't * report vring states. */ @@ -5068,6 +5173,30 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) netdev_dpdk_remap_txqs(dev); + /* Reset all stats if number of queues changed. */ + if (dev->up.n_txq != old_n_txq || dev->up.n_rxq != old_n_rxq) { + struct netdev_dpdk_vhost_q_stats *old_txq_stats, *new_txq_stats; + struct netdev_dpdk_vhost_q_stats *old_rxq_stats, *new_rxq_stats; + + new_txq_stats = xcalloc(dev->up.n_txq, sizeof *dev->vhost_txq_stats); + new_rxq_stats = xcalloc(dev->up.n_rxq, sizeof *dev->vhost_rxq_stats); + + rte_spinlock_lock(&dev->stats_lock); + + memset(&dev->stats, 0, sizeof dev->stats); + memset(dev->sw_stats, 0, sizeof *dev->sw_stats); + + old_txq_stats = dev->vhost_txq_stats; + dev->vhost_txq_stats = new_txq_stats; + old_rxq_stats = dev->vhost_rxq_stats; + dev->vhost_rxq_stats = new_rxq_stats; + + rte_spinlock_unlock(&dev->stats_lock); + + free(old_txq_stats); + free(old_rxq_stats); + } + err = netdev_dpdk_mempool_configure(dev); if (!err) { /* A new mempool was created or re-used. */ @@ -5473,7 +5602,7 @@ static const struct netdev_class dpdk_vhost_class = { .send = netdev_dpdk_vhost_send, .get_carrier = netdev_dpdk_vhost_get_carrier, .get_stats = netdev_dpdk_vhost_get_stats, - .get_custom_stats = netdev_dpdk_get_sw_custom_stats, + .get_custom_stats = netdev_dpdk_vhost_get_custom_stats, .get_status = netdev_dpdk_vhost_user_get_status, .reconfigure = netdev_dpdk_vhost_reconfigure, .rxq_recv = netdev_dpdk_vhost_rxq_recv, @@ -5489,7 +5618,7 @@ static const struct netdev_class dpdk_vhost_client_class = { .send = netdev_dpdk_vhost_send, .get_carrier = netdev_dpdk_vhost_get_carrier, .get_stats = netdev_dpdk_vhost_get_stats, - .get_custom_stats = netdev_dpdk_get_sw_custom_stats, + .get_custom_stats = netdev_dpdk_vhost_get_custom_stats, .get_status = netdev_dpdk_vhost_user_get_status, .reconfigure = netdev_dpdk_vhost_client_reconfigure, .rxq_recv = netdev_dpdk_vhost_rxq_recv, From patchwork Wed Jan 5 08:19:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 1575532 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ZcRgkaya; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JTMq56KG6z9sRR for ; Wed, 5 Jan 2022 19:20:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D8FC782C7C; Wed, 5 Jan 2022 08:19:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K8gxMQB3KI0s; Wed, 5 Jan 2022 08:19:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4D8E982C21; Wed, 5 Jan 2022 08:19:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8CE71C0078; Wed, 5 Jan 2022 08:19:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BCCCC0030 for ; Wed, 5 Jan 2022 08:19:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 078154053C for ; Wed, 5 Jan 2022 08:19:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ft5Auf7g_ozx for ; Wed, 5 Jan 2022 08:19:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id D3D59400A6 for ; Wed, 5 Jan 2022 08:19:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641370792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kFZyDzj2ydjd+mXvL9t8eIIbSX/vu4K1Hy3p1w0dx5Q=; b=ZcRgkayamiLcc9s+84AYIFjr29Md6FXPvha3McrgtQlSpFGhAyc7y136Zc56zYJhD1zseO xTwHsn1d21TvdhOrtYDcPAL6FTpi+WHaJ2zQc0zL7ekDeYrRp9aesYMdmuGTMknSj+hUNS flJi0h/1zi/U9iAnY/MgaKgD1dlrvBc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-438-SgdJjGEoNU2Wz-oBay_bPw-1; Wed, 05 Jan 2022 03:19:49 -0500 X-MC-Unique: SgdJjGEoNU2Wz-oBay_bPw-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CC3B0100CCC0; Wed, 5 Jan 2022 08:19:48 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2084216A4D; Wed, 5 Jan 2022 08:19:46 +0000 (UTC) From: Maxime Coquelin To: dev@openvswitch.org Date: Wed, 5 Jan 2022 09:19:23 +0100 Message-Id: <20220105081926.613684-3-maxime.coquelin@redhat.com> In-Reply-To: <20220105081926.613684-1-maxime.coquelin@redhat.com> References: <20220105081926.613684-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, Maxime Coquelin , i.maximets@ovn.org, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v5 2/5] netdev-dummy: Introduce per rxq/txq statistics. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds Rx and Tx per-queue statistics. It will be used to test hash-based Tx packet steering. Only "bytes", and "packets" per-queue custom statistics are added, as there are no global "errors" counters in netdev-dummy. Signed-off-by: Maxime Coquelin Reviewed-by: David Marchand --- lib/netdev-dummy.c | 87 +++++++++++++++++++++++++++++++++++++++++----- tests/ofproto.at | 3 +- 2 files changed, 80 insertions(+), 10 deletions(-) diff --git a/lib/netdev-dummy.c b/lib/netdev-dummy.c index 1f386b81b..65d57402d 100644 --- a/lib/netdev-dummy.c +++ b/lib/netdev-dummy.c @@ -101,6 +101,11 @@ struct offloaded_flow { uint32_t mark; }; +struct netdev_dummy_q_stats { + uint64_t bytes; + uint64_t packets; +}; + /* Protects 'dummy_list'. */ static struct ovs_mutex dummy_list_mutex = OVS_MUTEX_INITIALIZER; @@ -121,6 +126,8 @@ struct netdev_dummy { int mtu OVS_GUARDED; struct netdev_stats stats OVS_GUARDED; struct netdev_custom_counter custom_stats[C_STATS_SIZE] OVS_GUARDED; + struct netdev_dummy_q_stats *rxq_stats OVS_GUARDED; + struct netdev_dummy_q_stats *txq_stats OVS_GUARDED; enum netdev_flags flags OVS_GUARDED; int ifindex OVS_GUARDED; int numa_id OVS_GUARDED; @@ -707,6 +714,9 @@ netdev_dummy_construct(struct netdev *netdev_) ovs_strlcpy(netdev->custom_stats[1].name, "rx_custom_packets_2", NETDEV_CUSTOM_STATS_NAME_SIZE); + netdev->rxq_stats = xcalloc(netdev->up.n_rxq, sizeof *netdev->rxq_stats); + netdev->txq_stats = xcalloc(netdev->up.n_rxq, sizeof *netdev->txq_stats); + dummy_packet_conn_init(&netdev->conn); ovs_list_init(&netdev->rxes); @@ -731,6 +741,8 @@ netdev_dummy_destruct(struct netdev *netdev_) ovs_mutex_unlock(&dummy_list_mutex); ovs_mutex_lock(&netdev->mutex); + free(netdev->rxq_stats); + free(netdev->txq_stats); if (netdev->rxq_pcap) { ovs_pcap_close(netdev->rxq_pcap); } @@ -954,6 +966,8 @@ static int netdev_dummy_reconfigure(struct netdev *netdev_) { struct netdev_dummy *netdev = netdev_dummy_cast(netdev_); + int old_n_txq = netdev_->n_txq; + int old_n_rxq = netdev_->n_rxq; ovs_mutex_lock(&netdev->mutex); @@ -961,6 +975,26 @@ netdev_dummy_reconfigure(struct netdev *netdev_) netdev_->n_rxq = netdev->requested_n_rxq; netdev->numa_id = netdev->requested_numa_id; + if (netdev_->n_txq != old_n_txq || netdev_->n_rxq != old_n_rxq) { + /* Resize the per queue stats arrays. */ + netdev->txq_stats = xrealloc(netdev->txq_stats, + netdev_->n_txq * + sizeof *netdev->txq_stats); + netdev->rxq_stats = xrealloc(netdev->rxq_stats, + netdev_->n_rxq * + sizeof *netdev->rxq_stats); + + /* Reset all stats for consistency between per-queue and global + * counters */ + memset(&netdev->stats, 0, sizeof netdev->stats); + netdev->custom_stats[0].value = 0; + netdev->custom_stats[1].value = 0; + memset(netdev->txq_stats, 0, + netdev_->n_txq * sizeof *netdev->txq_stats); + memset(netdev->rxq_stats, 0, + netdev_->n_rxq * sizeof *netdev->rxq_stats); + } + ovs_mutex_unlock(&netdev->mutex); return 0; } @@ -1049,7 +1083,9 @@ netdev_dummy_rxq_recv(struct netdev_rxq *rxq_, struct dp_packet_batch *batch, } ovs_mutex_lock(&netdev->mutex); netdev->stats.rx_packets++; + netdev->rxq_stats[rxq_->queue_id].packets++; netdev->stats.rx_bytes += dp_packet_size(packet); + netdev->rxq_stats[rxq_->queue_id].bytes += dp_packet_size(packet); netdev->custom_stats[0].value++; netdev->custom_stats[1].value++; ovs_mutex_unlock(&netdev->mutex); @@ -1096,7 +1132,7 @@ netdev_dummy_rxq_drain(struct netdev_rxq *rxq_) } static int -netdev_dummy_send(struct netdev *netdev, int qid OVS_UNUSED, +netdev_dummy_send(struct netdev *netdev, int qid, struct dp_packet_batch *batch, bool concurrent_txq OVS_UNUSED) { @@ -1135,7 +1171,9 @@ netdev_dummy_send(struct netdev *netdev, int qid OVS_UNUSED, ovs_mutex_lock(&dev->mutex); dev->stats.tx_packets++; + dev->txq_stats[qid].packets++; dev->stats.tx_bytes += size; + dev->txq_stats[qid].bytes += size; dummy_packet_conn_send(&dev->conn, buffer, size); @@ -1252,22 +1290,53 @@ static int netdev_dummy_get_custom_stats(const struct netdev *netdev, struct netdev_custom_stats *custom_stats) { - int i; + int i, j; struct netdev_dummy *dev = netdev_dummy_cast(netdev); - custom_stats->size = 2; - custom_stats->counters = - (struct netdev_custom_counter *) xcalloc(C_STATS_SIZE, - sizeof(struct netdev_custom_counter)); - ovs_mutex_lock(&dev->mutex); + +#define DUMMY_Q_STATS \ + DUMMY_Q_STAT(bytes) \ + DUMMY_Q_STAT(packets) + + custom_stats->size = C_STATS_SIZE; +#define DUMMY_Q_STAT(NAME) + netdev->n_rxq + custom_stats->size += DUMMY_Q_STATS; +#undef DUMMY_Q_STAT +#define DUMMY_Q_STAT(NAME) + netdev->n_txq + custom_stats->size += DUMMY_Q_STATS; +#undef DUMMY_Q_STAT + + custom_stats->counters = xcalloc(custom_stats->size, + sizeof(struct netdev_custom_counter)); + + j = 0; for (i = 0 ; i < C_STATS_SIZE ; i++) { - custom_stats->counters[i].value = dev->custom_stats[i].value; - ovs_strlcpy(custom_stats->counters[i].name, + custom_stats->counters[j].value = dev->custom_stats[i].value; + ovs_strlcpy(custom_stats->counters[j++].name, dev->custom_stats[i].name, NETDEV_CUSTOM_STATS_NAME_SIZE); } + + for (i = 0; i < netdev->n_rxq; i++) { +#define DUMMY_Q_STAT(NAME) \ + snprintf(custom_stats->counters[j].name, \ + NETDEV_CUSTOM_STATS_NAME_SIZE, "rx_q%d_"#NAME, i); \ + custom_stats->counters[j++].value = dev->rxq_stats[i].NAME; + DUMMY_Q_STATS +#undef DUMMY_Q_STAT + } + + for (i = 0; i < netdev->n_txq; i++) { +#define DUMMY_Q_STAT(NAME) \ + snprintf(custom_stats->counters[j].name, \ + NETDEV_CUSTOM_STATS_NAME_SIZE, "tx_q%d_"#NAME, i); \ + custom_stats->counters[j++].value = dev->txq_stats[i].NAME; + DUMMY_Q_STATS +#undef DUMMY_Q_STAT + } + ovs_mutex_unlock(&dev->mutex); return 0; diff --git a/tests/ofproto.at b/tests/ofproto.at index 08c0a20b6..156d3e058 100644 --- a/tests/ofproto.at +++ b/tests/ofproto.at @@ -112,7 +112,8 @@ OFPST_PORT reply (OF1.4): 1 ports tx pkts=0, bytes=0, drop=?, errs=?, coll=? duration=?s CUSTOM Statistics - rx_custom_packets_1=0, rx_custom_packets_2=0, + rx_custom_packets_1=0, rx_custom_packets_2=0, rx_q0_bytes=0, + rx_q0_packets=0, tx_q0_bytes=0, tx_q0_packets=0, ]) OVS_VSWITCHD_STOP AT_CLEANUP From patchwork Wed Jan 5 08:19:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 1575533 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D5wMuwKY; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JTMq83PBGz9sRR for ; Wed, 5 Jan 2022 19:20:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5FD1E4071B; Wed, 5 Jan 2022 08:20:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E4IphIAHwI2V; Wed, 5 Jan 2022 08:20:00 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 96523408CD; Wed, 5 Jan 2022 08:19:59 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 72A61C0030; Wed, 5 Jan 2022 08:19:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5C12FC001E for ; Wed, 5 Jan 2022 08:19:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 407F94160D for ; Wed, 5 Jan 2022 08:19:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dt_yY9F3kUoP for ; Wed, 5 Jan 2022 08:19:56 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5ADDB41607 for ; Wed, 5 Jan 2022 08:19:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641370794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cdUQxHFBYBnzKgVdiJonXAqGITHqc2wYAW3suaaofI8=; b=D5wMuwKYieiCUcr6mPKZVQ+Z6VMsaujDbTxlFTqSuJMjaLhJxdf52G2sgf/J7jCGZPNCEQ gwOISqawLd1/HwgnbtFZA9cyG2Il6/AlYGksirwZq71xv/uDZn3yR+36okV5mBQ4pF6v0B hvu5A/3dyBYafWVEyb0reqJrkFThfN8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-186-T2jIpLfDNjucNBD6Y88oPA-1; Wed, 05 Jan 2022 03:19:51 -0500 X-MC-Unique: T2jIpLfDNjucNBD6Y88oPA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E0C02100CCC2; Wed, 5 Jan 2022 08:19:50 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 271A216A51; Wed, 5 Jan 2022 08:19:48 +0000 (UTC) From: Maxime Coquelin To: dev@openvswitch.org Date: Wed, 5 Jan 2022 09:19:24 +0100 Message-Id: <20220105081926.613684-4-maxime.coquelin@redhat.com> In-Reply-To: <20220105081926.613684-1-maxime.coquelin@redhat.com> References: <20220105081926.613684-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, Maxime Coquelin , i.maximets@ovn.org, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v5 3/5] dpif-netdev: Introduce Tx queue mode. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" A boolean is currently used to differenciate between the static and XPS Tx queue modes. Since we are going to introduce a new steering mode, replace this boolean with an enum. This patch does not introduce functionnal changes. Signed-off-by: Maxime Coquelin Reviewed-by: David Marchand --- lib/dpif-netdev.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index a790df5fd..13b7d2835 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -385,10 +385,15 @@ struct dp_netdev_rxq { atomic_ullong cycles_intrvl[PMD_INTERVAL_MAX]; }; +enum txq_mode { + TXQ_MODE_STATIC, + TXQ_MODE_XPS, +}; + /* A port in a netdev-based datapath. */ struct dp_netdev_port { odp_port_t port_no; - bool dynamic_txqs; /* If true XPS will be used. */ + enum txq_mode txq_mode; /* static, XPS */ bool need_reconfigure; /* True if we should reconfigure netdev. */ struct netdev *netdev; struct hmap_node node; /* Node in dp_netdev's 'ports'. */ @@ -4591,24 +4596,25 @@ dp_netdev_pmd_flush_output_on_port(struct dp_netdev_pmd_thread *pmd, int i; int tx_qid; int output_cnt; - bool dynamic_txqs; + bool concurrent_txqs; struct cycle_timer timer; uint64_t cycles; uint32_t tx_flush_interval; cycle_timer_start(&pmd->perf_stats, &timer); - dynamic_txqs = p->port->dynamic_txqs; - if (dynamic_txqs) { + if (p->port->txq_mode == TXQ_MODE_XPS) { tx_qid = dpif_netdev_xps_get_tx_qid(pmd, p); + concurrent_txqs = true; } else { tx_qid = pmd->static_tx_qid; + concurrent_txqs = false; } output_cnt = dp_packet_batch_size(&p->output_pkts); ovs_assert(output_cnt > 0); - netdev_send(p->port->netdev, tx_qid, &p->output_pkts, dynamic_txqs); + netdev_send(p->port->netdev, tx_qid, &p->output_pkts, concurrent_txqs); dp_packet_batch_init(&p->output_pkts); /* Update time of the next flush. */ @@ -5761,14 +5767,14 @@ reconfigure_datapath(struct dp_netdev *dp) * 'port->need_reconfigure', because netdev_is_reconf_required() can * change at any time. * Also mark for reconfiguration all ports which will likely change their - * 'dynamic_txqs' parameter. It's required to stop using them before + * 'txq_mode' parameter. It's required to stop using them before * changing this setting and it's simpler to mark ports here and allow * 'pmd_remove_stale_ports' to remove them from threads. There will be * no actual reconfiguration in 'port_reconfigure' because it's * unnecessary. */ HMAP_FOR_EACH (port, node, &dp->ports) { if (netdev_is_reconf_required(port->netdev) - || (port->dynamic_txqs + || ((port->txq_mode == TXQ_MODE_XPS) != (netdev_n_txq(port->netdev) < wanted_txqs))) { port->need_reconfigure = true; } @@ -5804,7 +5810,8 @@ reconfigure_datapath(struct dp_netdev *dp) seq_change(dp->port_seq); port_destroy(port); } else { - port->dynamic_txqs = netdev_n_txq(port->netdev) < wanted_txqs; + port->txq_mode = (netdev_n_txq(port->netdev) < wanted_txqs) + ? TXQ_MODE_XPS : TXQ_MODE_STATIC; } } @@ -7812,7 +7819,7 @@ dpif_netdev_xps_revalidate_pmd(const struct dp_netdev_pmd_thread *pmd, long long interval; HMAP_FOR_EACH (tx, node, &pmd->send_port_cache) { - if (!tx->port->dynamic_txqs) { + if (tx->port->txq_mode != TXQ_MODE_XPS) { continue; } interval = pmd->ctx.now - tx->last_used; From patchwork Wed Jan 5 08:19:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 1575536 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=D4L1oIqS; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JTMqr4slGz9sRR for ; Wed, 5 Jan 2022 19:20:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BC35D82EA5; Wed, 5 Jan 2022 08:20:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kbL8YYl270xy; Wed, 5 Jan 2022 08:20:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8698282C91; Wed, 5 Jan 2022 08:20:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 591B4C0030; Wed, 5 Jan 2022 08:20:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4F949C001E for ; Wed, 5 Jan 2022 08:20:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B5C0D82E9C for ; Wed, 5 Jan 2022 08:20:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gSeN5nkRD1ks for ; Wed, 5 Jan 2022 08:20:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1F77D82C91 for ; Wed, 5 Jan 2022 08:20:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641370800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=266n8PsUdYn4m6gZFPkzJDEuvQGRQCjQ5dddNj2oSZQ=; b=D4L1oIqSCxqnPiRJeHWPycqZdDGjhPr4ZhTOJt6S63mzmroF8zxYF5T/y2vgdf3AgX+RIN 6+H6Hn3aLCI+siOUZIEKsok7m1LKBd8CJ1P/EO0GJUKA7d7BGuML1DylvCwaAHKSsODb/m zj/qNPF2J4nJUm6yraT/Az7tCFZvwEk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-345-lhQtQr2rPRqjbESFj-O2Bg-1; Wed, 05 Jan 2022 03:19:57 -0500 X-MC-Unique: lhQtQr2rPRqjbESFj-O2Bg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A4DD218B9ECA; Wed, 5 Jan 2022 08:19:56 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4538416A4D; Wed, 5 Jan 2022 08:19:51 +0000 (UTC) From: Maxime Coquelin To: dev@openvswitch.org Date: Wed, 5 Jan 2022 09:19:25 +0100 Message-Id: <20220105081926.613684-5-maxime.coquelin@redhat.com> In-Reply-To: <20220105081926.613684-1-maxime.coquelin@redhat.com> References: <20220105081926.613684-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, Maxime Coquelin , i.maximets@ovn.org, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v5 4/5] dpif-netdev: Introduce hash-based Tx packet steering mode. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch adds a new hash Tx steering mode that distributes the traffic on all the Tx queues, whatever the number of PMD threads. It would be useful for guests expecting traffic to be distributed on all the vCPUs. The idea here is to re-use the 5-tuple hash of the packets, already computed to build the flows batches (and so it does not provide flexibility on which fields are part of the hash). There are also no user-configurable indirection table, given the feature is transparent to the guest. The queue selection is just a modulo operation between the packet hash and the number of Tx queues. There are no (at least intentionnally) functionnal changes for the existing XPS and static modes. There should not be noticeable performance changes for these modes (only one more branch in the hot path). For the hash mode, performance could be impacted due to locking when multiple PMD threads are in use (same as XPS mode) and also because of the second level of batching. Regarding the batching, the existing Tx port output_pkts is not modified. It means that at maximum, NETDEV_MAX_BURST can be batched for all the Tx queues. A second level of batching is done in dp_netdev_pmd_flush_output_on_port(), only for this hash mode. Signed-off-by: Maxime Coquelin Reviewed-by: David Marchand --- Documentation/automake.mk | 1 + Documentation/topics/index.rst | 1 + .../topics/userspace-tx-steering.rst | 73 ++++++++++ lib/dpif-netdev.c | 131 +++++++++++++++--- 4 files changed, 183 insertions(+), 23 deletions(-) create mode 100644 Documentation/topics/userspace-tx-steering.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 137cc57c5..01e3c4f9e 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -58,6 +58,7 @@ DOC_SOURCE = \ Documentation/topics/record-replay.rst \ Documentation/topics/tracing.rst \ Documentation/topics/userspace-tso.rst \ + Documentation/topics/userspace-tx-steering.rst \ Documentation/topics/windows.rst \ Documentation/howto/index.rst \ Documentation/howto/dpdk.rst \ diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst index d8ccbd757..3699fd5c4 100644 --- a/Documentation/topics/index.rst +++ b/Documentation/topics/index.rst @@ -55,3 +55,4 @@ OVS userspace-tso idl-compound-indexes ovs-extensions + userspace-tx-steering diff --git a/Documentation/topics/userspace-tx-steering.rst b/Documentation/topics/userspace-tx-steering.rst new file mode 100644 index 000000000..339cfbef3 --- /dev/null +++ b/Documentation/topics/userspace-tx-steering.rst @@ -0,0 +1,73 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +============================ +Userspace Tx packet steering +============================ + +The userspace datapath supports two transmit packets steering modes. + +Thread mode +~~~~~~~~~~~ + +This mode is automatically selected when the port ``tx-steering`` option is set +to ``thread`` or unset. + +Depending on the port number of Tx queues being greater or equal than the +number of PMD threads, static txq mapping or XPS will be used. + +This is the recommended mode for performance reasons if the number of Tx queues +is greater or equal to the number of PMD threads, because the Tx lock is not +acquired. + +If the number of Tx queues is greater than the number of PMDs, the +remaining Tx queues will not be used. + +This mode is enabled by default. + +Hash mode +~~~~~~~~~ + +Hash-based Tx packets steering mode distributes the packets on all the port's +transmit queues, whatever the number of PMD threads. The queue selection is +based on the 5-tuples hash already computed to build the flows batches, the +selected queue being the modulo between the hash and the number of Tx queues of +the port. + +Hash mode may be used for example with Vhost-user ports, when the number of +vCPUs and queues of thevguest are greater than the number of PMD threads. +Without hash mode, the Tx queues used would be limited to the number of PMD. + +Hash-based Tx packet steering may have an impact on the performance, given the +Tx lock acquisition is required and a second level of batching is performed. + +Usage +~~~~~ + +To enable hash mode:: + + $ ovs-vsctl set Interface other_config:tx-steering=hash + +To disable hash mode:: + + $ ovs-vsctl set Interface other_config:tx-steering=thread diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 13b7d2835..06077a360 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -385,15 +385,21 @@ struct dp_netdev_rxq { atomic_ullong cycles_intrvl[PMD_INTERVAL_MAX]; }; +enum txq_req_mode { + TXQ_REQ_MODE_THREAD, + TXQ_REQ_MODE_HASH, +}; + enum txq_mode { TXQ_MODE_STATIC, TXQ_MODE_XPS, + TXQ_MODE_HASH, }; /* A port in a netdev-based datapath. */ struct dp_netdev_port { odp_port_t port_no; - enum txq_mode txq_mode; /* static, XPS */ + enum txq_mode txq_mode; /* static, XPS, HASH. */ bool need_reconfigure; /* True if we should reconfigure netdev. */ struct netdev *netdev; struct hmap_node node; /* Node in dp_netdev's 'ports'. */ @@ -405,6 +411,7 @@ struct dp_netdev_port { bool emc_enabled; /* If true EMC will be used. */ char *type; /* Port type as requested by user. */ char *rxq_affinity_list; /* Requested affinity of rx queues. */ + enum txq_req_mode txq_requested_mode; }; static bool dp_netdev_flow_ref(struct dp_netdev_flow *); @@ -440,6 +447,7 @@ struct tx_port { struct hmap_node node; long long flush_time; struct dp_packet_batch output_pkts; + struct dp_packet_batch *txq_pkts; /* Only for hash mode */ struct dp_netdev_rxq *output_pkts_rxqs[NETDEV_MAX_BURST]; }; @@ -4435,6 +4443,8 @@ dpif_netdev_port_set_config(struct dpif *dpif, odp_port_t port_no, int error = 0; const char *affinity_list = smap_get(cfg, "pmd-rxq-affinity"); bool emc_enabled = smap_get_bool(cfg, "emc-enable", true); + const char *tx_steering_mode = smap_get(cfg, "tx-steering"); + enum txq_req_mode txq_mode; ovs_mutex_lock(&dp->port_mutex); error = get_port_by_number(dp, port_no, &port); @@ -4476,19 +4486,33 @@ dpif_netdev_port_set_config(struct dpif *dpif, odp_port_t port_no, } /* Checking for RXq affinity changes. */ - if (!netdev_is_pmd(port->netdev) - || nullable_string_is_equal(affinity_list, port->rxq_affinity_list)) { - goto unlock; + if (netdev_is_pmd(port->netdev) + && !nullable_string_is_equal(affinity_list, port->rxq_affinity_list)) { + + error = dpif_netdev_port_set_rxq_affinity(port, affinity_list); + if (error) { + goto unlock; + } + free(port->rxq_affinity_list); + port->rxq_affinity_list = nullable_xstrdup(affinity_list); + + dp_netdev_request_reconfigure(dp); } - error = dpif_netdev_port_set_rxq_affinity(port, affinity_list); - if (error) { - goto unlock; + if (nullable_string_is_equal(tx_steering_mode, "hash")) { + txq_mode = TXQ_REQ_MODE_HASH; + } else { + txq_mode = TXQ_REQ_MODE_THREAD; + } + + if (txq_mode != port->txq_requested_mode) { + port->txq_requested_mode = txq_mode; + VLOG_INFO("%s: Txq mode has been set to %s.", + netdev_get_name(port->netdev), + (txq_mode == TXQ_REQ_MODE_THREAD) ? "thread" : "hash"); + dp_netdev_request_reconfigure(dp); } - free(port->rxq_affinity_list); - port->rxq_affinity_list = nullable_xstrdup(affinity_list); - dp_netdev_request_reconfigure(dp); unlock: ovs_mutex_unlock(&dp->port_mutex); return error; @@ -4603,18 +4627,45 @@ dp_netdev_pmd_flush_output_on_port(struct dp_netdev_pmd_thread *pmd, cycle_timer_start(&pmd->perf_stats, &timer); - if (p->port->txq_mode == TXQ_MODE_XPS) { - tx_qid = dpif_netdev_xps_get_tx_qid(pmd, p); - concurrent_txqs = true; - } else { - tx_qid = pmd->static_tx_qid; - concurrent_txqs = false; - } - output_cnt = dp_packet_batch_size(&p->output_pkts); ovs_assert(output_cnt > 0); - netdev_send(p->port->netdev, tx_qid, &p->output_pkts, concurrent_txqs); + if (p->port->txq_mode == TXQ_MODE_HASH) { + int n_txq = netdev_n_txq(p->port->netdev); + + /* Re-batch per txq based on packet hash. */ + for (i = 0; i < output_cnt; i++) { + struct dp_packet *packet = p->output_pkts.packets[i]; + struct flow flow; + uint32_t hash; + + if (OVS_LIKELY(dp_packet_rss_valid(packet))) { + hash = dp_packet_get_rss_hash(packet); + } else { + flow_extract(packet, &flow); + hash = flow_hash_5tuple(&flow, 0); + } + dp_packet_batch_add(&p->txq_pkts[hash % n_txq], packet); + } + + /* Flush batches of each Tx queues. */ + for (i = 0; i < n_txq; i++) { + if (dp_packet_batch_is_empty(&p->txq_pkts[i])) { + continue; + } + netdev_send(p->port->netdev, i, &p->txq_pkts[i], true); + dp_packet_batch_init(&p->txq_pkts[i]); + } + } else { + if (p->port->txq_mode == TXQ_MODE_XPS) { + tx_qid = dpif_netdev_xps_get_tx_qid(pmd, p); + concurrent_txqs = true; + } else { + tx_qid = pmd->static_tx_qid; + concurrent_txqs = false; + } + netdev_send(p->port->netdev, tx_qid, &p->output_pkts, concurrent_txqs); + } dp_packet_batch_init(&p->output_pkts); /* Update time of the next flush. */ @@ -5775,7 +5826,9 @@ reconfigure_datapath(struct dp_netdev *dp) HMAP_FOR_EACH (port, node, &dp->ports) { if (netdev_is_reconf_required(port->netdev) || ((port->txq_mode == TXQ_MODE_XPS) - != (netdev_n_txq(port->netdev) < wanted_txqs))) { + != (netdev_n_txq(port->netdev) < wanted_txqs)) + || ((port->txq_mode == TXQ_MODE_HASH) + != (port->txq_requested_mode == TXQ_REQ_MODE_HASH))) { port->need_reconfigure = true; } } @@ -5810,8 +5863,15 @@ reconfigure_datapath(struct dp_netdev *dp) seq_change(dp->port_seq); port_destroy(port); } else { - port->txq_mode = (netdev_n_txq(port->netdev) < wanted_txqs) - ? TXQ_MODE_XPS : TXQ_MODE_STATIC; + /* With a single queue, there is no point in using hash mode. */ + if (port->txq_requested_mode == TXQ_REQ_MODE_HASH && + netdev_n_txq(port->netdev) > 1) { + port->txq_mode = TXQ_MODE_HASH; + } else if (netdev_n_txq(port->netdev) < wanted_txqs) { + port->txq_mode = TXQ_MODE_XPS; + } else { + port->txq_mode = TXQ_MODE_STATIC; + } } } @@ -6100,9 +6160,11 @@ pmd_free_cached_ports(struct dp_netdev_pmd_thread *pmd) dpif_netdev_xps_revalidate_pmd(pmd, true); HMAP_FOR_EACH_POP (tx_port_cached, node, &pmd->tnl_port_cache) { + free(tx_port_cached->txq_pkts); free(tx_port_cached); } HMAP_FOR_EACH_POP (tx_port_cached, node, &pmd->send_port_cache) { + free(tx_port_cached->txq_pkts); free(tx_port_cached); } } @@ -6122,14 +6184,27 @@ pmd_load_cached_ports(struct dp_netdev_pmd_thread *pmd) hmap_shrink(&pmd->tnl_port_cache); HMAP_FOR_EACH (tx_port, node, &pmd->tx_ports) { + int n_txq = netdev_n_txq(tx_port->port->netdev); + struct dp_packet_batch *txq_pkts_cached; + if (netdev_has_tunnel_push_pop(tx_port->port->netdev)) { tx_port_cached = xmemdup(tx_port, sizeof *tx_port_cached); + if (tx_port->txq_pkts) { + txq_pkts_cached = xmemdup(tx_port->txq_pkts, + n_txq * sizeof *tx_port->txq_pkts); + tx_port_cached->txq_pkts = txq_pkts_cached; + } hmap_insert(&pmd->tnl_port_cache, &tx_port_cached->node, hash_port_no(tx_port_cached->port->port_no)); } - if (netdev_n_txq(tx_port->port->netdev)) { + if (n_txq) { tx_port_cached = xmemdup(tx_port, sizeof *tx_port_cached); + if (tx_port->txq_pkts) { + txq_pkts_cached = xmemdup(tx_port->txq_pkts, + n_txq * sizeof *tx_port->txq_pkts); + tx_port_cached->txq_pkts = txq_pkts_cached; + } hmap_insert(&pmd->send_port_cache, &tx_port_cached->node, hash_port_no(tx_port_cached->port->port_no)); } @@ -6911,6 +6986,7 @@ dp_netdev_pmd_clear_ports(struct dp_netdev_pmd_thread *pmd) free(poll); } HMAP_FOR_EACH_POP (port, node, &pmd->tx_ports) { + free(port->txq_pkts); free(port); } ovs_mutex_unlock(&pmd->port_mutex); @@ -6981,6 +7057,14 @@ dp_netdev_add_port_tx_to_pmd(struct dp_netdev_pmd_thread *pmd, tx->flush_time = 0LL; dp_packet_batch_init(&tx->output_pkts); + if (tx->port->txq_mode == TXQ_MODE_HASH) { + int i, n_txq = netdev_n_txq(tx->port->netdev); + tx->txq_pkts = xzalloc(n_txq * sizeof *tx->txq_pkts); + for (i = 0; i < n_txq; i++) { + dp_packet_batch_init(&tx->txq_pkts[i]); + } + } + hmap_insert(&pmd->tx_ports, &tx->node, hash_port_no(tx->port->port_no)); pmd->need_reload = true; } @@ -6993,6 +7077,7 @@ dp_netdev_del_port_tx_from_pmd(struct dp_netdev_pmd_thread *pmd, OVS_REQUIRES(pmd->port_mutex) { hmap_remove(&pmd->tx_ports, &tx->node); + free(tx->txq_pkts); free(tx); pmd->need_reload = true; } From patchwork Wed Jan 5 08:19:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxime Coquelin X-Patchwork-Id: 1575535 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AsZD9eOx; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JTMqP5bQ2z9sRR for ; Wed, 5 Jan 2022 19:20:17 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 00EEC4099E; Wed, 5 Jan 2022 08:20:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LUuGlRpxzaMN; Wed, 5 Jan 2022 08:20:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id AF715408C7; Wed, 5 Jan 2022 08:20:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 93406C0038; Wed, 5 Jan 2022 08:20:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2ACFCC0030 for ; Wed, 5 Jan 2022 08:20:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C20446FB41 for ; Wed, 5 Jan 2022 08:20:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08HLhvhqA-U4 for ; Wed, 5 Jan 2022 08:20:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 010D16FB28 for ; Wed, 5 Jan 2022 08:20:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641370802; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ySKiS3xGwdbUG8QTaa9FKANv9e7Pysq9MxJ89YsFgJE=; b=AsZD9eOxt6hXiEBFECudm2t4vWa5uNr/1yNIoatdoubMSnlGxX+PLuc3B0QF9DJGzEJXSm jApGiCq31P/NsYTT9g+2Y4VK9LrOJpHTR6/nTS8HbVc7XbYzDGP/swmNYDuDR7YRaqRudW BplTk5HD6ritY8mrNi6uvTqUgG5Uj00= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-355-zWMH_gr1MiaQuYP_ZxtcKQ-1; Wed, 05 Jan 2022 03:20:00 -0500 X-MC-Unique: zWMH_gr1MiaQuYP_ZxtcKQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A14E05F9C5; Wed, 5 Jan 2022 08:19:58 +0000 (UTC) Received: from max-t490s.redhat.com (unknown [10.39.208.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id F28F916A51; Wed, 5 Jan 2022 08:19:56 +0000 (UTC) From: Maxime Coquelin To: dev@openvswitch.org Date: Wed, 5 Jan 2022 09:19:26 +0100 Message-Id: <20220105081926.613684-6-maxime.coquelin@redhat.com> In-Reply-To: <20220105081926.613684-1-maxime.coquelin@redhat.com> References: <20220105081926.613684-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, Maxime Coquelin , i.maximets@ovn.org, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v5 5/5] dpif-netdev.at: Add test for Tx packets steering. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch introduces a new test for Tx packets steering modes. First test validates the static mode, by checking that all packets are transmitted on a single queue (single PMD thread), then it tests the same with enabling hash based packet steering, ensuring packets are transmitted on both queues. Signed-off-by: Maxime Coquelin Reviewed-by: David Marchand --- tests/dpif-netdev.at | 67 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/tests/dpif-netdev.at b/tests/dpif-netdev.at index 53eee185a..b7f1c3f8b 100644 --- a/tests/dpif-netdev.at +++ b/tests/dpif-netdev.at @@ -635,3 +635,70 @@ OVS_WAIT_UNTIL([grep "flow: in_port is not an exact match" ovs-vswitchd.log]) OVS_VSWITCHD_STOP(["/flow: in_port is not an exact match/d /failed to put/d"]) AT_CLEANUP + +# SEND_UDP_PKTS([p_name], [p_ofport]) +# +# Sends 128 packets to port 'p_name' with different UDP destination ports. +m4_define([SEND_UDP_PKTS], + [ + for i in `seq 1 128`; do + pkt="in_port($2),eth(src=50:54:00:00:00:05,dst=50:54:00:00:01:00),eth_type(0x0800),ipv4(src=10.0.1.1,dst=10.0.0.1,proto=17),udp(src=1000,dst=$i)" + ovs-appctl netdev-dummy/receive $1 $pkt --len 256 + done + ] +) + +AT_SETUP([dpif-netdev - tx packets steering]) +OVS_VSWITCHD_START( + [add-port br0 p1 -- set Interface p1 type=dummy-pmd ofport_request=1], [], [], [--dummy-numa 0]) + +dnl 'thread' mode, packets are expected to be transmitted on a single +dnl queue since there is only one PMD thread. + +AT_CHECK([ovs-vsctl add-port br0 p2 -- set Interface p2 type=dummy-pmd ofport_request=2 options:n_txq=2 other_config:tx-steering=thread]) +AT_CHECK([ovs-ofctl add-flow br0 in_port=1,actions=output:2]) + +AT_CHECK([SEND_UDP_PKTS([p1], [1])]) + +OVS_WAIT_UNTIL([test `ovs-vsctl get Interface p2 statistics:tx_packets` -eq 128]) +AT_CHECK([ovs-vsctl get Interface p2 statistics], [], [stdout]) +AT_CHECK([test `ovs-vsctl get Interface p2 statistics:tx_q0_packets` -eq 0 -a dnl + `ovs-vsctl get Interface p2 statistics:tx_q1_packets` -eq 128 || dnl + test `ovs-vsctl get Interface p2 statistics:tx_q0_packets` -eq 128 -a dnl + `ovs-vsctl get Interface p2 statistics:tx_q1_packets` -eq 0]) + +AT_CHECK([ovs-vsctl del-port p2]) + +dnl 'hash' mode, packets are expected to be transmitted on both +dnl queues, based on their hash value. + +AT_CHECK([ovs-vsctl add-port br0 p2 -- set Interface p2 type=dummy-pmd ofport_request=2 options:n_txq=2 other_config:tx-steering=hash]) +AT_CHECK([ovs-ofctl add-flow br0 in_port=1,actions=output:2]) + +AT_CHECK([SEND_UDP_PKTS([p1], [1])]) + +OVS_WAIT_UNTIL([test `ovs-vsctl get Interface p2 statistics:tx_packets` -eq 128]) +AT_CHECK([ovs-vsctl get Interface p2 statistics], [], [stdout]) + +AT_CHECK([test `ovs-vsctl get Interface p2 statistics:tx_q0_packets` -gt 0 -a dnl + `ovs-vsctl get Interface p2 statistics:tx_q1_packets` -gt 0]) + +AT_CHECK([ovs-vsctl del-port p2]) + +dnl 'hash' mode with hw-offload enabled, packets are expected to be transmitted on both +dnl queues, based on their hash value. + +AT_CHECK([ovs-vsctl set Open_vSwitch . other_config:hw-offload=true]) +AT_CHECK([ovs-vsctl add-port br0 p2 -- set Interface p2 type=dummy-pmd ofport_request=2 options:n_txq=2 other_config:tx-steering=hash]) +AT_CHECK([ovs-ofctl add-flow br0 in_port=1,actions=output:2]) + +AT_CHECK([SEND_UDP_PKTS([p1], [1])]) + +OVS_WAIT_UNTIL([test `ovs-vsctl get Interface p2 statistics:tx_packets` -eq 128]) +AT_CHECK([ovs-vsctl get Interface p2 statistics], [], [stdout]) + +AT_CHECK([test `ovs-vsctl get Interface p2 statistics:tx_q0_packets` -gt 0 -a dnl + `ovs-vsctl get Interface p2 statistics:tx_q1_packets` -gt 0]) + +OVS_VSWITCHD_STOP +AT_CLEANUP