From patchwork Mon May 17 14:04:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1479522 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FkLS05k5dz9sWp for ; Tue, 18 May 2021 00:02:40 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3DDC360AA7; Mon, 17 May 2021 14:02:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id il5nkOJxWXMU; Mon, 17 May 2021 14:02:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTP id 496F060AAA; Mon, 17 May 2021 14:02:31 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 51398C0022; Mon, 17 May 2021 14:02:29 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 65001C0031 for ; Mon, 17 May 2021 14:02:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7967060A89 for ; Mon, 17 May 2021 14:02:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GjIExoCOwwq1 for ; Mon, 17 May 2021 14:02:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp3.osuosl.org (Postfix) with ESMTPS id 11DBC60A83 for ; Mon, 17 May 2021 14:02:21 +0000 (UTC) IronPort-SDR: iT6Ybqmk6Hv6k7rgxulJk4Twa1AeXOyPsEVpkj32/2N1F6/hemY2IaDQqpCZ8AjLVoNsw0NW9N /Vcky3TJt7ug== X-IronPort-AV: E=McAfee;i="6200,9189,9986"; a="180747418" X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="180747418" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2021 07:02:20 -0700 IronPort-SDR: HfEvyNuKSM9/1I6UieQW1pJogxDgCFi5CLUJfuqPY0xXibyYQz0qKjuRMCtzThhg7fh0OkUEtq DePUHt+nt8BQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="460318614" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.175]) by fmsmga004.fm.intel.com with ESMTP; 17 May 2021 07:02:18 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Mon, 17 May 2021 15:04:26 +0100 Message-Id: <20210517140434.59555-9-cian.ferriter@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210517140434.59555-1-cian.ferriter@intel.com> References: <20210517140434.59555-1-cian.ferriter@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v12 08/16] dpif-netdev: Add a partial HWOL PMD statistic. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" It is possible for packets traversing the userspace datapath to match a flow before hitting on EMC by using a mark ID provided by a NIC. Add a PMD statistic for this hit. Signed-off-by: Cian Ferriter --- NEWS | 2 ++ lib/dpif-netdev-avx512.c | 3 +++ lib/dpif-netdev-perf.c | 3 +++ lib/dpif-netdev-perf.h | 1 + lib/dpif-netdev.c | 9 +++++++-- tests/pmd.at | 6 ++++-- 6 files changed, 20 insertions(+), 4 deletions(-) diff --git a/NEWS b/NEWS index 402ce5969..3eab5f4fa 100644 --- a/NEWS +++ b/NEWS @@ -5,6 +5,8 @@ Post-v2.15.0 - Userspace datapath: * Auto load balancing of PMDs now partially supports cross-NUMA polling cases, e.g if all PMD threads are running on the same NUMA node. + * Add a partial HWOL PMD statistic counting hits similar to existing + * EMC/SMC/DPCLS stats. - ovs-ctl: * New option '--no-record-hostname' to disable hostname configuration in ovsdb on startup. diff --git a/lib/dpif-netdev-avx512.c b/lib/dpif-netdev-avx512.c index e255c9d17..5c6f54799 100644 --- a/lib/dpif-netdev-avx512.c +++ b/lib/dpif-netdev-avx512.c @@ -114,6 +114,7 @@ dp_netdev_input_outer_avx512(struct dp_netdev_pmd_thread *pmd, const uint32_t emc_enabled = pmd->ctx.emc_insert_min != 0; const uint32_t smc_enabled = pmd->ctx.smc_enable_db; + uint32_t phwol_hits = 0; uint32_t emc_hits = 0; uint32_t smc_hits = 0; @@ -147,6 +148,7 @@ dp_netdev_input_outer_avx512(struct dp_netdev_pmd_thread *pmd, pkt_meta[i].tcp_flags = parse_tcp_flags(packet); pkt_meta[i].bytes = dp_packet_size(packet); + phwol_hits++; hwol_emc_smc_hitmask |= (1 << i); continue; } @@ -235,6 +237,7 @@ dp_netdev_input_outer_avx512(struct dp_netdev_pmd_thread *pmd, /* At this point we don't return error anymore, so commit stats here. */ pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_RECV, batch_size); + pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_PHWOL_HIT, phwol_hits); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_EXACT_HIT, emc_hits); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_SMC_HIT, smc_hits); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_MASKED_HIT, diff --git a/lib/dpif-netdev-perf.c b/lib/dpif-netdev-perf.c index 9560e7c3c..7103a2d4d 100644 --- a/lib/dpif-netdev-perf.c +++ b/lib/dpif-netdev-perf.c @@ -246,6 +246,7 @@ pmd_perf_format_overall_stats(struct ds *str, struct pmd_perf_stats *s, ds_put_format(str, " Rx packets: %12"PRIu64" (%.0f Kpps, %.0f cycles/pkt)\n" " Datapath passes: %12"PRIu64" (%.2f passes/pkt)\n" + " - PHWOL hits: %12"PRIu64" (%5.1f %%)\n" " - EMC hits: %12"PRIu64" (%5.1f %%)\n" " - SMC hits: %12"PRIu64" (%5.1f %%)\n" " - Megaflow hits: %12"PRIu64" (%5.1f %%, %.2f " @@ -255,6 +256,8 @@ pmd_perf_format_overall_stats(struct ds *str, struct pmd_perf_stats *s, rx_packets, (rx_packets / duration) / 1000, 1.0 * stats[PMD_CYCLES_ITER_BUSY] / rx_packets, passes, rx_packets ? 1.0 * passes / rx_packets : 0, + stats[PMD_STAT_PHWOL_HIT], + 100.0 * stats[PMD_STAT_PHWOL_HIT] / passes, stats[PMD_STAT_EXACT_HIT], 100.0 * stats[PMD_STAT_EXACT_HIT] / passes, stats[PMD_STAT_SMC_HIT], diff --git a/lib/dpif-netdev-perf.h b/lib/dpif-netdev-perf.h index 72645b6b3..8b1a52387 100644 --- a/lib/dpif-netdev-perf.h +++ b/lib/dpif-netdev-perf.h @@ -56,6 +56,7 @@ extern "C" { /* Set of counter types maintained in pmd_perf_stats. */ enum pmd_stat_type { + PMD_STAT_PHWOL_HIT, /* Packets that had a partial HWOL hit (phwol). */ PMD_STAT_EXACT_HIT, /* Packets that had an exact match (emc). */ PMD_STAT_SMC_HIT, /* Packets that had a sig match hit (SMC). */ PMD_STAT_MASKED_HIT, /* Packets that matched in the flow table. */ diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index b35ccbe3b..f161ae7f5 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -660,6 +660,7 @@ pmd_info_show_stats(struct ds *reply, " packets received: %"PRIu64"\n" " packet recirculations: %"PRIu64"\n" " avg. datapath passes per packet: %.02f\n" + " phwol hits: %"PRIu64"\n" " emc hits: %"PRIu64"\n" " smc hits: %"PRIu64"\n" " megaflow hits: %"PRIu64"\n" @@ -668,7 +669,8 @@ pmd_info_show_stats(struct ds *reply, " miss with failed upcall: %"PRIu64"\n" " avg. packets per output batch: %.02f\n", total_packets, stats[PMD_STAT_RECIRC], - passes_per_pkt, stats[PMD_STAT_EXACT_HIT], + passes_per_pkt, stats[PMD_STAT_PHWOL_HIT], + stats[PMD_STAT_EXACT_HIT], stats[PMD_STAT_SMC_HIT], stats[PMD_STAT_MASKED_HIT], lookups_per_hit, stats[PMD_STAT_MISS], stats[PMD_STAT_LOST], @@ -1685,6 +1687,7 @@ dpif_netdev_get_stats(const struct dpif *dpif, struct dpif_dp_stats *stats) CMAP_FOR_EACH (pmd, node, &dp->poll_threads) { stats->n_flows += cmap_count(&pmd->flow_table); pmd_perf_read_counters(&pmd->perf_stats, pmd_stats); + stats->n_hit += pmd_stats[PMD_STAT_PHWOL_HIT]; stats->n_hit += pmd_stats[PMD_STAT_EXACT_HIT]; stats->n_hit += pmd_stats[PMD_STAT_SMC_HIT]; stats->n_hit += pmd_stats[PMD_STAT_MASKED_HIT]; @@ -6697,7 +6700,7 @@ dfc_processing(struct dp_netdev_pmd_thread *pmd, bool md_is_valid, odp_port_t port_no) { struct netdev_flow_key *key = &keys[0]; - size_t n_missed = 0, n_emc_hit = 0; + size_t n_missed = 0, n_emc_hit = 0, n_phwol_hit = 0; struct dfc_cache *cache = &pmd->flow_cache; struct dp_packet *packet; const size_t cnt = dp_packet_batch_size(packets_); @@ -6739,6 +6742,7 @@ dfc_processing(struct dp_netdev_pmd_thread *pmd, flow = mark_to_flow_find(pmd, mark); if (OVS_LIKELY(flow)) { tcp_flags = parse_tcp_flags(packet); + n_phwol_hit++; if (OVS_LIKELY(batch_enable)) { dp_netdev_queue_batches(packet, flow, tcp_flags, batches, n_batches); @@ -6801,6 +6805,7 @@ dfc_processing(struct dp_netdev_pmd_thread *pmd, /* Count of packets which are not flow batched. */ *n_flows = map_cnt; + pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_PHWOL_HIT, n_phwol_hit); pmd_perf_update_counter(&pmd->perf_stats, PMD_STAT_EXACT_HIT, n_emc_hit); if (!smc_enable_db) { diff --git a/tests/pmd.at b/tests/pmd.at index cc5371d5a..34a59d502 100644 --- a/tests/pmd.at +++ b/tests/pmd.at @@ -202,11 +202,12 @@ dummy@ovs-dummy: hit:0 missed:0 p0 7/1: (dummy-pmd: configured_rx_queues=4, configured_tx_queues=, requested_rx_queues=4, requested_tx_queues=) ]) -AT_CHECK([ovs-appctl dpif-netdev/pmd-stats-show | sed SED_NUMA_CORE_PATTERN | sed '/cycles/d' | grep pmd -A 9], [0], [dnl +AT_CHECK([ovs-appctl dpif-netdev/pmd-stats-show | sed SED_NUMA_CORE_PATTERN | sed '/cycles/d' | grep pmd -A 10], [0], [dnl pmd thread numa_id core_id : packets received: 0 packet recirculations: 0 avg. datapath passes per packet: 0.00 + phwol hits: 0 emc hits: 0 smc hits: 0 megaflow hits: 0 @@ -233,11 +234,12 @@ AT_CHECK([cat ovs-vswitchd.log | filter_flow_install | strip_xout], [0], [dnl recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth(src=50:54:00:00:00:77,dst=50:54:00:00:01:78),eth_type(0x0800),ipv4(frag=no), actions: ]) -AT_CHECK([ovs-appctl dpif-netdev/pmd-stats-show | sed SED_NUMA_CORE_PATTERN | sed '/cycles/d' | grep pmd -A 9], [0], [dnl +AT_CHECK([ovs-appctl dpif-netdev/pmd-stats-show | sed SED_NUMA_CORE_PATTERN | sed '/cycles/d' | grep pmd -A 10], [0], [dnl pmd thread numa_id core_id : packets received: 20 packet recirculations: 0 avg. datapath passes per packet: 1.00 + phwol hits: 0 emc hits: 19 smc hits: 0 megaflow hits: 0