From patchwork Wed Nov 30 17:39:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1710621 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NAACCfB0; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NMmgl4SxSz23nT for ; Thu, 1 Dec 2022 04:40:19 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8B9996104B; Wed, 30 Nov 2022 17:40:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8B9996104B Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NAACCfB0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VnXYhqffs0pk; Wed, 30 Nov 2022 17:40:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 17FE961041; Wed, 30 Nov 2022 17:40:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 17FE961041 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D908BC0035; Wed, 30 Nov 2022 17:40:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 966B5C002D for ; Wed, 30 Nov 2022 17:40:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 64E4A40BE8 for ; Wed, 30 Nov 2022 17:40:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 64E4A40BE8 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=NAACCfB0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TFmOfRe0qSNq for ; Wed, 30 Nov 2022 17:40:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A30DA4000B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id A30DA4000B for ; Wed, 30 Nov 2022 17:40:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669830007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8UOvG1sEz5/ePIbZYducTgK22L0mp9LQ39T9VmyXy7Y=; b=NAACCfB0gdeE5BO8fF7h1tyT4PNe0cl+P2kL66VQ6VLBGFLZB4a17nbYWO/Ys1v+vKjleF Im9PYJ+W+eBZVaJaJR+CUXjSSVSk5SB2yrwBx3MgoTTDI5pSdfYXYJn9LryVfeIgMMkUwp zz1VFwN04uBZSdpwuHRb5cVl+86EWyk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-637-HYRiIdVBPsywqm3FbTPk6g-1; Wed, 30 Nov 2022 12:40:02 -0500 X-MC-Unique: HYRiIdVBPsywqm3FbTPk6g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 865E487B2A4 for ; Wed, 30 Nov 2022 17:40:02 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.13]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AEA3C15BA4; Wed, 30 Nov 2022 17:40:00 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Wed, 30 Nov 2022 17:39:52 +0000 Message-Id: <20221130173954.1043885-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: david.marchand@redhat.com Subject: [ovs-dev] [PATCH v3 1/3] dpif-netdev: Make pmd-rxq-show time configurable. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" pmd-rxq-show shows the Rx queue to pmd assignments as well as the pmd usage of each Rx queue. Up until now a tail length of 60 seconds pmd usage was shown for each Rx queue, as this is the value used during rebalance to avoid any spike effects. When debugging or tuning, it is also convenient to display the pmd usage of an Rx queue over a shorter time frame, so any changes config or traffic that impact pmd usage can be evaluated more quickly. A parameter is added that allows pmd-rxq-show stats pmd usage to be shown for a shorter time frame. Values are rounded up to the nearest 5 seconds as that is the measurement granularity and the value used is displayed. e.g. $ ovs-appctl dpif-netdev/pmd-rxq-show -secs 5 Displaying last 5 seconds pmd usage % pmd thread numa_id 0 core_id 4: isolated : false port: dpdk0 queue-id: 0 (enabled) pmd usage: 95 % overhead: 4 % The default time frame has not changed and the maximum value is limited to the maximum stored tail length (60 seconds). Signed-off-by: Kevin Traynor Reviewed-by: David Marchand --- v2: - fixed comments from David's review - Squashed new unit tests into this patch - docs can be squashed later --- lib/dpif-netdev-private-thread.h | 2 +- lib/dpif-netdev.c | 98 ++++++++++++++++++++++++-------- tests/pmd.at | 62 ++++++++++++++++++++ 3 files changed, 138 insertions(+), 24 deletions(-) diff --git a/lib/dpif-netdev-private-thread.h b/lib/dpif-netdev-private-thread.h index 4472b199d..1ec3cd794 100644 --- a/lib/dpif-netdev-private-thread.h +++ b/lib/dpif-netdev-private-thread.h @@ -115,5 +115,5 @@ struct dp_netdev_pmd_thread { /* Write index for 'busy_cycles_intrvl'. */ - unsigned int intrvl_idx; + atomic_count intrvl_idx; /* Busy cycles in last PMD_INTERVAL_MAX intervals. */ atomic_ullong *busy_cycles_intrvl; diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 2c08a71c8..74d265a0b 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -161,9 +161,11 @@ static struct odp_support dp_netdev_support = { /* Time in microseconds of the interval in which rxq processing cycles used * in rxq to pmd assignments is measured and stored. */ -#define PMD_INTERVAL_LEN 10000000LL +#define PMD_INTERVAL_LEN 5000000LL +/* For converting PMD_INTERVAL_LEN to secs. */ +#define INTERVAL_USEC_TO_SEC 1000000LL /* Number of intervals for which cycles are stored * and used during rxq to pmd assignment. */ -#define PMD_INTERVAL_MAX 6 +#define PMD_INTERVAL_MAX 12 /* Time in microseconds to try RCU quiescing. */ @@ -429,5 +431,5 @@ struct dp_netdev_rxq { queue doesn't need to be pinned to a particular core. */ - unsigned intrvl_idx; /* Write index for 'cycles_intrvl'. */ + atomic_count intrvl_idx; /* Write index for 'cycles_intrvl'. */ struct dp_netdev_pmd_thread *pmd; /* pmd thread that polls this queue. */ bool is_vhost; /* Is rxq of a vhost port. */ @@ -616,4 +618,7 @@ dp_netdev_rxq_set_intrvl_cycles(struct dp_netdev_rxq *rx, static uint64_t dp_netdev_rxq_get_intrvl_cycles(struct dp_netdev_rxq *rx, unsigned idx); +static uint64_t +get_interval_values(atomic_ullong *source, atomic_count *cur_idx, + int num_to_read); static void dpif_netdev_xps_revalidate_pmd(const struct dp_netdev_pmd_thread *pmd, @@ -870,5 +875,6 @@ sorted_poll_list(struct dp_netdev_pmd_thread *pmd, struct rxq_poll **list, static void -pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd) +pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd, + int secs) { if (pmd->core_id != NON_PMD_CORE_ID) { @@ -878,4 +884,5 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd) uint64_t busy_cycles = 0; uint64_t total_rxq_proc_cycles = 0; + unsigned int intervals; ds_put_format(reply, @@ -889,13 +896,12 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd) /* Get the total pmd cycles for an interval. */ atomic_read_relaxed(&pmd->intrvl_cycles, &total_cycles); + /* Calculate how many intervals are to be used. */ + intervals = DIV_ROUND_UP(secs, + PMD_INTERVAL_LEN / INTERVAL_USEC_TO_SEC); /* Estimate the cycles to cover all intervals. */ - total_cycles *= PMD_INTERVAL_MAX; - - for (int j = 0; j < PMD_INTERVAL_MAX; j++) { - uint64_t cycles; - - atomic_read_relaxed(&pmd->busy_cycles_intrvl[j], &cycles); - busy_cycles += cycles; - } + total_cycles *= intervals; + busy_cycles = get_interval_values(pmd->busy_cycles_intrvl, + &pmd->intrvl_idx, + intervals); if (busy_cycles > total_cycles) { busy_cycles = total_cycles; @@ -907,7 +913,7 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd) uint64_t rxq_proc_cycles = 0; - for (int j = 0; j < PMD_INTERVAL_MAX; j++) { - rxq_proc_cycles += dp_netdev_rxq_get_intrvl_cycles(rxq, j); - } + rxq_proc_cycles = get_interval_values(rxq->cycles_intrvl, + &rxq->intrvl_idx, + intervals); total_rxq_proc_cycles += rxq_proc_cycles; ds_put_format(reply, " port: %-16s queue-id: %2d", name, @@ -1423,4 +1429,8 @@ dpif_netdev_pmd_info(struct unixctl_conn *conn, int argc, const char *argv[], bool filter_on_pmd = false; size_t n; + unsigned int secs = 0; + unsigned long long max_secs = (PMD_INTERVAL_LEN * PMD_INTERVAL_MAX) + / INTERVAL_USEC_TO_SEC; + bool first_show_rxq = true; ovs_mutex_lock(&dp_netdev_mutex); @@ -1433,4 +1443,12 @@ dpif_netdev_pmd_info(struct unixctl_conn *conn, int argc, const char *argv[], argc -= 2; argv += 2; + } else if (type == PMD_INFO_SHOW_RXQ && + !strcmp(argv[1], "-secs") && + argc > 2) { + if (!str_to_uint(argv[2], 10, &secs)) { + secs = max_secs; + } + argc -= 2; + argv += 2; } else { dp = shash_find_data(&dp_netdevs, argv[1]); @@ -1462,5 +1480,16 @@ dpif_netdev_pmd_info(struct unixctl_conn *conn, int argc, const char *argv[], } if (type == PMD_INFO_SHOW_RXQ) { - pmd_info_show_rxq(&reply, pmd); + if (first_show_rxq) { + if (!secs || secs > max_secs) { + secs = max_secs; + } else { + secs = ROUND_UP(secs, + PMD_INTERVAL_LEN / INTERVAL_USEC_TO_SEC); + } + ds_put_format(&reply, "Displaying last %u seconds " + "pmd usage %%\n", secs); + first_show_rxq = false; + } + pmd_info_show_rxq(&reply, pmd, secs); } else if (type == PMD_INFO_CLEAR_STATS) { pmd_perf_stats_clear(&pmd->perf_stats); @@ -1577,6 +1606,7 @@ dpif_netdev_init(void) 0, 3, dpif_netdev_pmd_info, (void *)&clear_aux); - unixctl_command_register("dpif-netdev/pmd-rxq-show", "[-pmd core] [dp]", - 0, 3, dpif_netdev_pmd_info, + unixctl_command_register("dpif-netdev/pmd-rxq-show", "[-pmd core] " + "[-secs secs] [dp]", + 0, 5, dpif_netdev_pmd_info, (void *)&poll_aux); unixctl_command_register("dpif-netdev/pmd-perf-show", @@ -5150,5 +5180,5 @@ dp_netdev_rxq_set_intrvl_cycles(struct dp_netdev_rxq *rx, unsigned long long cycles) { - unsigned int idx = rx->intrvl_idx++ % PMD_INTERVAL_MAX; + unsigned int idx = atomic_count_inc(&rx->intrvl_idx) % PMD_INTERVAL_MAX; atomic_store_relaxed(&rx->cycles_intrvl[idx], cycles); } @@ -6890,4 +6920,7 @@ reload: atomic_count_init(&pmd->pmd_overloaded, 0); + pmd->intrvl_tsc_prev = 0; + atomic_store_relaxed(&pmd->intrvl_cycles, 0); + if (!dpdk_attached) { dpdk_attached = dpdk_attach_thread(pmd->core_id); @@ -6921,10 +6954,8 @@ reload: } - pmd->intrvl_tsc_prev = 0; - atomic_store_relaxed(&pmd->intrvl_cycles, 0); for (i = 0; i < PMD_INTERVAL_MAX; i++) { atomic_store_relaxed(&pmd->busy_cycles_intrvl[i], 0); } - pmd->intrvl_idx = 0; + atomic_count_set(&pmd->intrvl_idx, 0); cycles_counter_update(s); @@ -9907,5 +9938,5 @@ dp_netdev_pmd_try_optimize(struct dp_netdev_pmd_thread *pmd, curr_tsc - pmd->intrvl_tsc_prev); } - idx = pmd->intrvl_idx++ % PMD_INTERVAL_MAX; + idx = atomic_count_inc(&pmd->intrvl_idx) % PMD_INTERVAL_MAX; atomic_store_relaxed(&pmd->busy_cycles_intrvl[idx], tot_proc); pmd->intrvl_tsc_prev = curr_tsc; @@ -9930,4 +9961,25 @@ dp_netdev_pmd_try_optimize(struct dp_netdev_pmd_thread *pmd, } +/* Returns the sum of a specified number of newest to + * oldest interval values. 'cur_idx' is where the next + * write will be and wrap around needs to be handled. + */ +static uint64_t +get_interval_values(atomic_ullong *source, atomic_count *cur_idx, + int num_to_read) { + unsigned int i; + uint64_t total = 0; + + i = atomic_count_get(cur_idx) % PMD_INTERVAL_MAX; + for (int read = 0; read < num_to_read; read++) { + uint64_t interval_value; + + i = i ? i - 1 : PMD_INTERVAL_MAX - 1; + atomic_read_relaxed(&source[i], &interval_value); + total += interval_value; + } + return total; +} + /* Insert 'rule' into 'cls'. */ static void diff --git a/tests/pmd.at b/tests/pmd.at index 10879a349..ed90f88c4 100644 --- a/tests/pmd.at +++ b/tests/pmd.at @@ -71,4 +71,5 @@ CHECK_PMD_THREADS_CREATED() AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id core_id : isolated : false @@ -103,4 +104,5 @@ dummy@ovs-dummy: hit:0 missed:0 AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id core_id : isolated : false @@ -135,4 +137,5 @@ dummy@ovs-dummy: hit:0 missed:0 AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id core_id : isolated : false @@ -184,4 +187,5 @@ CHECK_PMD_THREADS_CREATED([1], [], [+$TMP]) AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id core_id : isolated : false @@ -216,4 +220,5 @@ dummy@ovs-dummy: hit:0 missed:0 AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | sed SED_NUMA_CORE_PATTERN], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id core_id : isolated : false @@ -281,4 +286,5 @@ OVS_WAIT_UNTIL([tail -n +$TMP ovs-vswitchd.log | grep "Performing pmd to rx queu AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id 1 core_id 1: isolated : false @@ -303,4 +309,5 @@ OVS_WAIT_UNTIL([tail -n +$TMP ovs-vswitchd.log | grep "Performing pmd to rx queu AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id 1 core_id 1: isolated : false @@ -323,4 +330,5 @@ OVS_WAIT_UNTIL([tail -n +$TMP ovs-vswitchd.log | grep "Performing pmd to rx queu AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id 1 core_id 1: isolated : false @@ -344,4 +352,5 @@ CHECK_PMD_THREADS_CREATED([1], [1], [+$TMP]) AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show], [0], [dnl +Displaying last 60 seconds pmd usage % pmd thread numa_id 1 core_id 0: isolated : false @@ -472,4 +481,57 @@ OVS_VSWITCHD_STOP AT_CLEANUP +AT_SETUP([PMD - pmd-rxq-show pmd usage time]) +OVS_VSWITCHD_START([add-port br0 p0 -- set Interface p0 type=dummy-pmd], [], [], [DUMMY_NUMA]) + +#CHECK_CPU_DISCOVERED() +#CHECK_PMD_THREADS_CREATED() + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs -1 | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 0 | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 1 | grep Displaying], [0], [dnl +Displaying last 5 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 5 | grep Displaying], [0], [dnl +Displaying last 5 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 6 | grep Displaying], [0], [dnl +Displaying last 10 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 51 | grep Displaying], [0], [dnl +Displaying last 55 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 55 | grep Displaying], [0], [dnl +Displaying last 55 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 56 | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 60 | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +AT_CHECK([ovs-appctl dpif-netdev/pmd-rxq-show -secs 61 | grep Displaying], [0], [dnl +Displaying last 60 seconds pmd usage % +]) + +OVS_VSWITCHD_STOP +AT_CLEANUP + dnl Reconfigure the number of rx queues of a port, make sure that all the dnl queues are polled by the datapath and try to send a couple of packets. From patchwork Wed Nov 30 17:39:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1710620 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ipjLI8jX; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NMmgj0qwcz23nT for ; Thu, 1 Dec 2022 04:40:17 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2F4B1417DE; Wed, 30 Nov 2022 17:40:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2F4B1417DE Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ipjLI8jX X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cttd0KRCLGwp; Wed, 30 Nov 2022 17:40:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 4655E41733; Wed, 30 Nov 2022 17:40:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4655E41733 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F67EC0033; Wed, 30 Nov 2022 17:40:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 76179C002D for ; Wed, 30 Nov 2022 17:40:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5D23740BE9 for ; Wed, 30 Nov 2022 17:40:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5D23740BE9 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ipjLI8jX X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G_97_LFCqM1b for ; Wed, 30 Nov 2022 17:40:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2F1BD40BE8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2F1BD40BE8 for ; Wed, 30 Nov 2022 17:40:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669830007; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PCirwVUXppfKBdoL4gH4QH2q63y3VSAWZ6BSqPwTwio=; b=ipjLI8jXQAudjNuxUxPyGU+f9/SJVj+RdxKemhnAkUfwts7HeDjkttsaOagoKRfSDCHE3d D7Gy9k0fJmYjUfaSLbSs2X02qm1nizFUacV3zEHHB0v7NNGzRgpQy2Z8hpiZc2RkY5kch3 qpVhIxPC5CaA1trxk70ttAQ8P+QU/L8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-523-ap64ovPOM7OTqU5EbZ3wzg-1; Wed, 30 Nov 2022 12:40:06 -0500 X-MC-Unique: ap64ovPOM7OTqU5EbZ3wzg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2ED1C185A794 for ; Wed, 30 Nov 2022 17:40:06 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.13]) by smtp.corp.redhat.com (Postfix) with ESMTP id 21371C15BA4; Wed, 30 Nov 2022 17:40:05 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Wed, 30 Nov 2022 17:39:53 +0000 Message-Id: <20221130173954.1043885-2-ktraynor@redhat.com> In-Reply-To: <20221130173954.1043885-1-ktraynor@redhat.com> References: <20221130173954.1043885-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: david.marchand@redhat.com Subject: [ovs-dev] [PATCH v3 2/3] docs: Add documention for pmd-rxq-show secs parameter. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add description of new '-secs' parameter in docs. Also, add to NEWS as it is a user facing change. Signed-off-by: Kevin Traynor Reviewed-by: David Marchand --- v3: - My prediction that NEWS would cause a conflict was correct but I did not think it would be when submitting the patch :s - Rebased NEWS entry --- Documentation/topics/dpdk/pmd.rst | 23 ++++++++++++++++++----- NEWS | 3 +++ 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/Documentation/topics/dpdk/pmd.rst b/Documentation/topics/dpdk/pmd.rst index b259cc8b3..88457f366 100644 --- a/Documentation/topics/dpdk/pmd.rst +++ b/Documentation/topics/dpdk/pmd.rst @@ -102,10 +102,18 @@ core cycles for each Rx queue:: .. note:: - A history of one minute is recorded and shown for each Rx queue to allow for - traffic pattern spikes. Any changes in the Rx queue's PMD core cycles usage, - due to traffic pattern or reconfig changes, will take one minute to be fully - reflected in the stats. + By default a history of one minute is recorded and shown for each Rx queue + to allow for traffic pattern spikes. Any changes in the Rx queue's PMD core + cycles usage, due to traffic pattern or reconfig changes, will take one + minute to be fully reflected in the stats by default. - .. versionchanged:: 2.6.0 +PMD thread usage of an Rx queue can be displayed for a shorter period of time, +from the last 5 seconds up to the default 60 seconds in 5 second steps. + +To see the port/Rx queue assignment and the last 5 secs of measured usage +history of PMD core cycles for each Rx queue:: + + $ ovs-appctl dpif-netdev/pmd-rxq-show -secs 5 + +.. versionchanged:: 2.6.0 The ``pmd-rxq-show`` command was added in OVS 2.6.0. @@ -116,4 +124,9 @@ core cycles for each Rx queue:: cycles inherently consumed by the OVS PMD processing loop. +.. versionchanged:: 3.1.0 + + The ``-secs`` parameter was added to the dpif-netdev/pmd-rxq-show + command. + Rx queue to PMD assignment takes place whenever there are configuration changes or can be triggered by using:: diff --git a/NEWS b/NEWS index f6caf1ca7..a2ff388b0 100644 --- a/NEWS +++ b/NEWS @@ -31,4 +31,7 @@ Post-v3.0.0 determined. Previously it was 10 Mbps. Values can still be overridden by specifying 'max-rate' or '[r]stp-path-cost' accordingly. + - Userspace datapath: + * Add '-secs' argument to appctl 'dpif-netdev/pmd-rxq-show' to show + the pmd usage of an Rx queue over a configurable time period. From patchwork Wed Nov 30 17:39:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1710622 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WIcMPPa4; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4NMmgp3hnPz23nT for ; Thu, 1 Dec 2022 04:40:22 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5694F40BF3; Wed, 30 Nov 2022 17:40:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5694F40BF3 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WIcMPPa4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hrVB6Zk6u77H; Wed, 30 Nov 2022 17:40:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 206E640BFD; Wed, 30 Nov 2022 17:40:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 206E640BFD Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C8EE5C0035; Wed, 30 Nov 2022 17:40:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 499FFC0033 for ; Wed, 30 Nov 2022 17:40:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3208C6103E for ; Wed, 30 Nov 2022 17:40:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3208C6103E Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WIcMPPa4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f45QBxTl1dB3 for ; Wed, 30 Nov 2022 17:40:13 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D461C6101E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id D461C6101E for ; Wed, 30 Nov 2022 17:40:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669830011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=COlN2EOyBuQoywsX4MDC49DCtmh4T06/7KegGDFPJ+E=; b=WIcMPPa4LKyBOF7vdGm9sg7vm0cY+xK0Ehcz0y3yYIJmHqrpo5l9T3LrCETIRS0VJmdCC/ 65NWvEpYmlhTMQ84Wq9zTuUIi/KawLYQFaTpj2jrkdARNnrgyhNdm3511Sctf4vdAQnD1E qktgbmhKYFopRa8GovVX3MZivNdABj4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-663-ZJqTn0NUP96yDNYufpT8GA-1; Wed, 30 Nov 2022 12:40:08 -0500 X-MC-Unique: ZJqTn0NUP96yDNYufpT8GA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 47510811E87 for ; Wed, 30 Nov 2022 17:40:08 +0000 (UTC) Received: from rh.Home (unknown [10.39.192.13]) by smtp.corp.redhat.com (Postfix) with ESMTP id 56CA4C15BA4; Wed, 30 Nov 2022 17:40:07 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Wed, 30 Nov 2022 17:39:54 +0000 Message-Id: <20221130173954.1043885-3-ktraynor@redhat.com> In-Reply-To: <20221130173954.1043885-1-ktraynor@redhat.com> References: <20221130173954.1043885-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: david.marchand@redhat.com Subject: [ovs-dev] [PATCH v3 3/3] dpif-netdev: Rename pmd_info_show_rxq variables. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" There are some similar readings taken for pmds and Rx queues in this function and a few of the variable names are ambiguous. Improve the readability of the code by updating some variables names to indicate that they are readings related to the pmd. Signed-off-by: Kevin Traynor Reviewed-by: David Marchand --- lib/dpif-netdev.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 74d265a0b..cb3eb02e5 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -881,6 +881,6 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd, struct rxq_poll *list; size_t n_rxq; - uint64_t total_cycles = 0; - uint64_t busy_cycles = 0; + uint64_t total_pmd_cycles = 0; + uint64_t busy_pmd_cycles = 0; uint64_t total_rxq_proc_cycles = 0; unsigned int intervals; @@ -895,15 +895,15 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd, /* Get the total pmd cycles for an interval. */ - atomic_read_relaxed(&pmd->intrvl_cycles, &total_cycles); + atomic_read_relaxed(&pmd->intrvl_cycles, &total_pmd_cycles); /* Calculate how many intervals are to be used. */ intervals = DIV_ROUND_UP(secs, PMD_INTERVAL_LEN / INTERVAL_USEC_TO_SEC); /* Estimate the cycles to cover all intervals. */ - total_cycles *= intervals; - busy_cycles = get_interval_values(pmd->busy_cycles_intrvl, - &pmd->intrvl_idx, - intervals); - if (busy_cycles > total_cycles) { - busy_cycles = total_cycles; + total_pmd_cycles *= intervals; + busy_pmd_cycles = get_interval_values(pmd->busy_cycles_intrvl, + &pmd->intrvl_idx, + intervals); + if (busy_pmd_cycles > total_pmd_cycles) { + busy_pmd_cycles = total_pmd_cycles; } @@ -922,7 +922,7 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd, ? "(enabled) " : "(disabled)"); ds_put_format(reply, " pmd usage: "); - if (total_cycles) { + if (total_pmd_cycles) { ds_put_format(reply, "%2"PRIu64"", - rxq_proc_cycles * 100 / total_cycles); + rxq_proc_cycles * 100 / total_pmd_cycles); ds_put_cstr(reply, " %"); } else { @@ -934,12 +934,12 @@ pmd_info_show_rxq(struct ds *reply, struct dp_netdev_pmd_thread *pmd, if (n_rxq > 0) { ds_put_cstr(reply, " overhead: "); - if (total_cycles) { + if (total_pmd_cycles) { uint64_t overhead_cycles = 0; - if (total_rxq_proc_cycles < busy_cycles) { - overhead_cycles = busy_cycles - total_rxq_proc_cycles; + if (total_rxq_proc_cycles < busy_pmd_cycles) { + overhead_cycles = busy_pmd_cycles - total_rxq_proc_cycles; } ds_put_format(reply, "%2"PRIu64" %%", - overhead_cycles * 100 / total_cycles); + overhead_cycles * 100 / total_pmd_cycles); } else { ds_put_cstr(reply, "NOT AVAIL");