From patchwork Wed Jun 28 12:42:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 781652 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wyMtG4ps7z9s82 for ; Wed, 28 Jun 2017 22:44:06 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 42C6FAA6; Wed, 28 Jun 2017 12:43:26 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 47E23982 for ; Wed, 28 Jun 2017 12:43:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 82B0718F for ; Wed, 28 Jun 2017 12:43:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DDC3340F0E; Wed, 28 Jun 2017 12:43:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DDC3340F0E Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ktraynor@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com DDC3340F0E Received: from ktraynor.remote.csb (ovpn-117-248.ams2.redhat.com [10.36.117.248]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5B62688FF3; Wed, 28 Jun 2017 12:43:21 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org, ian.stokes@intel.com, jan.scheurich@ericsson.com, bhanuprakash.bodireddy@intel.com, ciara.loftus@intel.com Date: Wed, 28 Jun 2017 13:42:46 +0100 Message-Id: <1498653773-13757-2-git-send-email-ktraynor@redhat.com> In-Reply-To: <1498653773-13757-1-git-send-email-ktraynor@redhat.com> References: <1494002063-12269-1-git-send-email-ktraynor@redhat.com> <1498653773-13757-1-git-send-email-ktraynor@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 28 Jun 2017 12:43:23 +0000 (UTC) X-Spam-Status: No, score=-6.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 1/8] dpif-netdev: Change definitions of 'idle' & 'processing' cycles X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Ciara Loftus Instead of counting all polling cycles as processing cycles, only count the cycles where packets were received from the polling. Signed-off-by: Georg Schmuecking Signed-off-by: Ciara Loftus Co-authored-by: Georg Schmuecking Acked-by: Kevin Traynor --- lib/dpif-netdev.c | 58 +++++++++++++++++++++++++++++++++++----------- vswitchd/ovs-vswitchd.8.in | 5 +++- 2 files changed, 48 insertions(+), 15 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 4e29085..6bae4ac 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -327,6 +327,7 @@ enum dp_stat_type { enum pmd_cycles_counter_type { - PMD_CYCLES_POLLING, /* Cycles spent polling NICs. */ - PMD_CYCLES_PROCESSING, /* Cycles spent processing packets */ + PMD_CYCLES_IDLE, /* Cycles spent idle or unsuccessful polling */ + PMD_CYCLES_PROCESSING, /* Cycles spent successfully polling and + * processing polled packets */ PMD_N_CYCLES }; @@ -805,8 +806,8 @@ pmd_info_show_stats(struct ds *reply, ds_put_format(reply, - "\tpolling cycles:%"PRIu64" (%.02f%%)\n" + "\tidle cycles:%"PRIu64" (%.02f%%)\n" "\tprocessing cycles:%"PRIu64" (%.02f%%)\n", - cycles[PMD_CYCLES_POLLING], - cycles[PMD_CYCLES_POLLING] / (double)total_cycles * 100, + cycles[PMD_CYCLES_IDLE], + cycles[PMD_CYCLES_IDLE] / (double)total_cycles * 100, cycles[PMD_CYCLES_PROCESSING], cycles[PMD_CYCLES_PROCESSING] / (double)total_cycles * 100); @@ -3080,5 +3081,18 @@ cycles_count_end(struct dp_netdev_pmd_thread *pmd, } -static void +/* Calculate the intermediate cycle result and add to the counter 'type' */ +static inline void +cycles_count_intermediate(struct dp_netdev_pmd_thread *pmd, + enum pmd_cycles_counter_type type) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + unsigned long long new_cycles = cycles_counter(); + unsigned long long interval = new_cycles - pmd->last_cycles; + pmd->last_cycles = new_cycles; + + non_atomic_ullong_add(&pmd->cycles.n[type], interval); +} + +static int dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, struct netdev_rxq *rx, @@ -3087,15 +3101,13 @@ dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, struct dp_packet_batch batch; int error; + int batch_cnt = 0; dp_packet_batch_init(&batch); - cycles_count_start(pmd); error = netdev_rxq_recv(rx, &batch); - cycles_count_end(pmd, PMD_CYCLES_POLLING); if (!error) { *recirc_depth_get() = 0; - cycles_count_start(pmd); + batch_cnt = batch.count; dp_netdev_input(pmd, &batch, port_no); - cycles_count_end(pmd, PMD_CYCLES_PROCESSING); } else if (error != EAGAIN && error != EOPNOTSUPP) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); @@ -3104,4 +3116,6 @@ dp_netdev_process_rxq_port(struct dp_netdev_pmd_thread *pmd, netdev_rxq_get_name(rx), ovs_strerror(error)); } + + return batch_cnt; } @@ -3566,4 +3580,5 @@ dpif_netdev_run(struct dpif *dpif) struct dp_netdev_pmd_thread *non_pmd; uint64_t new_tnl_seq; + int process_packets = 0; ovs_mutex_lock(&dp->port_mutex); @@ -3571,4 +3586,5 @@ dpif_netdev_run(struct dpif *dpif) if (non_pmd) { ovs_mutex_lock(&dp->non_pmd_mutex); + cycles_count_start(non_pmd); HMAP_FOR_EACH (port, node, &dp->ports) { if (!netdev_is_pmd(port->netdev)) { @@ -3576,9 +3592,15 @@ dpif_netdev_run(struct dpif *dpif) for (i = 0; i < port->n_rxq; i++) { - dp_netdev_process_rxq_port(non_pmd, port->rxqs[i].rx, - port->port_no); + process_packets = + dp_netdev_process_rxq_port(non_pmd, + port->rxqs[i].rx, + port->port_no); + cycles_count_intermediate(non_pmd, process_packets ? + PMD_CYCLES_PROCESSING + : PMD_CYCLES_IDLE); } } } + cycles_count_end(non_pmd, PMD_CYCLES_IDLE); dpif_netdev_xps_revalidate_pmd(non_pmd, time_msec(), false); ovs_mutex_unlock(&dp->non_pmd_mutex); @@ -3705,4 +3727,5 @@ pmd_thread_main(void *f_) int poll_cnt; int i; + int process_packets = 0; poll_list = NULL; @@ -3731,8 +3754,13 @@ reload: } + cycles_count_start(pmd); for (;;) { for (i = 0; i < poll_cnt; i++) { - dp_netdev_process_rxq_port(pmd, poll_list[i].rx, - poll_list[i].port_no); + process_packets = + dp_netdev_process_rxq_port(pmd, poll_list[i].rx, + poll_list[i].port_no); + cycles_count_intermediate(pmd, + process_packets ? PMD_CYCLES_PROCESSING + : PMD_CYCLES_IDLE); } @@ -3755,4 +3783,6 @@ reload: } + cycles_count_end(pmd, PMD_CYCLES_IDLE); + poll_cnt = pmd_load_queues_and_ports(pmd, &poll_list); exiting = latch_is_set(&pmd->exit_latch); diff --git a/vswitchd/ovs-vswitchd.8.in b/vswitchd/ovs-vswitchd.8.in index 9178737..bd138bf 100644 --- a/vswitchd/ovs-vswitchd.8.in +++ b/vswitchd/ovs-vswitchd.8.in @@ -270,5 +270,8 @@ packets received by the datapath. Cycles are counted using the TSC or similar facilities (when available on the platform). To reset these counters use \fBdpif-netdev/pmd-stats-clear\fR. The duration of one cycle depends on the -measuring infrastructure. +measuring infrastructure. ``idle cycles'' refers to cycles spent polling +devices but not receiving any packets. ``processing cycles'' refers to cycles +spent polling devices and sucessfully receiving packets, plus the cycles +spent processing said packets. .IP "\fBdpif-netdev/pmd-stats-clear\fR [\fIdp\fR]" Resets to zero the per pmd thread performance numbers shown by the