From patchwork Thu Aug 24 23:38:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 805699 X-Patchwork-Delegate: dlu998@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xdgq24nQsz9t3f for ; Fri, 25 Aug 2017 09:43:42 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id A9BEAB0B; Thu, 24 Aug 2017 23:42:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 85544AE7 for ; Thu, 24 Aug 2017 23:42:21 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0C9B3471 for ; Thu, 24 Aug 2017 23:39:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DDDADC0587DB; Thu, 24 Aug 2017 23:38:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com DDDADC0587DB Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=ktraynor@redhat.com Received: from ktraynor.remote.csb (unknown [10.36.118.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8CC7E7F38E; Thu, 24 Aug 2017 23:38:28 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org, dball@vmware.com, ian.stokes@intel.com, jan.scheurich@ericsson.com, bhanuprakash.bodireddy@intel.com, mark.b.kavanagh@intel.com, gvrose8192@gmail.com, fbl@redhat.com Date: Fri, 25 Aug 2017 00:38:01 +0100 Message-Id: <1503617885-32501-3-git-send-email-ktraynor@redhat.com> In-Reply-To: <1503617885-32501-1-git-send-email-ktraynor@redhat.com> References: <1503495281-1741-1-git-send-email-ktraynor@redhat.com> <1503617885-32501-1-git-send-email-ktraynor@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 24 Aug 2017 23:38:32 +0000 (UTC) X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v6 2/6] dpif-netdev: Add rxq processing cycle counters. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Add counters to dp_netdev_rxq which will later be used for storing the processing cycles of an rxq. Processing cycles will be stored in reference to a defined time interval. We will store the cycles of the current in progress interval, a number of completed intervals and the sum of the completed intervals. cycles_count_intermediate was used to count cycles for a pmd. With some small additions we can also use it to count the cycles used for processing an rxq. Signed-off-by: Kevin Traynor --- lib/dpif-netdev.c | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index f35c079..8731435 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -182,4 +182,8 @@ struct emc_cache { #define DPCLS_OPTIMIZATION_INTERVAL 1000 +/* Number of intervals for which cycles are stored + * and used during rxq to pmd assignment. */ +#define PMD_RXQ_INTERVAL_MAX 6 + struct dpcls { struct cmap_node node; /* Within dp_netdev_pmd_thread.classifiers */ @@ -340,4 +344,13 @@ enum pmd_cycles_counter_type { }; +enum rxq_cycles_counter_type { + RXQ_CYCLES_PROC_CURR, /* Cycles spent successfully polling and + processing packets during the current + interval. */ + RXQ_CYCLES_PROC_HIST, /* Total cycles of all intervals that are used + during rxq to pmd assignment. */ + RXQ_N_CYCLES +}; + #define XPS_TIMEOUT_MS 500LL @@ -351,4 +364,11 @@ struct dp_netdev_rxq { particular core. */ struct dp_netdev_pmd_thread *pmd; /* pmd thread that polls this queue. */ + + /* Counters of cycles spent successfully polling and processing pkts. */ + atomic_ullong cycles[RXQ_N_CYCLES]; + /* We store PMD_RXQ_INTERVAL_MAX intervals of data for an rxq and then + sum them to yield the cycles used for an rxq. */ + atomic_ullong cycles_intrvl[PMD_RXQ_INTERVAL_MAX]; + unsigned intrvl_idx; /* Write index for 'cycles_intrvl'. */ }; @@ -677,5 +697,4 @@ static void pmd_load_cached_ports(struct dp_netdev_pmd_thread *pmd) static inline void dp_netdev_pmd_try_optimize(struct dp_netdev_pmd_thread *pmd); - static void dpif_netdev_xps_revalidate_pmd(const struct dp_netdev_pmd_thread *pmd, @@ -3092,4 +3111,5 @@ cycles_count_end(struct dp_netdev_pmd_thread *pmd, static inline void cycles_count_intermediate(struct dp_netdev_pmd_thread *pmd, + struct dp_netdev_rxq *rxq, enum pmd_cycles_counter_type type) OVS_NO_THREAD_SAFETY_ANALYSIS @@ -3100,4 +3120,8 @@ cycles_count_intermediate(struct dp_netdev_pmd_thread *pmd, non_atomic_ullong_add(&pmd->cycles.n[type], interval); + if (rxq && (type == PMD_CYCLES_PROCESSING)) { + /* Add to the amount of current processing cycles. */ + non_atomic_ullong_add(&rxq->cycles[RXQ_CYCLES_PROC_CURR], interval); + } } @@ -3668,5 +3692,5 @@ dpif_netdev_run(struct dpif *dpif) port->rxqs[i].rx, port->port_no); - cycles_count_intermediate(non_pmd, process_packets ? + cycles_count_intermediate(non_pmd, NULL, process_packets ? PMD_CYCLES_PROCESSING : PMD_CYCLES_IDLE); @@ -3855,5 +3879,5 @@ reload: dp_netdev_process_rxq_port(pmd, poll_list[i].rxq->rx, poll_list[i].port_no); - cycles_count_intermediate(pmd, + cycles_count_intermediate(pmd, NULL, process_packets ? PMD_CYCLES_PROCESSING : PMD_CYCLES_IDLE);