From patchwork Wed Aug 23 13:34:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 804980 X-Patchwork-Delegate: dlu998@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xcpSg1B5Fz9s83 for ; Wed, 23 Aug 2017 23:39:47 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 71384B3F; Wed, 23 Aug 2017 13:36:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A3F12B1E for ; Wed, 23 Aug 2017 13:36:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id D2A56458 for ; Wed, 23 Aug 2017 13:36:03 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 51639C04B930; Wed, 23 Aug 2017 13:36:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 51639C04B930 Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=ktraynor@redhat.com Received: from ktraynor.remote.csb (unknown [10.36.118.65]) by smtp.corp.redhat.com (Postfix) with ESMTP id C1C1160C28; Wed, 23 Aug 2017 13:35:58 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org, dball@vmware.com, ian.stokes@intel.com, jan.scheurich@ericsson.com, bhanuprakash.bodireddy@intel.com, mark.b.kavanagh@intel.com, gvrose8192@gmail.com, fbl@redhat.com Date: Wed, 23 Aug 2017 14:34:41 +0100 Message-Id: <1503495281-1741-7-git-send-email-ktraynor@redhat.com> In-Reply-To: <1503495281-1741-1-git-send-email-ktraynor@redhat.com> References: <1502293530-10783-1-git-send-email-ktraynor@redhat.com> <1503495281-1741-1-git-send-email-ktraynor@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Wed, 23 Aug 2017 13:36:03 +0000 (UTC) Subject: [ovs-dev] [PATCH v5 6/6] dpif-netdev: Add ovs-appctl dpif-netdev/pmd-rxq-rebalance. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Rxqs consumed processing cycles are used to improve the balance of how rxqs are assigned to pmds. Currently some reconfiguration is needed to perform a reassignment. Add an ovs-appctl command to perform a new assignment in order to balance based on the latest rxq processing cycle information. Note: Jan requested this for testing purposes. Suggested-by: Jan Scheurich Signed-off-by: Kevin Traynor --- Documentation/howto/dpdk.rst | 5 ++++- lib/dpif-netdev.c | 35 +++++++++++++++++++++++++++++++++++ vswitchd/ovs-vswitchd.8.in | 2 ++ 3 files changed, 41 insertions(+), 1 deletion(-) diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 493e215..5d009bd 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -140,5 +140,8 @@ Core 7: Q4 (70%) | Q5 (10%) core 8: Q0 (60%) | Q0 (30%) -Rxq to pmds assignment takes place whenever there are configuration changes. +Rxq to pmds assignment takes place whenever there are configuration changes +or can be triggered by using:: + + $ ovs-appctl dpif-netdev/pmd-rxq-rebalance QoS diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 6cc0a1e..4ea1c07 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -723,4 +723,6 @@ static inline bool emc_entry_alive(struct emc_entry *ce); static void emc_clear_entry(struct emc_entry *ce); +static void dp_netdev_request_reconfigure(struct dp_netdev *dp); + static void emc_cache_init(struct emc_cache *flow_cache) @@ -1016,4 +1018,34 @@ sorted_poll_thread_list(struct dp_netdev *dp, static void +dpif_netdev_pmd_rebalance(struct unixctl_conn *conn, int argc, + const char *argv[], void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + struct dp_netdev *dp = NULL; + + ovs_mutex_lock(&dp_netdev_mutex); + + if (argc == 2) { + dp = shash_find_data(&dp_netdevs, argv[1]); + } else if (shash_count(&dp_netdevs) == 1) { + /* There's only one datapath */ + dp = shash_first(&dp_netdevs)->data; + } + + if (!dp) { + ovs_mutex_unlock(&dp_netdev_mutex); + unixctl_command_reply_error(conn, + "please specify an existing datapath"); + return; + } + + dp_netdev_request_reconfigure(dp); + ovs_mutex_unlock(&dp_netdev_mutex); + ds_put_cstr(&reply, "pmd rxq rebalance requested.\n"); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + +static void dpif_netdev_pmd_info(struct unixctl_conn *conn, int argc, const char *argv[], void *aux) @@ -1093,4 +1125,7 @@ dpif_netdev_init(void) 0, 1, dpif_netdev_pmd_info, (void *)&poll_aux); + unixctl_command_register("dpif-netdev/pmd-rxq-rebalance", "[dp]", + 0, 1, dpif_netdev_pmd_rebalance, + NULL); return 0; } diff --git a/vswitchd/ovs-vswitchd.8.in b/vswitchd/ovs-vswitchd.8.in index dfd209e..c18baf6 100644 --- a/vswitchd/ovs-vswitchd.8.in +++ b/vswitchd/ovs-vswitchd.8.in @@ -281,4 +281,6 @@ bridge statistics, only the values shown by the above command. For each pmd thread of the datapath \fIdp\fR shows list of queue-ids with port names, which this thread polls. +.IP "\fBdpif-netdev/pmd-rxq-rebalance\fR [\fIdp\fR]" +Reassigns rxqs to pmds in the datapath \fIdp\fR based on their current usage. . .so ofproto/ofproto-dpif-unixctl.man