From patchwork Fri Nov 10 01:28:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chandran, Sugesh" X-Patchwork-Id: 836579 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yY2Ws6Xyjz9t2R for ; Fri, 10 Nov 2017 12:29:45 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 78F91A92; Fri, 10 Nov 2017 01:28:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 964FAA74 for ; Fri, 10 Nov 2017 01:28:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 46981136 for ; Fri, 10 Nov 2017 01:28:39 +0000 (UTC) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga105.jf.intel.com with ESMTP; 09 Nov 2017 17:28:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,371,1505804400"; d="scan'208";a="148132068" Received: from silpixa00389820.ir.intel.com ([10.237.222.155]) by orsmga004.jf.intel.com with ESMTP; 09 Nov 2017 17:28:38 -0800 From: Sugesh Chandran To: dev@openvswitch.org, blp@ovn.org Date: Fri, 10 Nov 2017 01:28:33 +0000 Message-Id: <1510277313-44817-3-git-send-email-sugesh.chandran@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1510277313-44817-1-git-send-email-sugesh.chandran@intel.com> References: <1510277313-44817-1-git-send-email-sugesh.chandran@intel.com> X-Spam-Status: No, score=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 2/2] Adding configuration option to whitelist DPDK physical ports. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Adding a OVS configuration option to whitelist DPDK physical ports. By default running multiple instances of DPDK on a single platform cannot use physical ports at the same time even though they are distinct. The eal init scans all the ports that are bound to DPDK and initialize the drivers accordingly. This happens for every DPDK process init. On a multi instance deployment usecase, it causes issues for using physical NIC ports. Consider a two DPDK process that are running on a single platform, the second DPDK primary process will try to initialize the drivers for all the physical ports even though it may be used in first DPDK process. To avoid this situation user can whitelist the ports for each DPDK application. Whitelisting of ports/PCI-ID in a DPDK process will limit the eal-init only on those ports. To whitelist two physical ports "0000:06:00.0" and "0000:06:00.1", the configuration option in OVS would be ovs-vsctl set Open_vSwitch . other_config:dpdk-whitelist-pci-ids="0000:06:00.0,0000:06:00.1" To update the whitelist ports, OVS daemon has to be restarted. Signed-off-by: Sugesh Chandran --- lib/dpdk.c | 29 +++++++++++++++++++++++++++++ vswitchd/vswitch.xml | 21 +++++++++++++++++++++ 2 files changed, 50 insertions(+) diff --git a/lib/dpdk.c b/lib/dpdk.c index 9d187c7..0f11977 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -323,6 +323,34 @@ dpdk_isolate_rte_mem_config(const struct smap *ovs_other_config, } static void +dpdk_whitelist_pci_ids(const struct smap *ovs_other_config, char ***argv, + int *argc) +{ + const char *pci_ids; + char *pci_dev; + int len; + int i; + pci_ids = smap_get(ovs_other_config, "dpdk-whitelist-pci-ids"); + if (!pci_ids) { + return; + } + len = strlen(pci_ids); + do { + i = strcspn(pci_ids, ","); + pci_dev = xmemdup0(pci_ids, i); + if (!strlen(pci_dev)) { + break; + } + *argv = grow_argv(argv, *argc, 2); + (*argv)[(*argc)++] = xstrdup("-w"); + (*argv)[(*argc)++] = pci_dev; + i++; + pci_ids += i; + len -= i; + } while (pci_ids && len > 0); +} + +static void dpdk_init__(const struct smap *ovs_other_config) { char **argv = NULL, **argv_to_release = NULL; @@ -409,6 +437,7 @@ dpdk_init__(const struct smap *ovs_other_config) } dpdk_isolate_rte_mem_config(ovs_other_config, &argv, &argc); + dpdk_whitelist_pci_ids(ovs_other_config, &argv, &argc); argv = grow_argv(&argv, argc, 1); argv[argc] = NULL; diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 7462b30..0b64b25 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -442,6 +442,27 @@

+ +

+ Specifies list of pci-ids separated by , for whitelisting available + physical NIC ports in OVS. The option valid only when DPDK is enabled + in OVS and the ports are already bound to DPDK userspace driver. +

+

+ By default, all the DPDK bound ports are initialized at the time of + vswitchd start. Whitelisting a list of pci-ids is used to limit the + initialization only to specific ports. Changing this value requires + restarting the daemon. +

+

+ This option allow user to run multiple instance of DPDK including + vswitchd simultaneously by exclusively allocating the physical NIC + ports between them. Its impossible to use the NIC ports in + multiple primary DPDK processes without whitelisting them + even the application is using distinct ports. +

+ +