From patchwork Wed Jan 10 03:05:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yuanhan Liu X-Patchwork-Id: 857885 X-Patchwork-Delegate: ian.stokes@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=fridaylinux.org header.i=@fridaylinux.org header.b="MnACg0kk"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.b="gPUwlEXl"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zGYmN6WhGz9s4s for ; Wed, 10 Jan 2018 14:05:40 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 9DCBCE1D; Wed, 10 Jan 2018 03:05:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 605CADF1 for ; Wed, 10 Jan 2018 03:05:37 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 7576FA3 for ; Wed, 10 Jan 2018 03:05:36 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 4647020CF8; Tue, 9 Jan 2018 22:05:35 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Tue, 09 Jan 2018 22:05:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fridaylinux.org; h=cc:date:from:message-id:subject:to:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=EleAjJYxWLbM4x78rl2rWSs5ELUFXU4g2CRhat4Xu IE=; b=MnACg0kkH15GwvX9/K7GxVJA6x3rw8xCeBOvkzS48u6tnqdlcTP1A6GFt GPErS6qaaTjiDkw5ptoFWCganJl+Dvbw8Zn6tVrcZCV29A0rCSopg8kbNsRPc8mA sa1F1MZEIATTj0xkwAyA5aWdfa7/FyXqDTZ5b7q7y6FfkFNKtlMTRQFO3u3JGHjK 8uwkVA0RPdQubkj4i9l9Fj2jFOe5xMspytPmW7G171WgU8x+5jHg3rDfi610q8hW rHj/qQ6J6b+dlDNVO0MMLvYMyLBOFYkiWs1BLO1NhFC7cHKiPP2kVUTknIg8F8B7 SajMrR5FOfkjO4ZY54nkOxPc2FIWw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:message-id:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=EleAjJYxWLbM4x78r l2rWSs5ELUFXU4g2CRhat4XuIE=; b=gPUwlEXlG2TeeVq8l1J9pmoa8ZmH/p0ue +wsndCJFMKQMSr0CzUwRx60+sD7YCmBJ7/qJHJE1BQho38gfOBQvXt100O5jWjTY e7/vpSBQ93xPBjXyybwXoSNDNDVFazr9bDIgaBVCTM3ZaO86uWhhwvfpURdSy53b XfswsG0yw8jq+ZGey1Wp2AziMaZdYzF++ZybmfK/wcH56tfa2ZlDlX479Dm8TS82 ihZXh0cuY398oWwd8fwoy/FKhCRLaH4UZyhoKUldG5ZKB8be/EaTxr/N1le/pEil T8+Z/CjT1tAuLK31y1nms7uzxDGJmhEpYOOs7dY0/a+XdPgfrUnoQ== X-ME-Sender: Received: from yliu-mob.mtl.com (unknown [115.148.91.233]) by mail.messagingengine.com (Postfix) with ESMTPA id BB13924599; Tue, 9 Jan 2018 22:05:32 -0500 (EST) From: Yuanhan Liu To: dev@openvswitch.org Date: Wed, 10 Jan 2018 11:05:29 +0800 Message-Id: <1515553529-5857-1-git-send-email-yliu@fridaylinux.org> X-Mailer: git-send-email 2.7.4 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Shahaf Shuler , Thomas Monjalon Subject: [ovs-dev] [PATCH v3] netdev-dpdk: fix port addition for ports sharing same PCI id X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Some NICs have only one PCI address associated with multiple ports. This patch extends the dpdk-devargs option's format to cater for such devices. To achieve that, this patch uses a new syntax that will be adapted and implemented in future DPDK release (likely, v18.05): http://dpdk.org/ml/archives/dev/2017-December/084234.html And since it's the DPDK duty to parse the (complete and full) syntax and this patch is more likely to serve as an intermediate workaround, here I take a simpler and shorter syntax from it (note it's allowed to have only one category being provided): class=eth,mac=00:11:22:33:44:55:66 Also, old compatibility is kept. Users can still go on with using the PCI id to add a port (if that's enough for them). Meaning, this patch will not break anything. This patch is basically based on the one from Ciara: https://mail.openvswitch.org/pipermail/ovs-dev/2017-October/339496.html Cc: Loftus Ciara Cc: Thomas Monjalon Cc: Kevin Traynor Signed-off-by: Yuanhan Liu --- v3: - reuse eth helper functions from OVS, suggested by Ilya Maximets v2: - added doc for the new devags syntax - added error log for wrong mac format --- Documentation/howto/dpdk.rst | 12 ++++++++ lib/netdev-dpdk.c | 70 ++++++++++++++++++++++++++++++++++---------- 2 files changed, 67 insertions(+), 15 deletions(-) diff --git a/Documentation/howto/dpdk.rst b/Documentation/howto/dpdk.rst index 2393c2f..e7cb7dd 100644 --- a/Documentation/howto/dpdk.rst +++ b/Documentation/howto/dpdk.rst @@ -48,6 +48,18 @@ number of dpdk devices found in the log file:: $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ options:dpdk-devargs=0000:01:00.1 +Some NICs (i.e. Mellanox ConnectX-3) have only one PCI address associated +with multiple ports. Using a PCI device like above won't work. Instead, below +usage is suggested:: + + $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \ + options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:01" + $ ovs-vsctl add-port br0 dpdk-p1 -- set Interface dpdk-p1 type=dpdk \ + options:dpdk-devargs="class=eth,mac=00:11:22:33:44:55:02" + +Note: such syntax won't support hotplug. The hotplug is supposed to work with +future DPDK release, v18.05. + After the DPDK ports get added to switch, a polling thread continuously polls DPDK devices and consumes 100% of the core, as can be checked from ``top`` and ``ps`` commands:: diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 364f545..c4f0f5e 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1203,29 +1203,69 @@ netdev_dpdk_lookup_by_port_id(dpdk_port_t port_id) } static dpdk_port_t +netdev_dpdk_get_port_by_mac(const char *mac_str) +{ + dpdk_port_t port_id; + struct eth_addr mac, port_mac; + + if (!eth_addr_from_string(mac_str, &mac)) { + VLOG_ERR("invalid mac: %s", mac_str); + return DPDK_ETH_PORT_ID_INVALID; + } + + RTE_ETH_FOREACH_DEV (port_id) { + struct ether_addr ea; + + rte_eth_macaddr_get(port_id, &ea); + memcpy(port_mac.ea, ea.addr_bytes, ETH_ADDR_LEN); + if (eth_addr_equals(mac, port_mac)) { + return port_id; + } + } + + return DPDK_ETH_PORT_ID_INVALID; +} + +/* + * Normally, a PCI id is enough for identifying a specific DPDK port. + * However, for some NICs having multiple ports sharing the same PCI + * id, using PCI id won't work then. + * + * To fix that, here one more method is introduced: "class=eth,mac=$MAC". + * + * Note that the compatibility is fully kept: user can still use the + * PCI id for adding ports (when it's enough for them). + */ +static dpdk_port_t netdev_dpdk_process_devargs(struct netdev_dpdk *dev, const char *devargs, char **errp) { - /* Get the name up to the first comma. */ - char *name = xmemdup0(devargs, strcspn(devargs, ",")); + char *name; dpdk_port_t new_port_id = DPDK_ETH_PORT_ID_INVALID; - if (rte_eth_dev_get_port_by_name(name, &new_port_id) - || !rte_eth_dev_is_valid_port(new_port_id)) { - /* Device not found in DPDK, attempt to attach it */ - if (!rte_eth_dev_attach(devargs, &new_port_id)) { - /* Attach successful */ - dev->attached = true; - VLOG_INFO("Device '%s' attached to DPDK", devargs); - } else { - /* Attach unsuccessful */ - new_port_id = DPDK_ETH_PORT_ID_INVALID; - VLOG_WARN_BUF(errp, "Error attaching device '%s' to DPDK", - devargs); + if (strncmp(devargs, "class=eth,mac=", 14) == 0) { + new_port_id = netdev_dpdk_get_port_by_mac(&devargs[14]); + } else { + name = xmemdup0(devargs, strcspn(devargs, ",")); + if (rte_eth_dev_get_port_by_name(name, &new_port_id) + || !rte_eth_dev_is_valid_port(new_port_id)) { + /* Device not found in DPDK, attempt to attach it */ + if (!rte_eth_dev_attach(devargs, &new_port_id)) { + /* Attach successful */ + dev->attached = true; + VLOG_INFO("Device '%s' attached to DPDK", devargs); + } else { + /* Attach unsuccessful */ + new_port_id = DPDK_ETH_PORT_ID_INVALID; + } } + free(name); + } + + if (new_port_id == DPDK_ETH_PORT_ID_INVALID) { + VLOG_WARN_BUF(errp, "Error attaching device '%s' to DPDK", devargs); } - free(name); return new_port_id; }