From patchwork Wed Oct 18 16:01:26 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827685 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHGzB70Z9z9t5q for ; Thu, 19 Oct 2017 03:02:14 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 04867BE8; Wed, 18 Oct 2017 16:01:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 71AADBE4 for ; Wed, 18 Oct 2017 16:01:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A8E1351D for ; Wed, 18 Oct 2017 16:01:37 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223467" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:35 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:26 +0100 Message-Id: <1508342491-21949-2-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 1/6] netdev-dpdk: fix management of pre-existing mempools. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Fix an issue on reconfiguration of pre-existing mempools. This patch avoids to call dpdk_mp_put() - and erroneously release the mempool - when it already exists. CC: Mark B Kavanagh CC: Kevin Traynor CC: Aaron Conole CC: Darrell Ball Reported-by: Ciara Loftus Tested-by: Ciara Loftus Reported-by: Róbert Mulik Fixes: d555d9bded5f ("netdev-dpdk: Create separate memory pool for each port.") Signed-off-by: Antonio Fischetti Acked-by: Kevin Traynor --- I've tested this patch by - changing at run-time the number of Rx queues: ovs-vsctl set Interface dpdk0 type=dpdk options:n_rxq=4 - reducing the MTU of the dpdk ports of 1 byte to force the configuration of an existing mempool: ovs-vsctl set Interface dpdk0 mtu_request=1499 This issue was observed in a PVP test topology with dpdkvhostuserclient ports. It can happen also with dpdk type ports, eg by reducing the MTU of 1 byte. To replicate the bug scenario in the PVP case it's sufficient to set 1 dpdkvhostuserclient port, and just boot the VM. Below some more details on my own test setup. PVP test setup -------------- CLIENT_SOCK_DIR=/tmp SOCK0=dpdkvhostuser0 SOCK1=dpdkvhostuser1 1 PMD Add 2 dpdk ports, n_rxq=1 Add 2 vhu ports both of type dpdkvhostuserclient and specify vhost-server-path ovs-vsctl set Interface dpdkvhostuser0 options:vhost-server-path="$CLIENT_SOCK_DIR/$SOCK0" ovs-vsctl set Interface dpdkvhostuser1 options:vhost-server-path="$CLIENT_SOCK_DIR/$SOCK1" Set port-based rules: dpdk0 <--> vhu0 and dpdk1 <--> vhu1 add-flow br0 in_port=1,action=output:3 add-flow br0 in_port=3,action=output:1 add-flow br0 in_port=4,action=output:2 add-flow br0 in_port=2,action=output:4 Launch QEMU ----------- As OvS vhu ports are acting as clients, we must specify 'server' in the next command. VM_IMAGE= sudo -E taskset 0x3F00 $QEMU_DIR/x86_64-softmmu/qemu-system-x86_64 -name us-vhost-vm1 -cpu host -enable-kvm -m 4096M -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc -smp 4 -drive file=$VM_IMAGE -chardev socket,id=char0,path=$CLIENT_SOCK_DIR/$SOCK0,server -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off -chardev socket,id=char1,path=$CLIENT_SOCK_DIR/$SOCK1,server -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off --nographic Expected behavior ----------------- With this fix OvS shouldn't crash. --- lib/netdev-dpdk.c | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index c60f46f..d49afd8 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -508,12 +508,13 @@ dpdk_mp_name(struct dpdk_mp *dmp) } static struct dpdk_mp * -dpdk_mp_create(struct netdev_dpdk *dev, int mtu) +dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) { struct dpdk_mp *dmp = dpdk_rte_mzalloc(sizeof *dmp); if (!dmp) { return NULL; } + *mp_exists = false; dmp->socket_id = dev->requested_socket_id; dmp->mtu = mtu; ovs_strzcpy(dmp->if_name, dev->up.name, IFNAMSIZ); @@ -530,8 +531,6 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu) + MIN(RTE_MAX_LCORE, dev->requested_n_rxq) * NETDEV_MAX_BURST + MIN_NB_MBUF; - bool mp_exists = false; - do { char *mp_name = dpdk_mp_name(dmp); @@ -559,7 +558,7 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu) /* As the mempool create returned EEXIST we can expect the * lookup has returned a valid pointer. If for some reason * that's not the case we keep track of it. */ - mp_exists = true; + *mp_exists = true; } else { VLOG_ERR("Failed mempool \"%s\" create request of %u mbufs", mp_name, dmp->mp_size); @@ -573,7 +572,7 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu) rte_mempool_obj_iter(dmp->mp, ovs_rte_pktmbuf_init, NULL); return dmp; } - } while (!mp_exists && + } while (!(*mp_exists) && (rte_errno == ENOMEM && (dmp->mp_size /= 2) >= MIN_NB_MBUF)); rte_free(dmp); @@ -581,12 +580,12 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu) } static struct dpdk_mp * -dpdk_mp_get(struct netdev_dpdk *dev, int mtu) +dpdk_mp_get(struct netdev_dpdk *dev, int mtu, bool *mp_exists) { struct dpdk_mp *dmp; ovs_mutex_lock(&dpdk_mp_mutex); - dmp = dpdk_mp_create(dev, mtu); + dmp = dpdk_mp_create(dev, mtu, mp_exists); ovs_mutex_unlock(&dpdk_mp_mutex); return dmp; @@ -610,9 +609,11 @@ dpdk_mp_put(struct dpdk_mp *dmp) ovs_mutex_unlock(&dpdk_mp_mutex); } -/* Tries to allocate new mempool on requested_socket_id with - * mbuf size corresponding to requested_mtu. - * On success new configuration will be applied. +/* Tries to allocate a new mempool - or re-use an existing one where + * appropriate - on requested_socket_id with mbuf size corresponding to + * requested_mtu. + * On success - or when re-using an existing mempool - the new configuration + * will be applied. * On error, device will be left unchanged. */ static int netdev_dpdk_mempool_configure(struct netdev_dpdk *dev) @@ -620,14 +621,22 @@ netdev_dpdk_mempool_configure(struct netdev_dpdk *dev) { uint32_t buf_size = dpdk_buf_size(dev->requested_mtu); struct dpdk_mp *mp; + bool mp_exists; - mp = dpdk_mp_get(dev, FRAME_LEN_TO_MTU(buf_size)); + mp = dpdk_mp_get(dev, FRAME_LEN_TO_MTU(buf_size), &mp_exists); if (!mp) { VLOG_ERR("Failed to create memory pool for netdev " "%s, with MTU %d on socket %d: %s\n", dev->up.name, dev->requested_mtu, dev->requested_socket_id, rte_strerror(rte_errno)); return rte_errno; + } else if (mp_exists) { + /* If a new MTU was requested and its rounded value equals the one + * that is currently used, then the existing mempool is returned. + * Update dev with the new values. */ + dev->mtu = dev->requested_mtu; + dev->max_packet_len = MTU_TO_FRAME_LEN(dev->mtu); + return EEXIST; } else { dpdk_mp_put(dev->dpdk_mp); dev->dpdk_mp = mp; @@ -3207,7 +3216,7 @@ netdev_dpdk_reconfigure(struct netdev *netdev) rte_eth_dev_stop(dev->port_id); err = netdev_dpdk_mempool_configure(dev); - if (err) { + if (err && err != EEXIST) { goto out; } @@ -3247,12 +3256,12 @@ dpdk_vhost_reconfigure_helper(struct netdev_dpdk *dev) netdev_dpdk_remap_txqs(dev); err = netdev_dpdk_mempool_configure(dev); - if (err) { - return err; - } else { + if (!err) { + /* A new mempool was created. */ netdev_change_seq_changed(&dev->up); + } else if (err != EEXIST){ + return err; } - if (netdev_dpdk_get_vid(dev) >= 0) { if (dev->vhost_reconfigured == false) { dev->vhost_reconfigured = true; From patchwork Wed Oct 18 16:01:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827686 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHH015GvNz9t4P for ; Thu, 19 Oct 2017 03:02:57 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 143FEBEA; Wed, 18 Oct 2017 16:01:41 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 9FFB6BF0 for ; Wed, 18 Oct 2017 16:01:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 4C7F251D for ; Wed, 18 Oct 2017 16:01:39 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223483" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:37 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:27 +0100 Message-Id: <1508342491-21949-3-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 2/6] netdev-dpdk: Fix mempool names to reflect socket id. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Create mempool names by considering also the NUMA socket number. So a name reflects what socket the mempool is allocated on. This change is needed for the NUMA-awareness feature. CC: Mark B Kavanagh CC: Kevin Traynor CC: Aaron Conole Reported-by: Ciara Loftus Tested-by: Ciara Loftus Fixes: d555d9bded5f ("netdev-dpdk: Create separate memory pool for each port.") Signed-off-by: Antonio Fischetti Acked-by: Kevin Traynor --- Mempool names now contains the requested socket id and become like: "ovs_4adb057e_1_2030_20512". Tested with DPDK 17.05.2 (from dpdk-stable branch). NUMA-awareness feature enabled (DPDK/config/common_base). Created 1 single dpdkvhostuser port type. OvS pmd-cpu-mask=FF00003 # enable cores on both numa nodes QEMU core mask = 0xFC000 # cores for qemu on numa node 1 only Before launching the VM: ------------------------ ovs-appctl dpif-netdev/pmd-rxq-show shows core #1 is serving the vhu port. pmd thread numa_id 0 core_id 1: isolated : false port: dpdkvhostuser0 queue-id: 0 After launching the VM: ----------------------- the vhu port is now managed by core #27 pmd thread numa_id 1 core_id 27: isolated : false port: dpdkvhostuser0 queue-id: 0 and the log shows a new mempool is allocated on NUMA node 1, while the previous one is deleted: 2017-10-06T14:04:55Z|00105|netdev_dpdk|DBG|Allocated "ovs_4adb057e_1_2030_20512" mempool with 20512 mbufs 2017-10-06T14:04:55Z|00106|netdev_dpdk|DBG|Releasing "ovs_4adb057e_0_2030_20512" mempool --- --- lib/netdev-dpdk.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index d49afd8..3155505 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -499,8 +499,8 @@ dpdk_mp_name(struct dpdk_mp *dmp) { uint32_t h = hash_string(dmp->if_name, 0); char *mp_name = xcalloc(RTE_MEMPOOL_NAMESIZE, sizeof *mp_name); - int ret = snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "ovs_%x_%d_%u", - h, dmp->mtu, dmp->mp_size); + int ret = snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "ovs_%x_%d_%d_%u", + h, dmp->socket_id, dmp->mtu, dmp->mp_size); if (ret < 0 || ret >= RTE_MEMPOOL_NAMESIZE) { return NULL; } @@ -535,9 +535,10 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) char *mp_name = dpdk_mp_name(dmp); VLOG_DBG("Requesting a mempool of %u mbufs for netdev %s " - "with %d Rx and %d Tx queues.", + "with %d Rx and %d Tx queues, socket id:%d.", dmp->mp_size, dev->up.name, - dev->requested_n_rxq, dev->requested_n_txq); + dev->requested_n_rxq, dev->requested_n_txq, + dev->requested_socket_id); dmp->mp = rte_pktmbuf_pool_create(mp_name, dmp->mp_size, MP_CACHE_SZ, From patchwork Wed Oct 18 16:01:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827687 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHH0y0wdmz9t4c for ; Thu, 19 Oct 2017 03:03:46 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id E3925BFF; Wed, 18 Oct 2017 16:01:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0C144BFA for ; Wed, 18 Oct 2017 16:01:42 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 78512528 for ; Wed, 18 Oct 2017 16:01:41 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:41 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223513" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:39 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:28 +0100 Message-Id: <1508342491-21949-4-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 3/6] netdev-dpdk: skip init for existing mempools. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Skip initialization of mempool packet areas if this was already done in a previous call to dpdk_mp_create. CC: Darrell Ball CC: Ciara Loftus CC: Kevin Traynor CC: Aaron Conole Acked-by: Mark Kavanagh Signed-off-by: Antonio Fischetti Acked-by: Kevin Traynor --- lib/netdev-dpdk.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 3155505..7c550f2 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -550,6 +550,11 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) if (dmp->mp) { VLOG_DBG("Allocated \"%s\" mempool with %u mbufs", mp_name, dmp->mp_size); + /* rte_pktmbuf_pool_create has done some initialization of the + * rte_mbuf part of each dp_packet. Some OvS specific fields + * of the packet still need to be initialized by + * ovs_rte_pktmbuf_init. */ + rte_mempool_obj_iter(dmp->mp, ovs_rte_pktmbuf_init, NULL); } else if (rte_errno == EEXIST) { /* A mempool with the same name already exists. We just * retrieve its pointer to be returned to the caller. */ @@ -566,11 +571,6 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) } free(mp_name); if (dmp->mp) { - /* rte_pktmbuf_pool_create has done some initialization of the - * rte_mbuf part of each dp_packet, while ovs_rte_pktmbuf_init - * initializes some OVS specific fields of dp_packet. - */ - rte_mempool_obj_iter(dmp->mp, ovs_rte_pktmbuf_init, NULL); return dmp; } } while (!(*mp_exists) && From patchwork Wed Oct 18 16:01:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827688 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHH240lWdz9t4P for ; Thu, 19 Oct 2017 03:04:44 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 2C8A8C0F; Wed, 18 Oct 2017 16:02:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7C55AC09 for ; Wed, 18 Oct 2017 16:02:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 26D1D52A for ; Wed, 18 Oct 2017 16:02:03 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:43 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223527" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:41 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:29 +0100 Message-Id: <1508342491-21949-5-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 4/6] netdev-dpdk: manage failure in mempool name creation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org In case a mempool name could not be generated log a message and return a null mempool pointer to the caller. CC: Mark B Kavanagh CC: Darrell Ball CC: Ciara Loftus CC: Kevin Traynor CC: Aaron Conole Signed-off-by: Antonio Fischetti --- lib/netdev-dpdk.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 7c550f2..a3b4638 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -502,6 +502,9 @@ dpdk_mp_name(struct dpdk_mp *dmp) int ret = snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "ovs_%x_%d_%d_%u", h, dmp->socket_id, dmp->mtu, dmp->mp_size); if (ret < 0 || ret >= RTE_MEMPOOL_NAMESIZE) { + VLOG_DBG("Failed to generate a mempool name for \"%s\". " + "Hash:0x%x, mtu:%d, mbufs:%u, %s", + dmp->if_name, h, dmp->mtu, dmp->mp_size, ovs_strerror(ret)); return NULL; } return mp_name; @@ -533,6 +536,10 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) do { char *mp_name = dpdk_mp_name(dmp); + if (!mp_name) { + rte_free(dmp); + return NULL; + } VLOG_DBG("Requesting a mempool of %u mbufs for netdev %s " "with %d Rx and %d Tx queues, socket id:%d.", From patchwork Wed Oct 18 16:01:30 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHH2m5MkHz9t4P for ; Thu, 19 Oct 2017 03:05:20 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 20FD7C0C; Wed, 18 Oct 2017 16:02:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 479F2BFD for ; Wed, 18 Oct 2017 16:02:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C686251D for ; Wed, 18 Oct 2017 16:02:04 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223537" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:43 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:30 +0100 Message-Id: <1508342491-21949-6-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 5/6] netdev-dpdk: Reword mp_size as n_mbufs. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org For code readability purposes mp_size is renamed as n_mbufs in dpdk_mp structure. This parameter is passed to rte mempool creation functions and is meant to contain the number of elements inside the requested mempool. CC: Ciara Loftus CC: Kevin Traynor CC: Aaron Conole Acked-by: Mark Kavanagh Signed-off-by: Antonio Fischetti Acked-by: Kevin Traynor --- lib/netdev-dpdk.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index a3b4638..0057e6b 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -308,7 +308,7 @@ struct dpdk_mp { int mtu; int socket_id; char if_name[IFNAMSIZ]; - unsigned mp_size; + unsigned n_mbufs; /* Number of mbufs inside the mempool. */ struct ovs_list list_node OVS_GUARDED_BY(dpdk_mp_mutex); }; @@ -500,11 +500,11 @@ dpdk_mp_name(struct dpdk_mp *dmp) uint32_t h = hash_string(dmp->if_name, 0); char *mp_name = xcalloc(RTE_MEMPOOL_NAMESIZE, sizeof *mp_name); int ret = snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "ovs_%x_%d_%d_%u", - h, dmp->socket_id, dmp->mtu, dmp->mp_size); + h, dmp->socket_id, dmp->mtu, dmp->n_mbufs); if (ret < 0 || ret >= RTE_MEMPOOL_NAMESIZE) { VLOG_DBG("Failed to generate a mempool name for \"%s\". " "Hash:0x%x, mtu:%d, mbufs:%u, %s", - dmp->if_name, h, dmp->mtu, dmp->mp_size, ovs_strerror(ret)); + dmp->if_name, h, dmp->mtu, dmp->n_mbufs, ovs_strerror(ret)); return NULL; } return mp_name; @@ -523,13 +523,13 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) ovs_strzcpy(dmp->if_name, dev->up.name, IFNAMSIZ); /* - * XXX: rough estimation of memory required for port: + * XXX: rough estimation of number of mbufs required for this port: * * + * + * + */ - dmp->mp_size = dev->requested_n_rxq * dev->requested_rxq_size + dmp->n_mbufs = dev->requested_n_rxq * dev->requested_rxq_size + dev->requested_n_txq * dev->requested_txq_size + MIN(RTE_MAX_LCORE, dev->requested_n_rxq) * NETDEV_MAX_BURST + MIN_NB_MBUF; @@ -543,11 +543,11 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) VLOG_DBG("Requesting a mempool of %u mbufs for netdev %s " "with %d Rx and %d Tx queues, socket id:%d.", - dmp->mp_size, dev->up.name, + dmp->n_mbufs, dev->up.name, dev->requested_n_rxq, dev->requested_n_txq, dev->requested_socket_id); - dmp->mp = rte_pktmbuf_pool_create(mp_name, dmp->mp_size, + dmp->mp = rte_pktmbuf_pool_create(mp_name, dmp->n_mbufs, MP_CACHE_SZ, sizeof (struct dp_packet) - sizeof (struct rte_mbuf), @@ -556,7 +556,7 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) dmp->socket_id); if (dmp->mp) { VLOG_DBG("Allocated \"%s\" mempool with %u mbufs", mp_name, - dmp->mp_size); + dmp->n_mbufs); /* rte_pktmbuf_pool_create has done some initialization of the * rte_mbuf part of each dp_packet. Some OvS specific fields * of the packet still need to be initialized by @@ -574,14 +574,14 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) *mp_exists = true; } else { VLOG_ERR("Failed mempool \"%s\" create request of %u mbufs", - mp_name, dmp->mp_size); + mp_name, dmp->n_mbufs); } free(mp_name); if (dmp->mp) { return dmp; } } while (!(*mp_exists) && - (rte_errno == ENOMEM && (dmp->mp_size /= 2) >= MIN_NB_MBUF)); + (rte_errno == ENOMEM && (dmp->n_mbufs /= 2) >= MIN_NB_MBUF)); rte_free(dmp); return NULL; From patchwork Wed Oct 18 16:01:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Fischetti, Antonio" X-Patchwork-Id: 827690 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yHH3W0cKWz9t4c for ; Thu, 19 Oct 2017 03:05:59 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 2D5FDC16; Wed, 18 Oct 2017 16:02:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BB2E1BFD for ; Wed, 18 Oct 2017 16:02:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 67F1651D for ; Wed, 18 Oct 2017 16:02:05 +0000 (UTC) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Oct 2017 09:01:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.43,397,1503385200"; d="scan'208"; a="1207223549" Received: from sivswdev02.ir.intel.com (HELO localhost.localdomain) ([10.237.217.46]) by fmsmga001.fm.intel.com with ESMTP; 18 Oct 2017 09:01:44 -0700 From: antonio.fischetti@intel.com To: dev@openvswitch.org Date: Wed, 18 Oct 2017 17:01:31 +0100 Message-Id: <1508342491-21949-7-git-send-email-antonio.fischetti@intel.com> X-Mailer: git-send-email 1.7.0.7 In-Reply-To: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> References: <1508342491-21949-1-git-send-email-antonio.fischetti@intel.com> X-Spam-Status: No, score=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH v7 6/6] netdev-dpdk: Rename dpdk_mp_put as dpdk_mp_free. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org For readability purposes dpdk_mp_put is renamed as dpdk_mp_free. Some other comments are also added to mempool functions. CC: Mark B Kavanagh CC: Darrell Ball CC: Ciara Loftus CC: Kevin Traynor CC: Aaron Conole Signed-off-by: Antonio Fischetti --- lib/netdev-dpdk.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 0057e6b..411cb06 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -587,6 +587,10 @@ dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool *mp_exists) return NULL; } +/* Returns a valid pointer when either of the following is true: + * - a new mempool was just created + * - a matching mempool already exists + */ static struct dpdk_mp * dpdk_mp_get(struct netdev_dpdk *dev, int mtu, bool *mp_exists) { @@ -599,8 +603,9 @@ dpdk_mp_get(struct netdev_dpdk *dev, int mtu, bool *mp_exists) return dmp; } +/* Release an existing mempool. */ static void -dpdk_mp_put(struct dpdk_mp *dmp) +dpdk_mp_free(struct dpdk_mp *dmp) { char *mp_name; @@ -618,8 +623,8 @@ dpdk_mp_put(struct dpdk_mp *dmp) } /* Tries to allocate a new mempool - or re-use an existing one where - * appropriate - on requested_socket_id with mbuf size corresponding to - * requested_mtu. + * appropriate - on requested_socket_id with a size determined by + * requested_mtu and requested Rx/Tx queues. * On success - or when re-using an existing mempool - the new configuration * will be applied. * On error, device will be left unchanged. */ @@ -646,7 +651,8 @@ netdev_dpdk_mempool_configure(struct netdev_dpdk *dev) dev->max_packet_len = MTU_TO_FRAME_LEN(dev->mtu); return EEXIST; } else { - dpdk_mp_put(dev->dpdk_mp); + /* A new mempool was created, release the previous one. */ + dpdk_mp_free(dev->dpdk_mp); dev->dpdk_mp = mp; dev->mtu = dev->requested_mtu; dev->socket_id = dev->requested_socket_id; @@ -1091,7 +1097,7 @@ common_destruct(struct netdev_dpdk *dev) OVS_EXCLUDED(dev->mutex) { rte_free(dev->tx_q); - dpdk_mp_put(dev->dpdk_mp); + dpdk_mp_free(dev->dpdk_mp); ovs_list_remove(&dev->list_node); free(ovsrcu_get_protected(struct ingress_policer *,