From patchwork Mon Sep 23 14:15:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Flavio Leitner X-Patchwork-Id: 1166079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=sysclose.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sysclose.org header.i=@sysclose.org header.b="GqrgBEB/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sysclose.org header.i=@sysclose.org header.b="QkOGE0c2"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46cRDj0gt2z9sP6 for ; Tue, 24 Sep 2019 00:15:32 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id D718AC83; Mon, 23 Sep 2019 14:15:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 33E01BA4 for ; Mon, 23 Sep 2019 14:15:29 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sysclose.org (smtp.sysclose.org [69.164.214.230]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id AF1EE89D for ; Mon, 23 Sep 2019 14:15:28 +0000 (UTC) Received: by sysclose.org (Postfix, from userid 5001) id 433596526; Mon, 23 Sep 2019 14:15:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 sysclose.org 433596526 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sysclose.org; s=201903; t=1569248145; bh=WtgfIJJJ45hia7MkmWzYwmqxH5hV9vNyerEO1R7n8so=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GqrgBEB/lDLlZGGrPWqyAR0KmIoiJRlNz/qxKmdE34ipOTL3cd/O9+OlA1JHMLf+W 5pdOQat6CDUfjfY4h0X+PeFIXfIjKmhtTr2BzF7Qx6PfEbTypnd19Np2Q5Y36GloNT WpnXxrWS24YBNv+c0IEp9bhROSlZXS54HYagtvsjST1KC9uvYk9BgvrmmtqwTCCNnn mDkihFyJ2D39w+Q7vpVDXK+m2RiV0M9J8T7FmTHvB3qM5RwPupTBiRGDGK8XhOZC87 CE0ys9TZOsuOV/MQsKqf1m2QNYVqp44XpF1sm7UC7bzhAQXaa4kDSxCqAWAj5F1y/T 5R68t2euxBa6A== X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU autolearn=ham version=3.3.1 Received: from localhost (unknown [177.183.215.210]) by sysclose.org (Postfix) with ESMTPSA id 3DE0A6510; Mon, 23 Sep 2019 14:15:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 sysclose.org 3DE0A6510 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sysclose.org; s=201903; t=1569248144; bh=WtgfIJJJ45hia7MkmWzYwmqxH5hV9vNyerEO1R7n8so=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QkOGE0c2wlc72UGwhKEk01o95NSvwpxqg6AoV0rSzpNXTJIie+jfZ3r9sUYINfBXB Zdn3BBV5GbcDAv92rb1aCPJPvvFJsxVVw7RhoR0jVAPfaJWkelBSgjhStL/f3TAmXl 9Q3cbT1UCeKpPXCQopKglDEVj5HrAPo4YqisB917FJ/ItmOInHOYdSLGowIAQHcSHs ATVWyT5xFHYd/UbJOwvTU4ukJ5IY5801hJNGrn4HVbAak2zYqOoMjhgqPq90YIFTVs 8+fS420VesCq1mng0Z2cJP9Xm9e1F8gozub9geBRzsnwUenJ9W6lhKLKp5rU6qvrZY 9RfeINJePMpnQ== From: Flavio Leitner To: dev@openvswitch.org Date: Mon, 23 Sep 2019 11:15:08 -0300 Message-Id: <20190923141510.10303-1-fbl@sysclose.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190923140606.GA3856@p50.lan> References: <20190923140606.GA3856@p50.lan> MIME-Version: 1.0 Cc: Flavio Leitner Subject: [ovs-dev] [PATCH 1/3] support for large mpool. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Signed-off-by: Flavio Leitner --- lib/netdev-dpdk.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 48057835f..7eb294366 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -323,8 +323,11 @@ static struct ovs_mutex dpdk_mp_mutex OVS_ACQ_AFTER(dpdk_mutex) static struct ovs_list dpdk_mp_list OVS_GUARDED_BY(dpdk_mp_mutex) = OVS_LIST_INITIALIZER(&dpdk_mp_list); +static struct rte_mempool *shared_mp64k = NULL; + struct dpdk_mp { struct rte_mempool *mp; + struct rte_mempool *mp64k; int mtu; int socket_id; int refcount; @@ -637,6 +640,40 @@ dpdk_calculate_mbufs(struct netdev_dpdk *dev, int mtu, bool per_port_mp) return n_mbufs; } +static struct dpdk_mp * +dpdk_mp64k_create(uint32_t socket_id) +{ + struct rte_mempool *mp; + char name[] = "64k"; + uint32_t n_mbufs; + uint32_t mbuf_priv_data_len; + uint32_t mbuf_size; + uint32_t aligned_mbuf_size = 0; + uint32_t pkt_size = 0; + + + /* 64k */ + mbuf_size = 0xffff; + mbuf_priv_data_len = 584; + n_mbufs = 337; + mp = rte_pktmbuf_pool_create(name, n_mbufs, 0, + mbuf_priv_data_len, + mbuf_size, + socket_id); + if (mp) { + rte_mempool_obj_iter(mp, ovs_rte_pktmbuf_init, NULL); + VLOG_INFO("mempool of 64k mbufs allocated"); + } else { + VLOG_ERR("Failed to create mempool mp_64k"); + VLOG_ERR("mp_64k: size: %d entries: %d (%d bytes)", mbuf_size * n_mbufs, + n_mbufs, mbuf_size); + VLOG_ERR("mp_64k: rte_errno=%d mbuf_priv_data_len=%d", rte_errno, + mbuf_priv_data_len); + } + + return mp; +} + static struct dpdk_mp * dpdk_mp_create(struct netdev_dpdk *dev, int mtu, bool per_port_mp) { @@ -755,6 +792,11 @@ dpdk_mp_get(struct netdev_dpdk *dev, int mtu, bool per_port_mp) bool reuse = false; ovs_mutex_lock(&dpdk_mp_mutex); + + if (!shared_mp64k) { + shared_mp64k = dpdk_mp64k_create(dev->requested_socket_id); + } + /* Check if shared memory is being used, if so check existing mempools * to see if reuse is possible. */ if (!per_port_mp) { @@ -859,6 +901,7 @@ netdev_dpdk_mempool_configure(struct netdev_dpdk *dev) dpdk_mp_put(dev->dpdk_mp); } dev->dpdk_mp = dmp; + dev->dpdk_mp->mp64k = shared_mp64k; dev->mtu = dev->requested_mtu; dev->socket_id = dev->requested_socket_id; dev->max_packet_len = MTU_TO_FRAME_LEN(dev->mtu); @@ -2214,6 +2257,7 @@ netdev_dpdk_vhost_rxq_recv(struct netdev_rxq *rxq, } nb_rx = rte_vhost_dequeue_burst(vid, qid, dev->dpdk_mp->mp, + dev->dpdk_mp->mp64k, (struct rte_mbuf **) batch->packets, NETDEV_MAX_BURST); if (!nb_rx) {