From patchwork Mon Sep 23 14:16:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Flavio Leitner X-Patchwork-Id: 1166083 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=sysclose.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sysclose.org header.i=@sysclose.org header.b="AR+x6vHM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=sysclose.org header.i=@sysclose.org header.b="AtVyaorP"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46cRHB593Nz9sP6 for ; Tue, 24 Sep 2019 00:17:42 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 13DF3CC4; Mon, 23 Sep 2019 14:16:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2BEAACB2 for ; Mon, 23 Sep 2019 14:16:20 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sysclose.org (smtp.sysclose.org [69.164.214.230]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6D3E289A for ; Mon, 23 Sep 2019 14:16:19 +0000 (UTC) Received: by sysclose.org (Postfix, from userid 5001) id 19A9B6525; Mon, 23 Sep 2019 14:16:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 sysclose.org 19A9B6525 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sysclose.org; s=201903; t=1569248196; bh=dsMysXJuVb72JU8mQ1yGuBQGVwjLDduxB4wg0h0Pae4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AR+x6vHM97/yrqcx6xRPC0jkqAhZ7bUBXFX5LkHzSJXfnkC8RiyQ1XwHSdqOrcOLC aVKy3PB5HqvzVrmk3CEAIxEfjFlLjamPCgN8s8TqvgHqWOn3ENFgDX+fMsptnvz4na zjjyp1ciNjbEevdbzYteLG+oR/88Qrspu/bero316qyecLYM8jF9A4GG6h8oGjjhZr L0Qn1gFtaWeAz/XRL9zgi04GCnJFflwdxzYo59+hBaNEtntnvj0okCCsyGcH3GO6Ow I4h0MkQde4ThWKlMiW43WdriR6F/K+Hha3OtqWfH1pGdTZ6+bfuuuPqdckv7r+W4fU XoFeRV+UaKMkw== X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU autolearn=ham version=3.3.1 Received: from localhost (unknown [177.183.215.210]) by sysclose.org (Postfix) with ESMTPSA id D7F46650B; Mon, 23 Sep 2019 14:16:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 sysclose.org D7F46650B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sysclose.org; s=201903; t=1569248195; bh=dsMysXJuVb72JU8mQ1yGuBQGVwjLDduxB4wg0h0Pae4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AtVyaorPpmQSXZiQXn+rah0k/wYOzoeUopFEYoYgwvbI6hZHMQWCiG1c2dbZ+Tv8O aEv30RXYOiHz0VUzgL38lZLANl6vi20EqsZBim99TE292bOuZLCkXxrfQqGtwf/9qr 8tkse1U/3rG8jjOLfMhkMIQHn+D2yC87AMAjGgLVMGiXt3SGvSbGj7iHXfVGHWyC3T EPTWGfVt7GwBOrUnD5eKFVQwA9eckxfeQiTQjkGiPCilvkMf6c7+fOCp8zMWKoYjHK GkeTlXjkLL2W8ReEy7ZugeupxjmDI+AIszzp1jSmAflfs4/eC4ln/RDQrPy27yDMv/ 1SumKLVpOVXMQ== From: Flavio Leitner To: dev@openvswitch.org Date: Mon, 23 Sep 2019 11:16:03 -0300 Message-Id: <20190923141603.10414-1-fbl@sysclose.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190923140606.GA3856@p50.lan> References: <20190923140606.GA3856@p50.lan> MIME-Version: 1.0 Cc: Flavio Leitner Subject: [ovs-dev] [PATCH] vhost: support for optional large mpool. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Signed-off-by: Flavio Leitner --- drivers/net/vhost/rte_eth_vhost.c | 4 +-- examples/tep_termination/main.c | 2 +- examples/vhost/main.c | 2 +- lib/librte_vhost/rte_vhost.h | 5 +++- lib/librte_vhost/virtio_net.c | 47 +++++++++++++++++++++---------- 5 files changed, 40 insertions(+), 20 deletions(-) diff --git a/drivers/net/vhost/rte_eth_vhost.c b/drivers/net/vhost/rte_eth_vhost.c index b2cda0483..0de103363 100644 --- a/drivers/net/vhost/rte_eth_vhost.c +++ b/drivers/net/vhost/rte_eth_vhost.c @@ -389,8 +389,8 @@ eth_vhost_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) VHOST_MAX_PKT_BURST); nb_pkts = rte_vhost_dequeue_burst(r->vid, r->virtqueue_id, - r->mb_pool, &bufs[nb_rx], - num); + r->mb_pool, NULL, + &bufs[nb_rx], num); nb_rx += nb_pkts; nb_receive -= nb_pkts; diff --git a/examples/tep_termination/main.c b/examples/tep_termination/main.c index d6379e31e..4674e1fde 100644 --- a/examples/tep_termination/main.c +++ b/examples/tep_termination/main.c @@ -702,7 +702,7 @@ switch_worker(__rte_unused void *arg) if (likely(!vdev->remove)) { /* Handle guest TX*/ tx_count = rte_vhost_dequeue_burst(vdev->vid, - VIRTIO_TXQ, mbuf_pool, + VIRTIO_TXQ, mbuf_pool, NULL, pkts_burst, MAX_PKT_BURST); /* If this is the first received packet we need to learn the MAC */ if (unlikely(vdev->ready == DEVICE_MAC_LEARNING) && tx_count) { diff --git a/examples/vhost/main.c b/examples/vhost/main.c index 2261f089f..6807efa7c 100644 --- a/examples/vhost/main.c +++ b/examples/vhost/main.c @@ -1091,7 +1091,7 @@ drain_virtio_tx(struct vhost_dev *vdev) pkts, MAX_PKT_BURST); } else { count = rte_vhost_dequeue_burst(vdev->vid, VIRTIO_TXQ, - mbuf_pool, pkts, MAX_PKT_BURST); + mbuf_pool, NULL, pkts, MAX_PKT_BURST); } /* setup VMDq for the first packet */ diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h index 5905e2401..780413097 100644 --- a/lib/librte_vhost/rte_vhost.h +++ b/lib/librte_vhost/rte_vhost.h @@ -523,6 +523,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t queue_id, * virtio queue index in mq case * @param mbuf_pool * mbuf_pool where host mbuf is allocated. + * @param mbuf_pool64k + * mbuf_pool where host large mbuf is allocated. * @param pkts * array to contain packets to be dequeued * @param count @@ -531,7 +533,8 @@ uint16_t rte_vhost_enqueue_burst(int vid, uint16_t queue_id, * num of packets dequeued */ uint16_t rte_vhost_dequeue_burst(int vid, uint16_t queue_id, - struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count); + struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool64k, + struct rte_mbuf **pkts, uint16_t count); /** * Get guest mem table: a list of memory regions. diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c index a6576891a..72b6a8539 100644 --- a/lib/librte_vhost/virtio_net.c +++ b/lib/librte_vhost/virtio_net.c @@ -1327,7 +1327,8 @@ get_zmbuf(struct vhost_virtqueue *vq) static __rte_always_inline uint16_t virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, - struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count) + struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool64k, + struct rte_mbuf **pkts, uint16_t count) { uint16_t i; uint16_t free_entries; @@ -1379,14 +1380,15 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, for (i = 0; i < count; i++) { struct buf_vector buf_vec[BUF_VECTOR_MAX]; uint16_t head_idx; - uint32_t dummy_len; + uint32_t buf_len; uint16_t nr_vec = 0; + struct rte_mempool *mpool; int err; if (unlikely(fill_vec_buf_split(dev, vq, vq->last_avail_idx + i, &nr_vec, buf_vec, - &head_idx, &dummy_len, + &head_idx, &buf_len, VHOST_ACCESS_RO) < 0)) break; @@ -1395,15 +1397,20 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, rte_prefetch0((void *)(uintptr_t)buf_vec[0].buf_addr); - pkts[i] = rte_pktmbuf_alloc(mbuf_pool); + // FIXME: if (mbuf_pool64k && buf_len > mbuf_pool->elt_size) + if (mbuf_pool64k && buf_len > 1500) + mpool = mbuf_pool64k; + else + mpool = mbuf_pool; + + pkts[i] = rte_pktmbuf_alloc(mpool); if (unlikely(pkts[i] == NULL)) { RTE_LOG(ERR, VHOST_DATA, "Failed to allocate memory for mbuf.\n"); break; } - err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i], - mbuf_pool); + err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i], mpool); if (unlikely(err)) { rte_pktmbuf_free(pkts[i]); break; @@ -1449,7 +1456,8 @@ virtio_dev_tx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, static __rte_always_inline uint16_t virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, - struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count) + struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool64k, + struct rte_mbuf **pkts, uint16_t count) { uint16_t i; @@ -1491,14 +1499,15 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, for (i = 0; i < count; i++) { struct buf_vector buf_vec[BUF_VECTOR_MAX]; uint16_t buf_id; - uint32_t dummy_len; + uint32_t buf_len; uint16_t desc_count, nr_vec = 0; + struct rte_mempool *mpool; int err; if (unlikely(fill_vec_buf_packed(dev, vq, vq->last_avail_idx, &desc_count, buf_vec, &nr_vec, - &buf_id, &dummy_len, + &buf_id, &buf_len, VHOST_ACCESS_RO) < 0)) break; @@ -1508,15 +1517,20 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, rte_prefetch0((void *)(uintptr_t)buf_vec[0].buf_addr); - pkts[i] = rte_pktmbuf_alloc(mbuf_pool); + // FIXME: if (mbuf_pool64k && buf_len > mbuf_pool->elt_size) + if (mbuf_pool64k && buf_len > 1500) + mpool = mbuf_pool64k; + else + mpool = mbuf_pool; + + pkts[i] = rte_pktmbuf_alloc(mpool); if (unlikely(pkts[i] == NULL)) { RTE_LOG(ERR, VHOST_DATA, "Failed to allocate memory for mbuf.\n"); break; } - err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i], - mbuf_pool); + err = copy_desc_to_mbuf(dev, vq, buf_vec, nr_vec, pkts[i], mpool); if (unlikely(err)) { rte_pktmbuf_free(pkts[i]); break; @@ -1568,7 +1582,8 @@ virtio_dev_tx_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, uint16_t rte_vhost_dequeue_burst(int vid, uint16_t queue_id, - struct rte_mempool *mbuf_pool, struct rte_mbuf **pkts, uint16_t count) + struct rte_mempool *mbuf_pool, struct rte_mempool *mbuf_pool64k, + struct rte_mbuf **pkts, uint16_t count) { struct virtio_net *dev; struct rte_mbuf *rarp_mbuf = NULL; @@ -1640,9 +1655,11 @@ rte_vhost_dequeue_burst(int vid, uint16_t queue_id, } if (vq_is_packed(dev)) - count = virtio_dev_tx_packed(dev, vq, mbuf_pool, pkts, count); + count = virtio_dev_tx_packed(dev, vq, mbuf_pool, mbuf_pool64k, pkts, + count); else - count = virtio_dev_tx_split(dev, vq, mbuf_pool, pkts, count); + count = virtio_dev_tx_split(dev, vq, mbuf_pool, mbuf_pool64k, pkts, + count); out: if (dev->features & (1ULL << VIRTIO_F_IOMMU_PLATFORM))