From patchwork Wed Sep 11 08:08:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Obrembski X-Patchwork-Id: 1160743 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46SvkN4Jn9z9s00 for ; Wed, 11 Sep 2019 18:11:40 +1000 (AEST) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id B6911F27; Wed, 11 Sep 2019 08:09:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E84B4F01 for ; Wed, 11 Sep 2019 08:09:22 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1443B7D2 for ; Wed, 11 Sep 2019 08:09:22 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Sep 2019 01:09:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,492,1559545200"; d="scan'208";a="214598463" Received: from mobrembx-mobl.ger.corp.intel.com ([10.103.104.26]) by fmsmga002.fm.intel.com with ESMTP; 11 Sep 2019 01:09:19 -0700 From: Obrembski To: dev@openvswitch.org Date: Wed, 11 Sep 2019 10:08:16 +0200 Message-Id: <20190911080828.2087-4-michalx.obrembski@intel.com> X-Mailer: git-send-email 2.23.0.windows.1 In-Reply-To: <20190911080828.2087-1-michalx.obrembski@intel.com> References: <20190911080828.2087-1-michalx.obrembski@intel.com> MIME-Version: 1.0 X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Flavio Leitner Subject: [ovs-dev] [PATCH v15 03/15] dp-packet: Handle multi-seg mbufs in helper funcs. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tiago Lam Most helper functions in dp-packet assume that the data held by a dp_packet is contiguous, and perform operations such as pointer arithmetic under that assumption. However, with the introduction of multi-segment mbufs, where data is non-contiguous, such assumptions are no longer possible. Some examples of Such helper functions are dp_packet_tail(), dp_packet_tailroom(), dp_packet_end(), dp_packet_get_allocated() and dp_packet_at(). Thus, instead of assuming contiguous data in dp_packet, they now iterate over the (non-contiguous) data in mbufs to perform their calculations. Finally, dp_packet_use__() has also been modified to perform the initialisation of the packet (and setting the source) before continuing to set its size and data length, which now depends on the type of packet. Co-authored-by: Mark Kavanagh Signed-off-by: Mark Kavanagh Signed-off-by: Tiago Lam Acked-by: Eelco Chaudron Acked-by: Flavio Leitner Signed-off-by: Michal Obrembski --- lib/dp-packet.c | 4 +- lib/dp-packet.h | 171 ++++++++++++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 151 insertions(+), 24 deletions(-) diff --git a/lib/dp-packet.c b/lib/dp-packet.c index 62d7faa..582e24d 100644 --- a/lib/dp-packet.c +++ b/lib/dp-packet.c @@ -43,11 +43,11 @@ static void dp_packet_use__(struct dp_packet *b, void *base, size_t allocated, enum dp_packet_source source) { + dp_packet_init__(b, allocated, source); + dp_packet_set_base(b, base); dp_packet_set_data(b, base); dp_packet_set_size(b, 0); - - dp_packet_init__(b, allocated, source); } /* Initializes 'b' as an empty dp_packet that contains the 'allocated' bytes of diff --git a/lib/dp-packet.h b/lib/dp-packet.h index 08c4a84..321eeb6 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -153,6 +153,10 @@ static inline void *dp_packet_at(const struct dp_packet *, size_t offset, size_t size); static inline void *dp_packet_at_assert(const struct dp_packet *, size_t offset, size_t size); +#ifdef DPDK_NETDEV +static inline const struct rte_mbuf * +dp_packet_mbuf_from_offset(const struct dp_packet *b, size_t *offset); +#endif static inline void *dp_packet_tail(const struct dp_packet *); static inline void *dp_packet_end(const struct dp_packet *); @@ -206,13 +210,28 @@ dp_packet_delete(struct dp_packet *b) } /* If 'b' contains at least 'offset + size' bytes of data, returns a pointer to - * byte 'offset'. Otherwise, returns a null pointer. */ + * byte 'offset'. Otherwise, returns a null pointer. For DPDK packets, this + * means the 'offset' + 'size' must fall within the same mbuf (not necessarily + * the first mbuf), otherwise null is returned */ static inline void * dp_packet_at(const struct dp_packet *b, size_t offset, size_t size) { - return offset + size <= dp_packet_size(b) - ? (char *) dp_packet_data(b) + offset - : NULL; + if (offset + size > dp_packet_size(b)) { + return NULL; + } + +#ifdef DPDK_NETDEV + if (b->source == DPBUF_DPDK) { + const struct rte_mbuf *mbuf = dp_packet_mbuf_from_offset(b, &offset); + + if (!mbuf || offset + size > mbuf->data_len) { + return NULL; + } + + return rte_pktmbuf_mtod_offset(mbuf, char *, offset); + } +#endif + return (char *) dp_packet_data(b) + offset; } /* Returns a pointer to byte 'offset' in 'b', which must contain at least @@ -221,13 +240,23 @@ static inline void * dp_packet_at_assert(const struct dp_packet *b, size_t offset, size_t size) { ovs_assert(offset + size <= dp_packet_size(b)); - return ((char *) dp_packet_data(b)) + offset; + return dp_packet_at(b, offset, size); } /* Returns a pointer to byte following the last byte of data in use in 'b'. */ static inline void * dp_packet_tail(const struct dp_packet *b) { +#ifdef DPDK_NETDEV + if (b->source == DPBUF_DPDK) { + struct rte_mbuf *buf = CONST_CAST(struct rte_mbuf *, &b->mbuf); + /* Find last segment where data ends, meaning the tail of the chained + * mbufs must be there */ + buf = rte_pktmbuf_lastseg(buf); + + return rte_pktmbuf_mtod_offset(buf, void *, buf->data_len); + } +#endif return (char *) dp_packet_data(b) + dp_packet_size(b); } @@ -236,6 +265,15 @@ dp_packet_tail(const struct dp_packet *b) static inline void * dp_packet_end(const struct dp_packet *b) { +#ifdef DPDK_NETDEV + if (b->source == DPBUF_DPDK) { + struct rte_mbuf *buf = CONST_CAST(struct rte_mbuf *, &(b->mbuf)); + + buf = rte_pktmbuf_lastseg(buf); + + return (char *) buf->buf_addr + buf->buf_len; + } +#endif return (char *) dp_packet_base(b) + dp_packet_get_allocated(b); } @@ -261,6 +299,15 @@ dp_packet_tailroom(const struct dp_packet *b) static inline void dp_packet_clear(struct dp_packet *b) { +#ifdef DPDK_NETDEV + if (b->source == DPBUF_DPDK) { + /* sets pkt_len and data_len to zero and frees unused mbufs */ + dp_packet_set_size(b, 0); + rte_pktmbuf_reset(&b->mbuf); + + return; + } +#endif dp_packet_set_data(b, dp_packet_base(b)); dp_packet_set_size(b, 0); } @@ -273,28 +320,51 @@ dp_packet_pull(struct dp_packet *b, size_t size) void *data = dp_packet_data(b); ovs_assert(dp_packet_size(b) - dp_packet_l2_pad_size(b) >= size); dp_packet_set_data(b, (char *) dp_packet_data(b) + size); - dp_packet_set_size(b, dp_packet_size(b) - size); +#ifdef DPDK_NETDEV + b->mbuf.pkt_len -= size; +#else + b->size_ -= size; +#endif + return data; } +/* Similar to dp_packet_try_pull() but doesn't actually pull any data, only + * checks if it could and returns 'true' or 'false', accordingly. For DPDK + * packets, 'true' is only returned in case the 'offset' + 'size' falls within + * the first mbuf, otherwise 'false' is returned */ +static inline bool +dp_packet_may_pull(const struct dp_packet *b, uint16_t offset, size_t size) +{ + if (offset == UINT16_MAX) { + return false; + } +#ifdef DPDK_NETDEV + /* Offset needs to be within the first mbuf */ + if (offset + size > b->mbuf.data_len) { + return false; + } +#endif + return (offset + size > dp_packet_size(b)) ? false : true; +} + /* If 'b' has at least 'size' bytes of data, removes that many bytes from the * head end of 'b' and returns the first byte removed. Otherwise, returns a * null pointer without modifying 'b'. */ static inline void * dp_packet_try_pull(struct dp_packet *b, size_t size) { +#ifdef DPDK_NETDEV + if (!dp_packet_may_pull(b, 0, size)) { + return NULL; + } +#endif + return dp_packet_size(b) - dp_packet_l2_pad_size(b) >= size ? dp_packet_pull(b, size) : NULL; } static inline bool -dp_packet_equal(const struct dp_packet *a, const struct dp_packet *b) -{ - return dp_packet_size(a) == dp_packet_size(b) && - !memcmp(dp_packet_data(a), dp_packet_data(b), dp_packet_size(a)); -} - -static inline bool dp_packet_is_eth(const struct dp_packet *b) { return b->packet_type == htonl(PT_ETH); @@ -393,10 +463,12 @@ dp_packet_l3_size(const struct dp_packet *b) static inline size_t dp_packet_l4_size(const struct dp_packet *b) { - return OVS_LIKELY(b->l4_ofs != UINT16_MAX) - ? (const char *)dp_packet_tail(b) - (const char *)dp_packet_l4(b) - - dp_packet_l2_pad_size(b) - : 0; + if (!dp_packet_may_pull(b, b->l4_ofs, 0)) { + return 0; + } + + size_t l4_size = dp_packet_size(b) - b->l4_ofs; + return l4_size - dp_packet_l2_pad_size(b); } static inline const void * @@ -409,7 +481,8 @@ dp_packet_get_tcp_payload(const struct dp_packet *b) int tcp_len = TCP_OFFSET(tcp->tcp_ctl) * 4; if (OVS_LIKELY(tcp_len >= TCP_HEADER_LEN && tcp_len <= l4_size)) { - return (const char *)tcp + tcp_len; + tcp = dp_packet_at(b, b->l4_ofs, tcp_len); + return (tcp == NULL) ? NULL : tcp + tcp_len; } } return NULL; @@ -456,6 +529,54 @@ dp_packet_init_specific(struct dp_packet *p) p->mbuf.next = NULL; } +static inline const struct rte_mbuf * +dp_packet_mbuf_from_offset(const struct dp_packet *b, size_t *offset) { + const struct rte_mbuf *mbuf = &b->mbuf; + while (mbuf && *offset >= mbuf->data_len) { + *offset -= mbuf->data_len; + + mbuf = mbuf->next; + } + + return mbuf; +} + +static inline bool +dp_packet_equal(const struct dp_packet *a, const struct dp_packet *b) +{ + if (dp_packet_size(a) != dp_packet_size(b)) { + return false; + } + + const struct rte_mbuf *m_a = NULL; + const struct rte_mbuf *m_b = NULL; + size_t abs_off_a = 0; + size_t abs_off_b = 0; + size_t len = 0; + while (m_a != NULL && m_b != NULL) { + size_t rel_off_a = abs_off_a; + size_t rel_off_b = abs_off_b; + m_a = dp_packet_mbuf_from_offset(a, &rel_off_a); + m_b = dp_packet_mbuf_from_offset(b, &rel_off_b); + if (!m_a || !m_b) { + break; + } + + len = MIN(m_a->data_len - rel_off_a, m_b->data_len - rel_off_b); + + if (memcmp(rte_pktmbuf_mtod_offset(m_a, char *, rel_off_a), + rte_pktmbuf_mtod_offset(m_b, char *, rel_off_b), + len)) { + return false; + } + + abs_off_a += len; + abs_off_b += len; + } + + return (!m_a && !m_b) ? true : false; +} + static inline void * dp_packet_base(const struct dp_packet *b) { @@ -582,7 +703,7 @@ __packet_set_data(struct dp_packet *b, uint16_t v) static inline uint16_t dp_packet_get_allocated(const struct dp_packet *b) { - return b->mbuf.buf_len; + return b->mbuf.nb_segs * b->mbuf.buf_len; } static inline void @@ -666,6 +787,13 @@ dp_packet_set_flow_mark(struct dp_packet *p, uint32_t mark) #else /* DPDK_NETDEV */ +static inline bool +dp_packet_equal(const struct dp_packet *a, const struct dp_packet *b) +{ + return dp_packet_size(a) == dp_packet_size(b) && + !memcmp(dp_packet_data(a), dp_packet_data(b), dp_packet_size(a)); +} + static inline void dp_packet_init_specific(struct dp_packet *p OVS_UNUSED) { @@ -841,10 +969,9 @@ dp_packet_set_data(struct dp_packet *b, void *data) } static inline void -dp_packet_reset_packet(struct dp_packet *b, int off) +dp_packet_reset_packet(struct dp_packet *b, size_t off) { - dp_packet_set_size(b, dp_packet_size(b) - off); - dp_packet_set_data(b, ((unsigned char *) dp_packet_data(b) + off)); + dp_packet_try_pull(b, off); dp_packet_reset_offsets(b); }