From patchwork Mon May 17 14:04:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Ferriter, Cian" X-Patchwork-Id: 1479515 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FkLRP0bYPz9sV5 for ; Tue, 18 May 2021 00:02:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2AA9F40452; Mon, 17 May 2021 14:02:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id of6VHtiyUvfa; Mon, 17 May 2021 14:02:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id 9E8D640305; Mon, 17 May 2021 14:02:00 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7C7F6C000D; Mon, 17 May 2021 14:02:00 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id EDB8DC0001 for ; Mon, 17 May 2021 14:01:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CE47F608BA for ; Mon, 17 May 2021 14:01:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vhLAUQbDaUfc for ; Mon, 17 May 2021 14:01:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6F03160803 for ; Mon, 17 May 2021 14:01:54 +0000 (UTC) IronPort-SDR: k9dOFAI97xsjg8fiKCoYjHs0nCdGm38zJ+tUa7XKep/F+mLl7c198S8yb0ht1g5yWzp8mOYXul oVqqttKvLXXA== X-IronPort-AV: E=McAfee;i="6200,9189,9986"; a="180068974" X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="180068974" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 May 2021 07:01:52 -0700 IronPort-SDR: pkRDUU2QOW3pW8MWpYnKO/I1MCPhGQWo/y0bmLqVvhKwQXR07T9dC3AaQ5nYxPTo9PH4SrY56N maEyfICNPCnA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,307,1613462400"; d="scan'208";a="460318221" Received: from silpixa00399779.ir.intel.com (HELO silpixa00399779.ger.corp.intel.com) ([10.237.223.175]) by fmsmga004.fm.intel.com with ESMTP; 17 May 2021 07:01:50 -0700 From: Cian Ferriter To: ovs-dev@openvswitch.org Date: Mon, 17 May 2021 15:04:18 +0100 Message-Id: <20210517140434.59555-1-cian.ferriter@intel.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v12 00/16] DPIF Framework + Optimizations X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" v12 Summary: - Add a partial HWOL PMD statistic. This is added for both the scalar and AVX512 DPIFs. v11 Summary: - Improve the dp_netdev_impl_get_default() function so PMD threads created after running "dpif-set" command will use the DPIF implementation that was set. - Fix small comment formatting issues. v10 Summary: - Removed AVX512 POC work for DPIF and MFEX which was added in v9 -- MFEX patches will be sent separately - Rebase additions to NEWS entries - Update copyright notices v9 Summary: - Added AVX512 POC work for DPIF and MFEX in single patch at end -- Note that the AVX512 MFEX is for Ether()/IP()/UDP() traffic. -- A significant performance boost is possible with these optimizations. v8 Summary: - Added NEWS entries for significant changes - Added scalar optimizations for datapath TX - Patchset is now ready for merge in my opinion. v7 summary: - OVS Conference included DPIF overview, youtube link: --- https://youtu.be/5dWyPxiXEhg - Rebased and tested on the DPDK 20.11 v4 patch --- Link: https://patchwork.ozlabs.org/project/openvswitch/list/?series=220645 --- Tested this series for shared/static builds --- Tested this series with/without -march= - Minor code improvements in DPIF component (see commits for details) - Improved CPU ISA checks, caching results - Commit message improvements (.'s etc) - Added performance data of patchset --- Note that the benchmark below does not utilize the AVX512-vpopcntdq --- optimizations, and performance is expected to improve when used. --- Further optimizations are planned that continue. Benchmark Details & Results =========================== Intel® Xeon® Gold 6230 CPU @2.10GHz OVS*-DPDK* Phy-Phy Performance 4x 25G Ports - Total 1 million flows 1C1T-4P, 64-byte frame size, performance in mpps: Results Table: ------------------------------------------- DPIF | Scalar | Scalar | AVX512 | AVX512 | DPCLS | Scalar | AVX512 | Scalar | AVX512 | ------------------------------------------- mpps | 6.955 | 7.530 | 7.530 | 7.962 | By enabling both AVX512 DPIF and DPCLS, packet forwarding is 7.962 / 6.955 = 1.1447x faster, aka 14% speedup. v6 summary: - Rebase to DPDK 20.11 enabling patch --- This creates a dependency, expect CI build failures on the last patch in this series if it is not applied! - Small improvements to DPIF layer --- EMC/SMC enabling in AVX512 DPIF cleanups - CPU ISA flags are cached, lowering overhead - Wilcard Classifier DPCLS --- Refactor and cleanups for function names --- Enable more subtable specializations --- Enable AVX512 vpopcount instruction v5 summary: - Dropped MFEX optimizations, re-targetting to a later release --- This allows focus of community reviews & development on DPIF --- Note OVS Conference talk still introduces both DPIF and MFEX topics - DPIF improvements --- Better EMC/SMC handling --- HWOL is enabled in the avx512 DPIF --- Documentation & NEWS items added --- Various smaller improvements v4 summary: - Updated and improve DPIF component --- SMC now implemented --- EMC handling improved --- Novel batching method using AVX512 implemented --- see commits for details - Updated Miniflow Extract component --- Improved AVX512 code path performance --- Implemented multiple TODO item's in v3 --- Add "disable" implementation to return to scalar miniflow only --- More fixes planned for v5/future revisions: ---- Rename command to better reflect usage ---- Improve dynamicness of patterns ---- Add more demo protocols to show usage - Future work --- Documentation/NEWS items --- Statistics for optimized MFEX - Note that this patchset will be discussed/presented at OvsConf soon :) v3 update summary: (Cian Ferriter helping with rebases, review and code cleanups) - Split out partially related changes (these will be sent separately) --- netdev output action optimization --- avx512 dpcls 16-block support optimization - Squash commit which moves netdev struct flow into the refactor commit: --- Squash dpif-netdev: move netdev flow struct to header --- Into dpif-netdev: Refactor to multiple header files - Implement Miniflow extract for AVX-512 DPIF --- A generic method of matching patterns and packets is implemented, providing traffic-pattern specific miniflow-extract acceleration. --- The patterns today are hard-coded, however in a future patchset it is intended to make these runtime configurable, allowing users to optimize the SIMD miniflow extract for active traffic types. - Notes: --- 32 bit builds will be fixed in next release by adding flexible miniflow extract optimization selection. --- AVX-512 VBMI ISA is not yet supported in OVS due to requiring the DPDK 20.11 update for RTE_CPUFLAG_*. Once on a newer DPDK this will be added. v2 updates: - Includes DPIF command switching at runtime - Includes AVX512 DPIF implementation - Includes some partially related changes (can be split out of set?) --- netdev output action optimization --- avx512 dpcls 16-block support optimization This patchset is a v7 for making the DPIF components of the userspace datapath more flexible. It has been refactored to be more modular to encourage code-reuse, and scalable in that ISA optimized implementations can be added and selected at runtime. The same approach as has been previously used for DPCLS is used here, where a function pointer allows selection of an implementation at runtime. Datapath features such as EMC, SMC and HWOL are shared between implementations, hence they are refactored into seperate header files. The file splitting also improves maintainability, as dpif_netdev.c has ~9000 LOC, and very hard to modify due to many structs defined locally in the .c file, ruling out re-usability in other .c files. Questions welcomed! Regards, -Harry Cian Ferriter (1): docs/dpdk/bridge: Add dpif performance section. Harry van Haaren (14): dpif-netdev: Refactor to multiple header files. dpif-netdev: Split HWOL out to own header file. dpif-netdev: Add function pointer for netdev input. dpif-avx512: Add ISA implementation of dpif. dpif-avx512: Add HWOL support to avx512 dpif. dpif-netdev: Add command to switch dpif implementation. dpif-netdev: Add command to get dpif implementations. dpif-netdev/dpcls: Refactor function names to dpcls. dpif-netdev/dpcls-avx512: enable 16 block processing. dpif-netdev/dpcls: specialize more subtable signatures. dpdk: Cache result of CPU ISA checks. dpcls-avx512: enabling avx512 vector popcount instruction. dpif-netdev: Optimize dp output action netdev: Optimize netdev_send_prepare_batch Documentation/topics/dpdk/bridge.rst | 37 ++ NEWS | 14 + acinclude.m4 | 15 + configure.ac | 1 + lib/automake.mk | 11 +- lib/dpdk.c | 29 +- lib/dpif-netdev-avx512.c | 300 ++++++++++ lib/dpif-netdev-lookup-autovalidator.c | 1 - lib/dpif-netdev-lookup-avx512-gather.c | 278 ++++++--- lib/dpif-netdev-lookup-generic.c | 7 +- lib/dpif-netdev-lookup.h | 2 +- lib/dpif-netdev-private-dfc.h | 252 ++++++++ lib/dpif-netdev-private-dpcls.h | 127 ++++ lib/dpif-netdev-private-dpif.c | 99 ++++ lib/dpif-netdev-private-dpif.h | 79 +++ lib/dpif-netdev-private-flow.h | 162 +++++ lib/dpif-netdev-private-hwol.h | 63 ++ lib/dpif-netdev-private-thread.h | 215 +++++++ lib/dpif-netdev-private.h | 123 ++-- lib/dpif-netdev.c | 779 +++++++------------------ lib/netdev.c | 31 +- 21 files changed, 1871 insertions(+), 754 deletions(-) create mode 100644 lib/dpif-netdev-avx512.c create mode 100644 lib/dpif-netdev-private-dfc.h create mode 100644 lib/dpif-netdev-private-dpcls.h create mode 100644 lib/dpif-netdev-private-dpif.c create mode 100644 lib/dpif-netdev-private-dpif.h create mode 100644 lib/dpif-netdev-private-flow.h create mode 100644 lib/dpif-netdev-private-hwol.h create mode 100644 lib/dpif-netdev-private-thread.h