From patchwork Mon Apr 12 15:19:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1465333 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=bBMGFFIJ; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=QsJ4ZOMB; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4FJsr65Pzjz9rx6 for ; Tue, 13 Apr 2021 01:20:38 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 29AA5404DF; Mon, 12 Apr 2021 15:20:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4A4-CxZHwqfx; Mon, 12 Apr 2021 15:20:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTP id B622A404EE; Mon, 12 Apr 2021 15:20:31 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7BC85C0022; Mon, 12 Apr 2021 15:20:28 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 078B0C0011 for ; Mon, 12 Apr 2021 15:20:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 24F6F403FF for ; Mon, 12 Apr 2021 15:20:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=u256.net header.b="bBMGFFIJ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.b="QsJ4ZOMB" Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pttm659_VTpM for ; Mon, 12 Apr 2021 15:20:21 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) by smtp2.osuosl.org (Postfix) with ESMTPS id BAEBB40001 for ; Mon, 12 Apr 2021 15:20:21 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id DC065181E; Mon, 12 Apr 2021 11:20:18 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Mon, 12 Apr 2021 11:20:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:mime-version :content-transfer-encoding; s=fm1; bh=AsT3BA46+bTpaogpBOami5Y2Rl II8eT6h9L9vSzts3E=; b=bBMGFFIJgjIjzkYLGB1P7iIszY1kNI3soucRX4gM0x sqFBVbWCouEX24aCTY+X7IvFPr1i77sIP7Zm88sI85D8tcH11GP1hAldxmR3ZCnh JOf+VvmMxRnj2Gr/5rfR7EDkwKYFssSyJBNG97pV0X5YppZ0mtnCwygyiMfewJtH S3bHiXw3yglc3nBTFlZ6v5He7WxHxtejxwQ0zg1zqVA6dDo3kRakCtraSQ0/uanJ +F+4P0CerzrfW0jCPh1zlQgEBarqBwko+/pjboboFv9zCVTuo/wo++YuOShcxk5K x+bOevMpw3CqeK+4rCNbXE7nJ+GTM+sE/jifRo0yWWBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :message-id:mime-version:subject:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=AsT3BA46+bTpaogpB Oami5Y2RlII8eT6h9L9vSzts3E=; b=QsJ4ZOMBqtTvHpnclt8/rvoXqAxeU7nU+ CJfikIwWyYbiUPKbuNWgOnJ6YjEb0ebYA6Y51Tk2WNoFNOuo1S43M9IIk1d+Z5K+ CnKDGNtnG/B3khjUFtm4Dr85598MS9X9Q52c2Gf2WtDcf8smx83vmMkW1ZtSHzRI gLHOZVIUWn/dk75rY3wV7ZsNV3izpNyLp786gT64u7qnEY/TVT4QSjp/HzxwkT2X zlfPoI4mCXrx4Jb8g0+ZQHLfW29dypeSX5tJg5g6PBdyiF0d0o2agQ/QRGxC07rM X73SFQMsDgAaF+Ds82CR1WwXiTxc3/MKE4d/G66tIDe4NBL/OPqfQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudekjedgkeekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgggfestdekredtre dttdenucfhrhhomhepifgrvghtrghnucftihhvvghtuceoghhrihhvvgesuhdvheeirdhn vghtqeenucggtffrrghtthgvrhhnpeeukefhledufeeiueeuheevgeelveeutdefvdetve dtudeiudffudfhkeegfeekheenucffohhmrghinhepghhithhhuhgsrdgtohhmnecukfhp peekiedrvdehgedrvdegfedrudegieenucevlhhushhtvghrufhiiigvpedtnecurfgrrh grmhepmhgrihhlfhhrohhmpehgrhhivhgvsehuvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-930-146.w86-254.abo.wanadoo.fr [86.254.243.146]) by mail.messagingengine.com (Postfix) with ESMTPA id 7D48824006D; Mon, 12 Apr 2021 11:20:17 -0400 (EDT) From: Gaetan Rivet To: ovs-dev@openvswitch.org Date: Mon, 12 Apr 2021 17:19:46 +0200 Message-Id: X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Cc: maxime.coquelin@redhat.com, david.marchand@redhat.com Subject: [ovs-dev] [PATCH v2 00/28] dpif-netdev: Parallel offload processing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This patch series aims to improve the performance of the management of hw-offloads in dpif-netdev. In the current version, some setup will experience high memory usage and poor latency between a flow decision and its execution regarding hardware offloading. This series starts by measuring key metrics regarding both issues Those patches are introduced first to compare the current status with each improvements introduced. Offloads enqueued and inserted, as well as the latency from queue insertion to hardware insertion is measured. A new command 'ovs-appctl dpctl/offload-stats-show' is introduced to show the current measure. In my current performance test setup I am measuring an average latency hovering between 1~2 seconds. After the optimizations, it is reduced to 500~900 ms. Finally when using multiple threads and with proper driver support[1], it is measured in the order of 1 ms. A few modules are introduced: * An ID pool with reduced capabilities, simplifying its operations and allowing better performances in both single and multi-thread setup. * A lockless queue between PMDs / revalidators and offload thread(s). As the number of PMDs increases, contention can be high on the shared queue. This queue is designed to serve as message queue between threads. * A bounded lockless MPMC ring and some helpers for calculating moving averages. * A moving average module for Cumulative and Exponential moving averages. The netdev-offload-dpdk module is made thread-safe. Internal maps are made per-netdev instead, and locks are taken for shorter critical sections within the module. CI result: https://github.com/grivet/ovs/actions/runs/554918929 [1]: The rte_flow API was made thread-safe in the 20.11 DPDK release. Drivers that do not implement those operations concurrently are protected by a lock. Others will allow better concurrency, that improve the result of this series. v2: * Improved the MPSC queue API to simplify usage. * Moved flush operation from initiator thread to offload thread(s). This ensures offload metadata are shared only among the offload thread pool. * Flush operation needs additional thread synchronization. The ovs_barrier currently triggers a UAF. Add a unit-test to validate its operations and a fix for the UAF. CI result: https://github.com/grivet/ovs/actions/runs/741430135 The error comes from a failure to download 'automake' on osx, unrelated to any change in this series. Gaetan Rivet (28): netdev: Add flow API uninit function netdev-offload-dpdk: Use per-netdev offload metadata netdev-offload-dpdk: Implement hw-offload statistics read dpctl: Add function to read hardware offload statistics dpif-netdev: Rename flow offload thread dpif-netdev: Rename offload thread structure mov-avg: Add a moving average helper structure dpif-netdev: Implement hardware offloads stats query ovs-atomic: Expose atomic exchange operation mpsc-queue: Module for lock-free message passing llring: Add lockless MPMC bounded queue structure seq-pool: Module for faster ID generation netdev-offload: Add multi-thread API dpif-netdev: Quiesce offload thread periodically dpif-netdev: Postpone flow offload item freeing dpif-netdev: Use seq-pool for mark allocation dpif-netdev: Introduce tagged union of offload requests tests: Add ovs-barrier unit test ovs-thread: Fix barrier use-after-free dpif-netdev: Execute flush from offload thread netdev-offload-dpdk: Use per-thread HW offload stats netdev-offload-dpdk: Lock rte_flow map access netdev-offload-dpdk: Protect concurrent offload destroy/query dpif-netdev: Use lockless queue to manage offloads dpif-netdev: Make megaflow and mark mappings thread objects dpif-netdev: Replace port mutex by rwlock dpif-netdev: Use one or more offload threads netdev-dpdk: Remove rte-flow API access locks lib/automake.mk | 7 + lib/dpctl.c | 36 ++ lib/dpif-netdev.c | 742 +++++++++++++++++++++++++--------- lib/dpif-netlink.c | 1 + lib/dpif-provider.h | 7 + lib/dpif.c | 8 + lib/dpif.h | 9 + lib/llring.c | 153 +++++++ lib/llring.h | 76 ++++ lib/mov-avg.h | 171 ++++++++ lib/mpsc-queue.c | 251 ++++++++++++ lib/mpsc-queue.h | 189 +++++++++ lib/netdev-dpdk.c | 6 - lib/netdev-offload-dpdk.c | 277 +++++++++++-- lib/netdev-offload-provider.h | 4 + lib/netdev-offload.c | 92 ++++- lib/netdev-offload.h | 21 + lib/ovs-atomic-c++.h | 3 + lib/ovs-atomic-clang.h | 5 + lib/ovs-atomic-gcc4+.h | 5 + lib/ovs-atomic-gcc4.7+.h | 5 + lib/ovs-atomic-i586.h | 5 + lib/ovs-atomic-locked.h | 9 + lib/ovs-atomic-msvc.h | 22 + lib/ovs-atomic-pthreads.h | 5 + lib/ovs-atomic-x86_64.h | 5 + lib/ovs-atomic.h | 8 +- lib/ovs-thread.c | 61 ++- lib/ovs-thread.h | 6 +- lib/seq-pool.c | 198 +++++++++ lib/seq-pool.h | 66 +++ tests/automake.mk | 3 + tests/library.at | 15 + tests/test-barrier.c | 264 ++++++++++++ tests/test-mpsc-queue.c | 727 +++++++++++++++++++++++++++++++++ tests/test-seq-pool.c | 543 +++++++++++++++++++++++++ vswitchd/vswitch.xml | 16 + 37 files changed, 3788 insertions(+), 233 deletions(-) create mode 100644 lib/llring.c create mode 100644 lib/llring.h create mode 100644 lib/mov-avg.h create mode 100644 lib/mpsc-queue.c create mode 100644 lib/mpsc-queue.h create mode 100644 lib/seq-pool.c create mode 100644 lib/seq-pool.h create mode 100644 tests/test-barrier.c create mode 100644 tests/test-mpsc-queue.c create mode 100644 tests/test-seq-pool.c --- 2.31.1