From patchwork Tue Jun 15 23:22:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1492518 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm2 header.b=bGNhP03b; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm3 header.b=TXmgrEfO; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G4PWQ5h5lz9sX2 for ; Wed, 16 Jun 2021 09:23:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6EC3283C2C; Tue, 15 Jun 2021 23:23:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CupJaLpLnYTl; Tue, 15 Jun 2021 23:23:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 93A2A8356E; Tue, 15 Jun 2021 23:23:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6B820C000E; Tue, 15 Jun 2021 23:23:10 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3EF88C000B for ; Tue, 15 Jun 2021 23:23:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2E0A483C5E for ; Tue, 15 Jun 2021 23:23:07 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LjFtC_gZDh4y for ; Tue, 15 Jun 2021 23:23:05 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by smtp1.osuosl.org (Postfix) with ESMTPS id EAA2B83AFD for ; Tue, 15 Jun 2021 23:23:04 +0000 (UTC) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 2455F5C00C8 for ; Tue, 15 Jun 2021 19:23:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 15 Jun 2021 19:23:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:subject:date:message-id:mime-version :content-transfer-encoding; s=fm2; bh=fhHG98ksIx5JzP7a2GszI78BVo 0cuIPzWUDAZfMrNuU=; b=bGNhP03b5huJ++YBtQCGY9Wf7sbjOfm3YET7kGakCt vr+Apkew6pK+L+qxNF+XvunmYsR1Vv8j74EHF+BAW0fITDo4WSrlSNjnlK2pVA0P NZ0DLpTHg9OcHQS58qsOmsSHEmXL9s0oD07MOd7IR4i/oAiALfKjw04qX+5lm8aR GSV+b3FwlSsc0WMhSGCvw2+JXTyFFNvl/ty6y1yNzS7BgpPFzp3jYXuAVNtcHtr5 ex3EW0EPwFXgUMJpTYklbtxOXhDBL7LuW46oseOGYPfYJa5kubI4TlZgbrJg6BJr lWNdY/uFbkMwunPvXpvYOANNRGyUU+TRIJR1Pwob4MtQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:date:from :message-id:mime-version:subject:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=fhHG98ksIx5JzP7a2 GszI78BVo0cuIPzWUDAZfMrNuU=; b=TXmgrEfOy1jTRj090mMvpWu1QglXdfBU2 huNkD8ADXBf1nnlmcr82mvBJo/c4ca3IPWWevvmj4YaxdYwQO8GY4YyM9CeCf3YP DS5vbjQEQfxtt3AtHnlcNE+SZkw7mBkbLg6a7LpVNp5uCJ7l2RW5fTGvdEqNc2L6 k2Hp87mzxHCPFxOlOODpWXBq6K4cXgrT/VEYxURYsQVfSk1ajqe7zsFCEs9RnbYJ DrjIhPrEEL2Y/x29W57BhkigJYRamfy0Lem38Qjk8NvOg+EDLJHLcP3dt24dqG6M a97Ry+Kpy7STd/646giOhV1gIeROLrE7Aj5Vxo6UmoOoIbjagtZkQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfedvkedgvddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefhvffufffkofgggfestdekredtre dttdenucfhrhhomhepifgrvghtrghnucftihhvvghtuceoghhrihhvvgesuhdvheeirdhn vghtqeenucggtffrrghtthgvrhhnpeevueffjeduvdejkeevieeujefftdefheefffehke dtgfdvgeeigeevkeeuhedujeenucffohhmrghinhepghhithhhuhgsrdgtohhmpdhoiihl rggsshdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehgrhhivhgvsehuvdehiedrnhgvth X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 15 Jun 2021 19:23:01 -0400 (EDT) From: Gaetan Rivet To: ovs-dev@openvswitch.org Date: Wed, 16 Jun 2021 01:22:45 +0200 Message-Id: X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v3 0/8] conntrack: improve multithread scalability X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Conntracks are executed within the datapath. Locks along this path are crucial and their critical section should be minimal. The global 'ct_lock' is necessary before any action taken on connection states. This lock is needed for many operations on the conntrack, slowing down the datapath. The cleanup thread 'ct_clean' will take it to do its job. As it can hold it a long time, the thread is limited in amount of connection cleaned per round, and calls are rate-limited. * Timeout policies locking is contrived to avoid deadlock. Anytime a connection state is updated, during its update it is unlocked, 'ct_lock' is taken, then the connection is locked again. Then the reverse is done for unlock. * Scalability is poor. The global ct_lock needs to be taken before applying any change to a conn object. This is backward: local changes to smaller objects should be independent, then the global lock should only be taken once the rest of the work is done, the goal being to have the smallest possible critical section. It can be improved. Using RCU-friendly structures for connections, zone limits and timeout policies, read-first workload is improved and the precedence of the global 'ct_lock' and local 'conn->lock' can be inversed. Running the conntrack benchmark we see these changes: ./tests/ovstest test-conntrack benchmark 3000000 32 code \ N 1 2 4 8 Before 2310 2766 6117 19838 (ms) After 2072 2084 2653 4541 (ms) One thread in the benchmark executes the task of a PMD, while the 'ct_clean' thread runs in background as well. Github actions: https://github.com/grivet/ovs/actions/runs/574446345 v2: An mpsc-queue is used instead of rculist to manage connection expirations lists. PMDs and ct_clean all act as producers, while ct_clean is the sole consumer thread. A PMD now needs to take the 'ct_lock' only when creating a new connection, and only while inserting it in the conn CMAP. For any updates, only the conn lock is now required, to properly change its state. The mpsc-queue implementation is identical to the one from the parallel offload series [1]. CI: https://github.com/grivet/ovs/actions/runs/772118640 [1]: https://patchwork.ozlabs.org/project/openvswitch/list/?series=238779 v3: The last part of the series modifying the rate limit of conntrack_clean is dropped. It is not necessary to improve scalability and can be done later if needed. CI: https://github.com/grivet/ovs/actions/runs/940610003 On my local development laptop, the benchmark gives different numbers since v1: ./tests/ovstest test-conntrack benchmark 3000000 32 code \ N 1 2 4 8 Before 598 1656 12612 39301 (ms) After 293 337 427 893 (ms) I replicate the numbers on a 24-cores machine as well. The benchmark is not very accurate as no core pinning and no isolation is done. Gaetan Rivet (8): conntrack: Init hash basis first at creation ovs-atomic: Expose atomic exchange operation mpsc-queue: Module for lock-free message passing conntrack: Use mpsc-queue to store conn expirations conntrack: Use a cmap to store zone limits conntrack-tp: Use a cmap to store timeout policies conntrack: Inverse conn and ct lock precedence conntrack: Use an atomic conn expiration value lib/automake.mk | 2 + lib/conntrack-private.h | 97 +++-- lib/conntrack-tp.c | 100 ++--- lib/conntrack.c | 278 ++++++++++---- lib/conntrack.h | 4 +- lib/dpif-netdev.c | 5 +- lib/mpsc-queue.c | 251 +++++++++++++ lib/mpsc-queue.h | 190 ++++++++++ lib/ovs-atomic-c++.h | 3 + lib/ovs-atomic-clang.h | 5 + lib/ovs-atomic-gcc4+.h | 5 + lib/ovs-atomic-gcc4.7+.h | 5 + lib/ovs-atomic-i586.h | 5 + lib/ovs-atomic-locked.h | 9 + lib/ovs-atomic-msvc.h | 22 ++ lib/ovs-atomic-pthreads.h | 5 + lib/ovs-atomic-x86_64.h | 5 + lib/ovs-atomic.h | 8 +- tests/automake.mk | 1 + tests/library.at | 5 + tests/test-mpsc-queue.c | 772 ++++++++++++++++++++++++++++++++++++++ 21 files changed, 1608 insertions(+), 169 deletions(-) create mode 100644 lib/mpsc-queue.c create mode 100644 lib/mpsc-queue.h create mode 100644 tests/test-mpsc-queue.c --- 2.31.1