From patchwork Wed Jun 9 13:09:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1489868 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm2 header.b=fSwJildz; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm3 header.b=RMnsxOb3; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G0SCB6FDSz9sW8 for ; Wed, 9 Jun 2021 23:10:30 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5572B4059A; Wed, 9 Jun 2021 13:10:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RybYVkQhM2xj; Wed, 9 Jun 2021 13:10:23 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9D68B4058B; Wed, 9 Jun 2021 13:10:17 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 44F9AC0029; Wed, 9 Jun 2021 13:10:17 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0A4D9C002B for ; Wed, 9 Jun 2021 13:10:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7FA4783CD3 for ; Wed, 9 Jun 2021 13:10:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=u256.net header.b="fSwJildz"; dkim=pass (2048-bit key) header.d=messagingengine.com header.b="RMnsxOb3" Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EJcI4Ncl86jx for ; Wed, 9 Jun 2021 13:10:08 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by smtp1.osuosl.org (Postfix) with ESMTPS id DE60E83CCF for ; Wed, 9 Jun 2021 13:10:02 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 445DF2657; Wed, 9 Jun 2021 09:10:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 09 Jun 2021 09:10:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm2; bh=Yvl4r9kQh64HU ASVvgT1bAiJYkfZycRowrVksd8jWC8=; b=fSwJildzxBAYjxoOFgqyp/2jAP/s4 WsR6ql89v9ym2Tcfu+UIb2WR4e4/eslUYRzGNzdgYDsMzRlpPwPOO3YBY6Lc1vHB N2VSHGm2b5xtSlG1DcgTe3meD5kRyy58qI2WG/p8MnX1KyNNxUyjGdsuPc8+VlZl i+Heu1tSYI01Ijo2brIXVtRJoIfVI/gWu/H7OVGf/2BI3syMBWRADn1+Xqm7PiK3 nhTUOFafoET7X8Y8gLrsIq1YUOlX1R79AmU/9UiJzm+ZE365XDXRFu9HWI/rWfYg ND0bj+pVl1TegMyWnh8RXmtSFrajIK2G26oz26okp0/rmwHr2b8zN6dqA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=Yvl4r9kQh64HUASVvgT1bAiJYkfZycRowrVksd8jWC8=; b=RMnsxOb3 VggMq/MVoOiEsxe1zjbaSkhbRyphVri1x/U6CJ16zKvgVPAvJGw+qFwaL6zgTqsD R1JlAgNMUgp1RH2odVXtrZAMMfuFxr09rWvStTTLSUl6LSh4Nf7VOiux/jHzDt3A N3/MAQ5LeVTRgLqvRr6RDz7lRDo3Supa3uZAV1nQTxUYiRHllrED4EOv008Rhurm 13YUyPdYS6nMW3x79UJ/FK1kb/1HM/+wK2OsLEIGoQrXM9zRb52BsvIRmUf5jjuW BCjkjWtqMmOPYyc99Q9o7yu7FNFVx1Gs23Wx59i5dGZTM6UID1w64iQKi4hQRiN6 AzTRw8woNlG1mQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeduuddgieefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefirggvthgr nhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedrnhgvtheqnecuggftrfgrthhtvghrnh ephefgveffkeetheetfeeifedvheelfeejfeehveduteejhfekuedtkeeiuedvteehnecu vehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepghhrihhvvg esuhdvheeirdhnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 9 Jun 2021 09:10:01 -0400 (EDT) From: Gaetan Rivet To: ovs-dev@openvswitch.org Date: Wed, 9 Jun 2021 15:09:19 +0200 Message-Id: <3ab29aa015e2c114a826a104d3ac9d6af5dd5b3e.1623234822.git.grive@u256.net> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v4 11/27] ovs-atomic: Expose atomic exchange operation X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The atomic exchange operation is a useful primitive that should be available as well. Most compiler already expose or offer a way to use it, but a single symbol needs to be defined. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/ovs-atomic-c++.h | 3 +++ lib/ovs-atomic-clang.h | 5 +++++ lib/ovs-atomic-gcc4+.h | 5 +++++ lib/ovs-atomic-gcc4.7+.h | 5 +++++ lib/ovs-atomic-i586.h | 5 +++++ lib/ovs-atomic-locked.h | 9 +++++++++ lib/ovs-atomic-msvc.h | 22 ++++++++++++++++++++++ lib/ovs-atomic-pthreads.h | 5 +++++ lib/ovs-atomic-x86_64.h | 5 +++++ lib/ovs-atomic.h | 8 +++++++- 10 files changed, 71 insertions(+), 1 deletion(-) diff --git a/lib/ovs-atomic-c++.h b/lib/ovs-atomic-c++.h index d47b8dd39..8605fa9d3 100644 --- a/lib/ovs-atomic-c++.h +++ b/lib/ovs-atomic-c++.h @@ -29,6 +29,9 @@ using std::atomic_compare_exchange_strong_explicit; using std::atomic_compare_exchange_weak; using std::atomic_compare_exchange_weak_explicit; +using std::atomic_exchange; +using std::atomic_exchange_explicit; + #define atomic_read(SRC, DST) \ atomic_read_explicit(SRC, DST, memory_order_seq_cst) #define atomic_read_explicit(SRC, DST, ORDER) \ diff --git a/lib/ovs-atomic-clang.h b/lib/ovs-atomic-clang.h index 34cc2faa7..cdf02a512 100644 --- a/lib/ovs-atomic-clang.h +++ b/lib/ovs-atomic-clang.h @@ -67,6 +67,11 @@ typedef enum { #define atomic_compare_exchange_weak_explicit(DST, EXP, SRC, ORD1, ORD2) \ __c11_atomic_compare_exchange_weak(DST, EXP, SRC, ORD1, ORD2) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + __c11_atomic_exchange(RMW, ARG, ORDER) + #define atomic_add(RMW, ARG, ORIG) \ atomic_add_explicit(RMW, ARG, ORIG, memory_order_seq_cst) #define atomic_sub(RMW, ARG, ORIG) \ diff --git a/lib/ovs-atomic-gcc4+.h b/lib/ovs-atomic-gcc4+.h index 25bcf20a0..f9accde1a 100644 --- a/lib/ovs-atomic-gcc4+.h +++ b/lib/ovs-atomic-gcc4+.h @@ -128,6 +128,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + __sync_lock_test_and_set(DST, SRC) +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + #define atomic_op__(RMW, OP, ARG, ORIG) \ ({ \ typeof(RMW) rmw__ = (RMW); \ diff --git a/lib/ovs-atomic-gcc4.7+.h b/lib/ovs-atomic-gcc4.7+.h index 4c197ebe0..846e05775 100644 --- a/lib/ovs-atomic-gcc4.7+.h +++ b/lib/ovs-atomic-gcc4.7+.h @@ -61,6 +61,11 @@ typedef enum { #define atomic_compare_exchange_weak_explicit(DST, EXP, SRC, ORD1, ORD2) \ __atomic_compare_exchange_n(DST, EXP, SRC, true, ORD1, ORD2) +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + __atomic_exchange_n(DST, SRC, ORDER) +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + #define atomic_add(RMW, OPERAND, ORIG) \ atomic_add_explicit(RMW, OPERAND, ORIG, memory_order_seq_cst) #define atomic_sub(RMW, OPERAND, ORIG) \ diff --git a/lib/ovs-atomic-i586.h b/lib/ovs-atomic-i586.h index 9a385ce84..35a0959ff 100644 --- a/lib/ovs-atomic-i586.h +++ b/lib/ovs-atomic-i586.h @@ -400,6 +400,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + atomic_exchange__(RMW, ARG, ORDER) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) + #define atomic_add__(RMW, ARG, CLOB) \ asm volatile("lock; xadd %0,%1 ; " \ "# atomic_add__ " \ diff --git a/lib/ovs-atomic-locked.h b/lib/ovs-atomic-locked.h index f8f0ba2a5..bf38c4a43 100644 --- a/lib/ovs-atomic-locked.h +++ b/lib/ovs-atomic-locked.h @@ -31,6 +31,15 @@ void atomic_unlock__(void *); atomic_unlock__(DST), \ false))) +#define atomic_exchange_locked(DST, SRC) \ + ({ \ + atomic_lock__(DST); \ + typeof(*(DST)) __tmp = *(DST); \ + *(DST) = SRC; \ + atomic_unlock__(DST); \ + __tmp; \ + }) + #define atomic_op_locked_add += #define atomic_op_locked_sub -= #define atomic_op_locked_or |= diff --git a/lib/ovs-atomic-msvc.h b/lib/ovs-atomic-msvc.h index 9def887d3..ef8310269 100644 --- a/lib/ovs-atomic-msvc.h +++ b/lib/ovs-atomic-msvc.h @@ -345,6 +345,28 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +/* While intrinsics offering different memory ordering + * are available in MSVC C compiler, they are not defined + * in the C++ compiler. Ignore for compatibility. + * + * Use nested ternary operators as the GNU extension ({}) + * is not available. + */ + +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + ((sizeof *(DST) == 1) ? \ + _InterlockedExchange8((char volatile *) DST, SRC) \ + : (sizeof *(DST) == 2) ? \ + _InterlockedExchange16((short volatile *) DST, SRC) \ + : (sizeof *(DST) == 4) ? \ + _InterlockedExchange((long int volatile *) DST, SRC) \ + : (sizeof *(DST) == 8) ? \ + _InterlockedExchange64((__int64 volatile *) DST, SRC) \ + : (ovs_abort(), 0)) + +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + /* MSVCs c++ compiler implements c11 atomics and looking through its * implementation (in xatomic.h), orders are ignored for x86 platform. * Do the same here. */ diff --git a/lib/ovs-atomic-pthreads.h b/lib/ovs-atomic-pthreads.h index 12234e79e..570a67fe4 100644 --- a/lib/ovs-atomic-pthreads.h +++ b/lib/ovs-atomic-pthreads.h @@ -77,6 +77,11 @@ atomic_signal_fence(memory_order order OVS_UNUSED) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange(DST, SRC) \ + atomic_exchange_locked(DST, SRC) +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + ((void) (ORDER), atomic_exchange(DST, SRC)) + #define atomic_add(RMW, ARG, ORIG) atomic_op_locked(RMW, add, ARG, ORIG) #define atomic_sub(RMW, ARG, ORIG) atomic_op_locked(RMW, sub, ARG, ORIG) #define atomic_or( RMW, ARG, ORIG) atomic_op_locked(RMW, or, ARG, ORIG) diff --git a/lib/ovs-atomic-x86_64.h b/lib/ovs-atomic-x86_64.h index 1e7d42707..3bdaf2f08 100644 --- a/lib/ovs-atomic-x86_64.h +++ b/lib/ovs-atomic-x86_64.h @@ -274,6 +274,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + atomic_exchange__(RMW, ARG, ORDER) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) + #define atomic_add__(RMW, ARG, CLOB) \ asm volatile("lock; xadd %0,%1 ; " \ "# atomic_add__ " \ diff --git a/lib/ovs-atomic.h b/lib/ovs-atomic.h index 11fa19268..8fdce0cf8 100644 --- a/lib/ovs-atomic.h +++ b/lib/ovs-atomic.h @@ -210,7 +210,7 @@ * In this section, A is an atomic type and C is the corresponding non-atomic * type. * - * The "store" and "compare_exchange" primitives match C11: + * The "store", "exchange", and "compare_exchange" primitives match C11: * * void atomic_store(A *object, C value); * void atomic_store_explicit(A *object, C value, memory_order); @@ -244,6 +244,12 @@ * efficiently, so it should be used if the application will need to * loop anyway. * + * C atomic_exchange(A *object, C desired); + * C atomic_exchange_explicit(A *object, C desired, memory_order); + * + * Atomically stores 'desired' into '*object', returning the value + * previously held. + * * The following primitives differ from the C11 ones (and have different names) * because there does not appear to be a way to implement the standard * primitives in standard C: