From patchwork Mon Mar 27 11:25:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eelco Chaudron X-Patchwork-Id: 1761521 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fD8FdUHk; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PlVqF3d10z1yXv for ; Mon, 27 Mar 2023 22:25:29 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 14AA34030A; Mon, 27 Mar 2023 11:25:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 14AA34030A Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fD8FdUHk X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BdkwbBXfkO5k; Mon, 27 Mar 2023 11:25:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id A7E1F402B8; Mon, 27 Mar 2023 11:25:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A7E1F402B8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7AE9BC0035; Mon, 27 Mar 2023 11:25:23 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9BB44C0032 for ; Mon, 27 Mar 2023 11:25:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 699C081C40 for ; Mon, 27 Mar 2023 11:25:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 699C081C40 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=fD8FdUHk X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E3-MqdTFlPcc for ; Mon, 27 Mar 2023 11:25:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9B8DA81BE3 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9B8DA81BE3 for ; Mon, 27 Mar 2023 11:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679916319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TjjAZ+3h0u4mOUPq89YcBpWpM43N+6A6qTj+fxLYd/8=; b=fD8FdUHkGFxkNLlidxjw4mMnUS7C+RXut9z+dBs+DcsfkoJlmdcslA5OU1fZyuPNlkOB0d 5YyFKA6oJUGcgHEEYBzDTqeKzWKdS3MyGk/X1ZWgEe7e6VD0FHN7rYz4kLR0rtkBbJur7g Z/lb3lwQFaRgVIsUctn+5yWUBcyRSLw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-MJeUMviMNgOheAZCPr4oUg-1; Mon, 27 Mar 2023 07:25:18 -0400 X-MC-Unique: MJeUMviMNgOheAZCPr4oUg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1079A185A791 for ; Mon, 27 Mar 2023 11:25:18 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.193.129]) by smtp.corp.redhat.com (Postfix) with ESMTP id B16D81121330 for ; Mon, 27 Mar 2023 11:25:17 +0000 (UTC) From: Eelco Chaudron To: dev@openvswitch.org Date: Mon, 27 Mar 2023 13:25:06 +0200 Message-Id: <167991626686.4046542.5458159147307773259.stgit@ebuild.local> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH] seq: Make read of the current value atomic X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Make the read of the current seq->value atomic, i.e., not needing to acquire the global mutex when reading it. On 64-bit systems, this incurs no overhead, and it will avoid the mutex and potentially a system call. For incrementing the value followed by waking up the threads, we are still taking the mutex, so the current behavior is not changing. The seq_read() behavior is already defined as, "Returns seq's current sequence number (which could change immediately)". So the change should not impact the current behavior. Signed-off-by: Eelco Chaudron Acked-by: Mike Pattrick --- lib/ovs-rcu.c | 2 +- lib/seq.c | 37 ++++++++++++++++--------------------- lib/seq.h | 1 - 3 files changed, 17 insertions(+), 23 deletions(-) diff --git a/lib/ovs-rcu.c b/lib/ovs-rcu.c index 946aa04d1..9e07d9bab 100644 --- a/lib/ovs-rcu.c +++ b/lib/ovs-rcu.c @@ -170,7 +170,7 @@ ovsrcu_try_quiesce(void) ovs_assert(!single_threaded()); perthread = ovsrcu_perthread_get(); if (!seq_try_lock()) { - perthread->seqno = seq_read_protected(global_seqno); + perthread->seqno = seq_read(global_seqno); if (perthread->cbset) { ovsrcu_flush_cbset__(perthread, true); } diff --git a/lib/seq.c b/lib/seq.c index 99e5bf8bd..22646f3f8 100644 --- a/lib/seq.c +++ b/lib/seq.c @@ -32,7 +32,7 @@ COVERAGE_DEFINE(seq_change); /* A sequence number object. */ struct seq { - uint64_t value OVS_GUARDED; + atomic_uint64_t value; struct hmap waiters OVS_GUARDED; /* Contains 'struct seq_waiter's. */ }; @@ -72,6 +72,7 @@ static void seq_wake_waiters(struct seq *) OVS_REQUIRES(seq_mutex); struct seq * OVS_EXCLUDED(seq_mutex) seq_create(void) { + uint64_t seq_value; struct seq *seq; seq_init(); @@ -81,7 +82,8 @@ seq_create(void) COVERAGE_INC(seq_change); ovs_mutex_lock(&seq_mutex); - seq->value = seq_next++; + seq_value = seq_next++; + atomic_store_relaxed(&seq->value, seq_value); hmap_init(&seq->waiters); ovs_mutex_unlock(&seq_mutex); @@ -126,9 +128,11 @@ void seq_change_protected(struct seq *seq) OVS_REQUIRES(seq_mutex) { + uint64_t seq_value = seq_next++; + COVERAGE_INC(seq_change); - seq->value = seq_next++; + atomic_store_relaxed(&seq->value, seq_value); seq_wake_waiters(seq); } @@ -143,18 +147,6 @@ seq_change(struct seq *seq) ovs_mutex_unlock(&seq_mutex); } -/* Returns 'seq''s current sequence number (which could change immediately). - * - * seq_read() and seq_wait() can be used together to yield a race-free wakeup - * when an object changes, even without an ability to lock the object. See - * Usage in seq.h for details. */ -uint64_t -seq_read_protected(const struct seq *seq) - OVS_REQUIRES(seq_mutex) -{ - return seq->value; -} - /* Returns 'seq''s current sequence number (which could change immediately). * * seq_read() and seq_wait() can be used together to yield a race-free wakeup @@ -162,14 +154,15 @@ seq_read_protected(const struct seq *seq) * Usage in seq.h for details. */ uint64_t seq_read(const struct seq *seq) - OVS_EXCLUDED(seq_mutex) { uint64_t value; - ovs_mutex_lock(&seq_mutex); - value = seq_read_protected(seq); - ovs_mutex_unlock(&seq_mutex); - + /* We use atomic_read(), memory_order_seq_cst, to simulate the full + * memory barrier you would get with locking and unlocking the global + * mutex. + * + * Note that the odd CONST_CAST() is here to keep sparse happy. */ + atomic_read(&CONST_CAST(struct seq *, seq)->value, &value); return value; } @@ -224,9 +217,11 @@ seq_wait_at(const struct seq *seq_, uint64_t value, const char *where) OVS_EXCLUDED(seq_mutex) { struct seq *seq = CONST_CAST(struct seq *, seq_); + uint64_t seq_value; ovs_mutex_lock(&seq_mutex); - if (value == seq->value) { + atomic_read_relaxed(&seq->value, &seq_value); + if (value == seq_value) { seq_wait__(seq, value, where); } else { poll_immediate_wake_at(where); diff --git a/lib/seq.h b/lib/seq.h index c88b9d1c8..fcfa01037 100644 --- a/lib/seq.h +++ b/lib/seq.h @@ -128,7 +128,6 @@ void seq_unlock(void); /* For observers. */ uint64_t seq_read(const struct seq *); -uint64_t seq_read_protected(const struct seq *); void seq_wait_at(const struct seq *, uint64_t value, const char *where); #define seq_wait(seq, value) seq_wait_at(seq, value, OVS_SOURCE_LOCATOR)