From patchwork Thu Apr 4 12:09:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Maximets X-Patchwork-Id: 1919763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4V9L512rH5z1yYn for ; Thu, 4 Apr 2024 23:09:07 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9112F417F4; Thu, 4 Apr 2024 12:09:05 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id VfP0LH8wLLTR; Thu, 4 Apr 2024 12:09:04 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EAE074063D Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id EAE074063D; Thu, 4 Apr 2024 12:09:03 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BAF47C0072; Thu, 4 Apr 2024 12:09:03 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 86173C0037 for ; Thu, 4 Apr 2024 12:09:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6C20C40640 for ; Thu, 4 Apr 2024 12:09:01 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 8TjbQmGuR9m0 for ; Thu, 4 Apr 2024 12:09:00 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=217.70.183.194; helo=relay2-d.mail.gandi.net; envelope-from=i.maximets@ovn.org; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org D48314063D Authentication-Results: smtp4.osuosl.org; dmarc=none (p=none dis=none) header.from=ovn.org DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D48314063D Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp4.osuosl.org (Postfix) with ESMTPS id D48314063D for ; Thu, 4 Apr 2024 12:08:59 +0000 (UTC) Received: by mail.gandi.net (Postfix) with ESMTPSA id 09EAE40006; Thu, 4 Apr 2024 12:08:56 +0000 (UTC) From: Ilya Maximets To: ovs-dev@openvswitch.org Date: Thu, 4 Apr 2024 14:09:37 +0200 Message-ID: <20240404120938.2207251-1-i.maximets@ovn.org> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-GND-Sasl: i.maximets@ovn.org Cc: Ilya Maximets Subject: [ovs-dev] [PATCH] ofproto-dpif-upcall: Fix ukey installation failure logs and counters. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" ukey_install() returns boolean signaling if the ukey was installed or not. Installation may fail for a few reasons: 1. Conflicting ukey. 2. Mutex contention while trying to replace existing ukey. 3. The same ukey already exists and active. Only the first case here signals an actual problem. Third one is a little odd for userspace datapath, but harmless. Second is the most common one that can easily happen during normal operation since other threads like revalidators may be currently working on this ukey preventing an immediate access. Since only the first case is actually worth logging and it already has its own log message, removing the 'upcall installation fails' warning from the upcall_cb(). This should fix most of the random failures of userspace system tests in CI. While at it, also fixing coverage counters. Mutex contention was mistakenly counted as a duplicate upcall. ukey contention for revalidators was counted only in one of two places. New counter added for the ukey contention on replace. We should not re-use existing upcall_ukey_contention counter for this, since it may lead to double counting. Fixes: 67f08985d769 ("upcall: Replace ukeys for deleted flows.") Fixes: 9cec8274ed9a ("ofproto-dpif-upcall: Add VLOG_WARN_RL logs for upcall_cb() error.") Signed-off-by: Ilya Maximets Acked-by: Eelco Chaudron --- ofproto/ofproto-dpif-upcall.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c index d88195636..73901b651 100644 --- a/ofproto/ofproto-dpif-upcall.c +++ b/ofproto/ofproto-dpif-upcall.c @@ -59,6 +59,7 @@ COVERAGE_DEFINE(handler_duplicate_upcall); COVERAGE_DEFINE(revalidate_missed_dp_flow); COVERAGE_DEFINE(ukey_dp_change); COVERAGE_DEFINE(ukey_invalid_stat_reset); +COVERAGE_DEFINE(ukey_replace_contention); COVERAGE_DEFINE(upcall_flow_limit_grew); COVERAGE_DEFINE(upcall_flow_limit_hit); COVERAGE_DEFINE(upcall_flow_limit_kill); @@ -1449,8 +1450,6 @@ upcall_cb(const struct dp_packet *packet, const struct flow *flow, ovs_u128 *ufi } if (upcall.ukey && !ukey_install(udpif, upcall.ukey)) { - static struct vlog_rate_limit rll = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rll, "upcall_cb failure: ukey installation fails"); error = ENOSPC; } out: @@ -1948,15 +1947,15 @@ try_ukey_replace(struct umap *umap, struct udpif_key *old_ukey, transition_ukey(old_ukey, UKEY_DELETED); transition_ukey(new_ukey, UKEY_VISIBLE); replaced = true; + COVERAGE_INC(upcall_ukey_replace); + } else { + COVERAGE_INC(handler_duplicate_upcall); } ovs_mutex_unlock(&old_ukey->mutex); - } - - if (replaced) { - COVERAGE_INC(upcall_ukey_replace); } else { - COVERAGE_INC(handler_duplicate_upcall); + COVERAGE_INC(ukey_replace_contention); } + return replaced; } @@ -3009,6 +3008,7 @@ revalidator_sweep__(struct revalidator *revalidator, bool purge) /* Handler threads could be holding a ukey lock while it installs a * new flow, so don't hang around waiting for access to it. */ if (ovs_mutex_trylock(&ukey->mutex)) { + COVERAGE_INC(upcall_ukey_contention); continue; } ukey_state = ukey->state;