From patchwork Fri Sep 23 16:29:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng He X-Patchwork-Id: 1681768 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ZGyROvvW; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MZH9Z2XWcz1yqM for ; Sat, 24 Sep 2022 15:08:49 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BD5DD60D95; Sat, 24 Sep 2022 05:08:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BD5DD60D95 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ZGyROvvW X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h0nc_KUK5-Eb; Sat, 24 Sep 2022 05:08:46 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9D39660E07; Sat, 24 Sep 2022 05:08:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9D39660E07 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 617A7C0070; Sat, 24 Sep 2022 05:08:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0AAC0C002D for ; Sat, 24 Sep 2022 05:08:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C0C4541B5F for ; Sat, 24 Sep 2022 05:08:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C0C4541B5F Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=ZGyROvvW X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AY5xcp0eLmf1 for ; Sat, 24 Sep 2022 05:08:43 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 1A14C41B5D Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1A14C41B5D for ; Sat, 24 Sep 2022 05:08:43 +0000 (UTC) Received: by mail-pj1-x1031.google.com with SMTP id x1-20020a17090ab00100b001fda21bbc90so7684508pjq.3 for ; Fri, 23 Sep 2022 22:08:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date; bh=yDTgknB+Yx8fauACd/B1W8M0qilLiOhYSyraWmujGUg=; b=ZGyROvvW1/UzuBMyTcKYet1ND8tlM6SQv67J86k8DiF3NdujIaxGQYf5WDKUUM8wSA luUpKQvLzLVQGT4yilbqB/MHH1oWLGxKcBJeQNNVSoVO73GqnBEgasdqio9UvKAtIlGB kkV7DivYa3whNmUYVTSogad3vi63u89kgRLL5Lbu9qUGxLaxmmSpYlDr5L26wlcGmcVf cY+44GHfHB3aAms3xhBHhckjStkyAZVER0i+sjSglUqltPSIzJEvOeHl7iTZbzBSd4BZ l47LF6VS31dEEDC3QloZrh6VpMK67cUph48CaLDXWVr8wWbWFEUT5BnriyE097inhk4g UcPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date; bh=yDTgknB+Yx8fauACd/B1W8M0qilLiOhYSyraWmujGUg=; b=3asBOs7DqByipl7E7t+xB+a0lB7/Gra7KsoLB6W2Y82O7BcW4CvnJU3YNgUQZ3uNde USCGuM1TNIR6WcHztKDPZGTtvv7nA+kB9Emoctzo/ILK69Yn7UQQ6YPdkspusTg2JYX7 r7i91vHKHSkd0sR8WpDYTnVDipTiCEKh4SLSsTA1WLaOn8BAVy7DZXojyV9mp596nZxz 8qtQLtSGHZcEu6kpmULiTc6XAtEK/GDLMCgvioeUU3hAIU/XXZxF2DW/5Cs25hTB8Khm SSFsBuzbfDStmT2259zvPhQ6qD7GHYDc/WmVqCR2k1ZmY4goWgFL6witGHwg48w/o4BM jzWg== X-Gm-Message-State: ACrzQf0C5fUoKBEo6BsOi6Xv0ZWT7gQDkAF3TNtuIFHILbTKLLqT/ekM aHIW2SAbJeWD/IOxF17r9IdzH21iUCeRE5ZD X-Google-Smtp-Source: AMsMyM77yR8WIiYT5n9o7gjc4UMmyBb8NmzEoEVZOti1UP0dwlkyOevkbIYJrcD5p55KDgT46540ow== X-Received: by 2002:a17:90a:7843:b0:203:5861:fc3d with SMTP id y3-20020a17090a784300b002035861fc3dmr13105562pjl.87.1663996121981; Fri, 23 Sep 2022 22:08:41 -0700 (PDT) Received: from localhost ([139.177.225.230]) by smtp.gmail.com with ESMTPSA id w22-20020aa79556000000b0053e0d6f353esm7467986pfq.27.2022.09.23.22.08.41 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 22:08:41 -0700 (PDT) From: Peng He X-Google-Original-From: Peng He To: dev@openvswitch.org Date: Fri, 23 Sep 2022 16:29:19 +0000 Message-Id: <20220923162920.612492-3-hepeng.0320@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220923162920.612492-1-hepeng.0320@bytedance.com> References: <20220923162920.612492-1-hepeng.0320@bytedance.com> MIME-Version: 1.0 Subject: [ovs-dev] [ovs-dev v3 3/4] ofproto-dpif-upcall: new ukey needs to take the old ukey's dump seq X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The userspace datapath mananges all the magaflows by a cmap. The cmap data structrue will grow/shrink during the datapath processing and it will re-position megaflows. This might result in two revalidator threads might process a same megaflow during one dump stage. Consider a situation that, revalidator 1 processes a megaflow A, and decides to delete it from the datapath, at the mean time, this megaflow A is also queued in the process batch of revalidator 2. Normally it's ok for revalidators to process the same megaflow multiple times, as the dump_seq shows it's already dumped and the stats will not be contributed twice. Assume that right after A is deleted, a PMD thread generates again a new megaflow B which has the same match and action of A. The ukey of megaflow B will replace the one of megaflow A. Now the ukey B is new to the revalidator system and its dump seq is 0. Now since the dump seq of ukey B is 0, when processing megaflow A, the revalidator 2 will not identify this megaflow A has already been dumped by revalidator 1 and will contribute the old megaflow A's stats again, this results in an inconsistent stats between ukeys and megaflows. To fix this, the newly generated the ukey B should take the dump_seq of the replaced ukey A to avoid a same megaflow being revalidated twice in one dump stage. We observe in the production environment, the OpenFlow rules' stats sometimes are amplified compared to the actual value. I believe this is also the reason that why somtimes there is mismatch between the ukey and megaflow in stats value. The Eelco's patch [ovs-dev] [PATCH v2 09/10] revalidator: Fix datapath statistics update tried to fix it in the past. Signed-off-by: Peng He --- ofproto/ofproto-dpif-upcall.c | 1 + 1 file changed, 1 insertion(+) diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c index e8bbcfeaf..89fad1bdf 100644 --- a/ofproto/ofproto-dpif-upcall.c +++ b/ofproto/ofproto-dpif-upcall.c @@ -1877,6 +1877,7 @@ try_ukey_replace(struct umap *umap, struct udpif_key *old_ukey, ovs_mutex_lock(&new_ukey->mutex); cmap_replace(&umap->cmap, &old_ukey->cmap_node, &new_ukey->cmap_node, new_ukey->hash); + new_ukey->dump_seq = old_ukey->dump_seq; ovsrcu_postpone(ukey_delete__, old_ukey); transition_ukey(old_ukey, UKEY_DELETED); transition_ukey(new_ukey, UKEY_VISIBLE);