From patchwork Wed Jun 9 13:09:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1489872 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm2 header.b=IgDtx5zT; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm3 header.b=sQ4YPmOO; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4G0SD12ncqz9sWD for ; Wed, 9 Jun 2021 23:11:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7AFF560A58; Wed, 9 Jun 2021 13:11:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HdhodJ_WoIdI; Wed, 9 Jun 2021 13:11:06 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id A6A7D60A3F; Wed, 9 Jun 2021 13:10:46 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7347DC000B; Wed, 9 Jun 2021 13:10:46 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 25232C000D for ; Wed, 9 Jun 2021 13:10:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 28EAB83CED for ; Wed, 9 Jun 2021 13:10:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=u256.net header.b="IgDtx5zT"; dkim=pass (2048-bit key) header.d=messagingengine.com header.b="sQ4YPmOO" Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WPi6Fzgad86l for ; Wed, 9 Jun 2021 13:10:19 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by smtp1.osuosl.org (Postfix) with ESMTPS id B830183D28 for ; Wed, 9 Jun 2021 13:10:09 +0000 (UTC) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 89A4A31E; Wed, 9 Jun 2021 09:10:08 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 09 Jun 2021 09:10:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm2; bh=FW999+fbN+b07 wCBEw8Y1fEtb4/PK284WWVtaM6yq0U=; b=IgDtx5zT82M4bav9PyTsmt34Gltnr tftNVxzlh6tXESDYTXmELJBiLbU2OlF5alPwDtKlm3Z+D3p0DazUVmFlYAsebVud lstYWISdoFZovX4hxlcWPU8VAhjy/xifgNcafJUVBXJV7J8cHgdD9kzLeAMhky9j Fg2LKuNI+oddRHAUqvxW8pcw4A13Z3BeEp5iqH6U6U+s209sA5z7gnW38l6y66ih moiHwthmVsVg/GWYA0XUN2+suufizgSF6qYBl0vbGnF3TfSNySguiDRw+SU1V40k LXqq6V42mCy8ZBug2Z4x+QEe1W2NV0BoFOnePsxKol1dPucuY7JjC+UOw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=FW999+fbN+b07wCBEw8Y1fEtb4/PK284WWVtaM6yq0U=; b=sQ4YPmOO 68h9c5foH313l3X3ueEjoaYXwm9yz9bkpE8+gl+3wGtVFr5GydyFo066Mt+YiHX3 wRJ/nZR4bFV+JsgsF9N9aXj0lHMQysicihVpZpJFnRTd458xzLCvTUmMEfVINsAC 3KauqSalsglDob1pttG+Wd6LN4tkAP4WMQMuTvshe6vdXOulxijkP+OiVpHz+U0L vx4xFr/4ZTsthpGxbxI0xrMGny/H5toWh9bC4eVRI4ABIZN5j4oJB9SFv6whSUIp TB+nzuRU3FXGmuEBnrqaQv+NiQHYiho5BKEdbbyZJVo1LIjUhdV2+J/o5Pk05BWq +pxsNhBykI77Cw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeduuddgieefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefirggvthgr nhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedrnhgvtheqnecuggftrfgrthhtvghrnh ephefgveffkeetheetfeeifedvheelfeejfeehveduteejhfekuedtkeeiuedvteehnecu vehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepghhrihhvvg esuhdvheeirdhnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 9 Jun 2021 09:10:07 -0400 (EDT) From: Gaetan Rivet To: ovs-dev@openvswitch.org Date: Wed, 9 Jun 2021 15:09:23 +0200 Message-Id: X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v4 15/27] dpif-netdev: Quiesce offload thread periodically X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Similar to what was done for the PMD threads [1], reduce the performance impact of quiescing too often in the offload thread. After each processed offload, the offload thread currently quiesce and will sync with RCU. This synchronization can be lengthy and make the thread unnecessary slow. Instead attempt to quiesce every 10 ms at most. While the queue is empty, the offload thread remains quiescent. [1]: 81ac8b3b194c ("dpif-netdev: Do RCU synchronization at fixed interval in PMD main loop.") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Reviewed-by: Maxime Coquelin --- lib/dpif-netdev.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index a20eeda4d..e403e461a 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2740,15 +2740,20 @@ err_free: return -1; } +#define DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US (10 * 1000) /* 10 ms */ + static void * dp_netdev_flow_offload_main(void *data OVS_UNUSED) { struct dp_offload_thread_item *offload; struct ovs_list *list; long long int latency_us; + long long int next_rcu; + long long int now; const char *op; int ret; + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; for (;;) { ovs_mutex_lock(&dp_offload_thread.mutex); if (ovs_list_is_empty(&dp_offload_thread.list)) { @@ -2756,6 +2761,7 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) ovs_mutex_cond_wait(&dp_offload_thread.cond, &dp_offload_thread.mutex); ovsrcu_quiesce_end(); + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; } list = ovs_list_pop_front(&dp_offload_thread.list); dp_offload_thread.enqueued_item--; @@ -2779,7 +2785,9 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) OVS_NOT_REACHED(); } - latency_us = time_usec() - offload->timestamp; + now = time_usec(); + + latency_us = now - offload->timestamp; mov_avg_cma_update(&dp_offload_thread.cma, latency_us); mov_avg_ema_update(&dp_offload_thread.ema, latency_us); @@ -2787,7 +2795,12 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) ret == 0 ? "succeed" : "failed", op, UUID_ARGS((struct uuid *) &offload->flow->mega_ufid)); dp_netdev_free_flow_offload(offload); - ovsrcu_quiesce(); + + /* Do RCU synchronization at fixed interval. */ + if (now > next_rcu) { + ovsrcu_quiesce(); + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; + } } return NULL;