From patchwork Wed Feb 10 14:57:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439002 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=u3xCeitZ; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=MwMe6P1O; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbNQM4bDnz9rx6 for ; Thu, 11 Feb 2021 02:06:51 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 39ED18693C; Wed, 10 Feb 2021 15:06:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sKQdEm7F4rDV; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id AFF6E867DA; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 90D4EC0FA7; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18756C013A for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 01F71867DA for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3q7LIIes7FmL for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id 0AB8286776 for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.west.internal (Postfix) with ESMTP id 17424B4C; Wed, 10 Feb 2021 09:57:50 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 10 Feb 2021 09:57:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=VMVjj5Usg0XW1 57jTOeIK07NZ0u1hdz+xRQDfUaRyYw=; b=u3xCeitZmGxivGYJbFmKEKfjEyUI2 ywy0hF6mtVjIa4V3hJIFQMEJCGspdd4emiyguaHDyuZMSB2RgeiLMf1nArm4TcXw b9Jc1N05jwpuq7JOl/927jH0JsMFaqp355/aqmh864aS80eFX3ZSueB32ZHcWHos mKgUp+LeS2W/W4BP1HPqK3h2jtATD7ljLuV8PH7XpyYQ/kZkdsBaq5t7IF7LDKxn tSkq9+Atz0GRrLlshHa+ffmvk6JmICYhnWB+be53St+gBRLBZGBV43e+qX946wy5 bXyJm1hO2So4UOByYvWvsdMYLBcDxsiX7pNr5iDob2MPbd0A1iQyuIdkA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=VMVjj5Usg0XW157jTOeIK07NZ0u1hdz+xRQDfUaRyYw=; b=MwMe6P1O jgTDQf9YWCoXHMw5lcropeCGAo5k1atvLOZn2rMoc5y8kpYYxmB0XM/JQ5I3ZO8M 3Xsx8dePzsr76yJHcZTDJnV9tPY7Rw+hcS4ofhJXahiTcMKWl6iN/zVflIuDGWW4 mTmixeYDB0YWrLPq5yTbjuqWyKz1FhGbf8jHoDoEW7e1I+YSXOThtJtBjeEiizy5 4zgdm1pyq62rd2fBA4hjwnIOyJYYxQgkKOAkI0NLKs76IEtJs2IPZ1Xs7+CXDJt4 OsC+tWPZvEk5r9uhkSbPrgKIGtd5jgkdvBXZHnHybUlZiavRdcQy8rWwOPzAjlTS gUu6MFtBvMJQJg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpefirggvthgrnhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedr nhgvtheqnecuggftrfgrthhtvghrnhephefgveffkeetheetfeeifedvheelfeejfeehve duteejhfekuedtkeeiuedvteehnecukfhppeekiedrvdehgedrudeigedrudejgeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvse huvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id A4FEE24005E; Wed, 10 Feb 2021 09:57:47 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 15:57:19 +0100 Message-Id: <4d3025b0ee9939dec137dc7fd28d0ea577180181.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: elibr@nvidia.com, ameerm@nvidia.com, i.maximets@ovn.org, majd@nvidia.com Subject: [ovs-dev] [PATCH v1 01/23] netdev: Add flow API de-init function X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a new operation for flow API providers to deinitializes when the API is disassociated from a netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-provider.h | 3 +++ lib/netdev-offload.c | 4 ++++ 2 files changed, 7 insertions(+) diff --git a/lib/netdev-offload-provider.h b/lib/netdev-offload-provider.h index cf859d1b4..4815a1bd2 100644 --- a/lib/netdev-offload-provider.h +++ b/lib/netdev-offload-provider.h @@ -90,6 +90,9 @@ struct netdev_flow_api { /* Initializies the netdev flow api. * Return 0 if successful, otherwise returns a positive errno value. */ int (*init_flow_api)(struct netdev *); + + /* Deinitializes the netdev flow api. */ + void (*deinit_flow_api)(struct netdev *); }; int netdev_register_flow_api_provider(const struct netdev_flow_api *); diff --git a/lib/netdev-offload.c b/lib/netdev-offload.c index 6237667c3..3092f8d55 100644 --- a/lib/netdev-offload.c +++ b/lib/netdev-offload.c @@ -320,6 +320,10 @@ netdev_uninit_flow_api(struct netdev *netdev) return; } + if (flow_api->deinit_flow_api) { + flow_api->deinit_flow_api(netdev); + } + ovsrcu_set(&netdev->flow_api, NULL); rfa = netdev_lookup_flow_api(flow_api->type); ovs_refcount_unref(&rfa->refcnt); From patchwork Wed Feb 10 14:57:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439006 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=MylS5HCj; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=JQUV8BR6; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbNQn21pgz9sB4 for ; Thu, 11 Feb 2021 02:07:13 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5CAD06F79C for ; Wed, 10 Feb 2021 15:07:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PZT6T7yjlaxE for ; Wed, 10 Feb 2021 15:07:09 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 90DC96F6DE; Wed, 10 Feb 2021 15:07:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTP id D005F6F62B; Wed, 10 Feb 2021 15:06:46 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 82AAEC1DA9; Wed, 10 Feb 2021 15:06:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4EAF7C013A for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 3E8A986892 for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d--OdaH-wG8a for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 0FE7C86521 for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 26397C89; Wed, 10 Feb 2021 09:57:50 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 09:57:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=0jH5ZRgvVDRqY xCdZw6rWVpGOq9a655iIyHUK3sekDc=; b=MylS5HCjASf4Rw9ixaZlWMXssU/j5 z/icUpw/gfY3kmu1o3v/9t4xsyphD1RPOVfG6kvFllM4TTl5ITtWDXyV43WKFLGr xaeAVV7Vy3mMwWzkG04G3Gl8MzwHeuizixEBCyWCh2CaktrZ5E4LZRBAKqZST4zF R8uyaFTqHZwdH4jk9vsr3qL593rMmOl1AkoLLPVWbRv9P3/sRDFAGvGOQtfQCiN5 J1xIDqEvFHMzQWf/OIEtEtgSoXwZjA+Pepovb3Lz03Tr4WjeD89OuPIJVumnkcwS 6xKPaNJZg/PiTQiHMKJyOSrL9YTTc46M1zHFZPeEuQIJ+bimixRMnFUiw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=0jH5ZRgvVDRqYxCdZw6rWVpGOq9a655iIyHUK3sekDc=; b=JQUV8BR6 cC5G5Q0L4ceuC3sWUsQF63EPEOQ9Mqb7KtCEhm+gV2FNFmODZxCqEKgZRlnMM3sw +1cVcILhMK1kEHkKhTZBUhziEr7MQMFay6iaf7Jix6q9kjhbugKmaUo83EV20CRY n9+47nrkEuIRB9wsr6bJ3RjzuTTeai3gqU9t3xecQsGbh9OGxe5H0rHH6cMC2Cpj pEM79e+0DEUj2hVaYNZglkZ7v7fn61D27+P5D5aBJr9zq9k1L8rSTa7SJxfUAq4Y +IkjgBcr7TrEFzTX1bNmqACckSW4jjTXow6medcHEot3hqipwSR95pXaOtGQtWZ8 UBUSBP7H29xdIg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpefirggvthgrnhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedr nhgvtheqnecuggftrfgrthhtvghrnhephefgveffkeetheetfeeifedvheelfeejfeehve duteejhfekuedtkeeiuedvteehnecukfhppeekiedrvdehgedrudeigedrudejgeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvse huvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 88F0D240065; Wed, 10 Feb 2021 09:57:48 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 15:57:20 +0100 Message-Id: <6c66b5bb17c5be666b555b1ee47968b4e15dff2a.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: elibr@nvidia.com, ameerm@nvidia.com, i.maximets@ovn.org, majd@nvidia.com Subject: [ovs-dev] [PATCH v1 02/23] netdev-offload-dpdk: Use per-netdev offload metadata X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a per-netdev offload data field as part of netdev hw_info structure. Use this field in netdev-offload-dpdk to map offload metadata (ufid to rte_flow). Use flow API deinit ops to destroy the per-netdev metadata when deallocating a netdev. Use RCU primitives to ensure coherency during port deletion. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-dpdk.c | 131 ++++++++++++++++++++++++++++++++------ lib/netdev-offload.h | 2 + 2 files changed, 114 insertions(+), 19 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index f2413f5be..47e56af23 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -26,6 +26,7 @@ #include "netdev-provider.h" #include "openvswitch/match.h" #include "openvswitch/vlog.h" +#include "ovs-rcu.h" #include "packets.h" #include "uuid.h" @@ -52,7 +53,6 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(100, 5); /* * A mapping from ufid to dpdk rte_flow. */ -static struct cmap ufid_to_rte_flow = CMAP_INITIALIZER; struct ufid_to_rte_flow_data { struct cmap_node node; @@ -63,14 +63,81 @@ struct ufid_to_rte_flow_data { struct dpif_flow_stats stats; }; +struct netdev_offload_dpdk_data { + struct cmap ufid_to_rte_flow; +}; + +static int +offload_data_init(struct netdev *netdev) +{ + struct netdev_offload_dpdk_data *data; + + data = xzalloc(sizeof *data); + cmap_init(&data->ufid_to_rte_flow); + + ovsrcu_set(&netdev->hw_info.offload_data, (void *) data); + + return 0; +} + +static void +offload_data_destroy__(struct netdev_offload_dpdk_data *data) +{ + free(data); +} + +static void +offload_data_destroy(struct netdev *netdev) +{ + struct netdev_offload_dpdk_data *data; + struct ufid_to_rte_flow_data *node; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + if (data == NULL) { + return; + } + + if (!cmap_is_empty(&data->ufid_to_rte_flow)) { + VLOG_ERR("Incomplete flush: %s contains rte_flow elements", + netdev_get_name(netdev)); + } + + CMAP_FOR_EACH (node, node, &data->ufid_to_rte_flow) { + ovsrcu_postpone(free, node); + } + + cmap_destroy(&data->ufid_to_rte_flow); + ovsrcu_postpone(offload_data_destroy__, data); + + ovsrcu_set(&netdev->hw_info.offload_data, NULL); +} + +static struct cmap * +offload_data_map(struct netdev *netdev) +{ + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + + return data ? &data->ufid_to_rte_flow : NULL; +} + /* Find rte_flow with @ufid. */ static struct ufid_to_rte_flow_data * -ufid_to_rte_flow_data_find(const ovs_u128 *ufid, bool warn) +ufid_to_rte_flow_data_find(struct netdev *netdev, + const ovs_u128 *ufid, bool warn) { size_t hash = hash_bytes(ufid, sizeof *ufid, 0); struct ufid_to_rte_flow_data *data; + struct cmap *map = offload_data_map(netdev); + + if (!map) { + return NULL; + } - CMAP_FOR_EACH_WITH_HASH (data, node, hash, &ufid_to_rte_flow) { + CMAP_FOR_EACH_WITH_HASH (data, node, hash, map) { if (ovs_u128_equals(*ufid, data->ufid)) { return data; } @@ -85,12 +152,19 @@ ufid_to_rte_flow_data_find(const ovs_u128 *ufid, bool warn) } static inline struct ufid_to_rte_flow_data * -ufid_to_rte_flow_associate(const ovs_u128 *ufid, struct netdev *netdev, +ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, struct rte_flow *rte_flow, bool actions_offloaded) { size_t hash = hash_bytes(ufid, sizeof *ufid, 0); - struct ufid_to_rte_flow_data *data = xzalloc(sizeof *data); + struct cmap *map = offload_data_map(netdev); struct ufid_to_rte_flow_data *data_prev; + struct ufid_to_rte_flow_data *data; + + if (!map) { + return NULL; + } + + data = xzalloc(sizeof *data); /* * We should not simply overwrite an existing rte flow. @@ -98,7 +172,7 @@ ufid_to_rte_flow_associate(const ovs_u128 *ufid, struct netdev *netdev, * Thus, if following assert triggers, something is wrong: * the rte_flow is not destroyed. */ - data_prev = ufid_to_rte_flow_data_find(ufid, false); + data_prev = ufid_to_rte_flow_data_find(netdev, ufid, false); if (data_prev) { ovs_assert(data_prev->rte_flow == NULL); } @@ -108,8 +182,7 @@ ufid_to_rte_flow_associate(const ovs_u128 *ufid, struct netdev *netdev, data->rte_flow = rte_flow; data->actions_offloaded = actions_offloaded; - cmap_insert(&ufid_to_rte_flow, - CONST_CAST(struct cmap_node *, &data->node), hash); + cmap_insert(map, CONST_CAST(struct cmap_node *, &data->node), hash); return data; } @@ -117,9 +190,13 @@ static inline void ufid_to_rte_flow_disassociate(struct ufid_to_rte_flow_data *data) { size_t hash = hash_bytes(&data->ufid, sizeof data->ufid, 0); + struct cmap *map = offload_data_map(data->netdev); + + if (!map) { + return; + } - cmap_remove(&ufid_to_rte_flow, - CONST_CAST(struct cmap_node *, &data->node), hash); + cmap_remove(map, CONST_CAST(struct cmap_node *, &data->node), hash); netdev_close(data->netdev); ovsrcu_postpone(free, data); } @@ -1421,7 +1498,7 @@ netdev_offload_dpdk_add_flow(struct netdev *netdev, if (!flow) { goto out; } - flows_data = ufid_to_rte_flow_associate(ufid, netdev, flow, + flows_data = ufid_to_rte_flow_associate(netdev, ufid, flow, actions_offloaded); VLOG_DBG("%s: installed flow %p by ufid "UUID_FMT, netdev_get_name(netdev), flow, UUID_ARGS((struct uuid *)ufid)); @@ -1478,7 +1555,7 @@ netdev_offload_dpdk_flow_put(struct netdev *netdev, struct match *match, * Here destroy the old rte flow first before adding a new one. * Keep the stats for the newly created rule. */ - rte_flow_data = ufid_to_rte_flow_data_find(ufid, false); + rte_flow_data = ufid_to_rte_flow_data_find(netdev, ufid, false); if (rte_flow_data && rte_flow_data->rte_flow) { old_stats = rte_flow_data->stats; modification = true; @@ -1509,7 +1586,7 @@ netdev_offload_dpdk_flow_del(struct netdev *netdev OVS_UNUSED, { struct ufid_to_rte_flow_data *rte_flow_data; - rte_flow_data = ufid_to_rte_flow_data_find(ufid, true); + rte_flow_data = ufid_to_rte_flow_data_find(netdev, ufid, true); if (!rte_flow_data || !rte_flow_data->rte_flow) { return -1; } @@ -1523,7 +1600,21 @@ netdev_offload_dpdk_flow_del(struct netdev *netdev OVS_UNUSED, static int netdev_offload_dpdk_init_flow_api(struct netdev *netdev) { - return netdev_dpdk_flow_api_supported(netdev) ? 0 : EOPNOTSUPP; + int ret = EOPNOTSUPP; + + if (netdev_dpdk_flow_api_supported(netdev)) { + ret = offload_data_init(netdev); + } + + return ret; +} + +static void +netdev_offload_dpdk_deinit_flow_api(struct netdev *netdev) +{ + if (netdev_dpdk_flow_api_supported(netdev)) { + offload_data_destroy(netdev); + } } static int @@ -1540,7 +1631,7 @@ netdev_offload_dpdk_flow_get(struct netdev *netdev, struct rte_flow_error error; int ret = 0; - rte_flow_data = ufid_to_rte_flow_data_find(ufid, false); + rte_flow_data = ufid_to_rte_flow_data_find(netdev, ufid, false); if (!rte_flow_data || !rte_flow_data->rte_flow) { ret = -1; goto out; @@ -1575,13 +1666,14 @@ out: static int netdev_offload_dpdk_flow_flush(struct netdev *netdev) { + struct cmap *map = offload_data_map(netdev); struct ufid_to_rte_flow_data *data; - CMAP_FOR_EACH (data, node, &ufid_to_rte_flow) { - if (data->netdev != netdev) { - continue; - } + if (!map) { + return -1; + } + CMAP_FOR_EACH (data, node, map) { netdev_offload_dpdk_flow_destroy(data); } @@ -1593,6 +1685,7 @@ const struct netdev_flow_api netdev_offload_dpdk = { .flow_put = netdev_offload_dpdk_flow_put, .flow_del = netdev_offload_dpdk_flow_del, .init_flow_api = netdev_offload_dpdk_init_flow_api, + .deinit_flow_api = netdev_offload_dpdk_deinit_flow_api, .flow_get = netdev_offload_dpdk_flow_get, .flow_flush = netdev_offload_dpdk_flow_flush, }; diff --git a/lib/netdev-offload.h b/lib/netdev-offload.h index 18b48790f..d820e23ed 100644 --- a/lib/netdev-offload.h +++ b/lib/netdev-offload.h @@ -20,6 +20,7 @@ #include "openvswitch/netdev.h" #include "openvswitch/types.h" +#include "ovs-rcu.h" #include "packets.h" #include "flow.h" @@ -45,6 +46,7 @@ struct netdev_hw_info { bool oor; /* Out of Offload Resources ? */ int offload_count; /* Pending (non-offloaded) flow count */ int pending_count; /* Offloaded flow count */ + OVSRCU_TYPE(void *) offload_data; /* Offload metadata. */ }; enum hw_info_type { From patchwork Wed Feb 10 14:57:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439005 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=BU8+qddF; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=GzAq0zPF; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbNQS2Myxz9sCD for ; Thu, 11 Feb 2021 02:06:56 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CDF9486A90; Wed, 10 Feb 2021 15:06:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nOjXvpmsZoUh; Wed, 10 Feb 2021 15:06:48 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 74D2686A0D; Wed, 10 Feb 2021 15:06:48 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5290AC0FA7; Wed, 10 Feb 2021 15:06:48 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 60042C013A for ; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4D6F76F568 for ; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5HotnHLSWJkY for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 7A4266F651; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2F37C6F604 for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailnew.west.internal (Postfix) with ESMTP id E7A64CBA; Wed, 10 Feb 2021 09:57:50 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Wed, 10 Feb 2021 09:57:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=dh+ZzcZg3wPmY biCYDe3uYEPVfsrTyiDTP1u1Mekb3k=; b=BU8+qddFgrwpDPH4l0LuA3dNr60G6 lBDgF3ia4bXV4WbdBFPbvVXMW+q/SxMG5OZGVdan2Bjnx9AIahlcbveKQ3n9mKcl II6JS+Ct+2blDOCnzhALdfqBv/uXFlVOqZO3PgFbU1AcFeWhE+/qqoGvmT8ki9Sc DTV4iqLkOi9EtrIJQ5Ju2JSHh2Xq92aVG+5wJ23IfweK/g+tQw8kOOKeklYywJh9 xYELv77G0cwLTtujbtaX51VjNYQb9PzU29vay2CjxeJn2exMm2Aq9ICPwQNfi05Y puRkpORCyybh8ydX50oXP1t9qdo1QdHsmh+tO5fh/rbJLWjv8lxMiGF5A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=dh+ZzcZg3wPmYbiCYDe3uYEPVfsrTyiDTP1u1Mekb3k=; b=GzAq0zPF m624im4zxrGlg7HZNIjBpRgfqoOcJAFOuOHWiC4UpRhBcYOyiGY7rw2mauhC1LE+ oYoERT/eoqRa/5y2Jldw6+LDU6DXIFXiUXocoVSNwHRwaGgTcst771tn8FFHpzsa UAm2GrQBOx/VdRaXjNrWeiXQbJH+wSuHj/rZm6evQvSKYODJiUtaTLq1xUvCfsR2 lTSg1kjxfh56ZFUaISQp7gXgnIJWYWjKLoKrmWy+lQig9Q3tsuhDY7myFLHXixqv cu2T7FxT/BYcibmlISh3WRBEI+tEH28QfyNtUtylFd2ipQsxt0etur7CVOG86ZsE HvNak5dHUj2sDw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpefirggvthgrnhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedr nhgvtheqnecuggftrfgrthhtvghrnhephefgveffkeetheetfeeifedvheelfeejfeehve duteejhfekuedtkeeiuedvteehnecukfhppeekiedrvdehgedrudeigedrudejgeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvse huvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 6C29F240064; Wed, 10 Feb 2021 09:57:49 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 15:57:21 +0100 Message-Id: <0aeb4afbe2ccabcbf1b5fffc645842ba210add08.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: elibr@nvidia.com, ameerm@nvidia.com, i.maximets@ovn.org, majd@nvidia.com Subject: [ovs-dev] [PATCH v1 03/23] netdev-offload-dpdk: Implement hw-offload statistics read X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In the DPDK offload provider, keep track of inserted rte_flow and report it when queried. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-dpdk.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 47e56af23..1d60c5a81 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -65,6 +65,7 @@ struct ufid_to_rte_flow_data { struct netdev_offload_dpdk_data { struct cmap ufid_to_rte_flow; + uint64_t rte_flow_counter; }; static int @@ -644,6 +645,12 @@ netdev_offload_dpdk_flow_create(struct netdev *netdev, flow = netdev_dpdk_rte_flow_create(netdev, attr, items, actions, error); if (flow) { + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + data->rte_flow_counter++; + if (!VLOG_DROP_DBG(&rl)) { dump_flow(&s, &s_extra, attr, items, actions); extra_str = ds_cstr(&s_extra); @@ -1524,6 +1531,12 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) ret = netdev_dpdk_rte_flow_destroy(netdev, rte_flow, &error); if (ret == 0) { + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + data->rte_flow_counter--; + ufid_to_rte_flow_disassociate(rte_flow_data); VLOG_DBG_RL(&rl, "%s: rte_flow 0x%"PRIxPTR " flow destroy %d ufid " UUID_FMT, @@ -1680,6 +1693,23 @@ netdev_offload_dpdk_flow_flush(struct netdev *netdev) return 0; } +static int +netdev_offload_dpdk_get_n_flows(struct netdev *netdev, + uint64_t *n_flows) +{ + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + if (!data) { + return -1; + } + + *n_flows = data->rte_flow_counter; + + return 0; +} + const struct netdev_flow_api netdev_offload_dpdk = { .type = "dpdk_flow_api", .flow_put = netdev_offload_dpdk_flow_put, @@ -1688,4 +1718,5 @@ const struct netdev_flow_api netdev_offload_dpdk = { .deinit_flow_api = netdev_offload_dpdk_deinit_flow_api, .flow_get = netdev_offload_dpdk_flow_get, .flow_flush = netdev_offload_dpdk_flow_flush, + .flow_get_n_flows = netdev_offload_dpdk_get_n_flows, }; From patchwork Wed Feb 10 14:57:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439004 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=qKQAy54u; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=RJH32xQg; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbNQR0TzLz9sBJ for ; Thu, 11 Feb 2021 02:06:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 8736687382; Wed, 10 Feb 2021 15:06:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KU0tgoKhb1IO; Wed, 10 Feb 2021 15:06:50 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id F25278736A; Wed, 10 Feb 2021 15:06:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E187FC1E70; Wed, 10 Feb 2021 15:06:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id E5D33C0891 for ; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C20446F568 for ; Wed, 10 Feb 2021 15:06:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TeYcMhFJ2pfX for ; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id AEBF06F604; Wed, 10 Feb 2021 15:06:44 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2F3F86F62B for ; Wed, 10 Feb 2021 15:06:43 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.west.internal (Postfix) with ESMTP id A02B47C7; Wed, 10 Feb 2021 09:57:51 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Wed, 10 Feb 2021 09:57:52 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=vCtK8dz5Co99w 7BtzVSaqUa8G33jrCGZqh46nOSg2Z4=; b=qKQAy54uHZqcWC3D/UsIuTGe06yqg cK8thbkKux2x1ptGRuc+Jqg/jE7Q2YYEawylo8sSxNu/+RzcNrB7dBfpO10lgC+q ZMRDQO9TgqwznazA6EyizgnHQNxSdt/6UyguzKpZFm2gR28iIc8Yd0h6xPlwibqu qdF/GXqdih9PTfMf/GbeAJaxMJ738+oLuWTgRwdh+GMAVYDt5tDPadO5YHMlNabF 3EJWfbaHJ0tXfaxgKPAcu4bppMdu2ZTR/86GcVYAw0oL3HL0iLpcc6iOryl4UpIQ E/AgTcwBoEXCXHdVJSprabh+qOt0sQhUEJB0ekU1+uoKtSmSLdwr8XYLA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=vCtK8dz5Co99w7BtzVSaqUa8G33jrCGZqh46nOSg2Z4=; b=RJH32xQg vjrL1k0vRDEZSLlMs6XIhjSa77E9PAJZ+HOFnfOVo3HpCC4vVTkfX83ElUbYWOcQ +9gqXmk5VYxhLmUc99jrVWjC1pUdleEG1rZS3cGIRsQqdOgC+xGbQdhlL0gMBcXS Xm84ePkdwU2cNqZuNd0p3brbwUfr17CWUBIqnik2R201ash6kN+4G2VSjOt3sFpc ndh2CRgrlNEwyLOQuF2rXT6c3aZePvjCSCSSPYGkx8ly2ny5+joUHVPehBYFzi6D SjiTO3c7y/vLTbmGphz5NR3/WJiKbX3fq9j/ZLQy8Z0ZvwtfRPhQJGeZb2vXMri8 clJdI0ZP8MMYIg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpefirggvthgrnhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedr nhgvtheqnecuggftrfgrthhtvghrnhephefgveffkeetheetfeeifedvheelfeejfeehve duteejhfekuedtkeeiuedvteehnecukfhppeekiedrvdehgedrudeigedrudejgeenucev lhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvse huvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 4F0DB240057; Wed, 10 Feb 2021 09:57:50 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 15:57:22 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: elibr@nvidia.com, ameerm@nvidia.com, i.maximets@ovn.org, majd@nvidia.com Subject: [ovs-dev] [PATCH v1 04/23] dpctl: Add function to read hardware offload statistics X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Expose a function to query datapath offload statistics. This function is separate from the current one in netdev-offload as it exposes more detailed statistics from the datapath, instead of only from the netdev-offload provider. Each datapath is meant to use the custom counters as it sees fit for its handling of hardware offloads. Call the new API from dpctl. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpctl.c | 36 ++++++++++++++++++++++++++++++++++++ lib/dpif-netdev.c | 1 + lib/dpif-netlink.c | 1 + lib/dpif-provider.h | 7 +++++++ lib/dpif.c | 8 ++++++++ lib/dpif.h | 9 +++++++++ 6 files changed, 62 insertions(+) diff --git a/lib/dpctl.c b/lib/dpctl.c index ef8ae7402..6ff73e2d9 100644 --- a/lib/dpctl.c +++ b/lib/dpctl.c @@ -1541,6 +1541,40 @@ dpctl_del_flows(int argc, const char *argv[], struct dpctl_params *dpctl_p) return error; } +static int +dpctl_offload_stats_show(int argc, const char *argv[], + struct dpctl_params *dpctl_p) +{ + struct netdev_custom_stats stats; + struct dpif *dpif; + int error; + size_t i; + + error = opt_dpif_open(argc, argv, dpctl_p, 2, &dpif); + if (error) { + return error; + } + + memset(&stats, 0, sizeof(stats)); + error = dpif_offload_stats_get(dpif, &stats); + if (error) { + dpctl_error(dpctl_p, error, "retrieving offload statistics"); + goto close_dpif; + } + + dpctl_print(dpctl_p, "HW Offload stats:\n"); + for (i = 0; i < stats.size; i++) { + dpctl_print(dpctl_p, " %s: %6" PRIu64 "\n", + stats.counters[i].name, stats.counters[i].value); + } + + netdev_free_custom_stats_counters(&stats); + +close_dpif: + dpif_close(dpif); + return error; +} + static int dpctl_help(int argc OVS_UNUSED, const char *argv[] OVS_UNUSED, struct dpctl_params *dpctl_p) @@ -2697,6 +2731,8 @@ static const struct dpctl_command all_commands[] = { { "add-flows", "[dp] file", 1, 2, dpctl_process_flows, DP_RW }, { "mod-flows", "[dp] file", 1, 2, dpctl_process_flows, DP_RW }, { "del-flows", "[dp] [file]", 0, 2, dpctl_del_flows, DP_RW }, + { "offload-stats-show", "[dp]", + 0, 1, dpctl_offload_stats_show, DP_RO }, { "dump-conntrack", "[-m] [-s] [dp] [zone=N]", 0, 4, dpctl_dump_conntrack, DP_RO }, { "flush-conntrack", "[dp] [zone=N] [ct-tuple]", 0, 3, diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index e3fd0a07f..ef347d3b9 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -8458,6 +8458,7 @@ const struct dpif_class dpif_netdev_class = { dpif_netdev_flow_dump_thread_destroy, dpif_netdev_flow_dump_next, dpif_netdev_operate, + NULL, /* offload_stats_get */ NULL, /* recv_set */ NULL, /* handlers_set */ dpif_netdev_set_config, diff --git a/lib/dpif-netlink.c b/lib/dpif-netlink.c index ceb56c685..924b0bd12 100644 --- a/lib/dpif-netlink.c +++ b/lib/dpif-netlink.c @@ -3972,6 +3972,7 @@ const struct dpif_class dpif_netlink_class = { dpif_netlink_flow_dump_thread_destroy, dpif_netlink_flow_dump_next, dpif_netlink_operate, + NULL, /* offload_stats_get */ dpif_netlink_recv_set, dpif_netlink_handlers_set, NULL, /* set_config */ diff --git a/lib/dpif-provider.h b/lib/dpif-provider.h index b817fceac..36dfa8e71 100644 --- a/lib/dpif-provider.h +++ b/lib/dpif-provider.h @@ -330,6 +330,13 @@ struct dpif_class { void (*operate)(struct dpif *dpif, struct dpif_op **ops, size_t n_ops, enum dpif_offload_type offload_type); + /* Get hardware-offloads activity counters from a dataplane. + * Those counters are not offload statistics (which are accessible through + * netdev statistics), but a status of hardware offload management: + * how many offloads are currently waiting, inserted, etc. */ + int (*offload_stats_get)(struct dpif *dpif, + struct netdev_custom_stats *stats); + /* Enables or disables receiving packets with dpif_recv() for 'dpif'. * Turning packet receive off and then back on is allowed to change Netlink * PID assignments (see ->port_get_pid()). The client is responsible for diff --git a/lib/dpif.c b/lib/dpif.c index 56d0b4a65..30fc2db51 100644 --- a/lib/dpif.c +++ b/lib/dpif.c @@ -1426,6 +1426,14 @@ dpif_operate(struct dpif *dpif, struct dpif_op **ops, size_t n_ops, } } +int dpif_offload_stats_get(struct dpif *dpif, + struct netdev_custom_stats *stats) +{ + return (dpif->dpif_class->offload_stats_get + ? dpif->dpif_class->offload_stats_get(dpif, stats) + : EOPNOTSUPP); +} + /* Returns a string that represents 'type', for use in log messages. */ const char * dpif_upcall_type_to_string(enum dpif_upcall_type type) diff --git a/lib/dpif.h b/lib/dpif.h index ecda896c7..89af01a5b 100644 --- a/lib/dpif.h +++ b/lib/dpif.h @@ -787,6 +787,15 @@ struct dpif_op { void dpif_operate(struct dpif *, struct dpif_op **ops, size_t n_ops, enum dpif_offload_type); + +/* Queries the datapath for hardware offloads stats. + * + * Statistics are written in 'stats' following the 'netdev_custom_stats' + * format. They are allocated on the heap and must be freed by the caller, + * using 'netdev_free_custom_stats_counters'. + */ +int dpif_offload_stats_get(struct dpif *dpif, + struct netdev_custom_stats *stats); /* Upcalls. */ From patchwork Wed Feb 10 15:10:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439027 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=QMArTn6D; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=EbBTmFf/; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbNZ7656Nz9sXV for ; Thu, 11 Feb 2021 02:13:32 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 5EE1986EC1; Wed, 10 Feb 2021 15:12:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yxw0G-qlh+mQ; Wed, 10 Feb 2021 15:12:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 369358698C; Wed, 10 Feb 2021 15:11:09 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E26E6C1ECB; Wed, 10 Feb 2021 15:11:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 38CD1C1EC8 for ; Wed, 10 Feb 2021 15:11:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 2DC8282558 for ; Wed, 10 Feb 2021 15:11:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Bu35qzTUAK26 for ; Wed, 10 Feb 2021 15:11:03 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by hemlock.osuosl.org (Postfix) with ESMTPS id BE819873AC for ; Wed, 10 Feb 2021 15:11:01 +0000 (UTC) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailnew.west.internal (Postfix) with ESMTP id C6B9AB64; Wed, 10 Feb 2021 10:11:00 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Wed, 10 Feb 2021 10:11:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=0eoaOtGxAoEhD b/DHDy3Mp2+QBow2eIAIkCaapBLa54=; b=QMArTn6DqfT0f9CI9wXeX4V3WC8fL WOC8DrDvUcW7E0GDSBUkLHrI2qx9YzBUfqDoKlBaU2jjnsqfKWtxCOf1JziRkxDL drsQyQC7dU5gkJArb8IoS+xkDbyPoYDcHqNfkdyCtCXYtzB2MiPAUwId0iRk4C0S c0EDSRkFZJdVLahuWgLpBPOoSIUPyCYSlViEYmsjrpLC9pR+LjuDqv8ZqnDHS8yW iIs4iBVDl3NbLDYSMigOne32DzOfMmjiPeOZjsQ85LUGfaZVrQBFAOIeFRp1rRdP gw90NykAWRLXPhzMHUi/h02gr9vUIN2ZoZZGZABews7QyicUeRRxgUQ6w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=0eoaOtGxAoEhDb/DHDy3Mp2+QBow2eIAIkCaapBLa54=; b=EbBTmFf/ 40iae7yhWKpuTnJj1EG7x7subJ4ZiHxA/grgoMv7ls5Q/l470fKCSKFXJwbDlHbI Vj3Z53zELKiKA1IYJ09hhQvM8UHGqVXbuDPG+asaTLw6aabxdW2LoFyewUFK+lld kQ+A8xxvTgZmHK196Y21xozADO7l5enuSeBhieOwK9KDaczc5w6mWqqZbhcVZbxv vYnzkEdFiT570mLw3gsEPzLrmJdvor4imE8MxgXoJ9CKHFQFIj7DpLGrHG3HVULK zYyPdZAc1seB9XjZR/mtMI6/njZWU1fdyHz3j4SmiqJF1VlhSCskQWdSl/iTLMIn OX2nCLBh1jD6Zw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkffojghfggfgsedtkeertd ertddtnecuhfhrohhmpefirggvthgrnhcutfhivhgvthcuoehgrhhivhgvsehuvdehiedr nhgvtheqnecuggftrfgrthhtvghrnhephefgveffkeetheetfeeifedvheelfeejfeehve duteejhfekuedtkeeiuedvteehnecukfhppeekiedrvdehgedrudeigedrudejgeenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvse huvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 6D3931080069; Wed, 10 Feb 2021 10:10:58 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:10:36 +0100 Message-Id: <87715afbe1fd682d5c8cb8cb624b814616e61cf1.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: elibr@nvidia.com, ameerm@nvidia.com, i.maximets@ovn.org, majd@nvidia.com Subject: [ovs-dev] [PATCH v1 05/23] dpif-netdev: Rename offload thread structure X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The offload management in userspace is done through a separate thread. The naming of the structure holding the objects used for synchronization with the dataplane is generic and nondescript. Clarify the object function by renaming it. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 52 +++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index ef347d3b9..7a0e8f24e 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -423,7 +423,7 @@ enum { DP_NETDEV_FLOW_OFFLOAD_OP_DEL, }; -struct dp_flow_offload_item { +struct dp_offload_thread_item { struct dp_netdev_pmd_thread *pmd; struct dp_netdev_flow *flow; int op; @@ -434,15 +434,15 @@ struct dp_flow_offload_item { struct ovs_list node; }; -struct dp_flow_offload { +struct dp_offload_thread { struct ovs_mutex mutex; struct ovs_list list; pthread_cond_t cond; }; -static struct dp_flow_offload dp_flow_offload = { +static struct dp_offload_thread dp_offload_thread = { .mutex = OVS_MUTEX_INITIALIZER, - .list = OVS_LIST_INITIALIZER(&dp_flow_offload.list), + .list = OVS_LIST_INITIALIZER(&dp_offload_thread.list), }; static struct ovsthread_once offload_thread_once @@ -2598,12 +2598,12 @@ mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd, return NULL; } -static struct dp_flow_offload_item * +static struct dp_offload_thread_item * dp_netdev_alloc_flow_offload(struct dp_netdev_pmd_thread *pmd, struct dp_netdev_flow *flow, int op) { - struct dp_flow_offload_item *offload; + struct dp_offload_thread_item *offload; offload = xzalloc(sizeof(*offload)); offload->pmd = pmd; @@ -2617,7 +2617,7 @@ dp_netdev_alloc_flow_offload(struct dp_netdev_pmd_thread *pmd, } static void -dp_netdev_free_flow_offload(struct dp_flow_offload_item *offload) +dp_netdev_free_flow_offload(struct dp_offload_thread_item *offload) { dp_netdev_pmd_unref(offload->pmd); dp_netdev_flow_unref(offload->flow); @@ -2627,16 +2627,16 @@ dp_netdev_free_flow_offload(struct dp_flow_offload_item *offload) } static void -dp_netdev_append_flow_offload(struct dp_flow_offload_item *offload) +dp_netdev_append_flow_offload(struct dp_offload_thread_item *offload) { - ovs_mutex_lock(&dp_flow_offload.mutex); - ovs_list_push_back(&dp_flow_offload.list, &offload->node); - xpthread_cond_signal(&dp_flow_offload.cond); - ovs_mutex_unlock(&dp_flow_offload.mutex); + ovs_mutex_lock(&dp_offload_thread.mutex); + ovs_list_push_back(&dp_offload_thread.list, &offload->node); + xpthread_cond_signal(&dp_offload_thread.cond); + ovs_mutex_unlock(&dp_offload_thread.mutex); } static int -dp_netdev_flow_offload_del(struct dp_flow_offload_item *offload) +dp_netdev_flow_offload_del(struct dp_offload_thread_item *offload) { return mark_to_flow_disassociate(offload->pmd, offload->flow); } @@ -2653,7 +2653,7 @@ dp_netdev_flow_offload_del(struct dp_flow_offload_item *offload) * valid, thus only item 2 needed. */ static int -dp_netdev_flow_offload_put(struct dp_flow_offload_item *offload) +dp_netdev_flow_offload_put(struct dp_offload_thread_item *offload) { struct dp_netdev_pmd_thread *pmd = offload->pmd; struct dp_netdev_flow *flow = offload->flow; @@ -2733,22 +2733,22 @@ err_free: static void * dp_netdev_flow_offload_main(void *data OVS_UNUSED) { - struct dp_flow_offload_item *offload; + struct dp_offload_thread_item *offload; struct ovs_list *list; const char *op; int ret; for (;;) { - ovs_mutex_lock(&dp_flow_offload.mutex); - if (ovs_list_is_empty(&dp_flow_offload.list)) { + ovs_mutex_lock(&dp_offload_thread.mutex); + if (ovs_list_is_empty(&dp_offload_thread.list)) { ovsrcu_quiesce_start(); - ovs_mutex_cond_wait(&dp_flow_offload.cond, - &dp_flow_offload.mutex); + ovs_mutex_cond_wait(&dp_offload_thread.cond, + &dp_offload_thread.mutex); ovsrcu_quiesce_end(); } - list = ovs_list_pop_front(&dp_flow_offload.list); - offload = CONTAINER_OF(list, struct dp_flow_offload_item, node); - ovs_mutex_unlock(&dp_flow_offload.mutex); + list = ovs_list_pop_front(&dp_offload_thread.list); + offload = CONTAINER_OF(list, struct dp_offload_thread_item, node); + ovs_mutex_unlock(&dp_offload_thread.mutex); switch (offload->op) { case DP_NETDEV_FLOW_OFFLOAD_OP_ADD: @@ -2781,10 +2781,10 @@ static void queue_netdev_flow_del(struct dp_netdev_pmd_thread *pmd, struct dp_netdev_flow *flow) { - struct dp_flow_offload_item *offload; + struct dp_offload_thread_item *offload; if (ovsthread_once_start(&offload_thread_once)) { - xpthread_cond_init(&dp_flow_offload.cond, NULL); + xpthread_cond_init(&dp_offload_thread.cond, NULL); ovs_thread_create("dp_netdev_flow_offload", dp_netdev_flow_offload_main, NULL); ovsthread_once_done(&offload_thread_once); @@ -2800,7 +2800,7 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, struct dp_netdev_flow *flow, struct match *match, const struct nlattr *actions, size_t actions_len) { - struct dp_flow_offload_item *offload; + struct dp_offload_thread_item *offload; int op; if (!netdev_is_flow_api_enabled()) { @@ -2808,7 +2808,7 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, } if (ovsthread_once_start(&offload_thread_once)) { - xpthread_cond_init(&dp_flow_offload.cond, NULL); + xpthread_cond_init(&dp_offload_thread.cond, NULL); ovs_thread_create("dp_netdev_flow_offload", dp_netdev_flow_offload_main, NULL); ovsthread_once_done(&offload_thread_once); From patchwork Wed Feb 10 15:33:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439068 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=IzB+ld97; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=aLZzeujt; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2G2GgQz9sB4 for ; Thu, 11 Feb 2021 02:34:30 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CFE7C86CAF; Wed, 10 Feb 2021 15:34:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZgY0TqjKQiYQ; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 890B386C58; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 537D5C1E6F; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 738E5C013A for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 61D6A866A3 for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 26sxNXQPE6Xf for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id 1E48A861CF for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id B5CC51C7; Wed, 10 Feb 2021 10:34:14 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=FrCF66J+vLOKI tAWpv6bRpjfOfAYb3O/d7y2oqREgms=; b=IzB+ld975bs/mNfyBRSJLwWxKiJTa xWT/IC5u9OcrasQAyt9fqAs5EyNiiKzEB4LXqjn7PEBMUZpcMnA0s8bGqImJfUpe uv5NjwCh/Ah0KtB7AYXsX6PrTuM96DbUSq8NNKY+Y5kry+lc1p9q1i0TyUlPT9Yd dTMU+h+IQZU9Ogt4rKiY/aLRyvbiicaiKZeyi9EwyixJSoVZkyMxGyUuQLhP5z7s ypwTLveu54lELEj8y1+RBuhMLS7Rsa3gTmzZ50RInpf8r1Z4M10pDPXI/Nr6BnuX H1XEMmhT2uTAIN+dbHvZgWbyPybyECFKQ6OkIfHS5csCjVBmyIUuvAxXg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=FrCF66J+vLOKItAWpv6bRpjfOfAYb3O/d7y2oqREgms=; b=aLZzeujt CtKZ3l9GFS4I0lcL1PvbNfG87H/Sts9ocoPlY26sqiQYrmXH+Ba9Sz7D68b2jLOd kJnD7m6AwoF0xnppuOu0b4zHHXLUITWSlJasIi6qk0/CBWfAueP88ioijJ4ElH25 ifX9Q34kd3+EngN9vbjJ4orpDThsvR6XeMQ565WdK457waBn26JFLRr1oNgsgu6q Ee01FBaVdRC0Q67VZTKlmFO0h/Jm6NJZTxtei4NQnKD5rmvKONm1rGx2YWYorpr/ 3Nj1ak1NSZ5FdRq3Y2MMEpkSEzmAuNhn20MbKk/Kdl2YrbySfex3AnYo0pEMr/1z mUISRvXvXqrcOg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe elgefftddtfedtvefhieekgeevueegheelledtudeigedvvdevleegueegkefhgfenucff ohhmrghinheprghprggthhgvrdhorhhgpdifihhkihhpvgguihgrrdhorhhgnecukfhppe ekiedrvdehgedrudeigedrudejgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgr mhepmhgrihhlfhhrohhmpehgrhhivhgvsehuvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 09E84240057; Wed, 10 Feb 2021 10:34:11 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:52 +0100 Message-Id: <2812000f3d5627a00b4c0f2a0ee57d3079a23947.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 06/23] mov-avg: Add a moving average helper structure X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a new module offering a helper to compute the Cumulative Moving Average (CMA) and the Exponential Moving Average (EMA) of a series of values. Use the new helpers to add latency metrics in dpif-netdev. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 1 + lib/mov-avg.h | 171 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 172 insertions(+) create mode 100644 lib/mov-avg.h diff --git a/lib/automake.mk b/lib/automake.mk index 39afbff9d..e072bed71 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -166,6 +166,7 @@ lib_libopenvswitch_la_SOURCES = \ lib/memory.c \ lib/memory.h \ lib/meta-flow.c \ + lib/mov-avg.h \ lib/multipath.c \ lib/multipath.h \ lib/namemap.c \ diff --git a/lib/mov-avg.h b/lib/mov-avg.h new file mode 100644 index 000000000..4a7e62c18 --- /dev/null +++ b/lib/mov-avg.h @@ -0,0 +1,171 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef _MOV_AVG_H +#define _MOV_AVG_H 1 + +#include + +/* Moving average helpers. */ + +/* Cumulative Moving Average. + * + * Computes the arithmetic mean over a whole series of value. + * Online equivalent of sum(V) / len(V). + * + * As all values have equal weight, this average will + * be slow to show recent changes in the series. + * + */ + +struct mov_avg_cma { + unsigned long long int count; + double mean; + double sum_dsquared; +}; + +#define MOV_AVG_CMA_INITIALIZER \ + { .count = 0, .mean = .0, .sum_dsquared = .0 } + +static inline void +mov_avg_cma_init(struct mov_avg_cma *cma) +{ + *cma = (struct mov_avg_cma) MOV_AVG_CMA_INITIALIZER; +} + +static inline void +mov_avg_cma_update(struct mov_avg_cma *cma, double new_val) +{ + double new_mean; + + cma->count++; + new_mean = cma->mean + (new_val - cma->mean) / cma->count; + + cma->sum_dsquared += (new_val - new_mean) * (new_val - cma->mean); + cma->mean = new_mean; +} + +static inline double +mov_avg_cma(struct mov_avg_cma *cma) +{ + return cma->mean; +} + +static inline double +mov_avg_cma_std_dev(struct mov_avg_cma *cma) +{ + double variance = 0.0; + + if (cma->count > 1) { + variance = cma->sum_dsquared / (cma->count - 1); + } + + return sqrt(variance); +} + +/* Exponential Moving Average. + * + * Each value in the series has an exponentially decreasing weight, + * the older they get the less weight they have. + * + * The smoothing factor 'alpha' must be within 0 < alpha < 1. + * The closer this factor to zero, the more equal the weight between + * recent and older values. As it approaches one, the more recent values + * will have more weight. + * + * The EMA can be thought of as an estimator for the next value when measures + * are dependent. In this case, it can make sense to consider the mean square + * error of the prediction. An 'alpha' minimizing this error would be the + * better choice to improve the estimation. + * + * A common way to choose 'alpha' is to use the following formula: + * + * a = 2 / (N + 1) + * + * With this 'alpha', the EMA will have the same 'center of mass' as an + * equivalent N-values Simple Moving Average. + * + * When using this factor, the N last values of the EMA will have a sum weight + * converging toward 0.8647, meaning that those values will account for 86% of + * the average[1]. + * + * [1] https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average + */ + +struct mov_avg_ema { + double alpha; /* 'Smoothing' factor. */ + double mean; + double variance; + bool initialized; +}; + +/* Choose alpha explicitly. */ +#define MOV_AVG_EMA_INITIALIZER_ALPHA(a) { \ + .initialized = false, \ + .alpha = (a), .variance = 0.0, .mean = 0.0 \ +} + +/* Choose alpha to consider 'N' past periods as 86% of the EMA. */ +#define MOV_AVG_EMA_INITIALIZER(n_elem) \ + MOV_AVG_EMA_INITIALIZER_ALPHA(2.0 / ((double)(n_elem) + 1.0)) + +static inline void +mov_avg_ema_init_alpha(struct mov_avg_ema *ema, + double alpha) +{ + *ema = (struct mov_avg_ema) MOV_AVG_EMA_INITIALIZER_ALPHA(alpha); +} + +static inline void +mov_avg_ema_init(struct mov_avg_ema *ema, + unsigned long long int n_elem) +{ + *ema = (struct mov_avg_ema) MOV_AVG_EMA_INITIALIZER(n_elem); +} + +static inline void +mov_avg_ema_update(struct mov_avg_ema *ema, double new_val) +{ + const double alpha = ema->alpha; + double alpha_diff; + double diff; + + if (!ema->initialized) { + ema->initialized = true; + ema->mean = new_val; + return; + } + + diff = new_val - ema->mean; + alpha_diff = alpha * diff; + + ema->variance = (1.0 - alpha) * (ema->variance + alpha_diff * diff); + ema->mean = ema->mean + alpha_diff; +} + +static inline double +mov_avg_ema(struct mov_avg_ema *ema) +{ + return ema->mean; +} + +static inline double +mov_avg_ema_std_dev(struct mov_avg_ema *ema) +{ + return sqrt(ema->variance); +} + +#endif /* _MOV_AVG_H */ From patchwork Wed Feb 10 15:33:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439067 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=YBmvNutV; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=ZKLzifx7; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2941hBz9sCD for ; Thu, 11 Feb 2021 02:34:25 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0828B86BBC; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id suggBoMmetOh; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0286386B22; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D41ADC1E6F; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 35C88C013A for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 244346F490 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KpIl0r6HbwH0 for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 062DF600B8; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9DB9A600B8 for ; Wed, 10 Feb 2021 15:34:14 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id C2515B4F; Wed, 10 Feb 2021 10:34:13 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=EwqzCITp2yhe7 xDVd14MwE1QW0un9K+C1LXz4Doiwqs=; b=YBmvNutVpkAHvRVnTWyKGv6j+2j9i vb2apOFvWoRIvX19BptMzDKqenwHIZDR/tWKaDI3pVLldwS+qg76Ogh0MH++tnoV 9lBcC1sEKBCLMCEW/cWQ9rJsj8OHpQQtHlPY3/f/1QUcugun6m6WMwMwMEht25wL newnGFHNNps/qj2+kAvtNBmdd3PBZu3Q7HqRAFkc0q5Xpg2JVCkqS90ZX8wguSch k6qecwGeUDvCRmn/oIs+0+4oppAJ95OAwyPctT/0Pnl+cgFsooVo7bOLsWwhCfJ/ uPpLHwgzpSRYhzl4/MDMHv+LflcSZ3nJ30eytY8tbmiOKcJtn6DI2lecw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=EwqzCITp2yhe7xDVd14MwE1QW0un9K+C1LXz4Doiwqs=; b=ZKLzifx7 4rC+X8lmYFgVgtQrMnLsvB273OtaznpIlvBNO9rCnJ1Fc/yRLmAm4nQzuamSdL5l Axoch3Eek15Alhw1A9QkZFHEA2FIsOIeD4b0PdMeHHRmkrRR7DTRjaQ1qqN8pMTB dj+CMGqRRmFxEMcb5CZDzGh4WEmQZyd4WelBsWl3GsYC/STi1XvveEtVgjK2k6gi CSlUfkB2J08t+zLNQp0Zi3OdC/C+2BstZW660lTc0CSyMhdsAi41oohBliiK6ff0 634/2kLjYd3JKVLPXwF8o1F2y1QmVnmGrXP0ZqDBvEjYDT6BXy+IUyGwoC3rFIdV w3JF8Zud7BBRFg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 903AD24005D; Wed, 10 Feb 2021 10:34:12 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:53 +0100 Message-Id: <2f83fd1e7389412a1691f1488bf73e34dbc1677e.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 07/23] dpif-netdev: Implement hardware offloads stats query X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In the netdev datapath, keep track of the enqueued offloads between the PMDs and the offload thread. Additionally, query each netdev for their hardware offload counters. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 89 insertions(+), 1 deletion(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 7a0e8f24e..bf994e304 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -51,6 +51,7 @@ #include "hmapx.h" #include "id-pool.h" #include "ipf.h" +#include "mov-avg.h" #include "netdev.h" #include "netdev-offload.h" #include "netdev-provider.h" @@ -430,6 +431,7 @@ struct dp_offload_thread_item { struct match match; struct nlattr *actions; size_t actions_len; + long long int timestamp; struct ovs_list node; }; @@ -437,12 +439,18 @@ struct dp_offload_thread_item { struct dp_offload_thread { struct ovs_mutex mutex; struct ovs_list list; + uint64_t enqueued_item; + struct mov_avg_cma cma; + struct mov_avg_ema ema; pthread_cond_t cond; }; static struct dp_offload_thread dp_offload_thread = { .mutex = OVS_MUTEX_INITIALIZER, .list = OVS_LIST_INITIALIZER(&dp_offload_thread.list), + .enqueued_item = 0, + .cma = MOV_AVG_CMA_INITIALIZER, + .ema = MOV_AVG_EMA_INITIALIZER(100), }; static struct ovsthread_once offload_thread_once @@ -2631,6 +2639,7 @@ dp_netdev_append_flow_offload(struct dp_offload_thread_item *offload) { ovs_mutex_lock(&dp_offload_thread.mutex); ovs_list_push_back(&dp_offload_thread.list, &offload->node); + dp_offload_thread.enqueued_item++; xpthread_cond_signal(&dp_offload_thread.cond); ovs_mutex_unlock(&dp_offload_thread.mutex); } @@ -2735,6 +2744,7 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) { struct dp_offload_thread_item *offload; struct ovs_list *list; + long long int latency_us; const char *op; int ret; @@ -2747,6 +2757,7 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) ovsrcu_quiesce_end(); } list = ovs_list_pop_front(&dp_offload_thread.list); + dp_offload_thread.enqueued_item--; offload = CONTAINER_OF(list, struct dp_offload_thread_item, node); ovs_mutex_unlock(&dp_offload_thread.mutex); @@ -2767,6 +2778,10 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) OVS_NOT_REACHED(); } + latency_us = time_usec() - offload->timestamp; + mov_avg_cma_update(&dp_offload_thread.cma, latency_us); + mov_avg_ema_update(&dp_offload_thread.ema, latency_us); + VLOG_DBG("%s to %s netdev flow "UUID_FMT, ret == 0 ? "succeed" : "failed", op, UUID_ARGS((struct uuid *) &offload->flow->mega_ufid)); @@ -2792,6 +2807,7 @@ queue_netdev_flow_del(struct dp_netdev_pmd_thread *pmd, offload = dp_netdev_alloc_flow_offload(pmd, flow, DP_NETDEV_FLOW_OFFLOAD_OP_DEL); + offload->timestamp = pmd->ctx.now; dp_netdev_append_flow_offload(offload); } @@ -2825,6 +2841,7 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, memcpy(offload->actions, actions, actions_len); offload->actions_len = actions_len; + offload->timestamp = pmd->ctx.now; dp_netdev_append_flow_offload(offload); } @@ -4201,6 +4218,77 @@ dpif_netdev_operate(struct dpif *dpif, struct dpif_op **ops, size_t n_ops, } } +static int +dpif_netdev_offload_stats_get(struct dpif *dpif, + struct netdev_custom_stats *stats) +{ + enum { + DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED, + DP_NETDEV_HW_OFFLOADS_STATS_INSERTED, + DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN, + DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV, + DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN, + DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV, + }; + const char *names[] = { + [DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED] = + " Enqueued offloads", + [DP_NETDEV_HW_OFFLOADS_STATS_INSERTED] = + " Inserted offloads", + [DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN] = + " Cumulative Average latency (us)", + [DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV] = + " Cumulative Latency stddev (us)", + [DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN] = + " Exponential Average latency (us)", + [DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV] = + " Exponential Latency stddev (us)", + }; + struct dp_netdev *dp = get_dp_netdev(dpif); + struct dp_netdev_port *port; + uint64_t nb_offloads; + size_t i; + + if (!netdev_is_flow_api_enabled()) { + return EINVAL; + } + + stats->size = ARRAY_SIZE(names); + stats->counters = xcalloc(stats->size, sizeof *stats->counters); + + nb_offloads = 0; + + ovs_mutex_lock(&dp->port_mutex); + HMAP_FOR_EACH (port, node, &dp->ports) { + uint64_t port_nb_offloads = 0; + + /* Do not abort on read error from a port, just report 0. */ + if (!netdev_flow_get_n_flows(port->netdev, &port_nb_offloads)) { + nb_offloads += port_nb_offloads; + } + } + ovs_mutex_unlock(&dp->port_mutex); + + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED].value = + dp_offload_thread.enqueued_item; + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_INSERTED].value = nb_offloads; + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN].value = + mov_avg_cma(&dp_offload_thread.cma); + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV].value = + mov_avg_cma_std_dev(&dp_offload_thread.cma); + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN].value = + mov_avg_ema(&dp_offload_thread.ema); + stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV].value = + mov_avg_ema_std_dev(&dp_offload_thread.ema); + + for (i = 0; i < ARRAY_SIZE(names); i++) { + snprintf(stats->counters[i].name, sizeof(stats->counters[i].name), + "%s", names[i]); + } + + return 0; +} + /* Enable or Disable PMD auto load balancing. */ static void set_pmd_auto_lb(struct dp_netdev *dp, bool always_log) @@ -8458,7 +8546,7 @@ const struct dpif_class dpif_netdev_class = { dpif_netdev_flow_dump_thread_destroy, dpif_netdev_flow_dump_next, dpif_netdev_operate, - NULL, /* offload_stats_get */ + dpif_netdev_offload_stats_get, NULL, /* recv_set */ NULL, /* handlers_set */ dpif_netdev_set_config, From patchwork Wed Feb 10 15:33:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439065 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=UarweCDA; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=iQfxgYHh; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP240C16z9sB4 for ; Thu, 11 Feb 2021 02:34:20 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8FAFC86ADD; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yXyTJxsKJbD3; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7F17286ACA; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 57C03C0FA7; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CB4E6C013A for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id BA922867DA for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6fV5tM7K8ROt for ; Wed, 10 Feb 2021 15:34:14 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id CB812866A3 for ; Wed, 10 Feb 2021 15:34:14 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id 60749547; Wed, 10 Feb 2021 10:34:14 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=0tTUZl0Zw0ps/ 2HR/rJY/JWOBOgAztt9Ip8J8UAlnDU=; b=UarweCDAv9aeP3+0WL2Mniejlqr2N qkjGZV4GLp4tdSPNbMqYiIeZYwI4C5n8Pzw6HJirhzVcWzARzG4F3BRn5cvNldw1 kwEhT7gZ+lb8Hr6kcEBemfIdzs9bDax+biGcP9ni+IhCQuBmGjkl7r+F4nj+52mm hXUdTNoW5Lx0JVLTfCCDdJxZwUdxZA3jngUqnpMAgn2B+Cxshr9xC/ZayuO9fXY3 rh7tTrUvJourYkU9Vl1+6qn+DcQdMNVRuByNlBsws6NkLORTH5UT178ME375VCvu 6vFzKskOZJaHtzYM1Yao6tbnZ7CQgwrqtUr2kDeUYtZCJQPpr0F7tyHnw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=0tTUZl0Zw0ps/2HR/rJY/JWOBOgAztt9Ip8J8UAlnDU=; b=iQfxgYHh nuUYetqKiVBErx8HajBOEl1+LEAFifCP4gSgSdGGEY1eotJ0iL9wTiud94Edfhko TExp7s8SsFISte2WZzgeUP1Cr4wMCAhjjUGDFGlUHXkUdxMcgnN0YJKtm5fWoP7i KYDLLwySDoQ5lI1xRnbM8WiS58yqcwrbn0jBukmEfCNhoFDQx2avhO/Z8kRn4I3B 9IcsZIUby5hnVxFd+au9QfRlI5BJ/+EZCGacNQ9nFEpgSRTTK6msa1FSioDEMhDa F5ov75wpfXDZCpZq8u7IBEj/VExH5owq/IdUuQDYfRMwSxw1Khs1VEJxirpIX4rp iXNEdfMteCrKqA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 1E3BA240064; Wed, 10 Feb 2021 10:34:13 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:54 +0100 Message-Id: <48225a2793aa9f61a8d23c97d96307815afd2354.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 08/23] ovs-atomic: Expose atomic exchange operation X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The atomic exchange operation is a useful primitive that should be available as well. Most compiler already expose or offer a way to use it, but a single symbol needs to be defined. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/ovs-atomic-c++.h | 3 +++ lib/ovs-atomic-clang.h | 5 +++++ lib/ovs-atomic-gcc4+.h | 5 +++++ lib/ovs-atomic-gcc4.7+.h | 5 +++++ lib/ovs-atomic-i586.h | 5 +++++ lib/ovs-atomic-locked.h | 9 +++++++++ lib/ovs-atomic-msvc.h | 22 ++++++++++++++++++++++ lib/ovs-atomic-pthreads.h | 5 +++++ lib/ovs-atomic-x86_64.h | 5 +++++ lib/ovs-atomic.h | 8 +++++++- 10 files changed, 71 insertions(+), 1 deletion(-) diff --git a/lib/ovs-atomic-c++.h b/lib/ovs-atomic-c++.h index d47b8dd39..8605fa9d3 100644 --- a/lib/ovs-atomic-c++.h +++ b/lib/ovs-atomic-c++.h @@ -29,6 +29,9 @@ using std::atomic_compare_exchange_strong_explicit; using std::atomic_compare_exchange_weak; using std::atomic_compare_exchange_weak_explicit; +using std::atomic_exchange; +using std::atomic_exchange_explicit; + #define atomic_read(SRC, DST) \ atomic_read_explicit(SRC, DST, memory_order_seq_cst) #define atomic_read_explicit(SRC, DST, ORDER) \ diff --git a/lib/ovs-atomic-clang.h b/lib/ovs-atomic-clang.h index 34cc2faa7..cdf02a512 100644 --- a/lib/ovs-atomic-clang.h +++ b/lib/ovs-atomic-clang.h @@ -67,6 +67,11 @@ typedef enum { #define atomic_compare_exchange_weak_explicit(DST, EXP, SRC, ORD1, ORD2) \ __c11_atomic_compare_exchange_weak(DST, EXP, SRC, ORD1, ORD2) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + __c11_atomic_exchange(RMW, ARG, ORDER) + #define atomic_add(RMW, ARG, ORIG) \ atomic_add_explicit(RMW, ARG, ORIG, memory_order_seq_cst) #define atomic_sub(RMW, ARG, ORIG) \ diff --git a/lib/ovs-atomic-gcc4+.h b/lib/ovs-atomic-gcc4+.h index 25bcf20a0..f9accde1a 100644 --- a/lib/ovs-atomic-gcc4+.h +++ b/lib/ovs-atomic-gcc4+.h @@ -128,6 +128,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + __sync_lock_test_and_set(DST, SRC) +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + #define atomic_op__(RMW, OP, ARG, ORIG) \ ({ \ typeof(RMW) rmw__ = (RMW); \ diff --git a/lib/ovs-atomic-gcc4.7+.h b/lib/ovs-atomic-gcc4.7+.h index 4c197ebe0..846e05775 100644 --- a/lib/ovs-atomic-gcc4.7+.h +++ b/lib/ovs-atomic-gcc4.7+.h @@ -61,6 +61,11 @@ typedef enum { #define atomic_compare_exchange_weak_explicit(DST, EXP, SRC, ORD1, ORD2) \ __atomic_compare_exchange_n(DST, EXP, SRC, true, ORD1, ORD2) +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + __atomic_exchange_n(DST, SRC, ORDER) +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + #define atomic_add(RMW, OPERAND, ORIG) \ atomic_add_explicit(RMW, OPERAND, ORIG, memory_order_seq_cst) #define atomic_sub(RMW, OPERAND, ORIG) \ diff --git a/lib/ovs-atomic-i586.h b/lib/ovs-atomic-i586.h index 9a385ce84..35a0959ff 100644 --- a/lib/ovs-atomic-i586.h +++ b/lib/ovs-atomic-i586.h @@ -400,6 +400,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + atomic_exchange__(RMW, ARG, ORDER) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) + #define atomic_add__(RMW, ARG, CLOB) \ asm volatile("lock; xadd %0,%1 ; " \ "# atomic_add__ " \ diff --git a/lib/ovs-atomic-locked.h b/lib/ovs-atomic-locked.h index f8f0ba2a5..bf38c4a43 100644 --- a/lib/ovs-atomic-locked.h +++ b/lib/ovs-atomic-locked.h @@ -31,6 +31,15 @@ void atomic_unlock__(void *); atomic_unlock__(DST), \ false))) +#define atomic_exchange_locked(DST, SRC) \ + ({ \ + atomic_lock__(DST); \ + typeof(*(DST)) __tmp = *(DST); \ + *(DST) = SRC; \ + atomic_unlock__(DST); \ + __tmp; \ + }) + #define atomic_op_locked_add += #define atomic_op_locked_sub -= #define atomic_op_locked_or |= diff --git a/lib/ovs-atomic-msvc.h b/lib/ovs-atomic-msvc.h index 9def887d3..ef8310269 100644 --- a/lib/ovs-atomic-msvc.h +++ b/lib/ovs-atomic-msvc.h @@ -345,6 +345,28 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +/* While intrinsics offering different memory ordering + * are available in MSVC C compiler, they are not defined + * in the C++ compiler. Ignore for compatibility. + * + * Use nested ternary operators as the GNU extension ({}) + * is not available. + */ + +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + ((sizeof *(DST) == 1) ? \ + _InterlockedExchange8((char volatile *) DST, SRC) \ + : (sizeof *(DST) == 2) ? \ + _InterlockedExchange16((short volatile *) DST, SRC) \ + : (sizeof *(DST) == 4) ? \ + _InterlockedExchange((long int volatile *) DST, SRC) \ + : (sizeof *(DST) == 8) ? \ + _InterlockedExchange64((__int64 volatile *) DST, SRC) \ + : (ovs_abort(), 0)) + +#define atomic_exchange(DST, SRC) \ + atomic_exchange_explicit(DST, SRC, memory_order_seq_cst) + /* MSVCs c++ compiler implements c11 atomics and looking through its * implementation (in xatomic.h), orders are ignored for x86 platform. * Do the same here. */ diff --git a/lib/ovs-atomic-pthreads.h b/lib/ovs-atomic-pthreads.h index 12234e79e..570a67fe4 100644 --- a/lib/ovs-atomic-pthreads.h +++ b/lib/ovs-atomic-pthreads.h @@ -77,6 +77,11 @@ atomic_signal_fence(memory_order order OVS_UNUSED) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange(DST, SRC) \ + atomic_exchange_locked(DST, SRC) +#define atomic_exchange_explicit(DST, SRC, ORDER) \ + ((void) (ORDER), atomic_exchange(DST, SRC)) + #define atomic_add(RMW, ARG, ORIG) atomic_op_locked(RMW, add, ARG, ORIG) #define atomic_sub(RMW, ARG, ORIG) atomic_op_locked(RMW, sub, ARG, ORIG) #define atomic_or( RMW, ARG, ORIG) atomic_op_locked(RMW, or, ARG, ORIG) diff --git a/lib/ovs-atomic-x86_64.h b/lib/ovs-atomic-x86_64.h index 1e7d42707..3bdaf2f08 100644 --- a/lib/ovs-atomic-x86_64.h +++ b/lib/ovs-atomic-x86_64.h @@ -274,6 +274,11 @@ atomic_signal_fence(memory_order order) #define atomic_compare_exchange_weak_explicit \ atomic_compare_exchange_strong_explicit +#define atomic_exchange_explicit(RMW, ARG, ORDER) \ + atomic_exchange__(RMW, ARG, ORDER) +#define atomic_exchange(RMW, ARG) \ + atomic_exchange_explicit(RMW, ARG, memory_order_seq_cst) + #define atomic_add__(RMW, ARG, CLOB) \ asm volatile("lock; xadd %0,%1 ; " \ "# atomic_add__ " \ diff --git a/lib/ovs-atomic.h b/lib/ovs-atomic.h index 11fa19268..8fdce0cf8 100644 --- a/lib/ovs-atomic.h +++ b/lib/ovs-atomic.h @@ -210,7 +210,7 @@ * In this section, A is an atomic type and C is the corresponding non-atomic * type. * - * The "store" and "compare_exchange" primitives match C11: + * The "store", "exchange", and "compare_exchange" primitives match C11: * * void atomic_store(A *object, C value); * void atomic_store_explicit(A *object, C value, memory_order); @@ -244,6 +244,12 @@ * efficiently, so it should be used if the application will need to * loop anyway. * + * C atomic_exchange(A *object, C desired); + * C atomic_exchange_explicit(A *object, C desired, memory_order); + * + * Atomically stores 'desired' into '*object', returning the value + * previously held. + * * The following primitives differ from the C11 ones (and have different names) * because there does not appear to be a way to implement the standard * primitives in standard C: From patchwork Wed Feb 10 15:33:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439072 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=PxcTBIEg; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=b3UTwLo3; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2T4tt4z9rx8 for ; Thu, 11 Feb 2021 02:34:41 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 416BE86D41; Wed, 10 Feb 2021 15:34:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vssJPEAKdNUo; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4DF7986C6C; Wed, 10 Feb 2021 15:34:27 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 295E4C1E6F; Wed, 10 Feb 2021 15:34:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 88834C1E70 for ; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4AF4B6F62B for ; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JfvOLLfdgdin for ; Wed, 10 Feb 2021 15:34:19 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 548DF6F628; Wed, 10 Feb 2021 15:34:19 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 42E816F604 for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id A5AD0E11; Wed, 10 Feb 2021 10:34:15 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=ZoW35I6WLPHVf 2R5wlnG0yWUsK2uYZzIhj4hYOqnfbk=; b=PxcTBIEgcqTzvKrKLwQ1DkO10+wGD GK2IauT5px32ZUlllJjG7Ki7HJRuvSXPT0s5pj+tBqc/NquPT2gFeYw8uDOyv1S0 aSZpvXbN2zw1EzQ2KJnkHtH9KZY2POxVFByTE3TmzxXvtpGxeSdaM9yewzDw+X+m 5cYXIja3vcF5XIPx4OmxG13rqc/dDL2s15kvcoAgQiUhXgNxG33Txnj6Mh5FTQyP oIFQOvjV3I/nvL2tZP0LSq8J+KdIOxhKtKsiTJTIwfdXg/RnMOx3z5ADTopqFG4I pFYnDaxdMGsfZJHxKKiUZeNSG76S/pqb9hvjWm+zfG4CKyP45xmcdZ+pA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=ZoW35I6WLPHVf2R5wlnG0yWUsK2uYZzIhj4hYOqnfbk=; b=b3UTwLo3 xfvfmEBUwHyriMDq56N/gyYdPjl8TJxGCK9NzfcZyFFcMpAEItIdQahBhzIMos0Y zywg+G3rwwvevZyf11SneBOB3toWCQPraYL8KvLioTKYFopRL1F2HjeaNDFItk70 a8iQRqIVez0ecsUwDBukVde93G1FryAQmMxlsO+TQ7lMdg5lnesQNEFAjpblO8Bq LzymPFleJyxk1lA8wMG0Xekx69C6uxknZCxeL4NkC/7I/OwadrVV/Ji7pHnrUIeI q/P41fLZLVVKUBCCGX2o6KkZseEoIn0UQxDvvxNSP5JlA06nP0bfr3ZGiCltggEB 7Frnb0Ye3YAGXQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe euhffftefftdfftdehheetheeivdekueefhedvffettedvgeduudffuddtjeehgfenucff ohhmrghinhepuddtvdegtghorhgvshdrnhgvthdprhhotghhvghsthgvrhdrvgguuhdprg hprggthhgvrdhorhhgnecukfhppeekiedrvdehgedrudeigedrudejgeenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhivhgvsehuvdehie drnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id A325A240069; Wed, 10 Feb 2021 10:34:13 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:55 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 09/23] mpsc-queue: Module for lock-free message passing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a lockless multi-producer/single-consumer (MPSC), linked-list based, intrusive, unbounded queue that does not require deferred memory management. The queue is an implementation of the structure described by Dmitri Vyukov[1]. It adds a slightly more explicit API explaining the proper use of the queue. Alternatives were considered such as a Treiber Stack [2] or a Michael-Scott queue [3], but this one is faster, simpler and scalable. [1]: http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue [2]: R. K. Treiber. Systems programming: Coping with parallelism. Technical Report RJ 5118, IBM Almaden Research Center, April 1986. [3]: M. M. Michael, Simple, Fast, and Practical Non-Blocking and Blocking Concurrent Queue Algorithms. https://www.cs.rochester.edu/research/synchronization/pseudocode/queues.html The queue is designed to improve the specific MPSC setup. A benchmark accompanies the unit tests to measure the difference in this configuration. A single reader thread polls the queue while N writers enqueue elements as fast as possible. The mpsc-queue is compared against the regular ovs-list as well as the guarded list. The latter usually offers a slight improvement by batching the element removal, however the mpsc-queue is faster. The average is of each producer threads time: $ ./tests/ovstest test-mpsc-queue benchmark 3000000 1 Benchmarking n=3000000 on 1 + 1 threads. type\thread: Reader 1 Avg mpsc-queue: 161 161 161 ms list: 803 803 803 ms guarded list: 665 665 665 ms $ ./tests/ovstest test-mpsc-queue benchmark 3000000 2 Benchmarking n=3000000 on 1 + 2 threads. type\thread: Reader 1 2 Avg mpsc-queue: 102 101 97 99 ms list: 246 212 246 229 ms guarded list: 264 263 214 238 ms $ ./tests/ovstest test-mpsc-queue benchmark 3000000 3 Benchmarking n=3000000 on 1 + 3 threads. type\thread: Reader 1 2 3 Avg mpsc-queue: 92 91 92 91 91 ms list: 520 517 515 520 517 ms guarded list: 405 395 401 404 400 ms $ ./tests/ovstest test-mpsc-queue benchmark 3000000 4 Benchmarking n=3000000 on 1 + 4 threads. type\thread: Reader 1 2 3 4 Avg mpsc-queue: 77 73 73 77 75 74 ms list: 371 359 361 287 370 344 ms guarded list: 389 388 359 363 357 366 ms Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 2 + lib/mpsc-queue.c | 190 +++++++++++++ lib/mpsc-queue.h | 149 +++++++++++ tests/automake.mk | 1 + tests/library.at | 5 + tests/test-mpsc-queue.c | 580 ++++++++++++++++++++++++++++++++++++++++ 6 files changed, 927 insertions(+) create mode 100644 lib/mpsc-queue.c create mode 100644 lib/mpsc-queue.h create mode 100644 tests/test-mpsc-queue.c diff --git a/lib/automake.mk b/lib/automake.mk index e072bed71..cbdda460a 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -167,6 +167,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/memory.h \ lib/meta-flow.c \ lib/mov-avg.h \ + lib/mpsc-queue.c \ + lib/mpsc-queue.h \ lib/multipath.c \ lib/multipath.h \ lib/namemap.c \ diff --git a/lib/mpsc-queue.c b/lib/mpsc-queue.c new file mode 100644 index 000000000..9280d81f6 --- /dev/null +++ b/lib/mpsc-queue.c @@ -0,0 +1,190 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "ovs-atomic.h" + +#include "mpsc-queue.h" + +/* Multi-producer, single-consumer queue + * ===================================== + * + * This an implementation of the MPSC queue described by Dmitri Vyukov [1]. + * + * One atomic exchange operation is done per insertion. Removal in most cases + * will not require atomic operation and will use one atomic exchange to close + * the queue chain. + * + * Insertion + * ========= + * + * The queue is implemented using a linked-list. Insertion is done at the + * back of the queue, by swapping the current end with the new node atomically, + * then pointing the previous end toward the new node. To follow Vyukov + * nomenclature, the end-node of the chain is called head. A producer will + * only manipulate the head. + * + * The head swap is atomic, however the link from the previous head to the new + * one is done in a separate operation. This means that the chain is + * momentarily broken, when the previous head still points to NULL and the + * current head has been inserted. + * + * Considering a series of insertions, the queue state will remain consistent + * and the insertions order is compatible with their precedence, thus the + * queue is serializable. However, because an insertion consists in two + * separate memory transactions, it is not linearizable. + * + * Removal + * ======= + * + * The consumer must deal with the queue inconsistency. It will manipulate + * the tail of the queue and move it along the latest consumed elements. + * When an end of the chain of elements is found (the next pointer is NULL), + * the tail is compared with the head. + * + * If both points to different addresses, then the queue is in an inconsistent + * state: the tail cannot move forward as the next is NULL, but the head is not + * the last element in the chain: this can only happen if the chain is broken. + * + * In this case, the consumer must wait for the producer to finish writing the + * next pointer of its current tail: 'MPSC_QUEUE_RETRY' is returned. + * + * Removal is thus in most cases (when there are elements in the queue) + * accomplished without using atomics, until the last element of the queue. + * There, the head is atomically loaded. If the queue is in a consistent state, + * the head is moved back to the queue stub by inserting the stub in the queue: + * ending the queue is the same as an insertion, which is one atomic XCHG. + * + * Limitations + * =========== + * + * The chain will remain broken as long as a producer is not finished writing + * its next pointer. If a producer is cancelled for example, the queue could + * remain broken for any future readings. This queue should either be used + * with cooperative threads or outside any cancellable sections. + * + * Performances + * ============ + * + * In benchmarks this structure was better than alternatives such as: + * + * * A reversed Treiber stack [2], using 1 CAS per operations + * and requiring reversal of the node list on removal. + * + * * Michael-Scott lock-free queue [3], using 2 CAS per operations. + * + * While it is not linearizable, this queue is well-suited for message passing. + * If a proper hardware XCHG operation is used, it scales better than + * CAS-based implementations. + * + * References + * ========== + * + * [1]: http://www.1024cores.net/home/lock-free-algorithms/queues/intrusive-mpsc-node-based-queue + * + * [2]: R. K. Treiber. Systems programming: Coping with parallelism. + * Technical Report RJ 5118, IBM Almaden Research Center, April 1986. + * + * [3]: M. M. Michael, Simple, Fast, and Practical Non-Blocking and + * Blocking Concurrent Queue Algorithms + * [3]: https://www.cs.rochester.edu/research/synchronization/pseudocode/queues.html + * + */ + +void +mpsc_queue_init(struct mpsc_queue *queue) +{ + atomic_store_relaxed(&queue->head, &queue->stub); + atomic_store_relaxed(&queue->tail, &queue->stub); + atomic_store_relaxed(&queue->stub.next, NULL); + + ovs_mutex_init(&queue->read_lock); +} + +void +mpsc_queue_destroy(struct mpsc_queue *queue) +{ + ovs_mutex_destroy(&queue->read_lock); +} + +int +mpsc_queue_acquire(struct mpsc_queue *queue) + OVS_TRY_LOCK(1, queue->read_lock) +{ + return !ovs_mutex_trylock(&queue->read_lock); +} + +void +mpsc_queue_release(struct mpsc_queue *queue) + OVS_RELEASES(queue->read_lock) +{ + ovs_mutex_unlock(&queue->read_lock); +} + +enum mpsc_queue_poll_result +mpsc_queue_poll(struct mpsc_queue *queue, struct mpsc_queue_node **node) + OVS_REQUIRES(queue->read_lock) +{ + struct mpsc_queue_node *tail; + struct mpsc_queue_node *next; + struct mpsc_queue_node *head; + + atomic_read_relaxed(&queue->tail, &tail); + atomic_read_explicit(&tail->next, &next, memory_order_acquire); + + if (tail == &queue->stub) { + if (next == NULL) { + return MPSC_QUEUE_EMPTY; + } + + atomic_store_relaxed(&queue->tail, next); + tail = next; + atomic_read_explicit(&tail->next, &next, memory_order_acquire); + } + + if (next != NULL) { + atomic_store_relaxed(&queue->tail, next); + *node = tail; + return MPSC_QUEUE_ITEM; + } + + atomic_read_explicit(&queue->head, &head, memory_order_acquire); + if (tail != head) { + return MPSC_QUEUE_RETRY; + } + + mpsc_queue_insert(queue, &queue->stub); + + atomic_read_explicit(&tail->next, &next, memory_order_acquire); + if (next != NULL) { + atomic_store_relaxed(&queue->tail, next); + *node = tail; + return MPSC_QUEUE_ITEM; + } + + return MPSC_QUEUE_EMPTY; +} + +void +mpsc_queue_insert(struct mpsc_queue *queue, struct mpsc_queue_node *node) +{ + struct mpsc_queue_node *prev; + + atomic_store_relaxed(&node->next, NULL); + prev = atomic_exchange_explicit(&queue->head, node, memory_order_acq_rel); + atomic_store_explicit(&prev->next, node, memory_order_release); +} diff --git a/lib/mpsc-queue.h b/lib/mpsc-queue.h new file mode 100644 index 000000000..8ee59409f --- /dev/null +++ b/lib/mpsc-queue.h @@ -0,0 +1,149 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef MPSC_QUEUE_H +#define MPSC_QUEUE_H 1 + +#include +#include +#include + +#include +#include + +#include "ovs-atomic.h" + +/* Multi-producer, single-consumer queue + * ===================================== + * + * This data structure is a lockless queue implementation with + * the following properties: + * + * * Multi-producer: multiple threads can write concurrently. + * Insertion in the queue is thread-safe, no inter-thread + * synchronization is necessary. + * + * * Single-consumer: only a single thread can safely remove + * nodes from the queue. The queue must be 'acquired' using + * 'mpsc_queue_acquire()' before removing nodes. + * + * * Unbounded: the queue is backed by a linked-list and is not + * limited in number of elements. + * + * * Intrusive: queue elements are allocated as part of larger + * objects. Objects are retrieved by offset manipulation. + * + * * per-producer FIFO: Elements in the queue are kept in the + * order their producer inserted them. The consumer retrieves + * them in in the same insertion order. When multiple + * producers insert at the same time, either will proceed. + * + * This queue is well-suited for message passing between threads, + * where any number of thread can insert a message and a single + * thread is meant to receive and process them. + * + * Thread-safety + * ============= + * + * The consumer thread must acquire the queue using 'mpsc_queue_acquire()'. + * If no error is returned, the thread can call 'mpsc_queue_poll()'. + * When a thread is finished with reading the queue, it can release the + * reader lock using 'mpsc_queue_release()'. + * + * Producers can always insert elements in the queue, even if no consumer + * acquired the reader lock. No inter-producer synchronization (e.g. using a + * lock) is needed. + * + * The consumer thread is also allowed to insert elements while it holds the + * reader lock. + * + * Producer threads must never be cancelled while writing to the queue. + * This will block the consumer, that will then lose any subsequent writes + * to the queue. Producers should ideally be cooperatively managed or + * the queue insertion should be within non-cancellable sections. + * + * Queue state + * =========== + * + * When polling the queue, three states can be observed: 'empty', 'non-empty', + * and 'inconsistent'. Three polling results are defined, respectively: + * + * * MPSC_QUEUE_EMPTY: the queue is empty. + * * MPSC_QUEUE_ITEM: an item was available and has been removed. + * * MPSC_QUEUE_RETRY: the queue is inconsistent. + * + * If 'MPSC_QUEUE_RETRY' is returned, then a producer has not yet finished + * writing to the queue and the list of nodes is not coherent. The consumer + * can retry shortly to check if the producer has finished. + * + * This behavior is the reason the removal function is called + * 'mpsc_queue_poll()'. + * + */ + +struct mpsc_queue_node { + ATOMIC(struct mpsc_queue_node *) next; +}; + +struct mpsc_queue { + ATOMIC(struct mpsc_queue_node *) head; + ATOMIC(struct mpsc_queue_node *) tail; + struct mpsc_queue_node stub; + struct ovs_mutex read_lock; +}; + +#define MPSC_QUEUE_INITIALIZER(Q) { \ + .head = ATOMIC_VAR_INIT(&(Q)->stub), \ + .tail = ATOMIC_VAR_INIT(&(Q)->stub), \ + .stub = { .next = ATOMIC_VAR_INIT(NULL) }, \ + .read_lock = OVS_MUTEX_INITIALIZER, \ +} + +/* Consumer API. */ + +/* Initialize the queue. Not necessary is 'MPSC_QUEUE_INITIALIZER' was used. */ +void mpsc_queue_init(struct mpsc_queue *queue); +/* The reader lock must be released prior to destroying the queue. */ +void mpsc_queue_destroy(struct mpsc_queue *queue); +/* Acquire the reader lock if 1 is returned. */ +int mpsc_queue_acquire(struct mpsc_queue *queue); +/* Release the reader lock. */ +void mpsc_queue_release(struct mpsc_queue *queue); + +enum mpsc_queue_poll_result { + /* Queue is empty. */ + MPSC_QUEUE_EMPTY, + /* Polling the queue returned an item. */ + MPSC_QUEUE_ITEM, + /* Data has been enqueued but one or more producer thread have not + * finished writing it. The queue is in an inconsistent state. + * Retrying shortly, if the producer threads are still active, will + * return the data. + */ + MPSC_QUEUE_RETRY, +}; + +/* Set 'node' to a removed item from the queue if 'MPSC_QUEUE_ITEM' is + * returned, otherwise 'node' is not set. + */ +enum mpsc_queue_poll_result +mpsc_queue_poll(struct mpsc_queue *queue, struct mpsc_queue_node **node); + +/* Producer API. */ + +void mpsc_queue_insert(struct mpsc_queue *queue, struct mpsc_queue_node *node); + +#endif /* MPSC_QUEUE_H */ diff --git a/tests/automake.mk b/tests/automake.mk index 677b99a6b..d7ae5df90 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -460,6 +460,7 @@ tests_ovstest_SOURCES = \ tests/test-list.c \ tests/test-lockfile.c \ tests/test-multipath.c \ + tests/test-mpsc-queue.c \ tests/test-netflow.c \ tests/test-odp.c \ tests/test-ofpbuf.c \ diff --git a/tests/library.at b/tests/library.at index 1702b7556..ac24921f8 100644 --- a/tests/library.at +++ b/tests/library.at @@ -254,3 +254,8 @@ AT_SETUP([stopwatch module]) AT_CHECK([ovstest test-stopwatch], [0], [...... ], [ignore]) AT_CLEANUP + +AT_SETUP([mpsc-queue module]) +AT_CHECK([ovstest test-mpsc-queue check], [0], [.. +]) +AT_CLEANUP diff --git a/tests/test-mpsc-queue.c b/tests/test-mpsc-queue.c new file mode 100644 index 000000000..791715848 --- /dev/null +++ b/tests/test-mpsc-queue.c @@ -0,0 +1,580 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#undef NDEBUG +#include +#include +#include + +#include + +#include "command-line.h" +#include "guarded-list.h" +#include "mpsc-queue.h" +#include "openvswitch/list.h" +#include "openvswitch/util.h" +#include "ovs-thread.h" +#include "ovstest.h" +#include "timeval.h" +#include "util.h" + +struct element { + union { + struct mpsc_queue_node mpscq; + struct ovs_list list; + } node; + uint64_t mark; +}; + +static void +test_mpsc_queue_mark_element(struct mpsc_queue_node *node, + uint64_t mark, + unsigned int *counter) +{ + struct element *elem; + + elem = CONTAINER_OF(node, struct element, node.mpscq); + elem->mark = mark; + *counter += 1; +} + +static void +test_mpsc_queue_insert(void) +{ + struct element elements[100]; + struct mpsc_queue_node *node; + struct mpsc_queue queue; + unsigned int counter; + size_t i; + + memset(elements, 0, sizeof(elements)); + mpsc_queue_init(&queue); + ignore(mpsc_queue_acquire(&queue)); + + for (i = 0; i < ARRAY_SIZE(elements); i++) { + mpsc_queue_insert(&queue, &elements[i].node.mpscq); + } + + counter = 0; + while (mpsc_queue_poll(&queue, &node) == MPSC_QUEUE_ITEM) { + test_mpsc_queue_mark_element(node, 1, &counter); + } + + mpsc_queue_release(&queue); + mpsc_queue_destroy(&queue); + + ovs_assert(counter == ARRAY_SIZE(elements)); + for (i = 0; i < ARRAY_SIZE(elements); i++) { + ovs_assert(elements[i].mark == 1); + } + + printf("."); +} + +static void +test_mpsc_queue_flush_is_fifo(void) +{ + struct element elements[100]; + struct mpsc_queue_node *node; + struct mpsc_queue queue; + unsigned int counter; + size_t i; + + memset(elements, 0, sizeof(elements)); + + mpsc_queue_init(&queue); + ignore(mpsc_queue_acquire(&queue)); + + for (i = 0; i < ARRAY_SIZE(elements); i++) { + mpsc_queue_insert(&queue, &elements[i].node.mpscq); + } + + /* Elements are in the same order in the list as they + * were declared / initialized. + */ + counter = 0; + while (mpsc_queue_poll(&queue, &node) == MPSC_QUEUE_ITEM) { + test_mpsc_queue_mark_element(node, counter, &counter); + } + + /* The list is valid once extracted from the queue, + * the queue can be destroyed here. + */ + mpsc_queue_release(&queue); + mpsc_queue_destroy(&queue); + + for (i = 0; i < ARRAY_SIZE(elements) - 1; i++) { + struct element *e1, *e2; + + e1 = &elements[i]; + e2 = &elements[i + 1]; + + ovs_assert(e1->mark < e2->mark); + } + + printf("."); +} + +static void +run_tests(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + /* Verify basic insertion worked. */ + test_mpsc_queue_insert(); + /* Verify flush() happens in FIFO if configured. */ + test_mpsc_queue_flush_is_fifo(); + printf("\n"); +} + +static struct element *elements; +static uint64_t *thread_working_ms; /* Measured work time. */ + +static unsigned int n_threads; +static unsigned int n_elems; + +static struct ovs_barrier barrier; +static volatile bool working; + +static int +elapsed(const struct timeval *start) +{ + struct timeval end; + + xgettimeofday(&end); + return timeval_to_msec(&end) - timeval_to_msec(start); +} + +struct mpscq_aux { + struct mpsc_queue *queue; + atomic_uint thread_id; +}; + +static void * +mpsc_queue_insert_thread(void *aux_) +{ + unsigned int n_elems_per_thread; + struct element *th_elements; + struct mpscq_aux *aux = aux_; + struct timeval start; + unsigned int id; + size_t i; + + atomic_add(&aux->thread_id, 1u, &id); + n_elems_per_thread = n_elems / n_threads; + th_elements = &elements[id * n_elems_per_thread]; + + ovs_barrier_block(&barrier); + xgettimeofday(&start); + + for (i = 0; i < n_elems_per_thread; i++) { + mpsc_queue_insert(aux->queue, &th_elements[i].node.mpscq); + } + + thread_working_ms[id] = elapsed(&start); + ovs_barrier_block(&barrier); + + working = false; + + return NULL; +} + +static void +benchmark_mpsc_queue(void) +{ + struct mpsc_queue_node *node; + struct mpsc_queue queue; + struct timeval start; + unsigned int counter; + bool work_complete; + pthread_t *threads; + struct mpscq_aux aux; + uint64_t epoch; + uint64_t avg; + size_t i; + + memset(elements, 0, n_elems & sizeof *elements); + memset(thread_working_ms, 0, n_threads & sizeof *thread_working_ms); + + mpsc_queue_init(&queue); + + aux.queue = &queue; + atomic_store(&aux.thread_id, 0); + + for (i = n_elems - (n_elems % n_threads); i < n_elems; i++) { + mpsc_queue_insert(&queue, &elements[i].node.mpscq); + } + + working = true; + + threads = xmalloc(n_threads * sizeof *threads); + ovs_barrier_init(&barrier, n_threads); + + for (i = 0; i < n_threads; i++) { + threads[i] = ovs_thread_create("sc_queue_insert", + mpsc_queue_insert_thread, &aux); + } + + ignore(mpsc_queue_acquire(&queue)); + xgettimeofday(&start); + + counter = 0; + epoch = 1; + do { + while (mpsc_queue_poll(&queue, &node) == MPSC_QUEUE_ITEM) { + test_mpsc_queue_mark_element(node, epoch, &counter); + } + if (epoch == UINT64_MAX) { + epoch = 0; + } + epoch++; + } while (working); + + avg = 0; + for (i = 0; i < n_threads; i++) { + xpthread_join(threads[i], NULL); + avg += thread_working_ms[i]; + } + avg /= n_threads; + + /* Elements might have been inserted before threads were joined. */ + while (mpsc_queue_poll(&queue, &node) == MPSC_QUEUE_ITEM) { + test_mpsc_queue_mark_element(node, epoch, &counter); + } + + printf(" mpsc-queue: %6d", elapsed(&start)); + for (i = 0; i < n_threads; i++) { + printf(" %6" PRIu64, thread_working_ms[i]); + } + printf(" %6" PRIu64 " ms\n", avg); + + mpsc_queue_release(&queue); + mpsc_queue_destroy(&queue); + ovs_barrier_destroy(&barrier); + free(threads); + + work_complete = true; + for (i = 0; i < n_elems; i++) { + if (elements[i].mark == 0) { + printf("Element %" PRIuSIZE " was never consumed.\n", i); + work_complete = false; + } + } + ovs_assert(work_complete); + ovs_assert(counter == n_elems); +} + +struct list_aux { + struct ovs_list *list; + struct ovs_mutex *lock; + atomic_uint thread_id; +}; + +static void * +locked_list_insert_thread(void *aux_) +{ + unsigned int n_elems_per_thread; + struct element *th_elements; + struct list_aux *aux = aux_; + struct timeval start; + unsigned int id; + size_t i; + + atomic_add(&aux->thread_id, 1u, &id); + n_elems_per_thread = n_elems / n_threads; + th_elements = &elements[id * n_elems_per_thread]; + + ovs_barrier_block(&barrier); + xgettimeofday(&start); + + for (i = 0; i < n_elems_per_thread; i++) { + ovs_mutex_lock(aux->lock); + ovs_list_push_front(aux->list, &th_elements[i].node.list); + ovs_mutex_unlock(aux->lock); + } + + thread_working_ms[id] = elapsed(&start); + ovs_barrier_block(&barrier); + + working = false; + + return NULL; +} + +static void +benchmark_list(void) +{ + struct ovs_mutex lock; + struct ovs_list list; + struct element *elem; + struct timeval start; + unsigned int counter; + bool work_complete; + pthread_t *threads; + struct list_aux aux; + uint64_t epoch; + uint64_t avg; + size_t i; + + memset(elements, 0, n_elems * sizeof *elements); + memset(thread_working_ms, 0, n_threads * sizeof *thread_working_ms); + + ovs_mutex_init(&lock); + ovs_list_init(&list); + + aux.list = &list; + aux.lock = &lock; + atomic_store(&aux.thread_id, 0); + + ovs_mutex_lock(&lock); + for (i = n_elems - (n_elems % n_threads); i < n_elems; i++) { + ovs_list_push_front(&list, &elements[i].node.list); + } + ovs_mutex_unlock(&lock); + + working = true; + + threads = xmalloc(n_threads * sizeof *threads); + ovs_barrier_init(&barrier, n_threads); + + for (i = 0; i < n_threads; i++) { + threads[i] = ovs_thread_create("locked_list_insert", + locked_list_insert_thread, &aux); + } + + xgettimeofday(&start); + + counter = 0; + epoch = 1; + do { + ovs_mutex_lock(&lock); + LIST_FOR_EACH_POP (elem, node.list, &list) { + elem->mark = epoch; + counter++; + } + ovs_mutex_unlock(&lock); + if (epoch == UINT64_MAX) { + epoch = 0; + } + epoch++; + } while (working); + + avg = 0; + for (i = 0; i < n_threads; i++) { + xpthread_join(threads[i], NULL); + avg += thread_working_ms[i]; + } + avg /= n_threads; + + /* Elements might have been inserted before threads were joined. */ + ovs_mutex_lock(&lock); + LIST_FOR_EACH_POP (elem, node.list, &list) { + elem->mark = epoch; + counter++; + } + ovs_mutex_unlock(&lock); + + printf(" list: %6d", elapsed(&start)); + for (i = 0; i < n_threads; i++) { + printf(" %6" PRIu64, thread_working_ms[i]); + } + printf(" %6" PRIu64 " ms\n", avg); + ovs_barrier_destroy(&barrier); + free(threads); + + work_complete = true; + for (i = 0; i < n_elems; i++) { + if (elements[i].mark == 0) { + printf("Element %" PRIuSIZE " was never consumed.\n", i); + work_complete = false; + } + } + ovs_assert(work_complete); + ovs_assert(counter == n_elems); +} + +struct guarded_list_aux { + struct guarded_list *glist; + atomic_uint thread_id; +}; + +static void * +guarded_list_insert_thread(void *aux_) +{ + unsigned int n_elems_per_thread; + struct element *th_elements; + struct guarded_list_aux *aux = aux_; + struct timeval start; + unsigned int id; + size_t i; + + atomic_add(&aux->thread_id, 1u, &id); + n_elems_per_thread = n_elems / n_threads; + th_elements = &elements[id * n_elems_per_thread]; + + ovs_barrier_block(&barrier); + xgettimeofday(&start); + + for (i = 0; i < n_elems_per_thread; i++) { + guarded_list_push_back(aux->glist, &th_elements[i].node.list, n_elems); + } + + thread_working_ms[id] = elapsed(&start); + ovs_barrier_block(&barrier); + + working = false; + + return NULL; +} + +static void +benchmark_guarded_list(void) +{ + struct guarded_list_aux aux; + struct ovs_list extracted; + struct guarded_list glist; + struct element *elem; + struct timeval start; + unsigned int counter; + bool work_complete; + pthread_t *threads; + uint64_t epoch; + uint64_t avg; + size_t i; + + memset(elements, 0, n_elems * sizeof *elements); + memset(thread_working_ms, 0, n_threads * sizeof *thread_working_ms); + + guarded_list_init(&glist); + ovs_list_init(&extracted); + + aux.glist = &glist; + atomic_store(&aux.thread_id, 0); + + for (i = n_elems - (n_elems % n_threads); i < n_elems; i++) { + guarded_list_push_back(&glist, &elements[i].node.list, n_elems); + } + + working = true; + + threads = xmalloc(n_threads * sizeof *threads); + ovs_barrier_init(&barrier, n_threads); + + for (i = 0; i < n_threads; i++) { + threads[i] = ovs_thread_create("guarded_list_insert", + guarded_list_insert_thread, &aux); + } + + xgettimeofday(&start); + + counter = 0; + epoch = 1; + do { + guarded_list_pop_all(&glist, &extracted); + LIST_FOR_EACH_POP (elem, node.list, &extracted) { + elem->mark = epoch; + counter++; + } + if (epoch == UINT64_MAX) { + epoch = 0; + } + epoch++; + } while (working); + + avg = 0; + for (i = 0; i < n_threads; i++) { + xpthread_join(threads[i], NULL); + avg += thread_working_ms[i]; + } + avg /= n_threads; + + /* Elements might have been inserted before threads were joined. */ + guarded_list_pop_all(&glist, &extracted); + LIST_FOR_EACH_POP (elem, node.list, &extracted) { + elem->mark = epoch; + counter++; + } + + printf("guarded list: %6d", elapsed(&start)); + for (i = 0; i < n_threads; i++) { + printf(" %6" PRIu64, thread_working_ms[i]); + } + printf(" %6" PRIu64 " ms\n", avg); + ovs_barrier_destroy(&barrier); + free(threads); + guarded_list_destroy(&glist); + + work_complete = true; + for (i = 0; i < n_elems; i++) { + if (elements[i].mark == 0) { + printf("Element %" PRIuSIZE " was never consumed.\n", i); + work_complete = false; + } + } + ovs_assert(work_complete); + ovs_assert(counter == n_elems); +} + +static void +run_benchmarks(struct ovs_cmdl_context *ctx) +{ + long int l_threads; + long int l_elems; + size_t i; + + l_elems = strtol(ctx->argv[1], NULL, 10); + l_threads = strtol(ctx->argv[2], NULL, 10); + ovs_assert(l_elems > 0 && l_threads > 0); + + n_elems = l_elems; + n_threads = l_threads; + + elements = xcalloc(n_elems, sizeof *elements); + thread_working_ms = xcalloc(n_threads, sizeof *thread_working_ms); + + printf("Benchmarking n=%u on 1 + %u threads.\n", n_elems, n_threads); + + printf(" type\\thread: Reader "); + for (i = 0; i < n_threads; i++) { + printf(" %3" PRIuSIZE " ", i + 1); + } + printf(" Avg\n"); + + benchmark_mpsc_queue(); + benchmark_list(); + benchmark_guarded_list(); + + free(thread_working_ms); + free(elements); +} + +static const struct ovs_cmdl_command commands[] = { + {"check", NULL, 0, 0, run_tests, OVS_RO}, + {"benchmark", " ", 2, 2, run_benchmarks, OVS_RO}, + {NULL, NULL, 0, 0, NULL, OVS_RO}, +}; + +static void +test_mpsc_queue_main(int argc, char *argv[]) +{ + struct ovs_cmdl_context ctx = { + .argc = argc - optind, + .argv = argv + optind, + }; + + set_program_name(argv[0]); + ovs_cmdl_run_command(&ctx, commands); +} + +OVSTEST_REGISTER("test-mpsc-queue", test_mpsc_queue_main); From patchwork Wed Feb 10 15:33:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439066 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=rJdgChs4; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=VW8Jt44R; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP276gCZz9sB4 for ; Thu, 11 Feb 2021 02:34:23 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7904286B5A; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yLH41fLY6AVH; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 2159586B12; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 086ACC0891; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id BB28DC013A for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id B8025866A3 for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Tt-dWJWCOsqd for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id DE16E867DA for ; Wed, 10 Feb 2021 15:34:15 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 75372547; Wed, 10 Feb 2021 10:34:15 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=4Kk/7rxs970IE aLomp6ToFrwZiWIdgoG0ACpXkzxoWw=; b=rJdgChs4msS9N9QV6SV6LWGA3nTN6 c4N8iF2bBmqh8j+AVWDszaLXyYrjk5VUBit67Re/o8D7TbL1ni+HnVS7lL+djqsO zZDT9ythoapjvN6s7TvGEeELMFSZI7PNie0gQn5o2oTH9+LyMTgk/khqk+wLMZhJ DhDI+4j3Hehqq7oTz+dcRGBg9gyDytRPPmQUo4Zq8FyBvwSSak7T/4Jv4Un7+Oul 8IwOjyn8hwho/WxledgtftNiCK33Tc8lA+7bZWuFS3m41fNGqWzDLeoaeJ8s4iWv OKVSLtxpjFb5zvchsG/t8pfk4rAZ1WPzZvYfqrLpfSTJFXtnspR8ukA8Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=4Kk/7rxs970IEaLomp6ToFrwZiWIdgoG0ACpXkzxoWw=; b=VW8Jt44R GuHBu1zHQYsw9X0ueSeoWb6vzd87EWxEysoUI4Hvcoa3NkwLPdI9RI4QJA7fQz/q APd5M8gbtWEX/dbCXGvWSCk28Sal3vSNlyK4a+Jn6DjAaY+Ed58Z/UF5b8U5fCmA Hxa5nU5mnaxfONyV47tT8TJlcttUVnszNE1P1qjxGpKW/yovjYpjHRYhNAykxNcE smkguirjTjWyoHMJbILoeMD8Jc+OGM4RR09uYMFW5nY/ZAjktTAhjgmp9xSFVvhE m4hdjB+JztYltMVCWHjGcbHXFdmSzN+ieFDEdHZiZ9lzadDHnpHb5Iz9l0VHrZkD 7+7mpgKNqQ7GGA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe duhfefieekieeiudetgfehkeeludektdekjeehudehffetieduheeigfekvdelfeenucff ohhmrghinheprghprggthhgvrdhorhhgnecukfhppeekiedrvdehgedrudeigedrudejge enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhi vhgvsehuvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 4DE5224005A; Wed, 10 Feb 2021 10:34:14 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:56 +0100 Message-Id: <80b89f3f0806b3124d3bb23754f99395d20308e0.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 10/23] llring: Add lockless MPMC bounded queue structure X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a lockless multi-producer/multi-consumer, array-based, non-intrusive, bounded queue that will fail on overflow. Each operation (enqueue, dequeue) uses a CAS(). As such, both producer and consumer sides guarantee lock-free forward progress. If the queue is full, enqueuing will fail. Conversely, if the queue is empty, dequeueing will fail. The bound of the queue are restricted to power-of-twos, to allow simpler overflow on unsigned position markers. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 2 + lib/llring.c | 153 ++++++++++++++++++++++++++++++++++++++++++++++++ lib/llring.h | 76 ++++++++++++++++++++++++ 3 files changed, 231 insertions(+) create mode 100644 lib/llring.c create mode 100644 lib/llring.h diff --git a/lib/automake.mk b/lib/automake.mk index cbdda460a..45948e519 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -156,6 +156,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/learn.h \ lib/learning-switch.c \ lib/learning-switch.h \ + lib/llring.c \ + lib/llring.h \ lib/lockfile.c \ lib/lockfile.h \ lib/mac-learning.c \ diff --git a/lib/llring.c b/lib/llring.c new file mode 100644 index 000000000..1fb930017 --- /dev/null +++ b/lib/llring.c @@ -0,0 +1,153 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "ovs-atomic.h" + +#include "llring.h" + +/* A queue element. + * Calling 'llring_create' will allocate an array of such elements, + * that will hold the inserted data. + */ +struct llring_node { + atomic_uint32_t seq; + uint32_t data; +}; + +/* A ring description. + * The head and tail of the ring are padded to avoid false-sharing, + * which improves slightly multi-thread performance, at the cost + * of some memory. + */ +struct llring { + PADDED_MEMBERS(CACHE_LINE_SIZE, atomic_uint32_t head;); + PADDED_MEMBERS(CACHE_LINE_SIZE, atomic_uint32_t tail;); + uint32_t mask; + struct llring_node nodes[0]; +}; + +struct llring * +llring_create(uint32_t size) +{ + struct llring *r; + uint32_t i; + + if (size < 2 || !IS_POW2(size)) { + return NULL; + } + + r = xmalloc(sizeof *r + size * sizeof r->nodes[0]); + + r->mask = size - 1; + for (i = 0; i < size; i++) { + atomic_store_relaxed(&r->nodes[i].seq, i); + } + atomic_store_relaxed(&r->head, 0); + atomic_store_relaxed(&r->tail, 0); + + return r; +} + +void +llring_destroy(struct llring *r) +{ + free(r); +} + +bool +llring_enqueue(struct llring *r, uint32_t data) +{ + struct llring_node *node; + uint32_t pos; + + atomic_read_relaxed(&r->head, &pos); + while (true) { + int64_t diff; + uint32_t seq; + + node = &r->nodes[pos & r->mask]; + atomic_read_explicit(&node->seq, &seq, memory_order_acquire); + diff = (int64_t) seq - (int64_t) pos; + + if (diff < 0) { + /* Current ring[head].seq is from previous ring generation, + * ring is full and enqueue fails. */ + return false; + } + + if (diff == 0) { + /* If head == ring[head].seq, then the slot is free, + * attempt to take it by moving the head, if no one moved it since. + */ + if (atomic_compare_exchange_weak_explicit(&r->head, &pos, pos + 1, + memory_order_relaxed, + memory_order_relaxed)) { + break; + } + } else { + /* Someone changed the head since last read, retry. */ + atomic_read_relaxed(&r->head, &pos); + } + } + + node->data = data; + atomic_store_explicit(&node->seq, pos + 1, memory_order_release); + return true; +} + +bool +llring_dequeue(struct llring *r, uint32_t *data) +{ + struct llring_node *node; + uint32_t pos; + + atomic_read_relaxed(&r->tail, &pos); + while (true) { + int64_t diff; + uint32_t seq; + + node = &r->nodes[pos & r->mask]; + atomic_read_explicit(&node->seq, &seq, memory_order_acquire); + diff = (int64_t) seq - (int64_t) (pos + 1); + + if (diff < 0) { + /* Current ring[tail + 1].seq is from previous ring generation, + * ring is empty and dequeue fails. */ + return false; + } + + if (diff == 0) { + /* If tail + 1 == ring[tail + 1].seq, then the slot is allocated, + * attempt to free it by moving the tail, if no one moved it since. + */ + if (atomic_compare_exchange_weak_explicit(&r->tail, &pos, pos + 1, + memory_order_relaxed, + memory_order_relaxed)) { + break; + } + } else { + /* Someone changed the tail since last read, retry. */ + atomic_read_relaxed(&r->tail, &pos); + } + } + + *data = node->data; + /* Advance the slot to next gen by adding r->mask + 1 to its sequence. */ + atomic_store_explicit(&node->seq, pos + r->mask + 1, memory_order_release); + return true; +} diff --git a/lib/llring.h b/lib/llring.h new file mode 100644 index 000000000..f97baa343 --- /dev/null +++ b/lib/llring.h @@ -0,0 +1,76 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "ovs-atomic.h" + +/* Bounded lockless queue + * ====================== + * + * A lockless FIFO queue bounded to a known size. + * Each operation (insert, remove) uses one CAS(). + * + * The structure is: + * + * Multi-producer: multiple threads can write to it + * concurrently. + * + * Multi-consumer: multiple threads can read from it + * concurrently. + * + * Bounded: the queue is backed by external memory. + * No new allocation is made on insertion, only the + * used elements in the queue are marked as such. + * The boundary of the queue is defined as the size given + * at init, which must be a power of two. + * + * Failing: when an operation (enqueue, dequeue) cannot + * be performed due to the queue being full/empty, the + * operation immediately fails, instead of waiting on + * a state change. + * + * Non-intrusive: queue elements are allocated prior to + * initialization. Data is shallow-copied to those + * allocated elements. + * + * Thread safety + * ============= + * + * The queue is thread-safe for MPMC case. + * No lock is taken by the queue. The queue guarantees + * lock-free forward progress for each of its operations. + * + */ + +/* Create a circular lockless ring. + * The 'size' parameter must be a power-of-two higher than 2, + * otherwise allocation will fail. + */ +struct llring; +struct llring *llring_create(uint32_t size); + +/* Free a lockless ring. */ +void llring_destroy(struct llring *r); + +/* 'data' is copied to the latest free slot in the queue. */ +bool llring_enqueue(struct llring *r, uint32_t data); + +/* The value within the oldest slot taken in the queue is copied + * to the address pointed by 'data'. + */ +bool llring_dequeue(struct llring *r, uint32_t *data); From patchwork Wed Feb 10 15:33:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439080 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=Ffc+NyGX; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=oOuecTaY; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP481k5Nz9sB4 for ; Thu, 11 Feb 2021 02:36:08 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id B2E1586D8D; Wed, 10 Feb 2021 15:36:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ulfOXhW2Hur5; Wed, 10 Feb 2021 15:35:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 241A287070; Wed, 10 Feb 2021 15:34:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DC497C1E73; Wed, 10 Feb 2021 15:34:35 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7E677C1E70 for ; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 623B286FA6 for ; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x8AGtsoXkK41 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id A79CF86B16 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id C966DA1E; Wed, 10 Feb 2021 10:34:16 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=UI3SZ4owHN8FV vJ4Z+b6wrgq4PQsYs8CzWujrVb2k9g=; b=Ffc+NyGXxbECWkSLZSc+UHlPrzC64 LUJuzJN0H8LODU6bYlFcA02IJWgEttf2RY2IJU9+uGO+75vgSXa0Y6L0zckaDN+K INb+9pyI4tkyhh1SB0J1xmnCnhJCJlbkpufP7eypKmIe/Ryv9QJNmGfmx4sihM62 e8gAfdICIOzwIqBif8xRvWv8JdBHlTljdyV+nVGJxJrxKZmGreSEZdS/Coq9KQfl AeIQu6sZ4uzKa56cFn3glrLplsKG9nvdY7ZDWBLY/vVmM/XvZMu0GVg9EwAzWKlt T0n8NK1+TgtKe9opWcTn81jU2nTHWkdSuBFLmfmDdJW/m+gh+sOn/9p4A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=UI3SZ4owHN8FVvJ4Z+b6wrgq4PQsYs8CzWujrVb2k9g=; b=oOuecTaY V2vzyFzF8WVXRac3toFt72TDhpNseTNIbgI2ULJmaiK/0GJApizOKXD+7358HEj4 pPGInDGDEX0zbWYZq3r6edsOmM03XdNIfASQ2YL4MgVAVPIRKOe8OV4V8N6YY0dJ 3qtC/xNGMP+/uK6b3QCnKIggp0ssp58jJbizBcF5Rorzv1uaJNSdAwDiWmtag0xz KwJzmHomnsS9A4CVD/dXH7XuNXLxeMkvIH3iQexKEYQ7X/1NC0O6cHB7MV1588Sy l1nGGOPVioFyhSVJz16njp3RpkhuDh/SP0owCPqO65Q9Q3CZEm0MGQmO9Xqu/WcT AISzxlt8JcFMtg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe duhfefieekieeiudetgfehkeeludektdekjeehudehffetieduheeigfekvdelfeenucff ohhmrghinheprghprggthhgvrdhorhhgnecukfhppeekiedrvdehgedrudeigedrudejge enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrhhi vhgvsehuvdehiedrnhgvth X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id D4194240064; Wed, 10 Feb 2021 10:34:14 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:57 +0100 Message-Id: <6e199c03be14af89cddcd490e08f9dde10a0c5cd.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 11/23] seq-pool: Module for faster ID generation X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The current id-pool module is slow to allocate the next valid ID, and can be optimized when restricting some properties of the pool. Those restrictions are: * No ability to add a random ID to the pool. * A new ID is no more the smallest possible ID. It is however guaranteed to be in the range of [base, next_id]. Multiple users of the pool are registered, each with a thread-local cache for better scalability and the next_id is one after the latest ID added to any user cache. The allocation range can be written as: [base, last_alloc + nb-user * cache-size + 1]. * A user should never free an ID that is not allocated. No checks are done and doing so will duplicate the spurious ID. Refcounting or other memory management scheme should be used to ensure an object and its ID are only freed once. This pool is designed to scale reasonably well in multi-thread setup. As it is aimed at being a faster replacement to the current id-pool, a benchmark has been implemented alongside unit tests. The benchmark is composed of 4 rounds: 'new', 'del', 'mix', and 'rnd'. Respectively + 'new': only allocate IDs + 'del': only free IDs + 'mix': allocate, sequential free, then allocate ID. + 'rnd': allocate, random free, allocate ID. Randomized freeing is done by swapping the latest allocated ID with any from the range of currently allocated ID, which is reminiscent of the Fisher-Yates shuffle. This evaluates freeing non-sequential IDs, which is the more natural use-case. For this specific round, the id-pool performance is such that a timeout of 10 seconds is added to the benchmark: $ ./tests/ovstest test-seq-pool benchmark 10000 1 Benchmarking n=10000 on 1 thread. type\thread: 1 Avg seq-pool new: 1 1 ms seq-pool del: 0 0 ms seq-pool mix: 1 1 ms seq-pool rnd: 1 1 ms id-pool new: 0 0 ms id-pool del: 1 1 ms id-pool mix: 1 1 ms id-pool rnd: 1201 1201 ms $ ./tests/ovstest test-seq-pool benchmark 100000 1 Benchmarking n=100000 on 1 thread. type\thread: 1 Avg seq-pool new: 2 2 ms seq-pool del: 5 5 ms seq-pool mix: 5 5 ms seq-pool rnd: 5 5 ms id-pool new: 8 8 ms id-pool del: 5 5 ms id-pool mix: 11 11 ms id-pool rnd: 10000+ ****** ms $ ./tests/ovstest test-seq-pool benchmark 1000000 1 Benchmarking n=1000000 on 1 thread. type\thread: 1 Avg seq-pool new: 23 23 ms seq-pool del: 49 49 ms seq-pool mix: 53 53 ms seq-pool rnd: 53 53 ms id-pool new: 190 190 ms id-pool del: 173 173 ms id-pool mix: 273 273 ms id-pool rnd: 10042+ ****** ms $ ./tests/ovstest test-seq-pool benchmark 1000000 2 Benchmarking n=1000000 on 2 threads. type\thread: 1 2 Avg seq-pool new: 40 39 39 ms seq-pool del: 33 33 33 ms seq-pool mix: 89 91 90 ms seq-pool rnd: 146 151 148 ms id-pool new: 485 485 485 ms id-pool del: 541 542 541 ms id-pool mix: 550 600 575 ms id-pool rnd: 10048+ 10003+ ****** ms $ ./tests/ovstest test-seq-pool benchmark 1000000 4 Benchmarking n=1000000 on 4 threads. type\thread: 1 2 3 4 Avg seq-pool new: 40 39 40 40 39 ms seq-pool del: 24 28 28 30 27 ms seq-pool mix: 60 63 69 69 65 ms seq-pool rnd: 195 197 202 202 199 ms id-pool new: 478 471 482 485 479 ms id-pool del: 474 469 467 474 471 ms id-pool mix: 558 558 611 545 568 ms id-pool rnd: 10121+ 10076+ 10030+ 10167+ ****** ms Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/automake.mk | 2 + lib/seq-pool.c | 198 +++++++++++++++ lib/seq-pool.h | 66 +++++ tests/automake.mk | 1 + tests/library.at | 5 + tests/test-seq-pool.c | 543 ++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 815 insertions(+) create mode 100644 lib/seq-pool.c create mode 100644 lib/seq-pool.h create mode 100644 tests/test-seq-pool.c diff --git a/lib/automake.mk b/lib/automake.mk index 45948e519..2577c6130 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -295,6 +295,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/sat-math.h \ lib/seq.c \ lib/seq.h \ + lib/seq-pool.c \ + lib/seq-pool.h \ lib/sha1.c \ lib/sha1.h \ lib/shash.c \ diff --git a/lib/seq-pool.c b/lib/seq-pool.c new file mode 100644 index 000000000..4426d11d8 --- /dev/null +++ b/lib/seq-pool.c @@ -0,0 +1,198 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "openvswitch/list.h" +#include "openvswitch/thread.h" +#include "openvswitch/util.h" +#include "ovs-atomic.h" +#include "llring.h" +#include "seq-pool.h" + +#define SEQPOOL_CACHE_SIZE 32 +BUILD_ASSERT_DECL(IS_POW2(SEQPOOL_CACHE_SIZE)); + +struct seq_node { + struct ovs_list list_node; + uint32_t id; +}; + +struct seq_pool { + uint32_t next_id; + struct llring **cache; /* per-user id cache. */ + size_t nb_user; /* Number of user threads. */ + struct ovs_mutex lock; /* Protects free_ids access. */ + struct ovs_list free_ids; /* Set of currently free IDs. */ + uint32_t base; /* IDs in the range of [base, base + n_ids). */ + uint32_t n_ids; /* Total number of ids in the pool. */ +}; + +struct seq_pool * +seq_pool_create(unsigned int nb_user, uint32_t base, uint32_t n_ids) +{ + struct seq_pool *pool; + size_t i; + + ovs_assert(nb_user != 0); + ovs_assert(base <= UINT32_MAX - n_ids); + + pool = xmalloc(sizeof *pool); + + pool->cache = xcalloc(nb_user, sizeof *pool->cache); + for (i = 0; i < nb_user; i++) { + pool->cache[i] = llring_create(SEQPOOL_CACHE_SIZE); + } + pool->nb_user = nb_user; + + pool->next_id = base; + pool->base = base; + pool->n_ids = n_ids; + + ovs_mutex_init(&pool->lock); + ovs_list_init(&pool->free_ids); + + return pool; +} + +void +seq_pool_destroy(struct seq_pool *pool) +{ + struct seq_node *node; + struct seq_node *next; + size_t i; + + if (!pool) { + return; + } + + ovs_mutex_lock(&pool->lock); + LIST_FOR_EACH_SAFE (node, next, list_node, &pool->free_ids) { + free(node); + } + ovs_list_poison(&pool->free_ids); + ovs_mutex_unlock(&pool->lock); + ovs_mutex_destroy(&pool->lock); + + for (i = 0; i < pool->nb_user; i++) { + llring_destroy(pool->cache[i]); + } + free(pool->cache); + + free(pool); +} + +bool +seq_pool_new_id(struct seq_pool *pool, unsigned int uid, uint32_t *id) +{ + struct llring *cache; + struct ovs_list *front; + struct seq_node *node; + + uid %= pool->nb_user; + cache = pool->cache[uid]; + + if (llring_dequeue(cache, id)) { + return true; + } + + ovs_mutex_lock(&pool->lock); + + while (!ovs_list_is_empty(&pool->free_ids)) { + front = ovs_list_front(&pool->free_ids); + node = CONTAINER_OF(front, struct seq_node, list_node); + if (llring_enqueue(cache, node->id)) { + ovs_list_remove(front); + free(node); + } else { + break; + } + } + + while (pool->next_id < pool->base + pool->n_ids) { + if (llring_enqueue(cache, pool->next_id)) { + pool->next_id++; + } else { + break; + } + } + + ovs_mutex_unlock(&pool->lock); + + if (llring_dequeue(cache, id)) { + return true; + } else { + struct llring *c2; + size_t i; + + /* If no ID was available either from shared counter, + * free-list or local cache, steal an ID from another + * user cache. + */ + for (i = 0; i < pool->nb_user; i++) { + if (i == uid) { + continue; + } + c2 = pool->cache[i]; + if (llring_dequeue(c2, id)) { + return true; + } + } + } + + return false; +} + +void +seq_pool_free_id(struct seq_pool *pool, unsigned int uid, uint32_t id) +{ + struct seq_node *nodes[SEQPOOL_CACHE_SIZE + 1]; + struct llring *cache; + uint32_t node_id; + size_t i; + + if (id < pool->base || id >= pool->base + pool->n_ids) { + return; + } + + uid %= pool->nb_user; + cache = pool->cache[uid]; + + if (llring_enqueue(cache, id)) { + return; + } + + /* Flush the cache. */ + for (i = 0; llring_dequeue(cache, &node_id); i++) { + nodes[i] = xmalloc(sizeof *nodes[i]); + nodes[i]->id = node_id; + } + + /* Finish with the last freed node. */ + nodes[i] = xmalloc(sizeof **nodes); + nodes[i]->id = id; + i++; + + if (i < ARRAY_SIZE(nodes)) { + nodes[i] = NULL; + } + + ovs_mutex_lock(&pool->lock); + for (i = 0; i < ARRAY_SIZE(nodes) && nodes[i] != NULL; i++) { + ovs_list_push_back(&pool->free_ids, &nodes[i]->list_node); + } + ovs_mutex_unlock(&pool->lock); +} diff --git a/lib/seq-pool.h b/lib/seq-pool.h new file mode 100644 index 000000000..c992a0988 --- /dev/null +++ b/lib/seq-pool.h @@ -0,0 +1,66 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef SEQ_POOL_H +#define SEQ_POOL_H + +#include +#include +#include + +/* + * Sequential ID pool. + * =================== + * + * Pool of unique 32bits IDs. + * + * Multiple users are registered at initialization. Each user uses a cache + * of ID. When each thread using the pool uses its own user ID, the pool + * scales reasonably for concurrent allocation. + * + * New IDs are always in the range of '[base, next_id]', where 'next_id' is + * in the range of '[last_alloc_ID + nb_user * cache_size + 1]'. + * This means that a new ID is not always the smallest available ID, but it is + * still from a limited range. + * + * Users should ensure that an ID is *never* freed twice. Not doing so will + * have the effect of double-allocating such ID afterward. + * + * Thread-safety + * ============= + * + * APIs are thread safe. + * + * Multiple threads can share the same user ID if necessary, but it can hurt + * performance if threads are not otherwise synchronized. + */ + +struct seq_pool; + +/* nb_user is the number of expected users of the pool, + * in terms of execution threads. */ +struct seq_pool *seq_pool_create(unsigned int nb_user, + uint32_t base, uint32_t n_ids); +void seq_pool_destroy(struct seq_pool *pool); + +/* uid is the thread user-id. It should be within '[0, nb_user['. */ +bool seq_pool_new_id(struct seq_pool *pool, unsigned int uid, uint32_t *id); + +/* uid is the thread user-id. It should be within '[0, nb_user['. + * An allocated ID must *never* be freed twice. + */ +void seq_pool_free_id(struct seq_pool *pool, unsigned int uid, uint32_t id); +#endif /* seq-pool.h */ diff --git a/tests/automake.mk b/tests/automake.mk index d7ae5df90..f34016a24 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -469,6 +469,7 @@ tests_ovstest_SOURCES = \ tests/test-rcu.c \ tests/test-reconnect.c \ tests/test-rstp.c \ + tests/test-seq-pool.c \ tests/test-sflow.c \ tests/test-sha1.c \ tests/test-skiplist.c \ diff --git a/tests/library.at b/tests/library.at index ac24921f8..a312a448f 100644 --- a/tests/library.at +++ b/tests/library.at @@ -259,3 +259,8 @@ AT_SETUP([mpsc-queue module]) AT_CHECK([ovstest test-mpsc-queue check], [0], [.. ]) AT_CLEANUP + +AT_SETUP([seq-pool module]) +AT_CHECK([ovstest test-seq-pool check], [0], [.... +]) +AT_CLEANUP diff --git a/tests/test-seq-pool.c b/tests/test-seq-pool.c new file mode 100644 index 000000000..a1e0d5500 --- /dev/null +++ b/tests/test-seq-pool.c @@ -0,0 +1,543 @@ +/* + * Copyright (c) 2020 NVIDIA Corporation. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#undef NDEBUG +#include +#include +#include + +#include + +#include "command-line.h" +#include "id-pool.h" +#include "openvswitch/util.h" +#include "ovs-thread.h" +#include "ovs-rcu.h" +#include "ovstest.h" +#include "random.h" +#include "seq-pool.h" +#include "timeval.h" +#include "util.h" + +#define SEQ_POOL_CACHE_SIZE 32 + +#define N_IDS 100 + +static void +test_seq_pool_alloc_full_range(void) +{ + bool ids[N_IDS]; + struct seq_pool *pool; + size_t i; + + memset(ids, 0, sizeof ids); + pool = seq_pool_create(1, 0, N_IDS); + + for (i = 0; i < N_IDS; i++) { + uint32_t id; + + ovs_assert(seq_pool_new_id(pool, 0, &id)); + /* No double alloc.*/ + ovs_assert(ids[id] == false); + ids[id] = true; + } + + for (i = 0; i < N_IDS; i++) { + ovs_assert(ids[i]); + } + + seq_pool_destroy(pool); + printf("."); +} + +static void +test_seq_pool_alloc_steal(void) +{ + /* N must be less than a pool cache size to force the second user + * to steal from the first. + */ +#define N (SEQ_POOL_CACHE_SIZE / 4) + bool ids[N]; + struct seq_pool *pool; + uint32_t id; + size_t i; + + memset(ids, 0, sizeof ids); + pool = seq_pool_create(2, 0, N); + + /* Fill up user 0 cache. */ + ovs_assert(seq_pool_new_id(pool, 0, &id)); + for (i = 0; i < N - 1; i++) { + /* Check that user 1 can still alloc from user 0 cache. */ + ovs_assert(seq_pool_new_id(pool, 1, &id)); + } + + seq_pool_destroy(pool); + printf("."); +#undef N +} + +static void +test_seq_pool_alloc_monotonic(void) +{ + uint32_t ids[N_IDS]; + struct seq_pool *pool; + size_t i; + + memset(ids, 0, sizeof ids); + pool = seq_pool_create(1, 0, N_IDS); + + for (i = 0; i < N_IDS; i++) { + ovs_assert(seq_pool_new_id(pool, 0, &ids[i])); + ovs_assert(ids[i] == i); + } + + seq_pool_destroy(pool); + printf("."); +} + +static void +test_seq_pool_alloc_under_limit(void) +{ + uint32_t ids[N_IDS]; + unsigned int limit; + struct seq_pool *pool; + size_t i; + + memset(ids, 0, sizeof ids); + pool = seq_pool_create(1, 0, N_IDS); + + for (limit = 1; limit < N_IDS; limit++) { + /* Allocate until arbitrary limit then free allocated ids. */ + for (i = 0; i < limit; i++) { + ovs_assert(seq_pool_new_id(pool, 0, &ids[i])); + } + for (i = 0; i < limit; i++) { + seq_pool_free_id(pool, 0, ids[i]); + } + /* Verify that the N='limit' next allocations are under limit. */ + for (i = 0; i < limit; i++) { + ovs_assert(seq_pool_new_id(pool, 0, &ids[i])); + ovs_assert(ids[i] < limit + SEQ_POOL_CACHE_SIZE); + } + for (i = 0; i < limit; i++) { + seq_pool_free_id(pool, 0, ids[i]); + } + } + + seq_pool_destroy(pool); + printf("."); +} + +static void +run_tests(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + /* Check that all ids can be allocated. */ + test_seq_pool_alloc_full_range(); + /* Check that all ids can be allocated with multiple users. */ + test_seq_pool_alloc_steal(); + /* Check that id allocation is always increasing. */ + test_seq_pool_alloc_monotonic(); + /* Check that id allocation stays under some limit. */ + test_seq_pool_alloc_under_limit(); + printf("\n"); +} + +static uint32_t *ids; +static uint64_t *thread_working_ms; /* Measured work time. */ + +static unsigned int n_threads; +static unsigned int n_ids; + +static struct ovs_barrier barrier; + +#define TIMEOUT_MS (10 * 1000) /* 10 sec timeout */ +static int running_time_ms; +static volatile bool stop = false; + +static int +elapsed(int *start) +{ + return running_time_ms - *start; +} + +static void +swap_u32(uint32_t *a, uint32_t *b) +{ + uint32_t t; + t = *a; + *a = *b; + *b = t; +} + +static void +shuffle(uint32_t *p, size_t n) +{ + for (; n > 1; n--, p++) { + uint32_t *q = &p[random_range(n)]; + swap_u32(p, q); + } +} + +static void +print_result(const char *prefix) +{ + uint64_t avg; + size_t i; + + avg = 0; + for (i = 0; i < n_threads; i++) { + avg += thread_working_ms[i]; + } + avg /= n_threads; + printf("%s: ", prefix); + for (i = 0; i < n_threads; i++) { + if (thread_working_ms[i] >= TIMEOUT_MS) { + printf("%6" PRIu64 "+", thread_working_ms[i]); + } else { + printf(" %6" PRIu64, thread_working_ms[i]); + } + } + if (avg >= TIMEOUT_MS) { + printf(" ****** ms\n"); + } else { + printf(" %6" PRIu64 " ms\n", avg); + } +} + +struct seq_pool_aux { + struct seq_pool *pool; + atomic_uint thread_id; +}; + +static void * +seq_pool_thread(void *aux_) +{ + unsigned int n_ids_per_thread; + struct seq_pool_aux *aux = aux_; + uint32_t *th_ids; + unsigned int tid; + int start; + size_t i; + + atomic_add(&aux->thread_id, 1u, &tid); + n_ids_per_thread = n_ids / n_threads; + th_ids = &ids[tid * n_ids_per_thread]; + + /* NEW / ALLOC */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + ignore(seq_pool_new_id(aux->pool, tid, &th_ids[i])); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* DEL */ + + shuffle(th_ids, n_ids_per_thread); + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + seq_pool_free_id(aux->pool, tid, th_ids[i]); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* MIX */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + ignore(seq_pool_new_id(aux->pool, tid, &th_ids[i])); + seq_pool_free_id(aux->pool, tid, th_ids[i]); + ignore(seq_pool_new_id(aux->pool, tid, &th_ids[i])); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* MIX SHUFFLED */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + if (elapsed(&start) >= TIMEOUT_MS) { + break; + } + ignore(seq_pool_new_id(aux->pool, tid, &th_ids[i])); + swap_u32(&th_ids[i], &th_ids[random_range(i + 1)]); + seq_pool_free_id(aux->pool, tid, th_ids[i]); + ignore(seq_pool_new_id(aux->pool, tid, &th_ids[i])); + } + thread_working_ms[tid] = elapsed(&start); + + return NULL; +} + +static void +benchmark_seq_pool(void) +{ + pthread_t *threads; + struct seq_pool_aux aux; + size_t i; + + memset(ids, 0, n_ids & sizeof *ids); + memset(thread_working_ms, 0, n_threads & sizeof *thread_working_ms); + + aux.pool = seq_pool_create(n_threads, 0, n_ids); + atomic_store(&aux.thread_id, 0); + + for (i = n_ids - (n_ids % n_threads); i < n_ids; i++) { + uint32_t id; + + seq_pool_new_id(aux.pool, 0, &id); + ids[i] = id; + } + + threads = xmalloc(n_threads * sizeof *threads); + ovs_barrier_init(&barrier, n_threads + 1); + + for (i = 0; i < n_threads; i++) { + threads[i] = ovs_thread_create("seq_pool_alloc", + seq_pool_thread, &aux); + } + + ovs_barrier_block(&barrier); + + print_result("seq-pool new"); + + ovs_barrier_block(&barrier); + + print_result("seq-pool del"); + + ovs_barrier_block(&barrier); + + print_result("seq-pool mix"); + + for (i = 0; i < n_threads; i++) { + xpthread_join(threads[i], NULL); + } + + print_result("seq-pool rnd"); + + seq_pool_destroy(aux.pool); + ovs_barrier_destroy(&barrier); + free(threads); +} + +struct id_pool_aux { + struct id_pool *pool; + struct ovs_mutex *lock; + atomic_uint thread_id; +}; + +static void * +id_pool_thread(void *aux_) +{ + unsigned int n_ids_per_thread; + struct id_pool_aux *aux = aux_; + uint32_t *th_ids; + unsigned int tid; + int start; + size_t i; + + atomic_add(&aux->thread_id, 1u, &tid); + n_ids_per_thread = n_ids / n_threads; + th_ids = &ids[tid * n_ids_per_thread]; + + /* NEW */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + ovs_mutex_lock(aux->lock); + ovs_assert(id_pool_alloc_id(aux->pool, &th_ids[i])); + ovs_mutex_unlock(aux->lock); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* DEL */ + + shuffle(th_ids, n_ids_per_thread); + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + ovs_mutex_lock(aux->lock); + id_pool_free_id(aux->pool, th_ids[i]); + ovs_mutex_unlock(aux->lock); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* MIX */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + ovs_mutex_lock(aux->lock); + ignore(id_pool_alloc_id(aux->pool, &th_ids[i])); + id_pool_free_id(aux->pool, th_ids[i]); + ignore(id_pool_alloc_id(aux->pool, &th_ids[i])); + ovs_mutex_unlock(aux->lock); + } + thread_working_ms[tid] = elapsed(&start); + + ovs_barrier_block(&barrier); + + /* MIX SHUFFLED */ + + start = running_time_ms; + for (i = 0; i < n_ids_per_thread; i++) { + if (elapsed(&start) >= TIMEOUT_MS) { + break; + } + ovs_mutex_lock(aux->lock); + ignore(id_pool_alloc_id(aux->pool, &th_ids[i])); + swap_u32(&th_ids[i], &th_ids[random_range(i + 1)]); + id_pool_free_id(aux->pool, th_ids[i]); + ignore(id_pool_alloc_id(aux->pool, &th_ids[i])); + ovs_mutex_unlock(aux->lock); + } + thread_working_ms[tid] = elapsed(&start); + + return NULL; +} + +static void +benchmark_id_pool(void) +{ + pthread_t *threads; + struct id_pool_aux aux; + struct ovs_mutex lock; + size_t i; + + memset(ids, 0, n_ids & sizeof *ids); + memset(thread_working_ms, 0, n_threads & sizeof *thread_working_ms); + + aux.pool = id_pool_create(0, n_ids); + aux.lock = &lock; + ovs_mutex_init(&lock); + atomic_store(&aux.thread_id, 0); + + for (i = n_ids - (n_ids % n_threads); i < n_ids; i++) { + id_pool_alloc_id(aux.pool, &ids[i]); + } + + threads = xmalloc(n_threads * sizeof *threads); + ovs_barrier_init(&barrier, n_threads + 1); + + for (i = 0; i < n_threads; i++) { + threads[i] = ovs_thread_create("id_pool_alloc", id_pool_thread, &aux); + } + + ovs_barrier_block(&barrier); + + print_result(" id-pool new"); + + ovs_barrier_block(&barrier); + + print_result(" id-pool del"); + + ovs_barrier_block(&barrier); + + print_result(" id-pool mix"); + + for (i = 0; i < n_threads; i++) { + xpthread_join(threads[i], NULL); + } + + print_result(" id-pool rnd"); + + id_pool_destroy(aux.pool); + ovs_barrier_destroy(&barrier); + free(threads); +} + +static void * +clock_main(void *arg OVS_UNUSED) +{ + struct timeval start; + struct timeval end; + + xgettimeofday(&start); + while (!stop) { + xgettimeofday(&end); + running_time_ms = timeval_to_msec(&end) - timeval_to_msec(&start); + xnanosleep(1000); + } + + return NULL; +} + +static void +run_benchmarks(struct ovs_cmdl_context *ctx) +{ + pthread_t clock; + long int l_threads; + long int l_ids; + size_t i; + + l_ids = strtol(ctx->argv[1], NULL, 10); + l_threads = strtol(ctx->argv[2], NULL, 10); + ovs_assert(l_ids > 0 && l_threads > 0); + + n_ids = l_ids; + n_threads = l_threads; + + ids = xcalloc(n_ids, sizeof *ids); + thread_working_ms = xcalloc(n_threads, sizeof *thread_working_ms); + + clock = ovs_thread_create("clock", clock_main, NULL); + + printf("Benchmarking n=%u on %u thread%s.\n", n_ids, n_threads, + n_threads > 1 ? "s" : ""); + + printf(" type\\thread: "); + for (i = 0; i < n_threads; i++) { + printf(" %3" PRIuSIZE " ", i + 1); + } + printf(" Avg\n"); + + benchmark_seq_pool(); + benchmark_id_pool(); + + stop = true; + + free(thread_working_ms); + xpthread_join(clock, NULL); +} + +static const struct ovs_cmdl_command commands[] = { + {"check", NULL, 0, 0, run_tests, OVS_RO}, + {"benchmark", " ", 2, 2, run_benchmarks, OVS_RO}, + {NULL, NULL, 0, 0, NULL, OVS_RO}, +}; + +static void +test_seq_pool_main(int argc, char *argv[]) +{ + struct ovs_cmdl_context ctx = { + .argc = argc - optind, + .argv = argv + optind, + }; + + set_program_name(argv[0]); + ovs_cmdl_run_command(&ctx, commands); +} + +OVSTEST_REGISTER("test-seq-pool", test_seq_pool_main); From patchwork Wed Feb 10 15:33:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439075 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=y314Qnx9; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=VEeJS0ZL; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2b3gR3z9sB4 for ; Thu, 11 Feb 2021 02:34:47 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 594596F5CD for ; Wed, 10 Feb 2021 15:34:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P1AHznc2-BRL for ; Wed, 10 Feb 2021 15:34:43 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 7403B6F8C8; Wed, 10 Feb 2021 15:34:43 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTP id 54E336F67B; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 34CBAC0891; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id AF9BCC0891 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 97574866A3 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SnSKVqEbucQO for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id B6DC1861CF for ; Wed, 10 Feb 2021 15:34:16 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 5DDACE21; Wed, 10 Feb 2021 10:34:16 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=JZa2x9D4RCZRU OkeMsr68oXC17B3/4HBRvNxij5vC5k=; b=y314Qnx9dztypvxLlAX8d5SZDnCtj bFyjL+nr5cFyXejl1V1rTEgCV+uZoZbTzd1KKYY4EvsixgFaz7D4jHyEC/bKgIr7 kTwI9p6h2xGSaLIBTp7R6Hx3nDD9ix1dc/yx9dk/3ZIT4M/v9Neb6a2kNqtmmMS2 pG0K+D8xohr97N1zWDh9klNW+H5HlV1PPKqORLQxJrhTUsWF470cAiSMHW9xb4s+ JT+ceSzj09XO4pLAmCjI7TRC5e4n6yuN3RyNm87V79qg4rs4lxs28gbZJnl2Woyh 5aSSrH8lLlaXYbX/YWqLMsycmjVTHjlDNiFZSVclOVgakCHObbSwSaIvA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=JZa2x9D4RCZRUOkeMsr68oXC17B3/4HBRvNxij5vC5k=; b=VEeJS0ZL nEixDKZDZVO2CC2gbABvGfPgl6THwNCZYQrHxY6gKYEWHJ3H849vpgXcxJlfMkX/ TmR7/4qs5NOEJvOvVQ8mN+jkga2wewnIJxZQmQw6Ocxo42n1pXuREUUbXM+deNzj +RlB2e7S9vB9zml70j3CuyhFehPTZt6vpdbE1aHAWmakBqHd0iECMZvbFeIxYQrd GNtdvFqdg+idwN+eEVEh8QiX3zpkKH8BFi7X4cEkI9cRxg6IfMGvFax7ghdP+iI+ /1s8pPgnsm5zVt30DwgykkxUn939Wl9N3KEgDFXjiOvrRixRDNXN9auCnUQO96r1 DOQ1M74SMTvQog== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 66EFF24005D; Wed, 10 Feb 2021 10:34:15 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:58 +0100 Message-Id: <289c99b5e10631ab6c28c9505667706484888f56.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 12/23] netdev-offload: Add multi-thread API X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Expose functions reporting user configuration of offloading threads, as well as utility functions for multithreading. This will only expose the configuration knob to the user, while no datapath will implement the multiple thread request. This will allow implementations to use this API for offload thread management in relevant layers before enabling the actual dataplane implementation. The offload thread ID is lazily allocated and can as such be in a different order than the offload thread start sequence. The RCU thread will sometime access hardware-offload objects from a provider for reclamation purposes. In such case, it will get a default offload thread ID of 0. Care must be taken that using this thread ID is safe concurrently with the offload threads. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-provider.h | 1 + lib/netdev-offload.c | 88 ++++++++++++++++++++++++++++++++++- lib/netdev-offload.h | 19 ++++++++ vswitchd/vswitch.xml | 16 +++++++ 4 files changed, 122 insertions(+), 2 deletions(-) diff --git a/lib/netdev-offload-provider.h b/lib/netdev-offload-provider.h index 4815a1bd2..f8b66961d 100644 --- a/lib/netdev-offload-provider.h +++ b/lib/netdev-offload-provider.h @@ -84,6 +84,7 @@ struct netdev_flow_api { struct dpif_flow_stats *); /* Get the number of flows offloaded to netdev. + * 'n_flows' is an array of counters, one per offload thread. * Return 0 if successful, otherwise returns a positive errno value. */ int (*flow_get_n_flows)(struct netdev *, uint64_t *n_flows); diff --git a/lib/netdev-offload.c b/lib/netdev-offload.c index 3092f8d55..c30377b91 100644 --- a/lib/netdev-offload.c +++ b/lib/netdev-offload.c @@ -60,6 +60,12 @@ VLOG_DEFINE_THIS_MODULE(netdev_offload); static bool netdev_flow_api_enabled = false; +#define DEFAULT_OFFLOAD_THREAD_NB 1 +#define MAX_OFFLOAD_THREAD_NB 10 + +static unsigned int offload_thread_nb = DEFAULT_OFFLOAD_THREAD_NB; +DEFINE_EXTERN_PER_THREAD_DATA(netdev_offload_thread_id, OVSTHREAD_ID_UNSET); + /* Protects 'netdev_flow_apis'. */ static struct ovs_mutex netdev_flow_api_provider_mutex = OVS_MUTEX_INITIALIZER; @@ -436,6 +442,64 @@ netdev_is_flow_api_enabled(void) return netdev_flow_api_enabled; } +unsigned int +netdev_offload_thread_nb(void) +{ + return offload_thread_nb; +} + +unsigned int +netdev_offload_ufid_to_thread_id(const ovs_u128 ufid) +{ + uint32_t ufid_hash; + + if (netdev_offload_thread_nb() == 1) { + return 0; + } + + ufid_hash = hash_words64_inline( + (const uint64_t [2]){ ufid.u64.lo, + ufid.u64.hi }, 2, 1); + return ufid_hash % netdev_offload_thread_nb(); +} + +unsigned int +netdev_offload_thread_init(void) +{ + static atomic_count next_id = ATOMIC_COUNT_INIT(0); + bool thread_is_hw_offload; + bool thread_is_rcu; + + thread_is_hw_offload = !strncmp(get_subprogram_name(), + "hw_offload", strlen("hw_offload")); + thread_is_rcu = !strncmp(get_subprogram_name(), "urcu", strlen("urcu")); + + /* Panic if any other thread besides offload and RCU tries + * to initialize their thread ID. */ + ovs_assert(thread_is_hw_offload || thread_is_rcu); + + if (*netdev_offload_thread_id_get() == OVSTHREAD_ID_UNSET) { + unsigned int id; + + if (thread_is_rcu) { + /* RCU will compete with other threads for shared object access. + * Reclamation functions using a thread ID must be thread-safe. + * For that end, and because RCU must consider all potential shared + * objects anyway, its thread-id can be whichever, so return 0. + */ + id = 0; + } else { + /* Only the actual offload threads have their own ID. */ + id = atomic_count_inc(&next_id); + } + /* Panic if any offload thread is getting a spurious ID. */ + ovs_assert(id < netdev_offload_thread_nb()); + return *netdev_offload_thread_id_get() = id; + } else { + return *netdev_offload_thread_id_get(); + } +} + void netdev_ports_flow_flush(const char *dpif_type) { @@ -627,7 +691,16 @@ netdev_ports_get_n_flows(const char *dpif_type, odp_port_t port_no, ovs_rwlock_rdlock(&netdev_hmap_rwlock); data = netdev_ports_lookup(port_no, dpif_type); if (data) { - ret = netdev_flow_get_n_flows(data->netdev, n_flows); + uint64_t thread_n_flows[MAX_OFFLOAD_THREAD_NB] = {0}; + unsigned int tid; + + ret = netdev_flow_get_n_flows(data->netdev, thread_n_flows); + *n_flows = 0; + if (!ret) { + for (tid = 0; tid < netdev_offload_thread_nb(); tid++) { + *n_flows += thread_n_flows[tid]; + } + } } ovs_rwlock_unlock(&netdev_hmap_rwlock); return ret; @@ -680,7 +753,18 @@ netdev_set_flow_api_enabled(const struct smap *ovs_other_config) if (ovsthread_once_start(&once)) { netdev_flow_api_enabled = true; - VLOG_INFO("netdev: Flow API Enabled"); + offload_thread_nb = smap_get_ullong(ovs_other_config, + "n-offload-threads", + DEFAULT_OFFLOAD_THREAD_NB); + if (offload_thread_nb > MAX_OFFLOAD_THREAD_NB) { + VLOG_WARN("netdev: Invalid number of threads requested: %u", + offload_thread_nb); + offload_thread_nb = DEFAULT_OFFLOAD_THREAD_NB; + } + + VLOG_INFO("netdev: Flow API Enabled, using %u thread%s", + offload_thread_nb, + offload_thread_nb > 1 ? "s" : ""); #ifdef __linux__ tc_set_policy(smap_get_def(ovs_other_config, "tc-policy", diff --git a/lib/netdev-offload.h b/lib/netdev-offload.h index d820e23ed..6e72456e1 100644 --- a/lib/netdev-offload.h +++ b/lib/netdev-offload.h @@ -21,6 +21,7 @@ #include "openvswitch/netdev.h" #include "openvswitch/types.h" #include "ovs-rcu.h" +#include "ovs-thread.h" #include "packets.h" #include "flow.h" @@ -80,6 +81,24 @@ struct offload_info { * to delete the original flow. */ }; +DECLARE_EXTERN_PER_THREAD_DATA(unsigned int, netdev_offload_thread_id); + +unsigned int netdev_offload_thread_nb(void); +unsigned int netdev_offload_thread_init(void); +unsigned int netdev_offload_ufid_to_thread_id(const ovs_u128 ufid); + +static inline unsigned int +netdev_offload_thread_id(void) +{ + unsigned int id = *netdev_offload_thread_id_get(); + + if (OVS_UNLIKELY(id == OVSTHREAD_ID_UNSET)) { + id = netdev_offload_thread_init(); + } + + return id; +} + int netdev_flow_flush(struct netdev *); int netdev_flow_dump_create(struct netdev *, struct netdev_flow_dump **dump, bool terse); diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index a2ad84ede..fc66a6d3e 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -247,6 +247,22 @@

+ +

+ Set this value to the number of threads created to manage hardware + offloads. +

+

+ The default value is 1. Changing this value requires + restarting the daemon. +

+

+ This is only relevant if + is enabled. +

+ + From patchwork Wed Feb 10 15:33:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439076 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=WOe5glFE; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=sttozY8Y; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2d5nf1z9rx8 for ; Thu, 11 Feb 2021 02:34:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 390E186CDB; Wed, 10 Feb 2021 15:34:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KXwXyEM7uBx8; Wed, 10 Feb 2021 15:34:43 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 3F53E86D1B; Wed, 10 Feb 2021 15:34:23 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20874C1E70; Wed, 10 Feb 2021 15:34:23 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 012D7C0891 for ; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D746187411 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SUGZmNR+RFhk for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by hemlock.osuosl.org (Postfix) with ESMTPS id 383FD873E4 for ; Wed, 10 Feb 2021 15:34:17 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id C98FAB43; Wed, 10 Feb 2021 10:34:16 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=nkPFUMAbDadOL mcWz59UYB90U0p3mXElFkQPGTMBzio=; b=WOe5glFE+JcTVArdn8iBKMAYhiT+3 r9VoAy0eBOoIaqRAay92350WjqW/RN9jP/kIKHEum89DeOIcDQl/T8JTVTzBi7z/ NzEgIa6m17q3h8HVL1HRJ0rnxiCy+uvLtEBf7mRho90C1sotec/A9yb1GWqN6Tfk hglI/H76JDGie0r8uUAidiLxL7DZ4a2S3fuhnX4uxX8p0RionUtwubEqEyGQxxA+ CaY1rbVDWC3WDLOI4gkx4EmIcusGRC94sLD3gWW8C8Lvi61cy4Xj6jh14VMRUQts WBVvHdjoo4/cBP81iuYgfLGeFGSuisIWnpmCDEpolkyCgt9F0Fi/ul0Pg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=nkPFUMAbDadOLmcWz59UYB90U0p3mXElFkQPGTMBzio=; b=sttozY8Y aAcPWw/ynkXkwT1/XIdVBID6PkUfndD7lOM/cmQXTaOk6IgTIfgIOiLrjgeRgGzg bm1tdq3rbdn25vpy4tEVjzKsi1KR6nwPw+iKuZUgrbSDuh0yL/KID+q5RR4xzxiU L+8h+KSX+6hFrv6mfBXKrc11Z/meuwF5IVrMQlFWGKwktmfNQvvSv07gnVWEYIxh wRWjmJoQBksbKhvkDhBlp4PGNDFUO18iKo9nGHXa9Ws50f02YKRq9PSwBpjFJJ3v UHrbtx+vwXy9e85I7FF50velztg6F1VEsPDvF/Swn2fvgkfRXZnyGg3+Bv1eQGqI AXgFMvM49TYWbA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id E8EDA24005A; Wed, 10 Feb 2021 10:34:15 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:33:59 +0100 Message-Id: <89eb59af01fcad7484b224cfef7d3677b3f9ce92.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 13/23] dpif-netdev: Quiesce offload thread periodically X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Similar to what was done for the PMD threads [1], reduce the performance impact of quiescing too often in the offload thread. After each processed offload, the offload thread currently quiesce and will sync with RCU. This synchronization can be lengthy and make the thread unnecessary slow. Instead attempt to quiesce every 10 ms at most. While the queue is empty, the offload thread remains quiescent. [1]: 81ac8b3b194c ("dpif-netdev: Do RCU synchronization at fixed interval in PMD main loop.") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index bf994e304..18e71446a 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2739,15 +2739,20 @@ err_free: return -1; } +#define DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US (10 * 1000) /* 10 ms */ + static void * dp_netdev_flow_offload_main(void *data OVS_UNUSED) { struct dp_offload_thread_item *offload; struct ovs_list *list; long long int latency_us; + long long int next_rcu; + long long int now; const char *op; int ret; + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; for (;;) { ovs_mutex_lock(&dp_offload_thread.mutex); if (ovs_list_is_empty(&dp_offload_thread.list)) { @@ -2755,6 +2760,7 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) ovs_mutex_cond_wait(&dp_offload_thread.cond, &dp_offload_thread.mutex); ovsrcu_quiesce_end(); + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; } list = ovs_list_pop_front(&dp_offload_thread.list); dp_offload_thread.enqueued_item--; @@ -2778,7 +2784,9 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) OVS_NOT_REACHED(); } - latency_us = time_usec() - offload->timestamp; + now = time_usec(); + + latency_us = now - offload->timestamp; mov_avg_cma_update(&dp_offload_thread.cma, latency_us); mov_avg_ema_update(&dp_offload_thread.ema, latency_us); @@ -2786,7 +2794,13 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) ret == 0 ? "succeed" : "failed", op, UUID_ARGS((struct uuid *) &offload->flow->mega_ufid)); dp_netdev_free_flow_offload(offload); - ovsrcu_quiesce(); + + /* Do RCU synchronization at fixed interval. */ + if (now > next_rcu) { + if (!ovsrcu_try_quiesce()) { + next_rcu += DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; + } + } } return NULL; From patchwork Wed Feb 10 15:34:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439069 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=zqNYDNnh; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=AYcHkW6w; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2J5CGyz9sCD for ; Thu, 11 Feb 2021 02:34:32 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0BCD886B8D; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eLYFHa1mU0XD; Wed, 10 Feb 2021 15:34:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 57CDE86C3B; Wed, 10 Feb 2021 15:34:26 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3D4C6C1E72; Wed, 10 Feb 2021 15:34:26 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6C6FEC013A for ; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 4AD9887459 for ; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FDMP-04jLMai for ; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by hemlock.osuosl.org (Postfix) with ESMTPS id 182458743C for ; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.west.internal (Postfix) with ESMTP id 5FF5CB43; Wed, 10 Feb 2021 10:34:18 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 10 Feb 2021 10:34:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=k/2i5k1DMGV3o pqkcijqadhLN4IHEdk2ldhA+nhOjFk=; b=zqNYDNnhibnBhuBMG3ln5XEV9K0zI 31/iCtBydm7/D+9ugGsMHPRhaWKvB5UUfzrCqLlFGZo6wDzXoHqTcvXt3ODJFq0z 3zNQOFgcV+4/uSbhACyDfXbdX1htuo1i/MxsJtCkXWmhbhU4cEh1hgk8i4Rwl2qe F73j2DgIDJC7iG+SI1zIL8HhlY0rtk2xh5sLuae25sfZe/be5zAsYw2IfBaDlte2 B1ONe1CBkFj+x+hziUdtO7JRZOwcwzdfneMLG9hP5IG9Fah7HbvaXdAcxaxNZ8ju Pi/5hFw8xeMsqzmU5eY92423V8GH3anvcivgC/BEPijYUULl6Hjm5ScWw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=k/2i5k1DMGV3opqkcijqadhLN4IHEdk2ldhA+nhOjFk=; b=AYcHkW6w GikPGcFHcgz7L2yn81aLmirHAy4jogg2HlwvTURtoYgCuzLBPabFnadO1qrsgdUZ ztx9szY2uikNFf+gg5dq4yXVT8ARa4eaavJw3zIfHY0ZLqyueDn+kfykWl99d2V+ uK7Av4dEVtdwaem1f4It0naDow09zHasVHRAeHUZaHNFVRzfwVVKxNtVA8cHhdoJ FH/s37RbP1xX5ksbu9n8Arwh2aoUTUo9nP37eohftcatqAtOQhhuRG/2kkN1WZW4 D2SNkRc/mtn8oq8+LBRrbLUhoZad4NzNPgOInZ/KA9C5aeBmjN6KGwKLjp4O2iIm f8egrsRrRSaHmg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 79CCB240065; Wed, 10 Feb 2021 10:34:16 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:00 +0100 Message-Id: <56e753427e9d8b07c0ca89a73c3a2192e4295eb7.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 14/23] dpif-netdev: Postpone flow offload item freeing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Profiling the HW offload thread, the flow offload freeing takes approximatively 25% of the time. Most of this time is spent waiting on the futex used by the libc free(), as it triggers a syscall and reschedule the thread. Avoid the syscall and its expensive context switch. Batch the offload messages freeing using the RCU. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 18e71446a..dc2828659 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2624,14 +2624,19 @@ dp_netdev_alloc_flow_offload(struct dp_netdev_pmd_thread *pmd, return offload; } +static void +dp_netdev_free_flow_offload__(struct dp_offload_thread_item *offload) +{ + free(offload->actions); + free(offload); +} + static void dp_netdev_free_flow_offload(struct dp_offload_thread_item *offload) { dp_netdev_pmd_unref(offload->pmd); dp_netdev_flow_unref(offload->flow); - - free(offload->actions); - free(offload); + ovsrcu_postpone(dp_netdev_free_flow_offload__, offload); } static void From patchwork Wed Feb 10 15:34:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439078 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=RmA/MR9+; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=IIwX5jd2; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2q10XMz9rx8 for ; Thu, 11 Feb 2021 02:34:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id A4AA186CA6; Wed, 10 Feb 2021 15:34:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id e+UYNBnFbC+4; Wed, 10 Feb 2021 15:34:56 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 57EFC86DEF; Wed, 10 Feb 2021 15:34:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 184E4C1E70; Wed, 10 Feb 2021 15:34:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 622BDC013A for ; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 2D6BA8745D for ; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2-7Pz6su3-Jf for ; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by hemlock.osuosl.org (Postfix) with ESMTPS id C588D87434 for ; Wed, 10 Feb 2021 15:34:18 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 115FA547; Wed, 10 Feb 2021 10:34:18 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=l+g1fXsZuRv0e XhDFQuL0Ej54g7PvAgL2cMYuGtXCTI=; b=RmA/MR9+upZlM5mTGogdG0V/fh/Yv VubJDe68km0cjMH512/i+2TG5fEZoIoCcP2TcyMYc8y3sWSGYZQ8n0MHgGtj5dkd 6dBoHFTvwA/bO50mdwlXUeVeIPpUn7tZIf+iPlBjg7R8NswOIGarH51umgtpqGBH jBt1RElnDWNTQUQz3Wn8c12IVYLiek0eI9DB9/iJKcEmzveINS3nt7l5+hQagv8U h/pIblDVZt0wKo1QwqYwTWLgZYAAU7xuGS8veaPkt04tOzv+5xinqk90J3Y4Z6V/ qzlBGLhDgFpgi41JH8JLj0UtE3aEVBD4ho0AwtpSKG0OGLeQy4BR/2RaA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=l+g1fXsZuRv0eXhDFQuL0Ej54g7PvAgL2cMYuGtXCTI=; b=IIwX5jd2 HqiexpS1wPorQWyDqKu1MyQgGBGadxEP7gwDMHvAKd22Wn5Id6Z0h2pbgyctf3aR xFBWe6EdT9NZ0MjV+j1IA2K6lPtK9dBV//heU2Pm9PEuST2s78nFCI+03CpiZVmt bN5Gs5rFG78in3q4hTKIZynHaP6flMuKc78HACPFCXoTMcoR1XD/K97GX+hMQKEE Nc5GRbMVBeWYDys7mNghIv98IxF/ctpuC+ZSD7HpdV6cclcg1OVA9moj7RzdYABy phZHBXKLCtvxSc/k5lzeTAqL9cD5y1qp2Our29IiHb+BfELGZYpI5yBK4AZC3cMw CcUQwdGtdEB1lA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgepfeenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 080A4240068; Wed, 10 Feb 2021 10:34:16 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:01 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 15/23] dpif-netdev: Use seq-pool for mark allocation X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Use the netdev-offload multithread API to allow multiple thread allocating marks concurrently. Initialize only once the pool in a multithread context by using the ovsthread_once type. Use the seq-pool module for faster concurrent ID allocation. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index dc2828659..9f9898caa 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -74,6 +74,7 @@ #include "pvector.h" #include "random.h" #include "seq.h" +#include "seq-pool.h" #include "smap.h" #include "sset.h" #include "timeval.h" @@ -2417,7 +2418,7 @@ struct megaflow_to_mark_data { struct flow_mark { struct cmap megaflow_to_mark; struct cmap mark_to_flow; - struct id_pool *pool; + struct seq_pool *pool; }; static struct flow_mark flow_mark = { @@ -2428,14 +2429,18 @@ static struct flow_mark flow_mark = { static uint32_t flow_mark_alloc(void) { + static struct ovsthread_once pool_init = OVSTHREAD_ONCE_INITIALIZER; + unsigned int tid = netdev_offload_thread_id(); uint32_t mark; - if (!flow_mark.pool) { + if (ovsthread_once_start(&pool_init)) { /* Haven't initiated yet, do it here */ - flow_mark.pool = id_pool_create(1, MAX_FLOW_MARK); + flow_mark.pool = seq_pool_create(netdev_offload_thread_nb(), + 1, MAX_FLOW_MARK); + ovsthread_once_done(&pool_init); } - if (id_pool_alloc_id(flow_mark.pool, &mark)) { + if (seq_pool_new_id(flow_mark.pool, tid, &mark)) { return mark; } @@ -2445,7 +2450,9 @@ flow_mark_alloc(void) static void flow_mark_free(uint32_t mark) { - id_pool_free_id(flow_mark.pool, mark); + unsigned int tid = netdev_offload_thread_id(); + + seq_pool_free_id(flow_mark.pool, tid, mark); } /* associate megaflow with a mark, which is a 1:1 mapping */ From patchwork Wed Feb 10 15:34:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439073 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=G1eHXiOn; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=twQdCJTv; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2X0SFnz9rx8 for ; Thu, 11 Feb 2021 02:34:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 95527874F3; Wed, 10 Feb 2021 15:34:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q+iU0LqS23YZ; Wed, 10 Feb 2021 15:34:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 780BC874AB; Wed, 10 Feb 2021 15:34:34 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 468EFC1E70; Wed, 10 Feb 2021 15:34:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id B699FC1DA9 for ; Wed, 10 Feb 2021 15:34:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 9734586EB5 for ; Wed, 10 Feb 2021 15:34:28 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lCxZYKlTv9Wk for ; Wed, 10 Feb 2021 15:34:23 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id 1A45986CAF for ; Wed, 10 Feb 2021 15:34:19 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id 99DC5E7E; Wed, 10 Feb 2021 10:34:18 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=q3DancdlLFD+k gS0adLeNIlLrzHC2nnZpS2QLEBSiOA=; b=G1eHXiOnvJTxP7IFkyNnHvd4A0oQ/ 9fRhpqgfD/bEiThcMQY86ZDY+kKQCz8BSr/qk32NYoFmT37g0b+WpblKU3mZKop5 VpyZlhtRPQ9xDb2SkzpwImJpVw5GcjMYVduP3BpeIjkWZflMTlw1ZwN3D3FWxXRb c2Z5xvvrG3IlVhzQnuVtWAHz/poMUVouV4EAUEIM/3myVgpMrfc8+xMVnqFluQv2 ZaRrXvmB7eWHIaGZdaYEk8KnSJWQeRZSI7bNWOwlZ0HMajctv/jUR5R0+wwOYXrM EJPE7Lm3qH7McoShpHLLe+JqDRtod5TvvYcU+zNsJIeqar38sgVIR3iZw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=q3DancdlLFD+kgS0adLeNIlLrzHC2nnZpS2QLEBSiOA=; b=twQdCJTv IWw/27vJ6uIagVVliVPOyAvigIW3OMgAIMzjio9Dk509WKaMhjFvuX1J7mYlDZc4 BckYUFTDhrHb5KZ5cX9+1tA/Z0GUE6wNzaQrk4rXFGklwdAkgngVRZMNPSWjVXbp y2o8/CO9EB95iQbXqfwLg32uQxcLabsHmF2+AFofoWOMZaM9WqH+TJgN+UwHXRYR UyRhknRtkTXH4/F+bSagyS+MadBfZdCOJLeJkfpbOFbUM0/SPb7Z8P0ehbb7e3y7 hbydnfA01Oo5Kpu4h0eFC75WkY3cRABMPZw3PGtVjluBv6JWA503Ht2DHkVX2AvH X4gomaowICmvNQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgepudenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 8B54024005B; Wed, 10 Feb 2021 10:34:17 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:02 +0100 Message-Id: <0f95979461825d788abd0b8ad20e30436c6f5d7e.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 16/23] netdev-offload-dpdk: Use per-thread HW offload stats X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The implementation of hardware offload counters in currently meant to be managed by a single thread. Use the offload thread pool API to manage one counter per thread. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-dpdk.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 1d60c5a81..338ca4dc6 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -65,7 +65,7 @@ struct ufid_to_rte_flow_data { struct netdev_offload_dpdk_data { struct cmap ufid_to_rte_flow; - uint64_t rte_flow_counter; + uint64_t *rte_flow_counters; }; static int @@ -75,6 +75,8 @@ offload_data_init(struct netdev *netdev) data = xzalloc(sizeof *data); cmap_init(&data->ufid_to_rte_flow); + data->rte_flow_counters = xcalloc(netdev_offload_thread_nb(), + sizeof *data->rte_flow_counters); ovsrcu_set(&netdev->hw_info.offload_data, (void *) data); @@ -84,6 +86,7 @@ offload_data_init(struct netdev *netdev) static void offload_data_destroy__(struct netdev_offload_dpdk_data *data) { + free(data->rte_flow_counters); free(data); } @@ -646,10 +649,11 @@ netdev_offload_dpdk_flow_create(struct netdev *netdev, flow = netdev_dpdk_rte_flow_create(netdev, attr, items, actions, error); if (flow) { struct netdev_offload_dpdk_data *data; + unsigned int tid = netdev_offload_thread_id(); data = (struct netdev_offload_dpdk_data *) ovsrcu_get(void *, &netdev->hw_info.offload_data); - data->rte_flow_counter++; + data->rte_flow_counters[tid]++; if (!VLOG_DROP_DBG(&rl)) { dump_flow(&s, &s_extra, attr, items, actions); @@ -1532,10 +1536,11 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) if (ret == 0) { struct netdev_offload_dpdk_data *data; + unsigned int tid = netdev_offload_thread_id(); data = (struct netdev_offload_dpdk_data *) ovsrcu_get(void *, &netdev->hw_info.offload_data); - data->rte_flow_counter--; + data->rte_flow_counters[tid]--; ufid_to_rte_flow_disassociate(rte_flow_data); VLOG_DBG_RL(&rl, "%s: rte_flow 0x%"PRIxPTR @@ -1698,6 +1703,7 @@ netdev_offload_dpdk_get_n_flows(struct netdev *netdev, uint64_t *n_flows) { struct netdev_offload_dpdk_data *data; + unsigned int tid; data = (struct netdev_offload_dpdk_data *) ovsrcu_get(void *, &netdev->hw_info.offload_data); @@ -1705,7 +1711,9 @@ netdev_offload_dpdk_get_n_flows(struct netdev *netdev, return -1; } - *n_flows = data->rte_flow_counter; + for (tid = 0; tid < netdev_offload_thread_nb(); tid++) { + n_flows[tid] = data->rte_flow_counters[tid]; + } return 0; } From patchwork Wed Feb 10 15:34:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=ZvvGuDaf; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=TtZphAXM; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP4W0xX8z9rx8 for ; Thu, 11 Feb 2021 02:36:27 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 9DB4B86CF4; Wed, 10 Feb 2021 15:36:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NfarffScgxeu; Wed, 10 Feb 2021 15:36:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 180AB8714A; Wed, 10 Feb 2021 15:34:40 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E7828C1E73; Wed, 10 Feb 2021 15:34:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5B3DCC013A for ; Wed, 10 Feb 2021 15:34:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 362E886C99 for ; Wed, 10 Feb 2021 15:34:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BDm4sXVcem1s for ; Wed, 10 Feb 2021 15:34:25 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id 670C886CAB for ; Wed, 10 Feb 2021 15:34:19 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 0DA8C547; Wed, 10 Feb 2021 10:34:18 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=aeEgB+HbqGqYh qKB2avgbRIz+uXdWdyEx30Dj27A4WU=; b=ZvvGuDafOFnAq/M5Y9SOq3OJMV1R3 mVS1Jq6uZ3O7U3kuLzrYfx1Out7Z3kEEIHQCpam6owlokckF8WLExdNptTeYA2bv Sr2S87IIxEIY7ixwPIgwGkmTUVDdgEJFzfQCceAHQYBC0hnQvk33lxcLFIeuCuJw lWcmyRNi7gulLzy4zrAZ9MGsQfJKXCNMDx1EO1LMN0T9O1WjVo9A6MorjbUog9AH M+T6TEdD6Yyd/nMbLIYpZuHidL61vCjDAYgciVe8ScPxZ2f1lgsEkfBaYcikjlI0 SqvK9WeVg3h4qsu3Omiv+DruGZCjDv+Qq3pk5t2oNidazJcNrChnzLSFw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=aeEgB+HbqGqYhqKB2avgbRIz+uXdWdyEx30Dj27A4WU=; b=TtZphAXM n/P9bgiNspj9a/Aium3PWuyIsbNcVTWBqcRAoeUc70/5qZ+C7bOcvSniQrrwtmYk SvEIeLRBwBq65G+3UDuaSKe8Oy/9S6j7AG9Hjafks1bKJkyeS0aVvUP3UIsXbus5 M0OwAG96af3/1edKQJ+XDYVIlwg7XNHtuSqi5EIAA8xvqvvuopjhrFRzMSKFu7Q/ 7SRk+8WMuAcbe/o7xh4cpPt/80MUIYryQrnfhQkbIjMwhqVtzr8PyPfJgT1PCsuz 12Nf/Z7O72suRCmQckEprRHF4oKvrQCW15c2SqtEML1i5EiqsfIlms1YDO+rgqld eBNHMa2BJjqx2g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 1AC49240069; Wed, 10 Feb 2021 10:34:18 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:03 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 17/23] netdev-offload-dpdk: Lock rte_flow map access X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add a lock to access the ufid to rte_flow map. This will protect it from concurrent write accesses when multiple threads attempt it. At this point, the reason for taking the lock is not to fullfill the needs of the DPDK offload implementation anymore. Rewrite the comments to reflect this change. The lock is still needed to protect against changes to netdev port mapping. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 8 ++--- lib/netdev-offload-dpdk.c | 61 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 61 insertions(+), 8 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 9f9898caa..8b9115609 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2566,7 +2566,7 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, port = netdev_ports_get(in_port, dpif_type_str); if (port) { /* Taking a global 'port_mutex' to fulfill thread safety - * restrictions for the netdev-offload-dpdk module. */ + * restrictions regarding netdev port mapping. */ ovs_mutex_lock(&pmd->dp->port_mutex); ret = netdev_flow_del(port, &flow->mega_ufid, NULL); ovs_mutex_unlock(&pmd->dp->port_mutex); @@ -2722,8 +2722,8 @@ dp_netdev_flow_offload_put(struct dp_offload_thread_item *offload) netdev_close(port); goto err_free; } - /* Taking a global 'port_mutex' to fulfill thread safety restrictions for - * the netdev-offload-dpdk module. */ + /* Taking a global 'port_mutex' to fulfill thread safety + * restrictions regarding the netdev port mapping. */ ovs_mutex_lock(&pmd->dp->port_mutex); ret = netdev_flow_put(port, &offload->match, CONST_CAST(struct nlattr *, offload->actions), @@ -3407,7 +3407,7 @@ dpif_netdev_get_flow_offload_status(const struct dp_netdev *dp, } ofpbuf_use_stack(&buf, &act_buf, sizeof act_buf); /* Taking a global 'port_mutex' to fulfill thread safety - * restrictions for the netdev-offload-dpdk module. + * restrictions regarding netdev port mapping. * * XXX: Main thread will try to pause/stop all revalidators during datapath * reconfiguration via datapath purge callback (dp_purge_cb) while diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 338ca4dc6..617f87eca 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -38,9 +38,6 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(100, 5); * * Below API is NOT thread safe in following terms: * - * - The caller must be sure that none of these functions will be called - * simultaneously. Even for different 'netdev's. - * * - The caller must be sure that 'netdev' will not be destructed/deallocated. * * - The caller must be sure that 'netdev' configuration will not be changed. @@ -66,6 +63,7 @@ struct ufid_to_rte_flow_data { struct netdev_offload_dpdk_data { struct cmap ufid_to_rte_flow; uint64_t *rte_flow_counters; + struct ovs_mutex map_lock; }; static int @@ -74,6 +72,7 @@ offload_data_init(struct netdev *netdev) struct netdev_offload_dpdk_data *data; data = xzalloc(sizeof *data); + ovs_mutex_init(&data->map_lock); cmap_init(&data->ufid_to_rte_flow); data->rte_flow_counters = xcalloc(netdev_offload_thread_nb(), sizeof *data->rte_flow_counters); @@ -86,6 +85,7 @@ offload_data_init(struct netdev *netdev) static void offload_data_destroy__(struct netdev_offload_dpdk_data *data) { + ovs_mutex_destroy(&data->map_lock); free(data->rte_flow_counters); free(data); } @@ -117,6 +117,34 @@ offload_data_destroy(struct netdev *netdev) ovsrcu_set(&netdev->hw_info.offload_data, NULL); } +static void +offload_data_lock(struct netdev *netdev) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + if (!data) { + return; + } + ovs_mutex_lock(&data->map_lock); +} + +static void +offload_data_unlock(struct netdev *netdev) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + struct netdev_offload_dpdk_data *data; + + data = (struct netdev_offload_dpdk_data *) + ovsrcu_get(void *, &netdev->hw_info.offload_data); + if (!data) { + return; + } + ovs_mutex_unlock(&data->map_lock); +} + static struct cmap * offload_data_map(struct netdev *netdev) { @@ -155,6 +183,24 @@ ufid_to_rte_flow_data_find(struct netdev *netdev, return NULL; } +/* Find rte_flow with @ufid, lock-protected. */ +static struct ufid_to_rte_flow_data * +ufid_to_rte_flow_data_find_protected(struct netdev *netdev, + const ovs_u128 *ufid) +{ + size_t hash = hash_bytes(ufid, sizeof *ufid, 0); + struct ufid_to_rte_flow_data *data; + struct cmap *map = offload_data_map(netdev); + + CMAP_FOR_EACH_WITH_HASH_PROTECTED (data, node, hash, map) { + if (ovs_u128_equals(*ufid, data->ufid)) { + return data; + } + } + + return NULL; +} + static inline struct ufid_to_rte_flow_data * ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, struct rte_flow *rte_flow, bool actions_offloaded) @@ -170,13 +216,15 @@ ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, data = xzalloc(sizeof *data); + offload_data_lock(netdev); + /* * We should not simply overwrite an existing rte flow. * We should have deleted it first before re-adding it. * Thus, if following assert triggers, something is wrong: * the rte_flow is not destroyed. */ - data_prev = ufid_to_rte_flow_data_find(netdev, ufid, false); + data_prev = ufid_to_rte_flow_data_find_protected(netdev, ufid); if (data_prev) { ovs_assert(data_prev->rte_flow == NULL); } @@ -187,6 +235,8 @@ ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, data->actions_offloaded = actions_offloaded; cmap_insert(map, CONST_CAST(struct cmap_node *, &data->node), hash); + + offload_data_unlock(netdev); return data; } @@ -200,7 +250,10 @@ ufid_to_rte_flow_disassociate(struct ufid_to_rte_flow_data *data) return; } + offload_data_lock(data->netdev); cmap_remove(map, CONST_CAST(struct cmap_node *, &data->node), hash); + offload_data_unlock(data->netdev); + netdev_close(data->netdev); ovsrcu_postpone(free, data); } From patchwork Wed Feb 10 15:34:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=vPkKihoa; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=NIXdiHF0; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP4N1lNnz9sB4 for ; Thu, 11 Feb 2021 02:36:20 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id B3E0986CEF; Wed, 10 Feb 2021 15:36:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rRFs8fr9vO7p; Wed, 10 Feb 2021 15:36:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id BF66D87103; Wed, 10 Feb 2021 15:34:38 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9974FC1E73; Wed, 10 Feb 2021 15:34:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3E6ABC013A for ; Wed, 10 Feb 2021 15:34:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 33AF38701C for ; Wed, 10 Feb 2021 15:34:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h-Tl4snTqwkZ for ; Wed, 10 Feb 2021 15:34:25 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by whitealder.osuosl.org (Postfix) with ESMTPS id 2229286CCE for ; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id 9ED5F1C7; Wed, 10 Feb 2021 10:34:19 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=luEXNMEVxqwJQ qJHaRbWeFxaO3kz3/1Y2zpN9CMenEU=; b=vPkKihoaQcIH4B7NPVtwLaR+8Zd7e AQuEhypOt5b/OlmlCZcraJFEyn57q+M7qOUvyXmQDbCXiJOljXZYUkzQSJKitTzt gjvE+1L91LzJDb7uJ8Ah8JL6LlqN1mMrxx1byOeHIXthqEnmvy7k+L/785r/Skpb omsNxCJLJnx/aGRA9rAuE9K1V/Z0dmhDkBwS4C4Pc4pAB424hfjwY5rrBDcP/b6N OsmoSdxRFRVrZGh2sfl0WvG/zQvMZSruFkvTw3BDS1LBaXVTARyR16TnbnYJYAkZ +vhbMv7c8P9pXCCj5M+RB833I9w41Pvpu82/xkjeiwa6Yh1DmsPdKKi1w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=luEXNMEVxqwJQqJHaRbWeFxaO3kz3/1Y2zpN9CMenEU=; b=NIXdiHF0 oHn44V4RX6D4QFURCdA3NiOP0/sx/jB5jUcTkOZf9W57k7aVC8pp2LB6DWHIcjaI XR3gXY5NVcLyALJKkyMes4UUs1Jm0kUfQXTeGrYsFZhXc4lR84F/MMK9t+mgpEtL Mws/zpxwV28CGI03w9mOXsTPDjUKe8yuVfaUHMD6Yg3cNF6X8zmZJc8HbR8N8O+W Q0D5ogJOYSXAlpfouFxGglyNlJUjbVU7bapwn1RMAx/40HBCLDdaJYSWVKw83Ofd 1ZWPpRsu9vPuXKxTMYOvkfNow2oNdCjPO9pf+A15dghGPWn91qG3QCSGEKSibJIw 7cuIqizZIdDAPA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgepudenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 9EE6F24005A; Wed, 10 Feb 2021 10:34:18 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:04 +0100 Message-Id: <3c8ccc6a4c0b609103bbb72d54c763e56138afe0.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 18/23] netdev-offload-dpdk: Protect concurrent offload destroy/query X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The rte_flow API in DPDK is now thread safe for insertion and deletion. It is not however safe for concurrent query while the offload is being inserted or deleted. Insertion is not an issue as the rte_flow handle will be published to other threads only once it has been inserted in the hardware, so the query will only be able to proceed once it is already available. For the deletion path however, offload status queries can be made while an offload is being destroyed. This would create race conditions and use-after-free if not properly protected. As a pre-step before removing the OVS-level locks on the rte_flow API, mutually exclude offload query and deletion from concurrent execution. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-offload-dpdk.c | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 617f87eca..72cff5688 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -58,6 +58,8 @@ struct ufid_to_rte_flow_data { struct rte_flow *rte_flow; bool actions_offloaded; struct dpif_flow_stats stats; + struct ovs_mutex lock; + bool dead; }; struct netdev_offload_dpdk_data { @@ -233,6 +235,7 @@ ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, data->netdev = netdev_ref(netdev); data->rte_flow = rte_flow; data->actions_offloaded = actions_offloaded; + ovs_mutex_init(&data->lock); cmap_insert(map, CONST_CAST(struct cmap_node *, &data->node), hash); @@ -240,8 +243,16 @@ ufid_to_rte_flow_associate(struct netdev *netdev, const ovs_u128 *ufid, return data; } +static void +rte_flow_data_unref(struct ufid_to_rte_flow_data *data) +{ + ovs_mutex_destroy(&data->lock); + free(data); +} + static inline void ufid_to_rte_flow_disassociate(struct ufid_to_rte_flow_data *data) + OVS_REQUIRES(data->lock) { size_t hash = hash_bytes(&data->ufid, sizeof data->ufid, 0); struct cmap *map = offload_data_map(data->netdev); @@ -255,7 +266,7 @@ ufid_to_rte_flow_disassociate(struct ufid_to_rte_flow_data *data) offload_data_unlock(data->netdev); netdev_close(data->netdev); - ovsrcu_postpone(free, data); + ovsrcu_postpone(rte_flow_data_unref, data); } /* @@ -1581,6 +1592,15 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) ovs_u128 *ufid; int ret; + ovs_mutex_lock(&rte_flow_data->lock); + + if (rte_flow_data->dead) { + ovs_mutex_unlock(&rte_flow_data->lock); + return 0; + } + + rte_flow_data->dead = true; + rte_flow = rte_flow_data->rte_flow; netdev = rte_flow_data->netdev; ufid = &rte_flow_data->ufid; @@ -1607,6 +1627,8 @@ netdev_offload_dpdk_flow_destroy(struct ufid_to_rte_flow_data *rte_flow_data) UUID_ARGS((struct uuid *) ufid)); } + ovs_mutex_unlock(&rte_flow_data->lock); + return ret; } @@ -1702,8 +1724,19 @@ netdev_offload_dpdk_flow_get(struct netdev *netdev, struct rte_flow_error error; int ret = 0; + attrs->dp_extra_info = NULL; + rte_flow_data = ufid_to_rte_flow_data_find(netdev, ufid, false); - if (!rte_flow_data || !rte_flow_data->rte_flow) { + if (!rte_flow_data || !rte_flow_data->rte_flow || + rte_flow_data->dead || ovs_mutex_trylock(&rte_flow_data->lock)) { + return -1; + } + + /* Check again whether the data is dead, as it could have been + * updated while the lock was not yet taken. The first check above + * was only to avoid unnecessary locking if possible. + */ + if (rte_flow_data->dead) { ret = -1; goto out; } @@ -1730,7 +1763,7 @@ netdev_offload_dpdk_flow_get(struct netdev *netdev, } memcpy(stats, &rte_flow_data->stats, sizeof *stats); out: - attrs->dp_extra_info = NULL; + ovs_mutex_unlock(&rte_flow_data->lock); return ret; } From patchwork Wed Feb 10 15:34:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439070 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=O/7yD6lW; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=oi+Y6USY; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2L4p4Fz9sB4 for ; Thu, 11 Feb 2021 02:34:34 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3C7B487492; Wed, 10 Feb 2021 15:34:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5f-91ah4YdDy; Wed, 10 Feb 2021 15:34:29 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 0B05887499; Wed, 10 Feb 2021 15:34:28 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EBE18C1DA9; Wed, 10 Feb 2021 15:34:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id D5471C013A for ; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 9DD208747F for ; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OMprCTNToljm for ; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by hemlock.osuosl.org (Postfix) with ESMTPS id 1CEE787431 for ; Wed, 10 Feb 2021 15:34:20 +0000 (UTC) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailnew.west.internal (Postfix) with ESMTP id 5D339A1E; Wed, 10 Feb 2021 10:34:20 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 10 Feb 2021 10:34:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=vLGAQfiuY49kl o1CHPjM8xKm40sp/sEgR/0lOx0BPM0=; b=O/7yD6lW/pLwVFgW8Qid931CczoUD ke6YCsKSCKduTAfi67O1jcOomyViGgIVbkF3Es2xEM+gOwlE0f0zMPH6V3+xTBeX Fhkz2K+5dm8aojQ9OvjIml1lXce4PFt7Q38C/vlyZTZLG+7tsyoD1wKW98OKNIHI rcp0gD8d+9/FPLHTMZ+0tf8clGiaUvvw+dmLOk319OQRc7JCMEEOa8IKwmhwvZ3e TMRiUVpn/nsDeAHoAiC9bpAo9/iHXu1yumdxSlovZsPfpoUHJlQgVB3vANzwnj0a c3XVu1e8XNU/FRx2O6TniH4MNlNGuSrQxkaGkoA3Jke0Kr+hU+6Xiwqng== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=vLGAQfiuY49klo1CHPjM8xKm40sp/sEgR/0lOx0BPM0=; b=oi+Y6USY i5gl0dDRkV7BjUEevoVKcrMBHuQ4IG5i9UiUSzZtIhVuOVJjGQ5n5FmnC43BSEGG ndqFSSwUM8q25k2K16gnrtk+k74RvhjVGkKso7ML+ZcN/WB6CaU+NwA7vgsoszn3 eI8DGV6DJVJhZ8le3a6O0H8h4a993q+p7XPKCaHaXTHZPnjakmP2PfNIf81Wdux3 wK/AOW3su9auCVCthjKFWGxGo7HKeEWOYlWGirKKbcmBsOkd95ZQQAMgoxC9Sfds iIJe9bCfg9nXxUdJVxoRWG26O1reX2JQ+5m13pyDf6lgPrzpyUAHG7MiKKv0NG+q 7x+FNPtkxrQkXQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 325C1240064; Wed, 10 Feb 2021 10:34:19 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:05 +0100 Message-Id: <4f26bd9c6667264d961176f7e2be46c2a6fd492e.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 19/23] dpif-netdev: Use lockless queue to manage offloads X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The dataplane threads (PMDs) send offloading commands to a dedicated offload management thread. The current implementation uses a lock and benchmarks show a high contention on the queue in some cases. With high-contention, the mutex will more often lead to the locking thread yielding in wait, using a syscall. This should be avoided in a userland dataplane. The mpsc-queue can be used instead. It uses less cycles and has lower latency. Benchmarks show better behavior as multiple revalidators and one or multiple PMDs writes to a single queue while another thread polls it. One trade-off with the new scheme however is to be forced to poll the queue from the offload thread. Without mutex, a cond_wait cannot be used for signaling. The offload thread is implementing an exponential backoff and will sleep in short increments when no data is available. This makes the thread yield, at the price of some latency to manage offloads after an inactivity period. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 78 ++++++++++++++++++++++++++++------------------- 1 file changed, 46 insertions(+), 32 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 8b9115609..09d62a3d5 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -52,6 +52,7 @@ #include "id-pool.h" #include "ipf.h" #include "mov-avg.h" +#include "mpsc-queue.h" #include "netdev.h" #include "netdev-offload.h" #include "netdev-provider.h" @@ -434,22 +435,19 @@ struct dp_offload_thread_item { size_t actions_len; long long int timestamp; - struct ovs_list node; + struct mpsc_queue_node node; }; struct dp_offload_thread { - struct ovs_mutex mutex; - struct ovs_list list; - uint64_t enqueued_item; + struct mpsc_queue queue; + atomic_uint64_t enqueued_item; struct mov_avg_cma cma; struct mov_avg_ema ema; - pthread_cond_t cond; }; static struct dp_offload_thread dp_offload_thread = { - .mutex = OVS_MUTEX_INITIALIZER, - .list = OVS_LIST_INITIALIZER(&dp_offload_thread.list), - .enqueued_item = 0, + .queue = MPSC_QUEUE_INITIALIZER(&dp_offload_thread.queue), + .enqueued_item = ATOMIC_VAR_INIT(0), .cma = MOV_AVG_CMA_INITIALIZER, .ema = MOV_AVG_EMA_INITIALIZER(100), }; @@ -2649,11 +2647,8 @@ dp_netdev_free_flow_offload(struct dp_offload_thread_item *offload) static void dp_netdev_append_flow_offload(struct dp_offload_thread_item *offload) { - ovs_mutex_lock(&dp_offload_thread.mutex); - ovs_list_push_back(&dp_offload_thread.list, &offload->node); - dp_offload_thread.enqueued_item++; - xpthread_cond_signal(&dp_offload_thread.cond); - ovs_mutex_unlock(&dp_offload_thread.mutex); + mpsc_queue_insert(&dp_offload_thread.queue, &offload->node); + atomic_count_inc64(&dp_offload_thread.enqueued_item); } static int @@ -2751,33 +2746,48 @@ err_free: return -1; } +#define DP_NETDEV_OFFLOAD_BACKOFF_MIN 1 +#define DP_NETDEV_OFFLOAD_BACKOFF_MAX 64 #define DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US (10 * 1000) /* 10 ms */ static void * dp_netdev_flow_offload_main(void *data OVS_UNUSED) { struct dp_offload_thread_item *offload; - struct ovs_list *list; + enum mpsc_queue_poll_result poll_result; + struct mpsc_queue_node *node; + struct mpsc_queue *queue; long long int latency_us; long long int next_rcu; long long int now; + uint64_t backoff; const char *op; int ret; + queue = &dp_offload_thread.queue; + if (!mpsc_queue_acquire(queue)) { + VLOG_ERR("failed to register as consumer of the offload queue"); + return NULL; + } + +sleep_until_next: + backoff = DP_NETDEV_OFFLOAD_BACKOFF_MIN; + while ((poll_result = mpsc_queue_poll(queue, &node)) == MPSC_QUEUE_EMPTY) { + xnanosleep(backoff * 1E6); + if (backoff < DP_NETDEV_OFFLOAD_BACKOFF_MAX) { + backoff <<= 1; + } + } + next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; - for (;;) { - ovs_mutex_lock(&dp_offload_thread.mutex); - if (ovs_list_is_empty(&dp_offload_thread.list)) { - ovsrcu_quiesce_start(); - ovs_mutex_cond_wait(&dp_offload_thread.cond, - &dp_offload_thread.mutex); - ovsrcu_quiesce_end(); - next_rcu = time_usec() + DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; - } - list = ovs_list_pop_front(&dp_offload_thread.list); - dp_offload_thread.enqueued_item--; - offload = CONTAINER_OF(list, struct dp_offload_thread_item, node); - ovs_mutex_unlock(&dp_offload_thread.mutex); + + do { + while (poll_result == MPSC_QUEUE_RETRY) { + poll_result = mpsc_queue_poll(queue, &node); + } + + offload = CONTAINER_OF(node, struct dp_offload_thread_item, node); + atomic_count_dec64(&dp_offload_thread.enqueued_item); switch (offload->op) { case DP_NETDEV_FLOW_OFFLOAD_OP_ADD: @@ -2813,7 +2823,11 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) next_rcu += DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US; } } - } + + poll_result = mpsc_queue_poll(queue, &node); + } while (poll_result != MPSC_QUEUE_EMPTY); + + goto sleep_until_next; return NULL; } @@ -2825,7 +2839,7 @@ queue_netdev_flow_del(struct dp_netdev_pmd_thread *pmd, struct dp_offload_thread_item *offload; if (ovsthread_once_start(&offload_thread_once)) { - xpthread_cond_init(&dp_offload_thread.cond, NULL); + mpsc_queue_init(&dp_offload_thread.queue); ovs_thread_create("dp_netdev_flow_offload", dp_netdev_flow_offload_main, NULL); ovsthread_once_done(&offload_thread_once); @@ -2850,7 +2864,7 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, } if (ovsthread_once_start(&offload_thread_once)) { - xpthread_cond_init(&dp_offload_thread.cond, NULL); + mpsc_queue_init(&dp_offload_thread.queue); ovs_thread_create("dp_netdev_flow_offload", dp_netdev_flow_offload_main, NULL); ovsthread_once_done(&offload_thread_once); @@ -4295,8 +4309,8 @@ dpif_netdev_offload_stats_get(struct dpif *dpif, } ovs_mutex_unlock(&dp->port_mutex); - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED].value = - dp_offload_thread.enqueued_item; + atomic_read_relaxed(&dp_offload_thread.enqueued_item, + &stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED].value); stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_INSERTED].value = nb_offloads; stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN].value = mov_avg_cma(&dp_offload_thread.cma); From patchwork Wed Feb 10 15:34:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439077 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=MYyIumba; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=E1RmMWDT; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP2h6xPhz9rx8 for ; Thu, 11 Feb 2021 02:34:52 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 84BDF87513; Wed, 10 Feb 2021 15:34:51 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q+8goIFrDinh; Wed, 10 Feb 2021 15:34:48 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 50C73874D9; Wed, 10 Feb 2021 15:34:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A16BC1E6F; Wed, 10 Feb 2021 15:34:37 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id EA46EC1E70 for ; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id B41636F6A5 for ; Wed, 10 Feb 2021 15:34:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sk2SaK0ZCt67 for ; Wed, 10 Feb 2021 15:34:29 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 6AFAB6F79A; Wed, 10 Feb 2021 15:34:29 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 49B796F66B for ; Wed, 10 Feb 2021 15:34:21 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id BA04BE18; Wed, 10 Feb 2021 10:34:20 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=uapF96GLjk13k e5x2Q/30LebnRbPauH8wMHhKw/nAxc=; b=MYyIumbaAEzqgv/1Ni+efykGsvjve joCviAU8VuEldhDVBMSM6YhG5EbvH0HzuxMpBo3bSoeAwzltmuRUl0TmN/eHbByH FCq5vcd8wv4RPQ0Rs0jvsd+SvsqHC4gCBxw5NEKN2OYg+dhytCafsUH+J45jZEyo irSfEbg2+UmzT/9fzHCCkPQnES7Xv78rfKXBCzVnpMdb8RVRwS27eUyLKlXT2bu9 S4Pn4ap7y9a94XeUQmJkhM/WVMRpEV1ZQXpZ5NTbHt5NERClR6Cb/xExvPL2en5T BQSYdhMqR0+KPMMnRBLZvw8d1Zh7HMh/vj5f/UHKQynp0IHt08rEDEJOQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=uapF96GLjk13ke5x2Q/30LebnRbPauH8wMHhKw/nAxc=; b=E1RmMWDT a3mCYmSUsX0PL7EDH7EjNH9jw7UtNJhT3JQId+4AUP5gnbM5s+P0BRptVV+3bKzE uoWZwszUotD1RwVqtYRShut9l2e5hLm+8UDcGn4lM1+GCg2t67ZHY8AZ+y4Ss/38 k/CkqAhFAm4zOGVe8hkL/x1QB47hDIR+4S4Dg2wKKF46773XKdoni4+Lo1ba5pf4 aU7BugBGEh7zNz5yGJ1Psxa+Ey+9fAit4x4cj0r6zQ32o6EnkJVJBp7MbUaVHdhg nOWDBF6LOE93LVGg/wU4OJWWI+XSFlp9SvW3i7SGuAt/G1A46eKdFE74mgLmY2TG io0J5SZ730aLnQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id B8065240066; Wed, 10 Feb 2021 10:34:19 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:06 +0100 Message-Id: <1bb4361ecaf090ddab368955eca248c291cb202e.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 20/23] dpif-netdev: Make megaflow and mark mappings thread objects X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In later commits hardware offloads are managed in several threads. Each offload is managed by a thread determined by its flow's 'mega_ufid'. As megaflow to mark and mark to flow mappings are 1:1 and 1:N respectively, then a single mark exists for a single 'mega_ufid', and multiple flows uses the same 'mega_ufid'. Because the managing thread will be choosen using the 'mega_ufid', then each mapping does not need to be shared with other offload threads. The mappings are kept as cmap as upcalls will sometimes query them before enqueuing orders to the offload threads. To prepare this change, move the mappings within the offload thread structure. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 41 +++++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 09d62a3d5..913edff27 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -441,12 +441,16 @@ struct dp_offload_thread_item { struct dp_offload_thread { struct mpsc_queue queue; atomic_uint64_t enqueued_item; + struct cmap megaflow_to_mark; + struct cmap mark_to_flow; struct mov_avg_cma cma; struct mov_avg_ema ema; }; static struct dp_offload_thread dp_offload_thread = { .queue = MPSC_QUEUE_INITIALIZER(&dp_offload_thread.queue), + .megaflow_to_mark = CMAP_INITIALIZER, + .mark_to_flow = CMAP_INITIALIZER, .enqueued_item = ATOMIC_VAR_INIT(0), .cma = MOV_AVG_CMA_INITIALIZER, .ema = MOV_AVG_EMA_INITIALIZER(100), @@ -2413,16 +2417,7 @@ struct megaflow_to_mark_data { uint32_t mark; }; -struct flow_mark { - struct cmap megaflow_to_mark; - struct cmap mark_to_flow; - struct seq_pool *pool; -}; - -static struct flow_mark flow_mark = { - .megaflow_to_mark = CMAP_INITIALIZER, - .mark_to_flow = CMAP_INITIALIZER, -}; +static struct seq_pool *flow_mark_pool; static uint32_t flow_mark_alloc(void) @@ -2433,12 +2428,12 @@ flow_mark_alloc(void) if (ovsthread_once_start(&pool_init)) { /* Haven't initiated yet, do it here */ - flow_mark.pool = seq_pool_create(netdev_offload_thread_nb(), + flow_mark_pool = seq_pool_create(netdev_offload_thread_nb(), 1, MAX_FLOW_MARK); ovsthread_once_done(&pool_init); } - if (seq_pool_new_id(flow_mark.pool, tid, &mark)) { + if (seq_pool_new_id(flow_mark_pool, tid, &mark)) { return mark; } @@ -2450,7 +2445,7 @@ flow_mark_free(uint32_t mark) { unsigned int tid = netdev_offload_thread_id(); - seq_pool_free_id(flow_mark.pool, tid, mark); + seq_pool_free_id(flow_mark_pool, tid, mark); } /* associate megaflow with a mark, which is a 1:1 mapping */ @@ -2463,7 +2458,7 @@ megaflow_to_mark_associate(const ovs_u128 *mega_ufid, uint32_t mark) data->mega_ufid = *mega_ufid; data->mark = mark; - cmap_insert(&flow_mark.megaflow_to_mark, + cmap_insert(&dp_offload_thread.megaflow_to_mark, CONST_CAST(struct cmap_node *, &data->node), hash); } @@ -2474,9 +2469,10 @@ megaflow_to_mark_disassociate(const ovs_u128 *mega_ufid) size_t hash = dp_netdev_flow_hash(mega_ufid); struct megaflow_to_mark_data *data; - CMAP_FOR_EACH_WITH_HASH (data, node, hash, &flow_mark.megaflow_to_mark) { + CMAP_FOR_EACH_WITH_HASH (data, node, hash, + &dp_offload_thread.megaflow_to_mark) { if (ovs_u128_equals(*mega_ufid, data->mega_ufid)) { - cmap_remove(&flow_mark.megaflow_to_mark, + cmap_remove(&dp_offload_thread.megaflow_to_mark, CONST_CAST(struct cmap_node *, &data->node), hash); ovsrcu_postpone(free, data); return; @@ -2493,7 +2489,8 @@ megaflow_to_mark_find(const ovs_u128 *mega_ufid) size_t hash = dp_netdev_flow_hash(mega_ufid); struct megaflow_to_mark_data *data; - CMAP_FOR_EACH_WITH_HASH (data, node, hash, &flow_mark.megaflow_to_mark) { + CMAP_FOR_EACH_WITH_HASH (data, node, hash, + &dp_offload_thread.megaflow_to_mark) { if (ovs_u128_equals(*mega_ufid, data->mega_ufid)) { return data->mark; } @@ -2510,7 +2507,7 @@ mark_to_flow_associate(const uint32_t mark, struct dp_netdev_flow *flow) { dp_netdev_flow_ref(flow); - cmap_insert(&flow_mark.mark_to_flow, + cmap_insert(&dp_offload_thread.mark_to_flow, CONST_CAST(struct cmap_node *, &flow->mark_node), hash_int(mark, 0)); flow->mark = mark; @@ -2525,7 +2522,7 @@ flow_mark_has_no_ref(uint32_t mark) struct dp_netdev_flow *flow; CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0), - &flow_mark.mark_to_flow) { + &dp_offload_thread.mark_to_flow) { if (flow->mark == mark) { return false; } @@ -2550,7 +2547,7 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, return EINVAL; } - cmap_remove(&flow_mark.mark_to_flow, mark_node, hash_int(mark, 0)); + cmap_remove(&dp_offload_thread.mark_to_flow, mark_node, hash_int(mark, 0)); flow->mark = INVALID_FLOW_MARK; /* @@ -2587,7 +2584,7 @@ flow_mark_flush(struct dp_netdev_pmd_thread *pmd) { struct dp_netdev_flow *flow; - CMAP_FOR_EACH (flow, mark_node, &flow_mark.mark_to_flow) { + CMAP_FOR_EACH (flow, mark_node, &dp_offload_thread.mark_to_flow) { if (flow->pmd_id == pmd->core_id) { queue_netdev_flow_del(pmd, flow); } @@ -2601,7 +2598,7 @@ mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd, struct dp_netdev_flow *flow; CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0), - &flow_mark.mark_to_flow) { + &dp_offload_thread.mark_to_flow) { if (flow->mark == mark && flow->pmd_id == pmd->core_id && flow->dead == false) { return flow; From patchwork Wed Feb 10 15:34:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=inumsFBz; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=umYrSKeJ; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP525Fh2z9sB4 for ; Thu, 11 Feb 2021 02:36:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 4476186D7E; Wed, 10 Feb 2021 15:36:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7OcdFZuTJq-x; Wed, 10 Feb 2021 15:36:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 25F2887221; Wed, 10 Feb 2021 15:34:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F1719C1E79; Wed, 10 Feb 2021 15:34:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0EA18C1DA9 for ; Wed, 10 Feb 2021 15:34:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E788D6F8B6 for ; Wed, 10 Feb 2021 15:34:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MAR2GWj9gIdJ for ; Wed, 10 Feb 2021 15:34:34 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 619306F78A; Wed, 10 Feb 2021 15:34:32 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3A1416F6CE for ; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id 850B6E18; Wed, 10 Feb 2021 10:34:21 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=h1umTmu4reqT9 PMfEWLOmsQNx6cMx0n8Z7sg6yCJtw4=; b=inumsFBzBh5TVytHNDVcClyfGzWvs SxBZZ/QVKjClC+5XgKrXzGDOKZKCgy/qjV3Y5lbsgSdDRmImpYwz+0LylKJMNSBF 07M2Cq3T7lje/gPw/M7/gHM9wYabVNabQ1BDf28L/7H2GHSeMtEgG/4QqgQ9cwkz FEvdayJBPzIkKHOggQ/IMNjz9OgQUPbILll7Ta1mp4XMPlh71jVfrQ+ESI/ISA3L nVIUblAluCCSavDqfFU7gmSLsKz0cCypDpAhGUofk8yjQIUmI5OQv3qybJLSkf22 OVyNnv1qYb/BdweZI03zpM0QeTrhIJPyKDYpfY8bgxpXVpRQAvRV//FmA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=h1umTmu4reqT9PMfEWLOmsQNx6cMx0n8Z7sg6yCJtw4=; b=umYrSKeJ cgOnuLoj6wqH6XgCzdnQeE1QUR4Ym2VuNE9MzwFxJ515tFC+Z6hdGVHdJl9uMIT4 28cwzHWYWZm9CobCXZCUcbiJ+5PpQlC9A0zzr4iAta6cCN3uBsTHXvZXtVfH1CbJ xPy4q4CsNC3LIFsWCus9KjCT4r8dlkPwCYgPtu+I0LRxskOxtJQmdRbjNsAQ64iq kWPhjRfpd++pjxPWzpbJ8f+otPfenzZoiYwCQF+F3FFlwnQv9AZaB8gl9ZiI6D4B 0dhOOoTJkaGMgOOAfK5D9K0u5gLgsE+eXFQVeUyLNDND3uHkkqQ/h6/xd7QPVaHp joCQfLH6SdIfjQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgepfeenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 4561524005A; Wed, 10 Feb 2021 10:34:20 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:07 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 21/23] dpif-netdev: Replace port mutex by rwlock X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The port mutex protects the netdev mapping, that can be changed by port addition or port deletion. HW offloads operations can be considered read operations on the port mapping itself. Use a rwlock to differentiate between read and write operations, allowing concurrent queries and offload insertions. Because offload queries, deletion, and reconfigure_datapath() calls are all rdlock, the deadlock fixed by [1] is still avoided, as the rdlock side is recursive as prescribed by the POSIX standard. Executing 'reconfigure_datapath()' only requires a rdlock taken, but it is sometimes executed in contexts where wrlock is taken ('do_add_port()' and 'do_del_port()'). This means that the deadlock described in [2] is still valid and should be mitigated. The rdlock is taken using 'tryrdlock()' during offload query, keeping the current behavior. [1]: 81e89d5c2645 ("dpif-netdev: Make datapath port mutex recursive.") [2]: 12d0edd75eba ("dpif-netdev: Avoid deadlock with offloading during PMD thread deletion."). Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 127 +++++++++++++++++++------------------- lib/netdev-offload-dpdk.c | 2 +- 2 files changed, 65 insertions(+), 64 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 913edff27..81d6ce94f 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -332,8 +332,8 @@ struct dp_netdev { /* Ports. * * Any lookup into 'ports' or any access to the dp_netdev_ports found - * through 'ports' requires taking 'port_mutex'. */ - struct ovs_mutex port_mutex; + * through 'ports' requires taking 'port_rwlock'. */ + struct ovs_rwlock port_rwlock; struct hmap ports; struct seq *port_seq; /* Incremented whenever a port changes. */ @@ -409,7 +409,7 @@ static void meter_unlock(const struct dp_netdev *dp, uint32_t meter_id) static struct dp_netdev_port *dp_netdev_lookup_port(const struct dp_netdev *dp, odp_port_t) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_RDLOCK(dp->port_rwlock); enum rxq_cycles_counter_type { RXQ_CYCLES_PROC_CURR, /* Cycles spent successfully polling and @@ -830,17 +830,17 @@ struct dpif_netdev { static int get_port_by_number(struct dp_netdev *dp, odp_port_t port_no, struct dp_netdev_port **portp) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_RDLOCK(dp->port_rwlock); static int get_port_by_name(struct dp_netdev *dp, const char *devname, struct dp_netdev_port **portp) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_RDLOCK(dp->port_rwlock); static void dp_netdev_free(struct dp_netdev *) OVS_REQUIRES(dp_netdev_mutex); static int do_add_port(struct dp_netdev *dp, const char *devname, const char *type, odp_port_t port_no) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_WRLOCK(dp->port_rwlock); static void do_del_port(struct dp_netdev *dp, struct dp_netdev_port *) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_WRLOCK(dp->port_rwlock); static int dpif_netdev_open(const struct dpif_class *, const char *name, bool create, struct dpif **); static void dp_netdev_execute_actions(struct dp_netdev_pmd_thread *pmd, @@ -861,7 +861,7 @@ static void dp_netdev_configure_pmd(struct dp_netdev_pmd_thread *pmd, int numa_id); static void dp_netdev_destroy_pmd(struct dp_netdev_pmd_thread *pmd); static void dp_netdev_set_nonpmd(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_WRLOCK(dp->port_rwlock); static void *pmd_thread_main(void *); static struct dp_netdev_pmd_thread *dp_netdev_get_pmd(struct dp_netdev *dp, @@ -895,7 +895,7 @@ static void dp_netdev_del_bond_tx_from_pmd(struct dp_netdev_pmd_thread *pmd, OVS_EXCLUDED(pmd->bond_mutex); static void reconfigure_datapath(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex); + OVS_REQ_RDLOCK(dp->port_rwlock); static bool dp_netdev_pmd_try_ref(struct dp_netdev_pmd_thread *pmd); static void dp_netdev_pmd_unref(struct dp_netdev_pmd_thread *pmd); static void dp_netdev_pmd_flow_flush(struct dp_netdev_pmd_thread *pmd); @@ -1401,8 +1401,8 @@ dpif_netdev_subtable_lookup_set(struct unixctl_conn *conn, int argc, struct dp_netdev_pmd_thread **pmd_list; sorted_poll_thread_list(dp, &pmd_list, &n); - /* take port mutex as HMAP iters over them. */ - ovs_mutex_lock(&dp->port_mutex); + /* take port rwlock as HMAP iters over them. */ + ovs_rwlock_rdlock(&dp->port_rwlock); for (size_t i = 0; i < n; i++) { struct dp_netdev_pmd_thread *pmd = pmd_list[i]; @@ -1425,8 +1425,8 @@ dpif_netdev_subtable_lookup_set(struct unixctl_conn *conn, int argc, } } - /* release port mutex before netdev mutex. */ - ovs_mutex_unlock(&dp->port_mutex); + /* release port rwlock before netdev mutex. */ + ovs_rwlock_unlock(&dp->port_rwlock); ovs_mutex_unlock(&dp_netdev_mutex); struct ds reply = DS_EMPTY_INITIALIZER; @@ -1719,7 +1719,7 @@ create_dpif_netdev(struct dp_netdev *dp) * Return ODPP_NONE on failure. */ static odp_port_t choose_port(struct dp_netdev *dp, const char *name) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { uint32_t port_no; @@ -1782,7 +1782,7 @@ create_dp_netdev(const char *name, const struct dpif_class *class, ovs_refcount_init(&dp->ref_cnt); atomic_flag_clear(&dp->destroyed); - ovs_mutex_init_recursive(&dp->port_mutex); + ovs_rwlock_init(&dp->port_rwlock); hmap_init(&dp->ports); dp->port_seq = seq_create(); ovs_mutex_init(&dp->bond_mutex); @@ -1817,7 +1817,7 @@ create_dp_netdev(const char *name, const struct dpif_class *class, ovs_mutex_init_recursive(&dp->non_pmd_mutex); ovsthread_key_create(&dp->per_pmd_key, NULL); - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); /* non-PMD will be created before all other threads and will * allocate static_tx_qid = 0. */ dp_netdev_set_nonpmd(dp); @@ -1825,7 +1825,7 @@ create_dp_netdev(const char *name, const struct dpif_class *class, error = do_add_port(dp, name, dpif_netdev_port_open_type(dp->class, "internal"), ODPP_LOCAL); - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); if (error) { dp_netdev_free(dp); return error; @@ -1911,11 +1911,11 @@ dp_netdev_free(struct dp_netdev *dp) shash_find_and_delete(&dp_netdevs, dp->name); - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); HMAP_FOR_EACH_SAFE (port, next, node, &dp->ports) { do_del_port(dp, port); } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); ovs_mutex_lock(&dp->bond_mutex); CMAP_FOR_EACH (bond, node, &dp->tx_bonds) { @@ -1940,7 +1940,7 @@ dp_netdev_free(struct dp_netdev *dp) seq_destroy(dp->port_seq); hmap_destroy(&dp->ports); - ovs_mutex_destroy(&dp->port_mutex); + ovs_rwlock_destroy(&dp->port_rwlock); cmap_destroy(&dp->tx_bonds); ovs_mutex_destroy(&dp->bond_mutex); @@ -2108,7 +2108,7 @@ out: static int do_add_port(struct dp_netdev *dp, const char *devname, const char *type, odp_port_t port_no) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_WRLOCK(dp->port_rwlock) { struct netdev_saved_flags *sf; struct dp_netdev_port *port; @@ -2160,7 +2160,7 @@ dpif_netdev_port_add(struct dpif *dpif, struct netdev *netdev, odp_port_t port_no; int error; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); dpif_port = netdev_vport_get_dpif_port(netdev, namebuf, sizeof namebuf); if (*port_nop != ODPP_NONE) { port_no = *port_nop; @@ -2173,7 +2173,7 @@ dpif_netdev_port_add(struct dpif *dpif, struct netdev *netdev, *port_nop = port_no; error = do_add_port(dp, dpif_port, netdev_get_type(netdev), port_no); } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return error; } @@ -2184,7 +2184,7 @@ dpif_netdev_port_del(struct dpif *dpif, odp_port_t port_no) struct dp_netdev *dp = get_dp_netdev(dpif); int error; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); if (port_no == ODPP_LOCAL) { error = EINVAL; } else { @@ -2195,7 +2195,7 @@ dpif_netdev_port_del(struct dpif *dpif, odp_port_t port_no) do_del_port(dp, port); } } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return error; } @@ -2208,7 +2208,7 @@ is_valid_port_number(odp_port_t port_no) static struct dp_netdev_port * dp_netdev_lookup_port(const struct dp_netdev *dp, odp_port_t port_no) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; @@ -2223,7 +2223,7 @@ dp_netdev_lookup_port(const struct dp_netdev *dp, odp_port_t port_no) static int get_port_by_number(struct dp_netdev *dp, odp_port_t port_no, struct dp_netdev_port **portp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { if (!is_valid_port_number(port_no)) { *portp = NULL; @@ -2258,7 +2258,7 @@ port_destroy(struct dp_netdev_port *port) static int get_port_by_name(struct dp_netdev *dp, const char *devname, struct dp_netdev_port **portp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; @@ -2277,7 +2277,7 @@ get_port_by_name(struct dp_netdev *dp, /* Returns 'true' if there is a port with pmd netdev. */ static bool has_pmd_port(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; @@ -2292,7 +2292,7 @@ has_pmd_port(struct dp_netdev *dp) static void do_del_port(struct dp_netdev *dp, struct dp_netdev_port *port) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_WRLOCK(dp->port_rwlock) { netdev_flow_flush(port->netdev); netdev_uninit_flow_api(port->netdev); @@ -2321,12 +2321,12 @@ dpif_netdev_port_query_by_number(const struct dpif *dpif, odp_port_t port_no, struct dp_netdev_port *port; int error; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); error = get_port_by_number(dp, port_no, &port); if (!error && dpif_port) { answer_port_query(port, dpif_port); } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return error; } @@ -2339,12 +2339,12 @@ dpif_netdev_port_query_by_name(const struct dpif *dpif, const char *devname, struct dp_netdev_port *port; int error; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_rdlock(&dp->port_rwlock); error = get_port_by_name(dp, devname, &port); if (!error && dpif_port) { answer_port_query(port, dpif_port); } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return error; } @@ -2562,9 +2562,9 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, if (port) { /* Taking a global 'port_mutex' to fulfill thread safety * restrictions regarding netdev port mapping. */ - ovs_mutex_lock(&pmd->dp->port_mutex); + ovs_rwlock_rdlock(&pmd->dp->port_rwlock); ret = netdev_flow_del(port, &flow->mega_ufid, NULL); - ovs_mutex_unlock(&pmd->dp->port_mutex); + ovs_rwlock_unlock(&pmd->dp->port_rwlock); netdev_close(port); } @@ -2716,12 +2716,12 @@ dp_netdev_flow_offload_put(struct dp_offload_thread_item *offload) } /* Taking a global 'port_mutex' to fulfill thread safety * restrictions regarding the netdev port mapping. */ - ovs_mutex_lock(&pmd->dp->port_mutex); + ovs_rwlock_rdlock(&pmd->dp->port_rwlock); ret = netdev_flow_put(port, &offload->match, CONST_CAST(struct nlattr *, offload->actions), offload->actions_len, &flow->mega_ufid, &info, NULL); - ovs_mutex_unlock(&pmd->dp->port_mutex); + ovs_rwlock_unlock(&pmd->dp->port_rwlock); netdev_close(port); if (ret) { @@ -2949,7 +2949,7 @@ dpif_netdev_port_dump_next(const struct dpif *dpif, void *state_, struct hmap_node *node; int retval; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_rdlock(&dp->port_rwlock); node = hmap_at_position(&dp->ports, &state->position); if (node) { struct dp_netdev_port *port; @@ -2966,7 +2966,7 @@ dpif_netdev_port_dump_next(const struct dpif *dpif, void *state_, } else { retval = EOF; } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return retval; } @@ -3417,24 +3417,24 @@ dpif_netdev_get_flow_offload_status(const struct dp_netdev *dp, return false; } ofpbuf_use_stack(&buf, &act_buf, sizeof act_buf); - /* Taking a global 'port_mutex' to fulfill thread safety + /* Taking a global 'port_rwlock' to fulfill thread safety * restrictions regarding netdev port mapping. * * XXX: Main thread will try to pause/stop all revalidators during datapath * reconfiguration via datapath purge callback (dp_purge_cb) while - * holding 'dp->port_mutex'. So we're not waiting for mutex here. - * Otherwise, deadlock is possible, bcause revalidators might sleep + * rw-holding 'dp->port_rwlock'. So we're not waiting for lock here. + * Otherwise, deadlock is possible, because revalidators might sleep * waiting for the main thread to release the lock and main thread * will wait for them to stop processing. * This workaround might make statistics less accurate. Especially * for flow deletion case, since there will be no other attempt. */ - if (!ovs_mutex_trylock(&dp->port_mutex)) { + if (!ovs_rwlock_tryrdlock(&dp->port_rwlock)) { ret = netdev_flow_get(netdev, &match, &actions, &netdev_flow->mega_ufid, stats, attrs, &buf); /* Storing statistics and attributes from the last request for * later use on mutex contention. */ dp_netdev_flow_set_last_stats_attrs(netdev_flow, stats, attrs, ret); - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); } else { dp_netdev_flow_get_last_stats_attrs(netdev_flow, stats, attrs, &ret); if (!ret && !attrs->dp_layer) { @@ -4295,7 +4295,7 @@ dpif_netdev_offload_stats_get(struct dpif *dpif, nb_offloads = 0; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_rdlock(&dp->port_rwlock); HMAP_FOR_EACH (port, node, &dp->ports) { uint64_t port_nb_offloads = 0; @@ -4304,7 +4304,7 @@ dpif_netdev_offload_stats_get(struct dpif *dpif, nb_offloads += port_nb_offloads; } } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); atomic_read_relaxed(&dp_offload_thread.enqueued_item, &stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED].value); @@ -4610,7 +4610,7 @@ dpif_netdev_port_set_config(struct dpif *dpif, odp_port_t port_no, const char *affinity_list = smap_get(cfg, "pmd-rxq-affinity"); bool emc_enabled = smap_get_bool(cfg, "emc-enable", true); - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); error = get_port_by_number(dp, port_no, &port); if (error) { goto unlock; @@ -4664,7 +4664,7 @@ dpif_netdev_port_set_config(struct dpif *dpif, odp_port_t port_no, dp_netdev_request_reconfigure(dp); unlock: - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); return error; } @@ -5148,7 +5148,8 @@ compare_rxq_cycles(const void *a, const void *b) * The function doesn't touch the pmd threads, it just stores the assignment * in the 'pmd' member of each rxq. */ static void -rxq_scheduling(struct dp_netdev *dp, bool pinned) OVS_REQUIRES(dp->port_mutex) +rxq_scheduling(struct dp_netdev *dp, bool pinned) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; struct rr_numa_list rr; @@ -5292,7 +5293,7 @@ reload_affected_pmds(struct dp_netdev *dp) static void reconfigure_pmd_threads(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_pmd_thread *pmd; struct ovs_numa_dump *pmd_cores; @@ -5390,7 +5391,7 @@ static void pmd_remove_stale_ports(struct dp_netdev *dp, struct dp_netdev_pmd_thread *pmd) OVS_EXCLUDED(pmd->port_mutex) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct rxq_poll *poll, *poll_next; struct tx_port *tx, *tx_next; @@ -5420,7 +5421,7 @@ pmd_remove_stale_ports(struct dp_netdev *dp, * rxqs and assigns all rxqs/txqs to pmd threads. */ static void reconfigure_datapath(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct hmapx busy_threads = HMAPX_INITIALIZER(&busy_threads); struct dp_netdev_pmd_thread *pmd; @@ -5604,7 +5605,7 @@ reconfigure_datapath(struct dp_netdev *dp) /* Returns true if one of the netdevs in 'dp' requires a reconfiguration */ static bool ports_require_restart(const struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; @@ -5655,7 +5656,7 @@ variance(uint64_t a[], int n) static bool get_dry_run_variance(struct dp_netdev *dp, uint32_t *core_list, uint32_t num_pmds, uint64_t *predicted_variance) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_port *port; struct dp_netdev_pmd_thread *pmd; @@ -5771,7 +5772,7 @@ cleanup: * better distribution of load on PMDs. */ static bool pmd_rebalance_dry_run(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_RDLOCK(dp->port_rwlock) { struct dp_netdev_pmd_thread *pmd; uint64_t *curr_pmd_usage; @@ -5866,7 +5867,7 @@ dpif_netdev_run(struct dpif *dpif) long long int now = time_msec(); struct dp_netdev_pmd_thread *pmd; - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_rdlock(&dp->port_rwlock); non_pmd = dp_netdev_get_pmd(dp, NON_PMD_CORE_ID); if (non_pmd) { ovs_mutex_lock(&dp->non_pmd_mutex); @@ -5938,7 +5939,7 @@ dpif_netdev_run(struct dpif *dpif) if (dp_netdev_is_reconf_required(dp) || ports_require_restart(dp)) { reconfigure_datapath(dp); } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); tnl_neigh_cache_run(); tnl_port_map_run(); @@ -5958,7 +5959,7 @@ dpif_netdev_wait(struct dpif *dpif) struct dp_netdev *dp = get_dp_netdev(dpif); ovs_mutex_lock(&dp_netdev_mutex); - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_rdlock(&dp->port_rwlock); HMAP_FOR_EACH (port, node, &dp->ports) { netdev_wait_reconf_required(port->netdev); if (!netdev_is_pmd(port->netdev)) { @@ -5969,7 +5970,7 @@ dpif_netdev_wait(struct dpif *dpif) } } } - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); ovs_mutex_unlock(&dp_netdev_mutex); seq_wait(tnl_conf_seq, dp->last_tnl_conf_seq); } @@ -6586,7 +6587,7 @@ dp_netdev_get_pmd(struct dp_netdev *dp, unsigned core_id) /* Sets the 'struct dp_netdev_pmd_thread' for non-pmd threads. */ static void dp_netdev_set_nonpmd(struct dp_netdev *dp) - OVS_REQUIRES(dp->port_mutex) + OVS_REQ_WRLOCK(dp->port_rwlock) { struct dp_netdev_pmd_thread *non_pmd; @@ -8649,7 +8650,7 @@ dpif_dummy_change_port_number(struct unixctl_conn *conn, int argc OVS_UNUSED, ovs_refcount_ref(&dp->ref_cnt); ovs_mutex_unlock(&dp_netdev_mutex); - ovs_mutex_lock(&dp->port_mutex); + ovs_rwlock_wrlock(&dp->port_rwlock); if (get_port_by_name(dp, argv[2], &port)) { unixctl_command_reply_error(conn, "unknown port"); goto exit; @@ -8678,7 +8679,7 @@ dpif_dummy_change_port_number(struct unixctl_conn *conn, int argc OVS_UNUSED, unixctl_command_reply(conn, NULL); exit: - ovs_mutex_unlock(&dp->port_mutex); + ovs_rwlock_unlock(&dp->port_rwlock); dp_netdev_unref(dp); } diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 72cff5688..9c6b3fea8 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -45,7 +45,7 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(100, 5); * 'netdev' is forbidden. * * For current implementation all above restrictions could be fulfilled by - * taking the datapath 'port_mutex' in lib/dpif-netdev.c. */ + * taking the datapath 'port_rwlock' in lib/dpif-netdev.c. */ /* * A mapping from ufid to dpdk rte_flow. From patchwork Wed Feb 10 15:34:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439086 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=e0MsozIR; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=nrGrZ58E; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP5D4xZJz9rx8 for ; Thu, 11 Feb 2021 02:37:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D85988752A; Wed, 10 Feb 2021 15:36:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q9vWqxJoYiB6; Wed, 10 Feb 2021 15:36:47 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id CE4E687242; Wed, 10 Feb 2021 15:34:44 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A68F6C1E75; Wed, 10 Feb 2021 15:34:44 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id D8B8BC1E6F for ; Wed, 10 Feb 2021 15:34:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id B2BDE6F7A0 for ; Wed, 10 Feb 2021 15:34:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cJW6i9bYQuBY for ; Wed, 10 Feb 2021 15:34:37 +0000 (UTC) Received: by smtp3.osuosl.org (Postfix, from userid 1001) id 3FDA96F8A6; Wed, 10 Feb 2021 15:34:37 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by smtp3.osuosl.org (Postfix) with ESMTPS id D28916F714 for ; Wed, 10 Feb 2021 15:34:23 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailnew.west.internal (Postfix) with ESMTP id 3FF96E89; Wed, 10 Feb 2021 10:34:23 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Wed, 10 Feb 2021 10:34:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=vjOImy0c4G8M/ myIHuQs3555lrjKVJJmYu0QRzVrIto=; b=e0MsozIRvb2XJpuIwZk/7MXllapkP m64JJYwOROlM+eMT9zTPWqAxKgeHCoi1uGqHoajVYSOF0DEJEPb6wfXnHxzRAn8v /zwEkvmb2CLseJ5xALRAkB3GZlFwVnlMdTYB4Gr5yxDEx1GP8WG6haSH/XUsoMVs sLr2PMYfo5CwXAzeT7x2pr8spw0yq9VlUyUFw4+N8CybGCDctzn96FDEJp5EMdRT vYG02MjWx1DNmIzsomBrlYIYKG2vm/a+Gj56Uu3p6ZzSDeoE20diG3F6+76DTUga lSNxhNpJwym8FpP6kjJQAl5RM5WTpsoGgy6dhD2UYEEqsbb09vjeL/76w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=vjOImy0c4G8M/myIHuQs3555lrjKVJJmYu0QRzVrIto=; b=nrGrZ58E ko06G+h6qkhpH7iXqEgUGF9ZvYqqklhKhQ7HSUaOikwgoF2Oif4IGi7VJI+S6wh+ VM+Wx80QeQrM9TUMb5Q/K6UoYhMXmgv5UYQGBOXg7OctOAYjVLFjGsEkULbtoa+B hWUgrP4w5D6+qLl1gQ0ah5xGpxLzYST6AqDzCcdqj8RVGkpW+eBneUmxNlkAr+ay cwgTWgI/gbJKQaFzjYYEEpJ9/StUSGA9lGOOjvvy4EpILBcGnymtGz7Jc0tfOzxg /IujJfEZUj+lmPwsLr+2qfzSG3qN0CA26AtkGrXhxFRbAqxPPMqQom8re+BmFaRB fp2H5YPAt/EuFQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe eklefhfffgkedugffgudelueekgfehgfdthfdvkeeuhfffuedtteevveduiedvgfenucff ohhmrghinhepmhgvrghnrdhtohhtrghlpdhsthguuggvvhdrthhothgrlhenucfkphepke eirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghm pehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id CA0BA240064; Wed, 10 Feb 2021 10:34:20 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:08 +0100 Message-Id: <4b8c5893c7431f6967b958ca03babf8a8b5cb543.1612968146.git.grive@u256.net> X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 22/23] dpif-netdev: Use one or more offload threads X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Read the user configuration in the netdev-offload module to modify the number of threads used to manage hardware offload requests. This allows processing insertion, deletion and modification concurrently. The offload thread structure was modified to contain all needed elements. This structure is multiplied by the number of requested threads and used separately. ovs_strlcpy silently fails to copy the threads name if it is too long. Rename the flow offload thread to differentiate it from the main thread. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/dpif-netdev.c | 237 ++++++++++++++++++++++++++++++---------------- 1 file changed, 157 insertions(+), 80 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 81d6ce94f..ea295893d 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -439,25 +439,47 @@ struct dp_offload_thread_item { }; struct dp_offload_thread { - struct mpsc_queue queue; - atomic_uint64_t enqueued_item; - struct cmap megaflow_to_mark; - struct cmap mark_to_flow; - struct mov_avg_cma cma; - struct mov_avg_ema ema; + PADDED_MEMBERS(CACHE_LINE_SIZE, + struct mpsc_queue queue; + atomic_uint64_t enqueued_item; + struct cmap megaflow_to_mark; + struct cmap mark_to_flow; + struct mov_avg_cma cma; + struct mov_avg_ema ema; + ); }; +static struct dp_offload_thread *dp_offload_threads; +static void *dp_netdev_flow_offload_main(void *arg); -static struct dp_offload_thread dp_offload_thread = { - .queue = MPSC_QUEUE_INITIALIZER(&dp_offload_thread.queue), - .megaflow_to_mark = CMAP_INITIALIZER, - .mark_to_flow = CMAP_INITIALIZER, - .enqueued_item = ATOMIC_VAR_INIT(0), - .cma = MOV_AVG_CMA_INITIALIZER, - .ema = MOV_AVG_EMA_INITIALIZER(100), -}; +static void +dp_netdev_offload_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + unsigned int nb_offload_thread = netdev_offload_thread_nb(); + unsigned int tid; + + if (!ovsthread_once_start(&once)) { + return; + } -static struct ovsthread_once offload_thread_once - = OVSTHREAD_ONCE_INITIALIZER; + dp_offload_threads = xcalloc(nb_offload_thread, + sizeof *dp_offload_threads); + + for (tid = 0; tid < nb_offload_thread; tid++) { + struct dp_offload_thread *thread; + + thread = &dp_offload_threads[tid]; + mpsc_queue_init(&thread->queue); + cmap_init(&thread->megaflow_to_mark); + cmap_init(&thread->mark_to_flow); + atomic_init(&thread->enqueued_item, 0); + mov_avg_cma_init(&thread->cma); + mov_avg_ema_init(&thread->ema, 100); + ovs_thread_create("hw_offload", dp_netdev_flow_offload_main, thread); + } + + ovsthread_once_done(&once); +} #define XPS_TIMEOUT 500000LL /* In microseconds. */ @@ -2454,11 +2476,12 @@ megaflow_to_mark_associate(const ovs_u128 *mega_ufid, uint32_t mark) { size_t hash = dp_netdev_flow_hash(mega_ufid); struct megaflow_to_mark_data *data = xzalloc(sizeof(*data)); + unsigned int tid = netdev_offload_thread_id(); data->mega_ufid = *mega_ufid; data->mark = mark; - cmap_insert(&dp_offload_thread.megaflow_to_mark, + cmap_insert(&dp_offload_threads[tid].megaflow_to_mark, CONST_CAST(struct cmap_node *, &data->node), hash); } @@ -2468,11 +2491,12 @@ megaflow_to_mark_disassociate(const ovs_u128 *mega_ufid) { size_t hash = dp_netdev_flow_hash(mega_ufid); struct megaflow_to_mark_data *data; + unsigned int tid = netdev_offload_thread_id(); CMAP_FOR_EACH_WITH_HASH (data, node, hash, - &dp_offload_thread.megaflow_to_mark) { + &dp_offload_threads[tid].megaflow_to_mark) { if (ovs_u128_equals(*mega_ufid, data->mega_ufid)) { - cmap_remove(&dp_offload_thread.megaflow_to_mark, + cmap_remove(&dp_offload_threads[tid].megaflow_to_mark, CONST_CAST(struct cmap_node *, &data->node), hash); ovsrcu_postpone(free, data); return; @@ -2488,9 +2512,10 @@ megaflow_to_mark_find(const ovs_u128 *mega_ufid) { size_t hash = dp_netdev_flow_hash(mega_ufid); struct megaflow_to_mark_data *data; + unsigned int tid = netdev_offload_thread_id(); CMAP_FOR_EACH_WITH_HASH (data, node, hash, - &dp_offload_thread.megaflow_to_mark) { + &dp_offload_threads[tid].megaflow_to_mark) { if (ovs_u128_equals(*mega_ufid, data->mega_ufid)) { return data->mark; } @@ -2505,9 +2530,10 @@ megaflow_to_mark_find(const ovs_u128 *mega_ufid) static void mark_to_flow_associate(const uint32_t mark, struct dp_netdev_flow *flow) { + unsigned int tid = netdev_offload_thread_id(); dp_netdev_flow_ref(flow); - cmap_insert(&dp_offload_thread.mark_to_flow, + cmap_insert(&dp_offload_threads[tid].mark_to_flow, CONST_CAST(struct cmap_node *, &flow->mark_node), hash_int(mark, 0)); flow->mark = mark; @@ -2519,10 +2545,11 @@ mark_to_flow_associate(const uint32_t mark, struct dp_netdev_flow *flow) static bool flow_mark_has_no_ref(uint32_t mark) { + unsigned int tid = netdev_offload_thread_id(); struct dp_netdev_flow *flow; CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0), - &dp_offload_thread.mark_to_flow) { + &dp_offload_threads[tid].mark_to_flow) { if (flow->mark == mark) { return false; } @@ -2538,6 +2565,7 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, const char *dpif_type_str = dpif_normalize_type(pmd->dp->class->type); struct cmap_node *mark_node = CONST_CAST(struct cmap_node *, &flow->mark_node); + unsigned int tid = netdev_offload_thread_id(); uint32_t mark = flow->mark; int ret = 0; @@ -2547,7 +2575,8 @@ mark_to_flow_disassociate(struct dp_netdev_pmd_thread *pmd, return EINVAL; } - cmap_remove(&dp_offload_thread.mark_to_flow, mark_node, hash_int(mark, 0)); + cmap_remove(&dp_offload_threads[tid].mark_to_flow, + mark_node, hash_int(mark, 0)); flow->mark = INVALID_FLOW_MARK; /* @@ -2583,10 +2612,18 @@ static void flow_mark_flush(struct dp_netdev_pmd_thread *pmd) { struct dp_netdev_flow *flow; + unsigned int tid; - CMAP_FOR_EACH (flow, mark_node, &dp_offload_thread.mark_to_flow) { - if (flow->pmd_id == pmd->core_id) { - queue_netdev_flow_del(pmd, flow); + if (dp_offload_threads == NULL) { + return; + } + + for (tid = 0; tid < netdev_offload_thread_nb(); tid++) { + CMAP_FOR_EACH (flow, mark_node, + &dp_offload_threads[tid].mark_to_flow) { + if (flow->pmd_id == pmd->core_id) { + queue_netdev_flow_del(pmd, flow); + } } } } @@ -2596,12 +2633,21 @@ mark_to_flow_find(const struct dp_netdev_pmd_thread *pmd, const uint32_t mark) { struct dp_netdev_flow *flow; + unsigned int tid; + size_t hash; - CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash_int(mark, 0), - &dp_offload_thread.mark_to_flow) { - if (flow->mark == mark && flow->pmd_id == pmd->core_id && - flow->dead == false) { - return flow; + if (dp_offload_threads == NULL) { + return NULL; + } + + hash = hash_int(mark, 0); + for (tid = 0; tid < netdev_offload_thread_nb(); tid++) { + CMAP_FOR_EACH_WITH_HASH (flow, mark_node, hash, + &dp_offload_threads[tid].mark_to_flow) { + if (flow->mark == mark && flow->pmd_id == pmd->core_id && + flow->dead == false) { + return flow; + } } } @@ -2644,8 +2690,13 @@ dp_netdev_free_flow_offload(struct dp_offload_thread_item *offload) static void dp_netdev_append_flow_offload(struct dp_offload_thread_item *offload) { - mpsc_queue_insert(&dp_offload_thread.queue, &offload->node); - atomic_count_inc64(&dp_offload_thread.enqueued_item); + unsigned int i; + + dp_netdev_offload_init(); + + i = netdev_offload_ufid_to_thread_id(offload->flow->mega_ufid); + mpsc_queue_insert(&dp_offload_threads[i].queue, &offload->node); + atomic_count_inc64(&dp_offload_threads[i].enqueued_item); } static int @@ -2748,8 +2799,9 @@ err_free: #define DP_NETDEV_OFFLOAD_QUIESCE_INTERVAL_US (10 * 1000) /* 10 ms */ static void * -dp_netdev_flow_offload_main(void *data OVS_UNUSED) +dp_netdev_flow_offload_main(void *arg) { + struct dp_offload_thread *ofl_thread = arg; struct dp_offload_thread_item *offload; enum mpsc_queue_poll_result poll_result; struct mpsc_queue_node *node; @@ -2761,7 +2813,7 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) const char *op; int ret; - queue = &dp_offload_thread.queue; + queue = &ofl_thread->queue; if (!mpsc_queue_acquire(queue)) { VLOG_ERR("failed to register as consumer of the offload queue"); return NULL; @@ -2784,7 +2836,7 @@ sleep_until_next: } offload = CONTAINER_OF(node, struct dp_offload_thread_item, node); - atomic_count_dec64(&dp_offload_thread.enqueued_item); + atomic_count_dec64(&ofl_thread->enqueued_item); switch (offload->op) { case DP_NETDEV_FLOW_OFFLOAD_OP_ADD: @@ -2806,8 +2858,8 @@ sleep_until_next: now = time_usec(); latency_us = now - offload->timestamp; - mov_avg_cma_update(&dp_offload_thread.cma, latency_us); - mov_avg_ema_update(&dp_offload_thread.ema, latency_us); + mov_avg_cma_update(&ofl_thread->cma, latency_us); + mov_avg_ema_update(&ofl_thread->ema, latency_us); VLOG_DBG("%s to %s netdev flow "UUID_FMT, ret == 0 ? "succeed" : "failed", op, @@ -2835,13 +2887,6 @@ queue_netdev_flow_del(struct dp_netdev_pmd_thread *pmd, { struct dp_offload_thread_item *offload; - if (ovsthread_once_start(&offload_thread_once)) { - mpsc_queue_init(&dp_offload_thread.queue); - ovs_thread_create("dp_netdev_flow_offload", - dp_netdev_flow_offload_main, NULL); - ovsthread_once_done(&offload_thread_once); - } - offload = dp_netdev_alloc_flow_offload(pmd, flow, DP_NETDEV_FLOW_OFFLOAD_OP_DEL); offload->timestamp = pmd->ctx.now; @@ -2860,13 +2905,6 @@ queue_netdev_flow_put(struct dp_netdev_pmd_thread *pmd, return; } - if (ovsthread_once_start(&offload_thread_once)) { - mpsc_queue_init(&dp_offload_thread.queue); - ovs_thread_create("dp_netdev_flow_offload", - dp_netdev_flow_offload_main, NULL); - ovsthread_once_done(&offload_thread_once); - } - if (flow->mark != INVALID_FLOW_MARK) { op = DP_NETDEV_FLOW_OFFLOAD_OP_MOD; } else { @@ -4267,60 +4305,99 @@ dpif_netdev_offload_stats_get(struct dpif *dpif, DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN, DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV, }; - const char *names[] = { + struct { + const char *name; + uint64_t total; + } hwol_stats[] = { [DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED] = - " Enqueued offloads", + { " Enqueued offloads", 0 }, [DP_NETDEV_HW_OFFLOADS_STATS_INSERTED] = - " Inserted offloads", + { " Inserted offloads", 0 }, [DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN] = - " Cumulative Average latency (us)", + { " Cumulative Average latency (us)", 0 }, [DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV] = - " Cumulative Latency stddev (us)", + { " Cumulative Latency stddev (us)", 0 }, [DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN] = - " Exponential Average latency (us)", + { " Exponential Average latency (us)", 0 }, [DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV] = - " Exponential Latency stddev (us)", + { " Exponential Latency stddev (us)", 0 }, }; struct dp_netdev *dp = get_dp_netdev(dpif); struct dp_netdev_port *port; - uint64_t nb_offloads; + unsigned int nb_thread; + uint64_t *port_nb_offloads; + uint64_t *nb_offloads; + unsigned int tid; size_t i; if (!netdev_is_flow_api_enabled()) { return EINVAL; } - stats->size = ARRAY_SIZE(names); + nb_thread = netdev_offload_thread_nb(); + /* nb_thread counters for the overall total as well. */ + stats->size = ARRAY_SIZE(hwol_stats) * (nb_thread + 1); stats->counters = xcalloc(stats->size, sizeof *stats->counters); - nb_offloads = 0; + nb_offloads = xcalloc(nb_thread, sizeof *nb_offloads); + port_nb_offloads = xcalloc(nb_thread, sizeof *port_nb_offloads); ovs_rwlock_rdlock(&dp->port_rwlock); HMAP_FOR_EACH (port, node, &dp->ports) { - uint64_t port_nb_offloads = 0; - + memset(port_nb_offloads, 0, nb_thread * sizeof *port_nb_offloads); /* Do not abort on read error from a port, just report 0. */ - if (!netdev_flow_get_n_flows(port->netdev, &port_nb_offloads)) { - nb_offloads += port_nb_offloads; + if (!netdev_flow_get_n_flows(port->netdev, port_nb_offloads)) { + for (i = 0; i < nb_thread; i++) { + nb_offloads[i] += port_nb_offloads[i]; + } } } ovs_rwlock_unlock(&dp->port_rwlock); - atomic_read_relaxed(&dp_offload_thread.enqueued_item, - &stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED].value); - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_INSERTED].value = nb_offloads; - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN].value = - mov_avg_cma(&dp_offload_thread.cma); - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV].value = - mov_avg_cma_std_dev(&dp_offload_thread.cma); - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN].value = - mov_avg_ema(&dp_offload_thread.ema); - stats->counters[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV].value = - mov_avg_ema_std_dev(&dp_offload_thread.ema); - - for (i = 0; i < ARRAY_SIZE(names); i++) { + free(port_nb_offloads); + + for (tid = 0; tid < nb_thread; tid++) { + uint64_t counts[ARRAY_SIZE(hwol_stats)]; + size_t idx = ((tid + 1) * ARRAY_SIZE(hwol_stats)); + + memset(counts, 0, sizeof counts); + counts[DP_NETDEV_HW_OFFLOADS_STATS_INSERTED] = nb_offloads[tid]; + if (dp_offload_threads != NULL) { + atomic_read_relaxed(&dp_offload_threads[tid].enqueued_item, + &counts[DP_NETDEV_HW_OFFLOADS_STATS_ENQUEUED]); + + counts[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN] = + mov_avg_cma(&dp_offload_threads[tid].cma); + counts[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV] = + mov_avg_cma_std_dev(&dp_offload_threads[tid].cma); + + counts[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN] = + mov_avg_ema(&dp_offload_threads[tid].ema); + counts[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV] = + mov_avg_ema_std_dev(&dp_offload_threads[tid].ema); + } + + for (i = 0; i < ARRAY_SIZE(hwol_stats); i++) { + snprintf(stats->counters[idx + i].name, + sizeof(stats->counters[idx + i].name), + " [%3u] %s", tid, hwol_stats[i].name); + stats->counters[idx + i].value = counts[i]; + hwol_stats[i].total += counts[i]; + } + } + + free(nb_offloads); + + /* Do an average of the average for the aggregate. */ + hwol_stats[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_MEAN].total /= nb_thread; + hwol_stats[DP_NETDEV_HW_OFFLOADS_STATS_LAT_CMA_STDDEV].total /= nb_thread; + hwol_stats[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_MEAN].total /= nb_thread; + hwol_stats[DP_NETDEV_HW_OFFLOADS_STATS_LAT_EMA_STDDEV].total /= nb_thread; + + for (i = 0; i < ARRAY_SIZE(hwol_stats); i++) { snprintf(stats->counters[i].name, sizeof(stats->counters[i].name), - "%s", names[i]); + " Total %s", hwol_stats[i].name); + stats->counters[i].value = hwol_stats[i].total; } return 0; From patchwork Wed Feb 10 15:34:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gaetan Rivet X-Patchwork-Id: 1439079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=u256.net header.i=@u256.net header.a=rsa-sha256 header.s=fm1 header.b=lX2LR2Cf; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm2 header.b=AetysXB1; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4DbP3b3Nrpz9rx8 for ; Thu, 11 Feb 2021 02:35:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id F3FA586ED2; Wed, 10 Feb 2021 15:35:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id etm6UDASctnJ; Wed, 10 Feb 2021 15:35:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 75A0486EDF; Wed, 10 Feb 2021 15:34:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4FC9AC013A; Wed, 10 Feb 2021 15:34:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1B4D4C1E71 for ; Wed, 10 Feb 2021 15:34:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id DE57E86C2B for ; Wed, 10 Feb 2021 15:34:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8i23nrTMYbRA for ; Wed, 10 Feb 2021 15:34:24 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from wnew4-smtp.messagingengine.com (wnew4-smtp.messagingengine.com [64.147.123.18]) by fraxinus.osuosl.org (Postfix) with ESMTPS id C1F1B86B7B for ; Wed, 10 Feb 2021 15:34:22 +0000 (UTC) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.west.internal (Postfix) with ESMTP id 4F4E3E18; Wed, 10 Feb 2021 10:34:22 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Wed, 10 Feb 2021 10:34:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=u256.net; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=fm1; bh=lw1LlZfDYk1c7 YNj0iFjAedK8DxRon4LeHwj1we7bYY=; b=lX2LR2Cf448JWAkqVWF8swfXpNwrb kivTlXPOBfqCtr5PVjTDngFuHofbSJJDL2C+clnKqkXVtNvGA1h+e12Zrv1wLKYC duh1UHvrhLEXpiUAM3M6yaqQG8UzBI7rpIqLmxagQUsJj/khL4Cxw+YBrYl4S4Ja hBX082FXo6jmBru92eE+PZ+NSk/6ng3xq+u1kZT5OwA08CICGk++LVQFy0Vp0kjH xlHesfvvWtEWLrpCI+pBc6hqkBdPdHyaTAXrsjuwWevaL8LomxLL8jdwEazDCB1p SHavhiwdGHJuZxbVE7zGIBNlDrFT85swH84ukT4EohYuGmXfmRBItgvUw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:subject:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=lw1LlZfDYk1c7YNj0iFjAedK8DxRon4LeHwj1we7bYY=; b=AetysXB1 kTzt3DkwGF2FByr7xwRFjQUddqhm6SOcybeQUM5IqwADzRlr/MiCRkmgpYF2PJIA JGZyVULnMW/po0Aq5JdyLDa1o7o/C17rcl2j1mACYPFep2t12DIGelMDBf6BEmeV Y7cXSn8/rtsnTBO3+7C1z6l8/Qv5mgRgBu7zuUbJfu4jCXZ8TOSv2Bn3Ujeas08h ZOWxgW3Xu7djyGMqJi4GtE7q54//fVXiJ1TbXuedHEk7QN/3dP0Vd1aed4oPWEza OdQ1JNxZd+jNd0oQxOWzuYGSOVWbFmHQjQ9qsM7YlMEeKQLL7zakE6iQNns8f7zn W065Cueo3bItpw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrheejgdejkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkofgjfhgggfestdekredtredttdenucfhrhhomhepifgrvghtrghn ucftihhvvghtuceoghhrihhvvgesuhdvheeirdhnvghtqeenucggtffrrghtthgvrhhnpe ehgfevffekteehteefieefvdehleefjeefheevudetjefhkeeutdekieeuvdetheenucfk phepkeeirddvheegrdduieegrddujeegnecuvehluhhsthgvrhfuihiivgepfeenucfrrg hrrghmpehmrghilhhfrhhomhepghhrihhvvgesuhdvheeirdhnvght X-ME-Proxy: Received: from inocybe.home (lfbn-poi-1-842-174.w86-254.abo.wanadoo.fr [86.254.164.174]) by mail.messagingengine.com (Postfix) with ESMTPA id 596FD240066; Wed, 10 Feb 2021 10:34:21 -0500 (EST) From: Gaetan Rivet To: dev@openvswitch.org Date: Wed, 10 Feb 2021 16:34:09 +0100 Message-Id: X-Mailer: git-send-email 2.30.0 In-Reply-To: References: MIME-Version: 1.0 Cc: Eli Britstein Subject: [ovs-dev] [PATCH v1 23/23] netdev-dpdk: Remove rte-flow API access locks X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The rte_flow DPDK API was made thread-safe [1] in release 20.11. Now that the DPDK offload provider in OVS is thread safe, remove the locks. [1]: 80d1a9aff7f6 ("ethdev: make flow API thread safe") Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein --- lib/netdev-dpdk.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 9d8096668..c7ebeb4d5 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -5239,9 +5239,7 @@ netdev_dpdk_rte_flow_destroy(struct netdev *netdev, struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); int ret; - ovs_mutex_lock(&dev->mutex); ret = rte_flow_destroy(dev->port_id, rte_flow, error); - ovs_mutex_unlock(&dev->mutex); return ret; } @@ -5255,9 +5253,7 @@ netdev_dpdk_rte_flow_create(struct netdev *netdev, struct rte_flow *flow; struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); - ovs_mutex_lock(&dev->mutex); flow = rte_flow_create(dev->port_id, attr, items, actions, error); - ovs_mutex_unlock(&dev->mutex); return flow; } @@ -5285,9 +5281,7 @@ netdev_dpdk_rte_flow_query_count(struct netdev *netdev, } dev = netdev_dpdk_cast(netdev); - ovs_mutex_lock(&dev->mutex); ret = rte_flow_query(dev->port_id, rte_flow, actions, query, error); - ovs_mutex_unlock(&dev->mutex); return ret; }