From patchwork Tue May 10 14:21:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629184 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=KklUNXGT; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyD2c27z9sCq for ; Wed, 11 May 2022 00:23:00 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2171240AF2; Tue, 10 May 2022 14:22:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id v6rLfv8bIg36; Tue, 10 May 2022 14:22:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2270F40AE9; Tue, 10 May 2022 14:22:56 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EAC29C0032; Tue, 10 May 2022 14:22:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 118E6C002D for ; Tue, 10 May 2022 14:22:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0068740872 for ; Tue, 10 May 2022 14:22:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fTsOlx2l6GQt for ; Tue, 10 May 2022 14:22:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by smtp4.osuosl.org (Postfix) with ESMTPS id E4CB64059D for ; Tue, 10 May 2022 14:22:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192573; x=1683728573; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=N2FBrF6sRe/ec+wgFisyU7DBQDSiBozDJS1WywLhWCQ=; b=KklUNXGTcQZeKXHpB3UX9nkUJ1Qx9cxrHNemmJIZGt6eB+vbwZxI18h0 PSgBXDocLDtc0UqttO2mfY61upVwY9kW8KvyEV0VhwYreUL0j7WQuRG5k ihjGX+RNCpX5LGoBaoVsHYGG2seeVS2HqE4oxqYhBiUjO0AuqhTy/cFT4 9SYuMMgM9MtksRpT661y8PmRXLkcvEG+TgBgVQt23Eoo/xdpuDOcOwUmp /Dioxg4MLXHfYOBISqeWK+zlPKMc8niphJwFQdzfI4jbNBoMO7ydy93t7 AqRkbd/HyhM+EGZIHxGNqKK9VD6Z5aEa5Pwcv5MjZAPW0Bs9sEIcb9JPl w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="249916944" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="249916944" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:22:53 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796811" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:22:51 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:52 +0000 Message-Id: <20220510142202.1087967-2-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 01/11] ofproto-dpif: Fix incorrect checksums in input packets X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The IP checksum field was invalid in the input packets for some unit tests. The unit tests will still pass without a valid checksum, however we should still fix these. Signed-off-by: Emma Finn Acked-by: Eelco Chaudron --- This was found using the autovalidator introduced later in this series. --- --- tests/ofproto-dpif.at | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at index dbb3b6dda..935ae80e0 100644 --- a/tests/ofproto-dpif.at +++ b/tests/ofproto-dpif.at @@ -2009,7 +2009,7 @@ dnl Checksum UDP. AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 ; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 1C 00 00 00 00 00 11 00 00 C0 A8 00 01 C0 A8 00 02 00 08 00 0B 00 00 12 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 1C 00 00 00 00 00 11 39 7E C0 A8 00 01 C0 A8 00 02 00 08 00 0B 00 00 12 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 18]) OVS_APP_EXIT_AND_WAIT([ovs-ofctl]) @@ -2079,7 +2079,7 @@ dnl Checksum SCTP. AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 ; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 24 00 00 00 00 00 84 00 00 C0 A8 00 01 C0 A8 00 02 04 58 08 af 00 00 00 00 d9 d7 91 57 01 00 00 34 cf 28 ec 4e 00 01 40 00 00 0a ff ff b7 53 24 19 00 05 00 08 7f 00 00 01 00 05 00 08 c0 a8 02 07 00 0c 00 06 00 05 00 00 80 00 00 04 c0 00 00 04' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 24 00 00 00 00 00 84 39 03 C0 A8 00 01 C0 A8 00 02 04 58 08 af 00 00 00 00 d9 d7 91 57 01 00 00 34 cf 28 ec 4e 00 01 40 00 00 0a ff ff b7 53 24 19 00 05 00 08 7f 00 00 01 00 05 00 08 c0 a8 02 07 00 0c 00 06 00 05 00 00 80 00 00 04 c0 00 00 04' done AT_CHECK([ovs-appctl time/warp 1000], [0], [ignore]) @@ -2951,7 +2951,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x7744 (correct), AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 01 88 48 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3b 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 01 88 48 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) @@ -2979,7 +2979,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x7744 (correct), AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 10 88 47 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3b 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 10 88 47 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) OVS_APP_EXIT_AND_WAIT(ovs-ofctl) @@ -3356,7 +3356,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x77ec (correct) AT_CHECK([ovs-ofctl -O OpenFlow12 monitor br0 65534 -P standard --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 01 60 66 66 66 00 08 88 48 00 01 41 20 45 20 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 01 60 66 66 66 00 08 88 48 00 01 41 20 45 20 00 2c 00 00 00 00 ff 06 3a 58 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) OVS_APP_EXIT_AND_WAIT(ovs-ofctl) From patchwork Tue May 10 14:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629186 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=EzeLa89B; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyS56PHz9sCq for ; Wed, 11 May 2022 00:23:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4243760FE5; Tue, 10 May 2022 14:23:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mu0FqAMvL6Go; Tue, 10 May 2022 14:23:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 02C8861001; Tue, 10 May 2022 14:23:07 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CC8E8C0032; Tue, 10 May 2022 14:23:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03B62C002D for ; Tue, 10 May 2022 14:23:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6803482A0E for ; Tue, 10 May 2022 14:23:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a5RlBf80c_gs for ; Tue, 10 May 2022 14:23:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id C8F6882983 for ; Tue, 10 May 2022 14:23:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192580; x=1683728580; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zpNgUvUVrfdX1L3EU+gk/uRfBnIcoQk3WUWp0PyE/gs=; b=EzeLa89BMQ4ViHrD29rcp93E5KY7aK6NEXuyZIeef+JvO0YA4AtDcg1G uMqf0GV6Egme7Ir8xA9j/0b7R5lwmEz8r+XF9suW3tpkEjU99ay2jUC5B EAxV6W/rUQagv1IVi50mnHl8GzxHIaTrjsJ1X1gzAIt3qO32cOJe2v1fa dmET+xnwFTm/R5EWDTQXQE3jqwzZG8iu5b3wN0oQB55P6OlG5IHJdPHlx s0Uy2zznpuXBXwhHFJBbL14l3LQ9i/CtH/JVeoOnzxe7PVbxwat+YwE38 nwx2i5oF81982aE9ZRgI7O6iJ7M7QbruB5NGyN/hBRynedIuWtWOJK4XB g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975832" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975832" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:00 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796850" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:22:57 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:53 +0000 Message-Id: <20220510142202.1087967-3-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 02/11] odp-execute: Add function pointers to odp-execute for different action implementations. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduces the initial infrastructure required to allow different implementations for OvS actions. The patch introduces action function pointers which allows user to switch between different action implementations available. This will allow for more performance and flexibility so the user can choose the action implementation to best suite their use case. Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- lib/automake.mk | 2 + lib/dpif-netdev.c | 4 ++ lib/odp-execute-private.c | 94 ++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 96 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute.c | 39 ++++++++++++++-- lib/odp-execute.h | 4 ++ 6 files changed, 234 insertions(+), 5 deletions(-) create mode 100644 lib/odp-execute-private.c create mode 100644 lib/odp-execute-private.h diff --git a/lib/automake.mk b/lib/automake.mk index a23cdc4ad..625c0d9c9 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -210,6 +210,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/object-collection.h \ lib/odp-execute.c \ lib/odp-execute.h \ + lib/odp-execute-private.c \ + lib/odp-execute-private.h \ lib/odp-util.c \ lib/odp-util.h \ lib/ofp-actions.c \ diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 61929049c..f74f8b864 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1686,6 +1686,10 @@ create_dpif_netdev(struct dp_netdev *dp) dpif->dp = dp; dpif->last_port_seq = seq_read(dp->port_seq); + /* Called once at initialization time. This handles setting up the state + * of the actions functions at init time. */ + odp_execute_init(); + return &dpif->dpif; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c new file mode 100644 index 000000000..b3a02745c --- /dev/null +++ b/lib/odp-execute-private.c @@ -0,0 +1,94 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include + +#include "dpdk.h" +#include "dp-packet.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "odp-util.h" +#include "openvswitch/vlog.h" + + +int32_t action_autoval_init(struct odp_execute_action_impl *self); +VLOG_DEFINE_THIS_MODULE(odp_execute_private); + +static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_SCALAR] = { + .available = 1, + .name = "scalar", + .probe = NULL, + .init_func = NULL, + }, +}; + +static void +action_impl_init_funcs(struct odp_execute_action_impl *to) +{ + for (uint32_t i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + atomic_init(&to->funcs[i], NULL); + } +} + +static void +action_impl_copy_funcs(struct odp_execute_action_impl *to, + const struct odp_execute_action_impl *from) +{ + for (uint32_t i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + atomic_store_relaxed(&to->funcs[i], from->funcs[i]); + } +} + +void +odp_execute_action_init(void) +{ + /* Call probe on each impl, and cache the result. */ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + bool avail = true; + if (action_impls[i].probe) { + /* Return zero is success, non-zero means error. */ + avail = (action_impls[i].probe() == 0); + } + VLOG_INFO("Action implementation %s (available: %s)\n", + action_impls[i].name, avail ? "available" : "not available"); + action_impls[i].available = avail; + } + + uint32_t i; + for (i = 0; i < ACTION_IMPL_MAX; i++) { + /* Initialize Actions function pointers. */ + action_impl_init_funcs(&action_impls[i]); + + /* Each impl's function array is initialized to reflect the scalar + * implementation. This simplifies adding optimized implementations, + * as the autovalidator can always compare all actions. + * + * Below copies the scalar functions to all other implementations. + */ + if (i != ACTION_IMPL_SCALAR) { + action_impl_copy_funcs(&action_impls[i], + &action_impls[ACTION_IMPL_SCALAR]); + } + + if (action_impls[i].init_func) { + action_impls[i].init_func(&action_impls[i]); + } + } +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h new file mode 100644 index 000000000..869478ce9 --- /dev/null +++ b/lib/odp-execute-private.h @@ -0,0 +1,96 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef ODP_EXTRACT_PRIVATE +#define ODP_EXTRACT_PRIVATE 1 + +#include "dp-packet.h" +#include "odp-execute.h" +#include "odp-netlink.h" +#include "ovs-atomic.h" + +/* Forward declaration for typedef. */ +struct odp_execute_action_impl; + +/* Typedef for an initialization function that can initialize each + * implementation, checking requirements such as CPU ISA. + */ +typedef int32_t (*odp_execute_action_init_func) + (struct odp_execute_action_impl *self); + +/* Probe function is used to detect if this CPU has the ISA required + * to run the optimized action implementation. + * Returns zero on successful probe and available will be true. + * Returns negative errno on failure and available will be false. + */ +typedef int (*odp_execute_action_probe)(void); + +/* Structure represents an implementation of the odp actions. */ +struct odp_execute_action_impl { + /* When set, the CPU ISA required for this implementation is available + * and the implementation can be used. + */ + bool available; + + /* Name of the implementation. */ + const char *name; + + /* Probe function is used to detect if this CPU has the ISA required + * to run the optimized miniflow implementation. It is optional and + * if it is not used, then it must be null. + */ + odp_execute_action_probe probe; + + /* Called to check requirements and if usable, initializes the + * implementation for use. + */ + odp_execute_action_init_func init_func; + + /* An array of callback functions, one for each action. */ + ATOMIC(odp_execute_cb) funcs[__OVS_KEY_ATTR_MAX]; +}; + +/* Order of Actions implementations. */ +enum odp_execute_action_impl_idx { + ACTION_IMPL_SCALAR, + /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. + * Do not change the autovalidator position in this list without updating + * the define below. + */ + + ACTION_IMPL_MAX, +}; + +/* Index to start verifying implementations from. */ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); + +/* Odp execute init handles setting up the state of the actions functions at + * initialization time. It cannot return errors, as it must always succeed in + * initializing the scalar/generic codepath. + */ +void odp_execute_action_init(void); + +/* Update the current active functions to those requested in name. */ +void odp_execute_action_get(struct ds *name); +int32_t odp_execute_action_set(const char *name, + struct odp_execute_action_impl *active); + +/* Init function for the scalar implementation. Calls into the odp-execute.c + * file, and initializes the function pointers for optimized action types. + */ +int32_t odp_action_scalar_init(struct odp_execute_action_impl *self); + +#endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 7da56793d..165386e66 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -17,6 +17,7 @@ #include #include "odp-execute.h" +#include "odp-execute-private.h" #include #include #include @@ -833,6 +834,23 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. + */ +static struct odp_execute_action_impl actions_active_impl; + +void +odp_execute_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + if (ovsthread_once_start(&once)) { + odp_execute_action_init(); + ovsthread_once_done(&once); + } +} + + /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and * definitely free the packets in 'batch', otherwise leaves 'batch' unchanged. @@ -858,13 +876,12 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, NL_ATTR_FOR_EACH_UNSAFE (a, left, actions, actions_len) { int type = nl_attr_type(a); bool last_action = (left <= NLA_ALIGN(a->nla_len)); + /* Allow 'dp_execute_action' to steal the packet data if we do + * not need it any more. */ + bool should_steal = steal && last_action; if (requires_datapath_assistance(a)) { if (dp_execute_action) { - /* Allow 'dp_execute_action' to steal the packet data if we do - * not need it any more. */ - bool should_steal = steal && last_action; - dp_execute_action(dp, batch, a, should_steal); if (last_action || dp_packet_batch_is_empty(batch)) { @@ -879,8 +896,20 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, continue; } - switch ((enum ovs_action_attr) type) { + /* If type is set in the active actions implementation, call the + * function-pointer and continue to the next action. + */ + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; + if (actions_active_impl.funcs[attr_type]) { + actions_active_impl.funcs[attr_type](NULL, batch, a, should_steal); + continue; + } + + /* If the action was not handled by the active function pointers above, + * process them by switching on the type below. + */ + switch (attr_type) { case OVS_ACTION_ATTR_HASH: { const struct ovs_action_hash *hash_act = nl_attr_get(a); diff --git a/lib/odp-execute.h b/lib/odp-execute.h index a3578a575..c4f5303e7 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -28,6 +28,10 @@ struct dp_packet; struct pkt_metadata; struct dp_packet_batch; + +/* Called once at initialization time. */ +void odp_execute_init(void); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); From patchwork Tue May 10 14:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629187 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Nup1kCtm; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyV2msjz9sCq for ; Wed, 11 May 2022 00:23:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4D616827AF; Tue, 10 May 2022 14:23:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K5da6blqUxLl; Tue, 10 May 2022 14:23:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4FAD482A0E; Tue, 10 May 2022 14:23:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C59ECC0081; Tue, 10 May 2022 14:23:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1C628C0084 for ; Tue, 10 May 2022 14:23:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C0ACC82A4F for ; Tue, 10 May 2022 14:23:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mIEuyVSWXUxa for ; Tue, 10 May 2022 14:23:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1315982A2E for ; Tue, 10 May 2022 14:23:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192584; x=1683728584; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=t6qCbKBOlsiv8mQuo2FQgrYUYv+nO4hnndNACGOehMM=; b=Nup1kCtmVmzjw332O1sUYUdcMBFJJ9aHeU0CDV1WdmfAb8iZCbLIvHhz Y2tJ81Q0E+FlS2NHyIK3eiWyHunN6qi5SYd+pxAZZfn6mT4kZk1b89Qem JO2AIBEDRqdO/7m5NkB9GwceKLAcBQX791TJ/in2IwPYOKhRr3kDAIGOz 0ml4JlLxUjS4CT+Dk8LrFxFgADyouVpH/6K25MoUsL+PiSKCH483CXbGo RIXEBCP0kE8DiqtOQc7oE2KKwFdMMYjmknNQAolbrJiLGPo3mkuSP5x7P +C0GBvq3Rx/88BGKa/bQfrx5V5OcbFSNSXTLobRnZMBnIovX61iU6hW0w g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975847" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975847" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796865" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:01 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:54 +0000 Message-Id: <20220510142202.1087967-4-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 03/11] odp-execute: Add function pointer for pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit removes the pop_vlan action from the large switch and creates a separate function for batched processing. A function pointer is also added to call the new batched function for the pop_vlan action. Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- lib/odp-execute-private.c | 19 +++++++++++++++++- lib/odp-execute.c | 41 +++++++++++++++++++++++++++++++++------ lib/odp-execute.h | 2 ++ 3 files changed, 55 insertions(+), 7 deletions(-) diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index b3a02745c..996de0bf6 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -29,13 +29,14 @@ int32_t action_autoval_init(struct odp_execute_action_impl *self); VLOG_DEFINE_THIS_MODULE(odp_execute_private); +static uint32_t active_action_impl_index; static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_SCALAR] = { .available = 1, .name = "scalar", .probe = NULL, - .init_func = NULL, + .init_func = odp_action_scalar_init, }, }; @@ -56,6 +57,22 @@ action_impl_copy_funcs(struct odp_execute_action_impl *to, } } +int32_t +odp_execute_action_set(const char *name, + struct odp_execute_action_impl *active) +{ + uint32_t i; + for (i = 0; i < ACTION_IMPL_MAX; i++) { + /* String compare, and set ptrs atomically. */ + if (strcmp(action_impls[i].name, name) == 0) { + action_impl_copy_funcs(active, &action_impls[i]); + active_action_impl_index = i; + return 0; + } + } + return -1; +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 165386e66..c2be74454 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -834,6 +834,28 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +static void +action_pop_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + struct dp_packet *packet; + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_pop_vlan(packet); + } +} + +/* Implementation of the scalar actions impl init function. Build up the + * array of func ptrs here. + */ +int32_t +odp_action_scalar_init(struct odp_execute_action_impl *self) +{ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + + return 0; +} + /* The active function pointers on the datapath. ISA optimized implementations * are enabled by plugging them into this static arary, which is consulted when * applying actions on the datapath. @@ -846,10 +868,22 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); + odp_actions_impl_set("scalar"); ovsthread_once_done(&once); } } +int32_t +odp_actions_impl_set(const char *name) +{ + + int err = odp_execute_action_set(name, &actions_active_impl); + if (err) { + VLOG_ERR("error %d from action set to %s\n", err, name); + return -1; + } + return 0; +} /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and @@ -965,12 +999,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_POP_VLAN: - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_pop_vlan(packet); - } - break; - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1114,6 +1142,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, } case OVS_ACTION_ATTR_OUTPUT: case OVS_ACTION_ATTR_LB_OUTPUT: + case OVS_ACTION_ATTR_POP_VLAN: case OVS_ACTION_ATTR_TUNNEL_PUSH: case OVS_ACTION_ATTR_TUNNEL_POP: case OVS_ACTION_ATTR_USERSPACE: diff --git a/lib/odp-execute.h b/lib/odp-execute.h index c4f5303e7..6441392b9 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -32,6 +32,8 @@ struct dp_packet_batch; /* Called once at initialization time. */ void odp_execute_init(void); +int32_t odp_actions_impl_set(const char *name); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); From patchwork Tue May 10 14:21:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629189 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=LHyOweug; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyg6497z9sCq for ; Wed, 11 May 2022 00:23:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D646682BD6; Tue, 10 May 2022 14:23:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vzm_wLugqhrO; Tue, 10 May 2022 14:23:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id DB39A82BA1; Tue, 10 May 2022 14:23:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B0AF9C0032; Tue, 10 May 2022 14:23:19 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 00D55C002D for ; Tue, 10 May 2022 14:23:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D85B1829EE for ; Tue, 10 May 2022 14:23:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id n-x1bhgfuS1z for ; Tue, 10 May 2022 14:23:08 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5D53182AA9 for ; Tue, 10 May 2022 14:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192588; x=1683728588; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A7BWjqagNTxOykU8KE05y0ZLgSA8b27COAzDWM72C5M=; b=LHyOweugaznWidk3bhzWlh4NeKoueVVWgdL0+S6F5mW7Ugax2LjRS5UR XH2AVrxQgpWw7jp6YJ+I1+FvvYyCsqxndhg9uOoOvIN/kahiayXmFTV6j XKysF9MM6xkuxL2pxt/Yym/qVVTtpA1xaPFVEpUzaHFSBmADjIV6JCkMM ucuWlXduGoliMfPWrmTdECTzgUE7Id90NUAxBEkDoz7jeAIzP+qSehLpy FCHGogl0JcXno0bPXtgmE9soS3ChCZFWyQHHhGsLwEbkzDsrggQR/tw8/ YiSMweBzWK9RUwT+8kaf2papDILGO/QwOoYdQhmN+KHY39Rm5bL32KCC1 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975858" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975858" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796881" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:06 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:55 +0000 Message-Id: <20220510142202.1087967-5-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 04/11] odp-execute: Add auto validation function for actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduced the auto-validation function which allows users to compare the batch of packets obtained from different action implementations against the linear action implementation. The autovalidator function can be triggered at runtime using the following command: $ ovs-appctl dpif-netdev/action-impl-set autovalidator Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- NEWS | 2 + lib/dp-packet.c | 23 +++++++++ lib/dp-packet.h | 4 ++ lib/odp-execute-private.c | 99 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 3 ++ 5 files changed, 131 insertions(+) diff --git a/NEWS b/NEWS index eece0d0b2..8539a03b6 100644 --- a/NEWS +++ b/NEWS @@ -58,6 +58,8 @@ v2.17.0 - 17 Feb 2022 * Add support for DPDK 21.11. * Forbid use of DPDK multiprocess feature. * Add support for running threads on cores >= RTE_MAX_LCORE. + * Add actions auto-validator function to compare different actions + implementations against default implementation. - Python: * For SSL support, the use of the pyOpenSSL library has been replaced with the native 'ssl' module. diff --git a/lib/dp-packet.c b/lib/dp-packet.c index 35c72542a..b71c68ed0 100644 --- a/lib/dp-packet.c +++ b/lib/dp-packet.c @@ -506,3 +506,26 @@ dp_packet_resize_l2(struct dp_packet *b, int increment) dp_packet_adjust_layer_offset(&b->l2_5_ofs, increment); return dp_packet_data(b); } + +bool +dp_packet_compare_and_log(struct dp_packet *good, struct dp_packet *test, + struct ds *err_str) +{ + if ((good->l2_pad_size != test->l2_pad_size) || + (good->l2_5_ofs != test->l2_5_ofs) || + (good->l3_ofs != test->l3_ofs) || + (good->l4_ofs != test->l4_ofs)) { + ds_put_format(err_str, "Autovalidation packet offsets failed" + "\n"); + ds_put_format(err_str, "Good offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + good->l2_pad_size, good->l2_5_ofs, + good->l3_ofs, good->l4_ofs); + ds_put_format(err_str, "Test offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + test->l2_pad_size, test->l2_5_ofs, + test->l3_ofs, test->l4_ofs); + return false; + } + return true; +} diff --git a/lib/dp-packet.h b/lib/dp-packet.h index bddaa2b5d..bf7ee61a5 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -236,6 +236,10 @@ void *dp_packet_steal_data(struct dp_packet *); static inline bool dp_packet_equal(const struct dp_packet *, const struct dp_packet *); +bool dp_packet_compare_and_log(struct dp_packet *good, + struct dp_packet *test, + struct ds *err_str); + /* Frees memory that 'b' points to, as well as 'b' itself. */ static inline void diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 996de0bf6..e85aed636 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -30,8 +30,16 @@ int32_t action_autoval_init(struct odp_execute_action_impl *self); VLOG_DEFINE_THIS_MODULE(odp_execute_private); static uint32_t active_action_impl_index; +static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_AUTOVALIDATOR] = { + .available = 1, + .name = "autovalidator", + .probe = NULL, + .init_func = action_autoval_init, + }, + [ACTION_IMPL_SCALAR] = { .available = 1, .name = "scalar", @@ -109,3 +117,94 @@ odp_execute_action_init(void) } } } + +/* Init sequence required to be scalar first to pick up the default scalar +* implementations, allowing over-riding of the optimized functions later. +*/ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +/* Loop over packets, and validate each one for the given action. */ +static void +action_autoval_generic(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a, bool should_steal) +{ + uint32_t failed = 0; + + int type = nl_attr_type(a); + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; + + struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; + + struct dp_packet_batch good_batch; + dp_packet_batch_clone(&good_batch, batch); + + scalar->funcs[attr_type](NULL, &good_batch, a, should_steal); + + for (uint32_t impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { + /* Clone original batch and execute implementation under test. */ + struct dp_packet_batch test_batch; + dp_packet_batch_clone(&test_batch, batch); + action_impls[impl].funcs[attr_type](NULL, &test_batch, a, + should_steal); + + /* Loop over implementations, checking each one. */ + for (uint32_t pidx = 0; pidx < batch->count; pidx++) { + struct dp_packet *good_pkt = good_batch.packets[pidx]; + struct dp_packet *test_pkt = test_batch.packets[pidx]; + + struct ds log_msg = DS_EMPTY_INITIALIZER; + + /* Compare packet length and payload contents. */ + bool eq = dp_packet_equal(good_pkt, test_pkt); + + if (!eq) { + ds_put_format(&log_msg, "Packet: %d\nAction : ", pidx); + format_odp_actions(&log_msg, a, a->nla_len, NULL); + ds_put_format(&log_msg, "\nGood hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(good_pkt), + dp_packet_size(good_pkt), 0, false); + ds_put_format(&log_msg, "Test hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(test_pkt), + dp_packet_size(test_pkt), 0, false); + + failed = 1; + } + + /* Compare offsets and RSS */ + if (!dp_packet_compare_and_log(good_pkt, test_pkt, &log_msg)) { + failed = 1; + } + + uint32_t good_hash = dp_packet_get_rss_hash(good_pkt); + uint32_t test_hash = dp_packet_get_rss_hash(test_pkt); + + if (good_hash != test_hash) { + ds_put_format(&log_msg, "Autovalidation rss hash failed" + "\n"); + ds_put_format(&log_msg, "Good RSS hash : %u\n", good_hash); + ds_put_format(&log_msg, "Test RSS hash : %u\n", test_hash); + + failed = 1; + } + + if (failed) { + VLOG_ERR_RL(&rl, "\nAutovalidation failed details:\n%s", + ds_cstr(&log_msg)); + } + } + dp_packet_delete_batch(&test_batch, 1); + } + dp_packet_delete_batch(&good_batch, 1); + + /* Apply the action to the original batch for continued processing. */ + scalar->funcs[attr_type](NULL, batch, a, should_steal); +} + +int32_t +action_autoval_init(struct odp_execute_action_impl *self) +{ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; + + return 0; +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 869478ce9..fed20930d 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -66,6 +66,7 @@ struct odp_execute_action_impl { /* Order of Actions implementations. */ enum odp_execute_action_impl_idx { ACTION_IMPL_SCALAR, + ACTION_IMPL_AUTOVALIDATOR, /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. * Do not change the autovalidator position in this list without updating * the define below. @@ -76,6 +77,8 @@ enum odp_execute_action_impl_idx { /* Index to start verifying implementations from. */ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); +#define ACTION_IMPL_BEGIN (ACTION_IMPL_AUTOVALIDATOR + 1) /* Odp execute init handles setting up the state of the actions functions at * initialization time. It cannot return errors, as it must always succeed in From patchwork Tue May 10 14:21:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629190 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Rza4tFs1; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyv0xL0z9sCq for ; Wed, 11 May 2022 00:23:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6DA1441774; Tue, 10 May 2022 14:23:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vBx8l2lBFXhA; Tue, 10 May 2022 14:23:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id ED84C4176E; Tue, 10 May 2022 14:23:24 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C1598C0032; Tue, 10 May 2022 14:23:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0F14EC002D for ; Tue, 10 May 2022 14:23:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7B5C782AF8 for ; Tue, 10 May 2022 14:23:13 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kfl2ieGupWFq for ; Tue, 10 May 2022 14:23:12 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5820782983 for ; Tue, 10 May 2022 14:23:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192592; x=1683728592; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9KS1UPUxdFkUQ1b3Ke340GNSXrAQU392A/g5Wok8lwY=; b=Rza4tFs1IDeOHDMYbkEYBu3fJgCmfTold0yyTIRxqK9nvmjszQsAD9R+ isHgpVMkLCDb2sDBUblL0dSVr4sB4y5AkKoPnKwW0bmwtP3dFPp4v07Nf 51Sv4r66E3Isd3Suotu3YXvpVRBI0gD/FIzyggM5/lvBjZ8nGGrCBSNPD Ko0OzzksY9seZDXzgRS14jW/oXU6Xu0owxak/7BGaWW/sXWRwyBTyylbP 48iPF0IY3ugh1lqXJvnUPM05In921DKDbjdEsaavFzfO3pgleV00I3xSH cnfV1KZ6Qzgb2z/6UI1r2DvQEkXhM9ciFIYzHRqJgOHz59K70j2SsT2vU g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975868" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975868" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796920" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:10 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:56 +0000 Message-Id: <20220510142202.1087967-6-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 05/11] odp-execute: Add command to switch action implementation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds a new command to allow the user to switch the active action implementation at runtime. A probe function is executed before switching the implementation, to ensure the CPU is capable of running the ISA required. Usage: $ ovs-appctl dpif-netdev/action-impl-set scalar This commit also adds a new command to retrieve the list of available action implementations. This can be used by to check what implementations of actions are available and what implementation is active during runtime. Usage: $ ovs-appctl dpif-netdev/action-impl-get Added separate test-case for ovs-actions get/set commands: 1023: PMD - ovs-actions configuration Signed-off-by: Emma Finn Co-authored-by: Kumar Amber Signed-off-by: Kumar Amber Acked-by: Harry van Haaren --- NEWS | 2 ++ lib/dpif-netdev-unixctl.man | 6 ++++++ lib/dpif-netdev.c | 39 +++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 14 +++++++++++++ lib/odp-execute.h | 3 +++ tests/pmd.at | 21 ++++++++++++++++++++ 6 files changed, 85 insertions(+) diff --git a/NEWS b/NEWS index 8539a03b6..eecdcda5e 100644 --- a/NEWS +++ b/NEWS @@ -60,6 +60,8 @@ v2.17.0 - 17 Feb 2022 * Add support for running threads on cores >= RTE_MAX_LCORE. * Add actions auto-validator function to compare different actions implementations against default implementation. + * Add command line option to switch between different actions + implementations available at run time. - Python: * For SSL support, the use of the pyOpenSSL library has been replaced with the native 'ssl' module. diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man index 8cd847416..500daf4de 100644 --- a/lib/dpif-netdev-unixctl.man +++ b/lib/dpif-netdev-unixctl.man @@ -262,3 +262,9 @@ PMDs in the case where no value is specified. By default "scalar" is used. \fIstudy_cnt\fR defaults to 128 and indicates the number of packets that the "study" miniflow implementation must parse before choosing an optimal implementation. + +.IP "\fBdpif-netdev/action-impl-get\fR +Lists the actions implementations that are available. +. +.IP "\fBdpif-netdev/action-impl-set\fR \fIaction_impl\fR" +Sets the action to be used to \fIaction_impl\fR. By default "scalar" is used. diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index f74f8b864..0a6437b0c 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -63,6 +63,7 @@ #include "netdev-vport.h" #include "netlink.h" #include "odp-execute.h" +#include "odp-execute-private.h" #include "odp-util.h" #include "openvswitch/dynamic-string.h" #include "openvswitch/list.h" @@ -1398,6 +1399,38 @@ error: ds_destroy(&reply); } +static void +action_impl_set(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + int32_t err = odp_actions_impl_set(argv[1]); + if (err) { + ds_put_format(&reply, "action implementation %s not found.\n", + argv[1]); + const char *reply_str = ds_cstr(&reply); + unixctl_command_reply_error(conn, reply_str); + VLOG_ERR("%s", reply_str); + ds_destroy(&reply); + return; + } + + ds_put_format(&reply, "action implementation set to %s.\n", argv[1]); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + +static void +action_impl_get(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + odp_execute_action_get(&reply); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + static void dpif_netdev_pmd_rebalance(struct unixctl_conn *conn, int argc, const char *argv[], void *aux OVS_UNUSED) @@ -1635,6 +1668,12 @@ dpif_netdev_init(void) unixctl_command_register("dpif-netdev/miniflow-parser-get", "", 0, 0, dpif_miniflow_extract_impl_get, NULL); + unixctl_command_register("dpif-netdev/action-impl-set", "name", + 1, 1, action_impl_set, + NULL); + unixctl_command_register("dpif-netdev/action-impl-get", "", + 0, 0, action_impl_get, + NULL); return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index e85aed636..8f5f8723f 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -81,6 +81,20 @@ odp_execute_action_set(const char *name, return -1; } +void +odp_execute_action_get(struct ds *string) +{ + uint32_t i; + + ds_put_cstr(string, "Available Actions implementations:\n"); + for (i = 0; i < ACTION_IMPL_MAX; i++) { + ds_put_format(string, " %s (available: %s, active: %s)\n", + action_impls[i].name, + action_impls[i].available ? "True" : "False", + i == active_action_impl_index ? "True" : "False"); + } +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 6441392b9..4f4cdc4ac 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -23,6 +23,7 @@ #include #include "openvswitch/types.h" +struct ds; struct nlattr; struct dp_packet; struct pkt_metadata; @@ -32,6 +33,8 @@ struct dp_packet_batch; /* Called once at initialization time. */ void odp_execute_init(void); +/* Runtime update get/set functionality. */ +int32_t odp_actions_impl_get(struct ds *name); int32_t odp_actions_impl_set(const char *name); typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, diff --git a/tests/pmd.at b/tests/pmd.at index 0a451f33c..65b4376e4 100644 --- a/tests/pmd.at +++ b/tests/pmd.at @@ -1200,3 +1200,24 @@ ovs-appctl: ovs-vswitchd: server returned an error OVS_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([PMD - ovs-actions configuration]) +OVS_VSWITCHD_START([], [], [], [--dummy-numa 0,0]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dummy-pmd]) + +dnl Set the scalar first, so we always have the scalar impl as Active. +AT_CHECK([ovs-appctl dpif-netdev/action-impl-set scalar], [0], [dnl +action implementation set to scalar. +]) + +AT_CHECK([ovs-vsctl show], [], [stdout]) +AT_CHECK([ovs-appctl dpif-netdev/action-impl-get | grep "scalar"], [], [dnl + scalar (available: True, active: True) +]) + +AT_CHECK([ovs-appctl dpif-netdev/action-impl-get | grep "autovalidator"], [], [dnl + autovalidator (available: True, active: False) +]) + +OVS_VSWITCHD_STOP +AT_CLEANUP \ No newline at end of file From patchwork Tue May 10 14:21:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629191 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=l9EXLH7d; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKyx5Y4lz9sCq for ; Wed, 11 May 2022 00:23:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A1AC741778; Tue, 10 May 2022 14:23:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J1CDXWrzMz2e; Tue, 10 May 2022 14:23:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1733F41763; Tue, 10 May 2022 14:23:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C43C4C0032; Tue, 10 May 2022 14:23:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 04771C0032 for ; Tue, 10 May 2022 14:23:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D72A482ACD for ; Tue, 10 May 2022 14:23:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FnONJ-yO-WsJ for ; Tue, 10 May 2022 14:23:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2BCAF827CE for ; Tue, 10 May 2022 14:23:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192596; x=1683728596; h=from:to:subject:date:message-id:in-reply-to:references: mime-version:content-transfer-encoding; bh=w6aBqKMV+YrUh7Z2/U2myrp/PuJ2HLq5Js+LdXUAmPc=; b=l9EXLH7dG47lV8gVtwK/zODpkEsJW/XmXTyojXaC7//UvrqdE/nK8GUF MwomTrQlp9JS4huN5BRVo8l5rjRF4bUyoCQuB0uo1FwVNx4T7lDrvpdWB 1O06I5aNe6y5uflggpuC3uCvF3k83+BAXIKhrahkY5+jhWbWYN8KPRsP9 sO0J7jR6scXS1pQc8Px5f6Gisvtj8NgT0542MuUxvwecoD0zRLWJZQ+Ks tnEmkeU3zrgAwee2cF/bqBYz92yWhHCRgiZpXtYn1mJN3Wua0GJvH+A9V zpsvGGAQAZGtZZ88XeuM5GHbKM5E0kwMvbklRdEsaaqAIiTDXrpxiWL+R g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975901" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975901" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796942" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:13 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:57 +0000 Message-Id: <20220510142202.1087967-7-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 06/11] dpif-netdev: Add configure option to enable actions autovalidator at build time. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Kumar Amber This commit adds a new command to allow the user to enable the actions autovalidator by default at build time thus allowing for running unit test by default. $ ./configure --enable-actions-default-autovalidator Signed-off-by: Kumar Amber Acked-by: Harry van Haaren --- NEWS | 2 ++ acinclude.m4 | 17 +++++++++++++++++ configure.ac | 1 + lib/odp-execute.c | 4 ++++ 4 files changed, 24 insertions(+) diff --git a/NEWS b/NEWS index eecdcda5e..73796e4d5 100644 --- a/NEWS +++ b/NEWS @@ -62,6 +62,8 @@ v2.17.0 - 17 Feb 2022 implementations against default implementation. * Add command line option to switch between different actions implementations available at run time. + * Add build time configure command to enable auto-validator as default + actions implementation at build time. - Python: * For SSL support, the use of the pyOpenSSL library has been replaced with the native 'ssl' module. diff --git a/acinclude.m4 b/acinclude.m4 index 61e88105f..0e80a17a6 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -14,6 +14,23 @@ # See the License for the specific language governing permissions and # limitations under the License. +dnl Set OVS Actions Autovalidator as default action at compile time? +dnl This enables automatically running all unit tests with all actions +dnl implementations. +AC_DEFUN([OVS_CHECK_ACTIONS_AUTOVALIDATOR], [ + AC_ARG_ENABLE([actions-default-autovalidator], + [AC_HELP_STRING([--enable-actions-default-autovalidator], [Enable actions autovalidator as default ovs actions implementation.])], + [autovalidator=yes],[autovalidator=no]) + AC_MSG_CHECKING([whether actions Autovalidator is default implementation]) + if test "$autovalidator" != yes; then + AC_MSG_RESULT([no]) + else + OVS_CFLAGS="$OVS_CFLAGS -DACTIONS_AUTOVALIDATOR_DEFAULT" + AC_MSG_RESULT([yes]) + fi +]) + + dnl Set OVS MFEX Autovalidator as default miniflow extract at compile time? dnl This enables automatically running all unit tests with all MFEX dnl implementations. diff --git a/configure.ac b/configure.ac index a79109bda..9bb3d637b 100644 --- a/configure.ac +++ b/configure.ac @@ -187,6 +187,7 @@ OVS_CTAGS_IDENTIFIERS OVS_CHECK_DPCLS_AUTOVALIDATOR OVS_CHECK_DPIF_AVX512_DEFAULT OVS_CHECK_MFEX_AUTOVALIDATOR +OVS_CHECK_ACTIONS_AUTOVALIDATOR OVS_CHECK_AVX512 AC_ARG_VAR(KARCH, [Kernel Architecture String]) diff --git a/lib/odp-execute.c b/lib/odp-execute.c index c2be74454..ba532101f 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -868,7 +868,11 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); +#ifdef ACTIONS_AUTOVALIDATOR_DEFAULT + odp_actions_impl_set("autovalidator"); +#else odp_actions_impl_set("scalar"); +#endif ovsthread_once_done(&once); } } From patchwork Tue May 10 14:21:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629192 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=RVT0xcev; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKz75hNnz9sCq for ; Wed, 11 May 2022 00:23:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1601361022; Tue, 10 May 2022 14:23:46 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xo9brmg19Atc; Tue, 10 May 2022 14:23:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id DED026103A; Tue, 10 May 2022 14:23:43 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C1570C0032; Tue, 10 May 2022 14:23:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 356E2C002D for ; Tue, 10 May 2022 14:23:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7331A828AA for ; Tue, 10 May 2022 14:23:21 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BLCui_Ft7yTy for ; Tue, 10 May 2022 14:23:20 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id 314738289B for ; Tue, 10 May 2022 14:23:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192600; x=1683728600; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xsxLe1UBgPf/ZZK5kxRGNH8JU1+dUTj+OcdhLRdictg=; b=RVT0xcevKYoIz4wc6mJsFodfAy7vy4Te5hZlOVHUgPwibAitXnB0W9dU zQFF7DDLMulCkMjTX5TgH+NuiVzysp0DGokhhwQlMBAy+g6SpA04nzhlB hQVJawTIhpTTAEy5fryIh7hpzeAfqV4/9TjsYVRXq3BTOBxqcozXq8SBP yB8QpE1YkNKh7XifJh9T5WR/PGkHpQKhM91gCM1r0by2vAoUoIdEjPVEk RuSIb1a0urmpM3dPcm+pwhDqWE5Gm06wncpiy4f9yLRghHjZATsC2gGmV XHHn3bNTskBXvlBAgmi8HvAHNUZxYZJF49ExFcDrRmS7SrIg6fBMHCHzw g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975924" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975924" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541796981" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:17 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:58 +0000 Message-Id: <20220510142202.1087967-8-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 07/11] odp-execute: Add ISA implementation of actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the action functionality. Usage: $ ovs-appctl dpif-netdev/action-impl-set avx512 Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- Documentation/topics/dpdk/bridge.rst | 25 +++++++++++ Documentation/topics/testing.rst | 20 ++++++--- NEWS | 1 + lib/automake.mk | 4 +- lib/cpu.c | 1 + lib/cpu.h | 1 + lib/odp-execute-avx512.c | 67 ++++++++++++++++++++++++++++ lib/odp-execute-private.c | 9 ++++ lib/odp-execute-private.h | 9 ++++ 9 files changed, 129 insertions(+), 8 deletions(-) create mode 100644 lib/odp-execute-avx512.c diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst index ceee91015..67089e08f 100644 --- a/Documentation/topics/dpdk/bridge.rst +++ b/Documentation/topics/dpdk/bridge.rst @@ -321,3 +321,28 @@ following command:: ``scalar`` can be selected on core ``3`` by the following command:: $ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 scalar + +Actions Performance +------------------- + +Actions are used in OpenFlow flows to describe what to do when the flow +matches a packet. Just like with the datapath interface, SIMD instructions +can be applied to the action implementation to improve performance. + +OVS provides multiple implementations of the actions. +Available implementations can be listed with the following command:: + + $ ovs-appctl dpif-netdev/action-impl-get + Available Actions implementations: + scalar (available: True, active: True) + autovalidator (available: True, active: False) + avx512 (available: True, active: False) + +By default, ``scalar`` is used. Implementations can be selected by +name:: + + $ ovs-appctl dpif-netdev/action-impl-set avx512 + action implementation set to avx512. + + $ ovs-appctl dpif-netdev/action-impl-set scalar + action implementation set to scalar. diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index c15d5b38f..10d0ecc48 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -361,12 +361,12 @@ testsuite. Userspace datapath: Testing and Validation of CPU-specific Optimizations '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -As multiple versions of the datapath classifier and packet parsing functions -can co-exist, each with different CPU ISA optimizations, it is important to -validate that they all give the exact same results. To easily test all the -implementations, an ``autovalidator`` implementation of them exists. This -implementation runs all other available implementations, and verifies that the -results are identical. +As multiple versions of the datapath classifier, packet parsing functions and +actions can co-exist, each with different CPU ISA optimizations, it is +important to validate that they all give the exact same results. To easily +test all the implementations, an ``autovalidator`` implementation of them +exists. This implementation runs all other available implementations, and +verifies that the results are identical. Running the OVS unit tests with the autovalidator enabled ensures all implementations provide the same results. Note that the performance of the @@ -382,18 +382,24 @@ To set the autovalidator for the packet parser, use this command:: $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator +To set the autovalidator for actions, use this command:: + + $ ovs-appctl dpif-netdev/action-impl-set autovalidator + To run the OVS unit test suite with the autovalidator as the default implementation, it is required to recompile OVS. During the recompilation, the default priority of the `autovalidator` implementation is set to the maximum priority, ensuring every test will be run with every implementation:: - $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator + $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator \ + --enable-actions-default-autovalidator The following line should be seen in the configuration log when the above options are used:: checking whether DPCLS Autovalidator is default implementation... yes checking whether MFEX Autovalidator is default implementation... yes + checking whether actions Autovalidator is default implementation... yes Compile OVS in debug mode to have `ovs_assert` statements error out if there is a mis-match in the datapath classifier lookup or packet parser diff --git a/NEWS b/NEWS index 73796e4d5..35d6b0f4a 100644 --- a/NEWS +++ b/NEWS @@ -64,6 +64,7 @@ v2.17.0 - 17 Feb 2022 implementations available at run time. * Add build time configure command to enable auto-validator as default actions implementation at build time. + * Add AVX512 implementation of actions. - Python: * For SSL support, the use of the pyOpenSSL library has been replaced with the native 'ssl' module. diff --git a/lib/automake.mk b/lib/automake.mk index 625c0d9c9..2973ec12d 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -32,6 +32,7 @@ lib_libopenvswitch_la_LIBADD += lib/libopenvswitchavx512.la lib_libopenvswitchavx512_la_CFLAGS = \ -mavx512f \ -mavx512bw \ + -mavx512vl \ -mavx512dq \ -mbmi \ -mbmi2 \ @@ -42,7 +43,8 @@ lib_libopenvswitchavx512_la_SOURCES = \ lib/cpu.h \ lib/dpif-netdev-lookup-avx512-gather.c \ lib/dpif-netdev-extract-avx512.c \ - lib/dpif-netdev-avx512.c + lib/dpif-netdev-avx512.c \ + lib/odp-execute-avx512.c lib_libopenvswitchavx512_la_LDFLAGS = \ -static endif diff --git a/lib/cpu.c b/lib/cpu.c index 2df003c51..0292f715e 100644 --- a/lib/cpu.c +++ b/lib/cpu.c @@ -53,6 +53,7 @@ X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 16, OVS_CPU_ISA_X86_AVX512F) X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 30, OVS_CPU_ISA_X86_AVX512BW) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 1, OVS_CPU_ISA_X86_AVX512VBMI) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 14, OVS_CPU_ISA_X86_VPOPCNTDQ) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 31, OVS_CPU_ISA_X86_AVX512VL) #endif bool diff --git a/lib/cpu.h b/lib/cpu.h index 92897bb71..3215229bc 100644 --- a/lib/cpu.h +++ b/lib/cpu.h @@ -25,6 +25,7 @@ enum ovs_cpu_isa { OVS_CPU_ISA_X86_AVX512F, OVS_CPU_ISA_X86_AVX512BW, OVS_CPU_ISA_X86_AVX512VBMI, + OVS_CPU_ISA_X86_AVX512VL, OVS_CPU_ISA_X86_VPOPCNTDQ, OVS_CPU_ISA_X86_LAST = OVS_CPU_ISA_X86_VPOPCNTDQ, }; diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c new file mode 100644 index 000000000..84f68d378 --- /dev/null +++ b/lib/odp-execute-avx512.c @@ -0,0 +1,67 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "cpu.h" +#include "dp-packet.h" +#include "immintrin.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "openvswitch/vlog.h" + +/* Probe functions to check ISA requirements. */ +static int32_t +avx512_isa_probe(uint32_t needs_vbmi) +{ + static enum ovs_cpu_isa isa_required[] = { + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_BMI2, + OVS_CPU_ISA_X86_AVX512VL + }; + + int32_t ret = 0; + for (uint32_t i = 0; i < ARRAY_SIZE(isa_required); i++) { + if (!cpu_has_isa(isa_required[i])) { + ret = -ENOTSUP; + } + } + + if (needs_vbmi) { + if (!cpu_has_isa(OVS_CPU_ISA_X86_AVX512VBMI)) { + ret = -ENOTSUP; + } + } + + return ret; +} + +int32_t +action_avx512_probe(void) +{ + const uint32_t needs_vbmi = 0; + return avx512_isa_probe(needs_vbmi); +} + + +int32_t +action_avx512_init(void) +{ + avx512_isa_probe(0); + return 0; +} diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 8f5f8723f..2bfa84152 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -46,6 +46,15 @@ static struct odp_execute_action_impl action_impls[] = { .probe = NULL, .init_func = odp_action_scalar_init, }, + + #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) + [ACTION_IMPL_AVX512] = { + .available = 1, + .name = "avx512", + .probe = action_avx512_probe, + .init_func = NULL, + }, + #endif }; static void diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index fed20930d..13fc74e52 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -71,6 +71,9 @@ enum odp_execute_action_impl_idx { * Do not change the autovalidator position in this list without updating * the define below. */ + #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) + ACTION_IMPL_AVX512, + #endif ACTION_IMPL_MAX, }; @@ -96,4 +99,10 @@ int32_t odp_execute_action_set(const char *name, */ int32_t odp_action_scalar_init(struct odp_execute_action_impl *self); +/* Init function for the optimized with AVX512 actions. */ +int32_t action_avx512_init(void); + +/* Probe function to check ISA requirements. */ +int32_t action_avx512_probe(void); + #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Tue May 10 14:21:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629193 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bmg3dMmQ; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyKzf1s2Gz9sCq for ; Wed, 11 May 2022 00:24:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7AA9F82F57; Tue, 10 May 2022 14:24:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FlVWC_EWcvxu; Tue, 10 May 2022 14:24:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3EFF282EDB; Tue, 10 May 2022 14:24:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EF660C0032; Tue, 10 May 2022 14:24:09 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 09544C0032 for ; Tue, 10 May 2022 14:24:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A55FC416F4 for ; Tue, 10 May 2022 14:23:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CQReeh_bRzrO for ; Tue, 10 May 2022 14:23:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0A70141762 for ; Tue, 10 May 2022 14:23:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192604; x=1683728604; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=maBAFBzIttNNAIrY0tnFvrNhGBPwSvqvf2O44nZEZXA=; b=bmg3dMmQT7Wq35x9t7B5w0QYLlfMtLrLISqOaaTyyjb7OOLfbDv+W4la ygKJuR8AGgZivUCBe1GqnlDLvtcFGWXam00qLqd0AD7tnpggnNR4l83rm 0jO4Pv9rFK7UuFdLmZqmxiK7UtMRTEP/KtEp7QudYUJ5gXv9EjyRnT24p oqZk2Nf+yFyFs69oEEeSOxWEaYbJaQLQKH7kAt9DJn1548OzRn+bCvNwF mv4WWxy5NkzQYk22w86FJ5XximVdOxtQOqjYWGyXEMmpaUC1EH+05NURx 6h1SU15wdEYYUvmMz3W8dQbCm1JJX6ZPZwJm36OZHzcbTC4GGvI0OPYdf w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975939" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975939" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541797025" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:21 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:21:59 +0000 Message-Id: <20220510142202.1087967-9-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 08/11] odp-execute: Add ISA implementation of pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the pop_vlan action. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 73 ++++++++++++++++++++++++++++++++++++++- lib/odp-execute-private.c | 2 +- lib/odp-execute-private.h | 2 +- 3 files changed, 74 insertions(+), 3 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 84f68d378..637956236 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -14,6 +14,11 @@ * limitations under the License. */ +#ifdef __x86_64__ +/* Sparse cannot handle the AVX512 instructions. */ +#if !defined(__CHECKER__) + + #include #include @@ -24,6 +29,67 @@ #include "odp-netlink.h" #include "openvswitch/vlog.h" +VLOG_DEFINE_THIS_MODULE(odp_execute_avx512); +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_5_ofs) + + MEMBER_SIZEOF(struct dp_packet, l2_5_ofs) == + offsetof(struct dp_packet, l3_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == + offsetof(struct dp_packet, l4_ofs)); + +static inline void ALWAYS_INLINE +avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) +{ + /* Update packet size/data pointers */ + dp_packet_set_data(b, (char *) dp_packet_data(b) - resize_by_bytes); + dp_packet_set_size(b, dp_packet_size(b) + resize_by_bytes); + + /* Increment u16 packet offset values */ + const __m128i v_zeros = _mm_setzero_si128(); + const __m128i v_u16_max = _mm_cmpeq_epi16(v_zeros, v_zeros); + + /* Only these lanes can be incremented/decremented for L2. */ + const uint8_t k_lanes = 0b1110; + __m128i v_offset = _mm_set1_epi16(VLAN_HEADER_LEN); + + /* Load packet and compare with UINT16_MAX */ + void *adjust_ptr = &b->l2_pad_size; + __m128i v_adjust_src = _mm_loadu_si128(adjust_ptr); + + /* Generate K mask to use for updating offset values of + * the packet buffer. */ + __mmask8 k_cmp = _mm_mask_cmpneq_epu16_mask(k_lanes, v_adjust_src, + v_u16_max); + + /* Update VLAN_HEADER_LEN using compare mask, store results. */ + __m128i v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + _mm_storeu_si128(adjust_ptr, v_adjust_wip); +} + +static void +action_avx512_pop_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct vlan_eth_header *veh = dp_packet_eth(packet); + + if (veh && dp_packet_size(packet) >= sizeof *veh && + eth_type_vlan(veh->veth_type)) { + + __m128i v_ether = _mm_loadu_si128((void *) veh); + __m128i v_realign = _mm_alignr_epi8(v_ether, _mm_setzero_si128(), + 16 - VLAN_HEADER_LEN); + _mm_storeu_si128((void *) veh, v_realign); + avx512_dp_packet_resize_l2(packet, -VLAN_HEADER_LEN); + } + } +} + /* Probe functions to check ISA requirements. */ static int32_t avx512_isa_probe(uint32_t needs_vbmi) @@ -60,8 +126,13 @@ action_avx512_probe(void) int32_t -action_avx512_init(void) +action_avx512_init(struct odp_execute_action_impl *self) { avx512_isa_probe(0); + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + return 0; } + +#endif +#endif diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 2bfa84152..8257bba80 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -52,7 +52,7 @@ static struct odp_execute_action_impl action_impls[] = { .available = 1, .name = "avx512", .probe = action_avx512_probe, - .init_func = NULL, + .init_func = action_avx512_init, }, #endif }; diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 13fc74e52..231d72492 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -100,7 +100,7 @@ int32_t odp_execute_action_set(const char *name, int32_t odp_action_scalar_init(struct odp_execute_action_impl *self); /* Init function for the optimized with AVX512 actions. */ -int32_t action_avx512_init(void); +int32_t action_avx512_init(struct odp_execute_action_impl *self); /* Probe function to check ISA requirements. */ int32_t action_avx512_probe(void); From patchwork Tue May 10 14:22:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629194 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=IXAgho18; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyL0c3Vqbz9sCq for ; Wed, 11 May 2022 00:25:03 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 195E060FF6; Tue, 10 May 2022 14:25:01 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3MKcIGPhWG3v; Tue, 10 May 2022 14:25:00 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 3098560FE3; Tue, 10 May 2022 14:24:59 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01E8EC0032; Tue, 10 May 2022 14:24:59 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4AFB9C002D for ; Tue, 10 May 2022 14:24:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0EDC141796 for ; Tue, 10 May 2022 14:23:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UNMeujdxgcWe for ; Tue, 10 May 2022 14:23:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp4.osuosl.org (Postfix) with ESMTPS id 7DD88417B7 for ; Tue, 10 May 2022 14:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192617; x=1683728617; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X7/kkvSo38fULXNALLwQEdCfTd0uLHsKBscan9v9bkM=; b=IXAgho18IrFqBOLcMfnWLOA9d3m48MugMzJDJ6fZIMNYHIMgRos/iCPR KqSV3is8TpQXiA8cnjZAfv7vBIjplt3nRMlKH+u+5lgQNU4/qC/VnFt9H pnXMGzqMaH5npcsJChEDmRGKOB2wY9mX5aPn/SNXPUb3l3uBRQeW5AR/6 6ghkf0lnxUAkpuD/9rvgWOjmqUlRPnjJqYj+rKSog/5pyfKL9r3cIrLYt LorVtkt3DQOlDxyD75zgkbZhJKJz5ScRITqPfNgfR7KNRFA6vJwKqw6S6 bcLPlQ6lMJgncv/SwS5P8Bkqs7x0oUtEMvCtFvqFOd+uie75vwF7GvajT w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975945" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975945" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541797049" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:25 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:22:00 +0000 Message-Id: <20220510142202.1087967-10-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 09/11] odp-execute: Add ISA implementation of push_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the push_vlan action. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 56 ++++++++++++++++++++++++++++++++++++--- lib/odp-execute-private.c | 1 + lib/odp-execute.c | 24 ++++++++++------- 3 files changed, 69 insertions(+), 12 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 637956236..5d095f867 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -42,6 +42,13 @@ static inline void ALWAYS_INLINE avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) { /* Update packet size/data pointers */ + if (resize_by_bytes >= 0) { + dp_packet_prealloc_headroom(b, resize_by_bytes); + } else { + ovs_assert(dp_packet_size(b) - dp_packet_l2_pad_size(b) >= + -resize_by_bytes); + } + dp_packet_set_data(b, (char *) dp_packet_data(b) - resize_by_bytes); dp_packet_set_size(b, dp_packet_size(b) + resize_by_bytes); @@ -51,7 +58,7 @@ avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) /* Only these lanes can be incremented/decremented for L2. */ const uint8_t k_lanes = 0b1110; - __m128i v_offset = _mm_set1_epi16(VLAN_HEADER_LEN); + __m128i v_offset = _mm_set1_epi16(abs(resize_by_bytes)); /* Load packet and compare with UINT16_MAX */ void *adjust_ptr = &b->l2_pad_size; @@ -63,8 +70,16 @@ avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) v_u16_max); /* Update VLAN_HEADER_LEN using compare mask, store results. */ - __m128i v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, - v_adjust_src, v_offset); + __m128i v_adjust_wip; + + if (resize_by_bytes >= 0) { + v_adjust_wip = _mm_mask_add_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } else { + v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } + _mm_storeu_si128(adjust_ptr, v_adjust_wip); } @@ -90,6 +105,40 @@ action_avx512_pop_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static void +action_avx512_push_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a, + bool should_steal OVS_UNUSED) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + ovs_be16 tpid, tci; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + tpid = vlan->vlan_tpid; + tci = vlan->vlan_tci; + + avx512_dp_packet_resize_l2(packet, VLAN_HEADER_LEN); + + /* Build up the VLAN TCI/TPID, and merge with the moving of Ether. */ + char *pkt_data = (char *) dp_packet_data(packet); + const uint16_t tci_proc = tci & htons(~VLAN_CFI); + const uint32_t tpid_tci = (tci_proc << 16) | tpid; + + static const uint8_t vlan_push_shuffle_mask[16] = { + 4, 5, 6, 7, 8, 9, 10, 11, + 12, 13, 14, 15, 0xFF, 0xFF, 0xFF, 0xFF + }; + + __m128i v_ether = _mm_loadu_si128((void *) pkt_data); + __m128i v_index = _mm_loadu_si128((void *) vlan_push_shuffle_mask); + __m128i v_shift = _mm_shuffle_epi8(v_ether, v_index); + __m128i v_vlan_hdr = _mm_insert_epi32(v_shift, tpid_tci, 3); + _mm_storeu_si128((void *) pkt_data, v_vlan_hdr); + + } +} + /* Probe functions to check ISA requirements. */ static int32_t avx512_isa_probe(uint32_t needs_vbmi) @@ -130,6 +179,7 @@ action_avx512_init(struct odp_execute_action_impl *self) { avx512_isa_probe(0); self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 8257bba80..6b09ad353 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -228,6 +228,7 @@ int32_t action_autoval_init(struct odp_execute_action_impl *self) { self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_autoval_generic; return 0; } diff --git a/lib/odp-execute.c b/lib/odp-execute.c index ba532101f..690e7e1ce 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -845,6 +845,19 @@ action_pop_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static void +action_push_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -852,6 +865,7 @@ int32_t odp_action_scalar_init(struct odp_execute_action_impl *self) { self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; return 0; } @@ -994,15 +1008,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_PUSH_VLAN: { - const struct ovs_action_push_vlan *vlan = nl_attr_get(a); - - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); - } - break; - } - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1147,6 +1152,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case OVS_ACTION_ATTR_OUTPUT: case OVS_ACTION_ATTR_LB_OUTPUT: case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_PUSH_VLAN: case OVS_ACTION_ATTR_TUNNEL_PUSH: case OVS_ACTION_ATTR_TUNNEL_POP: case OVS_ACTION_ATTR_USERSPACE: From patchwork Tue May 10 14:22:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629196 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=lk01kDn+; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyL0t1XKQz9sCq for ; Wed, 11 May 2022 00:25:18 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6E6FD41740; Tue, 10 May 2022 14:25:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z-LN4uxb0B9A; Tue, 10 May 2022 14:25:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 9E5024172A; Tue, 10 May 2022 14:25:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 61B2BC007B; Tue, 10 May 2022 14:25:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 221E0C007B for ; Tue, 10 May 2022 14:25:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 391E04172A for ; Tue, 10 May 2022 14:23:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lOD8CE3gDK0U for ; Tue, 10 May 2022 14:23:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp4.osuosl.org (Postfix) with ESMTPS id BBBAA417BD for ; Tue, 10 May 2022 14:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192617; x=1683728617; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e9ZgHhzFwIlK+8mghiHLvujArKJiFzPcnJ3/a1i6Iuc=; b=lk01kDn+MogRYrSM54nTDcdwZCvl97IYbrg8LDFTRh+XF29OoZf2ajxS KUCo6wg4/yUDGrBuVSyXqIjEqUspayDyK8DzOrWlqIc91kp9MGDj8Azme v4fLzrMRjY40FpCRaUjetzVtHPsGogdzVeTadeU6ZVwvnBbQvnDrZk53l MnMwTCSQ1aUb3NNKES2zqo2cbUCyw5qUjPkRPDXm6W5lfQsNyaxWwUhmW qyojEF86MflfjE1l3UNozFBxtT29DQ1Jzk/jrIYHcay0PrzCZCoLC7srT 31vy6k0T4zT0HCBRK+7kWMLpG7+STKZmzMsHGnNZuWPtX6lg2StqXqnK/ g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975952" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975952" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541797059" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:28 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:22:01 +0000 Message-Id: <20220510142202.1087967-11-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 10/11] odp-execute: Add ISA implementation of set_masked ETH action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit includes infrastructure changes for enabling set_masked_X actions and also adds support for the AVX512 implementation of the eth_set_addrs action. Signed-off-by: Emma Finn --- .../linux/compat/include/linux/openvswitch.h | 2 +- lib/odp-execute-avx512.c | 56 ++++++++++++++- lib/odp-execute-private.c | 68 ++++++++++++++++-- lib/odp-execute-private.h | 5 +- lib/odp-execute.c | 69 ++++++++++++++----- lib/odp-execute.h | 4 +- 6 files changed, 174 insertions(+), 30 deletions(-) diff --git a/datapath/linux/compat/include/linux/openvswitch.h b/datapath/linux/compat/include/linux/openvswitch.h index 8bb5abdc8..ccb54d6c6 100644 --- a/datapath/linux/compat/include/linux/openvswitch.h +++ b/datapath/linux/compat/include/linux/openvswitch.h @@ -473,8 +473,8 @@ enum ovs_frag_type { #define OVS_FRAG_TYPE_MAX (__OVS_FRAG_TYPE_MAX - 1) struct ovs_key_ethernet { - __u8 eth_src[ETH_ALEN]; __u8 eth_dst[ETH_ALEN]; + __u8 eth_src[ETH_ALEN]; }; struct ovs_key_mpls { diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 5d095f867..ede00b750 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -38,6 +38,12 @@ BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == offsetof(struct dp_packet, l4_ofs)); +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_dst) + + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_dst) == + offsetof(struct ovs_key_ethernet, eth_src)); + +static struct odp_execute_action_impl active_impl; + static inline void ALWAYS_INLINE avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) { @@ -139,6 +145,51 @@ action_avx512_push_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static void +action_avx512_eth_set_addrs(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + const struct ovs_key_ethernet *key = nl_attr_get(a); + const struct ovs_key_ethernet *mask = get_mask(a, struct ovs_key_ethernet); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + + struct eth_header *eh = dp_packet_eth(packet); + + if (!eh) { + continue; + } + + __m128i v_src = _mm_maskz_loadu_epi16(0x3F, key); + __m128i v_mask = _mm_maskz_loadu_epi16(0x3F, mask); + __m128i v_dst = _mm_maskz_loadu_epi16(0xFF, eh); + + __m128i dst_masked = _mm_andnot_si128(v_mask, v_dst); + __m128i res = _mm_or_si128(v_src, dst_masked); + + __m128i res_blend = _mm_blend_epi16(v_dst, res, 0x3F); + _mm_storeu_si128((void *) eh, res_blend); + } +} + +static void +action_avx512_set_masked(void *dp OVS_UNUSED, + struct dp_packet_batch *batch OVS_UNUSED, + const struct nlattr *a, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(a); + + if (active_impl.set_masked_funcs[attr_type]) { + active_impl.set_masked_funcs[attr_type](NULL, batch, a, should_steal); + } + +} + /* Probe functions to check ISA requirements. */ static int32_t avx512_isa_probe(uint32_t needs_vbmi) @@ -173,13 +224,16 @@ action_avx512_probe(void) return avx512_isa_probe(needs_vbmi); } - int32_t action_avx512_init(struct odp_execute_action_impl *self) { avx512_isa_probe(0); self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_avx512_set_masked; + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = + action_avx512_eth_set_addrs; + active_impl = *self; return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 6b09ad353..34f13523a 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -31,6 +31,8 @@ int32_t action_autoval_init(struct odp_execute_action_impl *self); VLOG_DEFINE_THIS_MODULE(odp_execute_private); static uint32_t active_action_impl_index; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); +static struct odp_execute_action_impl active_impl; +static bool set_masked = false; static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AUTOVALIDATOR] = { @@ -63,6 +65,11 @@ action_impl_init_funcs(struct odp_execute_action_impl *to) for (uint32_t i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { atomic_init(&to->funcs[i], NULL); } + + for (uint32_t i = 0; i < __OVS_KEY_ATTR_MAX; i++) { + atomic_init(&to->set_masked_funcs[i], NULL); + } + } static void @@ -72,6 +79,11 @@ action_impl_copy_funcs(struct odp_execute_action_impl *to, for (uint32_t i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { atomic_store_relaxed(&to->funcs[i], from->funcs[i]); } + + for (uint32_t i = 0; i < __OVS_KEY_ATTR_MAX; i++) { + atomic_store_relaxed(&to->set_masked_funcs[i], + from->set_masked_funcs[i]); + } } int32_t @@ -155,21 +167,40 @@ action_autoval_generic(void *dp OVS_UNUSED, struct dp_packet_batch *batch, uint32_t failed = 0; int type = nl_attr_type(a); - enum ovs_action_attr attr_type = (enum ovs_action_attr) type; + enum ovs_action_attr action_attr_type = (enum ovs_action_attr) type; + enum ovs_key_attr key_attr_type = (enum ovs_key_attr) type; + + if (action_attr_type == OVS_ACTION_ATTR_SET_MASKED) { + set_masked = true; + const struct nlattr *key = nl_attr_get(a); + key_attr_type = nl_attr_type(key); + } struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; struct dp_packet_batch good_batch; dp_packet_batch_clone(&good_batch, batch); - - scalar->funcs[attr_type](NULL, &good_batch, a, should_steal); + if (!set_masked) { + scalar->funcs[action_attr_type](NULL, &good_batch, a, should_steal); + } else { + scalar->set_masked_funcs[key_attr_type](NULL, &good_batch, a, + should_steal); + } for (uint32_t impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { /* Clone original batch and execute implementation under test. */ struct dp_packet_batch test_batch; dp_packet_batch_clone(&test_batch, batch); - action_impls[impl].funcs[attr_type](NULL, &test_batch, a, - should_steal); + + if (!set_masked) { + action_impls[impl].funcs[action_attr_type](NULL, &test_batch, a, + should_steal); + } else { + action_impls[impl].set_masked_funcs[key_attr_type](NULL, + &test_batch, + a, + should_steal); + } /* Loop over implementations, checking each one. */ for (uint32_t pidx = 0; pidx < batch->count; pidx++) { @@ -221,7 +252,29 @@ action_autoval_generic(void *dp OVS_UNUSED, struct dp_packet_batch *batch, dp_packet_delete_batch(&good_batch, 1); /* Apply the action to the original batch for continued processing. */ - scalar->funcs[attr_type](NULL, batch, a, should_steal); + if (!set_masked) { + scalar->funcs[action_attr_type](NULL, batch, a, should_steal); + } else { + scalar->set_masked_funcs[key_attr_type](NULL, batch, a, should_steal); + } + + set_masked = false; +} + +static void +action_set_masked_init(void *dp OVS_UNUSED, + struct dp_packet_batch *batch OVS_UNUSED, + const struct nlattr *a, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(a); + + if (active_impl.set_masked_funcs[attr_type]) { + set_masked = true; + active_impl.set_masked_funcs[attr_type](NULL, batch, a, should_steal); + } + } int32_t @@ -229,6 +282,9 @@ action_autoval_init(struct odp_execute_action_impl *self) { self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_autoval_generic; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked_init; + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_autoval_generic; + active_impl = *self; return 0; } diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 231d72492..2c0233f10 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -60,7 +60,9 @@ struct odp_execute_action_impl { odp_execute_action_init_func init_func; /* An array of callback functions, one for each action. */ - ATOMIC(odp_execute_cb) funcs[__OVS_KEY_ATTR_MAX]; + ATOMIC(odp_execute_cb) funcs[__OVS_ACTION_ATTR_MAX]; + /* An array of callback functions, one for each key. */ + ATOMIC(odp_execute_cb) set_masked_funcs[__OVS_KEY_ATTR_MAX]; }; /* Order of Actions implementations. */ @@ -93,7 +95,6 @@ void odp_execute_action_init(void); void odp_execute_action_get(struct ds *name); int32_t odp_execute_action_set(const char *name, struct odp_execute_action_impl *active); - /* Init function for the scalar implementation. Calls into the odp-execute.c * file, and initializes the function pointers for optimized action types. */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 690e7e1ce..5c4dd8e33 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -561,8 +561,6 @@ odp_execute_set_action(struct dp_packet *packet, const struct nlattr *a) } } -#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) - static void odp_execute_masked_set_action(struct dp_packet *packet, const struct nlattr *a) @@ -582,11 +580,6 @@ odp_execute_masked_set_action(struct dp_packet *packet, | (md->pkt_mark & ~*get_mask(a, uint32_t)); break; - case OVS_KEY_ATTR_ETHERNET: - odp_eth_set_addrs(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ethernet)); - break; - case OVS_KEY_ATTR_NSH: { odp_set_nsh(packet, a, true); break; @@ -661,6 +654,7 @@ odp_execute_masked_set_action(struct dp_packet *packet, case OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV4: case OVS_KEY_ATTR_CT_ORIG_TUPLE_IPV6: case OVS_KEY_ATTR_ENCAP: + case OVS_KEY_ATTR_ETHERNET: case OVS_KEY_ATTR_ETHERTYPE: case OVS_KEY_ATTR_IN_PORT: case OVS_KEY_ATTR_VLAN: @@ -833,6 +827,11 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. + */ +static struct odp_execute_action_impl actions_active_impl; static void action_pop_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, @@ -858,6 +857,41 @@ action_push_vlan(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static void +action_set_masked(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + struct dp_packet *packet; + + const struct nlattr *key = nl_attr_get(a); + enum ovs_key_attr key_type = nl_attr_type(key); + + if (actions_active_impl.set_masked_funcs[key_type]) { + actions_active_impl.set_masked_funcs[key_type](NULL, batch, a, + should_steal); + } else { + a = nl_attr_get(a); + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_execute_masked_set_action(packet, a); + } + } +} + +static void +action_mod_eth(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_eth_set_addrs(packet, nl_attr_get(a), + get_mask(a, struct ovs_key_ethernet)); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -866,15 +900,13 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) { self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked; + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_mod_eth; + actions_active_impl = *self; return 0; } -/* The active function pointers on the datapath. ISA optimized implementations - * are enabled by plugging them into this static arary, which is consulted when - * applying actions on the datapath. - */ -static struct odp_execute_action_impl actions_active_impl; void odp_execute_init(void) @@ -952,7 +984,11 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, * function-pointer and continue to the next action. */ enum ovs_action_attr attr_type = (enum ovs_action_attr) type; - if (actions_active_impl.funcs[attr_type]) { + + if (attr_type == OVS_ACTION_ATTR_SET_MASKED) { + action_set_masked(NULL, batch, a, should_steal); + continue; + } else if (actions_active_impl.funcs[attr_type]) { actions_active_impl.funcs[attr_type](NULL, batch, a, should_steal); continue; } @@ -1029,12 +1065,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, } break; - case OVS_ACTION_ATTR_SET_MASKED: - DP_PACKET_BATCH_FOR_EACH(i, packet, batch) { - odp_execute_masked_set_action(packet, nl_attr_get(a)); - } - break; - case OVS_ACTION_ATTR_SAMPLE: DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { odp_execute_sample(dp, packet, steal && last_action, a, @@ -1153,6 +1183,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case OVS_ACTION_ATTR_LB_OUTPUT: case OVS_ACTION_ATTR_POP_VLAN: case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_SET_MASKED: case OVS_ACTION_ATTR_TUNNEL_PUSH: case OVS_ACTION_ATTR_TUNNEL_POP: case OVS_ACTION_ATTR_USERSPACE: diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 4f4cdc4ac..a14af2b4e 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -34,7 +34,6 @@ struct dp_packet_batch; void odp_execute_init(void); /* Runtime update get/set functionality. */ -int32_t odp_actions_impl_get(struct ds *name); int32_t odp_actions_impl_set(const char *name); typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, @@ -48,4 +47,7 @@ void odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, const struct nlattr *actions, size_t actions_len, odp_execute_cb dp_execute_action); + +#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) + #endif From patchwork Tue May 10 14:22:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1629195 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=ADHU/nb9; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KyL0d5vW1z9sCq for ; Wed, 11 May 2022 00:25:05 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 28C3782BBC; Tue, 10 May 2022 14:25:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tJ8A_1PR-6vf; Tue, 10 May 2022 14:25:03 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0DE3A82ADD; Tue, 10 May 2022 14:25:01 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C4852C0081; Tue, 10 May 2022 14:25:01 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8C8F7C0032 for ; Tue, 10 May 2022 14:25:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8914A417B8 for ; Tue, 10 May 2022 14:23:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JAaeVP5phAqa for ; Tue, 10 May 2022 14:23:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp4.osuosl.org (Postfix) with ESMTPS id 24765417BA for ; Tue, 10 May 2022 14:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652192618; x=1683728618; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=58ba4AyjFL2g+yQ5HjN7VZnWAZ5CYXBwO3uaX4clKIM=; b=ADHU/nb9fywIwLfzP81nd4Ajgftw1cB0K5krmLxBQmTvj9ZJhnRgEKta 3WbGXgxz4qD1DDJff1yNUnQgseJiYj432Dj2RnLFUj6iHanpjnbhCP+gh ZWV32qVpdN7FKQSXwHEYo79nNAchjq8MWpzoFIOARLHyPvwcfQOD+iiCS em3WLRo0yvInkKhNY5FsJYmd8C5whiqqL5FQhulmZyWJnCETNaOuF/Zlo TvXpPOc3QfaEnAx3wugDBtz3tzRioD8xccHR7E23tAEdPtUdjUZVquDVq yYCopAUGAKXIcP8PBPaehJpkf6+b/rtw0wtYrjTwM23mLPY6wNn+3MFte g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="266975961" X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="266975961" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 May 2022 07:23:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,214,1647327600"; d="scan'208";a="541797100" Received: from silpixa00400899.ir.intel.com ([10.243.22.42]) by orsmga006.jf.intel.com with ESMTP; 10 May 2022 07:23:32 -0700 From: Emma Finn To: echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com, ian.stokes@intel.com, dev@openvswitch.org Date: Tue, 10 May 2022 14:22:02 +0000 Message-Id: <20220510142202.1087967-12-emma.finn@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220510142202.1087967-1-emma.finn@intel.com> References: <20220112094244.81402-1-emma.finn@intel.com> <20220510142202.1087967-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [v6 11/11] odp-execute: Add ISA implementation of set_masked IPv4 action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds support for the AVX512 implementation of the ipv4_set_addrs action as well as an AVX512 implementation of updating the checksums. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 194 ++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 1 + lib/odp-execute.c | 21 ++++- 3 files changed, 211 insertions(+), 5 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index ede00b750..618fa37a7 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -22,6 +22,7 @@ #include #include +#include "csum.h" #include "cpu.h" #include "dp-packet.h" #include "immintrin.h" @@ -175,6 +176,197 @@ action_avx512_eth_set_addrs(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static inline uint16_t ALWAYS_INLINE +avx512_l4_update_csum(struct ip_header *old_header, __m256i res) +{ + uint16_t tmp_checksum; + __m256i v_zeros = _mm256_setzero_si256(); + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xffff, 0x0302, 0xffff, + 0x0504, 0xffff, 0x0706, 0xffff, + 0x0100, 0xffff, 0x0302, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xffff, 0xffff, 0xffff, + 0x0d0c, 0xffff, 0x0f0e, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + + __m256i oh = _mm256_loadu_si256((void *) old_header); + oh = _mm256_mask_blend_epi16(0x3C0, oh, res); + __m256i v_shuf1 = _mm256_shuffle_epi8(oh, v_swap16a); + __m256i v_shuf2 = _mm256_shuffle_epi8(oh, v_swap16b); + + /* Add field values. */ + __m256i v_sum = _mm256_add_epi32(v_shuf1, v_shuf2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final hadd. */ + v_sum = _mm256_permutexvar_epi32(v_swap32a, v_sum); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi16(v_sum, v_zeros); + + /* Extract checksum value. */ + tmp_checksum = _mm256_extract_epi16(v_sum, 0); + + return ~tmp_checksum; +} + +static inline uint16_t ALWAYS_INLINE +avx512_ipv4_recalc_csum(__m256i res) +{ + uint32_t new_checksum; + __m256i v_zeros = _mm256_setzero_si256(); + + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xffff, 0x0302, 0xffff, + 0x0504, 0xffff, 0x0706, 0xffff, + 0x0100, 0xffff, 0x0302, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xffff, 0xffff, 0xffff, + 0x0d0c, 0xffff, 0x0f0e, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + + __m256i v_shuf1 = _mm256_shuffle_epi8(res, v_swap16a); + __m256i v_shuf2 = _mm256_shuffle_epi8(res, v_swap16b); + + /* Add field values. */ + __m256i v_sum = _mm256_add_epi32(v_shuf1, v_shuf2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final hadd. */ + v_sum = _mm256_permutexvar_epi32(v_swap32a, v_sum); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi16(v_sum, v_zeros); + + /* Extract new checksum value. */ + new_checksum = _mm256_extract_epi16(v_sum, 0); + + return ~new_checksum; +} + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_src) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_src) == + offsetof(struct ovs_key_ipv4, ipv4_dst)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_dst) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_dst) == + offsetof(struct ovs_key_ipv4, ipv4_proto)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_proto) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_proto) == + offsetof(struct ovs_key_ipv4, ipv4_tos)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_tos) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_tos) == + offsetof(struct ovs_key_ipv4, ipv4_ttl)); + +static void +action_avx512_ipv4_set_addrs(void *dp OVS_UNUSED, + struct dp_packet_batch *batch, + const struct nlattr *a, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + const struct ovs_key_ipv4 *key = nl_attr_get(a); + const struct ovs_key_ipv4 *mask = get_mask(a, struct ovs_key_ipv4); + struct dp_packet *packet; + ovs_be16 old_csum; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct ip_header *nh = dp_packet_l3(packet); + old_csum = nh->ip_csum; + + __m256i v_key = _mm256_loadu_si256((void *) key); + __m256i v_mask = _mm256_loadu_si256((void *) mask); + __m256i v_packet = _mm256_loadu_si256((void *) nh); + + /* Shuffle key and mask to match ip_header struct layout. */ + static const uint8_t ip_shuffle_mask[32] = { + 0xFF, 5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 6, 0xFF, 0xFF, 0xFF, 0, 1, 2, 3, + 0, 1, 2, 3, 0xFF, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + __m256i v_shuf32 = _mm256_setr_epi32(0x0, 0x2, 0xF, 0xF, + 0x1, 0xF, 0xF, 0xF); + + __m256i v_shuffle = _mm256_loadu_si256((void *) ip_shuffle_mask); + + __m256i v_key_shuf = _mm256_permutexvar_epi32(v_shuf32, v_key); + v_key_shuf = _mm256_shuffle_epi8(v_key_shuf, v_shuffle); + + __m256i v_mask_shuf = _mm256_permutexvar_epi32(v_shuf32, v_mask); + v_mask_shuf = _mm256_shuffle_epi8(v_mask_shuf, v_shuffle); + + __m256i v_pkt_masked = _mm256_andnot_si256(v_mask_shuf, v_packet); + __m256i v_res = _mm256_or_si256(v_key_shuf, v_pkt_masked); + + /* Update checksum. */ + uint16_t checksum = avx512_ipv4_recalc_csum(v_res); + + /* Insert new checksum. */ + v_res = _mm256_insert_epi16(v_res, checksum, 5); + + /* If ip_src or ip_dst has been modified, L4 checksum needs to + * be updated too. + */ + int update_mask = _mm256_movemask_epi8(v_mask); + if (update_mask & 0xFF) { + + uint16_t tmp_checksum = avx512_l4_update_csum(nh, v_res); + tmp_checksum = ~tmp_checksum; + uint16_t csum; + + if (nh->ip_proto == IPPROTO_UDP) { + /* New UDP checksum. */ + struct udp_header *uh = dp_packet_l4(packet); + if (uh->udp_csum) { + uint16_t old_udp_checksum = ~uh->udp_csum; + + uint32_t udp_checksum = old_csum + tmp_checksum; + udp_checksum = csum_finish(udp_checksum); + uint16_t udp_csum = ~udp_checksum; + + uint32_t nw_udp_checksum = udp_csum + old_udp_checksum; + + csum = csum_finish(nw_udp_checksum); + + /* Insert new udp checksum. */ + v_res = _mm256_insert_epi16(v_res, csum, 13); + } + } + if (nh->ip_proto == IPPROTO_TCP) { + /* New TCP checksum. */ + struct tcp_header *th = dp_packet_l4(packet); + uint16_t old_tcp_checksum = ~th->tcp_csum; + + uint32_t tcp_checksum = old_csum + tmp_checksum; + tcp_checksum = csum_finish(tcp_checksum); + uint16_t tcp_csum = ~tcp_checksum; + + uint32_t nw_tcp_checksum = tcp_csum + old_tcp_checksum; + + csum = csum_finish(nw_tcp_checksum); + + th->tcp_csum = csum; + } + } + + /* Store new IP header. */ + _mm256_storeu_si256((void *) nh, v_res); + } +} + static void action_avx512_set_masked(void *dp OVS_UNUSED, struct dp_packet_batch *batch OVS_UNUSED, @@ -233,6 +425,8 @@ action_avx512_init(struct odp_execute_action_impl *self) self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_avx512_set_masked; self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = + action_avx512_ipv4_set_addrs; active_impl = *self; return 0; diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 34f13523a..cb77bab31 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -284,6 +284,7 @@ action_autoval_init(struct odp_execute_action_impl *self) self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_autoval_generic; self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked_init; self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_autoval_generic; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_autoval_generic; active_impl = *self; return 0; diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 5c4dd8e33..cbf528f93 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -585,11 +585,6 @@ odp_execute_masked_set_action(struct dp_packet *packet, break; } - case OVS_KEY_ATTR_IPV4: - odp_set_ipv4(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv4)); - break; - case OVS_KEY_ATTR_IPV6: odp_set_ipv6(packet, nl_attr_get(a), get_mask(a, struct ovs_key_ipv6)); @@ -657,6 +652,7 @@ odp_execute_masked_set_action(struct dp_packet *packet, case OVS_KEY_ATTR_ETHERNET: case OVS_KEY_ATTR_ETHERTYPE: case OVS_KEY_ATTR_IN_PORT: + case OVS_KEY_ATTR_IPV4: case OVS_KEY_ATTR_VLAN: case OVS_KEY_ATTR_ICMP: case OVS_KEY_ATTR_ICMPV6: @@ -892,6 +888,20 @@ action_mod_eth(void *dp OVS_UNUSED, struct dp_packet_batch *batch, } } +static void +action_mod_ipv4(void *dp OVS_UNUSED, struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED, + bool should_steal OVS_UNUSED) +{ + a = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_set_ipv4(packet, nl_attr_get(a), + get_mask(a, struct ovs_key_ipv4)); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -902,6 +912,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked; self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_mod_eth; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_mod_ipv4; actions_active_impl = *self; return 0;