From patchwork Tue Jun 14 11:57:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643260 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=UQx7SKTU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn554JCpz9sGG for ; Tue, 14 Jun 2022 21:58:17 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 26EDC82ED2; Tue, 14 Jun 2022 11:58:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GbLAUA_jW9qo; Tue, 14 Jun 2022 11:58:15 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4E4C282F03; Tue, 14 Jun 2022 11:58:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 22E2FC0081; Tue, 14 Jun 2022 11:58:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1AEC7C0032 for ; Tue, 14 Jun 2022 11:58:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 188E960B18 for ; Tue, 14 Jun 2022 11:58:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 84iZK9I-8yqU for ; Tue, 14 Jun 2022 11:58:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2E48160C33 for ; Tue, 14 Jun 2022 11:58:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207890; x=1686743890; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pj8H8rd6cK37ZcXLfA+YkEtDaDCqUhRDTTvJtf8pnbg=; b=UQx7SKTUDsYyFPHarlzsWaNV0dSbynEgOG/VEzNVMtfG1KiWWXIYk2ky oB/BkRx6YPiI3SbclO6hcJurFX1tzF5GrMK8rw/nHdVJiSQ83o3eMZdCL Tzvsx1rtxOTWVOd7MTWK6nggplOI3q+gvbCAefvfjE4vIjcIn51E7S/rS UwckutS1N9i5Zfc1BXbcT6voxw0u6RZkslnTYQjlbeFvlEwGxMOVyygyp +LvxjlJ1sqFFARyUNamVHEqDqb/PcMMbBL69KdnFBQy7VfGfjcYOfXEyc /jBeQdn12CMXvoRbJmmml7G157gfO2nJRi3SWDDDbJqx7heQZDqnEe+wS w==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="279632289" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="279632289" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665149" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:08 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:33 +0000 Message-Id: <20220614115743.1143341-2-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 01/11] ofproto-dpif: Fix incorrect checksums in input packets X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The IP checksum field was invalid in the input packets for some unit tests. The unit tests will still pass without a valid checksum, however we should still fix these. Signed-off-by: Emma Finn Acked-by: Eelco Chaudron --- This was found using the autovalidator introduced later in this series. --- --- tests/ofproto-dpif.at | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at index dbb3b6dda..935ae80e0 100644 --- a/tests/ofproto-dpif.at +++ b/tests/ofproto-dpif.at @@ -2009,7 +2009,7 @@ dnl Checksum UDP. AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 ; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 1C 00 00 00 00 00 11 00 00 C0 A8 00 01 C0 A8 00 02 00 08 00 0B 00 00 12 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 1C 00 00 00 00 00 11 39 7E C0 A8 00 01 C0 A8 00 02 00 08 00 0B 00 00 12 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 18]) OVS_APP_EXIT_AND_WAIT([ovs-ofctl]) @@ -2079,7 +2079,7 @@ dnl Checksum SCTP. AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 ; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 24 00 00 00 00 00 84 00 00 C0 A8 00 01 C0 A8 00 02 04 58 08 af 00 00 00 00 d9 d7 91 57 01 00 00 34 cf 28 ec 4e 00 01 40 00 00 0a ff ff b7 53 24 19 00 05 00 08 7f 00 00 01 00 05 00 08 c0 a8 02 07 00 0c 00 06 00 05 00 00 80 00 00 04 c0 00 00 04' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 20 22 22 22 22 22 08 00 45 00 00 24 00 00 00 00 00 84 39 03 C0 A8 00 01 C0 A8 00 02 04 58 08 af 00 00 00 00 d9 d7 91 57 01 00 00 34 cf 28 ec 4e 00 01 40 00 00 0a ff ff b7 53 24 19 00 05 00 08 7f 00 00 01 00 05 00 08 c0 a8 02 07 00 0c 00 06 00 05 00 00 80 00 00 04 c0 00 00 04' done AT_CHECK([ovs-appctl time/warp 1000], [0], [ignore]) @@ -2951,7 +2951,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x7744 (correct), AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 01 88 48 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3b 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 01 88 48 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) @@ -2979,7 +2979,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x7744 (correct), AT_CHECK([ovs-ofctl monitor br0 65534 -P nxt_packet_in --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 10 88 47 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3b 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 07 60 66 66 66 02 10 88 47 00 01 40 20 00 01 41 1f 45 00 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) OVS_APP_EXIT_AND_WAIT(ovs-ofctl) @@ -3356,7 +3356,7 @@ dnl 192.168.0.1.80 > 192.168.0.2.0: Flags [none], cksum 0x77ec (correct) AT_CHECK([ovs-ofctl -O OpenFlow12 monitor br0 65534 -P standard --detach --no-chdir --pidfile 2> ofctl_monitor.log]) for i in 1 2 3; do - ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 01 60 66 66 66 00 08 88 48 00 01 41 20 45 20 00 2c 00 00 00 00 ff 06 3a 78 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' + ovs-appctl netdev-dummy/receive p1 '50 54 00 00 00 01 60 66 66 66 00 08 88 48 00 01 41 20 45 20 00 2c 00 00 00 00 ff 06 3a 58 c0 a8 00 01 c0 a8 00 02 00 50 00 00 00 00 00 2a 00 00 00 2a 50 00 27 10 77 44 00 00 48 4f 47 45' done OVS_WAIT_UNTIL([test `wc -l < ofctl_monitor.log` -ge 6]) OVS_APP_EXIT_AND_WAIT(ovs-ofctl) From patchwork Tue Jun 14 11:57:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643261 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=AmfYaeI2; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn670NFBz9sGG for ; Tue, 14 Jun 2022 21:59:11 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9619D60A98; Tue, 14 Jun 2022 11:59:09 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y4M9Z5JKVffN; Tue, 14 Jun 2022 11:59:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7EAAD60F4B; Tue, 14 Jun 2022 11:59:07 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4E3B6C0032; Tue, 14 Jun 2022 11:59:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id B14F9C002D for ; Tue, 14 Jun 2022 11:59:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A9B6760F4B for ; Tue, 14 Jun 2022 11:58:54 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5S6jhilV4eAq for ; Tue, 14 Jun 2022 11:58:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 852C86002E for ; Tue, 14 Jun 2022 11:58:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207933; x=1686743933; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1+AY7I2tCJtRTuxtAV/Ld1W5LgyVBSFw909ek4J2o00=; b=AmfYaeI272JB0menRxxK8QajAoI7yjp2aZm0sc9iML8/PzWclWL/cES3 Im+6CE9w7wtJPFU0dxY8D6AGQgSYKZ51pUBz0XiIOVzTfvpGTswWxbE6s u2W4da/2QX8LPm+/yyP8sPt0K1Xtr24Y11dj9rSUDktoHNVTcMHrJsB67 6iE6Tv/IU/68SaWSBFdS7ouoxhCyYYGE722ye3eevn7FybMm9F3m7vW08 ri5XCksjTmxpQXvmmlJ7ttio5ki7ZIdQWAS0zKb3UH1sybMo/X9FutRz6 75g2NsY4z0xOaW06b7yoZUHfQVLZeRDbV/IC8/hPI5B5ivBg93hpKth6v g==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137253" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137253" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665669" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:51 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:34 +0000 Message-Id: <20220614115743.1143341-3-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 02/11] odp-execute: Add function pointers to odp-execute for different action implementations. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduces the initial infrastructure required to allow different implementations for OvS actions. The patch introduces action function pointers which allows user to switch between different action implementations available. This will allow for more performance and flexibility so the user can choose the action implementation to best suite their use case. Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- lib/automake.mk | 2 + lib/dpif-netdev.c | 4 ++ lib/odp-execute-private.c | 80 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 74 ++++++++++++++++++++++++++++++++++++ lib/odp-execute.c | 41 +++++++++++++++++--- lib/odp-execute.h | 7 ++++ 6 files changed, 203 insertions(+), 5 deletions(-) create mode 100644 lib/odp-execute-private.c create mode 100644 lib/odp-execute-private.h diff --git a/lib/automake.mk b/lib/automake.mk index cb50578eb..1a49dd30b 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -213,6 +213,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/object-collection.h \ lib/odp-execute.c \ lib/odp-execute.h \ + lib/odp-execute-private.c \ + lib/odp-execute-private.h \ lib/odp-util.c \ lib/odp-util.h \ lib/ofp-actions.c \ diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index ff57b3961..47dd7a1a6 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -1675,6 +1675,10 @@ create_dpif_netdev(struct dp_netdev *dp) dpif->dp = dp; dpif->last_port_seq = seq_read(dp->port_seq); + /* Called once at initialization time. This handles setting up the state + * of the actions functions at init time. */ + odp_execute_init(); + return &dpif->dpif; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c new file mode 100644 index 000000000..92db2386c --- /dev/null +++ b/lib/odp-execute-private.c @@ -0,0 +1,80 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include + +#include "dpdk.h" +#include "dp-packet.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "odp-util.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(odp_execute_impl); + +static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_SCALAR] = { + .available = false, + .name = "scalar", + .init_func = NULL, + }, +}; + +static void +action_impl_copy_funcs(struct odp_execute_action_impl *src, + const struct odp_execute_action_impl *dst) +{ + for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + atomic_store_relaxed(&src->funcs[i], dst->funcs[i]); + } +} + +void +odp_execute_action_init(void) +{ + /* Each impl's function array is initialized to reflect the scalar + * implementation. This simplifies adding optimized implementations, + * as the autovalidator can always compare all actions. + * + * Below will check if impl is available and copies the scalar functions + * to all other implementations. + */ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + bool avail = true; + + if (action_impls[i].init_func) { + /* Return zero is success, non-zero means error. */ + avail = (action_impls[i].init_func(&action_impls[i]) == 0); + } + + action_impls[i].available = avail; + + if (i != ACTION_IMPL_SCALAR) { + action_impl_copy_funcs(&action_impls[i], + &action_impls[ACTION_IMPL_SCALAR]); + } + + if (action_impls[i].available == true) { + action_impls[i].init_func(&action_impls[i]); + } + + VLOG_INFO("Action implementation %s (available: %s)\n", + action_impls[i].name, avail ? "available" : "not available"); + } +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h new file mode 100644 index 000000000..08d1faf38 --- /dev/null +++ b/lib/odp-execute-private.h @@ -0,0 +1,74 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef ODP_EXTRACT_PRIVATE +#define ODP_EXTRACT_PRIVATE 1 + +#include "dp-packet.h" +#include "odp-execute.h" +#include "odp-netlink.h" +#include "ovs-atomic.h" + +/* Forward declaration for typedef. */ +struct odp_execute_action_impl; + +/* Typedef for an initialization function that can initialize each + * implementation, checking requirements such as CPU ISA. + */ +typedef int (*odp_execute_action_init_func) + (struct odp_execute_action_impl *self); + +/* Structure represents an implementation of the odp actions. */ +struct odp_execute_action_impl { + /* When set, the CPU ISA required for this implementation is available + * and the implementation can be used. + */ + bool available; + + /* Name of the implementation. */ + const char *name; + + /* Function is used to detect if this CPU has the ISA required + * to run the optimized action implementation and if available, initializes + * the implementation for use. + */ + odp_execute_action_init_func init_func; + + /* An array of callback functions, one for each action. */ + ATOMIC(odp_execute_action_cb) funcs[__OVS_ACTION_ATTR_MAX]; +}; + +/* Order of Actions implementations. */ +enum odp_execute_action_impl_idx { + ACTION_IMPL_SCALAR, + /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. + * Do not change the autovalidator position in this list without updating + * the define below. + */ + + ACTION_IMPL_MAX, +}; + +/* Index to start verifying implementations from. */ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); + +/* Odp execute init handles setting up the state of the actions functions at + * initialization time. It cannot return errors, as it must always succeed in + * initializing the scalar/generic codepath. + */ +void odp_execute_action_init(void); + +#endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 7da56793d..67ebbe195 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -17,6 +17,7 @@ #include #include "odp-execute.h" +#include "odp-execute-private.h" #include #include #include @@ -833,6 +834,23 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. + */ +static struct odp_execute_action_impl actions_active_impl; + +void +odp_execute_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + if (ovsthread_once_start(&once)) { + odp_execute_action_init(); + ovsthread_once_done(&once); + } +} + + /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and * definitely free the packets in 'batch', otherwise leaves 'batch' unchanged. @@ -857,14 +875,12 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, NL_ATTR_FOR_EACH_UNSAFE (a, left, actions, actions_len) { int type = nl_attr_type(a); + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; bool last_action = (left <= NLA_ALIGN(a->nla_len)); + bool should_steal = steal && last_action; if (requires_datapath_assistance(a)) { if (dp_execute_action) { - /* Allow 'dp_execute_action' to steal the packet data if we do - * not need it any more. */ - bool should_steal = steal && last_action; - dp_execute_action(dp, batch, a, should_steal); if (last_action || dp_packet_batch_is_empty(batch)) { @@ -879,8 +895,20 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, continue; } - switch ((enum ovs_action_attr) type) { + /* If type is set in the active actions implementation, call the + * function-pointer and continue to the next action. + */ + if (actions_active_impl.funcs[attr_type] && + attr_type <= OVS_ACTION_ATTR_MAX) { + actions_active_impl.funcs[attr_type](batch, a); + continue; + } + + /* If the action was not handled by the active function pointers above, + * process them by switching on the type below. + */ + switch (attr_type) { case OVS_ACTION_ATTR_HASH: { const struct ovs_action_hash *hash_act = nl_attr_get(a); @@ -1094,6 +1122,9 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: OVS_NOT_REACHED(); } + + /* Do not add any generic processing here, as it won't be executed when + * an ISA-specific action implementation exists. */ } dp_packet_delete_batch(batch, steal); diff --git a/lib/odp-execute.h b/lib/odp-execute.h index a3578a575..0921ee924 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -28,6 +28,13 @@ struct dp_packet; struct pkt_metadata; struct dp_packet_batch; + +/* Called once at initialization time. */ +void odp_execute_init(void); + +typedef void (*odp_execute_action_cb)(struct dp_packet_batch *batch, + const struct nlattr *action); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); From patchwork Tue Jun 14 11:57:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643262 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=buYzRSW4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6B0Zr3z9sGG for ; Tue, 14 Jun 2022 21:59:14 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2522460F4B; Tue, 14 Jun 2022 11:59:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sNlmY5S1vEaV; Tue, 14 Jun 2022 11:59:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1B0DE60F6D; Tue, 14 Jun 2022 11:59:10 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 23298C0083; Tue, 14 Jun 2022 11:59:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2F437C0032 for ; Tue, 14 Jun 2022 11:59:06 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 72A2160035 for ; Tue, 14 Jun 2022 11:58:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OxBuJC9cJPhO for ; Tue, 14 Jun 2022 11:58:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id B51136002E for ; Tue, 14 Jun 2022 11:58:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207934; x=1686743934; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lOhGyJwI2De5TQI6L5whqIN07FtFbQY7VGBWPgH7+ds=; b=buYzRSW4KLSTvey5sZOdGcWoQ68aaap7hx7uFt9QDYfzPYhOCN3/MrCg lk8Ifyoukj8NiUtG9wyQZBZ+Yv+tAFcBHnL/Vgx8/kAHba19mpyjvBUA4 ipgUM+YzFlgmPDffuFpS+lM9mp+QZuVbUa516petvr5HSR/Nr7fhGbkAl 8NSe1CPgZQBitmI/Mzg/HyjBHNOSapQV7JFuy5hPUFrcHp1RYyzEUrJ3K Cvs4Wl4Cacc6K+8E8gV9dxCteif6Lhh87hZJwGLtRWQq/v+Gwj0K0qaVS dshsOlCuo5RRm7bJ7FJmzLVYJHnukFuQ+rXTlMQWRIM+p3JZg16mfqrnC g==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137263" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137263" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665691" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:52 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:35 +0000 Message-Id: <20220614115743.1143341-4-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 03/11] odp-execute: Add function pointer for pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit removes the pop_vlan action from the large switch and creates a separate function for batched processing. A function pointer is also added to call the new batched function for the pop_vlan action. Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- lib/odp-execute-private.c | 18 +++++++++++++++- lib/odp-execute-private.h | 8 +++++++ lib/odp-execute.c | 44 +++++++++++++++++++++++++++++++++------ lib/odp-execute.h | 2 ++ 4 files changed, 65 insertions(+), 7 deletions(-) diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 92db2386c..25dbbfefc 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -27,12 +27,13 @@ #include "openvswitch/vlog.h" VLOG_DEFINE_THIS_MODULE(odp_execute_impl); +static int active_action_impl_index; static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", - .init_func = NULL, + .init_func = odp_action_scalar_init, }, }; @@ -45,6 +46,21 @@ action_impl_copy_funcs(struct odp_execute_action_impl *src, } } +int +odp_execute_action_set(const char *name, + struct odp_execute_action_impl *active) +{ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + /* String compare, and set ptrs atomically. */ + if (!strcmp(action_impls[i].name, name)) { + action_impl_copy_funcs(active, &action_impls[i]);; + active_action_impl_index = i; + return 0; + } + } + return -EINVAL; +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 08d1faf38..c5ab00b07 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -71,4 +71,12 @@ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); */ void odp_execute_action_init(void); +/* Init functions for the action implementations. Initializes the function + * pointers for optimized action types. + */ +int odp_action_scalar_init(struct odp_execute_action_impl *self); + +int odp_execute_action_set(const char *name, + struct odp_execute_action_impl *active); + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 67ebbe195..eff80d93f 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -834,6 +834,30 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +static void +action_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_pop_vlan(packet); + } +} + +/* Implementation of the scalar actions impl init function. Build up the + * array of func ptrs here. + */ +int +odp_action_scalar_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + + return 0; +} + /* The active function pointers on the datapath. ISA optimized implementations * are enabled by plugging them into this static arary, which is consulted when * applying actions on the datapath. @@ -846,10 +870,22 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); + odp_actions_impl_set("scalar"); ovsthread_once_done(&once); } } +int +odp_actions_impl_set(const char *name) +{ + + int err = odp_execute_action_set(name, &actions_active_impl); + if (err) { + VLOG_ERR("Failed setting action implementation to %s, error %d", + name, err); + } + return err; +} /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and @@ -964,12 +1000,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_POP_VLAN: - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_pop_vlan(packet); - } - break; - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1120,6 +1150,8 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case OVS_ACTION_ATTR_CT: case OVS_ACTION_ATTR_UNSPEC: case __OVS_ACTION_ATTR_MAX: + /* The following actions are handled by the scalar implementation. */ + case OVS_ACTION_ATTR_POP_VLAN: OVS_NOT_REACHED(); } diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 0921ee924..50d47b716 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -35,6 +35,8 @@ void odp_execute_init(void); typedef void (*odp_execute_action_cb)(struct dp_packet_batch *batch, const struct nlattr *action); +int odp_actions_impl_set(const char *name); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); From patchwork Tue Jun 14 11:57:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643263 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=E+YtDUx/; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6C6S43z9sGG for ; Tue, 14 Jun 2022 21:59:15 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5327141883; Tue, 14 Jun 2022 11:59:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P2ElFLY8tZEX; Tue, 14 Jun 2022 11:59:12 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8213141857; Tue, 14 Jun 2022 11:59:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BC72FC0087; Tue, 14 Jun 2022 11:59:08 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5C997C007A for ; Tue, 14 Jun 2022 11:59:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 56B3360F5D for ; Tue, 14 Jun 2022 11:58:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WwY9ehoNNbb0 for ; Tue, 14 Jun 2022 11:58:56 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8595C6002E for ; Tue, 14 Jun 2022 11:58:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207936; x=1686743936; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YtYf1XhgAyvAxwMYQTjCJHcyBUnPqNdpkPRKTh+QNpc=; b=E+YtDUx/wI9H7zirEd3SEAvBv8bc4oHWij6o9/46PPjAJI8TgeK5GH15 JmLwKoRp4eL0Ia+qjt5Y2TB3hPiGL0OXShqqnBmVOWCwUWhrTFCcj/q24 JYsCwBGInMmeJ46Zg7J7b7L88PvtsKkpvt34vYPAGLRi9pBYNo/EtRbNP vCMDbGvaU/Fi/+9rg9+DpaaiTfQBR5Hfl9G4V49I9qexBaqdjJD5u7brH Fi4ud6rbD/p9e4LEz10g7D0GNJNkgk/Dl1R7/+Rqy43LTEoiPKBmeX3VT C/qgR0AiAM/NQdFiR23q7FAZVCJ3wmpgTI4OGGLue2axy5fTITXBsbUvO Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137268" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137268" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665721" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:54 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:36 +0000 Message-Id: <20220614115743.1143341-5-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 04/11] odp-execute: Add auto validation function for actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduced the auto-validation function which allows users to compare the batch of packets obtained from different action implementations against the linear action implementation. The autovalidator function can be triggered at runtime using the following command: $ ovs-appctl dpif-netdev/action-impl-set autovalidator Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- NEWS | 3 ++ lib/dp-packet.c | 23 +++++++++ lib/dp-packet.h | 4 ++ lib/odp-execute-private.c | 98 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 6 +++ 5 files changed, 134 insertions(+) diff --git a/NEWS b/NEWS index 9fe3f44f4..3a25f3035 100644 --- a/NEWS +++ b/NEWS @@ -32,6 +32,9 @@ Post-v2.17.0 - DPDK: * OVS validated with DPDK 21.11.1. It is recommended to use this version until further releases. + - Userspace datapath: + * Add actions auto-validator function to compare different actions + implementations against default implementation. v2.17.0 - 17 Feb 2022 diff --git a/lib/dp-packet.c b/lib/dp-packet.c index 35c72542a..237dcf19e 100644 --- a/lib/dp-packet.c +++ b/lib/dp-packet.c @@ -506,3 +506,26 @@ dp_packet_resize_l2(struct dp_packet *b, int increment) dp_packet_adjust_layer_offset(&b->l2_5_ofs, increment); return dp_packet_data(b); } + +bool +dp_packet_compare_offsets(struct dp_packet *b1, struct dp_packet *b2, + struct ds *err_str) +{ + if ((b1->l2_pad_size != b2->l2_pad_size) || + (b1->l2_5_ofs != b2->l2_5_ofs) || + (b1->l3_ofs != b2->l3_ofs) || + (b1->l4_ofs != b2->l4_ofs)) { + ds_put_format(err_str, "Packet offset comparison failed" + "\n"); + ds_put_format(err_str, "Buffer 1 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b1->l2_pad_size, b1->l2_5_ofs, + b1->l3_ofs, b1->l4_ofs); + ds_put_format(err_str, "Buffer 2 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b2->l2_pad_size, b2->l2_5_ofs, + b2->l3_ofs, b2->l4_ofs); + return false; + } + return true; +} diff --git a/lib/dp-packet.h b/lib/dp-packet.h index bddaa2b5d..1776c3bfe 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -236,6 +236,10 @@ void *dp_packet_steal_data(struct dp_packet *); static inline bool dp_packet_equal(const struct dp_packet *, const struct dp_packet *); +bool dp_packet_compare_offsets(struct dp_packet *good, + struct dp_packet *test, + struct ds *err_str); + /* Frees memory that 'b' points to, as well as 'b' itself. */ static inline void diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 25dbbfefc..267f32c3e 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -28,8 +28,15 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; +static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_AUTOVALIDATOR] = { + .available = false, + .name = "autovalidator", + .init_func = action_autoval_init, + }, + [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", @@ -94,3 +101,94 @@ odp_execute_action_init(void) action_impls[i].name, avail ? "available" : "not available"); } } + +/* Init sequence required to be scalar first to pick up the default scalar +* implementations, allowing over-riding of the optimized functions later. +*/ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +/* Loop over packets, and validate each one for the given action. */ +static void +action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) +{ + bool failed = false; + int type = nl_attr_type(a); + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; + struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; + struct dp_packet_batch good_batch; + + dp_packet_batch_clone(&good_batch, batch); + + scalar->funcs[attr_type](&good_batch, a); + + for (int impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { + /* Clone original batch and execute implementation under test. */ + struct dp_packet_batch test_batch; + + dp_packet_batch_clone(&test_batch, batch); + action_impls[impl].funcs[attr_type](&test_batch, a); + + /* Loop over implementations, checking each one. */ + for (int pidx = 0; pidx < batch->count; pidx++) { + struct dp_packet *good_pkt = good_batch.packets[pidx]; + struct dp_packet *test_pkt = test_batch.packets[pidx]; + + struct ds log_msg = DS_EMPTY_INITIALIZER; + + /* Compare packet length and payload contents. */ + bool eq = dp_packet_equal(good_pkt, test_pkt); + + if (!eq) { + ds_put_format(&log_msg, "Packet: %d\nAction : ", pidx); + format_odp_actions(&log_msg, a, a->nla_len, NULL); + ds_put_format(&log_msg, "\nGood hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(good_pkt), + dp_packet_size(good_pkt), 0, false); + ds_put_format(&log_msg, "Test hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(test_pkt), + dp_packet_size(test_pkt), 0, false); + + failed = true; + } + + /* Compare offsets and RSS */ + if (!dp_packet_compare_offsets(good_pkt, test_pkt, &log_msg)) { + failed = true; + } + + uint32_t good_hash = dp_packet_get_rss_hash(good_pkt); + uint32_t test_hash = dp_packet_get_rss_hash(test_pkt); + + if (good_hash != test_hash) { + ds_put_format(&log_msg, "Autovalidation rss hash failed\n"); + ds_put_format(&log_msg, "Good RSS hash : %u\n", good_hash); + ds_put_format(&log_msg, "Test RSS hash : %u\n", test_hash); + + failed = true; + } + + if (failed) { + VLOG_ERR_RL(&rl, "Autovalidation of %s failed. Details:\n%s", + action_impls[impl].name, ds_cstr(&log_msg)); + ds_destroy(&log_msg); + failed = false; + } + } + dp_packet_delete_batch(&test_batch, 1); + } + dp_packet_delete_batch(&good_batch, 1); + + /* Apply the action to the original batch for continued processing. */ + scalar->funcs[attr_type](batch, a); +} + +int +action_autoval_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; + + return 0; +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index c5ab00b07..d3dc669d1 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -54,6 +54,7 @@ struct odp_execute_action_impl { /* Order of Actions implementations. */ enum odp_execute_action_impl_idx { ACTION_IMPL_SCALAR, + ACTION_IMPL_AUTOVALIDATOR, /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. * Do not change the autovalidator position in this list without updating * the define below. @@ -64,6 +65,9 @@ enum odp_execute_action_impl_idx { /* Index to start verifying implementations from. */ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +#define ACTION_IMPL_BEGIN (ACTION_IMPL_AUTOVALIDATOR + 1) /* Odp execute init handles setting up the state of the actions functions at * initialization time. It cannot return errors, as it must always succeed in @@ -76,6 +80,8 @@ void odp_execute_action_init(void); */ int odp_action_scalar_init(struct odp_execute_action_impl *self); +int action_autoval_init(struct odp_execute_action_impl *self); + int odp_execute_action_set(const char *name, struct odp_execute_action_impl *active); From patchwork Tue Jun 14 11:57:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643264 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=iQocK/I8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6G1jfnz9sGG for ; Tue, 14 Jun 2022 21:59:18 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 4BA7660FD0; Tue, 14 Jun 2022 11:59:16 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FtCbAo_i319P; Tue, 14 Jun 2022 11:59:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2449E60F9B; Tue, 14 Jun 2022 11:59:13 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BEED4C0081; Tue, 14 Jun 2022 11:59:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8A935C0032 for ; Tue, 14 Jun 2022 11:59:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3FE7160B18 for ; Tue, 14 Jun 2022 11:58:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JkzmIv389E_C for ; Tue, 14 Jun 2022 11:58:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 58C946002E for ; Tue, 14 Jun 2022 11:58:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207938; x=1686743938; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NiWcoNbfv8TWchEUNWZXLKkSlqMGZOfHLucm40RbNHg=; b=iQocK/I8z4ThDumhS/oSrjYwzDe3RvlEzf0wbBJWM/2E/aud8N8q+x0A BNHasXrMTix/5uA4TBQtxkFdVa8GLTW59c0O5fwByq1UPccUddNs7Uukw kf2auigqnDv0AH6VZkDwBrLe/lhIqHsK2t5waizpCCfCfKdNb6K4xruzA m9/zuVLTyxxCeA3N28nTx6EpVy0xKrR/mByFs9AbokHmjG1HfpEdoCbfz es0AJBmuWkDjvw9FBhv3QXs+lJqAxwDpDa1VTHbsIRCKTY1+n8iSbpG8z RsdE41r/GBE/07WgngxZQ9yANP70cG6or3fPpVlfug3cN0J78OBAoHzBF Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137276" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137276" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665753" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:56 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:37 +0000 Message-Id: <20220614115743.1143341-6-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: Kumar Amber Subject: [ovs-dev] [PATCH v7 05/11] odp-execute: Add command to switch action implementation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds a new command to allow the user to switch the active action implementation at runtime. A probe function is executed before switching the implementation, to ensure the CPU is capable of running the ISA required. Usage: $ ovs-appctl dpif-netdev/action-impl-set scalar This commit also adds a new command to retrieve the list of available action implementations. This can be used by to check what implementations of actions are available and what implementation is active during runtime. Usage: $ ovs-appctl dpif-netdev/action-impl-show Added separate test-case for ovs-actions show/set commands: 1023: PMD - ovs-actions configuration Signed-off-by: Emma Finn Co-authored-by: Kumar Amber Signed-off-by: Kumar Amber Acked-by: Harry van Haaren --- NEWS | 3 +++ lib/dpif-netdev-unixctl.man | 8 ++++++++ lib/dpif-netdev.c | 38 +++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 12 ++++++++++++ lib/odp-execute-private.h | 3 +++ lib/odp-execute.h | 2 ++ tests/pmd.at | 30 +++++++++++++++++++++++++++++ 7 files changed, 96 insertions(+) diff --git a/NEWS b/NEWS index 3a25f3035..90ceabd63 100644 --- a/NEWS +++ b/NEWS @@ -35,6 +35,9 @@ Post-v2.17.0 - Userspace datapath: * Add actions auto-validator function to compare different actions implementations against default implementation. + * Add command line option to switch between different actions + implementations available at run time. + v2.17.0 - 17 Feb 2022 diff --git a/lib/dpif-netdev-unixctl.man b/lib/dpif-netdev-unixctl.man index 8cd847416..81ef7d856 100644 --- a/lib/dpif-netdev-unixctl.man +++ b/lib/dpif-netdev-unixctl.man @@ -262,3 +262,11 @@ PMDs in the case where no value is specified. By default "scalar" is used. \fIstudy_cnt\fR defaults to 128 and indicates the number of packets that the "study" miniflow implementation must parse before choosing an optimal implementation. + +.IP "\fBdpif-netdev/action-impl-show\fR +Lists the actions implementations that are available and highlights the +currently enabled one. +. +.IP "\fBdpif-netdev/action-impl-set\fR \fIaction_impl\fR" +Sets the action implementation to any available implementation. By default +"scalar" is used. diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 47dd7a1a6..5a35c7ce5 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -63,6 +63,7 @@ #include "netdev-vport.h" #include "netlink.h" #include "odp-execute.h" +#include "odp-execute-private.h" #include "odp-util.h" #include "openvswitch/dynamic-string.h" #include "openvswitch/list.h" @@ -1387,6 +1388,37 @@ error: ds_destroy(&reply); } +static void +action_impl_set(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + int err = odp_actions_impl_set(argv[1]); + if (err) { + ds_put_format(&reply, + "Error: unknown action implementation, %s, specified!\n", + argv[1]); + unixctl_command_reply_error(conn, ds_cstr(&reply)); + } else { + ds_put_format(&reply, "Action implementation set to %s.\n", argv[1]); + unixctl_command_reply(conn, ds_cstr(&reply)); + } + + ds_destroy(&reply); +} + +static void +action_impl_show(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + odp_execute_action_get_info(&reply); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + static void dpif_netdev_pmd_rebalance(struct unixctl_conn *conn, int argc, const char *argv[], void *aux OVS_UNUSED) @@ -1624,6 +1656,12 @@ dpif_netdev_init(void) unixctl_command_register("dpif-netdev/miniflow-parser-get", "", 0, 0, dpif_miniflow_extract_impl_get, NULL); + unixctl_command_register("dpif-netdev/action-impl-set", "name", + 1, 1, action_impl_set, + NULL); + unixctl_command_register("dpif-netdev/action-impl-show", "", + 0, 0, action_impl_show, + NULL); return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 267f32c3e..f8d0896b5 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -68,6 +68,18 @@ odp_execute_action_set(const char *name, return -EINVAL; } +void +odp_execute_action_get_info(struct ds *string) +{ + ds_put_cstr(string, "Available Actions implementations:\n"); + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + ds_put_format(string, " %s (available: %s, active: %s)\n", + action_impls[i].name, + action_impls[i].available ? "Yes" : "No", + i == active_action_impl_index ? "Yes" : "No"); + } +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index d3dc669d1..5322eb8df 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -85,4 +85,7 @@ int action_autoval_init(struct odp_execute_action_impl *self); int odp_execute_action_set(const char *name, struct odp_execute_action_impl *active); +void odp_execute_action_get_info(struct ds *name); + + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 50d47b716..8668ab73f 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -23,6 +23,7 @@ #include #include "openvswitch/types.h" +struct ds; struct nlattr; struct dp_packet; struct pkt_metadata; @@ -36,6 +37,7 @@ typedef void (*odp_execute_action_cb)(struct dp_packet_batch *batch, const struct nlattr *action); int odp_actions_impl_set(const char *name); +int odp_actions_impl_get(struct ds *name); typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); diff --git a/tests/pmd.at b/tests/pmd.at index e6b173dab..ac05f5f7d 100644 --- a/tests/pmd.at +++ b/tests/pmd.at @@ -1200,3 +1200,33 @@ ovs-appctl: ovs-vswitchd: server returned an error OVS_VSWITCHD_STOP AT_CLEANUP + +AT_SETUP([PMD - ovs-actions configuration]) +OVS_VSWITCHD_START([], [], [], [--dummy-numa 0,0]) +AT_CHECK([ovs-vsctl add-port br0 p1 -- set Interface p1 type=dummy-pmd]) + +dnl Scalar impl is set by default. +AT_CHECK([ovs-vsctl show], [], [stdout]) +AT_CHECK([ovs-appctl dpif-netdev/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: Yes) +]) + +AT_CHECK([ovs-appctl dpif-netdev/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: No) +]) + +dnl Set the autovalidator impl to active. +AT_CHECK([ovs-appctl dpif-netdev/action-impl-set autovalidator], [0], [dnl +Action implementation set to autovalidator. +]) + +AT_CHECK([ovs-appctl dpif-netdev/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: No) +]) + +AT_CHECK([ovs-appctl dpif-netdev/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: Yes) +]) + +OVS_VSWITCHD_STOP +AT_CLEANUP From patchwork Tue Jun 14 11:57:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643265 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=PAmBTprC; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6J3MVwz9sGG for ; Tue, 14 Jun 2022 21:59:20 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D4D9D831C1; Tue, 14 Jun 2022 11:59:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 88U4egbvPjhQ; Tue, 14 Jun 2022 11:59:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id BB390831A9; Tue, 14 Jun 2022 11:59:14 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id ACCACC008B; Tue, 14 Jun 2022 11:59:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 611AAC008A for ; Tue, 14 Jun 2022 11:59:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BD23460A98 for ; Tue, 14 Jun 2022 11:59:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S5b-LlHy19Ui for ; Tue, 14 Jun 2022 11:59:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1DE866002E for ; Tue, 14 Jun 2022 11:59:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207940; x=1686743940; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DnEln5PjcCVGRwt5ign+3sz/luOIjjwBQUsfyZOy+Ic=; b=PAmBTprCKVkBS/SJVPjRxeR+Gv8cICiNjxdzfeaGkqfDCBZLiCcJYV/e unOl4pku8AQNW2tN6zgEpKwwwGho0vvK72xXuJpqPcAaoIBtjess9xEyh M5wm6hjXTZzQmvBU6s0VixPLlbEJPKlCAXmvbAtpKKke8PbhY3hob7dbe 7quFiY1BnGlTnjd+mShwDbsO9FA8CW6Dlg1YmMBU6b47khuCAWGse7xaG sNH0FFDFeH1aHSO0tagZdwWeLKcjyRctv1MJIoFYzOKxR2NPAhZj6P2x+ a5/B0TuJ3oL3/d14pAmTJGBCH7/lzOvXW90QJprUZ/Ls02XETvolzZrsf A==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137282" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137282" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:58:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665788" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:58:58 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:38 +0000 Message-Id: <20220614115743.1143341-7-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: Kumar Amber Subject: [ovs-dev] [PATCH v7 06/11] dpif-netdev: Add configure option to enable actions autovalidator at build time. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Kumar Amber This commit adds a new command to allow the user to enable the actions autovalidator by default at build time thus allowing for running unit test by default. $ ./configure --enable-actions-default-autovalidator Signed-off-by: Kumar Amber Acked-by: Harry van Haaren --- NEWS | 3 +++ acinclude.m4 | 21 +++++++++++++++++++++ configure.ac | 1 + lib/odp-execute.c | 4 ++++ 4 files changed, 29 insertions(+) diff --git a/NEWS b/NEWS index 90ceabd63..7ae4d87bb 100644 --- a/NEWS +++ b/NEWS @@ -37,6 +37,9 @@ Post-v2.17.0 implementations against default implementation. * Add command line option to switch between different actions implementations available at run time. + * Add build time configure command to enable auto-validator as default + actions implementation at build time. + diff --git a/acinclude.m4 b/acinclude.m4 index 7b2889a40..98f4599b1 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -14,6 +14,27 @@ # See the License for the specific language governing permissions and # limitations under the License. +dnl Set OVS Actions Autovalidator as the default action implementation +at compile time? +dnl This enables automatically running all unit tests with all actions +dnl implementations. +AC_DEFUN([OVS_CHECK_ACTIONS_AUTOVALIDATOR], [ + AC_ARG_ENABLE([actions-default-autovalidator], + [AC_HELP_STRING([--enable-actions-default-autovalidator], + [Enable actions autovalidator as default + ovs actions implementation.])], + [autovalidator=yes],[autovalidator=no]) + AC_MSG_CHECKING([whether actions Autovalidator is default implementation]) + if test "$autovalidator" != yes; then + AC_MSG_RESULT([no]) + else + AC_DEFINE([MFEX_AUTOVALIDATOR_DEFAULT], [1], + [Autovalidator for actions is a default implementation.]) + AC_MSG_RESULT([yes]) + fi +]) + + dnl Set OVS MFEX Autovalidator as default miniflow extract at compile time? dnl This enables automatically running all unit tests with all MFEX dnl implementations. diff --git a/configure.ac b/configure.ac index 59ea0a281..ab8e1bd12 100644 --- a/configure.ac +++ b/configure.ac @@ -184,6 +184,7 @@ OVS_CONDITIONAL_CC_OPTION([-Wno-unused-parameter], [HAVE_WNO_UNUSED_PARAMETER]) OVS_ENABLE_WERROR OVS_ENABLE_SPARSE OVS_CTAGS_IDENTIFIERS +OVS_CHECK_ACTIONS_AUTOVALIDATOR OVS_CHECK_DPCLS_AUTOVALIDATOR OVS_CHECK_DPIF_AVX512_DEFAULT OVS_CHECK_MFEX_AUTOVALIDATOR diff --git a/lib/odp-execute.c b/lib/odp-execute.c index eff80d93f..a49b331ef 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -870,7 +870,11 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); +#ifdef ACTIONS_AUTOVALIDATOR_DEFAULT + odp_actions_impl_set("autovalidator"); +#else odp_actions_impl_set("scalar"); +#endif ovsthread_once_done(&once); } } From patchwork Tue Jun 14 11:57:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643266 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=GzlapBqG; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6K3L9Dz9sGH for ; Tue, 14 Jun 2022 21:59:21 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 586FD60FFB; Tue, 14 Jun 2022 11:59:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vU1mmE3s50gr; Tue, 14 Jun 2022 11:59:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 664F560FD1; Tue, 14 Jun 2022 11:59:16 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 773D8C008E; Tue, 14 Jun 2022 11:59:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19FB6C008A for ; Tue, 14 Jun 2022 11:59:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 68B4C60B6E for ; Tue, 14 Jun 2022 11:59:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id exOY0cPwY79l for ; Tue, 14 Jun 2022 11:59:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 351AD6002E for ; Tue, 14 Jun 2022 11:59:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207942; x=1686743942; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MXJRlN+y4FmeFiZE0gVnYwejuuAiyMSov9pb9/tI9ws=; b=GzlapBqGClpgZO1anxnB3I/Cwa3NvOwO/1r8BycweNZcAX88wFVHDIz6 h1ivZ0E+RWYD1i6Li203zb7tYx/yjgqMKal1o9s76K9MMMehVpHx1RzZX y6ZWpKjRuJISrz0nrr5d/0NH+OKwcUEsfxt6dsdiah+itOas6YDfFsmzZ SjPLEVRFDZY6QubgYHQ4OLS8oEhwnY+e6W8M3F+t9WeHxpMnvfhUL1H8O bo6ouBO3t0/Ff/AvYcqf4SPoV/naNksPpm3XSkYYhXnRmaz+yWTBzVmxu R9XWtIpWFBkRPzgiKX/usQ2QjXTqMOLsGPmUUa9YRbFPKDsWwI+sa/3vz w==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137289" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137289" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:59:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665820" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:59:00 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:39 +0000 Message-Id: <20220614115743.1143341-8-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 07/11] odp-execute: Add ISA implementation of actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the action functionality. Usage: $ ovs-appctl dpif-netdev/action-impl-set avx512 Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- Documentation/ref/ovs-actions.7.rst | 26 ++++++++++++++ Documentation/topics/testing.rst | 24 ++++++++----- NEWS | 2 ++ lib/automake.mk | 6 +++- lib/cpu.c | 1 + lib/cpu.h | 1 + lib/odp-execute-avx512.c | 56 +++++++++++++++++++++++++++++ lib/odp-execute-private.c | 8 +++++ lib/odp-execute-private.h | 6 +++- 9 files changed, 120 insertions(+), 10 deletions(-) create mode 100644 lib/odp-execute-avx512.c diff --git a/Documentation/ref/ovs-actions.7.rst b/Documentation/ref/ovs-actions.7.rst index b59b7634f..a363d1571 100644 --- a/Documentation/ref/ovs-actions.7.rst +++ b/Documentation/ref/ovs-actions.7.rst @@ -2354,3 +2354,29 @@ never leads to recursion. The ``resubmit`` extension action is more flexible. **Conformance** OpenFlow 1.1 introduced ``goto_table``. Open vSwitch 2.1 added support for ``goto_table``. + +Actions Implementations +----------------------- + +Actions are used in OpenFlow flows to describe what to do when the flow +matches a packet. Just like with the datapath interface, SIMD instructions +with the userspace datapath can be applied to the action implementation to +improve performance. + +OVS provides multiple implementations of the actions. +Available implementations can be listed with the following command:: + + $ ovs-appctl dpif-netdev/action-impl-show + Available Actions implementations: + scalar (available: Yes, active: Yes) + autovalidator (available: Yes, active: No) + avx512 (available: Yes, active: No) + +By default, ``scalar`` is used. Implementations can be selected by +name:: + + $ ovs-appctl dpif-netdev/action-impl-set avx512 + Action implementation set to avx512. + + $ ovs-appctl dpif-netdev/action-impl-set scalar + Action implementation set to scalar. diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index c15d5b38f..d279131b8 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -361,12 +361,12 @@ testsuite. Userspace datapath: Testing and Validation of CPU-specific Optimizations '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -As multiple versions of the datapath classifier and packet parsing functions -can co-exist, each with different CPU ISA optimizations, it is important to -validate that they all give the exact same results. To easily test all the -implementations, an ``autovalidator`` implementation of them exists. This -implementation runs all other available implementations, and verifies that the -results are identical. +As multiple versions of the datapath classifier, packet parsing functions and +actions can co-exist, each with different CPU ISA optimizations, it is +important to validate that they all give the exact same results. To easily +test all the implementations, an ``autovalidator`` implementation of them +exists. This implementation runs all other available implementations, and +verifies that the results are identical. Running the OVS unit tests with the autovalidator enabled ensures all implementations provide the same results. Note that the performance of the @@ -382,18 +382,26 @@ To set the autovalidator for the packet parser, use this command:: $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator +To set the autovalidator for actions, use this command:: + + $ ovs-appctl dpif-netdev/action-impl-set autovalidator + To run the OVS unit test suite with the autovalidator as the default implementation, it is required to recompile OVS. During the recompilation, the default priority of the `autovalidator` implementation is set to the -maximum priority, ensuring every test will be run with every implementation:: +maximum priority, ensuring every test will be run with every implementation. +Priority is only related to mfex autovalidator and not the actions +autovalidator.:: - $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator + $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator \ + --enable-actions-default-autovalidator The following line should be seen in the configuration log when the above options are used:: checking whether DPCLS Autovalidator is default implementation... yes checking whether MFEX Autovalidator is default implementation... yes + checking whether actions Autovalidator is default implementation... yes Compile OVS in debug mode to have `ovs_assert` statements error out if there is a mis-match in the datapath classifier lookup or packet parser diff --git a/NEWS b/NEWS index 7ae4d87bb..927b7eed8 100644 --- a/NEWS +++ b/NEWS @@ -39,6 +39,8 @@ Post-v2.17.0 implementations available at run time. * Add build time configure command to enable auto-validator as default actions implementation at build time. + * Add AVX512 implementation of actions. + diff --git a/lib/automake.mk b/lib/automake.mk index 1a49dd30b..ad3ea3fb1 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -31,6 +31,9 @@ lib_LTLIBRARIES += lib/libopenvswitchavx512.la lib_libopenvswitch_la_LIBADD += lib/libopenvswitchavx512.la lib_libopenvswitchavx512_la_CFLAGS = \ -mavx512f \ + -mavx512bw \ + -mavx512vl \ + -mavx512dq \ -mbmi \ -mbmi2 \ -fPIC \ @@ -44,7 +47,8 @@ lib_libopenvswitchavx512_la_CFLAGS += \ -mavx512bw lib_libopenvswitchavx512_la_SOURCES += \ lib/dpif-netdev-extract-avx512.c \ - lib/dpif-netdev-lookup-avx512-gather.c + lib/dpif-netdev-lookup-avx512-gather.c \ + lib/odp-execute-avx512.c endif # HAVE_AVX512BW lib_libopenvswitchavx512_la_LDFLAGS = \ -static diff --git a/lib/cpu.c b/lib/cpu.c index 2df003c51..0292f715e 100644 --- a/lib/cpu.c +++ b/lib/cpu.c @@ -53,6 +53,7 @@ X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 16, OVS_CPU_ISA_X86_AVX512F) X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 30, OVS_CPU_ISA_X86_AVX512BW) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 1, OVS_CPU_ISA_X86_AVX512VBMI) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 14, OVS_CPU_ISA_X86_VPOPCNTDQ) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 31, OVS_CPU_ISA_X86_AVX512VL) #endif bool diff --git a/lib/cpu.h b/lib/cpu.h index 92897bb71..3215229bc 100644 --- a/lib/cpu.h +++ b/lib/cpu.h @@ -25,6 +25,7 @@ enum ovs_cpu_isa { OVS_CPU_ISA_X86_AVX512F, OVS_CPU_ISA_X86_AVX512BW, OVS_CPU_ISA_X86_AVX512VBMI, + OVS_CPU_ISA_X86_AVX512VL, OVS_CPU_ISA_X86_VPOPCNTDQ, OVS_CPU_ISA_X86_LAST = OVS_CPU_ISA_X86_VPOPCNTDQ, }; diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c new file mode 100644 index 000000000..1fb334689 --- /dev/null +++ b/lib/odp-execute-avx512.c @@ -0,0 +1,56 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "cpu.h" +#include "dp-packet.h" +#include "immintrin.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "openvswitch/vlog.h" + +/* Probe functions to check ISA requirements. */ +static bool +avx512_isa_probe(void) +{ + static enum ovs_cpu_isa isa_required[] = { + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_BMI2, + OVS_CPU_ISA_X86_AVX512VL, + }; + + bool ret = true; + for (int i = 0; i < ARRAY_SIZE(isa_required); i++) { + if (!cpu_has_isa(isa_required[i])) { + ret = -ENOTSUP; + } + } + + return ret; +} + +int +action_avx512_init(struct odp_execute_action_impl *self) +{ + if (!avx512_isa_probe()) { + return -ENOTSUP; + } + + return 0; +} diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index f8d0896b5..de2e4dfc4 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -42,6 +42,14 @@ static struct odp_execute_action_impl action_impls[] = { .name = "scalar", .init_func = odp_action_scalar_init, }, + + #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) + [ACTION_IMPL_AVX512] = { + .available = false, + .name = "avx512", + .init_func = action_avx512_init, + }, + #endif }; static void diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 5322eb8df..e4724b8b2 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -59,6 +59,9 @@ enum odp_execute_action_impl_idx { * Do not change the autovalidator position in this list without updating * the define below. */ + #if (__x86_64__ && HAVE_AVX512F && HAVE_LD_AVX512_GOOD && __SSE4_2__) + ACTION_IMPL_AVX512, + #endif ACTION_IMPL_MAX, }; @@ -82,10 +85,11 @@ int odp_action_scalar_init(struct odp_execute_action_impl *self); int action_autoval_init(struct odp_execute_action_impl *self); +int action_avx512_init(struct odp_execute_action_impl *self); + int odp_execute_action_set(const char *name, struct odp_execute_action_impl *active); void odp_execute_action_get_info(struct ds *name); - #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Tue Jun 14 11:57:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643267 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=M523lLyK; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6Q0XTTz9sGG for ; Tue, 14 Jun 2022 21:59:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 53A0340BC3; Tue, 14 Jun 2022 11:59:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4XPIkrBfVBno; Tue, 14 Jun 2022 11:59:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1373040B53; Tue, 14 Jun 2022 11:59:19 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0318AC009A; Tue, 14 Jun 2022 11:59:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 11A97C007A for ; Tue, 14 Jun 2022 11:59:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7BD4660B69 for ; Tue, 14 Jun 2022 11:59:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mtbK2yQdqdsL for ; Tue, 14 Jun 2022 11:59:03 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id D8A836002E for ; Tue, 14 Jun 2022 11:59:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207943; x=1686743943; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d1qIzC67UCjH9kSBiWaTQx32rp2M1c6Y7e/Ld5qE8u4=; b=M523lLyKe+qBY5RDn2/nZjcaXKrneisfEucW68fALoqp1TnSZZVG+Ujp WQaWMJMI3nm02dLmKmKkFHd/SHFt7O+rDylFaZYDIq4F6olQy5kmqUOLG UX/JfoDqbLcy9H5GdIypkMWxJLnJJ51zRE3c/7x6Hfu/KlnaKMoYa4Y4q /O6sFy9wTnYPigK/UG15AXk9vTLanOJzsfRCGqJr4f75KT608O2vL6cVK 5eIInXUWrPBdibIevua6XpsFvLVRGEsLBm2mxkEsTVyZNaJPnGnsM/b2f 4Thu6k7Tfzl0xE2J6EUpWo2g2sgFVBYPg48Hs2Hn0dbz0behthza92FME A==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137293" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137293" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:59:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665845" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:59:02 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:40 +0000 Message-Id: <20220614115743.1143341-9-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 08/11] odp-execute: Add ISA implementation of pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the pop_vlan action. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 91 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 1fb334689..f9e2b1727 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -14,6 +14,11 @@ * limitations under the License. */ +#ifdef __x86_64__ +/* Sparse cannot handle the AVX512 instructions. */ +#if !defined(__CHECKER__) + + #include #include @@ -24,6 +29,86 @@ #include "odp-netlink.h" #include "openvswitch/vlog.h" +VLOG_DEFINE_THIS_MODULE(odp_execute_avx512); +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_5_ofs) + + MEMBER_SIZEOF(struct dp_packet, l2_5_ofs) == + offsetof(struct dp_packet, l3_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == + offsetof(struct dp_packet, l4_ofs)); + +/* Adjust the size of the l2 portion of the dp_packet, updating the l2 + * pointer and the layer offsets. The function will broadcast resize_by_bytes + * across a register and uses a kmask to identify which lanes should be + * incremented/decremented. Either an add or subtract will be performed + * and the result is stored back to the original packet. */ +static inline void ALWAYS_INLINE +avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) +{ + /* Update packet size/data pointers */ + if (resize_by_bytes >= 0) { + dp_packet_prealloc_headroom(b, resize_by_bytes); + } else { + ovs_assert(dp_packet_size(b) - dp_packet_l2_pad_size(b) >= + -resize_by_bytes); + } + + dp_packet_set_data(b, (char *) dp_packet_data(b) - resize_by_bytes); + dp_packet_set_size(b, dp_packet_size(b) + resize_by_bytes); + + const __m128i v_zeros = _mm_setzero_si128(); + const __m128i v_u16_max = _mm_cmpeq_epi16(v_zeros, v_zeros); + + const uint8_t k_lanes = 0b1110; + __m128i v_offset = _mm_set1_epi16(abs(resize_by_bytes)); + + /* Load 128 bits from the dp_packet structure starting at the l2_pad_size + * offset. */ + void *adjust_ptr = &b->l2_pad_size; + __m128i v_adjust_src = _mm_loadu_si128(adjust_ptr); + + __mmask8 k_cmp = _mm_mask_cmpneq_epu16_mask(k_lanes, v_adjust_src, + v_u16_max); + + __m128i v_adjust_wip; + + if (resize_by_bytes >= 0) { + v_adjust_wip = _mm_mask_add_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } else { + v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } + + _mm_storeu_si128(adjust_ptr, v_adjust_wip); +} + +/* This function will load the entire vlan_eth_header into a 128-bit wide + * register. Then use an 8-byte realign to shift the header right by 12 bytes + * to remove the vlan header and store the results back to the orginal header. + */ +static void +action_avx512_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct vlan_eth_header *veh = dp_packet_eth(packet); + + if (veh && dp_packet_size(packet) >= sizeof *veh && + eth_type_vlan(veh->veth_type)) { + + __m128i v_ether = _mm_loadu_si128((void *) veh); + __m128i v_realign = _mm_alignr_epi8(v_ether, _mm_setzero_si128(), + 16 - VLAN_HEADER_LEN); + _mm_storeu_si128((void *) veh, v_realign); + avx512_dp_packet_resize_l2(packet, -VLAN_HEADER_LEN); + } + } +} + /* Probe functions to check ISA requirements. */ static bool avx512_isa_probe(void) @@ -52,5 +137,11 @@ action_avx512_init(struct odp_execute_action_impl *self) return -ENOTSUP; } + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; return 0; } + +#endif +#endif From patchwork Tue Jun 14 11:57:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643268 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=QLdCqh62; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6W1F1zz9sGG for ; Tue, 14 Jun 2022 21:59:31 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 91A5541934; Tue, 14 Jun 2022 11:59:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sl02L7o7lQX2; Tue, 14 Jun 2022 11:59:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 1981F418ED; Tue, 14 Jun 2022 11:59:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C86C7C009D; Tue, 14 Jun 2022 11:59:14 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 23D78C0090 for ; Tue, 14 Jun 2022 11:59:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 44A5760035 for ; Tue, 14 Jun 2022 11:59:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cdEoTfVsUHoe for ; Tue, 14 Jun 2022 11:59:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 84C9260C33 for ; Tue, 14 Jun 2022 11:59:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207945; x=1686743945; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nwuiMUI4AwYsoWJ8tNty9haSYOXesi1EVKye8j0ojaI=; b=QLdCqh629D2WzloUXGeECGEcGI9MG+INmnHOKRmmVeI+3RIsz4BD79Zt kPHXRvYd20pFwRhJ4SGojqia0q70jr5f3OUOIj9k2F6zUOb4qEE4elU7j qA9xVMo0TGpJkMjyd+kvFqku38ZrLfAoEOLRvnuPdPWhPO0CZN3rD7kWV SQf+ziTedJpHZaah1/AoPjmGtU3JEbPrP9lW2ZDKSjqLAriaIXQthLeNm bVJxQmbx8DLNrQafpwx44pP3BWv+FtqcAx/XC9Ja2cv1RYelsWQq1Gxqe 3H+o9EDlPpfaiJq5FwL5h1FyBjoswIFZ5hCm0InKt253N8L1Ps59JnVtm g==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137299" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137299" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:59:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665872" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:59:03 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:41 +0000 Message-Id: <20220614115743.1143341-10-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 09/11] odp-execute: Add ISA implementation of push_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the push_vlan action. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 37 +++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 1 + lib/odp-execute.c | 22 +++++++++++++--------- 3 files changed, 51 insertions(+), 9 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index f9e2b1727..bb178cbac 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -109,6 +109,41 @@ action_avx512_pop_vlan(struct dp_packet_batch *batch, } } +/* This function will load the entire eth_header into a 128-bit wide register. + * Then use an 8-byte shuffle to shift the data left to make room for + * the vlan header. Insert the new vlan header and then store back to the + * original packet. */ +static void +action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + ovs_be16 tpid, tci; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + tpid = vlan->vlan_tpid; + tci = vlan->vlan_tci; + + avx512_dp_packet_resize_l2(packet, VLAN_HEADER_LEN); + + /* Build up the VLAN TCI/TPID, and merge with the moving of Ether. */ + char *pkt_data = (char *) dp_packet_data(packet); + const uint16_t tci_proc = tci & htons(~VLAN_CFI); + const uint32_t tpid_tci = (tci_proc << 16) | tpid; + + static const uint8_t vlan_push_shuffle_mask[16] = { + 4, 5, 6, 7, 8, 9, 10, 11, + 12, 13, 14, 15, 0xFF, 0xFF, 0xFF, 0xFF + }; + + __m128i v_ether = _mm_loadu_si128((void *) pkt_data); + __m128i v_index = _mm_loadu_si128((void *) vlan_push_shuffle_mask); + __m128i v_shift = _mm_shuffle_epi8(v_ether, v_index); + __m128i v_vlan_hdr = _mm_insert_epi32(v_shift, tpid_tci, 3); + _mm_storeu_si128((void *) pkt_data, v_vlan_hdr); + } +} + /* Probe functions to check ISA requirements. */ static bool avx512_isa_probe(void) @@ -140,6 +175,8 @@ action_avx512_init(struct odp_execute_action_impl *self) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index de2e4dfc4..751a68fe3 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -209,6 +209,7 @@ action_autoval_init(struct odp_execute_action_impl *self) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_autoval_generic; return 0; } diff --git a/lib/odp-execute.c b/lib/odp-execute.c index a49b331ef..59f6bdc64 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -845,6 +845,17 @@ action_pop_vlan(struct dp_packet_batch *batch, } } +static void +action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -854,6 +865,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; return 0; } @@ -995,15 +1007,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_PUSH_VLAN: { - const struct ovs_action_push_vlan *vlan = nl_attr_get(a); - - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); - } - break; - } - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1156,6 +1159,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_PUSH_VLAN: OVS_NOT_REACHED(); } From patchwork Tue Jun 14 11:57:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643269 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Un8VXjNi; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6X31vMz9sGG for ; Tue, 14 Jun 2022 21:59:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 70EEC8338B; Tue, 14 Jun 2022 11:59:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5w420H2aSNSv; Tue, 14 Jun 2022 11:59:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id B1FCF833A8; Tue, 14 Jun 2022 11:59:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AFAE7C0082; Tue, 14 Jun 2022 11:59:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1B820C0082 for ; Tue, 14 Jun 2022 11:59:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7CC5860F4D for ; Tue, 14 Jun 2022 11:59:08 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp3.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W5UCs-ikeof9 for ; Tue, 14 Jun 2022 11:59:07 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7E08760F4A for ; Tue, 14 Jun 2022 11:59:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207947; x=1686743947; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yFNh4cSzPLRFmus4NGFo8SqUv+WkUj67ECYJKzWFt5s=; b=Un8VXjNi9+5JOOprM5oHzZUKNp7vH6McF+gBm/K7Oge6W6Dak76qYXYX UwfF5rMOIMHxDMZOngd/pwjEKeSIgnztu7az20Nydy2nOVURQFLJ+jcVo IHBy0Js2/IZOeeIA9dY4kF3gyrKOZ0MjFqLAJlpcB1m0K9AtRMY8kKzoj jJOTGDSLPsA4mvLC1PTDxG04IthLRKhCGW3KDgcAvSjLOpz+R1HFr8I0u coKTiXRrsSUld6iUCpsVyUe0lxKXCCygvRAcCyejwNvihIooh6HUrHIs+ 3Er0KL8U6sMVTlh+c9/381wPtvVDGBojiVizdLGNBsVOboSeqAEyL1Toa g==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137309" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137309" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:59:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665900" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:59:05 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:42 +0000 Message-Id: <20220614115743.1143341-11-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 10/11] odp-execute: Add ISA implementation of set_masked ETH X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit includes infrastructure changes for enabling set_masked_X actions and also adds support for the AVX512 implementation of the eth_set_addrs action. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 69 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 56 +++++++++++++++++++++++++++++-- lib/odp-execute-private.h | 4 +++ lib/odp-execute.c | 65 +++++++++++++++++++++++++----------- lib/odp-execute.h | 3 ++ 5 files changed, 175 insertions(+), 22 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index bb178cbac..ffe25b41d 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -38,6 +38,12 @@ BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == offsetof(struct dp_packet, l4_ofs)); +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_src) + + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_src) == + offsetof(struct ovs_key_ethernet, eth_dst)); + +static struct odp_execute_action_impl avx512_impl; + /* Adjust the size of the l2 portion of the dp_packet, updating the l2 * pointer and the layer offsets. The function will broadcast resize_by_bytes * across a register and uses a kmask to identify which lanes should be @@ -144,6 +150,61 @@ action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +/* This function will load the contents of eth_header into a 128-bit wide + * register. Then an 8-byte shuffle is required to shuffle both key and + * mask to match the layout of the eth_header struct. A bitwise ANDNOT and OR + * is performed on the entire header and results are stored back. */ +static void +action_avx512_eth_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + a = nl_attr_get(a); + const struct ovs_key_ethernet *key = nl_attr_get(a); + const struct ovs_key_ethernet *mask = get_mask(a, struct ovs_key_ethernet); + struct dp_packet *packet; + + __m128i v_src = _mm_loadu_si128((void *) key); + __m128i v_mask = _mm_loadu_si128((void *) mask); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + + struct eth_header *eh = dp_packet_eth(packet); + + if (!eh) { + continue; + } + + static const uint8_t eth_shuffle[16] = { + 6, 7, 8, 9, 10, 11, 0, 1, + 2, 3, 4, 5, 12, 13, 14, 15 + }; + + __m128i v_dst = _mm_loadu_si128((void *) eh); + __m128i v_shuf = _mm_loadu_si128((void *) eth_shuffle); + + v_src = _mm_shuffle_epi8(v_src, v_shuf); + v_mask = _mm_shuffle_epi8(v_mask, v_shuf); + + __m128i dst_masked = _mm_andnot_si128(v_mask, v_dst); + __m128i res = _mm_or_si128(v_src, dst_masked); + + __m128i res_blend = _mm_blend_epi16(v_dst, res, 0x3F); + _mm_storeu_si128((void *) eh, res_blend); + } +} + +static void +action_avx512_set_masked(struct dp_packet_batch *batch OVS_UNUSED, + const struct nlattr *a) +{ + a = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(a); + + if (avx512_impl.set_masked_funcs[attr_type]) { + avx512_impl.set_masked_funcs[attr_type](batch, a); + } +} + /* Probe functions to check ISA requirements. */ static bool avx512_isa_probe(void) @@ -176,6 +237,14 @@ action_avx512_init(struct odp_execute_action_impl *self) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_avx512_set_masked; + + /* Set function pointers that need a 2nd-level function. SET_MASKED action + * requires further processing for action type. Note that 2nd level items + * are identified by OVS_KEY_ATTR_*. */ + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = + action_avx512_eth_set_addrs; + avx512_impl = *self; return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 751a68fe3..e2d650779 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -29,6 +29,8 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); +static struct odp_execute_action_impl autoval_impl; +static bool set_masked = false; static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AUTOVALIDATOR] = { @@ -59,6 +61,11 @@ action_impl_copy_funcs(struct odp_execute_action_impl *src, for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { atomic_store_relaxed(&src->funcs[i], dst->funcs[i]); } + + for (uint32_t i = 0; i < __OVS_KEY_ATTR_MAX; i++) { + atomic_store_relaxed(&src->set_masked_funcs[i], + dst->set_masked_funcs[i]); + } } int @@ -135,19 +142,36 @@ action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) bool failed = false; int type = nl_attr_type(a); enum ovs_action_attr attr_type = (enum ovs_action_attr) type; + enum ovs_key_attr key_attr_type = (enum ovs_key_attr) type; + + if (attr_type == OVS_ACTION_ATTR_SET_MASKED) { + set_masked = true; + const struct nlattr *key = nl_attr_get(a); + key_attr_type = nl_attr_type(key); + } + struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; struct dp_packet_batch good_batch; dp_packet_batch_clone(&good_batch, batch); - scalar->funcs[attr_type](&good_batch, a); + if (!set_masked) { + scalar->funcs[attr_type](&good_batch, a); + } else { + scalar->set_masked_funcs[key_attr_type](&good_batch, a); + } for (int impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { /* Clone original batch and execute implementation under test. */ struct dp_packet_batch test_batch; dp_packet_batch_clone(&test_batch, batch); - action_impls[impl].funcs[attr_type](&test_batch, a); + + if (!set_masked) { + action_impls[impl].funcs[attr_type](&test_batch, a); + } else { + action_impls[impl].set_masked_funcs[key_attr_type](&test_batch, a); + } /* Loop over implementations, checking each one. */ for (int pidx = 0; pidx < batch->count; pidx++) { @@ -200,7 +224,26 @@ action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) dp_packet_delete_batch(&good_batch, 1); /* Apply the action to the original batch for continued processing. */ - scalar->funcs[attr_type](batch, a); + if (!set_masked) { + scalar->funcs[attr_type](batch, a); + } else { + scalar->set_masked_funcs[key_attr_type](batch, a); + } + + set_masked = false; +} + +static void +action_set_masked_init(struct dp_packet_batch *batch OVS_UNUSED, + const struct nlattr *a) +{ + const struct nlattr *type = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(type); + + if (autoval_impl.set_masked_funcs[attr_type]) { + set_masked = true; + autoval_impl.set_masked_funcs[attr_type](batch, a); + } } int @@ -210,6 +253,13 @@ action_autoval_init(struct odp_execute_action_impl *self) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_autoval_generic; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_autoval_generic; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked_init; + + /* Set function pointers that need a 2nd-level function. SET_MASKED action + * requires further processing for action type. Note that 2nd level items + * are identified by OVS_KEY_ATTR_*. */ + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_autoval_generic; + autoval_impl = *self; return 0; } diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index e4724b8b2..1f4d614ca 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -49,6 +49,10 @@ struct odp_execute_action_impl { /* An array of callback functions, one for each action. */ ATOMIC(odp_execute_action_cb) funcs[__OVS_ACTION_ATTR_MAX]; + + /* An array of callback functions, one for each action type. */ + ATOMIC(odp_execute_action_cb) set_masked_funcs[__OVS_KEY_ATTR_MAX]; + }; /* Order of Actions implementations. */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 59f6bdc64..db6e1ec03 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -561,8 +561,6 @@ odp_execute_set_action(struct dp_packet *packet, const struct nlattr *a) } } -#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) - static void odp_execute_masked_set_action(struct dp_packet *packet, const struct nlattr *a) @@ -582,11 +580,6 @@ odp_execute_masked_set_action(struct dp_packet *packet, | (md->pkt_mark & ~*get_mask(a, uint32_t)); break; - case OVS_KEY_ATTR_ETHERNET: - odp_eth_set_addrs(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ethernet)); - break; - case OVS_KEY_ATTR_NSH: { odp_set_nsh(packet, a, true); break; @@ -669,6 +662,8 @@ odp_execute_masked_set_action(struct dp_packet *packet, case OVS_KEY_ATTR_TCP_FLAGS: case OVS_KEY_ATTR_TUNNEL_INFO: case __OVS_KEY_ATTR_MAX: + /* The following action types are handled by the scalar implementation. */ + case OVS_KEY_ATTR_ETHERNET: default: OVS_NOT_REACHED(); } @@ -834,6 +829,12 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. + */ +static struct odp_execute_action_impl actions_active_impl; + static void action_pop_vlan(struct dp_packet_batch *batch, const struct nlattr *a OVS_UNUSED) @@ -856,6 +857,36 @@ action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +static void +action_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + + const struct nlattr *key = nl_attr_get(a); + enum ovs_key_attr key_type = nl_attr_type(key); + + if (actions_active_impl.set_masked_funcs[key_type]) { + actions_active_impl.set_masked_funcs[key_type](batch, a); + } else { + a = nl_attr_get(a); + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_execute_masked_set_action(packet, a); + } + } +} + +static void +action_mod_eth(struct dp_packet_batch *batch, const struct nlattr *a) +{ + a = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_eth_set_addrs(packet, nl_attr_get(a), + get_mask(a, struct ovs_key_ethernet)); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -866,16 +897,17 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked; + + /* Set function pointers that need a 2nd-level function. SET_MASKED action + * requires further processing for action type. Note that 2nd level items + * are identified by OVS_KEY_ATTR_*. */ + self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_mod_eth; + actions_active_impl = *self; return 0; } -/* The active function pointers on the datapath. ISA optimized implementations - * are enabled by plugging them into this static arary, which is consulted when - * applying actions on the datapath. - */ -static struct odp_execute_action_impl actions_active_impl; - void odp_execute_init(void) { @@ -1028,12 +1060,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, } break; - case OVS_ACTION_ATTR_SET_MASKED: - DP_PACKET_BATCH_FOR_EACH(i, packet, batch) { - odp_execute_masked_set_action(packet, nl_attr_get(a)); - } - break; - case OVS_ACTION_ATTR_SAMPLE: DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { odp_execute_sample(dp, packet, steal && last_action, a, @@ -1160,6 +1186,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_SET_MASKED: OVS_NOT_REACHED(); } diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 8668ab73f..762b99473 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -50,4 +50,7 @@ void odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, const struct nlattr *actions, size_t actions_len, odp_execute_cb dp_execute_action); + +#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) + #endif From patchwork Tue Jun 14 11:57:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1643270 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=lNjs7lD2; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LMn6b23Ncz9sGH for ; Tue, 14 Jun 2022 21:59:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 18F7261077; Tue, 14 Jun 2022 11:59:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tDpkBhpIN4Xb; Tue, 14 Jun 2022 11:59:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id BA16A61028; Tue, 14 Jun 2022 11:59:24 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BB928C008D; Tue, 14 Jun 2022 11:59:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0EC10C008A for ; Tue, 14 Jun 2022 11:59:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 504B060F74 for ; Tue, 14 Jun 2022 11:59:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LU8RuGhBglig for ; Tue, 14 Jun 2022 11:59:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5100060F5D for ; Tue, 14 Jun 2022 11:59:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1655207949; x=1686743949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kcGTVmrDCadXPfeF00LJllOLj2bTBhGeVFFrFkZREeU=; b=lNjs7lD2B9J/4UCqDj5l3lMqyK78t+sgwv7iNBxBXSC5mUroUGnqdmDO uOO+o1zOz6wFsFjKjekb7uzF1/5iopAwq+PER07fOPanh4pNHOClfVq84 fGd5xHXpczFothuRQWaRkZttxDkECi9W296iRgzRDNLys4zX+DGitYiMu 2wZs8UrSaALqsFYdq66jXniHNkssJ0S34hj2fmO9Xp8G3X5KYdJ/Re6hQ HDob2bLiXY8syON+fLhng9CL9yNADae4djdAHK01MJ6Dm9GVW6DoB20uj kH41PtrvkTVFNO/qG8IUI/zkCQ7oGDzZOMRfSfuSksw3zZH6eIdcxCicM w==; X-IronPort-AV: E=McAfee;i="6400,9594,10377"; a="276137313" X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="276137313" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2022 04:59:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,300,1647327600"; d="scan'208";a="582665926" Received: from silpixa00401384.ir.intel.com ([10.243.23.50]) by orsmga007.jf.intel.com with ESMTP; 14 Jun 2022 04:59:07 -0700 From: Emma Finn To: ian.stokes@intel.com, echaudro@redhat.com, harry.van.haaren@intel.com, dev@openvswitch.org Date: Tue, 14 Jun 2022 11:57:43 +0000 Message-Id: <20220614115743.1143341-12-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220614115743.1143341-1-emma.finn@intel.com> References: <20220510142202.1087967-1-emma.finn@intel.com> <20220614115743.1143341-1-emma.finn@intel.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH v7 11/11] odp-execute: Add ISA implementation of set_masked IPv4 action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds support for the AVX512 implementation of the ipv4_set_addrs action as well as an AVX512 implementation of updating the checksums. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 210 ++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 1 + lib/odp-execute.c | 19 +++- 3 files changed, 225 insertions(+), 5 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index ffe25b41d..5cba14b92 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -22,6 +22,7 @@ #include #include +#include "csum.h" #include "cpu.h" #include "dp-packet.h" #include "immintrin.h" @@ -193,6 +194,213 @@ action_avx512_eth_set_addrs(struct dp_packet_batch *batch, } } +/* Calculate delta checksum by summing only ip_src and ip_dst fields of + * ip_header. Resulting checksum will be used for updating L4 checksum */ +static inline uint16_t ALWAYS_INLINE +avx512_l4_update_csum(struct ip_header *old_header, __m256i res) +{ + uint16_t tmp_checksum; + __m256i v_zeros = _mm256_setzero_si256(); + + /* Each field needs to be shuffle into 16- bit granularity and across + * lanes. */ + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xffff, 0x0302, 0xffff, + 0x0504, 0xffff, 0x0706, 0xffff, + 0x0100, 0xffff, 0x0302, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xffff, 0xffff, 0xffff, + 0x0d0c, 0xffff, 0x0f0e, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + + __m256i oh = _mm256_loadu_si256((void *) old_header); + oh = _mm256_mask_blend_epi16(0x3C0, oh, res); + __m256i v_shuf1 = _mm256_shuffle_epi8(oh, v_swap16a); + __m256i v_shuf2 = _mm256_shuffle_epi8(oh, v_swap16b); + + /* Add field values. */ + __m256i v_sum = _mm256_add_epi32(v_shuf1, v_shuf2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final hadd. */ + v_sum = _mm256_permutexvar_epi32(v_swap32a, v_sum); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi16(v_sum, v_zeros); + + /* Extract checksum value. */ + tmp_checksum = _mm256_extract_epi16(v_sum, 0); + + return ~tmp_checksum; +} + +/* Calculate checksum by summing entire contents of ip_header leaving out + * current checksum field. */ +static inline uint16_t ALWAYS_INLINE +avx512_ipv4_recalc_csum(__m256i res) +{ + uint32_t new_checksum; + __m256i v_zeros = _mm256_setzero_si256(); + + /* Each field needs to be shuffle into 16-bit granularity and across + * lanes. */ + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xffff, 0x0302, 0xffff, + 0x0504, 0xffff, 0x0706, 0xffff, + 0x0100, 0xffff, 0x0302, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xffff, 0xffff, 0xffff, + 0x0d0c, 0xffff, 0x0f0e, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff, + 0xffff, 0xffff, 0xffff, 0xffff); + + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + + __m256i v_shuf1 = _mm256_shuffle_epi8(res, v_swap16a); + __m256i v_shuf2 = _mm256_shuffle_epi8(res, v_swap16b); + + /* Add field values. */ + __m256i v_sum = _mm256_add_epi32(v_shuf1, v_shuf2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final hadd. */ + v_sum = _mm256_permutexvar_epi32(v_swap32a, v_sum); + v_sum = _mm256_hadd_epi32(v_sum, v_zeros); + v_sum = _mm256_hadd_epi16(v_sum, v_zeros); + + /* Extract new checksum value. */ + new_checksum = _mm256_extract_epi16(v_sum, 0); + + return ~new_checksum; +} + +/* The shuffles used in action_avx512_ipv4_set_addrs() require the ovs_key_ipv4 + * struct to be in this layout. If struct changes, shuffle mask also needs to + * be updated. */ +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_src) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_src) == + offsetof(struct ovs_key_ipv4, ipv4_dst)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_dst) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_dst) == + offsetof(struct ovs_key_ipv4, ipv4_proto)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_proto) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_proto) == + offsetof(struct ovs_key_ipv4, ipv4_tos)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_tos) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_tos) == + offsetof(struct ovs_key_ipv4, ipv4_ttl)); + +static void +action_avx512_ipv4_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + a = nl_attr_get(a); + const struct ovs_key_ipv4 *key = nl_attr_get(a); + const struct ovs_key_ipv4 *mask = get_mask(a, struct ovs_key_ipv4); + struct dp_packet *packet; + ovs_be16 old_csum; + + __m256i v_key = _mm256_loadu_si256((void *) key); + __m256i v_mask = _mm256_loadu_si256((void *) mask); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct ip_header *nh = dp_packet_l3(packet); + old_csum = nh->ip_csum; + + __m256i v_packet = _mm256_loadu_si256((void *) nh); + + /* Shuffle key and mask to match ip_header struct layout. */ + static const uint8_t ip_shuffle_mask[32] = { + 0xFF, 5, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 6, 0xFF, 0xFF, 0xFF, 0, 1, 2, 3, + 0, 1, 2, 3, 0xFF, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + __m256i v_shuf32 = _mm256_setr_epi32(0x0, 0x2, 0xF, 0xF, + 0x1, 0xF, 0xF, 0xF); + + __m256i v_shuffle = _mm256_loadu_si256((void *) ip_shuffle_mask); + + /* Two shuffles are required for key and mask to match the layout of + * the ip_header struct. The _shuffle_epi8 only works within 128-bit + * lanes, so a permute is required to move src and dst into the correct + * lanes. And then a shuffle is used to move the fields into the right + * order. + */ + __m256i v_key_shuf = _mm256_permutexvar_epi32(v_shuf32, v_key); + v_key_shuf = _mm256_shuffle_epi8(v_key_shuf, v_shuffle); + + __m256i v_mask_shuf = _mm256_permutexvar_epi32(v_shuf32, v_mask); + v_mask_shuf = _mm256_shuffle_epi8(v_mask_shuf, v_shuffle); + + __m256i v_pkt_masked = _mm256_andnot_si256(v_mask_shuf, v_packet); + __m256i v_res = _mm256_or_si256(v_key_shuf, v_pkt_masked); + + /* Recalculate the ip_csum based on updated values. */ + uint16_t checksum = avx512_ipv4_recalc_csum(v_res); + + /* Insert new checksum. */ + v_res = _mm256_insert_epi16(v_res, checksum, 5); + + /* If ip_src or ip_dst has been modified, L4 checksum needs to + * be updated too. */ + int update_mask = _mm256_movemask_epi8(v_mask); + if (update_mask & 0xFF) { + + uint16_t tmp_checksum = avx512_l4_update_csum(nh, v_res); + tmp_checksum = ~tmp_checksum; + uint16_t csum; + + if (nh->ip_proto == IPPROTO_UDP) { + /* New UDP checksum. */ + struct udp_header *uh = dp_packet_l4(packet); + if (!uh->udp_csum) { + uh->udp_csum = htons(0xffff); + } else { + uint16_t old_udp_checksum = ~uh->udp_csum; + + uint32_t udp_checksum = old_csum + tmp_checksum; + udp_checksum = csum_finish(udp_checksum); + uint16_t udp_csum = ~udp_checksum; + + uint32_t nw_udp_checksum = udp_csum + old_udp_checksum; + + csum = csum_finish(nw_udp_checksum); + + /* Insert new udp checksum. */ + v_res = _mm256_insert_epi16(v_res, csum, 13); + } + } else if (nh->ip_proto == IPPROTO_TCP) { + /* New TCP checksum. */ + struct tcp_header *th = dp_packet_l4(packet); + uint16_t old_tcp_checksum = ~th->tcp_csum; + + uint32_t tcp_checksum = old_csum + tmp_checksum; + tcp_checksum = csum_finish(tcp_checksum); + uint16_t tcp_csum = ~tcp_checksum; + + uint32_t nw_tcp_checksum = tcp_csum + old_tcp_checksum; + + csum = csum_finish(nw_tcp_checksum); + + th->tcp_csum = csum; + } + } + /* Store new IP header. */ + _mm256_storeu_si256((void *) nh, v_res); + } +} + static void action_avx512_set_masked(struct dp_packet_batch *batch OVS_UNUSED, const struct nlattr *a) @@ -244,6 +452,8 @@ action_avx512_init(struct odp_execute_action_impl *self) * are identified by OVS_KEY_ATTR_*. */ self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = + action_avx512_ipv4_set_addrs; avx512_impl = *self; return 0; diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index e2d650779..763c8afc7 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -259,6 +259,7 @@ action_autoval_init(struct odp_execute_action_impl *self) * requires further processing for action type. Note that 2nd level items * are identified by OVS_KEY_ATTR_*. */ self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_autoval_generic; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_autoval_generic; autoval_impl = *self; return 0; diff --git a/lib/odp-execute.c b/lib/odp-execute.c index db6e1ec03..8da009ce9 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -585,11 +585,6 @@ odp_execute_masked_set_action(struct dp_packet *packet, break; } - case OVS_KEY_ATTR_IPV4: - odp_set_ipv4(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv4)); - break; - case OVS_KEY_ATTR_IPV6: odp_set_ipv6(packet, nl_attr_get(a), get_mask(a, struct ovs_key_ipv6)); @@ -664,6 +659,7 @@ odp_execute_masked_set_action(struct dp_packet *packet, case __OVS_KEY_ATTR_MAX: /* The following action types are handled by the scalar implementation. */ case OVS_KEY_ATTR_ETHERNET: + case OVS_KEY_ATTR_IPV4: default: OVS_NOT_REACHED(); } @@ -887,6 +883,18 @@ action_mod_eth(struct dp_packet_batch *batch, const struct nlattr *a) } } +static void +action_mod_ipv4(struct dp_packet_batch *batch, const struct nlattr *a) +{ + a = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_set_ipv4(packet, nl_attr_get(a), + get_mask(a, struct ovs_key_ipv4)); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ @@ -903,6 +911,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) * requires further processing for action type. Note that 2nd level items * are identified by OVS_KEY_ATTR_*. */ self->set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_mod_eth; + self->set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_mod_ipv4; actions_active_impl = *self; return 0;