From patchwork Thu Jul 14 17:51:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656576 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=EhIhaZQe; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMWl706Rz9sFs for ; Fri, 15 Jul 2022 03:52:18 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BAB8D81763; Thu, 14 Jul 2022 17:52:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org BAB8D81763 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=EhIhaZQe X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yxolviMXjMPl; Thu, 14 Jul 2022 17:52:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 610C781367; Thu, 14 Jul 2022 17:52:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 610C781367 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 39791C0035; Thu, 14 Jul 2022 17:52:13 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5B987C002D for ; Thu, 14 Jul 2022 17:52:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3645D813B6 for ; Thu, 14 Jul 2022 17:52:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3645D813B6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8aX4Kgm25IbQ for ; Thu, 14 Jul 2022 17:52:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 78DE381367 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by smtp1.osuosl.org (Postfix) with ESMTPS id 78DE381367 for ; Thu, 14 Jul 2022 17:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821129; x=1689357129; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DBOJkVuKLfGssYTxb2RWojFH/xn7VRgnxUIfDZPQD4Q=; b=EhIhaZQeOs8p9uzBoNPFku1zII5yw4SnjA3cxvS20Sjl01BHirtwNgy+ LIa6RACyfpo+jgkOurx4UIIzu53p60koLkxbZA1xKhrt/do2K83nu82Ky jneN731IRNdQun3MDN4h9jYevCWlktGa5VaPjtEPTuOmCT5yKlJu3kM9y ra2X0m/+GHQawxCzEB/YGOppfUnNR7ws+oTl/rROJ/CkAhcuLS1TzHxhV Kkc3LupAauFZHbpdcD2Bb/BoCZFPqMQs1S8Tfn9OQnlkkFzaEOGs+ttsX aNNchj3HaAlc4ngA8jVpgSRNJslVcVdb5DjMdQXEEbsrQsTS8AtILuq+F w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268618437" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="268618437" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232073" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:07 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:49 +0000 Message-Id: <20220714175158.3709150-2-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 01/10] odp-execute: Add function pointers to odp-execute for different action implementations. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduces the initial infrastructure required to allow different implementations for OvS actions. The patch introduces action function pointers which allows user to switch between different action implementations available. This will allow for more performance and flexibility so the user can choose the action implementation to best suite their use case. Signed-off-by: Emma Finn Acked-by: Harry van Haaren --- v10: - switch from < __OVS_ATTR_MAX back to <= OVS_ATTR_MAX (Eelco) v9: - rebase conflicts on NEWS - As Actions is initialized in vswitchd/bridge.c now, logs are output on startup, which have been added to the tests/ofproto-macros. --- lib/automake.mk | 2 + lib/odp-execute-private.c | 91 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 71 ++++++++++++++++++++++++++++++ lib/odp-execute.c | 58 ++++++++++++++++++++++++- lib/odp-execute.h | 7 +++ tests/ofproto-macros.at | 1 + vswitchd/bridge.c | 3 ++ 7 files changed, 232 insertions(+), 1 deletion(-) create mode 100644 lib/odp-execute-private.c create mode 100644 lib/odp-execute-private.h diff --git a/lib/automake.mk b/lib/automake.mk index 1d00cfa20..23ba4fab0 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -216,6 +216,8 @@ lib_libopenvswitch_la_SOURCES = \ lib/object-collection.h \ lib/odp-execute.c \ lib/odp-execute.h \ + lib/odp-execute-private.c \ + lib/odp-execute-private.h \ lib/odp-util.c \ lib/odp-util.h \ lib/ofp-actions.c \ diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c new file mode 100644 index 000000000..ba4aee09b --- /dev/null +++ b/lib/odp-execute-private.c @@ -0,0 +1,91 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include + +#include "dpdk.h" +#include "dp-packet.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "odp-util.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(odp_execute_impl); +static int active_action_impl_index; + +static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_SCALAR] = { + .available = false, + .name = "scalar", + .init_func = NULL, + }, +}; + +static void +action_impl_copy_funcs(struct odp_execute_action_impl *dest, + const struct odp_execute_action_impl *src) +{ + for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + atomic_store_relaxed(&dest->funcs[i], src->funcs[i]); + } +} + +struct odp_execute_action_impl * +odp_execute_action_set(const char *name) +{ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + /* String compare, and set ptrs atomically. */ + if (!strcmp(action_impls[i].name, name)) { + active_action_impl_index = i; + + VLOG_INFO("Action implementation set to %s", name); + return &action_impls[i]; + } + } + return NULL; +} + +void +odp_execute_action_init(void) +{ + /* Each impl's function array is initialized to reflect the scalar + * implementation. This simplifies adding optimized implementations, + * as the autovalidator can always compare all actions. + * + * Below will check if impl is available and copies the scalar functions + * to all other implementations. */ + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + bool avail = true; + + if (i != ACTION_IMPL_SCALAR) { + action_impl_copy_funcs(&action_impls[i], + &action_impls[ACTION_IMPL_SCALAR]); + } + + if (action_impls[i].init_func) { + /* Return zero is success, non-zero means error. */ + avail = (action_impls[i].init_func(&action_impls[i]) == 0); + } + + action_impls[i].available = avail; + + VLOG_INFO("Action implementation %s (available: %s)", + action_impls[i].name, avail ? "Yes" : "No"); + } +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h new file mode 100644 index 000000000..25a003e3d --- /dev/null +++ b/lib/odp-execute-private.h @@ -0,0 +1,71 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef ODP_EXTRACT_PRIVATE +#define ODP_EXTRACT_PRIVATE 1 + +#include "dp-packet.h" +#include "odp-execute.h" +#include "odp-netlink.h" +#include "ovs-atomic.h" + +/* Forward declaration for typedef. */ +struct odp_execute_action_impl; + +/* Typedef for an initialization function that can initialize each + * implementation, checking requirements such as CPU ISA. */ +typedef int (*odp_execute_action_init_func) + (struct odp_execute_action_impl *self); + +/* Structure represents an implementation of the odp actions. */ +struct odp_execute_action_impl { + /* When set, the CPU ISA required for this implementation is available + * and the implementation can be used. */ + bool available; + + /* Name of the implementation. */ + const char *name; + + /* Function is used to detect if this CPU has the ISA required + * to run the optimized action implementation and if available, initializes + * the implementation for use. */ + odp_execute_action_init_func init_func; + + /* An array of callback functions, one for each action. */ + ATOMIC(odp_execute_action_cb) funcs[__OVS_ACTION_ATTR_MAX]; +}; + +/* Order of Actions implementations. */ +enum odp_execute_action_impl_idx { + ACTION_IMPL_SCALAR, + /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. + * Do not change the autovalidator position in this list without updating + * the define below. */ + + ACTION_IMPL_MAX, +}; + +/* Index to start verifying implementations from. */ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); + +/* Odp execute init handles setting up the state of the actions functions at + * initialization time. It cannot return errors, as it must always succeed in + * initializing the scalar/generic codepath. */ +void odp_execute_action_init(void); + +struct odp_execute_action_impl * odp_execute_action_set(const char *name); + +#endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 7da56793d..8a120223e 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -17,6 +17,7 @@ #include #include "odp-execute.h" +#include "odp-execute-private.h" #include #include #include @@ -833,6 +834,37 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +/* The active function pointers on the datapath. ISA optimized implementations + * are enabled by plugging them into this static arary, which is consulted when + * applying actions on the datapath. */ +static struct odp_execute_action_impl *actions_active_impl; + +static int +odp_actions_impl_set(const char *name) +{ + struct odp_execute_action_impl *active; + active = odp_execute_action_set(name); + if (!active) { + VLOG_ERR("Failed setting action implementation to %s", name); + return 1; + } + + actions_active_impl = active; + return 0; + +} + +void +odp_execute_init(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + if (ovsthread_once_start(&once)) { + odp_execute_action_init(); + odp_actions_impl_set("scalar"); + ovsthread_once_done(&once); + } +} + /* Executes all of the 'actions_len' bytes of datapath actions in 'actions' on * the packets in 'batch'. If 'steal' is true, possibly modifies and * definitely free the packets in 'batch', otherwise leaves 'batch' unchanged. @@ -857,6 +889,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, NL_ATTR_FOR_EACH_UNSAFE (a, left, actions, actions_len) { int type = nl_attr_type(a); + enum ovs_action_attr attr_type = (enum ovs_action_attr) type; bool last_action = (left <= NLA_ALIGN(a->nla_len)); if (requires_datapath_assistance(a)) { @@ -879,8 +912,28 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, continue; } - switch ((enum ovs_action_attr) type) { + /* If type is set in the active actions implementation, call the + * function-pointer and continue to the next action. */ + if (attr_type <= OVS_ACTION_ATTR_MAX) { + /* Read the action function pointer atomically to avoid non-atomic + * read causing corruption if being written by another thread + * simultaneously. */ + odp_execute_action_cb action_func; + atomic_uintptr_t *active_action_func = + (void *)&actions_active_impl->funcs[attr_type]; + atomic_read_relaxed(active_action_func, + (uintptr_t *) &action_func); + + if (action_func) { + action_func(batch, a); + continue; + } + } + /* If the action was not handled by the active function pointers above, + * process them by switching on the type below. */ + + switch (attr_type) { case OVS_ACTION_ATTR_HASH: { const struct ovs_action_hash *hash_act = nl_attr_get(a); @@ -1094,6 +1147,9 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: OVS_NOT_REACHED(); } + + /* Do not add any generic processing here, as it won't be executed when + * an ISA-specific action implementation exists. */ } dp_packet_delete_batch(batch, steal); diff --git a/lib/odp-execute.h b/lib/odp-execute.h index a3578a575..0921ee924 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -28,6 +28,13 @@ struct dp_packet; struct pkt_metadata; struct dp_packet_batch; + +/* Called once at initialization time. */ +void odp_execute_init(void); + +typedef void (*odp_execute_action_cb)(struct dp_packet_batch *batch, + const struct nlattr *action); + typedef void (*odp_execute_cb)(void *dp, struct dp_packet_batch *batch, const struct nlattr *action, bool should_steal); diff --git a/tests/ofproto-macros.at b/tests/ofproto-macros.at index 84f07c108..676d55aa9 100644 --- a/tests/ofproto-macros.at +++ b/tests/ofproto-macros.at @@ -182,6 +182,7 @@ m4_define([_OVS_VSWITCHD_START], on_exit "kill_ovs_vswitchd `cat ovs-vswitchd.pid`" AT_CHECK([[sed < stderr ' /ovs_numa|INFO|Discovered /d +/odp_execute_impl|INFO|Action implementation /d /vlog|INFO|opened log file/d /vswitchd|INFO|ovs-vswitchd (Open vSwitch)/d /reconnect|INFO|/d diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c index e328d8ead..b704410d3 100644 --- a/vswitchd/bridge.c +++ b/vswitchd/bridge.c @@ -40,6 +40,7 @@ #include "netdev.h" #include "netdev-offload.h" #include "nx-match.h" +#include "odp-execute.h" #include "ofproto/bond.h" #include "ofproto/ofproto.h" #include "openvswitch/dynamic-string.h" @@ -530,6 +531,8 @@ bridge_init(const char *remote) stp_init(); lldp_init(); rstp_init(); + odp_execute_init(); + ifaces_changed = seq_create(); last_ifaces_changed = seq_read(ifaces_changed); ifnotifier = if_notifier_create(if_change_cb, NULL); From patchwork Thu Jul 14 17:51:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656581 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hz4eN7mr; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMX24p0Fz9sFs for ; Fri, 15 Jul 2022 03:52:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A5D376165E; Thu, 14 Jul 2022 17:52:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A5D376165E Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hz4eN7mr X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qg0VqNRkZR3N; Thu, 14 Jul 2022 17:52:30 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id AB6C86164D; Thu, 14 Jul 2022 17:52:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AB6C86164D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E2B3FC007F; Thu, 14 Jul 2022 17:52:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D62AC007E for ; Thu, 14 Jul 2022 17:52:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 71A954177A for ; Thu, 14 Jul 2022 17:52:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 71A954177A Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=hz4eN7mr X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DY1BozO2psE0 for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 96E60417E6 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by smtp2.osuosl.org (Postfix) with ESMTPS id 96E60417E6 for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821139; x=1689357139; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ehSf31HnQhpxYP50SXESWxoQT9PFGqqA92GIFuR6OJ8=; b=hz4eN7mrirVt1126UGkTkWp/+UIcfbbcFPGH4CIt0EzNV0tu62PcBbTk rC2IEaoOdhzhn4oZkvuaCZahQ52b/vfNTEVYmBW8Rn30qRf89GSYqi5QW nWYgtRU3691Gt6qKMvMzj4HF82qtQ8OoyfgQ75HJakVIqbFUNS1fiuZsk ltA/+fanz8WFZq64CxwD+8dHoKO/8Dnfoo58SK2lE6aEXz0e98DL/ST54 pAXRNEIeGrJdrREB7Kac/3pkQDImD56dYnBYEmHQXbexsIInEa9JIyhq1 lh7yP+j1YZLn1WjYCv+STSHLRGP2BdF0LDf1l0HPvCwvlJCcBrtRo0eOR Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268618446" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="268618446" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232100" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:09 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:50 +0000 Message-Id: <20220714175158.3709150-3-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 02/10] odp-execute: Add function pointer for pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit removes the pop_vlan action from the large switch and creates a separate function for batched processing. A function pointer is also added to call the new batched function for the pop_vlan action. Signed-off-by: Emma Finn Acked-by: Harry van Haaren Acked-by: Eelco Chaudron Acked-by: Eelco Chaudron --- lib/odp-execute-private.c | 16 +++++++++++++++- lib/odp-execute-private.h | 4 ++++ lib/odp-execute.c | 31 +++++++++++++++++++++++++------ 3 files changed, 44 insertions(+), 7 deletions(-) diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index ba4aee09b..47cc1b4bc 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -33,7 +33,7 @@ static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", - .init_func = NULL, + .init_func = odp_action_scalar_init, }, }; @@ -87,5 +87,19 @@ odp_execute_action_init(void) VLOG_INFO("Action implementation %s (available: %s)", action_impls[i].name, avail ? "Yes" : "No"); + + /* The following is a run-time check to make sure a scalar + * implementation exists for the given ISA implementation. This is to + * make sure the autovalidator works as expected. */ + if (avail && i != ACTION_IMPL_SCALAR) { + for (int j = 0; j < __OVS_ACTION_ATTR_MAX; j++) { + /* No ovs_assert(), as it can be compiled out. */ + if (action_impls[ACTION_IMPL_SCALAR].funcs[j] == NULL + && action_impls[i].funcs[j] != NULL) { + ovs_assert_failure(OVS_SOURCE_LOCATOR, __func__, + "Missing scalar action function!"); + } + } + } } } diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 25a003e3d..f890e0cf1 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -66,6 +66,10 @@ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); * initializing the scalar/generic codepath. */ void odp_execute_action_init(void); +/* Init functions for the action implementations. Initializes the function + * pointers for optimized action types. */ +int odp_action_scalar_init(struct odp_execute_action_impl *self); + struct odp_execute_action_impl * odp_execute_action_set(const char *name); #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 8a120223e..4ef82da76 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -834,6 +834,29 @@ requires_datapath_assistance(const struct nlattr *a) return false; } +static void +action_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_pop_vlan(packet); + } +} + +/* Implementation of the scalar actions impl init function. Build up the + * array of func ptrs here. */ +int +odp_action_scalar_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + + return 0; +} + /* The active function pointers on the datapath. ISA optimized implementations * are enabled by plugging them into this static arary, which is consulted when * applying actions on the datapath. */ @@ -989,12 +1012,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_POP_VLAN: - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_pop_vlan(packet); - } - break; - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1145,6 +1162,8 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case OVS_ACTION_ATTR_CT: case OVS_ACTION_ATTR_UNSPEC: case __OVS_ACTION_ATTR_MAX: + /* The following actions are handled by the scalar implementation. */ + case OVS_ACTION_ATTR_POP_VLAN: OVS_NOT_REACHED(); } From patchwork Thu Jul 14 17:51:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656578 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=nYwSnL9d; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMWr4mj9z9sFs for ; Fri, 15 Jul 2022 03:52:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 86A23417EB; Thu, 14 Jul 2022 17:52:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 86A23417EB Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=nYwSnL9d X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rCW7rV0j-dmd; Thu, 14 Jul 2022 17:52:21 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 5DE5841775; Thu, 14 Jul 2022 17:52:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5DE5841775 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 21018C0071; Thu, 14 Jul 2022 17:52:20 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6DE74C002D for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 555BC4169B for ; Thu, 14 Jul 2022 17:52:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 555BC4169B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wQmBFnnEmQh2 for ; Thu, 14 Jul 2022 17:52:14 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 00AE64091F Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by smtp2.osuosl.org (Postfix) with ESMTPS id 00AE64091F for ; Thu, 14 Jul 2022 17:52:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821134; x=1689357134; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=akJDrgbFtAoV0iOResQv01WLuwF/1/P5hyMAhyg6rZ4=; b=nYwSnL9dWBliHmrptxigz1Qna5DfLi9RB6YtfhDtbsGhqatwy3C/5PV/ 8V3gtg1FNKQS3RBHYKcsTA/X8kV6e8XbAF5LPqU/dxMKgKnOheqeFjG3E hJo86NjmcwSKxjKvzYTNVTm80POAUPCXKHvpVsyuYndAbwvxg9kcSTDFF hrDKkrHvQPN9/d9fvR91qNHg3kYzenVfwUjjAsMlrLxUdC2SUQnO120k7 L0Q/R6USo6Xlq+0kGgAciaT/L6EP6EImCJON8+ZTRMAbVI8trll159wok QSsrOCxebtBbTfVp+OO2zUqhBOkN+Oja8TCaiRdZ8hB+DHz3fzMb3heQr g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="347269745" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="347269745" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232121" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:11 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:51 +0000 Message-Id: <20220714175158.3709150-4-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 03/10] odp-execute: Add auto validation function for actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit introduced the auto-validation function which allows users to compare the batch of packets obtained from different action implementations against the linear action implementation. The autovalidator function can be triggered at runtime using the following command: $ ovs-appctl odp-execute/action-impl-set autovalidator Signed-off-by: Emma Finn Acked-by: Harry van Haaren Acked-by: Eelco Chaudron Acked-by: Eelco Chaudron --- NEWS | 2 + lib/dp-packet.c | 24 ++++++++++ lib/dp-packet.h | 4 ++ lib/odp-execute-private.c | 98 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.h | 6 +++ 5 files changed, 134 insertions(+) diff --git a/NEWS b/NEWS index 403cc7fb3..311c23e0c 100644 --- a/NEWS +++ b/NEWS @@ -55,6 +55,8 @@ Post-v2.17.0 * 'dpif-netdev/subtable-lookup-prio-get' appctl command renamed to 'dpif-netdev/subtable-lookup-info-get' to better reflect its purpose. The old variant is kept for backward compatibility. + * Add actions auto-validator function to compare different actions + implementations against default implementation. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/lib/dp-packet.c b/lib/dp-packet.c index 35c72542a..4538d2a61 100644 --- a/lib/dp-packet.c +++ b/lib/dp-packet.c @@ -506,3 +506,27 @@ dp_packet_resize_l2(struct dp_packet *b, int increment) dp_packet_adjust_layer_offset(&b->l2_5_ofs, increment); return dp_packet_data(b); } + +bool +dp_packet_compare_offsets(struct dp_packet *b1, struct dp_packet *b2, + struct ds *err_str) +{ + if ((b1->l2_pad_size != b2->l2_pad_size) || + (b1->l2_5_ofs != b2->l2_5_ofs) || + (b1->l3_ofs != b2->l3_ofs) || + (b1->l4_ofs != b2->l4_ofs)) { + if (err_str) { + ds_put_format(err_str, "Packet offset comparison failed\n"); + ds_put_format(err_str, "Buffer 1 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b1->l2_pad_size, b1->l2_5_ofs, + b1->l3_ofs, b1->l4_ofs); + ds_put_format(err_str, "Buffer 2 offsets: l2_pad_size %u," + " l2_5_ofs : %u l3_ofs %u, l4_ofs %u\n", + b2->l2_pad_size, b2->l2_5_ofs, + b2->l3_ofs, b2->l4_ofs); + } + return false; + } + return true; +} diff --git a/lib/dp-packet.h b/lib/dp-packet.h index eea5a9215..55eeaab2c 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -236,6 +236,10 @@ void *dp_packet_steal_data(struct dp_packet *); static inline bool dp_packet_equal(const struct dp_packet *, const struct dp_packet *); +bool dp_packet_compare_offsets(struct dp_packet *good, + struct dp_packet *test, + struct ds *err_str); + /* Frees memory that 'b' points to, as well as 'b' itself. */ static inline void diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 47cc1b4bc..604855b1b 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -30,6 +30,12 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; static struct odp_execute_action_impl action_impls[] = { + [ACTION_IMPL_AUTOVALIDATOR] = { + .available = false, + .name = "autovalidator", + .init_func = action_autoval_init, + }, + [ACTION_IMPL_SCALAR] = { .available = false, .name = "scalar", @@ -103,3 +109,95 @@ odp_execute_action_init(void) } } } + +/* Init sequence required to be scalar first to pick up the default scalar + * implementations, allowing over-riding of the optimized functions later. */ +BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +/* Loop over packets, and validate each one for the given action. */ +static void +action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct odp_execute_action_impl *scalar = &action_impls[ACTION_IMPL_SCALAR]; + enum ovs_action_attr attr_type = nl_attr_type(a); + struct dp_packet_batch original_batch; + bool failed = false; + + dp_packet_batch_clone(&original_batch, batch); + + scalar->funcs[attr_type](batch, a); + + for (int impl = ACTION_IMPL_BEGIN; impl < ACTION_IMPL_MAX; impl++) { + /* Clone original batch and execute implementation under test. */ + struct dp_packet_batch test_batch; + + dp_packet_batch_clone(&test_batch, &original_batch); + action_impls[impl].funcs[attr_type](&test_batch, a); + + /* Loop over implementations, checking each one. */ + for (int pidx = 0; pidx < original_batch.count; pidx++) { + struct dp_packet *good_pkt = batch->packets[pidx]; + struct dp_packet *test_pkt = test_batch.packets[pidx]; + + struct ds log_msg = DS_EMPTY_INITIALIZER; + + /* Compare packet length and payload contents. */ + bool eq = dp_packet_equal(good_pkt, test_pkt); + + if (!eq) { + ds_put_format(&log_msg, "Packet: %d\nAction : ", pidx); + format_odp_actions(&log_msg, a, a->nla_len, NULL); + ds_put_format(&log_msg, "\nGood hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(good_pkt), + dp_packet_size(good_pkt), 0, false); + ds_put_format(&log_msg, "Test hex:\n"); + ds_put_hex_dump(&log_msg, dp_packet_data(test_pkt), + dp_packet_size(test_pkt), 0, false); + + failed = true; + } + + /* Compare offsets and RSS */ + if (!dp_packet_compare_offsets(good_pkt, test_pkt, &log_msg)) { + failed = true; + } + + if (dp_packet_rss_valid(good_pkt)) { + uint32_t good_hash = dp_packet_get_rss_hash(good_pkt); + uint32_t test_hash = dp_packet_get_rss_hash(test_pkt); + + if (good_hash != test_hash) { + ds_put_format(&log_msg, + "Autovalidation rss hash failed\n"); + ds_put_format(&log_msg, "Good RSS hash : %u\n", good_hash); + ds_put_format(&log_msg, "Test RSS hash : %u\n", test_hash); + + failed = true; + } + } + + if (failed) { + VLOG_ERR("Autovalidation of %s failed. Details:\n%s", + action_impls[impl].name, ds_cstr(&log_msg)); + ds_destroy(&log_msg); + failed = false; + } + } + dp_packet_delete_batch(&test_batch, true); + } + dp_packet_delete_batch(&original_batch, true); +} + +int +action_autoval_init(struct odp_execute_action_impl *self) +{ + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + for (int i = 0; i < __OVS_ACTION_ATTR_MAX; i++) { + if (action_impls[ACTION_IMPL_SCALAR].funcs[i]) { + self->funcs[i] = action_autoval_generic; + } + } + return 0; +} diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index f890e0cf1..a155d534f 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -51,6 +51,7 @@ struct odp_execute_action_impl { /* Order of Actions implementations. */ enum odp_execute_action_impl_idx { ACTION_IMPL_SCALAR, + ACTION_IMPL_AUTOVALIDATOR, /* See ACTION_IMPL_BEGIN below, for "first to-be-validated" impl. * Do not change the autovalidator position in this list without updating * the define below. */ @@ -60,6 +61,9 @@ enum odp_execute_action_impl_idx { /* Index to start verifying implementations from. */ BUILD_ASSERT_DECL(ACTION_IMPL_SCALAR == 0); +BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); + +#define ACTION_IMPL_BEGIN (ACTION_IMPL_AUTOVALIDATOR + 1) /* Odp execute init handles setting up the state of the actions functions at * initialization time. It cannot return errors, as it must always succeed in @@ -72,4 +76,6 @@ int odp_action_scalar_init(struct odp_execute_action_impl *self); struct odp_execute_action_impl * odp_execute_action_set(const char *name); +int action_autoval_init(struct odp_execute_action_impl *self); + #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Thu Jul 14 17:51:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656579 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NE6EmM5E; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMWw55rmz9sFs for ; Fri, 15 Jul 2022 03:52:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3F58E8484F; Thu, 14 Jul 2022 17:52:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3F58E8484F Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NE6EmM5E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vp0hTL3JqvF8; Thu, 14 Jul 2022 17:52:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id B79538471B; Thu, 14 Jul 2022 17:52:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B79538471B Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7E5CAC007E; Thu, 14 Jul 2022 17:52:23 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3B7F1C007E for ; Thu, 14 Jul 2022 17:52:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 238D741765 for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 238D741765 Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NE6EmM5E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5GJ-Ij8ZWbDf for ; Thu, 14 Jul 2022 17:52:15 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6CD7941771 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6CD7941771 for ; Thu, 14 Jul 2022 17:52:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821136; x=1689357136; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gxkZFua2rZY7gFLRRWNSGF2dLcSjroIx+M1BdupRrbw=; b=NE6EmM5EznChX1VbWPD9p0k2OE9Y+oiGf2tIMW1swPx3ov++tOja1L+S z+LlvQsXiDAcmnwjWRFzkRiU/sUeC19ILrkEEuCWyfob36yx2ckr6+E2+ tsFIBmg4EqU9WQ8lmVbqMG9Y7AGeoTqL35AuSZI29bnaVw9W503VovXE+ V7vy83+zgXUwHhVY5CiNDnn88BMBdzsj3CnSxHB2A7yotYlneZzjkXYro j6VeCJbGvqu0exsE10GgKom5Iumpxnq5edjq9yEGZdlztE4Zo1ow6/uEg ArQw+OcvfWsMdDTsC9fJWaSXUfnBDwulZmyQ0mWJru7983zPiTQWhSy42 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="347269751" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="347269751" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232155" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:13 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:52 +0000 Message-Id: <20220714175158.3709150-5-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 04/10] odp-execute: Add command to switch action implementation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds a new command to allow the user to switch the active action implementation at runtime. Usage: $ ovs-appctl odp-execute/action-impl-set scalar This commit also adds a new command to retrieve the list of available action implementations. This can be used by to check what implementations of actions are available and what implementation is active during runtime. Usage: $ ovs-appctl odp-execute/action-impl-show Added separate test-case for ovs-actions show/set commands: odp-execute - actions implementation Signed-off-by: Emma Finn Signed-off-by: Kumar Amber Signed-off-by: Sunil Pai G Co-authored-by: Kumar Amber Co-authored-by: Sunil Pai G Acked-by: Harry van Haaren Acked-by: Eelco Chaudron --- NEWS | 2 ++ lib/automake.mk | 1 + lib/odp-execute-private.c | 12 ++++++++++ lib/odp-execute-private.h | 2 ++ lib/odp-execute-unixctl.man | 10 +++++++++ lib/odp-execute.c | 44 +++++++++++++++++++++++++++++++++++++ tests/odp.at | 39 ++++++++++++++++++++++++++++++++ vswitchd/ovs-vswitchd.8.in | 1 + 8 files changed, 111 insertions(+) create mode 100644 lib/odp-execute-unixctl.man diff --git a/NEWS b/NEWS index 311c23e0c..0b95c0bf2 100644 --- a/NEWS +++ b/NEWS @@ -57,6 +57,8 @@ Post-v2.17.0 The old variant is kept for backward compatibility. * Add actions auto-validator function to compare different actions implementations against default implementation. + * Add command line option to switch between different actions + implementations available at run time. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/lib/automake.mk b/lib/automake.mk index 23ba4fab0..5c3b05f6b 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -584,6 +584,7 @@ MAN_FRAGMENTS += \ lib/netdev-dpdk-unixctl.man \ lib/dpif-netdev-unixctl.man \ lib/dpif-netlink-unixctl.man \ + lib/odp-execute-unixctl.man \ lib/ofp-version.man \ lib/ovs.tmac \ lib/ovs-replay.man \ diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 604855b1b..60f202cad 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -67,6 +67,18 @@ odp_execute_action_set(const char *name) return NULL; } +void +odp_execute_action_get_info(struct ds *string) +{ + ds_put_cstr(string, "Available Actions implementations:\n"); + for (int i = 0; i < ACTION_IMPL_MAX; i++) { + ds_put_format(string, " %s (available: %s, active: %s)\n", + action_impls[i].name, + action_impls[i].available ? "Yes" : "No", + i == active_action_impl_index ? "Yes" : "No"); + } +} + void odp_execute_action_init(void) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index a155d534f..8c2ec3854 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -78,4 +78,6 @@ struct odp_execute_action_impl * odp_execute_action_set(const char *name); int action_autoval_init(struct odp_execute_action_impl *self); +void odp_execute_action_get_info(struct ds *name); + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute-unixctl.man b/lib/odp-execute-unixctl.man new file mode 100644 index 000000000..82d51e1d3 --- /dev/null +++ b/lib/odp-execute-unixctl.man @@ -0,0 +1,10 @@ +.SS "ODP-EXECUTE COMMANDS" +These commands manage the "odp-execute" component. + +.IP "\fBodp-execute/action-impl-show\fR +Lists the actions implementations that are available and highlights the +currently enabled one. +. +.IP "\fBodp-execute/action-impl-set\fR \fIaction_impl\fR" +Sets the action implementation to any available implementation. By default +"scalar" is used. diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 4ef82da76..2a6239aaf 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -39,6 +39,7 @@ #include "csum.h" #include "conntrack.h" #include "openvswitch/vlog.h" +#include "unixctl.h" VLOG_DEFINE_THIS_MODULE(odp_execute); COVERAGE_DEFINE(datapath_drop_sample_error); @@ -877,6 +878,48 @@ odp_actions_impl_set(const char *name) } +static void +action_impl_set(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + int err = odp_actions_impl_set(argv[1]); + if (err) { + ds_put_format(&reply, + "Error: unknown action implementation, %s, specified!", + argv[1]); + unixctl_command_reply_error(conn, ds_cstr(&reply)); + } else { + ds_put_format(&reply, "Action implementation set to %s.", argv[1]); + unixctl_command_reply(conn, ds_cstr(&reply)); + } + + ds_destroy(&reply); +} + +static void +action_impl_show(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED) +{ + struct ds reply = DS_EMPTY_INITIALIZER; + + odp_execute_action_get_info(&reply); + unixctl_command_reply(conn, ds_cstr(&reply)); + ds_destroy(&reply); +} + +static void +odp_execute_unixctl_init(void) +{ + unixctl_command_register("odp-execute/action-impl-set", "name", + 1, 1, action_impl_set, + NULL); + unixctl_command_register("odp-execute/action-impl-show", "", + 0, 0, action_impl_show, + NULL); +} + void odp_execute_init(void) { @@ -884,6 +927,7 @@ odp_execute_init(void) if (ovsthread_once_start(&once)) { odp_execute_action_init(); odp_actions_impl_set("scalar"); + odp_execute_unixctl_init(); ovsthread_once_done(&once); } } diff --git a/tests/odp.at b/tests/odp.at index 4d08c59ca..7a1cf3b2c 100644 --- a/tests/odp.at +++ b/tests/odp.at @@ -472,3 +472,42 @@ AT_CHECK_UNQUOTED([ovstest test-odp parse-keys < odp-in.txt], [0], [dnl odp_flow_from_string: error (syntax error at encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap(encap()))))))))))))))))))))))))))))))))) ]) AT_CLEANUP + +AT_BANNER([datapath actions in userspace]) +AT_SETUP([odp-execute - actions implementation]) +OVS_VSWITCHD_START() + +AT_CHECK([ovs-vsctl show], [], [stdout]) + +dnl Set the scalar first, so we always have the scalar impl as Active. +AT_CHECK([ovs-appctl odp-execute/action-impl-set scalar], [0], [dnl +Action implementation set to scalar. +]) +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: Yes) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: No) +]) + +dnl Set the autovalidator impl to active. +AT_CHECK([ovs-appctl odp-execute/action-impl-set autovalidator], [0], [dnl +Action implementation set to autovalidator. +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "scalar"], [], [dnl + scalar (available: Yes, active: No) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-show | grep "autovalidator"], [], [dnl + autovalidator (available: Yes, active: Yes) +]) + +AT_CHECK([ovs-appctl odp-execute/action-impl-set invalid_implementation], [2], [], [dnl +Error: unknown action implementation, invalid_implementation, specified! +ovs-appctl: ovs-vswitchd: server returned an error +]) + +OVS_VSWITCHD_STOP(["/Failed setting action implementation to invalid_implementation/d"]) +AT_CLEANUP diff --git a/vswitchd/ovs-vswitchd.8.in b/vswitchd/ovs-vswitchd.8.in index 1a32402be..9569265fc 100644 --- a/vswitchd/ovs-vswitchd.8.in +++ b/vswitchd/ovs-vswitchd.8.in @@ -282,6 +282,7 @@ type). .so lib/dpif-netdev-unixctl.man .so lib/dpif-netlink-unixctl.man .so lib/netdev-dpdk-unixctl.man +.so lib/odp-execute-unixctl.man .so ofproto/ofproto-dpif-unixctl.man .so ofproto/ofproto-unixctl.man .so lib/vlog-unixctl.man From patchwork Thu Jul 14 17:51:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656580 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=IARwnFBo; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMX00S0yz9sFs for ; Fri, 15 Jul 2022 03:52:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9578460D59; Thu, 14 Jul 2022 17:52:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9578460D59 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=IARwnFBo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Eei-z8E5ZyXe; Thu, 14 Jul 2022 17:52:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 68AD561647; Thu, 14 Jul 2022 17:52:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 68AD561647 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9BCA4C002D; Thu, 14 Jul 2022 17:52:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 218F5C0085 for ; Thu, 14 Jul 2022 17:52:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 688914176F for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 688914176F Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=IARwnFBo X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JjMEmpUAkDgy for ; Thu, 14 Jul 2022 17:52:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2EF8F41779 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2EF8F41779 for ; Thu, 14 Jul 2022 17:52:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821137; x=1689357137; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+tJEpxaPTfXBhK6mgJbnI46XtAU1jY9sIttAgiYcHDE=; b=IARwnFBotyGyNSzv265on6+pO51E3t6FC1VgwCasaBaFmzyO/sqhtfqw Nmufj30z6ZsSGqXNZ/qswNvB15sajjbE1UFrlMOZkNa6gv9tWhuJ8pQxp cKatoOxtH4O36+bqniCAdJLUnN+Ag02IhREtxkVslo6wvbLlpp8S5lQP3 iTxYIgyJuPjt3wUwPznTWiAyT6lXVnEm7QZ297HHFN1Ldxw0ZNEOja3kC olLRGSp1hbyS4VJx8DM4Tj1XRuvr3jsRT5lYTdu/7p9zDQ2hmXTXMdqEX bOObcPrnOUv/4B3eE/ieCAtJk+o+v4zbjAekY776+lxywoLUsutLobO35 w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="347269758" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="347269758" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232181" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:15 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:53 +0000 Message-Id: <20220714175158.3709150-6-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 05/10] acinclude: Add configure option to enable actions autovalidator at build time. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Kumar Amber This commit adds a new command to allow the user to enable the actions autovalidator by default at build time thus allowing for running unit test by default. $ ./configure --enable-actions-default-autovalidator Signed-off-by: Kumar Amber Acked-by: Harry van Haaren Acked-by: Eelco Chaudron Acked-by: Eelco Chaudron --- NEWS | 2 ++ acinclude.m4 | 20 ++++++++++++++++++++ configure.ac | 1 + lib/odp-execute.c | 4 ++++ 4 files changed, 27 insertions(+) diff --git a/NEWS b/NEWS index 0b95c0bf2..cf8e8a290 100644 --- a/NEWS +++ b/NEWS @@ -59,6 +59,8 @@ Post-v2.17.0 implementations against default implementation. * Add command line option to switch between different actions implementations available at run time. + * Add build time configure command to enable auto-validator as default + actions implementation at build time. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/acinclude.m4 b/acinclude.m4 index d15f11a4e..21c505fbd 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -14,6 +14,26 @@ # See the License for the specific language governing permissions and # limitations under the License. +dnl Set OVS Actions Autovalidator as the default action implementation +dnl at compile time. This enables automatically running all unit tests +dnl with all actions implementations. +AC_DEFUN([OVS_CHECK_ACTIONS_AUTOVALIDATOR], [ + AC_ARG_ENABLE([actions-default-autovalidator], + [AC_HELP_STRING([--enable-actions-default-autovalidator], + [Enable actions autovalidator as default + ovs actions implementation.])], + [autovalidator=yes],[autovalidator=no]) + AC_MSG_CHECKING([whether actions Autovalidator is default implementation]) + if test "$autovalidator" != yes; then + AC_MSG_RESULT([no]) + else + AC_DEFINE([ACTIONS_AUTOVALIDATOR_DEFAULT], [1], + [Autovalidator for actions is a default implementation.]) + AC_MSG_RESULT([yes]) + fi +]) + + dnl Set OVS MFEX Autovalidator as default miniflow extract at compile time? dnl This enables automatically running all unit tests with all MFEX dnl implementations. diff --git a/configure.ac b/configure.ac index 6f8679d7c..6c51e48ce 100644 --- a/configure.ac +++ b/configure.ac @@ -184,6 +184,7 @@ OVS_CONDITIONAL_CC_OPTION([-Wno-unused-parameter], [HAVE_WNO_UNUSED_PARAMETER]) OVS_ENABLE_WERROR_TOP OVS_ENABLE_SPARSE OVS_CTAGS_IDENTIFIERS +OVS_CHECK_ACTIONS_AUTOVALIDATOR OVS_CHECK_DPCLS_AUTOVALIDATOR OVS_CHECK_DPIF_AVX512_DEFAULT OVS_CHECK_MFEX_AUTOVALIDATOR diff --git a/lib/odp-execute.c b/lib/odp-execute.c index 2a6239aaf..ff16e9bf3 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -926,7 +926,11 @@ odp_execute_init(void) static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; if (ovsthread_once_start(&once)) { odp_execute_action_init(); +#ifdef ACTIONS_AUTOVALIDATOR_DEFAULT + odp_actions_impl_set("autovalidator"); +#else odp_actions_impl_set("scalar"); +#endif odp_execute_unixctl_init(); ovsthread_once_done(&once); } From patchwork Thu Jul 14 17:51:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656582 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=m1JGguLh; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMX56znzz9sFs for ; Fri, 15 Jul 2022 03:52:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 46AFF4185C; Thu, 14 Jul 2022 17:52:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 46AFF4185C Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=m1JGguLh X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FvJjdC9uLuA3; Thu, 14 Jul 2022 17:52:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 37D514187F; Thu, 14 Jul 2022 17:52:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 37D514187F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 05A62C0035; Thu, 14 Jul 2022 17:52:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BE03C007F for ; Thu, 14 Jul 2022 17:52:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 8A57D417E6 for ; Thu, 14 Jul 2022 17:52:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8A57D417E6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PQlcQV9DHFVt for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3451C4176E Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3451C4176E for ; Thu, 14 Jul 2022 17:52:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821139; x=1689357139; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vkoL2J6RB1Y0zBDnu0L1w7Q8uz9taKZIlQq9xcf6L1o=; b=m1JGguLh2AaJ7w7RPrNoE/RppIYcDC7uHlbGc8MgD96V5JYHE5HVTbNl aNOritLP/Z8Z4qaLXXptt4mTKItgQES/k5zOdUUJyzYUJe5vaSlsfjgva xFVZpFAy+av0k4FKs2XSOOHbO/IxunUSMJvg04BWedbyXOHBYSR6pqfAA xZvh2/tCD9l05rd/ZHqarEv+iDdMi5lIZSL5bdKOSgjzFgtoegUzA+0gs OmoRYvImnB2xArZhLVA338rVNRgPH8gI60bADqfUYWFw5gO28bFg1nKSo sOUlBzvwuR72kw8SoszdXxxx0ndFKiPNmIIqrHaMdEpob7MfYCrUNu71j g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="347269761" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="347269761" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232202" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:16 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:54 +0000 Message-Id: <20220714175158.3709150-7-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 06/10] odp-execute: Add ISA implementation of actions. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the action functionality. Usage: $ ovs-appctl odp-execute/action-impl-set avx512 Signed-off-by: Emma Finn Signed-off-by: Harry van Haaren Acked-by: Eelco Chaudron --- Documentation/topics/dpdk/bridge.rst | 30 ++++++++++++++++++++++++++++ Documentation/topics/testing.rst | 24 ++++++++++++++-------- NEWS | 1 + acinclude.m4 | 1 + lib/cpu.c | 1 + lib/cpu.h | 1 + lib/odp-execute-private.c | 8 ++++++++ lib/odp-execute-private.h | 12 +++++++++++ m4/openvswitch.m4 | 29 +++++++++++++++++++++++++++ 9 files changed, 99 insertions(+), 8 deletions(-) diff --git a/Documentation/topics/dpdk/bridge.rst b/Documentation/topics/dpdk/bridge.rst index 1f626c7c2..354f1ced1 100644 --- a/Documentation/topics/dpdk/bridge.rst +++ b/Documentation/topics/dpdk/bridge.rst @@ -321,3 +321,33 @@ following command:: ``scalar`` can be selected on core ``3`` by the following command:: $ ovs-appctl dpif-netdev/miniflow-parser-set -pmd 3 scalar + + +Actions Implementations (Experimental) +-------------------------------------- + +Actions describe what processing or modification should be performed on a +packet when it matches a given flow. Similar to the datapath interface, +DPCLS and MFEX (see above), the implementation of these actions can be +accelerated using SIMD instructions, resulting in improved performance. + +OVS provides multiple implementations of the actions, however some +implementations requiring a CPU capable of executing the required SIMD +instructions. + +Available implementations can be listed with the following command:: + + $ ovs-appctl odp-execute/action-impl-show + Available Actions implementations: + scalar (available: Yes, active: Yes) + autovalidator (available: Yes, active: No) + avx512 (available: Yes, active: No) + +By default, ``scalar`` is used. Implementations can be selected by +name:: + + $ ovs-appctl odp-execute/action-impl-set avx512 + Action implementation set to avx512. + + $ ovs-appctl odp-execute/action-impl-set scalar + Action implementation set to scalar. diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index c15d5b38f..a6c747b18 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -361,12 +361,12 @@ testsuite. Userspace datapath: Testing and Validation of CPU-specific Optimizations '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' -As multiple versions of the datapath classifier and packet parsing functions -can co-exist, each with different CPU ISA optimizations, it is important to -validate that they all give the exact same results. To easily test all the -implementations, an ``autovalidator`` implementation of them exists. This -implementation runs all other available implementations, and verifies that the -results are identical. +As multiple versions of the datapath classifier, packet parsing functions and +actions can co-exist, each with different CPU ISA optimizations, it is +important to validate that they all give the exact same results. To easily +test all the implementations, an ``autovalidator`` implementation of them +exists. This implementation runs all other available implementations, and +verifies that the results are identical. Running the OVS unit tests with the autovalidator enabled ensures all implementations provide the same results. Note that the performance of the @@ -382,18 +382,26 @@ To set the autovalidator for the packet parser, use this command:: $ ovs-appctl dpif-netdev/miniflow-parser-set autovalidator +To set the autovalidator for actions, use this command:: + + $ ovs-appctl odp-execute/action-impl-set autovalidator + To run the OVS unit test suite with the autovalidator as the default implementation, it is required to recompile OVS. During the recompilation, the default priority of the `autovalidator` implementation is set to the -maximum priority, ensuring every test will be run with every implementation:: +maximum priority, ensuring every test will be run with every implementation. +Priority is only related to mfex autovalidator and not the actions +autovalidator.:: - $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator + $ ./configure --enable-autovalidator --enable-mfex-default-autovalidator \ + --enable-actions-default-autovalidator The following line should be seen in the configuration log when the above options are used:: checking whether DPCLS Autovalidator is default implementation... yes checking whether MFEX Autovalidator is default implementation... yes + checking whether actions Autovalidator is default implementation... yes Compile OVS in debug mode to have `ovs_assert` statements error out if there is a mis-match in the datapath classifier lookup or packet parser diff --git a/NEWS b/NEWS index cf8e8a290..610cf362b 100644 --- a/NEWS +++ b/NEWS @@ -61,6 +61,7 @@ Post-v2.17.0 implementations available at run time. * Add build time configure command to enable auto-validator as default actions implementation at build time. + * Add AVX512 implementation of actions. - Linux datapath: * Add offloading meter tc police. * Add support for offloading the check_pkt_len action. diff --git a/acinclude.m4 b/acinclude.m4 index 21c505fbd..81e4c5ad2 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -96,6 +96,7 @@ dnl dnl Checks if compiler and binutils supports various AVX512 ISA. AC_DEFUN([OVS_CHECK_AVX512], [ OVS_CHECK_BINUTILS_AVX512 + OVS_CHECK_GCC_AVX512VL OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512f], [HAVE_AVX512F]) OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512bw], [HAVE_AVX512BW]) OVS_CONDITIONAL_CC_OPTION_DEFINE([-mavx512vl], [HAVE_AVX512VL]) diff --git a/lib/cpu.c b/lib/cpu.c index 2df003c51..0292f715e 100644 --- a/lib/cpu.c +++ b/lib/cpu.c @@ -53,6 +53,7 @@ X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 16, OVS_CPU_ISA_X86_AVX512F) X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 30, OVS_CPU_ISA_X86_AVX512BW) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 1, OVS_CPU_ISA_X86_AVX512VBMI) X86_ISA(X86_EXT_FEATURES_LEAF, ECX, 14, OVS_CPU_ISA_X86_VPOPCNTDQ) +X86_ISA(X86_EXT_FEATURES_LEAF, EBX, 31, OVS_CPU_ISA_X86_AVX512VL) #endif bool diff --git a/lib/cpu.h b/lib/cpu.h index 92897bb71..3215229bc 100644 --- a/lib/cpu.h +++ b/lib/cpu.h @@ -25,6 +25,7 @@ enum ovs_cpu_isa { OVS_CPU_ISA_X86_AVX512F, OVS_CPU_ISA_X86_AVX512BW, OVS_CPU_ISA_X86_AVX512VBMI, + OVS_CPU_ISA_X86_AVX512VL, OVS_CPU_ISA_X86_VPOPCNTDQ, OVS_CPU_ISA_X86_LAST = OVS_CPU_ISA_X86_VPOPCNTDQ, }; diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 60f202cad..feccdaa43 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -41,6 +41,14 @@ static struct odp_execute_action_impl action_impls[] = { .name = "scalar", .init_func = odp_action_scalar_init, }, + +#if ACTION_IMPL_AVX512_CHECK + [ACTION_IMPL_AVX512] = { + .available = false, + .name = "avx512", + .init_func = NULL, + }, +#endif }; static void diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 8c2ec3854..dc01a3f9b 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -22,6 +22,14 @@ #include "odp-netlink.h" #include "ovs-atomic.h" +/* Combine all required ISA and Linker checks into a single #define + * for readability and simplicity where the checks are needed. Note + * that it is always #defined, so code must use the #if preprocesor + * directive (not #ifdef). */ +#define ACTION_IMPL_AVX512_CHECK (__x86_64__ && HAVE_AVX512F \ + && HAVE_LD_AVX512_GOOD && __SSE4_2__ && HAVE_AVX512BW && HAVE_AVX512VL \ + && HAVE_GCC_AVX512VL_GOOD) + /* Forward declaration for typedef. */ struct odp_execute_action_impl; @@ -56,6 +64,10 @@ enum odp_execute_action_impl_idx { * Do not change the autovalidator position in this list without updating * the define below. */ +#if ACTION_IMPL_AVX512_CHECK + ACTION_IMPL_AVX512, +#endif + ACTION_IMPL_MAX, }; diff --git a/m4/openvswitch.m4 b/m4/openvswitch.m4 index 4c3bace6e..fe51455b4 100644 --- a/m4/openvswitch.m4 +++ b/m4/openvswitch.m4 @@ -421,6 +421,35 @@ AC_DEFUN([OVS_CHECK_SPHINX], AC_ARG_VAR([SPHINXBUILD]) AM_CONDITIONAL([HAVE_SPHINX], [test "$SPHINXBUILD" != none])]) + +dnl Checks for compiler correctly emitting AVX512-VL vpermd instruction. +dnl GCC5 says it exports AVX512-VL, but it doesn't implement "vpermd" instruction +dnl resulting in compilation failures. To workaround this "reported vs actual" +dnl mismatch, we compile a small snippet, and conditionally enable AVX512-VL. +AC_DEFUN([OVS_CHECK_GCC_AVX512VL], [ + AC_MSG_CHECKING([whether compiler correctly emits AVX512-VL]) + AC_COMPILE_IFELSE( + [AC_LANG_PROGRAM([#include + static void __attribute__((__target__("avx512vl"))) + check_permutexvar(void) + { + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, + 0xF, 0xF, 0xF, + 0xF, 0xF); + v_swap32a = _mm256_permutexvar_epi32(v_swap32a, + v_swap32a); + }],[])], + [AC_MSG_RESULT([yes]) + ovs_cv_gcc_avx512vl_good=yes], + [AC_MSG_RESULT([no]) + ovs_cv_gcc_avx512vl_good=no]) + if test "$ovs_cv_gcc_avx512vl_good" = yes; then + AC_DEFINE([HAVE_GCC_AVX512VL_GOOD], [1], + [Define to 1 if gcc implements the vpermd instruction.]) + fi + AM_CONDITIONAL([HAVE_GCC_AVX512VL_GOOD], + [test "$ovs_cv_gcc_avx512vl_good" = yes])]) + dnl Checks for binutils/assembler known issue with AVX512. dnl Due to backports, we probe assembling a reproducer instead of checking dnl binutils version string. More details, including ASM dumps and debug here: From patchwork Thu Jul 14 17:51:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656585 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=GPrGSDgz; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMXx2Ssqz9sFs for ; Fri, 15 Jul 2022 03:53:21 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 79F5B6166E; Thu, 14 Jul 2022 17:53:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 79F5B6166E Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=GPrGSDgz X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XGeNR22OGCUj; Thu, 14 Jul 2022 17:53:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 532C261662; Thu, 14 Jul 2022 17:53:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 532C261662 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E7549C007E; Thu, 14 Jul 2022 17:53:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 96282C0035 for ; Thu, 14 Jul 2022 17:53:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A0C4F41896 for ; Thu, 14 Jul 2022 17:52:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A0C4F41896 Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=GPrGSDgz X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NWkBCohTSkn4 for ; Thu, 14 Jul 2022 17:52:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CAE534189C Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp2.osuosl.org (Postfix) with ESMTPS id CAE534189C for ; Thu, 14 Jul 2022 17:52:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821153; x=1689357153; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GnpZyZV9e6XOTadjX+ILhKEYNGDBQKyd6M66PdIB6DI=; b=GPrGSDgzHLklSOZZKjguS/q3fBX63u7A6GbwJaE++qEBtjL52EIbP0X5 KUaBAi+7RdBOyqPE5ZeTr13Epu1HPisjAC9XikH+pSiz3hgoQCk+zfObr ops2pQOEickBJbq/Dd7GB8gyJcPFkNyWroH2C9zuS/nled5eoMgZgNCfy s1drFWK8ZGL9HlCT96drYbaNUhf4dPIjXhEa8dSBW7h5fsNjapNrzdjeN ooVaJpMiy/q2MsFDqt+pC/WIuffoN0DRImprhUtATAHzwo9Zm0TMXAq3k rqAIVyboql0Sh50pZEAa7yrCT/w3GAL2DfT3LvGg3vsQ/xpxe19HTJa0E Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265380380" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="265380380" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232209" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:19 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:55 +0000 Message-Id: <20220714175158.3709150-8-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 07/10] odp-execute: Add ISA implementation of pop_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Harry van Haaren This commit adds the AVX512 implementation of the pop_vlan action. Signed-off-by: Emma Finn --- lib/automake.mk | 4 + lib/odp-execute-avx512.c | 186 ++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 32 ++++++- lib/odp-execute-private.h | 4 + 4 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 lib/odp-execute-avx512.c diff --git a/lib/automake.mk b/lib/automake.mk index 5c3b05f6b..a76de6dbf 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -45,6 +45,10 @@ lib_libopenvswitchavx512_la_CFLAGS += \ lib_libopenvswitchavx512_la_SOURCES += \ lib/dpif-netdev-extract-avx512.c \ lib/dpif-netdev-lookup-avx512-gather.c +if HAVE_GCC_AVX512VL_GOOD +lib_libopenvswitchavx512_la_SOURCES += \ + lib/odp-execute-avx512.c +endif # HAVE_GCC_AVX512VL_GOOD endif # HAVE_AVX512VL endif # HAVE_AVX512BW lib_libopenvswitchavx512_la_LDFLAGS = \ diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c new file mode 100644 index 000000000..d929abe68 --- /dev/null +++ b/lib/odp-execute-avx512.c @@ -0,0 +1,186 @@ +/* + * Copyright (c) 2022 Intel. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifdef __x86_64__ +/* Sparse cannot handle the AVX512 instructions. */ +#if !defined(__CHECKER__) + +#include +#include + +#include "dp-packet.h" +#include "immintrin.h" +#include "odp-execute-private.h" +#include "odp-netlink.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(odp_execute_avx512); + +/* The below three build asserts make sure that l2_5_ofs, l3_ofs, and l4_ofs + * fields remain in the same order and offset to l2_padd_size. This is needed + * as the avx512_dp_packet_resize_l2() function will manipulate those fields at + * a fixed memory index based on the l2_padd_size offset. */ +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_pad_size) + + MEMBER_SIZEOF(struct dp_packet, l2_pad_size) == + offsetof(struct dp_packet, l2_5_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l2_5_ofs) + + MEMBER_SIZEOF(struct dp_packet, l2_5_ofs) == + offsetof(struct dp_packet, l3_ofs)); + +BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + + MEMBER_SIZEOF(struct dp_packet, l3_ofs) == + offsetof(struct dp_packet, l4_ofs)); + +/* The below build assert makes sure it's safe to read/write 128-bits starting + * at the l2_pad_size location. */ +BUILD_ASSERT_DECL(sizeof(struct dp_packet) - + offsetof(struct dp_packet, l2_pad_size) >= sizeof(__m128i)); + +static inline void ALWAYS_INLINE +avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) +{ + /* Update packet size/data pointers, same as the scalar implementation. */ + if (resize_by_bytes >= 0) { + dp_packet_push_uninit(b, resize_by_bytes); + } else { + dp_packet_pull(b, -resize_by_bytes); + } + + /* The next step is to update the l2_5_ofs, l3_ofs and l4_ofs fields which + * the scalar implementation does with the dp_packet_adjust_layer_offset() + * function. */ + + /* Set the v_zero register to all zero's. */ + const __m128i v_zeros = _mm_setzero_si128(); + + /* Set the v_u16_max register to all one's. */ + const __m128i v_u16_max = _mm_cmpeq_epi16(v_zeros, v_zeros); + + /* Each lane represents 16 bits in a 12-bit register. In this case the + * first three 16-bit values, which will map to the l2_5_ofs, l3_ofs and + * l4_ofs fields. */ + const uint8_t k_lanes = 0b1110; + + /* Set all 16-bit words in the 128-bits v_offset register to the value we + * need to add/substract from the l2_5_ofs, l3_ofs, and l4_ofs fields. */ + __m128i v_offset = _mm_set1_epi16(abs(resize_by_bytes)); + + /* Load 128 bits from the dp_packet structure starting at the l2_pad_size + * offset. */ + void *adjust_ptr = &b->l2_pad_size; + __m128i v_adjust_src = _mm_loadu_si128(adjust_ptr); + + /* Here is the tricky part, we only need to update the value of the three + * fields if they are not UINT16_MAX. The following function will return + * a mask of lanes (read fields) that are not UINT16_MAX. It will do this + * by comparing only the lanes we requested, k_lanes, and if they match + * v_u16_max, the bit will be set. */ + __mmask8 k_cmp = _mm_mask_cmpneq_epu16_mask(k_lanes, v_adjust_src, + v_u16_max); + + /* Based on the bytes adjust (positive, or negative) it will do the actual + * add or subtraction. These functions will only operate on the lanes + * (fields) requested based on k_cmp, i.e: + * k_cmp = [l2_5_ofs, l3_ofs, l4_ofs] + * for field in kcmp + * v_adjust_src[field] = v_adjust_src[field] + v_offset + */ + __m128i v_adjust_wip; + + if (resize_by_bytes >= 0) { + v_adjust_wip = _mm_mask_add_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } else { + v_adjust_wip = _mm_mask_sub_epi16(v_adjust_src, k_cmp, + v_adjust_src, v_offset); + } + + /* Here we write back the full 128-bits. */ + _mm_storeu_si128(adjust_ptr, v_adjust_wip); +} + +/* This function performs the same operation on each packet in the batch as + * the scalar eth_pop_vlan() function. */ +static void +action_avx512_pop_vlan(struct dp_packet_batch *batch, + const struct nlattr *a OVS_UNUSED) +{ + struct dp_packet *packet; + + /* Set the v_zero register to all zero's. */ + const __m128i v_zeros = _mm_setzero_si128(); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct vlan_eth_header *veh = dp_packet_eth(packet); + + if (veh && dp_packet_size(packet) >= sizeof *veh && + eth_type_vlan(veh->veth_type)) { + + /* Load the first 128-bits of l2 header into the v_ether register. + * This result in the veth_dst/src and veth_type/tci of the + * vlan_eth_header structure to be loaded. */ + __m128i v_ether = _mm_loadu_si128((void *) veh); + + /* This creates a 256-bit value containing the first four fields + * of the vlan_eth_header plus 128 zero-bit. The result will be the + * lowest 128-bits after the right shift, hence we shift the data + * 128(zero)-bits minus the VLAN_HEADER_LEN, so we are left with + * only the veth_dst and veth_src fields. */ + __m128i v_realign = _mm_alignr_epi8(v_ether, v_zeros, + sizeof(__m128i) - + VLAN_HEADER_LEN); + + /* Write back the modified ethernet header. */ + _mm_storeu_si128((void *) veh, v_realign); + + /* As we removed the VLAN_HEADER we now need to adjust all the + * offsets. */ + avx512_dp_packet_resize_l2(packet, -VLAN_HEADER_LEN); + } + } +} + +int +action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) +{ + if (!action_avx512_isa_probe()) { + return -ENOTSUP; + } + + /* Set function pointers for actions that can be applied directly, these + * are identified by OVS_ACTION_ATTR_*. */ + self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + return 0; +} + +#endif /* Sparse */ + +#else /* __x86_64__ */ + +#include +#include "odp-execute-private.h" +/* Function itself is required to be called, even in e.g. 32-bit builds. + * This dummy init function ensures 32-bit builds succeed too. + */ + +int +action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) +{ + return -ENOTSUP; +} + +#endif diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index feccdaa43..265e3205f 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -19,6 +19,7 @@ #include #include +#include "cpu.h" #include "dpdk.h" #include "dp-packet.h" #include "odp-execute-private.h" @@ -29,6 +30,35 @@ VLOG_DEFINE_THIS_MODULE(odp_execute_impl); static int active_action_impl_index; +#if ACTION_IMPL_AVX512_CHECK +/* Probe functions to check ISA requirements. */ +bool +action_avx512_isa_probe(void) +{ + static enum ovs_cpu_isa isa_required[] = { + OVS_CPU_ISA_X86_AVX512F, + OVS_CPU_ISA_X86_AVX512BW, + OVS_CPU_ISA_X86_BMI2, + OVS_CPU_ISA_X86_AVX512VL, + }; + for (int i = 0; i < ARRAY_SIZE(isa_required); i++) { + if (!cpu_has_isa(isa_required[i])) { + return false; + } + } + return true; +} + +#else + +bool +action_avx512_isa_probe(void) +{ + return false; +} + +#endif + static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AUTOVALIDATOR] = { .available = false, @@ -46,7 +76,7 @@ static struct odp_execute_action_impl action_impls[] = { [ACTION_IMPL_AVX512] = { .available = false, .name = "avx512", - .init_func = NULL, + .init_func = action_avx512_init, }, #endif }; diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index dc01a3f9b..5c0c5a25f 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -77,6 +77,8 @@ BUILD_ASSERT_DECL(ACTION_IMPL_AUTOVALIDATOR == 1); #define ACTION_IMPL_BEGIN (ACTION_IMPL_AUTOVALIDATOR + 1) +bool action_avx512_isa_probe(void); + /* Odp execute init handles setting up the state of the actions functions at * initialization time. It cannot return errors, as it must always succeed in * initializing the scalar/generic codepath. */ @@ -90,6 +92,8 @@ struct odp_execute_action_impl * odp_execute_action_set(const char *name); int action_autoval_init(struct odp_execute_action_impl *self); +int action_avx512_init(struct odp_execute_action_impl *self); + void odp_execute_action_get_info(struct ds *name); #endif /* ODP_EXTRACT_PRIVATE */ From patchwork Thu Jul 14 17:51:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656583 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=eN7z3Lq9; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMXk60rwz9sFs for ; Fri, 15 Jul 2022 03:53:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0E7516160F; Thu, 14 Jul 2022 17:53:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0E7516160F Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=eN7z3Lq9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XX3sA9EO4nvm; Thu, 14 Jul 2022 17:53:08 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1AD616163E; Thu, 14 Jul 2022 17:53:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1AD616163E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B25CEC007D; Thu, 14 Jul 2022 17:53:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2892AC0035 for ; Thu, 14 Jul 2022 17:53:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E993341850 for ; Thu, 14 Jul 2022 17:52:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E993341850 Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=eN7z3Lq9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 84eTZq6Q2ger for ; Thu, 14 Jul 2022 17:52:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 33C82418A1 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp2.osuosl.org (Postfix) with ESMTPS id 33C82418A1 for ; Thu, 14 Jul 2022 17:52:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821154; x=1689357154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rwK7Sj1aiSRVy1TFbnB3vfJ+H4EPg6PqoFJVonH9uXU=; b=eN7z3Lq9B/qMI1+6l0lIvH2uxulTadt5s6wExxdNyDmUV7JADv6FrrGp t4fbuKIuybQCGLbzFEq80Hnm3LkG3wG9s41lui+z5LPBly9FLtc/yMq4T Zg41uCGyZta3vRv/07j/y+PN6cBCSeGChNF3sLfiwaBAmKTPmK3+HS2lE DhrjwvXMcfNAItxSzRibgbJbC69nu1LxpmjNtdOQ8k6Rir17FdFVAoB8U 58BPvQ4TJc6WempB2A0ob1SM27NRp6Q/6R4zVAP1hZp7KJHZaa8k5jSrw XVy26JUx24DvR130zhL7NulXgfp7PJ+8s/NL4Z4WudIxzsN2X8rzLbEfL A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265380388" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="265380388" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232216" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:21 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:56 +0000 Message-Id: <20220714175158.3709150-9-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 08/10] odp-execute: Add ISA implementation of push_vlan action. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds the AVX512 implementation of the push_vlan action. Signed-off-by: Emma Finn Acked-by: Eelco Chaudron Acked-by: Eelco Chaudron --- lib/odp-execute-avx512.c | 54 ++++++++++++++++++++++++++++++++++++++++ lib/odp-execute.c | 22 +++++++++------- 2 files changed, 67 insertions(+), 9 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index d929abe68..90a5a7416 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -154,6 +154,58 @@ action_avx512_pop_vlan(struct dp_packet_batch *batch, } } +/* This function performs the same operation on each packet in the batch as + * the scalar eth_push_vlan() function. */ +static void +action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + ovs_be16 tpid, tci; + + /* This shuffle mask is used below, and each position tells where to + * move the bytes to. So here, the fourth byte in v_ether is moved to + * byte location 0 in v_shift. The fifth is moved to 1, etc., etc. + * The 0xFF is special it tells to fill that position with 0. */ + static const uint8_t vlan_push_shuffle_mask[16] = { + 4, 5, 6, 7, 8, 9, 10, 11, + 12, 13, 14, 15, 0xFF, 0xFF, 0xFF, 0xFF + }; + + /* Load the shuffle mask in v_index. */ + __m128i v_index = _mm_loadu_si128((void *) vlan_push_shuffle_mask); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + tpid = vlan->vlan_tpid; + tci = vlan->vlan_tci; + + /* As we are about to insert the VLAN_HEADER we now need to adjust all + * the offsets. */ + avx512_dp_packet_resize_l2(packet, VLAN_HEADER_LEN); + + char *pkt_data = (char *) dp_packet_data(packet); + + /* Build up the VLAN TCI/TPID in a single uint32_t. */ + const uint32_t tci_proc = tci & htons(~VLAN_CFI); + const uint32_t tpid_tci = (tci_proc << 16) | tpid; + + /* Load the first 128-bits of the packet into the v_ether register. + * Note that this includes the 4 unused bytes (VLAN_HEADER_LEN). */ + __m128i v_ether = _mm_loadu_si128((void *) pkt_data); + + /* Move(shuffle) the veth_dst and veth_src data to create room for + * the vlan header. */ + __m128i v_shift = _mm_shuffle_epi8(v_ether, v_index); + + /* Copy(insert) the 32-bit VLAN header, tpid_tci, at the 3rd 32-bit + * word offset, i.e. ofssetof(vlan_eth_header, veth_type) */ + __m128i v_vlan_hdr = _mm_insert_epi32(v_shift, tpid_tci, 3); + + /* Write back the modified ethernet header. */ + _mm_storeu_si128((void *) pkt_data, v_vlan_hdr); + } +} + int action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) { @@ -164,6 +216,8 @@ action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + return 0; } diff --git a/lib/odp-execute.c b/lib/odp-execute.c index ff16e9bf3..a65110138 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -846,6 +846,17 @@ action_pop_vlan(struct dp_packet_batch *batch, } } +static void +action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) +{ + struct dp_packet *packet; + const struct ovs_action_push_vlan *vlan = nl_attr_get(a); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ int @@ -854,6 +865,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) /* Set function pointers for actions that can be applied directly, these * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; + self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; return 0; } @@ -1051,15 +1063,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, break; } - case OVS_ACTION_ATTR_PUSH_VLAN: { - const struct ovs_action_push_vlan *vlan = nl_attr_get(a); - - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - eth_push_vlan(packet, vlan->vlan_tpid, vlan->vlan_tci); - } - break; - } - case OVS_ACTION_ATTR_PUSH_MPLS: { const struct ovs_action_push_mpls *mpls = nl_attr_get(a); @@ -1212,6 +1215,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, case __OVS_ACTION_ATTR_MAX: /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: + case OVS_ACTION_ATTR_PUSH_VLAN: OVS_NOT_REACHED(); } From patchwork Thu Jul 14 17:51:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656584 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=SQMMD7bA; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMXs0qSPz9sFs for ; Fri, 15 Jul 2022 03:53:16 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id EA55A427FC; Thu, 14 Jul 2022 17:53:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EA55A427FC Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=SQMMD7bA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p3YTLmVJTzYE; Thu, 14 Jul 2022 17:53:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 136E2427F0; Thu, 14 Jul 2022 17:53:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 136E2427F0 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D1920C0035; Thu, 14 Jul 2022 17:53:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A48EC0035 for ; Thu, 14 Jul 2022 17:53:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 160D06163A for ; Thu, 14 Jul 2022 17:52:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 160D06163A Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=SQMMD7bA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WsYzLzmySmw7 for ; Thu, 14 Jul 2022 17:52:35 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8A27561630 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8A27561630 for ; Thu, 14 Jul 2022 17:52:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821154; x=1689357154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YXARITG8/mMyxj/RXmrlCg2Y+IcgTw/fG17UJ5JQBQw=; b=SQMMD7bA5yfSPOn/DgNTVpN2fnxdw17lyuMbSHk2Vt9H2FVAwenshvTF 37smqXxR3I3f1Ulo9YSOaOE53s6AGCmaWfGqpJyBRGaOOQormFIpcSeMN dhTgVq10gASnmg2F+wpTbJ1rd0xQP/YTTrlr6i+U6u3yH62Byct7PoHJZ Yzo97g5O0NkcOokcBGmKRTVv9tpgAKKLCIl8RTQ6OT5SfjAinztqQn9B4 Iuu/cMxUID3dEKj1e8m6yLDjtQ/bIb1p+CTqWh7SRbGOvqSXAK2Ab+asK 4X21E/fjYgfkUIWl/wwK0IYfdWA3ugEwJI0bCIr2o2bEb+jA0UVCrjZF/ Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265380398" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="265380398" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232228" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:22 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:57 +0000 Message-Id: <20220714175158.3709150-10-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 09/10] odp-execute: Add ISA implementation of set_masked ETH X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit includes infrastructure changes for enabling set_masked_X actions and also adds support for the AVX512 implementation of the eth_set_addrs action. Signed-off-by: Emma Finn Acked-by: Eelco Chaudron --- lib/odp-execute-avx512.c | 90 +++++++++++++++++++++++++++++++++++++++ lib/odp-execute-private.c | 14 ++++++ lib/odp-execute-private.h | 3 ++ lib/odp-execute.c | 49 +++++++++++---------- lib/odp-execute.h | 3 ++ 5 files changed, 137 insertions(+), 22 deletions(-) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 90a5a7416..02e26cf26 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -23,6 +23,7 @@ #include "dp-packet.h" #include "immintrin.h" +#include "odp-execute.h" #include "odp-execute-private.h" #include "odp-netlink.h" #include "openvswitch/vlog.h" @@ -50,6 +51,16 @@ BUILD_ASSERT_DECL(offsetof(struct dp_packet, l3_ofs) + BUILD_ASSERT_DECL(sizeof(struct dp_packet) - offsetof(struct dp_packet, l2_pad_size) >= sizeof(__m128i)); +/* The below build assert makes sure the order of the fields needed by + * the set masked functions shuffle operations do not change. This should not + * happen as these are defined under the Linux uapi. */ +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_src) + + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_src) == + offsetof(struct ovs_key_ethernet, eth_dst)); + +/* Array of callback functions, one for each masked operation. */ +odp_execute_action_cb impl_set_masked_funcs[__OVS_KEY_ATTR_MAX]; + static inline void ALWAYS_INLINE avx512_dp_packet_resize_l2(struct dp_packet *b, int resize_by_bytes) { @@ -206,6 +217,80 @@ action_avx512_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +/* This function performs the same operation on each packet in the batch as + * the scalar odp_eth_set_addrs() function. */ +static void +action_avx512_eth_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + const struct ovs_key_ethernet *key, *mask; + struct dp_packet *packet; + + a = nl_attr_get(a); + key = nl_attr_get(a); + mask = odp_get_key_mask(a, struct ovs_key_ethernet); + + /* Read the content of the key(src) and mask in the respective registers. + * We only load the src and dest addresses, which is only 96-bits and not + * 128-bits. */ + __m128i v_src = _mm_maskz_loadu_epi32(0x7,(void *) key); + __m128i v_mask = _mm_maskz_loadu_epi32(0x7, (void *) mask); + + + /* These shuffle masks are used below, and each position tells where to + * move the bytes to. So here, the fourth sixth byte in + * ovs_key_ethernet is moved to byte location 0 in v_src/v_mask. + * The seventh is moved to 1, etc., etc. + * This swap is needed to move the src and dest MAC addresses in the + * same order as in the ethernet packet. */ + static const uint8_t eth_shuffle[16] = { + 6, 7, 8, 9, 10, 11, 0, 1, + 2, 3, 4, 5, 0xFF, 0xFF, 0xFF, 0xFF + }; + + /* Load the shuffle mask in v_shuf. */ + __m128i v_shuf = _mm_loadu_si128((void *) eth_shuffle); + + /* Swap the key/mask src and dest addresses to the ethernet order. */ + v_src = _mm_shuffle_epi8(v_src, v_shuf); + v_mask = _mm_shuffle_epi8(v_mask, v_shuf); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + + struct eth_header *eh = dp_packet_eth(packet); + + if (!eh) { + continue; + } + + /* Load the first 128-bits of the packet into the v_ether register. */ + __m128i v_dst = _mm_loadu_si128((void *) eh); + + /* AND the v_mask to the packet data (v_dst). */ + __m128i dst_masked = _mm_andnot_si128(v_mask, v_dst); + + /* OR the new addresses (v_src) with the masked packet addresses + * (dst_masked). */ + __m128i res = _mm_or_si128(v_src, dst_masked); + + /* Write back the modified ethernet addresses. */ + _mm_storeu_si128((void *) eh, res); + } +} + +static void +action_avx512_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) +{ + const struct nlattr *mask = nl_attr_get(a); + enum ovs_key_attr attr_type = nl_attr_type(mask); + + if (attr_type <= OVS_KEY_ATTR_MAX && impl_set_masked_funcs[attr_type]) { + impl_set_masked_funcs[attr_type](batch, a); + } else { + odp_execute_scalar_action(batch, a); + } +} + int action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) { @@ -217,6 +302,11 @@ action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_avx512_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_avx512_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_avx512_set_masked; + + /* Set function pointers for the individual operations supported by the + * SET_MASKED action. */ + impl_set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; return 0; } diff --git a/lib/odp-execute-private.c b/lib/odp-execute-private.c index 265e3205f..bec49206e 100644 --- a/lib/odp-execute-private.c +++ b/lib/odp-execute-private.c @@ -22,6 +22,7 @@ #include "cpu.h" #include "dpdk.h" #include "dp-packet.h" +#include "odp-execute.h" #include "odp-execute-private.h" #include "odp-netlink.h" #include "odp-util.h" @@ -239,6 +240,19 @@ action_autoval_generic(struct dp_packet_batch *batch, const struct nlattr *a) dp_packet_delete_batch(&original_batch, true); } +void +odp_execute_scalar_action(struct dp_packet_batch *batch, + const struct nlattr *action) +{ + enum ovs_action_attr type = nl_attr_type(action); + + if (type <= OVS_ACTION_ATTR_MAX && + action_impls[ACTION_IMPL_SCALAR].funcs[type]) { + + action_impls[ACTION_IMPL_SCALAR].funcs[type](batch, action); + } +} + int action_autoval_init(struct odp_execute_action_impl *self) { diff --git a/lib/odp-execute-private.h b/lib/odp-execute-private.h index 5c0c5a25f..940180c99 100644 --- a/lib/odp-execute-private.h +++ b/lib/odp-execute-private.h @@ -96,4 +96,7 @@ int action_avx512_init(struct odp_execute_action_impl *self); void odp_execute_action_get_info(struct ds *name); +void odp_execute_scalar_action(struct dp_packet_batch *batch, + const struct nlattr *action); + #endif /* ODP_EXTRACT_PRIVATE */ diff --git a/lib/odp-execute.c b/lib/odp-execute.c index a65110138..bb530aa63 100644 --- a/lib/odp-execute.c +++ b/lib/odp-execute.c @@ -562,8 +562,6 @@ odp_execute_set_action(struct dp_packet *packet, const struct nlattr *a) } } -#define get_mask(a, type) ((const type *)(const void *)(a + 1) + 1) - static void odp_execute_masked_set_action(struct dp_packet *packet, const struct nlattr *a) @@ -575,17 +573,17 @@ odp_execute_masked_set_action(struct dp_packet *packet, switch (type) { case OVS_KEY_ATTR_PRIORITY: md->skb_priority = nl_attr_get_u32(a) - | (md->skb_priority & ~*get_mask(a, uint32_t)); + | (md->skb_priority & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_SKB_MARK: md->pkt_mark = nl_attr_get_u32(a) - | (md->pkt_mark & ~*get_mask(a, uint32_t)); + | (md->pkt_mark & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_ETHERNET: odp_eth_set_addrs(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ethernet)); + odp_get_key_mask(a, struct ovs_key_ethernet)); break; case OVS_KEY_ATTR_NSH: { @@ -595,27 +593,27 @@ odp_execute_masked_set_action(struct dp_packet *packet, case OVS_KEY_ATTR_IPV4: odp_set_ipv4(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv4)); + odp_get_key_mask(a, struct ovs_key_ipv4)); break; case OVS_KEY_ATTR_IPV6: odp_set_ipv6(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_ipv6)); + odp_get_key_mask(a, struct ovs_key_ipv6)); break; case OVS_KEY_ATTR_TCP: odp_set_tcp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_tcp)); + odp_get_key_mask(a, struct ovs_key_tcp)); break; case OVS_KEY_ATTR_UDP: odp_set_udp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_udp)); + odp_get_key_mask(a, struct ovs_key_udp)); break; case OVS_KEY_ATTR_SCTP: odp_set_sctp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_sctp)); + odp_get_key_mask(a, struct ovs_key_sctp)); break; case OVS_KEY_ATTR_MPLS: @@ -623,33 +621,33 @@ odp_execute_masked_set_action(struct dp_packet *packet, if (mh) { put_16aligned_be32(&mh->mpls_lse, nl_attr_get_be32(a) | (get_16aligned_be32(&mh->mpls_lse) - & ~*get_mask(a, ovs_be32))); + & ~*odp_get_key_mask(a, ovs_be32))); } break; case OVS_KEY_ATTR_ARP: set_arp(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_arp)); + odp_get_key_mask(a, struct ovs_key_arp)); break; case OVS_KEY_ATTR_ND: odp_set_nd(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_nd)); + odp_get_key_mask(a, struct ovs_key_nd)); break; case OVS_KEY_ATTR_ND_EXTENSIONS: odp_set_nd_ext(packet, nl_attr_get(a), - get_mask(a, struct ovs_key_nd_extensions)); + odp_get_key_mask(a, struct ovs_key_nd_extensions)); break; case OVS_KEY_ATTR_DP_HASH: md->dp_hash = nl_attr_get_u32(a) - | (md->dp_hash & ~*get_mask(a, uint32_t)); + | (md->dp_hash & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_RECIRC_ID: md->recirc_id = nl_attr_get_u32(a) - | (md->recirc_id & ~*get_mask(a, uint32_t)); + | (md->recirc_id & ~*odp_get_key_mask(a, uint32_t)); break; case OVS_KEY_ATTR_TUNNEL: /* Masked data not supported for tunnel. */ @@ -857,6 +855,17 @@ action_push_vlan(struct dp_packet_batch *batch, const struct nlattr *a) } } +static void +action_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) +{ + const struct nlattr *key = nl_attr_get(a); + struct dp_packet *packet; + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + odp_execute_masked_set_action(packet, key); + } +} + /* Implementation of the scalar actions impl init function. Build up the * array of func ptrs here. */ int @@ -866,6 +875,7 @@ odp_action_scalar_init(struct odp_execute_action_impl *self) * are identified by OVS_ACTION_ATTR_*. */ self->funcs[OVS_ACTION_ATTR_POP_VLAN] = action_pop_vlan; self->funcs[OVS_ACTION_ATTR_PUSH_VLAN] = action_push_vlan; + self->funcs[OVS_ACTION_ATTR_SET_MASKED] = action_set_masked; return 0; } @@ -1084,12 +1094,6 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, } break; - case OVS_ACTION_ATTR_SET_MASKED: - DP_PACKET_BATCH_FOR_EACH(i, packet, batch) { - odp_execute_masked_set_action(packet, nl_attr_get(a)); - } - break; - case OVS_ACTION_ATTR_SAMPLE: DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { odp_execute_sample(dp, packet, steal && last_action, a, @@ -1216,6 +1220,7 @@ odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, /* The following actions are handled by the scalar implementation. */ case OVS_ACTION_ATTR_POP_VLAN: case OVS_ACTION_ATTR_PUSH_VLAN: + case OVS_ACTION_ATTR_SET_MASKED: OVS_NOT_REACHED(); } diff --git a/lib/odp-execute.h b/lib/odp-execute.h index 0921ee924..2ba1ec5d2 100644 --- a/lib/odp-execute.h +++ b/lib/odp-execute.h @@ -46,4 +46,7 @@ void odp_execute_actions(void *dp, struct dp_packet_batch *batch, bool steal, const struct nlattr *actions, size_t actions_len, odp_execute_cb dp_execute_action); + +#define odp_get_key_mask(a, type) ((const type *)(const void *)(a + 1) + 1) + #endif From patchwork Thu Jul 14 17:51:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emma Finn X-Patchwork-Id: 1656586 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bkeR4PxP; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LkMY16Gxgz9sFs for ; Fri, 15 Jul 2022 03:53:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id F258E42801; Thu, 14 Jul 2022 17:53:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org F258E42801 Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bkeR4PxP X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OKrWGrCNSo1l; Thu, 14 Jul 2022 17:53:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 57C4741997; Thu, 14 Jul 2022 17:53:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 57C4741997 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id ED3C3C0035; Thu, 14 Jul 2022 17:53:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 227BCC007D for ; Thu, 14 Jul 2022 17:53:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 75B384187B for ; Thu, 14 Jul 2022 17:52:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 75B384187B Authentication-Results: smtp2.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bkeR4PxP X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SF5vR8pq0FS6 for ; Thu, 14 Jul 2022 17:52:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 85CF6417E3 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by smtp2.osuosl.org (Postfix) with ESMTPS id 85CF6417E3 for ; Thu, 14 Jul 2022 17:52:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657821154; x=1689357154; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=n4rn/u3dt6rITjQ+8uMXxgamIJTYJy+4Khg5b5z807o=; b=bkeR4PxPAN159DYyR7uGAmJ3NwDQvjdPA1SR1hqxfcm6pB1syHjoS1by w6oShBhkbBqgz2tlJwlLB62F/Dd6ud5R1b+nzXILs9z6Yzq0wVzkjohiW H7GiL1DLFeqz9VrYWLysm/mmaZ+jT7AJynYhOinsHClLSYfyd28DnbmKo rN8BDgFL07rrMEKhlBjcjpGovIbucZ1G9j0mFqLWdU+OeUtY8EbAko4KJ Wk/RC423fYPemJ7w6nTMjFazZRt0wtVKvQD4tT168eICuu0bJCki+7FQ0 OxQH9HpMpUGWV9Py7GQFywKZLZaaaQG6xP3ku3SfGQnVUG/WkMghFS2ox Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265380404" X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="265380404" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:52:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,272,1650956400"; d="scan'208";a="842232232" Received: from silpixa00401384.ir.intel.com ([10.243.22.75]) by fmsmga006.fm.intel.com with ESMTP; 14 Jul 2022 10:52:25 -0700 From: Emma Finn To: dev@openvswitch.org, echaudro@redhat.com, harry.van.haaren@intel.com, kumar.amber@intel.com Date: Thu, 14 Jul 2022 17:51:58 +0000 Message-Id: <20220714175158.3709150-11-emma.finn@intel.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220714175158.3709150-1-emma.finn@intel.com> References: <20220713182807.3416578-1-harry.van.haaren@intel.com> <20220714175158.3709150-1-emma.finn@intel.com> MIME-Version: 1.0 Cc: i.maximets@ovn.org Subject: [ovs-dev] [v11 10/10] odp-execute: Add ISA implementation of set_masked IPv4 action X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This commit adds support for the AVX512 implementation of the ipv4_set_addrs action as well as an AVX512 implementation of updating the checksums. Signed-off-by: Emma Finn --- lib/odp-execute-avx512.c | 216 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 216 insertions(+) diff --git a/lib/odp-execute-avx512.c b/lib/odp-execute-avx512.c index 02e26cf26..2e0bc32a9 100644 --- a/lib/odp-execute-avx512.c +++ b/lib/odp-execute-avx512.c @@ -21,6 +21,7 @@ #include #include +#include "csum.h" #include "dp-packet.h" #include "immintrin.h" #include "odp-execute.h" @@ -58,6 +59,22 @@ BUILD_ASSERT_DECL(offsetof(struct ovs_key_ethernet, eth_src) + MEMBER_SIZEOF(struct ovs_key_ethernet, eth_src) == offsetof(struct ovs_key_ethernet, eth_dst)); +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_src) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_src) == + offsetof(struct ovs_key_ipv4, ipv4_dst)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_dst) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_dst) == + offsetof(struct ovs_key_ipv4, ipv4_proto)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_proto) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_proto) == + offsetof(struct ovs_key_ipv4, ipv4_tos)); + +BUILD_ASSERT_DECL(offsetof(struct ovs_key_ipv4, ipv4_tos) + + MEMBER_SIZEOF(struct ovs_key_ipv4, ipv4_tos) == + offsetof(struct ovs_key_ipv4, ipv4_ttl)); + /* Array of callback functions, one for each masked operation. */ odp_execute_action_cb impl_set_masked_funcs[__OVS_KEY_ATTR_MAX]; @@ -278,6 +295,204 @@ action_avx512_eth_set_addrs(struct dp_packet_batch *batch, } } +static inline uint16_t ALWAYS_INLINE +avx512_get_delta(__m256i old_header, __m256i new_header) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* These two shuffle masks, v_swap16a and v_swap16b, are to shuffle the + * old and new header to add padding after each 16-bit value for the + * following carry over addition. */ + __m256i v_swap16a = _mm256_setr_epi16(0x0100, 0xFFFF, 0x0302, 0xFFFF, + 0x0504, 0xFFFF, 0x0706, 0xFFFF, + 0x0100, 0xFFFF, 0x0302, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF); + __m256i v_swap16b = _mm256_setr_epi16(0x0908, 0xFFFF, 0x0B0A, 0xFFFF, + 0x0D0C, 0xFFFF, 0x0F0E, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, + 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF); + __m256i v_shuf_old1 = _mm256_shuffle_epi8(old_header, v_swap16a); + __m256i v_shuf_old2 = _mm256_shuffle_epi8(old_header, v_swap16b); + __m256i v_shuf_new1 = _mm256_shuffle_epi8(new_header, v_swap16a); + __m256i v_shuf_new2 = _mm256_shuffle_epi8(new_header, v_swap16b); + + /* Add each part of the old and new headers together. */ + __m256i v_delta1 = _mm256_add_epi32(v_shuf_old1, v_shuf_new1); + __m256i v_delta2 = _mm256_add_epi32(v_shuf_old2, v_shuf_new2); + + /* Add old and new header. */ + __m256i v_delta = _mm256_add_epi32(v_delta1, v_delta2); + + /* Perform horizontal add to go from 8x32-bits to 2x32-bits. */ + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + + /* Shuffle 32-bit value from 3rd lane into first lane for final + * horizontal add. */ + __m256i v_swap32a = _mm256_setr_epi32(0x0, 0x4, 0xF, 0xF, + 0xF, 0xF, 0xF, 0xF); + v_delta = _mm256_permutexvar_epi32(v_swap32a, v_delta); + + v_delta = _mm256_hadd_epi32(v_delta, v_zeros); + v_delta = _mm256_hadd_epi16(v_delta, v_zeros); + + /* Extract delta value. */ + delta = _mm256_extract_epi16(v_delta, 0); + + return delta; +} + +/* This function will calculate the csum delta for the IPv4 addresses in the + * new_header and old_header, assuming the csum field on the new_header was + * updated. */ +static inline uint16_t ALWAYS_INLINE +avx512_ipv4_addr_csum_delta(__m256i old_header, __m256i new_header) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* Set the v_ones register to all one's. */ + __m256i v_ones = _mm256_cmpeq_epi16(v_zeros, v_zeros); + + /* Combine the old and new header, i.e. adding in the new IP addresses + * in the old header (oh). This is done by using the 0x03C 16-bit mask, + * picking 16-bit word 7 till 10. */ + __m256i v_blend_new = _mm256_mask_blend_epi16(0x03C0, old_header, + new_header); + + /* Invert the old_header register. */ + old_header =_mm256_andnot_si256(old_header, v_ones); + + /* Calculate the delta between the old and new header. */ + delta = avx512_get_delta(old_header, v_blend_new); + + return delta; + +} + +/* This function will calculate the csum delta between the new_header and + * old_header, assuming the csum field on the new_header was not yet updated + * or reset. It also assumes headers contain the first 20-bytes of the IPv4 + * header data, and the rest is zeroed out. */ +static inline uint16_t ALWAYS_INLINE +avx512_ipv4_hdr_csum_delta(__m256i old_header, __m256i new_header) +{ + __m256i v_zeros = _mm256_setzero_si256(); + uint16_t delta; + + /* Set the v_ones register to all one's. */ + __m256i v_ones = _mm256_cmpeq_epi16(v_zeros, v_zeros); + + /* Invert the old_header register. */ + old_header =_mm256_andnot_si256(old_header, v_ones); + + /* Calculate the delta between the old and new header. */ + delta = avx512_get_delta(old_header, new_header); + + return delta; +} + +/* This function performs the same operation on each packet in the batch as + * the scalar odp_set_ipv4() function. */ +static void +action_avx512_ipv4_set_addrs(struct dp_packet_batch *batch, + const struct nlattr *a) +{ + const struct ovs_key_ipv4 *key, *mask; + struct dp_packet *packet; + a = nl_attr_get(a); + key = nl_attr_get(a); + mask = odp_get_key_mask(a, struct ovs_key_ipv4); + + /* Read the content of the key(src) and mask in the respective registers. + * We only load the size of the actual structure, which is only 96-bits. */ + __m256i v_key = _mm256_maskz_loadu_epi32(0x7, (void *) key); + __m256i v_mask = _mm256_maskz_loadu_epi32(0x7, (void *) mask); + + /* This two shuffle masks, v_shuf32, v_shuffle, are to shuffle key and + * mask to match the ip_header structure layout. */ + static const uint8_t ip_shuffle_mask[32] = { + 0xFF, 0x05, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, + 0x06, 0xFF, 0xFF, 0xFF, 0x00, 0x01, 0x02, 0x03, + 0x00, 0x01, 0x02, 0x03, 0xFF, 0xFF, 0xFF, 0xFF, + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; + + __m256i v_shuf32 = _mm256_setr_epi32(0x0, 0x2, 0xF, 0xF, + 0x1, 0xF, 0xF, 0xF); + + __m256i v_shuffle = _mm256_loadu_si256((void *) ip_shuffle_mask); + + /* Two shuffles are required for key and mask to match the layout of + * the ip_header struct. The _shuffle_epi8 only works within 128-bit + * lanes, so a permute is required to move src and dst into the correct + * lanes. And then a shuffle is used to move the fields into the right + * order. */ + __m256i v_key_shuf = _mm256_permutexvar_epi32(v_shuf32, v_key); + v_key_shuf = _mm256_shuffle_epi8(v_key_shuf, v_shuffle); + + __m256i v_mask_shuf = _mm256_permutexvar_epi32(v_shuf32, v_mask); + v_mask_shuf = _mm256_shuffle_epi8(v_mask_shuf, v_shuffle); + + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + struct ip_header *nh = dp_packet_l3(packet); + ovs_be16 old_csum = ~nh->ip_csum; + + /* Load the 20 bytes of the IPv4 header. Without options, which is the + * most common case it's 20 bytes, but can be up to 60 bytes. */ + __m256i v_packet = _mm256_maskz_loadu_epi32(0x1F, (void *) nh); + + /* AND the v_pkt_mask to the packet data (v_packet). */ + __m256i v_pkt_masked = _mm256_andnot_si256(v_mask_shuf, v_packet); + + /* OR the new addresses (v_key_shuf) with the masked packet addresses + * (v_pkt_masked). */ + __m256i v_new_hdr = _mm256_or_si256(v_key_shuf, v_pkt_masked); + + /* Update the IP checksum based on updated IP values. */ + uint16_t delta = avx512_ipv4_hdr_csum_delta(v_packet, v_new_hdr); + uint32_t new_csum = old_csum + delta; + delta = csum_finish(new_csum); + + /* Insert new checksum. */ + v_new_hdr = _mm256_insert_epi16(v_new_hdr, delta, 5); + + /* If ip_src or ip_dst has been modified, L4 checksum needs to + * be updated too. */ + if (mask->ipv4_src || mask->ipv4_dst) { + + uint16_t delta_checksum = avx512_ipv4_addr_csum_delta(v_packet, + v_new_hdr); + + if (nh->ip_proto == IPPROTO_UDP) { + /* New UDP checksum. */ + struct udp_header *uh = dp_packet_l4(packet); + if (uh->udp_csum) { + uint16_t old_udp_checksum = ~uh->udp_csum; + uint32_t udp_checksum = old_udp_checksum + delta_checksum; + udp_checksum = csum_finish(udp_checksum); + + if (!udp_checksum) { + udp_checksum = htons(0xffff); + } + /* Insert new udp checksum. */ + uh->udp_csum = udp_checksum; + } + } else if (nh->ip_proto == IPPROTO_TCP) { + /* New TCP checksum. */ + struct tcp_header *th = dp_packet_l4(packet); + uint16_t old_tcp_checksum = ~th->tcp_csum; + uint32_t tcp_checksum = old_tcp_checksum + delta_checksum; + tcp_checksum = csum_finish(tcp_checksum); + + th->tcp_csum = tcp_checksum; + } + } + /* Write back the modified IPv4 addresses. */ + _mm256_mask_storeu_epi32((void *) nh, 0x1F, v_new_hdr); + } +} + static void action_avx512_set_masked(struct dp_packet_batch *batch, const struct nlattr *a) { @@ -307,6 +522,7 @@ action_avx512_init(struct odp_execute_action_impl *self OVS_UNUSED) /* Set function pointers for the individual operations supported by the * SET_MASKED action. */ impl_set_masked_funcs[OVS_KEY_ATTR_ETHERNET] = action_avx512_eth_set_addrs; + impl_set_masked_funcs[OVS_KEY_ATTR_IPV4] = action_avx512_ipv4_set_addrs; return 0; }