From patchwork Wed Mar 29 07:20:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Mi X-Patchwork-Id: 1762599 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=ZE5RcQzJ; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4PmdK84MqPz1yYr for ; Wed, 29 Mar 2023 18:21:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6297341CCD; Wed, 29 Mar 2023 07:21:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6297341CCD Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (2048-bit key, unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=ZE5RcQzJ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5HDem5RMC5Cn; Wed, 29 Mar 2023 07:21:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id DF8AF41CB0; Wed, 29 Mar 2023 07:21:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org DF8AF41CB0 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AE64CC0037; Wed, 29 Mar 2023 07:21:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D2DBC002F for ; Wed, 29 Mar 2023 07:21:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D84A841D9B for ; Wed, 29 Mar 2023 07:21:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D84A841D9B Authentication-Results: smtp4.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=Nvidia.com header.i=@Nvidia.com header.a=rsa-sha256 header.s=selector2 header.b=ZE5RcQzJ X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s7lbytHPNdt0 for ; Wed, 29 Mar 2023 07:21:39 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3B2AB41D85 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2062e.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe59::62e]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3B2AB41D85 for ; Wed, 29 Mar 2023 07:21:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aglqqshtLI6tbZ/zvRJnvEO4wmUFhukMdoJs28e1fhATy7plU+GbBbsFhRoXNo5kPUyYHvsFa0fQVaRkgsfB9jURFOfI4t8lBqs99aeWhBimPrXLaa81GtfTb6aHccHKYYXeCqI2l9U2ARtuJat+86xl69y5nF5/qxjpv4H+8L2Cdr1mRDy+Z5muDqdH5tA09HQ9sMbQgMkYfqfTNYBHIo3fn0+8XXXSqC6YocY+Wq7TTwik2hN0t976D/yh4OLVqEUj7n0/cNj47Su3ERqoRN7FNlVk5JTnISrG+fwnalo6jydmuqIMzLhBu4uT2VcJ1hF/A5McOEVcVGQ8Ynv4HA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w+Hb87Y0fuqUeswDjZ0ZisNb8b/W9KxlCnQAK7k2DI8=; b=KILQ6RFpk6cQn2bS4WtoOOyKpjDl+qoKjMoWUQ8az5UcO0a2aPQzzFW+JTG6S5T4iDYf2lzPi3WY5XZYHZdDnpv77gzzw+BY6kcUsaqczcnGpn0x8z7naK5X8Wi6PsoegBPhmhMPyD8xM/g31wdIsoHXM3DFs2bF38Hh7Y8mr6MffGlWCXpVWAi+1AbobyoJ2v0ravKcn/eLW+B7qK7rQHHGh+ofR2LCQtg3Pp8GETNgAyB2HqX1TDE1bVhGBxKPJ5pj+1zpWgPvngNF0Vrvj3sBYnGfncfAJH/XJHHydEj7ewZ6m8eg3IEGWDqvuaxUoHmLjcaN2NhRZjvoFJaQ2g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=openvswitch.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w+Hb87Y0fuqUeswDjZ0ZisNb8b/W9KxlCnQAK7k2DI8=; b=ZE5RcQzJaido6lScyWqS/ahB+e8aQarfLQTFA30r1EUCAl6xou8f2r6K83UxyR7D1HEEkm7tvWTnTJ/3B9i6UR2nmtCKyIgl6gyYDSOGAe4LailT37MWAVHGvOsZ96mS//OQqxV6f6Fi/ZV276hDTe4lSlKKrCsjQ4PcjrytYJySLpmtsxiKAzMaSRrXa+5Wnt67hAc4sy8kOkd6dDkQq7dp/3wMwpIGlUlBQeNI7x7nzof4JD1HA5J+MieHRsEmXg+rSt2mKWsOhM4OqTXW4BWHEZ1QLaDMyb+ErVCOqMTr/mtrERZIhesQj7ngn4OvQfvAl51mabFvsm3/q+rumA== Received: from BLAPR03CA0024.namprd03.prod.outlook.com (2603:10b6:208:32b::29) by DS7PR12MB5981.namprd12.prod.outlook.com (2603:10b6:8:7c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6222.35; Wed, 29 Mar 2023 07:21:36 +0000 Received: from BL02EPF000100D3.namprd05.prod.outlook.com (2603:10b6:208:32b:cafe::35) by BLAPR03CA0024.outlook.office365.com (2603:10b6:208:32b::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.20 via Frontend Transport; Wed, 29 Mar 2023 07:21:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL02EPF000100D3.mail.protection.outlook.com (10.167.241.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.30 via Frontend Transport; Wed, 29 Mar 2023 07:21:34 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Wed, 29 Mar 2023 00:21:16 -0700 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Wed, 29 Mar 2023 00:21:15 -0700 Received: from c-237-115-60-065.mtl.labs.mlnx (10.127.8.10) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.5 via Frontend Transport; Wed, 29 Mar 2023 00:21:12 -0700 To: Date: Wed, 29 Mar 2023 10:20:54 +0300 Message-ID: <20230329072059.78569-4-cmi@nvidia.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20230329072059.78569-1-cmi@nvidia.com> References: <20230329072059.78569-1-cmi@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF000100D3:EE_|DS7PR12MB5981:EE_ X-MS-Office365-Filtering-Correlation-Id: 84499ac6-67cc-4fc8-d421-08db30263a62 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0RljLN+danbt+D6kOCe+xxKnEnaDuafnIudV9oz6WMOOgWiASXmBkZdoMxRMFqgiba2PSt9dheJhXZrx8RVKyDxqidrpf1XGBqrT8fL6ey0GuM3CcmndEkW2XZmZUWkhj+IAGo8Q7sz7tTudO4BzhMT5cHNz4cgLaPI5tRILgffvMbvGo/p7c+Lt44459wbsIrdeWVnbQzKW61Qfot/aw+YOsfn2gk9Z9dVwlWoVbbxsPJ29U0iKDEQetU9xfOTWtARA6AU3Dbed4FoR7DLgkswL7jOF+aWalocdjxeg3Z1U3+8poiDGHHmn14MGC7H5upwn7TzC7mGHFtZ2UZrEGnCazmT7kQTNDCkZDyCVe8a542AOREj4pG1+LxTZRKIBUheTauGKEbp7KIoGNg5dzNZeLt8YvL8bTyLJT3czNZz3s/yg+QJkVKQA+7tn0+L1Sg8AQjzv7xQdWfw1ehPsjhPtVE3NJSu+7s5i2psj5N/B4QG5Yk7LrLYKL1RoZbn//NPv4RifSp+YVJFfwR4QsErduNL7OOglo8b6BgC+iHdRY/oKfhaQbwbG/PcH2b5LsERaVb+WcIPGZ+MLNwVRGfrb42K49/KqV1vvPJIsQWBDuHyNATe6iy+TQKoAnfL0HJY83GTLpbfyF5I1Tn/vE7VBcpj/HQfqEg3N7r8dhXfxEloUlQWSxn9kft276mdxB1aWwObz12GOQtfxpa6JMP7gz2sCI0mi6XsU1mDDodCGyy96ji4bgvTweMyi9lMT X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(396003)(376002)(346002)(136003)(451199021)(46966006)(36840700001)(40470700004)(426003)(83380400001)(2616005)(336012)(47076005)(26005)(40480700001)(1076003)(82310400005)(86362001)(186003)(356005)(7636003)(107886003)(82740400003)(36860700001)(41300700001)(8936002)(36756003)(5660300002)(6666004)(70586007)(478600001)(70206006)(4326008)(6916009)(8676002)(40460700003)(54906003)(316002)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Mar 2023 07:21:34.7471 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84499ac6-67cc-4fc8-d421-08db30263a62 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF000100D3.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5981 Cc: elibr@nvidia.com, simon.horman@corigine.com, roniba@nvidia.com, i.maximets@ovn.org, konguyen@redhat.com, majd@nvidia.com, maord@nvidia.com Subject: [ovs-dev] [PATCH v25 3/8] netdev-offload-tc: Introduce group ID management API X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Chris Mi via dev From: Chris Mi Reply-To: Chris Mi Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When offloading sample action to TC, userspace creates a unique ID to map sample action and tunnel info and passes this ID to kernel instead of the sample info. Kernel will send this ID and sampled packet to userspace. Using the ID, userspace can recover the sample info and send sampled packet to the right sample monitoring host. Signed-off-by: Chris Mi Reviewed-by: Roi Dayan --- lib/netdev-offload-tc.c | 118 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) diff --git a/lib/netdev-offload-tc.c b/lib/netdev-offload-tc.c index 247c1ff8b..6ac40b663 100644 --- a/lib/netdev-offload-tc.c +++ b/lib/netdev-offload-tc.c @@ -41,6 +41,7 @@ #include "unaligned.h" #include "util.h" #include "dpif-provider.h" +#include "cmap.h" VLOG_DEFINE_THIS_MODULE(netdev_offload_tc); @@ -103,6 +104,107 @@ static void parse_tc_flower_to_stats(struct tc_flower *flower, static int get_ufid_adjust_stats(const ovs_u128 *ufid, struct dpif_flow_stats *stats); +/* When offloading sample action to TC, userspace creates a unique ID + * to map sample action and tunnel info and passes this ID to kernel + * instead of the sample info. Kernel will send this ID and sampled + * packet to userspace. Using the ID, userspace can recover the sample + * info and send sampled packet to the right sample monitoring host. */ +struct offload_sample { + uint16_t type; /* enum user_action_cookie_type. */ + struct nlattr *action; /* Sample action. Used in flow_get. */ + struct nlattr *userdata; /* Struct user_action_cookie. */ + struct nlattr *actions; /* All actions to get output tunnel. */ + struct flow_tnl *tunnel; /* Input tunnel. */ +}; + +/* This maps a sample group ID to struct offload_sample. */ +struct sgid_node { + struct cmap_node id_node; + uint32_t id; + struct offload_sample sample; +}; + +static struct ovs_mutex sgid_lock = OVS_MUTEX_INITIALIZER; +static struct cmap sgid_map = CMAP_INITIALIZER; +static struct id_pool *sample_group_ids OVS_GUARDED_BY(sgid_lock); + +static void +sgid_node_free(struct sgid_node *node) +{ + if (node) { + free(node->sample.tunnel); + free(node->sample.action); + free(node->sample.actions); + free(node->sample.userdata); + free(node); + } +} + +static struct sgid_node * +sgid_find(uint32_t id) +{ + const struct cmap_node *node = cmap_find(&sgid_map, id); + + return node ? CONTAINER_OF(node, struct sgid_node, id_node) : NULL; +} + +static void +offload_sample_clone(struct offload_sample *new, + const struct offload_sample *old) +{ + new->type = old->type; + new->action = xmemdup(old->action, old->action->nla_len); + new->actions = old->actions + ? xmemdup(old->actions, old->actions->nla_len) + : NULL; + new->userdata = xmemdup(old->userdata, old->userdata->nla_len); + new->tunnel = old->tunnel + ? xmemdup(old->tunnel, sizeof *old->tunnel) + : NULL; +} + +/* Allocate a unique group id for the given set of flow metadata. The id + * space is 2^^32 - 1. 0 is reserved. */ +static uint32_t +sgid_alloc(const struct offload_sample *sample) +{ + struct sgid_node *node = xzalloc(sizeof *node); + + offload_sample_clone(&node->sample, sample); + ovs_mutex_lock(&sgid_lock); + if (!id_pool_alloc_id(sample_group_ids, &node->id)) { + VLOG_ERR("Can't find a free sample group ID"); + sgid_node_free(node); + ovs_mutex_unlock(&sgid_lock); + return 0; + } + cmap_insert(&sgid_map, &node->id_node, node->id); + ovs_mutex_unlock(&sgid_lock); + + return node ? node->id : 0; +} + +static void +sgid_free(uint32_t id) +{ + struct sgid_node *node; + + if (!id) { + return; + } + + ovs_mutex_lock(&sgid_lock); + node = sgid_find(id); + if (node) { + id_pool_free_id(sample_group_ids, node->id); + cmap_remove(&sgid_map, &node->id_node, node->id); + ovsrcu_postpone(sgid_node_free, node); + } else { + VLOG_ERR("Freeing nonexistent sample group ID: %"PRIu32, id); + } + ovs_mutex_unlock(&sgid_lock); +} + static bool is_internal_port(const char *type) { @@ -192,12 +294,17 @@ static struct netlink_field set_flower_map[][4] = { static struct ovs_mutex ufid_lock = OVS_MUTEX_INITIALIZER; +#define MAX_TC_SAMPLES_PER_FLOW 1 + /** * struct ufid_tc_data - data entry for ufid-tc hashmaps. * @ufid_to_tc_node: Element in @ufid_to_tc hash table by ufid key. * @tc_to_ufid_node: Element in @tc_to_ufid hash table by tcf_id key. * @ufid: ufid assigned to the flow * @id: tc filter id (tcf_id) + * @sample_group_id: mapping id for sample actions. Currently only support + sFlow. Use array to support other sample types in the + future. * @netdev: netdev associated with the tc rule * @adjust_stats: When flow gets updated with new actions, we need to adjust * the reported stats to include previous values as the hardware @@ -208,6 +315,7 @@ struct ufid_tc_data { struct hmap_node tc_to_ufid_node; ovs_u128 ufid; struct tcf_id id; + uint32_t sample_group_id[MAX_TC_SAMPLES_PER_FLOW]; struct netdev *netdev; struct dpif_flow_stats adjust_stats; }; @@ -231,6 +339,8 @@ del_ufid_tc_mapping_unlocked(const ovs_u128 *ufid) hmap_remove(&ufid_to_tc, &data->ufid_to_tc_node); hmap_remove(&tc_to_ufid, &data->tc_to_ufid_node); netdev_close(data->netdev); + sgid_free(data->sample_group_id[0]); + data->sample_group_id[0] = 0; free(data); } @@ -2128,6 +2238,11 @@ netdev_tc_parse_nl_actions(struct netdev *netdev, struct tc_flower *flower, if (err) { return err; } + } else if (nl_attr_type(nla) == OVS_ACTION_ATTR_SAMPLE) { + struct offload_sample sample; + + memset(&sample, 0, sizeof sample); + sgid_alloc(&sample); } else { VLOG_DBG_RL(&rl, "unsupported put action type: %d", nl_attr_type(nla)); @@ -2871,6 +2986,9 @@ netdev_tc_init_flow_api(struct netdev *netdev) tc_cleanup_policer_actions(meter_police_ids, METER_POLICE_IDS_BASE, METER_POLICE_IDS_MAX); ovs_mutex_unlock(&meter_police_ids_mutex); + ovs_mutex_lock(&sgid_lock); + sample_group_ids = id_pool_create(1, UINT32_MAX - 1); + ovs_mutex_unlock(&sgid_lock); ovsthread_once_done(&once); }