From patchwork Mon Dec 28 09:25:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: taoyunxiang X-Patchwork-Id: 1420990 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cmss.chinamobile.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4D4Qd60XlFz9sVv for ; Tue, 29 Dec 2020 05:12:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A63068751C; Mon, 28 Dec 2020 18:12:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pFaFb38ndyWS; Mon, 28 Dec 2020 18:12:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 9BB1687293; Mon, 28 Dec 2020 18:11:42 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4C2BDC1EA3; Mon, 28 Dec 2020 18:11:42 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A397CC0893 for ; Mon, 28 Dec 2020 09:26:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 8B1CC85F50 for ; Mon, 28 Dec 2020 09:26:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lBTYmhYXEEke for ; Mon, 28 Dec 2020 09:26:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from cmccmta3.chinamobile.com (cmccmta3.chinamobile.com [221.176.66.81]) by fraxinus.osuosl.org (Postfix) with ESMTP id E8E8385F4E for ; Mon, 28 Dec 2020 09:26:17 +0000 (UTC) Received: from spf.mail.chinamobile.com (unknown[172.16.121.3]) by rmmx-syy-dmz-app10-12010 (RichMail) with SMTP id 2eea5fe9a4aaa10-c3410; Mon, 28 Dec 2020 17:26:02 +0800 (CST) X-RM-TRANSID: 2eea5fe9a4aaa10-c3410 X-RM-TagInfo: emlType=0 X-RM-SPAM-FLAG: 00000000 Received: from yun.localdomain (unknown[112.25.154.146]) by rmsmtp-syy-appsvr02-12002 (RichMail) with SMTP id 2ee25fe9a48a90b-7b916; Mon, 28 Dec 2020 17:26:02 +0800 (CST) X-RM-TRANSID: 2ee25fe9a48a90b-7b916 From: Tao YunXiang To: ovs-dev@openvswitch.org Date: Mon, 28 Dec 2020 01:25:15 -0800 Message-Id: <20201228092520.11807-58-taoyunxiang@cmss.chinamobile.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201228092520.11807-1-taoyunxiang@cmss.chinamobile.com> References: <8> <20201228092520.11807-1-taoyunxiang@cmss.chinamobile.com> MIME-Version: 1.0 X-Mailman-Approved-At: Mon, 28 Dec 2020 18:10:34 +0000 Cc: Taoyunxiang Subject: [ovs-dev] [PATCH 57/62] netdev-offload-dpdk: Add flow_flush api X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Taoyunxiang Code Source From: Self Code Description: This commit is to solve HW flow delete error when SW call port delete. Now when we del port on bridge, all the offloaded flows correspond to this port has to be flushed. Also we add netdev name into ufid_to_rte_flow mapping, so we can flush SW cmap node when we delete HW flows Jira: #[Optional] 市场项目编号（名称）：[Optional] --- lib/dpif-netdev.c | 20 +++++++++++++++++--- lib/netdev-dpdk.c | 13 +++++++++++++ lib/netdev-offload-dpdk.c | 45 +++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 73 insertions(+), 5 deletions(-) diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 85e1fe4..1eaad3e 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -2066,6 +2066,18 @@ do_del_port(struct dp_netdev *dp, struct dp_netdev_port *port) hmap_remove(&dp->ports, &port->node); seq_change(dp->port_seq); + if (netdev_is_flow_api_enabled()) { + int error; + /*Before del port, we should delete correspond hw flows*/ + error = netdev_flow_flush(port->netdev); + if (error) { + VLOG_ERR("Failed to flush hw flow on port %s!\n", + port->netdev->name); + } else { + VLOG_INFO("Success to flush hw flow on port %s!\n", + port->netdev->name); + } + } reconfigure_datapath(dp); port_destroy(port); @@ -2513,7 +2525,6 @@ dp_netdev_flow_offload_put(struct dp_flow_offload_item *offload) info.ori_nw_dst = offload->ori_nw_dst; info.ct_enable = false; info.group_id = 0; - info.dpif_class = pmd->dp->class; port = netdev_ports_get(in_port, dpif_type_str); if (!port) { @@ -2596,8 +2607,9 @@ dp_netdev_flow_offload_main(void *data OVS_UNUSED) OVS_NOT_REACHED(); } - VLOG_DBG("%s to %s netdev flow\n", - ret == 0 ? "succeed" : "failed", op); + VLOG_DBG("%s to %s netdev flow "UUID_FMT"\n", + ret == 0 ? "succeed" : "failed", op, + UUID_ARGS((struct uuid *) &offload->flow->ufid)); dp_netdev_free_flow_offload(offload); ovsrcu_quiesce(); } @@ -3650,6 +3662,8 @@ dp_netdev_flow_add(struct dp_netdev_pmd_thread *pmd, ds_put_cstr(&ds, "flow_add: "); odp_format_ufid(ufid, &ds); + ds_put_cstr(&ds, " mega_"); + odp_format_ufid(&flow->mega_ufid, &ds); ds_put_cstr(&ds, " "); odp_flow_format(key_buf.data, key_buf.size, mask_buf.data, mask_buf.size, diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 9f382ea..5c5edb9 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -5422,6 +5422,19 @@ netdev_dpdk_rte_flow_destroy(struct netdev *netdev, return ret; } +int +netdev_dpdk_rte_flow_flush(struct netdev *netdev, + struct rte_flow_error *error) +{ + struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); + int ret; + + ovs_mutex_lock(&dev->mutex); + ret = rte_flow_flush(dev->port_id, error); + ovs_mutex_unlock(&dev->mutex); + return ret; +} + struct rte_flow * netdev_dpdk_rte_flow_create(struct netdev *netdev, const struct rte_flow_attr *attr, diff --git a/lib/netdev-offload-dpdk.c b/lib/netdev-offload-dpdk.c index 9fc8cb2..bef2349 100644 --- a/lib/netdev-offload-dpdk.c +++ b/lib/netdev-offload-dpdk.c @@ -20,6 +20,7 @@ #include #include +#include #include #include @@ -32,6 +33,7 @@ #include "openvswitch/vlog.h" #include "packets.h" #include "uuid.h" +#include "netdev-vport-private.h" #include #include "id-pool.h" #include "odp-util.h" @@ -68,6 +70,7 @@ struct ufid_to_rte_flow_data { struct rte_flow *rte_flow; bool actions_offloaded; struct dpif_flow_stats stats; + char *netdev_name; }; /* maybe little-endian */ @@ -126,7 +129,9 @@ ufid_to_rte_flow_data_find(const ovs_u128 *ufid) static inline void ufid_to_rte_flow_associate(const ovs_u128 *ufid, - struct rte_flow *rte_flow, bool actions_offloaded) + struct rte_flow *rte_flow, + bool actions_offloaded, + char *netdev_name) { size_t hash = hash_bytes(ufid, sizeof *ufid, 0); struct ufid_to_rte_flow_data *data = xzalloc(sizeof *data); @@ -146,6 +151,7 @@ ufid_to_rte_flow_associate(const ovs_u128 *ufid, data->ufid = *ufid; data->rte_flow = rte_flow; data->actions_offloaded = actions_offloaded; + data->netdev_name = netdev_name; cmap_insert(&ufid_to_rte_flow, CONST_CAST(struct cmap_node *, &data->node), hash); @@ -170,6 +176,22 @@ ufid_to_rte_flow_disassociate(const ovs_u128 *ufid) UUID_ARGS((struct uuid *) ufid)); } +static inline void +ufid_to_rte_flow_disassociate_by_name(char *netdev_name) +{ + struct ufid_to_rte_flow_data *data; + + CMAP_FOR_EACH (data, node, &ufid_to_rte_flow) { + if (!strcmp(netdev_name, data->netdev_name)) { + size_t hash = hash_bytes(&(data->ufid), sizeof(ovs_u128), 0); + cmap_remove(&ufid_to_rte_flow, + CONST_CAST(struct cmap_node *, &data->node), hash); + ovsrcu_postpone(free, data); + } + } + + VLOG_INFO("%s is not associated with any rte flow\n", netdev_name); +} /* * To avoid individual xrealloc calls for each new element, a 'curent_max' * is used to keep track of current allocated number of elements. Starts @@ -2227,7 +2249,7 @@ netdev_offload_dpdk_add_flow(struct dpif *dpif, struct netdev *netdev, ret = -1; goto out; } - ufid_to_rte_flow_associate(ufid, flow, actions_offloaded); + ufid_to_rte_flow_associate(ufid, flow, actions_offloaded,netdev->name); VLOG_DBG("%s: installed flow %p by ufid "UUID_FMT"\n", netdev_get_name(netdev), flow, UUID_ARGS((struct uuid *)ufid)); @@ -2446,6 +2468,24 @@ netdev_offload_dpdk_flow_del(struct netdev *netdev, const ovs_u128 *ufid, } static int +netdev_offload_dpdk_flow_flush(struct netdev *netdev) +{ + int ret = -1; + struct rte_flow_error error; + + ret = netdev_dpdk_rte_flow_flush(netdev, &error); + if (ret == 0) { + ufid_to_rte_flow_disassociate_by_name(netdev->name); + VLOG_DBG("removed rte flow associated with %s\n", + netdev_get_name(netdev)); + } else { + VLOG_ERR("%s: rte flow flush error: %u : message : %s\n", + netdev_get_name(netdev), error.type, error.message); + } + return ret; +} + +static int netdev_offload_dpdk_init_flow_api(struct netdev *netdev) { return netdev_dpdk_flow_api_supported(netdev) ? 0 : EOPNOTSUPP; @@ -2547,6 +2587,7 @@ const struct netdev_flow_api netdev_offload_dpdk = { .type = "dpdk_flow_api", .flow_put = netdev_offload_dpdk_flow_put, .flow_del = netdev_offload_dpdk_flow_del, + .flow_flush = netdev_offload_dpdk_flow_flush, .init_flow_api = netdev_offload_dpdk_init_flow_api, .flow_get = netdev_offload_dpdk_flow_get, .flow_dump_create = netdev_offload_dpdk_flow_dump_create,