From patchwork Fri Nov 1 14:23:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="MiW1zCH0"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474PbQ6mTBz9sP4 for ; Sat, 2 Nov 2019 01:24:50 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 8ACCA18B3; Fri, 1 Nov 2019 14:24:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E973D18A9 for ; Fri, 1 Nov 2019 14:24:09 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 3B964876 for ; Fri, 1 Nov 2019 14:24:09 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id y24so4437503plr.12 for ; Fri, 01 Nov 2019 07:24:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=V689n7VXWa92NVH0qxFYpfev7qJ8KWsxZVfHA8bdL+k=; b=MiW1zCH0wKiu3KVbSGz6UJ450uF0zWr0nXOcZq9yKWpZ93xc/lxy1i2ZP6gsL2hxpS iBPZWq4QaFR8sygfh84+OV6oTYFTuJI3DYqqp0dm18QmfYNHj9CaqtxOIPURqLfAggbY KYCjdaISosw3nR3goqhg8mQoKRG/MslDn1ppKEL7VB1j8BK+gw2TLe0VfAX+5LLglepZ 5HrK2+fxYioYpthhPSAuXZCIhbkb73rw3dJkoW4nhfjS5+M5Y6Q1Ydts1pTmQnjv+3nN 7UmF871sv2DqlVdOPEsBQzLB501Cvu83cyneSgXOeTXtbkQN4CJxVdHa/YearnOiVCRB 3o7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=V689n7VXWa92NVH0qxFYpfev7qJ8KWsxZVfHA8bdL+k=; b=Rv5kkzlVosa741LcddXI6PaBx7in4tPCof4KKIUXfP/R5p0lHpb4GaEBdu68M8CyNK pyaJrqvzSdgLoC8Shl09oaSXhDs9vT7mx3blluoo5JljpwOzj9ouATsOUAUQ8H4Nvm4C lw/W/1tgHtw0t+KtTRkRwaf9UAAWMzj5/9vWiuZxwZ5H/WHAUscgbPoEyhEkrZ5/IJ1F JIc3Bd7/I7xMBJbrkc2LKyAsd4/OEiX02dijWjz7KFeK+OxH3FVO0pyc4CprBO6tqree cH9TG75Vw5Zv14nhHh0W3Gjzq57m8LJ0OEjsRN6Ku+oNipMNlov8yX7O60V7rd8GSoiU W/lQ== X-Gm-Message-State: APjAAAWjQhXtYqrDfXBHZsdXE8PHECTw3q2c8XjE/DvktOZjqI8ujYBv 39BVST2jwJUYLfed+6i7vsMUVUN4DD4= X-Google-Smtp-Source: APXvYqwYp4XyZDbkiry2Z/g8S9Tt6u1pNx5CS0SriSjbu2gQPK85xsg4ZddCcuQasrJkqhhhov065w== X-Received: by 2002:a17:902:8a8b:: with SMTP id p11mr4321295plo.152.1572618248618; Fri, 01 Nov 2019 07:24:08 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:07 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:45 +0800 Message-Id: <1572618234-6904-2-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 01/10] net: openvswitch: add flow-mask cache for performance X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang The idea of this optimization comes from a patch which is committed in 2014, openvswitch community. The author is Pravin B Shelar. In order to get high performance, I implement it again. Later patches will use it. Pravin B Shelar, says: | On every packet OVS needs to lookup flow-table with every | mask until it finds a match. The packet flow-key is first | masked with mask in the list and then the masked key is | looked up in flow-table. Therefore number of masks can | affect packet processing performance. Link: https://github.com/openvswitch/ovs/commit/5604935e4e1cbc16611d2d97f50b717aa31e8ec5 Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Signed-off-by: Pravin B Shelar --- net/openvswitch/datapath.c | 3 +- net/openvswitch/flow_table.c | 109 +++++++++++++++++++++++++++++++++++++------ net/openvswitch/flow_table.h | 11 ++++- 3 files changed, 107 insertions(+), 16 deletions(-) diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index f30e406..9fea7e1 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -227,7 +227,8 @@ void ovs_dp_process_packet(struct sk_buff *skb, struct sw_flow_key *key) stats = this_cpu_ptr(dp->stats_percpu); /* Look up flow. */ - flow = ovs_flow_tbl_lookup_stats(&dp->table, key, &n_mask_hit); + flow = ovs_flow_tbl_lookup_stats(&dp->table, key, skb_get_hash(skb), + &n_mask_hit); if (unlikely(!flow)) { struct dp_upcall_info upcall; diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index cf3582c..3d515c0 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -36,6 +36,10 @@ #define TBL_MIN_BUCKETS 1024 #define REHASH_INTERVAL (10 * 60 * HZ) +#define MC_HASH_SHIFT 8 +#define MC_HASH_ENTRIES (1u << MC_HASH_SHIFT) +#define MC_HASH_SEGS ((sizeof(uint32_t) * 8) / MC_HASH_SHIFT) + static struct kmem_cache *flow_cache; struct kmem_cache *flow_stats_cache __read_mostly; @@ -168,10 +172,15 @@ int ovs_flow_tbl_init(struct flow_table *table) { struct table_instance *ti, *ufid_ti; - ti = table_instance_alloc(TBL_MIN_BUCKETS); + table->mask_cache = __alloc_percpu(sizeof(struct mask_cache_entry) * + MC_HASH_ENTRIES, + __alignof__(struct mask_cache_entry)); + if (!table->mask_cache) + return -ENOMEM; + ti = table_instance_alloc(TBL_MIN_BUCKETS); if (!ti) - return -ENOMEM; + goto free_mask_cache; ufid_ti = table_instance_alloc(TBL_MIN_BUCKETS); if (!ufid_ti) @@ -187,6 +196,8 @@ int ovs_flow_tbl_init(struct flow_table *table) free_ti: __table_instance_destroy(ti); +free_mask_cache: + free_percpu(table->mask_cache); return -ENOMEM; } @@ -243,6 +254,7 @@ void ovs_flow_tbl_destroy(struct flow_table *table) struct table_instance *ti = rcu_dereference_raw(table->ti); struct table_instance *ufid_ti = rcu_dereference_raw(table->ufid_ti); + free_percpu(table->mask_cache); table_instance_destroy(ti, ufid_ti, false); } @@ -425,7 +437,8 @@ static bool ovs_flow_cmp_unmasked_key(const struct sw_flow *flow, static struct sw_flow *masked_flow_lookup(struct table_instance *ti, const struct sw_flow_key *unmasked, - const struct sw_flow_mask *mask) + const struct sw_flow_mask *mask, + u32 *n_mask_hit) { struct sw_flow *flow; struct hlist_head *head; @@ -435,6 +448,8 @@ static struct sw_flow *masked_flow_lookup(struct table_instance *ti, ovs_flow_mask_key(&masked_key, unmasked, false, mask); hash = flow_hash(&masked_key, &mask->range); head = find_bucket(ti, hash); + (*n_mask_hit)++; + hlist_for_each_entry_rcu(flow, head, flow_table.node[ti->node_ver]) { if (flow->mask == mask && flow->flow_table.hash == hash && flow_cmp_masked_key(flow, &masked_key, &mask->range)) @@ -443,30 +458,97 @@ static struct sw_flow *masked_flow_lookup(struct table_instance *ti, return NULL; } -struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, - const struct sw_flow_key *key, - u32 *n_mask_hit) +static struct sw_flow *flow_lookup(struct flow_table *tbl, + struct table_instance *ti, + const struct sw_flow_key *key, + u32 *n_mask_hit) { - struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); struct sw_flow_mask *mask; struct sw_flow *flow; - *n_mask_hit = 0; list_for_each_entry_rcu(mask, &tbl->mask_list, list) { - (*n_mask_hit)++; - flow = masked_flow_lookup(ti, key, mask); + flow = masked_flow_lookup(ti, key, mask, n_mask_hit); if (flow) /* Found */ return flow; } return NULL; } +/* + * mask_cache maps flow to probable mask. This cache is not tightly + * coupled cache, It means updates to mask list can result in inconsistent + * cache entry in mask cache. + * This is per cpu cache and is divided in MC_HASH_SEGS segments. + * In case of a hash collision the entry is hashed in next segment. + * */ +struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, + const struct sw_flow_key *key, + u32 skb_hash, + u32 *n_mask_hit) +{ + struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); + struct mask_cache_entry *entries, *ce, *del; + struct sw_flow *flow; + u32 hash = skb_hash; + int seg; + + *n_mask_hit = 0; + if (unlikely(!skb_hash)) + return flow_lookup(tbl, ti, key, n_mask_hit); + + del = NULL; + entries = this_cpu_ptr(tbl->mask_cache); + + for (seg = 0; seg < MC_HASH_SEGS; seg++) { + int index; + + index = hash & (MC_HASH_ENTRIES - 1); + ce = &entries[index]; + + if (ce->skb_hash == skb_hash) { + struct sw_flow_mask *mask; + int i; + + i = 0; + list_for_each_entry_rcu(mask, &tbl->mask_list, list) { + if (ce->mask_index == i++) { + flow = masked_flow_lookup(ti, key, mask, + n_mask_hit); + if (flow) /* Found */ + return flow; + + break; + } + } + + del = ce; + break; + } + + if (!del || (del->skb_hash && !ce->skb_hash)) { + del = ce; + } + + hash >>= MC_HASH_SHIFT; + } + + flow = flow_lookup(tbl, ti, key, n_mask_hit); + + if (flow) { + del->skb_hash = skb_hash; + del->mask_index = (*n_mask_hit - 1); + } + + return flow; +} + struct sw_flow *ovs_flow_tbl_lookup(struct flow_table *tbl, const struct sw_flow_key *key) { + struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); u32 __always_unused n_mask_hit; - return ovs_flow_tbl_lookup_stats(tbl, key, &n_mask_hit); + return flow_lookup(tbl, ti, key, &n_mask_hit); } struct sw_flow *ovs_flow_tbl_lookup_exact(struct flow_table *tbl, @@ -475,10 +557,11 @@ struct sw_flow *ovs_flow_tbl_lookup_exact(struct flow_table *tbl, struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); struct sw_flow_mask *mask; struct sw_flow *flow; + u32 __always_unused n_mask_hit; /* Always called under ovs-mutex. */ list_for_each_entry(mask, &tbl->mask_list, list) { - flow = masked_flow_lookup(ti, match->key, mask); + flow = masked_flow_lookup(ti, match->key, mask, &n_mask_hit); if (flow && ovs_identifier_is_key(&flow->id) && ovs_flow_cmp_unmasked_key(flow, match)) return flow; @@ -631,7 +714,7 @@ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, return -ENOMEM; mask->key = new->key; mask->range = new->range; - list_add_rcu(&mask->list, &tbl->mask_list); + list_add_tail_rcu(&mask->list, &tbl->mask_list); } else { BUG_ON(!mask->ref_count); mask->ref_count++; diff --git a/net/openvswitch/flow_table.h b/net/openvswitch/flow_table.h index bc52045..04b6b1c 100644 --- a/net/openvswitch/flow_table.h +++ b/net/openvswitch/flow_table.h @@ -22,6 +22,11 @@ #include "flow.h" +struct mask_cache_entry { + u32 skb_hash; + u32 mask_index; +}; + struct table_instance { struct hlist_head *buckets; unsigned int n_buckets; @@ -34,6 +39,7 @@ struct table_instance { struct flow_table { struct table_instance __rcu *ti; struct table_instance __rcu *ufid_ti; + struct mask_cache_entry __percpu *mask_cache; struct list_head mask_list; unsigned long last_rehash; unsigned int count; @@ -60,8 +66,9 @@ int ovs_flow_tbl_insert(struct flow_table *table, struct sw_flow *flow, struct sw_flow *ovs_flow_tbl_dump_next(struct table_instance *table, u32 *bucket, u32 *idx); struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *, - const struct sw_flow_key *, - u32 *n_mask_hit); + const struct sw_flow_key *, + u32 skb_hash, + u32 *n_mask_hit); struct sw_flow *ovs_flow_tbl_lookup(struct flow_table *, const struct sw_flow_key *); struct sw_flow *ovs_flow_tbl_lookup_exact(struct flow_table *tbl, From patchwork Fri Nov 1 14:23:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188080 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="hnrD7iKp"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474Pcc5PzFz9sP4 for ; Sat, 2 Nov 2019 01:25:52 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 8429018F2; Fri, 1 Nov 2019 14:24:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 45D9518D9 for ; Fri, 1 Nov 2019 14:24:14 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 67CA787C for ; Fri, 1 Nov 2019 14:24:13 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id e4so6610700pgs.1 for ; Fri, 01 Nov 2019 07:24:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7lvU6hsWiLcJK3Uk47iKQUMRaNI1QbE9PkkUaY0AFKk=; b=hnrD7iKpjrIb5zGayqyT1BAJ/Dd4QFByJfZQq+TJHDb/5aUES9h7FHWj8puI0f6Llr MxSWkTXqze4mJWR4w027pwdZsbSErUjJOrSA3sLgG1X4gXk+PRqb7BR0CoFNy5p30tpQ VH3h9sFla5O7EW3aAKMkLz4fZjl8vEDKMjk0r3AEdAjJFN/cDKa9nzo/fHuYqHx6FmcR ZydBTITZbMShYQZkUOB8MFhmH0ePL1ZQfmLl2r+EUFmAUBMwCOgQ505/LIbIIycId6Mk flDnd8vUQF4urAXtVzpSMHeP4Mq08hnSlhDBLmxI6rQcc6ym40vNk0SRV30uLwPAos3L Z3cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7lvU6hsWiLcJK3Uk47iKQUMRaNI1QbE9PkkUaY0AFKk=; b=KBYRrdXBbjbKpfeBj8uHiXPN1tOHMSVh2JOdBxCLOVf/cABzSs9xH9FNXDemqOhz3U 8cuPUWmMDCbjiLjCM4NufKCRTc+rOcpyE0QkSfohveRm4hoZ4L3rQuqmr7KNcBAm4ABO z0OHjryb5rdKaDWIZEb8vhKHbXQUg5YLuGSyaw6iXOtnxaMveWePVPS2THFcoGPCFI0c vLt52j+DRQ2Ttq6cy8BGrJRpoyjt4vPoEyNQERHDf+nhh1OyCVAF+s/XJfU82G7tSkq8 Ggb5UU7mzdE+Ndohi1oiV0o8d5ZxCmmwUgF6JyCBWWnfGJwkVAvxgu5mJAbu8BOcTNcZ 6Zmw== X-Gm-Message-State: APjAAAU8MtvspFFCo+/ifHdsjtjV08PYCK/qF3hBFNENKty+pm6ZcfJ8 79mekKqoIRksPkksw+Tyx/NBes6zd9Y= X-Google-Smtp-Source: APXvYqyyNKOFw2ZzNtygp+3MPAJrA3/6KPH7GV1gJIEYqoDjvSKzNCUL5Xgk4QFiGvE22p1WdmqaXg== X-Received: by 2002:aa7:8249:: with SMTP id e9mr13836642pfn.118.1572618252855; Fri, 01 Nov 2019 07:24:12 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:12 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:46 +0800 Message-Id: <1572618234-6904-3-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 02/10] net: openvswitch: convert mask list in mask array X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang Port the codes to linux upstream and with little changes. Pravin B Shelar, says: | mask caches index of mask in mask_list. On packet recv OVS | need to traverse mask-list to get cached mask. Therefore array | is better for retrieving cached mask. This also allows better | cache replacement algorithm by directly checking mask's existence. Link: https://github.com/openvswitch/ovs/commit/d49fc3ff53c65e4eca9cabd52ac63396746a7ef5 Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Signed-off-by: Pravin B Shelar --- net/openvswitch/flow.h | 1 - net/openvswitch/flow_table.c | 209 +++++++++++++++++++++++++++++++++---------- net/openvswitch/flow_table.h | 8 +- 3 files changed, 167 insertions(+), 51 deletions(-) diff --git a/net/openvswitch/flow.h b/net/openvswitch/flow.h index b830d5f..8080518 100644 --- a/net/openvswitch/flow.h +++ b/net/openvswitch/flow.h @@ -166,7 +166,6 @@ struct sw_flow_key_range { struct sw_flow_mask { int ref_count; struct rcu_head rcu; - struct list_head list; struct sw_flow_key_range range; struct sw_flow_key key; }; diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index 3d515c0..92efa23 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -34,6 +34,7 @@ #include #define TBL_MIN_BUCKETS 1024 +#define MASK_ARRAY_SIZE_MIN 16 #define REHASH_INTERVAL (10 * 60 * HZ) #define MC_HASH_SHIFT 8 @@ -168,9 +169,51 @@ static struct table_instance *table_instance_alloc(int new_size) return ti; } +static struct mask_array *tbl_mask_array_alloc(int size) +{ + struct mask_array *new; + + size = max(MASK_ARRAY_SIZE_MIN, size); + new = kzalloc(sizeof(struct mask_array) + + sizeof(struct sw_flow_mask *) * size, GFP_KERNEL); + if (!new) + return NULL; + + new->count = 0; + new->max = size; + + return new; +} + +static int tbl_mask_array_realloc(struct flow_table *tbl, int size) +{ + struct mask_array *old; + struct mask_array *new; + + new = tbl_mask_array_alloc(size); + if (!new) + return -ENOMEM; + + old = ovsl_dereference(tbl->mask_array); + if (old) { + int i; + + for (i = 0; i < old->max; i++) { + if (ovsl_dereference(old->masks[i])) + new->masks[new->count++] = old->masks[i]; + } + } + + rcu_assign_pointer(tbl->mask_array, new); + kfree_rcu(old, rcu); + + return 0; +} + int ovs_flow_tbl_init(struct flow_table *table) { struct table_instance *ti, *ufid_ti; + struct mask_array *ma; table->mask_cache = __alloc_percpu(sizeof(struct mask_cache_entry) * MC_HASH_ENTRIES, @@ -178,9 +221,13 @@ int ovs_flow_tbl_init(struct flow_table *table) if (!table->mask_cache) return -ENOMEM; + ma = tbl_mask_array_alloc(MASK_ARRAY_SIZE_MIN); + if (!ma) + goto free_mask_cache; + ti = table_instance_alloc(TBL_MIN_BUCKETS); if (!ti) - goto free_mask_cache; + goto free_mask_array; ufid_ti = table_instance_alloc(TBL_MIN_BUCKETS); if (!ufid_ti) @@ -188,7 +235,7 @@ int ovs_flow_tbl_init(struct flow_table *table) rcu_assign_pointer(table->ti, ti); rcu_assign_pointer(table->ufid_ti, ufid_ti); - INIT_LIST_HEAD(&table->mask_list); + rcu_assign_pointer(table->mask_array, ma); table->last_rehash = jiffies; table->count = 0; table->ufid_count = 0; @@ -196,6 +243,8 @@ int ovs_flow_tbl_init(struct flow_table *table) free_ti: __table_instance_destroy(ti); +free_mask_array: + kfree(ma); free_mask_cache: free_percpu(table->mask_cache); return -ENOMEM; @@ -255,6 +304,7 @@ void ovs_flow_tbl_destroy(struct flow_table *table) struct table_instance *ufid_ti = rcu_dereference_raw(table->ufid_ti); free_percpu(table->mask_cache); + kfree_rcu(rcu_dereference_raw(table->mask_array), rcu); table_instance_destroy(ti, ufid_ti, false); } @@ -460,17 +510,27 @@ static struct sw_flow *masked_flow_lookup(struct table_instance *ti, static struct sw_flow *flow_lookup(struct flow_table *tbl, struct table_instance *ti, + struct mask_array *ma, const struct sw_flow_key *key, - u32 *n_mask_hit) + u32 *n_mask_hit, + u32 *index) { - struct sw_flow_mask *mask; struct sw_flow *flow; + int i; - list_for_each_entry_rcu(mask, &tbl->mask_list, list) { - flow = masked_flow_lookup(ti, key, mask, n_mask_hit); - if (flow) /* Found */ - return flow; + for (i = 0; i < ma->max; i++) { + struct sw_flow_mask *mask; + + mask = rcu_dereference_ovsl(ma->masks[i]); + if (mask) { + flow = masked_flow_lookup(ti, key, mask, n_mask_hit); + if (flow) { /* Found */ + *index = i; + return flow; + } + } } + return NULL; } @@ -486,6 +546,7 @@ struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, u32 skb_hash, u32 *n_mask_hit) { + struct mask_array *ma = rcu_dereference_ovsl(tbl->mask_array); struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); struct mask_cache_entry *entries, *ce, *del; struct sw_flow *flow; @@ -493,8 +554,11 @@ struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, int seg; *n_mask_hit = 0; - if (unlikely(!skb_hash)) - return flow_lookup(tbl, ti, key, n_mask_hit); + if (unlikely(!skb_hash)) { + u32 __always_unused mask_index; + + return flow_lookup(tbl, ti, ma, key, n_mask_hit, &mask_index); + } del = NULL; entries = this_cpu_ptr(tbl->mask_cache); @@ -507,37 +571,33 @@ struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, if (ce->skb_hash == skb_hash) { struct sw_flow_mask *mask; - int i; - - i = 0; - list_for_each_entry_rcu(mask, &tbl->mask_list, list) { - if (ce->mask_index == i++) { - flow = masked_flow_lookup(ti, key, mask, - n_mask_hit); - if (flow) /* Found */ - return flow; - - break; - } + struct sw_flow *flow; + + mask = rcu_dereference_ovsl(ma->masks[ce->mask_index]); + if (mask) { + flow = masked_flow_lookup(ti, key, mask, + n_mask_hit); + if (flow) /* Found */ + return flow; } del = ce; break; } - if (!del || (del->skb_hash && !ce->skb_hash)) { + if (!del || (del->skb_hash && !ce->skb_hash) || + (rcu_dereference_ovsl(ma->masks[del->mask_index]) && + !rcu_dereference_ovsl(ma->masks[ce->mask_index]))) { del = ce; } hash >>= MC_HASH_SHIFT; } - flow = flow_lookup(tbl, ti, key, n_mask_hit); + flow = flow_lookup(tbl, ti, ma, key, n_mask_hit, &del->mask_index); - if (flow) { + if (flow) del->skb_hash = skb_hash; - del->mask_index = (*n_mask_hit - 1); - } return flow; } @@ -546,26 +606,38 @@ struct sw_flow *ovs_flow_tbl_lookup(struct flow_table *tbl, const struct sw_flow_key *key) { struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); + struct mask_array *ma = rcu_dereference_ovsl(tbl->mask_array); + u32 __always_unused n_mask_hit; + u32 __always_unused index; - return flow_lookup(tbl, ti, key, &n_mask_hit); + return flow_lookup(tbl, ti, ma, key, &n_mask_hit, &index); } struct sw_flow *ovs_flow_tbl_lookup_exact(struct flow_table *tbl, const struct sw_flow_match *match) { - struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); - struct sw_flow_mask *mask; - struct sw_flow *flow; - u32 __always_unused n_mask_hit; + struct mask_array *ma = ovsl_dereference(tbl->mask_array); + int i; /* Always called under ovs-mutex. */ - list_for_each_entry(mask, &tbl->mask_list, list) { + for (i = 0; i < ma->max; i++) { + struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); + u32 __always_unused n_mask_hit; + struct sw_flow_mask *mask; + struct sw_flow *flow; + + mask = ovsl_dereference(ma->masks[i]); + if (!mask) + continue; + flow = masked_flow_lookup(ti, match->key, mask, &n_mask_hit); if (flow && ovs_identifier_is_key(&flow->id) && - ovs_flow_cmp_unmasked_key(flow, match)) + ovs_flow_cmp_unmasked_key(flow, match)) { return flow; + } } + return NULL; } @@ -611,13 +683,9 @@ struct sw_flow *ovs_flow_tbl_lookup_ufid(struct flow_table *tbl, int ovs_flow_tbl_num_masks(const struct flow_table *table) { - struct sw_flow_mask *mask; - int num = 0; + struct mask_array *ma = rcu_dereference_ovsl(table->mask_array); - list_for_each_entry(mask, &table->mask_list, list) - num++; - - return num; + return ma->count; } static struct table_instance *table_instance_expand(struct table_instance *ti, @@ -638,8 +706,19 @@ static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) mask->ref_count--; if (!mask->ref_count) { - list_del_rcu(&mask->list); - kfree_rcu(mask, rcu); + struct mask_array *ma; + int i; + + ma = ovsl_dereference(tbl->mask_array); + for (i = 0; i < ma->max; i++) { + if (mask == ovsl_dereference(ma->masks[i])) { + RCU_INIT_POINTER(ma->masks[i], NULL); + ma->count--; + kfree_rcu(mask, rcu); + return; + } + } + BUG(); } } } @@ -689,13 +768,16 @@ static bool mask_equal(const struct sw_flow_mask *a, static struct sw_flow_mask *flow_mask_find(const struct flow_table *tbl, const struct sw_flow_mask *mask) { - struct list_head *ml; + struct mask_array *ma; + int i; - list_for_each(ml, &tbl->mask_list) { - struct sw_flow_mask *m; - m = container_of(ml, struct sw_flow_mask, list); - if (mask_equal(mask, m)) - return m; + ma = ovsl_dereference(tbl->mask_array); + for (i = 0; i < ma->max; i++) { + struct sw_flow_mask *t; + t = ovsl_dereference(ma->masks[i]); + + if (t && mask_equal(mask, t)) + return t; } return NULL; @@ -706,15 +788,44 @@ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, const struct sw_flow_mask *new) { struct sw_flow_mask *mask; + mask = flow_mask_find(tbl, new); if (!mask) { + struct mask_array *ma; + int i; + /* Allocate a new mask if none exsits. */ mask = mask_alloc(); if (!mask) return -ENOMEM; mask->key = new->key; mask->range = new->range; - list_add_tail_rcu(&mask->list, &tbl->mask_list); + + /* Add mask to mask-list. */ + ma = ovsl_dereference(tbl->mask_array); + if (ma->count >= ma->max) { + int err; + + err = tbl_mask_array_realloc(tbl, ma->max + + MASK_ARRAY_SIZE_MIN); + if (err) { + kfree(mask); + return err; + } + + ma = ovsl_dereference(tbl->mask_array); + } + + for (i = 0; i < ma->max; i++) { + const struct sw_flow_mask *t; + + t = ovsl_dereference(ma->masks[i]); + if (!t) { + rcu_assign_pointer(ma->masks[i], mask); + ma->count++; + break; + } + } } else { BUG_ON(!mask->ref_count); mask->ref_count++; diff --git a/net/openvswitch/flow_table.h b/net/openvswitch/flow_table.h index 04b6b1c..8a5cea6 100644 --- a/net/openvswitch/flow_table.h +++ b/net/openvswitch/flow_table.h @@ -27,6 +27,12 @@ struct mask_cache_entry { u32 mask_index; }; +struct mask_array { + struct rcu_head rcu; + int count, max; + struct sw_flow_mask __rcu *masks[]; +}; + struct table_instance { struct hlist_head *buckets; unsigned int n_buckets; @@ -40,7 +46,7 @@ struct flow_table { struct table_instance __rcu *ti; struct table_instance __rcu *ufid_ti; struct mask_cache_entry __percpu *mask_cache; - struct list_head mask_list; + struct mask_array __rcu *mask_array; unsigned long last_rehash; unsigned int count; unsigned int ufid_count; From patchwork Fri Nov 1 14:23:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="XLCZ4+G3"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474Pdh01sJz9sP4 for ; Sat, 2 Nov 2019 01:26:47 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 2F3921908; Fri, 1 Nov 2019 14:24:19 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 688D418EF for ; Fri, 1 Nov 2019 14:24:17 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 19114876 for ; Fri, 1 Nov 2019 14:24:17 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id q21so4428244plr.13 for ; Fri, 01 Nov 2019 07:24:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=99ayDtrGXogeYemVBZ8XosUkb5qFRdKURNlUuH0mkvk=; b=XLCZ4+G3rlEvjo41pe5K/B/gYE+MVta4weFFcGoJpxF2/LDaGtt6WhIMczbk2iLYng bSMY6ytLLW2fUdM73R73QPf41dUzApo2OoCBTajk4URs4AwOuSbu8YZx/8id7pZikKDk QOg/X1fje8BhKGy6WVesyymScb9/0Pk89VpN8k/ovY7NnVxGyhnExYlq6nbBpoQayT0/ wpOer/WhwJ9rLkiX09AuzQ8O1rpVByNoZVrITxLGJHurWeXOk6jXyl73PKYafR4ve6Ao OZDqeozXRPbmfvkiCvRBexeStkwePuxbHQk1ERuEkW+Vs3zCM2JY7ebgz2U7SRjiaBD/ fGtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=99ayDtrGXogeYemVBZ8XosUkb5qFRdKURNlUuH0mkvk=; b=SybV1XE/dVCl5UGcmbi11ylgZ4xGrNTsQ/D4RPXJbtz8zNqRknPWPbsSGYg6fBKluJ 2+EhqmMA+z1PDm/A4b2VfLElwMfoHKFe6OYZu2Z60lD7ovvapedN5gMYKf8j5JQQnA5W HQ8X4+9S57fQJmt182BsaDBP+hUWDcG3s578ZAU7iYZwbOt2okSzIzsG4GGU80MX7NZD lCBbr1rEyQrkK1rYmfsmUVitDVNajmiDoYm5AcaNp7nIT24ZnbuVedEuGmbIY7pQHm8W 2WAyDxywDw/bwwbNTbxn/XNRwFhzec2CmWgJf3PUysGlnsTkTnYOi/cGMHeUIdyF7Cp+ F0tA== X-Gm-Message-State: APjAAAXkxQSKoNXeFihOkj+qCKXq+NbJYJmkAI87M35AwEh7rWZaZznG I9/5DzUqS3JLEnZA7iOrZgY= X-Google-Smtp-Source: APXvYqzmWG71bMr0d5ph7h6xvhzZ5uJPFbhBwxPU55iW8lWTHhX6KSpvrneMRb48kbvA4TtVrfj1gw== X-Received: by 2002:a17:902:76ca:: with SMTP id j10mr13135868plt.58.1572618256453; Fri, 01 Nov 2019 07:24:16 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:16 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:47 +0800 Message-Id: <1572618234-6904-4-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 03/10] net: openvswitch: shrink the mask array if necessary X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang When creating and inserting flow-mask, if there is no available flow-mask, we realloc the mask array. When removing flow-mask, if necessary, we shrink mask array. Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Acked-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index 92efa23..0c0fcd6 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -694,6 +694,23 @@ static struct table_instance *table_instance_expand(struct table_instance *ti, return table_instance_rehash(ti, ti->n_buckets * 2, ufid); } +static void tbl_mask_array_delete_mask(struct mask_array *ma, + struct sw_flow_mask *mask) +{ + int i; + + /* Remove the deleted mask pointers from the array */ + for (i = 0; i < ma->max; i++) { + if (mask == ovsl_dereference(ma->masks[i])) { + RCU_INIT_POINTER(ma->masks[i], NULL); + ma->count--; + kfree_rcu(mask, rcu); + return; + } + } + BUG(); +} + /* Remove 'mask' from the mask list, if it is not needed any more. */ static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) { @@ -707,18 +724,14 @@ static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) if (!mask->ref_count) { struct mask_array *ma; - int i; ma = ovsl_dereference(tbl->mask_array); - for (i = 0; i < ma->max; i++) { - if (mask == ovsl_dereference(ma->masks[i])) { - RCU_INIT_POINTER(ma->masks[i], NULL); - ma->count--; - kfree_rcu(mask, rcu); - return; - } - } - BUG(); + tbl_mask_array_delete_mask(ma, mask); + + /* Shrink the mask array if necessary. */ + if (ma->max >= (MASK_ARRAY_SIZE_MIN * 2) && + ma->count <= (ma->max / 3)) + tbl_mask_array_realloc(tbl, ma->max / 2); } } } From patchwork Fri Nov 1 14:23:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lRHG3p2r"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474Pff6DF9z9sPF for ; Sat, 2 Nov 2019 01:27:38 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id C0857193A; Fri, 1 Nov 2019 14:24:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 914061929 for ; Fri, 1 Nov 2019 14:24:20 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1F4DF14D for ; Fri, 1 Nov 2019 14:24:20 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id r4so7186957pfl.7 for ; Fri, 01 Nov 2019 07:24:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ly8ZGZldGeMEyOyDTGYNxgf6VANdx6un83/HEOyfeC0=; b=lRHG3p2r0kRBY8xghWYmD3bEwoa+ehTWIbvzfwSmuJWexQEg2Z3BBtc5mmJOS+ptZ6 NDmmPKZQ0duJ7/nxWmt5q9l412RL4mxCs/pgQLXhyaTJzFWgEI+Z61P/qZ/EaYg376G5 rpfmWNAX+Tme9NMNQC65tf43majk+bRY1dFXsv6F8RMyc8/Cpvq9u9EK5rf3amOFFLHn WOT8A6NjKjJqa6GJG5WyHl53Ti8H04kUp2n7ZKh+PL1wLjgUrMAgR4d8TTTnELch1cnD uKJ5Qe+ckrNVTWoMv6UO3m6vYn6ZtNTe8HOf5S30rHpWqe+O3W/xOeBKw5vtNFbnPzoj zPbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ly8ZGZldGeMEyOyDTGYNxgf6VANdx6un83/HEOyfeC0=; b=rw3LRr18vjWXeAfFDCFAiLzomypWPOoEk/RvCSyVD1CtuHt5Knu46Bq8TdlVW7IGLR XXS0hHNP5JFeUVOGvEF2JtB7GDZJUBbpOBKwbJF886gRFsK74vwJKbiUmJsXJI523dWL ADJvL8qdrlCkVWCD8CYpsEObNYbT/k+mHbsLD2S1JK3/p8g6otcXSoaDHnlvwB+m1EDu ZUOIgHjbquAskrLweV/YCoWPsT9TJSN44xFzrxEgkO9QfLiKw+lbkPewcx7LP3Wm3amX CaOloUFWltSZlwWrSnUtEYKhZRdDpLnmO69Tsp2rzO0PdxVkIElJA4Za2UzzAUDn104q BSxQ== X-Gm-Message-State: APjAAAWBtoRWQzyeT1xzB1jVGnddh/vQyq5xJTn0I4EfJWlmVVBdyycL di4qIkINwAl/i//04aCKa8o= X-Google-Smtp-Source: APXvYqzjhMqjN6xslx/4IMEEMMuZqJhG83v0iWTh2oe6RUeOP2kDcUhsxPWL85jD1IR5At6znuhJRQ== X-Received: by 2002:a63:1e59:: with SMTP id p25mr13415474pgm.361.1572618259700; Fri, 01 Nov 2019 07:24:19 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:19 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:48 +0800 Message-Id: <1572618234-6904-5-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 04/10] net: openvswitch: optimize flow mask cache hash collision X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang Port the codes to linux upstream and with little changes. Pravin B Shelar, says: | In case hash collision on mask cache, OVS does extra flow | lookup. Following patch avoid it. Link: https://github.com/openvswitch/ovs/commit/0e6efbe2712da03522532dc5e84806a96f6a0dd1 Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Signed-off-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 95 ++++++++++++++++++++++++-------------------- 1 file changed, 53 insertions(+), 42 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index 0c0fcd6..c7ba435 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -508,6 +508,9 @@ static struct sw_flow *masked_flow_lookup(struct table_instance *ti, return NULL; } +/* Flow lookup does full lookup on flow table. It starts with + * mask from index passed in *index. + */ static struct sw_flow *flow_lookup(struct flow_table *tbl, struct table_instance *ti, struct mask_array *ma, @@ -515,19 +518,32 @@ static struct sw_flow *flow_lookup(struct flow_table *tbl, u32 *n_mask_hit, u32 *index) { + struct sw_flow_mask *mask; struct sw_flow *flow; int i; - for (i = 0; i < ma->max; i++) { - struct sw_flow_mask *mask; - - mask = rcu_dereference_ovsl(ma->masks[i]); + if (*index < ma->max) { + mask = rcu_dereference_ovsl(ma->masks[*index]); if (mask) { flow = masked_flow_lookup(ti, key, mask, n_mask_hit); - if (flow) { /* Found */ - *index = i; + if (flow) return flow; - } + } + } + + for (i = 0; i < ma->max; i++) { + + if (i == *index) + continue; + + mask = rcu_dereference_ovsl(ma->masks[i]); + if (!mask) + continue; + + flow = masked_flow_lookup(ti, key, mask, n_mask_hit); + if (flow) { /* Found */ + *index = i; + return flow; } } @@ -546,58 +562,54 @@ struct sw_flow *ovs_flow_tbl_lookup_stats(struct flow_table *tbl, u32 skb_hash, u32 *n_mask_hit) { - struct mask_array *ma = rcu_dereference_ovsl(tbl->mask_array); - struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); - struct mask_cache_entry *entries, *ce, *del; + struct mask_array *ma = rcu_dereference(tbl->mask_array); + struct table_instance *ti = rcu_dereference(tbl->ti); + struct mask_cache_entry *entries, *ce; struct sw_flow *flow; - u32 hash = skb_hash; + u32 hash; int seg; *n_mask_hit = 0; if (unlikely(!skb_hash)) { - u32 __always_unused mask_index; + u32 mask_index = 0; return flow_lookup(tbl, ti, ma, key, n_mask_hit, &mask_index); } - del = NULL; + /* Pre and post recirulation flows usually have the same skb_hash + * value. To avoid hash collisions, rehash the 'skb_hash' with + * 'recirc_id'. */ + if (key->recirc_id) + skb_hash = jhash_1word(skb_hash, key->recirc_id); + + ce = NULL; + hash = skb_hash; entries = this_cpu_ptr(tbl->mask_cache); + /* Find the cache entry 'ce' to operate on. */ for (seg = 0; seg < MC_HASH_SEGS; seg++) { - int index; - - index = hash & (MC_HASH_ENTRIES - 1); - ce = &entries[index]; - - if (ce->skb_hash == skb_hash) { - struct sw_flow_mask *mask; - struct sw_flow *flow; - - mask = rcu_dereference_ovsl(ma->masks[ce->mask_index]); - if (mask) { - flow = masked_flow_lookup(ti, key, mask, - n_mask_hit); - if (flow) /* Found */ - return flow; - } - - del = ce; - break; + int index = hash & (MC_HASH_ENTRIES - 1); + struct mask_cache_entry *e; + + e = &entries[index]; + if (e->skb_hash == skb_hash) { + flow = flow_lookup(tbl, ti, ma, key, n_mask_hit, + &e->mask_index); + if (!flow) + e->skb_hash = 0; + return flow; } - if (!del || (del->skb_hash && !ce->skb_hash) || - (rcu_dereference_ovsl(ma->masks[del->mask_index]) && - !rcu_dereference_ovsl(ma->masks[ce->mask_index]))) { - del = ce; - } + if (!ce || e->skb_hash < ce->skb_hash) + ce = e; /* A better replacement cache candidate. */ hash >>= MC_HASH_SHIFT; } - flow = flow_lookup(tbl, ti, ma, key, n_mask_hit, &del->mask_index); - + /* Cache miss, do full lookup. */ + flow = flow_lookup(tbl, ti, ma, key, n_mask_hit, &ce->mask_index); if (flow) - del->skb_hash = skb_hash; + ce->skb_hash = skb_hash; return flow; } @@ -607,9 +619,8 @@ struct sw_flow *ovs_flow_tbl_lookup(struct flow_table *tbl, { struct table_instance *ti = rcu_dereference_ovsl(tbl->ti); struct mask_array *ma = rcu_dereference_ovsl(tbl->mask_array); - u32 __always_unused n_mask_hit; - u32 __always_unused index; + u32 index = 0; return flow_lookup(tbl, ti, ma, key, &n_mask_hit, &index); } From patchwork Fri Nov 1 14:23:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VhXfJh+N"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474PgY1Q8wz9sP4 for ; Sat, 2 Nov 2019 01:28:25 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 6B3271923; Fri, 1 Nov 2019 14:24:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5AC5418AB for ; Fri, 1 Nov 2019 14:24:24 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C7A6C14D for ; Fri, 1 Nov 2019 14:24:23 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id d13so7203996pfq.2 for ; Fri, 01 Nov 2019 07:24:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+6y7OChHTvMPWpmBp1Ktoa0+GosQY6LfAXE2ork61rA=; b=VhXfJh+N4bmOKZLDKZ65/HcyEtHvXWWpYHsNmI4lli4UAcZgGcn2IlNO6sZyNLFmIA +Wh/P6hhELu5bh1cMHMRzoiH0XjEoOyd5LC1LNiCDS8Kd5TRBA6unCVS3PxOnZdd9PoT IO0NNZRnEnTJQmKrfKOQzUj6aECH/tEbw+ugG5GzT327hJ+8vpGiKJZKjEK2MwJNY1no Ag18wgB7tKCx67w0Iqb1yMxgaqT9ZmPtdE1qhLbmT5tuZ1BKXkAPJzU+uQhbXC8i6dXX 3zyZhhO159+K28wwTOgawnCtI6TCB9L5cMDUJAm/QfBiV0plDRyoEHLXgvkFGrssFp+O 2E9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+6y7OChHTvMPWpmBp1Ktoa0+GosQY6LfAXE2ork61rA=; b=qwQFszkQ3pI30gngWt6pyppxsRP4NEQprdTfUSeOKEeZndGEMJMLLZQ1fZe4lLu/BT rlJhmPlB3/Nfg2Hf5zGn1WJf7GUTJyzAzWkY1o5JqLMI4En3wXLpcZM5t2qfcCc5CD9Q m14Yi+8Ioo0BzhnzbcqTfdPPS95JBuFCfjLjj3Xo8aHHOl0m792LWmZ0I4mO2eNtacuh oMU+Nfs7SmcI0qsLiQ2oHVYZgML5zzAmZKdwd01ak7QZVXvjcUs7HuG4F8MmSfnWYKWO 9vxujHb2117EZvTdnYPhi54UwhbcEjK9K0ZhFWBuRIu4VhdD3Fm18Oi1f27T4kQA7oiS KlEw== X-Gm-Message-State: APjAAAVO3gzrW5Pvx3cFmHGKhyvarkLtWesr+fVuXHnXnbGJDaaTeVGy 7u/9P3ABrysb+6Sg/u/YLfY= X-Google-Smtp-Source: APXvYqzkEh4hxpliGiW8ZaZM8eTAEPuQW/LzHyotTlB67yYhCz8Nt1rpooXNpiOMsJJg+6NeKNJvqQ== X-Received: by 2002:a17:90b:3105:: with SMTP id gc5mr4361862pjb.30.1572618263399; Fri, 01 Nov 2019 07:24:23 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:22 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:49 +0800 Message-Id: <1572618234-6904-6-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 05/10] net: openvswitch: optimize flow-mask looking up X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang The full looking up on flow table traverses all mask array. If mask-array is too large, the number of invalid flow-mask increase, performance will be drop. One bad case, for example: M means flow-mask is valid and NULL of flow-mask means deleted. +-------------------------------------------+ | M | NULL | ... | NULL | M| +-------------------------------------------+ In that case, without this patch, openvswitch will traverses all mask array, because there will be one flow-mask in the tail. This patch changes the way of flow-mask inserting and deleting, and the mask array will be keep as below: there is not a NULL hole. In the fast path, we can "break" "for" (not "continue") in flow_lookup when we get a NULL flow-mask. "break" v +-------------------------------------------+ | M | M | NULL |... | NULL | NULL| +-------------------------------------------+ This patch don't optimize slow or control path, still using ma->max to traverse. Slow path: * tbl_mask_array_realloc * ovs_flow_tbl_lookup_exact * flow_mask_find Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 104 ++++++++++++++++++++++--------------------- 1 file changed, 53 insertions(+), 51 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index c7ba435..a10d421 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -518,8 +518,8 @@ static struct sw_flow *flow_lookup(struct flow_table *tbl, u32 *n_mask_hit, u32 *index) { - struct sw_flow_mask *mask; struct sw_flow *flow; + struct sw_flow_mask *mask; int i; if (*index < ma->max) { @@ -538,7 +538,7 @@ static struct sw_flow *flow_lookup(struct flow_table *tbl, mask = rcu_dereference_ovsl(ma->masks[i]); if (!mask) - continue; + break; flow = masked_flow_lookup(ti, key, mask, n_mask_hit); if (flow) { /* Found */ @@ -695,8 +695,7 @@ struct sw_flow *ovs_flow_tbl_lookup_ufid(struct flow_table *tbl, int ovs_flow_tbl_num_masks(const struct flow_table *table) { struct mask_array *ma = rcu_dereference_ovsl(table->mask_array); - - return ma->count; + return READ_ONCE(ma->count); } static struct table_instance *table_instance_expand(struct table_instance *ti, @@ -705,21 +704,33 @@ static struct table_instance *table_instance_expand(struct table_instance *ti, return table_instance_rehash(ti, ti->n_buckets * 2, ufid); } -static void tbl_mask_array_delete_mask(struct mask_array *ma, - struct sw_flow_mask *mask) +static void tbl_mask_array_del_mask(struct flow_table *tbl, + struct sw_flow_mask *mask) { - int i; + struct mask_array *ma = ovsl_dereference(tbl->mask_array); + int i, ma_count = READ_ONCE(ma->count); /* Remove the deleted mask pointers from the array */ - for (i = 0; i < ma->max; i++) { - if (mask == ovsl_dereference(ma->masks[i])) { - RCU_INIT_POINTER(ma->masks[i], NULL); - ma->count--; - kfree_rcu(mask, rcu); - return; - } + for (i = 0; i < ma_count; i++) { + if (mask == ovsl_dereference(ma->masks[i])) + goto found; } + BUG(); + return; + +found: + WRITE_ONCE(ma->count, ma_count -1); + + rcu_assign_pointer(ma->masks[i], ma->masks[ma_count -1]); + RCU_INIT_POINTER(ma->masks[ma_count -1], NULL); + + kfree_rcu(mask, rcu); + + /* Shrink the mask array if necessary. */ + if (ma->max >= (MASK_ARRAY_SIZE_MIN * 2) && + ma_count <= (ma->max / 3)) + tbl_mask_array_realloc(tbl, ma->max / 2); } /* Remove 'mask' from the mask list, if it is not needed any more. */ @@ -733,17 +744,8 @@ static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) BUG_ON(!mask->ref_count); mask->ref_count--; - if (!mask->ref_count) { - struct mask_array *ma; - - ma = ovsl_dereference(tbl->mask_array); - tbl_mask_array_delete_mask(ma, mask); - - /* Shrink the mask array if necessary. */ - if (ma->max >= (MASK_ARRAY_SIZE_MIN * 2) && - ma->count <= (ma->max / 3)) - tbl_mask_array_realloc(tbl, ma->max / 2); - } + if (!mask->ref_count) + tbl_mask_array_del_mask(tbl, mask); } } @@ -807,6 +809,29 @@ static struct sw_flow_mask *flow_mask_find(const struct flow_table *tbl, return NULL; } +static int tbl_mask_array_add_mask(struct flow_table *tbl, + struct sw_flow_mask *new) +{ + struct mask_array *ma = ovsl_dereference(tbl->mask_array); + int err, ma_count = READ_ONCE(ma->count); + + if (ma_count >= ma->max) { + err = tbl_mask_array_realloc(tbl, ma->max + + MASK_ARRAY_SIZE_MIN); + if (err) + return err; + + ma = ovsl_dereference(tbl->mask_array); + } + + BUG_ON(ovsl_dereference(ma->masks[ma_count])); + + rcu_assign_pointer(ma->masks[ma_count], new); + WRITE_ONCE(ma->count, ma_count +1); + + return 0; +} + /* Add 'mask' into the mask list, if it is not already there. */ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, const struct sw_flow_mask *new) @@ -815,9 +840,6 @@ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, mask = flow_mask_find(tbl, new); if (!mask) { - struct mask_array *ma; - int i; - /* Allocate a new mask if none exsits. */ mask = mask_alloc(); if (!mask) @@ -826,29 +848,9 @@ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, mask->range = new->range; /* Add mask to mask-list. */ - ma = ovsl_dereference(tbl->mask_array); - if (ma->count >= ma->max) { - int err; - - err = tbl_mask_array_realloc(tbl, ma->max + - MASK_ARRAY_SIZE_MIN); - if (err) { - kfree(mask); - return err; - } - - ma = ovsl_dereference(tbl->mask_array); - } - - for (i = 0; i < ma->max; i++) { - const struct sw_flow_mask *t; - - t = ovsl_dereference(ma->masks[i]); - if (!t) { - rcu_assign_pointer(ma->masks[i], mask); - ma->count++; - break; - } + if (tbl_mask_array_add_mask(tbl, mask)) { + kfree(mask); + return -ENOMEM; } } else { BUG_ON(!mask->ref_count); From patchwork Fri Nov 1 14:23:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188085 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="iSmNz8NQ"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474Phf61M3z9sPj for ; Sat, 2 Nov 2019 01:29:22 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 3CD1EAE7; Fri, 1 Nov 2019 14:24:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B4356194E for ; Fri, 1 Nov 2019 14:24:27 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 740A1876 for ; Fri, 1 Nov 2019 14:24:27 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id 15so6594165pgt.7 for ; Fri, 01 Nov 2019 07:24:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9IHvnBSUrDy/0DYdv8nzJjC4Qb9xT876DUgC/nGixdg=; b=iSmNz8NQ0pgtEIqu4lbggfqBER/eCvRutF590EdldSWDRSnkSVEjNKHTMm2VYsvEn+ BpUBHRIGn6r8owK9RFKMwH5TVK/EfkQVJHlRq72Sto3gu70um8YQOY1raTnX8yg+dyMG 314mn3bVUgu8H1nRDuQwa4ckWU0ix9MddbOCClHrdXdxUmc1mMWQLsNJO4rWBkfeX6WN amPA6qTLFHAH9faCI1p/MJYLEt4ThxyFYD/9sctspbc4kfLdzpN/sJlKoSCcqCI1bL70 GKhUaQhnTSZE99SP4PxQldraRvm3fhZFVrnF65/kYfqmQ10VdksZ7JzsrTajt3mSFppS wTmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9IHvnBSUrDy/0DYdv8nzJjC4Qb9xT876DUgC/nGixdg=; b=NI4a0YQ+RD4csUqHDDoPPYup3YEhPLvOuQUfV+k+i+H9Bqv5Sxt9xKsoWI5mrshCaM y4JH+cHbJgWrRD/sgpAXlOCGnM1QI3/eTYkHGe2MfAgEoC/kXBuUSPhCjhZ+bBvl2+5B AS8klOLsvw4gwG8exzDIDUfN0aF2DDBpbsD8Y3aGDeLAX8Ci6jCWQ3yLCs+sN41x2Xu6 R5fd3XZn5vj80oo3N6Y7gu4hvomNZt7QjYuOLpbdthQvZ2eKQP7ceqUwcSnNmzfJCK3e LCyInBL9a/saeBjSVT0zABTspWHTSew3ABsnDks9eUiG0xS4ydFEHf+fbWKtd0lXRAmk LL5Q== X-Gm-Message-State: APjAAAV1bCGKtsNV6KX7yQ3rqwG0C9zec2PWQ+bSlrMxNG42bkgkmwQL BqxSpWZmUjeGZYIJr2svHA4JxIsJpPU= X-Google-Smtp-Source: APXvYqz1JsEJuFdZfU9gMAvyChxTiqLfmxBoHwp2jCP3aNEOAA4a+z2mvFIPVlma8P9lGppn5wyEdg== X-Received: by 2002:a63:f919:: with SMTP id h25mr5628710pgi.85.1572618267099; Fri, 01 Nov 2019 07:24:27 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:26 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:50 +0800 Message-Id: <1572618234-6904-7-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 06/10] net: openvswitch: simplify the flow_hash X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang Simplify the code and remove the unnecessary BUILD_BUG_ON. Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Acked-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index a10d421..96757e2 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -432,13 +432,10 @@ int ovs_flow_tbl_flush(struct flow_table *flow_table) static u32 flow_hash(const struct sw_flow_key *key, const struct sw_flow_key_range *range) { - int key_start = range->start; - int key_end = range->end; - const u32 *hash_key = (const u32 *)((const u8 *)key + key_start); - int hash_u32s = (key_end - key_start) >> 2; + const u32 *hash_key = (const u32 *)((const u8 *)key + range->start); /* Make sure number of hash bytes are multiple of u32. */ - BUILD_BUG_ON(sizeof(long) % sizeof(u32)); + int hash_u32s = range_n_bytes(range) >> 2; return jhash2(hash_key, hash_u32s, 0); } From patchwork Fri Nov 1 14:23:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188086 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="QdfwaJbo"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474PjT0D6Vz9sP4 for ; Sat, 2 Nov 2019 01:30:05 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id DEC87DAB; Fri, 1 Nov 2019 14:24:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id DD8DCDAB for ; Fri, 1 Nov 2019 14:24:30 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9ECA714D for ; Fri, 1 Nov 2019 14:24:30 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id p24so683186pfn.4 for ; Fri, 01 Nov 2019 07:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wcdpZmmeZAo/k2NkvkUCVVJa5T89dwfwECWke5u/oEE=; b=QdfwaJboLYLlCyiwyGwg7xfY/wZVKYhrHqbrec8/lp+RNryGrJZ584pxVa51L3KISq HlrlWy03VT2BubOJ9MyIZR7+x2GIFN6YeYW/0kpJvRjPwaaIjTgdw11f//TzLuj2qnWh eQYarQjkR+71jla6srqCL/pC9jQ5mQvVkBJsGzmmtABaDVA3rTqHNfcBVGrbBKKfbOSO /JdHbCL5u8k0/8TwjRt1KuQAWXLg0MgvfYqAZDXdy1JPhdirCj8iHrssxLjtWjdv/jVW fJarMAxiYsYCu+LwawCmuRamMkrBpW7308DWWL6L8zdZ/t6G+Y4w35d3UYA8lg58aJNq 54BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wcdpZmmeZAo/k2NkvkUCVVJa5T89dwfwECWke5u/oEE=; b=YP0+oqxoXSpLXCr2IBFUvHGIrkeZjgnEOqNSdJOcG107sfJmQIk/Rmv7hEMSOI0lKX JsLXphiwUdeo3h5WFu2WHQZKcKaurA5HgVXMl1Rwnc6bRVTLbiAVRjHXC2/NLlAbVXcc vEBzTafNk3OFYnGsYJEiNtDo5bzB/SvX/1nO9zB3sWA79cvC4YqqKrehSbJUR/n/G/vk jT2FyTYW5PV+DvpSFm3lTw54cXRlAydb3pbNZP5kzgv39ETuDT6aOETrOaLbBt74v6rH gs72sgwqEYoWf2C+L8YXVK+d89jUGpvoUOTTh6AbMRDxjgUmY9EfnchPIEHgroqWnCxJ g84w== X-Gm-Message-State: APjAAAUVtqdFRoKiE4Y48s8azh2YTqmNho+dbTayj5hos63syqbLaCfl 64CNSZ1VFX9raxz6mLnR0fs= X-Google-Smtp-Source: APXvYqwJmTFhmV7lDzszpmeZ3DLL5+wVdBKVPjRxmx71enTrftOOdaG5SYiajc5mj8DVK0wPM70W7A== X-Received: by 2002:a17:90a:24b:: with SMTP id t11mr15837371pje.77.1572618270220; Fri, 01 Nov 2019 07:24:30 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:29 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:51 +0800 Message-Id: <1572618234-6904-8-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 07/10] net: openvswitch: add likely in flow_lookup X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang The most case *index < ma->max, and flow-mask is not NULL. We add un/likely for performance. Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Acked-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index 96757e2..9f5a06e 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -519,7 +519,7 @@ static struct sw_flow *flow_lookup(struct flow_table *tbl, struct sw_flow_mask *mask; int i; - if (*index < ma->max) { + if (likely(*index < ma->max)) { mask = rcu_dereference_ovsl(ma->masks[*index]); if (mask) { flow = masked_flow_lookup(ti, key, mask, n_mask_hit); @@ -534,7 +534,7 @@ static struct sw_flow *flow_lookup(struct flow_table *tbl, continue; mask = rcu_dereference_ovsl(ma->masks[i]); - if (!mask) + if (unlikely(!mask)) break; flow = masked_flow_lookup(ti, key, mask, n_mask_hit); From patchwork Fri Nov 1 14:23:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="BxcJn5Fb"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474PkC0vTPz9sPj for ; Sat, 2 Nov 2019 01:30:43 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 792A2E1C; Fri, 1 Nov 2019 14:24:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id EB311DDD for ; Fri, 1 Nov 2019 14:24:34 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 50DA287B for ; Fri, 1 Nov 2019 14:24:34 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id 21so7178236pfj.9 for ; Fri, 01 Nov 2019 07:24:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6q680XjywUSv96PIeU7IzP0LK6sHMsve52sMUypjSwk=; b=BxcJn5Fb7H5m1n6lQyX9jmYail7aYR3sWyw5nQVYKWtk2bmH3kwv330m/X54hz4LKs QCX8ajlwWExTyJSRG0xDZwCQZGJey9aKGMUpTA7cVva1j/NxYEN9tkf3RlHsWyffJJrG jwEUM5Gbjk9VeSRuTdYqwHkXXSC8Fphbr/kC9IlMh+kkuos4ugI5vyDKvtTceqN9DBKO yDS/g9YglztLPgIQcHVfdulb3avV+RNF5IW87lH9QVba716w1Ggg4Q8PIgmxknFrzGF+ Jx+b8cCbe9ceYEWm8pOooCvfvIYH/vD7ZMdVlCAGvGObGc6gUzLaDL2NV7WlYR5DnX7a QWaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6q680XjywUSv96PIeU7IzP0LK6sHMsve52sMUypjSwk=; b=sjdbBKFO6NJ98lueh1u53tdXCUbQuiu4dKPe45p1iZdmXdMzv4o5BdwDkFUiSXKhCp EmGCMOQ/8pyhruQtf5BXwNSWbHh9j7MTkI/rtjTKy8mSIBMncy0qd0vx79Z+FIK0G1pE XoYrpHrfAlGscCPq+mypke/INqJGF9VJgEQGd2dlm8rB9LsA4C6vQtQfF8AJiaEsXexe S945QKa6bZGlthSP/pRadUFnroF/ly6uhL9hpPvG5Vo3OgbgdnGMmT3GFbXp2DM8ludi nJ8+1iVZp3fXXjYy5OVv/7xEE1FjezUFrAYxUVKroxqB/f1eYR0+lKCf9skkyVJx3kK+ K4UA== X-Gm-Message-State: APjAAAWK+PQGwbVS97lKQAROQHpgHSt4JUUdIFJ5YLTfQMRnl7lAiCV1 Z0Bcycuwn0g+wx6EOWLp/7M= X-Google-Smtp-Source: APXvYqwhYxXmaQeMf05f+UVKHkYaIZsmFjrWbeYnoZnADChJi18A1qxKC1fwMVx4ZcKrDMMUGzb+hw== X-Received: by 2002:a63:cf4d:: with SMTP id b13mr13396436pgj.396.1572618273848; Fri, 01 Nov 2019 07:24:33 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:33 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:52 +0800 Message-Id: <1572618234-6904-9-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 08/10] net: openvswitch: fix possible memleak on destroy flow-table X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang When we destroy the flow tables which may contain the flow_mask, so release the flow mask struct. Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: Pravin B Shelar --- net/openvswitch/flow_table.c | 186 +++++++++++++++++++++++-------------------- 1 file changed, 98 insertions(+), 88 deletions(-) diff --git a/net/openvswitch/flow_table.c b/net/openvswitch/flow_table.c index 9f5a06e..5904e93 100644 --- a/net/openvswitch/flow_table.c +++ b/net/openvswitch/flow_table.c @@ -210,6 +210,74 @@ static int tbl_mask_array_realloc(struct flow_table *tbl, int size) return 0; } +static int tbl_mask_array_add_mask(struct flow_table *tbl, + struct sw_flow_mask *new) +{ + struct mask_array *ma = ovsl_dereference(tbl->mask_array); + int err, ma_count = READ_ONCE(ma->count); + + if (ma_count >= ma->max) { + err = tbl_mask_array_realloc(tbl, ma->max + + MASK_ARRAY_SIZE_MIN); + if (err) + return err; + + ma = ovsl_dereference(tbl->mask_array); + } + + BUG_ON(ovsl_dereference(ma->masks[ma_count])); + + rcu_assign_pointer(ma->masks[ma_count], new); + WRITE_ONCE(ma->count, ma_count +1); + + return 0; +} + +static void tbl_mask_array_del_mask(struct flow_table *tbl, + struct sw_flow_mask *mask) +{ + struct mask_array *ma = ovsl_dereference(tbl->mask_array); + int i, ma_count = READ_ONCE(ma->count); + + /* Remove the deleted mask pointers from the array */ + for (i = 0; i < ma_count; i++) { + if (mask == ovsl_dereference(ma->masks[i])) + goto found; + } + + BUG(); + return; + +found: + WRITE_ONCE(ma->count, ma_count -1); + + rcu_assign_pointer(ma->masks[i], ma->masks[ma_count -1]); + RCU_INIT_POINTER(ma->masks[ma_count -1], NULL); + + kfree_rcu(mask, rcu); + + /* Shrink the mask array if necessary. */ + if (ma->max >= (MASK_ARRAY_SIZE_MIN * 2) && + ma_count <= (ma->max / 3)) + tbl_mask_array_realloc(tbl, ma->max / 2); +} + +/* Remove 'mask' from the mask list, if it is not needed any more. */ +static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) +{ + if (mask) { + /* ovs-lock is required to protect mask-refcount and + * mask list. + */ + ASSERT_OVSL(); + BUG_ON(!mask->ref_count); + mask->ref_count--; + + if (!mask->ref_count) + tbl_mask_array_del_mask(tbl, mask); + } +} + int ovs_flow_tbl_init(struct flow_table *table) { struct table_instance *ti, *ufid_ti; @@ -257,7 +325,28 @@ static void flow_tbl_destroy_rcu_cb(struct rcu_head *rcu) __table_instance_destroy(ti); } -static void table_instance_destroy(struct table_instance *ti, +static void table_instance_flow_free(struct flow_table *table, + struct table_instance *ti, + struct table_instance *ufid_ti, + struct sw_flow *flow, + bool count) +{ + hlist_del_rcu(&flow->flow_table.node[ti->node_ver]); + if (count) + table->count--; + + if (ovs_identifier_is_ufid(&flow->id)) { + hlist_del_rcu(&flow->ufid_table.node[ufid_ti->node_ver]); + + if (count) + table->ufid_count--; + } + + flow_mask_remove(table, flow->mask); +} + +static void table_instance_destroy(struct flow_table *table, + struct table_instance *ti, struct table_instance *ufid_ti, bool deferred) { @@ -274,13 +363,12 @@ static void table_instance_destroy(struct table_instance *ti, struct sw_flow *flow; struct hlist_head *head = &ti->buckets[i]; struct hlist_node *n; - int ver = ti->node_ver; - int ufid_ver = ufid_ti->node_ver; - hlist_for_each_entry_safe(flow, n, head, flow_table.node[ver]) { - hlist_del_rcu(&flow->flow_table.node[ver]); - if (ovs_identifier_is_ufid(&flow->id)) - hlist_del_rcu(&flow->ufid_table.node[ufid_ver]); + hlist_for_each_entry_safe(flow, n, head, + flow_table.node[ti->node_ver]) { + + table_instance_flow_free(table, ti, ufid_ti, + flow, false); ovs_flow_free(flow, deferred); } } @@ -305,7 +393,7 @@ void ovs_flow_tbl_destroy(struct flow_table *table) free_percpu(table->mask_cache); kfree_rcu(rcu_dereference_raw(table->mask_array), rcu); - table_instance_destroy(ti, ufid_ti, false); + table_instance_destroy(table, ti, ufid_ti, false); } struct sw_flow *ovs_flow_tbl_dump_next(struct table_instance *ti, @@ -421,7 +509,7 @@ int ovs_flow_tbl_flush(struct flow_table *flow_table) flow_table->count = 0; flow_table->ufid_count = 0; - table_instance_destroy(old_ti, old_ufid_ti, true); + table_instance_destroy(flow_table, old_ti, old_ufid_ti, true); return 0; err_free_ti: @@ -701,51 +789,6 @@ static struct table_instance *table_instance_expand(struct table_instance *ti, return table_instance_rehash(ti, ti->n_buckets * 2, ufid); } -static void tbl_mask_array_del_mask(struct flow_table *tbl, - struct sw_flow_mask *mask) -{ - struct mask_array *ma = ovsl_dereference(tbl->mask_array); - int i, ma_count = READ_ONCE(ma->count); - - /* Remove the deleted mask pointers from the array */ - for (i = 0; i < ma_count; i++) { - if (mask == ovsl_dereference(ma->masks[i])) - goto found; - } - - BUG(); - return; - -found: - WRITE_ONCE(ma->count, ma_count -1); - - rcu_assign_pointer(ma->masks[i], ma->masks[ma_count -1]); - RCU_INIT_POINTER(ma->masks[ma_count -1], NULL); - - kfree_rcu(mask, rcu); - - /* Shrink the mask array if necessary. */ - if (ma->max >= (MASK_ARRAY_SIZE_MIN * 2) && - ma_count <= (ma->max / 3)) - tbl_mask_array_realloc(tbl, ma->max / 2); -} - -/* Remove 'mask' from the mask list, if it is not needed any more. */ -static void flow_mask_remove(struct flow_table *tbl, struct sw_flow_mask *mask) -{ - if (mask) { - /* ovs-lock is required to protect mask-refcount and - * mask list. - */ - ASSERT_OVSL(); - BUG_ON(!mask->ref_count); - mask->ref_count--; - - if (!mask->ref_count) - tbl_mask_array_del_mask(tbl, mask); - } -} - /* Must be called with OVS mutex held. */ void ovs_flow_tbl_remove(struct flow_table *table, struct sw_flow *flow) { @@ -753,17 +796,7 @@ void ovs_flow_tbl_remove(struct flow_table *table, struct sw_flow *flow) struct table_instance *ufid_ti = ovsl_dereference(table->ufid_ti); BUG_ON(table->count == 0); - hlist_del_rcu(&flow->flow_table.node[ti->node_ver]); - table->count--; - if (ovs_identifier_is_ufid(&flow->id)) { - hlist_del_rcu(&flow->ufid_table.node[ufid_ti->node_ver]); - table->ufid_count--; - } - - /* RCU delete the mask. 'flow->mask' is not NULLed, as it should be - * accessible as long as the RCU read lock is held. - */ - flow_mask_remove(table, flow->mask); + table_instance_flow_free(table, ti, ufid_ti, flow, true); } static struct sw_flow_mask *mask_alloc(void) @@ -806,29 +839,6 @@ static struct sw_flow_mask *flow_mask_find(const struct flow_table *tbl, return NULL; } -static int tbl_mask_array_add_mask(struct flow_table *tbl, - struct sw_flow_mask *new) -{ - struct mask_array *ma = ovsl_dereference(tbl->mask_array); - int err, ma_count = READ_ONCE(ma->count); - - if (ma_count >= ma->max) { - err = tbl_mask_array_realloc(tbl, ma->max + - MASK_ARRAY_SIZE_MIN); - if (err) - return err; - - ma = ovsl_dereference(tbl->mask_array); - } - - BUG_ON(ovsl_dereference(ma->masks[ma_count])); - - rcu_assign_pointer(ma->masks[ma_count], new); - WRITE_ONCE(ma->count, ma_count +1); - - return 0; -} - /* Add 'mask' into the mask list, if it is not already there. */ static int flow_mask_insert(struct flow_table *tbl, struct sw_flow *flow, const struct sw_flow_mask *new) From patchwork Fri Nov 1 14:23:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188088 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="YMSQDLwC"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474Pks6gn4z9sP4 for ; Sat, 2 Nov 2019 01:31:17 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 10E97E43; Fri, 1 Nov 2019 14:24:40 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B683FE35 for ; Fri, 1 Nov 2019 14:24:38 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 66F4187B for ; Fri, 1 Nov 2019 14:24:38 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id f19so6572886pgn.13 for ; Fri, 01 Nov 2019 07:24:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qNMP2DUI9muUqsSG2DrAsJDDEz4pD/8p5Z2KkI73k2w=; b=YMSQDLwCkQ9D7Sjv3BZ2JEtUaIOKeQGpbV1eLbu67TlC1bacAWwPv/fyr/mDgdhA+0 IhXUI9YUAJohjcB+X/Jy2Ck4zaIO1VCQSzXtDDSnkJ/v8WfJwSLx5Zf/QjgeoqfLMlTz kBq+VurhdGkuXnz1M/IaQXBWNVzZ8kH1RKFO3zVxRNOhkdjuOgi2V9DjOkqzB+Li+ux8 V8GApF5/FA8849lTf407w1WGvWONnAgLJ3kYqFZi2qudJU9tEeMqBGrb46G6aeXv90a1 PzNOxtmf+DERlUeoaBND4JEtMN6Aqn29NE9d/bvZugRu1QjBGYiUEaxH1Fd5mdDOe7TD dtkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qNMP2DUI9muUqsSG2DrAsJDDEz4pD/8p5Z2KkI73k2w=; b=QZ2p0h0C0jgHwbpsoG/er1DQJd+cpvF5z0MFQjjimha1Ed6e8wEiAAAFnGLyOZYr+Q QMVaJlaSsOI/vJniYJmQesANhxrQBpHjIWycdqhThwEBdN2/j5POqVzBN0XYvG2HIsgw T5TtNximG0mnl/pYIvdWBwEwa/552ESH5bFh0Oq+wC80ZgMWc5k+yrLPr8kFQjRSWZ2O TRJWhx01Ka46DhbN/97avIFrqsAe1m1GiTnnmRWQQsBssPUe2s9QQ5n5oWqD0UdYhcDV s6hRxrzzn5EiRU85YG3uut5A+kkEQcLHXSvo99R6O4vX0Rw4WuTHtTjU1E2lE1Rq7D2s epNA== X-Gm-Message-State: APjAAAV+KkSGavU9Wgw2qdA42UvqemhW/thSMU6EUxg/pasden632u1m tq6L/7pEcr9mSU6TZqn55J8= X-Google-Smtp-Source: APXvYqxUJI5vNrS6FrGh2YItyKO5+LYyTSponiJn+0llW9qKZPgtvI449KFffgsYXumQU2JDWyumDg== X-Received: by 2002:a63:5b56:: with SMTP id l22mr14147735pgm.52.1572618278082; Fri, 01 Nov 2019 07:24:38 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:37 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:53 +0800 Message-Id: <1572618234-6904-10-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 09/10] net: openvswitch: don't unlock mutex when changing the user_features fails X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang Unlocking of a not locked mutex is not allowed. Other kernel thread may be in critical section while we unlock it because of setting user_feature fail. Fixes: 95a7233c4 ("net: openvswitch: Set OvS recirc_id from tc chain index") Cc: Paul Blakey Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: William Tu Acked-by: Pravin B Shelar --- net/openvswitch/datapath.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index 9fea7e1..aeb76e4 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -1657,6 +1657,7 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) ovs_dp_reset_user_features(skb, info); } + ovs_unlock(); goto err_destroy_meters; } @@ -1673,7 +1674,6 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) return 0; err_destroy_meters: - ovs_unlock(); ovs_meters_exit(dp); err_destroy_ports_array: kfree(dp->ports); From patchwork Fri Nov 1 14:23:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tonghao Zhang X-Patchwork-Id: 1188092 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="oCqb2Luf"; dkim-atps=neutral Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474PlT0qdyz9sR1 for ; Sat, 2 Nov 2019 01:31:49 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id AF8AEE8C; Fri, 1 Nov 2019 14:24:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5DC27E59 for ; Fri, 1 Nov 2019 14:24:43 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 016F114D for ; Fri, 1 Nov 2019 14:24:42 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id z24so2043596pgu.4 for ; Fri, 01 Nov 2019 07:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xFv3hcOcNKmj8ZwltKq36bAhvNe+rhJRgoAZb20SCBc=; b=oCqb2LufV+1s3CI4oUZsXddPpfQG3yK8NmQOsOgP1D7C1otBNdF1mkR2CPcbVAGwg0 W8J88nj3UGzEt+vAypas32iVCAtfvkrMK3Lxyyx7x3iJojqiiXcscsIAq/vLu3mjJe/n vxFvwAGlbrJt/hwvN9LhBVD79VxDN0co6ySWso+EpeKARZQMrz7sT2ZNGxBvA7SBeeKF BchF0pF61OADjSFPHg04zPOUgS0FfIMXCFFtN46Usahh/SMJcmDz8JzxRwHqBkrRjvBU W7cf9fyiGjVEr5AjNWs3veCif7RkWv/XTuamQBClacYQmm5qkS4ai5UB/jbC08x7stKU zPgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xFv3hcOcNKmj8ZwltKq36bAhvNe+rhJRgoAZb20SCBc=; b=YHFvpGdBBhC14LX7wOoQeka/5WLL/LI51UG/SzD7kDAcwqarwqwukR3nUNwTMz0c7J l+wm3pPW6L7wFxoVqd+I8JERxvICvAHQ1aEHU944UdcX3RQQA+CJ+TlpXgZfs2TdbnWl 5qf0GKt1EO5tYDaoTPXuu6wWX1pkex11KVtbgDxwf+tG8wS/VJ6utEBGRIRfs28FDgxe nWz8VZKEI9X0SBBVtWRzf7ReKP4SJaRkIX5g7GXTDaotMfDUdHA5o+1kwx5e9vo1z9A6 RtjoM/srlpiXrK+ZDM0EpZvjBQ2OxJGcOOhIvrdcZ8m+rNTgpgH3lNuPvC0GAhRbMJSq htPQ== X-Gm-Message-State: APjAAAV8id0e4esQBwIcSFm97Kj63Y5E7Jz/OM8YO66ZDcUDaYfDoI9G BXp/hwE7Ifq8BLJNHbx1fkk= X-Google-Smtp-Source: APXvYqwvt51AGny15fZk1/0fZXsCjGAGSSSDHnGOikoFlGz26J36RXn59Q139wewVSIPgL5087zyAA== X-Received: by 2002:a65:6290:: with SMTP id f16mr13990808pgv.40.1572618282644; Fri, 01 Nov 2019 07:24:42 -0700 (PDT) Received: from local.opencloud.tech.localdomain ([1.203.173.208]) by smtp.gmail.com with ESMTPSA id c12sm8296499pfp.67.2019.11.01.07.24.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 Nov 2019 07:24:42 -0700 (PDT) From: xiangxia.m.yue@gmail.com To: gvrose8192@gmail.com, pshelar@ovn.org, davem@davemloft.net Date: Fri, 1 Nov 2019 22:23:54 +0800 Message-Id: <1572618234-6904-11-git-send-email-xiangxia.m.yue@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> References: <1572618234-6904-1-git-send-email-xiangxia.m.yue@gmail.com> X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: dev@openvswitch.org, netdev@vger.kernel.org Subject: [ovs-dev] [PATCH net-next v6 10/10] net: openvswitch: simplify the ovs_dp_cmd_new X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org From: Tonghao Zhang use the specified functions to init resource. Signed-off-by: Tonghao Zhang Tested-by: Greg Rose Acked-by: Pravin B Shelar --- net/openvswitch/datapath.c | 60 +++++++++++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 22 deletions(-) diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index aeb76e4..4d48e48 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -1576,6 +1576,31 @@ static int ovs_dp_change(struct datapath *dp, struct nlattr *a[]) return 0; } +static int ovs_dp_stats_init(struct datapath *dp) +{ + dp->stats_percpu = netdev_alloc_pcpu_stats(struct dp_stats_percpu); + if (!dp->stats_percpu) + return -ENOMEM; + + return 0; +} + +static int ovs_dp_vport_init(struct datapath *dp) +{ + int i; + + dp->ports = kmalloc_array(DP_VPORT_HASH_BUCKETS, + sizeof(struct hlist_head), + GFP_KERNEL); + if (!dp->ports) + return -ENOMEM; + + for (i = 0; i < DP_VPORT_HASH_BUCKETS; i++) + INIT_HLIST_HEAD(&dp->ports[i]); + + return 0; +} + static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) { struct nlattr **a = info->attrs; @@ -1584,7 +1609,7 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) struct datapath *dp; struct vport *vport; struct ovs_net *ovs_net; - int err, i; + int err; err = -EINVAL; if (!a[OVS_DP_ATTR_NAME] || !a[OVS_DP_ATTR_UPCALL_PID]) @@ -1597,35 +1622,26 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) err = -ENOMEM; dp = kzalloc(sizeof(*dp), GFP_KERNEL); if (dp == NULL) - goto err_free_reply; + goto err_destroy_reply; ovs_dp_set_net(dp, sock_net(skb->sk)); /* Allocate table. */ err = ovs_flow_tbl_init(&dp->table); if (err) - goto err_free_dp; + goto err_destroy_dp; - dp->stats_percpu = netdev_alloc_pcpu_stats(struct dp_stats_percpu); - if (!dp->stats_percpu) { - err = -ENOMEM; + err = ovs_dp_stats_init(dp); + if (err) goto err_destroy_table; - } - dp->ports = kmalloc_array(DP_VPORT_HASH_BUCKETS, - sizeof(struct hlist_head), - GFP_KERNEL); - if (!dp->ports) { - err = -ENOMEM; - goto err_destroy_percpu; - } - - for (i = 0; i < DP_VPORT_HASH_BUCKETS; i++) - INIT_HLIST_HEAD(&dp->ports[i]); + err = ovs_dp_vport_init(dp); + if (err) + goto err_destroy_stats; err = ovs_meters_init(dp); if (err) - goto err_destroy_ports_array; + goto err_destroy_ports; /* Set up our datapath device. */ parms.name = nla_data(a[OVS_DP_ATTR_NAME]); @@ -1675,15 +1691,15 @@ static int ovs_dp_cmd_new(struct sk_buff *skb, struct genl_info *info) err_destroy_meters: ovs_meters_exit(dp); -err_destroy_ports_array: +err_destroy_ports: kfree(dp->ports); -err_destroy_percpu: +err_destroy_stats: free_percpu(dp->stats_percpu); err_destroy_table: ovs_flow_tbl_destroy(&dp->table); -err_free_dp: +err_destroy_dp: kfree(dp); -err_free_reply: +err_destroy_reply: kfree_skb(reply); err: return err;