From patchwork Fri Sep 25 09:57:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371132 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6V3XXyz9s0b for ; Fri, 25 Sep 2020 19:58:42 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id E621D2E18A; Fri, 25 Sep 2020 09:58:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Lh2ZLqCFnZJP; Fri, 25 Sep 2020 09:58:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id DA9902E149; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CA574C0890; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 89307C0859 for ; Fri, 25 Sep 2020 09:58:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 7900086C43 for ; Fri, 25 Sep 2020 09:58:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0ypBgEaI6CBD for ; Fri, 25 Sep 2020 09:58:22 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by fraxinus.osuosl.org (Postfix) with ESMTPS id E1D4486C3E for ; Fri, 25 Sep 2020 09:58:21 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUd-0000rX-Jx; Fri, 25 Sep 2020 09:58:20 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUZ-00059v-JN; Fri, 25 Sep 2020 10:58:17 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:57:59 +0100 Message-Id: <20200925095807.19358-2-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 1/9] ovn-libs: Add support for parallel processing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov This adds a set of functions and macros intended to process hashes in parallel. The principles of operation are documented in the fasthmap.h If these one day go into the OVS tree, the OVS tree versions would be used in preference. Signed-off-by: Anton Ivanov --- lib/automake.mk | 2 + lib/fasthmap.c | 373 ++++++++++++++++++++++++++++++++++++++++++++++++ lib/fasthmap.h | 217 ++++++++++++++++++++++++++++ 3 files changed, 592 insertions(+) create mode 100644 lib/fasthmap.c create mode 100644 lib/fasthmap.h diff --git a/lib/automake.mk b/lib/automake.mk index f3e9c8818..c9bf7e021 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -13,6 +13,8 @@ lib_libovn_la_SOURCES = \ lib/expr.c \ lib/extend-table.h \ lib/extend-table.c \ + lib/fasthmap.h \ + lib/fasthmap.c \ lib/ip-mcast-index.c \ lib/ip-mcast-index.h \ lib/mcast-group-index.c \ diff --git a/lib/fasthmap.c b/lib/fasthmap.c new file mode 100644 index 000000000..38b06d74a --- /dev/null +++ b/lib/fasthmap.c @@ -0,0 +1,373 @@ +/* + * Copyright (c) 2020 Red Hat, Inc. + * Copyright (c) 2008, 2009, 2010, 2012, 2013, 2015, 2019 Nicira, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include +#include +#include +#include "fatal-signal.h" +#include "util.h" +#include "openvswitch/vlog.h" +#include "openvswitch/hmap.h" +#include "openvswitch/thread.h" +#include "fasthmap.h" +#include "ovs-atomic.h" +#include "ovs-thread.h" +#include "ovs-numa.h" + +VLOG_DEFINE_THIS_MODULE(fasthmap); + + +static bool worker_pool_setup = false; +static bool workers_must_exit = false; + +static struct ovs_list worker_pools = OVS_LIST_INITIALIZER(&worker_pools); + +static struct ovs_mutex init_mutex = OVS_MUTEX_INITIALIZER; + +static int pool_size; + +static void worker_pool_hook(void *aux OVS_UNUSED) { + int i; + static struct worker_pool *pool; + workers_must_exit = true; /* all workers must honour this flag */ + atomic_thread_fence(memory_order_release); + LIST_FOR_EACH (pool, list_node, &worker_pools) { + for (i = 0; i < pool->size ; i++) { + sem_post(&pool->controls[i].fire); + } + } +} + +static void setup_worker_pools(void) { + int cores, nodes; + + nodes = ovs_numa_get_n_numas(); + if (nodes == OVS_NUMA_UNSPEC || nodes <= 0) { + nodes = 1; + } + cores = ovs_numa_get_n_cores(); + + /* If there is no NUMA config, use 4 cores. + * If there is NUMA config use half the cores on + * one node so that the OS does not start pushing + * threads to other nodes. + */ + if (cores == OVS_CORE_UNSPEC || cores <= 0) { + pool_size = 4; + } else { + pool_size = cores / nodes / 2; + } + fatal_signal_add_hook(worker_pool_hook, NULL, NULL, true); + worker_pool_setup = true; +} + +bool ovn_seize_fire(void) +{ + return workers_must_exit; +} + +struct worker_pool *ovn_add_worker_pool(void *(*start)(void *)){ + + struct worker_pool *new_pool = NULL; + struct worker_control *new_control; + int i; + + ovs_mutex_lock(&init_mutex); + + if (!worker_pool_setup) { + setup_worker_pools(); + } + + new_pool = xmalloc(sizeof(struct worker_pool)); + new_pool->size = pool_size; + sem_init(&new_pool->done, 0, 0); + + ovs_list_push_back(&worker_pools, &new_pool->list_node); + + new_pool->controls = + xmalloc(sizeof(struct worker_control) * new_pool->size); + + for (i = 0; i < new_pool->size; i++) { + new_control = &new_pool->controls[i]; + sem_init(&new_control->fire, 0, 0); + new_control->id = i; + new_control->done = &new_pool->done; + new_control->data = NULL; + ovs_mutex_init(&new_control->mutex); + new_control->finished = ATOMIC_VAR_INIT(false); + } + + for (i = 0; i < pool_size; i++) { + ovs_thread_create("worker pool helper", start, &new_pool->controls[i]); + } + ovs_mutex_unlock(&init_mutex); + return new_pool; +} + + +/* Initializes 'hmap' as an empty hash table with mask N. */ +void +ovn_fast_hmap_init(struct hmap *hmap, ssize_t mask) +{ + size_t i; + + hmap->buckets = xmalloc(sizeof (struct hmap_node *) * (mask + 1)); + hmap->one = NULL; + hmap->mask = mask; + hmap->n = 0; + for (i = 0; i <= hmap->mask; i++) { + hmap->buckets[i] = NULL; + } +} + +/* Initializes 'hmap' as an empty hash table of size X. + * Intended for use in parallel processing so that all + * fragments used to store results in a parallel job + * are the same size. + */ +void +ovn_fast_hmap_size_for(struct hmap *hmap, int size) +{ + size_t mask; + mask = size / 2; + mask |= mask >> 1; + mask |= mask >> 2; + mask |= mask >> 4; + mask |= mask >> 8; + mask |= mask >> 16; +#if SIZE_MAX > UINT32_MAX + mask |= mask >> 32; +#endif + + /* If we need to dynamically allocate buckets we might as well allocate at + * least 4 of them. */ + mask |= (mask & 1) << 1; + + fast_hmap_init(hmap, mask); +} + +/* Run a thread pool - basic, does not do results processing. + */ + +void ovn_run_pool(struct worker_pool *pool) +{ + int index, completed; + + atomic_thread_fence(memory_order_release); + + for (index = 0; index < pool->size; index++) { + sem_post(&pool->controls[index].fire); + } + + completed = 0; + + do { + bool test; + sem_wait(&pool->done); + for (index = 0; index < pool->size; index++) { + test = true; + if (atomic_compare_exchange_weak( + &pool->controls[index].finished, + &test, + false)) { + completed++; + pool->controls[index].data = NULL; + } + } + } while (completed < pool->size); +} + +/* Brute force merge of a hashmap into another hashmap. + * Intended for use in parallel processing. The destination + * hashmap MUST be the same size as the one being merged. + * + * This can be achieved by pre-allocating them to correct size + * and using hmap_insert_fast() instead of hmap_insert() + */ + +void ovn_fast_hmap_merge(struct hmap *dest, struct hmap *inc) +{ + size_t i; + + ovs_assert(inc->mask == dest->mask); + + if (!inc->n) { + /* Request to merge an empty frag, nothing to do */ + return; + } + + for (i = 0; i <= dest->mask; i++) { + struct hmap_node **dest_bucket = &dest->buckets[i]; + struct hmap_node **inc_bucket = &inc->buckets[i]; + if (*inc_bucket != NULL) { + struct hmap_node *last_node = *inc_bucket; + while (last_node->next != NULL) { + last_node = last_node->next; + } + last_node->next = *dest_bucket; + *dest_bucket = *inc_bucket; + *inc_bucket = NULL; + } + } + dest->n += inc->n; + inc->n = 0; +} + +/* Run a thread pool which gathers results in an array + * of hashes. Merge results. + */ + +void ovn_run_pool_hash( + struct worker_pool *pool, + struct hmap *result, + struct hmap *result_frags) +{ + int index, completed; + + atomic_thread_fence(memory_order_release); + + for (index = 0; index < pool->size; index++) { + sem_post(&pool->controls[index].fire); + } + + completed = 0; + + do { + bool test; + sem_wait(&pool->done); + for (index = 0; index < pool->size; index++) { + test = true; + if (atomic_compare_exchange_weak( + &pool->controls[index].finished, + &test, + false)) { + fast_hmap_merge(result, &result_frags[index]); + hmap_destroy(&result_frags[index]); + completed++; + pool->controls[index].data = NULL; + } + } + } while (completed < pool->size); +} + +void ovn_merge_lists(struct ovs_list **dest, struct ovs_list *inc) +{ + struct ovs_list *last, *first; + if (inc == NULL) { + return; + } + + if (* dest == NULL) { + * dest = inc; + return; + } + + if (ovs_list_is_empty(inc)) { + return; + } + + if (ovs_list_is_empty(*dest)) { + * dest = inc; + return; + } + + + last = inc->prev; + /* first element is not the list pointer itself, it is the ->next */ + first = inc->next; + + (*dest)->prev->next = first; + first->prev = (*dest)->prev; + + (*dest)->prev = last; + last->next = *dest; +} + +/* Run a thread pool which gathers results in an array + * of lists. Merge results + */ + +void ovn_run_pool_list( + struct worker_pool *pool, + struct ovs_list **result, + struct ovs_list **result_frags) +{ + int index, completed; + + atomic_thread_fence(memory_order_release); + + + for (index = 0; index < pool->size; index++) { + sem_post(&pool->controls[index].fire); + } + + completed = 0; + + + do { + bool test; + sem_wait(&pool->done); + for (index = 0; index < pool->size; index++) { + test = true; + if (atomic_compare_exchange_weak( + &pool->controls[index].finished, + &test, + false)) { + merge_lists(result, result_frags[index]); + completed++; + pool->controls[index].data = NULL; + } + } + } while (completed < pool->size); +} + +/* Run a thread pool which uses a callback function to process results + */ + +void ovn_run_pool_callback( + struct worker_pool *pool, + void *fin_result, + void (*helper_func)( + struct worker_pool *pool, void *fin_result, int index)) +{ + int index, completed; + + atomic_thread_fence(memory_order_release); + + for (index = 0; index < pool->size; index++) { + sem_post(&pool->controls[index].fire); + } + + completed = 0; + + do { + bool test; + sem_wait(&pool->done); + for (index = 0; index < pool->size; index++) { + test = true; + if (atomic_compare_exchange_weak( + &pool->controls[index].finished, + &test, + false)) { + (helper_func)(pool, fin_result, index); + completed++; + pool->controls[index].data = NULL; + } + } + } while (completed < pool->size); +} diff --git a/lib/fasthmap.h b/lib/fasthmap.h new file mode 100644 index 000000000..ccf005475 --- /dev/null +++ b/lib/fasthmap.h @@ -0,0 +1,217 @@ +/* + * Copyright (c) 2020 Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef HMAP_HAS_PARALLEL_MACROS +#define HMAP_HAS_PARALLEL_MACROS 1 + +/* if the parallel macros are defined by hmap.h or any other ovs define + * we skip over the ovn specific definitions. + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include +#include "openvswitch/util.h" +#include "openvswitch/hmap.h" +#include "openvswitch/thread.h" +#include "ovs-atomic.h" + +/* A version of the HMAP_FOR_EACH macro intended for iterating as part + * of parallel processing. + * Each worker thread has a different ThreadID in the range of 0..POOL_SIZE + * and will iterate hash buckets ThreadID, ThreadID + step, + * ThreadID + step * 2, etc. The actual macro accepts + * ThreadID + step * i as the JOBID parameter. + */ + +#define HMAP_FOR_EACH_IN_PARALLEL(NODE, MEMBER, JOBID, HMAP) \ + for (INIT_CONTAINER(NODE, hmap_first_in_bucket_num(HMAP, JOBID), MEMBER); \ + (NODE != OBJECT_CONTAINING(NULL, NODE, MEMBER)) \ + || ((NODE = NULL), false); \ + ASSIGN_CONTAINER(NODE, hmap_next_in_bucket(&(NODE)->MEMBER), MEMBER)) + +/* We do not have a SAFE version of the macro, because the hash size is not + * atomic and hash removal operations would need to be wrapped with + * locks. This will defeat most of the benefits from doing anything in + * parallel. + * If the code block inside FOR_EACH_IN_PARALLEL needs to remove elements, + * each thread should store them in a temporary list result instead, merging + * the lists into a combined result at the end */ + +/* Work "Handle" */ + +struct worker_control { + int id; /* Used as a modulo when iterating over a hash. */ + atomic_bool finished; /* Set to true after achunk of work is complete. */ + sem_t fire; /* Work start semaphore - sem_post starts the worker. */ + sem_t *done; /* Work completion semaphore - sem_post on completion. */ + struct ovs_mutex mutex; /* Guards the data. */ + void *data; /* Pointer to data to be processed. */ + void *workload; /* back-pointer to the worker pool structure. */ +}; + +struct worker_pool { + int size; /* Number of threads in the pool. */ + struct ovs_list list_node; /* List of pools - used in cleanup/exit. */ + struct worker_control *controls; /* "Handles" in this pool. */ + sem_t done; /* Work completion semaphorew. */ +}; + +/* Add a worker pool for thread function start() which expects a pointer to + * a worker_control structure as an argument. */ + +struct worker_pool *ovn_add_worker_pool(void *(*start)(void *)); + +/* Setting this to true will make all processing threads exit */ + +bool ovn_seize_fire(void); + +/* Build a hmap pre-sized for size elements */ + +void ovn_fast_hmap_size_for(struct hmap *hmap, int size); + +/* Build a hmap with a mask equals to size */ + +void ovn_fast_hmap_init(struct hmap *hmap, ssize_t size); + +/* Brute-force merge a hmap into hmap. + * Dest and inc have to have the same mask. The merge is performed + * by extending the element list for bucket N in the dest hmap with the list + * from bucket N in inc. + */ + +void ovn_fast_hmap_merge(struct hmap *dest, struct hmap *inc); + +/* Merge two lists. + * It is possible to achieve the same functionality using ovs_list_splice(). + * This ensures the splicing is exactly for tail of dest to head of inc. + */ + +void ovn_merge_lists(struct ovs_list **dest, struct ovs_list *inc); + +/* Run a pool, without any default processing of results. + */ + +void ovn_run_pool(struct worker_pool *pool); + +/* Run a pool, merge results from hash frags into a final hash result. + * The hash frags must be pre-sized to the same size. + */ + +void ovn_run_pool_hash(struct worker_pool *pool, + struct hmap *result, struct hmap *result_frags); + +/* Run a pool, merge results from lists into a final combined list. + * The hash frags must be pre-sized to the same size. + */ + +void ovn_run_pool_list(struct worker_pool *pool, + struct ovs_list **result, struct ovs_list **result_frags); + +/* Run a pool, call a callback function to perform processing of results. + */ + +void ovn_run_pool_callback(struct worker_pool *pool, void *fin_result, + void (*helper_func)(struct worker_pool *pool, + void *fin_result, int index)); + + +/* Returns the first node in 'hmap' in the bucket in which the given 'hash' + * would land, or a null pointer if that bucket is empty. */ + +static inline struct hmap_node * +hmap_first_in_bucket_num(const struct hmap *hmap, size_t num) +{ + return hmap->buckets[num]; +} + +static inline struct hmap_node * +parallel_hmap_next__(const struct hmap *hmap, size_t start, size_t pool_size) +{ + size_t i; + for (i = start; i <= hmap->mask; i+= pool_size) { + struct hmap_node *node = hmap->buckets[i]; + if (node) { + return node; + } + } + return NULL; +} + +/* Returns the first node in 'hmap', as expected by thread with job_id + * for parallel processing in arbitrary order, or a null pointer if + * the slice of 'hmap' for that job_id is empty. */ +static inline struct hmap_node * +parallel_hmap_first(const struct hmap *hmap, size_t job_id, size_t pool_size) +{ + return parallel_hmap_next__(hmap, job_id, pool_size); +} + +/* Returns the next node in the slice of 'hmap' following 'node', + * in arbitrary order, or a * null pointer if 'node' is the last node in + * the 'hmap' slice. + * + */ +static inline struct hmap_node * +parallel_hmap_next(const struct hmap *hmap, + const struct hmap_node *node, ssize_t pool_size) +{ + return (node->next + ? node->next + : parallel_hmap_next__(hmap, + (node->hash & hmap->mask) + pool_size, pool_size)); +} + +/* Use the OVN library functions for stuff which OVS has not defined + * If OVS has defined these, they will still compile using the OVN + * local names, but will be dropped by the linker in favour of the OVS + * supplied functions. + */ + +#define seize_fire() ovn_seize_fire() + +#define add_worker_pool(start) ovn_add_worker_pool(start) + +#define fast_hmap_size_for(hmap, size) ovn_fast_hmap_size_for(hmap, size) + +#define fast_hmap_init(hmap, size) ovn_fast_hmap_init(hmap, size) + +#define fast_hmap_merge(dest, inc) ovn_fast_hmap_merge(dest, inc) + +#define hmap_merge(dest, inc) ovn_hmap_merge(dest, inc) + +#define merge_lists(dest, inc) ovn_merge_lists(dest, inc) + +#define ovn_run_pool(pool) ovn_run_pool(pool) + +#define run_pool_hash(pool, result, result_frags) \ + ovn_run_pool_hash(pool, result, result_frags) + +#define run_pool_list(pool, result, result_frags) \ + ovn_run_pool_list(pool, result, result_frags) + +#define run_pool_callback(pool, fin_result, helper_func) \ + ovn_run_pool_callback(pool, fin_result, helper_func) + +#ifdef __cplusplus +} +#endif + +#endif /* lib/fast-hmap.h */ From patchwork Fri Sep 25 09:58:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371128 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6J2z83z9s0b for ; Fri, 25 Sep 2020 19:58:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id D17FC86B66; Fri, 25 Sep 2020 09:58:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uomXZZymZCDr; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 00B9986C6E; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E88EFC0890; Fri, 25 Sep 2020 09:58:27 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id E8DE2C1AD5 for ; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D9C9F86A25 for ; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3fZdQApdbaDn for ; Fri, 25 Sep 2020 09:58:23 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by whitealder.osuosl.org (Postfix) with ESMTPS id DA88E86860 for ; Fri, 25 Sep 2020 09:58:22 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUe-0000rc-RO; Fri, 25 Sep 2020 09:58:21 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUb-00059v-Vk; Fri, 25 Sep 2020 10:58:19 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:00 +0100 Message-Id: <20200925095807.19358-3-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 2/9] ovn-northd: reorganize processing of lflows X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Merge lrouter and lswitch processing. 2. Move lrouter and lswitch lflow generation which uses the same iterator variables into common helpers 3. Set up structures to be used in parallel and sequential mode Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 191 ++++++++++++++++++++++++++------------------ 1 file changed, 114 insertions(+), 77 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 3324c9e81..66b6c2985 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -8874,24 +8874,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, struct ds actions = DS_EMPTY_INITIALIZER; struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, datapaths) { - build_adm_ctrl_flows_for_lrouter(od, lflows); - } - struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, ports) { - build_adm_ctrl_flows_for_lrouter_port(op, lflows, &match, &actions); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_neigh_learning_flows_for_lrouter( - od, lflows, &match, &actions); - } - - HMAP_FOR_EACH (op, key_node, ports) { - build_neigh_learning_flows_for_lrouter_port( - op, lflows, &match, &actions); - } /* Drop IP traffic destined to router owned IPs. Part of it is dropped * in stage "lr_in_ip_input" but traffic that could have been unSNATed @@ -9935,63 +9918,6 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, sset_destroy(&nat_entries); } - HMAP_FOR_EACH (op, key_node, ports) { - build_ND_RA_flows_for_lrouter_port(op, lflows, &match, &actions); - } - - /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS - * responder, by default goto next. (priority 0). */ - HMAP_FOR_EACH (od, key_node, datapaths) { - build_ND_RA_flows_for_lrouter(od, lflows); - } - - HMAP_FOR_EACH (op, key_node, ports) { - build_ip_routing_flows_for_lrouter_port(op, lflows); - } - - /* Convert the static routes to flows. */ - HMAP_FOR_EACH (od, key_node, datapaths) { - build_static_route_flows_for_lrouter(od, lflows, ports); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_mcast_lookup_flows_for_lrouter(od, lflows, &match, &actions); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_ingress_policy_flows_for_lrouter(od, lflows, ports); - } - - /* XXX destination unreachable */ - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_arp_resolve_flows_for_lrouter(od, lflows); - } - - HMAP_FOR_EACH (op, key_node, ports) { - build_arp_resolve_flows_for_lrouter_port( - op, lflows, ports, &match, &actions); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_check_pkt_len_flows_for_lrouter( - od, lflows, ports, &match, &actions); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_gateway_redirect_flows_for_lrouter( - od, lflows, &match, &actions); - } - - HMAP_FOR_EACH (od, key_node, datapaths) { - build_arp_request_flows_for_lrouter(od, lflows, &match, &actions); - } - - HMAP_FOR_EACH (op, key_node, ports) { - build_egress_delivery_flows_for_lrouter_port( - op, lflows, &match, &actions); - } - ds_destroy(&match); ds_destroy(&actions); } @@ -11389,6 +11315,117 @@ build_ipv6_input_flows_for_lrouter_port( } +struct lswitch_flow_build_info { + struct hmap *datapaths; + struct hmap *ports; + struct hmap *port_groups; + struct hmap *lflows; + struct hmap *mcgroups; + struct hmap *igmp_groups; + struct shash *meter_groups; + struct hmap *lbs; + char *svc_check_match; + struct ds match; + struct ds actions; +}; + +/* Helper function to combine all lflow generation which is iterated by + * datapath. + */ + +static void +build_lswitch_and_lrouter_iterate_by_od( + struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) +{ + + /* Build Logical Router Flows. */ + build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); + build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); + build_ND_RA_flows_for_lrouter(od, lsi->lflows); + build_static_route_flows_for_lrouter(od, lsi->lflows, lsi->ports); + build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); + build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->ports); + build_arp_resolve_flows_for_lrouter(od, lsi->lflows); + build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->ports, + &lsi->match, &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); + build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); +} + +/* Helper function to combine all lflow generation which is iterated by port. + */ + +static void +build_lswitch_and_lrouter_iterate_by_op( + struct ovn_port *op, + struct lswitch_flow_build_info *lsi) +{ + /* Build Logical Router Flows. */ + + build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + &lsi->actions); + build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + &lsi->actions); + build_ip_routing_flows_for_lrouter_port(op, lsi->lflows); + build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + &lsi->actions); + build_arp_resolve_flows_for_lrouter_port(op, lsi->lflows, lsi->ports, + &lsi->match, &lsi->actions); + build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + &lsi->actions); +} + +static void +build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, + struct hmap *port_groups, struct hmap *lflows, + struct hmap *mcgroups, struct hmap *igmp_groups, + struct shash *meter_groups, + struct hmap *lbs) +{ + + struct ovn_datapath *od; + struct ovn_port *op; + + char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); + + struct lswitch_flow_build_info lsi = { + .datapaths = datapaths, + .ports = ports, + .port_groups = port_groups, + .lflows = lflows, + .mcgroups = mcgroups, + .igmp_groups = igmp_groups, + .meter_groups = meter_groups, + .lbs = lbs, + .svc_check_match = svc_check_match, + .match = DS_EMPTY_INITIALIZER, + .actions = DS_EMPTY_INITIALIZER, + }; + + /* Combined build - all lflow generation from lswitch and lrouter + * will move here and will be reogranized by iterator type. + */ + HMAP_FOR_EACH (od, key_node, datapaths) { + build_lswitch_and_lrouter_iterate_by_od(od, &lsi); + } + HMAP_FOR_EACH (op, key_node, ports) { + build_lswitch_and_lrouter_iterate_by_op(op, &lsi); + } + free(svc_check_match); + + /* Legacy lswitch build - to be migrated. */ + build_lswitch_flows(datapaths, ports, port_groups, lflows, mcgroups, + igmp_groups, meter_groups, lbs); + + /* Legacy lrouter build - to be migrated. */ + build_lrouter_flows(datapaths, ports, lflows, meter_groups, lbs); +} + + /* Updates the Logical_Flow and Multicast_Group tables in the OVN_SB database, * constructing their contents based on the OVN_NB database. */ static void @@ -11400,9 +11437,9 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, { struct hmap lflows = HMAP_INITIALIZER(&lflows); - build_lswitch_flows(datapaths, ports, port_groups, &lflows, mcgroups, - igmp_groups, meter_groups, lbs); - build_lrouter_flows(datapaths, ports, &lflows, meter_groups, lbs); + build_lswitch_and_lrouter_flows(datapaths, ports, + port_groups, &lflows, mcgroups, + igmp_groups, meter_groups, lbs); /* Push changes to the Logical_Flow table to database. */ const struct sbrec_logical_flow *sbflow, *next_sbflow; From patchwork Fri Sep 25 09:58:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371129 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6K173Jz9sPB for ; Fri, 25 Sep 2020 19:58:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 5C20E84837; Fri, 25 Sep 2020 09:58:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yPgyqDh-E-fv; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id A4807844E7; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 940CAC1AD7; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 16F89C1AD5 for ; Fri, 25 Sep 2020 09:58:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id DAA5D2E152 for ; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8zGfgueyHpUN for ; Fri, 25 Sep 2020 09:58:24 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by silver.osuosl.org (Postfix) with ESMTPS id 449742E147 for ; Fri, 25 Sep 2020 09:58:24 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUg-0000ri-M2; Fri, 25 Sep 2020 09:58:23 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUd-00059v-PA; Fri, 25 Sep 2020 10:58:21 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:01 +0100 Message-Id: <20200925095807.19358-4-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 3/9] ovn-northd: introduce parallel lflow build X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Add support for parallel lflow build. 2. Move combined lflow generation to be build in parallel. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 197 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 171 insertions(+), 26 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 66b6c2985..225f4ca8e 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -48,6 +48,7 @@ #include "unixctl.h" #include "util.h" #include "uuid.h" +#include "fasthmap.h" #include "openvswitch/vlog.h" VLOG_DEFINE_THIS_MODULE(ovn_northd); @@ -4189,7 +4190,7 @@ ovn_lflow_add_at(struct hmap *lflow_map, struct ovn_datapath *od, ovn_lflow_init(lflow, od, stage, priority, xstrdup(match), xstrdup(actions), ovn_lflow_hint(stage_hint), where); - hmap_insert(lflow_map, &lflow->hmap_node, ovn_lflow_hash(lflow)); + hmap_insert_fast(lflow_map, &lflow->hmap_node, ovn_lflow_hash(lflow)); } /* Adds a row with the specified contents to the Logical_Flow table. */ @@ -11331,6 +11332,8 @@ struct lswitch_flow_build_info { /* Helper function to combine all lflow generation which is iterated by * datapath. + * Invoked by parallel build over a "chunk" of work or by single threaded + * build over a chunk which is initialized to contain "all" work. */ static void @@ -11357,6 +11360,8 @@ build_lswitch_and_lrouter_iterate_by_od( } /* Helper function to combine all lflow generation which is iterated by port. + * Invoked by parallel build over a "chunk" of work or by single threaded + * build over a chunk which is initialized to contain "all" work. */ static void @@ -11379,6 +11384,85 @@ build_lswitch_and_lrouter_iterate_by_op( &lsi->actions); } +struct lflows_thread_pool { + struct worker_pool *pool; +}; + +static void *build_lflows_thread(void *arg) { + struct worker_control *control = (struct worker_control *) arg; + struct lflows_thread_pool *workload; + struct lswitch_flow_build_info *lsi; + + struct ovn_datapath *od; + struct ovn_port *op; + int bnum; + + while (!seize_fire()) { + sem_wait(&control->fire); + workload = (struct lflows_thread_pool *) control->workload; + lsi = (struct lswitch_flow_build_info *) control->data; + if (lsi && workload) { + /* Iterate over bucket ThreadID, ThreadID+size, ... */ + for (bnum = control->id; + bnum <= lsi->datapaths->mask; + bnum += workload->pool->size) + { + HMAP_FOR_EACH_IN_PARALLEL ( + od, key_node, bnum, lsi->datapaths) { + if (seize_fire()) { + return NULL; + } + build_lswitch_and_lrouter_iterate_by_od(od, lsi); + } + } + for (bnum = control->id; + bnum <= lsi->ports->mask; + bnum += workload->pool->size) + { + HMAP_FOR_EACH_IN_PARALLEL ( + op, key_node, bnum, lsi->ports) { + if (seize_fire()) { + return NULL; + } + build_lswitch_and_lrouter_iterate_by_op(op, lsi); + } + } + atomic_store_relaxed(&control->finished, true); + atomic_thread_fence(memory_order_release); + } + sem_post(control->done); + } + return NULL; +} + +static struct lflows_thread_pool *build_lflows_pool = NULL; + +static void init_lflows_thread_pool(void) +{ + int index; + + if (!build_lflows_pool) { + build_lflows_pool = + xmalloc(sizeof(struct lflows_thread_pool)); + build_lflows_pool->pool = + add_worker_pool(build_lflows_thread); + + for (index = 0; index < build_lflows_pool->pool->size; index++) { + build_lflows_pool->pool->controls[index].workload = + build_lflows_pool; + } + } +} + +/* TODO: replace hard cutoffs by configurable via commands. These are + * temporary defines to determine single-thread to multi-thread processing + * cutoff. + * Setting to 1 forces "all parallel" lflow build. + */ + +#define OD_CUTOFF 1 +#define OP_CUTOFF 1 + static void build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, struct hmap *port_groups, struct hmap *lflows, @@ -11386,35 +11470,89 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, struct shash *meter_groups, struct hmap *lbs) { + char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); - struct ovn_datapath *od; - struct ovn_port *op; + if (hmap_count(datapaths) > OD_CUTOFF || hmap_count(ports) > OP_CUTOFF) { - char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); + struct hmap *lflow_segs; + struct lswitch_flow_build_info *lsiv; + int index; - struct lswitch_flow_build_info lsi = { - .datapaths = datapaths, - .ports = ports, - .port_groups = port_groups, - .lflows = lflows, - .mcgroups = mcgroups, - .igmp_groups = igmp_groups, - .meter_groups = meter_groups, - .lbs = lbs, - .svc_check_match = svc_check_match, - .match = DS_EMPTY_INITIALIZER, - .actions = DS_EMPTY_INITIALIZER, - }; + init_lflows_thread_pool(); + lsiv = xmalloc( + sizeof(struct lswitch_flow_build_info) * + build_lflows_pool->pool->size); + lflow_segs = xmalloc( + sizeof(struct hmap) * build_lflows_pool->pool->size); - /* Combined build - all lflow generation from lswitch and lrouter - * will move here and will be reogranized by iterator type. - */ - HMAP_FOR_EACH (od, key_node, datapaths) { - build_lswitch_and_lrouter_iterate_by_od(od, &lsi); - } - HMAP_FOR_EACH (op, key_node, ports) { - build_lswitch_and_lrouter_iterate_by_op(op, &lsi); + /* Set up "work chunks" for each thread to work on. */ + + for (index = 0; index < build_lflows_pool->pool->size; index++) { + fast_hmap_init(&lflow_segs[index], lflows->mask); + + lsiv[index].datapaths = datapaths; + lsiv[index].ports = ports; + lsiv[index].port_groups = port_groups; + lsiv[index].lflows = &lflow_segs[index]; + lsiv[index].mcgroups = mcgroups; + lsiv[index].igmp_groups = igmp_groups; + lsiv[index].meter_groups = meter_groups; + lsiv[index].lbs = lbs; + lsiv[index].svc_check_match = svc_check_match; + + /* This cast is needed. While you can initialize without + * casting, you cannot assign a struct at once without a cast + * and that is what DS_EMPTY_INITALIZER returns at present. */ + + lsiv[index].match = (struct ds) DS_EMPTY_INITIALIZER; + lsiv[index].actions = (struct ds) DS_EMPTY_INITIALIZER; + + build_lflows_pool->pool->controls[index].data = &lsiv[index]; + } + + /* Run thread pool. */ + + run_pool_hash(build_lflows_pool->pool, lflows, lflow_segs); + + for (index = 0; index < build_lflows_pool->pool->size; index++) { + ds_destroy(&lsiv[index].match); + ds_destroy(&lsiv[index].actions); + } + + free(lflow_segs); + free(lsiv); + } else { + struct ovn_datapath *od; + struct ovn_port *op; + + struct lswitch_flow_build_info lsi = { + .datapaths = datapaths, + .ports = ports, + .port_groups = port_groups, + .lflows = lflows, + .mcgroups = mcgroups, + .igmp_groups = igmp_groups, + .meter_groups = meter_groups, + .lbs = lbs, + .svc_check_match = svc_check_match, + .match = DS_EMPTY_INITIALIZER, + .actions = DS_EMPTY_INITIALIZER, + }; + + + /* Converged build - all lflow generation from lswitch and lrouter + * will move here and will be reogranized by iterator type. + */ + HMAP_FOR_EACH (od, key_node, datapaths) { + build_lswitch_and_lrouter_iterate_by_od(od, &lsi); + } + HMAP_FOR_EACH (op, key_node, ports) { + build_lswitch_and_lrouter_iterate_by_op(op, &lsi); + } + ds_destroy(&lsi.match); + ds_destroy(&lsi.actions); } + free(svc_check_match); /* Legacy lswitch build - to be migrated. */ @@ -11425,6 +11563,7 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, build_lrouter_flows(datapaths, ports, lflows, meter_groups, lbs); } +static ssize_t max_seen_lflow_size = 128; /* Updates the Logical_Flow and Multicast_Group tables in the OVN_SB database, * constructing their contents based on the OVN_NB database. */ @@ -11435,12 +11574,18 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, struct shash *meter_groups, struct hmap *lbs) { - struct hmap lflows = HMAP_INITIALIZER(&lflows); + struct hmap lflows; + + fast_hmap_size_for(&lflows, max_seen_lflow_size); build_lswitch_and_lrouter_flows(datapaths, ports, port_groups, &lflows, mcgroups, igmp_groups, meter_groups, lbs); + if (hmap_count(&lflows) > max_seen_lflow_size) { + max_seen_lflow_size = hmap_count(&lflows); + } + /* Push changes to the Logical_Flow table to database. */ const struct sbrec_logical_flow *sbflow, *next_sbflow; SBREC_LOGICAL_FLOW_FOR_EACH_SAFE (sbflow, next_sbflow, ctx->ovnsb_idl) { From patchwork Fri Sep 25 09:58:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371131 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6S1Zyjz9sPB for ; Fri, 25 Sep 2020 19:58:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 4716A875E7; Fri, 25 Sep 2020 09:58:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VIghoF-4Qdwq; Fri, 25 Sep 2020 09:58:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 01E56875E3; Fri, 25 Sep 2020 09:58:30 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B0781C1AD9; Fri, 25 Sep 2020 09:58:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5A869C0890 for ; Fri, 25 Sep 2020 09:58:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 41C7686C43 for ; Fri, 25 Sep 2020 09:58:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x8TpG6FfSjy3 for ; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by fraxinus.osuosl.org (Postfix) with ESMTPS id B47A386C59 for ; Fri, 25 Sep 2020 09:58:25 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUi-0000ro-9z; Fri, 25 Sep 2020 09:58:24 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUf-00059v-Jj; Fri, 25 Sep 2020 10:58:23 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:02 +0100 Message-Id: <20200925095807.19358-5-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 4/9] ovn-northd: Add commands to set/get parallelisation thresholds X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov Add commands to control and display single-threaded to multi- threaded cutoff. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 47 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 44 insertions(+), 3 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 225f4ca8e..c768431d0 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -59,6 +59,8 @@ static unixctl_cb_func ovn_northd_resume; static unixctl_cb_func ovn_northd_is_paused; static unixctl_cb_func ovn_northd_status; static unixctl_cb_func cluster_state_reset_cmd; +static unixctl_cb_func get_param_cutoff; +static unixctl_cb_func set_param_cutoff; struct northd_context { struct ovsdb_idl *ovnnb_idl; @@ -11460,8 +11462,8 @@ static void init_lflows_thread_pool(void) * Setting to 1 forces "all parallel" lflow build. */ -#define OD_CUTOFF 1 -#define OP_CUTOFF 1 +static ssize_t lflow_od_cuttoff = 1; +static ssize_t lflow_op_cutoff = 1; static void build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, @@ -11472,7 +11474,8 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, { char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); - if (hmap_count(datapaths) > OD_CUTOFF || hmap_count(ports) > OP_CUTOFF) { + if (hmap_count(datapaths) > lflow_od_cuttoff || + hmap_count(ports) > lflow_op_cutoff) { struct hmap *lflow_segs; struct lswitch_flow_build_info *lsiv; @@ -13157,6 +13160,14 @@ main(int argc, char *argv[]) unixctl_command_register("is-paused", "", 0, 0, ovn_northd_is_paused, &state); unixctl_command_register("status", "", 0, 0, ovn_northd_status, &state); + unixctl_command_register("set-datapath-cutoff", "", 0, 0, + set_param_cutoff, &lflow_od_cuttoff); + unixctl_command_register("set-port-cutoff", "", 0, 0, + set_param_cutoff, &lflow_op_cutoff); + unixctl_command_register("get-datapath-cutoff", "", 0, 0, + get_param_cutoff, &lflow_od_cuttoff); + unixctl_command_register("get-port-cutoff", "", 0, 0, + get_param_cutoff, &lflow_op_cutoff); bool reset_ovnsb_idl_min_index = false; unixctl_command_register("sb-cluster-state-reset", "", 0, 0, @@ -13582,3 +13593,33 @@ cluster_state_reset_cmd(struct unixctl_conn *conn, int argc OVS_UNUSED, poll_immediate_wake(); unixctl_command_reply(conn, NULL); } + +static void set_param_cutoff +(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[], + void *param_) +{ + long new_cutoff; + ssize_t *param = param_; + + if (str_to_long(argv[1], 10, &new_cutoff)) { + if (new_cutoff > 0) { + *param = new_cutoff; + return; + } + } + unixctl_command_reply_error(conn, "unsigned integer required"); +} + +static void get_param_cutoff +(struct unixctl_conn *conn, int argc OVS_UNUSED, + const char *argv[] OVS_UNUSED, + void *param_) +{ + struct ds ds = DS_EMPTY_INITIALIZER; + ssize_t *param = param_; + + ds_put_format(&ds, "%ld\n", *param); + unixctl_command_reply(conn, ds_cstr(&ds)); + ds_destroy(&ds); +} From patchwork Fri Sep 25 09:58:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371130 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6S01fXz9s0b for ; Fri, 25 Sep 2020 19:58:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 26E0786CA5; Fri, 25 Sep 2020 09:58:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3_MTkXfgy7pI; Fri, 25 Sep 2020 09:58:36 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 92EF986C71; Fri, 25 Sep 2020 09:58:35 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 61A18C1AD6; Fri, 25 Sep 2020 09:58:35 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id CAAEBC0859 for ; Fri, 25 Sep 2020 09:58:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id B256C2E162 for ; Fri, 25 Sep 2020 09:58:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JEdNnV9OhaVI for ; Fri, 25 Sep 2020 09:58:28 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by silver.osuosl.org (Postfix) with ESMTPS id 70F142E163 for ; Fri, 25 Sep 2020 09:58:27 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUk-0000ru-0f; Fri, 25 Sep 2020 09:58:26 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUh-00059v-AW; Fri, 25 Sep 2020 10:58:25 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:03 +0100 Message-Id: <20200925095807.19358-6-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 5/9] ovn-northd: move pre-acl and acl lswitch processing to a function X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov Move pre-acl and acl lswitch processing to a function Invoke the pre-acl and acl lswitch processing out of the converged processing od loop. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 52 +++++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 23 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index c768431d0..f75f619e6 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -6784,34 +6784,15 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, static void build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, - struct hmap *port_groups, struct hmap *lflows, - struct hmap *mcgroups, struct hmap *igmp_groups, - struct shash *meter_groups, - struct hmap *lbs) + struct hmap *lflows, struct hmap *mcgroups, + struct hmap *igmp_groups, struct hmap *lbs) { /* This flow table structure is documented in ovn-northd(8), so please * update ovn-northd.8.xml if you change anything. */ struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - - /* Build pre-ACL and ACL tables for both ingress and egress. - * Ingress tables 3 through 10. Egress tables 0 through 7. */ struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbs) { - continue; - } - - build_pre_acls(od, lflows); - build_pre_lb(od, lflows, meter_groups, lbs); - build_pre_stateful(od, lflows); - build_acl_hints(od, lflows); - build_acls(od, lflows, port_groups); - build_qos(od, lflows); - build_lb(od, lflows); - build_stateful(od, lflows, lbs); - } /* Build logical flows for the forwarding groups */ HMAP_FOR_EACH (od, key_node, datapaths) { @@ -7488,6 +7469,27 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ds_destroy(&actions); } +/* Build pre-ACL and ACL tables for both ingress and egress. + * Ingress tables 3 through 10. Egress tables 0 through 7. */ +static void + build_lswitch_lflows_pre_acl_and_acl( + struct ovn_datapath *od, struct hmap *port_groups, + struct hmap *lflows, struct shash *meter_groups, + struct hmap *lbs) +{ + if (od->nbs) { + build_pre_acls(od, lflows); + build_pre_lb(od, lflows, meter_groups, lbs); + build_pre_stateful(od, lflows); + build_acl_hints(od, lflows); + build_acls(od, lflows, port_groups); + build_qos(od, lflows); + build_lb(od, lflows); + build_stateful(od, lflows, lbs); + } +} + + /* Returns a string of the IP address of the router port 'op' that * overlaps with 'ip_s". If one is not found, returns NULL. * @@ -11342,6 +11344,10 @@ static void build_lswitch_and_lrouter_iterate_by_od( struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { + /* Build Logical Switch Flows. */ + + build_lswitch_lflows_pre_acl_and_acl(od, lsi->port_groups, lsi->lflows, + lsi->meter_groups, lsi->lbs); /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); @@ -11559,8 +11565,8 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports, free(svc_check_match); /* Legacy lswitch build - to be migrated. */ - build_lswitch_flows(datapaths, ports, port_groups, lflows, mcgroups, - igmp_groups, meter_groups, lbs); + build_lswitch_flows(datapaths, ports, lflows, mcgroups, + igmp_groups, lbs); /* Legacy lrouter build - to be migrated. */ build_lrouter_flows(datapaths, ports, lflows, meter_groups, lbs); From patchwork Fri Sep 25 09:58:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371133 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6W1p0vz9sPB for ; Fri, 25 Sep 2020 19:58:43 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id B7A30875C2; Fri, 25 Sep 2020 09:58:41 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jV8t8KBacaX4; Fri, 25 Sep 2020 09:58:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 24AEE8760E; Fri, 25 Sep 2020 09:58:34 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0A091C1AD5; Fri, 25 Sep 2020 09:58:34 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0BE52C1AD6 for ; Fri, 25 Sep 2020 09:58:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id F041F86BEF for ; Fri, 25 Sep 2020 09:58:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OuewBRBhiF4i for ; Fri, 25 Sep 2020 09:58:29 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 3B0E584837 for ; Fri, 25 Sep 2020 09:58:29 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUl-0000s0-PK; Fri, 25 Sep 2020 09:58:27 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUj-00059v-0x; Fri, 25 Sep 2020 10:58:26 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:04 +0100 Message-Id: <20200925095807.19358-7-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 6/9] ovn-northd: migrate build_fwd_group_lflows to build_converged X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Move the condition from the old HMAP_FOR_EACH loop into the build_fwd_group_lflows() function. 2. Move build_fwd_group_lflows() into the converged build helper function Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 112 ++++++++++++++++++++++---------------------- 1 file changed, 55 insertions(+), 57 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index f75f619e6..78a2618a8 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -6141,64 +6141,69 @@ build_stateful(struct ovn_datapath *od, struct hmap *lflows, struct hmap *lbs) ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); } +/* Build logical flows for the forwarding groups */ static void build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows) { - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - for (int i = 0; i < od->nbs->n_forwarding_groups; ++i) { - const struct nbrec_forwarding_group *fwd_group = NULL; - fwd_group = od->nbs->forwarding_groups[i]; - if (!fwd_group->n_child_port) { - continue; - } + if (!(!od->nbs || !od->nbs->n_forwarding_groups)) { + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; - /* ARP responder for the forwarding group's virtual IP */ - ds_put_format(&match, "arp.tpa == %s && arp.op == 1", - fwd_group->vip); - ds_put_format(&actions, - "eth.dst = eth.src; " - "eth.src = %s; " - "arp.op = 2; /* ARP reply */ " - "arp.tha = arp.sha; " - "arp.sha = %s; " - "arp.tpa = arp.spa; " - "arp.spa = %s; " - "outport = inport; " - "flags.loopback = 1; " - "output;", - fwd_group->vmac, fwd_group->vmac, fwd_group->vip); - - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + for (int i = 0; i < od->nbs->n_forwarding_groups; ++i) { + const struct nbrec_forwarding_group *fwd_group = NULL; + fwd_group = od->nbs->forwarding_groups[i]; + if (!fwd_group->n_child_port) { + continue; + } - /* L2 lookup for the forwarding group's virtual MAC */ - ds_clear(&match); - ds_put_format(&match, "eth.dst == %s", fwd_group->vmac); + /* ARP responder for the forwarding group's virtual IP */ + ds_put_format(&match, "arp.tpa == %s && arp.op == 1", + fwd_group->vip); + ds_put_format(&actions, + "eth.dst = eth.src; " + "eth.src = %s; " + "arp.op = 2; /* ARP reply */ " + "arp.tha = arp.sha; " + "arp.sha = %s; " + "arp.tpa = arp.spa; " + "arp.spa = %s; " + "outport = inport; " + "flags.loopback = 1; " + "output;", + fwd_group->vmac, fwd_group->vmac, fwd_group->vip); + + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(&match), ds_cstr(&actions), + &fwd_group->header_); - /* Create a comma separated string of child ports */ - struct ds group_ports = DS_EMPTY_INITIALIZER; - if (fwd_group->liveness) { - ds_put_cstr(&group_ports, "liveness=\"true\","); - } - ds_put_cstr(&group_ports, "childports="); - for (i = 0; i < (fwd_group->n_child_port - 1); ++i) { - ds_put_format(&group_ports, "\"%s\",", fwd_group->child_port[i]); + /* L2 lookup for the forwarding group's virtual MAC */ + ds_clear(&match); + ds_put_format(&match, "eth.dst == %s", fwd_group->vmac); + + /* Create a comma separated string of child ports */ + struct ds group_ports = DS_EMPTY_INITIALIZER; + if (fwd_group->liveness) { + ds_put_cstr(&group_ports, "liveness=\"true\","); + } + ds_put_cstr(&group_ports, "childports="); + for (i = 0; i < (fwd_group->n_child_port - 1); ++i) { + ds_put_format( + &group_ports, "\"%s\",", fwd_group->child_port[i]); + } + ds_put_format(&group_ports, "\"%s\"", + fwd_group->child_port[fwd_group->n_child_port - 1]); + + ds_clear(&actions); + ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(&match), ds_cstr(&actions), + &fwd_group->header_); } - ds_put_format(&group_ports, "\"%s\"", - fwd_group->child_port[fwd_group->n_child_port - 1]); - ds_clear(&actions); - ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + ds_destroy(&match); + ds_destroy(&actions); } - - ds_destroy(&match); - ds_destroy(&actions); } static void @@ -6794,15 +6799,6 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, struct ds actions = DS_EMPTY_INITIALIZER; struct ovn_datapath *od; - /* Build logical flows for the forwarding groups */ - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbs || !od->nbs->n_forwarding_groups) { - continue; - } - - build_fwd_group_lflows(od, lflows); - } - /* Logical switch ingress table 0: Admission control framework (priority * 100). */ HMAP_FOR_EACH (od, key_node, datapaths) { @@ -11349,6 +11345,8 @@ build_lswitch_and_lrouter_iterate_by_od( build_lswitch_lflows_pre_acl_and_acl(od, lsi->port_groups, lsi->lflows, lsi->meter_groups, lsi->lbs); + build_fwd_group_lflows(od, lsi->lflows); + /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, From patchwork Fri Sep 25 09:58:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371134 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS6c5zRwz9s0b for ; Fri, 25 Sep 2020 19:58:48 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 21853875F4; Fri, 25 Sep 2020 09:58:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5Fv-eOYxoHqj; Fri, 25 Sep 2020 09:58:45 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 9DDA5875D4; Fri, 25 Sep 2020 09:58:42 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 79841C1AD9; Fri, 25 Sep 2020 09:58:42 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 33B16C1AD6 for ; Fri, 25 Sep 2020 09:58:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 1B91186A03 for ; Fri, 25 Sep 2020 09:58:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id viihJ50Cqetp for ; Fri, 25 Sep 2020 09:58:36 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by whitealder.osuosl.org (Postfix) with ESMTPS id C5C9886C34 for ; Fri, 25 Sep 2020 09:58:30 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUn-0000s6-Ce; Fri, 25 Sep 2020 09:58:29 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUk-00059v-O6; Fri, 25 Sep 2020 10:58:28 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:05 +0100 Message-Id: <20200925095807.19358-8-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 7/9] ovn-northd: Move lswitch admission control to a helper function X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Move the admission control lflows to a helper function 2. Add the helper function to the converged build per-od loop. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 78a2618a8..0fb7c7821 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -6799,25 +6799,6 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, struct ds actions = DS_EMPTY_INITIALIZER; struct ovn_datapath *od; - /* Logical switch ingress table 0: Admission control framework (priority - * 100). */ - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbs) { - continue; - } - - /* Logical VLANs not supported. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100, "vlan.present", - "drop;"); - - /* Broadcast/multicast source address is invalid. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100, "eth.src[40]", - "drop;"); - - /* Port security flows have priority 50 (see below) and will continue - * to the next table if packet source is acceptable. */ - } - build_lswitch_input_port_sec(ports, datapaths, lflows); /* Ingress table 13: ARP/ND responder, skip requests coming from localnet @@ -7485,6 +7466,27 @@ static void } } +/* Logical switch ingress table 0: Admission control framework (priority + * 100). */ +static void + build_lswitch_lflows_admission_control( + struct ovn_datapath *od, struct hmap *lflows) +{ + if (od->nbs) { + + /* Logical VLANs not supported. */ + ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100, "vlan.present", + "drop;"); + + /* Broadcast/multicast source address is invalid. */ + ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_L2, 100, "eth.src[40]", + "drop;"); + + /* Port security flows have priority 50 (see below) and will continue + * to the next table if packet source is acceptable. */ + } +} + /* Returns a string of the IP address of the router port 'op' that * overlaps with 'ip_s". If one is not found, returns NULL. @@ -11346,6 +11348,7 @@ build_lswitch_and_lrouter_iterate_by_od( lsi->meter_groups, lsi->lbs); build_fwd_group_lflows(od, lsi->lflows); + build_lswitch_lflows_admission_control(od, lsi->lflows); /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); From patchwork Fri Sep 25 09:58:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371136 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=silver.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS7r40KLz9sPB for ; Fri, 25 Sep 2020 19:59:52 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id DFE8B2E183; Fri, 25 Sep 2020 09:59:50 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dT3Gcr9FmE3a; Fri, 25 Sep 2020 09:59:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id B6A9F2E1AD; Fri, 25 Sep 2020 09:58:46 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A1C02C0890; Fri, 25 Sep 2020 09:58:46 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id F17C8C1AD5 for ; Fri, 25 Sep 2020 09:58:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id DA8288756B for ; Fri, 25 Sep 2020 09:58:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6JQIA+svNC3F for ; Fri, 25 Sep 2020 09:58:39 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by hemlock.osuosl.org (Postfix) with ESMTPS id AC22A875E3 for ; Fri, 25 Sep 2020 09:58:36 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUs-0000sD-U2; Fri, 25 Sep 2020 09:58:35 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUm-00059v-Dm; Fri, 25 Sep 2020 10:58:30 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:06 +0100 Message-Id: <20200925095807.19358-9-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 8/9] ovn-northd: migrate input port security flows to converged build X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Split build_lswitch_input_port_sec into per-datapath and per-port iterators 2. Reuse common scratchpad in the per-port build 3. Migrate the two new functions to the converged build loops Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 102 +++++++++++++++++++++----------------------- 1 file changed, 49 insertions(+), 53 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 0fb7c7821..92f72e547 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -4795,70 +4795,64 @@ has_stateful_acl(struct ovn_datapath *od) return false; } +/* Logical switch ingress table 0: Ingress port security - L2 + * (priority 50). + * Ingress table 1: Ingress port security - IP (priority 90 and 80) + * Ingress table 2: Ingress port security - ND (priority 90 and 80) + */ static void -build_lswitch_input_port_sec(struct hmap *ports, struct hmap *datapaths, - struct hmap *lflows) +build_lswitch_input_port_sec_op( + struct ovn_port *op, struct hmap *lflows, + struct ds *actions, struct ds *match) { - /* Logical switch ingress table 0: Ingress port security - L2 - * (priority 50). - * Ingress table 1: Ingress port security - IP (priority 90 and 80) - * Ingress table 2: Ingress port security - ND (priority 90 and 80) - */ - struct ds actions = DS_EMPTY_INITIALIZER; - struct ds match = DS_EMPTY_INITIALIZER; - struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, ports) { - if (!op->nbsp) { - continue; - } + if (!op->nbsp) { + return; + } - if (!lsp_is_enabled(op->nbsp)) { - /* Drop packets from disabled logical ports (since logical flow - * tables are default-drop). */ - continue; - } + if (!lsp_is_enabled(op->nbsp)) { + /* Drop packets from disabled logical ports (since logical flow + * tables are default-drop). */ + return; + } - if (lsp_is_external(op->nbsp)) { - continue; - } + if (lsp_is_external(op->nbsp)) { + return; + } - ds_clear(&match); - ds_clear(&actions); - ds_put_format(&match, "inport == %s", op->json_key); - build_port_security_l2("eth.src", op->ps_addrs, op->n_ps_addrs, - &match); + ds_clear(match); + ds_clear(actions); + ds_put_format(match, "inport == %s", op->json_key); + build_port_security_l2("eth.src", op->ps_addrs, op->n_ps_addrs, + match); - const char *queue_id = smap_get(&op->sb->options, "qdisc_queue_id"); - if (queue_id) { - ds_put_format(&actions, "set_queue(%s); ", queue_id); - } - ds_put_cstr(&actions, "next;"); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_PORT_SEC_L2, 50, - ds_cstr(&match), ds_cstr(&actions), - &op->nbsp->header_); + const char *queue_id = smap_get(&op->sb->options, "qdisc_queue_id"); + if (queue_id) { + ds_put_format(actions, "set_queue(%s); ", queue_id); + } + ds_put_cstr(actions, "next;"); + ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_PORT_SEC_L2, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_); - if (op->nbsp->n_port_security) { - build_port_security_ip(P_IN, op, lflows, &op->nbsp->header_); - build_port_security_nd(op, lflows, &op->nbsp->header_); - } + if (op->nbsp->n_port_security) { + build_port_security_ip(P_IN, op, lflows, &op->nbsp->header_); + build_port_security_nd(op, lflows, &op->nbsp->header_); } +} - /* Ingress table 1 and 2: Port security - IP and ND, by default - * goto next. (priority 0) - */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbs) { - continue; - } +/* Ingress table 1 and 2: Port security - IP and ND, by default + * goto next. (priority 0) + */ +static void +build_lswitch_input_port_sec_od( + struct ovn_datapath *od, struct hmap *lflows) +{ + if (od->nbs) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_ND, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_PORT_SEC_IP, 0, "1", "next;"); } - - ds_destroy(&match); - ds_destroy(&actions); } static void @@ -6799,8 +6793,6 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, struct ds actions = DS_EMPTY_INITIALIZER; struct ovn_datapath *od; - build_lswitch_input_port_sec(ports, datapaths, lflows); - /* Ingress table 13: ARP/ND responder, skip requests coming from localnet * and vtep ports. (priority 100); see ovn-northd.8.xml for the * rationale. */ @@ -11349,6 +11341,7 @@ build_lswitch_and_lrouter_iterate_by_od( build_fwd_group_lflows(od, lsi->lflows); build_lswitch_lflows_admission_control(od, lsi->lflows); + build_lswitch_input_port_sec_od(od, lsi->lflows); /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); @@ -11378,8 +11371,11 @@ build_lswitch_and_lrouter_iterate_by_op( struct ovn_port *op, struct lswitch_flow_build_info *lsi) { - /* Build Logical Router Flows. */ + /* Build Logical Switch Flows. */ + build_lswitch_input_port_sec_op(op, lsi->lflows, &lsi->actions, + &lsi->match); + /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, From patchwork Fri Sep 25 09:58:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1371135 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cambridgegreys.com Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4ByS7Y2BZJz9sPB for ; Fri, 25 Sep 2020 19:59:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id BEFB686DAA; Fri, 25 Sep 2020 09:59:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rRDY8LLMLhOE; Fri, 25 Sep 2020 09:59:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id DD10B86A03; Fri, 25 Sep 2020 09:58:58 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CFF76C1AD5; Fri, 25 Sep 2020 09:58:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6BE16C1AD6 for ; Fri, 25 Sep 2020 09:58:57 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 4191C86DAF for ; Fri, 25 Sep 2020 09:58:57 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uBl9gHGgbSWm for ; Fri, 25 Sep 2020 09:58:49 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by whitealder.osuosl.org (Postfix) with ESMTPS id 528B286D8D for ; Fri, 25 Sep 2020 09:58:37 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLkUt-0000sI-Gq; Fri, 25 Sep 2020 09:58:35 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1kLkUo-00059v-4C; Fri, 25 Sep 2020 10:58:32 +0100 From: anton.ivanov@cambridgegreys.com To: dev@openvswitch.org Date: Fri, 25 Sep 2020 10:58:07 +0100 Message-Id: <20200925095807.19358-10-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> References: <20200925095807.19358-1-anton.ivanov@cambridgegreys.com> MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [PATCH ovn v4 9/9] ovn-northd: migrate lswitch arp responder to converged build X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov Move the processing for arp responder, arp responder known ips and arp responder skip to next to helper functions. Move the invocation of the helper functions to the converged build. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 396 +++++++++++++++++++++++--------------------- 1 file changed, 205 insertions(+), 191 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 92f72e547..b5405d0b3 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -6792,198 +6792,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; struct ovn_datapath *od; - - /* Ingress table 13: ARP/ND responder, skip requests coming from localnet - * and vtep ports. (priority 100); see ovn-northd.8.xml for the - * rationale. */ struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, ports) { - if (!op->nbsp) { - continue; - } - - if ((!strcmp(op->nbsp->type, "localnet")) || - (!strcmp(op->nbsp->type, "vtep"))) { - ds_clear(&match); - ds_put_format(&match, "inport == %s", op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, - 100, ds_cstr(&match), "next;", - &op->nbsp->header_); - } - } - - /* Ingress table 13: ARP/ND responder, reply for known IPs. - * (priority 50). */ - HMAP_FOR_EACH (op, key_node, ports) { - if (!op->nbsp) { - continue; - } - - if (!strcmp(op->nbsp->type, "virtual")) { - /* Handle - * - GARPs for virtual ip which belongs to a logical port - * of type 'virtual' and bind that port. - * - * - ARP reply from the virtual ip which belongs to a logical - * port of type 'virtual' and bind that port. - * */ - ovs_be32 ip; - const char *virtual_ip = smap_get(&op->nbsp->options, - "virtual-ip"); - const char *virtual_parents = smap_get(&op->nbsp->options, - "virtual-parents"); - if (!virtual_ip || !virtual_parents || - !ip_parse(virtual_ip, &ip)) { - continue; - } - - char *tokstr = xstrdup(virtual_parents); - char *save_ptr = NULL; - char *vparent; - for (vparent = strtok_r(tokstr, ",", &save_ptr); vparent != NULL; - vparent = strtok_r(NULL, ",", &save_ptr)) { - struct ovn_port *vp = ovn_port_find(ports, vparent); - if (!vp || vp->od != op->od) { - /* vparent name should be valid and it should belong - * to the same logical switch. */ - continue; - } - - ds_clear(&match); - ds_put_format(&match, "inport == \"%s\" && " - "((arp.op == 1 && arp.spa == %s && " - "arp.tpa == %s) || (arp.op == 2 && " - "arp.spa == %s))", - vparent, virtual_ip, virtual_ip, - virtual_ip); - ds_clear(&actions); - ds_put_format(&actions, - "bind_vport(%s, inport); " - "next;", - op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 100, - ds_cstr(&match), ds_cstr(&actions), - &vp->nbsp->header_); - } - - free(tokstr); - } else { - /* - * Add ARP/ND reply flows if either the - * - port is up and it doesn't have 'unknown' address defined or - * - port type is router or - * - port type is localport - */ - if (check_lsp_is_up && - !lsp_is_up(op->nbsp) && strcmp(op->nbsp->type, "router") && - strcmp(op->nbsp->type, "localport")) { - continue; - } - - if (lsp_is_external(op->nbsp) || op->has_unknown) { - continue; - } - - for (size_t i = 0; i < op->n_lsp_addrs; i++) { - for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) { - ds_clear(&match); - ds_put_format(&match, "arp.tpa == %s && arp.op == 1", - op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ds_clear(&actions); - ds_put_format(&actions, - "eth.dst = eth.src; " - "eth.src = %s; " - "arp.op = 2; /* ARP reply */ " - "arp.tha = arp.sha; " - "arp.sha = %s; " - "arp.tpa = arp.spa; " - "arp.spa = %s; " - "outport = inport; " - "flags.loopback = 1; " - "output;", - op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, - op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(&match), - ds_cstr(&actions), - &op->nbsp->header_); - - /* Do not reply to an ARP request from the port that owns - * the address (otherwise a DHCP client that ARPs to check - * for a duplicate address will fail). Instead, forward - * it the usual way. - * - * (Another alternative would be to simply drop the packet. - * If everything is working as it is configured, then this - * would produce equivalent results, since no one should - * reply to the request. But ARPing for one's own IP - * address is intended to detect situations where the - * network is not working as configured, so dropping the - * request would frustrate that intent.) */ - ds_put_format(&match, " && inport == %s", op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 100, - ds_cstr(&match), "next;", - &op->nbsp->header_); - } - - /* For ND solicitations, we need to listen for both the - * unicast IPv6 address and its all-nodes multicast address, - * but always respond with the unicast IPv6 address. */ - for (size_t j = 0; j < op->lsp_addrs[i].n_ipv6_addrs; j++) { - ds_clear(&match); - ds_put_format(&match, - "nd_ns && ip6.dst == {%s, %s} && nd.target == %s", - op->lsp_addrs[i].ipv6_addrs[j].addr_s, - op->lsp_addrs[i].ipv6_addrs[j].sn_addr_s, - op->lsp_addrs[i].ipv6_addrs[j].addr_s); - - ds_clear(&actions); - ds_put_format(&actions, - "%s { " - "eth.src = %s; " - "ip6.src = %s; " - "nd.target = %s; " - "nd.tll = %s; " - "outport = inport; " - "flags.loopback = 1; " - "output; " - "};", - !strcmp(op->nbsp->type, "router") ? - "nd_na_router" : "nd_na", - op->lsp_addrs[i].ea_s, - op->lsp_addrs[i].ipv6_addrs[j].addr_s, - op->lsp_addrs[i].ipv6_addrs[j].addr_s, - op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(&match), - ds_cstr(&actions), - &op->nbsp->header_); - - /* Do not reply to a solicitation from the port that owns - * the address (otherwise DAD detection will fail). */ - ds_put_format(&match, " && inport == %s", op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 100, - ds_cstr(&match), "next;", - &op->nbsp->header_); - } - } - } - } - - /* Ingress table 13: ARP/ND responder, by default goto next. - * (priority 0)*/ - HMAP_FOR_EACH (od, key_node, datapaths) { - if (!od->nbs) { - continue; - } - - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); - } /* Ingress table 13: ARP/ND responder for service monitor source ip. * (priority 110)*/ @@ -7479,6 +7288,205 @@ static void } } +/* Ingress table 13: ARP/ND responder, skip requests coming from localnet + * and vtep ports. (priority 100); see ovn-northd.8.xml for the + * rationale. */ + +static void + build_lswitch_arp_nd_responder( + struct ovn_port *op, struct hmap *lflows, + struct ds *match) +{ + if (op->nbsp) { + if ((!strcmp(op->nbsp->type, "localnet")) || + (!strcmp(op->nbsp->type, "vtep"))) { + ds_clear(match); + ds_put_format(match, "inport == %s", op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, + 100, ds_cstr(match), "next;", + &op->nbsp->header_); + } + } +} + +/* Ingress table 13: ARP/ND responder, reply for known IPs. + * (priority 50). */ +static void + build_lswitch_arp_nd_responder_known( + struct ovn_port *op, struct hmap *lflows, + struct hmap *ports, + struct ds *match, struct ds *actions) +{ + if (!op->nbsp) { + return; + } + if (!strcmp(op->nbsp->type, "virtual")) { + /* Handle + * - GARPs for virtual ip which belongs to a logical port + * of type 'virtual' and bind that port. + * + * - ARP reply from the virtual ip which belongs to a logical + * port of type 'virtual' and bind that port. + * */ + ovs_be32 ip; + const char *virtual_ip = smap_get(&op->nbsp->options, + "virtual-ip"); + const char *virtual_parents = smap_get(&op->nbsp->options, + "virtual-parents"); + if (!virtual_ip || !virtual_parents || + !ip_parse(virtual_ip, &ip)) { + return; + } + + char *tokstr = xstrdup(virtual_parents); + char *save_ptr = NULL; + char *vparent; + for (vparent = strtok_r(tokstr, ",", &save_ptr); vparent != NULL; + vparent = strtok_r(NULL, ",", &save_ptr)) { + struct ovn_port *vp = ovn_port_find(ports, vparent); + if (!vp || vp->od != op->od) { + /* vparent name should be valid and it should belong + * to the same logical switch. */ + continue; + } + + ds_clear(match); + ds_put_format(match, "inport == \"%s\" && " + "((arp.op == 1 && arp.spa == %s && " + "arp.tpa == %s) || (arp.op == 2 && " + "arp.spa == %s))", + vparent, virtual_ip, virtual_ip, + virtual_ip); + ds_clear(actions); + ds_put_format(actions, + "bind_vport(%s, inport); " + "next;", + op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 100, + ds_cstr(match), ds_cstr(actions), + &vp->nbsp->header_); + } + + free(tokstr); + } else { + /* + * Add ARP/ND reply flows if either the + * - port is up and it doesn't have 'unknown' address defined or + * - port type is router or + * - port type is localport + */ + if (check_lsp_is_up && + !lsp_is_up(op->nbsp) && strcmp(op->nbsp->type, "router") && + strcmp(op->nbsp->type, "localport")) { + return; + } + + if (lsp_is_external(op->nbsp) || op->has_unknown) { + return; + } + + for (size_t i = 0; i < op->n_lsp_addrs; i++) { + for (size_t j = 0; j < op->lsp_addrs[i].n_ipv4_addrs; j++) { + ds_clear(match); + ds_put_format(match, "arp.tpa == %s && arp.op == 1", + op->lsp_addrs[i].ipv4_addrs[j].addr_s); + ds_clear(actions); + ds_put_format(actions, + "eth.dst = eth.src; " + "eth.src = %s; " + "arp.op = 2; /* ARP reply */ " + "arp.tha = arp.sha; " + "arp.sha = %s; " + "arp.tpa = arp.spa; " + "arp.spa = %s; " + "outport = inport; " + "flags.loopback = 1; " + "output;", + op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, + op->lsp_addrs[i].ipv4_addrs[j].addr_s); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_); + + /* Do not reply to an ARP request from the port that owns + * the address (otherwise a DHCP client that ARPs to check + * for a duplicate address will fail). Instead, forward + * it the usual way. + * + * (Another alternative would be to simply drop the packet. + * If everything is working as it is configured, then this + * would produce equivalent results, since no one should + * reply to the request. But ARPing for one's own IP + * address is intended to detect situations where the + * network is not working as configured, so dropping the + * request would frustrate that intent.) */ + ds_put_format(match, " && inport == %s", op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 100, + ds_cstr(match), "next;", + &op->nbsp->header_); + } + + /* For ND solicitations, we need to listen for both the + * unicast IPv6 address and its all-nodes multicast address, + * but always respond with the unicast IPv6 address. */ + for (size_t j = 0; j < op->lsp_addrs[i].n_ipv6_addrs; j++) { + ds_clear(match); + ds_put_format(match, + "nd_ns && ip6.dst == {%s, %s} && nd.target == %s", + op->lsp_addrs[i].ipv6_addrs[j].addr_s, + op->lsp_addrs[i].ipv6_addrs[j].sn_addr_s, + op->lsp_addrs[i].ipv6_addrs[j].addr_s); + + ds_clear(actions); + ds_put_format(actions, + "%s { " + "eth.src = %s; " + "ip6.src = %s; " + "nd.target = %s; " + "nd.tll = %s; " + "outport = inport; " + "flags.loopback = 1; " + "output; " + "};", + !strcmp(op->nbsp->type, "router") ? + "nd_na_router" : "nd_na", + op->lsp_addrs[i].ea_s, + op->lsp_addrs[i].ipv6_addrs[j].addr_s, + op->lsp_addrs[i].ipv6_addrs[j].addr_s, + op->lsp_addrs[i].ea_s); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_); + + /* Do not reply to a solicitation from the port that owns + * the address (otherwise DAD detection will fail). */ + ds_put_format(match, " && inport == %s", op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 100, + ds_cstr(match), "next;", + &op->nbsp->header_); + } + } + } +} + +/* Ingress table 13: ARP/ND responder, by default goto next. + * (priority 0)*/ +static void + build_lswitch_arp_nd_responder_next( + struct ovn_datapath *od, struct hmap *lflows) +{ + if (od->nbs) { + ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); + } +} + /* Returns a string of the IP address of the router port 'op' that * overlaps with 'ip_s". If one is not found, returns NULL. @@ -11342,6 +11350,7 @@ build_lswitch_and_lrouter_iterate_by_od( build_fwd_group_lflows(od, lsi->lflows); build_lswitch_lflows_admission_control(od, lsi->lflows); build_lswitch_input_port_sec_od(od, lsi->lflows); + build_lswitch_arp_nd_responder_next(od, lsi->lflows); /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); @@ -11374,6 +11383,11 @@ build_lswitch_and_lrouter_iterate_by_op( /* Build Logical Switch Flows. */ build_lswitch_input_port_sec_op(op, lsi->lflows, &lsi->actions, &lsi->match); + build_lswitch_input_port_sec_op(op, lsi->lflows, &lsi->actions, + &lsi->match); + build_lswitch_arp_nd_responder(op, lsi->lflows, &lsi->match); + build_lswitch_arp_nd_responder_known(op, lsi->lflows, lsi->ports, + &lsi->match, &lsi->actions); /* Build Logical Router Flows. */ build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match,