Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/1523760/?format=api
{ "id": 1523760, "url": "http://patchwork.ozlabs.org/api/patches/1523760/?format=api", "web_url": "http://patchwork.ozlabs.org/project/ovn/patch/20210902132739.4175-3-anton.ivanov@cambridgegreys.com/", "project": { "id": 68, "url": "http://patchwork.ozlabs.org/api/projects/68/?format=api", "name": "Open Virtual Network development", "link_name": "ovn", "list_id": "ovs-dev.openvswitch.org", "list_email": "ovs-dev@openvswitch.org", "web_url": "http://openvswitch.org/", "scm_url": "", "webscm_url": "", "list_archive_url": "", "list_archive_url_format": "", "commit_url_format": "" }, "msgid": "<20210902132739.4175-3-anton.ivanov@cambridgegreys.com>", "list_archive_url": null, "date": "2021-09-02T13:27:38", "name": "[ovs-dev,v6,3/4] northd: Optimize dp groups operations", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "fad00db98a298940fcd5679e290eecfb850166cc", "submitter": { "id": 71996, "url": "http://patchwork.ozlabs.org/api/people/71996/?format=api", "name": "Anton Ivanov", "email": "anton.ivanov@cambridgegreys.com" }, "delegate": null, "mbox": "http://patchwork.ozlabs.org/project/ovn/patch/20210902132739.4175-3-anton.ivanov@cambridgegreys.com/mbox/", "series": [ { "id": 260747, "url": "http://patchwork.ozlabs.org/api/series/260747/?format=api", "web_url": "http://patchwork.ozlabs.org/project/ovn/list/?series=260747", "date": "2021-09-02T13:27:36", "name": "[ovs-dev,v6,1/4] northd: Disable parallel processing for logical_dp_groups", "version": 6, "mbox": "http://patchwork.ozlabs.org/series/260747/mbox/" } ], "comments": "http://patchwork.ozlabs.org/api/patches/1523760/comments/", "check": "fail", "checks": "http://patchwork.ozlabs.org/api/patches/1523760/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<ovs-dev-bounces@openvswitch.org>", "X-Original-To": [ "incoming@patchwork.ozlabs.org", "ovs-dev@openvswitch.org" ], "Delivered-To": [ "patchwork-incoming@bilbo.ozlabs.org", "ovs-dev@lists.linuxfoundation.org" ], "Authentication-Results": "ozlabs.org;\n spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org\n (client-ip=140.211.166.138; helo=smtp1.osuosl.org;\n envelope-from=ovs-dev-bounces@openvswitch.org; receiver=<UNKNOWN>)", "Received": [ "from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138])\n\t(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)\n\t key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest\n SHA256)\n\t(No client certificate requested)\n\tby ozlabs.org (Postfix) with ESMTPS id 4H0hZT2x75z9t54\n\tfor <incoming@patchwork.ozlabs.org>; Thu, 2 Sep 2021 23:28:16 +1000 (AEST)", "from localhost (localhost [127.0.0.1])\n\tby smtp1.osuosl.org (Postfix) with ESMTP id 76B0081066;\n\tThu, 2 Sep 2021 13:28:12 +0000 (UTC)", "from smtp1.osuosl.org ([127.0.0.1])\n\tby localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n\twith ESMTP id A5DGJkBS8zdQ; Thu, 2 Sep 2021 13:28:10 +0000 (UTC)", "from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56])\n\tby smtp1.osuosl.org (Postfix) with ESMTPS id 8075382CD3;\n\tThu, 2 Sep 2021 13:28:08 +0000 (UTC)", "from lf-lists.osuosl.org (localhost [127.0.0.1])\n\tby lists.linuxfoundation.org (Postfix) with ESMTP id 668D7C0010;\n\tThu, 2 Sep 2021 13:28:08 +0000 (UTC)", "from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137])\n by lists.linuxfoundation.org (Postfix) with ESMTP id 0C0C3C000E\n for <ovs-dev@openvswitch.org>; Thu, 2 Sep 2021 13:28:07 +0000 (UTC)", "from localhost (localhost [127.0.0.1])\n by smtp4.osuosl.org (Postfix) with ESMTP id F0C8A425AA\n for <ovs-dev@openvswitch.org>; Thu, 2 Sep 2021 13:28:06 +0000 (UTC)", "from smtp4.osuosl.org ([127.0.0.1])\n by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)\n with ESMTP id PDmGAACBDoac for <ovs-dev@openvswitch.org>;\n Thu, 2 Sep 2021 13:28:05 +0000 (UTC)", "from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42])\n by smtp4.osuosl.org (Postfix) with ESMTPS id BA712425BA\n for <ovs-dev@openvswitch.org>; Thu, 2 Sep 2021 13:28:04 +0000 (UTC)", "from tun252.jain.kot-begemot.co.uk ([192.168.18.6]\n helo=jain.kot-begemot.co.uk)\n by www.kot-begemot.co.uk with esmtps\n (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)\n (Exim 4.92) (envelope-from <anton.ivanov@cambridgegreys.com>)\n id 1mLml6-0004pO-3m\n for ovs-dev@openvswitch.org; Thu, 02 Sep 2021 13:28:03 +0000", "from jain.kot-begemot.co.uk ([192.168.3.3])\n by jain.kot-begemot.co.uk with esmtp (Exim 4.92)\n (envelope-from <anton.ivanov@cambridgegreys.com>)\n id 1mLmky-00016T-Lp; Thu, 02 Sep 2021 14:27:55 +0100" ], "X-Virus-Scanned": [ "amavisd-new at osuosl.org", "amavisd-new at osuosl.org" ], "X-Greylist": "from auto-whitelisted by SQLgrey-1.8.0", "From": "anton.ivanov@cambridgegreys.com", "To": "ovs-dev@openvswitch.org", "Date": "Thu, 2 Sep 2021 14:27:38 +0100", "Message-Id": "<20210902132739.4175-3-anton.ivanov@cambridgegreys.com>", "X-Mailer": "git-send-email 2.20.1", "In-Reply-To": "<20210902132739.4175-1-anton.ivanov@cambridgegreys.com>", "References": "<20210902132739.4175-1-anton.ivanov@cambridgegreys.com>", "MIME-Version": "1.0", "X-Clacks-Overhead": "GNU Terry Pratchett", "Cc": "Anton Ivanov <anton.ivanov@cambridgegreys.com>", "Subject": "[ovs-dev] [OVN Patch v6 3/4] northd: Optimize dp groups operations", "X-BeenThere": "ovs-dev@openvswitch.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "<ovs-dev.openvswitch.org>", "List-Unsubscribe": "<https://mail.openvswitch.org/mailman/options/ovs-dev>,\n <mailto:ovs-dev-request@openvswitch.org?subject=unsubscribe>", "List-Archive": "<http://mail.openvswitch.org/pipermail/ovs-dev/>", "List-Post": "<mailto:ovs-dev@openvswitch.org>", "List-Help": "<mailto:ovs-dev-request@openvswitch.org?subject=help>", "List-Subscribe": "<https://mail.openvswitch.org/mailman/listinfo/ovs-dev>,\n <mailto:ovs-dev-request@openvswitch.org?subject=subscribe>", "Content-Type": "text/plain; charset=\"us-ascii\"", "Content-Transfer-Encoding": "7bit", "Errors-To": "ovs-dev-bounces@openvswitch.org", "Sender": "\"dev\" <ovs-dev-bounces@openvswitch.org>" }, "content": "From: Anton Ivanov <anton.ivanov@cambridgegreys.com>\n\nRemove full hash walks to form lflow dp_groups and add them\nto the overall parallelizeable lflow build.\n\nMake processing of \"with dp groups\" and \"without\" in\nbuild_lflows independent to allow these to run in parallel\nafter the updates to the parallel API have been merged.\n\nSigned-off-by: Anton Ivanov <anton.ivanov@cambridgegreys.com>\n---\n northd/ovn-northd.c | 546 +++++++++++++++++++++++++++-----------------\n 1 file changed, 337 insertions(+), 209 deletions(-)", "diff": "diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c\nindex 6dfb4327a..1f903882b 100644\n--- a/northd/ovn-northd.c\n+++ b/northd/ovn-northd.c\n@@ -86,6 +86,11 @@ struct northd_state {\n bool paused;\n };\n \n+struct lflow_state {\n+ struct hmap single_od;\n+ struct hmap multiple_od;\n+};\n+\n static const char *ovnnb_db;\n static const char *ovnsb_db;\n static const char *unixctl_path;\n@@ -4306,8 +4311,15 @@ struct ovn_lflow {\n const char *where;\n };\n \n-static void ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow);\n-static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows,\n+static void ovn_lflow_destroy(struct lflow_state *lflows,\n+ struct ovn_lflow *lflow);\n+static struct ovn_lflow *do_ovn_lflow_find(const struct hmap *lflows,\n+ const struct ovn_datapath *od,\n+ enum ovn_stage stage,\n+ uint16_t priority, const char *match,\n+ const char *actions,\n+ const char *ctrl_meter, uint32_t hash);\n+static struct ovn_lflow *ovn_lflow_find(const struct lflow_state *lflows,\n const struct ovn_datapath *od,\n enum ovn_stage stage,\n uint16_t priority, const char *match,\n@@ -4366,7 +4378,7 @@ static struct hashrow_locks lflow_locks;\n * Version to use when locking is required.\n */\n static struct ovn_lflow *\n-do_ovn_lflow_add(struct hmap *lflow_map, struct ovn_datapath *od,\n+do_ovn_lflow_add(struct lflow_state *lflow_map, struct ovn_datapath *od,\n uint32_t hash, enum ovn_stage stage, uint16_t priority,\n const char *match, const char *actions, const char *io_port,\n const struct ovsdb_idl_row *stage_hint,\n@@ -4377,10 +4389,32 @@ do_ovn_lflow_add(struct hmap *lflow_map, struct ovn_datapath *od,\n struct ovn_lflow *lflow;\n \n if (use_logical_dp_groups) {\n- old_lflow = ovn_lflow_find(lflow_map, NULL, stage, priority, match,\n- actions, ctrl_meter, hash);\n+ old_lflow = do_ovn_lflow_find(&lflow_map->single_od, NULL, stage,\n+ priority, match,\n+ actions, ctrl_meter, hash);\n if (old_lflow) {\n hmapx_add(&old_lflow->od_group, od);\n+ /* Found, different, od count went up. Move to multiple od. */\n+ if (hmapx_count(&old_lflow->od_group) > 1) {\n+ hmap_remove(&lflow_map->single_od, &old_lflow->hmap_node);\n+ if (use_parallel_build) {\n+ hmap_insert_fast(&lflow_map->multiple_od,\n+ &old_lflow->hmap_node, hash);\n+ } else {\n+ hmap_insert(&lflow_map->multiple_od,\n+ &old_lflow->hmap_node, hash);\n+ }\n+ }\n+ } else {\n+ /* Not found, lookup in multiple od. */\n+ old_lflow = do_ovn_lflow_find(&lflow_map->multiple_od, NULL,\n+ stage, priority, match,\n+ actions, ctrl_meter, hash);\n+ if (old_lflow) {\n+ hmapx_add(&old_lflow->od_group, od);\n+ }\n+ }\n+ if (old_lflow) {\n return old_lflow;\n }\n }\n@@ -4395,16 +4429,20 @@ do_ovn_lflow_add(struct hmap *lflow_map, struct ovn_datapath *od,\n nullable_xstrdup(ctrl_meter),\n ovn_lflow_hint(stage_hint), where);\n hmapx_add(&lflow->od_group, od);\n+\n+ /* Insert \"fresh\" lflows into single_od. */\n+\n if (!use_parallel_build) {\n- hmap_insert(lflow_map, &lflow->hmap_node, hash);\n+ hmap_insert(&lflow_map->single_od, &lflow->hmap_node, hash);\n } else {\n- hmap_insert_fast(lflow_map, &lflow->hmap_node, hash);\n+ hmap_insert_fast(&lflow_map->single_od, &lflow->hmap_node, hash);\n }\n return lflow;\n }\n \n static struct ovn_lflow *\n-ovn_lflow_add_at_with_hash(struct hmap *lflow_map, struct ovn_datapath *od,\n+ovn_lflow_add_at_with_hash(struct lflow_state *lflow_map,\n+ struct ovn_datapath *od,\n enum ovn_stage stage, uint16_t priority,\n const char *match, const char *actions,\n const char *io_port, const char *ctrl_meter,\n@@ -4429,7 +4467,7 @@ ovn_lflow_add_at_with_hash(struct hmap *lflow_map, struct ovn_datapath *od,\n \n /* Adds a row with the specified contents to the Logical_Flow table. */\n static void\n-ovn_lflow_add_at(struct hmap *lflow_map, struct ovn_datapath *od,\n+ovn_lflow_add_at(struct lflow_state *lflow_map, struct ovn_datapath *od,\n enum ovn_stage stage, uint16_t priority,\n const char *match, const char *actions, const char *io_port,\n const char *ctrl_meter,\n@@ -4503,27 +4541,76 @@ ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref,\n ACTIONS, NULL, CTRL_METER, NULL)\n \n static struct ovn_lflow *\n-ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od,\n+do_ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od,\n enum ovn_stage stage, uint16_t priority,\n const char *match, const char *actions, const char *ctrl_meter,\n uint32_t hash)\n {\n struct ovn_lflow *lflow;\n- HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) {\n- if (ovn_lflow_equal(lflow, od, stage, priority, match, actions,\n- ctrl_meter)) {\n- return lflow;\n+ if (lflows) {\n+ HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) {\n+ if (ovn_lflow_equal(lflow, od, stage, priority, match, actions,\n+ ctrl_meter)) {\n+ return lflow;\n+ }\n }\n }\n return NULL;\n }\n+static struct ovn_lflow *\n+ovn_lflow_find(const struct lflow_state *lflows, const struct ovn_datapath *od,\n+ enum ovn_stage stage, uint16_t priority,\n+ const char *match, const char *actions, const char *ctrl_meter,\n+ uint32_t hash)\n+{\n+ struct ovn_lflow *lflow =\n+ do_ovn_lflow_find(&lflows->single_od, od, stage,\n+ priority, match, actions,\n+ ctrl_meter, hash);\n+ if (!lflow) {\n+ lflow = do_ovn_lflow_find(&lflows->multiple_od, od, stage,\n+ priority, match,\n+ actions, ctrl_meter, hash);\n+ }\n+\n+ return lflow;\n+}\n+\n+static inline bool\n+hmap_safe_remove(struct hmap *hmap, struct hmap_node *node)\n+{\n+ struct hmap_node **bucket = &hmap->buckets[node->hash & hmap->mask];\n+ while (*bucket != node && *bucket != NULL) {\n+ bucket = &(*bucket)->next;\n+ }\n+ if (*bucket == node) {\n+ *bucket = node->next;\n+ hmap->n--;\n+ return true;\n+ }\n+ return false;\n+}\n \n static void\n-ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow)\n+remove_lflow_from_lflows(struct lflow_state *lflows, struct ovn_lflow *lflow)\n+{\n+ if (use_logical_dp_groups && use_parallel_build) {\n+ lock_hash_row(&lflow_locks, lflow->hmap_node.hash);\n+ }\n+ if (!hmap_safe_remove(&lflows->multiple_od, &lflow->hmap_node)) {\n+ hmap_remove(&lflows->single_od, &lflow->hmap_node);\n+ }\n+ if (use_logical_dp_groups && use_parallel_build) {\n+ unlock_hash_row(&lflow_locks, lflow->hmap_node.hash);\n+ }\n+}\n+\n+static void\n+ovn_lflow_destroy(struct lflow_state *lflows, struct ovn_lflow *lflow)\n {\n if (lflow) {\n if (lflows) {\n- hmap_remove(lflows, &lflow->hmap_node);\n+ remove_lflow_from_lflows(lflows, lflow);\n }\n hmapx_destroy(&lflow->od_group);\n free(lflow->match);\n@@ -4535,6 +4622,7 @@ ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow)\n }\n }\n \n+\n /* Appends port security constraints on L2 address field 'eth_addr_field'\n * (e.g. \"eth.src\" or \"eth.dst\") to 'match'. 'ps_addrs', with 'n_ps_addrs'\n * elements, is the collection of port_security constraints from an\n@@ -4662,7 +4750,7 @@ build_port_security_ipv6_flow(\n * - Priority 80 flow to drop ARP and IPv6 ND packets.\n */\n static void\n-build_port_security_nd(struct ovn_port *op, struct hmap *lflows,\n+build_port_security_nd(struct ovn_port *op, struct lflow_state *lflows,\n const struct ovsdb_idl_row *stage_hint)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n@@ -4743,7 +4831,7 @@ build_port_security_nd(struct ovn_port *op, struct hmap *lflows,\n */\n static void\n build_port_security_ip(enum ovn_pipeline pipeline, struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n const struct ovsdb_idl_row *stage_hint)\n {\n char *port_direction;\n@@ -5147,7 +5235,7 @@ ls_get_acl_flags(struct ovn_datapath *od)\n */\n static void\n build_lswitch_input_port_sec_op(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *actions, struct ds *match)\n {\n \n@@ -5191,7 +5279,7 @@ build_lswitch_input_port_sec_op(\n */\n static void\n build_lswitch_input_port_sec_od(\n- struct ovn_datapath *od, struct hmap *lflows)\n+ struct ovn_datapath *od, struct lflow_state *lflows)\n {\n \n if (od->nbs) {\n@@ -5202,7 +5290,7 @@ build_lswitch_input_port_sec_od(\n \n static void\n build_lswitch_learn_fdb_op(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *actions, struct ds *match)\n {\n if (op->nbsp && !op->n_ps_addrs && !strcmp(op->nbsp->type, \"\") &&\n@@ -5229,7 +5317,7 @@ build_lswitch_learn_fdb_op(\n \n static void\n build_lswitch_learn_fdb_od(\n- struct ovn_datapath *od, struct hmap *lflows)\n+ struct ovn_datapath *od, struct lflow_state *lflows)\n {\n \n if (od->nbs) {\n@@ -5250,7 +5338,7 @@ build_lswitch_learn_fdb_od(\n */\n static void\n build_lswitch_output_port_sec_op(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match,\n struct ds *actions)\n {\n@@ -5295,7 +5383,7 @@ build_lswitch_output_port_sec_op(struct ovn_port *op,\n * (priority 100). */\n static void\n build_lswitch_output_port_sec_od(struct ovn_datapath *od,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (od->nbs) {\n ovn_lflow_add(lflows, od, S_SWITCH_OUT_PORT_SEC_IP, 0, \"1\", \"next;\");\n@@ -5307,7 +5395,7 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od,\n static void\n skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op,\n enum ovn_stage in_stage, enum ovn_stage out_stage,\n- uint16_t priority, struct hmap *lflows)\n+ uint16_t priority, struct lflow_state *lflows)\n {\n /* Can't use ct() for router ports. Consider the following configuration:\n * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a\n@@ -5337,7 +5425,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op,\n static void\n build_stateless_filter(struct ovn_datapath *od,\n const struct nbrec_acl *acl,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (!strcmp(acl->direction, \"from-lport\")) {\n ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_PRE_ACL,\n@@ -5356,7 +5444,7 @@ build_stateless_filter(struct ovn_datapath *od,\n \n static void\n build_stateless_filters(struct ovn_datapath *od, struct hmap *port_groups,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n for (size_t i = 0; i < od->nbs->n_acls; i++) {\n const struct nbrec_acl *acl = od->nbs->acls[i];\n@@ -5380,7 +5468,7 @@ build_stateless_filters(struct ovn_datapath *od, struct hmap *port_groups,\n \n static void\n build_pre_acls(struct ovn_datapath *od, struct hmap *port_groups,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are\n * allowed by default. */\n@@ -5509,7 +5597,7 @@ ls_has_lb_vip(struct ovn_datapath *od)\n }\n \n static void\n-build_pre_lb(struct ovn_datapath *od, struct hmap *lflows,\n+build_pre_lb(struct ovn_datapath *od, struct lflow_state *lflows,\n struct hmap *lbs)\n {\n /* Do not send ND packets to conntrack */\n@@ -5588,7 +5676,7 @@ build_pre_lb(struct ovn_datapath *od, struct hmap *lflows,\n }\n \n static void\n-build_pre_stateful(struct ovn_datapath *od, struct hmap *lflows)\n+build_pre_stateful(struct ovn_datapath *od, struct lflow_state *lflows)\n {\n /* Ingress and Egress pre-stateful Table (Priority 0): Packets are\n * allowed by default. */\n@@ -5640,7 +5728,7 @@ build_pre_stateful(struct ovn_datapath *od, struct hmap *lflows)\n }\n \n static void\n-build_acl_hints(struct ovn_datapath *od, struct hmap *lflows)\n+build_acl_hints(struct ovn_datapath *od, struct lflow_state *lflows)\n {\n /* This stage builds hints for the IN/OUT_ACL stage. Based on various\n * combinations of ct flags packets may hit only a subset of the logical\n@@ -5813,7 +5901,7 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl,\n }\n \n static void\n-build_reject_acl_rules(struct ovn_datapath *od, struct hmap *lflows,\n+build_reject_acl_rules(struct ovn_datapath *od, struct lflow_state *lflows,\n enum ovn_stage stage, struct nbrec_acl *acl,\n struct ds *extra_match, struct ds *extra_actions,\n const struct ovsdb_idl_row *stage_hint,\n@@ -5856,7 +5944,7 @@ build_reject_acl_rules(struct ovn_datapath *od, struct hmap *lflows,\n }\n \n static void\n-consider_acl(struct hmap *lflows, struct ovn_datapath *od,\n+consider_acl(struct lflow_state *lflows, struct ovn_datapath *od,\n struct nbrec_acl *acl, bool has_stateful,\n const struct shash *meter_groups, struct ds *match,\n struct ds *actions)\n@@ -6080,7 +6168,7 @@ build_port_group_lswitches(struct northd_context *ctx, struct hmap *pgs,\n }\n \n static void\n-build_acls(struct ovn_datapath *od, struct hmap *lflows,\n+build_acls(struct ovn_datapath *od, struct lflow_state *lflows,\n struct hmap *port_groups, const struct shash *meter_groups)\n {\n bool has_stateful = od->has_stateful_acl || od->has_lb_vip;\n@@ -6294,7 +6382,7 @@ build_acls(struct ovn_datapath *od, struct hmap *lflows,\n }\n \n static void\n-build_qos(struct ovn_datapath *od, struct hmap *lflows) {\n+build_qos(struct ovn_datapath *od, struct lflow_state *lflows) {\n struct ds action = DS_EMPTY_INITIALIZER;\n \n ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, \"1\", \"next;\");\n@@ -6355,7 +6443,7 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) {\n }\n \n static void\n-build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb,\n+build_lb_rules(struct lflow_state *lflows, struct ovn_northd_lb *lb,\n struct ds *match, struct ds *action,\n struct shash *meter_groups)\n {\n@@ -6436,7 +6524,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb,\n }\n \n static void\n-build_stateful(struct ovn_datapath *od, struct hmap *lflows)\n+build_stateful(struct ovn_datapath *od, struct lflow_state *lflows)\n {\n /* Ingress and Egress stateful Table (Priority 0): Packets are\n * allowed by default. */\n@@ -6475,7 +6563,7 @@ build_stateful(struct ovn_datapath *od, struct hmap *lflows)\n }\n \n static void\n-build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows)\n+build_lb_hairpin(struct ovn_datapath *od, struct lflow_state *lflows)\n {\n /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0).\n * Packets that don't need hairpinning should continue processing.\n@@ -6533,7 +6621,7 @@ build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows)\n \n /* Build logical flows for the forwarding groups */\n static void\n-build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows)\n+build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_state *lflows)\n {\n \n if (!(!od->nbs || !od->nbs->n_forwarding_groups)) {\n@@ -6746,7 +6834,7 @@ static void\n build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op,\n uint32_t priority,\n struct ovn_datapath *od,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct sset all_eth_addrs = SSET_INITIALIZER(&all_eth_addrs);\n struct ds eth_src = DS_EMPTY_INITIALIZER;\n@@ -6840,7 +6928,7 @@ arp_nd_ns_match(const char *ips, int addr_family, struct ds *match)\n static void\n build_lswitch_rport_arp_req_flow_for_reachable_ip(const char *ips,\n int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od,\n- uint32_t priority, struct hmap *lflows,\n+ uint32_t priority, struct lflow_state *lflows,\n const struct ovsdb_idl_row *stage_hint)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n@@ -6879,7 +6967,7 @@ build_lswitch_rport_arp_req_flow_for_reachable_ip(const char *ips,\n static void\n build_lswitch_rport_arp_req_flow_for_unreachable_ip(const char *ips,\n int addr_family, struct ovn_datapath *od, uint32_t priority,\n- struct hmap *lflows, const struct ovsdb_idl_row *stage_hint)\n+ struct lflow_state *lflows, const struct ovsdb_idl_row *stage_hint)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n \n@@ -6904,7 +6992,7 @@ static void\n build_lswitch_rport_arp_req_flows(struct ovn_port *op,\n struct ovn_datapath *sw_od,\n struct ovn_port *sw_op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n const struct ovsdb_idl_row *stage_hint)\n {\n if (!op || !op->nbrp) {\n@@ -7021,7 +7109,7 @@ build_dhcpv4_options_flows(struct ovn_port *op,\n struct lport_addresses *lsp_addrs,\n struct ovn_port *inport, bool is_external,\n struct shash *meter_groups,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n \n@@ -7114,7 +7202,7 @@ build_dhcpv6_options_flows(struct ovn_port *op,\n struct lport_addresses *lsp_addrs,\n struct ovn_port *inport, bool is_external,\n struct shash *meter_groups,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n \n@@ -7164,7 +7252,7 @@ build_dhcpv6_options_flows(struct ovn_port *op,\n static void\n build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op,\n const struct ovn_port *port,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n \n@@ -7229,7 +7317,7 @@ is_vlan_transparent(const struct ovn_datapath *od)\n }\n \n static void\n-build_lswitch_flows(struct hmap *datapaths, struct hmap *lflows)\n+build_lswitch_flows(struct hmap *datapaths, struct lflow_state *lflows)\n {\n /* This flow table structure is documented in ovn-northd(8), so please\n * update ovn-northd.8.xml if you change anything. */\n@@ -7264,7 +7352,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *lflows)\n static void\n build_lswitch_lflows_pre_acl_and_acl(struct ovn_datapath *od,\n struct hmap *port_groups,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups,\n struct hmap *lbs)\n {\n@@ -7287,7 +7375,7 @@ build_lswitch_lflows_pre_acl_and_acl(struct ovn_datapath *od,\n * 100). */\n static void\n build_lswitch_lflows_admission_control(struct ovn_datapath *od,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (od->nbs) {\n /* Logical VLANs not supported. */\n@@ -7313,7 +7401,7 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od,\n \n static void\n build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match)\n {\n if (op->nbsp) {\n@@ -7333,7 +7421,7 @@ build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op,\n * (priority 50). */\n static void\n build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct hmap *ports,\n struct shash *meter_groups,\n struct ds *actions,\n@@ -7553,7 +7641,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op,\n * (priority 0)*/\n static void\n build_lswitch_arp_nd_responder_default(struct ovn_datapath *od,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (od->nbs) {\n ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, \"1\", \"next;\");\n@@ -7564,7 +7652,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od,\n * (priority 110)*/\n static void\n build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *actions,\n struct ds *match)\n {\n@@ -7612,7 +7700,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb,\n * priority 100 flows. */\n static void\n build_lswitch_dhcp_options_and_response(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups)\n {\n if (op->nbsp) {\n@@ -7668,7 +7756,7 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op,\n * (priority 0). */\n static void\n build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (od->nbs) {\n ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, \"1\", \"next;\");\n@@ -7684,7 +7772,7 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od,\n */\n static void\n build_lswitch_dns_lookup_and_response(struct ovn_datapath *od,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups)\n {\n if (od->nbs && ls_has_dns_records(od->nbs)) {\n@@ -7713,7 +7801,7 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od,\n * binding the external ports. */\n static void\n build_lswitch_external_port(struct ovn_port *op,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n if (op->nbsp && lsp_is_external(op->nbsp)) {\n \n@@ -7728,7 +7816,7 @@ build_lswitch_external_port(struct ovn_port *op,\n * (priority 70 - 100). */\n static void\n build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -7820,7 +7908,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od,\n * (priority 90). */\n static void\n build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *actions,\n struct ds *match)\n {\n@@ -7897,7 +7985,7 @@ static struct ovs_mutex mcgroup_mutex = OVS_MUTEX_INITIALIZER;\n /* Ingress table 22: Destination lookup, unicast handling (priority 50), */\n static void\n build_lswitch_ip_unicast_lookup(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct hmap *mcgroups,\n struct ds *actions,\n struct ds *match)\n@@ -8317,7 +8405,7 @@ get_outport_for_routing_policy_nexthop(struct ovn_datapath *od,\n }\n \n static void\n-build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_routing_policy_flow(struct lflow_state *lflows, struct ovn_datapath *od,\n struct hmap *ports,\n const struct nbrec_logical_router_policy *rule,\n const struct ovsdb_idl_row *stage_hint)\n@@ -8382,8 +8470,8 @@ build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_ecmp_routing_policy_flows(struct hmap *lflows, struct ovn_datapath *od,\n- struct hmap *ports,\n+build_ecmp_routing_policy_flows(struct lflow_state *lflows,\n+ struct ovn_datapath *od, struct hmap *ports,\n const struct nbrec_logical_router_policy *rule,\n uint16_t ecmp_group_id)\n {\n@@ -8853,7 +8941,7 @@ find_static_route_outport(struct ovn_datapath *od, struct hmap *ports,\n }\n \n static void\n-add_ecmp_symmetric_reply_flows(struct hmap *lflows,\n+add_ecmp_symmetric_reply_flows(struct lflow_state *lflows,\n struct ovn_datapath *od,\n const char *port_ip,\n struct ovn_port *out_port,\n@@ -8933,7 +9021,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows,\n }\n \n static void\n-build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_ecmp_route_flow(struct lflow_state *lflows, struct ovn_datapath *od,\n struct hmap *ports, struct ecmp_groups_node *eg)\n \n {\n@@ -9016,7 +9104,7 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-add_route(struct hmap *lflows, struct ovn_datapath *od,\n+add_route(struct lflow_state *lflows, struct ovn_datapath *od,\n const struct ovn_port *op, const char *lrp_addr_s,\n const char *network_s, int plen, const char *gateway,\n bool is_src_route, const struct ovsdb_idl_row *stage_hint,\n@@ -9078,7 +9166,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_static_route_flow(struct lflow_state *lflows, struct ovn_datapath *od,\n struct hmap *ports,\n const struct parsed_route *route_)\n {\n@@ -9173,7 +9261,7 @@ static void\n build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip,\n struct ovn_northd_lb *lb,\n struct ovn_northd_lb_vip *vips_nb,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match, struct ds *action,\n struct shash *meter_groups)\n {\n@@ -9383,7 +9471,8 @@ next:\n }\n \n static void\n-build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows,\n+build_lswitch_flows_for_lb(struct ovn_northd_lb *lb,\n+ struct lflow_state *lflows,\n struct shash *meter_groups, struct ds *match,\n struct ds *action)\n {\n@@ -9434,7 +9523,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows,\n */\n static void\n build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match)\n {\n if (!lb->n_nb_lr) {\n@@ -9492,7 +9581,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb,\n static void\n build_lflows_for_unreachable_vips(struct ovn_northd_lb *lb,\n struct ovn_lb_vip *lb_vip,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match)\n {\n static const char *action = \"outport = \\\"_MC_flood\\\"; output;\";\n@@ -9553,9 +9642,10 @@ build_lflows_for_unreachable_vips(struct ovn_northd_lb *lb,\n }\n \n static void\n-build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows,\n- struct shash *meter_groups, struct ds *match,\n- struct ds *action)\n+build_lrouter_flows_for_lb(struct ovn_northd_lb *lb,\n+ struct lflow_state *lflows,\n+ struct shash *meter_groups,\n+ struct ds *match, struct ds *action)\n {\n if (!lb->n_nb_lr) {\n return;\n@@ -9706,7 +9796,7 @@ lrouter_nat_is_stateless(const struct nbrec_nat *nat)\n */\n static inline void\n lrouter_nat_add_ext_ip_match(struct ovn_datapath *od,\n- struct hmap *lflows, struct ds *match,\n+ struct lflow_state *lflows, struct ds *match,\n const struct nbrec_nat *nat,\n bool is_v6, bool is_src, ovs_be32 mask)\n {\n@@ -9773,7 +9863,7 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op,\n const char *ip_address, const char *eth_addr,\n struct ds *extra_match, bool drop, uint16_t priority,\n const struct ovsdb_idl_row *hint,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n struct ds actions = DS_EMPTY_INITIALIZER;\n@@ -9823,7 +9913,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op,\n const char *sn_ip_address, const char *eth_addr,\n struct ds *extra_match, bool drop, uint16_t priority,\n const struct ovsdb_idl_row *hint,\n- struct hmap *lflows, struct shash *meter_groups)\n+ struct lflow_state *lflows, struct shash *meter_groups)\n {\n struct ds match = DS_EMPTY_INITIALIZER;\n struct ds actions = DS_EMPTY_INITIALIZER;\n@@ -9877,7 +9967,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op,\n static void\n build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od,\n struct ovn_nat *nat_entry,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups)\n {\n struct lport_addresses *ext_addrs = &nat_entry->ext_addrs;\n@@ -9900,7 +9990,7 @@ build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od,\n static void\n build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,\n struct ovn_nat *nat_entry,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups)\n {\n struct lport_addresses *ext_addrs = &nat_entry->ext_addrs;\n@@ -9967,7 +10057,7 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,\n static void\n build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage,\n uint16_t priority, bool drop_snat_ip,\n- struct hmap *lflows)\n+ struct lflow_state *lflows)\n {\n struct ds match_ips = DS_EMPTY_INITIALIZER;\n \n@@ -10024,7 +10114,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage,\n }\n \n static void\n-build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_force_snat_flows(struct lflow_state *lflows,\n+ struct ovn_datapath *od,\n const char *ip_version, const char *ip_addr,\n const char *context)\n {\n@@ -10051,7 +10142,7 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od,\n \n static void\n build_lrouter_force_snat_flows_op(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (!op->nbrp || !op->peer || !op->od->lb_force_snat_router_ip) {\n@@ -10122,7 +10213,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op,\n }\n \n static void\n-build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op,\n+build_lrouter_bfd_flows(struct lflow_state *lflows, struct ovn_port *op,\n struct shash *meter_groups)\n {\n if (!op->has_bfd) {\n@@ -10177,7 +10268,7 @@ build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op,\n */\n static void\n build_adm_ctrl_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows)\n+ struct ovn_datapath *od, struct lflow_state *lflows)\n {\n if (od->nbr) {\n /* Logical VLANs not supported.\n@@ -10196,7 +10287,7 @@ build_check_pkt_len_action_string(struct ovn_port *op, struct ds *actions);\n */\n static void\n build_adm_ctrl_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (op->nbrp) {\n@@ -10248,7 +10339,7 @@ build_adm_ctrl_flows_for_lrouter_port(\n * lflows for logical routers. */\n static void\n build_neigh_learning_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -10352,7 +10443,7 @@ build_neigh_learning_flows_for_lrouter(\n * for logical router ports. */\n static void\n build_neigh_learning_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (op->nbrp) {\n@@ -10415,7 +10506,7 @@ build_neigh_learning_flows_for_lrouter_port(\n * Adv (RA) options and response. */\n static void\n build_ND_RA_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -10546,7 +10637,8 @@ build_ND_RA_flows_for_lrouter_port(\n /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS\n * responder, by default goto next. (priority 0). */\n static void\n-build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows)\n+build_ND_RA_flows_for_lrouter(struct ovn_datapath *od,\n+ struct lflow_state *lflows)\n {\n if (od->nbr) {\n ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, \"1\", \"next;\");\n@@ -10572,7 +10664,7 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows)\n */\n static void\n build_ip_routing_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *ports,struct hmap *lflows)\n+ struct ovn_port *op, struct hmap *ports,struct lflow_state *lflows)\n {\n if (op->nbrp) {\n \n@@ -10619,7 +10711,7 @@ build_ip_routing_flows_for_lrouter_port(\n \n static void\n build_static_route_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct hmap *ports, struct hmap *bfd_connections)\n {\n if (od->nbr) {\n@@ -10673,7 +10765,7 @@ build_static_route_flows_for_lrouter(\n */\n static void\n build_mcast_lookup_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (od->nbr) {\n@@ -10742,7 +10834,7 @@ build_mcast_lookup_flows_for_lrouter(\n * advances to the next table for ARP/ND resolution. */\n static void\n build_ingress_policy_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct hmap *ports)\n {\n if (od->nbr) {\n@@ -10776,7 +10868,7 @@ build_ingress_policy_flows_for_lrouter(\n /* Local router ingress table ARP_RESOLVE: ARP Resolution. */\n static void\n build_arp_resolve_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows)\n+ struct ovn_datapath *od, struct lflow_state *lflows)\n {\n if (od->nbr) {\n /* Multicast packets already have the outport set so just advance to\n@@ -10793,7 +10885,8 @@ build_arp_resolve_flows_for_lrouter(\n }\n \n static void\n-routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port,\n+routable_addresses_to_lflows(struct lflow_state *lflows,\n+ struct ovn_port *router_port,\n struct ovn_port *peer, struct ds *match,\n struct ds *actions)\n {\n@@ -10834,7 +10927,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port,\n */\n static void\n build_arp_resolve_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct hmap *ports,\n struct ds *match, struct ds *actions)\n {\n@@ -11157,7 +11250,8 @@ build_arp_resolve_flows_for_lrouter_port(\n }\n \n static void\n-build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows,\n+build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu,\n+ struct lflow_state *lflows,\n struct shash *meter_groups, struct ds *match,\n struct ds *actions, enum ovn_stage stage,\n struct ovn_port *outport)\n@@ -11250,7 +11344,8 @@ build_check_pkt_len_action_string(struct ovn_port *op, struct ds *actions)\n \n static void\n build_check_pkt_len_flows_for_lrp(struct ovn_port *op,\n- struct hmap *lflows, struct hmap *ports,\n+ struct lflow_state *lflows,\n+ struct hmap *ports,\n struct shash *meter_groups, struct ds *match,\n struct ds *actions)\n {\n@@ -11303,7 +11398,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op,\n * */\n static void\n build_check_pkt_len_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct hmap *ports,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n@@ -11338,7 +11433,7 @@ build_check_pkt_len_flows_for_lrouter(\n */\n static void\n build_gateway_redirect_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (!od->nbr) {\n@@ -11376,7 +11471,7 @@ build_gateway_redirect_flows_for_lrouter(\n * and sends an ARP/IPv6 NA request (priority 100). */\n static void\n build_arp_request_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows,\n+ struct ovn_datapath *od, struct lflow_state *lflows,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -11455,7 +11550,7 @@ build_arp_request_flows_for_lrouter(\n */\n static void\n build_egress_delivery_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match, struct ds *actions)\n {\n if (op->nbrp) {\n@@ -11497,7 +11592,7 @@ build_egress_delivery_flows_for_lrouter_port(\n \n static void\n build_misc_local_traffic_drop_flows_for_lrouter(\n- struct ovn_datapath *od, struct hmap *lflows)\n+ struct ovn_datapath *od, struct lflow_state *lflows)\n {\n if (od->nbr) {\n /* L3 admission control: drop multicast and broadcast source, localhost\n@@ -11552,7 +11647,7 @@ build_misc_local_traffic_drop_flows_for_lrouter(\n \n static void\n build_dhcpv6_reply_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match)\n {\n if (op->nbrp && (!op->l3dgw_port)) {\n@@ -11571,7 +11666,7 @@ build_dhcpv6_reply_flows_for_lrouter_port(\n \n static void\n build_ipv6_input_flows_for_lrouter_port(\n- struct ovn_port *op, struct hmap *lflows,\n+ struct ovn_port *op, struct lflow_state *lflows,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -11733,7 +11828,7 @@ build_ipv6_input_flows_for_lrouter_port(\n \n static void\n build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct shash *meter_groups)\n {\n if (od->nbr) {\n@@ -11783,7 +11878,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od,\n /* Logical router ingress table 3: IP Input for IPv4. */\n static void\n build_lrouter_ipv4_ip_input(struct ovn_port *op,\n- struct hmap *lflows,\n+ struct lflow_state *lflows,\n struct ds *match, struct ds *actions,\n struct shash *meter_groups)\n {\n@@ -12071,7 +12166,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op,\n }\n \n static void\n-build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_in_unsnat_flow(struct lflow_state *lflows,\n+ struct ovn_datapath *od,\n const struct nbrec_nat *nat, struct ds *match,\n struct ds *actions, bool distributed, bool is_v6)\n {\n@@ -12134,7 +12230,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_in_dnat_flow(struct lflow_state *lflows, struct ovn_datapath *od,\n const struct nbrec_nat *nat, struct ds *match,\n struct ds *actions, bool distributed,\n ovs_be32 mask, bool is_v6)\n@@ -12221,7 +12317,8 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_out_undnat_flow(struct lflow_state *lflows,\n+ struct ovn_datapath *od,\n const struct nbrec_nat *nat, struct ds *match,\n struct ds *actions, bool distributed,\n struct eth_addr mac, bool is_v6)\n@@ -12268,7 +12365,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_out_snat_flow(struct lflow_state *lflows,\n+ struct ovn_datapath *od,\n const struct nbrec_nat *nat, struct ds *match,\n struct ds *actions, bool distributed,\n struct eth_addr mac, ovs_be32 mask,\n@@ -12361,7 +12459,7 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od,\n }\n \n static void\n-build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od,\n+build_lrouter_ingress_flow(struct lflow_state *lflows, struct ovn_datapath *od,\n const struct nbrec_nat *nat, struct ds *match,\n struct ds *actions, struct eth_addr mac,\n bool distributed, bool is_v6)\n@@ -12497,7 +12595,8 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat,\n \n /* NAT, Defrag and load balancing. */\n static void\n-build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows,\n+build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od,\n+ struct lflow_state *lflows,\n struct hmap *ports, struct ds *match,\n struct ds *actions)\n {\n@@ -12719,7 +12818,7 @@ struct lswitch_flow_build_info {\n struct hmap *datapaths;\n struct hmap *ports;\n struct hmap *port_groups;\n- struct hmap *lflows;\n+ struct lflow_state *lflows;\n struct hmap *mcgroups;\n struct hmap *igmp_groups;\n struct shash *meter_groups;\n@@ -12940,12 +13039,6 @@ init_lflows_thread_pool(void)\n }\n }\n \n-/* TODO: replace hard cutoffs by configurable via commands. These are\n- * temporary defines to determine single-thread to multi-thread processing\n- * cutoff.\n- * Setting to 1 forces \"all parallel\" lflow build.\n- */\n-\n static void\n noop_callback(struct worker_pool *pool OVS_UNUSED,\n void *fin_result OVS_UNUSED,\n@@ -12955,10 +13048,24 @@ noop_callback(struct worker_pool *pool OVS_UNUSED,\n /* Do nothing */\n }\n \n+static void\n+lflow_merge_callback(struct worker_pool *pool OVS_UNUSED,\n+ void *fin_result,\n+ void *result_frags,\n+ int index)\n+{\n+ struct lflow_state *result = (struct lflow_state *)fin_result;\n+ struct lflow_state *res_frags = (struct lflow_state *)result_frags;\n+\n+ fast_hmap_merge(&result->single_od, &res_frags[index].single_od);\n+ hmap_destroy(&res_frags[index].single_od);\n+}\n+\n \n static void\n build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports,\n- struct hmap *port_groups, struct hmap *lflows,\n+ struct hmap *port_groups,\n+ struct lflow_state *lflows,\n struct hmap *mcgroups,\n struct hmap *igmp_groups,\n struct shash *meter_groups, struct hmap *lbs,\n@@ -12975,7 +13082,7 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports,\n }\n \n if (use_parallel_build && (!use_logical_dp_groups)) {\n- struct hmap *lflow_segs;\n+ struct lflow_state *lflow_segs;\n struct lswitch_flow_build_info *lsiv;\n int index;\n \n@@ -12994,7 +13101,9 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports,\n * on a per-bucket level instead of merging hash frags */\n lsiv[index].lflows = lflows;\n } else {\n- fast_hmap_init(&lflow_segs[index], lflows->mask);\n+ fast_hmap_init(&lflow_segs[index].single_od,\n+ lflows->single_od.mask);\n+ hmap_init(&lflow_segs[index].multiple_od);\n lsiv[index].lflows = &lflow_segs[index];\n }\n \n@@ -13017,7 +13126,9 @@ build_lswitch_and_lrouter_flows(struct hmap *datapaths, struct hmap *ports,\n if (use_logical_dp_groups) {\n run_pool_callback(build_lflows_pool->pool, NULL, NULL, noop_callback);\n } else {\n- run_pool_hash(build_lflows_pool->pool, lflows, lflow_segs);\n+ run_pool_callback(build_lflows_pool->pool,\n+ &lflows->single_od, lflow_segs,\n+ lflow_merge_callback);\n }\n \n for (index = 0; index < build_lflows_pool->pool->size; index++) {\n@@ -13157,6 +13268,57 @@ ovn_sb_set_lflow_logical_dp_group(\n \n static ssize_t max_seen_lflow_size = 128;\n \n+static void\n+reconcile_lflow(struct ovn_lflow *lflow, struct northd_context *ctx,\n+ struct hmap *dp_groups, struct lflow_state *lflows)\n+{\n+ const struct sbrec_logical_flow *sbflow;\n+ const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage);\n+ uint8_t table = ovn_stage_get_table(lflow->stage);\n+\n+ sbflow = sbrec_logical_flow_insert(ctx->ovnsb_txn);\n+ if (lflow->od) {\n+ sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb);\n+ }\n+ ovn_sb_set_lflow_logical_dp_group(ctx, dp_groups,\n+ sbflow, &lflow->od_group);\n+ sbrec_logical_flow_set_pipeline(sbflow, pipeline);\n+ sbrec_logical_flow_set_table_id(sbflow, table);\n+ sbrec_logical_flow_set_priority(sbflow, lflow->priority);\n+ sbrec_logical_flow_set_match(sbflow, lflow->match);\n+ sbrec_logical_flow_set_actions(sbflow, lflow->actions);\n+ if (lflow->io_port) {\n+ struct smap tags = SMAP_INITIALIZER(&tags);\n+ smap_add(&tags, \"in_out_port\", lflow->io_port);\n+ sbrec_logical_flow_set_tags(sbflow, &tags);\n+ smap_destroy(&tags);\n+ }\n+ sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter);\n+\n+ /* Trim the source locator lflow->where, which looks something like\n+ * \"ovn/northd/ovn-northd.c:1234\", down to just the part following the\n+ * last slash, e.g. \"ovn-northd.c:1234\". */\n+ const char *slash = strrchr(lflow->where, '/');\n+#if _WIN32\n+ const char *backslash = strrchr(lflow->where, '\\\\');\n+ if (!slash || backslash > slash) {\n+ slash = backslash;\n+ }\n+#endif\n+ const char *where = slash ? slash + 1 : lflow->where;\n+\n+ struct smap ids = SMAP_INITIALIZER(&ids);\n+ smap_add(&ids, \"stage-name\", ovn_stage_to_str(lflow->stage));\n+ smap_add(&ids, \"source\", where);\n+ if (lflow->stage_hint) {\n+ smap_add(&ids, \"stage-hint\", lflow->stage_hint);\n+ }\n+ sbrec_logical_flow_set_external_ids(sbflow, &ids);\n+ smap_destroy(&ids);\n+\n+ ovn_lflow_destroy(lflows, lflow);\n+}\n+\n /* Updates the Logical_Flow and Multicast_Group tables in the OVN_SB database,\n * constructing their contents based on the OVN_NB database. */\n static void\n@@ -13167,51 +13329,41 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths,\n struct hmap *lbs, struct hmap *bfd_connections,\n bool ovn_internal_version_changed)\n {\n- struct hmap lflows;\n+ struct lflow_state lflows;\n \n- fast_hmap_size_for(&lflows, max_seen_lflow_size);\n- if (use_parallel_build) {\n- update_hashrow_locks(&lflows, &lflow_locks);\n+ fast_hmap_size_for(&lflows.single_od, max_seen_lflow_size);\n+ fast_hmap_size_for(&lflows.multiple_od, max_seen_lflow_size);\n+ if (use_parallel_build && use_logical_dp_groups) {\n+ update_hashrow_locks(&lflows.single_od, &lflow_locks);\n }\n+\n+\n build_lswitch_and_lrouter_flows(datapaths, ports,\n port_groups, &lflows, mcgroups,\n igmp_groups, meter_groups, lbs,\n bfd_connections);\n \n /* Parallel build may result in a suboptimal hash. Resize the\n- * hash to a correct size before doing lookups */\n-\n- hmap_expand(&lflows);\n+ * hash to a correct size before doing lookups.\n+ * We need to do that only to the multiple_od hash. Single_od\n+ * is recreated during processing, it will be the correct size\n+ * at the point where reconciliation starts lookups.\n+ */\n \n- if (hmap_count(&lflows) > max_seen_lflow_size) {\n- max_seen_lflow_size = hmap_count(&lflows);\n- }\n+ hmap_expand(&lflows.multiple_od);\n \n stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec());\n- /* Collecting all unique datapath groups. */\n+ /* Collecting all unique datapath groups. Works only with\n+ * multiple_od hash and is not dependent on single_od.\n+ * Can run in a separate thread after OVN updates to the\n+ * newer version of the parallel API.\n+ */\n struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups);\n- struct hmapx single_dp_lflows = HMAPX_INITIALIZER(&single_dp_lflows);\n- struct ovn_lflow *lflow;\n- HMAP_FOR_EACH (lflow, hmap_node, &lflows) {\n+ struct ovn_lflow *lflow, *next_lflow;\n+\n+ HMAP_FOR_EACH (lflow, hmap_node, &lflows.multiple_od) {\n uint32_t hash = hash_int(hmapx_count(&lflow->od_group), 0);\n struct ovn_dp_group *dpg;\n-\n- ovs_assert(hmapx_count(&lflow->od_group));\n-\n- if (hmapx_count(&lflow->od_group) == 1) {\n- /* There is only one datapath, so it should be moved out of the\n- * group to a single 'od'. */\n- const struct hmapx_node *node;\n- HMAPX_FOR_EACH (node, &lflow->od_group) {\n- lflow->od = node->data;\n- break;\n- }\n- hmapx_clear(&lflow->od_group);\n- /* Logical flow should be re-hashed later to allow lookups. */\n- hmapx_add(&single_dp_lflows, lflow);\n- continue;\n- }\n-\n dpg = ovn_dp_group_find(&dp_groups, &lflow->od_group, hash);\n if (!dpg) {\n dpg = xzalloc(sizeof *dpg);\n@@ -13222,18 +13374,34 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths,\n }\n \n /* Adding datapath to the flow hash for logical flows that have only one,\n- * so they could be found by the southbound db record. */\n- const struct hmapx_node *node;\n+ * so they could be found by the southbound db record.\n+ * Not dependent on multiple_od - same comment as above applies here.\n+ */\n+\n+ struct hmap processed_single;\n+ hmap_init(&processed_single);\n+\n uint32_t hash;\n- HMAPX_FOR_EACH (node, &single_dp_lflows) {\n- lflow = node->data;\n+ struct hmapx_node *node;\n+ HMAP_FOR_EACH_POP (lflow, hmap_node, &lflows.single_od) {\n+ HMAPX_FOR_EACH (node, &lflow->od_group) {\n+ lflow->od = node->data;\n+ break;\n+ }\n+ hmapx_clear(&lflow->od_group);\n hash = hmap_node_hash(&lflow->hmap_node);\n- hmap_remove(&lflows, &lflow->hmap_node);\n hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid,\n hash);\n- hmap_insert(&lflows, &lflow->hmap_node, hash);\n+ hmap_insert(&processed_single, &lflow->hmap_node, hash);\n }\n- hmapx_destroy(&single_dp_lflows);\n+\n+ hmap_destroy(&lflows.single_od);\n+\n+ lflows.single_od = processed_single;\n+ hmap_moved(&lflows.single_od);\n+\n+ max_seen_lflow_size = MAX(hmap_count(&lflows.single_od),\n+ hmap_count(&lflows.multiple_od));\n \n /* Push changes to the Logical_Flow table to database. */\n const struct sbrec_logical_flow *sbflow, *next_sbflow;\n@@ -13366,54 +13534,14 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths,\n }\n \n stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec());\n- struct ovn_lflow *next_lflow;\n- HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows) {\n- const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage);\n- uint8_t table = ovn_stage_get_table(lflow->stage);\n-\n- sbflow = sbrec_logical_flow_insert(ctx->ovnsb_txn);\n- if (lflow->od) {\n- sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb);\n- }\n- ovn_sb_set_lflow_logical_dp_group(ctx, &dp_groups,\n- sbflow, &lflow->od_group);\n- sbrec_logical_flow_set_pipeline(sbflow, pipeline);\n- sbrec_logical_flow_set_table_id(sbflow, table);\n- sbrec_logical_flow_set_priority(sbflow, lflow->priority);\n- sbrec_logical_flow_set_match(sbflow, lflow->match);\n- sbrec_logical_flow_set_actions(sbflow, lflow->actions);\n- if (lflow->io_port) {\n- struct smap tags = SMAP_INITIALIZER(&tags);\n- smap_add(&tags, \"in_out_port\", lflow->io_port);\n- sbrec_logical_flow_set_tags(sbflow, &tags);\n- smap_destroy(&tags);\n- }\n- sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter);\n-\n- /* Trim the source locator lflow->where, which looks something like\n- * \"ovn/northd/ovn-northd.c:1234\", down to just the part following the\n- * last slash, e.g. \"ovn-northd.c:1234\". */\n- const char *slash = strrchr(lflow->where, '/');\n-#if _WIN32\n- const char *backslash = strrchr(lflow->where, '\\\\');\n- if (!slash || backslash > slash) {\n- slash = backslash;\n- }\n-#endif\n- const char *where = slash ? slash + 1 : lflow->where;\n-\n- struct smap ids = SMAP_INITIALIZER(&ids);\n- smap_add(&ids, \"stage-name\", ovn_stage_to_str(lflow->stage));\n- smap_add(&ids, \"source\", where);\n- if (lflow->stage_hint) {\n- smap_add(&ids, \"stage-hint\", lflow->stage_hint);\n- }\n- sbrec_logical_flow_set_external_ids(sbflow, &ids);\n- smap_destroy(&ids);\n-\n- ovn_lflow_destroy(&lflows, lflow);\n+ HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows.single_od) {\n+ reconcile_lflow(lflow, ctx, &dp_groups, &lflows);\n+ }\n+ HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows.multiple_od) {\n+ reconcile_lflow(lflow, ctx, &dp_groups, &lflows);\n }\n- hmap_destroy(&lflows);\n+ hmap_destroy(&lflows.single_od);\n+ hmap_destroy(&lflows.multiple_od);\n \n struct ovn_dp_group *dpg;\n HMAP_FOR_EACH_POP (dpg, node, &dp_groups) {\n", "prefixes": [ "ovs-dev", "v6", "3/4" ] }